2024-09-10T21:28:54.065 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T21:28:54.086 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T21:28:54.182 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899691 branch: wip-hemanth-testing-2024-09-10-0723 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} email: hyelloji@redhat.com first_in_suite: false job_id: '7899691' kernel: client: branch: testing flavor: default kdb: 1 sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-hemanth-testing-2024-09-10-0723 ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected - object missing on disk - Invalid tag char sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 kclient: syntax: v2 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-09-10-0723 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 6190 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 sleep_before_teardown: 0 subset: 479/1024 suite: fs suite_branch: wip-hemanth-testing-2024-09-10-0723 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi007.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBjDGS5nmwPi985Xovar9cxzJNEIjtgdGwuAJ0CFZXkw9GSDn9dqqagcE6Ym3nk6G1VnrQapo4JeRmfp5KNJYfU= smithi133.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIwVSnTQFwGtT2F70wr2qNvD0ndyipSXStfikPxAgevFOu/1/rsghAHgEVNqIGBC418XUMnXj2QK3WdqEa0SPDc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_16:59:47 tube: smithi user: hyelloji verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T21:28:54.182 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa; will attempt to use it 2024-09-10T21:28:54.183 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks 2024-09-10T21:28:54.183 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T21:28:54.185 INFO:teuthology.task.internal:Checking packages... 2024-09-10T21:28:54.205 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8f219c476cb4f26fc5719c661c0323719a45cb30' 2024-09-10T21:28:54.205 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T21:28:54.205 INFO:teuthology.packaging:ref: None 2024-09-10T21:28:54.205 INFO:teuthology.packaging:tag: None 2024-09-10T21:28:54.206 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-09-10-0723 2024-09-10T21:28:54.206 INFO:teuthology.packaging:sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:28:54.206 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-hemanth-testing-2024-09-10-0723 2024-09-10T21:28:54.352 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4854.g8f219c47 2024-09-10T21:28:54.354 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T21:28:54.360 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T21:28:54.360 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T21:28:54.498 INFO:teuthology.task.internal:Saving configuration 2024-09-10T21:28:54.516 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T21:28:54.522 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T21:28:54.552 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi007.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899691', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 21:23:59.902509', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBjDGS5nmwPi985Xovar9cxzJNEIjtgdGwuAJ0CFZXkw9GSDn9dqqagcE6Ym3nk6G1VnrQapo4JeRmfp5KNJYfU='} 2024-09-10T21:28:54.586 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi133.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899691', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 21:23:59.904474', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIwVSnTQFwGtT2F70wr2qNvD0ndyipSXStfikPxAgevFOu/1/rsghAHgEVNqIGBC418XUMnXj2QK3WdqEa0SPDc='} 2024-09-10T21:28:54.586 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T21:28:54.592 INFO:teuthology.task.internal:roles: ubuntu@smithi007.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-10T21:28:54.592 INFO:teuthology.task.internal:roles: ubuntu@smithi133.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-10T21:28:54.592 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T21:28:54.668 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9cac8d3be0>, signals=[15]) 2024-09-10T21:28:54.668 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T21:28:54.674 INFO:teuthology.task.internal:Opening connections... 2024-09-10T21:28:54.674 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:28:54.675 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:28:54.758 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi133.front.sepia.ceph.com 2024-09-10T21:28:54.759 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:28:54.843 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T21:28:54.850 DEBUG:teuthology.orchestra.run.smithi007:> uname -m 2024-09-10T21:28:54.868 INFO:teuthology.orchestra.run.smithi007.stdout:x86_64 2024-09-10T21:28:54.868 DEBUG:teuthology.orchestra.run.smithi007:> cat /etc/os-release 2024-09-10T21:28:54.925 INFO:teuthology.orchestra.run.smithi007.stdout:NAME="CentOS Stream" 2024-09-10T21:28:54.926 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION="9" 2024-09-10T21:28:54.926 INFO:teuthology.orchestra.run.smithi007.stdout:ID="centos" 2024-09-10T21:28:54.926 INFO:teuthology.orchestra.run.smithi007.stdout:ID_LIKE="rhel fedora" 2024-09-10T21:28:54.926 INFO:teuthology.orchestra.run.smithi007.stdout:VERSION_ID="9" 2024-09-10T21:28:54.926 INFO:teuthology.orchestra.run.smithi007.stdout:PLATFORM_ID="platform:el9" 2024-09-10T21:28:54.926 INFO:teuthology.orchestra.run.smithi007.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T21:28:54.926 INFO:teuthology.orchestra.run.smithi007.stdout:ANSI_COLOR="0;31" 2024-09-10T21:28:54.926 INFO:teuthology.orchestra.run.smithi007.stdout:LOGO="fedora-logo-icon" 2024-09-10T21:28:54.926 INFO:teuthology.orchestra.run.smithi007.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T21:28:54.926 INFO:teuthology.orchestra.run.smithi007.stdout:HOME_URL="https://centos.org/" 2024-09-10T21:28:54.926 INFO:teuthology.orchestra.run.smithi007.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T21:28:54.927 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T21:28:54.927 INFO:teuthology.orchestra.run.smithi007.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T21:28:54.927 INFO:teuthology.lock.ops:Updating smithi007.front.sepia.ceph.com on lock server 2024-09-10T21:28:54.955 DEBUG:teuthology.orchestra.run.smithi133:> uname -m 2024-09-10T21:28:54.974 INFO:teuthology.orchestra.run.smithi133.stdout:x86_64 2024-09-10T21:28:54.974 DEBUG:teuthology.orchestra.run.smithi133:> cat /etc/os-release 2024-09-10T21:28:55.032 INFO:teuthology.orchestra.run.smithi133.stdout:NAME="CentOS Stream" 2024-09-10T21:28:55.032 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION="9" 2024-09-10T21:28:55.032 INFO:teuthology.orchestra.run.smithi133.stdout:ID="centos" 2024-09-10T21:28:55.032 INFO:teuthology.orchestra.run.smithi133.stdout:ID_LIKE="rhel fedora" 2024-09-10T21:28:55.032 INFO:teuthology.orchestra.run.smithi133.stdout:VERSION_ID="9" 2024-09-10T21:28:55.032 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM_ID="platform:el9" 2024-09-10T21:28:55.032 INFO:teuthology.orchestra.run.smithi133.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T21:28:55.032 INFO:teuthology.orchestra.run.smithi133.stdout:ANSI_COLOR="0;31" 2024-09-10T21:28:55.032 INFO:teuthology.orchestra.run.smithi133.stdout:LOGO="fedora-logo-icon" 2024-09-10T21:28:55.032 INFO:teuthology.orchestra.run.smithi133.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T21:28:55.032 INFO:teuthology.orchestra.run.smithi133.stdout:HOME_URL="https://centos.org/" 2024-09-10T21:28:55.032 INFO:teuthology.orchestra.run.smithi133.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T21:28:55.033 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T21:28:55.033 INFO:teuthology.orchestra.run.smithi133.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T21:28:55.033 INFO:teuthology.lock.ops:Updating smithi133.front.sepia.ceph.com on lock server 2024-09-10T21:28:55.063 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T21:28:55.073 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T21:28:55.078 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T21:28:55.078 DEBUG:teuthology.orchestra.run.smithi007:> test '!' -e /home/ubuntu/cephtest 2024-09-10T21:28:55.080 DEBUG:teuthology.orchestra.run.smithi133:> test '!' -e /home/ubuntu/cephtest 2024-09-10T21:28:55.097 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T21:28:55.103 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T21:28:55.103 DEBUG:teuthology.orchestra.run.smithi007:> test -z $(ls -A /var/lib/ceph) 2024-09-10T21:28:55.138 DEBUG:teuthology.orchestra.run.smithi133:> test -z $(ls -A /var/lib/ceph) 2024-09-10T21:28:55.166 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T21:28:55.236 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T21:28:55.250 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'} 2024-09-10T21:28:55.250 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-10T21:28:55.250 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-10T21:28:55.251 DEBUG:teuthology.orchestra.run.smithi007:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T21:28:55.251 DEBUG:teuthology.orchestra.run.smithi133:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T21:28:55.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:28:55.268 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-10T21:28:55.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:28:55.279 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-10T21:28:55.448 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:28:55.448 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T21:28:55.449 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-09-10T21:28:55.450 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:28:55.450 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T21:28:55.450 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-09-10T21:28:55.468 INFO:teuthology.orchestra.run.smithi007.stdout:5.14.0-503.el9.x86_64 2024-09-10T21:28:55.468 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:28:55.468 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T21:28:55.469 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum upgrade -y linux-firmware 2024-09-10T21:28:55.470 INFO:teuthology.orchestra.run.smithi133.stdout:5.14.0-503.el9.x86_64 2024-09-10T21:28:55.470 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:28:55.470 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T21:28:55.470 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum upgrade -y linux-firmware 2024-09-10T21:28:58.037 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.2 MB 00:00 2024-09-10T21:29:00.290 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.2 MB 00:03 2024-09-10T21:29:01.163 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:01 2024-09-10T21:29:02.558 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - AppStream 18 MB/s | 20 MB 00:01 2024-09-10T21:29:06.446 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 6.5 MB 00:01 2024-09-10T21:29:07.233 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.5 MB 00:00 2024-09-10T21:29:08.047 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 73 kB/s | 19 kB 00:00 2024-09-10T21:29:09.309 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 20 MB/s | 23 MB 00:01 2024-09-10T21:29:09.575 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - Extras packages 17 kB/s | 19 kB 00:01 2024-09-10T21:29:11.146 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-10T21:29:14.957 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-10T21:29:16.560 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T21:29:17.134 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-10T21:29:17.135 INFO:teuthology.orchestra.run.smithi007.stdout:Nothing to do. 2024-09-10T21:29:17.135 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-10T21:29:17.251 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T21:29:17.252 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:29:17.427 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-10T21:29:17.427 DEBUG:teuthology.orchestra.run.smithi007:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-10T21:29:17.430 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi007.front.sepia.ceph.com to complete... 2024-09-10T21:29:18.680 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-09-10T21:29:18.681 INFO:teuthology.orchestra.run.smithi133.stdout:Nothing to do. 2024-09-10T21:29:18.681 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-09-10T21:29:18.800 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-10T21:29:18.801 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:29:18.961 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-10T21:29:18.961 DEBUG:teuthology.orchestra.run.smithi133:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-10T21:29:18.964 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi133.front.sepia.ceph.com to complete... 2024-09-10T21:29:26.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10 21:29:26 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-10T21:29:26.862 INFO:teuthology.orchestra.run.smithi007.stderr:FINISHED --2024-09-10 21:29:26-- 2024-09-10T21:29:26.863 INFO:teuthology.orchestra.run.smithi007.stderr:Total wall clock time: 9.4s 2024-09-10T21:29:26.863 INFO:teuthology.orchestra.run.smithi007.stderr:Downloaded: 1 files, 343M in 9.1s (37.8 MB/s) 2024-09-10T21:29:26.865 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-10T21:29:26.865 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-10T21:29:29.087 INFO:teuthology.orchestra.run.smithi007.stdout:Verifying... ######################################## 2024-09-10T21:29:29.217 INFO:teuthology.orchestra.run.smithi007.stdout:Preparing... ######################################## 2024-09-10T21:29:29.345 INFO:teuthology.orchestra.run.smithi007.stdout:Updating / installing... 2024-09-10T21:29:29.679 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10 21:29:29 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-10T21:29:29.679 INFO:teuthology.orchestra.run.smithi133.stderr:FINISHED --2024-09-10 21:29:29-- 2024-09-10T21:29:29.679 INFO:teuthology.orchestra.run.smithi133.stderr:Total wall clock time: 11s 2024-09-10T21:29:29.680 INFO:teuthology.orchestra.run.smithi133.stderr:Downloaded: 1 files, 343M in 10s (33.4 MB/s) 2024-09-10T21:29:29.682 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-10T21:29:29.682 DEBUG:teuthology.orchestra.run.smithi133:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-10T21:29:31.790 INFO:teuthology.orchestra.run.smithi133.stdout:Verifying... ######################################## 2024-09-10T21:29:31.921 INFO:teuthology.orchestra.run.smithi133.stdout:Preparing... ######################################## 2024-09-10T21:29:32.040 INFO:teuthology.orchestra.run.smithi133.stdout:Updating / installing... 2024-09-10T21:29:37.656 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-10T21:29:40.038 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-10T21:30:18.243 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi007.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-10T21:30:18.243 DEBUG:teuthology.orchestra.run.smithi007:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-10T21:30:18.336 INFO:teuthology.orchestra.run.smithi007.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:30:18.336 INFO:teuthology.orchestra.run.smithi007.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:30:18.336 INFO:teuthology.orchestra.run.smithi007.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:30:18.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:30:18.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-10T21:30:18.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-10T21:30:18.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-10T21:30:18.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-10T21:30:18.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-10T21:30:18.336 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-10T21:30:18.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T21:30:18.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T21:30:18.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T21:30:18.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T21:30:18.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T21:30:18.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T21:30:18.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T21:30:18.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-10T21:30:18.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T21:30:18.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T21:30:18.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T21:30:18.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T21:30:18.337 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T21:30:18.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T21:30:18.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T21:30:18.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T21:30:18.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T21:30:18.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T21:30:18.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T21:30:18.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T21:30:18.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-10T21:30:18.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-10T21:30:18.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-10T21:30:18.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T21:30:18.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-10T21:30:18.338 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T21:30:18.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T21:30:18.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T21:30:18.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-10T21:30:18.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T21:30:18.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T21:30:18.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T21:30:18.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-10T21:30:18.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T21:30:18.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T21:30:18.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T21:30:18.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T21:30:18.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T21:30:18.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T21:30:18.339 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T21:30:18.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T21:30:18.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T21:30:18.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T21:30:18.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T21:30:18.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T21:30:18.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T21:30:18.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T21:30:18.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T21:30:18.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T21:30:18.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T21:30:18.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T21:30:18.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T21:30:18.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T21:30:18.340 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T21:30:18.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T21:30:18.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T21:30:18.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-10T21:30:18.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-10T21:30:18.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T21:30:18.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T21:30:18.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T21:30:18.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T21:30:18.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T21:30:18.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T21:30:18.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T21:30:18.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T21:30:18.341 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T21:30:18.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T21:30:18.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T21:30:18.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T21:30:18.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T21:30:18.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T21:30:18.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T21:30:18.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T21:30:18.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T21:30:18.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T21:30:18.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T21:30:18.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T21:30:18.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T21:30:18.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T21:30:18.342 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-10T21:30:18.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-10T21:30:18.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T21:30:18.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T21:30:18.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T21:30:18.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T21:30:18.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T21:30:18.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T21:30:18.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T21:30:18.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-10T21:30:18.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T21:30:18.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T21:30:18.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-10T21:30:18.343 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T21:30:18.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T21:30:18.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T21:30:18.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T21:30:18.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T21:30:18.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T21:30:18.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T21:30:18.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T21:30:18.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T21:30:18.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T21:30:18.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T21:30:18.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T21:30:18.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T21:30:18.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T21:30:18.344 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T21:30:18.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T21:30:18.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T21:30:18.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T21:30:18.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T21:30:18.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T21:30:18.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T21:30:18.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T21:30:18.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T21:30:18.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T21:30:18.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T21:30:18.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T21:30:18.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T21:30:18.345 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T21:30:18.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T21:30:18.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T21:30:18.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T21:30:18.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T21:30:18.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T21:30:18.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T21:30:18.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T21:30:18.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-10T21:30:18.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T21:30:18.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T21:30:18.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T21:30:18.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T21:30:18.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T21:30:18.346 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T21:30:18.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T21:30:18.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T21:30:18.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T21:30:18.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T21:30:18.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T21:30:18.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T21:30:18.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T21:30:18.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T21:30:18.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T21:30:18.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T21:30:18.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T21:30:18.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-10T21:30:18.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T21:30:18.347 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-10T21:30:18.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-10T21:30:18.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-10T21:30:18.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-10T21:30:18.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-10T21:30:18.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T21:30:18.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-10T21:30:18.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-10T21:30:18.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T21:30:18.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-10T21:30:18.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-10T21:30:18.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T21:30:18.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T21:30:18.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-10T21:30:18.348 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-10T21:30:18.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T21:30:18.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T21:30:18.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-10T21:30:18.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T21:30:18.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T21:30:18.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T21:30:18.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-10T21:30:18.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-10T21:30:18.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T21:30:18.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-10T21:30:18.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-10T21:30:18.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T21:30:18.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-10T21:30:18.349 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T21:30:18.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-10T21:30:18.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T21:30:18.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T21:30:18.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T21:30:18.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T21:30:18.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T21:30:18.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-10T21:30:18.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T21:30:18.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T21:30:18.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T21:30:18.350 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T21:30:18.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T21:30:18.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T21:30:18.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T21:30:18.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-10T21:30:18.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T21:30:18.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T21:30:18.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T21:30:18.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-10T21:30:18.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-10T21:30:18.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T21:30:18.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T21:30:18.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T21:30:18.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-10T21:30:18.351 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-10T21:30:18.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-10T21:30:18.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-10T21:30:18.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-10T21:30:18.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-10T21:30:18.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T21:30:18.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T21:30:18.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-10T21:30:18.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-10T21:30:18.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T21:30:18.352 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-10T21:30:18.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-10T21:30:18.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T21:30:18.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T21:30:18.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-10T21:30:18.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-10T21:30:18.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-10T21:30:18.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T21:30:18.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-10T21:30:18.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T21:30:18.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T21:30:18.353 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T21:30:18.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-10T21:30:18.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T21:30:18.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-10T21:30:18.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T21:30:18.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-10T21:30:18.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-10T21:30:18.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-10T21:30:18.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-10T21:30:18.354 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T21:30:18.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-10T21:30:18.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T21:30:18.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-10T21:30:18.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T21:30:18.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T21:30:18.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T21:30:18.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T21:30:18.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T21:30:18.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T21:30:18.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T21:30:18.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T21:30:18.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T21:30:18.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T21:30:18.355 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T21:30:18.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T21:30:18.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T21:30:18.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-10T21:30:18.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-10T21:30:18.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T21:30:18.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-10T21:30:18.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T21:30:18.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T21:30:18.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-10T21:30:18.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T21:30:18.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T21:30:18.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T21:30:18.356 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T21:30:18.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T21:30:18.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T21:30:18.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T21:30:18.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T21:30:18.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T21:30:18.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T21:30:18.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T21:30:18.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T21:30:18.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T21:30:18.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T21:30:18.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T21:30:18.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T21:30:18.357 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T21:30:18.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T21:30:18.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T21:30:18.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T21:30:18.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T21:30:18.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T21:30:18.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T21:30:18.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T21:30:18.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T21:30:18.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T21:30:18.358 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T21:30:18.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T21:30:18.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T21:30:18.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T21:30:18.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T21:30:18.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T21:30:18.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T21:30:18.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T21:30:18.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T21:30:18.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T21:30:18.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T21:30:18.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T21:30:18.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T21:30:18.359 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T21:30:18.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T21:30:18.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T21:30:18.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T21:30:18.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T21:30:18.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T21:30:18.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T21:30:18.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T21:30:18.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T21:30:18.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T21:30:18.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T21:30:18.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T21:30:18.360 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T21:30:18.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T21:30:18.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T21:30:18.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T21:30:18.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T21:30:18.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T21:30:18.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T21:30:18.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T21:30:18.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T21:30:18.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T21:30:18.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T21:30:18.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T21:30:18.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T21:30:18.361 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-10T21:30:18.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T21:30:18.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T21:30:18.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T21:30:18.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-10T21:30:18.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T21:30:18.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T21:30:18.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T21:30:18.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T21:30:18.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T21:30:18.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T21:30:18.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T21:30:18.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T21:30:18.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T21:30:18.362 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T21:30:18.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T21:30:18.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T21:30:18.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T21:30:18.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T21:30:18.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T21:30:18.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T21:30:18.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T21:30:18.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T21:30:18.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T21:30:18.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T21:30:18.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T21:30:18.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T21:30:18.363 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T21:30:18.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T21:30:18.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T21:30:18.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T21:30:18.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T21:30:18.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T21:30:18.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T21:30:18.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T21:30:18.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T21:30:18.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T21:30:18.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T21:30:18.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T21:30:18.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T21:30:18.364 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T21:30:18.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T21:30:18.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T21:30:18.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T21:30:18.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T21:30:18.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T21:30:18.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T21:30:18.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T21:30:18.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T21:30:18.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T21:30:18.365 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T21:30:18.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T21:30:18.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T21:30:18.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T21:30:18.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T21:30:18.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T21:30:18.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T21:30:18.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T21:30:18.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T21:30:18.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T21:30:18.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T21:30:18.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T21:30:18.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T21:30:18.366 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T21:30:18.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T21:30:18.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T21:30:18.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T21:30:18.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T21:30:18.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T21:30:18.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T21:30:18.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T21:30:18.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T21:30:18.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T21:30:18.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T21:30:18.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T21:30:18.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T21:30:18.367 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T21:30:18.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T21:30:18.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T21:30:18.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T21:30:18.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T21:30:18.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T21:30:18.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T21:30:18.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T21:30:18.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T21:30:18.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T21:30:18.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T21:30:18.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T21:30:18.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T21:30:18.368 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T21:30:18.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T21:30:18.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T21:30:18.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T21:30:18.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-10T21:30:18.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T21:30:18.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T21:30:18.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T21:30:18.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T21:30:18.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T21:30:18.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T21:30:18.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T21:30:18.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T21:30:18.369 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T21:30:18.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T21:30:18.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T21:30:18.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T21:30:18.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T21:30:18.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T21:30:18.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T21:30:18.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T21:30:18.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T21:30:18.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T21:30:18.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T21:30:18.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T21:30:18.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T21:30:18.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T21:30:18.370 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T21:30:18.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T21:30:18.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T21:30:18.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T21:30:18.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T21:30:18.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T21:30:18.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T21:30:18.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T21:30:18.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T21:30:18.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T21:30:18.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T21:30:18.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T21:30:18.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T21:30:18.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T21:30:18.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T21:30:18.371 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T21:30:18.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T21:30:18.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T21:30:18.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T21:30:18.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-10T21:30:18.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-10T21:30:18.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T21:30:18.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T21:30:18.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-10T21:30:18.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T21:30:18.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T21:30:18.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T21:30:18.372 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T21:30:18.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T21:30:18.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T21:30:18.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T21:30:18.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T21:30:18.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T21:30:18.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T21:30:18.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T21:30:18.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T21:30:18.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T21:30:18.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T21:30:18.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T21:30:18.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T21:30:18.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T21:30:18.373 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T21:30:18.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T21:30:18.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T21:30:18.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T21:30:18.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T21:30:18.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T21:30:18.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-10T21:30:18.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T21:30:18.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-10T21:30:18.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T21:30:18.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T21:30:18.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-10T21:30:18.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-10T21:30:18.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T21:30:18.374 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T21:30:18.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T21:30:18.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T21:30:18.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T21:30:18.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T21:30:18.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T21:30:18.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-10T21:30:18.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-10T21:30:18.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T21:30:18.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-10T21:30:18.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T21:30:18.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-10T21:30:18.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T21:30:18.375 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-10T21:30:18.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T21:30:18.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-10T21:30:18.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-10T21:30:18.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T21:30:18.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-10T21:30:18.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T21:30:18.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-10T21:30:18.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T21:30:18.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-10T21:30:18.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T21:30:18.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-10T21:30:18.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T21:30:18.376 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-10T21:30:18.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-10T21:30:18.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T21:30:18.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T21:30:18.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T21:30:18.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T21:30:18.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-10T21:30:18.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T21:30:18.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-10T21:30:18.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T21:30:18.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T21:30:18.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T21:30:18.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T21:30:18.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T21:30:18.377 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T21:30:18.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T21:30:18.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T21:30:18.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-10T21:30:18.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T21:30:18.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-10T21:30:18.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T21:30:18.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T21:30:18.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T21:30:18.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T21:30:18.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T21:30:18.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-10T21:30:18.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T21:30:18.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T21:30:18.378 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T21:30:18.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T21:30:18.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-10T21:30:18.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T21:30:18.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T21:30:18.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T21:30:18.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T21:30:18.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T21:30:18.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T21:30:18.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T21:30:18.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T21:30:18.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-10T21:30:18.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-10T21:30:18.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T21:30:18.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T21:30:18.379 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T21:30:18.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T21:30:18.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T21:30:18.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T21:30:18.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T21:30:18.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T21:30:18.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T21:30:18.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T21:30:18.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T21:30:18.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T21:30:18.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T21:30:18.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T21:30:18.380 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T21:30:18.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T21:30:18.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-10T21:30:18.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T21:30:18.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T21:30:18.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T21:30:18.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T21:30:18.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T21:30:18.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-10T21:30:18.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-10T21:30:18.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T21:30:18.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T21:30:18.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T21:30:18.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T21:30:18.381 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T21:30:18.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T21:30:18.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T21:30:18.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-10T21:30:18.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T21:30:18.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T21:30:18.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-10T21:30:18.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T21:30:18.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T21:30:18.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T21:30:18.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T21:30:18.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-10T21:30:18.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-10T21:30:18.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T21:30:18.382 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T21:30:18.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T21:30:18.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T21:30:18.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-10T21:30:18.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T21:30:18.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-10T21:30:18.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T21:30:18.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T21:30:18.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T21:30:18.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T21:30:18.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T21:30:18.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T21:30:18.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T21:30:18.383 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-10T21:30:18.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T21:30:18.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T21:30:18.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-10T21:30:18.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T21:30:18.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T21:30:18.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T21:30:18.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-10T21:30:18.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-10T21:30:18.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-10T21:30:18.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-10T21:30:18.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-10T21:30:18.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-10T21:30:18.384 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T21:30:18.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T21:30:18.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T21:30:18.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T21:30:18.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-10T21:30:18.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T21:30:18.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-10T21:30:18.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-10T21:30:18.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T21:30:18.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-10T21:30:18.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-10T21:30:18.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T21:30:18.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T21:30:18.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T21:30:18.385 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-10T21:30:18.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T21:30:18.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T21:30:18.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T21:30:18.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-10T21:30:18.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-10T21:30:18.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T21:30:18.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T21:30:18.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T21:30:18.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T21:30:18.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-10T21:30:18.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T21:30:18.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T21:30:18.386 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T21:30:18.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T21:30:18.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-10T21:30:18.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T21:30:18.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-10T21:30:18.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T21:30:18.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-10T21:30:18.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T21:30:18.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-10T21:30:18.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T21:30:18.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-10T21:30:18.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T21:30:18.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T21:30:18.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T21:30:18.387 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-10T21:30:18.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-10T21:30:18.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T21:30:18.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-10T21:30:18.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T21:30:18.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-10T21:30:18.388 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T21:30:18.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-10T21:30:18.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T21:30:18.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T21:30:18.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-10T21:30:18.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T21:30:18.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T21:30:18.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-10T21:30:18.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T21:30:18.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T21:30:18.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-10T21:30:18.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T21:30:18.389 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T21:30:18.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-10T21:30:18.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-10T21:30:18.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T21:30:18.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-10T21:30:18.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-10T21:30:18.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T21:30:18.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-10T21:30:18.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T21:30:18.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-10T21:30:18.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T21:30:18.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-10T21:30:18.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T21:30:18.390 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-10T21:30:18.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T21:30:18.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-10T21:30:18.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-10T21:30:18.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T21:30:18.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-10T21:30:18.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-10T21:30:18.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T21:30:18.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T21:30:18.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T21:30:18.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T21:30:18.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T21:30:18.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T21:30:18.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-10T21:30:18.391 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T21:30:18.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T21:30:18.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T21:30:18.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-10T21:30:18.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-10T21:30:18.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T21:30:18.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T21:30:18.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T21:30:18.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-10T21:30:18.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T21:30:18.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-10T21:30:18.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T21:30:18.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-10T21:30:18.392 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T21:30:18.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-10T21:30:18.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T21:30:18.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-10T21:30:18.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T21:30:18.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-10T21:30:18.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T21:30:18.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-10T21:30:18.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T21:30:18.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-10T21:30:18.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T21:30:18.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-10T21:30:18.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T21:30:18.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-10T21:30:18.393 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T21:30:18.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-10T21:30:18.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T21:30:18.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-10T21:30:18.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T21:30:18.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T21:30:18.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-10T21:30:18.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T21:30:18.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T21:30:18.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T21:30:18.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-10T21:30:18.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-10T21:30:18.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T21:30:18.394 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T21:30:18.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-10T21:30:18.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-10T21:30:18.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T21:30:18.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-10T21:30:18.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T21:30:18.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-10T21:30:18.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-10T21:30:18.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T21:30:18.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-10T21:30:18.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T21:30:18.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T21:30:18.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-10T21:30:18.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T21:30:18.395 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-10T21:30:18.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T21:30:18.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-10T21:30:18.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T21:30:18.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T21:30:18.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-10T21:30:18.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-10T21:30:18.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T21:30:18.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T21:30:18.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-10T21:30:18.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T21:30:18.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-10T21:30:18.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T21:30:18.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-10T21:30:18.396 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T21:30:18.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T21:30:18.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T21:30:18.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T21:30:18.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-10T21:30:18.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-10T21:30:18.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-10T21:30:18.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T21:30:18.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-10T21:30:18.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T21:30:18.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-10T21:30:18.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T21:30:18.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T21:30:18.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-10T21:30:18.397 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T21:30:18.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T21:30:18.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T21:30:18.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T21:30:18.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-10T21:30:18.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-10T21:30:18.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T21:30:18.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T21:30:18.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-10T21:30:18.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T21:30:18.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T21:30:18.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T21:30:18.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T21:30:18.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-10T21:30:18.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T21:30:18.398 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-10T21:30:18.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T21:30:18.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-10T21:30:18.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T21:30:18.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T21:30:18.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-10T21:30:18.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T21:30:18.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T21:30:18.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-10T21:30:18.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T21:30:18.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T21:30:18.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-10T21:30:18.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T21:30:18.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-10T21:30:18.399 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T21:30:18.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T21:30:18.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-10T21:30:18.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T21:30:18.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T21:30:18.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T21:30:18.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-10T21:30:18.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T21:30:18.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T21:30:18.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T21:30:18.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T21:30:18.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T21:30:18.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-10T21:30:18.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T21:30:18.400 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T21:30:18.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-10T21:30:18.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T21:30:18.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T21:30:18.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-10T21:30:18.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T21:30:18.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T21:30:18.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-10T21:30:18.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T21:30:18.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T21:30:18.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T21:30:18.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T21:30:18.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T21:30:18.401 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-10T21:30:18.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T21:30:18.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T21:30:18.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T21:30:18.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T21:30:18.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T21:30:18.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T21:30:18.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T21:30:18.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T21:30:18.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T21:30:18.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T21:30:18.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T21:30:18.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T21:30:18.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-10T21:30:18.402 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T21:30:18.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T21:30:18.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T21:30:18.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-10T21:30:18.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T21:30:18.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T21:30:18.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T21:30:18.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T21:30:18.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T21:30:18.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-10T21:30:18.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-10T21:30:18.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T21:30:18.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T21:30:18.403 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T21:30:18.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-10T21:30:18.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T21:30:18.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T21:30:18.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T21:30:18.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T21:30:18.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T21:30:18.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T21:30:18.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-10T21:30:18.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T21:30:18.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-10T21:30:18.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-10T21:30:18.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T21:30:18.404 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-10T21:30:18.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T21:30:18.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-10T21:30:18.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T21:30:18.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-10T21:30:18.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-10T21:30:18.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T21:30:18.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-10T21:30:18.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-10T21:30:18.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T21:30:18.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T21:30:18.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-10T21:30:18.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T21:30:18.405 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T21:30:18.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T21:30:18.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T21:30:18.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-10T21:30:18.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-10T21:30:18.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T21:30:18.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-10T21:30:18.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-10T21:30:18.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T21:30:18.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-10T21:30:18.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T21:30:18.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T21:30:18.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T21:30:18.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-10T21:30:18.406 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-10T21:30:18.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T21:30:18.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-10T21:30:18.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-10T21:30:18.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T21:30:18.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T21:30:18.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T21:30:18.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T21:30:18.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T21:30:18.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T21:30:18.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T21:30:18.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T21:30:18.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T21:30:18.407 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T21:30:18.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T21:30:18.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T21:30:18.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-10T21:30:18.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T21:30:18.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T21:30:18.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T21:30:18.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T21:30:18.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T21:30:18.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T21:30:18.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T21:30:18.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T21:30:18.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T21:30:18.408 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T21:30:18.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T21:30:18.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T21:30:18.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-10T21:30:18.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T21:30:18.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T21:30:18.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-10T21:30:18.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-10T21:30:18.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T21:30:18.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T21:30:18.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T21:30:18.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T21:30:18.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-10T21:30:18.409 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T21:30:18.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T21:30:18.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-10T21:30:18.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-10T21:30:18.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-10T21:30:18.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T21:30:18.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T21:30:18.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-10T21:30:18.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T21:30:18.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T21:30:18.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T21:30:18.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T21:30:18.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T21:30:18.410 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T21:30:18.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T21:30:18.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T21:30:18.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T21:30:18.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T21:30:18.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T21:30:18.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T21:30:18.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T21:30:18.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T21:30:18.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T21:30:18.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T21:30:18.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T21:30:18.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T21:30:18.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T21:30:18.411 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T21:30:18.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T21:30:18.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T21:30:18.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T21:30:18.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T21:30:18.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T21:30:18.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T21:30:18.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T21:30:18.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T21:30:18.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T21:30:18.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T21:30:18.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T21:30:18.412 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-10T21:30:18.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T21:30:18.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T21:30:18.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T21:30:18.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T21:30:18.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T21:30:18.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-10T21:30:18.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T21:30:18.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T21:30:18.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-10T21:30:18.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T21:30:18.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T21:30:18.413 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T21:30:18.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-10T21:30:18.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T21:30:18.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T21:30:18.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-10T21:30:18.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T21:30:18.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-10T21:30:18.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T21:30:18.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-10T21:30:18.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T21:30:18.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-10T21:30:18.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T21:30:18.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T21:30:18.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-10T21:30:18.414 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T21:30:18.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-10T21:30:18.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-10T21:30:18.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T21:30:18.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-10T21:30:18.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T21:30:18.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T21:30:18.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T21:30:18.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-10T21:30:18.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-10T21:30:18.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-10T21:30:18.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-10T21:30:18.415 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-10T21:30:18.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T21:30:18.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-10T21:30:18.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T21:30:18.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T21:30:18.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-10T21:30:18.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T21:30:18.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T21:30:18.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-10T21:30:18.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T21:30:18.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T21:30:18.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T21:30:18.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T21:30:18.416 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T21:30:18.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T21:30:18.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T21:30:18.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-10T21:30:18.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T21:30:18.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T21:30:18.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T21:30:18.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-10T21:30:18.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T21:30:18.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T21:30:18.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T21:30:18.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-10T21:30:18.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T21:30:18.417 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-10T21:30:18.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T21:30:18.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-10T21:30:18.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T21:30:18.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T21:30:18.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T21:30:18.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T21:30:18.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-10T21:30:18.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T21:30:18.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-10T21:30:18.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T21:30:18.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T21:30:18.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-10T21:30:18.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T21:30:18.418 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T21:30:18.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T21:30:18.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-10T21:30:18.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T21:30:18.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T21:30:18.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-10T21:30:18.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T21:30:18.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T21:30:18.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T21:30:18.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T21:30:18.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T21:30:18.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T21:30:18.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T21:30:18.419 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T21:30:18.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T21:30:18.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T21:30:18.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T21:30:18.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-10T21:30:18.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T21:30:18.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T21:30:18.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T21:30:18.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T21:30:18.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-10T21:30:18.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T21:30:18.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-10T21:30:18.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-10T21:30:18.420 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T21:30:18.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-10T21:30:18.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T21:30:18.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T21:30:18.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T21:30:18.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T21:30:18.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T21:30:18.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T21:30:18.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-10T21:30:18.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T21:30:18.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-10T21:30:18.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-10T21:30:18.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-10T21:30:18.421 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T21:30:18.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T21:30:18.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T21:30:18.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T21:30:18.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T21:30:18.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T21:30:18.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T21:30:18.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T21:30:18.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T21:30:18.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T21:30:18.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T21:30:18.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T21:30:18.422 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T21:30:18.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T21:30:18.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T21:30:18.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T21:30:18.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T21:30:18.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T21:30:18.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-10T21:30:18.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-10T21:30:18.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T21:30:18.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T21:30:18.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T21:30:18.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T21:30:18.423 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-10T21:30:18.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-10T21:30:18.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T21:30:18.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T21:30:18.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-10T21:30:18.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T21:30:18.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-10T21:30:18.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-10T21:30:18.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T21:30:18.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-10T21:30:18.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T21:30:18.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-10T21:30:18.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T21:30:18.424 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T21:30:18.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T21:30:18.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T21:30:18.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T21:30:18.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-10T21:30:18.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-10T21:30:18.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T21:30:18.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T21:30:18.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T21:30:18.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T21:30:18.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-10T21:30:18.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T21:30:18.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-10T21:30:18.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T21:30:18.425 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T21:30:18.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-10T21:30:18.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T21:30:18.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T21:30:18.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T21:30:18.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T21:30:18.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T21:30:18.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T21:30:18.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T21:30:18.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T21:30:18.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T21:30:18.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T21:30:18.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-10T21:30:18.426 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T21:30:18.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T21:30:18.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T21:30:18.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T21:30:18.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T21:30:18.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T21:30:18.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T21:30:18.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-10T21:30:18.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T21:30:18.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-10T21:30:18.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T21:30:18.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T21:30:18.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T21:30:18.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T21:30:18.427 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T21:30:18.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T21:30:18.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T21:30:18.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T21:30:18.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T21:30:18.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T21:30:18.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T21:30:18.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T21:30:18.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T21:30:18.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T21:30:18.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T21:30:18.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T21:30:18.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T21:30:18.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T21:30:18.428 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T21:30:18.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T21:30:18.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T21:30:18.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T21:30:18.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T21:30:18.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T21:30:18.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T21:30:18.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T21:30:18.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T21:30:18.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T21:30:18.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T21:30:18.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T21:30:18.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T21:30:18.429 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T21:30:18.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T21:30:18.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T21:30:18.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T21:30:18.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T21:30:18.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T21:30:18.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T21:30:18.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T21:30:18.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T21:30:18.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T21:30:18.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T21:30:18.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T21:30:18.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T21:30:18.430 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T21:30:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-10T21:30:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T21:30:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T21:30:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T21:30:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T21:30:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T21:30:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T21:30:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T21:30:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T21:30:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T21:30:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T21:30:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T21:30:18.431 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T21:30:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T21:30:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T21:30:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T21:30:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-10T21:30:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T21:30:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T21:30:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T21:30:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-10T21:30:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-10T21:30:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T21:30:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T21:30:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T21:30:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T21:30:18.432 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-10T21:30:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T21:30:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T21:30:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T21:30:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T21:30:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-10T21:30:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-10T21:30:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T21:30:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T21:30:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T21:30:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-10T21:30:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-10T21:30:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-10T21:30:18.433 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-10T21:30:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-10T21:30:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-10T21:30:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-10T21:30:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-10T21:30:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-10T21:30:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-10T21:30:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T21:30:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T21:30:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-10T21:30:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T21:30:18.434 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T21:30:18.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T21:30:18.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T21:30:18.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T21:30:18.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T21:30:18.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T21:30:18.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T21:30:18.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T21:30:18.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T21:30:18.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T21:30:18.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T21:30:18.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-10T21:30:18.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T21:30:18.435 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T21:30:18.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T21:30:18.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T21:30:18.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T21:30:18.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T21:30:18.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T21:30:18.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T21:30:18.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T21:30:18.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T21:30:18.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T21:30:18.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T21:30:18.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T21:30:18.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T21:30:18.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T21:30:18.436 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T21:30:18.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T21:30:18.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T21:30:18.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T21:30:18.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T21:30:18.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T21:30:18.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T21:30:18.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T21:30:18.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T21:30:18.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T21:30:18.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T21:30:18.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-10T21:30:18.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-10T21:30:18.437 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T21:30:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T21:30:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T21:30:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-10T21:30:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T21:30:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T21:30:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-10T21:30:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T21:30:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-10T21:30:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-10T21:30:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T21:30:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-10T21:30:18.438 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T21:30:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-10T21:30:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T21:30:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T21:30:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-10T21:30:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T21:30:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-10T21:30:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T21:30:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-10T21:30:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T21:30:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-10T21:30:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T21:30:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-10T21:30:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T21:30:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-10T21:30:18.439 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T21:30:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-10T21:30:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-10T21:30:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-10T21:30:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T21:30:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-10T21:30:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-10T21:30:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T21:30:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-10T21:30:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-10T21:30:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T21:30:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-10T21:30:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-10T21:30:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T21:30:18.440 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-10T21:30:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T21:30:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-10T21:30:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T21:30:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-10T21:30:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T21:30:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-10T21:30:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T21:30:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-10T21:30:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T21:30:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-10T21:30:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T21:30:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-10T21:30:18.441 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T21:30:18.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-10T21:30:18.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-10T21:30:18.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T21:30:18.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-10T21:30:18.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T21:30:18.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-10T21:30:18.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T21:30:18.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T21:30:18.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T21:30:18.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T21:30:18.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-10T21:30:18.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T21:30:18.442 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T21:30:18.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-10T21:30:18.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T21:30:18.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-10T21:30:18.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T21:30:18.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-10T21:30:18.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T21:30:18.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T21:30:18.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T21:30:18.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T21:30:18.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T21:30:18.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T21:30:18.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T21:30:18.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T21:30:18.443 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T21:30:18.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T21:30:18.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T21:30:18.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T21:30:18.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T21:30:18.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T21:30:18.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T21:30:18.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T21:30:18.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T21:30:18.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T21:30:18.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T21:30:18.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T21:30:18.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T21:30:18.444 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T21:30:18.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T21:30:18.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T21:30:18.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T21:30:18.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T21:30:18.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T21:30:18.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T21:30:18.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T21:30:18.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T21:30:18.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T21:30:18.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T21:30:18.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T21:30:18.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T21:30:18.445 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T21:30:18.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T21:30:18.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T21:30:18.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T21:30:18.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T21:30:18.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T21:30:18.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T21:30:18.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T21:30:18.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T21:30:18.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T21:30:18.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T21:30:18.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T21:30:18.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T21:30:18.446 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T21:30:18.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T21:30:18.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T21:30:18.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-10T21:30:18.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-10T21:30:18.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T21:30:18.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-10T21:30:18.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T21:30:18.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-10T21:30:18.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T21:30:18.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T21:30:18.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T21:30:18.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T21:30:18.447 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T21:30:18.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-10T21:30:18.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T21:30:18.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-10T21:30:18.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T21:30:18.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-10T21:30:18.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T21:30:18.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-10T21:30:18.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T21:30:18.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-10T21:30:18.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-10T21:30:18.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T21:30:18.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-10T21:30:18.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T21:30:18.448 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T21:30:18.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-10T21:30:18.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T21:30:18.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-10T21:30:18.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T21:30:18.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-10T21:30:18.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T21:30:18.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-10T21:30:18.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T21:30:18.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-10T21:30:18.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T21:30:18.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-10T21:30:18.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-10T21:30:18.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-10T21:30:18.449 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T21:30:18.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-10T21:30:18.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-10T21:30:18.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-10T21:30:18.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-10T21:30:18.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T21:30:18.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T21:30:18.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-10T21:30:18.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-10T21:30:18.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T21:30:18.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T21:30:18.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T21:30:18.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T21:30:18.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-10T21:30:18.450 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T21:30:18.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T21:30:18.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-10T21:30:18.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T21:30:18.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-10T21:30:18.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T21:30:18.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-10T21:30:18.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T21:30:18.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T21:30:18.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T21:30:18.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T21:30:18.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-10T21:30:18.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-10T21:30:18.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-10T21:30:18.451 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T21:30:18.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-10T21:30:18.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-10T21:30:18.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T21:30:18.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T21:30:18.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T21:30:18.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T21:30:18.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T21:30:18.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T21:30:18.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T21:30:18.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T21:30:18.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-10T21:30:18.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T21:30:18.452 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T21:30:18.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T21:30:18.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T21:30:18.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T21:30:18.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-10T21:30:18.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T21:30:18.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-10T21:30:18.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T21:30:18.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T21:30:18.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T21:30:18.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T21:30:18.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-10T21:30:18.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T21:30:18.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-10T21:30:18.453 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T21:30:18.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T21:30:18.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-10T21:30:18.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T21:30:18.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T21:30:18.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T21:30:18.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T21:30:18.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T21:30:18.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T21:30:18.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T21:30:18.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T21:30:18.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T21:30:18.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T21:30:18.454 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T21:30:18.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T21:30:18.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T21:30:18.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T21:30:18.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T21:30:18.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T21:30:18.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T21:30:18.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T21:30:18.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T21:30:18.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T21:30:18.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T21:30:18.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-10T21:30:18.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T21:30:18.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T21:30:18.455 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-10T21:30:18.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T21:30:18.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-10T21:30:18.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-10T21:30:18.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T21:30:18.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-10T21:30:18.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T21:30:18.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T21:30:18.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T21:30:18.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T21:30:18.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-10T21:30:18.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-10T21:30:18.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-10T21:30:18.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T21:30:18.456 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T21:30:18.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T21:30:18.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T21:30:18.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T21:30:18.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-10T21:30:18.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-10T21:30:18.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T21:30:18.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T21:30:18.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T21:30:18.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-10T21:30:18.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T21:30:18.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T21:30:18.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T21:30:18.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T21:30:18.457 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T21:30:18.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T21:30:18.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T21:30:18.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T21:30:18.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T21:30:18.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T21:30:18.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-10T21:30:18.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T21:30:18.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T21:30:18.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T21:30:18.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T21:30:18.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T21:30:18.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T21:30:18.458 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T21:30:18.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T21:30:18.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T21:30:18.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T21:30:18.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T21:30:18.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T21:30:18.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T21:30:18.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T21:30:18.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T21:30:18.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T21:30:18.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T21:30:18.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T21:30:18.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T21:30:18.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T21:30:18.459 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T21:30:18.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T21:30:18.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T21:30:18.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T21:30:18.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T21:30:18.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T21:30:18.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T21:30:18.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T21:30:18.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T21:30:18.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T21:30:18.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T21:30:18.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T21:30:18.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T21:30:18.460 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T21:30:18.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T21:30:18.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T21:30:18.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T21:30:18.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T21:30:18.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-10T21:30:18.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T21:30:18.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T21:30:18.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T21:30:18.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T21:30:18.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T21:30:18.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T21:30:18.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T21:30:18.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-10T21:30:18.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T21:30:18.461 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T21:30:18.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T21:30:18.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T21:30:18.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T21:30:18.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T21:30:18.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T21:30:18.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T21:30:18.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T21:30:18.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T21:30:18.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T21:30:18.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T21:30:18.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T21:30:18.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T21:30:18.462 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T21:30:18.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T21:30:18.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T21:30:18.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T21:30:18.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T21:30:18.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T21:30:18.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T21:30:18.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T21:30:18.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T21:30:18.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T21:30:18.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T21:30:18.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T21:30:18.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T21:30:18.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-10T21:30:18.463 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T21:30:18.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-10T21:30:18.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T21:30:18.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T21:30:18.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T21:30:18.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T21:30:18.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T21:30:18.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T21:30:18.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T21:30:18.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-10T21:30:18.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T21:30:18.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-10T21:30:18.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T21:30:18.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-10T21:30:18.464 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T21:30:18.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-10T21:30:18.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-10T21:30:18.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T21:30:18.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T21:30:18.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T21:30:18.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T21:30:18.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T21:30:18.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T21:30:18.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T21:30:18.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T21:30:18.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T21:30:18.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T21:30:18.465 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T21:30:18.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T21:30:18.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T21:30:18.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T21:30:18.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T21:30:18.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T21:30:18.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-10T21:30:18.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T21:30:18.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T21:30:18.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T21:30:18.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T21:30:18.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T21:30:18.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T21:30:18.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T21:30:18.466 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T21:30:18.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T21:30:18.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T21:30:18.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T21:30:18.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T21:30:18.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T21:30:18.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T21:30:18.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T21:30:18.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T21:30:18.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T21:30:18.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T21:30:18.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T21:30:18.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T21:30:18.467 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T21:30:18.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T21:30:18.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T21:30:18.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T21:30:18.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T21:30:18.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T21:30:18.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T21:30:18.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T21:30:18.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T21:30:18.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T21:30:18.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T21:30:18.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T21:30:18.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T21:30:18.468 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T21:30:18.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T21:30:18.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T21:30:18.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T21:30:18.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T21:30:18.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T21:30:18.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T21:30:18.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T21:30:18.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T21:30:18.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T21:30:18.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T21:30:18.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T21:30:18.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T21:30:18.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T21:30:18.469 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T21:30:18.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T21:30:18.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T21:30:18.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T21:30:18.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T21:30:18.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T21:30:18.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T21:30:18.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T21:30:18.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T21:30:18.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T21:30:18.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T21:30:18.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-10T21:30:18.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T21:30:18.470 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T21:30:18.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T21:30:18.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T21:30:18.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-10T21:30:18.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T21:30:18.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T21:30:18.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T21:30:18.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T21:30:18.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T21:30:18.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T21:30:18.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T21:30:18.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T21:30:18.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T21:30:18.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T21:30:18.471 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T21:30:18.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T21:30:18.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T21:30:18.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T21:30:18.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T21:30:18.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T21:30:18.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T21:30:18.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T21:30:18.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T21:30:18.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T21:30:18.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T21:30:18.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T21:30:18.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T21:30:18.472 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T21:30:18.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T21:30:18.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T21:30:18.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T21:30:18.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T21:30:18.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T21:30:18.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T21:30:18.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T21:30:18.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T21:30:18.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T21:30:18.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T21:30:18.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T21:30:18.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T21:30:18.473 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T21:30:18.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T21:30:18.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T21:30:18.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T21:30:18.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T21:30:18.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T21:30:18.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T21:30:18.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T21:30:18.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T21:30:18.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T21:30:18.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T21:30:18.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T21:30:18.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T21:30:18.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T21:30:18.474 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T21:30:18.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T21:30:18.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T21:30:18.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T21:30:18.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T21:30:18.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T21:30:18.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T21:30:18.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T21:30:18.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T21:30:18.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T21:30:18.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T21:30:18.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T21:30:18.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T21:30:18.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T21:30:18.475 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T21:30:18.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T21:30:18.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T21:30:18.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T21:30:18.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T21:30:18.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T21:30:18.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T21:30:18.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-10T21:30:18.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T21:30:18.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-10T21:30:18.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T21:30:18.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-10T21:30:18.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T21:30:18.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T21:30:18.476 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T21:30:18.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T21:30:18.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-10T21:30:18.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T21:30:18.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-10T21:30:18.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T21:30:18.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T21:30:18.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T21:30:18.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-10T21:30:18.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T21:30:18.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T21:30:18.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T21:30:18.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T21:30:18.477 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T21:30:18.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T21:30:18.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T21:30:18.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T21:30:18.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T21:30:18.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T21:30:18.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T21:30:18.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T21:30:18.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T21:30:18.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T21:30:18.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T21:30:18.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T21:30:18.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T21:30:18.478 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T21:30:18.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T21:30:18.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T21:30:18.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T21:30:18.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T21:30:18.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T21:30:18.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T21:30:18.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T21:30:18.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T21:30:18.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T21:30:18.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T21:30:18.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T21:30:18.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T21:30:18.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T21:30:18.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T21:30:18.479 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T21:30:18.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T21:30:18.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T21:30:18.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T21:30:18.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T21:30:18.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T21:30:18.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T21:30:18.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T21:30:18.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T21:30:18.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T21:30:18.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T21:30:18.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-10T21:30:18.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T21:30:18.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T21:30:18.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-10T21:30:18.480 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-10T21:30:18.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T21:30:18.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T21:30:18.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T21:30:18.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-10T21:30:18.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T21:30:18.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-10T21:30:18.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T21:30:18.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T21:30:18.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-10T21:30:18.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T21:30:18.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-10T21:30:18.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T21:30:18.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T21:30:18.481 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T21:30:18.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T21:30:18.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T21:30:18.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-10T21:30:18.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T21:30:18.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-10T21:30:18.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-10T21:30:18.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-10T21:30:18.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-10T21:30:18.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-10T21:30:18.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-10T21:30:18.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T21:30:18.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T21:30:18.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T21:30:18.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T21:30:18.482 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T21:30:18.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T21:30:18.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T21:30:18.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T21:30:18.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T21:30:18.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T21:30:18.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T21:30:18.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T21:30:18.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T21:30:18.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-10T21:30:18.483 INFO:teuthology.orchestra.run.smithi007.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T21:30:18.487 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:30:18.487 DEBUG:teuthology.orchestra.run.smithi007:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-10T21:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout:postinstall scriptlet (using /bin/sh): 2024-09-10T21:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T21:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T21:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2024-09-10T21:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout:for file in vmlinuz System.map config; do 2024-09-10T21:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-10T21:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T21:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout: fi 2024-09-10T21:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout:done 2024-09-10T21:30:18.512 INFO:teuthology.orchestra.run.smithi007.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-10T21:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-10T21:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T21:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T21:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2024-09-10T21:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-10T21:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2024-09-10T21:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-10T21:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T21:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:30:18.513 INFO:teuthology.orchestra.run.smithi007.stdout:fi 2024-09-10T21:30:18.514 DEBUG:teuthology.orchestra.run.smithi007:> sudo rpm -qi grub2-tools 2024-09-10T21:30:18.590 INFO:teuthology.orchestra.run.smithi007.stdout:Name : grub2-tools 2024-09-10T21:30:18.590 INFO:teuthology.orchestra.run.smithi007.stdout:Epoch : 1 2024-09-10T21:30:18.590 INFO:teuthology.orchestra.run.smithi007.stdout:Version : 2.06 2024-09-10T21:30:18.590 INFO:teuthology.orchestra.run.smithi007.stdout:Release : 82.el9 2024-09-10T21:30:18.590 INFO:teuthology.orchestra.run.smithi007.stdout:Architecture: x86_64 2024-09-10T21:30:18.590 INFO:teuthology.orchestra.run.smithi007.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T21:30:18.590 INFO:teuthology.orchestra.run.smithi007.stdout:Group : Unspecified 2024-09-10T21:30:18.591 INFO:teuthology.orchestra.run.smithi007.stdout:Size : 8274670 2024-09-10T21:30:18.591 INFO:teuthology.orchestra.run.smithi007.stdout:License : GPLv3+ 2024-09-10T21:30:18.591 INFO:teuthology.orchestra.run.smithi007.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T21:30:18.591 INFO:teuthology.orchestra.run.smithi007.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T21:30:18.591 INFO:teuthology.orchestra.run.smithi007.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T21:30:18.591 INFO:teuthology.orchestra.run.smithi007.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T21:30:18.591 INFO:teuthology.orchestra.run.smithi007.stdout:Packager : builder@centos.org 2024-09-10T21:30:18.591 INFO:teuthology.orchestra.run.smithi007.stdout:Vendor : CentOS 2024-09-10T21:30:18.591 INFO:teuthology.orchestra.run.smithi007.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T21:30:18.591 INFO:teuthology.orchestra.run.smithi007.stdout:Summary : Support tools for GRUB. 2024-09-10T21:30:18.591 INFO:teuthology.orchestra.run.smithi007.stdout:Description : 2024-09-10T21:30:18.592 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:30:18.592 INFO:teuthology.orchestra.run.smithi007.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T21:30:18.592 INFO:teuthology.orchestra.run.smithi007.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T21:30:18.592 INFO:teuthology.orchestra.run.smithi007.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T21:30:18.592 INFO:teuthology.orchestra.run.smithi007.stdout:hardware devices. 2024-09-10T21:30:18.592 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:30:18.592 INFO:teuthology.orchestra.run.smithi007.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T21:30:18.593 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T21:30:18.593 INFO:teuthology.task.kernel:Updating grub on smithi007 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:30:18.593 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T21:30:19.233 INFO:teuthology.orchestra.run.smithi007.stderr:Generating grub configuration file ... 2024-09-10T21:30:19.871 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi133.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-10T21:30:19.871 DEBUG:teuthology.orchestra.run.smithi133:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-10T21:30:19.958 INFO:teuthology.orchestra.run.smithi133.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:30:19.958 INFO:teuthology.orchestra.run.smithi133.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:30:19.958 INFO:teuthology.orchestra.run.smithi133.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:30:19.958 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:30:19.958 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-10T21:30:19.958 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-10T21:30:19.958 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-10T21:30:19.958 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-10T21:30:19.958 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-10T21:30:19.959 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-10T21:30:19.959 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T21:30:19.959 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T21:30:19.959 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T21:30:19.959 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T21:30:19.959 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T21:30:19.959 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T21:30:19.959 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T21:30:19.959 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-10T21:30:19.959 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T21:30:19.959 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T21:30:19.959 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T21:30:19.959 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T21:30:19.960 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T21:30:19.960 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T21:30:19.960 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T21:30:19.960 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T21:30:19.960 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T21:30:19.960 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T21:30:19.960 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T21:30:19.960 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T21:30:19.960 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-10T21:30:19.960 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-10T21:30:19.960 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-10T21:30:19.960 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T21:30:19.960 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-10T21:30:19.961 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T21:30:19.961 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T21:30:19.961 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T21:30:19.961 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-10T21:30:19.961 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T21:30:19.961 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T21:30:19.961 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T21:30:19.961 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-10T21:30:19.961 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T21:30:19.961 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T21:30:19.961 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T21:30:19.961 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T21:30:19.961 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T21:30:19.961 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T21:30:19.962 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T21:30:19.962 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T21:30:19.962 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T21:30:19.962 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T21:30:19.962 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T21:30:19.962 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T21:30:19.962 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T21:30:19.962 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T21:30:19.962 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T21:30:19.962 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T21:30:19.962 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T21:30:19.962 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T21:30:19.962 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T21:30:19.963 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T21:30:19.963 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T21:30:19.963 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T21:30:19.963 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T21:30:19.963 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-10T21:30:19.963 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-10T21:30:19.963 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T21:30:19.963 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T21:30:19.963 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T21:30:19.963 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T21:30:19.963 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T21:30:19.963 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T21:30:19.963 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T21:30:19.963 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T21:30:19.964 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T21:30:19.964 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T21:30:19.964 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T21:30:19.964 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T21:30:19.964 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T21:30:19.964 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T21:30:19.964 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T21:30:19.964 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T21:30:19.964 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T21:30:19.964 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T21:30:19.964 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T21:30:19.964 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T21:30:19.964 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T21:30:19.964 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T21:30:19.965 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-10T21:30:19.965 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-10T21:30:19.965 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T21:30:19.965 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T21:30:19.965 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T21:30:19.965 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T21:30:19.965 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T21:30:19.965 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T21:30:19.965 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T21:30:19.965 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-10T21:30:19.965 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T21:30:19.965 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T21:30:19.965 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-10T21:30:19.965 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T21:30:19.966 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T21:30:19.966 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T21:30:19.966 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T21:30:19.966 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T21:30:19.966 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T21:30:19.966 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T21:30:19.966 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T21:30:19.966 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T21:30:19.966 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T21:30:19.966 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T21:30:19.966 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T21:30:19.966 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T21:30:19.966 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T21:30:19.966 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T21:30:19.967 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T21:30:19.967 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T21:30:19.967 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T21:30:19.967 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T21:30:19.967 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T21:30:19.967 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T21:30:19.967 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T21:30:19.967 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T21:30:19.967 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T21:30:19.967 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T21:30:19.967 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T21:30:19.967 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T21:30:19.968 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T21:30:19.968 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T21:30:19.968 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T21:30:19.968 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T21:30:19.968 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T21:30:19.968 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T21:30:19.968 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T21:30:19.968 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T21:30:19.968 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-10T21:30:19.968 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T21:30:19.968 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T21:30:19.968 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T21:30:19.968 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T21:30:19.968 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T21:30:19.969 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T21:30:19.969 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T21:30:19.969 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T21:30:19.969 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T21:30:19.969 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T21:30:19.969 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T21:30:19.969 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T21:30:19.969 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T21:30:19.969 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T21:30:19.969 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T21:30:19.969 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T21:30:19.969 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T21:30:19.969 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-10T21:30:19.969 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T21:30:19.970 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-10T21:30:19.970 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-10T21:30:19.970 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-10T21:30:19.970 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-10T21:30:19.970 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-10T21:30:19.970 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T21:30:19.970 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-10T21:30:19.970 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-10T21:30:19.970 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T21:30:19.970 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-10T21:30:19.970 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-10T21:30:19.970 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T21:30:19.970 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T21:30:19.970 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-10T21:30:19.971 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-10T21:30:19.971 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T21:30:19.971 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T21:30:19.971 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-10T21:30:19.971 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T21:30:19.971 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T21:30:19.971 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T21:30:19.971 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-10T21:30:19.971 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-10T21:30:19.971 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T21:30:19.971 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-10T21:30:19.971 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-10T21:30:19.971 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T21:30:19.971 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-10T21:30:19.972 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T21:30:19.972 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-10T21:30:19.972 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T21:30:19.972 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T21:30:19.972 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T21:30:19.972 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T21:30:19.972 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T21:30:19.972 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-10T21:30:19.972 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T21:30:19.972 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T21:30:19.972 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T21:30:19.972 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T21:30:19.972 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T21:30:19.972 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T21:30:19.973 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T21:30:19.973 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-10T21:30:19.973 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T21:30:19.973 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T21:30:19.973 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T21:30:19.973 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-10T21:30:19.973 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-10T21:30:19.973 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T21:30:19.973 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T21:30:19.973 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T21:30:19.973 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-10T21:30:19.973 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-10T21:30:19.973 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-10T21:30:19.973 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-10T21:30:19.974 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-10T21:30:19.974 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-10T21:30:19.974 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T21:30:19.974 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T21:30:19.974 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-10T21:30:19.974 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-10T21:30:19.974 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T21:30:19.974 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-10T21:30:19.974 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-10T21:30:19.974 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T21:30:19.974 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T21:30:19.974 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-10T21:30:19.974 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-10T21:30:19.974 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-10T21:30:19.975 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T21:30:19.975 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-10T21:30:19.975 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T21:30:19.975 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T21:30:19.975 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T21:30:19.975 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-10T21:30:19.975 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T21:30:19.975 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-10T21:30:19.975 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T21:30:19.975 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-10T21:30:19.975 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-10T21:30:19.975 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-10T21:30:19.975 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-10T21:30:19.976 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T21:30:19.976 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-10T21:30:19.976 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T21:30:19.976 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-10T21:30:19.976 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T21:30:19.976 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T21:30:19.976 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T21:30:19.976 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T21:30:19.976 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T21:30:19.976 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T21:30:19.976 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T21:30:19.976 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T21:30:19.976 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T21:30:19.977 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T21:30:19.977 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T21:30:19.977 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T21:30:19.977 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T21:30:19.977 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-10T21:30:19.977 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-10T21:30:19.977 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T21:30:19.977 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-10T21:30:19.977 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T21:30:19.977 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T21:30:19.977 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-10T21:30:19.977 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T21:30:19.977 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T21:30:19.978 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T21:30:19.978 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T21:30:19.978 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T21:30:19.978 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T21:30:19.978 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T21:30:19.978 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T21:30:19.978 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T21:30:19.978 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T21:30:19.978 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T21:30:19.978 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T21:30:19.978 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T21:30:19.978 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T21:30:19.979 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T21:30:19.979 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T21:30:19.979 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T21:30:19.979 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T21:30:19.979 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T21:30:19.979 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T21:30:19.979 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T21:30:19.979 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T21:30:19.979 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T21:30:19.979 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T21:30:19.979 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T21:30:19.979 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T21:30:19.979 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T21:30:19.979 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T21:30:19.979 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T21:30:19.980 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T21:30:19.980 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T21:30:19.980 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T21:30:19.980 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T21:30:19.980 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T21:30:19.980 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T21:30:19.980 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T21:30:19.980 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T21:30:19.980 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T21:30:19.980 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T21:30:19.980 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T21:30:19.980 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T21:30:19.980 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T21:30:19.981 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T21:30:19.981 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T21:30:19.981 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T21:30:19.981 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T21:30:19.981 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T21:30:19.981 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T21:30:19.981 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T21:30:19.981 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T21:30:19.981 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T21:30:19.981 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T21:30:19.981 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T21:30:19.981 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T21:30:19.981 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T21:30:19.981 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T21:30:19.982 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T21:30:19.982 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T21:30:19.982 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T21:30:19.982 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T21:30:19.982 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T21:30:19.982 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T21:30:19.982 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T21:30:19.982 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T21:30:19.982 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-10T21:30:19.982 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T21:30:19.982 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T21:30:19.982 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T21:30:19.982 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-10T21:30:19.982 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T21:30:19.983 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T21:30:19.983 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T21:30:19.983 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T21:30:19.983 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T21:30:19.983 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T21:30:19.983 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T21:30:19.983 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T21:30:19.983 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T21:30:19.983 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T21:30:19.983 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T21:30:19.983 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T21:30:19.983 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T21:30:19.983 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T21:30:19.984 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T21:30:19.984 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T21:30:19.984 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T21:30:19.984 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T21:30:19.984 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T21:30:19.984 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T21:30:19.984 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T21:30:19.984 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T21:30:19.984 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T21:30:19.984 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T21:30:19.984 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T21:30:19.984 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T21:30:19.984 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T21:30:19.984 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T21:30:19.985 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T21:30:19.985 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T21:30:19.985 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T21:30:19.985 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T21:30:19.985 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T21:30:19.985 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T21:30:19.985 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T21:30:19.985 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T21:30:19.985 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T21:30:19.985 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T21:30:19.985 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T21:30:19.985 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T21:30:19.985 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T21:30:19.985 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T21:30:19.986 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T21:30:19.986 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T21:30:19.986 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T21:30:19.986 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T21:30:19.986 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T21:30:19.986 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T21:30:19.986 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T21:30:19.986 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T21:30:19.986 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T21:30:19.986 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T21:30:19.986 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T21:30:19.986 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T21:30:19.986 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T21:30:19.986 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T21:30:19.987 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T21:30:19.987 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T21:30:19.987 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T21:30:19.987 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T21:30:19.987 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T21:30:19.987 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T21:30:19.987 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T21:30:19.987 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T21:30:19.987 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T21:30:19.987 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T21:30:19.987 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T21:30:19.987 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T21:30:19.987 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T21:30:19.987 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T21:30:19.988 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T21:30:19.988 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T21:30:19.988 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T21:30:19.988 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T21:30:19.988 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T21:30:19.988 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T21:30:19.988 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T21:30:19.988 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T21:30:19.988 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T21:30:19.988 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T21:30:19.988 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T21:30:19.988 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T21:30:19.988 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T21:30:19.989 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T21:30:19.989 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T21:30:19.989 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T21:30:19.989 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T21:30:19.989 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T21:30:19.989 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-10T21:30:19.989 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T21:30:19.989 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T21:30:19.989 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T21:30:19.989 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T21:30:19.989 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T21:30:19.989 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T21:30:19.989 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T21:30:19.989 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T21:30:19.990 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T21:30:19.990 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T21:30:19.990 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T21:30:19.990 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T21:30:19.990 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T21:30:19.990 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T21:30:19.990 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T21:30:19.990 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T21:30:19.990 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T21:30:19.990 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T21:30:19.990 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T21:30:19.990 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T21:30:19.990 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T21:30:19.990 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T21:30:19.990 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T21:30:19.991 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T21:30:19.991 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T21:30:19.991 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T21:30:19.991 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T21:30:19.991 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T21:30:19.991 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T21:30:19.991 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T21:30:19.991 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T21:30:19.991 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T21:30:19.991 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T21:30:19.991 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T21:30:19.991 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T21:30:19.991 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T21:30:19.992 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T21:30:19.992 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T21:30:19.992 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T21:30:19.992 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T21:30:19.992 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T21:30:19.992 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-10T21:30:19.992 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-10T21:30:19.992 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T21:30:19.992 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T21:30:19.992 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-10T21:30:19.992 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T21:30:19.992 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T21:30:19.992 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T21:30:19.993 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T21:30:19.993 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T21:30:19.993 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T21:30:19.993 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T21:30:19.993 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T21:30:19.993 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T21:30:19.993 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T21:30:19.993 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T21:30:19.993 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T21:30:19.993 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T21:30:19.993 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T21:30:19.993 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T21:30:19.993 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T21:30:19.993 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T21:30:19.994 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T21:30:19.994 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T21:30:19.994 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T21:30:19.994 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T21:30:19.994 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T21:30:19.994 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T21:30:19.994 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-10T21:30:19.994 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T21:30:19.994 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-10T21:30:19.994 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T21:30:19.994 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T21:30:19.994 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-10T21:30:19.994 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-10T21:30:19.995 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T21:30:19.995 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T21:30:19.995 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T21:30:19.995 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T21:30:19.995 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T21:30:19.995 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T21:30:19.995 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T21:30:19.995 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-10T21:30:19.995 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-10T21:30:19.995 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T21:30:19.995 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-10T21:30:19.995 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T21:30:19.995 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-10T21:30:19.995 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T21:30:19.996 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-10T21:30:19.996 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T21:30:19.996 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-10T21:30:19.996 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-10T21:30:19.996 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T21:30:19.996 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-10T21:30:19.996 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T21:30:19.996 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-10T21:30:19.996 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T21:30:19.996 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-10T21:30:19.996 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T21:30:19.996 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-10T21:30:19.996 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T21:30:19.996 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-10T21:30:19.997 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-10T21:30:19.997 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T21:30:19.997 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T21:30:19.997 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T21:30:19.997 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T21:30:19.997 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-10T21:30:19.997 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T21:30:19.997 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-10T21:30:19.997 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T21:30:19.997 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T21:30:19.997 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T21:30:19.997 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T21:30:19.997 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T21:30:19.997 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T21:30:19.998 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T21:30:19.998 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T21:30:19.998 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-10T21:30:19.998 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T21:30:19.998 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-10T21:30:19.998 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T21:30:19.998 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T21:30:19.998 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T21:30:19.998 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T21:30:19.998 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T21:30:19.998 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-10T21:30:19.998 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T21:30:19.998 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T21:30:19.998 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T21:30:19.999 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T21:30:19.999 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-10T21:30:19.999 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T21:30:19.999 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T21:30:19.999 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T21:30:19.999 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T21:30:19.999 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T21:30:19.999 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T21:30:19.999 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T21:30:19.999 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T21:30:19.999 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-10T21:30:19.999 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-10T21:30:19.999 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T21:30:19.999 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T21:30:19.999 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T21:30:20.000 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T21:30:20.000 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T21:30:20.000 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T21:30:20.000 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T21:30:20.000 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T21:30:20.000 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T21:30:20.000 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T21:30:20.000 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T21:30:20.000 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T21:30:20.000 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T21:30:20.000 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T21:30:20.000 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T21:30:20.000 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T21:30:20.001 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-10T21:30:20.001 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T21:30:20.001 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T21:30:20.001 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T21:30:20.001 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T21:30:20.001 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T21:30:20.001 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-10T21:30:20.001 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-10T21:30:20.001 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T21:30:20.001 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T21:30:20.001 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T21:30:20.001 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T21:30:20.001 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T21:30:20.002 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T21:30:20.002 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T21:30:20.002 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-10T21:30:20.002 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T21:30:20.002 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T21:30:20.002 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-10T21:30:20.002 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T21:30:20.002 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T21:30:20.002 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T21:30:20.002 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T21:30:20.002 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-10T21:30:20.002 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-10T21:30:20.002 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T21:30:20.002 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T21:30:20.003 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T21:30:20.003 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T21:30:20.003 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-10T21:30:20.003 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T21:30:20.003 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-10T21:30:20.003 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T21:30:20.003 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T21:30:20.003 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T21:30:20.003 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T21:30:20.003 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T21:30:20.003 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T21:30:20.003 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T21:30:20.003 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-10T21:30:20.003 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T21:30:20.004 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T21:30:20.004 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-10T21:30:20.004 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T21:30:20.004 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T21:30:20.004 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T21:30:20.004 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-10T21:30:20.004 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-10T21:30:20.004 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-10T21:30:20.004 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-10T21:30:20.004 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-10T21:30:20.004 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-10T21:30:20.004 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T21:30:20.004 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T21:30:20.004 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T21:30:20.005 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T21:30:20.005 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-10T21:30:20.005 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T21:30:20.005 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-10T21:30:20.005 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-10T21:30:20.005 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T21:30:20.005 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-10T21:30:20.005 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-10T21:30:20.005 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T21:30:20.005 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T21:30:20.005 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T21:30:20.005 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-10T21:30:20.005 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T21:30:20.005 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T21:30:20.006 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T21:30:20.006 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-10T21:30:20.006 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-10T21:30:20.006 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T21:30:20.006 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T21:30:20.006 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T21:30:20.006 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T21:30:20.006 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-10T21:30:20.006 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T21:30:20.006 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T21:30:20.006 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T21:30:20.006 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T21:30:20.006 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-10T21:30:20.006 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T21:30:20.007 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-10T21:30:20.007 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T21:30:20.007 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-10T21:30:20.007 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T21:30:20.007 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-10T21:30:20.007 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T21:30:20.007 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-10T21:30:20.007 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T21:30:20.007 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T21:30:20.007 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T21:30:20.007 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-10T21:30:20.007 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-10T21:30:20.007 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T21:30:20.007 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-10T21:30:20.008 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T21:30:20.008 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-10T21:30:20.008 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T21:30:20.008 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-10T21:30:20.008 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T21:30:20.008 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T21:30:20.008 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-10T21:30:20.008 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T21:30:20.008 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T21:30:20.008 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-10T21:30:20.008 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T21:30:20.008 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T21:30:20.008 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-10T21:30:20.008 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T21:30:20.009 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T21:30:20.009 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-10T21:30:20.009 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-10T21:30:20.009 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T21:30:20.009 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-10T21:30:20.009 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-10T21:30:20.009 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T21:30:20.009 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-10T21:30:20.009 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T21:30:20.009 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-10T21:30:20.009 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T21:30:20.009 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-10T21:30:20.009 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T21:30:20.010 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-10T21:30:20.010 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T21:30:20.010 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-10T21:30:20.010 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-10T21:30:20.010 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T21:30:20.010 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-10T21:30:20.010 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-10T21:30:20.010 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T21:30:20.010 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T21:30:20.010 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T21:30:20.010 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T21:30:20.010 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T21:30:20.010 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T21:30:20.010 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-10T21:30:20.011 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T21:30:20.011 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T21:30:20.011 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T21:30:20.011 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-10T21:30:20.011 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-10T21:30:20.011 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T21:30:20.011 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T21:30:20.011 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T21:30:20.011 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-10T21:30:20.011 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T21:30:20.011 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-10T21:30:20.011 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T21:30:20.011 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-10T21:30:20.011 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T21:30:20.012 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-10T21:30:20.012 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T21:30:20.012 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-10T21:30:20.012 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T21:30:20.012 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-10T21:30:20.012 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T21:30:20.012 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-10T21:30:20.012 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T21:30:20.012 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-10T21:30:20.012 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T21:30:20.012 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-10T21:30:20.012 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T21:30:20.012 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-10T21:30:20.012 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T21:30:20.013 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-10T21:30:20.013 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T21:30:20.013 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-10T21:30:20.013 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T21:30:20.013 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T21:30:20.013 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-10T21:30:20.013 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T21:30:20.013 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T21:30:20.013 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T21:30:20.013 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-10T21:30:20.013 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-10T21:30:20.013 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T21:30:20.013 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T21:30:20.013 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-10T21:30:20.014 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-10T21:30:20.014 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T21:30:20.014 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-10T21:30:20.014 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T21:30:20.014 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-10T21:30:20.014 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-10T21:30:20.014 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T21:30:20.014 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-10T21:30:20.014 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T21:30:20.014 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T21:30:20.014 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-10T21:30:20.014 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T21:30:20.014 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-10T21:30:20.014 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T21:30:20.015 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-10T21:30:20.015 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T21:30:20.015 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T21:30:20.015 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-10T21:30:20.015 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-10T21:30:20.015 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T21:30:20.015 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T21:30:20.015 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-10T21:30:20.015 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T21:30:20.015 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-10T21:30:20.015 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T21:30:20.015 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-10T21:30:20.015 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T21:30:20.015 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T21:30:20.016 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T21:30:20.016 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T21:30:20.016 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-10T21:30:20.016 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-10T21:30:20.016 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-10T21:30:20.016 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T21:30:20.016 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-10T21:30:20.016 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T21:30:20.016 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-10T21:30:20.016 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T21:30:20.016 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T21:30:20.016 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-10T21:30:20.016 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T21:30:20.016 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T21:30:20.017 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T21:30:20.017 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T21:30:20.017 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-10T21:30:20.017 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-10T21:30:20.017 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T21:30:20.017 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T21:30:20.017 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-10T21:30:20.017 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T21:30:20.017 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T21:30:20.017 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T21:30:20.017 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T21:30:20.017 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-10T21:30:20.017 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T21:30:20.018 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-10T21:30:20.018 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T21:30:20.018 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-10T21:30:20.018 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T21:30:20.018 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T21:30:20.018 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-10T21:30:20.018 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T21:30:20.018 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T21:30:20.018 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-10T21:30:20.018 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T21:30:20.018 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T21:30:20.018 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-10T21:30:20.018 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T21:30:20.018 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-10T21:30:20.019 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T21:30:20.019 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T21:30:20.019 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-10T21:30:20.019 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T21:30:20.019 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T21:30:20.019 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T21:30:20.019 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-10T21:30:20.019 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T21:30:20.019 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T21:30:20.019 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T21:30:20.019 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T21:30:20.019 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T21:30:20.019 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-10T21:30:20.020 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T21:30:20.020 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T21:30:20.020 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-10T21:30:20.020 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T21:30:20.020 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T21:30:20.020 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-10T21:30:20.020 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T21:30:20.020 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T21:30:20.020 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-10T21:30:20.020 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T21:30:20.020 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T21:30:20.020 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T21:30:20.020 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T21:30:20.020 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T21:30:20.021 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-10T21:30:20.021 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T21:30:20.021 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T21:30:20.021 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T21:30:20.021 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T21:30:20.021 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T21:30:20.021 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T21:30:20.021 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T21:30:20.021 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T21:30:20.022 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T21:30:20.022 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T21:30:20.022 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T21:30:20.022 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T21:30:20.022 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-10T21:30:20.022 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T21:30:20.022 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T21:30:20.022 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T21:30:20.022 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-10T21:30:20.022 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T21:30:20.022 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T21:30:20.022 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T21:30:20.022 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T21:30:20.022 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T21:30:20.022 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-10T21:30:20.023 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-10T21:30:20.023 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T21:30:20.023 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T21:30:20.023 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T21:30:20.023 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-10T21:30:20.023 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T21:30:20.023 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T21:30:20.023 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T21:30:20.023 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T21:30:20.023 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T21:30:20.023 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T21:30:20.023 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-10T21:30:20.023 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T21:30:20.024 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-10T21:30:20.024 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-10T21:30:20.024 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T21:30:20.024 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-10T21:30:20.024 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T21:30:20.024 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-10T21:30:20.024 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T21:30:20.024 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-10T21:30:20.024 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-10T21:30:20.024 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T21:30:20.024 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-10T21:30:20.024 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-10T21:30:20.024 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T21:30:20.024 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T21:30:20.025 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-10T21:30:20.025 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T21:30:20.025 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T21:30:20.025 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T21:30:20.025 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T21:30:20.025 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-10T21:30:20.025 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-10T21:30:20.025 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T21:30:20.025 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-10T21:30:20.025 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-10T21:30:20.025 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T21:30:20.025 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-10T21:30:20.025 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T21:30:20.025 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T21:30:20.026 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T21:30:20.026 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-10T21:30:20.026 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-10T21:30:20.026 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T21:30:20.026 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-10T21:30:20.026 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-10T21:30:20.026 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T21:30:20.026 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T21:30:20.026 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T21:30:20.026 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T21:30:20.026 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T21:30:20.026 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T21:30:20.026 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T21:30:20.027 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T21:30:20.027 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T21:30:20.027 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T21:30:20.027 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T21:30:20.027 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T21:30:20.027 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-10T21:30:20.027 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T21:30:20.027 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T21:30:20.027 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T21:30:20.027 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T21:30:20.027 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T21:30:20.027 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T21:30:20.027 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T21:30:20.027 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T21:30:20.028 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T21:30:20.028 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T21:30:20.028 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T21:30:20.028 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T21:30:20.028 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-10T21:30:20.028 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T21:30:20.028 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T21:30:20.028 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-10T21:30:20.028 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-10T21:30:20.028 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T21:30:20.028 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T21:30:20.028 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T21:30:20.028 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T21:30:20.029 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-10T21:30:20.029 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T21:30:20.029 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T21:30:20.029 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-10T21:30:20.029 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-10T21:30:20.029 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-10T21:30:20.029 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T21:30:20.029 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T21:30:20.029 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-10T21:30:20.029 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T21:30:20.029 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T21:30:20.029 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T21:30:20.029 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T21:30:20.029 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T21:30:20.030 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T21:30:20.030 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T21:30:20.030 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T21:30:20.030 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T21:30:20.030 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T21:30:20.030 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T21:30:20.030 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T21:30:20.030 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T21:30:20.030 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T21:30:20.030 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T21:30:20.030 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T21:30:20.030 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T21:30:20.030 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T21:30:20.030 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T21:30:20.031 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T21:30:20.031 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T21:30:20.031 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T21:30:20.031 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T21:30:20.031 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T21:30:20.031 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T21:30:20.031 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T21:30:20.031 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T21:30:20.031 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T21:30:20.031 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T21:30:20.031 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T21:30:20.031 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T21:30:20.031 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-10T21:30:20.031 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T21:30:20.032 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T21:30:20.032 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T21:30:20.032 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T21:30:20.032 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T21:30:20.032 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-10T21:30:20.032 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T21:30:20.032 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T21:30:20.032 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-10T21:30:20.032 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T21:30:20.032 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T21:30:20.032 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T21:30:20.032 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-10T21:30:20.033 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T21:30:20.033 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T21:30:20.033 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-10T21:30:20.033 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T21:30:20.033 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-10T21:30:20.033 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T21:30:20.033 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-10T21:30:20.033 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T21:30:20.033 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-10T21:30:20.033 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T21:30:20.033 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T21:30:20.033 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-10T21:30:20.033 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T21:30:20.033 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-10T21:30:20.034 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-10T21:30:20.034 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T21:30:20.034 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-10T21:30:20.034 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T21:30:20.034 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T21:30:20.034 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T21:30:20.034 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-10T21:30:20.034 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-10T21:30:20.034 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-10T21:30:20.034 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-10T21:30:20.034 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-10T21:30:20.034 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T21:30:20.034 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-10T21:30:20.034 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T21:30:20.035 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T21:30:20.035 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-10T21:30:20.035 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T21:30:20.035 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T21:30:20.035 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-10T21:30:20.035 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T21:30:20.035 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T21:30:20.035 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T21:30:20.035 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T21:30:20.035 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T21:30:20.035 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T21:30:20.035 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T21:30:20.035 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-10T21:30:20.035 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T21:30:20.036 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T21:30:20.036 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T21:30:20.036 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-10T21:30:20.036 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T21:30:20.036 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T21:30:20.036 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T21:30:20.036 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-10T21:30:20.036 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T21:30:20.036 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-10T21:30:20.036 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T21:30:20.036 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-10T21:30:20.036 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T21:30:20.036 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T21:30:20.037 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T21:30:20.037 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T21:30:20.037 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-10T21:30:20.037 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T21:30:20.037 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-10T21:30:20.037 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T21:30:20.037 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T21:30:20.037 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-10T21:30:20.037 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T21:30:20.037 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T21:30:20.037 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T21:30:20.037 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-10T21:30:20.037 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T21:30:20.037 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T21:30:20.038 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-10T21:30:20.038 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T21:30:20.038 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T21:30:20.038 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T21:30:20.038 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T21:30:20.038 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T21:30:20.038 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T21:30:20.038 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T21:30:20.038 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T21:30:20.038 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T21:30:20.038 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T21:30:20.038 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T21:30:20.039 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-10T21:30:20.039 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T21:30:20.039 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T21:30:20.039 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T21:30:20.039 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T21:30:20.039 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-10T21:30:20.039 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T21:30:20.039 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-10T21:30:20.039 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-10T21:30:20.039 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T21:30:20.039 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-10T21:30:20.039 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T21:30:20.040 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T21:30:20.040 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T21:30:20.040 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T21:30:20.040 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T21:30:20.040 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T21:30:20.040 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-10T21:30:20.040 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T21:30:20.040 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-10T21:30:20.040 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-10T21:30:20.040 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-10T21:30:20.040 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T21:30:20.040 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T21:30:20.040 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T21:30:20.041 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T21:30:20.041 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T21:30:20.041 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T21:30:20.041 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T21:30:20.041 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T21:30:20.041 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T21:30:20.041 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T21:30:20.041 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T21:30:20.041 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T21:30:20.041 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T21:30:20.041 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T21:30:20.041 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T21:30:20.041 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T21:30:20.042 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T21:30:20.042 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T21:30:20.042 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-10T21:30:20.042 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-10T21:30:20.042 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T21:30:20.042 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T21:30:20.042 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T21:30:20.042 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T21:30:20.042 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-10T21:30:20.042 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-10T21:30:20.042 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T21:30:20.042 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T21:30:20.042 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-10T21:30:20.042 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T21:30:20.043 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-10T21:30:20.043 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-10T21:30:20.043 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T21:30:20.043 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-10T21:30:20.043 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T21:30:20.043 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-10T21:30:20.043 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T21:30:20.043 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T21:30:20.043 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T21:30:20.043 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T21:30:20.043 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T21:30:20.043 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-10T21:30:20.043 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-10T21:30:20.043 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T21:30:20.044 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T21:30:20.044 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T21:30:20.044 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T21:30:20.044 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-10T21:30:20.044 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T21:30:20.044 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-10T21:30:20.044 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T21:30:20.044 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T21:30:20.044 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-10T21:30:20.044 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T21:30:20.044 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T21:30:20.044 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T21:30:20.044 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T21:30:20.045 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T21:30:20.045 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T21:30:20.045 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T21:30:20.045 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T21:30:20.045 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T21:30:20.045 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T21:30:20.045 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-10T21:30:20.045 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T21:30:20.045 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T21:30:20.045 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T21:30:20.045 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T21:30:20.045 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T21:30:20.045 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T21:30:20.046 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T21:30:20.046 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-10T21:30:20.046 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T21:30:20.046 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-10T21:30:20.046 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T21:30:20.046 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T21:30:20.046 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T21:30:20.046 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T21:30:20.046 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T21:30:20.046 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T21:30:20.046 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T21:30:20.046 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T21:30:20.046 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T21:30:20.046 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T21:30:20.047 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T21:30:20.047 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T21:30:20.047 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T21:30:20.047 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T21:30:20.047 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T21:30:20.047 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T21:30:20.047 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T21:30:20.047 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T21:30:20.047 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T21:30:20.047 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T21:30:20.047 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T21:30:20.047 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T21:30:20.047 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T21:30:20.047 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T21:30:20.048 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T21:30:20.048 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T21:30:20.048 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T21:30:20.048 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T21:30:20.048 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T21:30:20.048 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T21:30:20.048 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T21:30:20.048 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T21:30:20.048 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T21:30:20.048 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T21:30:20.048 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T21:30:20.048 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T21:30:20.048 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T21:30:20.049 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T21:30:20.049 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T21:30:20.049 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T21:30:20.049 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T21:30:20.049 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T21:30:20.049 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T21:30:20.049 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T21:30:20.049 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T21:30:20.049 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-10T21:30:20.049 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T21:30:20.049 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T21:30:20.049 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T21:30:20.049 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T21:30:20.049 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T21:30:20.050 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T21:30:20.050 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T21:30:20.050 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T21:30:20.050 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T21:30:20.050 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T21:30:20.050 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T21:30:20.050 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T21:30:20.050 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T21:30:20.050 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T21:30:20.050 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T21:30:20.050 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-10T21:30:20.050 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T21:30:20.050 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T21:30:20.050 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T21:30:20.051 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-10T21:30:20.051 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-10T21:30:20.051 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T21:30:20.051 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T21:30:20.051 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T21:30:20.051 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T21:30:20.051 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-10T21:30:20.051 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T21:30:20.051 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T21:30:20.051 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T21:30:20.051 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T21:30:20.051 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-10T21:30:20.051 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-10T21:30:20.051 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T21:30:20.052 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T21:30:20.052 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T21:30:20.052 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-10T21:30:20.052 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-10T21:30:20.052 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-10T21:30:20.052 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-10T21:30:20.052 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-10T21:30:20.052 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-10T21:30:20.052 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-10T21:30:20.052 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-10T21:30:20.052 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-10T21:30:20.052 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-10T21:30:20.052 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T21:30:20.052 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T21:30:20.053 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-10T21:30:20.053 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T21:30:20.053 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T21:30:20.053 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T21:30:20.053 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T21:30:20.053 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T21:30:20.053 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T21:30:20.053 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T21:30:20.053 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T21:30:20.053 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T21:30:20.053 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T21:30:20.053 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T21:30:20.053 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T21:30:20.054 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-10T21:30:20.054 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T21:30:20.054 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T21:30:20.054 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T21:30:20.054 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T21:30:20.054 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T21:30:20.054 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T21:30:20.054 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T21:30:20.054 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T21:30:20.054 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T21:30:20.054 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T21:30:20.054 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T21:30:20.054 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T21:30:20.054 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T21:30:20.055 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T21:30:20.055 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T21:30:20.055 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T21:30:20.055 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T21:30:20.055 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T21:30:20.055 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T21:30:20.055 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T21:30:20.055 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T21:30:20.055 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T21:30:20.055 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T21:30:20.055 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T21:30:20.055 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T21:30:20.055 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T21:30:20.056 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-10T21:30:20.056 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-10T21:30:20.056 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T21:30:20.056 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T21:30:20.056 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T21:30:20.056 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-10T21:30:20.056 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T21:30:20.056 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T21:30:20.056 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-10T21:30:20.056 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T21:30:20.056 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-10T21:30:20.056 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-10T21:30:20.056 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T21:30:20.056 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-10T21:30:20.057 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T21:30:20.057 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-10T21:30:20.057 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T21:30:20.057 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T21:30:20.057 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-10T21:30:20.057 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T21:30:20.057 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-10T21:30:20.057 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T21:30:20.057 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-10T21:30:20.057 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T21:30:20.057 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-10T21:30:20.057 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T21:30:20.057 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-10T21:30:20.058 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T21:30:20.058 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-10T21:30:20.058 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T21:30:20.058 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-10T21:30:20.058 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-10T21:30:20.058 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-10T21:30:20.058 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T21:30:20.058 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-10T21:30:20.058 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-10T21:30:20.058 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T21:30:20.058 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-10T21:30:20.058 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-10T21:30:20.058 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T21:30:20.058 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-10T21:30:20.059 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-10T21:30:20.059 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T21:30:20.059 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-10T21:30:20.059 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T21:30:20.059 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-10T21:30:20.059 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T21:30:20.059 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-10T21:30:20.059 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T21:30:20.059 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-10T21:30:20.059 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T21:30:20.059 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-10T21:30:20.059 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T21:30:20.059 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-10T21:30:20.059 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T21:30:20.060 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-10T21:30:20.060 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T21:30:20.060 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-10T21:30:20.060 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-10T21:30:20.060 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T21:30:20.060 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-10T21:30:20.060 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T21:30:20.060 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-10T21:30:20.060 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T21:30:20.060 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T21:30:20.060 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T21:30:20.060 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T21:30:20.060 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-10T21:30:20.061 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T21:30:20.061 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T21:30:20.061 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-10T21:30:20.061 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T21:30:20.061 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-10T21:30:20.061 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T21:30:20.061 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-10T21:30:20.061 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T21:30:20.061 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T21:30:20.061 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T21:30:20.061 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T21:30:20.061 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T21:30:20.061 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T21:30:20.061 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T21:30:20.062 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T21:30:20.062 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T21:30:20.062 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T21:30:20.062 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T21:30:20.062 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T21:30:20.062 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T21:30:20.062 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T21:30:20.062 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T21:30:20.062 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T21:30:20.062 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T21:30:20.062 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T21:30:20.062 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T21:30:20.062 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T21:30:20.062 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T21:30:20.063 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T21:30:20.063 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T21:30:20.063 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T21:30:20.063 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T21:30:20.063 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T21:30:20.063 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T21:30:20.063 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T21:30:20.063 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T21:30:20.063 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T21:30:20.063 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T21:30:20.063 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T21:30:20.063 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T21:30:20.063 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T21:30:20.063 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T21:30:20.064 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T21:30:20.064 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T21:30:20.064 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T21:30:20.064 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T21:30:20.064 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T21:30:20.064 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T21:30:20.064 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T21:30:20.064 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T21:30:20.064 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T21:30:20.064 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T21:30:20.064 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T21:30:20.064 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T21:30:20.064 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T21:30:20.065 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T21:30:20.065 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T21:30:20.065 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-10T21:30:20.065 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-10T21:30:20.065 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T21:30:20.065 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-10T21:30:20.065 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T21:30:20.065 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-10T21:30:20.065 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T21:30:20.065 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T21:30:20.065 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T21:30:20.065 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T21:30:20.065 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T21:30:20.065 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-10T21:30:20.066 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T21:30:20.066 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-10T21:30:20.066 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T21:30:20.066 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-10T21:30:20.066 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T21:30:20.066 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-10T21:30:20.066 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T21:30:20.066 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-10T21:30:20.066 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-10T21:30:20.066 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T21:30:20.066 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-10T21:30:20.066 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T21:30:20.066 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T21:30:20.066 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-10T21:30:20.067 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T21:30:20.067 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-10T21:30:20.067 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T21:30:20.067 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-10T21:30:20.067 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T21:30:20.067 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-10T21:30:20.067 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T21:30:20.067 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-10T21:30:20.067 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T21:30:20.067 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-10T21:30:20.067 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-10T21:30:20.067 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-10T21:30:20.067 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T21:30:20.067 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-10T21:30:20.068 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-10T21:30:20.068 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-10T21:30:20.068 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-10T21:30:20.068 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T21:30:20.068 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T21:30:20.068 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-10T21:30:20.068 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-10T21:30:20.068 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T21:30:20.068 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T21:30:20.068 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T21:30:20.068 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T21:30:20.068 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-10T21:30:20.068 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T21:30:20.068 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T21:30:20.069 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-10T21:30:20.069 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T21:30:20.069 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-10T21:30:20.069 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T21:30:20.069 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-10T21:30:20.069 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T21:30:20.069 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T21:30:20.069 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T21:30:20.069 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T21:30:20.069 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-10T21:30:20.069 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-10T21:30:20.069 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-10T21:30:20.069 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T21:30:20.069 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-10T21:30:20.070 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-10T21:30:20.070 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T21:30:20.070 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T21:30:20.070 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T21:30:20.070 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T21:30:20.070 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T21:30:20.070 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T21:30:20.070 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T21:30:20.070 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T21:30:20.070 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-10T21:30:20.070 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T21:30:20.070 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T21:30:20.070 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T21:30:20.070 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T21:30:20.071 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T21:30:20.071 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-10T21:30:20.071 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T21:30:20.071 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-10T21:30:20.071 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T21:30:20.071 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T21:30:20.071 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T21:30:20.071 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T21:30:20.071 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-10T21:30:20.071 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T21:30:20.071 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-10T21:30:20.071 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T21:30:20.071 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T21:30:20.071 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-10T21:30:20.072 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T21:30:20.072 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T21:30:20.072 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T21:30:20.072 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T21:30:20.072 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T21:30:20.072 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T21:30:20.072 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T21:30:20.072 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T21:30:20.072 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T21:30:20.072 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T21:30:20.072 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T21:30:20.072 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T21:30:20.072 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T21:30:20.072 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T21:30:20.073 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T21:30:20.073 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T21:30:20.073 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T21:30:20.073 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T21:30:20.073 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T21:30:20.073 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T21:30:20.073 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T21:30:20.073 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-10T21:30:20.073 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T21:30:20.073 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T21:30:20.073 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-10T21:30:20.073 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T21:30:20.073 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-10T21:30:20.073 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-10T21:30:20.074 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T21:30:20.074 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-10T21:30:20.074 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T21:30:20.074 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T21:30:20.074 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T21:30:20.074 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T21:30:20.074 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-10T21:30:20.074 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-10T21:30:20.074 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-10T21:30:20.074 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T21:30:20.074 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T21:30:20.074 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T21:30:20.074 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T21:30:20.075 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T21:30:20.075 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-10T21:30:20.075 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-10T21:30:20.075 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T21:30:20.075 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T21:30:20.075 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T21:30:20.075 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-10T21:30:20.075 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T21:30:20.075 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T21:30:20.075 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T21:30:20.075 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T21:30:20.075 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T21:30:20.075 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T21:30:20.075 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T21:30:20.076 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T21:30:20.076 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T21:30:20.076 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T21:30:20.076 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-10T21:30:20.076 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T21:30:20.076 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T21:30:20.076 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T21:30:20.076 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T21:30:20.076 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T21:30:20.076 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T21:30:20.076 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T21:30:20.076 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T21:30:20.076 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T21:30:20.077 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T21:30:20.077 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T21:30:20.077 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T21:30:20.077 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T21:30:20.077 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T21:30:20.077 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T21:30:20.077 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T21:30:20.077 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T21:30:20.077 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T21:30:20.077 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T21:30:20.077 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T21:30:20.077 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T21:30:20.077 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T21:30:20.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T21:30:20.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T21:30:20.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T21:30:20.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T21:30:20.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T21:30:20.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T21:30:20.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T21:30:20.080 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T21:30:20.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T21:30:20.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T21:30:20.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T21:30:20.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T21:30:20.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T21:30:20.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T21:30:20.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T21:30:20.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T21:30:20.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-10T21:30:20.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T21:30:20.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T21:30:20.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T21:30:20.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T21:30:20.081 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T21:30:20.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T21:30:20.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T21:30:20.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-10T21:30:20.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T21:30:20.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T21:30:20.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T21:30:20.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T21:30:20.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T21:30:20.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T21:30:20.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T21:30:20.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T21:30:20.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T21:30:20.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T21:30:20.082 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T21:30:20.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T21:30:20.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T21:30:20.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T21:30:20.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T21:30:20.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T21:30:20.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T21:30:20.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T21:30:20.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T21:30:20.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T21:30:20.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T21:30:20.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T21:30:20.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T21:30:20.083 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T21:30:20.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T21:30:20.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T21:30:20.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T21:30:20.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-10T21:30:20.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T21:30:20.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-10T21:30:20.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T21:30:20.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T21:30:20.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T21:30:20.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T21:30:20.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T21:30:20.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T21:30:20.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T21:30:20.084 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-10T21:30:20.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T21:30:20.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-10T21:30:20.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T21:30:20.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-10T21:30:20.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T21:30:20.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-10T21:30:20.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-10T21:30:20.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T21:30:20.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T21:30:20.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T21:30:20.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T21:30:20.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T21:30:20.085 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T21:30:20.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T21:30:20.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T21:30:20.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T21:30:20.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T21:30:20.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T21:30:20.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T21:30:20.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T21:30:20.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T21:30:20.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T21:30:20.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T21:30:20.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-10T21:30:20.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T21:30:20.086 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T21:30:20.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T21:30:20.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T21:30:20.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T21:30:20.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T21:30:20.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T21:30:20.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T21:30:20.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T21:30:20.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T21:30:20.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T21:30:20.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T21:30:20.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T21:30:20.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T21:30:20.087 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T21:30:20.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T21:30:20.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T21:30:20.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T21:30:20.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T21:30:20.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T21:30:20.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T21:30:20.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T21:30:20.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T21:30:20.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T21:30:20.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T21:30:20.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T21:30:20.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T21:30:20.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T21:30:20.088 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T21:30:20.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T21:30:20.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T21:30:20.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T21:30:20.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T21:30:20.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T21:30:20.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T21:30:20.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T21:30:20.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T21:30:20.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T21:30:20.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T21:30:20.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T21:30:20.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T21:30:20.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T21:30:20.089 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T21:30:20.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T21:30:20.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T21:30:20.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T21:30:20.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T21:30:20.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T21:30:20.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T21:30:20.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T21:30:20.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T21:30:20.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T21:30:20.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T21:30:20.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T21:30:20.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T21:30:20.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T21:30:20.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T21:30:20.090 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T21:30:20.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-10T21:30:20.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T21:30:20.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T21:30:20.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T21:30:20.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T21:30:20.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-10T21:30:20.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T21:30:20.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T21:30:20.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T21:30:20.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T21:30:20.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T21:30:20.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T21:30:20.091 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T21:30:20.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T21:30:20.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T21:30:20.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T21:30:20.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T21:30:20.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T21:30:20.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T21:30:20.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T21:30:20.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T21:30:20.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T21:30:20.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T21:30:20.092 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T21:30:20.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T21:30:20.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T21:30:20.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T21:30:20.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T21:30:20.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T21:30:20.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T21:30:20.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T21:30:20.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T21:30:20.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T21:30:20.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T21:30:20.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T21:30:20.093 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T21:30:20.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T21:30:20.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T21:30:20.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T21:30:20.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T21:30:20.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T21:30:20.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T21:30:20.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T21:30:20.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T21:30:20.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T21:30:20.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T21:30:20.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T21:30:20.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T21:30:20.094 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T21:30:20.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T21:30:20.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T21:30:20.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T21:30:20.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T21:30:20.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T21:30:20.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T21:30:20.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T21:30:20.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T21:30:20.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T21:30:20.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T21:30:20.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T21:30:20.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T21:30:20.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T21:30:20.095 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T21:30:20.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T21:30:20.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T21:30:20.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T21:30:20.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T21:30:20.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T21:30:20.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T21:30:20.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T21:30:20.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T21:30:20.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T21:30:20.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T21:30:20.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T21:30:20.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T21:30:20.096 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T21:30:20.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T21:30:20.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-10T21:30:20.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T21:30:20.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-10T21:30:20.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T21:30:20.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-10T21:30:20.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T21:30:20.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T21:30:20.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T21:30:20.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T21:30:20.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-10T21:30:20.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T21:30:20.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-10T21:30:20.097 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T21:30:20.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T21:30:20.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T21:30:20.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-10T21:30:20.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T21:30:20.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T21:30:20.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T21:30:20.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T21:30:20.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T21:30:20.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T21:30:20.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T21:30:20.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T21:30:20.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T21:30:20.098 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T21:30:20.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T21:30:20.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T21:30:20.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T21:30:20.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T21:30:20.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T21:30:20.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T21:30:20.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T21:30:20.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T21:30:20.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T21:30:20.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T21:30:20.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T21:30:20.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T21:30:20.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T21:30:20.099 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T21:30:20.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T21:30:20.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T21:30:20.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T21:30:20.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T21:30:20.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T21:30:20.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T21:30:20.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T21:30:20.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T21:30:20.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T21:30:20.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T21:30:20.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T21:30:20.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T21:30:20.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T21:30:20.100 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T21:30:20.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T21:30:20.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T21:30:20.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T21:30:20.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T21:30:20.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T21:30:20.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-10T21:30:20.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T21:30:20.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T21:30:20.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-10T21:30:20.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-10T21:30:20.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T21:30:20.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T21:30:20.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T21:30:20.101 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-10T21:30:20.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T21:30:20.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-10T21:30:20.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T21:30:20.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T21:30:20.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-10T21:30:20.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T21:30:20.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-10T21:30:20.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T21:30:20.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T21:30:20.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T21:30:20.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T21:30:20.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T21:30:20.102 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-10T21:30:20.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T21:30:20.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-10T21:30:20.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-10T21:30:20.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-10T21:30:20.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-10T21:30:20.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-10T21:30:20.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-10T21:30:20.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T21:30:20.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T21:30:20.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T21:30:20.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T21:30:20.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T21:30:20.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T21:30:20.103 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T21:30:20.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T21:30:20.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T21:30:20.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T21:30:20.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T21:30:20.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T21:30:20.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T21:30:20.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-10T21:30:20.104 INFO:teuthology.orchestra.run.smithi133.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T21:30:20.108 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:30:20.108 DEBUG:teuthology.orchestra.run.smithi133:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-10T21:30:20.134 INFO:teuthology.orchestra.run.smithi133.stdout:postinstall scriptlet (using /bin/sh): 2024-09-10T21:30:20.134 INFO:teuthology.orchestra.run.smithi133.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T21:30:20.134 INFO:teuthology.orchestra.run.smithi133.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-10T21:30:20.134 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2024-09-10T21:30:20.134 INFO:teuthology.orchestra.run.smithi133.stdout:for file in vmlinuz System.map config; do 2024-09-10T21:30:20.134 INFO:teuthology.orchestra.run.smithi133.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-10T21:30:20.134 INFO:teuthology.orchestra.run.smithi133.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T21:30:20.134 INFO:teuthology.orchestra.run.smithi133.stdout: fi 2024-09-10T21:30:20.135 INFO:teuthology.orchestra.run.smithi133.stdout:done 2024-09-10T21:30:20.135 INFO:teuthology.orchestra.run.smithi133.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-10T21:30:20.135 INFO:teuthology.orchestra.run.smithi133.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-10T21:30:20.135 INFO:teuthology.orchestra.run.smithi133.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T21:30:20.135 INFO:teuthology.orchestra.run.smithi133.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-10T21:30:20.135 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2024-09-10T21:30:20.135 INFO:teuthology.orchestra.run.smithi133.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-10T21:30:20.135 INFO:teuthology.orchestra.run.smithi133.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:30:20.135 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2024-09-10T21:30:20.135 INFO:teuthology.orchestra.run.smithi133.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-10T21:30:20.135 INFO:teuthology.orchestra.run.smithi133.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-10T21:30:20.135 INFO:teuthology.orchestra.run.smithi133.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:30:20.135 INFO:teuthology.orchestra.run.smithi133.stdout:fi 2024-09-10T21:30:20.136 DEBUG:teuthology.orchestra.run.smithi133:> sudo rpm -qi grub2-tools 2024-09-10T21:30:20.215 INFO:teuthology.orchestra.run.smithi133.stdout:Name : grub2-tools 2024-09-10T21:30:20.216 INFO:teuthology.orchestra.run.smithi133.stdout:Epoch : 1 2024-09-10T21:30:20.216 INFO:teuthology.orchestra.run.smithi133.stdout:Version : 2.06 2024-09-10T21:30:20.216 INFO:teuthology.orchestra.run.smithi133.stdout:Release : 82.el9 2024-09-10T21:30:20.216 INFO:teuthology.orchestra.run.smithi133.stdout:Architecture: x86_64 2024-09-10T21:30:20.216 INFO:teuthology.orchestra.run.smithi133.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T21:30:20.216 INFO:teuthology.orchestra.run.smithi133.stdout:Group : Unspecified 2024-09-10T21:30:20.216 INFO:teuthology.orchestra.run.smithi133.stdout:Size : 8274670 2024-09-10T21:30:20.216 INFO:teuthology.orchestra.run.smithi133.stdout:License : GPLv3+ 2024-09-10T21:30:20.216 INFO:teuthology.orchestra.run.smithi133.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T21:30:20.216 INFO:teuthology.orchestra.run.smithi133.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T21:30:20.216 INFO:teuthology.orchestra.run.smithi133.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T21:30:20.216 INFO:teuthology.orchestra.run.smithi133.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T21:30:20.217 INFO:teuthology.orchestra.run.smithi133.stdout:Packager : builder@centos.org 2024-09-10T21:30:20.217 INFO:teuthology.orchestra.run.smithi133.stdout:Vendor : CentOS 2024-09-10T21:30:20.217 INFO:teuthology.orchestra.run.smithi133.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T21:30:20.217 INFO:teuthology.orchestra.run.smithi133.stdout:Summary : Support tools for GRUB. 2024-09-10T21:30:20.217 INFO:teuthology.orchestra.run.smithi133.stdout:Description : 2024-09-10T21:30:20.217 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:30:20.217 INFO:teuthology.orchestra.run.smithi133.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T21:30:20.217 INFO:teuthology.orchestra.run.smithi133.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T21:30:20.217 INFO:teuthology.orchestra.run.smithi133.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T21:30:20.217 INFO:teuthology.orchestra.run.smithi133.stdout:hardware devices. 2024-09-10T21:30:20.217 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:30:20.217 INFO:teuthology.orchestra.run.smithi133.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T21:30:20.218 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T21:30:20.218 INFO:teuthology.task.kernel:Updating grub on smithi133 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:30:20.219 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T21:30:20.621 INFO:teuthology.orchestra.run.smithi007.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T21:30:20.636 INFO:teuthology.orchestra.run.smithi007.stderr:done 2024-09-10T21:30:20.638 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-09-10T21:30:20.653 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.hzHA5EakPB 2024-09-10T21:30:20.654 DEBUG:teuthology.orchestra.run.smithi007:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.hzHA5EakPB 2024-09-10T21:30:20.729 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.hzHA5EakPB 2024-09-10T21:30:20.865 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.hzHA5EakPB is 6KB 2024-09-10T21:30:20.907 INFO:teuthology.orchestra.run.smithi133.stderr:Generating grub configuration file ... 2024-09-10T21:30:20.914 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.hzHA5EakPB 2024-09-10T21:30:20.929 DEBUG:teuthology.orchestra.run.smithi007:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T21:30:20.999 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T21:30:20.999 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T21:30:20.999 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T21:30:20.999 INFO:teuthology.orchestra.run.smithi007.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-10T21:30:21.001 DEBUG:teuthology.orchestra.run.smithi007:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:30:21.167 DEBUG:teuthology.orchestra.run.smithi007:> sudo shutdown -r now 2024-09-10T21:30:22.309 INFO:teuthology.orchestra.run.smithi133.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T21:30:22.337 INFO:teuthology.orchestra.run.smithi133.stderr:done 2024-09-10T21:30:22.339 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-09-10T21:30:22.359 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.icQvLrZvjH 2024-09-10T21:30:22.359 DEBUG:teuthology.orchestra.run.smithi133:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.icQvLrZvjH 2024-09-10T21:30:22.425 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.icQvLrZvjH 2024-09-10T21:30:22.560 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.icQvLrZvjH is 6KB 2024-09-10T21:30:22.609 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.icQvLrZvjH 2024-09-10T21:30:22.624 DEBUG:teuthology.orchestra.run.smithi133:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T21:30:22.686 INFO:teuthology.orchestra.run.smithi133.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T21:30:22.686 INFO:teuthology.orchestra.run.smithi133.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T21:30:22.686 INFO:teuthology.orchestra.run.smithi133.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T21:30:22.686 INFO:teuthology.orchestra.run.smithi133.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-10T21:30:22.688 DEBUG:teuthology.orchestra.run.smithi133:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:30:22.854 DEBUG:teuthology.orchestra.run.smithi133:> sudo shutdown -r now 2024-09-10T21:30:51.198 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-10T21:30:51.198 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-10T21:30:51.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:30:52.858 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-10T21:30:52.858 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-10T21:30:52.859 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:31:09.757 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-09-10T21:31:11.293 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-09-10T21:31:18.766 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-10T21:31:18.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:31:20.295 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-10T21:31:20.296 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:31:21.853 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-09-10T21:31:23.357 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.133 2024-09-10T21:31:33.861 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-10T21:31:33.861 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:31:35.359 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-10T21:31:35.360 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:31:36.925 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.7 2024-09-10T21:31:51.937 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-10T21:31:51.938 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:32:07.424 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-09-10T21:32:07.952 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-10T21:32:07.953 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T21:32:07.953 DEBUG:teuthology.orchestra.run.smithi133:> uname -r 2024-09-10T21:32:07.989 INFO:teuthology.orchestra.run.smithi133.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:32:07.992 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:32:07.992 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T21:32:07.992 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T21:32:08.995 DEBUG:teuthology.parallel:result is None 2024-09-10T21:32:51.966 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T21:33:09.967 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-10T21:33:09.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:33:10.353 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-09-10T21:33:10.872 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-10T21:33:10.872 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T21:33:10.872 DEBUG:teuthology.orchestra.run.smithi007:> uname -r 2024-09-10T21:33:10.907 INFO:teuthology.orchestra.run.smithi007.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:33:10.907 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:33:10.907 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T21:33:10.907 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T21:33:11.910 DEBUG:teuthology.parallel:result is None 2024-09-10T21:33:11.910 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T21:33:11.917 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T21:33:11.918 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T21:33:11.922 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T21:33:11.963 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T21:33:11.972 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T21:33:11.978 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T21:33:11.978 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T21:33:12.001 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T21:33:12.056 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T21:33:12.063 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T21:33:12.063 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T21:33:12.068 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T21:33:12.154 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:33:12.185 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:33:12.188 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:33:12.215 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:33:12.218 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T21:33:12.225 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T21:33:12.225 DEBUG:teuthology.orchestra.run.smithi007:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T21:33:12.229 DEBUG:teuthology.orchestra.run.smithi133:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T21:33:12.285 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T21:33:12.294 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T21:33:12.294 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T21:33:12.297 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T21:33:12.332 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T21:33:12.439 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T21:33:12.511 DEBUG:teuthology.orchestra.run.smithi007:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T21:33:12.581 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T21:33:12.646 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:33:12.646 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T21:33:12.700 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T21:33:12.772 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T21:33:12.851 DEBUG:teuthology.orchestra.run.smithi133:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T21:33:12.923 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T21:33:12.984 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:33:12.984 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T21:33:13.038 DEBUG:teuthology.orchestra.run.smithi007:> sudo service rsyslog restart 2024-09-10T21:33:13.042 DEBUG:teuthology.orchestra.run.smithi133:> sudo service rsyslog restart 2024-09-10T21:33:13.122 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T21:33:13.173 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T21:33:13.645 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T21:33:13.653 INFO:teuthology.task.internal:Starting timer... 2024-09-10T21:33:13.653 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T21:33:13.663 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T21:33:13.671 DEBUG:teuthology.orchestra.run.smithi007:> sudo service auditd rotate 2024-09-10T21:33:13.769 INFO:teuthology.orchestra.run.smithi007.stdout:Rotating logs: 2024-09-10T21:33:13.773 DEBUG:teuthology.orchestra.run.smithi133:> sudo service auditd rotate 2024-09-10T21:33:13.869 INFO:teuthology.orchestra.run.smithi133.stdout:Rotating logs: 2024-09-10T21:33:13.873 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T21:33:13.873 DEBUG:teuthology.orchestra.run.smithi007:> /usr/sbin/getenforce 2024-09-10T21:33:13.926 INFO:teuthology.orchestra.run.smithi007.stdout:Permissive 2024-09-10T21:33:13.926 DEBUG:teuthology.orchestra.run.smithi133:> /usr/sbin/getenforce 2024-09-10T21:33:13.982 INFO:teuthology.orchestra.run.smithi133.stdout:Permissive 2024-09-10T21:33:13.982 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi007.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi133.front.sepia.ceph.com': 'permissive'} 2024-09-10T21:33:13.983 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T21:33:14.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:33:14.040 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T21:33:14.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:33:14.097 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T21:33:14.097 DEBUG:teuthology.orchestra.run.smithi007:> sudo /usr/sbin/setenforce permissive 2024-09-10T21:33:14.158 DEBUG:teuthology.orchestra.run.smithi133:> sudo /usr/sbin/setenforce permissive 2024-09-10T21:33:14.220 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T21:33:14.234 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T21:33:14.245 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T21:33:14.246 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi007.front.sepia.ceph.com,smithi133.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T21:37:58.900 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi007.front.sepia.ceph.com'), Remote(name='ubuntu@smithi133.front.sepia.ceph.com')] 2024-09-10T21:37:58.902 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-10T21:37:58.903 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi007.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:37:59.016 DEBUG:teuthology.orchestra.run.smithi007:> true 2024-09-10T21:37:59.135 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi007.front.sepia.ceph.com' 2024-09-10T21:37:59.136 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-10T21:37:59.137 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi133.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:37:59.228 DEBUG:teuthology.orchestra.run.smithi133:> true 2024-09-10T21:37:59.343 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi133.front.sepia.ceph.com' 2024-09-10T21:37:59.343 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T21:37:59.355 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T21:37:59.355 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T21:37:59.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:37:59.360 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T21:37:59.360 DEBUG:teuthology.orchestra.run.smithi133:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:37:59.419 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T21:37:59.442 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T21:37:59.449 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T21:37:59.471 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T21:37:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:sudo: ntpd: command not found 2024-09-10T21:37:59.522 INFO:teuthology.orchestra.run.smithi133.stderr:sudo: ntpd: command not found 2024-09-10T21:37:59.525 INFO:teuthology.orchestra.run.smithi007.stdout:506 Cannot talk to daemon 2024-09-10T21:37:59.546 INFO:teuthology.orchestra.run.smithi133.stdout:506 Cannot talk to daemon 2024-09-10T21:37:59.554 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T21:37:59.575 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T21:37:59.583 INFO:teuthology.orchestra.run.smithi007.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T21:37:59.604 INFO:teuthology.orchestra.run.smithi133.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T21:37:59.689 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-09-10T21:37:59.697 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T21:37:59.697 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-09-10T21:37:59.697 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:37:59.697 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:37:59.697 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:37:59.697 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:37:59.706 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-09-10T21:37:59.713 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T21:37:59.713 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-09-10T21:37:59.713 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:37:59.713 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:37:59.713 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:37:59.713 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:37:59.714 INFO:teuthology.run_tasks:Running task install... 2024-09-10T21:37:59.723 DEBUG:teuthology.task.install:project ceph 2024-09-10T21:37:59.723 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}} 2024-09-10T21:37:59.723 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'} 2024-09-10T21:37:59.723 INFO:teuthology.task.install:Using flavor: default 2024-09-10T21:37:59.728 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T21:37:59.729 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T21:37:59.729 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T21:37:59.729 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:37:59.731 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'tag': None, 'wait_for_package': False} 2024-09-10T21:37:59.731 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:37:59.884 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T21:37:59.884 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T21:37:59.896 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/ 2024-09-10T21:37:59.896 INFO:teuthology.task.install.rpm:Package version is 19.3.0-4854.g8f219c47 2024-09-10T21:38:00.003 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T21:38:00.003 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:38:00.003 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T21:38:00.014 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T21:38:00.014 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:38:00.014 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T21:38:00.061 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T21:38:00.061 DEBUG:teuthology.orchestra.run.smithi007:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T21:38:00.070 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-10T21:38:00.070 DEBUG:teuthology.orchestra.run.smithi133:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8f219c476cb4f26fc5719c661c0323719a45cb30/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T21:38:00.159 DEBUG:teuthology.orchestra.run.smithi007:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T21:38:00.167 DEBUG:teuthology.orchestra.run.smithi133:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T21:38:00.237 DEBUG:teuthology.orchestra.run.smithi007:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T21:38:00.245 DEBUG:teuthology.orchestra.run.smithi133:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T21:38:00.293 INFO:teuthology.orchestra.run.smithi007.stdout:check_obsoletes = 1 2024-09-10T21:38:00.296 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum clean all 2024-09-10T21:38:00.299 INFO:teuthology.orchestra.run.smithi133.stdout:check_obsoletes = 1 2024-09-10T21:38:00.302 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum clean all 2024-09-10T21:38:00.575 INFO:teuthology.orchestra.run.smithi133.stdout:45 files removed 2024-09-10T21:38:00.586 INFO:teuthology.orchestra.run.smithi007.stdout:45 files removed 2024-09-10T21:38:00.608 DEBUG:teuthology.orchestra.run.smithi133:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T21:38:00.621 DEBUG:teuthology.orchestra.run.smithi007:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-10T21:38:01.323 INFO:teuthology.orchestra.run.smithi133.stdout:ceph packages for x86_64 266 kB/s | 84 kB 00:00 2024-09-10T21:38:01.338 INFO:teuthology.orchestra.run.smithi007.stdout:ceph packages for x86_64 274 kB/s | 84 kB 00:00 2024-09-10T21:38:01.647 INFO:teuthology.orchestra.run.smithi007.stdout:ceph noarch packages 92 kB/s | 24 kB 00:00 2024-09-10T21:38:01.690 INFO:teuthology.orchestra.run.smithi133.stdout:ceph noarch packages 75 kB/s | 24 kB 00:00 2024-09-10T21:38:01.939 INFO:teuthology.orchestra.run.smithi007.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-10T21:38:02.025 INFO:teuthology.orchestra.run.smithi133.stdout:ceph source packages 6.5 kB/s | 1.9 kB 00:00 2024-09-10T21:38:02.883 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.2 MB 00:00 2024-09-10T21:38:02.897 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - BaseOS 9.0 MB/s | 8.2 MB 00:00 2024-09-10T21:38:06.060 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:02 2024-09-10T21:38:07.765 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - AppStream 5.5 MB/s | 20 MB 00:03 2024-09-10T21:38:10.543 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-10T21:38:12.269 INFO:teuthology.orchestra.run.smithi133.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-10T21:38:13.308 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - CRB 4.8 MB/s | 6.5 MB 00:01 2024-09-10T21:38:13.773 INFO:teuthology.orchestra.run.smithi133.stdout:Extra Packages for Enterprise Linux 16 MB/s | 23 MB 00:01 2024-09-10T21:38:15.100 INFO:teuthology.orchestra.run.smithi007.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-10T21:38:16.496 INFO:teuthology.orchestra.run.smithi007.stdout:Extra Packages for Enterprise Linux 18 MB/s | 23 MB 00:01 2024-09-10T21:38:20.188 INFO:teuthology.orchestra.run.smithi133.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-09-10T21:38:22.674 INFO:teuthology.orchestra.run.smithi133.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T21:38:22.675 INFO:teuthology.orchestra.run.smithi133.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T21:38:22.687 INFO:teuthology.orchestra.run.smithi133.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T21:38:22.688 INFO:teuthology.orchestra.run.smithi133.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T21:38:22.689 INFO:teuthology.orchestra.run.smithi133.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T21:38:22.690 INFO:teuthology.orchestra.run.smithi133.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T21:38:22.691 INFO:teuthology.orchestra.run.smithi133.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T21:38:22.694 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T21:38:22.695 INFO:teuthology.orchestra.run.smithi133.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T21:38:22.761 INFO:teuthology.orchestra.run.smithi133.stdout:Dependencies resolved. 2024-09-10T21:38:22.781 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================= 2024-09-10T21:38:22.781 INFO:teuthology.orchestra.run.smithi133.stdout: Package Arch Version Repository Size 2024-09-10T21:38:22.781 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================= 2024-09-10T21:38:22.781 INFO:teuthology.orchestra.run.smithi133.stdout:Installing: 2024-09-10T21:38:22.781 INFO:teuthology.orchestra.run.smithi133.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T21:38:22.782 INFO:teuthology.orchestra.run.smithi133.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T21:38:22.782 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T21:38:22.782 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T21:38:22.782 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T21:38:22.782 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T21:38:22.782 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T21:38:22.782 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T21:38:22.782 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T21:38:22.782 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T21:38:22.782 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T21:38:22.782 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T21:38:22.782 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T21:38:22.783 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T21:38:22.783 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T21:38:22.783 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T21:38:22.783 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T21:38:22.783 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T21:38:22.783 INFO:teuthology.orchestra.run.smithi133.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T21:38:22.783 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T21:38:22.783 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T21:38:22.783 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T21:38:22.783 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T21:38:22.783 INFO:teuthology.orchestra.run.smithi133.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T21:38:22.784 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T21:38:22.784 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T21:38:22.784 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T21:38:22.784 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T21:38:22.784 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T21:38:22.784 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T21:38:22.784 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T21:38:22.784 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T21:38:22.784 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T21:38:22.784 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T21:38:22.784 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T21:38:22.784 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T21:38:22.784 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T21:38:22.785 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrading: 2024-09-10T21:38:22.785 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T21:38:22.785 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T21:38:22.785 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T21:38:22.785 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T21:38:22.785 INFO:teuthology.orchestra.run.smithi133.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T21:38:22.785 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T21:38:22.785 INFO:teuthology.orchestra.run.smithi133.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T21:38:22.785 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T21:38:22.785 INFO:teuthology.orchestra.run.smithi133.stdout:Installing dependencies: 2024-09-10T21:38:22.785 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T21:38:22.785 INFO:teuthology.orchestra.run.smithi133.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T21:38:22.785 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T21:38:22.785 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T21:38:22.786 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T21:38:22.786 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T21:38:22.786 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T21:38:22.786 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T21:38:22.786 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T21:38:22.786 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T21:38:22.786 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T21:38:22.786 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T21:38:22.786 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T21:38:22.786 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T21:38:22.786 INFO:teuthology.orchestra.run.smithi133.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T21:38:22.786 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T21:38:22.786 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T21:38:22.786 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T21:38:22.787 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T21:38:22.787 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T21:38:22.787 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T21:38:22.787 INFO:teuthology.orchestra.run.smithi133.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T21:38:22.787 INFO:teuthology.orchestra.run.smithi133.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T21:38:22.787 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T21:38:22.787 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T21:38:22.787 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T21:38:22.787 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T21:38:22.787 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T21:38:22.787 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T21:38:22.787 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T21:38:22.787 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T21:38:22.788 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T21:38:22.788 INFO:teuthology.orchestra.run.smithi133.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T21:38:22.788 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T21:38:22.788 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T21:38:22.788 INFO:teuthology.orchestra.run.smithi133.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T21:38:22.788 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T21:38:22.788 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T21:38:22.788 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T21:38:22.788 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T21:38:22.788 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T21:38:22.788 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T21:38:22.788 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T21:38:22.789 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T21:38:22.789 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T21:38:22.789 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T21:38:22.789 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T21:38:22.789 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T21:38:22.789 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T21:38:22.789 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T21:38:22.789 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T21:38:22.789 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T21:38:22.789 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T21:38:22.789 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T21:38:22.789 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T21:38:22.789 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T21:38:22.789 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T21:38:22.789 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T21:38:22.790 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T21:38:22.790 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T21:38:22.790 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T21:38:22.790 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T21:38:22.790 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T21:38:22.790 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T21:38:22.790 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T21:38:22.790 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T21:38:22.790 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T21:38:22.790 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T21:38:22.790 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T21:38:22.790 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T21:38:22.790 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T21:38:22.790 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T21:38:22.791 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T21:38:22.791 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T21:38:22.791 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T21:38:22.791 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T21:38:22.791 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T21:38:22.791 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T21:38:22.791 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T21:38:22.791 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T21:38:22.791 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T21:38:22.791 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T21:38:22.791 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T21:38:22.791 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T21:38:22.791 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T21:38:22.791 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T21:38:22.792 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T21:38:22.792 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T21:38:22.792 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T21:38:22.792 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T21:38:22.792 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T21:38:22.792 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T21:38:22.792 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T21:38:22.792 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T21:38:22.792 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T21:38:22.792 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T21:38:22.792 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T21:38:22.792 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T21:38:22.792 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T21:38:22.793 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T21:38:22.793 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T21:38:22.793 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T21:38:22.793 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T21:38:22.793 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T21:38:22.793 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T21:38:22.793 INFO:teuthology.orchestra.run.smithi133.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T21:38:22.793 INFO:teuthology.orchestra.run.smithi133.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T21:38:22.793 INFO:teuthology.orchestra.run.smithi133.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T21:38:22.793 INFO:teuthology.orchestra.run.smithi133.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T21:38:22.793 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T21:38:22.793 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T21:38:22.794 INFO:teuthology.orchestra.run.smithi133.stdout:Installing weak dependencies: 2024-09-10T21:38:22.794 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T21:38:22.794 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:38:22.794 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction Summary 2024-09-10T21:38:22.794 INFO:teuthology.orchestra.run.smithi133.stdout:======================================================================================= 2024-09-10T21:38:22.794 INFO:teuthology.orchestra.run.smithi133.stdout:Install 149 Packages 2024-09-10T21:38:22.794 INFO:teuthology.orchestra.run.smithi133.stdout:Upgrade 8 Packages 2024-09-10T21:38:22.794 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:38:22.795 INFO:teuthology.orchestra.run.smithi133.stdout:Total download size: 240 M 2024-09-10T21:38:22.795 INFO:teuthology.orchestra.run.smithi133.stdout:Downloading Packages: 2024-09-10T21:38:23.110 INFO:teuthology.orchestra.run.smithi007.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-10T21:38:23.364 INFO:teuthology.orchestra.run.smithi133.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-09-10T21:38:23.889 INFO:teuthology.orchestra.run.smithi133.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 1.7 MB/s | 900 kB 00:00 2024-09-10T21:38:23.997 INFO:teuthology.orchestra.run.smithi133.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 1.3 MB/s | 145 kB 00:00 2024-09-10T21:38:24.981 INFO:teuthology.orchestra.run.smithi133.stdout:(4/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 2.4 MB/s | 2.4 MB 00:00 2024-09-10T21:38:25.156 INFO:teuthology.orchestra.run.smithi133.stdout:(5/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 2.7 MB/s | 5.5 MB 00:02 2024-09-10T21:38:25.607 INFO:teuthology.orchestra.run.smithi133.stdout:(6/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 2.4 MB/s | 1.5 MB 00:00 2024-09-10T21:38:25.751 INFO:teuthology.orchestra.run.smithi007.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T21:38:25.752 INFO:teuthology.orchestra.run.smithi007.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-10T21:38:25.764 INFO:teuthology.orchestra.run.smithi007.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-10T21:38:25.765 INFO:teuthology.orchestra.run.smithi007.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-10T21:38:25.765 INFO:teuthology.orchestra.run.smithi007.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-10T21:38:25.766 INFO:teuthology.orchestra.run.smithi007.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-10T21:38:25.768 INFO:teuthology.orchestra.run.smithi007.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-10T21:38:25.771 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-10T21:38:25.771 INFO:teuthology.orchestra.run.smithi007.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-10T21:38:25.842 INFO:teuthology.orchestra.run.smithi007.stdout:Dependencies resolved. 2024-09-10T21:38:25.870 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-09-10T21:38:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: Package Arch Version Repository Size 2024-09-10T21:38:25.870 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-09-10T21:38:25.870 INFO:teuthology.orchestra.run.smithi007.stdout:Installing: 2024-09-10T21:38:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-10T21:38:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: ceph x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 6.6 k 2024-09-10T21:38:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T21:38:25.870 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 900 k 2024-09-10T21:38:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 145 k 2024-09-10T21:38:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 1.5 M 2024-09-10T21:38:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 154 k 2024-09-10T21:38:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 14 M 2024-09-10T21:38:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 7.4 M 2024-09-10T21:38:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 49 k 2024-09-10T21:38:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 11 M 2024-09-10T21:38:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 50 M 2024-09-10T21:38:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 285 k 2024-09-10T21:38:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 783 k 2024-09-10T21:38:25.871 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 225 k 2024-09-10T21:38:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 27 k 2024-09-10T21:38:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 18 k 2024-09-10T21:38:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-10T21:38:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-10T21:38:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-10T21:38:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 32 k 2024-09-10T21:38:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 755 k 2024-09-10T21:38:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 127 k 2024-09-10T21:38:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-10T21:38:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-10T21:38:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-10T21:38:25.872 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-10T21:38:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-10T21:38:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-10T21:38:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 161 k 2024-09-10T21:38:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 321 k 2024-09-10T21:38:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 306 k 2024-09-10T21:38:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 99 k 2024-09-10T21:38:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 85 k 2024-09-10T21:38:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T21:38:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 171 k 2024-09-10T21:38:25.873 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-10T21:38:25.873 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrading: 2024-09-10T21:38:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-10T21:38:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-10T21:38:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-10T21:38:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-10T21:38:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: librados2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.5 M 2024-09-10T21:38:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 3.2 M 2024-09-10T21:38:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-10T21:38:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-10T21:38:25.874 INFO:teuthology.orchestra.run.smithi007.stdout:Installing dependencies: 2024-09-10T21:38:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-10T21:38:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-10T21:38:25.874 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T21:38:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 22 M 2024-09-10T21:38:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 34 k 2024-09-10T21:38:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 2.4 M 2024-09-10T21:38:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 291 k 2024-09-10T21:38:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 4.8 M 2024-09-10T21:38:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 17 M 2024-09-10T21:38:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts noarch 2:19.3.0-4854.g8f219c47.el9 ceph-noarch 16 k 2024-09-10T21:38:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 25 k 2024-09-10T21:38:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T21:38:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T21:38:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T21:38:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T21:38:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T21:38:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-10T21:38:25.875 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T21:38:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T21:38:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 162 k 2024-09-10T21:38:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T21:38:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-10T21:38:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T21:38:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T21:38:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T21:38:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T21:38:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 507 k 2024-09-10T21:38:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T21:38:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2 x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 5.5 M 2024-09-10T21:38:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T21:38:25.876 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-10T21:38:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T21:38:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-10T21:38:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-10T21:38:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-10T21:38:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T21:38:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T21:38:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T21:38:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-10T21:38:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-10T21:38:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-10T21:38:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T21:38:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-10T21:38:25.877 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T21:38:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T21:38:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T21:38:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 45 k 2024-09-10T21:38:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common x86_64 2:19.3.0-4854.g8f219c47.el9 ceph 143 k 2024-09-10T21:38:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T21:38:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T21:38:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T21:38:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T21:38:25.878 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-10T21:38:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-10T21:38:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T21:38:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T21:38:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T21:38:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-10T21:38:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-10T21:38:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T21:38:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T21:38:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T21:38:25.879 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T21:38:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T21:38:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T21:38:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T21:38:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T21:38:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T21:38:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T21:38:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T21:38:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T21:38:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T21:38:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T21:38:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T21:38:25.880 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T21:38:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T21:38:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T21:38:25.881 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T21:38:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T21:38:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T21:38:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T21:38:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-10T21:38:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T21:38:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T21:38:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T21:38:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T21:38:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-10T21:38:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T21:38:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T21:38:25.882 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T21:38:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T21:38:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T21:38:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T21:38:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T21:38:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T21:38:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T21:38:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T21:38:25.883 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T21:38:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-10T21:38:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T21:38:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T21:38:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T21:38:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T21:38:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-10T21:38:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-10T21:38:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T21:38:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T21:38:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T21:38:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-10T21:38:25.884 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T21:38:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-10T21:38:25.885 INFO:teuthology.orchestra.run.smithi007.stdout:Installing weak dependencies: 2024-09-10T21:38:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-10T21:38:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:38:25.885 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction Summary 2024-09-10T21:38:25.885 INFO:teuthology.orchestra.run.smithi007.stdout:======================================================================================= 2024-09-10T21:38:25.885 INFO:teuthology.orchestra.run.smithi007.stdout:Install 149 Packages 2024-09-10T21:38:25.885 INFO:teuthology.orchestra.run.smithi007.stdout:Upgrade 8 Packages 2024-09-10T21:38:25.885 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:38:25.886 INFO:teuthology.orchestra.run.smithi007.stdout:Total download size: 240 M 2024-09-10T21:38:25.886 INFO:teuthology.orchestra.run.smithi007.stdout:Downloading Packages: 2024-09-10T21:38:26.774 INFO:teuthology.orchestra.run.smithi133.stdout:(7/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 3.0 MB/s | 4.8 MB 00:01 2024-09-10T21:38:26.819 INFO:teuthology.orchestra.run.smithi007.stdout:(1/157): ceph-19.3.0-4854.g8f219c47.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-09-10T21:38:27.786 INFO:teuthology.orchestra.run.smithi007.stdout:(2/157): ceph-fuse-19.3.0-4854.g8f219c47.el9.x8 931 kB/s | 900 kB 00:00 2024-09-10T21:38:27.953 INFO:teuthology.orchestra.run.smithi007.stdout:(3/157): ceph-immutable-object-cache-19.3.0-485 870 kB/s | 145 kB 00:00 2024-09-10T21:38:28.754 INFO:teuthology.orchestra.run.smithi007.stdout:(4/157): ceph-base-19.3.0-4854.g8f219c47.el9.x8 2.5 MB/s | 5.5 MB 00:02 2024-09-10T21:38:29.188 INFO:teuthology.orchestra.run.smithi007.stdout:(5/157): ceph-mds-19.3.0-4854.g8f219c47.el9.x86 1.9 MB/s | 2.4 MB 00:01 2024-09-10T21:38:29.229 INFO:teuthology.orchestra.run.smithi007.stdout:(6/157): ceph-mgr-19.3.0-4854.g8f219c47.el9.x86 3.2 MB/s | 1.5 MB 00:00 2024-09-10T21:38:29.573 INFO:teuthology.orchestra.run.smithi133.stdout:(8/157): ceph-common-19.3.0-4854.g8f219c47.el9. 3.3 MB/s | 22 MB 00:06 2024-09-10T21:38:29.665 INFO:teuthology.orchestra.run.smithi133.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 275 kB/s | 25 kB 00:00 2024-09-10T21:38:30.723 INFO:teuthology.orchestra.run.smithi133.stdout:(10/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 2.8 MB/s | 11 MB 00:03 2024-09-10T21:38:30.923 INFO:teuthology.orchestra.run.smithi133.stdout:(11/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 3.2 MB/s | 17 MB 00:05 2024-09-10T21:38:30.957 INFO:teuthology.orchestra.run.smithi133.stdout:(12/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 964 kB/s | 225 kB 00:00 2024-09-10T21:38:31.016 INFO:teuthology.orchestra.run.smithi133.stdout:(13/157): cephfs-shell-19.3.0-4854.g8f219c47.el 292 kB/s | 27 kB 00:00 2024-09-10T21:38:31.049 INFO:teuthology.orchestra.run.smithi133.stdout:(14/157): libcephfs-devel-19.3.0-4854.g8f219c47 346 kB/s | 32 kB 00:00 2024-09-10T21:38:31.133 INFO:teuthology.orchestra.run.smithi133.stdout:(15/157): libcephsqlite-19.3.0-4854.g8f219c47.e 1.9 MB/s | 162 kB 00:00 2024-09-10T21:38:31.217 INFO:teuthology.orchestra.run.smithi133.stdout:(16/157): librados-devel-19.3.0-4854.g8f219c47. 1.5 MB/s | 127 kB 00:00 2024-09-10T21:38:31.238 INFO:teuthology.orchestra.run.smithi007.stdout:(7/157): ceph-common-19.3.0-4854.g8f219c47.el9. 4.6 MB/s | 22 MB 00:04 2024-09-10T21:38:31.309 INFO:teuthology.orchestra.run.smithi133.stdout:(17/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 2.5 MB/s | 755 kB 00:00 2024-09-10T21:38:31.330 INFO:teuthology.orchestra.run.smithi007.stdout:(8/157): ceph-mon-19.3.0-4854.g8f219c47.el9.x86 2.2 MB/s | 4.8 MB 00:02 2024-09-10T21:38:31.430 INFO:teuthology.orchestra.run.smithi007.stdout:(9/157): ceph-selinux-19.3.0-4854.g8f219c47.el9 252 kB/s | 25 kB 00:00 2024-09-10T21:38:31.476 INFO:teuthology.orchestra.run.smithi133.stdout:(18/157): libradosstriper1-19.3.0-4854.g8f219c4 1.9 MB/s | 507 kB 00:00 2024-09-10T21:38:31.584 INFO:teuthology.orchestra.run.smithi133.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 418 kB/s | 45 kB 00:00 2024-09-10T21:38:31.684 INFO:teuthology.orchestra.run.smithi133.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 1.4 MB/s | 143 kB 00:00 2024-09-10T21:38:31.785 INFO:teuthology.orchestra.run.smithi133.stdout:(21/157): python3-cephfs-19.3.0-4854.g8f219c47. 1.6 MB/s | 161 kB 00:00 2024-09-10T21:38:31.954 INFO:teuthology.orchestra.run.smithi133.stdout:(22/157): python3-rados-19.3.0-4854.g8f219c47.e 1.9 MB/s | 321 kB 00:00 2024-09-10T21:38:32.129 INFO:teuthology.orchestra.run.smithi133.stdout:(23/157): python3-rbd-19.3.0-4854.g8f219c47.el9 1.7 MB/s | 306 kB 00:00 2024-09-10T21:38:32.221 INFO:teuthology.orchestra.run.smithi133.stdout:(24/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.1 MB/s | 99 kB 00:00 2024-09-10T21:38:32.321 INFO:teuthology.orchestra.run.smithi133.stdout:(25/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 853 kB/s | 85 kB 00:00 2024-09-10T21:38:32.463 INFO:teuthology.orchestra.run.smithi007.stdout:(10/157): ceph-radosgw-19.3.0-4854.g8f219c47.el 8.9 MB/s | 11 MB 00:01 2024-09-10T21:38:32.688 INFO:teuthology.orchestra.run.smithi007.stdout:(11/157): ceph-osd-19.3.0-4854.g8f219c47.el9.x8 5.0 MB/s | 17 MB 00:03 2024-09-10T21:38:32.714 INFO:teuthology.orchestra.run.smithi007.stdout:(12/157): cephfs-mirror-19.3.0-4854.g8f219c47.e 900 kB/s | 225 kB 00:00 2024-09-10T21:38:32.797 INFO:teuthology.orchestra.run.smithi007.stdout:(13/157): cephfs-shell-19.3.0-4854.g8f219c47.el 247 kB/s | 27 kB 00:00 2024-09-10T21:38:32.848 INFO:teuthology.orchestra.run.smithi007.stdout:(14/157): libcephfs-devel-19.3.0-4854.g8f219c47 237 kB/s | 32 kB 00:00 2024-09-10T21:38:32.897 INFO:teuthology.orchestra.run.smithi133.stdout:(26/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 3.5 MB/s | 5.5 MB 00:01 2024-09-10T21:38:32.931 INFO:teuthology.orchestra.run.smithi007.stdout:(15/157): libcephsqlite-19.3.0-4854.g8f219c47.e 1.9 MB/s | 162 kB 00:00 2024-09-10T21:38:32.989 INFO:teuthology.orchestra.run.smithi133.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 1.8 MB/s | 171 kB 00:00 2024-09-10T21:38:33.024 INFO:teuthology.orchestra.run.smithi007.stdout:(16/157): libcephfs2-19.3.0-4854.g8f219c47.el9. 3.3 MB/s | 755 kB 00:00 2024-09-10T21:38:33.049 INFO:teuthology.orchestra.run.smithi007.stdout:(17/157): librados-devel-19.3.0-4854.g8f219c47. 1.1 MB/s | 127 kB 00:00 2024-09-10T21:38:33.072 INFO:teuthology.orchestra.run.smithi133.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 412 kB/s | 34 kB 00:00 2024-09-10T21:38:33.183 INFO:teuthology.orchestra.run.smithi007.stdout:(18/157): libradosstriper1-19.3.0-4854.g8f219c4 3.1 MB/s | 507 kB 00:00 2024-09-10T21:38:33.189 INFO:teuthology.orchestra.run.smithi133.stdout:(29/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 1.3 MB/s | 154 kB 00:00 2024-09-10T21:38:33.273 INFO:teuthology.orchestra.run.smithi133.stdout:(30/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 3.3 MB/s | 3.2 MB 00:00 2024-09-10T21:38:33.275 INFO:teuthology.orchestra.run.smithi007.stdout:(19/157): python3-ceph-argparse-19.3.0-4854.g8f 493 kB/s | 45 kB 00:00 2024-09-10T21:38:33.366 INFO:teuthology.orchestra.run.smithi007.stdout:(20/157): python3-ceph-common-19.3.0-4854.g8f21 1.5 MB/s | 143 kB 00:00 2024-09-10T21:38:33.458 INFO:teuthology.orchestra.run.smithi007.stdout:(21/157): python3-cephfs-19.3.0-4854.g8f219c47. 1.7 MB/s | 161 kB 00:00 2024-09-10T21:38:33.559 INFO:teuthology.orchestra.run.smithi007.stdout:(22/157): python3-rados-19.3.0-4854.g8f219c47.e 3.1 MB/s | 321 kB 00:00 2024-09-10T21:38:33.676 INFO:teuthology.orchestra.run.smithi007.stdout:(23/157): librgw2-19.3.0-4854.g8f219c47.el9.x86 8.8 MB/s | 5.5 MB 00:00 2024-09-10T21:38:33.701 INFO:teuthology.orchestra.run.smithi007.stdout:(24/157): python3-rbd-19.3.0-4854.g8f219c47.el9 2.1 MB/s | 306 kB 00:00 2024-09-10T21:38:33.760 INFO:teuthology.orchestra.run.smithi007.stdout:(25/157): python3-rgw-19.3.0-4854.g8f219c47.el9 1.2 MB/s | 99 kB 00:00 2024-09-10T21:38:33.785 INFO:teuthology.orchestra.run.smithi007.stdout:(26/157): rbd-fuse-19.3.0-4854.g8f219c47.el9.x8 1.0 MB/s | 85 kB 00:00 2024-09-10T21:38:33.885 INFO:teuthology.orchestra.run.smithi007.stdout:(27/157): rbd-nbd-19.3.0-4854.g8f219c47.el9.x86 1.7 MB/s | 171 kB 00:00 2024-09-10T21:38:33.977 INFO:teuthology.orchestra.run.smithi007.stdout:(28/157): ceph-grafana-dashboards-19.3.0-4854.g 373 kB/s | 34 kB 00:00 2024-09-10T21:38:34.119 INFO:teuthology.orchestra.run.smithi007.stdout:(29/157): rbd-mirror-19.3.0-4854.g8f219c47.el9. 8.8 MB/s | 3.2 MB 00:00 2024-09-10T21:38:34.145 INFO:teuthology.orchestra.run.smithi007.stdout:(30/157): ceph-mgr-cephadm-19.3.0-4854.g8f219c4 925 kB/s | 154 kB 00:00 2024-09-10T21:38:35.595 INFO:teuthology.orchestra.run.smithi007.stdout:(31/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 9.6 MB/s | 14 MB 00:01 2024-09-10T21:38:35.681 INFO:teuthology.orchestra.run.smithi133.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 3.1 MB/s | 7.4 MB 00:02 2024-09-10T21:38:35.687 INFO:teuthology.orchestra.run.smithi007.stdout:(32/157): ceph-mgr-diskprediction-local-19.3.0- 4.8 MB/s | 7.4 MB 00:01 2024-09-10T21:38:35.712 INFO:teuthology.orchestra.run.smithi007.stdout:(33/157): ceph-mgr-modules-core-19.3.0-4854.g8f 2.4 MB/s | 291 kB 00:00 2024-09-10T21:38:35.771 INFO:teuthology.orchestra.run.smithi007.stdout:(34/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 591 kB/s | 49 kB 00:00 2024-09-10T21:38:35.796 INFO:teuthology.orchestra.run.smithi007.stdout:(35/157): ceph-prometheus-alerts-19.3.0-4854.g8 197 kB/s | 16 kB 00:00 2024-09-10T21:38:35.823 INFO:teuthology.orchestra.run.smithi133.stdout:(32/157): ceph-mgr-modules-core-19.3.0-4854.g8f 2.0 MB/s | 291 kB 00:00 2024-09-10T21:38:35.923 INFO:teuthology.orchestra.run.smithi007.stdout:(36/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 6.0 MB/s | 783 kB 00:00 2024-09-10T21:38:35.931 INFO:teuthology.orchestra.run.smithi133.stdout:(33/157): ceph-mgr-rook-19.3.0-4854.g8f219c47.e 456 kB/s | 49 kB 00:00 2024-09-10T21:38:35.955 INFO:teuthology.orchestra.run.smithi007.stdout:(37/157): ceph-volume-19.3.0-4854.g8f219c47.el9 1.5 MB/s | 285 kB 00:00 2024-09-10T21:38:35.997 INFO:teuthology.orchestra.run.smithi007.stdout:(38/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 248 kB/s | 18 kB 00:00 2024-09-10T21:38:36.024 INFO:teuthology.orchestra.run.smithi133.stdout:(34/157): ceph-prometheus-alerts-19.3.0-4854.g8 179 kB/s | 16 kB 00:00 2024-09-10T21:38:36.106 INFO:teuthology.orchestra.run.smithi007.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 531 kB/s | 80 kB 00:00 2024-09-10T21:38:36.157 INFO:teuthology.orchestra.run.smithi133.stdout:(35/157): ceph-volume-19.3.0-4854.g8f219c47.el9 2.1 MB/s | 285 kB 00:00 2024-09-10T21:38:36.206 INFO:teuthology.orchestra.run.smithi007.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 191 kB 00:00 2024-09-10T21:38:36.273 INFO:teuthology.orchestra.run.smithi007.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 801 kB 00:00 2024-09-10T21:38:36.298 INFO:teuthology.orchestra.run.smithi007.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-09-10T21:38:36.349 INFO:teuthology.orchestra.run.smithi007.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-10T21:38:36.416 INFO:teuthology.orchestra.run.smithi007.stdout:(44/157): python3-cryptography-36.0.1-4.el9.x86 8.6 MB/s | 1.2 MB 00:00 2024-09-10T21:38:36.416 INFO:teuthology.orchestra.run.smithi133.stdout:(36/157): cephadm-19.3.0-4854.g8f219c47.el9.noa 3.0 MB/s | 783 kB 00:00 2024-09-10T21:38:36.441 INFO:teuthology.orchestra.run.smithi007.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-09-10T21:38:36.466 INFO:teuthology.orchestra.run.smithi007.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-10T21:38:36.500 INFO:teuthology.orchestra.run.smithi007.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-09-10T21:38:36.508 INFO:teuthology.orchestra.run.smithi133.stdout:(37/157): cephfs-top-19.3.0-4854.g8f219c47.el9. 199 kB/s | 18 kB 00:00 2024-09-10T21:38:36.658 INFO:teuthology.orchestra.run.smithi133.stdout:(38/157): ceph-mgr-dashboard-19.3.0-4854.g8f219 4.1 MB/s | 14 MB 00:03 2024-09-10T21:38:36.742 INFO:teuthology.orchestra.run.smithi133.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 342 kB/s | 80 kB 00:00 2024-09-10T21:38:36.809 INFO:teuthology.orchestra.run.smithi133.stdout:(40/157): libquadmath-11.5.0-2.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-10T21:38:36.859 INFO:teuthology.orchestra.run.smithi133.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 3.9 MB/s | 801 kB 00:00 2024-09-10T21:38:36.884 INFO:teuthology.orchestra.run.smithi133.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-09-10T21:38:36.935 INFO:teuthology.orchestra.run.smithi133.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-09-10T21:38:36.960 INFO:teuthology.orchestra.run.smithi133.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-10T21:38:36.985 INFO:teuthology.orchestra.run.smithi133.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-10T21:38:37.011 INFO:teuthology.orchestra.run.smithi133.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-10T21:38:37.036 INFO:teuthology.orchestra.run.smithi133.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-10T21:38:37.058 INFO:teuthology.orchestra.run.smithi007.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 1.1 MB/s | 689 kB 00:00 2024-09-10T21:38:37.167 INFO:teuthology.orchestra.run.smithi007.stdout:(49/157): bison-3.7.4-5.el9.x86_64.rpm 1.4 MB/s | 943 kB 00:00 2024-09-10T21:38:37.219 INFO:teuthology.orchestra.run.smithi133.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-09-10T21:38:37.270 INFO:teuthology.orchestra.run.smithi133.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-10T21:38:37.312 INFO:teuthology.orchestra.run.smithi133.stdout:(50/157): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-09-10T21:38:37.337 INFO:teuthology.orchestra.run.smithi133.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 323 kB/s | 22 kB 00:00 2024-09-10T21:38:37.371 INFO:teuthology.orchestra.run.smithi133.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 5.3 MB/s | 313 kB 00:00 2024-09-10T21:38:37.396 INFO:teuthology.orchestra.run.smithi133.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 562 kB/s | 33 kB 00:00 2024-09-10T21:38:37.434 INFO:teuthology.orchestra.run.smithi007.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 81 kB/s | 22 kB 00:00 2024-09-10T21:38:37.440 INFO:teuthology.orchestra.run.smithi133.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 435 kB/s | 18 kB 00:00 2024-09-10T21:38:37.480 INFO:teuthology.orchestra.run.smithi133.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 272 kB/s | 11 kB 00:00 2024-09-10T21:38:37.501 INFO:teuthology.orchestra.run.smithi007.stdout:(51/157): boost-program-options-1.75.0-8.el9.x8 242 kB/s | 107 kB 00:00 2024-09-10T21:38:37.539 INFO:teuthology.orchestra.run.smithi133.stdout:(56/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 18 MB/s | 3.0 MB 00:00 2024-09-10T21:38:37.564 INFO:teuthology.orchestra.run.smithi133.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.0 MB/s | 171 kB 00:00 2024-09-10T21:38:37.589 INFO:teuthology.orchestra.run.smithi133.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-10T21:38:37.614 INFO:teuthology.orchestra.run.smithi133.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 907 kB/s | 45 kB 00:00 2024-09-10T21:38:37.648 INFO:teuthology.orchestra.run.smithi133.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-10T21:38:37.682 INFO:teuthology.orchestra.run.smithi133.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-10T21:38:37.707 INFO:teuthology.orchestra.run.smithi133.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 419 kB/s | 24 kB 00:00 2024-09-10T21:38:37.732 INFO:teuthology.orchestra.run.smithi133.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 951 kB/s | 47 kB 00:00 2024-09-10T21:38:37.758 INFO:teuthology.orchestra.run.smithi133.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-10T21:38:37.783 INFO:teuthology.orchestra.run.smithi133.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T21:38:37.790 INFO:teuthology.orchestra.run.smithi007.stdout:(52/157): flexiblas-3.0.4-8.el9.x86_64.rpm 113 kB/s | 33 kB 00:00 2024-09-10T21:38:37.808 INFO:teuthology.orchestra.run.smithi133.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-10T21:38:37.925 INFO:teuthology.orchestra.run.smithi133.stdout:(67/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-10T21:38:37.960 INFO:teuthology.orchestra.run.smithi007.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 597 kB/s | 313 kB 00:00 2024-09-10T21:38:38.000 INFO:teuthology.orchestra.run.smithi133.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 23 MB/s | 4.9 MB 00:00 2024-09-10T21:38:38.043 INFO:teuthology.orchestra.run.smithi133.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-10T21:38:38.076 INFO:teuthology.orchestra.run.smithi133.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-10T21:38:38.101 INFO:teuthology.orchestra.run.smithi133.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-10T21:38:38.169 INFO:teuthology.orchestra.run.smithi133.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 15 MB/s | 1.0 MB 00:00 2024-09-10T21:38:38.244 INFO:teuthology.orchestra.run.smithi133.stdout:(73/157): postgresql-server-13.16-1.el9.x86_64. 35 MB/s | 5.8 MB 00:00 2024-09-10T21:38:38.252 INFO:teuthology.orchestra.run.smithi007.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 62 kB/s | 18 kB 00:00 2024-09-10T21:38:38.269 INFO:teuthology.orchestra.run.smithi133.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 876 kB/s | 87 kB 00:00 2024-09-10T21:38:38.311 INFO:teuthology.orchestra.run.smithi133.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-10T21:38:38.403 INFO:teuthology.orchestra.run.smithi133.stdout:(76/157): python3-babel-2.9.1-2.el9.noarch.rpm 38 MB/s | 6.0 MB 00:00 2024-09-10T21:38:38.436 INFO:teuthology.orchestra.run.smithi007.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 61 kB/s | 11 kB 00:00 2024-09-10T21:38:38.436 INFO:teuthology.orchestra.run.smithi133.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.9 MB/s | 245 kB 00:00 2024-09-10T21:38:38.464 INFO:teuthology.orchestra.run.smithi133.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-10T21:38:38.487 INFO:teuthology.orchestra.run.smithi133.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-09-10T21:38:38.521 INFO:teuthology.orchestra.run.smithi133.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 326 kB/s | 19 kB 00:00 2024-09-10T21:38:38.546 INFO:teuthology.orchestra.run.smithi133.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-09-10T21:38:38.571 INFO:teuthology.orchestra.run.smithi133.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 690 kB/s | 35 kB 00:00 2024-09-10T21:38:38.628 INFO:teuthology.orchestra.run.smithi007.stdout:(56/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.6 MB/s | 3.0 MB 00:00 2024-09-10T21:38:38.640 INFO:teuthology.orchestra.run.smithi133.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.6 MB/s | 250 kB 00:00 2024-09-10T21:38:38.714 INFO:teuthology.orchestra.run.smithi133.stdout:(84/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 30 MB/s | 5.1 MB 00:00 2024-09-10T21:38:38.739 INFO:teuthology.orchestra.run.smithi133.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-10T21:38:38.773 INFO:teuthology.orchestra.run.smithi133.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-10T21:38:38.798 INFO:teuthology.orchestra.run.smithi133.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 724 kB/s | 42 kB 00:00 2024-09-10T21:38:38.823 INFO:teuthology.orchestra.run.smithi133.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-10T21:38:38.848 INFO:teuthology.orchestra.run.smithi133.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-10T21:38:38.853 INFO:teuthology.orchestra.run.smithi007.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 409 kB/s | 171 kB 00:00 2024-09-10T21:38:38.876 INFO:teuthology.orchestra.run.smithi133.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-10T21:38:38.899 INFO:teuthology.orchestra.run.smithi133.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-10T21:38:38.924 INFO:teuthology.orchestra.run.smithi133.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-10T21:38:38.966 INFO:teuthology.orchestra.run.smithi133.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-10T21:38:39.008 INFO:teuthology.orchestra.run.smithi133.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-10T21:38:39.045 INFO:teuthology.orchestra.run.smithi007.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 384 kB/s | 160 kB 00:00 2024-09-10T21:38:39.058 INFO:teuthology.orchestra.run.smithi133.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-10T21:38:39.101 INFO:teuthology.orchestra.run.smithi133.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 865 kB/s | 36 kB 00:00 2024-09-10T21:38:39.196 INFO:teuthology.orchestra.run.smithi007.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 132 kB/s | 45 kB 00:00 2024-09-10T21:38:39.276 INFO:teuthology.orchestra.run.smithi133.stdout:(97/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 43 MB/s | 16 MB 00:00 2024-09-10T21:38:39.318 INFO:teuthology.orchestra.run.smithi133.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 218 kB/s | 47 kB 00:00 2024-09-10T21:38:39.351 INFO:teuthology.orchestra.run.smithi133.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 4.0 MB/s | 304 kB 00:00 2024-09-10T21:38:39.418 INFO:teuthology.orchestra.run.smithi133.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 578 kB/s | 58 kB 00:00 2024-09-10T21:38:39.444 INFO:teuthology.orchestra.run.smithi133.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.0 MB/s | 96 kB 00:00 2024-09-10T21:38:39.486 INFO:teuthology.orchestra.run.smithi133.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 952 kB/s | 64 kB 00:00 2024-09-10T21:38:39.527 INFO:teuthology.orchestra.run.smithi133.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 537 kB/s | 45 kB 00:00 2024-09-10T21:38:39.611 INFO:teuthology.orchestra.run.smithi133.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 178 kB/s | 22 kB 00:00 2024-09-10T21:38:39.646 INFO:teuthology.orchestra.run.smithi007.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 1.1 MB/s | 662 kB 00:00 2024-09-10T21:38:39.712 INFO:teuthology.orchestra.run.smithi133.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 5.4 MB/s | 548 kB 00:00 2024-09-10T21:38:39.745 INFO:teuthology.orchestra.run.smithi133.stdout:(106/157): protobuf-compiler-3.14.0-13.el9.x86_ 3.9 MB/s | 863 kB 00:00 2024-09-10T21:38:39.787 INFO:teuthology.orchestra.run.smithi133.stdout:(107/157): gperftools-libs-2.9.1-2.el9.x86_64.r 4.0 MB/s | 309 kB 00:00 2024-09-10T21:38:39.788 INFO:teuthology.orchestra.run.smithi007.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 982 kB/s | 581 kB 00:00 2024-09-10T21:38:39.837 INFO:teuthology.orchestra.run.smithi133.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-09-10T21:38:39.913 INFO:teuthology.orchestra.run.smithi007.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 92 kB/s | 24 kB 00:00 2024-09-10T21:38:40.038 INFO:teuthology.orchestra.run.smithi133.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 130 kB/s | 26 kB 00:00 2024-09-10T21:38:40.080 INFO:teuthology.orchestra.run.smithi133.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.1 MB/s | 49 kB 00:00 2024-09-10T21:38:40.114 INFO:teuthology.orchestra.run.smithi133.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-10T21:38:40.130 INFO:teuthology.orchestra.run.smithi007.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 139 kB/s | 47 kB 00:00 2024-09-10T21:38:40.139 INFO:teuthology.orchestra.run.smithi133.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-10T21:38:40.206 INFO:teuthology.orchestra.run.smithi133.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-10T21:38:40.273 INFO:teuthology.orchestra.run.smithi133.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 8.1 MB/s | 548 kB 00:00 2024-09-10T21:38:40.315 INFO:teuthology.orchestra.run.smithi133.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.0 MB/s | 43 kB 00:00 2024-09-10T21:38:40.357 INFO:teuthology.orchestra.run.smithi133.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 773 kB/s | 32 kB 00:00 2024-09-10T21:38:40.399 INFO:teuthology.orchestra.run.smithi133.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 340 kB/s | 14 kB 00:00 2024-09-10T21:38:40.422 INFO:teuthology.orchestra.run.smithi007.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 575 kB/s | 292 kB 00:00 2024-09-10T21:38:40.466 INFO:teuthology.orchestra.run.smithi133.stdout:(118/157): libarrow-9.0.0-11.el9.x86_64.rpm 6.5 MB/s | 4.4 MB 00:00 2024-09-10T21:38:40.491 INFO:teuthology.orchestra.run.smithi133.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-09-10T21:38:40.524 INFO:teuthology.orchestra.run.smithi133.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-10T21:38:40.575 INFO:teuthology.orchestra.run.smithi133.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.6 MB/s | 303 kB 00:00 2024-09-10T21:38:40.589 INFO:teuthology.orchestra.run.smithi007.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 411 kB/s | 188 kB 00:00 2024-09-10T21:38:40.600 INFO:teuthology.orchestra.run.smithi133.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 2.9 MB/s | 220 kB 00:00 2024-09-10T21:38:40.634 INFO:teuthology.orchestra.run.smithi133.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-10T21:38:40.684 INFO:teuthology.orchestra.run.smithi133.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 214 kB/s | 11 kB 00:00 2024-09-10T21:38:40.739 INFO:teuthology.orchestra.run.smithi007.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 121 kB/s | 38 kB 00:00 2024-09-10T21:38:40.742 INFO:teuthology.orchestra.run.smithi133.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 305 kB/s | 18 kB 00:00 2024-09-10T21:38:40.793 INFO:teuthology.orchestra.run.smithi133.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 465 kB/s | 23 kB 00:00 2024-09-10T21:38:40.843 INFO:teuthology.orchestra.run.smithi133.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 390 kB/s | 19 kB 00:00 2024-09-10T21:38:40.956 INFO:teuthology.orchestra.run.smithi133.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 175 kB/s | 20 kB 00:00 2024-09-10T21:38:41.027 INFO:teuthology.orchestra.run.smithi133.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.5 MB/s | 2.0 MB 00:00 2024-09-10T21:38:41.077 INFO:teuthology.orchestra.run.smithi133.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 935 kB/s | 46 kB 00:00 2024-09-10T21:38:41.103 INFO:teuthology.orchestra.run.smithi133.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-10T21:38:41.153 INFO:teuthology.orchestra.run.smithi133.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-10T21:38:41.195 INFO:teuthology.orchestra.run.smithi133.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 4.3 MB/s | 1.0 MB 00:00 2024-09-10T21:38:41.237 INFO:teuthology.orchestra.run.smithi133.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 399 kB/s | 16 kB 00:00 2024-09-10T21:38:41.262 INFO:teuthology.orchestra.run.smithi133.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-10T21:38:41.296 INFO:teuthology.orchestra.run.smithi133.stdout:(136/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.9 MB/s | 272 kB 00:00 2024-09-10T21:38:41.338 INFO:teuthology.orchestra.run.smithi133.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-09-10T21:38:41.380 INFO:teuthology.orchestra.run.smithi133.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 368 kB/s | 31 kB 00:00 2024-09-10T21:38:41.413 INFO:teuthology.orchestra.run.smithi133.stdout:(139/157): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-09-10T21:38:41.455 INFO:teuthology.orchestra.run.smithi133.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 788 kB/s | 59 kB 00:00 2024-09-10T21:38:41.497 INFO:teuthology.orchestra.run.smithi133.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 429 kB/s | 36 kB 00:00 2024-09-10T21:38:41.523 INFO:teuthology.orchestra.run.smithi133.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.1 MB/s | 74 kB 00:00 2024-09-10T21:38:41.531 INFO:teuthology.orchestra.run.smithi007.stdout:(67/157): openblas-openmp-0.3.26-2.el9.x86_64.r 5.2 MB/s | 4.9 MB 00:00 2024-09-10T21:38:41.548 INFO:teuthology.orchestra.run.smithi133.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.6 MB/s | 90 kB 00:00 2024-09-10T21:38:41.573 INFO:teuthology.orchestra.run.smithi133.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-10T21:38:41.615 INFO:teuthology.orchestra.run.smithi007.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 5.1 MB/s | 4.4 MB 00:00 2024-09-10T21:38:41.615 INFO:teuthology.orchestra.run.smithi133.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 482 kB/s | 20 kB 00:00 2024-09-10T21:38:41.649 INFO:teuthology.orchestra.run.smithi133.stdout:(146/157): re2-20211101-20.el9.x86_64.rpm 5.7 MB/s | 191 kB 00:00 2024-09-10T21:38:41.682 INFO:teuthology.orchestra.run.smithi133.stdout:(147/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-10T21:38:41.775 INFO:teuthology.orchestra.run.smithi133.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-10T21:38:42.241 INFO:teuthology.orchestra.run.smithi007.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 2.2 MB/s | 1.6 MB 00:00 2024-09-10T21:38:42.274 INFO:teuthology.orchestra.run.smithi007.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 1.3 MB/s | 910 kB 00:00 2024-09-10T21:38:42.674 INFO:teuthology.orchestra.run.smithi007.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 317 kB/s | 137 kB 00:00 2024-09-10T21:38:42.775 INFO:teuthology.orchestra.run.smithi133.stdout:(149/157): ceph-test-19.3.0-4854.g8f219c47.el9. 3.8 MB/s | 50 MB 00:13 2024-09-10T21:38:42.842 INFO:teuthology.orchestra.run.smithi133.stdout:(150/157): librados2-19.3.0-4854.g8f219c47.el9. 3.0 MB/s | 3.5 MB 00:01 2024-09-10T21:38:42.959 INFO:teuthology.orchestra.run.smithi133.stdout:(151/157): NetworkManager-1.49.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-09-10T21:38:43.017 INFO:teuthology.orchestra.run.smithi133.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 702 kB/s | 41 kB 00:00 2024-09-10T21:38:43.059 INFO:teuthology.orchestra.run.smithi133.stdout:(153/157): NetworkManager-tui-1.49.90-1.el9.x86 5.9 MB/s | 249 kB 00:00 2024-09-10T21:38:43.109 INFO:teuthology.orchestra.run.smithi133.stdout:(154/157): NetworkManager-libnm-1.49.90-1.el9.x 6.8 MB/s | 1.8 MB 00:00 2024-09-10T21:38:43.151 INFO:teuthology.orchestra.run.smithi133.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 15 MB/s | 1.4 MB 00:00 2024-09-10T21:38:43.218 INFO:teuthology.orchestra.run.smithi133.stdout:(156/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 2.2 MB/s | 3.2 MB 00:01 2024-09-10T21:38:43.285 INFO:teuthology.orchestra.run.smithi133.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 14 MB/s | 2.4 MB 00:00 2024-09-10T21:38:43.290 INFO:teuthology.orchestra.run.smithi133.stdout:-------------------------------------------------------------------------------- 2024-09-10T21:38:43.291 INFO:teuthology.orchestra.run.smithi133.stdout:Total 12 MB/s | 240 MB 00:20 2024-09-10T21:38:43.500 INFO:teuthology.orchestra.run.smithi007.stdout:(72/157): ceph-test-19.3.0-4854.g8f219c47.el9.x 4.2 MB/s | 50 MB 00:12 2024-09-10T21:38:43.741 INFO:teuthology.orchestra.run.smithi007.stdout:(73/157): postgresql-server-13.16-1.el9.x86_64. 3.9 MB/s | 5.8 MB 00:01 2024-09-10T21:38:43.867 INFO:teuthology.orchestra.run.smithi007.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 867 kB/s | 1.0 MB 00:01 2024-09-10T21:38:43.892 INFO:teuthology.orchestra.run.smithi007.stdout:(75/157): python3-attrs-20.3.0-7.el9.noarch.rpm 223 kB/s | 87 kB 00:00 2024-09-10T21:38:44.250 INFO:teuthology.orchestra.run.smithi007.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 160 kB/s | 61 kB 00:00 2024-09-10T21:38:44.392 INFO:teuthology.orchestra.run.smithi007.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 489 kB/s | 245 kB 00:00 2024-09-10T21:38:44.709 INFO:teuthology.orchestra.run.smithi007.stdout:(78/157): python3-babel-2.9.1-2.el9.noarch.rpm 6.2 MB/s | 6.0 MB 00:00 2024-09-10T21:38:44.776 INFO:teuthology.orchestra.run.smithi007.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 67 kB/s | 26 kB 00:00 2024-09-10T21:38:44.802 INFO:teuthology.orchestra.run.smithi007.stdout:(80/157): python3-jinja2-2.11.3-6.el9.noarch.rp 451 kB/s | 248 kB 00:00 2024-09-10T21:38:44.985 INFO:teuthology.orchestra.run.smithi007.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 70 kB/s | 19 kB 00:00 2024-09-10T21:38:45.077 INFO:teuthology.orchestra.run.smithi007.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 126 kB/s | 35 kB 00:00 2024-09-10T21:38:45.186 INFO:teuthology.orchestra.run.smithi007.stdout:(83/157): python3-mako-1.1.4-6.el9.noarch.rpm 420 kB/s | 172 kB 00:00 2024-09-10T21:38:45.377 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction check 2024-09-10T21:38:45.558 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction check succeeded. 2024-09-10T21:38:45.559 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction test 2024-09-10T21:38:45.578 INFO:teuthology.orchestra.run.smithi007.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 501 kB/s | 250 kB 00:00 2024-09-10T21:38:45.720 INFO:teuthology.orchestra.run.smithi007.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 416 kB/s | 222 kB 00:00 2024-09-10T21:38:45.920 INFO:teuthology.orchestra.run.smithi007.stdout:(86/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.5 MB/s | 5.1 MB 00:00 2024-09-10T21:38:45.954 INFO:teuthology.orchestra.run.smithi007.stdout:(87/157): python3-packaging-20.9-5.el9.noarch.r 205 kB/s | 77 kB 00:00 2024-09-10T21:38:46.063 INFO:teuthology.orchestra.run.smithi007.stdout:(88/157): python3-prettytable-0.7.2-27.el9.noar 123 kB/s | 42 kB 00:00 2024-09-10T21:38:46.404 INFO:teuthology.orchestra.run.smithi007.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 353 kB/s | 159 kB 00:00 2024-09-10T21:38:46.438 INFO:teuthology.orchestra.run.smithi007.stdout:(90/157): python3-protobuf-3.14.0-13.el9.noarch 521 kB/s | 269 kB 00:00 2024-09-10T21:38:46.572 INFO:teuthology.orchestra.run.smithi007.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 548 kB/s | 279 kB 00:00 2024-09-10T21:38:46.747 INFO:teuthology.orchestra.run.smithi007.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 149 kB/s | 51 kB 00:00 2024-09-10T21:38:46.780 INFO:teuthology.orchestra.run.smithi007.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 157 kB/s | 54 kB 00:00 2024-09-10T21:38:47.031 INFO:teuthology.orchestra.run.smithi007.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 147 kB/s | 42 kB 00:00 2024-09-10T21:38:47.348 INFO:teuthology.orchestra.run.smithi007.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 78 kB/s | 44 kB 00:00 2024-09-10T21:38:47.694 INFO:teuthology.orchestra.run.smithi007.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 104 kB/s | 36 kB 00:00 2024-09-10T21:38:47.807 INFO:teuthology.orchestra.run.smithi007.stdout:(97/157): qatlib-24.02.0-1.el9.x86_64.rpm 284 kB/s | 221 kB 00:00 2024-09-10T21:38:47.991 INFO:teuthology.orchestra.run.smithi133.stdout:Transaction test succeeded. 2024-09-10T21:38:47.991 INFO:teuthology.orchestra.run.smithi133.stdout:Running transaction 2024-09-10T21:38:48.040 INFO:teuthology.orchestra.run.smithi007.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 137 kB/s | 47 kB 00:00 2024-09-10T21:38:48.299 INFO:teuthology.orchestra.run.smithi007.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 619 kB/s | 304 kB 00:00 2024-09-10T21:38:48.399 INFO:teuthology.orchestra.run.smithi007.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 161 kB/s | 58 kB 00:00 2024-09-10T21:38:48.683 INFO:teuthology.orchestra.run.smithi007.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 250 kB/s | 96 kB 00:00 2024-09-10T21:38:48.758 INFO:teuthology.orchestra.run.smithi007.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 177 kB/s | 64 kB 00:00 2024-09-10T21:38:48.850 INFO:teuthology.orchestra.run.smithi007.stdout:(103/157): lua-devel-5.4.4-4.el9.x86_64.rpm 244 kB/s | 22 kB 00:00 2024-09-10T21:38:49.017 INFO:teuthology.orchestra.run.smithi007.stdout:(104/157): protobuf-compiler-3.14.0-13.el9.x86_ 5.1 MB/s | 863 kB 00:00 2024-09-10T21:38:49.042 INFO:teuthology.orchestra.run.smithi007.stdout:(105/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 125 kB/s | 45 kB 00:00 2024-09-10T21:38:49.126 INFO:teuthology.orchestra.run.smithi007.stdout:(106/157): gperftools-libs-2.9.1-2.el9.x86_64.r 3.6 MB/s | 309 kB 00:00 2024-09-10T21:38:49.151 INFO:teuthology.orchestra.run.smithi007.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 787 kB/s | 19 kB 00:00 2024-09-10T21:38:49.184 INFO:teuthology.orchestra.run.smithi007.stdout:(108/157): abseil-cpp-20211102.0-3.el9.x86_64.r 3.2 MB/s | 548 kB 00:00 2024-09-10T21:38:49.210 INFO:teuthology.orchestra.run.smithi007.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-10T21:38:49.244 INFO:teuthology.orchestra.run.smithi007.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-10T21:38:49.277 INFO:teuthology.orchestra.run.smithi007.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2024-09-10T21:38:49.311 INFO:teuthology.orchestra.run.smithi007.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2024-09-10T21:38:49.427 INFO:teuthology.orchestra.run.smithi007.stdout:(113/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.1 MB/s | 839 kB 00:00 2024-09-10T21:38:49.495 INFO:teuthology.orchestra.run.smithi007.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 7.9 MB/s | 548 kB 00:00 2024-09-10T21:38:49.512 INFO:teuthology.orchestra.run.smithi007.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 2.6 MB/s | 43 kB 00:00 2024-09-10T21:38:49.529 INFO:teuthology.orchestra.run.smithi007.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.9 MB/s | 32 kB 00:00 2024-09-10T21:38:49.546 INFO:teuthology.orchestra.run.smithi007.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 861 kB/s | 14 kB 00:00 2024-09-10T21:38:49.588 INFO:teuthology.orchestra.run.smithi007.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 4.0 MB/s | 172 kB 00:00 2024-09-10T21:38:49.666 INFO:teuthology.orchestra.run.smithi007.stdout:(119/157): libarrow-9.0.0-11.el9.x86_64.rpm 8.6 MB/s | 4.4 MB 00:00 2024-09-10T21:38:49.700 INFO:teuthology.orchestra.run.smithi007.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 3.1 MB/s | 358 kB 00:00 2024-09-10T21:38:49.725 INFO:teuthology.orchestra.run.smithi007.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-10T21:38:49.751 INFO:teuthology.orchestra.run.smithi007.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-10T21:38:49.784 INFO:teuthology.orchestra.run.smithi007.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-10T21:38:49.809 INFO:teuthology.orchestra.run.smithi007.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 431 kB/s | 11 kB 00:00 2024-09-10T21:38:49.835 INFO:teuthology.orchestra.run.smithi007.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 711 kB/s | 18 kB 00:00 2024-09-10T21:38:49.880 INFO:teuthology.orchestra.run.smithi007.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 518 kB/s | 23 kB 00:00 2024-09-10T21:38:49.911 INFO:teuthology.orchestra.run.smithi007.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 645 kB/s | 19 kB 00:00 2024-09-10T21:38:49.936 INFO:teuthology.orchestra.run.smithi007.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 792 kB/s | 20 kB 00:00 2024-09-10T21:38:49.994 INFO:teuthology.orchestra.run.smithi007.stdout:(129/157): python3-grpcio-1.46.7-10.el9.x86_64. 7.6 MB/s | 2.0 MB 00:00 2024-09-10T21:38:50.020 INFO:teuthology.orchestra.run.smithi007.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-10T21:38:50.053 INFO:teuthology.orchestra.run.smithi007.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-10T21:38:50.087 INFO:teuthology.orchestra.run.smithi007.stdout:(132/157): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2024-09-10T21:38:50.112 INFO:teuthology.orchestra.run.smithi007.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 987 kB/s | 58 kB 00:00 2024-09-10T21:38:50.129 INFO:teuthology.orchestra.run.smithi007.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 988 kB/s | 16 kB 00:00 2024-09-10T21:38:50.155 INFO:teuthology.orchestra.run.smithi007.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-10T21:38:50.188 INFO:teuthology.orchestra.run.smithi007.stdout:(136/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-10T21:38:50.205 INFO:teuthology.orchestra.run.smithi007.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 565 kB/s | 28 kB 00:00 2024-09-10T21:38:50.222 INFO:teuthology.orchestra.run.smithi007.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 915 kB/s | 31 kB 00:00 2024-09-10T21:38:50.247 INFO:teuthology.orchestra.run.smithi007.stdout:(139/157): python3-routes-2.5.1-5.el9.noarch.rp 4.4 MB/s | 188 kB 00:00 2024-09-10T21:38:50.281 INFO:teuthology.orchestra.run.smithi007.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.0 MB/s | 59 kB 00:00 2024-09-10T21:38:50.306 INFO:teuthology.orchestra.run.smithi007.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 610 kB/s | 36 kB 00:00 2024-09-10T21:38:50.365 INFO:teuthology.orchestra.run.smithi007.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 893 kB/s | 74 kB 00:00 2024-09-10T21:38:50.390 INFO:teuthology.orchestra.run.smithi007.stdout:(143/157): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-10T21:38:50.416 INFO:teuthology.orchestra.run.smithi007.stdout:(144/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-10T21:38:50.432 INFO:teuthology.orchestra.run.smithi007.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-10T21:38:50.475 INFO:teuthology.orchestra.run.smithi007.stdout:(146/157): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-09-10T21:38:50.508 INFO:teuthology.orchestra.run.smithi007.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 2.5 MB/s | 191 kB 00:00 2024-09-10T21:38:50.592 INFO:teuthology.orchestra.run.smithi007.stdout:(148/157): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-09-10T21:38:50.851 INFO:teuthology.orchestra.run.smithi007.stdout:(149/157): librados2-19.3.0-4854.g8f219c47.el9. 10 MB/s | 3.5 MB 00:00 2024-09-10T21:38:50.968 INFO:teuthology.orchestra.run.smithi007.stdout:(150/157): librbd1-19.3.0-4854.g8f219c47.el9.x8 8.6 MB/s | 3.2 MB 00:00 2024-09-10T21:38:51.010 INFO:teuthology.orchestra.run.smithi007.stdout:(151/157): NetworkManager-1.49.90-1.el9.x86_64. 15 MB/s | 2.3 MB 00:00 2024-09-10T21:38:51.060 INFO:teuthology.orchestra.run.smithi007.stdout:(152/157): NetworkManager-team-1.49.90-1.el9.x8 819 kB/s | 41 kB 00:00 2024-09-10T21:38:51.118 INFO:teuthology.orchestra.run.smithi007.stdout:(153/157): NetworkManager-tui-1.49.90-1.el9.x86 4.2 MB/s | 249 kB 00:00 2024-09-10T21:38:51.185 INFO:teuthology.orchestra.run.smithi007.stdout:(154/157): openssl-3.2.2-6.el9.x86_64.rpm 20 MB/s | 1.4 MB 00:00 2024-09-10T21:38:51.269 INFO:teuthology.orchestra.run.smithi007.stdout:(155/157): NetworkManager-libnm-1.49.90-1.el9.x 6.0 MB/s | 1.8 MB 00:00 2024-09-10T21:38:51.327 INFO:teuthology.orchestra.run.smithi007.stdout:(156/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 17 MB/s | 2.4 MB 00:00 2024-09-10T21:38:55.002 INFO:teuthology.orchestra.run.smithi133.stdout: Preparing : 1/1 2024-09-10T21:38:55.148 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T21:38:55.198 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T21:38:55.271 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T21:38:55.308 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T21:38:55.340 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T21:38:55.397 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T21:38:55.765 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T21:38:55.874 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T21:38:56.018 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T21:38:56.049 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T21:38:56.154 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T21:38:56.194 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T21:38:56.228 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T21:38:56.442 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T21:38:56.506 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T21:38:56.538 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T21:38:56.647 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T21:38:56.672 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T21:38:56.754 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T21:38:56.782 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T21:38:56.820 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T21:38:57.052 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T21:38:57.240 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T21:38:57.405 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T21:38:57.538 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T21:38:57.623 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T21:38:57.914 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T21:38:58.056 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T21:38:58.095 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T21:38:58.140 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T21:38:58.221 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T21:38:58.275 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T21:38:58.314 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T21:38:58.368 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T21:38:58.407 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T21:38:58.441 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T21:38:58.531 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T21:38:58.596 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T21:38:58.637 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T21:38:58.682 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T21:38:58.771 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T21:38:58.806 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T21:38:58.850 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T21:38:58.887 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T21:38:58.976 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T21:38:59.031 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T21:38:59.120 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T21:38:59.279 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T21:38:59.333 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T21:38:59.981 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T21:39:00.075 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T21:39:00.112 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T21:39:00.156 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T21:39:00.219 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T21:39:00.290 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T21:39:00.473 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T21:39:00.538 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T21:39:00.641 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T21:39:00.708 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T21:39:00.791 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T21:39:00.840 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T21:39:00.876 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T21:39:00.933 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T21:39:01.000 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T21:39:01.048 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T21:39:01.098 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T21:39:01.268 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T21:39:01.315 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T21:39:01.376 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T21:39:01.501 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T21:39:01.541 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T21:39:01.584 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T21:39:01.630 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T21:39:01.654 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T21:39:01.789 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T21:39:01.962 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T21:39:02.003 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T21:39:02.040 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T21:39:02.078 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T21:39:02.123 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T21:39:02.162 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T21:39:02.199 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T21:39:02.234 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T21:39:02.267 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T21:39:02.310 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T21:39:02.343 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T21:39:02.476 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T21:39:02.543 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T21:39:02.605 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T21:39:02.725 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T21:39:02.784 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T21:39:03.671 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T21:39:03.736 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T21:39:03.770 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T21:39:03.882 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T21:39:03.944 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T21:39:03.998 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T21:39:04.093 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T21:39:04.156 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T21:39:04.391 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T21:39:04.419 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T21:39:04.517 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T21:39:05.318 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T21:39:05.454 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T21:39:06.976 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T21:39:07.073 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T21:39:07.108 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T21:39:07.141 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T21:39:07.517 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T21:39:07.562 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T21:39:07.657 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T21:39:07.693 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T21:39:07.741 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T21:39:08.277 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T21:39:08.315 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T21:39:08.405 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T21:39:08.433 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T21:39:11.090 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T21:39:11.130 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T21:39:11.781 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T21:39:11.824 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T21:39:11.967 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T21:39:11.968 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T21:39:11.968 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T21:39:11.968 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:39:12.009 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T21:39:24.643 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T21:39:24.643 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /sys 2024-09-10T21:39:24.643 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /proc 2024-09-10T21:39:24.644 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /mnt 2024-09-10T21:39:24.644 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /var/tmp 2024-09-10T21:39:24.644 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /home 2024-09-10T21:39:24.644 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /root 2024-09-10T21:39:24.644 INFO:teuthology.orchestra.run.smithi133.stdout:skipping the directory /tmp 2024-09-10T21:39:24.644 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:39:24.871 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T21:39:24.907 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T21:39:24.907 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:39:24.908 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T21:39:24.908 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:39:26.515 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T21:39:26.574 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T21:39:26.574 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:39:26.574 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T21:39:26.575 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T21:39:26.575 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T21:39:26.575 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:39:26.816 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T21:39:26.867 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T21:39:26.867 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:39:26.867 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T21:39:26.867 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T21:39:26.867 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T21:39:26.867 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:39:27.358 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T21:39:27.408 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T21:39:27.409 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:39:27.409 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T21:39:27.409 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T21:39:27.409 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T21:39:27.409 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:39:27.533 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T21:39:27.592 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T21:39:27.656 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T21:39:27.882 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T21:39:27.953 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T21:39:28.040 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T21:39:28.148 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T21:39:28.311 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T21:39:28.369 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T21:39:28.409 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T21:39:28.435 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T21:39:29.135 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T21:39:29.168 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T21:39:29.210 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T21:39:29.291 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T21:39:29.332 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T21:39:32.934 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T21:39:32.996 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T21:39:34.083 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T21:39:34.163 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T21:39:34.315 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T21:39:34.493 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T21:39:34.532 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T21:39:34.583 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T21:39:34.583 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:39:34.583 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T21:39:34.583 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T21:39:34.583 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T21:39:34.583 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:39:34.641 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T21:39:34.667 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T21:39:34.774 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T21:39:34.839 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T21:39:35.869 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T21:39:35.910 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T21:39:35.961 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T21:39:35.961 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:39:35.962 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T21:39:35.962 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:39:35.962 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:39:35.962 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:39:36.019 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T21:39:36.069 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T21:39:36.069 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:39:36.069 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T21:39:36.069 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:39:36.128 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T21:39:36.179 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T21:39:36.179 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:39:36.179 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T21:39:36.179 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:39:36.179 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:39:36.179 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:39:36.520 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T21:39:36.571 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T21:39:36.571 INFO:teuthology.orchestra.run.smithi133.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:39:36.571 INFO:teuthology.orchestra.run.smithi133.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T21:39:36.571 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T21:39:36.571 INFO:teuthology.orchestra.run.smithi133.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T21:39:36.572 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:39:41.480 INFO:teuthology.orchestra.run.smithi007.stdout:(157/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 303 kB/s | 16 MB 00:54 2024-09-10T21:39:41.485 INFO:teuthology.orchestra.run.smithi007.stdout:-------------------------------------------------------------------------------- 2024-09-10T21:39:41.486 INFO:teuthology.orchestra.run.smithi007.stdout:Total 3.2 MB/s | 240 MB 01:15 2024-09-10T21:39:41.651 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T21:39:41.691 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T21:39:41.735 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T21:39:41.764 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T21:39:41.800 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T21:39:42.036 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T21:39:42.097 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T21:39:42.127 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T21:39:42.158 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T21:39:42.202 INFO:teuthology.orchestra.run.smithi133.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T21:39:43.031 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T21:39:43.074 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T21:39:43.116 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T21:39:43.597 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction check 2024-09-10T21:39:43.799 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction check succeeded. 2024-09-10T21:39:43.799 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction test 2024-09-10T21:39:44.376 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T21:39:44.480 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T21:39:44.536 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T21:39:44.643 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T21:39:44.682 INFO:teuthology.orchestra.run.smithi133.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T21:39:44.708 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T21:39:44.744 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T21:39:44.744 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T21:39:44.885 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T21:39:44.885 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T21:39:45.019 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T21:39:45.043 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T21:39:45.059 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T21:39:45.059 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T21:39:45.296 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T21:39:45.296 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T21:39:45.413 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T21:39:45.494 INFO:teuthology.orchestra.run.smithi133.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T21:39:45.699 INFO:teuthology.orchestra.run.smithi133.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T21:39:45.699 INFO:teuthology.orchestra.run.smithi133.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-10T21:39:45.699 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:39:46.361 INFO:teuthology.orchestra.run.smithi007.stdout:Transaction test succeeded. 2024-09-10T21:39:46.361 INFO:teuthology.orchestra.run.smithi007.stdout:Running transaction 2024-09-10T21:39:52.032 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T21:39:52.033 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T21:39:52.033 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T21:39:52.033 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T21:39:52.033 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T21:39:52.033 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T21:39:52.033 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T21:39:52.033 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T21:39:52.033 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T21:39:52.033 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T21:39:52.034 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T21:39:52.034 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T21:39:52.034 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T21:39:52.034 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T21:39:52.034 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T21:39:52.034 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T21:39:52.034 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T21:39:52.034 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T21:39:52.034 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T21:39:52.034 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T21:39:52.034 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T21:39:52.034 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T21:39:52.034 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T21:39:52.035 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T21:39:52.035 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T21:39:52.035 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T21:39:52.046 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T21:39:52.046 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T21:39:52.046 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T21:39:52.047 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T21:39:52.047 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T21:39:52.047 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T21:39:52.047 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T21:39:52.047 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T21:39:52.047 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T21:39:52.047 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T21:39:52.047 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T21:39:52.047 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T21:39:52.047 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T21:39:52.047 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T21:39:52.048 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T21:39:52.048 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T21:39:52.048 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T21:39:52.048 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T21:39:52.048 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T21:39:52.048 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T21:39:52.048 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T21:39:52.048 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T21:39:52.048 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T21:39:52.048 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T21:39:52.048 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T21:39:52.048 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T21:39:52.049 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T21:39:52.049 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T21:39:52.049 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T21:39:52.049 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T21:39:52.049 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T21:39:52.049 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T21:39:52.049 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T21:39:52.049 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T21:39:52.049 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T21:39:52.049 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T21:39:52.049 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T21:39:52.049 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T21:39:52.050 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T21:39:52.050 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T21:39:52.050 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T21:39:52.050 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T21:39:52.050 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T21:39:52.050 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T21:39:52.050 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T21:39:52.050 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T21:39:52.050 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T21:39:52.050 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T21:39:52.050 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T21:39:52.050 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T21:39:52.050 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T21:39:52.050 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T21:39:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T21:39:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T21:39:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T21:39:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T21:39:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T21:39:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T21:39:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T21:39:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T21:39:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T21:39:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T21:39:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T21:39:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T21:39:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T21:39:52.051 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T21:39:52.052 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T21:39:52.052 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T21:39:52.052 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T21:39:52.052 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T21:39:52.052 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T21:39:52.052 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T21:39:52.052 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T21:39:52.052 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T21:39:52.052 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T21:39:52.052 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T21:39:52.052 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T21:39:52.052 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T21:39:52.052 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T21:39:52.052 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T21:39:52.053 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T21:39:52.053 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T21:39:52.053 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T21:39:52.053 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T21:39:52.053 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T21:39:52.053 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T21:39:52.053 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T21:39:52.053 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T21:39:52.053 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T21:39:52.053 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T21:39:52.053 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T21:39:52.053 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T21:39:52.053 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T21:39:52.054 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T21:39:52.054 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T21:39:52.054 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T21:39:52.054 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T21:39:52.054 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T21:39:52.054 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T21:39:52.054 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T21:39:52.054 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T21:39:52.054 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T21:39:52.054 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T21:39:52.054 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T21:39:52.054 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T21:39:52.054 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T21:39:52.054 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T21:39:52.055 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T21:39:52.055 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T21:39:52.055 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T21:39:52.055 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T21:39:52.055 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T21:39:52.055 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T21:39:52.055 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T21:39:52.055 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T21:39:52.055 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T21:39:52.055 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T21:39:52.055 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T21:39:52.055 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T21:39:52.055 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T21:39:52.056 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T21:39:52.056 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T21:39:52.056 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T21:39:52.056 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T21:39:52.056 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T21:39:52.056 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T21:39:52.056 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T21:39:52.056 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T21:39:52.056 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T21:39:52.056 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T21:39:52.056 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T21:39:52.056 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T21:39:52.057 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T21:39:52.057 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T21:39:52.057 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T21:39:52.057 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T21:39:52.057 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T21:39:52.057 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T21:39:53.300 INFO:teuthology.orchestra.run.smithi007.stdout: Preparing : 1/1 2024-09-10T21:39:53.443 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-10T21:39:53.500 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-10T21:39:53.585 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-10T21:39:53.629 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-10T21:39:53.660 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-10T21:39:53.728 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-10T21:39:54.128 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-10T21:39:54.244 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T21:39:54.261 INFO:teuthology.orchestra.run.smithi133.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T21:39:54.261 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:39:54.261 INFO:teuthology.orchestra.run.smithi133.stdout:Upgraded: 2024-09-10T21:39:54.261 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T21:39:54.261 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T21:39:54.261 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T21:39:54.261 INFO:teuthology.orchestra.run.smithi133.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T21:39:54.261 INFO:teuthology.orchestra.run.smithi133.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.262 INFO:teuthology.orchestra.run.smithi133.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.262 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T21:39:54.262 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T21:39:54.262 INFO:teuthology.orchestra.run.smithi133.stdout:Installed: 2024-09-10T21:39:54.262 INFO:teuthology.orchestra.run.smithi133.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T21:39:54.262 INFO:teuthology.orchestra.run.smithi133.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T21:39:54.262 INFO:teuthology.orchestra.run.smithi133.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T21:39:54.262 INFO:teuthology.orchestra.run.smithi133.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T21:39:54.262 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.262 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.262 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.262 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.262 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:39:54.263 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.263 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.263 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.263 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:39:54.263 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:39:54.263 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:39:54.263 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:39:54.263 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:39:54.263 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.263 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.263 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:39:54.263 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.263 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.263 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.264 INFO:teuthology.orchestra.run.smithi133.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:39:54.264 INFO:teuthology.orchestra.run.smithi133.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:39:54.264 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.264 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.264 INFO:teuthology.orchestra.run.smithi133.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:39:54.264 INFO:teuthology.orchestra.run.smithi133.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T21:39:54.264 INFO:teuthology.orchestra.run.smithi133.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T21:39:54.264 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T21:39:54.264 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T21:39:54.264 INFO:teuthology.orchestra.run.smithi133.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T21:39:54.264 INFO:teuthology.orchestra.run.smithi133.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T21:39:54.264 INFO:teuthology.orchestra.run.smithi133.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T21:39:54.264 INFO:teuthology.orchestra.run.smithi133.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T21:39:54.264 INFO:teuthology.orchestra.run.smithi133.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T21:39:54.264 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T21:39:54.265 INFO:teuthology.orchestra.run.smithi133.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T21:39:54.265 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.265 INFO:teuthology.orchestra.run.smithi133.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.265 INFO:teuthology.orchestra.run.smithi133.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.265 INFO:teuthology.orchestra.run.smithi133.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T21:39:54.265 INFO:teuthology.orchestra.run.smithi133.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T21:39:54.265 INFO:teuthology.orchestra.run.smithi133.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T21:39:54.265 INFO:teuthology.orchestra.run.smithi133.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T21:39:54.265 INFO:teuthology.orchestra.run.smithi133.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T21:39:54.265 INFO:teuthology.orchestra.run.smithi133.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T21:39:54.265 INFO:teuthology.orchestra.run.smithi133.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.265 INFO:teuthology.orchestra.run.smithi133.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.266 INFO:teuthology.orchestra.run.smithi133.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T21:39:54.266 INFO:teuthology.orchestra.run.smithi133.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.266 INFO:teuthology.orchestra.run.smithi133.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T21:39:54.266 INFO:teuthology.orchestra.run.smithi133.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T21:39:54.266 INFO:teuthology.orchestra.run.smithi133.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T21:39:54.266 INFO:teuthology.orchestra.run.smithi133.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T21:39:54.266 INFO:teuthology.orchestra.run.smithi133.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T21:39:54.266 INFO:teuthology.orchestra.run.smithi133.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T21:39:54.266 INFO:teuthology.orchestra.run.smithi133.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T21:39:54.266 INFO:teuthology.orchestra.run.smithi133.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T21:39:54.266 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T21:39:54.266 INFO:teuthology.orchestra.run.smithi133.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T21:39:54.266 INFO:teuthology.orchestra.run.smithi133.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T21:39:54.267 INFO:teuthology.orchestra.run.smithi133.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T21:39:54.267 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T21:39:54.267 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T21:39:54.267 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T21:39:54.267 INFO:teuthology.orchestra.run.smithi133.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T21:39:54.267 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T21:39:54.267 INFO:teuthology.orchestra.run.smithi133.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T21:39:54.267 INFO:teuthology.orchestra.run.smithi133.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T21:39:54.267 INFO:teuthology.orchestra.run.smithi133.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T21:39:54.267 INFO:teuthology.orchestra.run.smithi133.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T21:39:54.267 INFO:teuthology.orchestra.run.smithi133.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T21:39:54.267 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T21:39:54.267 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.268 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.268 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.268 INFO:teuthology.orchestra.run.smithi133.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T21:39:54.268 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T21:39:54.268 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T21:39:54.268 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T21:39:54.268 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T21:39:54.268 INFO:teuthology.orchestra.run.smithi133.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T21:39:54.268 INFO:teuthology.orchestra.run.smithi133.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T21:39:54.268 INFO:teuthology.orchestra.run.smithi133.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T21:39:54.268 INFO:teuthology.orchestra.run.smithi133.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T21:39:54.268 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T21:39:54.268 INFO:teuthology.orchestra.run.smithi133.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T21:39:54.269 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T21:39:54.269 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T21:39:54.269 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T21:39:54.269 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T21:39:54.269 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T21:39:54.269 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T21:39:54.269 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T21:39:54.269 INFO:teuthology.orchestra.run.smithi133.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T21:39:54.269 INFO:teuthology.orchestra.run.smithi133.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T21:39:54.269 INFO:teuthology.orchestra.run.smithi133.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T21:39:54.269 INFO:teuthology.orchestra.run.smithi133.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T21:39:54.269 INFO:teuthology.orchestra.run.smithi133.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T21:39:54.269 INFO:teuthology.orchestra.run.smithi133.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T21:39:54.270 INFO:teuthology.orchestra.run.smithi133.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T21:39:54.270 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T21:39:54.270 INFO:teuthology.orchestra.run.smithi133.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T21:39:54.270 INFO:teuthology.orchestra.run.smithi133.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T21:39:54.270 INFO:teuthology.orchestra.run.smithi133.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T21:39:54.270 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T21:39:54.270 INFO:teuthology.orchestra.run.smithi133.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T21:39:54.270 INFO:teuthology.orchestra.run.smithi133.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T21:39:54.270 INFO:teuthology.orchestra.run.smithi133.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T21:39:54.270 INFO:teuthology.orchestra.run.smithi133.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T21:39:54.270 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T21:39:54.270 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T21:39:54.270 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T21:39:54.270 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T21:39:54.271 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T21:39:54.271 INFO:teuthology.orchestra.run.smithi133.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T21:39:54.271 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.271 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.271 INFO:teuthology.orchestra.run.smithi133.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T21:39:54.271 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T21:39:54.271 INFO:teuthology.orchestra.run.smithi133.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T21:39:54.271 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.271 INFO:teuthology.orchestra.run.smithi133.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T21:39:54.271 INFO:teuthology.orchestra.run.smithi133.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T21:39:54.271 INFO:teuthology.orchestra.run.smithi133.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T21:39:54.271 INFO:teuthology.orchestra.run.smithi133.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T21:39:54.271 INFO:teuthology.orchestra.run.smithi133.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T21:39:54.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T21:39:54.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T21:39:54.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T21:39:54.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T21:39:54.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T21:39:54.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T21:39:54.272 INFO:teuthology.orchestra.run.smithi133.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T21:39:54.272 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T21:39:54.272 INFO:teuthology.orchestra.run.smithi133.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T21:39:54.272 INFO:teuthology.orchestra.run.smithi133.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T21:39:54.272 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.272 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.272 INFO:teuthology.orchestra.run.smithi133.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:39:54.273 INFO:teuthology.orchestra.run.smithi133.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T21:39:54.273 INFO:teuthology.orchestra.run.smithi133.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T21:39:54.273 INFO:teuthology.orchestra.run.smithi133.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T21:39:54.273 INFO:teuthology.orchestra.run.smithi133.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T21:39:54.273 INFO:teuthology.orchestra.run.smithi133.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T21:39:54.273 INFO:teuthology.orchestra.run.smithi133.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T21:39:54.273 INFO:teuthology.orchestra.run.smithi133.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T21:39:54.273 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:39:54.273 INFO:teuthology.orchestra.run.smithi133.stdout:Complete! 2024-09-10T21:39:54.393 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T21:39:54.454 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T21:39:54.568 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T21:39:54.610 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T21:39:54.650 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-10T21:39:54.878 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-10T21:39:54.935 DEBUG:teuthology.parallel:result is None 2024-09-10T21:39:54.966 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-10T21:39:54.992 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T21:39:55.121 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T21:39:55.151 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T21:39:55.234 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 15/165 2024-09-10T21:39:55.261 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T21:39:55.312 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-10T21:39:55.580 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T21:39:55.750 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-10T21:39:55.936 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 18/165 2024-09-10T21:39:56.076 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T21:39:56.161 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T21:39:56.445 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 19/165 2024-09-10T21:39:56.599 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-10T21:39:56.633 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-10T21:39:56.678 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-10T21:39:56.764 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-10T21:39:56.823 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-10T21:39:56.879 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-10T21:39:56.941 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-10T21:39:56.980 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-10T21:39:57.013 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-10T21:39:57.106 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-10T21:39:57.176 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-10T21:39:57.209 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-10T21:39:57.264 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-10T21:39:57.357 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-10T21:39:57.396 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 34/165 2024-09-10T21:39:57.441 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 35/165 2024-09-10T21:39:57.468 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 36/165 2024-09-10T21:39:57.564 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-10T21:39:57.630 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-10T21:39:57.725 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-10T21:39:57.897 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-10T21:39:57.992 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-10T21:39:58.665 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-10T21:39:58.791 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-10T21:39:58.836 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-10T21:39:58.880 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-10T21:39:58.918 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 46/165 2024-09-10T21:39:58.971 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-10T21:39:59.161 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-10T21:39:59.255 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-10T21:39:59.373 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-10T21:39:59.447 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-10T21:39:59.532 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-10T21:39:59.569 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-10T21:39:59.614 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-10T21:39:59.682 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-10T21:39:59.730 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-10T21:39:59.776 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-10T21:39:59.826 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-10T21:40:00.004 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-10T21:40:00.050 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-10T21:40:00.096 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-10T21:40:00.257 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-10T21:40:00.300 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-10T21:40:00.343 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-10T21:40:00.373 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 65/165 2024-09-10T21:40:00.397 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-10T21:40:00.511 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-10T21:40:00.680 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-10T21:40:00.730 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-10T21:40:00.771 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-10T21:40:00.827 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-10T21:40:00.901 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-10T21:40:00.937 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-10T21:40:00.974 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-10T21:40:01.009 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-10T21:40:01.042 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-10T21:40:01.111 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-10T21:40:01.151 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-10T21:40:01.254 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-10T21:40:01.311 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-10T21:40:01.356 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-10T21:40:01.482 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-10T21:40:01.565 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-10T21:40:02.517 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-10T21:40:02.571 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-10T21:40:02.604 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-10T21:40:02.723 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-10T21:40:02.780 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-10T21:40:02.834 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-10T21:40:02.951 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-10T21:40:02.999 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-10T21:40:03.247 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-10T21:40:03.278 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-10T21:40:03.384 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-10T21:40:04.611 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-10T21:40:04.764 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-10T21:40:06.428 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-10T21:40:06.526 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-10T21:40:06.592 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-10T21:40:06.637 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-10T21:40:07.016 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-10T21:40:07.090 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T21:40:07.204 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 102/165 2024-09-10T21:40:07.247 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 103/165 2024-09-10T21:40:07.295 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-10T21:40:07.866 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-10T21:40:07.904 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T21:40:08.018 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 106/165 2024-09-10T21:40:08.045 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 107/165 2024-09-10T21:40:10.773 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T21:40:10.817 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T21:40:11.522 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 108/165 2024-09-10T21:40:11.572 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T21:40:11.718 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 109/165 2024-09-10T21:40:11.718 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T21:40:11.718 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T21:40:11.718 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:40:11.805 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T21:40:25.084 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 110/165 2024-09-10T21:40:25.084 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /sys 2024-09-10T21:40:25.085 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /proc 2024-09-10T21:40:25.085 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /mnt 2024-09-10T21:40:25.085 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /var/tmp 2024-09-10T21:40:25.085 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /home 2024-09-10T21:40:25.085 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /root 2024-09-10T21:40:25.085 INFO:teuthology.orchestra.run.smithi007.stdout:skipping the directory /tmp 2024-09-10T21:40:25.085 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:40:25.302 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T21:40:25.345 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 111/165 2024-09-10T21:40:25.345 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:40:25.345 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T21:40:25.345 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:40:27.047 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T21:40:27.106 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 112/165 2024-09-10T21:40:27.106 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:40:27.106 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T21:40:27.106 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T21:40:27.106 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T21:40:27.106 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:40:27.364 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T21:40:27.415 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 113/165 2024-09-10T21:40:27.415 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:40:27.415 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T21:40:27.415 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T21:40:27.415 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T21:40:27.416 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:40:27.940 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T21:40:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 114/165 2024-09-10T21:40:27.991 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:40:27.991 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T21:40:27.991 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T21:40:27.991 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T21:40:27.991 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:40:28.132 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-10T21:40:28.194 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-10T21:40:28.291 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-10T21:40:28.538 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-10T21:40:28.603 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-10T21:40:28.693 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-10T21:40:28.829 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-10T21:40:28.997 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-10T21:40:29.053 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-10T21:40:29.093 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-10T21:40:29.125 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-10T21:40:29.753 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T21:40:29.784 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 126/165 2024-09-10T21:40:29.819 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 127/165 2024-09-10T21:40:29.902 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 128/165 2024-09-10T21:40:29.931 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T21:40:33.833 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 129/165 2024-09-10T21:40:33.896 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T21:40:35.066 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 130/165 2024-09-10T21:40:35.096 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T21:40:35.256 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 131/165 2024-09-10T21:40:35.445 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 132/165 2024-09-10T21:40:35.473 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T21:40:35.524 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 133/165 2024-09-10T21:40:35.525 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:40:35.525 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T21:40:35.525 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T21:40:35.525 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T21:40:35.525 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:40:35.582 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T21:40:35.608 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 134/165 2024-09-10T21:40:35.751 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 135/165 2024-09-10T21:40:35.822 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 136/165 2024-09-10T21:40:36.922 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-10T21:40:36.951 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T21:40:37.004 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 138/165 2024-09-10T21:40:37.004 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:40:37.004 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T21:40:37.004 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:40:37.004 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:40:37.005 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:40:37.060 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T21:40:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-4854.g8f219 139/165 2024-09-10T21:40:37.111 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:40:37.111 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T21:40:37.111 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:40:37.161 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T21:40:37.213 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 140/165 2024-09-10T21:40:37.213 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:40:37.213 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-10T21:40:37.213 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:40:37.213 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:40:37.213 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:40:37.578 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T21:40:37.629 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 141/165 2024-09-10T21:40:37.630 INFO:teuthology.orchestra.run.smithi007.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T21:40:37.630 INFO:teuthology.orchestra.run.smithi007.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T21:40:37.630 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T21:40:37.630 INFO:teuthology.orchestra.run.smithi007.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T21:40:37.630 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:40:43.085 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 142/165 2024-09-10T21:40:43.135 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 143/165 2024-09-10T21:40:43.179 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 144/165 2024-09-10T21:40:43.215 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-10T21:40:43.251 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-10T21:40:43.505 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 147/165 2024-09-10T21:40:43.565 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-10T21:40:43.594 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 149/165 2024-09-10T21:40:43.626 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 150/165 2024-09-10T21:40:43.669 INFO:teuthology.orchestra.run.smithi007.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 151/165 2024-09-10T21:40:44.414 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T21:40:44.467 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T21:40:44.509 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-10T21:40:45.876 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 153/165 2024-09-10T21:40:45.974 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-10T21:40:46.019 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-10T21:40:46.133 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-10T21:40:46.185 INFO:teuthology.orchestra.run.smithi007.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-10T21:40:46.235 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-10T21:40:46.271 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-10T21:40:46.272 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T21:40:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-10T21:40:46.371 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T21:40:46.479 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-10T21:40:46.504 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-10T21:40:46.520 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T21:40:46.520 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T21:40:46.773 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-10T21:40:46.773 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T21:40:46.883 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-10T21:40:46.960 INFO:teuthology.orchestra.run.smithi007.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T21:40:47.153 INFO:teuthology.orchestra.run.smithi007.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T21:40:47.153 INFO:teuthology.orchestra.run.smithi007.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-10T21:40:47.153 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:40:53.975 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 1/165 2024-09-10T21:40:53.976 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2/165 2024-09-10T21:40:53.976 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 3/165 2024-09-10T21:40:53.976 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 4/165 2024-09-10T21:40:53.976 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-4854.g8f219 5/165 2024-09-10T21:40:53.976 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 6/165 2024-09-10T21:40:53.976 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 7/165 2024-09-10T21:40:53.976 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 8/165 2024-09-10T21:40:53.976 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 9/165 2024-09-10T21:40:53.976 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 10/165 2024-09-10T21:40:53.976 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 11/165 2024-09-10T21:40:53.977 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 12/165 2024-09-10T21:40:53.977 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 13/165 2024-09-10T21:40:53.977 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 14/165 2024-09-10T21:40:53.977 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_ 15/165 2024-09-10T21:40:53.977 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 16/165 2024-09-10T21:40:53.977 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 17/165 2024-09-10T21:40:53.977 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_6 18/165 2024-09-10T21:40:53.977 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86 19/165 2024-09-10T21:40:53.977 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 20/165 2024-09-10T21:40:53.977 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el 21/165 2024-09-10T21:40:53.977 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ceph-common-2:19.3.0-4854.g8f219c47.el9. 22/165 2024-09-10T21:40:53.977 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_6 23/165 2024-09-10T21:40:53.977 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 24/165 2024-09-10T21:40:53.978 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 25/165 2024-09-10T21:40:53.989 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 26/165 2024-09-10T21:40:53.990 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 27/165 2024-09-10T21:40:53.990 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 28/165 2024-09-10T21:40:53.990 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 29/165 2024-09-10T21:40:53.990 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47. 30/165 2024-09-10T21:40:53.990 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noa 31/165 2024-09-10T21:40:53.990 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.n 32/165 2024-09-10T21:40:53.990 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f2 33/165 2024-09-10T21:40:53.990 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el 34/165 2024-09-10T21:40:53.990 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 35/165 2024-09-10T21:40:53.990 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.e 36/165 2024-09-10T21:40:53.990 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 37/165 2024-09-10T21:40:53.991 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 38/165 2024-09-10T21:40:53.991 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 39/165 2024-09-10T21:40:53.991 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-10T21:40:53.991 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-10T21:40:53.991 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-10T21:40:53.991 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-10T21:40:53.991 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-10T21:40:53.991 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-10T21:40:53.991 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-10T21:40:53.991 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-10T21:40:53.991 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-10T21:40:53.991 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-10T21:40:53.991 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-10T21:40:53.991 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-10T21:40:53.992 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-10T21:40:53.992 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-10T21:40:53.992 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-10T21:40:53.992 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-10T21:40:53.992 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-10T21:40:53.992 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-10T21:40:53.992 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-10T21:40:53.992 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-10T21:40:53.992 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-10T21:40:53.992 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-10T21:40:53.992 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-10T21:40:53.992 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-10T21:40:53.993 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-10T21:40:53.993 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-10T21:40:53.993 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-10T21:40:53.993 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-10T21:40:53.993 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-10T21:40:53.993 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-10T21:40:53.993 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-10T21:40:53.993 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-10T21:40:53.993 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-10T21:40:53.993 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-10T21:40:53.993 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-10T21:40:53.993 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-10T21:40:53.993 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-10T21:40:53.994 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-10T21:40:53.994 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-10T21:40:53.994 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-10T21:40:53.994 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-10T21:40:53.994 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-10T21:40:53.994 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-10T21:40:53.994 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-10T21:40:53.994 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-10T21:40:53.994 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-10T21:40:53.994 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-10T21:40:53.994 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-10T21:40:53.994 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-10T21:40:53.994 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-10T21:40:53.994 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-10T21:40:53.995 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-10T21:40:53.995 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-10T21:40:53.995 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-10T21:40:53.995 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-10T21:40:53.995 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-10T21:40:53.995 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-10T21:40:53.995 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-10T21:40:53.995 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-10T21:40:53.995 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-10T21:40:53.995 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-10T21:40:53.995 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-10T21:40:53.995 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-10T21:40:53.995 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-10T21:40:53.996 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-10T21:40:53.996 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-10T21:40:53.996 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-10T21:40:53.996 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-10T21:40:53.996 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/165 2024-09-10T21:40:53.996 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-10T21:40:53.996 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-10T21:40:53.996 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-10T21:40:53.996 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-10T21:40:53.996 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-10T21:40:53.996 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-10T21:40:53.996 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-10T21:40:53.997 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-10T21:40:53.997 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-10T21:40:53.997 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-10T21:40:53.997 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-10T21:40:53.997 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-10T21:40:53.997 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-10T21:40:53.997 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-10T21:40:53.997 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-10T21:40:53.997 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-10T21:40:53.997 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-10T21:40:53.997 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-10T21:40:53.997 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-10T21:40:53.998 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-10T21:40:53.998 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-10T21:40:53.998 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-10T21:40:53.998 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-10T21:40:53.998 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-10T21:40:53.998 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-10T21:40:53.998 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-10T21:40:53.998 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-10T21:40:53.998 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-10T21:40:53.998 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-10T21:40:53.998 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-10T21:40:53.998 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-10T21:40:53.998 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-10T21:40:53.999 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-10T21:40:53.999 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-10T21:40:53.999 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-10T21:40:53.999 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-10T21:40:53.999 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-10T21:40:53.999 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-10T21:40:53.999 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-10T21:40:53.999 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-10T21:40:53.999 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-10T21:40:53.999 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 150/165 2024-09-10T21:40:53.999 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-10T21:40:53.999 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 152/165 2024-09-10T21:40:53.999 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-10T21:40:54.000 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 154/165 2024-09-10T21:40:54.000 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-10T21:40:54.000 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 156/165 2024-09-10T21:40:54.000 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-10T21:40:54.000 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 158/165 2024-09-10T21:40:54.000 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-10T21:40:54.000 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 160/165 2024-09-10T21:40:54.000 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-10T21:40:54.000 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-10T21:40:54.000 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-10T21:40:54.000 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-10T21:40:56.216 INFO:teuthology.orchestra.run.smithi007.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-10T21:40:56.217 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:40:56.217 INFO:teuthology.orchestra.run.smithi007.stdout:Upgraded: 2024-09-10T21:40:56.217 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-10T21:40:56.217 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-10T21:40:56.217 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-10T21:40:56.217 INFO:teuthology.orchestra.run.smithi007.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-10T21:40:56.217 INFO:teuthology.orchestra.run.smithi007.stdout: librados2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.217 INFO:teuthology.orchestra.run.smithi007.stdout: librbd1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.217 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-10T21:40:56.217 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-10T21:40:56.217 INFO:teuthology.orchestra.run.smithi007.stdout:Installed: 2024-09-10T21:40:56.218 INFO:teuthology.orchestra.run.smithi007.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-10T21:40:56.218 INFO:teuthology.orchestra.run.smithi007.stdout: automake-1.16.2-8.el9.noarch 2024-09-10T21:40:56.218 INFO:teuthology.orchestra.run.smithi007.stdout: bison-3.7.4-5.el9.x86_64 2024-09-10T21:40:56.218 INFO:teuthology.orchestra.run.smithi007.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T21:40:56.218 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.218 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-base-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.218 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.218 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.218 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-grafana-dashboards-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:40:56.218 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-immutable-object-cache-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.218 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mds-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.218 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.219 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:40:56.219 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-dashboard-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:40:56.219 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-diskprediction-local-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:40:56.219 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-modules-core-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:40:56.219 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mgr-rook-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:40:56.219 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-mon-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.219 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-osd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.219 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-prometheus-alerts-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:40:56.219 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-radosgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.219 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-selinux-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.219 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-test-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.220 INFO:teuthology.orchestra.run.smithi007.stdout: ceph-volume-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:40:56.220 INFO:teuthology.orchestra.run.smithi007.stdout: cephadm-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:40:56.220 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.220 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-shell-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.220 INFO:teuthology.orchestra.run.smithi007.stdout: cephfs-top-2:19.3.0-4854.g8f219c47.el9.noarch 2024-09-10T21:40:56.220 INFO:teuthology.orchestra.run.smithi007.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-10T21:40:56.220 INFO:teuthology.orchestra.run.smithi007.stdout: flex-2.6.4-9.el9.x86_64 2024-09-10T21:40:56.220 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T21:40:56.220 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T21:40:56.220 INFO:teuthology.orchestra.run.smithi007.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T21:40:56.220 INFO:teuthology.orchestra.run.smithi007.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-10T21:40:56.220 INFO:teuthology.orchestra.run.smithi007.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T21:40:56.220 INFO:teuthology.orchestra.run.smithi007.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-10T21:40:56.221 INFO:teuthology.orchestra.run.smithi007.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-10T21:40:56.221 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T21:40:56.221 INFO:teuthology.orchestra.run.smithi007.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T21:40:56.221 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.221 INFO:teuthology.orchestra.run.smithi007.stdout: libcephfs2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.221 INFO:teuthology.orchestra.run.smithi007.stdout: libcephsqlite-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.221 INFO:teuthology.orchestra.run.smithi007.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T21:40:56.221 INFO:teuthology.orchestra.run.smithi007.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-10T21:40:56.221 INFO:teuthology.orchestra.run.smithi007.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T21:40:56.221 INFO:teuthology.orchestra.run.smithi007.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T21:40:56.221 INFO:teuthology.orchestra.run.smithi007.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T21:40:56.221 INFO:teuthology.orchestra.run.smithi007.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T21:40:56.222 INFO:teuthology.orchestra.run.smithi007.stdout: librados-devel-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.222 INFO:teuthology.orchestra.run.smithi007.stdout: libradosstriper1-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.222 INFO:teuthology.orchestra.run.smithi007.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T21:40:56.222 INFO:teuthology.orchestra.run.smithi007.stdout: librgw2-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.222 INFO:teuthology.orchestra.run.smithi007.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-10T21:40:56.222 INFO:teuthology.orchestra.run.smithi007.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T21:40:56.222 INFO:teuthology.orchestra.run.smithi007.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-10T21:40:56.222 INFO:teuthology.orchestra.run.smithi007.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-10T21:40:56.222 INFO:teuthology.orchestra.run.smithi007.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T21:40:56.222 INFO:teuthology.orchestra.run.smithi007.stdout: lua-5.4.4-4.el9.x86_64 2024-09-10T21:40:56.222 INFO:teuthology.orchestra.run.smithi007.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-10T21:40:56.222 INFO:teuthology.orchestra.run.smithi007.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-10T21:40:56.223 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T21:40:56.223 INFO:teuthology.orchestra.run.smithi007.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T21:40:56.223 INFO:teuthology.orchestra.run.smithi007.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-10T21:40:56.223 INFO:teuthology.orchestra.run.smithi007.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T21:40:56.223 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-13.16-1.el9.x86_64 2024-09-10T21:40:56.223 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-10T21:40:56.223 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-10T21:40:56.223 INFO:teuthology.orchestra.run.smithi007.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-10T21:40:56.223 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-10T21:40:56.223 INFO:teuthology.orchestra.run.smithi007.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-10T21:40:56.223 INFO:teuthology.orchestra.run.smithi007.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T21:40:56.223 INFO:teuthology.orchestra.run.smithi007.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-10T21:40:56.223 INFO:teuthology.orchestra.run.smithi007.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T21:40:56.224 INFO:teuthology.orchestra.run.smithi007.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T21:40:56.224 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T21:40:56.224 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-argparse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.224 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ceph-common-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.224 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cephfs-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.224 INFO:teuthology.orchestra.run.smithi007.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T21:40:56.224 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T21:40:56.224 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T21:40:56.224 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T21:40:56.224 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-10T21:40:56.224 INFO:teuthology.orchestra.run.smithi007.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-10T21:40:56.224 INFO:teuthology.orchestra.run.smithi007.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T21:40:56.224 INFO:teuthology.orchestra.run.smithi007.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T21:40:56.225 INFO:teuthology.orchestra.run.smithi007.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T21:40:56.225 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-10T21:40:56.225 INFO:teuthology.orchestra.run.smithi007.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-10T21:40:56.225 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T21:40:56.225 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T21:40:56.225 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T21:40:56.225 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T21:40:56.225 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T21:40:56.225 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T21:40:56.226 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T21:40:56.226 INFO:teuthology.orchestra.run.smithi007.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T21:40:56.226 INFO:teuthology.orchestra.run.smithi007.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T21:40:56.226 INFO:teuthology.orchestra.run.smithi007.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T21:40:56.226 INFO:teuthology.orchestra.run.smithi007.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T21:40:56.226 INFO:teuthology.orchestra.run.smithi007.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T21:40:56.226 INFO:teuthology.orchestra.run.smithi007.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T21:40:56.226 INFO:teuthology.orchestra.run.smithi007.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T21:40:56.226 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T21:40:56.226 INFO:teuthology.orchestra.run.smithi007.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T21:40:56.226 INFO:teuthology.orchestra.run.smithi007.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T21:40:56.226 INFO:teuthology.orchestra.run.smithi007.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T21:40:56.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T21:40:56.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T21:40:56.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T21:40:56.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T21:40:56.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-10T21:40:56.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T21:40:56.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T21:40:56.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T21:40:56.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T21:40:56.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-10T21:40:56.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T21:40:56.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rados-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.227 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.228 INFO:teuthology.orchestra.run.smithi007.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T21:40:56.228 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T21:40:56.228 INFO:teuthology.orchestra.run.smithi007.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T21:40:56.228 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rgw-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.228 INFO:teuthology.orchestra.run.smithi007.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T21:40:56.228 INFO:teuthology.orchestra.run.smithi007.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T21:40:56.228 INFO:teuthology.orchestra.run.smithi007.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T21:40:56.228 INFO:teuthology.orchestra.run.smithi007.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T21:40:56.228 INFO:teuthology.orchestra.run.smithi007.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T21:40:56.228 INFO:teuthology.orchestra.run.smithi007.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T21:40:56.228 INFO:teuthology.orchestra.run.smithi007.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T21:40:56.228 INFO:teuthology.orchestra.run.smithi007.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-10T21:40:56.228 INFO:teuthology.orchestra.run.smithi007.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T21:40:56.229 INFO:teuthology.orchestra.run.smithi007.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T21:40:56.229 INFO:teuthology.orchestra.run.smithi007.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T21:40:56.229 INFO:teuthology.orchestra.run.smithi007.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T21:40:56.229 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-10T21:40:56.229 INFO:teuthology.orchestra.run.smithi007.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-10T21:40:56.229 INFO:teuthology.orchestra.run.smithi007.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-10T21:40:56.229 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-fuse-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.229 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-mirror-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.229 INFO:teuthology.orchestra.run.smithi007.stdout: rbd-nbd-2:19.3.0-4854.g8f219c47.el9.x86_64 2024-09-10T21:40:56.229 INFO:teuthology.orchestra.run.smithi007.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T21:40:56.229 INFO:teuthology.orchestra.run.smithi007.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T21:40:56.229 INFO:teuthology.orchestra.run.smithi007.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T21:40:56.229 INFO:teuthology.orchestra.run.smithi007.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-10T21:40:56.230 INFO:teuthology.orchestra.run.smithi007.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-10T21:40:56.230 INFO:teuthology.orchestra.run.smithi007.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T21:40:56.230 INFO:teuthology.orchestra.run.smithi007.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-10T21:40:56.230 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:40:56.230 INFO:teuthology.orchestra.run.smithi007.stdout:Complete! 2024-09-10T21:40:57.052 DEBUG:teuthology.parallel:result is None 2024-09-10T21:40:57.053 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:40:57.200 DEBUG:teuthology.orchestra.run.smithi007:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T21:40:57.245 INFO:teuthology.orchestra.run.smithi007.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T21:40:57.246 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T21:40:57.246 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T21:40:57.248 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:40:57.399 DEBUG:teuthology.orchestra.run.smithi133:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T21:40:57.448 INFO:teuthology.orchestra.run.smithi133.stdout:19.3.0-4854.g8f219c47.el9 2024-09-10T21:40:57.449 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854.g8f219c47.el9 2024-09-10T21:40:57.449 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854.g8f219c47 is installed. 2024-09-10T21:40:57.450 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T21:40:57.451 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:40:57.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T21:40:57.517 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:40:57.517 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T21:40:57.579 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T21:40:57.579 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:40:57.579 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T21:40:57.638 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T21:40:57.738 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:40:57.738 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T21:40:57.797 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T21:40:57.894 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T21:40:57.894 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:40:57.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T21:40:57.954 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T21:40:58.053 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:40:58.053 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T21:40:58.109 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T21:40:58.209 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T21:40:58.209 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:40:58.209 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T21:40:58.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T21:40:58.366 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:40:58.366 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T21:40:58.422 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T21:40:58.518 INFO:teuthology.run_tasks:Running task ceph... 2024-09-10T21:40:58.606 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-10T21:40:58.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 777 /var/log/ceph 2024-09-10T21:40:58.610 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 777 /var/log/ceph 2024-09-10T21:40:58.666 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-10T21:40:58.666 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T21:40:58.672 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T21:40:58.727 INFO:tasks.ceph:Creating extra log directories... 2024-09-10T21:40:58.728 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T21:40:58.731 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T21:40:58.792 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-10T21:40:58.792 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-10T21:40:58.792 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-10T21:40:58.792 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:40:58.792 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T21:40:58.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T21:40:59.018 DEBUG:teuthology.orchestra.run.smithi007:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-10T21:40:59.081 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:40:59.081 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T21:40:59.116 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T21:40:59.293 DEBUG:teuthology.orchestra.run.smithi133:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-10T21:40:59.354 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-10T21:40:59.354 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected', 'object missing on disk', 'Invalid tag char'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-10T21:40:59.354 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899691', 'branch': 'wip-hemanth-testing-2024-09-10-0723', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub}', 'email': 'hyelloji@redhat.com', 'first_in_suite': False, 'job_id': '7899691', 'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-hemanth-testing-2024-09-10-0723'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected', 'object missing on disk', 'Invalid tag char'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'kclient': {'syntax': 'v2'}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-hemanth-testing-2024-09-10-0723', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'owner': 'scheduled_hyelloji@teuthology', 'priority': 100, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 6190, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'sleep_before_teardown': 0, 'subset': '479/1024', 'suite': 'fs', 'suite_branch': 'wip-hemanth-testing-2024-09-10-0723', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi007.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBjDGS5nmwPi985Xovar9cxzJNEIjtgdGwuAJ0CFZXkw9GSDn9dqqagcE6Ym3nk6G1VnrQapo4JeRmfp5KNJYfU=', 'smithi133.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIwVSnTQFwGtT2F70wr2qNvD0ndyipSXStfikPxAgevFOu/1/rsghAHgEVNqIGBC418XUMnXj2QK3WdqEa0SPDc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected', 'object missing on disk', 'Invalid tag char'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_16:59:47', 'tube': 'smithi', 'user': 'hyelloji', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-10T21:40:59.355 DEBUG:teuthology.orchestra.run.smithi007:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T21:40:59.359 DEBUG:teuthology.orchestra.run.smithi133:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T21:40:59.394 DEBUG:teuthology.orchestra.run.smithi007:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T21:40:59.435 DEBUG:teuthology.orchestra.run.smithi133:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T21:40:59.490 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:40:59.491 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T21:40:59.526 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T21:40:59.526 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_1 2024-09-10T21:40:59.602 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T21:40:59.602 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:40:59.602 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 638 Links: 1 2024-09-10T21:40:59.602 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:40:59.603 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:40:59.603 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-10 21:40:47.505902938 +0000 2024-09-10T21:40:59.603 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-10 21:37:35.589824457 +0000 2024-09-10T21:40:59.603 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-10 21:37:35.589824457 +0000 2024-09-10T21:40:59.603 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-10 21:37:35.589824457 +0000 2024-09-10T21:40:59.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T21:40:59.695 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-10T21:40:59.695 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-10T21:40:59.695 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000235299 s, 2.2 MB/s 2024-09-10T21:40:59.697 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T21:40:59.734 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_2 2024-09-10T21:40:59.811 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T21:40:59.811 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:40:59.812 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 656 Links: 1 2024-09-10T21:40:59.812 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:40:59.812 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:40:59.812 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-10 21:40:47.506902923 +0000 2024-09-10T21:40:59.812 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-10 21:37:36.069817417 +0000 2024-09-10T21:40:59.812 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-10 21:37:36.069817417 +0000 2024-09-10T21:40:59.812 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-10 21:37:36.069817417 +0000 2024-09-10T21:40:59.812 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T21:40:59.904 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-10T21:40:59.904 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-10T21:40:59.904 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000160909 s, 3.2 MB/s 2024-09-10T21:40:59.907 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T21:40:59.944 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_3 2024-09-10T21:41:00.020 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T21:41:00.021 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:41:00.021 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 674 Links: 1 2024-09-10T21:41:00.021 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:41:00.021 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:41:00.021 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-10 21:40:47.507902908 +0000 2024-09-10T21:41:00.021 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-10 21:37:36.499811110 +0000 2024-09-10T21:41:00.021 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-10 21:37:36.499811110 +0000 2024-09-10T21:41:00.021 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-10 21:37:36.499811110 +0000 2024-09-10T21:41:00.022 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T21:41:00.112 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-10T21:41:00.122 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-10T21:41:00.122 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000176279 s, 2.9 MB/s 2024-09-10T21:41:00.123 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T21:41:00.160 DEBUG:teuthology.orchestra.run.smithi007:> stat /dev/vg_nvme/lv_4 2024-09-10T21:41:00.235 INFO:teuthology.orchestra.run.smithi007.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T21:41:00.236 INFO:teuthology.orchestra.run.smithi007.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:41:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:Device: 6h/6d Inode: 693 Links: 1 2024-09-10T21:41:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:41:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:41:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:Access: 2024-09-10 21:40:47.507902908 +0000 2024-09-10T21:41:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:Modify: 2024-09-10 21:37:36.964804291 +0000 2024-09-10T21:41:00.236 INFO:teuthology.orchestra.run.smithi007.stdout:Change: 2024-09-10 21:37:36.964804291 +0000 2024-09-10T21:41:00.236 INFO:teuthology.orchestra.run.smithi007.stdout: Birth: 2024-09-10 21:37:36.964804291 +0000 2024-09-10T21:41:00.236 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T21:41:00.328 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records in 2024-09-10T21:41:00.328 INFO:teuthology.orchestra.run.smithi007.stderr:1+0 records out 2024-09-10T21:41:00.329 INFO:teuthology.orchestra.run.smithi007.stderr:512 bytes copied, 0.000171755 s, 3.0 MB/s 2024-09-10T21:41:00.331 DEBUG:teuthology.orchestra.run.smithi007:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T21:41:00.367 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:41:00.367 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:41:00.367 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T21:41:00.407 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T21:41:00.407 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_1 2024-09-10T21:41:00.486 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T21:41:00.486 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:41:00.486 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 6h/6d Inode: 631 Links: 1 2024-09-10T21:41:00.486 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:41:00.486 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:41:00.486 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-10 21:39:46.050799982 +0000 2024-09-10T21:41:00.486 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-10 21:37:32.681374406 +0000 2024-09-10T21:41:00.486 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-10 21:37:32.681374406 +0000 2024-09-10T21:41:00.486 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-09-10 21:37:32.681374406 +0000 2024-09-10T21:41:00.486 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T21:41:00.582 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-10T21:41:00.583 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-10T21:41:00.583 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.0003011 s, 1.7 MB/s 2024-09-10T21:41:00.586 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T21:41:00.623 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_2 2024-09-10T21:41:00.699 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T21:41:00.699 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:41:00.699 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-09-10T21:41:00.699 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:41:00.700 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:41:00.700 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-10 21:39:46.050799982 +0000 2024-09-10T21:41:00.700 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-10 21:37:33.118365997 +0000 2024-09-10T21:41:00.700 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-10 21:37:33.118365997 +0000 2024-09-10T21:41:00.700 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-09-10 21:37:33.118365997 +0000 2024-09-10T21:41:00.700 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T21:41:00.792 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-10T21:41:00.792 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-10T21:41:00.792 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.00024336 s, 2.1 MB/s 2024-09-10T21:41:00.794 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T21:41:00.831 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_3 2024-09-10T21:41:00.905 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T21:41:00.906 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:41:00.906 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 6h/6d Inode: 672 Links: 1 2024-09-10T21:41:00.906 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:41:00.906 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:41:00.906 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-10 21:39:46.051799963 +0000 2024-09-10T21:41:00.906 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-10 21:37:33.607356587 +0000 2024-09-10T21:41:00.906 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-10 21:37:33.607356587 +0000 2024-09-10T21:41:00.906 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-09-10 21:37:33.607356587 +0000 2024-09-10T21:41:00.906 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T21:41:00.998 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-10T21:41:00.998 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-10T21:41:00.999 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000262327 s, 2.0 MB/s 2024-09-10T21:41:01.001 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T21:41:01.037 DEBUG:teuthology.orchestra.run.smithi133:> stat /dev/vg_nvme/lv_4 2024-09-10T21:41:01.112 INFO:teuthology.orchestra.run.smithi133.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T21:41:01.113 INFO:teuthology.orchestra.run.smithi133.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:41:01.113 INFO:teuthology.orchestra.run.smithi133.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-09-10T21:41:01.113 INFO:teuthology.orchestra.run.smithi133.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:41:01.113 INFO:teuthology.orchestra.run.smithi133.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T21:41:01.113 INFO:teuthology.orchestra.run.smithi133.stdout:Access: 2024-09-10 21:39:46.052799944 +0000 2024-09-10T21:41:01.113 INFO:teuthology.orchestra.run.smithi133.stdout:Modify: 2024-09-10 21:37:34.070347678 +0000 2024-09-10T21:41:01.113 INFO:teuthology.orchestra.run.smithi133.stdout:Change: 2024-09-10 21:37:34.070347678 +0000 2024-09-10T21:41:01.113 INFO:teuthology.orchestra.run.smithi133.stdout: Birth: 2024-09-10 21:37:34.070347678 +0000 2024-09-10T21:41:01.114 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T21:41:01.207 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records in 2024-09-10T21:41:01.207 INFO:teuthology.orchestra.run.smithi133.stderr:1+0 records out 2024-09-10T21:41:01.207 INFO:teuthology.orchestra.run.smithi133.stderr:512 bytes copied, 0.000273223 s, 1.9 MB/s 2024-09-10T21:41:01.209 DEBUG:teuthology.orchestra.run.smithi133:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T21:41:01.245 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:41:01.245 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi133.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T21:41:01.245 INFO:tasks.ceph:Generating config... 2024-09-10T21:41:01.247 INFO:tasks.ceph:[client] pid file = 2024-09-10T21:41:01.247 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-10T21:41:01.247 INFO:tasks.ceph:[mgr] client mount timeout = 30 2024-09-10T21:41:01.247 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-10T21:41:01.247 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-10T21:41:01.247 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-10T21:41:01.247 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-10T21:41:01.247 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-10T21:41:01.247 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-10T21:41:01.247 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-10T21:41:01.248 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-10T21:41:01.248 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-10T21:41:01.248 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-10T21:41:01.248 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-10T21:41:01.248 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-10T21:41:01.248 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-10T21:41:01.248 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-10T21:41:01.248 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-10T21:41:01.248 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-10T21:41:01.248 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-10T21:41:01.248 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-10T21:41:01.248 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-10T21:41:01.248 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-10T21:41:01.248 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-10T21:41:01.248 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-10T21:41:01.249 INFO:tasks.ceph:Setting up mon.a... 2024-09-10T21:41:01.249 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-10T21:41:01.349 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.keyring 2024-09-10T21:41:01.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-10T21:41:01.442 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T21:41:01.496 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.7'), ('mon.b', '172.21.15.133'), ('mon.c', '[v2:172.21.15.133:3301,v1:172.21.15.133:6790]')] 2024-09-10T21:41:01.497 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.7,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'client mount timeout': 30, 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-10T21:41:01.497 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:41:01.498 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T21:41:01.532 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.7 --add b 172.21.15.133 --addv c '[v2:172.21.15.133:3301,v1:172.21.15.133:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:41:01.649 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:41:01.649 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: generated fsid 2029451a-3c26-48c7-b874-0597197278ac 2024-09-10T21:41:01.649 INFO:teuthology.orchestra.run.smithi007.stdout:setting min_mon_release = quincy 2024-09-10T21:41:01.649 INFO:teuthology.orchestra.run.smithi007.stdout:epoch 0 2024-09-10T21:41:01.649 INFO:teuthology.orchestra.run.smithi007.stdout:fsid 2029451a-3c26-48c7-b874-0597197278ac 2024-09-10T21:41:01.650 INFO:teuthology.orchestra.run.smithi007.stdout:last_changed 2024-09-10T21:41:01.649225+0000 2024-09-10T21:41:01.650 INFO:teuthology.orchestra.run.smithi007.stdout:created 2024-09-10T21:41:01.649225+0000 2024-09-10T21:41:01.650 INFO:teuthology.orchestra.run.smithi007.stdout:min_mon_release 17 (quincy) 2024-09-10T21:41:01.650 INFO:teuthology.orchestra.run.smithi007.stdout:election_strategy: 1 2024-09-10T21:41:01.650 INFO:teuthology.orchestra.run.smithi007.stdout:0: [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] mon.a 2024-09-10T21:41:01.650 INFO:teuthology.orchestra.run.smithi007.stdout:1: [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] mon.b 2024-09-10T21:41:01.650 INFO:teuthology.orchestra.run.smithi007.stdout:2: [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] mon.c 2024-09-10T21:41:01.650 INFO:teuthology.orchestra.run.smithi007.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-10T21:41:01.652 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T21:41:01.686 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2029451a-3c26-48c7-b874-0597197278ac... 2024-09-10T21:41:01.687 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T21:41:01.731 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T21:41:01.825 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-09-10T21:41:01.825 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-09-10T21:41:01.825 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T21:41:01.825 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-09-10T21:41:01.825 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.825 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-09-10T21:41:01.825 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.825 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-09-10T21:41:01.825 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.825 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-09-10T21:41:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-09-10T21:41:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-09-10T21:41:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-09-10T21:41:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T21:41:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-09-10T21:41:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T21:41:01.826 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T21:41:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-09-10T21:41:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-09-10T21:41:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-09-10T21:41:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-09-10T21:41:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-09-10T21:41:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-09-10T21:41:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-09-10T21:41:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-09-10T21:41:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T21:41:01.827 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T21:41:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-09-10T21:41:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T21:41:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-09-10T21:41:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-09-10T21:41:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-09-10T21:41:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-09-10T21:41:01.828 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-09-10T21:41:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2024-09-10T21:41:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on skipped message = False 2024-09-10T21:41:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 2029451a-3c26-48c7-b874-0597197278ac 2024-09-10T21:41:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.829 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-09-10T21:41:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-09-10T21:41:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-09-10T21:41:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2024-09-10T21:41:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-09-10T21:41:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-09-10T21:41:01.829 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-09-10T21:41:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-09-10T21:41:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-09-10T21:41:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-09-10T21:41:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug trim objects = true 2024-09-10T21:41:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-09-10T21:41:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-09-10T21:41:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-09-10T21:41:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-09-10T21:41:01.830 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-09-10T21:41:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-09-10T21:41:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-09-10T21:41:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-09-10T21:41:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-09-10T21:41:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: bdev async discard = True 2024-09-10T21:41:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: bdev enable discard = True 2024-09-10T21:41:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore allocator = bitmap 2024-09-10T21:41:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-09-10T21:41:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-09-10T21:41:01.831 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 1/20 2024-09-10T21:41:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 1/20 2024-09-10T21:41:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-10T21:41:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-09-10T21:41:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 4/10 2024-09-10T21:41:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T21:41:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-09-10T21:41:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-09-10T21:41:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2024-09-10T21:41:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2024-09-10T21:41:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.832 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-09-10T21:41:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-10T21:41:01.832 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-09-10T21:41:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-10T21:41:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-10T21:41:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:41:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:41:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-09-10T21:41:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: client mount timeout = 30 2024-09-10T21:41:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-09-10T21:41:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.833 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-09-10T21:41:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-10T21:41:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-10T21:41:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-09-10T21:41:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-10T21:41:01.833 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-09-10T21:41:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-09-10T21:41:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:41:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-09-10T21:41:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-09-10T21:41:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:41:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T21:41:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T21:41:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T21:41:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T21:41:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T21:41:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T21:41:01.834 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-09-10T21:41:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-09-10T21:41:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-09-10T21:41:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: mon down mkfs grace = 300 2024-09-10T21:41:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:01.835 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-09-10T21:41:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-09-10T21:41:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-09-10T21:41:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-09-10T21:41:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T21:41:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T21:41:01.835 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = "" 2024-09-10T21:41:01.835 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-09-10T21:41:01.836 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-09-10T21:41:01.836 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-09-10T21:41:01.836 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-09-10T21:41:01.836 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-09-10T21:41:01.836 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-09-10T21:41:01.836 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-09-10T21:41:01.836 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2024-09-10T21:41:01.836 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2024-09-10T21:41:01.836 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T21:41:01.836 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2024-09-10T21:41:01.837 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.837 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2024-09-10T21:41:01.837 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.837 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2024-09-10T21:41:01.837 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.837 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2024-09-10T21:41:01.837 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2024-09-10T21:41:01.837 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.837 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2024-09-10T21:41:01.837 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2024-09-10T21:41:01.837 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.837 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T21:41:01.837 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2024-09-10T21:41:01.837 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.838 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T21:41:01.838 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T21:41:01.838 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.838 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2024-09-10T21:41:01.838 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.838 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2024-09-10T21:41:01.838 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2024-09-10T21:41:01.838 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2024-09-10T21:41:01.838 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2024-09-10T21:41:01.838 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2024-09-10T21:41:01.838 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2024-09-10T21:41:01.838 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2024-09-10T21:41:01.838 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T21:41:01.838 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T21:41:01.839 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2024-09-10T21:41:01.839 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.839 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T21:41:01.839 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.839 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2024-09-10T21:41:01.839 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.839 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2024-09-10T21:41:01.839 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.839 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2024-09-10T21:41:01.839 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2024-09-10T21:41:01.839 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2024-09-10T21:41:01.839 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.7,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2024-09-10T21:41:01.839 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on skipped message = False 2024-09-10T21:41:01.839 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 2029451a-3c26-48c7-b874-0597197278ac 2024-09-10T21:41:01.839 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.840 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2024-09-10T21:41:01.840 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2024-09-10T21:41:01.840 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.840 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2024-09-10T21:41:01.840 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2024-09-10T21:41:01.840 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2024-09-10T21:41:01.840 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.840 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2024-09-10T21:41:01.840 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2024-09-10T21:41:01.840 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.840 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2024-09-10T21:41:01.840 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2024-09-10T21:41:01.841 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2024-09-10T21:41:01.841 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug trim objects = true 2024-09-10T21:41:01.841 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.842 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2024-09-10T21:41:01.842 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2024-09-10T21:41:01.842 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.842 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2024-09-10T21:41:01.842 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.842 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2024-09-10T21:41:01.842 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.842 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2024-09-10T21:41:01.842 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.842 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2024-09-10T21:41:01.842 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2024-09-10T21:41:01.842 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.843 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2024-09-10T21:41:01.843 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2024-09-10T21:41:01.843 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2024-09-10T21:41:01.843 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2024-09-10T21:41:01.843 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = bitmap 2024-09-10T21:41:01.843 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2024-09-10T21:41:01.843 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2024-09-10T21:41:01.843 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 1/20 2024-09-10T21:41:01.843 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 1/20 2024-09-10T21:41:01.843 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-10T21:41:01.843 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2024-09-10T21:41:01.843 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 4/10 2024-09-10T21:41:01.843 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T21:41:01.843 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2024-09-10T21:41:01.843 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2024-09-10T21:41:01.844 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2024-09-10T21:41:01.844 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2024-09-10T21:41:01.844 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.844 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2024-09-10T21:41:01.844 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-10T21:41:01.844 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2024-09-10T21:41:01.844 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-09-10T21:41:01.844 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-09-10T21:41:01.844 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:41:01.844 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:41:01.844 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2024-09-10T21:41:01.844 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 30 2024-09-10T21:41:01.844 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2024-09-10T21:41:01.845 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.845 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2024-09-10T21:41:01.845 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-10T21:41:01.845 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-09-10T21:41:01.845 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2024-09-10T21:41:01.845 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-09-10T21:41:01.845 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2024-09-10T21:41:01.845 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2024-09-10T21:41:01.845 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:41:01.845 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2024-09-10T21:41:01.845 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2024-09-10T21:41:01.845 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:41:01.845 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.845 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T21:41:01.845 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T21:41:01.846 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T21:41:01.846 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.846 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T21:41:01.846 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T21:41:01.846 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T21:41:01.846 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.846 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2024-09-10T21:41:01.846 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2024-09-10T21:41:01.846 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.846 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2024-09-10T21:41:01.846 INFO:teuthology.orchestra.run.smithi133.stdout: mon down mkfs grace = 300 2024-09-10T21:41:01.846 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:41:01.846 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2024-09-10T21:41:01.846 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2024-09-10T21:41:01.846 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2024-09-10T21:41:01.847 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2024-09-10T21:41:01.847 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T21:41:01.847 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T21:41:01.847 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2024-09-10T21:41:01.847 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2024-09-10T21:41:01.847 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2024-09-10T21:41:01.847 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2024-09-10T21:41:01.847 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-09-10T21:41:01.847 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-09-10T21:41:01.847 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-09-10T21:41:01.847 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2024-09-10T21:41:01.855 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-10T21:41:01.855 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-10T21:41:01.953 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-10T21:41:01.953 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:41:01.954 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-10T21:41:01.988 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:41:01.988 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-10T21:41:02.063 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:41:02.063 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:41:02.063 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T21:41:02.063 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T21:41:02.181 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:41:02.181 DEBUG:teuthology.orchestra.run.smithi007:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:41:02.256 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi133.front.sepia.ceph.com 2024-09-10T21:41:02.256 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:41:02.256 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T21:41:02.256 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T21:41:02.336 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:41:02.336 DEBUG:teuthology.orchestra.run.smithi133:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:41:02.412 INFO:tasks.ceph:Setting up mon nodes... 2024-09-10T21:41:02.412 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-10T21:41:02.412 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T21:41:02.516 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T21:41:02.522 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T21:41:02.630 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T21:41:02.637 INFO:tasks.ceph:Setting up mds nodes... 2024-09-10T21:41:02.637 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T21:41:02.741 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T21:41:02.746 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-10T21:41:02.802 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T21:41:02.905 INFO:teuthology.orchestra.run.smithi007.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T21:41:02.911 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-10T21:41:02.967 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T21:41:03.070 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T21:41:03.076 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-10T21:41:03.131 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T21:41:03.234 INFO:teuthology.orchestra.run.smithi133.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T21:41:03.239 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-10T21:41:03.295 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-10T21:41:03.296 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T21:41:03.379 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-10T21:41:03.407 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-10T21:41:03.489 INFO:teuthology.orchestra.run.smithi007.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-10T21:41:03.516 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T21:41:03.596 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-10T21:41:03.623 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-10T21:41:03.703 INFO:teuthology.orchestra.run.smithi133.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-10T21:41:03.730 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-10T21:41:03.731 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi007.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi133.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T21:41:03.731 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-10T21:41:03.787 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:41:03.787 INFO:tasks.ceph:role: osd.0 2024-09-10T21:41:03.787 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:41:03.787 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T21:41:03.842 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:41:03.842 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:41:03.842 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:41:03.842 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T21:41:03.842 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:41:03.842 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:41:03.842 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:41:03.843 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T21:41:03.843 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:41:03.843 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:41:03.875 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-10T21:41:03.878 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-10T21:41:03.879 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-10T21:41:03.960 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-10T21:41:04.022 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-10T21:41:04.079 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:41:04.079 INFO:tasks.ceph:role: osd.1 2024-09-10T21:41:04.079 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:41:04.079 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T21:41:04.134 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:41:04.134 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:41:04.134 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:41:04.135 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T21:41:04.135 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:41:04.135 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:41:04.135 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:41:04.135 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T21:41:04.135 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:41:04.135 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:41:04.168 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-10T21:41:04.171 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-10T21:41:04.171 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-10T21:41:04.250 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-10T21:41:04.312 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-10T21:41:04.368 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:41:04.369 INFO:tasks.ceph:role: osd.2 2024-09-10T21:41:04.369 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:41:04.369 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T21:41:04.424 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:41:04.424 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:41:04.424 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:41:04.424 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T21:41:04.424 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:41:04.424 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:41:04.424 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:41:04.425 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T21:41:04.425 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:41:04.425 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:41:04.457 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-10T21:41:04.461 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-10T21:41:04.461 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-10T21:41:04.541 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-10T21:41:04.603 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-10T21:41:04.658 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:41:04.658 INFO:tasks.ceph:role: osd.3 2024-09-10T21:41:04.658 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:41:04.658 DEBUG:teuthology.orchestra.run.smithi007:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T21:41:04.713 INFO:teuthology.orchestra.run.smithi007.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:41:04.713 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:41:04.713 INFO:teuthology.orchestra.run.smithi007.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:41:04.714 INFO:teuthology.orchestra.run.smithi007.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T21:41:04.714 INFO:teuthology.orchestra.run.smithi007.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:41:04.714 INFO:teuthology.orchestra.run.smithi007.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:41:04.714 INFO:teuthology.orchestra.run.smithi007.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:41:04.714 INFO:teuthology.orchestra.run.smithi007.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T21:41:04.714 INFO:teuthology.orchestra.run.smithi007.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:41:04.714 INFO:teuthology.orchestra.run.smithi007.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:41:04.747 INFO:teuthology.orchestra.run.smithi007.stdout:Discarding blocks...Done. 2024-09-10T21:41:04.750 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi007.front.sepia.ceph.com -o noatime 2024-09-10T21:41:04.750 DEBUG:teuthology.orchestra.run.smithi007:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-10T21:41:04.829 DEBUG:teuthology.orchestra.run.smithi007:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-10T21:41:04.891 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:41:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:04.991+0000 7f7b44c12640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-10T21:41:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:04.992+0000 7f7b44c12640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-10T21:41:04.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:04.992+0000 7f7b44c12640 -1 bdev(0x555689da3800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-10T21:41:04.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:04.992+0000 7f7b44c12640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-10T21:41:08.025 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T21:41:08.084 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:41:08.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:08.174+0000 7f1a69215640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-10T21:41:08.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:08.174+0000 7f1a69215640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-10T21:41:08.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:08.174+0000 7f1a69215640 -1 bdev(0x5649b683d800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-10T21:41:08.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:08.175+0000 7f1a69215640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-10T21:41:11.184 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T21:41:11.244 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:41:11.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:11.337+0000 7f15ac215640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-10T21:41:11.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:11.337+0000 7f15ac215640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-10T21:41:11.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:11.337+0000 7f15ac215640 -1 bdev(0x5579717b7800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-10T21:41:11.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:11.338+0000 7f15ac215640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-10T21:41:14.385 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T21:41:14.444 DEBUG:teuthology.orchestra.run.smithi007:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:41:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:14.534+0000 7fad107c3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-10T21:41:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:14.534+0000 7fad107c3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-10T21:41:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:14.535+0000 7fad107c3640 -1 bdev(0x562b1d13d800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-10T21:41:14.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:14.535+0000 7fad107c3640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-10T21:41:17.814 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T21:41:17.875 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-10T21:41:17.937 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:41:17.937 INFO:tasks.ceph:role: osd.4 2024-09-10T21:41:17.937 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi133.front.sepia.ceph.com 2024-09-10T21:41:17.937 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T21:41:18.002 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:41:18.002 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:41:18.002 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:41:18.002 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T21:41:18.002 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:41:18.002 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:41:18.002 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:41:18.002 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T21:41:18.002 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:41:18.002 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:41:18.417 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-09-10T21:41:18.423 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-09-10T21:41:18.423 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-10T21:41:18.570 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-10T21:41:18.638 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-10T21:41:18.694 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:41:18.694 INFO:tasks.ceph:role: osd.5 2024-09-10T21:41:18.694 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi133.front.sepia.ceph.com 2024-09-10T21:41:18.694 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T21:41:18.757 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:41:18.757 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:41:18.757 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:41:18.757 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T21:41:18.757 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:41:18.757 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:41:18.757 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:41:18.757 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T21:41:18.757 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:41:18.757 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:41:19.115 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-09-10T21:41:19.119 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-09-10T21:41:19.119 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-10T21:41:19.204 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-10T21:41:19.265 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-10T21:41:19.320 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:41:19.320 INFO:tasks.ceph:role: osd.6 2024-09-10T21:41:19.320 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi133.front.sepia.ceph.com 2024-09-10T21:41:19.321 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T21:41:19.382 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:41:19.383 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:41:19.383 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:41:19.383 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T21:41:19.383 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:41:19.383 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:41:19.383 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:41:19.383 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T21:41:19.383 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:41:19.383 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:41:19.776 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-09-10T21:41:19.780 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-09-10T21:41:19.780 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-10T21:41:19.882 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-10T21:41:19.944 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-10T21:41:19.999 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:41:19.999 INFO:tasks.ceph:role: osd.7 2024-09-10T21:41:19.999 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi133.front.sepia.ceph.com 2024-09-10T21:41:20.000 DEBUG:teuthology.orchestra.run.smithi133:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T21:41:20.061 INFO:teuthology.orchestra.run.smithi133.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:41:20.062 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:41:20.062 INFO:teuthology.orchestra.run.smithi133.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:41:20.062 INFO:teuthology.orchestra.run.smithi133.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-10T21:41:20.062 INFO:teuthology.orchestra.run.smithi133.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:41:20.062 INFO:teuthology.orchestra.run.smithi133.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:41:20.062 INFO:teuthology.orchestra.run.smithi133.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:41:20.062 INFO:teuthology.orchestra.run.smithi133.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-10T21:41:20.062 INFO:teuthology.orchestra.run.smithi133.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:41:20.062 INFO:teuthology.orchestra.run.smithi133.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:41:20.484 INFO:teuthology.orchestra.run.smithi133.stdout:Discarding blocks...Done. 2024-09-10T21:41:20.487 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi133.front.sepia.ceph.com -o noatime 2024-09-10T21:41:20.488 DEBUG:teuthology.orchestra.run.smithi133:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-10T21:41:20.626 DEBUG:teuthology.orchestra.run.smithi133:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-10T21:41:20.693 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:41:20.856 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:20.854+0000 7f50cac18640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-10T21:41:20.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:20.855+0000 7f50cac18640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-10T21:41:20.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:20.855+0000 7f50cac18640 -1 bdev(0x55a4543cf800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-10T21:41:20.857 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:20.855+0000 7f50cac18640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-10T21:41:23.877 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T21:41:23.936 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:41:24.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:24.026+0000 7fb0f5545640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-10T21:41:24.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:24.027+0000 7fb0f5545640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-10T21:41:24.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:24.027+0000 7fb0f5545640 -1 bdev(0x55e02b54b800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-10T21:41:24.029 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:24.027+0000 7fb0f5545640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-10T21:41:27.074 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T21:41:27.130 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:41:27.220 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:27.218+0000 7f4ced840640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-10T21:41:27.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:27.218+0000 7f4ced840640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-10T21:41:27.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:27.218+0000 7f4ced840640 -1 bdev(0x5623005db800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-10T21:41:27.221 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:27.219+0000 7f4ced840640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-10T21:41:29.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:41:29.361 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:41:30.264 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-10T21:41:30.321 DEBUG:teuthology.orchestra.run.smithi133:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:41:30.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:30.409+0000 7f5b95213640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-10T21:41:30.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:30.410+0000 7f5b95213640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-10T21:41:30.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:30.410+0000 7f5b95213640 -1 bdev(0x55ad39e0b800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-10T21:41:30.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:30.410+0000 7f5b95213640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-10T21:41:33.431 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-10T21:41:33.488 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-10T21:41:33.488 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:41:33.488 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-10T21:41:33.550 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:41:33.550 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-10T21:41:33.648 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:41:33.648 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-10T21:41:33.745 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:41:33.746 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-10T21:41:33.842 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:41:33.842 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-10T21:41:33.937 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:41:33.937 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-10T21:41:34.032 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:41:34.032 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-10T21:41:34.128 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:41:34.128 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-10T21:41:34.185 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:41:34.185 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-10T21:41:34.280 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:41:34.280 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-10T21:41:34.376 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:41:34.376 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-10T21:41:34.470 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:41:34.471 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-10T21:41:34.565 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:41:34.565 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-10T21:41:34.660 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:41:34.660 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-10T21:41:34.756 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:41:34.756 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-10T21:41:34.793 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:41:34.793 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-10T21:41:34.870 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:41:34.870 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-10T21:41:34.904 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:41:34.905 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-10T21:41:34.980 INFO:tasks.ceph:Adding keys to all mons... 2024-09-10T21:41:34.981 DEBUG:teuthology.orchestra.run.smithi007:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T21:41:34.985 DEBUG:teuthology.orchestra.run.smithi133:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T21:41:35.035 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.y] 2024-09-10T21:41:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDuvOBmKBHLHhAA0W5My19tIhN1v2qev4cBTA== 2024-09-10T21:41:35.035 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-09-10T21:41:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDuvOBmgRMuLBAA88QT12AXv2qf11IxU20Xbg== 2024-09-10T21:41:35.035 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-09-10T21:41:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDuvOBmnq31NRAAj1eJUAIS986idxppfeXEAA== 2024-09-10T21:41:35.035 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.0] 2024-09-10T21:41:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDwvOBmzfgtOxAAk6K5fU0Uar4VW9nUUIuNdw== 2024-09-10T21:41:35.035 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.1] 2024-09-10T21:41:35.035 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQD0vOBmHcRzChAA3z+CaGgu5NkyUmjpqzmKjw== 2024-09-10T21:41:35.036 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.2] 2024-09-10T21:41:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQD3vOBm9sArFBAAGQlyiw6qEcyumEBICWEBKA== 2024-09-10T21:41:35.036 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.3] 2024-09-10T21:41:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQD6vOBmkO3rHxAAW0zPydYZRGUKjGh3yIyPCA== 2024-09-10T21:41:35.036 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr.x] 2024-09-10T21:41:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDuvOBmsu6OJRAAhdgDvpeHY/zDKUeUOv6xxA== 2024-09-10T21:41:35.036 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-09-10T21:41:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDvvOBmRUM2BBAAwcQM6z+hjZI4ZWBlVrYMwQ== 2024-09-10T21:41:35.036 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-09-10T21:41:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDvvOBmy43uDRAAzdKjLxZMGROuDABrsJ0zgQ== 2024-09-10T21:41:35.036 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.4] 2024-09-10T21:41:35.036 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAAveBmlPABMxAAeazVOzc1DhsVAwoHn87QIA== 2024-09-10T21:41:35.037 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.5] 2024-09-10T21:41:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAEveBmL9qzARAApu8nWIY9HVsDl4yUnfhH9w== 2024-09-10T21:41:35.037 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.6] 2024-09-10T21:41:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAHveBmJ4ofDRAAEiCFN8nGzVSR3u2gKoT+2Q== 2024-09-10T21:41:35.037 INFO:teuthology.orchestra.run.smithi007.stdout:[osd.7] 2024-09-10T21:41:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQAKveBmoUGEGBAAhXmkUaniuAQrBiYPWr8Tog== 2024-09-10T21:41:35.037 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-09-10T21:41:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDvvOBm0aqcFhAAtid2bnzjcpaYAbU7g0JcOA== 2024-09-10T21:41:35.037 INFO:teuthology.orchestra.run.smithi007.stdout:[client.2] 2024-09-10T21:41:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDvvOBmybQhHRAAZUwqnIsihEWnHZWQTYr+uQ== 2024-09-10T21:41:35.037 INFO:teuthology.orchestra.run.smithi007.stdout:[client.1] 2024-09-10T21:41:35.037 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDvvOBmqf+MIxAAib5hVnbBsw/bJPqsQszJwg== 2024-09-10T21:41:35.038 INFO:teuthology.orchestra.run.smithi007.stdout:[client.3] 2024-09-10T21:41:35.038 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDvvOBmDbrrKRAAUxb3EBfKIJ4FAGHW3QIsLg== 2024-09-10T21:41:35.072 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr.y] 2024-09-10T21:41:35.072 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDuvOBmKBHLHhAA0W5My19tIhN1v2qev4cBTA== 2024-09-10T21:41:35.072 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-09-10T21:41:35.072 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDuvOBmgRMuLBAA88QT12AXv2qf11IxU20Xbg== 2024-09-10T21:41:35.073 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-09-10T21:41:35.073 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDuvOBmnq31NRAAj1eJUAIS986idxppfeXEAA== 2024-09-10T21:41:35.073 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.0] 2024-09-10T21:41:35.073 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDwvOBmzfgtOxAAk6K5fU0Uar4VW9nUUIuNdw== 2024-09-10T21:41:35.073 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.1] 2024-09-10T21:41:35.073 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQD0vOBmHcRzChAA3z+CaGgu5NkyUmjpqzmKjw== 2024-09-10T21:41:35.073 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.2] 2024-09-10T21:41:35.073 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQD3vOBm9sArFBAAGQlyiw6qEcyumEBICWEBKA== 2024-09-10T21:41:35.073 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.3] 2024-09-10T21:41:35.073 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQD6vOBmkO3rHxAAW0zPydYZRGUKjGh3yIyPCA== 2024-09-10T21:41:35.073 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr.x] 2024-09-10T21:41:35.073 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDuvOBmsu6OJRAAhdgDvpeHY/zDKUeUOv6xxA== 2024-09-10T21:41:35.073 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-09-10T21:41:35.073 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDvvOBmRUM2BBAAwcQM6z+hjZI4ZWBlVrYMwQ== 2024-09-10T21:41:35.074 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2024-09-10T21:41:35.074 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDvvOBmy43uDRAAzdKjLxZMGROuDABrsJ0zgQ== 2024-09-10T21:41:35.074 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.4] 2024-09-10T21:41:35.074 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAAveBmlPABMxAAeazVOzc1DhsVAwoHn87QIA== 2024-09-10T21:41:35.074 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.5] 2024-09-10T21:41:35.074 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAEveBmL9qzARAApu8nWIY9HVsDl4yUnfhH9w== 2024-09-10T21:41:35.074 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.6] 2024-09-10T21:41:35.074 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAHveBmJ4ofDRAAEiCFN8nGzVSR3u2gKoT+2Q== 2024-09-10T21:41:35.074 INFO:teuthology.orchestra.run.smithi133.stdout:[osd.7] 2024-09-10T21:41:35.074 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQAKveBmoUGEGBAAhXmkUaniuAQrBiYPWr8Tog== 2024-09-10T21:41:35.074 INFO:teuthology.orchestra.run.smithi133.stdout:[client.0] 2024-09-10T21:41:35.074 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDvvOBm0aqcFhAAtid2bnzjcpaYAbU7g0JcOA== 2024-09-10T21:41:35.074 INFO:teuthology.orchestra.run.smithi133.stdout:[client.2] 2024-09-10T21:41:35.074 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDvvOBmybQhHRAAZUwqnIsihEWnHZWQTYr+uQ== 2024-09-10T21:41:35.075 INFO:teuthology.orchestra.run.smithi133.stdout:[client.1] 2024-09-10T21:41:35.075 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDvvOBmqf+MIxAAib5hVnbBsw/bJPqsQszJwg== 2024-09-10T21:41:35.075 INFO:teuthology.orchestra.run.smithi133.stdout:[client.3] 2024-09-10T21:41:35.075 INFO:teuthology.orchestra.run.smithi133.stdout: key = AQDvvOBmDbrrKRAAUxb3EBfKIJ4FAGHW3QIsLg== 2024-09-10T21:41:35.076 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T21:41:35.083 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T21:41:35.210 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:41:35.214 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:41:35.309 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:41:35.313 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:41:35.409 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:41:35.413 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:41:35.506 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:41:35.510 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:41:35.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:41:35.609 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:41:35.702 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:41:35.706 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:41:35.800 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T21:41:35.803 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T21:41:35.897 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:41:35.901 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:41:35.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:41:36.001 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:41:36.096 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:41:36.100 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:41:36.193 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:41:36.197 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:41:36.289 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:41:36.293 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:41:36.388 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:41:36.393 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:41:36.486 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:41:36.489 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:41:36.587 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:41:36.591 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:41:36.686 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:41:36.691 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:41:36.787 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:41:36.791 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:41:36.887 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-10T21:41:36.887 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-10T21:41:36.943 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T21:41:37.051 DEBUG:teuthology.orchestra.run.smithi007:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-10T21:41:37.107 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-10T21:41:37.163 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T21:41:37.271 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-10T21:41:37.326 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-10T21:41:37.381 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T21:41:37.490 DEBUG:teuthology.orchestra.run.smithi133:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-10T21:41:37.546 DEBUG:teuthology.orchestra.run.smithi007:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:41:37.550 DEBUG:teuthology.orchestra.run.smithi133:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:41:37.584 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-10T21:41:37.584 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T21:41:37.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T21:41:37.627 INFO:tasks.ceph.mon.a:Started 2024-09-10T21:41:37.627 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-10T21:41:37.627 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-10T21:41:37.631 INFO:tasks.ceph.mon.b:Started 2024-09-10T21:41:37.631 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T21:41:37.631 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T21:41:37.634 INFO:tasks.ceph.mon.c:Started 2024-09-10T21:41:37.635 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-10T21:41:37.635 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-10T21:41:37.635 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-10T21:41:37.638 INFO:tasks.ceph.mgr.y:Started 2024-09-10T21:41:37.639 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-10T21:41:37.639 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-10T21:41:37.642 INFO:tasks.ceph.mgr.x:Started 2024-09-10T21:41:37.642 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-10T21:41:37.643 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-10T21:41:37.646 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-10T21:41:37.647 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-10T21:41:37.691 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-10T21:41:37.692 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-10T21:41:37.739 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-10T21:41:37.739 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-10T21:41:37.786 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-10T21:41:37.786 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-10T21:41:37.832 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-10T21:41:37.832 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-10T21:41:37.879 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-10T21:41:37.879 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-10T21:41:37.926 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-10T21:41:37.926 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-10T21:41:37.973 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-10T21:41:37.973 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-10T21:41:38.020 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-10T21:41:38.020 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-10T21:41:38.065 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-10T21:41:38.065 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-10T21:41:38.110 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-10T21:41:38.111 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-10T21:41:38.156 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-10T21:41:38.156 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-10T21:41:38.204 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-10T21:41:38.204 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-10T21:41:38.248 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-10T21:41:38.249 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-10T21:41:38.293 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-10T21:41:38.294 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-10T21:41:38.338 INFO:tasks.ceph:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-10T21:41:38.339 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set mon mon_warn_on_pool_no_app False 2024-09-10T21:41:38.385 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-10T21:41:38.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-10T21:41:38.431 DEBUG:tasks.ceph:set 18 configs 2024-09-10T21:41:38.431 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi007') 2024-09-10T21:41:43.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.183+0000 7f940da00640 1 -- 172.21.15.7:0/1321788398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93e4003b20 msgr2=0x7f94081f3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:43.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.183+0000 7f940da00640 1 --2- 172.21.15.7:0/1321788398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93e4003b20 0x7f94081f3810 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f93f8005d00 tx=0x7f93f80039c0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.185+0000 7f940da00640 1 -- 172.21.15.7:0/1321788398 shutdown_connections 2024-09-10T21:41:43.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.185+0000 7f940da00640 1 --2- 172.21.15.7:0/1321788398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93e4004fd0 0x7f9408152d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.185+0000 7f940da00640 1 --2- 172.21.15.7:0/1321788398 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f940815e120 0x7f94080c8c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.186+0000 7f940da00640 1 --2- 172.21.15.7:0/1321788398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93e4003b20 0x7f94081f3810 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.186+0000 7f940da00640 1 -- 172.21.15.7:0/1321788398 >> 172.21.15.7:0/1321788398 conn(0x7f94080d3f10 msgr2=0x7f940815f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:43.189 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi007') 2024-09-10T21:41:43.189 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi007') 2024-09-10T21:41:43.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.188+0000 7fe516000640 1 -- 172.21.15.7:0/1676490616 <== mon.1 v2:172.21.15.133:3300/0 11 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v11)=0 v11) ==== 121+0+0 (secure 0 0 0) 0x7fe510024440 con 0x7fe504004060 2024-09-10T21:41:43.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.188+0000 7f940da00640 1 -- 172.21.15.7:0/1321788398 shutdown_connections 2024-09-10T21:41:43.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.188+0000 7f940da00640 1 -- 172.21.15.7:0/1321788398 wait complete. 2024-09-10T21:41:43.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.193+0000 7fe516000640 1 -- 172.21.15.7:0/1676490616 <== mon.1 v2:172.21.15.133:3300/0 12 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe510024660 con 0x7fe504004060 2024-09-10T21:41:43.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.194+0000 7fe50b400640 1 -- 172.21.15.7:0/1676490616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe504004060 msgr2=0x7fe520200110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:43.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.194+0000 7fe50b400640 1 --2- 172.21.15.7:0/1676490616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe504004060 0x7fe520200110 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe51000e680 tx=0x7fe51000eb40 comp rx=0 tx=0).stop 2024-09-10T21:41:43.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.194+0000 7f3dbe000640 1 -- 172.21.15.7:0/1074897810 <== mon.1 v2:172.21.15.133:3300/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3dc001b3e0 con 0x7f3dac005290 2024-09-10T21:41:43.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.194+0000 7fad8d600640 1 -- 172.21.15.7:0/1590278432 <== mon.0 v2:172.21.15.7:3300/0 11 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v12)=0 v12) ==== 123+0+0 (secure 0 0 0) 0x7fad8401b7a0 con 0x7fad700072c0 2024-09-10T21:41:43.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.195+0000 7f27f0c00640 1 -- 172.21.15.7:0/1798705140 <== mon.1 v2:172.21.15.133:3300/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27e0010460 con 0x7f27dc004f40 2024-09-10T21:41:43.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.195+0000 7f00e8c00640 1 -- 172.21.15.7:0/2610032507 <== mon.1 v2:172.21.15.133:3300/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00e4049450 con 0x7f00d4005290 2024-09-10T21:41:43.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.196+0000 7fad8d600640 1 -- 172.21.15.7:0/1590278432 <== mon.0 v2:172.21.15.7:3300/0 12 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad8401ba70 con 0x7fad700072c0 2024-09-10T21:41:43.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.197+0000 7f2ec4c00640 1 -- 172.21.15.7:0/758541930 <== mon.0 v2:172.21.15.7:3300/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ebc0266e0 con 0x7f2ea80041c0 2024-09-10T21:41:43.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.197+0000 7f2ece400640 1 -- 172.21.15.7:0/758541930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f2e9c0052f0 con 0x7f2ea80041c0 2024-09-10T21:41:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.196+0000 7fe50b400640 1 -- 172.21.15.7:0/1676490616 shutdown_connections 2024-09-10T21:41:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.196+0000 7fe50b400640 1 --2- 172.21.15.7:0/1676490616 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe5200b4bc0 0x7fe5200b7bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.196+0000 7fe50b400640 1 --2- 172.21.15.7:0/1676490616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe504004e60 0x7fe5200b7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.196+0000 7fe50b400640 1 --2- 172.21.15.7:0/1676490616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe504004060 0x7fe520200110 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.196+0000 7fe50b400640 1 -- 172.21.15.7:0/1676490616 >> 172.21.15.7:0/1676490616 conn(0x7fe52009d0d0 msgr2=0x7fe52006d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.197+0000 7fad96800640 1 -- 172.21.15.7:0/1590278432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad700072c0 msgr2=0x7fad900b8650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.197+0000 7fad96800640 1 --2- 172.21.15.7:0/1590278432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad700072c0 0x7fad900b8650 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fad8400e6d0 tx=0x7fad8400eb90 comp rx=0 tx=0).stop 2024-09-10T21:41:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.198+0000 7fad96800640 1 -- 172.21.15.7:0/1590278432 shutdown_connections 2024-09-10T21:41:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.198+0000 7fad96800640 1 --2- 172.21.15.7:0/1590278432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad78004d90 0x7fad90153fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.198+0000 7fad96800640 1 --2- 172.21.15.7:0/1590278432 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad78004010 0x7fad90157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.198+0000 7fad96800640 1 --2- 172.21.15.7:0/1590278432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad700072c0 0x7fad900b8650 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.198+0000 7fad96800640 1 -- 172.21.15.7:0/1590278432 >> 172.21.15.7:0/1590278432 conn(0x7fad9009d0d0 msgr2=0x7fad9006e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.198+0000 7fad96800640 1 -- 172.21.15.7:0/1590278432 shutdown_connections 2024-09-10T21:41:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.198+0000 7fad96800640 1 -- 172.21.15.7:0/1590278432 wait complete. 2024-09-10T21:41:43.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.199+0000 7fe50b400640 1 -- 172.21.15.7:0/1676490616 shutdown_connections 2024-09-10T21:41:43.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.199+0000 7fe50b400640 1 -- 172.21.15.7:0/1676490616 wait complete. 2024-09-10T21:41:43.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.202+0000 7f2ec4c00640 1 -- 172.21.15.7:0/758541930 <== mon.0 v2:172.21.15.7:3300/0 12 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v13)=0 v13) ==== 119+0+0 (secure 0 0 0) 0x7f2ebc02c070 con 0x7f2ea80041c0 2024-09-10T21:41:43.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.205+0000 7f2ece400640 1 -- 172.21.15.7:0/758541930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ea80041c0 msgr2=0x7f2ec815c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:43.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.205+0000 7f2ece400640 1 --2- 172.21.15.7:0/758541930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ea80041c0 0x7f2ec815c350 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00d770 tx=0x7f2ebc013e90 comp rx=0 tx=0).stop 2024-09-10T21:41:43.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.205+0000 7f2ece400640 1 -- 172.21.15.7:0/758541930 shutdown_connections 2024-09-10T21:41:43.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.205+0000 7f2ece400640 1 --2- 172.21.15.7:0/758541930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ea8004f40 0x7f2ec81f77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.205+0000 7f2ece400640 1 --2- 172.21.15.7:0/758541930 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ec80b3bb0 0x7f2ec81f7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.205+0000 7f2ece400640 1 --2- 172.21.15.7:0/758541930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ea80041c0 0x7f2ec815c350 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.205+0000 7f2ece400640 1 -- 172.21.15.7:0/758541930 >> 172.21.15.7:0/758541930 conn(0x7f2ec809d0d0 msgr2=0x7f2ec80ce9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:43.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.206+0000 7f2ece400640 1 -- 172.21.15.7:0/758541930 shutdown_connections 2024-09-10T21:41:43.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.206+0000 7f2ece400640 1 -- 172.21.15.7:0/758541930 wait complete. 2024-09-10T21:41:43.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.210+0000 7f3dccc00640 1 -- 172.21.15.7:0/1074897810 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f3d9c0052f0 con 0x7f3dac005290 2024-09-10T21:41:43.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.215+0000 7f3dbe000640 1 -- 172.21.15.7:0/1074897810 <== mon.1 v2:172.21.15.133:3300/0 12 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v14)=0 v14) ==== 120+0+0 (secure 0 0 0) 0x7f3dc0025a50 con 0x7f3dac005290 2024-09-10T21:41:43.218 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi007') 2024-09-10T21:41:43.218 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi007') 2024-09-10T21:41:43.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.217+0000 7f3dccc00640 1 -- 172.21.15.7:0/1074897810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3dac005290 msgr2=0x7f3dc80b82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:43.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.217+0000 7f3dccc00640 1 --2- 172.21.15.7:0/1074897810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3dac005290 0x7f3dc80b82d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3dc80698c0 tx=0x7f3dc0009ec0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.217+0000 7f3dccc00640 1 -- 172.21.15.7:0/1074897810 shutdown_connections 2024-09-10T21:41:43.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.217+0000 7f3dccc00640 1 --2- 172.21.15.7:0/1074897810 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3dac005290 0x7f3dc80b82d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.217+0000 7f3dccc00640 1 --2- 172.21.15.7:0/1074897810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3dac004800 0x7f3dc80b7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.217+0000 7f3dccc00640 1 --2- 172.21.15.7:0/1074897810 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3dac003a00 0x7f3dc815a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.217+0000 7f3dccc00640 1 -- 172.21.15.7:0/1074897810 >> 172.21.15.7:0/1074897810 conn(0x7f3dc80d3f10 msgr2=0x7f3dc80d08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:43.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.217+0000 7f3dccc00640 1 -- 172.21.15.7:0/1074897810 shutdown_connections 2024-09-10T21:41:43.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.218+0000 7f3dccc00640 1 -- 172.21.15.7:0/1074897810 wait complete. 2024-09-10T21:41:43.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.221+0000 7f27f9e00640 1 -- 172.21.15.7:0/1798705140 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f27c80052f0 con 0x7f27dc004f40 2024-09-10T21:41:43.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.223+0000 7f00f3600640 1 -- 172.21.15.7:0/2610032507 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f00c00052f0 con 0x7f00d4005290 2024-09-10T21:41:43.226 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi007') 2024-09-10T21:41:43.226 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi007') 2024-09-10T21:41:43.226 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi007') 2024-09-10T21:41:43.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.225+0000 7f27f0c00640 1 -- 172.21.15.7:0/1798705140 <== mon.1 v2:172.21.15.133:3300/0 12 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v15)=0 v15) ==== 121+0+0 (secure 0 0 0) 0x7f27e0010840 con 0x7f27dc004f40 2024-09-10T21:41:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.226+0000 7f27f9e00640 1 -- 172.21.15.7:0/1798705140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27dc004f40 msgr2=0x7f27f414f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.226+0000 7f27f9e00640 1 --2- 172.21.15.7:0/1798705140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27dc004f40 0x7f27f414f810 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f27e0005d00 tx=0x7f27e0013e90 comp rx=0 tx=0).stop 2024-09-10T21:41:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.226+0000 7f27f9e00640 1 -- 172.21.15.7:0/1798705140 shutdown_connections 2024-09-10T21:41:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.226+0000 7f27f9e00640 1 --2- 172.21.15.7:0/1798705140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f27dc004f40 0x7f27f414f810 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.226+0000 7f27f9e00640 1 --2- 172.21.15.7:0/1798705140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f27f40d0470 0x7f27f41372c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.226+0000 7f27f9e00640 1 --2- 172.21.15.7:0/1798705140 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27dc003aa0 0x7f27f41eb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.226+0000 7f27f9e00640 1 -- 172.21.15.7:0/1798705140 >> 172.21.15.7:0/1798705140 conn(0x7f27f406cd00 msgr2=0x7f27f40ce190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.227+0000 7f27f9e00640 1 -- 172.21.15.7:0/1798705140 shutdown_connections 2024-09-10T21:41:43.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.227+0000 7f27f9e00640 1 -- 172.21.15.7:0/1798705140 wait complete. 2024-09-10T21:41:43.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.228+0000 7f00e8c00640 1 -- 172.21.15.7:0/2610032507 <== mon.1 v2:172.21.15.133:3300/0 12 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v16)=0 v16) ==== 106+0+0 (secure 0 0 0) 0x7f00e40499e0 con 0x7f00d4005290 2024-09-10T21:41:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.230+0000 7f00f3600640 1 -- 172.21.15.7:0/2610032507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00d4005290 msgr2=0x7f00ec160a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.230+0000 7f00f3600640 1 --2- 172.21.15.7:0/2610032507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00d4005290 0x7f00ec160a40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0caf70 tx=0x7f00e4004a40 comp rx=0 tx=0).stop 2024-09-10T21:41:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.230+0000 7f00f3600640 1 -- 172.21.15.7:0/2610032507 shutdown_connections 2024-09-10T21:41:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.230+0000 7f00f3600640 1 --2- 172.21.15.7:0/2610032507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00d4005290 0x7f00ec160a40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.230+0000 7f00f3600640 1 --2- 172.21.15.7:0/2610032507 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00d4004800 0x7f00ec0b9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.230+0000 7f00f3600640 1 --2- 172.21.15.7:0/2610032507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00d4003a00 0x7f00ec0b51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.230+0000 7f00f3600640 1 -- 172.21.15.7:0/2610032507 >> 172.21.15.7:0/2610032507 conn(0x7f00ec0d3f10 msgr2=0x7f00ec15f030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.230+0000 7f00f3600640 1 -- 172.21.15.7:0/2610032507 shutdown_connections 2024-09-10T21:41:43.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:43.230+0000 7f00f3600640 1 -- 172.21.15.7:0/2610032507 wait complete. 2024-09-10T21:41:43.249 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi007') 2024-09-10T21:41:44.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.182+0000 7f1b1a200640 1 -- 172.21.15.7:0/3546170238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1af4007d70 msgr2=0x7f1af4008170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:44.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.182+0000 7f1b1a200640 1 --2- 172.21.15.7:0/3546170238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1af4007d70 0x7f1af4008170 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1b140b5050 tx=0x7f1b0002b9d0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.182+0000 7f1b1a200640 1 -- 172.21.15.7:0/3546170238 shutdown_connections 2024-09-10T21:41:44.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.182+0000 7f1b1a200640 1 --2- 172.21.15.7:0/3546170238 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1af4007d70 0x7f1af4008170 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.182+0000 7f1b1a200640 1 -- 172.21.15.7:0/3546170238 >> 172.21.15.7:0/3546170238 conn(0x7f1b1409d0d0 msgr2=0x7f1b140d3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:44.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.182+0000 7f1b1a200640 1 -- 172.21.15.7:0/3546170238 shutdown_connections 2024-09-10T21:41:44.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.182+0000 7f1b1a200640 1 -- 172.21.15.7:0/3546170238 wait complete. 2024-09-10T21:41:44.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.184+0000 7f1b1a200640 1 Processor -- start 2024-09-10T21:41:44.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.184+0000 7f1b1a200640 1 -- start start 2024-09-10T21:41:44.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.185+0000 7f1b1a200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1af4003b60 0x7f1b141f56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:44.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.185+0000 7f1b1a200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1af4007d70 0x7f1b140b8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:44.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.185+0000 7f1b1a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1af40086b0 0x7f1b140b8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:44.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.185+0000 7f1b1a200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1b140d0790 con 0x7f1af40086b0 2024-09-10T21:41:44.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.185+0000 7f1b1a200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1b140b2f70 con 0x7f1af4007d70 2024-09-10T21:41:44.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.185+0000 7f1b1a200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f1b140d0910 con 0x7f1af4003b60 2024-09-10T21:41:44.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.185+0000 7f1b19800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1af40086b0 0x7f1b140b8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:44.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.185+0000 7f1b18e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1af4003b60 0x7f1b141f56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:44.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.185+0000 7f1b19800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1af40086b0 0x7f1b140b8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45396/0 (socket says 172.21.15.7:45396) 2024-09-10T21:41:44.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.185+0000 7f1b13e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1af4007d70 0x7f1b140b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:44.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.185+0000 7f1b19800640 1 -- 172.21.15.7:0/2619943749 learned_addr learned my addr 172.21.15.7:0/2619943749 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:41:44.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.185+0000 7f1b18e00640 1 -- 172.21.15.7:0/2619943749 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1af4003b60 msgr2=0x7f1b141f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:41:44.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.185+0000 7f1b18e00640 1 -- 172.21.15.7:0/2619943749 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1af4003b60 msgr2=0x7f1b141f56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:41:44.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.185+0000 7f1b18e00640 1 --2- 172.21.15.7:0/2619943749 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1af4003b60 0x7f1b141f56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:41:44.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.185+0000 7f1b18e00640 1 --2- 172.21.15.7:0/2619943749 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1af4003b60 0x7f1b141f56b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:41:44.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.186+0000 7f1b19800640 1 -- 172.21.15.7:0/2619943749 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1af4003b60 msgr2=0x7f1b141f56b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:41:44.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.186+0000 7f1b19800640 1 --2- 172.21.15.7:0/2619943749 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1af4003b60 0x7f1b141f56b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.186+0000 7f1b19800640 1 -- 172.21.15.7:0/2619943749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1af4007d70 msgr2=0x7f1b140b8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:44.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.186+0000 7f1b19800640 1 --2- 172.21.15.7:0/2619943749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1af4007d70 0x7f1b140b8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.186+0000 7f1b19800640 1 -- 172.21.15.7:0/2619943749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b141f5bf0 con 0x7f1af40086b0 2024-09-10T21:41:44.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.186+0000 7f1b19800640 1 --2- 172.21.15.7:0/2619943749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1af40086b0 0x7f1b140b8ef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c005fd0 tx=0x7f1b0c013b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:44.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.186+0000 7f1b10c00640 1 -- 172.21.15.7:0/2619943749 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c011dd0 con 0x7f1af40086b0 2024-09-10T21:41:44.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.186+0000 7f1b10c00640 1 -- 172.21.15.7:0/2619943749 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b0c01c480 con 0x7f1af40086b0 2024-09-10T21:41:44.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.186+0000 7f1b10c00640 1 -- 172.21.15.7:0/2619943749 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b0c01c8f0 con 0x7f1af40086b0 2024-09-10T21:41:44.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.186+0000 7f1b1a200640 1 -- 172.21.15.7:0/2619943749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b140cb430 con 0x7f1af40086b0 2024-09-10T21:41:44.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.186+0000 7f1b1a200640 1 -- 172.21.15.7:0/2619943749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b140cb9f0 con 0x7f1af40086b0 2024-09-10T21:41:44.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.187+0000 7f1b10c00640 1 -- 172.21.15.7:0/2619943749 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f1b0c00f040 con 0x7f1af40086b0 2024-09-10T21:41:44.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.187+0000 7f1b10c00640 1 -- 172.21.15.7:0/2619943749 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1b0c010740 con 0x7f1af40086b0 2024-09-10T21:41:44.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.187+0000 7f1b1a200640 1 -- 172.21.15.7:0/2619943749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b140b4370 con 0x7f1af40086b0 2024-09-10T21:41:44.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.190+0000 7f1b10c00640 1 -- 172.21.15.7:0/2619943749 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f1b141f5bf0 con 0x7f1af40086b0 2024-09-10T21:41:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.235+0000 7f934ea00640 1 -- 172.21.15.7:0/1298381885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9328003b60 msgr2=0x7f9328006650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.235+0000 7f934ea00640 1 --2- 172.21.15.7:0/1298381885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9328003b60 0x7f9328006650 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f93480698c0 tx=0x7f933c02b930 comp rx=0 tx=0).stop 2024-09-10T21:41:44.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.235+0000 7f934ea00640 1 -- 172.21.15.7:0/1298381885 shutdown_connections 2024-09-10T21:41:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.235+0000 7f934ea00640 1 --2- 172.21.15.7:0/1298381885 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9328008690 0x7f932800d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.235+0000 7f934ea00640 1 --2- 172.21.15.7:0/1298381885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9328003b60 0x7f9328006650 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.235+0000 7f934ea00640 1 -- 172.21.15.7:0/1298381885 >> 172.21.15.7:0/1298381885 conn(0x7f93480d3e80 msgr2=0x7f93480d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.235+0000 7f934ea00640 1 -- 172.21.15.7:0/1298381885 shutdown_connections 2024-09-10T21:41:44.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.235+0000 7f934ea00640 1 -- 172.21.15.7:0/1298381885 wait complete. 2024-09-10T21:41:44.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.236+0000 7f934ea00640 1 Processor -- start 2024-09-10T21:41:44.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.237+0000 7f934ea00640 1 -- start start 2024-09-10T21:41:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.237+0000 7f934ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9328003b60 0x7f934815af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.237+0000 7f934ea00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9328006b90 0x7f9348153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.238+0000 7f934ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9328008690 0x7f93481544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.238+0000 7f934ea00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f93480d2a80 con 0x7f9328008690 2024-09-10T21:41:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.238+0000 7f934ea00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f93480d2900 con 0x7f9328003b60 2024-09-10T21:41:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.238+0000 7f934ea00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f93480b0a10 con 0x7f9328006b90 2024-09-10T21:41:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.238+0000 7f934d600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9328003b60 0x7f934815af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.238+0000 7f934e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9328008690 0x7f93481544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.238+0000 7f934d600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9328003b60 0x7f934815af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:34332/0 (socket says 172.21.15.7:34332) 2024-09-10T21:41:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.238+0000 7f934d600640 1 -- 172.21.15.7:0/4276856187 learned_addr learned my addr 172.21.15.7:0/4276856187 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:41:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.238+0000 7f934cc00640 1 --2- 172.21.15.7:0/4276856187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9328006b90 0x7f9348153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:44.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.238+0000 7f934cc00640 1 -- 172.21.15.7:0/4276856187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9328006b90 msgr2=0x7f9348153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:41:44.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.238+0000 7f934cc00640 1 -- 172.21.15.7:0/4276856187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9328006b90 msgr2=0x7f9348153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:41:44.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.238+0000 7f934cc00640 1 --2- 172.21.15.7:0/4276856187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9328006b90 0x7f9348153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:41:44.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.238+0000 7f934cc00640 1 --2- 172.21.15.7:0/4276856187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9328006b90 0x7f9348153f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:41:44.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.238+0000 7f934e000640 1 -- 172.21.15.7:0/4276856187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9328006b90 msgr2=0x7f9348153f60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:41:44.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.238+0000 7f934e000640 1 --2- 172.21.15.7:0/4276856187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9328006b90 0x7f9348153f60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.238+0000 7f934e000640 1 -- 172.21.15.7:0/4276856187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9328003b60 msgr2=0x7f934815af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:44.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.238+0000 7f934e000640 1 --2- 172.21.15.7:0/4276856187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9328003b60 0x7f934815af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.238+0000 7f934e000640 1 -- 172.21.15.7:0/4276856187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93481549e0 con 0x7f9328008690 2024-09-10T21:41:44.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.238+0000 7f934d600640 1 --2- 172.21.15.7:0/4276856187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9328003b60 0x7f934815af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:41:44.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.239+0000 7f934e000640 1 --2- 172.21.15.7:0/4276856187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9328008690 0x7f93481544a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f93480b9f10 tx=0x7f9340012e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:44.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.239+0000 7f9345600640 1 -- 172.21.15.7:0/4276856187 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340020020 con 0x7f9328008690 2024-09-10T21:41:44.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.239+0000 7f9345600640 1 -- 172.21.15.7:0/4276856187 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9340010e50 con 0x7f9328008690 2024-09-10T21:41:44.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.239+0000 7f9345600640 1 -- 172.21.15.7:0/4276856187 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9340004d80 con 0x7f9328008690 2024-09-10T21:41:44.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.239+0000 7f934ea00640 1 -- 172.21.15.7:0/4276856187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93480c9710 con 0x7f9328008690 2024-09-10T21:41:44.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.239+0000 7f934ea00640 1 -- 172.21.15.7:0/4276856187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9348158760 con 0x7f9328008690 2024-09-10T21:41:44.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.239+0000 7f9345600640 1 -- 172.21.15.7:0/4276856187 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f934000f040 con 0x7f9328008690 2024-09-10T21:41:44.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.240+0000 7f9345600640 1 -- 172.21.15.7:0/4276856187 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f934001ca70 con 0x7f9328008690 2024-09-10T21:41:44.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.240+0000 7f934ea00640 1 -- 172.21.15.7:0/4276856187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93480b6cd0 con 0x7f9328008690 2024-09-10T21:41:44.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.242+0000 7f9345600640 1 -- 172.21.15.7:0/4276856187 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f9340025530 con 0x7f9328008690 2024-09-10T21:41:44.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.258+0000 7f1b1a200640 1 -- 172.21.15.7:0/2619943749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f1b1406e410 con 0x7f1af40086b0 2024-09-10T21:41:44.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.260+0000 7f1b10c00640 1 -- 172.21.15.7:0/2619943749 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v17)=0 v17) ==== 115+0+0 (secure 0 0 0) 0x7f1b0c025cd0 con 0x7f1af40086b0 2024-09-10T21:41:44.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.261+0000 7f1b1a200640 1 -- 172.21.15.7:0/2619943749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1af40086b0 msgr2=0x7f1b140b8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:44.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.261+0000 7f1b1a200640 1 --2- 172.21.15.7:0/2619943749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1af40086b0 0x7f1b140b8ef0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c005fd0 tx=0x7f1b0c013b20 comp rx=0 tx=0).stop 2024-09-10T21:41:44.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.262+0000 7f1b1a200640 1 -- 172.21.15.7:0/2619943749 shutdown_connections 2024-09-10T21:41:44.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.262+0000 7f1b1a200640 1 --2- 172.21.15.7:0/2619943749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1af4007d70 0x7f1b140b8380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.262+0000 7f1b1a200640 1 --2- 172.21.15.7:0/2619943749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1af40086b0 0x7f1b140b8ef0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.262+0000 7f1b1a200640 1 --2- 172.21.15.7:0/2619943749 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1af4003b60 0x7f1b141f56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.262+0000 7f1b1a200640 1 -- 172.21.15.7:0/2619943749 >> 172.21.15.7:0/2619943749 conn(0x7f1b1409d0d0 msgr2=0x7f1b1406d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.262+0000 7f1b1a200640 1 -- 172.21.15.7:0/2619943749 shutdown_connections 2024-09-10T21:41:44.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.262+0000 7f1b1a200640 1 -- 172.21.15.7:0/2619943749 wait complete. 2024-09-10T21:41:44.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.274+0000 7fa121800640 1 -- 172.21.15.7:0/3465361417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0fc003b50 msgr2=0x7fa0fc006630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:44.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.274+0000 7fa121800640 1 --2- 172.21.15.7:0/3465361417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0fc003b50 0x7fa0fc006630 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fa10c006b20 tx=0x7fa10c02b890 comp rx=0 tx=0).stop 2024-09-10T21:41:44.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.274+0000 7fa121800640 1 -- 172.21.15.7:0/3465361417 shutdown_connections 2024-09-10T21:41:44.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.274+0000 7fa121800640 1 --2- 172.21.15.7:0/3465361417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0fc006b70 0x7fa0fc006f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.274+0000 7fa121800640 1 --2- 172.21.15.7:0/3465361417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0fc003b50 0x7fa0fc006630 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.274+0000 7fa121800640 1 -- 172.21.15.7:0/3465361417 >> 172.21.15.7:0/3465361417 conn(0x7fa11c0b55e0 msgr2=0x7fa11c0b59c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:44.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.274+0000 7fa121800640 1 -- 172.21.15.7:0/3465361417 shutdown_connections 2024-09-10T21:41:44.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.274+0000 7fa121800640 1 -- 172.21.15.7:0/3465361417 wait complete. 2024-09-10T21:41:44.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.276+0000 7fa121800640 1 Processor -- start 2024-09-10T21:41:44.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.276+0000 7fa121800640 1 -- start start 2024-09-10T21:41:44.278 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi007') 2024-09-10T21:41:44.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.277+0000 7fa121800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0fc003b50 0x7fa11c1f5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:44.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.277+0000 7fa121800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0fc006b70 0x7fa11c15c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:44.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.277+0000 7fa121800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0fc007490 0x7fa11c1ee810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:44.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.277+0000 7fa121800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c074440 con 0x7fa0fc007490 2024-09-10T21:41:44.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.277+0000 7fa121800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa11c0742c0 con 0x7fa0fc006b70 2024-09-10T21:41:44.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.277+0000 7fa121800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa11c0b3a90 con 0x7fa0fc003b50 2024-09-10T21:41:44.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.277+0000 7fa120e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0fc007490 0x7fa11c1ee810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:44.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.277+0000 7fa11be00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0fc003b50 0x7fa11c1f5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:44.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.277+0000 7fa120e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0fc007490 0x7fa11c1ee810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45422/0 (socket says 172.21.15.7:45422) 2024-09-10T21:41:44.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.277+0000 7fa120e00640 1 -- 172.21.15.7:0/1671389929 learned_addr learned my addr 172.21.15.7:0/1671389929 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:41:44.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.277+0000 7fa11b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0fc006b70 0x7fa11c15c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:44.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.277+0000 7fa11be00640 1 -- 172.21.15.7:0/1671389929 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0fc003b50 msgr2=0x7fa11c1f5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:41:44.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.277+0000 7fa11be00640 1 -- 172.21.15.7:0/1671389929 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0fc003b50 msgr2=0x7fa11c1f5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:41:44.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.277+0000 7fa11be00640 1 --2- 172.21.15.7:0/1671389929 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0fc003b50 0x7fa11c1f5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:41:44.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.277+0000 7fa11be00640 1 --2- 172.21.15.7:0/1671389929 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0fc003b50 0x7fa11c1f5640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:41:44.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.278+0000 7fa120e00640 1 -- 172.21.15.7:0/1671389929 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0fc003b50 msgr2=0x7fa11c1f5640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:41:44.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.278+0000 7fa120e00640 1 --2- 172.21.15.7:0/1671389929 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0fc003b50 0x7fa11c1f5640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.278+0000 7fa120e00640 1 -- 172.21.15.7:0/1671389929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0fc006b70 msgr2=0x7fa11c15c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:44.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.278+0000 7fa120e00640 1 --2- 172.21.15.7:0/1671389929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0fc006b70 0x7fa11c15c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.278+0000 7fa120e00640 1 -- 172.21.15.7:0/1671389929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa11c15cfe0 con 0x7fa0fc007490 2024-09-10T21:41:44.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.278+0000 7fa11b400640 1 --2- 172.21.15.7:0/1671389929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0fc006b70 0x7fa11c15c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:41:44.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.278+0000 7fa120e00640 1 --2- 172.21.15.7:0/1671389929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0fc007490 0x7fa11c1ee810 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa11c075090 tx=0x7fa11400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:44.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.278+0000 7fa113e00640 1 -- 172.21.15.7:0/1671389929 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa114017070 con 0x7fa0fc007490 2024-09-10T21:41:44.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.278+0000 7fa113e00640 1 -- 172.21.15.7:0/1671389929 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa114003c80 con 0x7fa0fc007490 2024-09-10T21:41:44.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.278+0000 7fa121800640 1 -- 172.21.15.7:0/1671389929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa11c15d2c0 con 0x7fa0fc007490 2024-09-10T21:41:44.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.278+0000 7fa113e00640 1 -- 172.21.15.7:0/1671389929 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1140052d0 con 0x7fa0fc007490 2024-09-10T21:41:44.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.278+0000 7fa121800640 1 -- 172.21.15.7:0/1671389929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa11c15d880 con 0x7fa0fc007490 2024-09-10T21:41:44.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.279+0000 7fa113e00640 1 -- 172.21.15.7:0/1671389929 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fa114005470 con 0x7fa0fc007490 2024-09-10T21:41:44.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.279+0000 7fa113e00640 1 -- 172.21.15.7:0/1671389929 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa11401c780 con 0x7fa0fc007490 2024-09-10T21:41:44.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.280+0000 7fa121800640 1 -- 172.21.15.7:0/1671389929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa11c0b47b0 con 0x7fa0fc007490 2024-09-10T21:41:44.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.283+0000 7fa113e00640 1 -- 172.21.15.7:0/1671389929 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fa114021020 con 0x7fa0fc007490 2024-09-10T21:41:44.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.312+0000 7f934ea00640 1 -- 172.21.15.7:0/4276856187 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f93480a1d80 con 0x7f9328008690 2024-09-10T21:41:44.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.314+0000 7f9345600640 1 -- 172.21.15.7:0/4276856187 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v18)=0 v18) ==== 122+0+0 (secure 0 0 0) 0x7f9340025710 con 0x7f9328008690 2024-09-10T21:41:44.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.315+0000 7f934ea00640 1 -- 172.21.15.7:0/4276856187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9328008690 msgr2=0x7f93481544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:44.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.315+0000 7f934ea00640 1 --2- 172.21.15.7:0/4276856187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9328008690 0x7f93481544a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f93480b9f10 tx=0x7f9340012e20 comp rx=0 tx=0).stop 2024-09-10T21:41:44.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.315+0000 7f934ea00640 1 -- 172.21.15.7:0/4276856187 shutdown_connections 2024-09-10T21:41:44.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.315+0000 7f934ea00640 1 --2- 172.21.15.7:0/4276856187 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9328008690 0x7f93481544a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.315+0000 7f934ea00640 1 --2- 172.21.15.7:0/4276856187 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9328006b90 0x7f9348153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.315+0000 7f934ea00640 1 --2- 172.21.15.7:0/4276856187 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9328003b60 0x7f934815af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.315+0000 7f934ea00640 1 -- 172.21.15.7:0/4276856187 >> 172.21.15.7:0/4276856187 conn(0x7f93480d3e80 msgr2=0x7f93480b7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:44.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.316+0000 7f934ea00640 1 -- 172.21.15.7:0/4276856187 shutdown_connections 2024-09-10T21:41:44.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.316+0000 7f934ea00640 1 -- 172.21.15.7:0/4276856187 wait complete. 2024-09-10T21:41:44.331 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi007') 2024-09-10T21:41:44.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.380+0000 7fa121800640 1 -- 172.21.15.7:0/1671389929 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7fa11c0b2a80 con 0x7fa0fc007490 2024-09-10T21:41:44.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.383+0000 7fa113e00640 1 -- 172.21.15.7:0/1671389929 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v19)=0 v19) ==== 121+0+0 (secure 0 0 0) 0x7fa11401c980 con 0x7fa0fc007490 2024-09-10T21:41:44.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.384+0000 7fa121800640 1 -- 172.21.15.7:0/1671389929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0fc007490 msgr2=0x7fa11c1ee810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.384+0000 7fa121800640 1 --2- 172.21.15.7:0/1671389929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0fc007490 0x7fa11c1ee810 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa11c075090 tx=0x7fa11400cb90 comp rx=0 tx=0).stop 2024-09-10T21:41:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.384+0000 7fa121800640 1 -- 172.21.15.7:0/1671389929 shutdown_connections 2024-09-10T21:41:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.384+0000 7fa121800640 1 --2- 172.21.15.7:0/1671389929 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0fc006b70 0x7fa11c15c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.384+0000 7fa121800640 1 --2- 172.21.15.7:0/1671389929 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa0fc007490 0x7fa11c1ee810 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.384+0000 7fa121800640 1 --2- 172.21.15.7:0/1671389929 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0fc003b50 0x7fa11c1f5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.384+0000 7fa121800640 1 -- 172.21.15.7:0/1671389929 >> 172.21.15.7:0/1671389929 conn(0x7fa11c0b55e0 msgr2=0x7fa11c0b1820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.384+0000 7fa121800640 1 -- 172.21.15.7:0/1671389929 shutdown_connections 2024-09-10T21:41:44.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.384+0000 7fa121800640 1 -- 172.21.15.7:0/1671389929 wait complete. 2024-09-10T21:41:44.400 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi007') 2024-09-10T21:41:44.401 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi007') 2024-09-10T21:41:44.401 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi007') 2024-09-10T21:41:44.401 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi007') 2024-09-10T21:41:44.401 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi007') 2024-09-10T21:41:44.401 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi007') 2024-09-10T21:41:44.401 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi007') 2024-09-10T21:41:44.401 INFO:tasks.ceph:Setting crush tunables to default 2024-09-10T21:41:44.402 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph osd crush tunables default 2024-09-10T21:41:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.555+0000 7f583c200640 1 -- 172.21.15.7:0/2710983559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340d1f90 msgr2=0x7f58340d23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.555+0000 7f583c200640 1 --2- 172.21.15.7:0/2710983559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340d1f90 0x7f58340d23e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f583408ca40 tx=0x7f582c02a9a0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.555+0000 7f583c200640 1 -- 172.21.15.7:0/2710983559 shutdown_connections 2024-09-10T21:41:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.556+0000 7f583c200640 1 --2- 172.21.15.7:0/2710983559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340d1f90 0x7f58340d23e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.556+0000 7f583c200640 1 --2- 172.21.15.7:0/2710983559 >> v2:172.21.15.133:3300/0 conn(0x7f583415e120 0x7f58340d4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.556+0000 7f583c200640 1 -- 172.21.15.7:0/2710983559 >> 172.21.15.7:0/2710983559 conn(0x7f58340d3f10 msgr2=0x7f58340d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:44.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.556+0000 7f583c200640 1 -- 172.21.15.7:0/2710983559 shutdown_connections 2024-09-10T21:41:44.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.556+0000 7f583c200640 1 -- 172.21.15.7:0/2710983559 wait complete. 2024-09-10T21:41:44.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.557+0000 7f583c200640 1 Processor -- start 2024-09-10T21:41:44.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.557+0000 7f583c200640 1 -- start start 2024-09-10T21:41:44.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.558+0000 7f583c200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58340d10d0 0x7f58340c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:44.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.558+0000 7f583c200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340d1f90 0x7f58340c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:44.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.558+0000 7f583c200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f583415e120 0x7f58341f7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:44.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.558+0000 7f583c200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58340b70f0 con 0x7f58340d1f90 2024-09-10T21:41:44.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.558+0000 7f583c200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f58340b6f70 con 0x7f583415e120 2024-09-10T21:41:44.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.558+0000 7f583c200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f58340b7270 con 0x7f58340d10d0 2024-09-10T21:41:44.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.559+0000 7f5838e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340d1f90 0x7f58340c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:44.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.559+0000 7f5838e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340d1f90 0x7f58340c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45436/0 (socket says 172.21.15.7:45436) 2024-09-10T21:41:44.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.559+0000 7f583a200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f583415e120 0x7f58341f7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:44.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.559+0000 7f5838e00640 1 -- 172.21.15.7:0/2115611469 learned_addr learned my addr 172.21.15.7:0/2115611469 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:41:44.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.559+0000 7f5839800640 1 --2- 172.21.15.7:0/2115611469 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58340d10d0 0x7f58340c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:44.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.559+0000 7f5839800640 1 -- 172.21.15.7:0/2115611469 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58340d10d0 msgr2=0x7f58340c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:41:44.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.559+0000 7f5839800640 1 -- 172.21.15.7:0/2115611469 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58340d10d0 msgr2=0x7f58340c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:41:44.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.559+0000 7f5839800640 1 --2- 172.21.15.7:0/2115611469 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58340d10d0 0x7f58340c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:41:44.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.559+0000 7f5839800640 1 --2- 172.21.15.7:0/2115611469 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58340d10d0 0x7f58340c9ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:41:44.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.559+0000 7f5838e00640 1 -- 172.21.15.7:0/2115611469 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58340d10d0 msgr2=0x7f58340c9ec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:41:44.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.559+0000 7f5838e00640 1 --2- 172.21.15.7:0/2115611469 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58340d10d0 0x7f58340c9ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.559+0000 7f5838e00640 1 -- 172.21.15.7:0/2115611469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f583415e120 msgr2=0x7f58341f7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:44.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.559+0000 7f5838e00640 1 --2- 172.21.15.7:0/2115611469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f583415e120 0x7f58341f7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.559+0000 7f5838e00640 1 -- 172.21.15.7:0/2115611469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58341f88b0 con 0x7f58340d1f90 2024-09-10T21:41:44.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.559+0000 7f583a200640 1 --2- 172.21.15.7:0/2115611469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f583415e120 0x7f58341f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:41:44.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.559+0000 7f5838e00640 1 --2- 172.21.15.7:0/2115611469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340d1f90 0x7f58340c8f40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f582000c4d0 tx=0x7f582000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:44.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.560+0000 7f5831600640 1 -- 172.21.15.7:0/2115611469 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820016020 con 0x7f58340d1f90 2024-09-10T21:41:44.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.560+0000 7f5831600640 1 -- 172.21.15.7:0/2115611469 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5820005310 con 0x7f58340d1f90 2024-09-10T21:41:44.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.560+0000 7f583c200640 1 -- 172.21.15.7:0/2115611469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58341f8b90 con 0x7f58340d1f90 2024-09-10T21:41:44.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.560+0000 7f5831600640 1 -- 172.21.15.7:0/2115611469 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5820005500 con 0x7f58340d1f90 2024-09-10T21:41:44.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.560+0000 7f583c200640 1 -- 172.21.15.7:0/2115611469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58340a0bc0 con 0x7f58340d1f90 2024-09-10T21:41:44.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.560+0000 7f5831600640 1 -- 172.21.15.7:0/2115611469 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f5820007030 con 0x7f58340d1f90 2024-09-10T21:41:44.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.560+0000 7f5831600640 1 -- 172.21.15.7:0/2115611469 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5820012dd0 con 0x7f58340d1f90 2024-09-10T21:41:44.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.561+0000 7f583c200640 1 -- 172.21.15.7:0/2115611469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58340b7800 con 0x7f58340d1f90 2024-09-10T21:41:44.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.563+0000 7f5831600640 1 -- 172.21.15.7:0/2115611469 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f5820020020 con 0x7f58340d1f90 2024-09-10T21:41:44.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.632+0000 7f583c200640 1 -- 172.21.15.7:0/2115611469 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f58340a0d90 con 0x7f58340d1f90 2024-09-10T21:41:44.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.634+0000 7f5831600640 1 -- 172.21.15.7:0/2115611469 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f5820007580 con 0x7f58340d1f90 2024-09-10T21:41:44.636 INFO:teuthology.orchestra.run.smithi007.stderr:adjusted tunables profile to default 2024-09-10T21:41:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.636+0000 7f583c200640 1 -- 172.21.15.7:0/2115611469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340d1f90 msgr2=0x7f58340c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.636+0000 7f583c200640 1 --2- 172.21.15.7:0/2115611469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340d1f90 0x7f58340c8f40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f582000c4d0 tx=0x7f582000c990 comp rx=0 tx=0).stop 2024-09-10T21:41:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.636+0000 7f583c200640 1 -- 172.21.15.7:0/2115611469 shutdown_connections 2024-09-10T21:41:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.636+0000 7f583c200640 1 --2- 172.21.15.7:0/2115611469 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f583415e120 0x7f58341f7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.636+0000 7f583c200640 1 --2- 172.21.15.7:0/2115611469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58340d1f90 0x7f58340c8f40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.636+0000 7f583c200640 1 --2- 172.21.15.7:0/2115611469 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58340d10d0 0x7f58340c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:44.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.636+0000 7f583c200640 1 -- 172.21.15.7:0/2115611469 >> 172.21.15.7:0/2115611469 conn(0x7f58340d3f10 msgr2=0x7f58340b89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:44.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.636+0000 7f583c200640 1 -- 172.21.15.7:0/2115611469 shutdown_connections 2024-09-10T21:41:44.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:44.636+0000 7f583c200640 1 -- 172.21.15.7:0/2115611469 wait complete. 2024-09-10T21:41:44.652 INFO:tasks.ceph:check_enable_crimson: False 2024-09-10T21:41:44.652 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-10T21:41:44.652 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:41:44.652 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-10T21:41:44.708 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:41:44.708 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-10T21:41:44.805 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:41:44.805 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-10T21:41:44.901 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:41:44.901 DEBUG:teuthology.orchestra.run.smithi007:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-10T21:41:44.998 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:41:44.998 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-10T21:41:45.054 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:41:45.054 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-10T21:41:45.150 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:41:45.150 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-10T21:41:45.246 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:41:45.247 DEBUG:teuthology.orchestra.run.smithi133:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-10T21:41:45.342 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new c49a2472-5948-485f-8fef-3848b9e3bce3 0 2024-09-10T21:41:45.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.536+0000 7f69bf200640 1 -- 172.21.15.133:0/3711245389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69a4004800 msgr2=0x7f69a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:45.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.536+0000 7f69bf200640 1 --2- 172.21.15.133:0/3711245389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69a4004800 0x7f69a4004c80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f69a8009b20 tx=0x7f69a4005830 comp rx=0 tx=0).stop 2024-09-10T21:41:45.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.536+0000 7f69bf200640 1 -- 172.21.15.133:0/3711245389 shutdown_connections 2024-09-10T21:41:45.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.536+0000 7f69bf200640 1 --2- 172.21.15.133:0/3711245389 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69a4004800 0x7f69a4004c80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:45.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.536+0000 7f69bf200640 1 -- 172.21.15.133:0/3711245389 >> 172.21.15.133:0/3711245389 conn(0x7f69c00d3de0 msgr2=0x7f69c00d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:45.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.536+0000 7f69bf200640 1 -- 172.21.15.133:0/3711245389 shutdown_connections 2024-09-10T21:41:45.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.536+0000 7f69bf200640 1 -- 172.21.15.133:0/3711245389 wait complete. 2024-09-10T21:41:45.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.537+0000 7f69bf200640 1 Processor -- start 2024-09-10T21:41:45.538 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.538+0000 7f69bf200640 1 -- start start 2024-09-10T21:41:45.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.538+0000 7f69bf200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69a4004800 0x7f69c015c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:45.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.538+0000 7f69bf200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01fc3c0 0x7f69c009db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:45.541 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.538+0000 7f69bf200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69c00b8de0 0x7f69c00b9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.539+0000 7f69bf200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f69c00d4c90 con 0x7f69c00b8de0 2024-09-10T21:41:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.539+0000 7f69bf200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f69c00d4b10 con 0x7f69c01fc3c0 2024-09-10T21:41:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.539+0000 7f69bf200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f69c00cf4e0 con 0x7f69a4004800 2024-09-10T21:41:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.539+0000 7f69bde00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69a4004800 0x7f69c015c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.539+0000 7f69bde00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69a4004800 0x7f69c015c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:55156/0 (socket says 172.21.15.133:55156) 2024-09-10T21:41:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.539+0000 7f69bde00640 1 -- 172.21.15.133:0/1215683176 learned_addr learned my addr 172.21.15.133:0/1215683176 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-10T21:41:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.539+0000 7f69b7e00640 1 --2- 172.21.15.133:0/1215683176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01fc3c0 0x7f69c009db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.539+0000 7f69bde00640 1 -- 172.21.15.133:0/1215683176 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69a4004800 msgr2=0x7f69c015c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:41:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.539+0000 7f69bde00640 1 -- 172.21.15.133:0/1215683176 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69a4004800 msgr2=0x7f69c015c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:41:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.539+0000 7f69bde00640 1 --2- 172.21.15.133:0/1215683176 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69a4004800 0x7f69c015c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:41:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.539+0000 7f69be800640 1 --2- 172.21.15.133:0/1215683176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69c00b8de0 0x7f69c00b9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.539+0000 7f69bde00640 1 --2- 172.21.15.133:0/1215683176 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69a4004800 0x7f69c015c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:41:45.542 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.539+0000 7f69b7e00640 1 -- 172.21.15.133:0/1215683176 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69a4004800 msgr2=0x7f69c015c370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:41:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.539+0000 7f69b7e00640 1 --2- 172.21.15.133:0/1215683176 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69a4004800 0x7f69c015c370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.539+0000 7f69b7e00640 1 -- 172.21.15.133:0/1215683176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69c00b8de0 msgr2=0x7f69c00b9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.539+0000 7f69b7e00640 1 --2- 172.21.15.133:0/1215683176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69c00b8de0 0x7f69c00b9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.539+0000 7f69b7e00640 1 -- 172.21.15.133:0/1215683176 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69c015c8b0 con 0x7f69c01fc3c0 2024-09-10T21:41:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.539+0000 7f69be800640 1 --2- 172.21.15.133:0/1215683176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69c00b8de0 0x7f69c00b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:41:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.539+0000 7f69b7e00640 1 --2- 172.21.15.133:0/1215683176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01fc3c0 0x7f69c009db10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f69a400c440 tx=0x7f69a80336f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.540+0000 7f69b6000640 1 -- 172.21.15.133:0/1215683176 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8038040 con 0x7f69c01fc3c0 2024-09-10T21:41:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.540+0000 7f69b6000640 1 -- 172.21.15.133:0/1215683176 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69a80338e0 con 0x7f69c01fc3c0 2024-09-10T21:41:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.540+0000 7f69b6000640 1 -- 172.21.15.133:0/1215683176 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69a8033d30 con 0x7f69c01fc3c0 2024-09-10T21:41:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.540+0000 7f69bf200640 1 -- 172.21.15.133:0/1215683176 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69c009e410 con 0x7f69c01fc3c0 2024-09-10T21:41:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.540+0000 7f69bf200640 1 -- 172.21.15.133:0/1215683176 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69c009e9d0 con 0x7f69c01fc3c0 2024-09-10T21:41:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.540+0000 7f69b6000640 1 -- 172.21.15.133:0/1215683176 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f69a803b030 con 0x7f69c01fc3c0 2024-09-10T21:41:45.543 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.540+0000 7f69b6000640 1 -- 172.21.15.133:0/1215683176 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f69a8004a70 con 0x7f69c01fc3c0 2024-09-10T21:41:45.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.541+0000 7f69bf200640 1 -- 172.21.15.133:0/1215683176 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69c00751b0 con 0x7f69c01fc3c0 2024-09-10T21:41:45.544 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.543+0000 7f69b6000640 1 -- 172.21.15.133:0/1215683176 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f69a804c020 con 0x7f69c01fc3c0 2024-09-10T21:41:45.619 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.618+0000 7f69bf200640 1 -- 172.21.15.133:0/1215683176 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c49a2472-5948-485f-8fef-3848b9e3bce3", "id": 0} v 0) -- 0x7f69c00d05d0 con 0x7f69c01fc3c0 2024-09-10T21:41:45.624 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.623+0000 7f69b6000640 1 -- 172.21.15.133:0/1215683176 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c49a2472-5948-485f-8fef-3848b9e3bce3", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f69c015c8b0 con 0x7f69c01fc3c0 2024-09-10T21:41:45.624 INFO:teuthology.orchestra.run.smithi133.stdout:0 2024-09-10T21:41:45.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.624+0000 7f69bf200640 1 -- 172.21.15.133:0/1215683176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01fc3c0 msgr2=0x7f69c009db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:45.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.624+0000 7f69bf200640 1 --2- 172.21.15.133:0/1215683176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01fc3c0 0x7f69c009db10 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f69a400c440 tx=0x7f69a80336f0 comp rx=0 tx=0).stop 2024-09-10T21:41:45.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.624+0000 7f69bf200640 1 -- 172.21.15.133:0/1215683176 shutdown_connections 2024-09-10T21:41:45.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.624+0000 7f69bf200640 1 --2- 172.21.15.133:0/1215683176 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69c01fc3c0 0x7f69c009db10 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:45.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.624+0000 7f69bf200640 1 --2- 172.21.15.133:0/1215683176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69c00b8de0 0x7f69c00b9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:45.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.624+0000 7f69bf200640 1 --2- 172.21.15.133:0/1215683176 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69a4004800 0x7f69c015c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:45.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.624+0000 7f69bf200640 1 -- 172.21.15.133:0/1215683176 >> 172.21.15.133:0/1215683176 conn(0x7f69c00d3de0 msgr2=0x7f69c00d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:45.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.624+0000 7f69bf200640 1 -- 172.21.15.133:0/1215683176 shutdown_connections 2024-09-10T21:41:45.626 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.624+0000 7f69bf200640 1 -- 172.21.15.133:0/1215683176 wait complete. 2024-09-10T21:41:45.640 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 6ad0e4c6-e412-4bf0-906f-aa8fdfecb405 1 2024-09-10T21:41:45.795 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.794+0000 7f25d9a00640 1 -- 172.21.15.133:0/3509585242 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25d40b4bc0 msgr2=0x7f25b8004c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.794+0000 7f25d9a00640 1 --2- 172.21.15.133:0/3509585242 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25d40b4bc0 0x7f25b8004c20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f25d40b5050 tx=0x7f25b8005830 comp rx=0 tx=0).stop 2024-09-10T21:41:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.794+0000 7f25d9000640 1 -- 172.21.15.133:0/3509585242 reap_dead start 2024-09-10T21:41:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.794+0000 7f25d9a00640 1 -- 172.21.15.133:0/3509585242 shutdown_connections 2024-09-10T21:41:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.794+0000 7f25d9a00640 1 -- 172.21.15.133:0/3509585242 >> 172.21.15.133:0/3509585242 conn(0x7f25d409d0d0 msgr2=0x7f25d40d3da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.794+0000 7f25d9a00640 1 -- 172.21.15.133:0/3509585242 shutdown_connections 2024-09-10T21:41:45.796 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.794+0000 7f25d9a00640 1 -- 172.21.15.133:0/3509585242 wait complete. 2024-09-10T21:41:45.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.796+0000 7f25d9a00640 1 Processor -- start 2024-09-10T21:41:45.797 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.796+0000 7f25d9a00640 1 -- start start 2024-09-10T21:41:45.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.797+0000 7f25d9a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25d40b4bc0 0x7f25d4159d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:45.799 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.797+0000 7f25d9a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25d41f8840 0x7f25d40ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:45.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.797+0000 7f25d9a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25d415c8e0 0x7f25d415cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:45.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.797+0000 7f25d9a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f25d40cf0c0 con 0x7f25d415c8e0 2024-09-10T21:41:45.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.797+0000 7f25d9a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f25d40b2f10 con 0x7f25d41f8840 2024-09-10T21:41:45.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.797+0000 7f25d9a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f25d40cf240 con 0x7f25d40b4bc0 2024-09-10T21:41:45.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.797+0000 7f25d3e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25d40b4bc0 0x7f25d4159d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:45.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.797+0000 7f25d9000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25d415c8e0 0x7f25d415cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:45.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.797+0000 7f25d3e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25d40b4bc0 0x7f25d4159d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:55176/0 (socket says 172.21.15.133:55176) 2024-09-10T21:41:45.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.797+0000 7f25cbe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25d41f8840 0x7f25d40ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:45.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.797+0000 7f25d3e00640 1 -- 172.21.15.133:0/2391198794 learned_addr learned my addr 172.21.15.133:0/2391198794 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-10T21:41:45.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.797+0000 7f25d3e00640 1 -- 172.21.15.133:0/2391198794 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25d40b4bc0 msgr2=0x7f25d4159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:41:45.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.797+0000 7f25d3e00640 1 -- 172.21.15.133:0/2391198794 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25d40b4bc0 msgr2=0x7f25d4159d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:41:45.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.797+0000 7f25d3e00640 1 --2- 172.21.15.133:0/2391198794 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25d40b4bc0 0x7f25d4159d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:41:45.800 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.797+0000 7f25d3e00640 1 --2- 172.21.15.133:0/2391198794 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25d40b4bc0 0x7f25d4159d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:41:45.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.797+0000 7f25cbe00640 1 -- 172.21.15.133:0/2391198794 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25d40b4bc0 msgr2=0x7f25d4159d90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:41:45.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.797+0000 7f25cbe00640 1 --2- 172.21.15.133:0/2391198794 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25d40b4bc0 0x7f25d4159d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:45.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.798+0000 7f25cbe00640 1 -- 172.21.15.133:0/2391198794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25d415c8e0 msgr2=0x7f25d415cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:45.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.798+0000 7f25cbe00640 1 --2- 172.21.15.133:0/2391198794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25d415c8e0 0x7f25d415cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:45.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.798+0000 7f25cbe00640 1 -- 172.21.15.133:0/2391198794 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25d415a420 con 0x7f25d41f8840 2024-09-10T21:41:45.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.798+0000 7f25d9000640 1 --2- 172.21.15.133:0/2391198794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25d415c8e0 0x7f25d415cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:41:45.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.798+0000 7f25cbe00640 1 --2- 172.21.15.133:0/2391198794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25d41f8840 0x7f25d40ca250 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f25bc001440 tx=0x7f25bc001e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:45.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.798+0000 7f25d0c00640 1 -- 172.21.15.133:0/2391198794 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc015070 con 0x7f25d41f8840 2024-09-10T21:41:45.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.798+0000 7f25d0c00640 1 -- 172.21.15.133:0/2391198794 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25bc011650 con 0x7f25d41f8840 2024-09-10T21:41:45.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.798+0000 7f25d9a00640 1 -- 172.21.15.133:0/2391198794 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25d415b880 con 0x7f25d41f8840 2024-09-10T21:41:45.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.798+0000 7f25d0c00640 1 -- 172.21.15.133:0/2391198794 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25bc011aa0 con 0x7f25d41f8840 2024-09-10T21:41:45.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.798+0000 7f25d9a00640 1 -- 172.21.15.133:0/2391198794 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25d415be00 con 0x7f25d41f8840 2024-09-10T21:41:45.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.798+0000 7f25d0c00640 1 -- 172.21.15.133:0/2391198794 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f25bc011c40 con 0x7f25d41f8840 2024-09-10T21:41:45.801 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.799+0000 7f25d0c00640 1 -- 172.21.15.133:0/2391198794 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f25bc023820 con 0x7f25d41f8840 2024-09-10T21:41:45.802 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.800+0000 7f25d9a00640 1 -- 172.21.15.133:0/2391198794 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d40ce9b0 con 0x7f25d41f8840 2024-09-10T21:41:45.803 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.802+0000 7f25d0c00640 1 -- 172.21.15.133:0/2391198794 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f25d415a420 con 0x7f25d41f8840 2024-09-10T21:41:45.897 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.895+0000 7f25d9a00640 1 -- 172.21.15.133:0/2391198794 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6ad0e4c6-e412-4bf0-906f-aa8fdfecb405", "id": 1} v 0) -- 0x7f25d40ce2d0 con 0x7f25d41f8840 2024-09-10T21:41:45.903 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.902+0000 7f25d0c00640 1 -- 172.21.15.133:0/2391198794 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6ad0e4c6-e412-4bf0-906f-aa8fdfecb405", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f25bc023af0 con 0x7f25d41f8840 2024-09-10T21:41:45.903 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-09-10T21:41:45.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.903+0000 7f25d9a00640 1 -- 172.21.15.133:0/2391198794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25d41f8840 msgr2=0x7f25d40ca250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:45.904 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.903+0000 7f25d9a00640 1 --2- 172.21.15.133:0/2391198794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25d41f8840 0x7f25d40ca250 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f25bc001440 tx=0x7f25bc001e40 comp rx=0 tx=0).stop 2024-09-10T21:41:45.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.903+0000 7f25d9a00640 1 -- 172.21.15.133:0/2391198794 shutdown_connections 2024-09-10T21:41:45.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.903+0000 7f25d9a00640 1 --2- 172.21.15.133:0/2391198794 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25d41f8840 0x7f25d40ca250 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:45.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.903+0000 7f25d9a00640 1 --2- 172.21.15.133:0/2391198794 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25d415c8e0 0x7f25d415cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:45.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.903+0000 7f25d9a00640 1 --2- 172.21.15.133:0/2391198794 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25d40b4bc0 0x7f25d4159d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:45.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.903+0000 7f25d9a00640 1 -- 172.21.15.133:0/2391198794 >> 172.21.15.133:0/2391198794 conn(0x7f25d409d0d0 msgr2=0x7f25d406d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:45.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.903+0000 7f25d9a00640 1 -- 172.21.15.133:0/2391198794 shutdown_connections 2024-09-10T21:41:45.905 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:45.903+0000 7f25d9a00640 1 -- 172.21.15.133:0/2391198794 wait complete. 2024-09-10T21:41:45.918 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 86e61267-ebb6-4e9f-b436-596efaa9eeb6 2 2024-09-10T21:41:46.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.070+0000 7f9f1ce00640 1 -- 172.21.15.133:0/1717517609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f180d22b0 msgr2=0x7f9f180d26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:46.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.070+0000 7f9f1ce00640 1 --2- 172.21.15.133:0/1717517609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f180d22b0 0x7f9f180d26f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009830 tx=0x7f9f0002a510 comp rx=0 tx=0).stop 2024-09-10T21:41:46.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.070+0000 7f9f1ce00640 1 -- 172.21.15.133:0/1717517609 shutdown_connections 2024-09-10T21:41:46.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.070+0000 7f9f1ce00640 1 --2- 172.21.15.133:0/1717517609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f180d22b0 0x7f9f180d26f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.070+0000 7f9f1ce00640 1 --2- 172.21.15.133:0/1717517609 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f1815df70 0x7f9f180b6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.070+0000 7f9f1ce00640 1 -- 172.21.15.133:0/1717517609 >> 172.21.15.133:0/1717517609 conn(0x7f9f180d3e80 msgr2=0x7f9f180d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:46.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.070+0000 7f9f1ce00640 1 -- 172.21.15.133:0/1717517609 shutdown_connections 2024-09-10T21:41:46.072 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.070+0000 7f9f1ce00640 1 -- 172.21.15.133:0/1717517609 wait complete. 2024-09-10T21:41:46.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.072+0000 7f9f1ce00640 1 Processor -- start 2024-09-10T21:41:46.073 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.072+0000 7f9f1ce00640 1 -- start start 2024-09-10T21:41:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.073+0000 7f9f1ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f180b0a10 0x7f9f180c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.073+0000 7f9f1ce00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f180d22b0 0x7f9f181f89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.073+0000 7f9f1ce00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f1815df70 0x7f9f1814ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.073+0000 7f9f1ce00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9f180b2960 con 0x7f9f180b0a10 2024-09-10T21:41:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.073+0000 7f9f1ce00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9f180b27e0 con 0x7f9f1815df70 2024-09-10T21:41:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.073+0000 7f9f1ce00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f9f180b2ae0 con 0x7f9f180d22b0 2024-09-10T21:41:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.073+0000 7f9f17400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f180b0a10 0x7f9f180c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.073+0000 7f9f0fe00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f180d22b0 0x7f9f181f89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:46.076 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.073+0000 7f9f17e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f1815df70 0x7f9f1814ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.073+0000 7f9f17400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f180b0a10 0x7f9f180c7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.133:44770/0 (socket says 172.21.15.133:44770) 2024-09-10T21:41:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.073+0000 7f9f17400640 1 -- 172.21.15.133:0/2358800809 learned_addr learned my addr 172.21.15.133:0/2358800809 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-10T21:41:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.073+0000 7f9f0fe00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f180d22b0 0x7f9f181f89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:55208/0 (socket says 172.21.15.133:55208) 2024-09-10T21:41:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.073+0000 7f9f0fe00640 1 -- 172.21.15.133:0/2358800809 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f180d22b0 msgr2=0x7f9f181f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:41:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.073+0000 7f9f0fe00640 1 -- 172.21.15.133:0/2358800809 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f180d22b0 msgr2=0x7f9f181f89f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:41:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.074+0000 7f9f0fe00640 1 --2- 172.21.15.133:0/2358800809 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f180d22b0 0x7f9f181f89f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:41:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.074+0000 7f9f0fe00640 1 --2- 172.21.15.133:0/2358800809 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f180d22b0 0x7f9f181f89f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:41:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.074+0000 7f9f17400640 1 -- 172.21.15.133:0/2358800809 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f180d22b0 msgr2=0x7f9f181f89f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:41:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.074+0000 7f9f17400640 1 --2- 172.21.15.133:0/2358800809 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f180d22b0 0x7f9f181f89f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.074+0000 7f9f17400640 1 -- 172.21.15.133:0/2358800809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f1815df70 msgr2=0x7f9f1814ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.074+0000 7f9f17400640 1 --2- 172.21.15.133:0/2358800809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f1815df70 0x7f9f1814ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.077 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.074+0000 7f9f17400640 1 -- 172.21.15.133:0/2358800809 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f180cc7b0 con 0x7f9f180b0a10 2024-09-10T21:41:46.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.074+0000 7f9f17400640 1 --2- 172.21.15.133:0/2358800809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f180b0a10 0x7f9f180c7b70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800d980 tx=0x7f9f0800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:46.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.074+0000 7f9f0f400640 1 -- 172.21.15.133:0/2358800809 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08017070 con 0x7f9f180b0a10 2024-09-10T21:41:46.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.074+0000 7f9f0f400640 1 -- 172.21.15.133:0/2358800809 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f08009d20 con 0x7f9f180b0a10 2024-09-10T21:41:46.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.074+0000 7f9f1ce00640 1 -- 172.21.15.133:0/2358800809 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f180ca870 con 0x7f9f180b0a10 2024-09-10T21:41:46.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.074+0000 7f9f0f400640 1 -- 172.21.15.133:0/2358800809 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f08004280 con 0x7f9f180b0a10 2024-09-10T21:41:46.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.074+0000 7f9f1ce00640 1 -- 172.21.15.133:0/2358800809 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f180ccd30 con 0x7f9f180b0a10 2024-09-10T21:41:46.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.074+0000 7f9f0f400640 1 -- 172.21.15.133:0/2358800809 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f9f080035f0 con 0x7f9f180b0a10 2024-09-10T21:41:46.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.075+0000 7f9f0f400640 1 -- 172.21.15.133:0/2358800809 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f9f08004420 con 0x7f9f180b0a10 2024-09-10T21:41:46.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.075+0000 7f9f1ce00640 1 -- 172.21.15.133:0/2358800809 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f180d26f0 con 0x7f9f180b0a10 2024-09-10T21:41:46.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.078+0000 7f9f0f400640 1 -- 172.21.15.133:0/2358800809 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f9f08003910 con 0x7f9f180b0a10 2024-09-10T21:41:46.153 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.152+0000 7f9f1ce00640 1 -- 172.21.15.133:0/2358800809 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd new", "uuid": "86e61267-ebb6-4e9f-b436-596efaa9eeb6", "id": 2} v 0) -- 0x7f9f181eac30 con 0x7f9f180b0a10 2024-09-10T21:41:46.154 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.154+0000 7f9f0f400640 1 -- 172.21.15.133:0/2358800809 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "86e61267-ebb6-4e9f-b436-596efaa9eeb6", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f9f08003bd0 con 0x7f9f180b0a10 2024-09-10T21:41:46.155 INFO:teuthology.orchestra.run.smithi133.stdout:2 2024-09-10T21:41:46.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.155+0000 7f9f1ce00640 1 -- 172.21.15.133:0/2358800809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f180b0a10 msgr2=0x7f9f180c7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:46.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.155+0000 7f9f1ce00640 1 --2- 172.21.15.133:0/2358800809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f180b0a10 0x7f9f180c7b70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800d980 tx=0x7f9f0800de40 comp rx=0 tx=0).stop 2024-09-10T21:41:46.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.155+0000 7f9f1ce00640 1 -- 172.21.15.133:0/2358800809 shutdown_connections 2024-09-10T21:41:46.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.155+0000 7f9f1ce00640 1 --2- 172.21.15.133:0/2358800809 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9f1815df70 0x7f9f1814ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.155+0000 7f9f1ce00640 1 --2- 172.21.15.133:0/2358800809 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9f180d22b0 0x7f9f181f89f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.155+0000 7f9f1ce00640 1 --2- 172.21.15.133:0/2358800809 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9f180b0a10 0x7f9f180c7b70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.155+0000 7f9f1ce00640 1 -- 172.21.15.133:0/2358800809 >> 172.21.15.133:0/2358800809 conn(0x7f9f180d3e80 msgr2=0x7f9f180b78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:46.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.155+0000 7f9f1ce00640 1 -- 172.21.15.133:0/2358800809 shutdown_connections 2024-09-10T21:41:46.156 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.155+0000 7f9f1ce00640 1 -- 172.21.15.133:0/2358800809 wait complete. 2024-09-10T21:41:46.170 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 1b85f36c-239a-4e3e-9bd8-a5b36382dcb9 3 2024-09-10T21:41:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.322+0000 7fa94e000640 1 -- 172.21.15.133:0/1052724851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480d22b0 msgr2=0x7fa9480d26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.322+0000 7fa94e000640 1 --2- 172.21.15.133:0/1052724851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480d22b0 0x7fa9480d26f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa9480d2db0 tx=0x7fa93802a5c0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.322+0000 7fa94e000640 1 -- 172.21.15.133:0/1052724851 shutdown_connections 2024-09-10T21:41:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.322+0000 7fa94e000640 1 --2- 172.21.15.133:0/1052724851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480d22b0 0x7fa9480d26f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.322+0000 7fa94e000640 1 -- 172.21.15.133:0/1052724851 >> 172.21.15.133:0/1052724851 conn(0x7fa9480d3e80 msgr2=0x7fa9480d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.323+0000 7fa94e000640 1 -- 172.21.15.133:0/1052724851 shutdown_connections 2024-09-10T21:41:46.324 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.323+0000 7fa94e000640 1 -- 172.21.15.133:0/1052724851 wait complete. 2024-09-10T21:41:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.324+0000 7fa94e000640 1 Processor -- start 2024-09-10T21:41:46.325 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.324+0000 7fa94e000640 1 -- start start 2024-09-10T21:41:46.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.325+0000 7fa94e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480b0a10 0x7fa948154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:46.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.325+0000 7fa94e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9480d22b0 0x7fa948154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:46.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.325+0000 7fa94e000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa94815df70 0x7fa9480c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:46.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.325+0000 7fa94e000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa9480b2b70 con 0x7fa9480b0a10 2024-09-10T21:41:46.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.325+0000 7fa94e000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa9480b29f0 con 0x7fa9480d22b0 2024-09-10T21:41:46.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.325+0000 7fa94e000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa9480b2cf0 con 0x7fa94815df70 2024-09-10T21:41:46.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.325+0000 7fa947e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9480d22b0 0x7fa948154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:46.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.325+0000 7fa94d600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa94815df70 0x7fa9480c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:46.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.325+0000 7fa947e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9480d22b0 0x7fa948154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36820/0 (socket says 172.21.15.133:36820) 2024-09-10T21:41:46.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.326+0000 7fa947e00640 1 -- 172.21.15.133:0/1956174653 learned_addr learned my addr 172.21.15.133:0/1956174653 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-10T21:41:46.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.326+0000 7fa94cc00640 1 --2- 172.21.15.133:0/1956174653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480b0a10 0x7fa948154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:46.328 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.326+0000 7fa94d600640 1 -- 172.21.15.133:0/1956174653 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa94815df70 msgr2=0x7fa9480c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T21:41:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.326+0000 7fa94d600640 1 -- 172.21.15.133:0/1956174653 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa94815df70 msgr2=0x7fa9480c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:41:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.326+0000 7fa94d600640 1 --2- 172.21.15.133:0/1956174653 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa94815df70 0x7fa9480c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:41:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.326+0000 7fa94d600640 1 --2- 172.21.15.133:0/1956174653 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa94815df70 0x7fa9480c9310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:41:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.326+0000 7fa94cc00640 1 -- 172.21.15.133:0/1956174653 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa94815df70 msgr2=0x7fa9480c9310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:41:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.326+0000 7fa94cc00640 1 --2- 172.21.15.133:0/1956174653 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa94815df70 0x7fa9480c9310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.326+0000 7fa94cc00640 1 -- 172.21.15.133:0/1956174653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9480d22b0 msgr2=0x7fa948154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.326+0000 7fa94cc00640 1 --2- 172.21.15.133:0/1956174653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9480d22b0 0x7fa948154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.326+0000 7fa94cc00640 1 -- 172.21.15.133:0/1956174653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9480c7b30 con 0x7fa9480b0a10 2024-09-10T21:41:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.326+0000 7fa947e00640 1 --2- 172.21.15.133:0/1956174653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9480d22b0 0x7fa948154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:41:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.326+0000 7fa94cc00640 1 --2- 172.21.15.133:0/1956174653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480b0a10 0x7fa948154e70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00c390 tx=0x7fa93c00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.326+0000 7fa944c00640 1 -- 172.21.15.133:0/1956174653 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c011790 con 0x7fa9480b0a10 2024-09-10T21:41:46.329 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.327+0000 7fa944c00640 1 -- 172.21.15.133:0/1956174653 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa93c017070 con 0x7fa9480b0a10 2024-09-10T21:41:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.327+0000 7fa94e000640 1 -- 172.21.15.133:0/1956174653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9480c7e70 con 0x7fa9480b0a10 2024-09-10T21:41:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.327+0000 7fa944c00640 1 -- 172.21.15.133:0/1956174653 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa93c011cc0 con 0x7fa9480b0a10 2024-09-10T21:41:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.327+0000 7fa94e000640 1 -- 172.21.15.133:0/1956174653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9480c86b0 con 0x7fa9480b0a10 2024-09-10T21:41:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.327+0000 7fa944c00640 1 -- 172.21.15.133:0/1956174653 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fa93c01c460 con 0x7fa9480b0a10 2024-09-10T21:41:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.327+0000 7fa944c00640 1 -- 172.21.15.133:0/1956174653 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fa93c01cc10 con 0x7fa9480b0a10 2024-09-10T21:41:46.330 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.328+0000 7fa94e000640 1 -- 172.21.15.133:0/1956174653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9480b70d0 con 0x7fa9480b0a10 2024-09-10T21:41:46.331 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.330+0000 7fa944c00640 1 -- 172.21.15.133:0/1956174653 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fa9480c7e70 con 0x7fa9480b0a10 2024-09-10T21:41:46.404 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.403+0000 7fa94e000640 1 -- 172.21.15.133:0/1956174653 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1b85f36c-239a-4e3e-9bd8-a5b36382dcb9", "id": 3} v 0) -- 0x7fa9480b89a0 con 0x7fa9480b0a10 2024-09-10T21:41:46.406 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.405+0000 7fa944c00640 1 -- 172.21.15.133:0/1956174653 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1b85f36c-239a-4e3e-9bd8-a5b36382dcb9", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fa93c01c780 con 0x7fa9480b0a10 2024-09-10T21:41:46.407 INFO:teuthology.orchestra.run.smithi133.stdout:3 2024-09-10T21:41:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.406+0000 7fa94e000640 1 -- 172.21.15.133:0/1956174653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480b0a10 msgr2=0x7fa948154e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.406+0000 7fa94e000640 1 --2- 172.21.15.133:0/1956174653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480b0a10 0x7fa948154e70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa93c00c390 tx=0x7fa93c00c790 comp rx=0 tx=0).stop 2024-09-10T21:41:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.406+0000 7fa94e000640 1 -- 172.21.15.133:0/1956174653 shutdown_connections 2024-09-10T21:41:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.406+0000 7fa94e000640 1 --2- 172.21.15.133:0/1956174653 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9480d22b0 0x7fa948154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.406+0000 7fa94e000640 1 --2- 172.21.15.133:0/1956174653 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa94815df70 0x7fa9480c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.406+0000 7fa94e000640 1 --2- 172.21.15.133:0/1956174653 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9480b0a10 0x7fa948154e70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.406+0000 7fa94e000640 1 -- 172.21.15.133:0/1956174653 >> 172.21.15.133:0/1956174653 conn(0x7fa9480d3e80 msgr2=0x7fa94815d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.407+0000 7fa94e000640 1 -- 172.21.15.133:0/1956174653 shutdown_connections 2024-09-10T21:41:46.408 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.407+0000 7fa94e000640 1 -- 172.21.15.133:0/1956174653 wait complete. 2024-09-10T21:41:46.422 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 4c5f74b5-0324-4f6d-a41c-a5176ea886e4 4 2024-09-10T21:41:46.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.575+0000 7f833b600640 1 -- 172.21.15.133:0/2994179950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83340d10d0 msgr2=0x7f83340d4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:46.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.575+0000 7f833b600640 1 --2- 172.21.15.133:0/2994179950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83340d10d0 0x7f83340d4a30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f831c009820 tx=0x7f831c02a480 comp rx=0 tx=0).stop 2024-09-10T21:41:46.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.575+0000 7f833b600640 1 -- 172.21.15.133:0/2994179950 shutdown_connections 2024-09-10T21:41:46.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.575+0000 7f833b600640 1 --2- 172.21.15.133:0/2994179950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83340d10d0 0x7f83340d4a30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.575+0000 7f833b600640 1 -- 172.21.15.133:0/2994179950 >> 172.21.15.133:0/2994179950 conn(0x7f83340d3f10 msgr2=0x7f83340d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:46.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.576+0000 7f833b600640 1 -- 172.21.15.133:0/2994179950 shutdown_connections 2024-09-10T21:41:46.577 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.576+0000 7f833b600640 1 -- 172.21.15.133:0/2994179950 wait complete. 2024-09-10T21:41:46.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.577+0000 7f833b600640 1 Processor -- start 2024-09-10T21:41:46.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.577+0000 7f833b600640 1 -- start start 2024-09-10T21:41:46.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.578+0000 7f833b600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83340d10d0 0x7f83340ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:46.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.578+0000 7f833b600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83340d1f90 0x7f83341f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:46.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.578+0000 7f833b600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833415e120 0x7f83340cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:46.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.578+0000 7f833b600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f83340b78e0 con 0x7f83340d1f90 2024-09-10T21:41:46.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.578+0000 7f833b600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f83340b7760 con 0x7f833415e120 2024-09-10T21:41:46.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.578+0000 7f833b600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f83340d23e0 con 0x7f83340d10d0 2024-09-10T21:41:46.581 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.578+0000 7f8338c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83340d10d0 0x7f83340ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:46.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.578+0000 7f8339600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833415e120 0x7f83340cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:46.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.578+0000 7f8338c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83340d10d0 0x7f83340ca100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:55236/0 (socket says 172.21.15.133:55236) 2024-09-10T21:41:46.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.578+0000 7f8339600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833415e120 0x7f83340cd020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36836/0 (socket says 172.21.15.133:36836) 2024-09-10T21:41:46.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.578+0000 7f8338c00640 1 -- 172.21.15.133:0/3259880041 learned_addr learned my addr 172.21.15.133:0/3259880041 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-10T21:41:46.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.578+0000 7f832be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83340d1f90 0x7f83341f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:46.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.579+0000 7f8338c00640 1 -- 172.21.15.133:0/3259880041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83340d10d0 msgr2=0x7f83340ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:41:46.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.579+0000 7f8338c00640 1 -- 172.21.15.133:0/3259880041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83340d10d0 msgr2=0x7f83340ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:41:46.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.579+0000 7f8338c00640 1 --2- 172.21.15.133:0/3259880041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83340d10d0 0x7f83340ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:41:46.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.579+0000 7f8338c00640 1 --2- 172.21.15.133:0/3259880041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83340d10d0 0x7f83340ca100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:41:46.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.579+0000 7f832be00640 1 -- 172.21.15.133:0/3259880041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83340d10d0 msgr2=0x7f83340ca100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:41:46.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.579+0000 7f832be00640 1 --2- 172.21.15.133:0/3259880041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83340d10d0 0x7f83340ca100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.579+0000 7f832be00640 1 -- 172.21.15.133:0/3259880041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833415e120 msgr2=0x7f83340cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:46.582 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.579+0000 7f832be00640 1 --2- 172.21.15.133:0/3259880041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833415e120 0x7f83340cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.579+0000 7f832be00640 1 -- 172.21.15.133:0/3259880041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83340cb180 con 0x7f83340d1f90 2024-09-10T21:41:46.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.579+0000 7f8339600640 1 --2- 172.21.15.133:0/3259880041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833415e120 0x7f83340cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:41:46.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.579+0000 7f832be00640 1 --2- 172.21.15.133:0/3259880041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83340d1f90 0x7f83341f52e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f831c031000 tx=0x7f831c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:46.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.580+0000 7f8331600640 1 -- 172.21.15.133:0/3259880041 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c005c30 con 0x7f83340d1f90 2024-09-10T21:41:46.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.580+0000 7f8331600640 1 -- 172.21.15.133:0/3259880041 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f831c004360 con 0x7f83340d1f90 2024-09-10T21:41:46.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.580+0000 7f833b600640 1 -- 172.21.15.133:0/3259880041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83340cb460 con 0x7f83340d1f90 2024-09-10T21:41:46.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.580+0000 7f8331600640 1 -- 172.21.15.133:0/3259880041 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c004a00 con 0x7f83340d1f90 2024-09-10T21:41:46.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.580+0000 7f833b600640 1 -- 172.21.15.133:0/3259880041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83340cba20 con 0x7f83340d1f90 2024-09-10T21:41:46.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.580+0000 7f8331600640 1 -- 172.21.15.133:0/3259880041 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f831c04c030 con 0x7f83340d1f90 2024-09-10T21:41:46.583 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.580+0000 7f8331600640 1 -- 172.21.15.133:0/3259880041 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f831c0035f0 con 0x7f83340d1f90 2024-09-10T21:41:46.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.581+0000 7f833b600640 1 -- 172.21.15.133:0/3259880041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8334074620 con 0x7f83340d1f90 2024-09-10T21:41:46.584 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.583+0000 7f8331600640 1 -- 172.21.15.133:0/3259880041 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f8334074620 con 0x7f83340d1f90 2024-09-10T21:41:46.658 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.657+0000 7f833b600640 1 -- 172.21.15.133:0/3259880041 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd new", "uuid": "4c5f74b5-0324-4f6d-a41c-a5176ea886e4", "id": 4} v 0) -- 0x7f83340d28b0 con 0x7f83340d1f90 2024-09-10T21:41:46.660 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.659+0000 7f8331600640 1 -- 172.21.15.133:0/3259880041 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4c5f74b5-0324-4f6d-a41c-a5176ea886e4", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f831c040030 con 0x7f83340d1f90 2024-09-10T21:41:46.660 INFO:teuthology.orchestra.run.smithi133.stdout:4 2024-09-10T21:41:46.661 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.660+0000 7f833b600640 1 -- 172.21.15.133:0/3259880041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83340d1f90 msgr2=0x7f83341f52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:46.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.660+0000 7f833b600640 1 --2- 172.21.15.133:0/3259880041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83340d1f90 0x7f83341f52e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f831c031000 tx=0x7f831c0052f0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.660+0000 7f833b600640 1 -- 172.21.15.133:0/3259880041 shutdown_connections 2024-09-10T21:41:46.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.660+0000 7f833b600640 1 --2- 172.21.15.133:0/3259880041 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f833415e120 0x7f83340cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.660+0000 7f833b600640 1 --2- 172.21.15.133:0/3259880041 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f83340d1f90 0x7f83341f52e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.660+0000 7f833b600640 1 --2- 172.21.15.133:0/3259880041 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f83340d10d0 0x7f83340ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.660+0000 7f833b600640 1 -- 172.21.15.133:0/3259880041 >> 172.21.15.133:0/3259880041 conn(0x7f83340d3f10 msgr2=0x7f83340b8280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:46.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.660+0000 7f833b600640 1 -- 172.21.15.133:0/3259880041 shutdown_connections 2024-09-10T21:41:46.662 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.660+0000 7f833b600640 1 -- 172.21.15.133:0/3259880041 wait complete. 2024-09-10T21:41:46.677 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 6b561866-3e02-4726-a0fb-a1dd3cdae62e 5 2024-09-10T21:41:46.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.832+0000 7ff19f200640 1 -- 172.21.15.133:0/795523194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1a00b1120 msgr2=0x7ff1a00d26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:46.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.832+0000 7ff19f200640 1 --2- 172.21.15.133:0/795523194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1a00b1120 0x7ff1a00d26f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff1a00d2db0 tx=0x7ff19802a920 comp rx=0 tx=0).stop 2024-09-10T21:41:46.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.832+0000 7ff19f200640 1 -- 172.21.15.133:0/795523194 shutdown_connections 2024-09-10T21:41:46.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.832+0000 7ff19f200640 1 --2- 172.21.15.133:0/795523194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1a00b1120 0x7ff1a00d26f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.832+0000 7ff19f200640 1 -- 172.21.15.133:0/795523194 >> 172.21.15.133:0/795523194 conn(0x7ff1a00d3e80 msgr2=0x7ff1a00d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:46.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.832+0000 7ff19f200640 1 -- 172.21.15.133:0/795523194 shutdown_connections 2024-09-10T21:41:46.834 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.832+0000 7ff19f200640 1 -- 172.21.15.133:0/795523194 wait complete. 2024-09-10T21:41:46.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.834+0000 7ff19f200640 1 Processor -- start 2024-09-10T21:41:46.835 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.834+0000 7ff19f200640 1 -- start start 2024-09-10T21:41:46.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.835+0000 7ff19f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1a00b1120 0x7ff1a00caec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:46.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.835+0000 7ff19f200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1a00d22b0 0x7ff1a015adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:46.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.835+0000 7ff19f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1a015df70 0x7ff1a006d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:46.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.835+0000 7ff19f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a00b2b70 con 0x7ff1a015df70 2024-09-10T21:41:46.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.835+0000 7ff19f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a00b29f0 con 0x7ff1a00b1120 2024-09-10T21:41:46.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.835+0000 7ff19f200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ff1a00b2cf0 con 0x7ff1a00d22b0 2024-09-10T21:41:46.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.835+0000 7ff19de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1a00b1120 0x7ff1a00caec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:46.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.835+0000 7ff19d400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1a00d22b0 0x7ff1a015adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:46.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.835+0000 7ff19e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1a015df70 0x7ff1a006d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:46.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.835+0000 7ff19d400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1a00d22b0 0x7ff1a015adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.133:55250/0 (socket says 172.21.15.133:55250) 2024-09-10T21:41:46.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.835+0000 7ff19de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1a00b1120 0x7ff1a00caec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36844/0 (socket says 172.21.15.133:36844) 2024-09-10T21:41:46.838 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.835+0000 7ff19d400640 1 -- 172.21.15.133:0/2156789516 learned_addr learned my addr 172.21.15.133:0/2156789516 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-10T21:41:46.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.835+0000 7ff19d400640 1 -- 172.21.15.133:0/2156789516 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1a00d22b0 msgr2=0x7ff1a015adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:41:46.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.835+0000 7ff19d400640 1 -- 172.21.15.133:0/2156789516 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1a00d22b0 msgr2=0x7ff1a015adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:41:46.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.835+0000 7ff19d400640 1 --2- 172.21.15.133:0/2156789516 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1a00d22b0 0x7ff1a015adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:41:46.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.835+0000 7ff19d400640 1 --2- 172.21.15.133:0/2156789516 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1a00d22b0 0x7ff1a015adb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:41:46.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.836+0000 7ff19e800640 1 -- 172.21.15.133:0/2156789516 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1a00d22b0 msgr2=0x7ff1a015adb0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:41:46.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.836+0000 7ff19e800640 1 --2- 172.21.15.133:0/2156789516 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1a00d22b0 0x7ff1a015adb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.836+0000 7ff19e800640 1 -- 172.21.15.133:0/2156789516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1a00b1120 msgr2=0x7ff1a00caec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:46.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.836+0000 7ff19e800640 1 --2- 172.21.15.133:0/2156789516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1a00b1120 0x7ff1a00caec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.836+0000 7ff19e800640 1 -- 172.21.15.133:0/2156789516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a00cb4c0 con 0x7ff1a015df70 2024-09-10T21:41:46.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.836+0000 7ff19de00640 1 --2- 172.21.15.133:0/2156789516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1a00b1120 0x7ff1a00caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:41:46.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.836+0000 7ff19e800640 1 --2- 172.21.15.133:0/2156789516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1a015df70 0x7ff1a006d410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff198009000 tx=0x7ff198003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:46.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.836+0000 7ff196000640 1 -- 172.21.15.133:0/2156789516 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff198004240 con 0x7ff1a015df70 2024-09-10T21:41:46.839 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.836+0000 7ff196000640 1 -- 172.21.15.133:0/2156789516 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1980043e0 con 0x7ff1a015df70 2024-09-10T21:41:46.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.836+0000 7ff196000640 1 -- 172.21.15.133:0/2156789516 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff198004830 con 0x7ff1a015df70 2024-09-10T21:41:46.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.836+0000 7ff19f200640 1 -- 172.21.15.133:0/2156789516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a00cb7a0 con 0x7ff1a015df70 2024-09-10T21:41:46.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.836+0000 7ff19f200640 1 -- 172.21.15.133:0/2156789516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a00cbd60 con 0x7ff1a015df70 2024-09-10T21:41:46.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.837+0000 7ff196000640 1 -- 172.21.15.133:0/2156789516 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7ff19803a450 con 0x7ff1a015df70 2024-09-10T21:41:46.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.837+0000 7ff196000640 1 -- 172.21.15.133:0/2156789516 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7ff19803acf0 con 0x7ff1a015df70 2024-09-10T21:41:46.840 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.837+0000 7ff19f200640 1 -- 172.21.15.133:0/2156789516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1a00b31a0 con 0x7ff1a015df70 2024-09-10T21:41:46.841 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.840+0000 7ff196000640 1 -- 172.21.15.133:0/2156789516 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7ff19802d070 con 0x7ff1a015df70 2024-09-10T21:41:46.915 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.914+0000 7ff19f200640 1 -- 172.21.15.133:0/2156789516 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6b561866-3e02-4726-a0fb-a1dd3cdae62e", "id": 5} v 0) -- 0x7ff1a00b7ed0 con 0x7ff1a015df70 2024-09-10T21:41:46.917 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.916+0000 7ff196000640 1 -- 172.21.15.133:0/2156789516 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6b561866-3e02-4726-a0fb-a1dd3cdae62e", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7ff19803a770 con 0x7ff1a015df70 2024-09-10T21:41:46.917 INFO:teuthology.orchestra.run.smithi133.stdout:5 2024-09-10T21:41:46.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.917+0000 7ff19f200640 1 -- 172.21.15.133:0/2156789516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1a015df70 msgr2=0x7ff1a006d410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:46.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.917+0000 7ff19f200640 1 --2- 172.21.15.133:0/2156789516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1a015df70 0x7ff1a006d410 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff198009000 tx=0x7ff198003c20 comp rx=0 tx=0).stop 2024-09-10T21:41:46.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.917+0000 7ff19f200640 1 -- 172.21.15.133:0/2156789516 shutdown_connections 2024-09-10T21:41:46.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.917+0000 7ff19f200640 1 --2- 172.21.15.133:0/2156789516 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff1a015df70 0x7ff1a006d410 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.917+0000 7ff19f200640 1 --2- 172.21.15.133:0/2156789516 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff1a00d22b0 0x7ff1a015adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.917+0000 7ff19f200640 1 --2- 172.21.15.133:0/2156789516 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff1a00b1120 0x7ff1a00caec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.917+0000 7ff19f200640 1 -- 172.21.15.133:0/2156789516 >> 172.21.15.133:0/2156789516 conn(0x7ff1a00d3e80 msgr2=0x7ff1a006ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:46.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.917+0000 7ff19f200640 1 -- 172.21.15.133:0/2156789516 shutdown_connections 2024-09-10T21:41:46.919 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:46.918+0000 7ff19f200640 1 -- 172.21.15.133:0/2156789516 wait complete. 2024-09-10T21:41:46.933 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new e0b15a28-b8bc-446a-8b02-3e555e9aa2c7 6 2024-09-10T21:41:47.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.084+0000 7f33ab200640 1 -- 172.21.15.133:0/2020650470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33ac0d28e0 msgr2=0x7f33ac0b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:47.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.084+0000 7f33ab200640 1 --2- 172.21.15.133:0/2020650470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33ac0d28e0 0x7f33ac0b3eb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f33a4005fd0 tx=0x7f33a402a810 comp rx=0 tx=0).stop 2024-09-10T21:41:47.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.084+0000 7f33ab200640 1 -- 172.21.15.133:0/2020650470 shutdown_connections 2024-09-10T21:41:47.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.084+0000 7f33ab200640 1 --2- 172.21.15.133:0/2020650470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33ac0d28e0 0x7f33ac0b3eb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.084+0000 7f33ab200640 1 --2- 172.21.15.133:0/2020650470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac0d0740 0x7f33ac0d23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.084+0000 7f33ab200640 1 -- 172.21.15.133:0/2020650470 >> 172.21.15.133:0/2020650470 conn(0x7f33ac09d0d0 msgr2=0x7f33ac0d3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:47.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.084+0000 7f33ab200640 1 -- 172.21.15.133:0/2020650470 shutdown_connections 2024-09-10T21:41:47.086 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.084+0000 7f33ab200640 1 -- 172.21.15.133:0/2020650470 wait complete. 2024-09-10T21:41:47.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.086+0000 7f33ab200640 1 Processor -- start 2024-09-10T21:41:47.087 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.086+0000 7f33ab200640 1 -- start start 2024-09-10T21:41:47.089 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.087+0000 7f33ab200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac0d0740 0x7f33ac157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:47.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.087+0000 7f33ab200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33ac0d28e0 0x7f33ac0b82f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:47.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.087+0000 7f33ab200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33ac15e1f0 0x7f33ac0b2230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:47.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.087+0000 7f33ab200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac0b4970 con 0x7f33ac15e1f0 2024-09-10T21:41:47.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.087+0000 7f33ab200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f33ac0b47f0 con 0x7f33ac0d28e0 2024-09-10T21:41:47.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.087+0000 7f33ab200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f33ac0b4af0 con 0x7f33ac0d0740 2024-09-10T21:41:47.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.087+0000 7f33a3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33ac0d28e0 0x7f33ac0b82f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:47.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.087+0000 7f33a9e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac0d0740 0x7f33ac157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:47.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.087+0000 7f33a3e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33ac0d28e0 0x7f33ac0b82f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36868/0 (socket says 172.21.15.133:36868) 2024-09-10T21:41:47.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.087+0000 7f33a3e00640 1 -- 172.21.15.133:0/2893262279 learned_addr learned my addr 172.21.15.133:0/2893262279 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-10T21:41:47.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.087+0000 7f33aa800640 1 --2- 172.21.15.133:0/2893262279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33ac15e1f0 0x7f33ac0b2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:47.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.088+0000 7f33a9e00640 1 -- 172.21.15.133:0/2893262279 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac0d0740 msgr2=0x7f33ac157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:41:47.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.088+0000 7f33a9e00640 1 -- 172.21.15.133:0/2893262279 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac0d0740 msgr2=0x7f33ac157640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:41:47.090 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.088+0000 7f33a9e00640 1 --2- 172.21.15.133:0/2893262279 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac0d0740 0x7f33ac157640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:41:47.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.088+0000 7f33a3e00640 1 -- 172.21.15.133:0/2893262279 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac0d0740 msgr2=0x7f33ac157640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:41:47.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.088+0000 7f33a9e00640 1 --2- 172.21.15.133:0/2893262279 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac0d0740 0x7f33ac157640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:41:47.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.088+0000 7f33a3e00640 1 --2- 172.21.15.133:0/2893262279 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac0d0740 0x7f33ac157640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.088+0000 7f33a3e00640 1 -- 172.21.15.133:0/2893262279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33ac15e1f0 msgr2=0x7f33ac0b2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:47.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.088+0000 7f33a3e00640 1 --2- 172.21.15.133:0/2893262279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33ac15e1f0 0x7f33ac0b2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.088+0000 7f33a3e00640 1 -- 172.21.15.133:0/2893262279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33ac0b8e60 con 0x7f33ac0d28e0 2024-09-10T21:41:47.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.088+0000 7f33a3e00640 1 --2- 172.21.15.133:0/2893262279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33ac0d28e0 0x7f33ac0b82f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f339400d720 tx=0x7f339400dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:47.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.088+0000 7f33a2000640 1 -- 172.21.15.133:0/2893262279 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3394017070 con 0x7f33ac0d28e0 2024-09-10T21:41:47.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.088+0000 7f33a2000640 1 -- 172.21.15.133:0/2893262279 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3394009d20 con 0x7f33ac0d28e0 2024-09-10T21:41:47.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.088+0000 7f33a2000640 1 -- 172.21.15.133:0/2893262279 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33940106c0 con 0x7f33ac0d28e0 2024-09-10T21:41:47.091 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.088+0000 7f33ab200640 1 -- 172.21.15.133:0/2893262279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33ac157b80 con 0x7f33ac0d28e0 2024-09-10T21:41:47.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.088+0000 7f33ab200640 1 -- 172.21.15.133:0/2893262279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33ac1f8620 con 0x7f33ac0d28e0 2024-09-10T21:41:47.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.088+0000 7f33a2000640 1 -- 172.21.15.133:0/2893262279 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f3394010860 con 0x7f33ac0d28e0 2024-09-10T21:41:47.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.089+0000 7f33a2000640 1 -- 172.21.15.133:0/2893262279 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f339401cce0 con 0x7f33ac0d28e0 2024-09-10T21:41:47.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.089+0000 7f33ab200640 1 -- 172.21.15.133:0/2893262279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33ac0b57d0 con 0x7f33ac0d28e0 2024-09-10T21:41:47.092 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.091+0000 7f33a2000640 1 -- 172.21.15.133:0/2893262279 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f3394010b80 con 0x7f33ac0d28e0 2024-09-10T21:41:47.166 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.165+0000 7f33ab200640 1 -- 172.21.15.133:0/2893262279 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e0b15a28-b8bc-446a-8b02-3e555e9aa2c7", "id": 6} v 0) -- 0x7f33ac0b33f0 con 0x7f33ac0d28e0 2024-09-10T21:41:47.170 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.169+0000 7f33a2000640 1 -- 172.21.15.133:0/2893262279 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e0b15a28-b8bc-446a-8b02-3e555e9aa2c7", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f3394010d60 con 0x7f33ac0d28e0 2024-09-10T21:41:47.170 INFO:teuthology.orchestra.run.smithi133.stdout:6 2024-09-10T21:41:47.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.170+0000 7f33ab200640 1 -- 172.21.15.133:0/2893262279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33ac0d28e0 msgr2=0x7f33ac0b82f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:47.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.170+0000 7f33ab200640 1 --2- 172.21.15.133:0/2893262279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33ac0d28e0 0x7f33ac0b82f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f339400d720 tx=0x7f339400dbe0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.171+0000 7f33ab200640 1 -- 172.21.15.133:0/2893262279 shutdown_connections 2024-09-10T21:41:47.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.171+0000 7f33ab200640 1 --2- 172.21.15.133:0/2893262279 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f33ac0d28e0 0x7f33ac0b82f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.171+0000 7f33ab200640 1 --2- 172.21.15.133:0/2893262279 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f33ac15e1f0 0x7f33ac0b2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.171+0000 7f33ab200640 1 --2- 172.21.15.133:0/2893262279 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f33ac0d0740 0x7f33ac157640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.171+0000 7f33ab200640 1 -- 172.21.15.133:0/2893262279 >> 172.21.15.133:0/2893262279 conn(0x7f33ac09d0d0 msgr2=0x7f33ac0b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:47.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.171+0000 7f33ab200640 1 -- 172.21.15.133:0/2893262279 shutdown_connections 2024-09-10T21:41:47.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.171+0000 7f33ab200640 1 -- 172.21.15.133:0/2893262279 wait complete. 2024-09-10T21:41:47.186 DEBUG:teuthology.orchestra.run.smithi133:> sudo ceph --cluster ceph osd new 13799424-09a2-4a3a-adbb-99db0da9d130 7 2024-09-10T21:41:47.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.342+0000 7efcaea00640 1 -- 172.21.15.133:0/348450019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc90003ff0 msgr2=0x7efc90004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:47.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.342+0000 7efcaea00640 1 --2- 172.21.15.133:0/348450019 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc90003ff0 0x7efc90004800 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc9400bad0 tx=0x7efc90005830 comp rx=0 tx=0).stop 2024-09-10T21:41:47.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.342+0000 7efcae000640 1 -- 172.21.15.133:0/348450019 reap_dead start 2024-09-10T21:41:47.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.342+0000 7efcaea00640 1 -- 172.21.15.133:0/348450019 shutdown_connections 2024-09-10T21:41:47.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.342+0000 7efcaea00640 1 -- 172.21.15.133:0/348450019 >> 172.21.15.133:0/348450019 conn(0x7efca80d3e80 msgr2=0x7efca80d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:47.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.342+0000 7efcaea00640 1 -- 172.21.15.133:0/348450019 shutdown_connections 2024-09-10T21:41:47.344 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.342+0000 7efcaea00640 1 -- 172.21.15.133:0/348450019 wait complete. 2024-09-10T21:41:47.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.344+0000 7efcaea00640 1 Processor -- start 2024-09-10T21:41:47.345 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.344+0000 7efcaea00640 1 -- start start 2024-09-10T21:41:47.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcaea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc90003ff0 0x7efca815afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:47.348 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcaea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efca80b4420 0x7efca81606f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcaea00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efca80c7c10 0x7efca806e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcaea00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efca80d2a80 con 0x7efc90003ff0 2024-09-10T21:41:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcaea00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efca80d2900 con 0x7efca80b4420 2024-09-10T21:41:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcaea00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7efca80b27e0 con 0x7efca80c7c10 2024-09-10T21:41:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcacc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efca80b4420 0x7efca81606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcae000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efca80c7c10 0x7efca806e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcad600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc90003ff0 0x7efca815afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcacc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efca80b4420 0x7efca81606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.133:36878/0 (socket says 172.21.15.133:36878) 2024-09-10T21:41:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcad600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc90003ff0 0x7efca815afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.133:44872/0 (socket says 172.21.15.133:44872) 2024-09-10T21:41:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcacc00640 1 -- 172.21.15.133:0/733856402 learned_addr learned my addr 172.21.15.133:0/733856402 (peer_addr_for_me v2:172.21.15.133:0/0) 2024-09-10T21:41:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcae000640 1 -- 172.21.15.133:0/733856402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efca80c7c10 msgr2=0x7efca806e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T21:41:47.349 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcae000640 1 -- 172.21.15.133:0/733856402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efca80c7c10 msgr2=0x7efca806e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:41:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcae000640 1 --2- 172.21.15.133:0/733856402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efca80c7c10 0x7efca806e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:41:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcae000640 1 --2- 172.21.15.133:0/733856402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efca80c7c10 0x7efca806e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:41:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcad600640 1 -- 172.21.15.133:0/733856402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efca80c7c10 msgr2=0x7efca806e5b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:41:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcad600640 1 --2- 172.21.15.133:0/733856402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efca80c7c10 0x7efca806e5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcad600640 1 -- 172.21.15.133:0/733856402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efca80b4420 msgr2=0x7efca81606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcad600640 1 --2- 172.21.15.133:0/733856402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efca80b4420 0x7efca81606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcad600640 1 -- 172.21.15.133:0/733856402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca80b49e0 con 0x7efc90003ff0 2024-09-10T21:41:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.345+0000 7efcacc00640 1 --2- 172.21.15.133:0/733856402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efca80b4420 0x7efca81606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:41:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.346+0000 7efcad600640 1 --2- 172.21.15.133:0/733856402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc90003ff0 0x7efca815afb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efc9c006630 tx=0x7efc9c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.346+0000 7efca5600640 1 -- 172.21.15.133:0/733856402 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c015070 con 0x7efc90003ff0 2024-09-10T21:41:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.346+0000 7efca5600640 1 -- 172.21.15.133:0/733856402 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc9c011900 con 0x7efc90003ff0 2024-09-10T21:41:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.346+0000 7efca5600640 1 -- 172.21.15.133:0/733856402 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9c011d50 con 0x7efc90003ff0 2024-09-10T21:41:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.346+0000 7efcaea00640 1 -- 172.21.15.133:0/733856402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca80b4cc0 con 0x7efc90003ff0 2024-09-10T21:41:47.350 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.346+0000 7efcaea00640 1 -- 172.21.15.133:0/733856402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca80b3110 con 0x7efc90003ff0 2024-09-10T21:41:47.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.346+0000 7efca5600640 1 -- 172.21.15.133:0/733856402 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7efc9c0103c0 con 0x7efc90003ff0 2024-09-10T21:41:47.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.346+0000 7efca5600640 1 -- 172.21.15.133:0/733856402 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7efc9c010d40 con 0x7efc90003ff0 2024-09-10T21:41:47.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.347+0000 7efcaea00640 1 -- 172.21.15.133:0/733856402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efca80b3310 con 0x7efc90003ff0 2024-09-10T21:41:47.351 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.350+0000 7efca5600640 1 -- 172.21.15.133:0/733856402 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7efca80b3310 con 0x7efc90003ff0 2024-09-10T21:41:47.424 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.423+0000 7efcaea00640 1 -- 172.21.15.133:0/733856402 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd new", "uuid": "13799424-09a2-4a3a-adbb-99db0da9d130", "id": 7} v 0) -- 0x7efca80b4f00 con 0x7efc90003ff0 2024-09-10T21:41:47.426 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.425+0000 7efca5600640 1 -- 172.21.15.133:0/733856402 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "13799424-09a2-4a3a-adbb-99db0da9d130", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7efc9c0107b0 con 0x7efc90003ff0 2024-09-10T21:41:47.426 INFO:teuthology.orchestra.run.smithi133.stdout:7 2024-09-10T21:41:47.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.426+0000 7efcaea00640 1 -- 172.21.15.133:0/733856402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc90003ff0 msgr2=0x7efca815afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:47.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.426+0000 7efcaea00640 1 --2- 172.21.15.133:0/733856402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc90003ff0 0x7efca815afb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7efc9c006630 tx=0x7efc9c0039c0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.426+0000 7efcaea00640 1 -- 172.21.15.133:0/733856402 shutdown_connections 2024-09-10T21:41:47.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.426+0000 7efcaea00640 1 --2- 172.21.15.133:0/733856402 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efca80b4420 0x7efca81606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.426+0000 7efcaea00640 1 --2- 172.21.15.133:0/733856402 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efca80c7c10 0x7efca806e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.427 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.426+0000 7efcaea00640 1 --2- 172.21.15.133:0/733856402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc90003ff0 0x7efca815afb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.426+0000 7efcaea00640 1 -- 172.21.15.133:0/733856402 >> 172.21.15.133:0/733856402 conn(0x7efca80d3e80 msgr2=0x7efca80b8200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:47.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.426+0000 7efcaea00640 1 -- 172.21.15.133:0/733856402 shutdown_connections 2024-09-10T21:41:47.428 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:41:47.426+0000 7efcaea00640 1 -- 172.21.15.133:0/733856402 wait complete. 2024-09-10T21:41:47.443 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-10T21:41:47.443 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-10T21:41:47.447 INFO:tasks.ceph.osd.0:Started 2024-09-10T21:41:47.447 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-10T21:41:47.447 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-10T21:41:47.451 INFO:tasks.ceph.osd.1:Started 2024-09-10T21:41:47.451 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-10T21:41:47.451 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-10T21:41:47.456 INFO:tasks.ceph.osd.2:Started 2024-09-10T21:41:47.456 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-10T21:41:47.457 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-10T21:41:47.461 INFO:tasks.ceph.osd.3:Started 2024-09-10T21:41:47.461 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-10T21:41:47.461 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-10T21:41:47.466 INFO:tasks.ceph.osd.4:Started 2024-09-10T21:41:47.466 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-10T21:41:47.466 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-10T21:41:47.471 INFO:tasks.ceph.osd.5:Started 2024-09-10T21:41:47.471 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-10T21:41:47.472 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-10T21:41:47.476 INFO:tasks.ceph.osd.6:Started 2024-09-10T21:41:47.477 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-10T21:41:47.477 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-10T21:41:47.482 INFO:tasks.ceph.osd.7:Started 2024-09-10T21:41:47.482 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:41:47.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.698+0000 7f803c200640 1 -- 172.21.15.7:0/2679451930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f803415aa30 msgr2=0x7f8034073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:47.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.698+0000 7f803c200640 1 --2- 172.21.15.7:0/2679451930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f803415aa30 0x7f8034073550 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f80340c3320 tx=0x7f802802a560 comp rx=0 tx=0).stop 2024-09-10T21:41:47.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.698+0000 7f803c200640 1 -- 172.21.15.7:0/2679451930 shutdown_connections 2024-09-10T21:41:47.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.698+0000 7f803c200640 1 --2- 172.21.15.7:0/2679451930 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f803415e650 0x7f803415eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.698+0000 7f803c200640 1 --2- 172.21.15.7:0/2679451930 >> v2:172.21.15.133:3300/0 conn(0x7f80340ce860 0x7f80340ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.698+0000 7f803c200640 1 --2- 172.21.15.7:0/2679451930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f803415aa30 0x7f8034073550 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.698+0000 7f803c200640 1 -- 172.21.15.7:0/2679451930 >> 172.21.15.7:0/2679451930 conn(0x7f80340b16d0 msgr2=0x7f80340b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:47.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.698+0000 7f803c200640 1 -- 172.21.15.7:0/2679451930 shutdown_connections 2024-09-10T21:41:47.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.698+0000 7f803c200640 1 -- 172.21.15.7:0/2679451930 wait complete. 2024-09-10T21:41:47.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.700+0000 7f803c200640 1 Processor -- start 2024-09-10T21:41:47.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.700+0000 7f803c200640 1 -- start start 2024-09-10T21:41:47.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.701+0000 7f803c200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80340ce860 0x7f80340c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:47.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.701+0000 7f803c200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f803415aa30 0x7f8034146bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:47.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.701+0000 7f803c200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f803415e650 0x7f8034155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:47.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.701+0000 7f803c200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f803406ee30 con 0x7f803415e650 2024-09-10T21:41:47.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.701+0000 7f803c200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f803406ecb0 con 0x7f803415aa30 2024-09-10T21:41:47.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.701+0000 7f803c200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f803415f020 con 0x7f80340ce860 2024-09-10T21:41:47.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.701+0000 7f8039800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80340ce860 0x7f80340c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:47.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.701+0000 7f8039800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80340ce860 0x7f80340c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:35752/0 (socket says 172.21.15.7:35752) 2024-09-10T21:41:47.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.701+0000 7f803a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f803415e650 0x7f8034155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:47.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.701+0000 7f8039800640 1 -- 172.21.15.7:0/3450872333 learned_addr learned my addr 172.21.15.7:0/3450872333 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:41:47.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.701+0000 7f8038e00640 1 --2- 172.21.15.7:0/3450872333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f803415aa30 0x7f8034146bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:47.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.701+0000 7f8039800640 1 -- 172.21.15.7:0/3450872333 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80340ce860 msgr2=0x7f80340c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:41:47.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.701+0000 7f8039800640 1 -- 172.21.15.7:0/3450872333 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80340ce860 msgr2=0x7f80340c9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:41:47.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.701+0000 7f8039800640 1 --2- 172.21.15.7:0/3450872333 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80340ce860 0x7f80340c9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:41:47.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.701+0000 7f8039800640 1 --2- 172.21.15.7:0/3450872333 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80340ce860 0x7f80340c9760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:41:47.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.702+0000 7f803a200640 1 -- 172.21.15.7:0/3450872333 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80340ce860 msgr2=0x7f80340c9760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:41:47.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.702+0000 7f803a200640 1 --2- 172.21.15.7:0/3450872333 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80340ce860 0x7f80340c9760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.702+0000 7f803a200640 1 -- 172.21.15.7:0/3450872333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f803415aa30 msgr2=0x7f8034146bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:47.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.702+0000 7f803a200640 1 --2- 172.21.15.7:0/3450872333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f803415aa30 0x7f8034146bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.702+0000 7f803a200640 1 -- 172.21.15.7:0/3450872333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8034150b80 con 0x7f803415e650 2024-09-10T21:41:47.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.702+0000 7f803a200640 1 --2- 172.21.15.7:0/3450872333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f803415e650 0x7f8034155ef0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f802c00e960 tx=0x7f802c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:47.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.702+0000 7f8031600640 1 -- 172.21.15.7:0/3450872333 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c004d20 con 0x7f803415e650 2024-09-10T21:41:47.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.702+0000 7f8031600640 1 -- 172.21.15.7:0/3450872333 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f802c00ca60 con 0x7f803415e650 2024-09-10T21:41:47.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.702+0000 7f8031600640 1 -- 172.21.15.7:0/3450872333 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f802c005020 con 0x7f803415e650 2024-09-10T21:41:47.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.702+0000 7f803c200640 1 -- 172.21.15.7:0/3450872333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8034150e60 con 0x7f803415e650 2024-09-10T21:41:47.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.702+0000 7f803c200640 1 -- 172.21.15.7:0/3450872333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80341470f0 con 0x7f803415e650 2024-09-10T21:41:47.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.703+0000 7f8031600640 1 -- 172.21.15.7:0/3450872333 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f802c00f040 con 0x7f803415e650 2024-09-10T21:41:47.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.703+0000 7f8031600640 1 -- 172.21.15.7:0/3450872333 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f802c011d90 con 0x7f803415e650 2024-09-10T21:41:47.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.704+0000 7f803c200640 1 -- 172.21.15.7:0/3450872333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8034073550 con 0x7f803415e650 2024-09-10T21:41:47.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.706+0000 7f8031600640 1 -- 172.21.15.7:0/3450872333 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f802c016030 con 0x7f803415e650 2024-09-10T21:41:47.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.790+0000 7f803c200640 1 -- 172.21.15.7:0/3450872333 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f803415ec40 con 0x7f803415e650 2024-09-10T21:41:47.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.790+0000 7f8031600640 1 -- 172.21.15.7:0/3450872333 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f802c011680 con 0x7f803415e650 2024-09-10T21:41:47.793 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:41:47.793 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":10,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:41:47.424461+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:41:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.791+0000 7f803c200640 1 -- 172.21.15.7:0/3450872333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f803415e650 msgr2=0x7f8034155ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.791+0000 7f803c200640 1 --2- 172.21.15.7:0/3450872333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f803415e650 0x7f8034155ef0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f802c00e960 tx=0x7f802c00ee20 comp rx=0 tx=0).stop 2024-09-10T21:41:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.792+0000 7f803c200640 1 -- 172.21.15.7:0/3450872333 shutdown_connections 2024-09-10T21:41:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.792+0000 7f803c200640 1 --2- 172.21.15.7:0/3450872333 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f803415aa30 0x7f8034146bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.792+0000 7f803c200640 1 --2- 172.21.15.7:0/3450872333 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f803415e650 0x7f8034155ef0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.792+0000 7f803c200640 1 --2- 172.21.15.7:0/3450872333 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80340ce860 0x7f80340c9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:47.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.792+0000 7f803c200640 1 -- 172.21.15.7:0/3450872333 >> 172.21.15.7:0/3450872333 conn(0x7f80340b16d0 msgr2=0x7f80340cf9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.792+0000 7f803c200640 1 -- 172.21.15.7:0/3450872333 shutdown_connections 2024-09-10T21:41:47.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:41:47.792+0000 7f803c200640 1 -- 172.21.15.7:0/3450872333 wait complete. 2024-09-10T21:41:47.808 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-10T21:41:47.809 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-10T21:41:48.614 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T21:41:48.758 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.756+0000 7f9c5d200640 1 -- 172.21.15.7:0/2299530759 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c580b55d0 msgr2=0x7f9c580cfc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:48.758 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.756+0000 7f9c5d200640 1 --2- 172.21.15.7:0/2299530759 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c580b55d0 0x7f9c580cfc40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009860 tx=0x7f9c4402a510 comp rx=0 tx=0).stop 2024-09-10T21:41:48.758 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.756+0000 7f9c5d200640 1 -- 172.21.15.7:0/2299530759 shutdown_connections 2024-09-10T21:41:48.758 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.756+0000 7f9c5d200640 1 --2- 172.21.15.7:0/2299530759 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c580b55d0 0x7f9c580cfc40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:48.759 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.756+0000 7f9c5d200640 1 -- 172.21.15.7:0/2299530759 >> 172.21.15.7:0/2299530759 conn(0x7f9c580d4020 msgr2=0x7f9c580d4430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:48.759 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.756+0000 7f9c5d200640 1 -- 172.21.15.7:0/2299530759 shutdown_connections 2024-09-10T21:41:48.759 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.756+0000 7f9c5d200640 1 -- 172.21.15.7:0/2299530759 wait complete. 2024-09-10T21:41:48.759 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.757+0000 7f9c5d200640 1 Processor -- start 2024-09-10T21:41:48.759 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.757+0000 7f9c5d200640 1 -- start start 2024-09-10T21:41:48.761 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.758+0000 7f9c5d200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c580b55d0 0x7f9c581f4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:48.761 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.758+0000 7f9c5d200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c580d1f20 0x7f9c581535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:48.761 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.758+0000 7f9c5d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c581618b0 0x7f9c580cc140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:48.761 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.758+0000 7f9c5d200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9c580b3d10 con 0x7f9c581618b0 2024-09-10T21:41:48.761 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.758+0000 7f9c5d200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9c580b3b90 con 0x7f9c580d1f20 2024-09-10T21:41:48.761 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.758+0000 7f9c5d200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f9c580b3e90 con 0x7f9c580b55d0 2024-09-10T21:41:48.761 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.759+0000 7f9c55400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c580d1f20 0x7f9c581535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:48.761 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.759+0000 7f9c56800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c581618b0 0x7f9c580cc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:48.761 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.759+0000 7f9c55e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c580b55d0 0x7f9c581f4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:48.761 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.759+0000 7f9c55400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c580d1f20 0x7f9c581535b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:34400/0 (socket says 172.21.15.7:34400) 2024-09-10T21:41:48.761 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.759+0000 7f9c55400640 1 -- 172.21.15.7:0/1064144903 learned_addr learned my addr 172.21.15.7:0/1064144903 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:41:48.762 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.759+0000 7f9c55e00640 1 -- 172.21.15.7:0/1064144903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c580d1f20 msgr2=0x7f9c581535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:48.763 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.759+0000 7f9c55e00640 1 --2- 172.21.15.7:0/1064144903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c580d1f20 0x7f9c581535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:48.763 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.759+0000 7f9c55e00640 1 -- 172.21.15.7:0/1064144903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c581618b0 msgr2=0x7f9c580cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:48.763 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.759+0000 7f9c55e00640 1 --2- 172.21.15.7:0/1064144903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c581618b0 0x7f9c580cc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:48.763 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.759+0000 7f9c55e00640 1 -- 172.21.15.7:0/1064144903 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c58153af0 con 0x7f9c580b55d0 2024-09-10T21:41:48.763 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.759+0000 7f9c55e00640 1 --2- 172.21.15.7:0/1064144903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c580b55d0 0x7f9c581f4ed0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800c3f0 tx=0x7f9c4800c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:48.763 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.760+0000 7f9c4e000640 1 -- 172.21.15.7:0/1064144903 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c48011790 con 0x7f9c580b55d0 2024-09-10T21:41:48.763 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.760+0000 7f9c4e000640 1 -- 172.21.15.7:0/1064144903 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c48017070 con 0x7f9c580b55d0 2024-09-10T21:41:48.763 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.760+0000 7f9c4e000640 1 -- 172.21.15.7:0/1064144903 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c48011ce0 con 0x7f9c580b55d0 2024-09-10T21:41:48.763 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.760+0000 7f9c5d200640 1 -- 172.21.15.7:0/1064144903 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5806ebd0 con 0x7f9c580b55d0 2024-09-10T21:41:48.763 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.760+0000 7f9c5d200640 1 -- 172.21.15.7:0/1064144903 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c580b7050 con 0x7f9c580b55d0 2024-09-10T21:41:48.764 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.761+0000 7f9c5d200640 1 -- 172.21.15.7:0/1064144903 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c5806f0f0 con 0x7f9c580b55d0 2024-09-10T21:41:48.764 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.761+0000 7f9c4e000640 1 -- 172.21.15.7:0/1064144903 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 2) ==== 50324+0+0 (secure 0 0 0) 0x7f9c4801c460 con 0x7f9c580b55d0 2024-09-10T21:41:48.764 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.761+0000 7f9c4e000640 1 -- 172.21.15.7:0/1064144903 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f9c48051850 con 0x7f9c580b55d0 2024-09-10T21:41:48.766 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.764+0000 7f9c4e000640 1 -- 172.21.15.7:0/1064144903 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f9c48012420 con 0x7f9c580b55d0 2024-09-10T21:41:48.832 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.830+0000 7f9c5d200640 1 -- 172.21.15.7:0/1064144903 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c580b6e70 con 0x7f9c580b55d0 2024-09-10T21:41:48.833 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.830+0000 7f9c4e000640 1 -- 172.21.15.7:0/1064144903 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f9c580b6e70 con 0x7f9c580b55d0 2024-09-10T21:41:48.833 INFO:teuthology.misc.health.smithi007.stdout: 2024-09-10T21:41:48.833 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":10,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:41:47.424461+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:41:48.834 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.831+0000 7f9c5d200640 1 -- 172.21.15.7:0/1064144903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c580b55d0 msgr2=0x7f9c581f4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:48.834 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.831+0000 7f9c5d200640 1 --2- 172.21.15.7:0/1064144903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c580b55d0 0x7f9c581f4ed0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9c4800c3f0 tx=0x7f9c4800c8b0 comp rx=0 tx=0).stop 2024-09-10T21:41:48.834 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.831+0000 7f9c5d200640 1 -- 172.21.15.7:0/1064144903 shutdown_connections 2024-09-10T21:41:48.834 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.831+0000 7f9c5d200640 1 --2- 172.21.15.7:0/1064144903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9c580d1f20 0x7f9c581535b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:48.834 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.831+0000 7f9c5d200640 1 --2- 172.21.15.7:0/1064144903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9c581618b0 0x7f9c580cc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:48.834 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.831+0000 7f9c5d200640 1 --2- 172.21.15.7:0/1064144903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9c580b55d0 0x7f9c581f4ed0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:48.834 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.831+0000 7f9c5d200640 1 -- 172.21.15.7:0/1064144903 >> 172.21.15.7:0/1064144903 conn(0x7f9c580d4020 msgr2=0x7f9c580b6060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:48.834 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.831+0000 7f9c5d200640 1 -- 172.21.15.7:0/1064144903 shutdown_connections 2024-09-10T21:41:48.834 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:48.832+0000 7f9c5d200640 1 -- 172.21.15.7:0/1064144903 wait complete. 2024-09-10T21:41:48.845 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-10T21:41:48.859 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-10T21:41:48.858+0000 7f6654600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T21:41:48.859 INFO:tasks.ceph.mgr.x.smithi133.stderr:2024-09-10T21:41:48.858+0000 7f6654600640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T21:41:49.044 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-10T21:41:49.042+0000 7f5a3a015640 -1 Falling back to public interface 2024-09-10T21:41:49.045 INFO:tasks.ceph.osd.7.smithi133.stderr:2024-09-10T21:41:49.045+0000 7fa0fa02c640 -1 Falling back to public interface 2024-09-10T21:41:49.054 INFO:tasks.ceph.osd.6.smithi133.stderr:2024-09-10T21:41:49.054+0000 7f4872a26640 -1 Falling back to public interface 2024-09-10T21:41:49.056 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-10T21:41:49.054+0000 7f59ceb4b640 -1 Falling back to public interface 2024-09-10T21:41:49.092 INFO:tasks.ceph.osd.5.smithi133.stderr:2024-09-10T21:41:49.091+0000 7f92dfa0f640 -1 Falling back to public interface 2024-09-10T21:41:49.240 INFO:tasks.ceph.osd.4.smithi133.stderr:2024-09-10T21:41:49.239+0000 7f536b85e640 -1 Falling back to public interface 2024-09-10T21:41:49.248 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-10T21:41:49.245+0000 7f1e2d734640 -1 Falling back to public interface 2024-09-10T21:41:49.267 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-10T21:41:49.265+0000 7f078940d640 -1 Falling back to public interface 2024-09-10T21:41:52.723 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-10T21:41:52.722+0000 7f5a3a015640 -1 osd.3 0 log_to_monitors true 2024-09-10T21:41:52.772 INFO:tasks.ceph.osd.7.smithi133.stderr:2024-09-10T21:41:52.771+0000 7fa0fa02c640 -1 osd.7 0 log_to_monitors true 2024-09-10T21:41:52.919 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-10T21:41:52.918+0000 7f1e2d734640 -1 osd.2 0 log_to_monitors true 2024-09-10T21:41:52.966 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-10T21:41:52.965+0000 7f59ceb4b640 -1 osd.0 0 log_to_monitors true 2024-09-10T21:41:53.079 INFO:tasks.ceph.osd.6.smithi133.stderr:2024-09-10T21:41:53.078+0000 7f4872a26640 -1 osd.6 0 log_to_monitors true 2024-09-10T21:41:53.140 INFO:tasks.ceph.osd.4.smithi133.stderr:2024-09-10T21:41:53.139+0000 7f536b85e640 -1 osd.4 0 log_to_monitors true 2024-09-10T21:41:53.359 INFO:tasks.ceph.osd.5.smithi133.stderr:2024-09-10T21:41:53.358+0000 7f92dfa0f640 -1 osd.5 0 log_to_monitors true 2024-09-10T21:41:53.446 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-10T21:41:53.444+0000 7f078940d640 -1 osd.1 0 log_to_monitors true 2024-09-10T21:41:54.782 INFO:tasks.ceph.osd.7.smithi133.stderr:2024-09-10T21:41:54.781+0000 7fa0f4400640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T21:41:54.788 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-10T21:41:54.787+0000 7f5a35400640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T21:41:54.795 INFO:tasks.ceph.osd.7.smithi133.stderr:2024-09-10T21:41:54.793+0000 7fa0ea400640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:41:54.801 INFO:tasks.ceph.osd.3.smithi007.stderr:2024-09-10T21:41:54.798+0000 7f5a2aa00640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:41:55.652 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T21:41:55.889 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.888+0000 7ff27ac00640 1 -- 172.21.15.7:0/1951741199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2740b31d0 msgr2=0x7ff27415c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:55.889 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.888+0000 7ff27ac00640 1 --2- 172.21.15.7:0/1951741199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2740b31d0 0x7ff27415c8b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ff2740d0c10 tx=0x7ff26c02a9c0 comp rx=0 tx=0).stop 2024-09-10T21:41:55.890 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.889+0000 7ff27ac00640 1 -- 172.21.15.7:0/1951741199 shutdown_connections 2024-09-10T21:41:55.890 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.889+0000 7ff27ac00640 1 --2- 172.21.15.7:0/1951741199 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2740b31d0 0x7ff27415c8b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:55.890 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.889+0000 7ff27ac00640 1 --2- 172.21.15.7:0/1951741199 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2741601a0 0x7ff2740b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:55.890 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.889+0000 7ff27ac00640 1 --2- 172.21.15.7:0/1951741199 >> v2:172.21.15.7:3300/0 conn(0x7ff2740d0400 0x7ff2740701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:55.891 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.889+0000 7ff27ac00640 1 -- 172.21.15.7:0/1951741199 >> 172.21.15.7:0/1951741199 conn(0x7ff27406cd10 msgr2=0x7ff27406d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:55.891 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.889+0000 7ff27ac00640 1 -- 172.21.15.7:0/1951741199 shutdown_connections 2024-09-10T21:41:55.891 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.889+0000 7ff27ac00640 1 -- 172.21.15.7:0/1951741199 wait complete. 2024-09-10T21:41:55.893 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.892+0000 7ff27ac00640 1 Processor -- start 2024-09-10T21:41:55.894 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.892+0000 7ff27ac00640 1 -- start start 2024-09-10T21:41:55.901 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.899+0000 7ff27ac00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2740b31d0 0x7ff27414e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:55.901 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.899+0000 7ff27ac00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2740d0400 0x7ff27413cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:55.901 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.899+0000 7ff27ac00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2741601a0 0x7ff274155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:55.901 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.899+0000 7ff27ac00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff27415d180 con 0x7ff2740d0400 2024-09-10T21:41:55.902 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.899+0000 7ff27ac00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff27415d000 con 0x7ff2741601a0 2024-09-10T21:41:55.902 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.899+0000 7ff27ac00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ff2740ae5c0 con 0x7ff2740b31d0 2024-09-10T21:41:55.902 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.900+0000 7ff273e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2740b31d0 0x7ff27414e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:55.902 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.900+0000 7ff273e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2740b31d0 0x7ff27414e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:42918/0 (socket says 172.21.15.7:42918) 2024-09-10T21:41:55.902 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.900+0000 7ff273e00640 1 -- 172.21.15.7:0/804636990 learned_addr learned my addr 172.21.15.7:0/804636990 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:41:55.903 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.901+0000 7ff273e00640 1 -- 172.21.15.7:0/804636990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2741601a0 msgr2=0x7ff274155320 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:41:55.903 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.901+0000 7ff273e00640 1 --2- 172.21.15.7:0/804636990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2741601a0 0x7ff274155320 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:55.903 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.901+0000 7ff273e00640 1 -- 172.21.15.7:0/804636990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2740d0400 msgr2=0x7ff27413cd20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:41:55.903 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.902+0000 7ff273e00640 1 --2- 172.21.15.7:0/804636990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2740d0400 0x7ff27413cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:55.904 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.902+0000 7ff273e00640 1 -- 172.21.15.7:0/804636990 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff274154560 con 0x7ff2740b31d0 2024-09-10T21:41:55.905 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.904+0000 7ff273e00640 1 --2- 172.21.15.7:0/804636990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2740b31d0 0x7ff27414e2b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff26400d950 tx=0x7ff26400de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:55.906 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.905+0000 7ff26be00640 1 -- 172.21.15.7:0/804636990 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264017070 con 0x7ff2740b31d0 2024-09-10T21:41:55.907 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.905+0000 7ff26be00640 1 -- 172.21.15.7:0/804636990 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff264009d20 con 0x7ff2740b31d0 2024-09-10T21:41:55.907 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.905+0000 7ff26be00640 1 -- 172.21.15.7:0/804636990 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2640106c0 con 0x7ff2740b31d0 2024-09-10T21:41:55.907 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.906+0000 7ff27ac00640 1 -- 172.21.15.7:0/804636990 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff274154840 con 0x7ff2740b31d0 2024-09-10T21:41:55.907 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.906+0000 7ff27ac00640 1 -- 172.21.15.7:0/804636990 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff274154e00 con 0x7ff2740b31d0 2024-09-10T21:41:55.909 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.908+0000 7ff27ac00640 1 -- 172.21.15.7:0/804636990 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2740701e0 con 0x7ff2740b31d0 2024-09-10T21:41:55.912 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.910+0000 7ff26be00640 1 -- 172.21.15.7:0/804636990 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 4) ==== 100545+0+0 (secure 0 0 0) 0x7ff264010860 con 0x7ff2740b31d0 2024-09-10T21:41:55.914 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.913+0000 7ff26be00640 1 --2- 172.21.15.7:0/804636990 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff250077ee0 0x7ff25007a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:55.915 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.913+0000 7ff26be00640 1 -- 172.21.15.7:0/804636990 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(14..14 src has 1..14) ==== 2791+0+0 (secure 0 0 0) 0x7ff2640994a0 con 0x7ff2740b31d0 2024-09-10T21:41:55.915 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.914+0000 7ff273400640 1 --2- 172.21.15.7:0/804636990 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff250077ee0 0x7ff25007a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:55.916 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.915+0000 7ff273400640 1 --2- 172.21.15.7:0/804636990 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff250077ee0 0x7ff25007a3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff274151fd0 tx=0x7ff25c007a50 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:55.926 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:55.925+0000 7ff26be00640 1 -- 172.21.15.7:0/804636990 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff264062c70 con 0x7ff2740b31d0 2024-09-10T21:41:56.286 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:56.285+0000 7ff27ac00640 1 -- 172.21.15.7:0/804636990 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff27409dba0 con 0x7ff2740b31d0 2024-09-10T21:41:56.288 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:56.287+0000 7ff26be00640 1 -- 172.21.15.7:0/804636990 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+8029 (secure 0 0 0) 0x7ff264062570 con 0x7ff2740b31d0 2024-09-10T21:41:56.291 INFO:teuthology.misc.health.smithi007.stdout: 2024-09-10T21:41:56.291 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":14,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:41:55.797313+0000","last_up_change":"2024-09-10T21:41:55.797313+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:41:56.292 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:56.290+0000 7ff27ac00640 1 -- 172.21.15.7:0/804636990 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff250077ee0 msgr2=0x7ff25007a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:56.292 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:56.290+0000 7ff27ac00640 1 --2- 172.21.15.7:0/804636990 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff250077ee0 0x7ff25007a3a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff274151fd0 tx=0x7ff25c007a50 comp rx=0 tx=0).stop 2024-09-10T21:41:56.292 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:56.290+0000 7ff27ac00640 1 -- 172.21.15.7:0/804636990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2740b31d0 msgr2=0x7ff27414e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:56.292 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:56.290+0000 7ff27ac00640 1 --2- 172.21.15.7:0/804636990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2740b31d0 0x7ff27414e2b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff26400d950 tx=0x7ff26400de10 comp rx=0 tx=0).stop 2024-09-10T21:41:56.292 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:56.290+0000 7ff27ac00640 1 -- 172.21.15.7:0/804636990 shutdown_connections 2024-09-10T21:41:56.292 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:56.290+0000 7ff27ac00640 1 --2- 172.21.15.7:0/804636990 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff250077ee0 0x7ff25007a3a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:56.292 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:56.290+0000 7ff27ac00640 1 --2- 172.21.15.7:0/804636990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2741601a0 0x7ff274155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:56.292 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:56.290+0000 7ff27ac00640 1 --2- 172.21.15.7:0/804636990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2740d0400 0x7ff27413cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:56.292 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:56.290+0000 7ff27ac00640 1 --2- 172.21.15.7:0/804636990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2740b31d0 0x7ff27414e2b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:56.292 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:56.290+0000 7ff27ac00640 1 -- 172.21.15.7:0/804636990 >> 172.21.15.7:0/804636990 conn(0x7ff27406cd10 msgr2=0x7ff27409ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:56.293 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:56.291+0000 7ff27ac00640 1 -- 172.21.15.7:0/804636990 shutdown_connections 2024-09-10T21:41:56.293 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:41:56.291+0000 7ff27ac00640 1 -- 172.21.15.7:0/804636990 wait complete. 2024-09-10T21:41:56.312 DEBUG:teuthology.misc:2 of 8 OSDs are up 2024-09-10T21:41:56.342 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-10T21:41:56.341+0000 7f1e28600640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T21:41:56.351 INFO:tasks.ceph.osd.2.smithi007.stderr:2024-09-10T21:41:56.350+0000 7f1e1e600640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:41:56.358 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-10T21:41:56.357+0000 7f59c9a00640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T21:41:56.370 INFO:tasks.ceph.osd.0.smithi007.stderr:2024-09-10T21:41:56.369+0000 7f59bfa00640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:41:56.386 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-10T21:41:56.385+0000 7f0783c00640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T21:41:56.396 INFO:tasks.ceph.osd.1.smithi007.stderr:2024-09-10T21:41:56.395+0000 7f0779c00640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:41:56.613 INFO:tasks.ceph.osd.5.smithi133.stderr:2024-09-10T21:41:56.612+0000 7f92dae00640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T21:41:56.620 INFO:tasks.ceph.osd.5.smithi133.stderr:2024-09-10T21:41:56.619+0000 7f92d0400640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:41:56.623 INFO:tasks.ceph.osd.6.smithi133.stderr:2024-09-10T21:41:56.623+0000 7f486d800640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T21:41:56.635 INFO:tasks.ceph.osd.6.smithi133.stderr:2024-09-10T21:41:56.634+0000 7f4863800640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:41:56.644 INFO:tasks.ceph.osd.4.smithi133.stderr:2024-09-10T21:41:56.643+0000 7f5366800640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T21:41:56.651 INFO:tasks.ceph.osd.4.smithi133.stderr:2024-09-10T21:41:56.650+0000 7f535c800640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:41:59.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:41:59.484 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:42:03.118 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T21:42:03.271 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.269+0000 7f56e3600640 1 -- 172.21.15.7:0/251488759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56dc1601a0 msgr2=0x7f56dc0b2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:03.271 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.269+0000 7f56e3600640 1 --2- 172.21.15.7:0/251488759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56dc1601a0 0x7f56dc0b2c90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f56dc074ad0 tx=0x7f56c402a4d0 comp rx=0 tx=0).stop 2024-09-10T21:42:03.271 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.269+0000 7f56e3600640 1 -- 172.21.15.7:0/251488759 shutdown_connections 2024-09-10T21:42:03.271 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.269+0000 7f56e3600640 1 --2- 172.21.15.7:0/251488759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56dc1601a0 0x7f56dc0b2c90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:03.271 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.269+0000 7f56e3600640 1 --2- 172.21.15.7:0/251488759 >> v2:172.21.15.133:3300/0 conn(0x7f56dc0d0400 0x7f56dc0701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:03.271 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.269+0000 7f56e3600640 1 -- 172.21.15.7:0/251488759 >> 172.21.15.7:0/251488759 conn(0x7f56dc06cd10 msgr2=0x7f56dc06d120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:03.271 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.269+0000 7f56e3600640 1 -- 172.21.15.7:0/251488759 shutdown_connections 2024-09-10T21:42:03.271 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.269+0000 7f56e3600640 1 -- 172.21.15.7:0/251488759 wait complete. 2024-09-10T21:42:03.272 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.271+0000 7f56e3600640 1 Processor -- start 2024-09-10T21:42:03.272 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.271+0000 7f56e3600640 1 -- start start 2024-09-10T21:42:03.275 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.272+0000 7f56e3600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56dc0b31d0 0x7f56dc1f8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:03.275 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.272+0000 7f56e3600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56dc0d0400 0x7f56dc0cf760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:03.275 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.272+0000 7f56e3600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56dc1601a0 0x7f56dc154300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:03.275 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.272+0000 7f56e3600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc0ae740 con 0x7f56dc1601a0 2024-09-10T21:42:03.275 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.272+0000 7f56e3600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f56dc0ae5c0 con 0x7f56dc0b31d0 2024-09-10T21:42:03.275 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.272+0000 7f56e3600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f56dc0ae8c0 con 0x7f56dc0d0400 2024-09-10T21:42:03.275 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.272+0000 7f56e1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56dc1601a0 0x7f56dc154300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:03.275 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.272+0000 7f56e1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56dc1601a0 0x7f56dc154300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37770/0 (socket says 172.21.15.7:37770) 2024-09-10T21:42:03.276 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.272+0000 7f56e1600640 1 -- 172.21.15.7:0/3669638579 learned_addr learned my addr 172.21.15.7:0/3669638579 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:03.276 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.272+0000 7f56e0c00640 1 --2- 172.21.15.7:0/3669638579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56dc0b31d0 0x7f56dc1f8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:03.276 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.273+0000 7f56d3e00640 1 --2- 172.21.15.7:0/3669638579 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56dc0d0400 0x7f56dc0cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:03.276 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.273+0000 7f56e1600640 1 -- 172.21.15.7:0/3669638579 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56dc0d0400 msgr2=0x7f56dc0cf760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:03.276 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.273+0000 7f56e1600640 1 --2- 172.21.15.7:0/3669638579 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56dc0d0400 0x7f56dc0cf760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:03.276 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.273+0000 7f56e1600640 1 -- 172.21.15.7:0/3669638579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56dc0b31d0 msgr2=0x7f56dc1f8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:03.276 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.273+0000 7f56e1600640 1 --2- 172.21.15.7:0/3669638579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56dc0b31d0 0x7f56dc1f8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:03.276 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.273+0000 7f56e1600640 1 -- 172.21.15.7:0/3669638579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56dc154840 con 0x7f56dc1601a0 2024-09-10T21:42:03.276 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.273+0000 7f56e1600640 1 --2- 172.21.15.7:0/3669638579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56dc1601a0 0x7f56dc154300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f56d400c780 tx=0x7f56d400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:03.276 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.273+0000 7f56d9600640 1 -- 172.21.15.7:0/3669638579 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d4004b60 con 0x7f56dc1601a0 2024-09-10T21:42:03.276 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.273+0000 7f56d9600640 1 -- 172.21.15.7:0/3669638579 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56d4004df0 con 0x7f56dc1601a0 2024-09-10T21:42:03.276 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.273+0000 7f56d9600640 1 -- 172.21.15.7:0/3669638579 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56d400aa70 con 0x7f56dc1601a0 2024-09-10T21:42:03.277 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.273+0000 7f56e3600640 1 -- 172.21.15.7:0/3669638579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56dc154ac0 con 0x7f56dc1601a0 2024-09-10T21:42:03.277 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.273+0000 7f56e3600640 1 -- 172.21.15.7:0/3669638579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56dc155080 con 0x7f56dc1601a0 2024-09-10T21:42:03.277 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.274+0000 7f56e3600640 1 -- 172.21.15.7:0/3669638579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56dc0aee50 con 0x7f56dc1601a0 2024-09-10T21:42:03.277 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.276+0000 7f56d9600640 1 -- 172.21.15.7:0/3669638579 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f56d400ac10 con 0x7f56dc1601a0 2024-09-10T21:42:03.278 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.277+0000 7f56d9600640 1 --2- 172.21.15.7:0/3669638579 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f56b8077f70 0x7f56b807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:03.278 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.277+0000 7f56d9600640 1 -- 172.21.15.7:0/3669638579 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f56d4099d10 con 0x7f56dc1601a0 2024-09-10T21:42:03.279 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.278+0000 7f56e0c00640 1 --2- 172.21.15.7:0/3669638579 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f56b8077f70 0x7f56b807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:03.279 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.278+0000 7f56e0c00640 1 --2- 172.21.15.7:0/3669638579 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f56b8077f70 0x7f56b807a430 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f56cc005fd0 tx=0x7f56cc005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:03.283 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.282+0000 7f56d9600640 1 -- 172.21.15.7:0/3669638579 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f56d409f020 con 0x7f56dc1601a0 2024-09-10T21:42:03.480 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.479+0000 7f56e3600640 1 -- 172.21.15.7:0/3669638579 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f56dc0b2fb0 con 0x7f56dc1601a0 2024-09-10T21:42:03.481 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.480+0000 7f56d9600640 1 -- 172.21.15.7:0/3669638579 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12860 (secure 0 0 0) 0x7f56d4020070 con 0x7f56dc1601a0 2024-09-10T21:42:03.481 INFO:teuthology.misc.health.smithi007.stdout: 2024-09-10T21:42:03.481 INFO:teuthology.misc.health.smithi007.stdout:{"epoch":17,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:41:58.852225+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:42:03.484 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.482+0000 7f56e3600640 1 -- 172.21.15.7:0/3669638579 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f56b8077f70 msgr2=0x7f56b807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:03.484 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.482+0000 7f56e3600640 1 --2- 172.21.15.7:0/3669638579 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f56b8077f70 0x7f56b807a430 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f56cc005fd0 tx=0x7f56cc005850 comp rx=0 tx=0).stop 2024-09-10T21:42:03.484 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.482+0000 7f56e3600640 1 -- 172.21.15.7:0/3669638579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56dc1601a0 msgr2=0x7f56dc154300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:03.484 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.482+0000 7f56e3600640 1 --2- 172.21.15.7:0/3669638579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56dc1601a0 0x7f56dc154300 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f56d400c780 tx=0x7f56d400cc40 comp rx=0 tx=0).stop 2024-09-10T21:42:03.484 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.482+0000 7f56e3600640 1 -- 172.21.15.7:0/3669638579 shutdown_connections 2024-09-10T21:42:03.484 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.482+0000 7f56e3600640 1 --2- 172.21.15.7:0/3669638579 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f56b8077f70 0x7f56b807a430 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:03.484 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.482+0000 7f56e3600640 1 --2- 172.21.15.7:0/3669638579 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56dc1601a0 0x7f56dc154300 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:03.484 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.482+0000 7f56e3600640 1 --2- 172.21.15.7:0/3669638579 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f56dc0d0400 0x7f56dc0cf760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:03.484 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.482+0000 7f56e3600640 1 --2- 172.21.15.7:0/3669638579 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f56dc0b31d0 0x7f56dc1f8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:03.484 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.482+0000 7f56e3600640 1 -- 172.21.15.7:0/3669638579 >> 172.21.15.7:0/3669638579 conn(0x7f56dc06cd10 msgr2=0x7f56dc09d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:03.484 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.482+0000 7f56e3600640 1 -- 172.21.15.7:0/3669638579 shutdown_connections 2024-09-10T21:42:03.484 INFO:teuthology.misc.health.smithi007.stderr:2024-09-10T21:42:03.482+0000 7f56e3600640 1 -- 172.21.15.7:0/3669638579 wait complete. 2024-09-10T21:42:03.496 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-10T21:42:03.496 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-10T21:42:03.497 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:42:03.497 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:42:03.500 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:42:03.501 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:42:03.501 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:42:03.504 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:42:03.505 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:42:03.505 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:42:03.509 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:42:03.509 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:42:03.509 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:42:03.513 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:42:03.513 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-10T21:42:03.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:42:03.632 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-09-10T21:42:03.633 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-09-10T21:42:03.647 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2024-09-10T21:42:03.648 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2024-09-10T21:42:03.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.681+0000 7f5f0f400640 1 -- 172.21.15.7:0/2141123724 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f0815a8e0 msgr2=0x7f5f080ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:03.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.681+0000 7f5f0f400640 1 --2- 172.21.15.7:0/2141123724 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f0815a8e0 0x7f5f080ce860 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8009830 tx=0x7f5ef802a5a0 comp rx=0 tx=0).stop 2024-09-10T21:42:03.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.681+0000 7f5f0f400640 1 -- 172.21.15.7:0/2141123724 shutdown_connections 2024-09-10T21:42:03.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.681+0000 7f5f0f400640 1 --2- 172.21.15.7:0/2141123724 >> v2:172.21.15.133:3300/0 conn(0x7f5f0815acc0 0x7f5f080cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:03.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.681+0000 7f5f0f400640 1 --2- 172.21.15.7:0/2141123724 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f0815a8e0 0x7f5f080ce860 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:03.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.681+0000 7f5f0f400640 1 -- 172.21.15.7:0/2141123724 >> 172.21.15.7:0/2141123724 conn(0x7f5f080d0c90 msgr2=0x7f5f080d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:03.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.681+0000 7f5f0f400640 1 -- 172.21.15.7:0/2141123724 shutdown_connections 2024-09-10T21:42:03.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.681+0000 7f5f0f400640 1 -- 172.21.15.7:0/2141123724 wait complete. 2024-09-10T21:42:03.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.683+0000 7f5f0f400640 1 Processor -- start 2024-09-10T21:42:03.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.683+0000 7f5f0f400640 1 -- start start 2024-09-10T21:42:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.684+0000 7f5f0f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0815a8e0 0x7f5f08151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.684+0000 7f5f0f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0815acc0 0x7f5f080c7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.684+0000 7f5f0f400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f080c6070 0x7f5f080b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.684+0000 7f5f0f400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5f08070be0 con 0x7f5f0815a8e0 2024-09-10T21:42:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.684+0000 7f5f0f400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5f08070a60 con 0x7f5f0815acc0 2024-09-10T21:42:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.684+0000 7f5f0f400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f5f08070d60 con 0x7f5f080c6070 2024-09-10T21:42:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.684+0000 7f5f0ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0815a8e0 0x7f5f08151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.684+0000 7f5f0ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0815a8e0 0x7f5f08151b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37832/0 (socket says 172.21.15.7:37832) 2024-09-10T21:42:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.684+0000 7f5f0ca00640 1 -- 172.21.15.7:0/2143982320 learned_addr learned my addr 172.21.15.7:0/2143982320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:03.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.684+0000 7f5f0d400640 1 --2- 172.21.15.7:0/2143982320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f080c6070 0x7f5f080b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:03.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.684+0000 7f5effe00640 1 --2- 172.21.15.7:0/2143982320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0815acc0 0x7f5f080c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:03.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.685+0000 7f5f0ca00640 1 -- 172.21.15.7:0/2143982320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f080c6070 msgr2=0x7f5f080b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:03.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.685+0000 7f5f0ca00640 1 --2- 172.21.15.7:0/2143982320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f080c6070 0x7f5f080b1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:03.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.685+0000 7f5f0ca00640 1 -- 172.21.15.7:0/2143982320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0815acc0 msgr2=0x7f5f080c7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:03.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.685+0000 7f5f0ca00640 1 --2- 172.21.15.7:0/2143982320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0815acc0 0x7f5f080c7590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:03.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.685+0000 7f5f0ca00640 1 -- 172.21.15.7:0/2143982320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f080c67d0 con 0x7f5f0815a8e0 2024-09-10T21:42:03.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.685+0000 7f5f0ca00640 1 --2- 172.21.15.7:0/2143982320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0815a8e0 0x7f5f08151b70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8000c00 tx=0x7f5ef8032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:03.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.685+0000 7f5f05600640 1 -- 172.21.15.7:0/2143982320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8040070 con 0x7f5f0815a8e0 2024-09-10T21:42:03.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.685+0000 7f5f05600640 1 -- 172.21.15.7:0/2143982320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ef80038e0 con 0x7f5f0815a8e0 2024-09-10T21:42:03.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.685+0000 7f5f05600640 1 -- 172.21.15.7:0/2143982320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef8005c90 con 0x7f5f0815a8e0 2024-09-10T21:42:03.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.685+0000 7f5f0f400640 1 -- 172.21.15.7:0/2143982320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f080c6ab0 con 0x7f5f0815a8e0 2024-09-10T21:42:03.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.685+0000 7f5f0f400640 1 -- 172.21.15.7:0/2143982320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f080c7070 con 0x7f5f0815a8e0 2024-09-10T21:42:03.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.687+0000 7f5f0f400640 1 -- 172.21.15.7:0/2143982320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f080ce860 con 0x7f5f0815a8e0 2024-09-10T21:42:03.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.687+0000 7f5f05600640 1 -- 172.21.15.7:0/2143982320 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5ef8045460 con 0x7f5f0815a8e0 2024-09-10T21:42:03.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.688+0000 7f5f05600640 1 --2- 172.21.15.7:0/2143982320 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5ee4077f70 0x7f5ee407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:03.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.688+0000 7f5f05600640 1 -- 172.21.15.7:0/2143982320 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f5ef80bb280 con 0x7f5f0815a8e0 2024-09-10T21:42:03.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.689+0000 7f5effe00640 1 --2- 172.21.15.7:0/2143982320 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5ee4077f70 0x7f5ee407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:03.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.689+0000 7f5effe00640 1 --2- 172.21.15.7:0/2143982320 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5ee4077f70 0x7f5ee407a430 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0007fd0 tx=0x7f5ef0001410 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:03.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.693+0000 7f5f05600640 1 -- 172.21.15.7:0/2143982320 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5ef8083a70 con 0x7f5f0815a8e0 2024-09-10T21:42:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.925+0000 7f5f0f400640 1 -- 172.21.15.7:0/2143982320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f08074720 con 0x7f5f0815a8e0 2024-09-10T21:42:03.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.926+0000 7f5f05600640 1 -- 172.21.15.7:0/2143982320 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1431 (secure 0 0 0) 0x7f5ef80884a0 con 0x7f5f0815a8e0 2024-09-10T21:42:03.927 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:03.927 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":2,"btime":"2024-09-10T21:42:03:639042+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4383,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3193801272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3193801272},{"type":"v1","addr":"172.21.15.7:6834","nonce":3193801272}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T21:42:03.928 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 2 2024-09-10T21:42:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.928+0000 7f5f0f400640 1 -- 172.21.15.7:0/2143982320 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5ee4077f70 msgr2=0x7f5ee407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.928+0000 7f5f0f400640 1 --2- 172.21.15.7:0/2143982320 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5ee4077f70 0x7f5ee407a430 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0007fd0 tx=0x7f5ef0001410 comp rx=0 tx=0).stop 2024-09-10T21:42:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.928+0000 7f5f0f400640 1 -- 172.21.15.7:0/2143982320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0815a8e0 msgr2=0x7f5f08151b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.928+0000 7f5f0f400640 1 --2- 172.21.15.7:0/2143982320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0815a8e0 0x7f5f08151b70 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8000c00 tx=0x7f5ef8032c10 comp rx=0 tx=0).stop 2024-09-10T21:42:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.928+0000 7f5f0f400640 1 -- 172.21.15.7:0/2143982320 shutdown_connections 2024-09-10T21:42:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.928+0000 7f5f0f400640 1 --2- 172.21.15.7:0/2143982320 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5ee4077f70 0x7f5ee407a430 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.928+0000 7f5f0f400640 1 --2- 172.21.15.7:0/2143982320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5f0815acc0 0x7f5f080c7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.928+0000 7f5f0f400640 1 --2- 172.21.15.7:0/2143982320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5f080c6070 0x7f5f080b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.928+0000 7f5f0f400640 1 --2- 172.21.15.7:0/2143982320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5f0815a8e0 0x7f5f08151b70 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.928+0000 7f5f0f400640 1 -- 172.21.15.7:0/2143982320 >> 172.21.15.7:0/2143982320 conn(0x7f5f080d0c90 msgr2=0x7f5f0815d530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.928+0000 7f5f0f400640 1 -- 172.21.15.7:0/2143982320 shutdown_connections 2024-09-10T21:42:03.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:03.928+0000 7f5f0f400640 1 -- 172.21.15.7:0/2143982320 wait complete. 2024-09-10T21:42:05.947 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:42:06.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.118+0000 7f343ee00640 1 -- 172.21.15.7:0/329723323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343815a4a0 msgr2=0x7f34380ce890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.118+0000 7f343ee00640 1 --2- 172.21.15.7:0/329723323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343815a4a0 0x7f34380ce890 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f34380b26d0 tx=0x7f342402a5f0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.118+0000 7f343ee00640 1 -- 172.21.15.7:0/329723323 shutdown_connections 2024-09-10T21:42:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.118+0000 7f343ee00640 1 --2- 172.21.15.7:0/329723323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f343815a4a0 0x7f34380ce890 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.118+0000 7f343ee00640 1 --2- 172.21.15.7:0/329723323 >> v2:172.21.15.133:3300/0 conn(0x7f343815a0c0 0x7f34380cb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.118+0000 7f343ee00640 1 -- 172.21.15.7:0/329723323 >> 172.21.15.7:0/329723323 conn(0x7f34380b16d0 msgr2=0x7f34380b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.118+0000 7f343ee00640 1 -- 172.21.15.7:0/329723323 shutdown_connections 2024-09-10T21:42:06.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.118+0000 7f343ee00640 1 -- 172.21.15.7:0/329723323 wait complete. 2024-09-10T21:42:06.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.120+0000 7f343ee00640 1 Processor -- start 2024-09-10T21:42:06.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.120+0000 7f343ee00640 1 -- start start 2024-09-10T21:42:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.121+0000 7f343ee00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34380cedd0 0x7f3438152380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.121+0000 7f343ee00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f343815a0c0 0x7f34380c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.121+0000 7f343ee00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f343815a4a0 0x7f3438150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.121+0000 7f343ee00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f34380cc600 con 0x7f34380cedd0 2024-09-10T21:42:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.121+0000 7f343ee00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f34380ca3e0 con 0x7f343815a0c0 2024-09-10T21:42:06.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.121+0000 7f343ee00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f34380cc780 con 0x7f343815a4a0 2024-09-10T21:42:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.121+0000 7f3437e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34380cedd0 0x7f3438152380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.121+0000 7f3437e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34380cedd0 0x7f3438152380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45194/0 (socket says 172.21.15.7:45194) 2024-09-10T21:42:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.121+0000 7f3437e00640 1 -- 172.21.15.7:0/2575737778 learned_addr learned my addr 172.21.15.7:0/2575737778 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.121+0000 7f3437400640 1 --2- 172.21.15.7:0/2575737778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f343815a0c0 0x7f34380c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.121+0000 7f343ce00640 1 --2- 172.21.15.7:0/2575737778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f343815a4a0 0x7f3438150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.121+0000 7f3437e00640 1 -- 172.21.15.7:0/2575737778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f343815a4a0 msgr2=0x7f3438150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.121+0000 7f3437e00640 1 --2- 172.21.15.7:0/2575737778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f343815a4a0 0x7f3438150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.121+0000 7f3437e00640 1 -- 172.21.15.7:0/2575737778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f343815a0c0 msgr2=0x7f34380c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.121+0000 7f3437e00640 1 --2- 172.21.15.7:0/2575737778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f343815a0c0 0x7f34380c8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.121+0000 7f3437e00640 1 -- 172.21.15.7:0/2575737778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3438156800 con 0x7f34380cedd0 2024-09-10T21:42:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.121+0000 7f343ce00640 1 --2- 172.21.15.7:0/2575737778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f343815a4a0 0x7f3438150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:06.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.122+0000 7f3437e00640 1 --2- 172.21.15.7:0/2575737778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34380cedd0 0x7f3438152380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f342800e9f0 tx=0x7f342800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.122+0000 7f342fe00640 1 -- 172.21.15.7:0/2575737778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342800cd80 con 0x7f34380cedd0 2024-09-10T21:42:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.122+0000 7f343ee00640 1 -- 172.21.15.7:0/2575737778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34381515e0 con 0x7f34380cedd0 2024-09-10T21:42:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.122+0000 7f342fe00640 1 -- 172.21.15.7:0/2575737778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3428010430 con 0x7f34380cedd0 2024-09-10T21:42:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.122+0000 7f342fe00640 1 -- 172.21.15.7:0/2575737778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3428010840 con 0x7f34380cedd0 2024-09-10T21:42:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.122+0000 7f343ee00640 1 -- 172.21.15.7:0/2575737778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3438151e80 con 0x7f34380cedd0 2024-09-10T21:42:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.123+0000 7f343ee00640 1 -- 172.21.15.7:0/2575737778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34381519d0 con 0x7f34380cedd0 2024-09-10T21:42:06.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.124+0000 7f342fe00640 1 -- 172.21.15.7:0/2575737778 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f34280109e0 con 0x7f34380cedd0 2024-09-10T21:42:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.125+0000 7f342fe00640 1 --2- 172.21.15.7:0/2575737778 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3418077f70 0x7f341807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:06.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.125+0000 7f342fe00640 1 -- 172.21.15.7:0/2575737778 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f342809a660 con 0x7f34380cedd0 2024-09-10T21:42:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.126+0000 7f3437400640 1 --2- 172.21.15.7:0/2575737778 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3418077f70 0x7f341807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:06.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.126+0000 7f3437400640 1 --2- 172.21.15.7:0/2575737778 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3418077f70 0x7f341807a430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3424033000 tx=0x7f3424004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:06.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.130+0000 7f342fe00640 1 -- 172.21.15.7:0/2575737778 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3428062e00 con 0x7f34380cedd0 2024-09-10T21:42:06.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.366+0000 7f343ee00640 1 -- 172.21.15.7:0/2575737778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f34380cb0c0 con 0x7f34380cedd0 2024-09-10T21:42:06.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.367+0000 7f342fe00640 1 -- 172.21.15.7:0/2575737778 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3908 (secure 0 0 0) 0x7f3428067830 con 0x7f34380cedd0 2024-09-10T21:42:06.368 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:06.368 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":3,"btime":"2024-09-10T21:42:04:640790+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4342,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3303727910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3303727910},{"type":"v1","addr":"172.21.15.7:6835","nonce":3303727910}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4383,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3193801272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3193801272},{"type":"v1","addr":"172.21.15.7:6834","nonce":3193801272}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4401,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1546148687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1546148687},{"type":"v1","addr":"172.21.15.133:6837","nonce":1546148687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":234179,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3769241160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3769241160},{"type":"v1","addr":"172.21.15.133:6836","nonce":3769241160}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T21:42:06.368 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 3 2024-09-10T21:42:06.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.369+0000 7f343ee00640 1 -- 172.21.15.7:0/2575737778 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3418077f70 msgr2=0x7f341807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:06.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.369+0000 7f343ee00640 1 --2- 172.21.15.7:0/2575737778 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3418077f70 0x7f341807a430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3424033000 tx=0x7f3424004730 comp rx=0 tx=0).stop 2024-09-10T21:42:06.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.369+0000 7f343ee00640 1 -- 172.21.15.7:0/2575737778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34380cedd0 msgr2=0x7f3438152380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:06.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.369+0000 7f343ee00640 1 --2- 172.21.15.7:0/2575737778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34380cedd0 0x7f3438152380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f342800e9f0 tx=0x7f342800eeb0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.369+0000 7f343ee00640 1 -- 172.21.15.7:0/2575737778 shutdown_connections 2024-09-10T21:42:06.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.369+0000 7f343ee00640 1 --2- 172.21.15.7:0/2575737778 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3418077f70 0x7f341807a430 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.369+0000 7f343ee00640 1 --2- 172.21.15.7:0/2575737778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f343815a0c0 0x7f34380c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.369+0000 7f343ee00640 1 --2- 172.21.15.7:0/2575737778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f343815a4a0 0x7f3438150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.369+0000 7f343ee00640 1 --2- 172.21.15.7:0/2575737778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f34380cedd0 0x7f3438152380 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.369+0000 7f343ee00640 1 -- 172.21.15.7:0/2575737778 >> 172.21.15.7:0/2575737778 conn(0x7f34380b16d0 msgr2=0x7f34380b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:06.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.369+0000 7f343ee00640 1 -- 172.21.15.7:0/2575737778 shutdown_connections 2024-09-10T21:42:06.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.369+0000 7f343ee00640 1 -- 172.21.15.7:0/2575737778 wait complete. 2024-09-10T21:42:06.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:42:06.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.545+0000 7f4108600640 1 -- 172.21.15.7:0/3700735630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41000cbd50 msgr2=0x7f41000cc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:06.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.545+0000 7f4108600640 1 --2- 172.21.15.7:0/3700735630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41000cbd50 0x7f41000cc190 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f41000ca610 tx=0x7f40ec02a560 comp rx=0 tx=0).stop 2024-09-10T21:42:06.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.545+0000 7f4108600640 1 -- 172.21.15.7:0/3700735630 shutdown_connections 2024-09-10T21:42:06.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.545+0000 7f4108600640 1 --2- 172.21.15.7:0/3700735630 >> v2:172.21.15.133:3300/0 conn(0x7f41000ae020 0x7f41000ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.545+0000 7f4108600640 1 --2- 172.21.15.7:0/3700735630 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41000cbd50 0x7f41000cc190 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.545+0000 7f4108600640 1 --2- 172.21.15.7:0/3700735630 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41001619e0 0x7f410015e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.545+0000 7f4108600640 1 -- 172.21.15.7:0/3700735630 >> 172.21.15.7:0/3700735630 conn(0x7f41000b16d0 msgr2=0x7f41000b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:06.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.545+0000 7f4108600640 1 -- 172.21.15.7:0/3700735630 shutdown_connections 2024-09-10T21:42:06.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.545+0000 7f4108600640 1 -- 172.21.15.7:0/3700735630 wait complete. 2024-09-10T21:42:06.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.547+0000 7f4108600640 1 Processor -- start 2024-09-10T21:42:06.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.547+0000 7f4108600640 1 -- start start 2024-09-10T21:42:06.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.548+0000 7f4108600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41000ae020 0x7f4100157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:06.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.548+0000 7f4108600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41000cbd50 0x7f4100159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:06.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.548+0000 7f4108600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41001619e0 0x7f41000b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:06.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.548+0000 7f4108600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f41000cdec0 con 0x7f41000cbd50 2024-09-10T21:42:06.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.548+0000 7f4108600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f41000cdd40 con 0x7f41000ae020 2024-09-10T21:42:06.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.548+0000 7f4108600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f41000ce040 con 0x7f41001619e0 2024-09-10T21:42:06.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.548+0000 7f4105200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41000cbd50 0x7f4100159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:06.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.548+0000 7f4105200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41000cbd50 0x7f4100159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45204/0 (socket says 172.21.15.7:45204) 2024-09-10T21:42:06.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.548+0000 7f4105200640 1 -- 172.21.15.7:0/2142169501 learned_addr learned my addr 172.21.15.7:0/2142169501 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:06.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.548+0000 7f4105c00640 1 --2- 172.21.15.7:0/2142169501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41000ae020 0x7f4100157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:06.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.548+0000 7f4106600640 1 --2- 172.21.15.7:0/2142169501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41001619e0 0x7f41000b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:06.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.549+0000 7f4105200640 1 -- 172.21.15.7:0/2142169501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41001619e0 msgr2=0x7f41000b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:06.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.549+0000 7f4105200640 1 --2- 172.21.15.7:0/2142169501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41001619e0 0x7f41000b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.549+0000 7f4105200640 1 -- 172.21.15.7:0/2142169501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41000ae020 msgr2=0x7f4100157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:06.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.549+0000 7f4105200640 1 --2- 172.21.15.7:0/2142169501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41000ae020 0x7f4100157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.549+0000 7f4105200640 1 -- 172.21.15.7:0/2142169501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41001f4f10 con 0x7f41000cbd50 2024-09-10T21:42:06.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.549+0000 7f4105200640 1 --2- 172.21.15.7:0/2142169501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41000cbd50 0x7f4100159b50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009850 tx=0x7f40ec003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:06.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.549+0000 7f40fd600640 1 -- 172.21.15.7:0/2142169501 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec03f070 con 0x7f41000cbd50 2024-09-10T21:42:06.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.549+0000 7f40fd600640 1 -- 172.21.15.7:0/2142169501 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40ec0047a0 con 0x7f41000cbd50 2024-09-10T21:42:06.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.549+0000 7f4108600640 1 -- 172.21.15.7:0/2142169501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41001f51d0 con 0x7f41000cbd50 2024-09-10T21:42:06.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.549+0000 7f40fd600640 1 -- 172.21.15.7:0/2142169501 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40ec035030 con 0x7f41000cbd50 2024-09-10T21:42:06.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.549+0000 7f4108600640 1 -- 172.21.15.7:0/2142169501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41001f5790 con 0x7f41000cbd50 2024-09-10T21:42:06.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.550+0000 7f4108600640 1 -- 172.21.15.7:0/2142169501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41000ca3e0 con 0x7f41000cbd50 2024-09-10T21:42:06.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.551+0000 7f40fd600640 1 -- 172.21.15.7:0/2142169501 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f40ec004bb0 con 0x7f41000cbd50 2024-09-10T21:42:06.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.552+0000 7f40fd600640 1 --2- 172.21.15.7:0/2142169501 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f40e0077f70 0x7f40e007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:06.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.553+0000 7f40fd600640 1 -- 172.21.15.7:0/2142169501 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f40ec0babe0 con 0x7f41000cbd50 2024-09-10T21:42:06.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.553+0000 7f4105c00640 1 --2- 172.21.15.7:0/2142169501 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f40e0077f70 0x7f40e007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:06.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.554+0000 7f4105c00640 1 --2- 172.21.15.7:0/2142169501 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f40e0077f70 0x7f40e007a430 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f40f4005d80 tx=0x7f40f400b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:06.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.557+0000 7f40fd600640 1 -- 172.21.15.7:0/2142169501 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f41000ca3e0 con 0x7f41000cbd50 2024-09-10T21:42:06.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.754+0000 7f4108600640 1 -- 172.21.15.7:0/2142169501 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f41000b1d60 con 0x7f41000cbd50 2024-09-10T21:42:06.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.755+0000 7f40fd600640 1 -- 172.21.15.7:0/2142169501 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f40ec083340 con 0x7f41000cbd50 2024-09-10T21:42:06.756 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:06.756 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-09-10T21:42:06.756 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:06.756 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-09-10T21:42:06.756 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-09-10T21:42:06.757 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:06.757 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-09-10T21:42:06.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.757+0000 7f4108600640 1 -- 172.21.15.7:0/2142169501 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f40e0077f70 msgr2=0x7f40e007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:06.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.757+0000 7f4108600640 1 --2- 172.21.15.7:0/2142169501 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f40e0077f70 0x7f40e007a430 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f40f4005d80 tx=0x7f40f400b390 comp rx=0 tx=0).stop 2024-09-10T21:42:06.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.757+0000 7f4108600640 1 -- 172.21.15.7:0/2142169501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41000cbd50 msgr2=0x7f4100159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:06.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.757+0000 7f4108600640 1 --2- 172.21.15.7:0/2142169501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41000cbd50 0x7f4100159b50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f40ec009850 tx=0x7f40ec003910 comp rx=0 tx=0).stop 2024-09-10T21:42:06.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.757+0000 7f4108600640 1 -- 172.21.15.7:0/2142169501 shutdown_connections 2024-09-10T21:42:06.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.757+0000 7f4108600640 1 --2- 172.21.15.7:0/2142169501 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f40e0077f70 0x7f40e007a430 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.757+0000 7f4108600640 1 --2- 172.21.15.7:0/2142169501 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41001619e0 0x7f41000b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.757+0000 7f4108600640 1 --2- 172.21.15.7:0/2142169501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41000cbd50 0x7f4100159b50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.757+0000 7f4108600640 1 --2- 172.21.15.7:0/2142169501 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41000ae020 0x7f4100157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.757+0000 7f4108600640 1 -- 172.21.15.7:0/2142169501 >> 172.21.15.7:0/2142169501 conn(0x7f41000b16d0 msgr2=0x7f41000ca6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:06.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.757+0000 7f4108600640 1 -- 172.21.15.7:0/2142169501 shutdown_connections 2024-09-10T21:42:06.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.757+0000 7f4108600640 1 -- 172.21.15.7:0/2142169501 wait complete. 2024-09-10T21:42:06.773 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:42:06.773 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:42:06.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.935+0000 7fe6e7e00640 1 -- 172.21.15.7:0/2873207673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e015bce0 msgr2=0x7fe6e006cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:06.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.935+0000 7fe6e7e00640 1 --2- 172.21.15.7:0/2873207673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e015bce0 0x7fe6e006cb20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0070940 tx=0x7fe6d802a980 comp rx=0 tx=0).stop 2024-09-10T21:42:06.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.935+0000 7fe6e7e00640 1 -- 172.21.15.7:0/2873207673 shutdown_connections 2024-09-10T21:42:06.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.935+0000 7fe6e7e00640 1 --2- 172.21.15.7:0/2873207673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e015bce0 0x7fe6e006cb20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.935+0000 7fe6e7e00640 1 --2- 172.21.15.7:0/2873207673 >> v2:172.21.15.133:3300/0 conn(0x7fe6e00ace80 0x7fe6e015f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.935+0000 7fe6e7e00640 1 --2- 172.21.15.7:0/2873207673 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6e015b900 0x7fe6e0070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.935+0000 7fe6e7e00640 1 -- 172.21.15.7:0/2873207673 >> 172.21.15.7:0/2873207673 conn(0x7fe6e00b1c10 msgr2=0x7fe6e00b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:06.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.935+0000 7fe6e7e00640 1 -- 172.21.15.7:0/2873207673 shutdown_connections 2024-09-10T21:42:06.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.935+0000 7fe6e7e00640 1 -- 172.21.15.7:0/2873207673 wait complete. 2024-09-10T21:42:06.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.937+0000 7fe6e7e00640 1 Processor -- start 2024-09-10T21:42:06.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.937+0000 7fe6e7e00640 1 -- start start 2024-09-10T21:42:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.938+0000 7fe6e7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e00ace80 0x7fe6e009d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.938+0000 7fe6e7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e015b900 0x7fe6e014f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.938+0000 7fe6e7e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6e015bce0 0x7fe6e01f23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.938+0000 7fe6e7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e006d460 con 0x7fe6e015b900 2024-09-10T21:42:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.938+0000 7fe6e7e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e006d2e0 con 0x7fe6e00ace80 2024-09-10T21:42:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.938+0000 7fe6e7e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e00aeb80 con 0x7fe6e015bce0 2024-09-10T21:42:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.938+0000 7fe6e4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e015b900 0x7fe6e014f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:06.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.938+0000 7fe6e4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e015b900 0x7fe6e014f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45236/0 (socket says 172.21.15.7:45236) 2024-09-10T21:42:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.938+0000 7fe6e4a00640 1 -- 172.21.15.7:0/2531342337 learned_addr learned my addr 172.21.15.7:0/2531342337 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.938+0000 7fe6e5e00640 1 --2- 172.21.15.7:0/2531342337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6e015bce0 0x7fe6e01f23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.938+0000 7fe6e5400640 1 --2- 172.21.15.7:0/2531342337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e00ace80 0x7fe6e009d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.939+0000 7fe6e4a00640 1 -- 172.21.15.7:0/2531342337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6e015bce0 msgr2=0x7fe6e01f23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.939+0000 7fe6e4a00640 1 --2- 172.21.15.7:0/2531342337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6e015bce0 0x7fe6e01f23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.939+0000 7fe6e4a00640 1 -- 172.21.15.7:0/2531342337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e00ace80 msgr2=0x7fe6e009d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.939+0000 7fe6e4a00640 1 --2- 172.21.15.7:0/2531342337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e00ace80 0x7fe6e009d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.939+0000 7fe6e4a00640 1 -- 172.21.15.7:0/2531342337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e014e610 con 0x7fe6e015b900 2024-09-10T21:42:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.939+0000 7fe6e5e00640 1 --2- 172.21.15.7:0/2531342337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6e015bce0 0x7fe6e01f23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:06.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.939+0000 7fe6e5400640 1 --2- 172.21.15.7:0/2531342337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e00ace80 0x7fe6e009d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:06.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.939+0000 7fe6e4a00640 1 --2- 172.21.15.7:0/2531342337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e015b900 0x7fe6e014f0b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00d980 tx=0x7fe6cc00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:06.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.939+0000 7fe6dd600640 1 -- 172.21.15.7:0/2531342337 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc017030 con 0x7fe6e015b900 2024-09-10T21:42:06.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.939+0000 7fe6dd600640 1 -- 172.21.15.7:0/2531342337 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6cc009b20 con 0x7fe6e015b900 2024-09-10T21:42:06.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.939+0000 7fe6dd600640 1 -- 172.21.15.7:0/2531342337 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc004020 con 0x7fe6e015b900 2024-09-10T21:42:06.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.939+0000 7fe6e7e00640 1 -- 172.21.15.7:0/2531342337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e01f2e90 con 0x7fe6e015b900 2024-09-10T21:42:06.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.939+0000 7fe6e7e00640 1 -- 172.21.15.7:0/2531342337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e014e8b0 con 0x7fe6e015b900 2024-09-10T21:42:06.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.941+0000 7fe6e7e00640 1 -- 172.21.15.7:0/2531342337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e01f33b0 con 0x7fe6e015b900 2024-09-10T21:42:06.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.942+0000 7fe6dd600640 1 -- 172.21.15.7:0/2531342337 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe6cc0041c0 con 0x7fe6e015b900 2024-09-10T21:42:06.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.943+0000 7fe6dd600640 1 --2- 172.21.15.7:0/2531342337 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe6c0077f70 0x7fe6c007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:06.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.943+0000 7fe6dd600640 1 -- 172.21.15.7:0/2531342337 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fe6cc099ba0 con 0x7fe6e015b900 2024-09-10T21:42:06.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.943+0000 7fe6e5400640 1 --2- 172.21.15.7:0/2531342337 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe6c0077f70 0x7fe6c007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:06.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.944+0000 7fe6e5400640 1 --2- 172.21.15.7:0/2531342337 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe6c0077f70 0x7fe6c007a430 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4005fd0 tx=0x7fe6d400b5f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:06.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:06.947+0000 7fe6dd600640 1 -- 172.21.15.7:0/2531342337 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe6e01f33b0 con 0x7fe6e015b900 2024-09-10T21:42:07.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.140+0000 7fe6e7e00640 1 -- 172.21.15.7:0/2531342337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fe6e0159ba0 con 0x7fe6e015b900 2024-09-10T21:42:07.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.661+0000 7fe6dd600640 1 -- 172.21.15.7:0/2531342337 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7fe6cc0629e0 con 0x7fe6e015b900 2024-09-10T21:42:07.663 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-09-10T21:42:07.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.665+0000 7fe6e7e00640 1 -- 172.21.15.7:0/2531342337 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe6c0077f70 msgr2=0x7fe6c007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:07.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.665+0000 7fe6e7e00640 1 --2- 172.21.15.7:0/2531342337 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe6c0077f70 0x7fe6c007a430 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4005fd0 tx=0x7fe6d400b5f0 comp rx=0 tx=0).stop 2024-09-10T21:42:07.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.665+0000 7fe6e7e00640 1 -- 172.21.15.7:0/2531342337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e015b900 msgr2=0x7fe6e014f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:07.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.665+0000 7fe6e7e00640 1 --2- 172.21.15.7:0/2531342337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e015b900 0x7fe6e014f0b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc00d980 tx=0x7fe6cc00de40 comp rx=0 tx=0).stop 2024-09-10T21:42:07.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.666+0000 7fe6e7e00640 1 -- 172.21.15.7:0/2531342337 shutdown_connections 2024-09-10T21:42:07.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.666+0000 7fe6e7e00640 1 --2- 172.21.15.7:0/2531342337 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe6c0077f70 0x7fe6c007a430 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:07.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.666+0000 7fe6e7e00640 1 --2- 172.21.15.7:0/2531342337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6e015bce0 0x7fe6e01f23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:07.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.666+0000 7fe6e7e00640 1 --2- 172.21.15.7:0/2531342337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e015b900 0x7fe6e014f0b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:07.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.666+0000 7fe6e7e00640 1 --2- 172.21.15.7:0/2531342337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e00ace80 0x7fe6e009d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:07.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.666+0000 7fe6e7e00640 1 -- 172.21.15.7:0/2531342337 >> 172.21.15.7:0/2531342337 conn(0x7fe6e00b1c10 msgr2=0x7fe6e014b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:07.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.666+0000 7fe6e7e00640 1 -- 172.21.15.7:0/2531342337 shutdown_connections 2024-09-10T21:42:07.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.666+0000 7fe6e7e00640 1 -- 172.21.15.7:0/2531342337 wait complete. 2024-09-10T21:42:07.689 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:42:07.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.858+0000 7fd5cfc00640 1 -- 172.21.15.7:0/2092510066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c806fcf0 msgr2=0x7fd5c8073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:07.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.858+0000 7fd5cfc00640 1 --2- 172.21.15.7:0/2092510066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c806fcf0 0x7fd5c8073d40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd5c80708e0 tx=0x7fd5b402a560 comp rx=0 tx=0).stop 2024-09-10T21:42:07.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.858+0000 7fd5cfc00640 1 -- 172.21.15.7:0/2092510066 shutdown_connections 2024-09-10T21:42:07.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.858+0000 7fd5cfc00640 1 --2- 172.21.15.7:0/2092510066 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5c8074280 0x7fd5c806e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:07.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.858+0000 7fd5cfc00640 1 --2- 172.21.15.7:0/2092510066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c806fcf0 0x7fd5c8073d40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:07.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.858+0000 7fd5cfc00640 1 --2- 172.21.15.7:0/2092510066 >> v2:172.21.15.133:3300/0 conn(0x7fd5c8155fb0 0x7fd5c81600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:07.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.858+0000 7fd5cfc00640 1 -- 172.21.15.7:0/2092510066 >> 172.21.15.7:0/2092510066 conn(0x7fd5c80b16d0 msgr2=0x7fd5c80b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:07.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.858+0000 7fd5cfc00640 1 -- 172.21.15.7:0/2092510066 shutdown_connections 2024-09-10T21:42:07.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.858+0000 7fd5cfc00640 1 -- 172.21.15.7:0/2092510066 wait complete. 2024-09-10T21:42:07.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.859+0000 7fd5cfc00640 1 Processor -- start 2024-09-10T21:42:07.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.860+0000 7fd5cfc00640 1 -- start start 2024-09-10T21:42:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.860+0000 7fd5cfc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c806fcf0 0x7fd5c80c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.860+0000 7fd5cfc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5c8074280 0x7fd5c80c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.860+0000 7fd5cfc00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5c8155fb0 0x7fd5c80c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:07.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.860+0000 7fd5cfc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c80cbc80 con 0x7fd5c806fcf0 2024-09-10T21:42:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.860+0000 7fd5cfc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c806eac0 con 0x7fd5c8074280 2024-09-10T21:42:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.860+0000 7fd5cfc00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fd5c80b3a20 con 0x7fd5c8155fb0 2024-09-10T21:42:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.861+0000 7fd5cd200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c806fcf0 0x7fd5c80c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.861+0000 7fd5cd200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c806fcf0 0x7fd5c80c6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45246/0 (socket says 172.21.15.7:45246) 2024-09-10T21:42:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.861+0000 7fd5cd200640 1 -- 172.21.15.7:0/2753504427 learned_addr learned my addr 172.21.15.7:0/2753504427 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.861+0000 7fd5cdc00640 1 --2- 172.21.15.7:0/2753504427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5c8155fb0 0x7fd5c80c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.861+0000 7fd5cd200640 1 -- 172.21.15.7:0/2753504427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5c8155fb0 msgr2=0x7fd5c80c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:07.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.861+0000 7fd5cd200640 1 --2- 172.21.15.7:0/2753504427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5c8155fb0 0x7fd5c80c9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:07.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.861+0000 7fd5cd200640 1 -- 172.21.15.7:0/2753504427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5c8074280 msgr2=0x7fd5c80c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:07.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.861+0000 7fd5cd200640 1 --2- 172.21.15.7:0/2753504427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5c8074280 0x7fd5c80c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:07.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.861+0000 7fd5cd200640 1 -- 172.21.15.7:0/2753504427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c80c99a0 con 0x7fd5c806fcf0 2024-09-10T21:42:07.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.861+0000 7fd5cdc00640 1 --2- 172.21.15.7:0/2753504427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5c8155fb0 0x7fd5c80c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:07.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.861+0000 7fd5cd200640 1 --2- 172.21.15.7:0/2753504427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c806fcf0 0x7fd5c80c6fb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00e7c0 tx=0x7fd5bc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:07.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.862+0000 7fd5c4c00640 1 -- 172.21.15.7:0/2753504427 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc00ccc0 con 0x7fd5c806fcf0 2024-09-10T21:42:07.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.862+0000 7fd5c4c00640 1 -- 172.21.15.7:0/2753504427 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5bc010430 con 0x7fd5c806fcf0 2024-09-10T21:42:07.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.862+0000 7fd5cfc00640 1 -- 172.21.15.7:0/2753504427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c80c9c80 con 0x7fd5c806fcf0 2024-09-10T21:42:07.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.862+0000 7fd5c4c00640 1 -- 172.21.15.7:0/2753504427 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc010860 con 0x7fd5c806fcf0 2024-09-10T21:42:07.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.862+0000 7fd5cfc00640 1 -- 172.21.15.7:0/2753504427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c81f44e0 con 0x7fd5c806fcf0 2024-09-10T21:42:07.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.863+0000 7fd5cfc00640 1 -- 172.21.15.7:0/2753504427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c80b3d80 con 0x7fd5c806fcf0 2024-09-10T21:42:07.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.864+0000 7fd5c4c00640 1 -- 172.21.15.7:0/2753504427 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd5bc010a00 con 0x7fd5c806fcf0 2024-09-10T21:42:07.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.865+0000 7fd5c4c00640 1 --2- 172.21.15.7:0/2753504427 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd5a8077f70 0x7fd5a807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:07.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.865+0000 7fd5c4c00640 1 -- 172.21.15.7:0/2753504427 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7fd5bc09a5c0 con 0x7fd5c806fcf0 2024-09-10T21:42:07.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.865+0000 7fd5c7e00640 1 --2- 172.21.15.7:0/2753504427 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd5a8077f70 0x7fd5a807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:07.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.866+0000 7fd5c7e00640 1 --2- 172.21.15.7:0/2753504427 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd5a8077f70 0x7fd5a807a430 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4033000 tx=0x7fd5b40046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:07.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:07.870+0000 7fd5c4c00640 1 -- 172.21.15.7:0/2753504427 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd5bc063200 con 0x7fd5c806fcf0 2024-09-10T21:42:08.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.067+0000 7fd5cfc00640 1 -- 172.21.15.7:0/2753504427 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fd5c806e580 con 0x7fd5c806fcf0 2024-09-10T21:42:08.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.669+0000 7fd5c4c00640 1 -- 172.21.15.7:0/2753504427 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) ==== 170+0+0 (secure 0 0 0) 0x7fd5bc062b00 con 0x7fd5c806fcf0 2024-09-10T21:42:08.670 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-09-10T21:42:08.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.672+0000 7fd5cfc00640 1 -- 172.21.15.7:0/2753504427 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd5a8077f70 msgr2=0x7fd5a807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:08.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.672+0000 7fd5cfc00640 1 --2- 172.21.15.7:0/2753504427 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd5a8077f70 0x7fd5a807a430 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4033000 tx=0x7fd5b40046c0 comp rx=0 tx=0).stop 2024-09-10T21:42:08.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.672+0000 7fd5cfc00640 1 -- 172.21.15.7:0/2753504427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c806fcf0 msgr2=0x7fd5c80c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:08.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.672+0000 7fd5cfc00640 1 --2- 172.21.15.7:0/2753504427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c806fcf0 0x7fd5c80c6fb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00e7c0 tx=0x7fd5bc00ec80 comp rx=0 tx=0).stop 2024-09-10T21:42:08.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.673+0000 7fd5cfc00640 1 -- 172.21.15.7:0/2753504427 shutdown_connections 2024-09-10T21:42:08.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.673+0000 7fd5cfc00640 1 --2- 172.21.15.7:0/2753504427 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd5a8077f70 0x7fd5a807a430 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:08.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.673+0000 7fd5cfc00640 1 --2- 172.21.15.7:0/2753504427 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd5c8074280 0x7fd5c80c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:08.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.673+0000 7fd5cfc00640 1 --2- 172.21.15.7:0/2753504427 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd5c8155fb0 0x7fd5c80c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:08.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.673+0000 7fd5cfc00640 1 --2- 172.21.15.7:0/2753504427 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd5c806fcf0 0x7fd5c80c6fb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:08.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.673+0000 7fd5cfc00640 1 -- 172.21.15.7:0/2753504427 >> 172.21.15.7:0/2753504427 conn(0x7fd5c80b16d0 msgr2=0x7fd5c80b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:08.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.675+0000 7fd5cfc00640 1 -- 172.21.15.7:0/2753504427 shutdown_connections 2024-09-10T21:42:08.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.676+0000 7fd5cfc00640 1 -- 172.21.15.7:0/2753504427 wait complete. 2024-09-10T21:42:08.710 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:42:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.875+0000 7f5a40800640 1 -- 172.21.15.7:0/3786043253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5a3806fcf0 msgr2=0x7f5a38073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.875+0000 7f5a40800640 1 --2- 172.21.15.7:0/3786043253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5a3806fcf0 0x7f5a38073d40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5a28005fd0 tx=0x7f5a2802a5d0 comp rx=0 tx=0).stop 2024-09-10T21:42:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.875+0000 7f5a40800640 1 -- 172.21.15.7:0/3786043253 shutdown_connections 2024-09-10T21:42:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.875+0000 7f5a40800640 1 --2- 172.21.15.7:0/3786043253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5a3806fcf0 0x7f5a38073d40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.875+0000 7f5a40800640 1 --2- 172.21.15.7:0/3786043253 >> v2:172.21.15.133:3300/0 conn(0x7f5a38155fb0 0x7f5a381600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.875+0000 7f5a40800640 1 -- 172.21.15.7:0/3786043253 >> 172.21.15.7:0/3786043253 conn(0x7f5a380b16d0 msgr2=0x7f5a380b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.875+0000 7f5a40800640 1 -- 172.21.15.7:0/3786043253 shutdown_connections 2024-09-10T21:42:08.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.875+0000 7f5a40800640 1 -- 172.21.15.7:0/3786043253 wait complete. 2024-09-10T21:42:08.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.877+0000 7f5a40800640 1 Processor -- start 2024-09-10T21:42:08.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.877+0000 7f5a40800640 1 -- start start 2024-09-10T21:42:08.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.878+0000 7f5a40800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a3806fcf0 0x7f5a380c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:08.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.878+0000 7f5a40800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a38074280 0x7f5a38148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:08.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.878+0000 7f5a40800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5a38155fb0 0x7f5a380ca1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:08.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.878+0000 7f5a40800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5a38156510 con 0x7f5a38074280 2024-09-10T21:42:08.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.878+0000 7f5a40800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5a38156390 con 0x7f5a3806fcf0 2024-09-10T21:42:08.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.878+0000 7f5a40800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f5a3806ec90 con 0x7f5a38155fb0 2024-09-10T21:42:08.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.878+0000 7f5a3d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a38074280 0x7f5a38148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:08.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.879+0000 7f5a3d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a38074280 0x7f5a38148ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45262/0 (socket says 172.21.15.7:45262) 2024-09-10T21:42:08.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.879+0000 7f5a3de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a3806fcf0 0x7f5a380c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:08.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.879+0000 7f5a3d400640 1 -- 172.21.15.7:0/2158168264 learned_addr learned my addr 172.21.15.7:0/2158168264 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:08.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.879+0000 7f5a3e800640 1 --2- 172.21.15.7:0/2158168264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5a38155fb0 0x7f5a380ca1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:08.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.879+0000 7f5a3de00640 1 -- 172.21.15.7:0/2158168264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5a38155fb0 msgr2=0x7f5a380ca1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.879+0000 7f5a3de00640 1 --2- 172.21.15.7:0/2158168264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5a38155fb0 0x7f5a380ca1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.879+0000 7f5a3de00640 1 -- 172.21.15.7:0/2158168264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a38074280 msgr2=0x7f5a38148ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.879+0000 7f5a3de00640 1 --2- 172.21.15.7:0/2158168264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a38074280 0x7f5a38148ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.879+0000 7f5a3de00640 1 -- 172.21.15.7:0/2158168264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a380c60e0 con 0x7f5a3806fcf0 2024-09-10T21:42:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.879+0000 7f5a3d400640 1 --2- 172.21.15.7:0/2158168264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a38074280 0x7f5a38148ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:42:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.879+0000 7f5a3de00640 1 --2- 172.21.15.7:0/2158168264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a3806fcf0 0x7f5a380c5ae0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00e9f0 tx=0x7f5a2c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.880+0000 7f5a32000640 1 -- 172.21.15.7:0/2158168264 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c00ccc0 con 0x7f5a3806fcf0 2024-09-10T21:42:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.880+0000 7f5a40800640 1 -- 172.21.15.7:0/2158168264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a380c63c0 con 0x7f5a3806fcf0 2024-09-10T21:42:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.880+0000 7f5a32000640 1 -- 172.21.15.7:0/2158168264 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a2c004020 con 0x7f5a3806fcf0 2024-09-10T21:42:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.880+0000 7f5a32000640 1 -- 172.21.15.7:0/2158168264 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c00ce60 con 0x7f5a3806fcf0 2024-09-10T21:42:08.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.880+0000 7f5a40800640 1 -- 172.21.15.7:0/2158168264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a38149bf0 con 0x7f5a3806fcf0 2024-09-10T21:42:08.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.881+0000 7f5a40800640 1 -- 172.21.15.7:0/2158168264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a3806f140 con 0x7f5a3806fcf0 2024-09-10T21:42:08.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.882+0000 7f5a32000640 1 -- 172.21.15.7:0/2158168264 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5a2c03b9c0 con 0x7f5a3806fcf0 2024-09-10T21:42:08.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.883+0000 7f5a32000640 1 --2- 172.21.15.7:0/2158168264 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5a1c077f70 0x7f5a1c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:08.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.883+0000 7f5a32000640 1 -- 172.21.15.7:0/2158168264 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f5a2c09a5b0 con 0x7f5a3806fcf0 2024-09-10T21:42:08.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.884+0000 7f5a3d400640 1 --2- 172.21.15.7:0/2158168264 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5a1c077f70 0x7f5a1c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:08.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.884+0000 7f5a3d400640 1 --2- 172.21.15.7:0/2158168264 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5a1c077f70 0x7f5a1c07a430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5a28000c00 tx=0x7f5a28004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:08.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:08.888+0000 7f5a32000640 1 -- 172.21.15.7:0/2158168264 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5a2c062a20 con 0x7f5a3806fcf0 2024-09-10T21:42:09.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.118+0000 7f5a40800640 1 -- 172.21.15.7:0/2158168264 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5a3815b100 con 0x7f5a3806fcf0 2024-09-10T21:42:09.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.676+0000 7f5a32000640 1 -- 172.21.15.7:0/2158168264 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f5a2c067450 con 0x7f5a3806fcf0 2024-09-10T21:42:09.679 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T21:42:09.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.685+0000 7f5a40800640 1 -- 172.21.15.7:0/2158168264 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5a1c077f70 msgr2=0x7f5a1c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:09.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.685+0000 7f5a40800640 1 --2- 172.21.15.7:0/2158168264 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5a1c077f70 0x7f5a1c07a430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5a28000c00 tx=0x7f5a28004730 comp rx=0 tx=0).stop 2024-09-10T21:42:09.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.685+0000 7f5a40800640 1 -- 172.21.15.7:0/2158168264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a3806fcf0 msgr2=0x7f5a380c5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:09.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.685+0000 7f5a40800640 1 --2- 172.21.15.7:0/2158168264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a3806fcf0 0x7f5a380c5ae0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c00e9f0 tx=0x7f5a2c00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:42:09.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.686+0000 7f5a40800640 1 -- 172.21.15.7:0/2158168264 shutdown_connections 2024-09-10T21:42:09.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.686+0000 7f5a40800640 1 --2- 172.21.15.7:0/2158168264 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5a1c077f70 0x7f5a1c07a430 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:09.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.686+0000 7f5a40800640 1 --2- 172.21.15.7:0/2158168264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5a38155fb0 0x7f5a380ca1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:09.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.686+0000 7f5a40800640 1 --2- 172.21.15.7:0/2158168264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5a38074280 0x7f5a38148ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:09.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.686+0000 7f5a40800640 1 --2- 172.21.15.7:0/2158168264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5a3806fcf0 0x7f5a380c5ae0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:09.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.686+0000 7f5a40800640 1 -- 172.21.15.7:0/2158168264 >> 172.21.15.7:0/2158168264 conn(0x7f5a380b16d0 msgr2=0x7f5a380b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:09.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.688+0000 7f5a40800640 1 -- 172.21.15.7:0/2158168264 shutdown_connections 2024-09-10T21:42:09.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.688+0000 7f5a40800640 1 -- 172.21.15.7:0/2158168264 wait complete. 2024-09-10T21:42:09.726 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:42:09.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.888+0000 7f42d0c00640 1 -- 172.21.15.7:0/517006512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42cc161f80 msgr2=0x7f42cc0b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:09.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.888+0000 7f42d0c00640 1 --2- 172.21.15.7:0/517006512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42cc161f80 0x7f42cc0b3e70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f42bc005fd0 tx=0x7f42bc02ab30 comp rx=0 tx=0).stop 2024-09-10T21:42:09.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.888+0000 7f42d0c00640 1 -- 172.21.15.7:0/517006512 shutdown_connections 2024-09-10T21:42:09.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.889+0000 7f42d0c00640 1 --2- 172.21.15.7:0/517006512 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42cc161f80 0x7f42cc0b3e70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:09.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.889+0000 7f42d0c00640 1 -- 172.21.15.7:0/517006512 >> 172.21.15.7:0/517006512 conn(0x7f42cc09d260 msgr2=0x7f42cc0d3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:09.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.889+0000 7f42d0c00640 1 -- 172.21.15.7:0/517006512 shutdown_connections 2024-09-10T21:42:09.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.889+0000 7f42d0c00640 1 -- 172.21.15.7:0/517006512 wait complete. 2024-09-10T21:42:09.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.890+0000 7f42d0c00640 1 Processor -- start 2024-09-10T21:42:09.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.890+0000 7f42d0c00640 1 -- start start 2024-09-10T21:42:09.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.891+0000 7f42d0c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42cc0b43b0 0x7f42cc1f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:09.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.891+0000 7f42d0c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42cc161f80 0x7f42cc15fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:09.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.891+0000 7f42d0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42cc162940 0x7f42cc0b8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:09.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.891+0000 7f42d0c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42cc0b1340 con 0x7f42cc162940 2024-09-10T21:42:09.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.891+0000 7f42d0c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f42cc162680 con 0x7f42cc0b43b0 2024-09-10T21:42:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.891+0000 7f42d0c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f42cc0b14c0 con 0x7f42cc161f80 2024-09-10T21:42:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.891+0000 7f42ca800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42cc162940 0x7f42cc0b8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.892+0000 7f42ca800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42cc162940 0x7f42cc0b8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45278/0 (socket says 172.21.15.7:45278) 2024-09-10T21:42:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.892+0000 7f42ca800640 1 -- 172.21.15.7:0/3742227523 learned_addr learned my addr 172.21.15.7:0/3742227523 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.892+0000 7f42c9e00640 1 --2- 172.21.15.7:0/3742227523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42cc0b43b0 0x7f42cc1f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.892+0000 7f42c1400640 1 --2- 172.21.15.7:0/3742227523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42cc161f80 0x7f42cc15fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.892+0000 7f42ca800640 1 -- 172.21.15.7:0/3742227523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42cc161f80 msgr2=0x7f42cc15fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.892+0000 7f42ca800640 1 --2- 172.21.15.7:0/3742227523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42cc161f80 0x7f42cc15fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.892+0000 7f42ca800640 1 -- 172.21.15.7:0/3742227523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42cc0b43b0 msgr2=0x7f42cc1f8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.892+0000 7f42ca800640 1 --2- 172.21.15.7:0/3742227523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42cc0b43b0 0x7f42cc1f8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.892+0000 7f42ca800640 1 -- 172.21.15.7:0/3742227523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42cc0b9750 con 0x7f42cc162940 2024-09-10T21:42:09.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.892+0000 7f42ca800640 1 --2- 172.21.15.7:0/3742227523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42cc162940 0x7f42cc0b8fe0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f42c400c400 tx=0x7f42c400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.892+0000 7f42c2a00640 1 -- 172.21.15.7:0/3742227523 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c4017070 con 0x7f42cc162940 2024-09-10T21:42:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.893+0000 7f42c2a00640 1 -- 172.21.15.7:0/3742227523 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42c40053c0 con 0x7f42cc162940 2024-09-10T21:42:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.893+0000 7f42d0c00640 1 -- 172.21.15.7:0/3742227523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42cc0b9a30 con 0x7f42cc162940 2024-09-10T21:42:09.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.893+0000 7f42c2a00640 1 -- 172.21.15.7:0/3742227523 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c4003850 con 0x7f42cc162940 2024-09-10T21:42:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.893+0000 7f42d0c00640 1 -- 172.21.15.7:0/3742227523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42cc0cfb70 con 0x7f42cc162940 2024-09-10T21:42:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.894+0000 7f42d0c00640 1 -- 172.21.15.7:0/3742227523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42cc0b3e70 con 0x7f42cc162940 2024-09-10T21:42:09.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.895+0000 7f42c2a00640 1 -- 172.21.15.7:0/3742227523 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f42c40039f0 con 0x7f42cc162940 2024-09-10T21:42:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.896+0000 7f42c2a00640 1 --2- 172.21.15.7:0/3742227523 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f42a8077f70 0x7f42a807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.896+0000 7f42c2a00640 1 -- 172.21.15.7:0/3742227523 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7f42c40994b0 con 0x7f42cc162940 2024-09-10T21:42:09.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.896+0000 7f42c9e00640 1 --2- 172.21.15.7:0/3742227523 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f42a8077f70 0x7f42a807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:09.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.897+0000 7f42c9e00640 1 --2- 172.21.15.7:0/3742227523 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f42a8077f70 0x7f42a807a430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f42bc035000 tx=0x7f42bc038040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:09.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:09.901+0000 7f42c2a00640 1 -- 172.21.15.7:0/3742227523 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f42c4061840 con 0x7f42cc162940 2024-09-10T21:42:10.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.096+0000 7f42d0c00640 1 -- 172.21.15.7:0/3742227523 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42cc08d6e0 con 0x7f42cc162940 2024-09-10T21:42:10.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.097+0000 7f42c2a00640 1 -- 172.21.15.7:0/3742227523 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16642 (secure 0 0 0) 0x7f42c4066270 con 0x7f42cc162940 2024-09-10T21:42:10.099 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:10.099 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:42:09.121054+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:42:07.149267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:42:08.077083+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:42:10.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.100+0000 7f42d0c00640 1 -- 172.21.15.7:0/3742227523 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f42a8077f70 msgr2=0x7f42a807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:10.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.100+0000 7f42d0c00640 1 --2- 172.21.15.7:0/3742227523 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f42a8077f70 0x7f42a807a430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f42bc035000 tx=0x7f42bc038040 comp rx=0 tx=0).stop 2024-09-10T21:42:10.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.100+0000 7f42d0c00640 1 -- 172.21.15.7:0/3742227523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42cc162940 msgr2=0x7f42cc0b8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:10.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.100+0000 7f42d0c00640 1 --2- 172.21.15.7:0/3742227523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42cc162940 0x7f42cc0b8fe0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f42c400c400 tx=0x7f42c400c8c0 comp rx=0 tx=0).stop 2024-09-10T21:42:10.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.100+0000 7f42d0c00640 1 -- 172.21.15.7:0/3742227523 shutdown_connections 2024-09-10T21:42:10.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.100+0000 7f42d0c00640 1 --2- 172.21.15.7:0/3742227523 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f42a8077f70 0x7f42a807a430 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:10.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.100+0000 7f42d0c00640 1 --2- 172.21.15.7:0/3742227523 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42cc162940 0x7f42cc0b8fe0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:10.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.100+0000 7f42d0c00640 1 --2- 172.21.15.7:0/3742227523 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42cc161f80 0x7f42cc15fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:10.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.100+0000 7f42d0c00640 1 --2- 172.21.15.7:0/3742227523 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42cc0b43b0 0x7f42cc1f8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:10.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.100+0000 7f42d0c00640 1 -- 172.21.15.7:0/3742227523 >> 172.21.15.7:0/3742227523 conn(0x7f42cc09d260 msgr2=0x7f42cc0d1b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:10.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.101+0000 7f42d0c00640 1 -- 172.21.15.7:0/3742227523 shutdown_connections 2024-09-10T21:42:10.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.101+0000 7f42d0c00640 1 -- 172.21.15.7:0/3742227523 wait complete. 2024-09-10T21:42:10.117 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:42:10.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.276+0000 7fac21e00640 1 -- 172.21.15.7:0/1560599286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac1c0ae020 msgr2=0x7fac1c0ae4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.276+0000 7fac21e00640 1 --2- 172.21.15.7:0/1560599286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac1c0ae020 0x7fac1c0ae4d0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fac14005fd0 tx=0x7fac1402a840 comp rx=0 tx=0).stop 2024-09-10T21:42:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.276+0000 7fac21e00640 1 -- 172.21.15.7:0/1560599286 shutdown_connections 2024-09-10T21:42:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.276+0000 7fac21e00640 1 --2- 172.21.15.7:0/1560599286 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac1c0ae020 0x7fac1c0ae4d0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.276+0000 7fac21e00640 1 --2- 172.21.15.7:0/1560599286 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac1c0cbd50 0x7fac1c0cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.276+0000 7fac21e00640 1 -- 172.21.15.7:0/1560599286 >> 172.21.15.7:0/1560599286 conn(0x7fac1c0b16d0 msgr2=0x7fac1c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.276+0000 7fac21e00640 1 -- 172.21.15.7:0/1560599286 shutdown_connections 2024-09-10T21:42:10.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.276+0000 7fac21e00640 1 -- 172.21.15.7:0/1560599286 wait complete. 2024-09-10T21:42:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.278+0000 7fac21e00640 1 Processor -- start 2024-09-10T21:42:10.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.278+0000 7fac21e00640 1 -- start start 2024-09-10T21:42:10.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.279+0000 7fac21e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac1c0ae020 0x7fac1c0c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:10.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.279+0000 7fac21e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac1c0cbd50 0x7fac1c0b5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:10.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.279+0000 7fac21e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac1c1619e0 0x7fac1c0c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:10.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.279+0000 7fac21e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c0ce100 con 0x7fac1c0cbd50 2024-09-10T21:42:10.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.279+0000 7fac21e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c0cdf80 con 0x7fac1c0ae020 2024-09-10T21:42:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.279+0000 7fac21e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fac1c161dc0 con 0x7fac1c1619e0 2024-09-10T21:42:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.279+0000 7fac12000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac1c0cbd50 0x7fac1c0b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.279+0000 7fac12000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac1c0cbd50 0x7fac1c0b5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45280/0 (socket says 172.21.15.7:45280) 2024-09-10T21:42:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.279+0000 7fac12000640 1 -- 172.21.15.7:0/1827256152 learned_addr learned my addr 172.21.15.7:0/1827256152 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.279+0000 7fac12000640 1 -- 172.21.15.7:0/1827256152 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac1c1619e0 msgr2=0x7fac1c0c6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.279+0000 7fac1b400640 1 --2- 172.21.15.7:0/1827256152 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac1c1619e0 0x7fac1c0c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.279+0000 7fac12000640 1 --2- 172.21.15.7:0/1827256152 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac1c1619e0 0x7fac1c0c6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.279+0000 7fac1aa00640 1 --2- 172.21.15.7:0/1827256152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac1c0ae020 0x7fac1c0c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.279+0000 7fac12000640 1 -- 172.21.15.7:0/1827256152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac1c0ae020 msgr2=0x7fac1c0c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.279+0000 7fac12000640 1 --2- 172.21.15.7:0/1827256152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac1c0ae020 0x7fac1c0c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.279+0000 7fac12000640 1 -- 172.21.15.7:0/1827256152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c1f4e20 con 0x7fac1c0cbd50 2024-09-10T21:42:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.279+0000 7fac1b400640 1 --2- 172.21.15.7:0/1827256152 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac1c1619e0 0x7fac1c0c6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:10.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.279+0000 7fac1aa00640 1 --2- 172.21.15.7:0/1827256152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac1c0ae020 0x7fac1c0c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.280+0000 7fac12000640 1 --2- 172.21.15.7:0/1827256152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac1c0cbd50 0x7fac1c0b5140 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fac0400e9f0 tx=0x7fac0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.280+0000 7fac13e00640 1 -- 172.21.15.7:0/1827256152 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac0400cd80 con 0x7fac1c0cbd50 2024-09-10T21:42:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.280+0000 7fac13e00640 1 -- 172.21.15.7:0/1827256152 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fac04004020 con 0x7fac1c0cbd50 2024-09-10T21:42:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.280+0000 7fac13e00640 1 -- 172.21.15.7:0/1827256152 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac04004430 con 0x7fac1c0cbd50 2024-09-10T21:42:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.280+0000 7fac21e00640 1 -- 172.21.15.7:0/1827256152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1c1f5100 con 0x7fac1c0cbd50 2024-09-10T21:42:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.280+0000 7fac21e00640 1 -- 172.21.15.7:0/1827256152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac1c1f56c0 con 0x7fac1c0cbd50 2024-09-10T21:42:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.281+0000 7fac21e00640 1 -- 172.21.15.7:0/1827256152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac1c0ae4d0 con 0x7fac1c0cbd50 2024-09-10T21:42:10.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.282+0000 7fac13e00640 1 -- 172.21.15.7:0/1827256152 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fac040035f0 con 0x7fac1c0cbd50 2024-09-10T21:42:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.283+0000 7fac13e00640 1 --2- 172.21.15.7:0/1827256152 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fabf8077f70 0x7fabf807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:10.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.283+0000 7fac13e00640 1 -- 172.21.15.7:0/1827256152 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7fac0409a8e0 con 0x7fac1c0cbd50 2024-09-10T21:42:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.283+0000 7fac1aa00640 1 --2- 172.21.15.7:0/1827256152 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fabf8077f70 0x7fabf807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:10.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.284+0000 7fac1aa00640 1 --2- 172.21.15.7:0/1827256152 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fabf8077f70 0x7fabf807a430 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fac0c001860 tx=0x7fac0c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:10.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.287+0000 7fac13e00640 1 -- 172.21.15.7:0/1827256152 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fac04063400 con 0x7fac1c0cbd50 2024-09-10T21:42:10.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.485+0000 7fac21e00640 1 -- 172.21.15.7:0/1827256152 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac1c0ae7a0 con 0x7fac1c0cbd50 2024-09-10T21:42:10.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.486+0000 7fac13e00640 1 -- 172.21.15.7:0/1827256152 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) ==== 74+0+16642 (secure 0 0 0) 0x7fac04062d00 con 0x7fac1c0cbd50 2024-09-10T21:42:10.487 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:10.487 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":20,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:42:09.121054+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:42:07.149267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:42:08.077083+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:42:10.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.488+0000 7fac21e00640 1 -- 172.21.15.7:0/1827256152 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fabf8077f70 msgr2=0x7fabf807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:10.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.488+0000 7fac21e00640 1 --2- 172.21.15.7:0/1827256152 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fabf8077f70 0x7fabf807a430 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fac0c001860 tx=0x7fac0c009040 comp rx=0 tx=0).stop 2024-09-10T21:42:10.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.488+0000 7fac21e00640 1 -- 172.21.15.7:0/1827256152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac1c0cbd50 msgr2=0x7fac1c0b5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:10.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.488+0000 7fac21e00640 1 --2- 172.21.15.7:0/1827256152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac1c0cbd50 0x7fac1c0b5140 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fac0400e9f0 tx=0x7fac0400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:42:10.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.488+0000 7fac21e00640 1 -- 172.21.15.7:0/1827256152 shutdown_connections 2024-09-10T21:42:10.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.488+0000 7fac21e00640 1 --2- 172.21.15.7:0/1827256152 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fabf8077f70 0x7fabf807a430 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:10.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.488+0000 7fac21e00640 1 --2- 172.21.15.7:0/1827256152 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fac1c1619e0 0x7fac1c0c6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:10.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.488+0000 7fac21e00640 1 --2- 172.21.15.7:0/1827256152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fac1c0cbd50 0x7fac1c0b5140 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:10.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.488+0000 7fac21e00640 1 --2- 172.21.15.7:0/1827256152 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fac1c0ae020 0x7fac1c0c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:10.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.488+0000 7fac21e00640 1 -- 172.21.15.7:0/1827256152 >> 172.21.15.7:0/1827256152 conn(0x7fac1c0b16d0 msgr2=0x7fac1c14ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:10.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.488+0000 7fac21e00640 1 -- 172.21.15.7:0/1827256152 shutdown_connections 2024-09-10T21:42:10.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.489+0000 7fac21e00640 1 -- 172.21.15.7:0/1827256152 wait complete. 2024-09-10T21:42:10.505 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:42:10.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.667+0000 7ff439c00640 1 -- 172.21.15.7:0/467077133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4340d22b0 msgr2=0x7ff4340d26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:10.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.667+0000 7ff439c00640 1 --2- 172.21.15.7:0/467077133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4340d22b0 0x7ff4340d26f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff41c005fd0 tx=0x7ff41c02a5d0 comp rx=0 tx=0).stop 2024-09-10T21:42:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.668+0000 7ff439c00640 1 -- 172.21.15.7:0/467077133 shutdown_connections 2024-09-10T21:42:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.668+0000 7ff439c00640 1 --2- 172.21.15.7:0/467077133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4340d22b0 0x7ff4340d26f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.668+0000 7ff439c00640 1 --2- 172.21.15.7:0/467077133 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff43415df70 0x7ff4340b6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.668+0000 7ff439c00640 1 -- 172.21.15.7:0/467077133 >> 172.21.15.7:0/467077133 conn(0x7ff4340d3e80 msgr2=0x7ff4340d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.668+0000 7ff439c00640 1 -- 172.21.15.7:0/467077133 shutdown_connections 2024-09-10T21:42:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.668+0000 7ff439c00640 1 -- 172.21.15.7:0/467077133 wait complete. 2024-09-10T21:42:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.669+0000 7ff439c00640 1 Processor -- start 2024-09-10T21:42:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.669+0000 7ff439c00640 1 -- start start 2024-09-10T21:42:10.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.670+0000 7ff439c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4340b0a10 0x7ff43414d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:10.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.670+0000 7ff439c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4340d22b0 0x7ff4340b2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:10.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.670+0000 7ff439c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff43415df70 0x7ff4340d0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:10.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.670+0000 7ff439c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4340b2960 con 0x7ff43415df70 2024-09-10T21:42:10.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.670+0000 7ff439c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff4340b27e0 con 0x7ff4340d22b0 2024-09-10T21:42:10.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.670+0000 7ff439c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ff4340b2ae0 con 0x7ff4340b0a10 2024-09-10T21:42:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.670+0000 7ff439200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff43415df70 0x7ff4340d0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.670+0000 7ff439200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff43415df70 0x7ff4340d0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45304/0 (socket says 172.21.15.7:45304) 2024-09-10T21:42:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.671+0000 7ff439200640 1 -- 172.21.15.7:0/2235843905 learned_addr learned my addr 172.21.15.7:0/2235843905 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.671+0000 7ff433e00640 1 --2- 172.21.15.7:0/2235843905 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4340b0a10 0x7ff43414d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.671+0000 7ff42be00640 1 --2- 172.21.15.7:0/2235843905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4340d22b0 0x7ff4340b2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.671+0000 7ff439200640 1 -- 172.21.15.7:0/2235843905 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4340b0a10 msgr2=0x7ff43414d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.671+0000 7ff439200640 1 --2- 172.21.15.7:0/2235843905 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4340b0a10 0x7ff43414d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.671+0000 7ff439200640 1 -- 172.21.15.7:0/2235843905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4340d22b0 msgr2=0x7ff4340b2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.671+0000 7ff439200640 1 --2- 172.21.15.7:0/2235843905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4340d22b0 0x7ff4340b2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.671+0000 7ff439200640 1 -- 172.21.15.7:0/2235843905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff434150070 con 0x7ff43415df70 2024-09-10T21:42:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.671+0000 7ff42be00640 1 --2- 172.21.15.7:0/2235843905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4340d22b0 0x7ff4340b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:10.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.671+0000 7ff439200640 1 --2- 172.21.15.7:0/2235843905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff43415df70 0x7ff4340d0c70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00c7d0 tx=0x7ff42c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.671+0000 7ff430c00640 1 -- 172.21.15.7:0/2235843905 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c003b70 con 0x7ff43415df70 2024-09-10T21:42:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.671+0000 7ff430c00640 1 -- 172.21.15.7:0/2235843905 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff42c003d30 con 0x7ff43415df70 2024-09-10T21:42:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.671+0000 7ff430c00640 1 -- 172.21.15.7:0/2235843905 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c00aab0 con 0x7ff43415df70 2024-09-10T21:42:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.671+0000 7ff439c00640 1 -- 172.21.15.7:0/2235843905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff434150350 con 0x7ff43415df70 2024-09-10T21:42:10.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.671+0000 7ff439c00640 1 -- 172.21.15.7:0/2235843905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff434150910 con 0x7ff43415df70 2024-09-10T21:42:10.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.672+0000 7ff439c00640 1 -- 172.21.15.7:0/2235843905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4340d26f0 con 0x7ff43415df70 2024-09-10T21:42:10.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.674+0000 7ff430c00640 1 -- 172.21.15.7:0/2235843905 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff42c00ac50 con 0x7ff43415df70 2024-09-10T21:42:10.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.674+0000 7ff430c00640 1 --2- 172.21.15.7:0/2235843905 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff410077f70 0x7ff41007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:10.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.675+0000 7ff430c00640 1 -- 172.21.15.7:0/2235843905 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff42c099ad0 con 0x7ff43415df70 2024-09-10T21:42:10.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.675+0000 7ff433e00640 1 --2- 172.21.15.7:0/2235843905 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff410077f70 0x7ff41007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:10.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.676+0000 7ff433e00640 1 --2- 172.21.15.7:0/2235843905 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff410077f70 0x7ff41007a430 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff42400bb50 tx=0x7ff42400b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:10.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.679+0000 7ff430c00640 1 -- 172.21.15.7:0/2235843905 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff42c0625c0 con 0x7ff43415df70 2024-09-10T21:42:10.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:10.910+0000 7ff439c00640 1 -- 172.21.15.7:0/2235843905 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ff43414dc00 con 0x7ff43415df70 2024-09-10T21:42:11.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.705+0000 7ff430c00640 1 -- 172.21.15.7:0/2235843905 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7ff42c061ec0 con 0x7ff43415df70 2024-09-10T21:42:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.708+0000 7ff439c00640 1 -- 172.21.15.7:0/2235843905 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff410077f70 msgr2=0x7ff41007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.708+0000 7ff439c00640 1 --2- 172.21.15.7:0/2235843905 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff410077f70 0x7ff41007a430 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff42400bb50 tx=0x7ff42400b2e0 comp rx=0 tx=0).stop 2024-09-10T21:42:11.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.708+0000 7ff439c00640 1 -- 172.21.15.7:0/2235843905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff43415df70 msgr2=0x7ff4340d0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.708+0000 7ff439c00640 1 --2- 172.21.15.7:0/2235843905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff43415df70 0x7ff4340d0c70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00c7d0 tx=0x7ff42c00cc90 comp rx=0 tx=0).stop 2024-09-10T21:42:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.708+0000 7ff439c00640 1 -- 172.21.15.7:0/2235843905 shutdown_connections 2024-09-10T21:42:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.708+0000 7ff439c00640 1 --2- 172.21.15.7:0/2235843905 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff410077f70 0x7ff41007a430 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.708+0000 7ff439c00640 1 --2- 172.21.15.7:0/2235843905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4340d22b0 0x7ff4340b2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.708+0000 7ff439c00640 1 --2- 172.21.15.7:0/2235843905 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff43415df70 0x7ff4340d0c70 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.708+0000 7ff439c00640 1 --2- 172.21.15.7:0/2235843905 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4340b0a10 0x7ff43414d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.708+0000 7ff439c00640 1 -- 172.21.15.7:0/2235843905 >> 172.21.15.7:0/2235843905 conn(0x7ff4340d3e80 msgr2=0x7ff4340b7450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.708+0000 7ff439c00640 1 -- 172.21.15.7:0/2235843905 shutdown_connections 2024-09-10T21:42:11.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.708+0000 7ff439c00640 1 -- 172.21.15.7:0/2235843905 wait complete. 2024-09-10T21:42:11.727 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:42:11.728 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:42:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.894+0000 7f99a2a00640 1 -- 172.21.15.7:0/2835641284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9980004d40 msgr2=0x7f99800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.894+0000 7f99a2a00640 1 --2- 172.21.15.7:0/2835641284 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9980004d40 0x7f99800051c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f999c06f1b0 tx=0x7f9980005830 comp rx=0 tx=0).stop 2024-09-10T21:42:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.894+0000 7f99a0a00640 1 -- 172.21.15.7:0/2835641284 reap_dead start 2024-09-10T21:42:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.894+0000 7f99a2a00640 1 -- 172.21.15.7:0/2835641284 shutdown_connections 2024-09-10T21:42:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.894+0000 7f99a2a00640 1 -- 172.21.15.7:0/2835641284 >> 172.21.15.7:0/2835641284 conn(0x7f999c0d1db0 msgr2=0x7f999c0d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.894+0000 7f99a2a00640 1 -- 172.21.15.7:0/2835641284 shutdown_connections 2024-09-10T21:42:11.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.894+0000 7f99a2a00640 1 -- 172.21.15.7:0/2835641284 wait complete. 2024-09-10T21:42:11.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.896+0000 7f99a2a00640 1 Processor -- start 2024-09-10T21:42:11.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.896+0000 7f99a2a00640 1 -- start start 2024-09-10T21:42:11.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.897+0000 7f99a2a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9980004d40 0x7f999c0b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.897+0000 7f99a2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f999c1f82a0 0x7f999c15d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.897+0000 7f99a2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999c1f4cc0 0x7f999c15fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.897+0000 7f99a2a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f999c0cf570 con 0x7f999c1f4cc0 2024-09-10T21:42:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.897+0000 7f99a2a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f999c0cf980 con 0x7f999c1f82a0 2024-09-10T21:42:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.897+0000 7f99a2a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f999c0cf6f0 con 0x7f9980004d40 2024-09-10T21:42:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.897+0000 7f99a0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999c1f4cc0 0x7f999c15fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.897+0000 7f99a0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999c1f4cc0 0x7f999c15fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45324/0 (socket says 172.21.15.7:45324) 2024-09-10T21:42:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.897+0000 7f99a0a00640 1 -- 172.21.15.7:0/3157454873 learned_addr learned my addr 172.21.15.7:0/3157454873 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.897+0000 7f9993400640 1 --2- 172.21.15.7:0/3157454873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f999c1f82a0 0x7f999c15d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.898+0000 7f999be00640 1 --2- 172.21.15.7:0/3157454873 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9980004d40 0x7f999c0b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.898+0000 7f99a0a00640 1 -- 172.21.15.7:0/3157454873 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9980004d40 msgr2=0x7f999c0b4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:11.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.898+0000 7f99a0a00640 1 --2- 172.21.15.7:0/3157454873 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9980004d40 0x7f999c0b4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.898+0000 7f99a0a00640 1 -- 172.21.15.7:0/3157454873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f999c1f82a0 msgr2=0x7f999c15d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.898+0000 7f99a0a00640 1 --2- 172.21.15.7:0/3157454873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f999c1f82a0 0x7f999c15d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.898+0000 7f99a0a00640 1 -- 172.21.15.7:0/3157454873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f999c0b6690 con 0x7f999c1f4cc0 2024-09-10T21:42:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.898+0000 7f99a0a00640 1 --2- 172.21.15.7:0/3157454873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999c1f4cc0 0x7f999c15fc80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f999402b4b0 tx=0x7f9994001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.898+0000 7f9998c00640 1 -- 172.21.15.7:0/3157454873 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9994039070 con 0x7f999c1f4cc0 2024-09-10T21:42:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.898+0000 7f9998c00640 1 -- 172.21.15.7:0/3157454873 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f999402bcc0 con 0x7f999c1f4cc0 2024-09-10T21:42:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.898+0000 7f9998c00640 1 -- 172.21.15.7:0/3157454873 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999400c280 con 0x7f999c1f4cc0 2024-09-10T21:42:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.898+0000 7f99a2a00640 1 -- 172.21.15.7:0/3157454873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f999c0b5b10 con 0x7f999c1f4cc0 2024-09-10T21:42:11.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.898+0000 7f99a2a00640 1 -- 172.21.15.7:0/3157454873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f999c0b5550 con 0x7f999c1f4cc0 2024-09-10T21:42:11.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.900+0000 7f99a2a00640 1 -- 172.21.15.7:0/3157454873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f999c0ce990 con 0x7f999c1f4cc0 2024-09-10T21:42:11.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.900+0000 7f9998c00640 1 -- 172.21.15.7:0/3157454873 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9994045030 con 0x7f999c1f4cc0 2024-09-10T21:42:11.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.901+0000 7f9998c00640 1 --2- 172.21.15.7:0/3157454873 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9978077fa0 0x7f997807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:11.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.901+0000 7f9998c00640 1 -- 172.21.15.7:0/3157454873 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f99940b9370 con 0x7f999c1f4cc0 2024-09-10T21:42:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.902+0000 7f999be00640 1 --2- 172.21.15.7:0/3157454873 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9978077fa0 0x7f997807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:11.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.903+0000 7f999be00640 1 --2- 172.21.15.7:0/3157454873 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9978077fa0 0x7f997807a460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f998c009b90 tx=0x7f998c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:11.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:11.905+0000 7f9998c00640 1 -- 172.21.15.7:0/3157454873 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9994081f70 con 0x7f999c1f4cc0 2024-09-10T21:42:12.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.136+0000 7f99a2a00640 1 -- 172.21.15.7:0/3157454873 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f999c0ce1d0 con 0x7f999c1f4cc0 2024-09-10T21:42:12.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.711+0000 7f9998c00640 1 -- 172.21.15.7:0/3157454873 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f9994081870 con 0x7f999c1f4cc0 2024-09-10T21:42:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.714+0000 7f99a2a00640 1 -- 172.21.15.7:0/3157454873 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9978077fa0 msgr2=0x7f997807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.714+0000 7f99a2a00640 1 --2- 172.21.15.7:0/3157454873 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9978077fa0 0x7f997807a460 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f998c009b90 tx=0x7f998c005f90 comp rx=0 tx=0).stop 2024-09-10T21:42:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.714+0000 7f99a2a00640 1 -- 172.21.15.7:0/3157454873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999c1f4cc0 msgr2=0x7f999c15fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.714+0000 7f99a2a00640 1 --2- 172.21.15.7:0/3157454873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999c1f4cc0 0x7f999c15fc80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f999402b4b0 tx=0x7f9994001aa0 comp rx=0 tx=0).stop 2024-09-10T21:42:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.714+0000 7f99a2a00640 1 -- 172.21.15.7:0/3157454873 shutdown_connections 2024-09-10T21:42:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.714+0000 7f99a2a00640 1 --2- 172.21.15.7:0/3157454873 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9978077fa0 0x7f997807a460 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.714+0000 7f99a2a00640 1 --2- 172.21.15.7:0/3157454873 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f999c1f82a0 0x7f999c15d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.714+0000 7f99a2a00640 1 --2- 172.21.15.7:0/3157454873 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f999c1f4cc0 0x7f999c15fc80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.714+0000 7f99a2a00640 1 --2- 172.21.15.7:0/3157454873 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9980004d40 0x7f999c0b4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.714+0000 7f99a2a00640 1 -- 172.21.15.7:0/3157454873 >> 172.21.15.7:0/3157454873 conn(0x7f999c0d1db0 msgr2=0x7f999c15e430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:12.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.714+0000 7f99a2a00640 1 -- 172.21.15.7:0/3157454873 shutdown_connections 2024-09-10T21:42:12.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.714+0000 7f99a2a00640 1 -- 172.21.15.7:0/3157454873 wait complete. 2024-09-10T21:42:12.734 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:42:12.734 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:42:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.899+0000 7f9a4ca00640 1 -- 172.21.15.7:0/3342547867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a2c0053c0 msgr2=0x7f9a2c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.899+0000 7f9a4ca00640 1 --2- 172.21.15.7:0/3342547867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a2c0053c0 0x7f9a2c00b9c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9a40005fd0 tx=0x7f9a2c005830 comp rx=0 tx=0).stop 2024-09-10T21:42:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.899+0000 7f9a4ca00640 1 -- 172.21.15.7:0/3342547867 shutdown_connections 2024-09-10T21:42:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.899+0000 7f9a4ca00640 1 --2- 172.21.15.7:0/3342547867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a2c0053c0 0x7f9a2c00b9c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.899+0000 7f9a4ca00640 1 -- 172.21.15.7:0/3342547867 >> 172.21.15.7:0/3342547867 conn(0x7f9a480b16d0 msgr2=0x7f9a480b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.899+0000 7f9a4ca00640 1 -- 172.21.15.7:0/3342547867 shutdown_connections 2024-09-10T21:42:12.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.899+0000 7f9a4ca00640 1 -- 172.21.15.7:0/3342547867 wait complete. 2024-09-10T21:42:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.901+0000 7f9a4ca00640 1 Processor -- start 2024-09-10T21:42:12.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.901+0000 7f9a4ca00640 1 -- start start 2024-09-10T21:42:12.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.902+0000 7f9a4ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a2c0053c0 0x7f9a48154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:12.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.902+0000 7f9a4ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a48153e40 0x7f9a480ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:12.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.902+0000 7f9a4ca00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9a480ca710 0x7f9a48151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:12.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.902+0000 7f9a4ca00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a480cd2d0 con 0x7f9a2c0053c0 2024-09-10T21:42:12.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.902+0000 7f9a4ca00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9a480cd150 con 0x7f9a48153e40 2024-09-10T21:42:12.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.902+0000 7f9a4ca00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f9a4815fda0 con 0x7f9a480ca710 2024-09-10T21:42:12.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.902+0000 7f9a45e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a2c0053c0 0x7f9a48154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:12.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.902+0000 7f9a45e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a2c0053c0 0x7f9a48154a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45340/0 (socket says 172.21.15.7:45340) 2024-09-10T21:42:12.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.902+0000 7f9a45e00640 1 -- 172.21.15.7:0/2184052775 learned_addr learned my addr 172.21.15.7:0/2184052775 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:12.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.902+0000 7f9a46800640 1 --2- 172.21.15.7:0/2184052775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9a480ca710 0x7f9a48151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:12.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.902+0000 7f9a3fe00640 1 --2- 172.21.15.7:0/2184052775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a48153e40 0x7f9a480ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:12.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.902+0000 7f9a45e00640 1 -- 172.21.15.7:0/2184052775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9a480ca710 msgr2=0x7f9a48151b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:12.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.902+0000 7f9a45e00640 1 --2- 172.21.15.7:0/2184052775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9a480ca710 0x7f9a48151b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:12.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.902+0000 7f9a45e00640 1 -- 172.21.15.7:0/2184052775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a48153e40 msgr2=0x7f9a480ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:12.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.902+0000 7f9a45e00640 1 --2- 172.21.15.7:0/2184052775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a48153e40 0x7f9a480ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:12.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.902+0000 7f9a45e00640 1 -- 172.21.15.7:0/2184052775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a480cabb0 con 0x7f9a2c0053c0 2024-09-10T21:42:12.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.902+0000 7f9a3fe00640 1 --2- 172.21.15.7:0/2184052775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a48153e40 0x7f9a480ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.903+0000 7f9a45e00640 1 --2- 172.21.15.7:0/2184052775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a2c0053c0 0x7f9a48154a80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9a38003ed0 tx=0x7f9a38001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.903+0000 7f9a3e000640 1 -- 172.21.15.7:0/2184052775 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38015070 con 0x7f9a2c0053c0 2024-09-10T21:42:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.903+0000 7f9a3e000640 1 -- 172.21.15.7:0/2184052775 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a38011840 con 0x7f9a2c0053c0 2024-09-10T21:42:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.903+0000 7f9a3e000640 1 -- 172.21.15.7:0/2184052775 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38011c90 con 0x7f9a2c0053c0 2024-09-10T21:42:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.903+0000 7f9a4ca00640 1 -- 172.21.15.7:0/2184052775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a480cae90 con 0x7f9a2c0053c0 2024-09-10T21:42:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.903+0000 7f9a4ca00640 1 -- 172.21.15.7:0/2184052775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a480cb450 con 0x7f9a2c0053c0 2024-09-10T21:42:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.904+0000 7f9a4ca00640 1 -- 172.21.15.7:0/2184052775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a48073730 con 0x7f9a2c0053c0 2024-09-10T21:42:12.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.905+0000 7f9a3e000640 1 -- 172.21.15.7:0/2184052775 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9a380103c0 con 0x7f9a2c0053c0 2024-09-10T21:42:12.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.906+0000 7f9a3e000640 1 --2- 172.21.15.7:0/2184052775 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9a24077f70 0x7f9a2407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:12.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.906+0000 7f9a3e000640 1 -- 172.21.15.7:0/2184052775 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f9a3809a7e0 con 0x7f9a2c0053c0 2024-09-10T21:42:12.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.907+0000 7f9a3fe00640 1 --2- 172.21.15.7:0/2184052775 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9a24077f70 0x7f9a2407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:12.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.907+0000 7f9a3fe00640 1 --2- 172.21.15.7:0/2184052775 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9a24077f70 0x7f9a2407a430 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9a30000fb0 tx=0x7f9a300067a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:12.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:12.910+0000 7f9a3e000640 1 -- 172.21.15.7:0/2184052775 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9a38062e00 con 0x7f9a2c0053c0 2024-09-10T21:42:13.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.141+0000 7f9a4ca00640 1 -- 172.21.15.7:0/2184052775 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a48072be0 con 0x7f9a2c0053c0 2024-09-10T21:42:13.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.141+0000 7f9a3e000640 1 -- 172.21.15.7:0/2184052775 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5227 (secure 0 0 0) 0x7f9a38067830 con 0x7f9a2c0053c0 2024-09-10T21:42:13.143 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:13.143 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"btime":"2024-09-10T21:42:12:706481+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4342,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3303727910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3303727910},{"type":"v1","addr":"172.21.15.7:6835","nonce":3303727910}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4383,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3193801272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3193801272},{"type":"v1","addr":"172.21.15.7:6834","nonce":3193801272}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4401,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1546148687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1546148687},{"type":"v1","addr":"172.21.15.133:6837","nonce":1546148687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:42:09.121017+0000","modified":"2024-09-10T21:42:12.137567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234179":{"gid":234179,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6836/3769241160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3769241160},{"type":"v1","addr":"172.21.15.133:6836","nonce":3769241160}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234179,"qdb_cluster":[234179]},"id":1}]} 2024-09-10T21:42:13.143 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-10T21:42:13.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.144+0000 7f9a4ca00640 1 -- 172.21.15.7:0/2184052775 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9a24077f70 msgr2=0x7f9a2407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:13.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.144+0000 7f9a4ca00640 1 --2- 172.21.15.7:0/2184052775 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9a24077f70 0x7f9a2407a430 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9a30000fb0 tx=0x7f9a300067a0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.144+0000 7f9a4ca00640 1 -- 172.21.15.7:0/2184052775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a2c0053c0 msgr2=0x7f9a48154a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:13.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.144+0000 7f9a4ca00640 1 --2- 172.21.15.7:0/2184052775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a2c0053c0 0x7f9a48154a80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9a38003ed0 tx=0x7f9a38001630 comp rx=0 tx=0).stop 2024-09-10T21:42:13.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.144+0000 7f9a4ca00640 1 -- 172.21.15.7:0/2184052775 shutdown_connections 2024-09-10T21:42:13.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.144+0000 7f9a4ca00640 1 --2- 172.21.15.7:0/2184052775 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9a24077f70 0x7f9a2407a430 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.144+0000 7f9a4ca00640 1 --2- 172.21.15.7:0/2184052775 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a48153e40 0x7f9a480ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.144+0000 7f9a4ca00640 1 --2- 172.21.15.7:0/2184052775 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9a480ca710 0x7f9a48151b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.144+0000 7f9a4ca00640 1 --2- 172.21.15.7:0/2184052775 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a2c0053c0 0x7f9a48154a80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.144+0000 7f9a4ca00640 1 -- 172.21.15.7:0/2184052775 >> 172.21.15.7:0/2184052775 conn(0x7f9a480b16d0 msgr2=0x7f9a480b4820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:13.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.144+0000 7f9a4ca00640 1 -- 172.21.15.7:0/2184052775 shutdown_connections 2024-09-10T21:42:13.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.144+0000 7f9a4ca00640 1 -- 172.21.15.7:0/2184052775 wait complete. 2024-09-10T21:42:13.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:42:13.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.321+0000 7feed0000640 1 -- 172.21.15.7:0/1881729609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feeb0004800 msgr2=0x7feeb0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:13.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.321+0000 7feed0000640 1 --2- 172.21.15.7:0/1881729609 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feeb0004800 0x7feeb0004c80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7feeb4001830 tx=0x7feeb0005830 comp rx=0 tx=0).stop 2024-09-10T21:42:13.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.321+0000 7feece000640 1 -- 172.21.15.7:0/1881729609 reap_dead start 2024-09-10T21:42:13.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.321+0000 7feed0000640 1 -- 172.21.15.7:0/1881729609 shutdown_connections 2024-09-10T21:42:13.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.321+0000 7feed0000640 1 -- 172.21.15.7:0/1881729609 >> 172.21.15.7:0/1881729609 conn(0x7feec80b16d0 msgr2=0x7feec80b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:13.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.321+0000 7feed0000640 1 -- 172.21.15.7:0/1881729609 shutdown_connections 2024-09-10T21:42:13.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.321+0000 7feed0000640 1 -- 172.21.15.7:0/1881729609 wait complete. 2024-09-10T21:42:13.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.323+0000 7feed0000640 1 Processor -- start 2024-09-10T21:42:13.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.323+0000 7feed0000640 1 -- start start 2024-09-10T21:42:13.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.324+0000 7feed0000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feeb0004800 0x7feec815e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.324+0000 7feed0000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feec8153320 0x7feec80c9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.324+0000 7feed0000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feec8152760 0x7feec8152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.324+0000 7feed0000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feec8070610 con 0x7feec8152760 2024-09-10T21:42:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.324+0000 7feed0000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feec80ccef0 con 0x7feeb0004800 2024-09-10T21:42:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.324+0000 7feed0000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7feec8153100 con 0x7feec8153320 2024-09-10T21:42:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.324+0000 7feece000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feec8152760 0x7feec8152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.324+0000 7feece000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feec8152760 0x7feec8152bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45364/0 (socket says 172.21.15.7:45364) 2024-09-10T21:42:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.324+0000 7feece000640 1 -- 172.21.15.7:0/388094015 learned_addr learned my addr 172.21.15.7:0/388094015 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.324+0000 7feeccc00640 1 --2- 172.21.15.7:0/388094015 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feec8153320 0x7feec80c9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.324+0000 7feecd600640 1 --2- 172.21.15.7:0/388094015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feeb0004800 0x7feec815e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.325+0000 7feece000640 1 -- 172.21.15.7:0/388094015 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feec8153320 msgr2=0x7feec80c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:13.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.325+0000 7feece000640 1 --2- 172.21.15.7:0/388094015 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feec8153320 0x7feec80c9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.325+0000 7feece000640 1 -- 172.21.15.7:0/388094015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feeb0004800 msgr2=0x7feec815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.325+0000 7feece000640 1 --2- 172.21.15.7:0/388094015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feeb0004800 0x7feec815e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.325+0000 7feece000640 1 -- 172.21.15.7:0/388094015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec815ef90 con 0x7feec8152760 2024-09-10T21:42:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.325+0000 7feece000640 1 --2- 172.21.15.7:0/388094015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feec8152760 0x7feec8152bc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7feec80c3650 tx=0x7feec0006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.325+0000 7feec5600640 1 -- 172.21.15.7:0/388094015 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0015020 con 0x7feec8152760 2024-09-10T21:42:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.325+0000 7feec5600640 1 -- 172.21.15.7:0/388094015 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feec0011900 con 0x7feec8152760 2024-09-10T21:42:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.325+0000 7feed0000640 1 -- 172.21.15.7:0/388094015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec81f55f0 con 0x7feec8152760 2024-09-10T21:42:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.325+0000 7feec5600640 1 -- 172.21.15.7:0/388094015 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0011d10 con 0x7feec8152760 2024-09-10T21:42:13.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.325+0000 7feed0000640 1 -- 172.21.15.7:0/388094015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec81f5bb0 con 0x7feec8152760 2024-09-10T21:42:13.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.326+0000 7feed0000640 1 -- 172.21.15.7:0/388094015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feec81601f0 con 0x7feec8152760 2024-09-10T21:42:13.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.328+0000 7feec5600640 1 -- 172.21.15.7:0/388094015 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7feec00103c0 con 0x7feec8152760 2024-09-10T21:42:13.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.328+0000 7feec5600640 1 --2- 172.21.15.7:0/388094015 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7feea8077f70 0x7feea807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.329+0000 7feec5600640 1 -- 172.21.15.7:0/388094015 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feec009a7c0 con 0x7feec8152760 2024-09-10T21:42:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.329+0000 7feecd600640 1 --2- 172.21.15.7:0/388094015 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7feea8077f70 0x7feea807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:13.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.330+0000 7feecd600640 1 --2- 172.21.15.7:0/388094015 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7feea8077f70 0x7feea807a430 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feec806dac0 tx=0x7feebc005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:13.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.333+0000 7feec5600640 1 -- 172.21.15.7:0/388094015 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7feec001f030 con 0x7feec8152760 2024-09-10T21:42:13.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.529+0000 7feed0000640 1 -- 172.21.15.7:0/388094015 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feec80008d0 con 0x7feec8152760 2024-09-10T21:42:13.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.531+0000 7feec5600640 1 -- 172.21.15.7:0/388094015 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16629 (secure 0 0 0) 0x7feec0062c60 con 0x7feec8152760 2024-09-10T21:42:13.532 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:13.532 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:42:10.684978+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:42:07.149267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:42:08.077083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:42:13.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.533+0000 7feed0000640 1 -- 172.21.15.7:0/388094015 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7feea8077f70 msgr2=0x7feea807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.533+0000 7feed0000640 1 --2- 172.21.15.7:0/388094015 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7feea8077f70 0x7feea807a430 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7feec806dac0 tx=0x7feebc005f90 comp rx=0 tx=0).stop 2024-09-10T21:42:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.533+0000 7feed0000640 1 -- 172.21.15.7:0/388094015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feec8152760 msgr2=0x7feec8152bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.533+0000 7feed0000640 1 --2- 172.21.15.7:0/388094015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feec8152760 0x7feec8152bc0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7feec80c3650 tx=0x7feec0006430 comp rx=0 tx=0).stop 2024-09-10T21:42:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.533+0000 7feed0000640 1 -- 172.21.15.7:0/388094015 shutdown_connections 2024-09-10T21:42:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.533+0000 7feed0000640 1 --2- 172.21.15.7:0/388094015 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7feea8077f70 0x7feea807a430 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.533+0000 7feed0000640 1 --2- 172.21.15.7:0/388094015 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feec8152760 0x7feec8152bc0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.533+0000 7feed0000640 1 --2- 172.21.15.7:0/388094015 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feec8153320 0x7feec80c9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.533+0000 7feed0000640 1 --2- 172.21.15.7:0/388094015 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feeb0004800 0x7feec815e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.533+0000 7feed0000640 1 -- 172.21.15.7:0/388094015 >> 172.21.15.7:0/388094015 conn(0x7feec80b16d0 msgr2=0x7feec80724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.533+0000 7feed0000640 1 -- 172.21.15.7:0/388094015 shutdown_connections 2024-09-10T21:42:13.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.533+0000 7feed0000640 1 -- 172.21.15.7:0/388094015 wait complete. 2024-09-10T21:42:13.550 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:42:13.550 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:42:13.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.708+0000 7f8067c00640 1 -- 172.21.15.7:0/466390116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f806015e040 msgr2=0x7f806015bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:13.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.708+0000 7f8067c00640 1 --2- 172.21.15.7:0/466390116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f806015e040 0x7f806015bd60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f80600d0a30 tx=0x7f805402a560 comp rx=0 tx=0).stop 2024-09-10T21:42:13.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.709+0000 7f8067c00640 1 -- 172.21.15.7:0/466390116 shutdown_connections 2024-09-10T21:42:13.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.709+0000 7f8067c00640 1 --2- 172.21.15.7:0/466390116 >> v2:172.21.15.133:3300/0 conn(0x7f8060073c80 0x7f806015c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.709+0000 7f8067c00640 1 --2- 172.21.15.7:0/466390116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f806015e040 0x7f806015bd60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.709+0000 7f8067c00640 1 -- 172.21.15.7:0/466390116 >> 172.21.15.7:0/466390116 conn(0x7f806006d400 msgr2=0x7f80600b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:13.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.709+0000 7f8067c00640 1 -- 172.21.15.7:0/466390116 shutdown_connections 2024-09-10T21:42:13.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.709+0000 7f8067c00640 1 -- 172.21.15.7:0/466390116 wait complete. 2024-09-10T21:42:13.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.710+0000 7f8067c00640 1 Processor -- start 2024-09-10T21:42:13.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.710+0000 7f8067c00640 1 -- start start 2024-09-10T21:42:13.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.711+0000 7f8067c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8060072990 0x7f80601f7d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:13.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.711+0000 7f8067c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8060073c80 0x7f80600cfc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:13.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.712+0000 7f8067c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f806015e040 0x7f80600cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:13.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.712+0000 7f8067c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80600b4100 con 0x7f8060072990 2024-09-10T21:42:13.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.712+0000 7f8067c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f80600b3f80 con 0x7f8060073c80 2024-09-10T21:42:13.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.712+0000 7f8067c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f80600b4280 con 0x7f806015e040 2024-09-10T21:42:13.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.712+0000 7f8065200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8060072990 0x7f80601f7d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:13.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.712+0000 7f805fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8060073c80 0x7f80600cfc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:13.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.712+0000 7f8065200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8060072990 0x7f80601f7d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45382/0 (socket says 172.21.15.7:45382) 2024-09-10T21:42:13.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.712+0000 7f8065200640 1 -- 172.21.15.7:0/2246031104 learned_addr learned my addr 172.21.15.7:0/2246031104 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:13.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.712+0000 7f8065c00640 1 --2- 172.21.15.7:0/2246031104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f806015e040 0x7f80600cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:13.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.712+0000 7f8065200640 1 -- 172.21.15.7:0/2246031104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f806015e040 msgr2=0x7f80600cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:13.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.712+0000 7f8065200640 1 --2- 172.21.15.7:0/2246031104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f806015e040 0x7f80600cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.712+0000 7f8065200640 1 -- 172.21.15.7:0/2246031104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8060073c80 msgr2=0x7f80600cfc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:13.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.712+0000 7f8065200640 1 --2- 172.21.15.7:0/2246031104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8060073c80 0x7f80600cfc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.712+0000 7f8065200640 1 -- 172.21.15.7:0/2246031104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80601f8340 con 0x7f8060072990 2024-09-10T21:42:13.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.713+0000 7f8065200640 1 --2- 172.21.15.7:0/2246031104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8060072990 0x7f80601f7d60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8054034040 tx=0x7f80540039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:13.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.715+0000 7f805cc00640 1 -- 172.21.15.7:0/2246031104 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f805403e030 con 0x7f8060072990 2024-09-10T21:42:13.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.715+0000 7f805cc00640 1 -- 172.21.15.7:0/2246031104 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80540047a0 con 0x7f8060072990 2024-09-10T21:42:13.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.715+0000 7f805cc00640 1 -- 172.21.15.7:0/2246031104 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8054035040 con 0x7f8060072990 2024-09-10T21:42:13.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.715+0000 7f8067c00640 1 -- 172.21.15.7:0/2246031104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80601f8620 con 0x7f8060072990 2024-09-10T21:42:13.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.715+0000 7f8067c00640 1 -- 172.21.15.7:0/2246031104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80600cd5d0 con 0x7f8060072990 2024-09-10T21:42:13.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.716+0000 7f8067c00640 1 -- 172.21.15.7:0/2246031104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80600b4750 con 0x7f8060072990 2024-09-10T21:42:13.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.717+0000 7f805cc00640 1 -- 172.21.15.7:0/2246031104 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8054004b70 con 0x7f8060072990 2024-09-10T21:42:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.718+0000 7f805cc00640 1 --2- 172.21.15.7:0/2246031104 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8040077f70 0x7f804007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.718+0000 7f805cc00640 1 -- 172.21.15.7:0/2246031104 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f80540ba690 con 0x7f8060072990 2024-09-10T21:42:13.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.718+0000 7f805fe00640 1 --2- 172.21.15.7:0/2246031104 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8040077f70 0x7f804007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:13.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.719+0000 7f805fe00640 1 --2- 172.21.15.7:0/2246031104 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8040077f70 0x7f804007a430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8060154be0 tx=0x7f804c009110 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:13.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.722+0000 7f805cc00640 1 -- 172.21.15.7:0/2246031104 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8054082b30 con 0x7f8060072990 2024-09-10T21:42:13.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.913+0000 7f8067c00640 1 -- 172.21.15.7:0/2246031104 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f806015e6d0 con 0x7f8040077f70 2024-09-10T21:42:13.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.919+0000 7f805cc00640 1 -- 172.21.15.7:0/2246031104 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251935 (secure 0 0 0) 0x7f806015e6d0 con 0x7f8040077f70 2024-09-10T21:42:13.924 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:13.925 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:42:13.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.923+0000 7f8067c00640 1 -- 172.21.15.7:0/2246031104 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8040077f70 msgr2=0x7f804007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:13.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.923+0000 7f8067c00640 1 --2- 172.21.15.7:0/2246031104 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8040077f70 0x7f804007a430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8060154be0 tx=0x7f804c009110 comp rx=0 tx=0).stop 2024-09-10T21:42:13.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.923+0000 7f8067c00640 1 -- 172.21.15.7:0/2246031104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8060072990 msgr2=0x7f80601f7d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:13.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.923+0000 7f8067c00640 1 --2- 172.21.15.7:0/2246031104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8060072990 0x7f80601f7d60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8054034040 tx=0x7f80540039e0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.923+0000 7f8067c00640 1 -- 172.21.15.7:0/2246031104 shutdown_connections 2024-09-10T21:42:13.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.923+0000 7f8067c00640 1 --2- 172.21.15.7:0/2246031104 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8040077f70 0x7f804007a430 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.923+0000 7f8067c00640 1 --2- 172.21.15.7:0/2246031104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8060073c80 0x7f80600cfc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.923+0000 7f8067c00640 1 --2- 172.21.15.7:0/2246031104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f806015e040 0x7f80600cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.923+0000 7f8067c00640 1 --2- 172.21.15.7:0/2246031104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8060072990 0x7f80601f7d60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:13.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.923+0000 7f8067c00640 1 -- 172.21.15.7:0/2246031104 >> 172.21.15.7:0/2246031104 conn(0x7f806006d400 msgr2=0x7f806015f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:13.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.923+0000 7f8067c00640 1 -- 172.21.15.7:0/2246031104 shutdown_connections 2024-09-10T21:42:13.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:13.923+0000 7f8067c00640 1 -- 172.21.15.7:0/2246031104 wait complete. 2024-09-10T21:42:13.940 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-10T21:42:13.856571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235704,"kb_used_data":2832,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739016,"statfs":{"total":773094113280,"available":772852752384,"internally_reserved":0,"allocated":2899968,"data_stored":1750452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001359"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782002+0000","last_change":"2024-09-10T21:42:09.712975+0000","last_active":"2024-09-10T21:42:11.782002+0000","last_peered":"2024-09-10T21:42:11.782002+0000","last_clean":"2024-09-10T21:42:11.782002+0000","last_became_active":"2024-09-10T21:42:09.712242+0000","last_became_peered":"2024-09-10T21:42:09.712242+0000","last_unstale":"2024-09-10T21:42:11.782002+0000","last_undegraded":"2024-09-10T21:42:11.782002+0000","last_fullsized":"2024-09-10T21:42:11.782002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:00.850316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695614+0000","last_change":"2024-09-10T21:42:09.697173+0000","last_active":"2024-09-10T21:42:10.695614+0000","last_peered":"2024-09-10T21:42:10.695614+0000","last_clean":"2024-09-10T21:42:10.695614+0000","last_became_active":"2024-09-10T21:42:09.696835+0000","last_became_peered":"2024-09-10T21:42:09.696835+0000","last_unstale":"2024-09-10T21:42:10.695614+0000","last_undegraded":"2024-09-10T21:42:10.695614+0000","last_fullsized":"2024-09-10T21:42:10.695614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:13.873778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975496+0000","last_change":"2024-09-10T21:42:09.693813+0000","last_active":"2024-09-10T21:42:11.975496+0000","last_peered":"2024-09-10T21:42:11.975496+0000","last_clean":"2024-09-10T21:42:11.975496+0000","last_became_active":"2024-09-10T21:42:09.693499+0000","last_became_peered":"2024-09-10T21:42:09.693499+0000","last_unstale":"2024-09-10T21:42:11.975496+0000","last_undegraded":"2024-09-10T21:42:11.975496+0000","last_fullsized":"2024-09-10T21:42:11.975496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:51.172800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702414+0000","last_change":"2024-09-10T21:42:09.698507+0000","last_active":"2024-09-10T21:42:10.702414+0000","last_peered":"2024-09-10T21:42:10.702414+0000","last_clean":"2024-09-10T21:42:10.702414+0000","last_became_active":"2024-09-10T21:42:09.697845+0000","last_became_peered":"2024-09-10T21:42:09.697845+0000","last_unstale":"2024-09-10T21:42:10.702414+0000","last_undegraded":"2024-09-10T21:42:10.702414+0000","last_fullsized":"2024-09-10T21:42:10.702414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:34.523014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781839+0000","last_change":"2024-09-10T21:42:09.703989+0000","last_active":"2024-09-10T21:42:11.781839+0000","last_peered":"2024-09-10T21:42:11.781839+0000","last_clean":"2024-09-10T21:42:11.781839+0000","last_became_active":"2024-09-10T21:42:09.703641+0000","last_became_peered":"2024-09-10T21:42:09.703641+0000","last_unstale":"2024-09-10T21:42:11.781839+0000","last_undegraded":"2024-09-10T21:42:11.781839+0000","last_fullsized":"2024-09-10T21:42:11.781839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:51.463128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701240+0000","last_change":"2024-09-10T21:42:09.691432+0000","last_active":"2024-09-10T21:42:10.701240+0000","last_peered":"2024-09-10T21:42:10.701240+0000","last_clean":"2024-09-10T21:42:10.701240+0000","last_became_active":"2024-09-10T21:42:09.691029+0000","last_became_peered":"2024-09-10T21:42:09.691029+0000","last_unstale":"2024-09-10T21:42:10.701240+0000","last_undegraded":"2024-09-10T21:42:10.701240+0000","last_fullsized":"2024-09-10T21:42:10.701240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:06.860570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781477+0000","last_change":"2024-09-10T21:42:09.716770+0000","last_active":"2024-09-10T21:42:11.781477+0000","last_peered":"2024-09-10T21:42:11.781477+0000","last_clean":"2024-09-10T21:42:11.781477+0000","last_became_active":"2024-09-10T21:42:09.716490+0000","last_became_peered":"2024-09-10T21:42:09.716490+0000","last_unstale":"2024-09-10T21:42:11.781477+0000","last_undegraded":"2024-09-10T21:42:11.781477+0000","last_fullsized":"2024-09-10T21:42:11.781477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:45.131035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.699076+0000","last_change":"2024-09-10T21:42:09.695560+0000","last_active":"2024-09-10T21:42:09.699076+0000","last_peered":"2024-09-10T21:42:09.699076+0000","last_clean":"2024-09-10T21:42:09.699076+0000","last_became_active":"2024-09-10T21:42:09.695252+0000","last_became_peered":"2024-09-10T21:42:09.695252+0000","last_unstale":"2024-09-10T21:42:09.699076+0000","last_undegraded":"2024-09-10T21:42:09.699076+0000","last_fullsized":"2024-09-10T21:42:09.699076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:49.273055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782198+0000","last_change":"2024-09-10T21:42:09.713256+0000","last_active":"2024-09-10T21:42:11.782198+0000","last_peered":"2024-09-10T21:42:11.782198+0000","last_clean":"2024-09-10T21:42:11.782198+0000","last_became_active":"2024-09-10T21:42:09.712700+0000","last_became_peered":"2024-09-10T21:42:09.712700+0000","last_unstale":"2024-09-10T21:42:11.782198+0000","last_undegraded":"2024-09-10T21:42:11.782198+0000","last_fullsized":"2024-09-10T21:42:11.782198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:19.953355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685329+0000","last_change":"2024-09-10T21:42:09.702384+0000","last_active":"2024-09-10T21:42:11.685329+0000","last_peered":"2024-09-10T21:42:11.685329+0000","last_clean":"2024-09-10T21:42:11.685329+0000","last_became_active":"2024-09-10T21:42:09.698800+0000","last_became_peered":"2024-09-10T21:42:09.698800+0000","last_unstale":"2024-09-10T21:42:11.685329+0000","last_undegraded":"2024-09-10T21:42:11.685329+0000","last_fullsized":"2024-09-10T21:42:11.685329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:13.873778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685153+0000","last_change":"2024-09-10T21:42:09.704731+0000","last_active":"2024-09-10T21:42:11.685153+0000","last_peered":"2024-09-10T21:42:11.685153+0000","last_clean":"2024-09-10T21:42:11.685153+0000","last_became_active":"2024-09-10T21:42:09.698600+0000","last_became_peered":"2024-09-10T21:42:09.698600+0000","last_unstale":"2024-09-10T21:42:11.685153+0000","last_undegraded":"2024-09-10T21:42:11.685153+0000","last_fullsized":"2024-09-10T21:42:11.685153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:06.304338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701569+0000","last_change":"2024-09-10T21:42:09.710639+0000","last_active":"2024-09-10T21:42:10.701569+0000","last_peered":"2024-09-10T21:42:10.701569+0000","last_clean":"2024-09-10T21:42:10.701569+0000","last_became_active":"2024-09-10T21:42:09.710337+0000","last_became_peered":"2024-09-10T21:42:09.710337+0000","last_unstale":"2024-09-10T21:42:10.701569+0000","last_undegraded":"2024-09-10T21:42:10.701569+0000","last_fullsized":"2024-09-10T21:42:10.701569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:47.171015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.708774+0000","last_change":"2024-09-10T21:42:09.706632+0000","last_active":"2024-09-10T21:42:09.708774+0000","last_peered":"2024-09-10T21:42:09.708774+0000","last_clean":"2024-09-10T21:42:09.708774+0000","last_became_active":"2024-09-10T21:42:09.706366+0000","last_became_peered":"2024-09-10T21:42:09.706366+0000","last_unstale":"2024-09-10T21:42:09.708774+0000","last_undegraded":"2024-09-10T21:42:09.708774+0000","last_fullsized":"2024-09-10T21:42:09.708774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:40.730728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.974972+0000","last_change":"2024-09-10T21:42:09.696596+0000","last_active":"2024-09-10T21:42:11.974972+0000","last_peered":"2024-09-10T21:42:11.974972+0000","last_clean":"2024-09-10T21:42:11.974972+0000","last_became_active":"2024-09-10T21:42:09.696292+0000","last_became_peered":"2024-09-10T21:42:09.696292+0000","last_unstale":"2024-09-10T21:42:11.974972+0000","last_undegraded":"2024-09-10T21:42:11.974972+0000","last_fullsized":"2024-09-10T21:42:11.974972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:49.235638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695329+0000","last_change":"2024-09-10T21:42:09.701498+0000","last_active":"2024-09-10T21:42:10.695329+0000","last_peered":"2024-09-10T21:42:10.695329+0000","last_clean":"2024-09-10T21:42:10.695329+0000","last_became_active":"2024-09-10T21:42:09.701149+0000","last_became_peered":"2024-09-10T21:42:09.701149+0000","last_unstale":"2024-09-10T21:42:10.695329+0000","last_undegraded":"2024-09-10T21:42:10.695329+0000","last_fullsized":"2024-09-10T21:42:10.695329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:46.339409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.700962+0000","last_change":"2024-09-10T21:42:09.694844+0000","last_active":"2024-09-10T21:42:10.700962+0000","last_peered":"2024-09-10T21:42:10.700962+0000","last_clean":"2024-09-10T21:42:10.700962+0000","last_became_active":"2024-09-10T21:42:09.694573+0000","last_became_peered":"2024-09-10T21:42:09.694573+0000","last_unstale":"2024-09-10T21:42:10.700962+0000","last_undegraded":"2024-09-10T21:42:10.700962+0000","last_fullsized":"2024-09-10T21:42:10.700962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:13.873778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702591+0000","last_change":"2024-09-10T21:42:09.707330+0000","last_active":"2024-09-10T21:42:10.702591+0000","last_peered":"2024-09-10T21:42:10.702591+0000","last_clean":"2024-09-10T21:42:10.702591+0000","last_became_active":"2024-09-10T21:42:09.707071+0000","last_became_peered":"2024-09-10T21:42:09.707071+0000","last_unstale":"2024-09-10T21:42:10.702591+0000","last_undegraded":"2024-09-10T21:42:10.702591+0000","last_fullsized":"2024-09-10T21:42:10.702591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:33.348954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428727+0000","last_change":"2024-09-10T21:42:09.709362+0000","last_active":"2024-09-10T21:42:11.428727+0000","last_peered":"2024-09-10T21:42:11.428727+0000","last_clean":"2024-09-10T21:42:11.428727+0000","last_became_active":"2024-09-10T21:42:09.707734+0000","last_became_peered":"2024-09-10T21:42:09.707734+0000","last_unstale":"2024-09-10T21:42:11.428727+0000","last_undegraded":"2024-09-10T21:42:11.428727+0000","last_fullsized":"2024-09-10T21:42:11.428727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:54.172897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782132+0000","last_change":"2024-09-10T21:42:09.709392+0000","last_active":"2024-09-10T21:42:11.782132+0000","last_peered":"2024-09-10T21:42:11.782132+0000","last_clean":"2024-09-10T21:42:11.782132+0000","last_became_active":"2024-09-10T21:42:09.708821+0000","last_became_peered":"2024-09-10T21:42:09.708821+0000","last_unstale":"2024-09-10T21:42:11.782132+0000","last_undegraded":"2024-09-10T21:42:11.782132+0000","last_fullsized":"2024-09-10T21:42:11.782132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:52.646979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685221+0000","last_change":"2024-09-10T21:42:09.711112+0000","last_active":"2024-09-10T21:42:11.685221+0000","last_peered":"2024-09-10T21:42:11.685221+0000","last_clean":"2024-09-10T21:42:11.685221+0000","last_became_active":"2024-09-10T21:42:09.710589+0000","last_became_peered":"2024-09-10T21:42:09.710589+0000","last_unstale":"2024-09-10T21:42:11.685221+0000","last_undegraded":"2024-09-10T21:42:11.685221+0000","last_fullsized":"2024-09-10T21:42:11.685221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:51.172800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975107+0000","last_change":"2024-09-10T21:42:09.707660+0000","last_active":"2024-09-10T21:42:11.975107+0000","last_peered":"2024-09-10T21:42:11.975107+0000","last_clean":"2024-09-10T21:42:11.975107+0000","last_became_active":"2024-09-10T21:42:09.707031+0000","last_became_peered":"2024-09-10T21:42:09.707031+0000","last_unstale":"2024-09-10T21:42:11.975107+0000","last_undegraded":"2024-09-10T21:42:11.975107+0000","last_fullsized":"2024-09-10T21:42:11.975107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:13.474744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701445+0000","last_change":"2024-09-10T21:42:09.708730+0000","last_active":"2024-09-10T21:42:10.701445+0000","last_peered":"2024-09-10T21:42:10.701445+0000","last_clean":"2024-09-10T21:42:10.701445+0000","last_became_active":"2024-09-10T21:42:09.708056+0000","last_became_peered":"2024-09-10T21:42:09.708056+0000","last_unstale":"2024-09-10T21:42:10.701445+0000","last_undegraded":"2024-09-10T21:42:10.701445+0000","last_fullsized":"2024-09-10T21:42:10.701445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:58.932679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.710662+0000","last_change":"2024-09-10T21:42:09.705245+0000","last_active":"2024-09-10T21:42:09.710662+0000","last_peered":"2024-09-10T21:42:09.710662+0000","last_clean":"2024-09-10T21:42:09.710662+0000","last_became_active":"2024-09-10T21:42:09.699361+0000","last_became_peered":"2024-09-10T21:42:09.699361+0000","last_unstale":"2024-09-10T21:42:09.710662+0000","last_undegraded":"2024-09-10T21:42:09.710662+0000","last_fullsized":"2024-09-10T21:42:09.710662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:45.084767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701128+0000","last_change":"2024-09-10T21:42:09.704790+0000","last_active":"2024-09-10T21:42:10.701128+0000","last_peered":"2024-09-10T21:42:10.701128+0000","last_clean":"2024-09-10T21:42:10.701128+0000","last_became_active":"2024-09-10T21:42:09.699333+0000","last_became_peered":"2024-09-10T21:42:09.699333+0000","last_unstale":"2024-09-10T21:42:10.701128+0000","last_undegraded":"2024-09-10T21:42:10.701128+0000","last_fullsized":"2024-09-10T21:42:10.701128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:48.304330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.714909+0000","last_change":"2024-09-10T21:42:09.712518+0000","last_active":"2024-09-10T21:42:09.714909+0000","last_peered":"2024-09-10T21:42:09.714909+0000","last_clean":"2024-09-10T21:42:09.714909+0000","last_became_active":"2024-09-10T21:42:09.711734+0000","last_became_peered":"2024-09-10T21:42:09.711734+0000","last_unstale":"2024-09-10T21:42:09.714909+0000","last_undegraded":"2024-09-10T21:42:09.714909+0000","last_fullsized":"2024-09-10T21:42:09.714909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:04.871576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685338+0000","last_change":"2024-09-10T21:42:09.713125+0000","last_active":"2024-09-10T21:42:11.685338+0000","last_peered":"2024-09-10T21:42:11.685338+0000","last_clean":"2024-09-10T21:42:11.685338+0000","last_became_active":"2024-09-10T21:42:09.711555+0000","last_became_peered":"2024-09-10T21:42:09.711555+0000","last_unstale":"2024-09-10T21:42:11.685338+0000","last_undegraded":"2024-09-10T21:42:11.685338+0000","last_fullsized":"2024-09-10T21:42:11.685338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:48.304330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695578+0000","last_change":"2024-09-10T21:42:09.698227+0000","last_active":"2024-09-10T21:42:10.695578+0000","last_peered":"2024-09-10T21:42:10.695578+0000","last_clean":"2024-09-10T21:42:10.695578+0000","last_became_active":"2024-09-10T21:42:09.695964+0000","last_became_peered":"2024-09-10T21:42:09.695964+0000","last_unstale":"2024-09-10T21:42:10.695578+0000","last_undegraded":"2024-09-10T21:42:10.695578+0000","last_fullsized":"2024-09-10T21:42:10.695578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:51.172800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.705104+0000","last_change":"2024-09-10T21:42:09.699013+0000","last_active":"2024-09-10T21:42:09.705104+0000","last_peered":"2024-09-10T21:42:09.705104+0000","last_clean":"2024-09-10T21:42:09.705104+0000","last_became_active":"2024-09-10T21:42:09.698647+0000","last_became_peered":"2024-09-10T21:42:09.698647+0000","last_unstale":"2024-09-10T21:42:09.705104+0000","last_undegraded":"2024-09-10T21:42:09.705104+0000","last_fullsized":"2024-09-10T21:42:09.705104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:00.427200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975129+0000","last_change":"2024-09-10T21:42:09.697577+0000","last_active":"2024-09-10T21:42:11.975129+0000","last_peered":"2024-09-10T21:42:11.975129+0000","last_clean":"2024-09-10T21:42:11.975129+0000","last_became_active":"2024-09-10T21:42:09.696978+0000","last_became_peered":"2024-09-10T21:42:09.696978+0000","last_unstale":"2024-09-10T21:42:11.975129+0000","last_undegraded":"2024-09-10T21:42:11.975129+0000","last_fullsized":"2024-09-10T21:42:11.975129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:37.025454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701665+0000","last_change":"2024-09-10T21:42:09.697210+0000","last_active":"2024-09-10T21:42:10.701665+0000","last_peered":"2024-09-10T21:42:10.701665+0000","last_clean":"2024-09-10T21:42:10.701665+0000","last_became_active":"2024-09-10T21:42:09.696910+0000","last_became_peered":"2024-09-10T21:42:09.696910+0000","last_unstale":"2024-09-10T21:42:10.701665+0000","last_undegraded":"2024-09-10T21:42:10.701665+0000","last_fullsized":"2024-09-10T21:42:10.701665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:17.732763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428577+0000","last_change":"2024-09-10T21:42:09.702664+0000","last_active":"2024-09-10T21:42:11.428577+0000","last_peered":"2024-09-10T21:42:11.428577+0000","last_clean":"2024-09-10T21:42:11.428577+0000","last_became_active":"2024-09-10T21:42:09.700834+0000","last_became_peered":"2024-09-10T21:42:09.700834+0000","last_unstale":"2024-09-10T21:42:11.428577+0000","last_undegraded":"2024-09-10T21:42:11.428577+0000","last_fullsized":"2024-09-10T21:42:11.428577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:35.668254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.684927+0000","last_change":"2024-09-10T21:42:09.707098+0000","last_active":"2024-09-10T21:42:11.684927+0000","last_peered":"2024-09-10T21:42:11.684927+0000","last_clean":"2024-09-10T21:42:11.684927+0000","last_became_active":"2024-09-10T21:42:09.706747+0000","last_became_peered":"2024-09-10T21:42:09.706747+0000","last_unstale":"2024-09-10T21:42:11.684927+0000","last_undegraded":"2024-09-10T21:42:11.684927+0000","last_fullsized":"2024-09-10T21:42:11.684927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:38.013273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701269+0000","last_change":"2024-09-10T21:42:09.694901+0000","last_active":"2024-09-10T21:42:10.701269+0000","last_peered":"2024-09-10T21:42:10.701269+0000","last_clean":"2024-09-10T21:42:10.701269+0000","last_became_active":"2024-09-10T21:42:09.694509+0000","last_became_peered":"2024-09-10T21:42:09.694509+0000","last_unstale":"2024-09-10T21:42:10.701269+0000","last_undegraded":"2024-09-10T21:42:10.701269+0000","last_fullsized":"2024-09-10T21:42:10.701269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:57.099056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.716471+0000","last_change":"2024-09-10T21:42:08.690816+0000","last_active":"2024-09-10T21:42:09.716471+0000","last_peered":"2024-09-10T21:42:09.716471+0000","last_clean":"2024-09-10T21:42:09.716471+0000","last_became_active":"2024-09-10T21:42:08.689282+0000","last_became_peered":"2024-09-10T21:42:08.689282+0000","last_unstale":"2024-09-10T21:42:09.716471+0000","last_undegraded":"2024-09-10T21:42:09.716471+0000","last_fullsized":"2024-09-10T21:42:09.716471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:17.328468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975323+0000","last_change":"2024-09-10T21:42:09.707990+0000","last_active":"2024-09-10T21:42:11.975323+0000","last_peered":"2024-09-10T21:42:11.975323+0000","last_clean":"2024-09-10T21:42:11.975323+0000","last_became_active":"2024-09-10T21:42:09.707404+0000","last_became_peered":"2024-09-10T21:42:09.707404+0000","last_unstale":"2024-09-10T21:42:11.975323+0000","last_undegraded":"2024-09-10T21:42:11.975323+0000","last_fullsized":"2024-09-10T21:42:11.975323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:40.556148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.684709+0000","last_change":"2024-09-10T21:42:08.697982+0000","last_active":"2024-09-10T21:42:11.684709+0000","last_peered":"2024-09-10T21:42:11.684709+0000","last_clean":"2024-09-10T21:42:11.684709+0000","last_became_active":"2024-09-10T21:42:08.696642+0000","last_became_peered":"2024-09-10T21:42:08.696642+0000","last_unstale":"2024-09-10T21:42:11.684709+0000","last_undegraded":"2024-09-10T21:42:11.684709+0000","last_fullsized":"2024-09-10T21:42:11.684709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:50.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.680064+0000","last_change":"2024-09-10T21:42:08.704347+0000","last_active":"2024-09-10T21:42:09.680064+0000","last_peered":"2024-09-10T21:42:09.680064+0000","last_clean":"2024-09-10T21:42:09.680064+0000","last_became_active":"2024-09-10T21:42:08.701532+0000","last_became_peered":"2024-09-10T21:42:08.701532+0000","last_unstale":"2024-09-10T21:42:09.680064+0000","last_undegraded":"2024-09-10T21:42:09.680064+0000","last_fullsized":"2024-09-10T21:42:09.680064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:05.016544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.703760+0000","last_change":"2024-09-10T21:42:09.698844+0000","last_active":"2024-09-10T21:42:09.703760+0000","last_peered":"2024-09-10T21:42:09.703760+0000","last_clean":"2024-09-10T21:42:09.703760+0000","last_became_active":"2024-09-10T21:42:09.698554+0000","last_became_peered":"2024-09-10T21:42:09.698554+0000","last_unstale":"2024-09-10T21:42:09.703760+0000","last_undegraded":"2024-09-10T21:42:09.703760+0000","last_fullsized":"2024-09-10T21:42:09.703760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:46.579215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701042+0000","last_change":"2024-09-10T21:42:08.685541+0000","last_active":"2024-09-10T21:42:10.701042+0000","last_peered":"2024-09-10T21:42:10.701042+0000","last_clean":"2024-09-10T21:42:10.701042+0000","last_became_active":"2024-09-10T21:42:08.685254+0000","last_became_peered":"2024-09-10T21:42:08.685254+0000","last_unstale":"2024-09-10T21:42:10.701042+0000","last_undegraded":"2024-09-10T21:42:10.701042+0000","last_fullsized":"2024-09-10T21:42:10.701042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:56.090844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695329+0000","last_change":"2024-09-10T21:42:09.695387+0000","last_active":"2024-09-10T21:42:10.695329+0000","last_peered":"2024-09-10T21:42:10.695329+0000","last_clean":"2024-09-10T21:42:10.695329+0000","last_became_active":"2024-09-10T21:42:09.695065+0000","last_became_peered":"2024-09-10T21:42:09.695065+0000","last_unstale":"2024-09-10T21:42:10.695329+0000","last_undegraded":"2024-09-10T21:42:10.695329+0000","last_fullsized":"2024-09-10T21:42:10.695329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:38.013273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685092+0000","last_change":"2024-09-10T21:42:08.692049+0000","last_active":"2024-09-10T21:42:11.685092+0000","last_peered":"2024-09-10T21:42:11.685092+0000","last_clean":"2024-09-10T21:42:11.685092+0000","last_became_active":"2024-09-10T21:42:08.691746+0000","last_became_peered":"2024-09-10T21:42:08.691746+0000","last_unstale":"2024-09-10T21:42:11.685092+0000","last_undegraded":"2024-09-10T21:42:11.685092+0000","last_fullsized":"2024-09-10T21:42:11.685092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:46.162803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702258+0000","last_change":"2024-09-10T21:42:09.703241+0000","last_active":"2024-09-10T21:42:10.702258+0000","last_peered":"2024-09-10T21:42:10.702258+0000","last_clean":"2024-09-10T21:42:10.702258+0000","last_became_active":"2024-09-10T21:42:09.702971+0000","last_became_peered":"2024-09-10T21:42:09.702971+0000","last_unstale":"2024-09-10T21:42:10.702258+0000","last_undegraded":"2024-09-10T21:42:10.702258+0000","last_fullsized":"2024-09-10T21:42:10.702258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:42.927407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702191+0000","last_change":"2024-09-10T21:42:08.698423+0000","last_active":"2024-09-10T21:42:10.702191+0000","last_peered":"2024-09-10T21:42:10.702191+0000","last_clean":"2024-09-10T21:42:10.702191+0000","last_became_active":"2024-09-10T21:42:08.696721+0000","last_became_peered":"2024-09-10T21:42:08.696721+0000","last_unstale":"2024-09-10T21:42:10.702191+0000","last_undegraded":"2024-09-10T21:42:10.702191+0000","last_fullsized":"2024-09-10T21:42:10.702191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:50.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975587+0000","last_change":"2024-09-10T21:42:09.713357+0000","last_active":"2024-09-10T21:42:11.975587+0000","last_peered":"2024-09-10T21:42:11.975587+0000","last_clean":"2024-09-10T21:42:11.975587+0000","last_became_active":"2024-09-10T21:42:09.713078+0000","last_became_peered":"2024-09-10T21:42:09.713078+0000","last_unstale":"2024-09-10T21:42:11.975587+0000","last_undegraded":"2024-09-10T21:42:11.975587+0000","last_fullsized":"2024-09-10T21:42:11.975587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:52.381281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695294+0000","last_change":"2024-09-10T21:42:08.696594+0000","last_active":"2024-09-10T21:42:10.695294+0000","last_peered":"2024-09-10T21:42:10.695294+0000","last_clean":"2024-09-10T21:42:10.695294+0000","last_became_active":"2024-09-10T21:42:08.696243+0000","last_became_peered":"2024-09-10T21:42:08.696243+0000","last_unstale":"2024-09-10T21:42:10.695294+0000","last_undegraded":"2024-09-10T21:42:10.695294+0000","last_fullsized":"2024-09-10T21:42:10.695294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:46.162803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701779+0000","last_change":"2024-09-10T21:42:09.704846+0000","last_active":"2024-09-10T21:42:10.701779+0000","last_peered":"2024-09-10T21:42:10.701779+0000","last_clean":"2024-09-10T21:42:10.701779+0000","last_became_active":"2024-09-10T21:42:09.704582+0000","last_became_peered":"2024-09-10T21:42:09.704582+0000","last_unstale":"2024-09-10T21:42:10.701779+0000","last_undegraded":"2024-09-10T21:42:10.701779+0000","last_fullsized":"2024-09-10T21:42:10.701779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:34.026318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685136+0000","last_change":"2024-09-10T21:42:08.694242+0000","last_active":"2024-09-10T21:42:11.685136+0000","last_peered":"2024-09-10T21:42:11.685136+0000","last_clean":"2024-09-10T21:42:11.685136+0000","last_became_active":"2024-09-10T21:42:08.693785+0000","last_became_peered":"2024-09-10T21:42:08.693785+0000","last_unstale":"2024-09-10T21:42:11.685136+0000","last_undegraded":"2024-09-10T21:42:11.685136+0000","last_fullsized":"2024-09-10T21:42:11.685136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:56.090844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781904+0000","last_change":"2024-09-10T21:42:09.709096+0000","last_active":"2024-09-10T21:42:11.781904+0000","last_peered":"2024-09-10T21:42:11.781904+0000","last_clean":"2024-09-10T21:42:11.781904+0000","last_became_active":"2024-09-10T21:42:09.708202+0000","last_became_peered":"2024-09-10T21:42:09.708202+0000","last_unstale":"2024-09-10T21:42:11.781904+0000","last_undegraded":"2024-09-10T21:42:11.781904+0000","last_fullsized":"2024-09-10T21:42:11.781904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:12:39.888041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975170+0000","last_change":"2024-09-10T21:42:08.694284+0000","last_active":"2024-09-10T21:42:11.975170+0000","last_peered":"2024-09-10T21:42:11.975170+0000","last_clean":"2024-09-10T21:42:11.975170+0000","last_became_active":"2024-09-10T21:42:08.693910+0000","last_became_peered":"2024-09-10T21:42:08.693910+0000","last_unstale":"2024-09-10T21:42:11.975170+0000","last_undegraded":"2024-09-10T21:42:11.975170+0000","last_fullsized":"2024-09-10T21:42:11.975170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:40.174118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975003+0000","last_change":"2024-09-10T21:42:09.712940+0000","last_active":"2024-09-10T21:42:11.975003+0000","last_peered":"2024-09-10T21:42:11.975003+0000","last_clean":"2024-09-10T21:42:11.975003+0000","last_became_active":"2024-09-10T21:42:09.709015+0000","last_became_peered":"2024-09-10T21:42:09.709015+0000","last_unstale":"2024-09-10T21:42:11.975003+0000","last_undegraded":"2024-09-10T21:42:11.975003+0000","last_fullsized":"2024-09-10T21:42:11.975003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:41.900249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701745+0000","last_change":"2024-09-10T21:42:08.689371+0000","last_active":"2024-09-10T21:42:10.701745+0000","last_peered":"2024-09-10T21:42:10.701745+0000","last_clean":"2024-09-10T21:42:10.701745+0000","last_became_active":"2024-09-10T21:42:08.688880+0000","last_became_peered":"2024-09-10T21:42:08.688880+0000","last_unstale":"2024-09-10T21:42:10.701745+0000","last_undegraded":"2024-09-10T21:42:10.701745+0000","last_fullsized":"2024-09-10T21:42:10.701745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:46.162803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781881+0000","last_change":"2024-09-10T21:42:09.713450+0000","last_active":"2024-09-10T21:42:11.781881+0000","last_peered":"2024-09-10T21:42:11.781881+0000","last_clean":"2024-09-10T21:42:11.781881+0000","last_became_active":"2024-09-10T21:42:09.713199+0000","last_became_peered":"2024-09-10T21:42:09.713199+0000","last_unstale":"2024-09-10T21:42:11.781881+0000","last_undegraded":"2024-09-10T21:42:11.781881+0000","last_fullsized":"2024-09-10T21:42:11.781881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:23.530344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.684649+0000","last_change":"2024-09-10T21:42:08.703573+0000","last_active":"2024-09-10T21:42:11.684649+0000","last_peered":"2024-09-10T21:42:11.684649+0000","last_clean":"2024-09-10T21:42:11.684649+0000","last_became_active":"2024-09-10T21:42:08.701580+0000","last_became_peered":"2024-09-10T21:42:08.701580+0000","last_unstale":"2024-09-10T21:42:11.684649+0000","last_undegraded":"2024-09-10T21:42:11.684649+0000","last_fullsized":"2024-09-10T21:42:11.684649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:16.471687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.684924+0000","last_change":"2024-09-10T21:42:09.700853+0000","last_active":"2024-09-10T21:42:11.684924+0000","last_peered":"2024-09-10T21:42:11.684924+0000","last_clean":"2024-09-10T21:42:11.684924+0000","last_became_active":"2024-09-10T21:42:09.700370+0000","last_became_peered":"2024-09-10T21:42:09.700370+0000","last_unstale":"2024-09-10T21:42:11.684924+0000","last_undegraded":"2024-09-10T21:42:11.684924+0000","last_fullsized":"2024-09-10T21:42:11.684924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:13.700823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428370+0000","last_change":"2024-09-10T21:42:08.697117+0000","last_active":"2024-09-10T21:42:11.428370+0000","last_peered":"2024-09-10T21:42:11.428370+0000","last_clean":"2024-09-10T21:42:11.428370+0000","last_became_active":"2024-09-10T21:42:08.695800+0000","last_became_peered":"2024-09-10T21:42:08.695800+0000","last_unstale":"2024-09-10T21:42:11.428370+0000","last_undegraded":"2024-09-10T21:42:11.428370+0000","last_fullsized":"2024-09-10T21:42:11.428370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:36.934204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781790+0000","last_change":"2024-09-10T21:42:09.707807+0000","last_active":"2024-09-10T21:42:11.781790+0000","last_peered":"2024-09-10T21:42:11.781790+0000","last_clean":"2024-09-10T21:42:11.781790+0000","last_became_active":"2024-09-10T21:42:09.707283+0000","last_became_peered":"2024-09-10T21:42:09.707283+0000","last_unstale":"2024-09-10T21:42:11.781790+0000","last_undegraded":"2024-09-10T21:42:11.781790+0000","last_fullsized":"2024-09-10T21:42:11.781790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:50.040141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701923+0000","last_change":"2024-09-10T21:42:08.693177+0000","last_active":"2024-09-10T21:42:10.701923+0000","last_peered":"2024-09-10T21:42:10.701923+0000","last_clean":"2024-09-10T21:42:10.701923+0000","last_became_active":"2024-09-10T21:42:08.692871+0000","last_became_peered":"2024-09-10T21:42:08.692871+0000","last_unstale":"2024-09-10T21:42:10.701923+0000","last_undegraded":"2024-09-10T21:42:10.701923+0000","last_fullsized":"2024-09-10T21:42:10.701923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:06.841814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781985+0000","last_change":"2024-09-10T21:42:09.710163+0000","last_active":"2024-09-10T21:42:11.781985+0000","last_peered":"2024-09-10T21:42:11.781985+0000","last_clean":"2024-09-10T21:42:11.781985+0000","last_became_active":"2024-09-10T21:42:09.709887+0000","last_became_peered":"2024-09-10T21:42:09.709887+0000","last_unstale":"2024-09-10T21:42:11.781985+0000","last_undegraded":"2024-09-10T21:42:11.781985+0000","last_fullsized":"2024-09-10T21:42:11.781985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:05:52.771916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428604+0000","last_change":"2024-09-10T21:42:08.686678+0000","last_active":"2024-09-10T21:42:11.428604+0000","last_peered":"2024-09-10T21:42:11.428604+0000","last_clean":"2024-09-10T21:42:11.428604+0000","last_became_active":"2024-09-10T21:42:08.686395+0000","last_became_peered":"2024-09-10T21:42:08.686395+0000","last_unstale":"2024-09-10T21:42:11.428604+0000","last_undegraded":"2024-09-10T21:42:11.428604+0000","last_fullsized":"2024-09-10T21:42:11.428604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:50.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782250+0000","last_change":"2024-09-10T21:42:09.714231+0000","last_active":"2024-09-10T21:42:11.782250+0000","last_peered":"2024-09-10T21:42:11.782250+0000","last_clean":"2024-09-10T21:42:11.782250+0000","last_became_active":"2024-09-10T21:42:09.713873+0000","last_became_peered":"2024-09-10T21:42:09.713873+0000","last_unstale":"2024-09-10T21:42:11.782250+0000","last_undegraded":"2024-09-10T21:42:11.782250+0000","last_fullsized":"2024-09-10T21:42:11.782250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:50.638349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695399+0000","last_change":"2024-09-10T21:42:08.694275+0000","last_active":"2024-09-10T21:42:10.695399+0000","last_peered":"2024-09-10T21:42:10.695399+0000","last_clean":"2024-09-10T21:42:10.695399+0000","last_became_active":"2024-09-10T21:42:08.694014+0000","last_became_peered":"2024-09-10T21:42:08.694014+0000","last_unstale":"2024-09-10T21:42:10.695399+0000","last_undegraded":"2024-09-10T21:42:10.695399+0000","last_fullsized":"2024-09-10T21:42:10.695399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:52.021550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702618+0000","last_change":"2024-09-10T21:42:09.698139+0000","last_active":"2024-09-10T21:42:10.702618+0000","last_peered":"2024-09-10T21:42:10.702618+0000","last_clean":"2024-09-10T21:42:10.702618+0000","last_became_active":"2024-09-10T21:42:09.697391+0000","last_became_peered":"2024-09-10T21:42:09.697391+0000","last_unstale":"2024-09-10T21:42:10.702618+0000","last_undegraded":"2024-09-10T21:42:10.702618+0000","last_fullsized":"2024-09-10T21:42:10.702618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:47.497053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428569+0000","last_change":"2024-09-10T21:42:08.691652+0000","last_active":"2024-09-10T21:42:11.428569+0000","last_peered":"2024-09-10T21:42:11.428569+0000","last_clean":"2024-09-10T21:42:11.428569+0000","last_became_active":"2024-09-10T21:42:08.691317+0000","last_became_peered":"2024-09-10T21:42:08.691317+0000","last_unstale":"2024-09-10T21:42:11.428569+0000","last_undegraded":"2024-09-10T21:42:11.428569+0000","last_fullsized":"2024-09-10T21:42:11.428569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:10.198175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781834+0000","last_change":"2024-09-10T21:42:09.720201+0000","last_active":"2024-09-10T21:42:11.781834+0000","last_peered":"2024-09-10T21:42:11.781834+0000","last_clean":"2024-09-10T21:42:11.781834+0000","last_became_active":"2024-09-10T21:42:09.719872+0000","last_became_peered":"2024-09-10T21:42:09.719872+0000","last_unstale":"2024-09-10T21:42:11.781834+0000","last_undegraded":"2024-09-10T21:42:11.781834+0000","last_fullsized":"2024-09-10T21:42:11.781834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:31.222577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781370+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:42:11.781370+0000","last_peered":"2024-09-10T21:42:11.781370+0000","last_clean":"2024-09-10T21:42:11.781370+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:42:11.781370+0000","last_undegraded":"2024-09-10T21:42:11.781370+0000","last_fullsized":"2024-09-10T21:42:11.781370+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701380+0000","last_change":"2024-09-10T21:42:08.698035+0000","last_active":"2024-09-10T21:42:10.701380+0000","last_peered":"2024-09-10T21:42:10.701380+0000","last_clean":"2024-09-10T21:42:10.701380+0000","last_became_active":"2024-09-10T21:42:08.695961+0000","last_became_peered":"2024-09-10T21:42:08.695961+0000","last_unstale":"2024-09-10T21:42:10.701380+0000","last_undegraded":"2024-09-10T21:42:10.701380+0000","last_fullsized":"2024-09-10T21:42:10.701380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:16.471687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.698912+0000","last_change":"2024-09-10T21:42:09.692488+0000","last_active":"2024-09-10T21:42:09.698912+0000","last_peered":"2024-09-10T21:42:09.698912+0000","last_clean":"2024-09-10T21:42:09.698912+0000","last_became_active":"2024-09-10T21:42:09.691349+0000","last_became_peered":"2024-09-10T21:42:09.691349+0000","last_unstale":"2024-09-10T21:42:09.698912+0000","last_undegraded":"2024-09-10T21:42:09.698912+0000","last_fullsized":"2024-09-10T21:42:09.698912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:41.624422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781332+0000","last_change":"2024-09-10T21:42:08.706510+0000","last_active":"2024-09-10T21:42:11.781332+0000","last_peered":"2024-09-10T21:42:11.781332+0000","last_clean":"2024-09-10T21:42:11.781332+0000","last_became_active":"2024-09-10T21:42:08.706080+0000","last_became_peered":"2024-09-10T21:42:08.706080+0000","last_unstale":"2024-09-10T21:42:11.781332+0000","last_undegraded":"2024-09-10T21:42:11.781332+0000","last_fullsized":"2024-09-10T21:42:11.781332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:25.172297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975060+0000","last_change":"2024-09-10T21:42:09.698977+0000","last_active":"2024-09-10T21:42:11.975060+0000","last_peered":"2024-09-10T21:42:11.975060+0000","last_clean":"2024-09-10T21:42:11.975060+0000","last_became_active":"2024-09-10T21:42:09.698557+0000","last_became_peered":"2024-09-10T21:42:09.698557+0000","last_unstale":"2024-09-10T21:42:11.975060+0000","last_undegraded":"2024-09-10T21:42:11.975060+0000","last_fullsized":"2024-09-10T21:42:11.975060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:40.541602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428457+0000","last_change":"2024-09-10T21:42:08.689207+0000","last_active":"2024-09-10T21:42:11.428457+0000","last_peered":"2024-09-10T21:42:11.428457+0000","last_clean":"2024-09-10T21:42:11.428457+0000","last_became_active":"2024-09-10T21:42:08.688941+0000","last_became_peered":"2024-09-10T21:42:08.688941+0000","last_unstale":"2024-09-10T21:42:11.428457+0000","last_undegraded":"2024-09-10T21:42:11.428457+0000","last_fullsized":"2024-09-10T21:42:11.428457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:56.090844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695552+0000","last_change":"2024-09-10T21:42:09.688301+0000","last_active":"2024-09-10T21:42:10.695552+0000","last_peered":"2024-09-10T21:42:10.695552+0000","last_clean":"2024-09-10T21:42:10.695552+0000","last_became_active":"2024-09-10T21:42:09.687944+0000","last_became_peered":"2024-09-10T21:42:09.687944+0000","last_unstale":"2024-09-10T21:42:10.695552+0000","last_undegraded":"2024-09-10T21:42:10.695552+0000","last_fullsized":"2024-09-10T21:42:10.695552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:37.025454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701842+0000","last_change":"2024-09-10T21:42:08.698803+0000","last_active":"2024-09-10T21:42:10.701842+0000","last_peered":"2024-09-10T21:42:10.701842+0000","last_clean":"2024-09-10T21:42:10.701842+0000","last_became_active":"2024-09-10T21:42:08.697174+0000","last_became_peered":"2024-09-10T21:42:08.697174+0000","last_unstale":"2024-09-10T21:42:10.701842+0000","last_undegraded":"2024-09-10T21:42:10.701842+0000","last_fullsized":"2024-09-10T21:42:10.701842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:47.619335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702033+0000","last_change":"2024-09-10T21:42:09.688575+0000","last_active":"2024-09-10T21:42:10.702033+0000","last_peered":"2024-09-10T21:42:10.702033+0000","last_clean":"2024-09-10T21:42:10.702033+0000","last_became_active":"2024-09-10T21:42:09.688284+0000","last_became_peered":"2024-09-10T21:42:09.688284+0000","last_unstale":"2024-09-10T21:42:10.702033+0000","last_undegraded":"2024-09-10T21:42:10.702033+0000","last_fullsized":"2024-09-10T21:42:10.702033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:53.029762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695680+0000","last_change":"2024-09-10T21:42:08.698800+0000","last_active":"2024-09-10T21:42:10.695680+0000","last_peered":"2024-09-10T21:42:10.695680+0000","last_clean":"2024-09-10T21:42:10.695680+0000","last_became_active":"2024-09-10T21:42:08.696627+0000","last_became_peered":"2024-09-10T21:42:08.696627+0000","last_unstale":"2024-09-10T21:42:10.695680+0000","last_undegraded":"2024-09-10T21:42:10.695680+0000","last_fullsized":"2024-09-10T21:42:10.695680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:56.090844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701667+0000","last_change":"2024-09-10T21:42:09.700048+0000","last_active":"2024-09-10T21:42:10.701667+0000","last_peered":"2024-09-10T21:42:10.701667+0000","last_clean":"2024-09-10T21:42:10.701667+0000","last_became_active":"2024-09-10T21:42:09.699085+0000","last_became_peered":"2024-09-10T21:42:09.699085+0000","last_unstale":"2024-09-10T21:42:10.701667+0000","last_undegraded":"2024-09-10T21:42:10.701667+0000","last_fullsized":"2024-09-10T21:42:10.701667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:22.835437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782141+0000","last_change":"2024-09-10T21:42:08.704257+0000","last_active":"2024-09-10T21:42:11.782141+0000","last_peered":"2024-09-10T21:42:11.782141+0000","last_clean":"2024-09-10T21:42:11.782141+0000","last_became_active":"2024-09-10T21:42:08.703837+0000","last_became_peered":"2024-09-10T21:42:08.703837+0000","last_unstale":"2024-09-10T21:42:11.782141+0000","last_undegraded":"2024-09-10T21:42:11.782141+0000","last_fullsized":"2024-09-10T21:42:11.782141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:44.076555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695679+0000","last_change":"2024-09-10T21:42:09.695578+0000","last_active":"2024-09-10T21:42:10.695679+0000","last_peered":"2024-09-10T21:42:10.695679+0000","last_clean":"2024-09-10T21:42:10.695679+0000","last_became_active":"2024-09-10T21:42:09.693969+0000","last_became_peered":"2024-09-10T21:42:09.693969+0000","last_unstale":"2024-09-10T21:42:10.695679+0000","last_undegraded":"2024-09-10T21:42:10.695679+0000","last_fullsized":"2024-09-10T21:42:10.695679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:48.304330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695614+0000","last_change":"2024-09-10T21:42:08.699017+0000","last_active":"2024-09-10T21:42:10.695614+0000","last_peered":"2024-09-10T21:42:10.695614+0000","last_clean":"2024-09-10T21:42:10.695614+0000","last_became_active":"2024-09-10T21:42:08.698540+0000","last_became_peered":"2024-09-10T21:42:08.698540+0000","last_unstale":"2024-09-10T21:42:10.695614+0000","last_undegraded":"2024-09-10T21:42:10.695614+0000","last_fullsized":"2024-09-10T21:42:10.695614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:33.018106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.714319+0000","last_change":"2024-09-10T21:42:09.709063+0000","last_active":"2024-09-10T21:42:09.714319+0000","last_peered":"2024-09-10T21:42:09.714319+0000","last_clean":"2024-09-10T21:42:09.714319+0000","last_became_active":"2024-09-10T21:42:09.708726+0000","last_became_peered":"2024-09-10T21:42:09.708726+0000","last_unstale":"2024-09-10T21:42:09.714319+0000","last_undegraded":"2024-09-10T21:42:09.714319+0000","last_fullsized":"2024-09-10T21:42:09.714319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:35.888935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685182+0000","last_change":"2024-09-10T21:42:08.697045+0000","last_active":"2024-09-10T21:42:11.685182+0000","last_peered":"2024-09-10T21:42:11.685182+0000","last_clean":"2024-09-10T21:42:11.685182+0000","last_became_active":"2024-09-10T21:42:08.696483+0000","last_became_peered":"2024-09-10T21:42:08.696483+0000","last_unstale":"2024-09-10T21:42:11.685182+0000","last_undegraded":"2024-09-10T21:42:11.685182+0000","last_fullsized":"2024-09-10T21:42:11.685182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:34.907546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975478+0000","last_change":"2024-09-10T21:42:09.694168+0000","last_active":"2024-09-10T21:42:11.975478+0000","last_peered":"2024-09-10T21:42:11.975478+0000","last_clean":"2024-09-10T21:42:11.975478+0000","last_became_active":"2024-09-10T21:42:09.693779+0000","last_became_peered":"2024-09-10T21:42:09.693779+0000","last_unstale":"2024-09-10T21:42:11.975478+0000","last_undegraded":"2024-09-10T21:42:11.975478+0000","last_fullsized":"2024-09-10T21:42:11.975478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:24:55.155776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781313+0000","last_change":"2024-09-10T21:42:08.697319+0000","last_active":"2024-09-10T21:42:11.781313+0000","last_peered":"2024-09-10T21:42:11.781313+0000","last_clean":"2024-09-10T21:42:11.781313+0000","last_became_active":"2024-09-10T21:42:08.697039+0000","last_became_peered":"2024-09-10T21:42:08.697039+0000","last_unstale":"2024-09-10T21:42:11.781313+0000","last_undegraded":"2024-09-10T21:42:11.781313+0000","last_fullsized":"2024-09-10T21:42:11.781313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:48.264843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701623+0000","last_change":"2024-09-10T21:42:09.689554+0000","last_active":"2024-09-10T21:42:10.701623+0000","last_peered":"2024-09-10T21:42:10.701623+0000","last_clean":"2024-09-10T21:42:10.701623+0000","last_became_active":"2024-09-10T21:42:09.689041+0000","last_became_peered":"2024-09-10T21:42:09.689041+0000","last_unstale":"2024-09-10T21:42:10.701623+0000","last_undegraded":"2024-09-10T21:42:10.701623+0000","last_fullsized":"2024-09-10T21:42:10.701623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:48.627547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695404+0000","last_change":"2024-09-10T21:42:08.700219+0000","last_active":"2024-09-10T21:42:10.695404+0000","last_peered":"2024-09-10T21:42:10.695404+0000","last_clean":"2024-09-10T21:42:10.695404+0000","last_became_active":"2024-09-10T21:42:08.696072+0000","last_became_peered":"2024-09-10T21:42:08.696072+0000","last_unstale":"2024-09-10T21:42:10.695404+0000","last_undegraded":"2024-09-10T21:42:10.695404+0000","last_fullsized":"2024-09-10T21:42:10.695404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:50.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.699090+0000","last_change":"2024-09-10T21:42:09.695584+0000","last_active":"2024-09-10T21:42:09.699090+0000","last_peered":"2024-09-10T21:42:09.699090+0000","last_clean":"2024-09-10T21:42:09.699090+0000","last_became_active":"2024-09-10T21:42:09.695169+0000","last_became_peered":"2024-09-10T21:42:09.695169+0000","last_unstale":"2024-09-10T21:42:09.699090+0000","last_undegraded":"2024-09-10T21:42:09.699090+0000","last_fullsized":"2024-09-10T21:42:09.699090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:26.180509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695459+0000","last_change":"2024-09-10T21:42:08.702389+0000","last_active":"2024-09-10T21:42:10.695459+0000","last_peered":"2024-09-10T21:42:10.695459+0000","last_clean":"2024-09-10T21:42:10.695459+0000","last_became_active":"2024-09-10T21:42:08.701730+0000","last_became_peered":"2024-09-10T21:42:08.701730+0000","last_unstale":"2024-09-10T21:42:10.695459+0000","last_undegraded":"2024-09-10T21:42:10.695459+0000","last_fullsized":"2024-09-10T21:42:10.695459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:47.619335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.698786+0000","last_change":"2024-09-10T21:42:09.691569+0000","last_active":"2024-09-10T21:42:09.698786+0000","last_peered":"2024-09-10T21:42:09.698786+0000","last_clean":"2024-09-10T21:42:09.698786+0000","last_became_active":"2024-09-10T21:42:09.691270+0000","last_became_peered":"2024-09-10T21:42:09.691270+0000","last_unstale":"2024-09-10T21:42:09.698786+0000","last_undegraded":"2024-09-10T21:42:09.698786+0000","last_fullsized":"2024-09-10T21:42:09.698786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:53.886934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782013+0000","last_change":"2024-09-10T21:42:08.702598+0000","last_active":"2024-09-10T21:42:11.782013+0000","last_peered":"2024-09-10T21:42:11.782013+0000","last_clean":"2024-09-10T21:42:11.782013+0000","last_became_active":"2024-09-10T21:42:08.702352+0000","last_became_peered":"2024-09-10T21:42:08.702352+0000","last_unstale":"2024-09-10T21:42:11.782013+0000","last_undegraded":"2024-09-10T21:42:11.782013+0000","last_fullsized":"2024-09-10T21:42:11.782013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:45.571003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685024+0000","last_change":"2024-09-10T21:42:09.708756+0000","last_active":"2024-09-10T21:42:11.685024+0000","last_peered":"2024-09-10T21:42:11.685024+0000","last_clean":"2024-09-10T21:42:11.685024+0000","last_became_active":"2024-09-10T21:42:09.708458+0000","last_became_peered":"2024-09-10T21:42:09.708458+0000","last_unstale":"2024-09-10T21:42:11.685024+0000","last_undegraded":"2024-09-10T21:42:11.685024+0000","last_fullsized":"2024-09-10T21:42:11.685024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:37.025454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701552+0000","last_change":"2024-09-10T21:42:09.692810+0000","last_active":"2024-09-10T21:42:10.701552+0000","last_peered":"2024-09-10T21:42:10.701552+0000","last_clean":"2024-09-10T21:42:10.701552+0000","last_became_active":"2024-09-10T21:42:09.692537+0000","last_became_peered":"2024-09-10T21:42:09.692537+0000","last_unstale":"2024-09-10T21:42:10.701552+0000","last_undegraded":"2024-09-10T21:42:10.701552+0000","last_fullsized":"2024-09-10T21:42:10.701552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:55.273108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781669+0000","last_change":"2024-09-10T21:42:08.705305+0000","last_active":"2024-09-10T21:42:11.781669+0000","last_peered":"2024-09-10T21:42:11.781669+0000","last_clean":"2024-09-10T21:42:11.781669+0000","last_became_active":"2024-09-10T21:42:08.704991+0000","last_became_peered":"2024-09-10T21:42:08.704991+0000","last_unstale":"2024-09-10T21:42:11.781669+0000","last_undegraded":"2024-09-10T21:42:11.781669+0000","last_fullsized":"2024-09-10T21:42:11.781669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:59.418988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702644+0000","last_change":"2024-09-10T21:42:09.704028+0000","last_active":"2024-09-10T21:42:10.702644+0000","last_peered":"2024-09-10T21:42:10.702644+0000","last_clean":"2024-09-10T21:42:10.702644+0000","last_became_active":"2024-09-10T21:42:09.703608+0000","last_became_peered":"2024-09-10T21:42:09.703608+0000","last_unstale":"2024-09-10T21:42:10.702644+0000","last_undegraded":"2024-09-10T21:42:10.702644+0000","last_fullsized":"2024-09-10T21:42:10.702644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:11.626660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782020+0000","last_change":"2024-09-10T21:42:08.704448+0000","last_active":"2024-09-10T21:42:11.782020+0000","last_peered":"2024-09-10T21:42:11.782020+0000","last_clean":"2024-09-10T21:42:11.782020+0000","last_became_active":"2024-09-10T21:42:08.704156+0000","last_became_peered":"2024-09-10T21:42:08.704156+0000","last_unstale":"2024-09-10T21:42:11.782020+0000","last_undegraded":"2024-09-10T21:42:11.782020+0000","last_fullsized":"2024-09-10T21:42:11.782020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:07.043672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975512+0000","last_change":"2024-09-10T21:42:09.693556+0000","last_active":"2024-09-10T21:42:11.975512+0000","last_peered":"2024-09-10T21:42:11.975512+0000","last_clean":"2024-09-10T21:42:11.975512+0000","last_became_active":"2024-09-10T21:42:09.693330+0000","last_became_peered":"2024-09-10T21:42:09.693330+0000","last_unstale":"2024-09-10T21:42:11.975512+0000","last_undegraded":"2024-09-10T21:42:11.975512+0000","last_fullsized":"2024-09-10T21:42:11.975512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:26:39.608345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.717090+0000","last_change":"2024-09-10T21:42:08.699157+0000","last_active":"2024-09-10T21:42:09.717090+0000","last_peered":"2024-09-10T21:42:09.717090+0000","last_clean":"2024-09-10T21:42:09.717090+0000","last_became_active":"2024-09-10T21:42:08.691408+0000","last_became_peered":"2024-09-10T21:42:08.691408+0000","last_unstale":"2024-09-10T21:42:09.717090+0000","last_undegraded":"2024-09-10T21:42:09.717090+0000","last_fullsized":"2024-09-10T21:42:09.717090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:52.464492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.708009+0000","last_change":"2024-09-10T21:42:09.704294+0000","last_active":"2024-09-10T21:42:09.708009+0000","last_peered":"2024-09-10T21:42:09.708009+0000","last_clean":"2024-09-10T21:42:09.708009+0000","last_became_active":"2024-09-10T21:42:09.703607+0000","last_became_peered":"2024-09-10T21:42:09.703607+0000","last_unstale":"2024-09-10T21:42:09.708009+0000","last_undegraded":"2024-09-10T21:42:09.708009+0000","last_fullsized":"2024-09-10T21:42:09.708009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:08.051884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975594+0000","last_change":"2024-09-10T21:42:08.702462+0000","last_active":"2024-09-10T21:42:11.975594+0000","last_peered":"2024-09-10T21:42:11.975594+0000","last_clean":"2024-09-10T21:42:11.975594+0000","last_became_active":"2024-09-10T21:42:08.702103+0000","last_became_peered":"2024-09-10T21:42:08.702103+0000","last_unstale":"2024-09-10T21:42:11.975594+0000","last_undegraded":"2024-09-10T21:42:11.975594+0000","last_fullsized":"2024-09-10T21:42:11.975594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:30.979951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":72277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":72313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":72187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":815104,"data_stored":663193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":794624,"data_stored":656837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":208896,"data_stored":67499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:42:13.945 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:42:14.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.139+0000 7f5befe00640 1 -- 172.21.15.7:0/851363105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf015dfd0 msgr2=0x7f5bf00d4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:14.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.139+0000 7f5befe00640 1 --2- 172.21.15.7:0/851363105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf015dfd0 0x7f5bf00d4870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009830 tx=0x7f5be002a530 comp rx=0 tx=0).stop 2024-09-10T21:42:14.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.139+0000 7f5befe00640 1 -- 172.21.15.7:0/851363105 shutdown_connections 2024-09-10T21:42:14.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.139+0000 7f5befe00640 1 --2- 172.21.15.7:0/851363105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf015dfd0 0x7f5bf00d4870 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.139+0000 7f5befe00640 1 -- 172.21.15.7:0/851363105 >> 172.21.15.7:0/851363105 conn(0x7f5bf00d3d50 msgr2=0x7f5bf00d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:14.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.139+0000 7f5befe00640 1 -- 172.21.15.7:0/851363105 shutdown_connections 2024-09-10T21:42:14.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.139+0000 7f5befe00640 1 -- 172.21.15.7:0/851363105 wait complete. 2024-09-10T21:42:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.140+0000 7f5befe00640 1 Processor -- start 2024-09-10T21:42:14.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.141+0000 7f5befe00640 1 -- start start 2024-09-10T21:42:14.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.142+0000 7f5befe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bf00b2b50 0x7f5bf00cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.142+0000 7f5befe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf00d0e70 0x7f5bf01f8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.142+0000 7f5befe00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5bf015dfd0 0x7f5bf0156860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.142+0000 7f5befe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf00d2770 con 0x7f5bf00b2b50 2024-09-10T21:42:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.142+0000 7f5befe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5bf00d25f0 con 0x7f5bf00d0e70 2024-09-10T21:42:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.142+0000 7f5befe00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f5bf00d28f0 con 0x7f5bf015dfd0 2024-09-10T21:42:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.142+0000 7f5beea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bf00b2b50 0x7f5bf00cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.142+0000 7f5beea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bf00b2b50 0x7f5bf00cae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54038/0 (socket says 172.21.15.7:54038) 2024-09-10T21:42:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.142+0000 7f5beea00640 1 -- 172.21.15.7:0/3911985853 learned_addr learned my addr 172.21.15.7:0/3911985853 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.142+0000 7f5bef400640 1 --2- 172.21.15.7:0/3911985853 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5bf015dfd0 0x7f5bf0156860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.142+0000 7f5bee000640 1 --2- 172.21.15.7:0/3911985853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf00d0e70 0x7f5bf01f8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.142+0000 7f5beea00640 1 -- 172.21.15.7:0/3911985853 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5bf015dfd0 msgr2=0x7f5bf0156860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:14.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.142+0000 7f5beea00640 1 --2- 172.21.15.7:0/3911985853 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5bf015dfd0 0x7f5bf0156860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.142+0000 7f5beea00640 1 -- 172.21.15.7:0/3911985853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf00d0e70 msgr2=0x7f5bf01f8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:14.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.142+0000 7f5beea00640 1 --2- 172.21.15.7:0/3911985853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf00d0e70 0x7f5bf01f8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.142+0000 7f5beea00640 1 -- 172.21.15.7:0/3911985853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bf00b8260 con 0x7f5bf00b2b50 2024-09-10T21:42:14.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.143+0000 7f5beea00640 1 --2- 172.21.15.7:0/3911985853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bf00b2b50 0x7f5bf00cae20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009950 tx=0x7f5be0033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:14.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.143+0000 7f5be6a00640 1 -- 172.21.15.7:0/3911985853 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0031b50 con 0x7f5bf00b2b50 2024-09-10T21:42:14.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.143+0000 7f5be6a00640 1 -- 172.21.15.7:0/3911985853 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5be0031de0 con 0x7f5bf00b2b50 2024-09-10T21:42:14.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.143+0000 7f5befe00640 1 -- 172.21.15.7:0/3911985853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bf00b8520 con 0x7f5bf00b2b50 2024-09-10T21:42:14.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.143+0000 7f5be6a00640 1 -- 172.21.15.7:0/3911985853 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5be0005280 con 0x7f5bf00b2b50 2024-09-10T21:42:14.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.143+0000 7f5befe00640 1 -- 172.21.15.7:0/3911985853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bf01f83b0 con 0x7f5bf00b2b50 2024-09-10T21:42:14.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.144+0000 7f5befe00640 1 -- 172.21.15.7:0/3911985853 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bf00d2d50 con 0x7f5bf00b2b50 2024-09-10T21:42:14.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.145+0000 7f5be6a00640 1 -- 172.21.15.7:0/3911985853 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5be0005420 con 0x7f5bf00b2b50 2024-09-10T21:42:14.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.146+0000 7f5be6a00640 1 --2- 172.21.15.7:0/3911985853 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5bd0077f70 0x7f5bd007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:14.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.146+0000 7f5be6a00640 1 -- 172.21.15.7:0/3911985853 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f5be00bae00 con 0x7f5bf00b2b50 2024-09-10T21:42:14.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.146+0000 7f5bee000640 1 --2- 172.21.15.7:0/3911985853 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5bd0077f70 0x7f5bd007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:14.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.147+0000 7f5bee000640 1 --2- 172.21.15.7:0/3911985853 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5bd0077f70 0x7f5bd007a430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00cb360 tx=0x7f5bdc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:14.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.150+0000 7f5be6a00640 1 -- 172.21.15.7:0/3911985853 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5be003f030 con 0x7f5bf00b2b50 2024-09-10T21:42:14.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.342+0000 7f5befe00640 1 -- 172.21.15.7:0/3911985853 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5bf00b9cd0 con 0x7f5bd0077f70 2024-09-10T21:42:14.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.347+0000 7f5be6a00640 1 -- 172.21.15.7:0/3911985853 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251935 (secure 0 0 0) 0x7f5bf00b9cd0 con 0x7f5bd0077f70 2024-09-10T21:42:14.351 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:14.352 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:42:14.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.351+0000 7f5befe00640 1 -- 172.21.15.7:0/3911985853 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5bd0077f70 msgr2=0x7f5bd007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:14.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.351+0000 7f5befe00640 1 --2- 172.21.15.7:0/3911985853 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5bd0077f70 0x7f5bd007a430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5bf00cb360 tx=0x7f5bdc009040 comp rx=0 tx=0).stop 2024-09-10T21:42:14.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.351+0000 7f5befe00640 1 -- 172.21.15.7:0/3911985853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bf00b2b50 msgr2=0x7f5bf00cae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:14.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.351+0000 7f5befe00640 1 --2- 172.21.15.7:0/3911985853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bf00b2b50 0x7f5bf00cae20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5be0009950 tx=0x7f5be0033ba0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.351+0000 7f5befe00640 1 -- 172.21.15.7:0/3911985853 shutdown_connections 2024-09-10T21:42:14.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.351+0000 7f5befe00640 1 --2- 172.21.15.7:0/3911985853 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5bd0077f70 0x7f5bd007a430 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.351+0000 7f5befe00640 1 --2- 172.21.15.7:0/3911985853 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5bf00d0e70 0x7f5bf01f8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.351+0000 7f5befe00640 1 --2- 172.21.15.7:0/3911985853 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5bf015dfd0 0x7f5bf0156860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.351+0000 7f5befe00640 1 --2- 172.21.15.7:0/3911985853 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5bf00b2b50 0x7f5bf00cae20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.351+0000 7f5befe00640 1 -- 172.21.15.7:0/3911985853 >> 172.21.15.7:0/3911985853 conn(0x7f5bf00d3d50 msgr2=0x7f5bf00d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:14.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.351+0000 7f5befe00640 1 -- 172.21.15.7:0/3911985853 shutdown_connections 2024-09-10T21:42:14.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.351+0000 7f5befe00640 1 -- 172.21.15.7:0/3911985853 wait complete. 2024-09-10T21:42:14.367 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2024-09-10T21:42:13.856571+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235704,"kb_used_data":2832,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754739016,"statfs":{"total":773094113280,"available":772852752384,"internally_reserved":0,"allocated":2899968,"data_stored":1750452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001359"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782002+0000","last_change":"2024-09-10T21:42:09.712975+0000","last_active":"2024-09-10T21:42:11.782002+0000","last_peered":"2024-09-10T21:42:11.782002+0000","last_clean":"2024-09-10T21:42:11.782002+0000","last_became_active":"2024-09-10T21:42:09.712242+0000","last_became_peered":"2024-09-10T21:42:09.712242+0000","last_unstale":"2024-09-10T21:42:11.782002+0000","last_undegraded":"2024-09-10T21:42:11.782002+0000","last_fullsized":"2024-09-10T21:42:11.782002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:00.850316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695614+0000","last_change":"2024-09-10T21:42:09.697173+0000","last_active":"2024-09-10T21:42:10.695614+0000","last_peered":"2024-09-10T21:42:10.695614+0000","last_clean":"2024-09-10T21:42:10.695614+0000","last_became_active":"2024-09-10T21:42:09.696835+0000","last_became_peered":"2024-09-10T21:42:09.696835+0000","last_unstale":"2024-09-10T21:42:10.695614+0000","last_undegraded":"2024-09-10T21:42:10.695614+0000","last_fullsized":"2024-09-10T21:42:10.695614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:13.873778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975496+0000","last_change":"2024-09-10T21:42:09.693813+0000","last_active":"2024-09-10T21:42:11.975496+0000","last_peered":"2024-09-10T21:42:11.975496+0000","last_clean":"2024-09-10T21:42:11.975496+0000","last_became_active":"2024-09-10T21:42:09.693499+0000","last_became_peered":"2024-09-10T21:42:09.693499+0000","last_unstale":"2024-09-10T21:42:11.975496+0000","last_undegraded":"2024-09-10T21:42:11.975496+0000","last_fullsized":"2024-09-10T21:42:11.975496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:51.172800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702414+0000","last_change":"2024-09-10T21:42:09.698507+0000","last_active":"2024-09-10T21:42:10.702414+0000","last_peered":"2024-09-10T21:42:10.702414+0000","last_clean":"2024-09-10T21:42:10.702414+0000","last_became_active":"2024-09-10T21:42:09.697845+0000","last_became_peered":"2024-09-10T21:42:09.697845+0000","last_unstale":"2024-09-10T21:42:10.702414+0000","last_undegraded":"2024-09-10T21:42:10.702414+0000","last_fullsized":"2024-09-10T21:42:10.702414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:34.523014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781839+0000","last_change":"2024-09-10T21:42:09.703989+0000","last_active":"2024-09-10T21:42:11.781839+0000","last_peered":"2024-09-10T21:42:11.781839+0000","last_clean":"2024-09-10T21:42:11.781839+0000","last_became_active":"2024-09-10T21:42:09.703641+0000","last_became_peered":"2024-09-10T21:42:09.703641+0000","last_unstale":"2024-09-10T21:42:11.781839+0000","last_undegraded":"2024-09-10T21:42:11.781839+0000","last_fullsized":"2024-09-10T21:42:11.781839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:51.463128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701240+0000","last_change":"2024-09-10T21:42:09.691432+0000","last_active":"2024-09-10T21:42:10.701240+0000","last_peered":"2024-09-10T21:42:10.701240+0000","last_clean":"2024-09-10T21:42:10.701240+0000","last_became_active":"2024-09-10T21:42:09.691029+0000","last_became_peered":"2024-09-10T21:42:09.691029+0000","last_unstale":"2024-09-10T21:42:10.701240+0000","last_undegraded":"2024-09-10T21:42:10.701240+0000","last_fullsized":"2024-09-10T21:42:10.701240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:06.860570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781477+0000","last_change":"2024-09-10T21:42:09.716770+0000","last_active":"2024-09-10T21:42:11.781477+0000","last_peered":"2024-09-10T21:42:11.781477+0000","last_clean":"2024-09-10T21:42:11.781477+0000","last_became_active":"2024-09-10T21:42:09.716490+0000","last_became_peered":"2024-09-10T21:42:09.716490+0000","last_unstale":"2024-09-10T21:42:11.781477+0000","last_undegraded":"2024-09-10T21:42:11.781477+0000","last_fullsized":"2024-09-10T21:42:11.781477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:45.131035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.699076+0000","last_change":"2024-09-10T21:42:09.695560+0000","last_active":"2024-09-10T21:42:09.699076+0000","last_peered":"2024-09-10T21:42:09.699076+0000","last_clean":"2024-09-10T21:42:09.699076+0000","last_became_active":"2024-09-10T21:42:09.695252+0000","last_became_peered":"2024-09-10T21:42:09.695252+0000","last_unstale":"2024-09-10T21:42:09.699076+0000","last_undegraded":"2024-09-10T21:42:09.699076+0000","last_fullsized":"2024-09-10T21:42:09.699076+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:49.273055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782198+0000","last_change":"2024-09-10T21:42:09.713256+0000","last_active":"2024-09-10T21:42:11.782198+0000","last_peered":"2024-09-10T21:42:11.782198+0000","last_clean":"2024-09-10T21:42:11.782198+0000","last_became_active":"2024-09-10T21:42:09.712700+0000","last_became_peered":"2024-09-10T21:42:09.712700+0000","last_unstale":"2024-09-10T21:42:11.782198+0000","last_undegraded":"2024-09-10T21:42:11.782198+0000","last_fullsized":"2024-09-10T21:42:11.782198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:19.953355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685329+0000","last_change":"2024-09-10T21:42:09.702384+0000","last_active":"2024-09-10T21:42:11.685329+0000","last_peered":"2024-09-10T21:42:11.685329+0000","last_clean":"2024-09-10T21:42:11.685329+0000","last_became_active":"2024-09-10T21:42:09.698800+0000","last_became_peered":"2024-09-10T21:42:09.698800+0000","last_unstale":"2024-09-10T21:42:11.685329+0000","last_undegraded":"2024-09-10T21:42:11.685329+0000","last_fullsized":"2024-09-10T21:42:11.685329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:13.873778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685153+0000","last_change":"2024-09-10T21:42:09.704731+0000","last_active":"2024-09-10T21:42:11.685153+0000","last_peered":"2024-09-10T21:42:11.685153+0000","last_clean":"2024-09-10T21:42:11.685153+0000","last_became_active":"2024-09-10T21:42:09.698600+0000","last_became_peered":"2024-09-10T21:42:09.698600+0000","last_unstale":"2024-09-10T21:42:11.685153+0000","last_undegraded":"2024-09-10T21:42:11.685153+0000","last_fullsized":"2024-09-10T21:42:11.685153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:06.304338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701569+0000","last_change":"2024-09-10T21:42:09.710639+0000","last_active":"2024-09-10T21:42:10.701569+0000","last_peered":"2024-09-10T21:42:10.701569+0000","last_clean":"2024-09-10T21:42:10.701569+0000","last_became_active":"2024-09-10T21:42:09.710337+0000","last_became_peered":"2024-09-10T21:42:09.710337+0000","last_unstale":"2024-09-10T21:42:10.701569+0000","last_undegraded":"2024-09-10T21:42:10.701569+0000","last_fullsized":"2024-09-10T21:42:10.701569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:47.171015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.708774+0000","last_change":"2024-09-10T21:42:09.706632+0000","last_active":"2024-09-10T21:42:09.708774+0000","last_peered":"2024-09-10T21:42:09.708774+0000","last_clean":"2024-09-10T21:42:09.708774+0000","last_became_active":"2024-09-10T21:42:09.706366+0000","last_became_peered":"2024-09-10T21:42:09.706366+0000","last_unstale":"2024-09-10T21:42:09.708774+0000","last_undegraded":"2024-09-10T21:42:09.708774+0000","last_fullsized":"2024-09-10T21:42:09.708774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:46:40.730728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.974972+0000","last_change":"2024-09-10T21:42:09.696596+0000","last_active":"2024-09-10T21:42:11.974972+0000","last_peered":"2024-09-10T21:42:11.974972+0000","last_clean":"2024-09-10T21:42:11.974972+0000","last_became_active":"2024-09-10T21:42:09.696292+0000","last_became_peered":"2024-09-10T21:42:09.696292+0000","last_unstale":"2024-09-10T21:42:11.974972+0000","last_undegraded":"2024-09-10T21:42:11.974972+0000","last_fullsized":"2024-09-10T21:42:11.974972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:49.235638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695329+0000","last_change":"2024-09-10T21:42:09.701498+0000","last_active":"2024-09-10T21:42:10.695329+0000","last_peered":"2024-09-10T21:42:10.695329+0000","last_clean":"2024-09-10T21:42:10.695329+0000","last_became_active":"2024-09-10T21:42:09.701149+0000","last_became_peered":"2024-09-10T21:42:09.701149+0000","last_unstale":"2024-09-10T21:42:10.695329+0000","last_undegraded":"2024-09-10T21:42:10.695329+0000","last_fullsized":"2024-09-10T21:42:10.695329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:46.339409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.700962+0000","last_change":"2024-09-10T21:42:09.694844+0000","last_active":"2024-09-10T21:42:10.700962+0000","last_peered":"2024-09-10T21:42:10.700962+0000","last_clean":"2024-09-10T21:42:10.700962+0000","last_became_active":"2024-09-10T21:42:09.694573+0000","last_became_peered":"2024-09-10T21:42:09.694573+0000","last_unstale":"2024-09-10T21:42:10.700962+0000","last_undegraded":"2024-09-10T21:42:10.700962+0000","last_fullsized":"2024-09-10T21:42:10.700962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:13.873778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702591+0000","last_change":"2024-09-10T21:42:09.707330+0000","last_active":"2024-09-10T21:42:10.702591+0000","last_peered":"2024-09-10T21:42:10.702591+0000","last_clean":"2024-09-10T21:42:10.702591+0000","last_became_active":"2024-09-10T21:42:09.707071+0000","last_became_peered":"2024-09-10T21:42:09.707071+0000","last_unstale":"2024-09-10T21:42:10.702591+0000","last_undegraded":"2024-09-10T21:42:10.702591+0000","last_fullsized":"2024-09-10T21:42:10.702591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:33.348954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428727+0000","last_change":"2024-09-10T21:42:09.709362+0000","last_active":"2024-09-10T21:42:11.428727+0000","last_peered":"2024-09-10T21:42:11.428727+0000","last_clean":"2024-09-10T21:42:11.428727+0000","last_became_active":"2024-09-10T21:42:09.707734+0000","last_became_peered":"2024-09-10T21:42:09.707734+0000","last_unstale":"2024-09-10T21:42:11.428727+0000","last_undegraded":"2024-09-10T21:42:11.428727+0000","last_fullsized":"2024-09-10T21:42:11.428727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:54.172897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782132+0000","last_change":"2024-09-10T21:42:09.709392+0000","last_active":"2024-09-10T21:42:11.782132+0000","last_peered":"2024-09-10T21:42:11.782132+0000","last_clean":"2024-09-10T21:42:11.782132+0000","last_became_active":"2024-09-10T21:42:09.708821+0000","last_became_peered":"2024-09-10T21:42:09.708821+0000","last_unstale":"2024-09-10T21:42:11.782132+0000","last_undegraded":"2024-09-10T21:42:11.782132+0000","last_fullsized":"2024-09-10T21:42:11.782132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:52.646979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685221+0000","last_change":"2024-09-10T21:42:09.711112+0000","last_active":"2024-09-10T21:42:11.685221+0000","last_peered":"2024-09-10T21:42:11.685221+0000","last_clean":"2024-09-10T21:42:11.685221+0000","last_became_active":"2024-09-10T21:42:09.710589+0000","last_became_peered":"2024-09-10T21:42:09.710589+0000","last_unstale":"2024-09-10T21:42:11.685221+0000","last_undegraded":"2024-09-10T21:42:11.685221+0000","last_fullsized":"2024-09-10T21:42:11.685221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:51.172800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975107+0000","last_change":"2024-09-10T21:42:09.707660+0000","last_active":"2024-09-10T21:42:11.975107+0000","last_peered":"2024-09-10T21:42:11.975107+0000","last_clean":"2024-09-10T21:42:11.975107+0000","last_became_active":"2024-09-10T21:42:09.707031+0000","last_became_peered":"2024-09-10T21:42:09.707031+0000","last_unstale":"2024-09-10T21:42:11.975107+0000","last_undegraded":"2024-09-10T21:42:11.975107+0000","last_fullsized":"2024-09-10T21:42:11.975107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:13.474744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701445+0000","last_change":"2024-09-10T21:42:09.708730+0000","last_active":"2024-09-10T21:42:10.701445+0000","last_peered":"2024-09-10T21:42:10.701445+0000","last_clean":"2024-09-10T21:42:10.701445+0000","last_became_active":"2024-09-10T21:42:09.708056+0000","last_became_peered":"2024-09-10T21:42:09.708056+0000","last_unstale":"2024-09-10T21:42:10.701445+0000","last_undegraded":"2024-09-10T21:42:10.701445+0000","last_fullsized":"2024-09-10T21:42:10.701445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:58.932679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.710662+0000","last_change":"2024-09-10T21:42:09.705245+0000","last_active":"2024-09-10T21:42:09.710662+0000","last_peered":"2024-09-10T21:42:09.710662+0000","last_clean":"2024-09-10T21:42:09.710662+0000","last_became_active":"2024-09-10T21:42:09.699361+0000","last_became_peered":"2024-09-10T21:42:09.699361+0000","last_unstale":"2024-09-10T21:42:09.710662+0000","last_undegraded":"2024-09-10T21:42:09.710662+0000","last_fullsized":"2024-09-10T21:42:09.710662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:45.084767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701128+0000","last_change":"2024-09-10T21:42:09.704790+0000","last_active":"2024-09-10T21:42:10.701128+0000","last_peered":"2024-09-10T21:42:10.701128+0000","last_clean":"2024-09-10T21:42:10.701128+0000","last_became_active":"2024-09-10T21:42:09.699333+0000","last_became_peered":"2024-09-10T21:42:09.699333+0000","last_unstale":"2024-09-10T21:42:10.701128+0000","last_undegraded":"2024-09-10T21:42:10.701128+0000","last_fullsized":"2024-09-10T21:42:10.701128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:48.304330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.714909+0000","last_change":"2024-09-10T21:42:09.712518+0000","last_active":"2024-09-10T21:42:09.714909+0000","last_peered":"2024-09-10T21:42:09.714909+0000","last_clean":"2024-09-10T21:42:09.714909+0000","last_became_active":"2024-09-10T21:42:09.711734+0000","last_became_peered":"2024-09-10T21:42:09.711734+0000","last_unstale":"2024-09-10T21:42:09.714909+0000","last_undegraded":"2024-09-10T21:42:09.714909+0000","last_fullsized":"2024-09-10T21:42:09.714909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:04.871576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685338+0000","last_change":"2024-09-10T21:42:09.713125+0000","last_active":"2024-09-10T21:42:11.685338+0000","last_peered":"2024-09-10T21:42:11.685338+0000","last_clean":"2024-09-10T21:42:11.685338+0000","last_became_active":"2024-09-10T21:42:09.711555+0000","last_became_peered":"2024-09-10T21:42:09.711555+0000","last_unstale":"2024-09-10T21:42:11.685338+0000","last_undegraded":"2024-09-10T21:42:11.685338+0000","last_fullsized":"2024-09-10T21:42:11.685338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:48.304330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695578+0000","last_change":"2024-09-10T21:42:09.698227+0000","last_active":"2024-09-10T21:42:10.695578+0000","last_peered":"2024-09-10T21:42:10.695578+0000","last_clean":"2024-09-10T21:42:10.695578+0000","last_became_active":"2024-09-10T21:42:09.695964+0000","last_became_peered":"2024-09-10T21:42:09.695964+0000","last_unstale":"2024-09-10T21:42:10.695578+0000","last_undegraded":"2024-09-10T21:42:10.695578+0000","last_fullsized":"2024-09-10T21:42:10.695578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:51.172800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.705104+0000","last_change":"2024-09-10T21:42:09.699013+0000","last_active":"2024-09-10T21:42:09.705104+0000","last_peered":"2024-09-10T21:42:09.705104+0000","last_clean":"2024-09-10T21:42:09.705104+0000","last_became_active":"2024-09-10T21:42:09.698647+0000","last_became_peered":"2024-09-10T21:42:09.698647+0000","last_unstale":"2024-09-10T21:42:09.705104+0000","last_undegraded":"2024-09-10T21:42:09.705104+0000","last_fullsized":"2024-09-10T21:42:09.705104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:00:00.427200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975129+0000","last_change":"2024-09-10T21:42:09.697577+0000","last_active":"2024-09-10T21:42:11.975129+0000","last_peered":"2024-09-10T21:42:11.975129+0000","last_clean":"2024-09-10T21:42:11.975129+0000","last_became_active":"2024-09-10T21:42:09.696978+0000","last_became_peered":"2024-09-10T21:42:09.696978+0000","last_unstale":"2024-09-10T21:42:11.975129+0000","last_undegraded":"2024-09-10T21:42:11.975129+0000","last_fullsized":"2024-09-10T21:42:11.975129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:37.025454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701665+0000","last_change":"2024-09-10T21:42:09.697210+0000","last_active":"2024-09-10T21:42:10.701665+0000","last_peered":"2024-09-10T21:42:10.701665+0000","last_clean":"2024-09-10T21:42:10.701665+0000","last_became_active":"2024-09-10T21:42:09.696910+0000","last_became_peered":"2024-09-10T21:42:09.696910+0000","last_unstale":"2024-09-10T21:42:10.701665+0000","last_undegraded":"2024-09-10T21:42:10.701665+0000","last_fullsized":"2024-09-10T21:42:10.701665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:17.732763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428577+0000","last_change":"2024-09-10T21:42:09.702664+0000","last_active":"2024-09-10T21:42:11.428577+0000","last_peered":"2024-09-10T21:42:11.428577+0000","last_clean":"2024-09-10T21:42:11.428577+0000","last_became_active":"2024-09-10T21:42:09.700834+0000","last_became_peered":"2024-09-10T21:42:09.700834+0000","last_unstale":"2024-09-10T21:42:11.428577+0000","last_undegraded":"2024-09-10T21:42:11.428577+0000","last_fullsized":"2024-09-10T21:42:11.428577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:35.668254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.684927+0000","last_change":"2024-09-10T21:42:09.707098+0000","last_active":"2024-09-10T21:42:11.684927+0000","last_peered":"2024-09-10T21:42:11.684927+0000","last_clean":"2024-09-10T21:42:11.684927+0000","last_became_active":"2024-09-10T21:42:09.706747+0000","last_became_peered":"2024-09-10T21:42:09.706747+0000","last_unstale":"2024-09-10T21:42:11.684927+0000","last_undegraded":"2024-09-10T21:42:11.684927+0000","last_fullsized":"2024-09-10T21:42:11.684927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:38.013273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701269+0000","last_change":"2024-09-10T21:42:09.694901+0000","last_active":"2024-09-10T21:42:10.701269+0000","last_peered":"2024-09-10T21:42:10.701269+0000","last_clean":"2024-09-10T21:42:10.701269+0000","last_became_active":"2024-09-10T21:42:09.694509+0000","last_became_peered":"2024-09-10T21:42:09.694509+0000","last_unstale":"2024-09-10T21:42:10.701269+0000","last_undegraded":"2024-09-10T21:42:10.701269+0000","last_fullsized":"2024-09-10T21:42:10.701269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:57.099056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.716471+0000","last_change":"2024-09-10T21:42:08.690816+0000","last_active":"2024-09-10T21:42:09.716471+0000","last_peered":"2024-09-10T21:42:09.716471+0000","last_clean":"2024-09-10T21:42:09.716471+0000","last_became_active":"2024-09-10T21:42:08.689282+0000","last_became_peered":"2024-09-10T21:42:08.689282+0000","last_unstale":"2024-09-10T21:42:09.716471+0000","last_undegraded":"2024-09-10T21:42:09.716471+0000","last_fullsized":"2024-09-10T21:42:09.716471+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:17.328468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975323+0000","last_change":"2024-09-10T21:42:09.707990+0000","last_active":"2024-09-10T21:42:11.975323+0000","last_peered":"2024-09-10T21:42:11.975323+0000","last_clean":"2024-09-10T21:42:11.975323+0000","last_became_active":"2024-09-10T21:42:09.707404+0000","last_became_peered":"2024-09-10T21:42:09.707404+0000","last_unstale":"2024-09-10T21:42:11.975323+0000","last_undegraded":"2024-09-10T21:42:11.975323+0000","last_fullsized":"2024-09-10T21:42:11.975323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:40.556148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.684709+0000","last_change":"2024-09-10T21:42:08.697982+0000","last_active":"2024-09-10T21:42:11.684709+0000","last_peered":"2024-09-10T21:42:11.684709+0000","last_clean":"2024-09-10T21:42:11.684709+0000","last_became_active":"2024-09-10T21:42:08.696642+0000","last_became_peered":"2024-09-10T21:42:08.696642+0000","last_unstale":"2024-09-10T21:42:11.684709+0000","last_undegraded":"2024-09-10T21:42:11.684709+0000","last_fullsized":"2024-09-10T21:42:11.684709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:50.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.680064+0000","last_change":"2024-09-10T21:42:08.704347+0000","last_active":"2024-09-10T21:42:09.680064+0000","last_peered":"2024-09-10T21:42:09.680064+0000","last_clean":"2024-09-10T21:42:09.680064+0000","last_became_active":"2024-09-10T21:42:08.701532+0000","last_became_peered":"2024-09-10T21:42:08.701532+0000","last_unstale":"2024-09-10T21:42:09.680064+0000","last_undegraded":"2024-09-10T21:42:09.680064+0000","last_fullsized":"2024-09-10T21:42:09.680064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:05.016544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.703760+0000","last_change":"2024-09-10T21:42:09.698844+0000","last_active":"2024-09-10T21:42:09.703760+0000","last_peered":"2024-09-10T21:42:09.703760+0000","last_clean":"2024-09-10T21:42:09.703760+0000","last_became_active":"2024-09-10T21:42:09.698554+0000","last_became_peered":"2024-09-10T21:42:09.698554+0000","last_unstale":"2024-09-10T21:42:09.703760+0000","last_undegraded":"2024-09-10T21:42:09.703760+0000","last_fullsized":"2024-09-10T21:42:09.703760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:46.579215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701042+0000","last_change":"2024-09-10T21:42:08.685541+0000","last_active":"2024-09-10T21:42:10.701042+0000","last_peered":"2024-09-10T21:42:10.701042+0000","last_clean":"2024-09-10T21:42:10.701042+0000","last_became_active":"2024-09-10T21:42:08.685254+0000","last_became_peered":"2024-09-10T21:42:08.685254+0000","last_unstale":"2024-09-10T21:42:10.701042+0000","last_undegraded":"2024-09-10T21:42:10.701042+0000","last_fullsized":"2024-09-10T21:42:10.701042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:56.090844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695329+0000","last_change":"2024-09-10T21:42:09.695387+0000","last_active":"2024-09-10T21:42:10.695329+0000","last_peered":"2024-09-10T21:42:10.695329+0000","last_clean":"2024-09-10T21:42:10.695329+0000","last_became_active":"2024-09-10T21:42:09.695065+0000","last_became_peered":"2024-09-10T21:42:09.695065+0000","last_unstale":"2024-09-10T21:42:10.695329+0000","last_undegraded":"2024-09-10T21:42:10.695329+0000","last_fullsized":"2024-09-10T21:42:10.695329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:38.013273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685092+0000","last_change":"2024-09-10T21:42:08.692049+0000","last_active":"2024-09-10T21:42:11.685092+0000","last_peered":"2024-09-10T21:42:11.685092+0000","last_clean":"2024-09-10T21:42:11.685092+0000","last_became_active":"2024-09-10T21:42:08.691746+0000","last_became_peered":"2024-09-10T21:42:08.691746+0000","last_unstale":"2024-09-10T21:42:11.685092+0000","last_undegraded":"2024-09-10T21:42:11.685092+0000","last_fullsized":"2024-09-10T21:42:11.685092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:46.162803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702258+0000","last_change":"2024-09-10T21:42:09.703241+0000","last_active":"2024-09-10T21:42:10.702258+0000","last_peered":"2024-09-10T21:42:10.702258+0000","last_clean":"2024-09-10T21:42:10.702258+0000","last_became_active":"2024-09-10T21:42:09.702971+0000","last_became_peered":"2024-09-10T21:42:09.702971+0000","last_unstale":"2024-09-10T21:42:10.702258+0000","last_undegraded":"2024-09-10T21:42:10.702258+0000","last_fullsized":"2024-09-10T21:42:10.702258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:42.927407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702191+0000","last_change":"2024-09-10T21:42:08.698423+0000","last_active":"2024-09-10T21:42:10.702191+0000","last_peered":"2024-09-10T21:42:10.702191+0000","last_clean":"2024-09-10T21:42:10.702191+0000","last_became_active":"2024-09-10T21:42:08.696721+0000","last_became_peered":"2024-09-10T21:42:08.696721+0000","last_unstale":"2024-09-10T21:42:10.702191+0000","last_undegraded":"2024-09-10T21:42:10.702191+0000","last_fullsized":"2024-09-10T21:42:10.702191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:50.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975587+0000","last_change":"2024-09-10T21:42:09.713357+0000","last_active":"2024-09-10T21:42:11.975587+0000","last_peered":"2024-09-10T21:42:11.975587+0000","last_clean":"2024-09-10T21:42:11.975587+0000","last_became_active":"2024-09-10T21:42:09.713078+0000","last_became_peered":"2024-09-10T21:42:09.713078+0000","last_unstale":"2024-09-10T21:42:11.975587+0000","last_undegraded":"2024-09-10T21:42:11.975587+0000","last_fullsized":"2024-09-10T21:42:11.975587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:52.381281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695294+0000","last_change":"2024-09-10T21:42:08.696594+0000","last_active":"2024-09-10T21:42:10.695294+0000","last_peered":"2024-09-10T21:42:10.695294+0000","last_clean":"2024-09-10T21:42:10.695294+0000","last_became_active":"2024-09-10T21:42:08.696243+0000","last_became_peered":"2024-09-10T21:42:08.696243+0000","last_unstale":"2024-09-10T21:42:10.695294+0000","last_undegraded":"2024-09-10T21:42:10.695294+0000","last_fullsized":"2024-09-10T21:42:10.695294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:46.162803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701779+0000","last_change":"2024-09-10T21:42:09.704846+0000","last_active":"2024-09-10T21:42:10.701779+0000","last_peered":"2024-09-10T21:42:10.701779+0000","last_clean":"2024-09-10T21:42:10.701779+0000","last_became_active":"2024-09-10T21:42:09.704582+0000","last_became_peered":"2024-09-10T21:42:09.704582+0000","last_unstale":"2024-09-10T21:42:10.701779+0000","last_undegraded":"2024-09-10T21:42:10.701779+0000","last_fullsized":"2024-09-10T21:42:10.701779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:34.026318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685136+0000","last_change":"2024-09-10T21:42:08.694242+0000","last_active":"2024-09-10T21:42:11.685136+0000","last_peered":"2024-09-10T21:42:11.685136+0000","last_clean":"2024-09-10T21:42:11.685136+0000","last_became_active":"2024-09-10T21:42:08.693785+0000","last_became_peered":"2024-09-10T21:42:08.693785+0000","last_unstale":"2024-09-10T21:42:11.685136+0000","last_undegraded":"2024-09-10T21:42:11.685136+0000","last_fullsized":"2024-09-10T21:42:11.685136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:56.090844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781904+0000","last_change":"2024-09-10T21:42:09.709096+0000","last_active":"2024-09-10T21:42:11.781904+0000","last_peered":"2024-09-10T21:42:11.781904+0000","last_clean":"2024-09-10T21:42:11.781904+0000","last_became_active":"2024-09-10T21:42:09.708202+0000","last_became_peered":"2024-09-10T21:42:09.708202+0000","last_unstale":"2024-09-10T21:42:11.781904+0000","last_undegraded":"2024-09-10T21:42:11.781904+0000","last_fullsized":"2024-09-10T21:42:11.781904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:12:39.888041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975170+0000","last_change":"2024-09-10T21:42:08.694284+0000","last_active":"2024-09-10T21:42:11.975170+0000","last_peered":"2024-09-10T21:42:11.975170+0000","last_clean":"2024-09-10T21:42:11.975170+0000","last_became_active":"2024-09-10T21:42:08.693910+0000","last_became_peered":"2024-09-10T21:42:08.693910+0000","last_unstale":"2024-09-10T21:42:11.975170+0000","last_undegraded":"2024-09-10T21:42:11.975170+0000","last_fullsized":"2024-09-10T21:42:11.975170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:40.174118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975003+0000","last_change":"2024-09-10T21:42:09.712940+0000","last_active":"2024-09-10T21:42:11.975003+0000","last_peered":"2024-09-10T21:42:11.975003+0000","last_clean":"2024-09-10T21:42:11.975003+0000","last_became_active":"2024-09-10T21:42:09.709015+0000","last_became_peered":"2024-09-10T21:42:09.709015+0000","last_unstale":"2024-09-10T21:42:11.975003+0000","last_undegraded":"2024-09-10T21:42:11.975003+0000","last_fullsized":"2024-09-10T21:42:11.975003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:41.900249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701745+0000","last_change":"2024-09-10T21:42:08.689371+0000","last_active":"2024-09-10T21:42:10.701745+0000","last_peered":"2024-09-10T21:42:10.701745+0000","last_clean":"2024-09-10T21:42:10.701745+0000","last_became_active":"2024-09-10T21:42:08.688880+0000","last_became_peered":"2024-09-10T21:42:08.688880+0000","last_unstale":"2024-09-10T21:42:10.701745+0000","last_undegraded":"2024-09-10T21:42:10.701745+0000","last_fullsized":"2024-09-10T21:42:10.701745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:46.162803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781881+0000","last_change":"2024-09-10T21:42:09.713450+0000","last_active":"2024-09-10T21:42:11.781881+0000","last_peered":"2024-09-10T21:42:11.781881+0000","last_clean":"2024-09-10T21:42:11.781881+0000","last_became_active":"2024-09-10T21:42:09.713199+0000","last_became_peered":"2024-09-10T21:42:09.713199+0000","last_unstale":"2024-09-10T21:42:11.781881+0000","last_undegraded":"2024-09-10T21:42:11.781881+0000","last_fullsized":"2024-09-10T21:42:11.781881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:23.530344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.684649+0000","last_change":"2024-09-10T21:42:08.703573+0000","last_active":"2024-09-10T21:42:11.684649+0000","last_peered":"2024-09-10T21:42:11.684649+0000","last_clean":"2024-09-10T21:42:11.684649+0000","last_became_active":"2024-09-10T21:42:08.701580+0000","last_became_peered":"2024-09-10T21:42:08.701580+0000","last_unstale":"2024-09-10T21:42:11.684649+0000","last_undegraded":"2024-09-10T21:42:11.684649+0000","last_fullsized":"2024-09-10T21:42:11.684649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:16.471687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.684924+0000","last_change":"2024-09-10T21:42:09.700853+0000","last_active":"2024-09-10T21:42:11.684924+0000","last_peered":"2024-09-10T21:42:11.684924+0000","last_clean":"2024-09-10T21:42:11.684924+0000","last_became_active":"2024-09-10T21:42:09.700370+0000","last_became_peered":"2024-09-10T21:42:09.700370+0000","last_unstale":"2024-09-10T21:42:11.684924+0000","last_undegraded":"2024-09-10T21:42:11.684924+0000","last_fullsized":"2024-09-10T21:42:11.684924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:13.700823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428370+0000","last_change":"2024-09-10T21:42:08.697117+0000","last_active":"2024-09-10T21:42:11.428370+0000","last_peered":"2024-09-10T21:42:11.428370+0000","last_clean":"2024-09-10T21:42:11.428370+0000","last_became_active":"2024-09-10T21:42:08.695800+0000","last_became_peered":"2024-09-10T21:42:08.695800+0000","last_unstale":"2024-09-10T21:42:11.428370+0000","last_undegraded":"2024-09-10T21:42:11.428370+0000","last_fullsized":"2024-09-10T21:42:11.428370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:36.934204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781790+0000","last_change":"2024-09-10T21:42:09.707807+0000","last_active":"2024-09-10T21:42:11.781790+0000","last_peered":"2024-09-10T21:42:11.781790+0000","last_clean":"2024-09-10T21:42:11.781790+0000","last_became_active":"2024-09-10T21:42:09.707283+0000","last_became_peered":"2024-09-10T21:42:09.707283+0000","last_unstale":"2024-09-10T21:42:11.781790+0000","last_undegraded":"2024-09-10T21:42:11.781790+0000","last_fullsized":"2024-09-10T21:42:11.781790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:50.040141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701923+0000","last_change":"2024-09-10T21:42:08.693177+0000","last_active":"2024-09-10T21:42:10.701923+0000","last_peered":"2024-09-10T21:42:10.701923+0000","last_clean":"2024-09-10T21:42:10.701923+0000","last_became_active":"2024-09-10T21:42:08.692871+0000","last_became_peered":"2024-09-10T21:42:08.692871+0000","last_unstale":"2024-09-10T21:42:10.701923+0000","last_undegraded":"2024-09-10T21:42:10.701923+0000","last_fullsized":"2024-09-10T21:42:10.701923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:06.841814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781985+0000","last_change":"2024-09-10T21:42:09.710163+0000","last_active":"2024-09-10T21:42:11.781985+0000","last_peered":"2024-09-10T21:42:11.781985+0000","last_clean":"2024-09-10T21:42:11.781985+0000","last_became_active":"2024-09-10T21:42:09.709887+0000","last_became_peered":"2024-09-10T21:42:09.709887+0000","last_unstale":"2024-09-10T21:42:11.781985+0000","last_undegraded":"2024-09-10T21:42:11.781985+0000","last_fullsized":"2024-09-10T21:42:11.781985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:05:52.771916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428604+0000","last_change":"2024-09-10T21:42:08.686678+0000","last_active":"2024-09-10T21:42:11.428604+0000","last_peered":"2024-09-10T21:42:11.428604+0000","last_clean":"2024-09-10T21:42:11.428604+0000","last_became_active":"2024-09-10T21:42:08.686395+0000","last_became_peered":"2024-09-10T21:42:08.686395+0000","last_unstale":"2024-09-10T21:42:11.428604+0000","last_undegraded":"2024-09-10T21:42:11.428604+0000","last_fullsized":"2024-09-10T21:42:11.428604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:50.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782250+0000","last_change":"2024-09-10T21:42:09.714231+0000","last_active":"2024-09-10T21:42:11.782250+0000","last_peered":"2024-09-10T21:42:11.782250+0000","last_clean":"2024-09-10T21:42:11.782250+0000","last_became_active":"2024-09-10T21:42:09.713873+0000","last_became_peered":"2024-09-10T21:42:09.713873+0000","last_unstale":"2024-09-10T21:42:11.782250+0000","last_undegraded":"2024-09-10T21:42:11.782250+0000","last_fullsized":"2024-09-10T21:42:11.782250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:50.638349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695399+0000","last_change":"2024-09-10T21:42:08.694275+0000","last_active":"2024-09-10T21:42:10.695399+0000","last_peered":"2024-09-10T21:42:10.695399+0000","last_clean":"2024-09-10T21:42:10.695399+0000","last_became_active":"2024-09-10T21:42:08.694014+0000","last_became_peered":"2024-09-10T21:42:08.694014+0000","last_unstale":"2024-09-10T21:42:10.695399+0000","last_undegraded":"2024-09-10T21:42:10.695399+0000","last_fullsized":"2024-09-10T21:42:10.695399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:52.021550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702618+0000","last_change":"2024-09-10T21:42:09.698139+0000","last_active":"2024-09-10T21:42:10.702618+0000","last_peered":"2024-09-10T21:42:10.702618+0000","last_clean":"2024-09-10T21:42:10.702618+0000","last_became_active":"2024-09-10T21:42:09.697391+0000","last_became_peered":"2024-09-10T21:42:09.697391+0000","last_unstale":"2024-09-10T21:42:10.702618+0000","last_undegraded":"2024-09-10T21:42:10.702618+0000","last_fullsized":"2024-09-10T21:42:10.702618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:47.497053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428569+0000","last_change":"2024-09-10T21:42:08.691652+0000","last_active":"2024-09-10T21:42:11.428569+0000","last_peered":"2024-09-10T21:42:11.428569+0000","last_clean":"2024-09-10T21:42:11.428569+0000","last_became_active":"2024-09-10T21:42:08.691317+0000","last_became_peered":"2024-09-10T21:42:08.691317+0000","last_unstale":"2024-09-10T21:42:11.428569+0000","last_undegraded":"2024-09-10T21:42:11.428569+0000","last_fullsized":"2024-09-10T21:42:11.428569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:10.198175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781834+0000","last_change":"2024-09-10T21:42:09.720201+0000","last_active":"2024-09-10T21:42:11.781834+0000","last_peered":"2024-09-10T21:42:11.781834+0000","last_clean":"2024-09-10T21:42:11.781834+0000","last_became_active":"2024-09-10T21:42:09.719872+0000","last_became_peered":"2024-09-10T21:42:09.719872+0000","last_unstale":"2024-09-10T21:42:11.781834+0000","last_undegraded":"2024-09-10T21:42:11.781834+0000","last_fullsized":"2024-09-10T21:42:11.781834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:31.222577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781370+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:42:11.781370+0000","last_peered":"2024-09-10T21:42:11.781370+0000","last_clean":"2024-09-10T21:42:11.781370+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:42:11.781370+0000","last_undegraded":"2024-09-10T21:42:11.781370+0000","last_fullsized":"2024-09-10T21:42:11.781370+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701380+0000","last_change":"2024-09-10T21:42:08.698035+0000","last_active":"2024-09-10T21:42:10.701380+0000","last_peered":"2024-09-10T21:42:10.701380+0000","last_clean":"2024-09-10T21:42:10.701380+0000","last_became_active":"2024-09-10T21:42:08.695961+0000","last_became_peered":"2024-09-10T21:42:08.695961+0000","last_unstale":"2024-09-10T21:42:10.701380+0000","last_undegraded":"2024-09-10T21:42:10.701380+0000","last_fullsized":"2024-09-10T21:42:10.701380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:16.471687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.698912+0000","last_change":"2024-09-10T21:42:09.692488+0000","last_active":"2024-09-10T21:42:09.698912+0000","last_peered":"2024-09-10T21:42:09.698912+0000","last_clean":"2024-09-10T21:42:09.698912+0000","last_became_active":"2024-09-10T21:42:09.691349+0000","last_became_peered":"2024-09-10T21:42:09.691349+0000","last_unstale":"2024-09-10T21:42:09.698912+0000","last_undegraded":"2024-09-10T21:42:09.698912+0000","last_fullsized":"2024-09-10T21:42:09.698912+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:41.624422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781332+0000","last_change":"2024-09-10T21:42:08.706510+0000","last_active":"2024-09-10T21:42:11.781332+0000","last_peered":"2024-09-10T21:42:11.781332+0000","last_clean":"2024-09-10T21:42:11.781332+0000","last_became_active":"2024-09-10T21:42:08.706080+0000","last_became_peered":"2024-09-10T21:42:08.706080+0000","last_unstale":"2024-09-10T21:42:11.781332+0000","last_undegraded":"2024-09-10T21:42:11.781332+0000","last_fullsized":"2024-09-10T21:42:11.781332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:25.172297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975060+0000","last_change":"2024-09-10T21:42:09.698977+0000","last_active":"2024-09-10T21:42:11.975060+0000","last_peered":"2024-09-10T21:42:11.975060+0000","last_clean":"2024-09-10T21:42:11.975060+0000","last_became_active":"2024-09-10T21:42:09.698557+0000","last_became_peered":"2024-09-10T21:42:09.698557+0000","last_unstale":"2024-09-10T21:42:11.975060+0000","last_undegraded":"2024-09-10T21:42:11.975060+0000","last_fullsized":"2024-09-10T21:42:11.975060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:40.541602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428457+0000","last_change":"2024-09-10T21:42:08.689207+0000","last_active":"2024-09-10T21:42:11.428457+0000","last_peered":"2024-09-10T21:42:11.428457+0000","last_clean":"2024-09-10T21:42:11.428457+0000","last_became_active":"2024-09-10T21:42:08.688941+0000","last_became_peered":"2024-09-10T21:42:08.688941+0000","last_unstale":"2024-09-10T21:42:11.428457+0000","last_undegraded":"2024-09-10T21:42:11.428457+0000","last_fullsized":"2024-09-10T21:42:11.428457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:56.090844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695552+0000","last_change":"2024-09-10T21:42:09.688301+0000","last_active":"2024-09-10T21:42:10.695552+0000","last_peered":"2024-09-10T21:42:10.695552+0000","last_clean":"2024-09-10T21:42:10.695552+0000","last_became_active":"2024-09-10T21:42:09.687944+0000","last_became_peered":"2024-09-10T21:42:09.687944+0000","last_unstale":"2024-09-10T21:42:10.695552+0000","last_undegraded":"2024-09-10T21:42:10.695552+0000","last_fullsized":"2024-09-10T21:42:10.695552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:37.025454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701842+0000","last_change":"2024-09-10T21:42:08.698803+0000","last_active":"2024-09-10T21:42:10.701842+0000","last_peered":"2024-09-10T21:42:10.701842+0000","last_clean":"2024-09-10T21:42:10.701842+0000","last_became_active":"2024-09-10T21:42:08.697174+0000","last_became_peered":"2024-09-10T21:42:08.697174+0000","last_unstale":"2024-09-10T21:42:10.701842+0000","last_undegraded":"2024-09-10T21:42:10.701842+0000","last_fullsized":"2024-09-10T21:42:10.701842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:47.619335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702033+0000","last_change":"2024-09-10T21:42:09.688575+0000","last_active":"2024-09-10T21:42:10.702033+0000","last_peered":"2024-09-10T21:42:10.702033+0000","last_clean":"2024-09-10T21:42:10.702033+0000","last_became_active":"2024-09-10T21:42:09.688284+0000","last_became_peered":"2024-09-10T21:42:09.688284+0000","last_unstale":"2024-09-10T21:42:10.702033+0000","last_undegraded":"2024-09-10T21:42:10.702033+0000","last_fullsized":"2024-09-10T21:42:10.702033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:53.029762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695680+0000","last_change":"2024-09-10T21:42:08.698800+0000","last_active":"2024-09-10T21:42:10.695680+0000","last_peered":"2024-09-10T21:42:10.695680+0000","last_clean":"2024-09-10T21:42:10.695680+0000","last_became_active":"2024-09-10T21:42:08.696627+0000","last_became_peered":"2024-09-10T21:42:08.696627+0000","last_unstale":"2024-09-10T21:42:10.695680+0000","last_undegraded":"2024-09-10T21:42:10.695680+0000","last_fullsized":"2024-09-10T21:42:10.695680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:56.090844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701667+0000","last_change":"2024-09-10T21:42:09.700048+0000","last_active":"2024-09-10T21:42:10.701667+0000","last_peered":"2024-09-10T21:42:10.701667+0000","last_clean":"2024-09-10T21:42:10.701667+0000","last_became_active":"2024-09-10T21:42:09.699085+0000","last_became_peered":"2024-09-10T21:42:09.699085+0000","last_unstale":"2024-09-10T21:42:10.701667+0000","last_undegraded":"2024-09-10T21:42:10.701667+0000","last_fullsized":"2024-09-10T21:42:10.701667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:22.835437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782141+0000","last_change":"2024-09-10T21:42:08.704257+0000","last_active":"2024-09-10T21:42:11.782141+0000","last_peered":"2024-09-10T21:42:11.782141+0000","last_clean":"2024-09-10T21:42:11.782141+0000","last_became_active":"2024-09-10T21:42:08.703837+0000","last_became_peered":"2024-09-10T21:42:08.703837+0000","last_unstale":"2024-09-10T21:42:11.782141+0000","last_undegraded":"2024-09-10T21:42:11.782141+0000","last_fullsized":"2024-09-10T21:42:11.782141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:44.076555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695679+0000","last_change":"2024-09-10T21:42:09.695578+0000","last_active":"2024-09-10T21:42:10.695679+0000","last_peered":"2024-09-10T21:42:10.695679+0000","last_clean":"2024-09-10T21:42:10.695679+0000","last_became_active":"2024-09-10T21:42:09.693969+0000","last_became_peered":"2024-09-10T21:42:09.693969+0000","last_unstale":"2024-09-10T21:42:10.695679+0000","last_undegraded":"2024-09-10T21:42:10.695679+0000","last_fullsized":"2024-09-10T21:42:10.695679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:48.304330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695614+0000","last_change":"2024-09-10T21:42:08.699017+0000","last_active":"2024-09-10T21:42:10.695614+0000","last_peered":"2024-09-10T21:42:10.695614+0000","last_clean":"2024-09-10T21:42:10.695614+0000","last_became_active":"2024-09-10T21:42:08.698540+0000","last_became_peered":"2024-09-10T21:42:08.698540+0000","last_unstale":"2024-09-10T21:42:10.695614+0000","last_undegraded":"2024-09-10T21:42:10.695614+0000","last_fullsized":"2024-09-10T21:42:10.695614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:33.018106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.714319+0000","last_change":"2024-09-10T21:42:09.709063+0000","last_active":"2024-09-10T21:42:09.714319+0000","last_peered":"2024-09-10T21:42:09.714319+0000","last_clean":"2024-09-10T21:42:09.714319+0000","last_became_active":"2024-09-10T21:42:09.708726+0000","last_became_peered":"2024-09-10T21:42:09.708726+0000","last_unstale":"2024-09-10T21:42:09.714319+0000","last_undegraded":"2024-09-10T21:42:09.714319+0000","last_fullsized":"2024-09-10T21:42:09.714319+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:35.888935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685182+0000","last_change":"2024-09-10T21:42:08.697045+0000","last_active":"2024-09-10T21:42:11.685182+0000","last_peered":"2024-09-10T21:42:11.685182+0000","last_clean":"2024-09-10T21:42:11.685182+0000","last_became_active":"2024-09-10T21:42:08.696483+0000","last_became_peered":"2024-09-10T21:42:08.696483+0000","last_unstale":"2024-09-10T21:42:11.685182+0000","last_undegraded":"2024-09-10T21:42:11.685182+0000","last_fullsized":"2024-09-10T21:42:11.685182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:34.907546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975478+0000","last_change":"2024-09-10T21:42:09.694168+0000","last_active":"2024-09-10T21:42:11.975478+0000","last_peered":"2024-09-10T21:42:11.975478+0000","last_clean":"2024-09-10T21:42:11.975478+0000","last_became_active":"2024-09-10T21:42:09.693779+0000","last_became_peered":"2024-09-10T21:42:09.693779+0000","last_unstale":"2024-09-10T21:42:11.975478+0000","last_undegraded":"2024-09-10T21:42:11.975478+0000","last_fullsized":"2024-09-10T21:42:11.975478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:24:55.155776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781313+0000","last_change":"2024-09-10T21:42:08.697319+0000","last_active":"2024-09-10T21:42:11.781313+0000","last_peered":"2024-09-10T21:42:11.781313+0000","last_clean":"2024-09-10T21:42:11.781313+0000","last_became_active":"2024-09-10T21:42:08.697039+0000","last_became_peered":"2024-09-10T21:42:08.697039+0000","last_unstale":"2024-09-10T21:42:11.781313+0000","last_undegraded":"2024-09-10T21:42:11.781313+0000","last_fullsized":"2024-09-10T21:42:11.781313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:48.264843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701623+0000","last_change":"2024-09-10T21:42:09.689554+0000","last_active":"2024-09-10T21:42:10.701623+0000","last_peered":"2024-09-10T21:42:10.701623+0000","last_clean":"2024-09-10T21:42:10.701623+0000","last_became_active":"2024-09-10T21:42:09.689041+0000","last_became_peered":"2024-09-10T21:42:09.689041+0000","last_unstale":"2024-09-10T21:42:10.701623+0000","last_undegraded":"2024-09-10T21:42:10.701623+0000","last_fullsized":"2024-09-10T21:42:10.701623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:48.627547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695404+0000","last_change":"2024-09-10T21:42:08.700219+0000","last_active":"2024-09-10T21:42:10.695404+0000","last_peered":"2024-09-10T21:42:10.695404+0000","last_clean":"2024-09-10T21:42:10.695404+0000","last_became_active":"2024-09-10T21:42:08.696072+0000","last_became_peered":"2024-09-10T21:42:08.696072+0000","last_unstale":"2024-09-10T21:42:10.695404+0000","last_undegraded":"2024-09-10T21:42:10.695404+0000","last_fullsized":"2024-09-10T21:42:10.695404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:50.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.699090+0000","last_change":"2024-09-10T21:42:09.695584+0000","last_active":"2024-09-10T21:42:09.699090+0000","last_peered":"2024-09-10T21:42:09.699090+0000","last_clean":"2024-09-10T21:42:09.699090+0000","last_became_active":"2024-09-10T21:42:09.695169+0000","last_became_peered":"2024-09-10T21:42:09.695169+0000","last_unstale":"2024-09-10T21:42:09.699090+0000","last_undegraded":"2024-09-10T21:42:09.699090+0000","last_fullsized":"2024-09-10T21:42:09.699090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:26.180509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695459+0000","last_change":"2024-09-10T21:42:08.702389+0000","last_active":"2024-09-10T21:42:10.695459+0000","last_peered":"2024-09-10T21:42:10.695459+0000","last_clean":"2024-09-10T21:42:10.695459+0000","last_became_active":"2024-09-10T21:42:08.701730+0000","last_became_peered":"2024-09-10T21:42:08.701730+0000","last_unstale":"2024-09-10T21:42:10.695459+0000","last_undegraded":"2024-09-10T21:42:10.695459+0000","last_fullsized":"2024-09-10T21:42:10.695459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:47.619335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.698786+0000","last_change":"2024-09-10T21:42:09.691569+0000","last_active":"2024-09-10T21:42:09.698786+0000","last_peered":"2024-09-10T21:42:09.698786+0000","last_clean":"2024-09-10T21:42:09.698786+0000","last_became_active":"2024-09-10T21:42:09.691270+0000","last_became_peered":"2024-09-10T21:42:09.691270+0000","last_unstale":"2024-09-10T21:42:09.698786+0000","last_undegraded":"2024-09-10T21:42:09.698786+0000","last_fullsized":"2024-09-10T21:42:09.698786+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:53.886934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782013+0000","last_change":"2024-09-10T21:42:08.702598+0000","last_active":"2024-09-10T21:42:11.782013+0000","last_peered":"2024-09-10T21:42:11.782013+0000","last_clean":"2024-09-10T21:42:11.782013+0000","last_became_active":"2024-09-10T21:42:08.702352+0000","last_became_peered":"2024-09-10T21:42:08.702352+0000","last_unstale":"2024-09-10T21:42:11.782013+0000","last_undegraded":"2024-09-10T21:42:11.782013+0000","last_fullsized":"2024-09-10T21:42:11.782013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:45.571003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685024+0000","last_change":"2024-09-10T21:42:09.708756+0000","last_active":"2024-09-10T21:42:11.685024+0000","last_peered":"2024-09-10T21:42:11.685024+0000","last_clean":"2024-09-10T21:42:11.685024+0000","last_became_active":"2024-09-10T21:42:09.708458+0000","last_became_peered":"2024-09-10T21:42:09.708458+0000","last_unstale":"2024-09-10T21:42:11.685024+0000","last_undegraded":"2024-09-10T21:42:11.685024+0000","last_fullsized":"2024-09-10T21:42:11.685024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:37.025454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701552+0000","last_change":"2024-09-10T21:42:09.692810+0000","last_active":"2024-09-10T21:42:10.701552+0000","last_peered":"2024-09-10T21:42:10.701552+0000","last_clean":"2024-09-10T21:42:10.701552+0000","last_became_active":"2024-09-10T21:42:09.692537+0000","last_became_peered":"2024-09-10T21:42:09.692537+0000","last_unstale":"2024-09-10T21:42:10.701552+0000","last_undegraded":"2024-09-10T21:42:10.701552+0000","last_fullsized":"2024-09-10T21:42:10.701552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:55.273108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781669+0000","last_change":"2024-09-10T21:42:08.705305+0000","last_active":"2024-09-10T21:42:11.781669+0000","last_peered":"2024-09-10T21:42:11.781669+0000","last_clean":"2024-09-10T21:42:11.781669+0000","last_became_active":"2024-09-10T21:42:08.704991+0000","last_became_peered":"2024-09-10T21:42:08.704991+0000","last_unstale":"2024-09-10T21:42:11.781669+0000","last_undegraded":"2024-09-10T21:42:11.781669+0000","last_fullsized":"2024-09-10T21:42:11.781669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:59.418988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702644+0000","last_change":"2024-09-10T21:42:09.704028+0000","last_active":"2024-09-10T21:42:10.702644+0000","last_peered":"2024-09-10T21:42:10.702644+0000","last_clean":"2024-09-10T21:42:10.702644+0000","last_became_active":"2024-09-10T21:42:09.703608+0000","last_became_peered":"2024-09-10T21:42:09.703608+0000","last_unstale":"2024-09-10T21:42:10.702644+0000","last_undegraded":"2024-09-10T21:42:10.702644+0000","last_fullsized":"2024-09-10T21:42:10.702644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:11.626660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782020+0000","last_change":"2024-09-10T21:42:08.704448+0000","last_active":"2024-09-10T21:42:11.782020+0000","last_peered":"2024-09-10T21:42:11.782020+0000","last_clean":"2024-09-10T21:42:11.782020+0000","last_became_active":"2024-09-10T21:42:08.704156+0000","last_became_peered":"2024-09-10T21:42:08.704156+0000","last_unstale":"2024-09-10T21:42:11.782020+0000","last_undegraded":"2024-09-10T21:42:11.782020+0000","last_fullsized":"2024-09-10T21:42:11.782020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:07.043672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975512+0000","last_change":"2024-09-10T21:42:09.693556+0000","last_active":"2024-09-10T21:42:11.975512+0000","last_peered":"2024-09-10T21:42:11.975512+0000","last_clean":"2024-09-10T21:42:11.975512+0000","last_became_active":"2024-09-10T21:42:09.693330+0000","last_became_peered":"2024-09-10T21:42:09.693330+0000","last_unstale":"2024-09-10T21:42:11.975512+0000","last_undegraded":"2024-09-10T21:42:11.975512+0000","last_fullsized":"2024-09-10T21:42:11.975512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:26:39.608345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.717090+0000","last_change":"2024-09-10T21:42:08.699157+0000","last_active":"2024-09-10T21:42:09.717090+0000","last_peered":"2024-09-10T21:42:09.717090+0000","last_clean":"2024-09-10T21:42:09.717090+0000","last_became_active":"2024-09-10T21:42:08.691408+0000","last_became_peered":"2024-09-10T21:42:08.691408+0000","last_unstale":"2024-09-10T21:42:09.717090+0000","last_undegraded":"2024-09-10T21:42:09.717090+0000","last_fullsized":"2024-09-10T21:42:09.717090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:52.464492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-10T21:42:09.708009+0000","last_change":"2024-09-10T21:42:09.704294+0000","last_active":"2024-09-10T21:42:09.708009+0000","last_peered":"2024-09-10T21:42:09.708009+0000","last_clean":"2024-09-10T21:42:09.708009+0000","last_became_active":"2024-09-10T21:42:09.703607+0000","last_became_peered":"2024-09-10T21:42:09.703607+0000","last_unstale":"2024-09-10T21:42:09.708009+0000","last_undegraded":"2024-09-10T21:42:09.708009+0000","last_fullsized":"2024-09-10T21:42:09.708009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:08.051884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975594+0000","last_change":"2024-09-10T21:42:08.702462+0000","last_active":"2024-09-10T21:42:11.975594+0000","last_peered":"2024-09-10T21:42:11.975594+0000","last_clean":"2024-09-10T21:42:11.975594+0000","last_became_active":"2024-09-10T21:42:08.702103+0000","last_became_peered":"2024-09-10T21:42:08.702103+0000","last_unstale":"2024-09-10T21:42:11.975594+0000","last_undegraded":"2024-09-10T21:42:11.975594+0000","last_fullsized":"2024-09-10T21:42:11.975594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:30.979951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509446,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":72277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":72313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":72187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509447,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":815104,"data_stored":663193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542148,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":794624,"data_stored":656837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":208896,"data_stored":67499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:42:14.371 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:42:14.372 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-10T21:42:14.372 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-10T21:42:14.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.569+0000 7f2114400640 1 -- 172.21.15.7:0/4209890057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c06e510 msgr2=0x7f210c073cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:14.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.569+0000 7f2114400640 1 --2- 172.21.15.7:0/4209890057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c06e510 0x7f210c073cd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f210c0b2620 tx=0x7f20f802a4d0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.569+0000 7f2114400640 1 -- 172.21.15.7:0/4209890057 shutdown_connections 2024-09-10T21:42:14.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.569+0000 7f2114400640 1 --2- 172.21.15.7:0/4209890057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c06e510 0x7f210c073cd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.569+0000 7f2114400640 1 --2- 172.21.15.7:0/4209890057 >> v2:172.21.15.133:3300/0 conn(0x7f210c06ff50 0x7f210c15cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.569+0000 7f2114400640 1 -- 172.21.15.7:0/4209890057 >> 172.21.15.7:0/4209890057 conn(0x7f210c0b1660 msgr2=0x7f210c0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:14.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.569+0000 7f2114400640 1 -- 172.21.15.7:0/4209890057 shutdown_connections 2024-09-10T21:42:14.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.569+0000 7f2114400640 1 -- 172.21.15.7:0/4209890057 wait complete. 2024-09-10T21:42:14.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.571+0000 7f2114400640 1 Processor -- start 2024-09-10T21:42:14.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.571+0000 7f2114400640 1 -- start start 2024-09-10T21:42:14.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.572+0000 7f2114400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c06e510 0x7f210c0adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.572+0000 7f2114400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f210c06e960 0x7f210c1f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.572+0000 7f2114400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f210c06ff50 0x7f210c0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.572+0000 7f2114400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f210c0745a0 con 0x7f210c06e510 2024-09-10T21:42:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.572+0000 7f2114400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f210c074420 con 0x7f210c06e960 2024-09-10T21:42:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.572+0000 7f2114400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f210c074720 con 0x7f210c06ff50 2024-09-10T21:42:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.572+0000 7f2111a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c06e510 0x7f210c0adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.572+0000 7f2111a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c06e510 0x7f210c0adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54046/0 (socket says 172.21.15.7:54046) 2024-09-10T21:42:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.572+0000 7f2111a00640 1 -- 172.21.15.7:0/1938599020 learned_addr learned my addr 172.21.15.7:0/1938599020 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.572+0000 7f2111000640 1 --2- 172.21.15.7:0/1938599020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f210c06e960 0x7f210c1f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.572+0000 7f2112400640 1 --2- 172.21.15.7:0/1938599020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f210c06ff50 0x7f210c0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.573+0000 7f2111a00640 1 -- 172.21.15.7:0/1938599020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f210c06ff50 msgr2=0x7f210c0af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.573+0000 7f2111a00640 1 --2- 172.21.15.7:0/1938599020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f210c06ff50 0x7f210c0af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.573+0000 7f2111a00640 1 -- 172.21.15.7:0/1938599020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f210c06e960 msgr2=0x7f210c1f24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.573+0000 7f2111a00640 1 --2- 172.21.15.7:0/1938599020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f210c06e960 0x7f210c1f24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.573+0000 7f2111a00640 1 -- 172.21.15.7:0/1938599020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f210c1f2f40 con 0x7f210c06e510 2024-09-10T21:42:14.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.573+0000 7f2111000640 1 --2- 172.21.15.7:0/1938599020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f210c06e960 0x7f210c1f24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:14.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.573+0000 7f2111a00640 1 --2- 172.21.15.7:0/1938599020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c06e510 0x7f210c0adfa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f210000d740 tx=0x7f210000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:14.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.573+0000 7f2109600640 1 -- 172.21.15.7:0/1938599020 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2100017070 con 0x7f210c06e510 2024-09-10T21:42:14.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.573+0000 7f2109600640 1 -- 172.21.15.7:0/1938599020 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2100009d20 con 0x7f210c06e510 2024-09-10T21:42:14.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.573+0000 7f2109600640 1 -- 172.21.15.7:0/1938599020 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21000106c0 con 0x7f210c06e510 2024-09-10T21:42:14.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.573+0000 7f2114400640 1 -- 172.21.15.7:0/1938599020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f210c1f3220 con 0x7f210c06e510 2024-09-10T21:42:14.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.573+0000 7f2114400640 1 -- 172.21.15.7:0/1938599020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f210c1f3500 con 0x7f210c06e510 2024-09-10T21:42:14.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.574+0000 7f2114400640 1 -- 172.21.15.7:0/1938599020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f210c160160 con 0x7f210c06e510 2024-09-10T21:42:14.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.576+0000 7f2109600640 1 -- 172.21.15.7:0/1938599020 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2100010860 con 0x7f210c06e510 2024-09-10T21:42:14.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.577+0000 7f2109600640 1 --2- 172.21.15.7:0/1938599020 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f20ec077f70 0x7f20ec07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:14.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.577+0000 7f2109600640 1 -- 172.21.15.7:0/1938599020 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f210009a160 con 0x7f210c06e510 2024-09-10T21:42:14.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.577+0000 7f2111000640 1 --2- 172.21.15.7:0/1938599020 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f20ec077f70 0x7f20ec07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:14.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.578+0000 7f2111000640 1 --2- 172.21.15.7:0/1938599020 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f20ec077f70 0x7f20ec07a430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f20f8033000 tx=0x7f20f8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:14.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.581+0000 7f2109600640 1 -- 172.21.15.7:0/1938599020 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2100062570 con 0x7f210c06e510 2024-09-10T21:42:14.656 INFO:tasks.ceph.mds.b.smithi133.stderr:2024-09-10T21:42:14.653+0000 7fd3eb600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:42:14.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.776+0000 7f2114400640 1 -- 172.21.15.7:0/1938599020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f210c0b1000 con 0x7f210c06e510 2024-09-10T21:42:14.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.777+0000 7f2109600640 1 -- 172.21.15.7:0/1938599020 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v27) ==== 86+0+160 (secure 0 0 0) 0x7f2100066fa0 con 0x7f210c06e510 2024-09-10T21:42:14.778 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.779+0000 7f2114400640 1 -- 172.21.15.7:0/1938599020 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f20ec077f70 msgr2=0x7f20ec07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.779+0000 7f2114400640 1 --2- 172.21.15.7:0/1938599020 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f20ec077f70 0x7f20ec07a430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f20f8033000 tx=0x7f20f8004730 comp rx=0 tx=0).stop 2024-09-10T21:42:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.779+0000 7f2114400640 1 -- 172.21.15.7:0/1938599020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c06e510 msgr2=0x7f210c0adfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.779+0000 7f2114400640 1 --2- 172.21.15.7:0/1938599020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c06e510 0x7f210c0adfa0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f210000d740 tx=0x7f210000dc00 comp rx=0 tx=0).stop 2024-09-10T21:42:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.779+0000 7f2114400640 1 -- 172.21.15.7:0/1938599020 shutdown_connections 2024-09-10T21:42:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.779+0000 7f2114400640 1 --2- 172.21.15.7:0/1938599020 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f20ec077f70 0x7f20ec07a430 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.779+0000 7f2114400640 1 --2- 172.21.15.7:0/1938599020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f210c06e960 0x7f210c1f24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.779+0000 7f2114400640 1 --2- 172.21.15.7:0/1938599020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f210c06ff50 0x7f210c0af4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.779+0000 7f2114400640 1 --2- 172.21.15.7:0/1938599020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210c06e510 0x7f210c0adfa0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.779+0000 7f2114400640 1 -- 172.21.15.7:0/1938599020 >> 172.21.15.7:0/1938599020 conn(0x7f210c0b1660 msgr2=0x7f210c0cc390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:14.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.779+0000 7f2114400640 1 -- 172.21.15.7:0/1938599020 shutdown_connections 2024-09-10T21:42:14.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.779+0000 7f2114400640 1 -- 172.21.15.7:0/1938599020 wait complete. 2024-09-10T21:42:14.795 INFO:teuthology.orchestra.run.smithi007.stdout:[{"version":27,"timestamp":"2024-09-10T21:41:56.644807+0000","name":"","changes":[{"name":"osd.4/osd_mclock_max_capacity_iops_ssd","new_value":"4630.109911"}]}] 2024-09-10T21:42:14.796 INFO:tasks.ceph_manager:config epoch is 27 2024-09-10T21:42:14.796 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T21:42:14.796 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-10T21:42:14.796 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-10T21:42:14.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.955+0000 7f62f2e00640 1 -- 172.21.15.7:0/4090249920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62ec0cc050 msgr2=0x7f62ec0b4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:14.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.955+0000 7f62f2e00640 1 --2- 172.21.15.7:0/4090249920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62ec0cc050 0x7f62ec0b4100 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f62e0009830 tx=0x7f62e002a510 comp rx=0 tx=0).stop 2024-09-10T21:42:14.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.955+0000 7f62f2e00640 1 -- 172.21.15.7:0/4090249920 shutdown_connections 2024-09-10T21:42:14.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.955+0000 7f62f2e00640 1 --2- 172.21.15.7:0/4090249920 >> v2:172.21.15.133:3300/0 conn(0x7f62ec06fa70 0x7f62ec0728d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.955+0000 7f62f2e00640 1 --2- 172.21.15.7:0/4090249920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62ec0cc050 0x7f62ec0b4100 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.955+0000 7f62f2e00640 1 -- 172.21.15.7:0/4090249920 >> 172.21.15.7:0/4090249920 conn(0x7f62ec0b1d70 msgr2=0x7f62ec0cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:14.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.955+0000 7f62f2e00640 1 -- 172.21.15.7:0/4090249920 shutdown_connections 2024-09-10T21:42:14.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.955+0000 7f62f2e00640 1 -- 172.21.15.7:0/4090249920 wait complete. 2024-09-10T21:42:14.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.957+0000 7f62f2e00640 1 Processor -- start 2024-09-10T21:42:14.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.957+0000 7f62f2e00640 1 -- start start 2024-09-10T21:42:14.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.958+0000 7f62f2e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62ec06fa70 0x7f62ec0b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:14.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.958+0000 7f62f2e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62ec0cc050 0x7f62ec1f45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:14.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.958+0000 7f62f2e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62ec15aff0 0x7f62ec0aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:14.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.958+0000 7f62f2e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec073290 con 0x7f62ec06fa70 2024-09-10T21:42:14.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.958+0000 7f62f2e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f62ec073110 con 0x7f62ec0cc050 2024-09-10T21:42:14.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.958+0000 7f62f2e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f62ec073410 con 0x7f62ec15aff0 2024-09-10T21:42:14.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.958+0000 7f62f1a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62ec06fa70 0x7f62ec0b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:14.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.958+0000 7f62f1a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62ec06fa70 0x7f62ec0b0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54054/0 (socket says 172.21.15.7:54054) 2024-09-10T21:42:14.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.958+0000 7f62f1a00640 1 -- 172.21.15.7:0/737245204 learned_addr learned my addr 172.21.15.7:0/737245204 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:14.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.958+0000 7f62f2400640 1 --2- 172.21.15.7:0/737245204 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62ec15aff0 0x7f62ec0aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:14.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.958+0000 7f62f1000640 1 --2- 172.21.15.7:0/737245204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62ec0cc050 0x7f62ec1f45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:14.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.958+0000 7f62f1a00640 1 -- 172.21.15.7:0/737245204 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62ec15aff0 msgr2=0x7f62ec0aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:14.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.959+0000 7f62f1a00640 1 --2- 172.21.15.7:0/737245204 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62ec15aff0 0x7f62ec0aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.959+0000 7f62f1a00640 1 -- 172.21.15.7:0/737245204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62ec0cc050 msgr2=0x7f62ec1f45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:14.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.959+0000 7f62f1a00640 1 --2- 172.21.15.7:0/737245204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62ec0cc050 0x7f62ec1f45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:14.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.959+0000 7f62f1a00640 1 -- 172.21.15.7:0/737245204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62ec1f4f10 con 0x7f62ec06fa70 2024-09-10T21:42:14.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.959+0000 7f62f2400640 1 --2- 172.21.15.7:0/737245204 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62ec15aff0 0x7f62ec0aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:14.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.959+0000 7f62f1000640 1 --2- 172.21.15.7:0/737245204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62ec0cc050 0x7f62ec1f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:14.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.959+0000 7f62f1a00640 1 --2- 172.21.15.7:0/737245204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62ec06fa70 0x7f62ec0b0f30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f62e002ab00 tx=0x7f62e0032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:14.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.959+0000 7f62e9600640 1 -- 172.21.15.7:0/737245204 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0040070 con 0x7f62ec06fa70 2024-09-10T21:42:14.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.959+0000 7f62e9600640 1 -- 172.21.15.7:0/737245204 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62e0005310 con 0x7f62ec06fa70 2024-09-10T21:42:14.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.959+0000 7f62f2e00640 1 -- 172.21.15.7:0/737245204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62ec1f51f0 con 0x7f62ec06fa70 2024-09-10T21:42:14.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.959+0000 7f62e9600640 1 -- 172.21.15.7:0/737245204 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62e0005c90 con 0x7f62ec06fa70 2024-09-10T21:42:14.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.959+0000 7f62f2e00640 1 -- 172.21.15.7:0/737245204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62ec0afa10 con 0x7f62ec06fa70 2024-09-10T21:42:14.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.960+0000 7f62f2e00640 1 -- 172.21.15.7:0/737245204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62ec073930 con 0x7f62ec06fa70 2024-09-10T21:42:14.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.961+0000 7f62e9600640 1 -- 172.21.15.7:0/737245204 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f62e004c030 con 0x7f62ec06fa70 2024-09-10T21:42:14.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.962+0000 7f62e9600640 1 --2- 172.21.15.7:0/737245204 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f62cc077f70 0x7f62cc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:14.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.963+0000 7f62e9600640 1 -- 172.21.15.7:0/737245204 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f62e00bad80 con 0x7f62ec06fa70 2024-09-10T21:42:14.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.963+0000 7f62f1000640 1 --2- 172.21.15.7:0/737245204 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f62cc077f70 0x7f62cc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:14.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.963+0000 7f62f1000640 1 --2- 172.21.15.7:0/737245204 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f62cc077f70 0x7f62cc07a430 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f62d8001220 tx=0x7f62d8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:14.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:14.966+0000 7f62e9600640 1 -- 172.21.15.7:0/737245204 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f62e00837e0 con 0x7f62ec06fa70 2024-09-10T21:42:15.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.211+0000 7f62f2e00640 1 -- 172.21.15.7:0/737245204 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f62ec06f260 con 0x7f62ec06fa70 2024-09-10T21:42:15.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.214+0000 7f62e9600640 1 -- 172.21.15.7:0/737245204 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+193250 (secure 0 0 0) 0x7f62e008a0e0 con 0x7f62ec06fa70 2024-09-10T21:42:15.218 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.217+0000 7f62f2e00640 1 -- 172.21.15.7:0/737245204 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f62cc077f70 msgr2=0x7f62cc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.217+0000 7f62f2e00640 1 --2- 172.21.15.7:0/737245204 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f62cc077f70 0x7f62cc07a430 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f62d8001220 tx=0x7f62d8009040 comp rx=0 tx=0).stop 2024-09-10T21:42:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.217+0000 7f62f2e00640 1 -- 172.21.15.7:0/737245204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62ec06fa70 msgr2=0x7f62ec0b0f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:15.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.217+0000 7f62f2e00640 1 --2- 172.21.15.7:0/737245204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62ec06fa70 0x7f62ec0b0f30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f62e002ab00 tx=0x7f62e0032c10 comp rx=0 tx=0).stop 2024-09-10T21:42:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.217+0000 7f62f2e00640 1 -- 172.21.15.7:0/737245204 shutdown_connections 2024-09-10T21:42:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.217+0000 7f62f2e00640 1 --2- 172.21.15.7:0/737245204 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f62cc077f70 0x7f62cc07a430 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.217+0000 7f62f2e00640 1 --2- 172.21.15.7:0/737245204 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62ec0cc050 0x7f62ec1f45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.217+0000 7f62f2e00640 1 --2- 172.21.15.7:0/737245204 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f62ec15aff0 0x7f62ec0aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.217+0000 7f62f2e00640 1 --2- 172.21.15.7:0/737245204 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62ec06fa70 0x7f62ec0b0f30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.217+0000 7f62f2e00640 1 -- 172.21.15.7:0/737245204 >> 172.21.15.7:0/737245204 conn(0x7f62ec0b1d70 msgr2=0x7f62ec15ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.218+0000 7f62f2e00640 1 -- 172.21.15.7:0/737245204 shutdown_connections 2024-09-10T21:42:15.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.218+0000 7f62f2e00640 1 -- 172.21.15.7:0/737245204 wait complete. 2024-09-10T21:42:15.235 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":5,"flags":0,"active_gid":4123,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6800","nonce":4032855124},{"type":"v1","addr":"172.21.15.133:6801","nonce":4032855124}]},"active_addr":"172.21.15.133:6801/4032855124","active_change":"2024-09-10T21:41:47.842751+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4170,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":3658492980}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":3552524572}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":3234272629}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.133:0","nonce":3599911299}]}]} 2024-09-10T21:42:15.239 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-10T21:42:15.239 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-10T21:42:15.239 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:42:15.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.397+0000 7fe3fae00640 1 -- 172.21.15.7:0/285721206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f40b2b50 msgr2=0x7fe3f40d4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:15.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.397+0000 7fe3fae00640 1 --2- 172.21.15.7:0/285721206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f40b2b50 0x7fe3f40d4870 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec005fd0 tx=0x7fe3ec02aaa0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.398+0000 7fe3fae00640 1 -- 172.21.15.7:0/285721206 shutdown_connections 2024-09-10T21:42:15.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.398+0000 7fe3fae00640 1 --2- 172.21.15.7:0/285721206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f40b2b50 0x7fe3f40d4870 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.398+0000 7fe3fae00640 1 -- 172.21.15.7:0/285721206 >> 172.21.15.7:0/285721206 conn(0x7fe3f40d3d50 msgr2=0x7fe3f40d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:15.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.398+0000 7fe3fae00640 1 -- 172.21.15.7:0/285721206 shutdown_connections 2024-09-10T21:42:15.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.398+0000 7fe3fae00640 1 -- 172.21.15.7:0/285721206 wait complete. 2024-09-10T21:42:15.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.399+0000 7fe3fae00640 1 Processor -- start 2024-09-10T21:42:15.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.399+0000 7fe3fae00640 1 -- start start 2024-09-10T21:42:15.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.400+0000 7fe3fae00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f40b2b50 0x7fe3f40ca4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:15.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.400+0000 7fe3fae00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3f40d0e70 0x7fe3f414d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.400+0000 7fe3fae00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3f415dfd0 0x7fe3f41f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.400+0000 7fe3fae00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f40d2770 con 0x7fe3f40b2b50 2024-09-10T21:42:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.400+0000 7fe3fae00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f40d25f0 con 0x7fe3f40d0e70 2024-09-10T21:42:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.400+0000 7fe3fae00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe3f40d28f0 con 0x7fe3f415dfd0 2024-09-10T21:42:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.401+0000 7fe3f9a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f40b2b50 0x7fe3f40ca4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.401+0000 7fe3f9a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f40b2b50 0x7fe3f40ca4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54062/0 (socket says 172.21.15.7:54062) 2024-09-10T21:42:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.401+0000 7fe3f9a00640 1 -- 172.21.15.7:0/2923863925 learned_addr learned my addr 172.21.15.7:0/2923863925 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.401+0000 7fe3fa400640 1 --2- 172.21.15.7:0/2923863925 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3f415dfd0 0x7fe3f41f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.401+0000 7fe3f9000640 1 --2- 172.21.15.7:0/2923863925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3f40d0e70 0x7fe3f414d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.401+0000 7fe3f9a00640 1 -- 172.21.15.7:0/2923863925 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3f415dfd0 msgr2=0x7fe3f41f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.401+0000 7fe3f9a00640 1 --2- 172.21.15.7:0/2923863925 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3f415dfd0 0x7fe3f41f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.401+0000 7fe3f9a00640 1 -- 172.21.15.7:0/2923863925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3f40d0e70 msgr2=0x7fe3f414d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:15.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.401+0000 7fe3f9a00640 1 --2- 172.21.15.7:0/2923863925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3f40d0e70 0x7fe3f414d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.401+0000 7fe3f9a00640 1 -- 172.21.15.7:0/2923863925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f40caa30 con 0x7fe3f40b2b50 2024-09-10T21:42:15.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.401+0000 7fe3f9000640 1 --2- 172.21.15.7:0/2923863925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3f40d0e70 0x7fe3f414d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:15.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.401+0000 7fe3f9a00640 1 --2- 172.21.15.7:0/2923863925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f40b2b50 0x7fe3f40ca4f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800c3a0 tx=0x7fe3e800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:15.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.402+0000 7fe3f1600640 1 -- 172.21.15.7:0/2923863925 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8017070 con 0x7fe3f40b2b50 2024-09-10T21:42:15.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.402+0000 7fe3f1600640 1 -- 172.21.15.7:0/2923863925 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3e80053c0 con 0x7fe3f40b2b50 2024-09-10T21:42:15.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.402+0000 7fe3fae00640 1 -- 172.21.15.7:0/2923863925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f40b7f10 con 0x7fe3f40b2b50 2024-09-10T21:42:15.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.402+0000 7fe3f1600640 1 -- 172.21.15.7:0/2923863925 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3e8003850 con 0x7fe3f40b2b50 2024-09-10T21:42:15.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.402+0000 7fe3fae00640 1 -- 172.21.15.7:0/2923863925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f40b84d0 con 0x7fe3f40b2b50 2024-09-10T21:42:15.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.403+0000 7fe3fae00640 1 -- 172.21.15.7:0/2923863925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3f40d2da0 con 0x7fe3f40b2b50 2024-09-10T21:42:15.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.404+0000 7fe3f1600640 1 -- 172.21.15.7:0/2923863925 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe3e80039f0 con 0x7fe3f40b2b50 2024-09-10T21:42:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.405+0000 7fe3f1600640 1 --2- 172.21.15.7:0/2923863925 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe3d4077fa0 0x7fe3d407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.405+0000 7fe3f1600640 1 -- 172.21.15.7:0/2923863925 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe3e8099390 con 0x7fe3f40b2b50 2024-09-10T21:42:15.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.406+0000 7fe3f9000640 1 --2- 172.21.15.7:0/2923863925 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe3d4077fa0 0x7fe3d407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:15.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.406+0000 7fe3f9000640 1 --2- 172.21.15.7:0/2923863925 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe3d4077fa0 0x7fe3d407a460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0007fd0 tx=0x7fe3e0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:15.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.409+0000 7fe3f1600640 1 -- 172.21.15.7:0/2923863925 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe3e80617a0 con 0x7fe3f40b2b50 2024-09-10T21:42:15.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.604+0000 7fe3fae00640 1 -- 172.21.15.7:0/2923863925 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3f40b9cd0 con 0x7fe3f40b2b50 2024-09-10T21:42:15.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.605+0000 7fe3f1600640 1 -- 172.21.15.7:0/2923863925 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16629 (secure 0 0 0) 0x7fe3e80661d0 con 0x7fe3f40b2b50 2024-09-10T21:42:15.606 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:15.607 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:42:10.684978+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:42:07.149267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:42:08.077083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:42:15.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.607+0000 7fe3fae00640 1 -- 172.21.15.7:0/2923863925 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe3d4077fa0 msgr2=0x7fe3d407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:15.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.607+0000 7fe3fae00640 1 --2- 172.21.15.7:0/2923863925 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe3d4077fa0 0x7fe3d407a460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0007fd0 tx=0x7fe3e0009040 comp rx=0 tx=0).stop 2024-09-10T21:42:15.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.607+0000 7fe3fae00640 1 -- 172.21.15.7:0/2923863925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f40b2b50 msgr2=0x7fe3f40ca4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:15.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.607+0000 7fe3fae00640 1 --2- 172.21.15.7:0/2923863925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f40b2b50 0x7fe3f40ca4f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800c3a0 tx=0x7fe3e800c860 comp rx=0 tx=0).stop 2024-09-10T21:42:15.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.607+0000 7fe3fae00640 1 -- 172.21.15.7:0/2923863925 shutdown_connections 2024-09-10T21:42:15.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.607+0000 7fe3fae00640 1 --2- 172.21.15.7:0/2923863925 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe3d4077fa0 0x7fe3d407a460 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.607+0000 7fe3fae00640 1 --2- 172.21.15.7:0/2923863925 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe3f40d0e70 0x7fe3f414d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.607+0000 7fe3fae00640 1 --2- 172.21.15.7:0/2923863925 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3f415dfd0 0x7fe3f41f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.607+0000 7fe3fae00640 1 --2- 172.21.15.7:0/2923863925 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3f40b2b50 0x7fe3f40ca4f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.607+0000 7fe3fae00640 1 -- 172.21.15.7:0/2923863925 >> 172.21.15.7:0/2923863925 conn(0x7fe3f40d3d50 msgr2=0x7fe3f40b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:15.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.608+0000 7fe3fae00640 1 -- 172.21.15.7:0/2923863925 shutdown_connections 2024-09-10T21:42:15.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.608+0000 7fe3fae00640 1 -- 172.21.15.7:0/2923863925 wait complete. 2024-09-10T21:42:15.624 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-10T21:42:15.625 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:42:15.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.784+0000 7fe6ee600640 1 -- 172.21.15.7:0/555284455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6e815f290 msgr2=0x7fe6e80cf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:15.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.784+0000 7fe6ee600640 1 --2- 172.21.15.7:0/555284455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6e815f290 0x7fe6e80cf8f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8005fd0 tx=0x7fe6d802a4b0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.784+0000 7fe6ee600640 1 -- 172.21.15.7:0/555284455 shutdown_connections 2024-09-10T21:42:15.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.784+0000 7fe6ee600640 1 --2- 172.21.15.7:0/555284455 >> v2:172.21.15.133:3300/0 conn(0x7fe6e80cfe30 0x7fe6e80cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.784+0000 7fe6ee600640 1 --2- 172.21.15.7:0/555284455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6e815f290 0x7fe6e80cf8f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.784+0000 7fe6ee600640 1 -- 172.21.15.7:0/555284455 >> 172.21.15.7:0/555284455 conn(0x7fe6e80d1d20 msgr2=0x7fe6e80d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:15.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.784+0000 7fe6ee600640 1 -- 172.21.15.7:0/555284455 shutdown_connections 2024-09-10T21:42:15.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.784+0000 7fe6ee600640 1 -- 172.21.15.7:0/555284455 wait complete. 2024-09-10T21:42:15.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.786+0000 7fe6ee600640 1 Processor -- start 2024-09-10T21:42:15.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.786+0000 7fe6ee600640 1 -- start start 2024-09-10T21:42:15.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.787+0000 7fe6ee600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6e80cd910 0x7fe6e80b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:15.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.787+0000 7fe6ee600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e80cfe30 0x7fe6e814d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:15.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.787+0000 7fe6ee600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e815f290 0x7fe6e80b6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:15.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.787+0000 7fe6ee600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e815f7f0 con 0x7fe6e815f290 2024-09-10T21:42:15.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.787+0000 7fe6ee600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e815f670 con 0x7fe6e80cfe30 2024-09-10T21:42:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.787+0000 7fe6ee600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe6e80ce3c0 con 0x7fe6e80cd910 2024-09-10T21:42:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.787+0000 7fe6e7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e815f290 0x7fe6e80b6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.787+0000 7fe6e7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e815f290 0x7fe6e80b6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54070/0 (socket says 172.21.15.7:54070) 2024-09-10T21:42:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.787+0000 7fe6e7e00640 1 -- 172.21.15.7:0/4239380132 learned_addr learned my addr 172.21.15.7:0/4239380132 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.787+0000 7fe6e7400640 1 --2- 172.21.15.7:0/4239380132 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6e80cd910 0x7fe6e80b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.787+0000 7fe6e6a00640 1 --2- 172.21.15.7:0/4239380132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e80cfe30 0x7fe6e814d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.788+0000 7fe6e7e00640 1 -- 172.21.15.7:0/4239380132 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6e80cd910 msgr2=0x7fe6e80b4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.788+0000 7fe6e7e00640 1 --2- 172.21.15.7:0/4239380132 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6e80cd910 0x7fe6e80b4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.788+0000 7fe6e7e00640 1 -- 172.21.15.7:0/4239380132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e80cfe30 msgr2=0x7fe6e814d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.788+0000 7fe6e7e00640 1 --2- 172.21.15.7:0/4239380132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e80cfe30 0x7fe6e814d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.788+0000 7fe6e7e00640 1 -- 172.21.15.7:0/4239380132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e80b6750 con 0x7fe6e815f290 2024-09-10T21:42:15.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.788+0000 7fe6e7e00640 1 --2- 172.21.15.7:0/4239380132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e815f290 0x7fe6e80b6210 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000c730 tx=0x7fe6e000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.788+0000 7fe6df400640 1 -- 172.21.15.7:0/4239380132 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0003b90 con 0x7fe6e815f290 2024-09-10T21:42:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.788+0000 7fe6df400640 1 -- 172.21.15.7:0/4239380132 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6e0003d50 con 0x7fe6e815f290 2024-09-10T21:42:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.788+0000 7fe6ee600640 1 -- 172.21.15.7:0/4239380132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e80b5170 con 0x7fe6e815f290 2024-09-10T21:42:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.788+0000 7fe6df400640 1 -- 172.21.15.7:0/4239380132 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e000aab0 con 0x7fe6e815f290 2024-09-10T21:42:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.789+0000 7fe6ee600640 1 -- 172.21.15.7:0/4239380132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e80b5730 con 0x7fe6e815f290 2024-09-10T21:42:15.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.790+0000 7fe6ee600640 1 -- 172.21.15.7:0/4239380132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6e80b5f00 con 0x7fe6e815f290 2024-09-10T21:42:15.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.791+0000 7fe6df400640 1 -- 172.21.15.7:0/4239380132 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe6e000ac50 con 0x7fe6e815f290 2024-09-10T21:42:15.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.792+0000 7fe6df400640 1 --2- 172.21.15.7:0/4239380132 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe6c8077f70 0x7fe6c807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:15.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.792+0000 7fe6df400640 1 -- 172.21.15.7:0/4239380132 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe6e0099a00 con 0x7fe6e815f290 2024-09-10T21:42:15.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.792+0000 7fe6e7400640 1 --2- 172.21.15.7:0/4239380132 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe6c8077f70 0x7fe6c807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:15.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.793+0000 7fe6e7400640 1 --2- 172.21.15.7:0/4239380132 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe6c8077f70 0x7fe6c807a430 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8033000 tx=0x7fe6d8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:15.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.796+0000 7fe6df400640 1 -- 172.21.15.7:0/4239380132 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe6e80b6750 con 0x7fe6e815f290 2024-09-10T21:42:15.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.990+0000 7fe6ee600640 1 -- 172.21.15.7:0/4239380132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6e80b33f0 con 0x7fe6e815f290 2024-09-10T21:42:15.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.992+0000 7fe6df400640 1 -- 172.21.15.7:0/4239380132 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16629 (secure 0 0 0) 0x7fe6e80b33f0 con 0x7fe6e815f290 2024-09-10T21:42:15.993 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:15.993 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:42:10.684978+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:42:07.149267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:42:08.077083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:42:15.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.994+0000 7fe6ee600640 1 -- 172.21.15.7:0/4239380132 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe6c8077f70 msgr2=0x7fe6c807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:15.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.994+0000 7fe6ee600640 1 --2- 172.21.15.7:0/4239380132 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe6c8077f70 0x7fe6c807a430 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8033000 tx=0x7fe6d8004730 comp rx=0 tx=0).stop 2024-09-10T21:42:15.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.994+0000 7fe6ee600640 1 -- 172.21.15.7:0/4239380132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e815f290 msgr2=0x7fe6e80b6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:15.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.994+0000 7fe6ee600640 1 --2- 172.21.15.7:0/4239380132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e815f290 0x7fe6e80b6210 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe6e000c730 tx=0x7fe6e000cbf0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.994+0000 7fe6ee600640 1 -- 172.21.15.7:0/4239380132 shutdown_connections 2024-09-10T21:42:15.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.994+0000 7fe6ee600640 1 --2- 172.21.15.7:0/4239380132 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe6c8077f70 0x7fe6c807a430 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.994+0000 7fe6ee600640 1 --2- 172.21.15.7:0/4239380132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe6e80cfe30 0x7fe6e814d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.994+0000 7fe6ee600640 1 --2- 172.21.15.7:0/4239380132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6e815f290 0x7fe6e80b6210 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.994+0000 7fe6ee600640 1 --2- 172.21.15.7:0/4239380132 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6e80cd910 0x7fe6e80b4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:15.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.994+0000 7fe6ee600640 1 -- 172.21.15.7:0/4239380132 >> 172.21.15.7:0/4239380132 conn(0x7fe6e80d1d20 msgr2=0x7fe6e8071a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:15.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.994+0000 7fe6ee600640 1 -- 172.21.15.7:0/4239380132 shutdown_connections 2024-09-10T21:42:15.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:15.994+0000 7fe6ee600640 1 -- 172.21.15.7:0/4239380132 wait complete. 2024-09-10T21:42:16.012 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T21:42:16.013 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T21:42:16.013 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T21:42:16.014 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-10T21:42:16.014 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-10T21:42:16.015 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-10T21:42:16.015 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-10T21:42:16.016 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-10T21:42:16.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.295+0000 7f0840800640 1 -- 172.21.15.7:0/1027996098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08380ccf50 msgr2=0x7f08380cbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.295+0000 7f0840800640 1 --2- 172.21.15.7:0/1027996098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08380ccf50 0x7f08380cbe20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0828009830 tx=0x7f082802a510 comp rx=0 tx=0).stop 2024-09-10T21:42:16.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.296+0000 7f0840800640 1 -- 172.21.15.7:0/1027996098 shutdown_connections 2024-09-10T21:42:16.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.297+0000 7f0840800640 1 --2- 172.21.15.7:0/1027996098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08380ccf50 0x7f08380cbe20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.297+0000 7f0840800640 1 -- 172.21.15.7:0/1027996098 >> 172.21.15.7:0/1027996098 conn(0x7f08380b1740 msgr2=0x7f08380b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.298+0000 7f0840800640 1 -- 172.21.15.7:0/1027996098 shutdown_connections 2024-09-10T21:42:16.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.298+0000 7f0840800640 1 -- 172.21.15.7:0/1027996098 wait complete. 2024-09-10T21:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.301+0000 7f0840800640 1 Processor -- start 2024-09-10T21:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.301+0000 7f0840800640 1 -- start start 2024-09-10T21:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.303+0000 7f0840800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08380ccb70 0x7f08381f8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.303+0000 7f0840800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08380ccf50 0x7f08380c2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.303+0000 7f0840800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08380cece0 0x7f08380c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.304+0000 7f083de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08380ccb70 0x7f08381f8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.304+0000 7f083d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08380ccf50 0x7f08380c2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.304+0000 7f083de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08380ccb70 0x7f08381f8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:48816/0 (socket says 172.21.15.7:48816) 2024-09-10T21:42:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.304+0000 7f083d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08380ccf50 0x7f08380c2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54088/0 (socket says 172.21.15.7:54088) 2024-09-10T21:42:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.305+0000 7f0840800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08380cf2d0 con 0x7f08380ccf50 2024-09-10T21:42:16.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.305+0000 7f0840800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f08380cf150 con 0x7f08380ccb70 2024-09-10T21:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.305+0000 7f0840800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f08380cf450 con 0x7f08380cece0 2024-09-10T21:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.305+0000 7f083f200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08380cece0 0x7f08380c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.305+0000 7f083f200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08380cece0 0x7f08380c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:50060/0 (socket says 172.21.15.7:50060) 2024-09-10T21:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.305+0000 7f083f200640 1 -- 172.21.15.7:0/142808410 learned_addr learned my addr 172.21.15.7:0/142808410 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:16.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.306+0000 7f083f200640 1 -- 172.21.15.7:0/142808410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08380ccb70 msgr2=0x7f08381f8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.306+0000 7f083f200640 1 --2- 172.21.15.7:0/142808410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08380ccb70 0x7f08381f8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.306+0000 7f083f200640 1 -- 172.21.15.7:0/142808410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08380ccf50 msgr2=0x7f08380c2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.307+0000 7f083f200640 1 --2- 172.21.15.7:0/142808410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08380ccf50 0x7f08380c2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.307+0000 7f083f200640 1 -- 172.21.15.7:0/142808410 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08380c6ff0 con 0x7f08380cece0 2024-09-10T21:42:16.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.307+0000 7f083d400640 1 --2- 172.21.15.7:0/142808410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08380ccf50 0x7f08380c2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:42:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.307+0000 7fa59be00640 1 -- 172.21.15.7:0/171437149 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa59403bf60 msgr2=0x7fa59403a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.307+0000 7fa59be00640 1 --2- 172.21.15.7:0/171437149 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa59403bf60 0x7fa59403a370 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa594053110 tx=0x7fa58c02a580 comp rx=0 tx=0).stop 2024-09-10T21:42:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.307+0000 7fa59be00640 1 -- 172.21.15.7:0/171437149 shutdown_connections 2024-09-10T21:42:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.307+0000 7fa59be00640 1 --2- 172.21.15.7:0/171437149 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa59403bf60 0x7fa59403a370 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.307+0000 7fa59be00640 1 --2- 172.21.15.7:0/171437149 >> v2:172.21.15.133:3300/0 conn(0x7fa594006b30 0x7fa59403ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.307+0000 7fa59be00640 1 --2- 172.21.15.7:0/171437149 >> v2:172.21.15.7:3300/0 conn(0x7fa5940e28c0 0x7fa5940538c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.307+0000 7fa59be00640 1 -- 172.21.15.7:0/171437149 >> 172.21.15.7:0/171437149 conn(0x7fa594055550 msgr2=0x7fa594055930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.308+0000 7fa59be00640 1 -- 172.21.15.7:0/171437149 shutdown_connections 2024-09-10T21:42:16.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.308+0000 7f083f200640 1 --2- 172.21.15.7:0/142808410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08380cece0 0x7f08380c8ca0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f083400b990 tx=0x7f083400be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.308+0000 7f0832000640 1 -- 172.21.15.7:0/142808410 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834019070 con 0x7f08380cece0 2024-09-10T21:42:16.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.308+0000 7fa59be00640 1 -- 172.21.15.7:0/171437149 wait complete. 2024-09-10T21:42:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.308+0000 7f0832000640 1 -- 172.21.15.7:0/142808410 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f083400cc90 con 0x7f08380cece0 2024-09-10T21:42:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.308+0000 7f0832000640 1 -- 172.21.15.7:0/142808410 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08340146a0 con 0x7f08380cece0 2024-09-10T21:42:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.309+0000 7f0840800640 1 -- 172.21.15.7:0/142808410 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08380c72d0 con 0x7f08380cece0 2024-09-10T21:42:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.309+0000 7f0840800640 1 -- 172.21.15.7:0/142808410 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f08380c7890 con 0x7f08380cece0 2024-09-10T21:42:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.309+0000 7f85d9200640 1 -- 172.21.15.7:0/2487734733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85d40d1f90 msgr2=0x7f85d40d4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.309+0000 7f85d9200640 1 --2- 172.21.15.7:0/2487734733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85d40d1f90 0x7f85d40d4a30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f85c0009830 tx=0x7f85c002a5b0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.310+0000 7f0840800640 1 -- 172.21.15.7:0/142808410 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f08380cbe20 con 0x7f08380cece0 2024-09-10T21:42:16.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.310+0000 7fe47da00640 1 -- 172.21.15.7:0/2067826039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe478164d80 msgr2=0x7fe478167170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.310+0000 7fe47da00640 1 --2- 172.21.15.7:0/2067826039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe478164d80 0x7fe478167170 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe470005fd0 tx=0x7fe47002a7c0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.310+0000 7fa59be00640 1 Processor -- start 2024-09-10T21:42:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.310+0000 7f85d9200640 1 -- 172.21.15.7:0/2487734733 shutdown_connections 2024-09-10T21:42:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.310+0000 7f85d9200640 1 --2- 172.21.15.7:0/2487734733 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85d40d1f90 0x7f85d40d4a30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.310+0000 7f85d9200640 1 -- 172.21.15.7:0/2487734733 >> 172.21.15.7:0/2487734733 conn(0x7f85d40d3f10 msgr2=0x7f85d40d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.310+0000 7f85d9200640 1 -- 172.21.15.7:0/2487734733 shutdown_connections 2024-09-10T21:42:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.310+0000 7f85d9200640 1 -- 172.21.15.7:0/2487734733 wait complete. 2024-09-10T21:42:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.311+0000 7fe47da00640 1 -- 172.21.15.7:0/2067826039 shutdown_connections 2024-09-10T21:42:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.311+0000 7fe47da00640 1 --2- 172.21.15.7:0/2067826039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe478164d80 0x7fe478167170 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.311+0000 7fe47da00640 1 -- 172.21.15.7:0/2067826039 >> 172.21.15.7:0/2067826039 conn(0x7fe4780d1d60 msgr2=0x7fe4780d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.312+0000 7fe47da00640 1 -- 172.21.15.7:0/2067826039 shutdown_connections 2024-09-10T21:42:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.311+0000 7fa59be00640 1 -- start start 2024-09-10T21:42:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.313+0000 7fa59be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa594006b30 0x7fa59404ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.313+0000 7fa59be00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa59403bf60 0x7fa59417cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.313+0000 7fa59be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5940e28c0 0x7fa5940d3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.313+0000 7fa59be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa59403acb0 con 0x7fa5940e28c0 2024-09-10T21:42:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.313+0000 7fa59be00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa59403ab30 con 0x7fa594006b30 2024-09-10T21:42:16.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.313+0000 7fa59be00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa594057c30 con 0x7fa59403bf60 2024-09-10T21:42:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.313+0000 7f85d9200640 1 Processor -- start 2024-09-10T21:42:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.313+0000 7f85d9200640 1 -- start start 2024-09-10T21:42:16.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.313+0000 7f0832000640 1 -- 172.21.15.7:0/142808410 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f083401d3b0 con 0x7f08380cece0 2024-09-10T21:42:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.313+0000 7fa59aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa594006b30 0x7fa59404ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.314+0000 7fa59b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5940e28c0 0x7fa5940d3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.314+0000 7fa59b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5940e28c0 0x7fa5940d3bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54102/0 (socket says 172.21.15.7:54102) 2024-09-10T21:42:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.312+0000 7fe47da00640 1 -- 172.21.15.7:0/2067826039 wait complete. 2024-09-10T21:42:16.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.314+0000 7fa59b400640 1 -- 172.21.15.7:0/1348337140 learned_addr learned my addr 172.21.15.7:0/1348337140 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.314+0000 7fe47da00640 1 Processor -- start 2024-09-10T21:42:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.314+0000 7fe47da00640 1 -- start start 2024-09-10T21:42:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.314+0000 7fa59a000640 1 --2- 172.21.15.7:0/1348337140 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa59403bf60 0x7fa59417cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.315+0000 7f85d9200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85d40cf5c0 0x7f85d40b9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.315+0000 7fa59b400640 1 -- 172.21.15.7:0/1348337140 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa59403bf60 msgr2=0x7fa59417cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.315+0000 7f85d9200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d40d1f90 0x7f85d415fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.316+0000 7fa59b400640 1 --2- 172.21.15.7:0/1348337140 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa59403bf60 0x7fa59417cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.316+0000 7fa59b400640 1 -- 172.21.15.7:0/1348337140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa594006b30 msgr2=0x7fa59404ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.316+0000 7f85d9200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85d4157c70 0x7f85d40c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.316+0000 7fe47da00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4780b2b90 0x7fe4780cf050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.316+0000 7f85d9200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f85d40cfbb0 con 0x7f85d40d1f90 2024-09-10T21:42:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.316+0000 7fe47da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe47815f440 0x7fe4781f6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.316+0000 7f85d9200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f85d40cfa30 con 0x7f85d40cf5c0 2024-09-10T21:42:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.316+0000 7f85d9200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f85d40cfd30 con 0x7f85d4157c70 2024-09-10T21:42:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.316+0000 7f85d2800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85d4157c70 0x7f85d40c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.316+0000 7fa59b400640 1 --2- 172.21.15.7:0/1348337140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa594006b30 0x7fa59404ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.316+0000 7f0832000640 1 --2- 172.21.15.7:0/142808410 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f081c078070 0x7f081c07a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.317+0000 7fa59b400640 1 -- 172.21.15.7:0/1348337140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5940e3080 con 0x7fa5940e28c0 2024-09-10T21:42:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.317+0000 7fe47da00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe478164d80 0x7fe4781f9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.317+0000 7fa59aa00640 1 --2- 172.21.15.7:0/1348337140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa594006b30 0x7fa59404ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:16.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.317+0000 7fa59a000640 1 --2- 172.21.15.7:0/1348337140 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa59403bf60 0x7fa59417cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.316+0000 7f85d2800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85d4157c70 0x7f85d40c77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:50088/0 (socket says 172.21.15.7:50088) 2024-09-10T21:42:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.316+0000 7f85d2800640 1 -- 172.21.15.7:0/375815459 learned_addr learned my addr 172.21.15.7:0/375815459 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.317+0000 7f85d1400640 1 --2- 172.21.15.7:0/375815459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d40d1f90 0x7f85d415fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.317+0000 7fe476a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4780b2b90 0x7fe4780cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.317+0000 7fe47da00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe4780b2830 con 0x7fe47815f440 2024-09-10T21:42:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.317+0000 7fe47da00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe47806dc30 con 0x7fe478164d80 2024-09-10T21:42:16.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.317+0000 7fe47da00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe4780cc840 con 0x7fe4780b2b90 2024-09-10T21:42:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.317+0000 7f0832000640 1 -- 172.21.15.7:0/142808410 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f083409a490 con 0x7f08380cece0 2024-09-10T21:42:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.317+0000 7f0832000640 1 --2- 172.21.15.7:0/142808410 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x7f081c080290 0x7f081c0826b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.317+0000 7fe477400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe478164d80 0x7fe4781f9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.317+0000 7fe477400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe478164d80 0x7fe4781f9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:48864/0 (socket says 172.21.15.7:48864) 2024-09-10T21:42:16.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.317+0000 7fe477400640 1 -- 172.21.15.7:0/4049770036 learned_addr learned my addr 172.21.15.7:0/4049770036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.317+0000 7f0832000640 1 -- 172.21.15.7:0/142808410 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0834012e50 con 0x7f081c080290 2024-09-10T21:42:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.317+0000 7f85d2800640 1 -- 172.21.15.7:0/375815459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85d40cf5c0 msgr2=0x7f85d40b9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.316+0000 7f85d1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85d40cf5c0 0x7f85d40b9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.317+0000 7f0832000640 1 -- 172.21.15.7:0/142808410 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f083409a880 con 0x7f08380cece0 2024-09-10T21:42:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.318+0000 7f083de00640 1 --2- 172.21.15.7:0/142808410 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f081c078070 0x7f081c07a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.318+0000 7f083d400640 1 --2- 172.21.15.7:0/142808410 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x7f081c080290 0x7f081c0826b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.318+0000 7f85d2800640 1 --2- 172.21.15.7:0/375815459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85d40cf5c0 0x7f85d40b9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.318+0000 7f85d2800640 1 -- 172.21.15.7:0/375815459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d40d1f90 msgr2=0x7f85d415fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.318+0000 7f85d2800640 1 --2- 172.21.15.7:0/375815459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d40d1f90 0x7f85d415fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.318+0000 7f85d2800640 1 -- 172.21.15.7:0/375815459 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d415d200 con 0x7f85d4157c70 2024-09-10T21:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.318+0000 7f85d1400640 1 --2- 172.21.15.7:0/375815459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f85d40d1f90 0x7f85d415fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:16.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.318+0000 7fe476000640 1 --2- 172.21.15.7:0/4049770036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe47815f440 0x7fe4781f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.318+0000 7fe476a00640 1 -- 172.21.15.7:0/4049770036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe478164d80 msgr2=0x7fe4781f9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.318+0000 7fe476a00640 1 --2- 172.21.15.7:0/4049770036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe478164d80 0x7fe4781f9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.318+0000 7fe476a00640 1 -- 172.21.15.7:0/4049770036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe47815f440 msgr2=0x7fe4781f6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.318+0000 7fa59b400640 1 --2- 172.21.15.7:0/1348337140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5940e28c0 0x7fa5940d3bd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009230 tx=0x7fa58c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.318+0000 7fe476a00640 1 --2- 172.21.15.7:0/4049770036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe47815f440 0x7fe4781f6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.318+0000 7fe476a00640 1 -- 172.21.15.7:0/4049770036 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe478152450 con 0x7fe4780b2b90 2024-09-10T21:42:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.318+0000 7fe476000640 1 --2- 172.21.15.7:0/4049770036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe47815f440 0x7fe4781f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.318+0000 7f85d1e00640 1 --2- 172.21.15.7:0/375815459 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f85d40cf5c0 0x7f85d40b9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.319+0000 7f85d2800640 1 --2- 172.21.15.7:0/375815459 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85d4157c70 0x7f85d40c77a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00c730 tx=0x7f85cc00cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.319+0000 7fa592a00640 1 -- 172.21.15.7:0/1348337140 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c03f070 con 0x7fa5940e28c0 2024-09-10T21:42:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.319+0000 7f083d400640 1 --2- 172.21.15.7:0/142808410 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x7f081c080290 0x7f081c0826b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.319+0000 7f083de00640 1 --2- 172.21.15.7:0/142808410 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f081c078070 0x7f081c07a530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f082c007fd0 tx=0x7f082c001410 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.319+0000 7fe476a00640 1 --2- 172.21.15.7:0/4049770036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4780b2b90 0x7fe4780cf050 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe46800c700 tx=0x7fe46800cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.320+0000 7fe46ea00640 1 -- 172.21.15.7:0/4049770036 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468017070 con 0x7fe4780b2b90 2024-09-10T21:42:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.320+0000 7fe47da00640 1 -- 172.21.15.7:0/4049770036 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4780cca90 con 0x7fe4780b2b90 2024-09-10T21:42:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.320+0000 7fe46ea00640 1 -- 172.21.15.7:0/4049770036 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe468005380 con 0x7fe4780b2b90 2024-09-10T21:42:16.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.320+0000 7fe46ea00640 1 -- 172.21.15.7:0/4049770036 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe46800a8a0 con 0x7fe4780b2b90 2024-09-10T21:42:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.319+0000 7fa59be00640 1 -- 172.21.15.7:0/1348337140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5940e3360 con 0x7fa5940e28c0 2024-09-10T21:42:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.319+0000 7f85ca000640 1 -- 172.21.15.7:0/375815459 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc003da0 con 0x7f85d4157c70 2024-09-10T21:42:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.319+0000 7f85ca000640 1 -- 172.21.15.7:0/375815459 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85cc00ce40 con 0x7f85d4157c70 2024-09-10T21:42:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.319+0000 7f85ca000640 1 -- 172.21.15.7:0/375815459 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc003da0 con 0x7f85d4157c70 2024-09-10T21:42:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.319+0000 7f85d9200640 1 -- 172.21.15.7:0/375815459 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d415f0a0 con 0x7f85d4157c70 2024-09-10T21:42:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.320+0000 7f85d9200640 1 -- 172.21.15.7:0/375815459 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d415f660 con 0x7f85d4157c70 2024-09-10T21:42:16.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.319+0000 7f2f03600640 1 -- 172.21.15.7:0/1674946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2efc0d10d0 msgr2=0x7f2efc0cf850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.319+0000 7f2f03600640 1 --2- 172.21.15.7:0/1674946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2efc0d10d0 0x7f2efc0cf850 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2ee8009820 tx=0x7f2ee802a2e0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.319+0000 7f2f03600640 1 -- 172.21.15.7:0/1674946 shutdown_connections 2024-09-10T21:42:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.319+0000 7f2f03600640 1 --2- 172.21.15.7:0/1674946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2efc0d10d0 0x7f2efc0cf850 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.319+0000 7f2f03600640 1 --2- 172.21.15.7:0/1674946 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2efc15e120 0x7f2efc0d4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.319+0000 7f2f03600640 1 -- 172.21.15.7:0/1674946 >> 172.21.15.7:0/1674946 conn(0x7f2efc0d3f10 msgr2=0x7f2efc0d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.320+0000 7fe47da00640 1 -- 172.21.15.7:0/4049770036 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4780cd050 con 0x7fe4780b2b90 2024-09-10T21:42:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.320+0000 7f2f03600640 1 -- 172.21.15.7:0/1674946 shutdown_connections 2024-09-10T21:42:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.321+0000 7fe47da00640 1 -- 172.21.15.7:0/4049770036 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe478074960 con 0x7fe4780b2b90 2024-09-10T21:42:16.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.319+0000 7fa59be00640 1 -- 172.21.15.7:0/1348337140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5940e3c00 con 0x7fa5940e28c0 2024-09-10T21:42:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.320+0000 7fa59be00640 1 -- 172.21.15.7:0/1348337140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa59403a370 con 0x7fa5940e28c0 2024-09-10T21:42:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.321+0000 7f85d9200640 1 -- 172.21.15.7:0/375815459 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f85d40d30e0 con 0x7f85d4157c70 2024-09-10T21:42:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.320+0000 7f2f03600640 1 -- 172.21.15.7:0/1674946 wait complete. 2024-09-10T21:42:16.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.323+0000 7f0832000640 1 -- 172.21.15.7:0/142808410 <== osd.0 v2:172.21.15.7:6808/651402908 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0834012e50 con 0x7f081c080290 2024-09-10T21:42:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.322+0000 7fa592a00640 1 -- 172.21.15.7:0/1348337140 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa59403a370 con 0x7fa5940e28c0 2024-09-10T21:42:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.322+0000 7fa592a00640 1 -- 172.21.15.7:0/1348337140 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c049060 con 0x7fa5940e28c0 2024-09-10T21:42:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.324+0000 7f85ca000640 1 -- 172.21.15.7:0/375815459 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f85cc005070 con 0x7f85d4157c70 2024-09-10T21:42:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.323+0000 7f2f03600640 1 Processor -- start 2024-09-10T21:42:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.323+0000 7f2f03600640 1 -- start start 2024-09-10T21:42:16.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.324+0000 7f2f03600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2efc0d10d0 0x7f2efc1f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.322+0000 7f99d5600640 1 -- 172.21.15.7:0/3843560246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99b4004800 msgr2=0x7f99b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.322+0000 7f99d5600640 1 --2- 172.21.15.7:0/3843560246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99b4004800 0x7f99b4004c80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00bdf0 tx=0x7f99b4005830 comp rx=0 tx=0).stop 2024-09-10T21:42:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.322+0000 7f99cf400640 1 -- 172.21.15.7:0/3843560246 reap_dead start 2024-09-10T21:42:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.322+0000 7f99d5600640 1 -- 172.21.15.7:0/3843560246 shutdown_connections 2024-09-10T21:42:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.322+0000 7f99d5600640 1 -- 172.21.15.7:0/3843560246 >> 172.21.15.7:0/3843560246 conn(0x7f99d00b1660 msgr2=0x7f99d00b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.323+0000 7f99d5600640 1 -- 172.21.15.7:0/3843560246 shutdown_connections 2024-09-10T21:42:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.323+0000 7f99d5600640 1 -- 172.21.15.7:0/3843560246 wait complete. 2024-09-10T21:42:16.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.324+0000 7f579f000640 1 -- 172.21.15.7:0/3728092987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5798161220 msgr2=0x7f5798161b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.324+0000 7f579f000640 1 --2- 172.21.15.7:0/3728092987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5798161220 0x7f5798161b00 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f578c009830 tx=0x7f578c02a520 comp rx=0 tx=0).stop 2024-09-10T21:42:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.324+0000 7f579f000640 1 -- 172.21.15.7:0/3728092987 shutdown_connections 2024-09-10T21:42:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.324+0000 7f579f000640 1 --2- 172.21.15.7:0/3728092987 >> v2:172.21.15.133:3300/0 conn(0x7f57980d4920 0x7f57980b54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.324+0000 7f579f000640 1 --2- 172.21.15.7:0/3728092987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5798161220 0x7f5798161b00 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.324+0000 7f579f000640 1 -- 172.21.15.7:0/3728092987 >> 172.21.15.7:0/3728092987 conn(0x7f57980d3e00 msgr2=0x7f57980d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.325+0000 7f2f03600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2efc0d1f90 0x7f2efc0c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.325+0000 7f579f000640 1 -- 172.21.15.7:0/3728092987 shutdown_connections 2024-09-10T21:42:16.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.325+0000 7f2f03600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2efc15e120 0x7f2efc1f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.325+0000 7f579f000640 1 -- 172.21.15.7:0/3728092987 wait complete. 2024-09-10T21:42:16.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.324+0000 7fe46ea00640 1 -- 172.21.15.7:0/4049770036 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe468038ac0 con 0x7fe4780b2b90 2024-09-10T21:42:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.325+0000 7f99d5600640 1 Processor -- start 2024-09-10T21:42:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.325+0000 7f2f01600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2efc15e120 0x7f2efc1f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.325+0000 7f2f01600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2efc15e120 0x7f2efc1f97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54150/0 (socket says 172.21.15.7:54150) 2024-09-10T21:42:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.326+0000 7f2efbe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2efc0d1f90 0x7f2efc0c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.326+0000 7f99d5600640 1 -- start start 2024-09-10T21:42:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.326+0000 7f2f01600640 1 -- 172.21.15.7:0/3293197901 learned_addr learned my addr 172.21.15.7:0/3293197901 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.326+0000 7f2f00c00640 1 --2- 172.21.15.7:0/3293197901 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2efc0d10d0 0x7f2efc1f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.326+0000 7f85ca000640 1 --2- 172.21.15.7:0/375815459 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f85b4078170 0x7f85b407a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.326+0000 7f85ca000640 1 -- 172.21.15.7:0/375815459 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f85cc099f40 con 0x7f85d4157c70 2024-09-10T21:42:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.326+0000 7f85ca000640 1 --2- 172.21.15.7:0/375815459 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x7f85b4080390 0x7f85b40827b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.326+0000 7f85ca000640 1 -- 172.21.15.7:0/375815459 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f85cc014e50 con 0x7f85b4080390 2024-09-10T21:42:16.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.326+0000 7f85ca000640 1 -- 172.21.15.7:0/375815459 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f85cc09a330 con 0x7f85d4157c70 2024-09-10T21:42:16.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.326+0000 7f2f03600640 1 -- 172.21.15.7:0/3293197901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc0b7800 con 0x7f2efc15e120 2024-09-10T21:42:16.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.325+0000 7fa592a00640 1 -- 172.21.15.7:0/1348337140 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa58c004ba0 con 0x7fa5940e28c0 2024-09-10T21:42:16.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.326+0000 7f2f03600640 1 -- 172.21.15.7:0/3293197901 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc0b7680 con 0x7f2efc0d1f90 2024-09-10T21:42:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.327+0000 7f85d1400640 1 --2- 172.21.15.7:0/375815459 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x7f85b4080390 0x7f85b40827b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.327+0000 7f85d1e00640 1 --2- 172.21.15.7:0/375815459 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f85b4078170 0x7f85b407a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.327+0000 7f2f03600640 1 -- 172.21.15.7:0/3293197901 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f2efc0d2440 con 0x7f2efc0d10d0 2024-09-10T21:42:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.327+0000 7ff623e00640 1 -- 172.21.15.7:0/2699159114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62406e4a0 msgr2=0x7ff62406e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.327+0000 7ff623e00640 1 --2- 172.21.15.7:0/2699159114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62406e4a0 0x7ff62406e8f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ff624051fa0 tx=0x7ff61c02a820 comp rx=0 tx=0).stop 2024-09-10T21:42:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.326+0000 7fe46ea00640 1 --2- 172.21.15.7:0/4049770036 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe458077f70 0x7fe45807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.327+0000 7fe46ea00640 1 -- 172.21.15.7:0/4049770036 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe468099600 con 0x7fe4780b2b90 2024-09-10T21:42:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.327+0000 7fe46ea00640 1 --2- 172.21.15.7:0/4049770036 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x7fe4580801f0 0x7fe458082610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.327+0000 7fe46ea00640 1 -- 172.21.15.7:0/4049770036 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe468016ec0 con 0x7fe4580801f0 2024-09-10T21:42:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.327+0000 7fe46ea00640 1 -- 172.21.15.7:0/4049770036 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fe4680999f0 con 0x7fe4780b2b90 2024-09-10T21:42:16.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.327+0000 7fa592a00640 1 --2- 172.21.15.7:0/1348337140 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa57c077fa0 0x7fa57c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.328+0000 7fe476000640 1 --2- 172.21.15.7:0/4049770036 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe458077f70 0x7fe45807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.328+0000 7f579f000640 1 Processor -- start 2024-09-10T21:42:16.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.328+0000 7fa592a00640 1 -- 172.21.15.7:0/1348337140 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa58c0bb2d0 con 0x7fa5940e28c0 2024-09-10T21:42:16.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.328+0000 7ff623e00640 1 -- 172.21.15.7:0/2699159114 shutdown_connections 2024-09-10T21:42:16.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.328+0000 7ff623e00640 1 --2- 172.21.15.7:0/2699159114 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff62406e4a0 0x7ff62406e8f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.328+0000 7ff623e00640 1 --2- 172.21.15.7:0/2699159114 >> v2:172.21.15.133:3300/0 conn(0x7ff624070530 0x7ff62415a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.328+0000 7ff623e00640 1 -- 172.21.15.7:0/2699159114 >> 172.21.15.7:0/2699159114 conn(0x7ff6240b16d0 msgr2=0x7ff6240b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.328+0000 7fe477400640 1 --2- 172.21.15.7:0/4049770036 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x7fe4580801f0 0x7fe458082610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.328+0000 7fa59aa00640 1 --2- 172.21.15.7:0/1348337140 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa57c077fa0 0x7fa57c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.328+0000 7f2f01600640 1 -- 172.21.15.7:0/3293197901 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2efc0d10d0 msgr2=0x7f2efc1f8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.328+0000 7f85d1400640 1 --2- 172.21.15.7:0/375815459 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x7f85b4080390 0x7f85b40827b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.328+0000 7f2f01600640 1 --2- 172.21.15.7:0/3293197901 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2efc0d10d0 0x7f2efc1f8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.328+0000 7f2f01600640 1 -- 172.21.15.7:0/3293197901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2efc0d1f90 msgr2=0x7f2efc0c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.328+0000 7f2f01600640 1 --2- 172.21.15.7:0/3293197901 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2efc0d1f90 0x7f2efc0c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.328+0000 7f2f01600640 1 -- 172.21.15.7:0/3293197901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2efc0c7c00 con 0x7f2efc15e120 2024-09-10T21:42:16.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.328+0000 7ff623e00640 1 -- 172.21.15.7:0/2699159114 shutdown_connections 2024-09-10T21:42:16.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.328+0000 7f99d5600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99b4004800 0x7f99d01fc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.329+0000 7f2f01600640 1 --2- 172.21.15.7:0/3293197901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2efc15e120 0x7f2efc1f97e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400caa0 tx=0x7f2ef400cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.329+0000 7f99d5600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99d01fc7f0 0x7f99d00c3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.329+0000 7fe476000640 1 --2- 172.21.15.7:0/4049770036 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe458077f70 0x7fe45807a430 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe464007fd0 tx=0x7fe464009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.329+0000 7fa592a00640 1 --2- 172.21.15.7:0/1348337140 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x7fa57c0801c0 0x7fa57c0825e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.329+0000 7f99ce000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99d01fc7f0 0x7f99d00c3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.329+0000 7fa592a00640 1 -- 172.21.15.7:0/1348337140 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa57c082c90 con 0x7fa57c0801c0 2024-09-10T21:42:16.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.330+0000 7f85d1e00640 1 --2- 172.21.15.7:0/375815459 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f85b4078170 0x7f85b407a630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f85c0034000 tx=0x7f85c0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.328+0000 7f579f000640 1 -- start start 2024-09-10T21:42:16.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.330+0000 7fa592a00640 1 -- 172.21.15.7:0/1348337140 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7fa58c0bb6c0 con 0x7fa5940e28c0 2024-09-10T21:42:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.329+0000 7f579f000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57980d4920 0x7f57980ca430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.329+0000 7f579f000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5798161220 0x7f57981fc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.329+0000 7f579f000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5798161600 0x7f57981fc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.329+0000 7f579f000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f57980b5d90 con 0x7f5798161220 2024-09-10T21:42:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.329+0000 7f579f000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f57980b5c10 con 0x7f57980d4920 2024-09-10T21:42:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.329+0000 7f579f000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f57980b5f10 con 0x7f5798161600 2024-09-10T21:42:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.330+0000 7fa59aa00640 1 --2- 172.21.15.7:0/1348337140 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa57c077fa0 0x7fa57c07a460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa588005fd0 tx=0x7fa58800a520 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.330+0000 7f85ca000640 1 -- 172.21.15.7:0/375815459 <== osd.3 v2:172.21.15.7:6800/2195665006 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f85cc014e50 con 0x7f85b4080390 2024-09-10T21:42:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.330+0000 7fa59a000640 1 --2- 172.21.15.7:0/1348337140 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x7fa57c0801c0 0x7fa57c0825e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.330+0000 7fe477400640 1 --2- 172.21.15.7:0/4049770036 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x7fe4580801f0 0x7fe458082610 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.329+0000 7f99d5600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99d00c4000 0x7f99d00c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.329+0000 7f99ce000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99d01fc7f0 0x7f99d00c3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:48898/0 (socket says 172.21.15.7:48898) 2024-09-10T21:42:16.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.330+0000 7f99cea00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99b4004800 0x7f99d01fc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.330+0000 7f99cea00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99b4004800 0x7f99d01fc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:50134/0 (socket says 172.21.15.7:50134) 2024-09-10T21:42:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.330+0000 7f579dc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57980d4920 0x7f57980ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.330+0000 7f579e600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5798161600 0x7f57981fc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.330+0000 7f579dc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57980d4920 0x7f57980ca430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:48912/0 (socket says 172.21.15.7:48912) 2024-09-10T21:42:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.330+0000 7f579dc00640 1 -- 172.21.15.7:0/3238756539 learned_addr learned my addr 172.21.15.7:0/3238756539 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:16.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.330+0000 7f579d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5798161220 0x7f57981fc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.331+0000 7f99cf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99d00c4000 0x7f99d00c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.331+0000 7f99cf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99d00c4000 0x7f99d00c4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54170/0 (socket says 172.21.15.7:54170) 2024-09-10T21:42:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.331+0000 7f99ce000640 1 -- 172.21.15.7:0/3839776363 learned_addr learned my addr 172.21.15.7:0/3839776363 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.331+0000 7f579e600640 1 -- 172.21.15.7:0/3238756539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57980d4920 msgr2=0x7f57980ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.331+0000 7f579e600640 1 --2- 172.21.15.7:0/3238756539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57980d4920 0x7f57980ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.331+0000 7f579e600640 1 -- 172.21.15.7:0/3238756539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5798161220 msgr2=0x7f57981fc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.331+0000 7f579e600640 1 --2- 172.21.15.7:0/3238756539 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5798161220 0x7f57981fc060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.331+0000 7f579e600640 1 -- 172.21.15.7:0/3238756539 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57981604f0 con 0x7f5798161600 2024-09-10T21:42:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.331+0000 7f579dc00640 1 --2- 172.21.15.7:0/3238756539 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f57980d4920 0x7f57980ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.331+0000 7fa59a000640 1 --2- 172.21.15.7:0/1348337140 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x7fa57c0801c0 0x7fa57c0825e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.331+0000 7f99d5600640 1 -- 172.21.15.7:0/3839776363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f99d0157db0 con 0x7f99d00c4000 2024-09-10T21:42:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.331+0000 7f99d5600640 1 -- 172.21.15.7:0/3839776363 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f99d0157c30 con 0x7f99d01fc7f0 2024-09-10T21:42:16.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.331+0000 7f579e600640 1 --2- 172.21.15.7:0/3238756539 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5798161600 0x7f57981fc5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f579000ca30 tx=0x7f579000cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.332+0000 7f99d5600640 1 -- 172.21.15.7:0/3839776363 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f99d00cc980 con 0x7f99b4004800 2024-09-10T21:42:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.332+0000 7f2ef8c00640 1 -- 172.21.15.7:0/3293197901 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4017070 con 0x7f2efc15e120 2024-09-10T21:42:16.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.329+0000 7ff623e00640 1 -- 172.21.15.7:0/2699159114 wait complete. 2024-09-10T21:42:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.332+0000 7f5795600640 1 -- 172.21.15.7:0/3238756539 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790007d90 con 0x7f5798161600 2024-09-10T21:42:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.331+0000 7ff623e00640 1 Processor -- start 2024-09-10T21:42:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.332+0000 7f2ef8c00640 1 -- 172.21.15.7:0/3293197901 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ef4005070 con 0x7f2efc15e120 2024-09-10T21:42:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.332+0000 7f2ef8c00640 1 -- 172.21.15.7:0/3293197901 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef40054c0 con 0x7f2efc15e120 2024-09-10T21:42:16.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.332+0000 7f5795600640 1 -- 172.21.15.7:0/3238756539 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5790004c10 con 0x7f5798161600 2024-09-10T21:42:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.332+0000 7f5795600640 1 -- 172.21.15.7:0/3238756539 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790005020 con 0x7f5798161600 2024-09-10T21:42:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.332+0000 7ff623e00640 1 -- start start 2024-09-10T21:42:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.332+0000 7f2f03600640 1 -- 172.21.15.7:0/3293197901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2efc1f9d20 con 0x7f2efc15e120 2024-09-10T21:42:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.332+0000 7f2f03600640 1 -- 172.21.15.7:0/3293197901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2efc0c8160 con 0x7f2efc15e120 2024-09-10T21:42:16.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.332+0000 7f579f000640 1 -- 172.21.15.7:0/3238756539 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57981607d0 con 0x7f5798161600 2024-09-10T21:42:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.333+0000 7f579f000640 1 -- 172.21.15.7:0/3238756539 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f57980ca970 con 0x7f5798161600 2024-09-10T21:42:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.333+0000 7f99cf400640 1 -- 172.21.15.7:0/3839776363 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99b4004800 msgr2=0x7f99d01fc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.333+0000 7f99cf400640 1 --2- 172.21.15.7:0/3839776363 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f99b4004800 0x7f99d01fc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.333+0000 7f99cf400640 1 -- 172.21.15.7:0/3839776363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99d01fc7f0 msgr2=0x7f99d00c3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.333+0000 7f99cf400640 1 --2- 172.21.15.7:0/3839776363 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f99d01fc7f0 0x7f99d00c3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.333+0000 7f99cf400640 1 -- 172.21.15.7:0/3839776363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d00c4bd0 con 0x7f99d00c4000 2024-09-10T21:42:16.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.332+0000 7fe46ea00640 1 -- 172.21.15.7:0/4049770036 <== osd.1 v2:172.21.15.7:6824/4175601737 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe468016ec0 con 0x7fe4580801f0 2024-09-10T21:42:16.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.333+0000 7f99cf400640 1 --2- 172.21.15.7:0/3839776363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99d00c4000 0x7f99d00c4460 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f99d00b5c80 tx=0x7f99c800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.334+0000 7fa592a00640 1 -- 172.21.15.7:0/1348337140 <== osd.2 v2:172.21.15.7:6816/4238877366 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa57c082c90 con 0x7fa57c0801c0 2024-09-10T21:42:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.334+0000 7ff623e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff62406e4a0 0x7ff6241e6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.334+0000 7ff623e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff624070530 0x7ff6241505b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.334+0000 7ff623e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6240cf0c0 0x7ff6240b0750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.334+0000 7ff623e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff62406f460 con 0x7ff6240cf0c0 2024-09-10T21:42:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.334+0000 7ff623e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff62406f2e0 con 0x7ff62406e4a0 2024-09-10T21:42:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.334+0000 7ff623e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ff62406f5e0 con 0x7ff624070530 2024-09-10T21:42:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.335+0000 7ff622a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff62406e4a0 0x7ff6241e6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.335+0000 7ff622a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff62406e4a0 0x7ff6241e6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:48928/0 (socket says 172.21.15.7:48928) 2024-09-10T21:42:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.335+0000 7ff622a00640 1 -- 172.21.15.7:0/1407784178 learned_addr learned my addr 172.21.15.7:0/1407784178 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.335+0000 7f2f03600640 1 -- 172.21.15.7:0/3293197901 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2efc0cf850 con 0x7f2efc15e120 2024-09-10T21:42:16.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.335+0000 7ff622a00640 1 -- 172.21.15.7:0/1407784178 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff624070530 msgr2=0x7ff6241505b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:42:16.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.335+0000 7ff622a00640 1 --2- 172.21.15.7:0/1407784178 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff624070530 0x7ff6241505b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.335+0000 7ff622a00640 1 -- 172.21.15.7:0/1407784178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6240cf0c0 msgr2=0x7ff6240b0750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:42:16.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.335+0000 7ff622a00640 1 --2- 172.21.15.7:0/1407784178 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6240cf0c0 0x7ff6240b0750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.335+0000 7ff622a00640 1 -- 172.21.15.7:0/1407784178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6241e6fb0 con 0x7ff62406e4a0 2024-09-10T21:42:16.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.336+0000 7f579f000640 1 -- 172.21.15.7:0/3238756539 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f57980b54c0 con 0x7f5798161600 2024-09-10T21:42:16.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.336+0000 7f99c6a00640 1 -- 172.21.15.7:0/3839776363 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8017070 con 0x7f99d00c4000 2024-09-10T21:42:16.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.336+0000 7f99c6a00640 1 -- 172.21.15.7:0/3839776363 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99c8005e30 con 0x7f99d00c4000 2024-09-10T21:42:16.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.336+0000 7f99c6a00640 1 -- 172.21.15.7:0/3839776363 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c800a7f0 con 0x7f99d00c4000 2024-09-10T21:42:16.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.336+0000 7ff622a00640 1 --2- 172.21.15.7:0/1407784178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff62406e4a0 0x7ff6241e6a70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff61400fb00 tx=0x7ff614009e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.337+0000 7f5795600640 1 -- 172.21.15.7:0/3238756539 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f57900051c0 con 0x7f5798161600 2024-09-10T21:42:16.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.337+0000 7ff61aa00640 1 -- 172.21.15.7:0/1407784178 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff61400dc90 con 0x7ff62406e4a0 2024-09-10T21:42:16.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.338+0000 7ff61aa00640 1 -- 172.21.15.7:0/1407784178 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff614005020 con 0x7ff62406e4a0 2024-09-10T21:42:16.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.338+0000 7ff61aa00640 1 -- 172.21.15.7:0/1407784178 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff614005410 con 0x7ff62406e4a0 2024-09-10T21:42:16.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.337+0000 7f2ef8c00640 1 -- 172.21.15.7:0/3293197901 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2ef4005660 con 0x7f2efc15e120 2024-09-10T21:42:16.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.338+0000 7f2ef8c00640 1 --2- 172.21.15.7:0/3293197901 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2edc077fa0 0x7f2edc07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.336+0000 7f99d5600640 1 -- 172.21.15.7:0/3839776363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d00c4eb0 con 0x7f99d00c4000 2024-09-10T21:42:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.336+0000 7f99d5600640 1 -- 172.21.15.7:0/3839776363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d01fa200 con 0x7f99d00c4000 2024-09-10T21:42:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.338+0000 7f5795600640 1 --2- 172.21.15.7:0/3238756539 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5778077fc0 0x7f577807a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.339+0000 7f5795600640 1 -- 172.21.15.7:0/3238756539 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f579005eeb0 con 0x7f5798161600 2024-09-10T21:42:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.339+0000 7f2ef8c00640 1 -- 172.21.15.7:0/3293197901 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2ef40992a0 con 0x7f2efc15e120 2024-09-10T21:42:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.339+0000 7f579dc00640 1 --2- 172.21.15.7:0/3238756539 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5778077fc0 0x7f577807a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.339+0000 7ff623e00640 1 -- 172.21.15.7:0/1407784178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff624201280 con 0x7ff62406e4a0 2024-09-10T21:42:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.339+0000 7ff623e00640 1 -- 172.21.15.7:0/1407784178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff624201510 con 0x7ff62406e4a0 2024-09-10T21:42:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.340+0000 7f5795600640 1 --2- 172.21.15.7:0/3238756539 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x7f5778080df0 0x7f5778083290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.340+0000 7f5795600640 1 -- 172.21.15.7:0/3238756539 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5778083960 con 0x7f5778080df0 2024-09-10T21:42:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.340+0000 7f5795600640 1 -- 172.21.15.7:0/3238756539 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f579005f430 con 0x7f5798161600 2024-09-10T21:42:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.340+0000 7f579d200640 1 --2- 172.21.15.7:0/3238756539 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x7f5778080df0 0x7f5778083290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.340+0000 7f2ef8c00640 1 --2- 172.21.15.7:0/3293197901 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x7f2edc0801c0 0x7f2edc0825e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.340+0000 7f2ef8c00640 1 -- 172.21.15.7:0/3293197901 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2ef4013e50 con 0x7f2edc0801c0 2024-09-10T21:42:16.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.340+0000 7f2ef8c00640 1 -- 172.21.15.7:0/3293197901 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f2ef40a0050 con 0x7f2efc15e120 2024-09-10T21:42:16.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.341+0000 7ff623e00640 1 -- 172.21.15.7:0/1407784178 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff624201eb0 con 0x7ff62406e4a0 2024-09-10T21:42:16.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.341+0000 7f579dc00640 1 --2- 172.21.15.7:0/3238756539 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5778077fc0 0x7f577807a480 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f578c009830 tx=0x7f578c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.341+0000 7f579d200640 1 --2- 172.21.15.7:0/3238756539 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x7f5778080df0 0x7f5778083290 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.341+0000 7f99d5600640 1 -- 172.21.15.7:0/3839776363 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f99a8001030 con 0x7f99d00c4000 2024-09-10T21:42:16.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.341+0000 7f2efbe00640 1 --2- 172.21.15.7:0/3293197901 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x7f2edc0801c0 0x7f2edc0825e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.342+0000 7f2f00c00640 1 --2- 172.21.15.7:0/3293197901 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2edc077fa0 0x7f2edc07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.342+0000 7f2efbe00640 1 --2- 172.21.15.7:0/3293197901 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x7f2edc0801c0 0x7f2edc0825e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.342+0000 7f2f00c00640 1 --2- 172.21.15.7:0/3293197901 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2edc077fa0 0x7f2edc07a460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0b7eb0 tx=0x7f2ef0009260 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.342+0000 7f5795600640 1 -- 172.21.15.7:0/3238756539 <== osd.7 v2:172.21.15.133:6802/2638399040 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5778083960 con 0x7f5778080df0 2024-09-10T21:42:16.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.343+0000 7f99c6a00640 1 -- 172.21.15.7:0/3839776363 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f99c800a990 con 0x7f99d00c4000 2024-09-10T21:42:16.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.343+0000 7f2ef8c00640 1 -- 172.21.15.7:0/3293197901 <== osd.4 v2:172.21.15.133:6826/1857710445 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2ef4013e50 con 0x7f2edc0801c0 2024-09-10T21:42:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.344+0000 7f99c6a00640 1 --2- 172.21.15.7:0/3839776363 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f99b0078070 0x7f99b007a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.345+0000 7f99c6a00640 1 -- 172.21.15.7:0/3839776363 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f99c80992a0 con 0x7f99d00c4000 2024-09-10T21:42:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.345+0000 7f99c6a00640 1 --2- 172.21.15.7:0/3839776363 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x7f99b0080210 0x7f99b0082630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.345+0000 7f99c6a00640 1 -- 172.21.15.7:0/3839776363 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f99c801de80 con 0x7f99b0080210 2024-09-10T21:42:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.345+0000 7f99c6a00640 1 -- 172.21.15.7:0/3839776363 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7f99c80a0050 con 0x7f99d00c4000 2024-09-10T21:42:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.346+0000 7f99ce000640 1 --2- 172.21.15.7:0/3839776363 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x7f99b0080210 0x7f99b0082630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.346+0000 7f99cea00640 1 --2- 172.21.15.7:0/3839776363 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f99b0078070 0x7f99b007a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.346+0000 7f99ce000640 1 --2- 172.21.15.7:0/3839776363 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x7f99b0080210 0x7f99b0082630 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.347+0000 7f99cea00640 1 --2- 172.21.15.7:0/3839776363 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f99b0078070 0x7f99b007a530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f99c00058c0 tx=0x7f99c0005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.347+0000 7f99c6a00640 1 -- 172.21.15.7:0/3839776363 <== osd.5 v2:172.21.15.133:6818/3838498382 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f99c801de80 con 0x7f99b0080210 2024-09-10T21:42:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.346+0000 7ff61aa00640 1 -- 172.21.15.7:0/1407784178 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff6140047a0 con 0x7ff62406e4a0 2024-09-10T21:42:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.347+0000 7ff61aa00640 1 --2- 172.21.15.7:0/1407784178 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff604078040 0x7ff60407a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.347+0000 7ff61aa00640 1 -- 172.21.15.7:0/1407784178 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7ff61409ae30 con 0x7ff62406e4a0 2024-09-10T21:42:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.348+0000 7ff622000640 1 --2- 172.21.15.7:0/1407784178 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff604078040 0x7ff60407a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.348+0000 7ff61aa00640 1 --2- 172.21.15.7:0/1407784178 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x7ff604080260 0x7ff604082680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.348+0000 7ff61aa00640 1 -- 172.21.15.7:0/1407784178 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff61401aec0 con 0x7ff604080260 2024-09-10T21:42:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.348+0000 7ff61aa00640 1 -- 172.21.15.7:0/1407784178 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) ==== 24+0+0 (secure 0 0 0) 0x7ff61409b220 con 0x7ff62406e4a0 2024-09-10T21:42:16.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.348+0000 7ff623400640 1 --2- 172.21.15.7:0/1407784178 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x7ff604080260 0x7ff604082680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.349+0000 7ff622000640 1 --2- 172.21.15.7:0/1407784178 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff604078040 0x7ff60407a500 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff624202470 tx=0x7ff610007990 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.349+0000 7ff623400640 1 --2- 172.21.15.7:0/1407784178 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x7ff604080260 0x7ff604082680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.350+0000 7ff61aa00640 1 -- 172.21.15.7:0/1407784178 <== osd.6 v2:172.21.15.133:6810/3365981160 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff61401aec0 con 0x7ff604080260 2024-09-10T21:42:16.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.353+0000 7f0840800640 1 -- 172.21.15.7:0/142808410 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f083806fd30 con 0x7f081c080290 2024-09-10T21:42:16.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.354+0000 7f0832000640 1 -- 172.21.15.7:0/142808410 <== osd.0 v2:172.21.15.7:6808/651402908 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f083806fd30 con 0x7f081c080290 2024-09-10T21:42:16.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.354+0000 7f0840800640 1 -- 172.21.15.7:0/142808410 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x7f081c080290 msgr2=0x7f081c0826b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.354+0000 7f0840800640 1 --2- 172.21.15.7:0/142808410 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x7f081c080290 0x7f081c0826b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.355+0000 7f0840800640 1 -- 172.21.15.7:0/142808410 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f081c078070 msgr2=0x7f081c07a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.355+0000 7f0840800640 1 --2- 172.21.15.7:0/142808410 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f081c078070 0x7f081c07a530 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f082c007fd0 tx=0x7f082c001410 comp rx=0 tx=0).stop 2024-09-10T21:42:16.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.355+0000 7f0840800640 1 -- 172.21.15.7:0/142808410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08380cece0 msgr2=0x7f08380c8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.355+0000 7f0840800640 1 --2- 172.21.15.7:0/142808410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08380cece0 0x7f08380c8ca0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f083400b990 tx=0x7f083400be50 comp rx=0 tx=0).stop 2024-09-10T21:42:16.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.355+0000 7f083f200640 1 -- 172.21.15.7:0/142808410 reap_dead start 2024-09-10T21:42:16.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.355+0000 7f0840800640 1 -- 172.21.15.7:0/142808410 shutdown_connections 2024-09-10T21:42:16.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.355+0000 7f0840800640 1 -- 172.21.15.7:0/142808410 >> 172.21.15.7:0/142808410 conn(0x7f08380b1740 msgr2=0x7f08380b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.355+0000 7f0840800640 1 -- 172.21.15.7:0/142808410 shutdown_connections 2024-09-10T21:42:16.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.355+0000 7f0840800640 1 -- 172.21.15.7:0/142808410 wait complete. 2024-09-10T21:42:16.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.359+0000 7f85d9200640 1 -- 172.21.15.7:0/375815459 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f85d40d4a30 con 0x7f85b4080390 2024-09-10T21:42:16.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.360+0000 7f85ca000640 1 -- 172.21.15.7:0/375815459 <== osd.3 v2:172.21.15.7:6800/2195665006 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f85d40d4a30 con 0x7f85b4080390 2024-09-10T21:42:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.363+0000 7fe47da00640 1 -- 172.21.15.7:0/4049770036 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe478157d90 con 0x7fe4580801f0 2024-09-10T21:42:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.363+0000 7f85d9200640 1 -- 172.21.15.7:0/375815459 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x7f85b4080390 msgr2=0x7f85b40827b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.363+0000 7f85d9200640 1 --2- 172.21.15.7:0/375815459 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x7f85b4080390 0x7f85b40827b0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.364+0000 7f85d9200640 1 -- 172.21.15.7:0/375815459 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f85b4078170 msgr2=0x7f85b407a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.364+0000 7f85d9200640 1 --2- 172.21.15.7:0/375815459 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f85b4078170 0x7f85b407a630 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f85c0034000 tx=0x7f85c0004730 comp rx=0 tx=0).stop 2024-09-10T21:42:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.364+0000 7f85d9200640 1 -- 172.21.15.7:0/375815459 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85d4157c70 msgr2=0x7f85d40c77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.364+0000 7f85d9200640 1 --2- 172.21.15.7:0/375815459 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f85d4157c70 0x7f85d40c77a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00c730 tx=0x7f85cc00cbf0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.364+0000 7f85d2800640 1 -- 172.21.15.7:0/375815459 reap_dead start 2024-09-10T21:42:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.364+0000 7f85d9200640 1 -- 172.21.15.7:0/375815459 shutdown_connections 2024-09-10T21:42:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.364+0000 7f85d9200640 1 -- 172.21.15.7:0/375815459 >> 172.21.15.7:0/375815459 conn(0x7f85d40d3f10 msgr2=0x7f85d40b7b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.365+0000 7f85d9200640 1 -- 172.21.15.7:0/375815459 shutdown_connections 2024-09-10T21:42:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.365+0000 7f85d9200640 1 -- 172.21.15.7:0/375815459 wait complete. 2024-09-10T21:42:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.365+0000 7fe46ea00640 1 -- 172.21.15.7:0/4049770036 <== osd.1 v2:172.21.15.7:6824/4175601737 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe478157d90 con 0x7fe4580801f0 2024-09-10T21:42:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.366+0000 7fe47da00640 1 -- 172.21.15.7:0/4049770036 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x7fe4580801f0 msgr2=0x7fe458082610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.366+0000 7fe47da00640 1 --2- 172.21.15.7:0/4049770036 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x7fe4580801f0 0x7fe458082610 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.366+0000 7fe47da00640 1 -- 172.21.15.7:0/4049770036 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe458077f70 msgr2=0x7fe45807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.366+0000 7fe47da00640 1 --2- 172.21.15.7:0/4049770036 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe458077f70 0x7fe45807a430 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fe464007fd0 tx=0x7fe464009040 comp rx=0 tx=0).stop 2024-09-10T21:42:16.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.366+0000 7fe47da00640 1 -- 172.21.15.7:0/4049770036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4780b2b90 msgr2=0x7fe4780cf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.366+0000 7fe47da00640 1 --2- 172.21.15.7:0/4049770036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4780b2b90 0x7fe4780cf050 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe46800c700 tx=0x7fe46800cbc0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.367+0000 7fe477400640 1 -- 172.21.15.7:0/4049770036 reap_dead start 2024-09-10T21:42:16.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.367+0000 7fe47da00640 1 -- 172.21.15.7:0/4049770036 shutdown_connections 2024-09-10T21:42:16.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.367+0000 7fe47da00640 1 -- 172.21.15.7:0/4049770036 >> 172.21.15.7:0/4049770036 conn(0x7fe4780d1d60 msgr2=0x7fe4781582b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.367+0000 7fe47da00640 1 -- 172.21.15.7:0/4049770036 shutdown_connections 2024-09-10T21:42:16.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.367+0000 7fe47da00640 1 -- 172.21.15.7:0/4049770036 wait complete. 2024-09-10T21:42:16.378 INFO:teuthology.orchestra.run.smithi007.stdout:64424509446 2024-09-10T21:42:16.378 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T21:42:16.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.377+0000 7f579f000640 1 -- 172.21.15.7:0/3238756539 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5798069680 con 0x7f5778080df0 2024-09-10T21:42:16.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.377+0000 7f99d5600640 1 -- 172.21.15.7:0/3839776363 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f99a8002dd0 con 0x7f99b0080210 2024-09-10T21:42:16.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.378+0000 7f5795600640 1 -- 172.21.15.7:0/3238756539 <== osd.7 v2:172.21.15.133:6802/2638399040 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5798069680 con 0x7f5778080df0 2024-09-10T21:42:16.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.378+0000 7f99c6a00640 1 -- 172.21.15.7:0/3839776363 <== osd.5 v2:172.21.15.133:6818/3838498382 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f99a8002dd0 con 0x7f99b0080210 2024-09-10T21:42:16.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.378+0000 7f579f000640 1 -- 172.21.15.7:0/3238756539 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x7f5778080df0 msgr2=0x7f5778083290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.378+0000 7f579f000640 1 --2- 172.21.15.7:0/3238756539 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x7f5778080df0 0x7f5778083290 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.384+0000 7fa59be00640 1 -- 172.21.15.7:0/1348337140 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa594005e30 con 0x7fa57c0801c0 2024-09-10T21:42:16.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.384+0000 7fa592a00640 1 -- 172.21.15.7:0/1348337140 <== osd.2 v2:172.21.15.7:6816/4238877366 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa594005e30 con 0x7fa57c0801c0 2024-09-10T21:42:16.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.381+0000 7f2f03600640 1 -- 172.21.15.7:0/3293197901 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2efc0b6ff0 con 0x7f2edc0801c0 2024-09-10T21:42:16.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.381+0000 7f2ef8c00640 1 -- 172.21.15.7:0/3293197901 <== osd.4 v2:172.21.15.133:6826/1857710445 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2efc0b6ff0 con 0x7f2edc0801c0 2024-09-10T21:42:16.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.382+0000 7f2f03600640 1 -- 172.21.15.7:0/3293197901 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x7f2edc0801c0 msgr2=0x7f2edc0825e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.382+0000 7f2f03600640 1 --2- 172.21.15.7:0/3293197901 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x7f2edc0801c0 0x7f2edc0825e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.382+0000 7f2f03600640 1 -- 172.21.15.7:0/3293197901 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2edc077fa0 msgr2=0x7f2edc07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.382+0000 7f2f03600640 1 --2- 172.21.15.7:0/3293197901 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2edc077fa0 0x7f2edc07a460 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2efc0b7eb0 tx=0x7f2ef0009260 comp rx=0 tx=0).stop 2024-09-10T21:42:16.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.382+0000 7f2f03600640 1 -- 172.21.15.7:0/3293197901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2efc15e120 msgr2=0x7f2efc1f97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.382+0000 7f2f03600640 1 --2- 172.21.15.7:0/3293197901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2efc15e120 0x7f2efc1f97e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400caa0 tx=0x7f2ef400cea0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.382+0000 7f2f01600640 1 -- 172.21.15.7:0/3293197901 reap_dead start 2024-09-10T21:42:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.382+0000 7f2f03600640 1 -- 172.21.15.7:0/3293197901 shutdown_connections 2024-09-10T21:42:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.382+0000 7f2f03600640 1 -- 172.21.15.7:0/3293197901 >> 172.21.15.7:0/3293197901 conn(0x7f2efc0d3f10 msgr2=0x7f2efc160720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.383+0000 7f2f03600640 1 -- 172.21.15.7:0/3293197901 shutdown_connections 2024-09-10T21:42:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.383+0000 7f2f03600640 1 -- 172.21.15.7:0/3293197901 wait complete. 2024-09-10T21:42:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.380+0000 7f99d5600640 1 -- 172.21.15.7:0/3839776363 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x7f99b0080210 msgr2=0x7f99b0082630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.380+0000 7f99d5600640 1 --2- 172.21.15.7:0/3839776363 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x7f99b0080210 0x7f99b0082630 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.380+0000 7f99d5600640 1 -- 172.21.15.7:0/3839776363 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f99b0078070 msgr2=0x7f99b007a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.380+0000 7f99d5600640 1 --2- 172.21.15.7:0/3839776363 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f99b0078070 0x7f99b007a530 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f99c00058c0 tx=0x7f99c0005850 comp rx=0 tx=0).stop 2024-09-10T21:42:16.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.380+0000 7f99d5600640 1 -- 172.21.15.7:0/3839776363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99d00c4000 msgr2=0x7f99d00c4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.380+0000 7f99d5600640 1 --2- 172.21.15.7:0/3839776363 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f99d00c4000 0x7f99d00c4460 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f99d00b5c80 tx=0x7f99c800cb90 comp rx=0 tx=0).stop 2024-09-10T21:42:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.380+0000 7f99cf400640 1 -- 172.21.15.7:0/3839776363 reap_dead start 2024-09-10T21:42:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.381+0000 7f99d5600640 1 -- 172.21.15.7:0/3839776363 shutdown_connections 2024-09-10T21:42:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.381+0000 7f99d5600640 1 -- 172.21.15.7:0/3839776363 >> 172.21.15.7:0/3839776363 conn(0x7f99d00b1660 msgr2=0x7f99d0156f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.382+0000 7f99d5600640 1 -- 172.21.15.7:0/3839776363 shutdown_connections 2024-09-10T21:42:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.382+0000 7f99d5600640 1 -- 172.21.15.7:0/3839776363 wait complete. 2024-09-10T21:42:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.382+0000 7ff623e00640 1 -- 172.21.15.7:0/1407784178 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff6241e7650 con 0x7ff604080260 2024-09-10T21:42:16.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.383+0000 7ff61aa00640 1 -- 172.21.15.7:0/1407784178 <== osd.6 v2:172.21.15.133:6810/3365981160 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff6241e7650 con 0x7ff604080260 2024-09-10T21:42:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.383+0000 7ff623e00640 1 -- 172.21.15.7:0/1407784178 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x7ff604080260 msgr2=0x7ff604082680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.383+0000 7ff623e00640 1 --2- 172.21.15.7:0/1407784178 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x7ff604080260 0x7ff604082680 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.384+0000 7ff623e00640 1 -- 172.21.15.7:0/1407784178 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff604078040 msgr2=0x7ff60407a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.384+0000 7ff623e00640 1 --2- 172.21.15.7:0/1407784178 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff604078040 0x7ff60407a500 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff624202470 tx=0x7ff610007990 comp rx=0 tx=0).stop 2024-09-10T21:42:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.384+0000 7ff623e00640 1 -- 172.21.15.7:0/1407784178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff62406e4a0 msgr2=0x7ff6241e6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.384+0000 7ff623e00640 1 --2- 172.21.15.7:0/1407784178 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff62406e4a0 0x7ff6241e6a70 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff61400fb00 tx=0x7ff614009e70 comp rx=0 tx=0).stop 2024-09-10T21:42:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.384+0000 7ff623400640 1 -- 172.21.15.7:0/1407784178 reap_dead start 2024-09-10T21:42:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.379+0000 7f579f000640 1 -- 172.21.15.7:0/3238756539 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5778077fc0 msgr2=0x7f577807a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.379+0000 7f579f000640 1 --2- 172.21.15.7:0/3238756539 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5778077fc0 0x7f577807a480 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f578c009830 tx=0x7f578c004730 comp rx=0 tx=0).stop 2024-09-10T21:42:16.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.379+0000 7f579f000640 1 -- 172.21.15.7:0/3238756539 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5798161600 msgr2=0x7f57981fc5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.379+0000 7f579f000640 1 --2- 172.21.15.7:0/3238756539 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5798161600 0x7f57981fc5a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f579000ca30 tx=0x7f579000cef0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.379+0000 7f579e600640 1 -- 172.21.15.7:0/3238756539 reap_dead start 2024-09-10T21:42:16.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.380+0000 7f579f000640 1 -- 172.21.15.7:0/3238756539 shutdown_connections 2024-09-10T21:42:16.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.380+0000 7f579f000640 1 -- 172.21.15.7:0/3238756539 >> 172.21.15.7:0/3238756539 conn(0x7f57980d3e00 msgr2=0x7f57980b9030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.380+0000 7f579f000640 1 -- 172.21.15.7:0/3238756539 shutdown_connections 2024-09-10T21:42:16.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.380+0000 7f579f000640 1 -- 172.21.15.7:0/3238756539 wait complete. 2024-09-10T21:42:16.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.385+0000 7ff623e00640 1 -- 172.21.15.7:0/1407784178 shutdown_connections 2024-09-10T21:42:16.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.385+0000 7ff623e00640 1 -- 172.21.15.7:0/1407784178 >> 172.21.15.7:0/1407784178 conn(0x7ff6240b16d0 msgr2=0x7ff6241e9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.386+0000 7ff623e00640 1 -- 172.21.15.7:0/1407784178 shutdown_connections 2024-09-10T21:42:16.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.386+0000 7ff623e00640 1 -- 172.21.15.7:0/1407784178 wait complete. 2024-09-10T21:42:16.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.388+0000 7fa59be00640 1 -- 172.21.15.7:0/1348337140 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x7fa57c0801c0 msgr2=0x7fa57c0825e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.388+0000 7fa59be00640 1 --2- 172.21.15.7:0/1348337140 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x7fa57c0801c0 0x7fa57c0825e0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.388+0000 7fa59be00640 1 -- 172.21.15.7:0/1348337140 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa57c077fa0 msgr2=0x7fa57c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.388+0000 7fa59be00640 1 --2- 172.21.15.7:0/1348337140 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa57c077fa0 0x7fa57c07a460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa588005fd0 tx=0x7fa58800a520 comp rx=0 tx=0).stop 2024-09-10T21:42:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.388+0000 7fa59be00640 1 -- 172.21.15.7:0/1348337140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5940e28c0 msgr2=0x7fa5940d3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.388+0000 7fa59be00640 1 --2- 172.21.15.7:0/1348337140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa5940e28c0 0x7fa5940d3bd0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa58c009230 tx=0x7fa58c003b90 comp rx=0 tx=0).stop 2024-09-10T21:42:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.388+0000 7fa59b400640 1 -- 172.21.15.7:0/1348337140 reap_dead start 2024-09-10T21:42:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.389+0000 7fa59be00640 1 -- 172.21.15.7:0/1348337140 shutdown_connections 2024-09-10T21:42:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.389+0000 7fa59be00640 1 -- 172.21.15.7:0/1348337140 >> 172.21.15.7:0/1348337140 conn(0x7fa594055550 msgr2=0x7fa594004e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.389+0000 7fa59be00640 1 -- 172.21.15.7:0/1348337140 shutdown_connections 2024-09-10T21:42:16.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.389+0000 7fa59be00640 1 -- 172.21.15.7:0/1348337140 wait complete. 2024-09-10T21:42:16.394 INFO:teuthology.orchestra.run.smithi007.stdout:60129542150 2024-09-10T21:42:16.394 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T21:42:16.399 INFO:teuthology.orchestra.run.smithi007.stdout:64424509448 2024-09-10T21:42:16.399 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T21:42:16.407 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-10T21:42:16.407 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T21:42:16.412 INFO:teuthology.orchestra.run.smithi007.stdout:64424509446 2024-09-10T21:42:16.412 INFO:teuthology.orchestra.run.smithi007.stdout:64424509447 2024-09-10T21:42:16.412 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T21:42:16.413 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T21:42:16.416 INFO:teuthology.orchestra.run.smithi007.stdout:64424509446 2024-09-10T21:42:16.416 INFO:teuthology.orchestra.run.smithi007.stdout:64424509447 2024-09-10T21:42:16.416 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T21:42:16.417 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T21:42:16.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.670+0000 7fa20d200640 1 -- 172.21.15.7:0/3558459558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1ec004800 msgr2=0x7fa1ec004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.670+0000 7fa204a00640 1 -- 172.21.15.7:0/3558459558 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f4003c10 con 0x7fa1ec004800 2024-09-10T21:42:16.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.670+0000 7fa20d200640 1 --2- 172.21.15.7:0/3558459558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1ec004800 0x7fa1ec004c80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa2000e2a10 tx=0x7fa1ec005830 comp rx=0 tx=0).stop 2024-09-10T21:42:16.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.671+0000 7fa206800640 1 -- 172.21.15.7:0/3558459558 reap_dead start 2024-09-10T21:42:16.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.671+0000 7fa20d200640 1 -- 172.21.15.7:0/3558459558 shutdown_connections 2024-09-10T21:42:16.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.671+0000 7fa20d200640 1 -- 172.21.15.7:0/3558459558 >> 172.21.15.7:0/3558459558 conn(0x7fa20001d7a0 msgr2=0x7fa200054590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.672+0000 7fa20d200640 1 -- 172.21.15.7:0/3558459558 shutdown_connections 2024-09-10T21:42:16.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.672+0000 7fa20d200640 1 -- 172.21.15.7:0/3558459558 wait complete. 2024-09-10T21:42:16.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.676+0000 7fa20d200640 1 Processor -- start 2024-09-10T21:42:16.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.676+0000 7fa20d200640 1 -- start start 2024-09-10T21:42:16.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.678+0000 7fa20d200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1ec004800 0x7fa200021800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.679+0000 7fa20d200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa2000de480 0x7fa2000313c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.679+0000 7fa20d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2000de900 0x7fa200031900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.679+0000 7fa205400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa2000de480 0x7fa2000313c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.679+0000 7fa205400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa2000de480 0x7fa2000313c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:50164/0 (socket says 172.21.15.7:50164) 2024-09-10T21:42:16.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.679+0000 7fa205e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1ec004800 0x7fa200021800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.680+0000 7fa205e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1ec004800 0x7fa200021800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:48952/0 (socket says 172.21.15.7:48952) 2024-09-10T21:42:16.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.681+0000 7fa205400640 1 -- 172.21.15.7:0/707327189 learned_addr learned my addr 172.21.15.7:0/707327189 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:16.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.681+0000 7fa206800640 1 --2- 172.21.15.7:0/707327189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2000de900 0x7fa200031900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.681+0000 7fa20d200640 1 -- 172.21.15.7:0/707327189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa200054e20 con 0x7fa2000de900 2024-09-10T21:42:16.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.681+0000 7fa20d200640 1 -- 172.21.15.7:0/707327189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa200054ca0 con 0x7fa1ec004800 2024-09-10T21:42:16.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.681+0000 7fa20d200640 1 -- 172.21.15.7:0/707327189 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa2000392e0 con 0x7fa2000de480 2024-09-10T21:42:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.682+0000 7fa205e00640 1 -- 172.21.15.7:0/707327189 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa2000de480 msgr2=0x7fa2000313c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.682+0000 7fa205e00640 1 --2- 172.21.15.7:0/707327189 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa2000de480 0x7fa2000313c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.682+0000 7fa205e00640 1 -- 172.21.15.7:0/707327189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2000de900 msgr2=0x7fa200031900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.683+0000 7fa205e00640 1 --2- 172.21.15.7:0/707327189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2000de900 0x7fa200031900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.683+0000 7fa205e00640 1 -- 172.21.15.7:0/707327189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa200021f30 con 0x7fa1ec004800 2024-09-10T21:42:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.683+0000 7fa206800640 1 --2- 172.21.15.7:0/707327189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2000de900 0x7fa200031900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:42:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.684+0000 7fa205e00640 1 --2- 172.21.15.7:0/707327189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1ec004800 0x7fa200021800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009a40 tx=0x7fa1f8009f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.684+0000 7fa1fe000640 1 -- 172.21.15.7:0/707327189 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8021070 con 0x7fa1ec004800 2024-09-10T21:42:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.684+0000 7fa1fe000640 1 -- 172.21.15.7:0/707327189 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1f8014310 con 0x7fa1ec004800 2024-09-10T21:42:16.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.684+0000 7fa20d200640 1 -- 172.21.15.7:0/707327189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20004c7c0 con 0x7fa1ec004800 2024-09-10T21:42:16.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.684+0000 7fa20d200640 1 -- 172.21.15.7:0/707327189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa20004d2a0 con 0x7fa1ec004800 2024-09-10T21:42:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.684+0000 7fa1fe000640 1 -- 172.21.15.7:0/707327189 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1f8008b40 con 0x7fa1ec004800 2024-09-10T21:42:16.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.686+0000 7fa20d200640 1 -- 172.21.15.7:0/707327189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e0005290 con 0x7fa1ec004800 2024-09-10T21:42:16.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.689+0000 7fa1fe000640 1 -- 172.21.15.7:0/707327189 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa1f801cce0 con 0x7fa1ec004800 2024-09-10T21:42:16.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.691+0000 7fa1fe000640 1 --2- 172.21.15.7:0/707327189 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa1e8077fc0 0x7fa1e807a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.691+0000 7fa1fe000640 1 -- 172.21.15.7:0/707327189 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa1f80a14a0 con 0x7fa1ec004800 2024-09-10T21:42:16.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.691+0000 7fe313800640 1 -- 172.21.15.7:0/2778005350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30c0cc520 msgr2=0x7fe30c15eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.691+0000 7fe313800640 1 --2- 172.21.15.7:0/2778005350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30c0cc520 0x7fe30c15eda0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8009830 tx=0x7fe2f802a540 comp rx=0 tx=0).stop 2024-09-10T21:42:16.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.692+0000 7fe313800640 1 -- 172.21.15.7:0/2778005350 shutdown_connections 2024-09-10T21:42:16.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.692+0000 7fe313800640 1 --2- 172.21.15.7:0/2778005350 >> v2:172.21.15.133:3300/0 conn(0x7fe30c0cc900 0x7fe30c161940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.692+0000 7fe313800640 1 --2- 172.21.15.7:0/2778005350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30c0cc520 0x7fe30c15eda0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.692+0000 7fe313800640 1 -- 172.21.15.7:0/2778005350 >> 172.21.15.7:0/2778005350 conn(0x7fe30c0b1660 msgr2=0x7fe30c0b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.692+0000 7fe313800640 1 -- 172.21.15.7:0/2778005350 shutdown_connections 2024-09-10T21:42:16.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.693+0000 7fe313800640 1 -- 172.21.15.7:0/2778005350 wait complete. 2024-09-10T21:42:16.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.693+0000 7fa205400640 1 --2- 172.21.15.7:0/707327189 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa1e8077fc0 0x7fa1e807a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.696+0000 7fe313800640 1 Processor -- start 2024-09-10T21:42:16.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.696+0000 7fe313800640 1 -- start start 2024-09-10T21:42:16.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.697+0000 7fa205400640 1 --2- 172.21.15.7:0/707327189 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa1e8077fc0 0x7fa1e807a480 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4007b50 tx=0x7fa1f400d040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.697+0000 7fa1fe000640 1 -- 172.21.15.7:0/707327189 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fa1f8069a30 con 0x7fa1ec004800 2024-09-10T21:42:16.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.697+0000 7fe313800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30c0cc520 0x7fe30c0b0960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.697+0000 7fe313800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe30c0cc900 0x7fe30c0b4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.697+0000 7fe313800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe30c161e80 0x7fe30c0c4a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.697+0000 7fe313800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe30c1551b0 con 0x7fe30c161e80 2024-09-10T21:42:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.697+0000 7fe313800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe30c155030 con 0x7fe30c0cc900 2024-09-10T21:42:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.697+0000 7fe313800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe30c072c20 con 0x7fe30c0cc520 2024-09-10T21:42:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.698+0000 7fe310e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30c0cc520 0x7fe30c0b0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.698+0000 7fe30be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe30c0cc900 0x7fe30c0b4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.699+0000 7fe310e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30c0cc520 0x7fe30c0b0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:50186/0 (socket says 172.21.15.7:50186) 2024-09-10T21:42:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.699+0000 7fe310e00640 1 -- 172.21.15.7:0/503837256 learned_addr learned my addr 172.21.15.7:0/503837256 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:16.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.699+0000 7fe311800640 1 --2- 172.21.15.7:0/503837256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe30c161e80 0x7fe30c0c4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.700+0000 7fe310e00640 1 -- 172.21.15.7:0/503837256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe30c0cc900 msgr2=0x7fe30c0b4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.700+0000 7fe310e00640 1 --2- 172.21.15.7:0/503837256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe30c0cc900 0x7fe30c0b4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.700+0000 7fe310e00640 1 -- 172.21.15.7:0/503837256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe30c161e80 msgr2=0x7fe30c0c4a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.700+0000 7fe310e00640 1 --2- 172.21.15.7:0/503837256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe30c161e80 0x7fe30c0c4a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.700+0000 7fe310e00640 1 -- 172.21.15.7:0/503837256 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe30c1f4930 con 0x7fe30c0cc520 2024-09-10T21:42:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.700+0000 7fe311800640 1 --2- 172.21.15.7:0/503837256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe30c161e80 0x7fe30c0c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:42:16.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.701+0000 7fe310e00640 1 --2- 172.21.15.7:0/503837256 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30c0cc520 0x7fe30c0b0960 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8000c00 tx=0x7fe2f8032b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.701+0000 7fe308c00640 1 -- 172.21.15.7:0/503837256 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8040070 con 0x7fe30c0cc520 2024-09-10T21:42:16.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.701+0000 7fe308c00640 1 -- 172.21.15.7:0/503837256 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2f80038e0 con 0x7fe30c0cc520 2024-09-10T21:42:16.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.701+0000 7fe308c00640 1 -- 172.21.15.7:0/503837256 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8003d10 con 0x7fe30c0cc520 2024-09-10T21:42:16.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.702+0000 7f1eab000640 1 -- 172.21.15.7:0/1569767304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ea40b3060 msgr2=0x7f1ea40d0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.702+0000 7f1eab000640 1 --2- 172.21.15.7:0/1569767304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ea40b3060 0x7f1ea40d0670 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40ce550 tx=0x7f1e9002a4d0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.703+0000 7fe313800640 1 -- 172.21.15.7:0/503837256 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe30c1f4ed0 con 0x7fe30c0cc520 2024-09-10T21:42:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.703+0000 7fe313800640 1 -- 172.21.15.7:0/503837256 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe30c1f5490 con 0x7fe30c0cc520 2024-09-10T21:42:16.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.703+0000 7f1eab000640 1 -- 172.21.15.7:0/1569767304 shutdown_connections 2024-09-10T21:42:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.703+0000 7f1eab000640 1 --2- 172.21.15.7:0/1569767304 >> v2:172.21.15.7:3300/0 conn(0x7f1ea40d0bb0 0x7f1ea40b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.703+0000 7f1eab000640 1 --2- 172.21.15.7:0/1569767304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ea40b3060 0x7f1ea40d0670 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.703+0000 7f1eab000640 1 -- 172.21.15.7:0/1569767304 >> 172.21.15.7:0/1569767304 conn(0x7f1ea409d270 msgr2=0x7f1ea40d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.704+0000 7f1eab000640 1 -- 172.21.15.7:0/1569767304 shutdown_connections 2024-09-10T21:42:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.704+0000 7f1eab000640 1 -- 172.21.15.7:0/1569767304 wait complete. 2024-09-10T21:42:16.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.704+0000 7fe313800640 1 -- 172.21.15.7:0/503837256 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe30c158350 con 0x7fe30c0cc520 2024-09-10T21:42:16.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.707+0000 7f1eab000640 1 Processor -- start 2024-09-10T21:42:16.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.707+0000 7fe308c00640 1 -- 172.21.15.7:0/503837256 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe2f803c420 con 0x7fe30c0cc520 2024-09-10T21:42:16.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.707+0000 7f1eab000640 1 -- start start 2024-09-10T21:42:16.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.707+0000 7fbd4b200640 1 -- 172.21.15.7:0/90623402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd2c0062d0 msgr2=0x7fbd2c003440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.707+0000 7fbd4b200640 1 --2- 172.21.15.7:0/90623402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd2c0062d0 0x7fbd2c003440 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbd34001760 tx=0x7fbd2c006940 comp rx=0 tx=0).stop 2024-09-10T21:42:16.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.708+0000 7fbd4b200640 1 -- 172.21.15.7:0/90623402 shutdown_connections 2024-09-10T21:42:16.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.708+0000 7fbd4b200640 1 --2- 172.21.15.7:0/90623402 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd2c0062d0 0x7fbd2c003440 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.708+0000 7fbd4b200640 1 -- 172.21.15.7:0/90623402 >> 172.21.15.7:0/90623402 conn(0x7fbd440b16d0 msgr2=0x7fbd440b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.709+0000 7fbd4b200640 1 -- 172.21.15.7:0/90623402 shutdown_connections 2024-09-10T21:42:16.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.709+0000 7f1eab000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ea40b3060 0x7f1ea41503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.709+0000 7f1eab000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ea40d0bb0 0x7f1ea415f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.709+0000 7f1eab000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea415e060 0x7f1ea41f7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.709+0000 7f1eab000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea40b4360 con 0x7f1ea415e060 2024-09-10T21:42:16.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.709+0000 7f1eab000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea40b41e0 con 0x7f1ea40d0bb0 2024-09-10T21:42:16.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.709+0000 7f1eab000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f1ea40b44e0 con 0x7f1ea40b3060 2024-09-10T21:42:16.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.710+0000 7f1ea3e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ea40b3060 0x7f1ea41503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.709+0000 7fe308c00640 1 --2- 172.21.15.7:0/503837256 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe2ec078040 0x7fe2ec07a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.709+0000 7fe308c00640 1 -- 172.21.15.7:0/503837256 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe2f80bb220 con 0x7fe30c0cc520 2024-09-10T21:42:16.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.710+0000 7f1ea3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ea40d0bb0 0x7f1ea415f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.710+0000 7f1ea3400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ea40d0bb0 0x7f1ea415f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:49030/0 (socket says 172.21.15.7:49030) 2024-09-10T21:42:16.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.710+0000 7f1ea3400640 1 -- 172.21.15.7:0/3671554452 learned_addr learned my addr 172.21.15.7:0/3671554452 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:16.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.710+0000 7f1ea9000640 1 --2- 172.21.15.7:0/3671554452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea415e060 0x7f1ea41f7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.709+0000 7fbd4b200640 1 -- 172.21.15.7:0/90623402 wait complete. 2024-09-10T21:42:16.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.710+0000 7fe30be00640 1 --2- 172.21.15.7:0/503837256 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe2ec078040 0x7fe2ec07a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.711+0000 7f1ea3e00640 1 -- 172.21.15.7:0/3671554452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ea40d0bb0 msgr2=0x7f1ea415f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.711+0000 7f1ea3e00640 1 --2- 172.21.15.7:0/3671554452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ea40d0bb0 0x7f1ea415f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.711+0000 7f1ea3e00640 1 -- 172.21.15.7:0/3671554452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea415e060 msgr2=0x7f1ea41f7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.711+0000 7f1ea3e00640 1 --2- 172.21.15.7:0/3671554452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea415e060 0x7f1ea41f7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.711+0000 7f1ea3e00640 1 -- 172.21.15.7:0/3671554452 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea4150910 con 0x7f1ea40b3060 2024-09-10T21:42:16.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.711+0000 7f1ea3400640 1 --2- 172.21.15.7:0/3671554452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ea40d0bb0 0x7f1ea415f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:42:16.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.711+0000 7f1ea3e00640 1 --2- 172.21.15.7:0/3671554452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ea40b3060 0x7f1ea41503d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00be50 tx=0x7f1e9c009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.711+0000 7fe30be00640 1 --2- 172.21.15.7:0/503837256 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe2ec078040 0x7fe2ec07a500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0c63b0 tx=0x7fe304009670 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.712+0000 7f4697a00640 1 -- 172.21.15.7:0/1455671807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f469006ff50 msgr2=0x7f469015cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.712+0000 7f4697a00640 1 --2- 172.21.15.7:0/1455671807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f469006ff50 0x7f469015cbe0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f46900b2620 tx=0x7f468402a370 comp rx=0 tx=0).stop 2024-09-10T21:42:16.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.712+0000 7fbd4b200640 1 Processor -- start 2024-09-10T21:42:16.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.712+0000 7f4697a00640 1 -- 172.21.15.7:0/1455671807 shutdown_connections 2024-09-10T21:42:16.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.712+0000 7f4697a00640 1 --2- 172.21.15.7:0/1455671807 >> v2:172.21.15.7:3300/0 conn(0x7f469006e960 0x7f4690074210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.712+0000 7f4697a00640 1 --2- 172.21.15.7:0/1455671807 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f469006e510 0x7f4690073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.712+0000 7f4697a00640 1 --2- 172.21.15.7:0/1455671807 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f469006ff50 0x7f469015cbe0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.712+0000 7f4697a00640 1 -- 172.21.15.7:0/1455671807 >> 172.21.15.7:0/1455671807 conn(0x7f46900b1660 msgr2=0x7f46900b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.712+0000 7fbd4b200640 1 -- start start 2024-09-10T21:42:16.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.713+0000 7f4697a00640 1 -- 172.21.15.7:0/1455671807 shutdown_connections 2024-09-10T21:42:16.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.713+0000 7f4697a00640 1 -- 172.21.15.7:0/1455671807 wait complete. 2024-09-10T21:42:16.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.713+0000 7f1e9be00640 1 -- 172.21.15.7:0/3671554452 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c019070 con 0x7f1ea40b3060 2024-09-10T21:42:16.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.714+0000 7f1e9be00640 1 -- 172.21.15.7:0/3671554452 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e9c014440 con 0x7f1ea40b3060 2024-09-10T21:42:16.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.714+0000 7f1e9be00640 1 -- 172.21.15.7:0/3671554452 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c019070 con 0x7f1ea40b3060 2024-09-10T21:42:16.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.714+0000 7f1eab000640 1 -- 172.21.15.7:0/3671554452 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea4160290 con 0x7f1ea40b3060 2024-09-10T21:42:16.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.714+0000 7f1eab000640 1 -- 172.21.15.7:0/3671554452 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea415ffb0 con 0x7f1ea40b3060 2024-09-10T21:42:16.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.714+0000 7fbd4b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd2c0062d0 0x7fbd4414e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.714+0000 7fbd4b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd44153fd0 0x7fbd441556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.714+0000 7fe308c00640 1 -- 172.21.15.7:0/503837256 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe30c158350 con 0x7fe30c0cc520 2024-09-10T21:42:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.714+0000 7fbd4b200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbd44156610 0x7fbd441594a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.714+0000 7fbd4b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4415a170 con 0x7fbd2c0062d0 2024-09-10T21:42:16.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.714+0000 7fbd4b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbd44070740 con 0x7fbd44153fd0 2024-09-10T21:42:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.715+0000 7feafec00640 1 -- 172.21.15.7:0/4142929197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feaf80cc520 msgr2=0x7feaf80cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.715+0000 7feafec00640 1 --2- 172.21.15.7:0/4142929197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feaf80cc520 0x7feaf80cc940 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7feae8005fd0 tx=0x7feae802a540 comp rx=0 tx=0).stop 2024-09-10T21:42:16.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.714+0000 7fbd4b200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fbd4415a2f0 con 0x7fbd44156610 2024-09-10T21:42:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.715+0000 7fbd49e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd2c0062d0 0x7fbd4414e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.715+0000 7fbd4a800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbd44156610 0x7fbd441594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.715+0000 7fbd49e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd2c0062d0 0x7fbd4414e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54252/0 (socket says 172.21.15.7:54252) 2024-09-10T21:42:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.715+0000 7fbd49e00640 1 -- 172.21.15.7:0/2337736543 learned_addr learned my addr 172.21.15.7:0/2337736543 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.715+0000 7f4697a00640 1 Processor -- start 2024-09-10T21:42:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.715+0000 7f4697a00640 1 -- start start 2024-09-10T21:42:16.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.716+0000 7fbd49400640 1 --2- 172.21.15.7:0/2337736543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd44153fd0 0x7fbd441556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.716+0000 7feafec00640 1 -- 172.21.15.7:0/4142929197 shutdown_connections 2024-09-10T21:42:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.716+0000 7fbd49e00640 1 -- 172.21.15.7:0/2337736543 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbd44156610 msgr2=0x7fbd441594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.716+0000 7fbd49e00640 1 --2- 172.21.15.7:0/2337736543 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbd44156610 0x7fbd441594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.716+0000 7fbd49e00640 1 -- 172.21.15.7:0/2337736543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd44153fd0 msgr2=0x7fbd441556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.716+0000 7fbd49e00640 1 --2- 172.21.15.7:0/2337736543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd44153fd0 0x7fbd441556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.716+0000 7fbd49e00640 1 -- 172.21.15.7:0/2337736543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd441f3750 con 0x7fbd2c0062d0 2024-09-10T21:42:16.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.716+0000 7feafec00640 1 --2- 172.21.15.7:0/4142929197 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feaf80cc520 0x7feaf80cc940 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.716+0000 7feafec00640 1 --2- 172.21.15.7:0/4142929197 >> v2:172.21.15.7:3300/0 conn(0x7feaf815e470 0x7feaf80ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.716+0000 7feafec00640 1 -- 172.21.15.7:0/4142929197 >> 172.21.15.7:0/4142929197 conn(0x7feaf80b1660 msgr2=0x7feaf80b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.716+0000 7fe31f000640 1 -- 172.21.15.7:0/2724398013 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe31815eda0 msgr2=0x7fe318161940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.716+0000 7fe31f000640 1 --2- 172.21.15.7:0/2724398013 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe31815eda0 0x7fe318161940 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe304009830 tx=0x7fe30402a520 comp rx=0 tx=0).stop 2024-09-10T21:42:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.716+0000 7fe31f000640 1 -- 172.21.15.7:0/2724398013 shutdown_connections 2024-09-10T21:42:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.716+0000 7fe31f000640 1 --2- 172.21.15.7:0/2724398013 >> v2:172.21.15.133:3300/0 conn(0x7fe318161e80 0x7fe318072510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.716+0000 7fe31f000640 1 --2- 172.21.15.7:0/2724398013 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe31815eda0 0x7fe318161940 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.716+0000 7fe31f000640 1 -- 172.21.15.7:0/2724398013 >> 172.21.15.7:0/2724398013 conn(0x7fe3180b1660 msgr2=0x7fe3180b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.716+0000 7fbd4a800640 1 --2- 172.21.15.7:0/2337736543 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbd44156610 0x7fbd441594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.716+0000 7fbd49e00640 1 --2- 172.21.15.7:0/2337736543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd2c0062d0 0x7fbd4414e800 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbd40010bc0 tx=0x7fbd4000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.717+0000 7feafec00640 1 -- 172.21.15.7:0/4142929197 shutdown_connections 2024-09-10T21:42:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.717+0000 7f4697a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f469006e510 0x7f469015a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.717+0000 7f4697a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f469006e960 0x7f469015b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.717+0000 7f4697a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ff50 0x7f46900c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.717+0000 7f4697a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f469015f890 con 0x7f469006ff50 2024-09-10T21:42:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.717+0000 7f4697a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f469015f710 con 0x7f469006e960 2024-09-10T21:42:16.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.717+0000 7f4697a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f469015fa10 con 0x7f469006e510 2024-09-10T21:42:16.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.717+0000 7fbd3e000640 1 -- 172.21.15.7:0/2337736543 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40003b70 con 0x7fbd2c0062d0 2024-09-10T21:42:16.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.717+0000 7fbd3e000640 1 -- 172.21.15.7:0/2337736543 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd40003d10 con 0x7fbd2c0062d0 2024-09-10T21:42:16.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.717+0000 7fbd3e000640 1 -- 172.21.15.7:0/2337736543 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd40004280 con 0x7fbd2c0062d0 2024-09-10T21:42:16.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.717+0000 7fbd4b200640 1 -- 172.21.15.7:0/2337736543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd44156ac0 con 0x7fbd2c0062d0 2024-09-10T21:42:16.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.717+0000 7fe31f000640 1 -- 172.21.15.7:0/2724398013 shutdown_connections 2024-09-10T21:42:16.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.717+0000 7f4695000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f469006e510 0x7f469015a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.717+0000 7f468fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f469006e960 0x7f469015b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.717+0000 7f1eab000640 1 -- 172.21.15.7:0/3671554452 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e7c005290 con 0x7f1ea40b3060 2024-09-10T21:42:16.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.718+0000 7feafec00640 1 -- 172.21.15.7:0/4142929197 wait complete. 2024-09-10T21:42:16.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.718+0000 7fbd4b200640 1 -- 172.21.15.7:0/2337736543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd441f3cd0 con 0x7fbd2c0062d0 2024-09-10T21:42:16.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.718+0000 7f4695000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f469006e510 0x7f469015a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:50246/0 (socket says 172.21.15.7:50246) 2024-09-10T21:42:16.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.718+0000 7f4695000640 1 -- 172.21.15.7:0/1138883994 learned_addr learned my addr 172.21.15.7:0/1138883994 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:16.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.718+0000 7f4695a00640 1 --2- 172.21.15.7:0/1138883994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ff50 0x7f46900c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.718+0000 7fe31f000640 1 -- 172.21.15.7:0/2724398013 wait complete. 2024-09-10T21:42:16.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.718+0000 7f1e9be00640 1 -- 172.21.15.7:0/3671554452 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1e9c01d3b0 con 0x7f1ea40b3060 2024-09-10T21:42:16.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.720+0000 7fbd4b200640 1 -- 172.21.15.7:0/2337736543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd44072520 con 0x7fbd2c0062d0 2024-09-10T21:42:16.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.718+0000 7f4695000640 1 -- 172.21.15.7:0/1138883994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f469006e960 msgr2=0x7f469015b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.718+0000 7f4695000640 1 --2- 172.21.15.7:0/1138883994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f469006e960 0x7f469015b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.718+0000 7f4695000640 1 -- 172.21.15.7:0/1138883994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ff50 msgr2=0x7f46900c52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.718+0000 7f4695000640 1 --2- 172.21.15.7:0/1138883994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ff50 0x7f46900c52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.718+0000 7f4695000640 1 -- 172.21.15.7:0/1138883994 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46900ca240 con 0x7f469006e510 2024-09-10T21:42:16.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.720+0000 7f4695000640 1 --2- 172.21.15.7:0/1138883994 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f469006e510 0x7f469015a9e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4684000c00 tx=0x7f4684033c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.720+0000 7feafec00640 1 Processor -- start 2024-09-10T21:42:16.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.720+0000 7fe31f000640 1 Processor -- start 2024-09-10T21:42:16.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.721+0000 7fe31f000640 1 -- start start 2024-09-10T21:42:16.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.722+0000 7fe31f000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3180cc520 0x7fe3180b2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.722+0000 7f468cc00640 1 -- 172.21.15.7:0/1138883994 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468403f020 con 0x7f469006e510 2024-09-10T21:42:16.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.722+0000 7f468cc00640 1 -- 172.21.15.7:0/1138883994 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4684033e10 con 0x7f469006e510 2024-09-10T21:42:16.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.722+0000 7f468cc00640 1 -- 172.21.15.7:0/1138883994 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684037040 con 0x7f469006e510 2024-09-10T21:42:16.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.722+0000 7fe31f000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe31815eda0 0x7fe3181fe4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.722+0000 7fe31f000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe318161e80 0x7fe318204740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.723+0000 7feafec00640 1 -- start start 2024-09-10T21:42:16.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.723+0000 7f4697a00640 1 -- 172.21.15.7:0/1138883994 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46901f9740 con 0x7f469006e510 2024-09-10T21:42:16.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.723+0000 7f4697a00640 1 -- 172.21.15.7:0/1138883994 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f46900ca7c0 con 0x7f469006e510 2024-09-10T21:42:16.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.722+0000 7f1e9be00640 1 --2- 172.21.15.7:0/3671554452 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1e84078170 0x7f1e8407a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.722+0000 7f1e9be00640 1 -- 172.21.15.7:0/3671554452 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f1e9c099cc0 con 0x7f1ea40b3060 2024-09-10T21:42:16.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.722+0000 7fe31f000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe318072a50 con 0x7fe3180cc520 2024-09-10T21:42:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.722+0000 7fe31f000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe318155240 con 0x7fe318161e80 2024-09-10T21:42:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.722+0000 7fe31f000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe318072bd0 con 0x7fe31815eda0 2024-09-10T21:42:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.723+0000 7fe317400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe31815eda0 0x7fe3181fe4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.723+0000 7fe317400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe31815eda0 0x7fe3181fe4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:50248/0 (socket says 172.21.15.7:50248) 2024-09-10T21:42:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.723+0000 7fe317400640 1 -- 172.21.15.7:0/3960647260 learned_addr learned my addr 172.21.15.7:0/3960647260 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.723+0000 7fe31d000640 1 --2- 172.21.15.7:0/3960647260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe318161e80 0x7fe318204740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.723+0000 7fe31d000640 1 -- 172.21.15.7:0/3960647260 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe31815eda0 msgr2=0x7fe3181fe4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.723+0000 7fe31d000640 1 --2- 172.21.15.7:0/3960647260 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe31815eda0 0x7fe3181fe4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.724+0000 7f1ea3400640 1 --2- 172.21.15.7:0/3671554452 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1e84078170 0x7f1e8407a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.723+0000 7fe31d000640 1 -- 172.21.15.7:0/3960647260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3180cc520 msgr2=0x7fe3180b2b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:42:16.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.723+0000 7fe317e00640 1 --2- 172.21.15.7:0/3960647260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3180cc520 0x7fe3180b2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.723+0000 7fe31d000640 1 --2- 172.21.15.7:0/3960647260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3180cc520 0x7fe3180b2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.723+0000 7fe31d000640 1 -- 172.21.15.7:0/3960647260 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3180b3050 con 0x7fe318161e80 2024-09-10T21:42:16.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.723+0000 7fbd3e000640 1 -- 172.21.15.7:0/2337736543 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fbd4000f040 con 0x7fbd2c0062d0 2024-09-10T21:42:16.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.724+0000 7fe31d000640 1 --2- 172.21.15.7:0/3960647260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe318161e80 0x7fe318204740 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe3180b2480 tx=0x7fe30800edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.725+0000 7f1ea3400640 1 --2- 172.21.15.7:0/3671554452 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1e84078170 0x7f1e8407a630 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40b2ee0 tx=0x7f1e9000b040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.725+0000 7f4697a00640 1 -- 172.21.15.7:0/1138883994 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f469015cbe0 con 0x7f469006e510 2024-09-10T21:42:16.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.724+0000 7fe317e00640 1 --2- 172.21.15.7:0/3960647260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3180cc520 0x7fe3180b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:16.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.725+0000 7feafec00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feaf80cc520 0x7feaf80c9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.725+0000 7feafec00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaf815bc20 0x7feaf81f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.725+0000 7feafec00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feaf815e470 0x7feaf81f8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.725+0000 7fe30fe00640 1 -- 172.21.15.7:0/3960647260 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30800ccc0 con 0x7fe318161e80 2024-09-10T21:42:16.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.725+0000 7fe30fe00640 1 -- 172.21.15.7:0/3960647260 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe308004020 con 0x7fe318161e80 2024-09-10T21:42:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.725+0000 7fe31f000640 1 -- 172.21.15.7:0/3960647260 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe318157bb0 con 0x7fe318161e80 2024-09-10T21:42:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.725+0000 7feafec00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feaf806f150 con 0x7feaf815bc20 2024-09-10T21:42:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.725+0000 7feafec00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feaf806efd0 con 0x7feaf815e470 2024-09-10T21:42:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.725+0000 7feafec00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7feaf806f2d0 con 0x7feaf80cc520 2024-09-10T21:42:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.726+0000 7feafcc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feaf815e470 0x7feaf81f8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.726+0000 7feafcc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feaf815e470 0x7feaf81f8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:49086/0 (socket says 172.21.15.7:49086) 2024-09-10T21:42:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.726+0000 7feafcc00640 1 -- 172.21.15.7:0/3094863317 learned_addr learned my addr 172.21.15.7:0/3094863317 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:16.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.726+0000 7feaf7e00640 1 --2- 172.21.15.7:0/3094863317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feaf80cc520 0x7feaf80c9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.726+0000 7fbd3e000640 1 --2- 172.21.15.7:0/2337736543 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbd28077fc0 0x7fbd2807a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.726+0000 7feafcc00640 1 -- 172.21.15.7:0/3094863317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feaf80cc520 msgr2=0x7feaf80c9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.726+0000 7feafcc00640 1 --2- 172.21.15.7:0/3094863317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feaf80cc520 0x7feaf80c9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.726+0000 7feafcc00640 1 -- 172.21.15.7:0/3094863317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaf815bc20 msgr2=0x7feaf81f81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.726+0000 7fbd3e000640 1 -- 172.21.15.7:0/2337736543 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fbd4009a8f0 con 0x7fbd2c0062d0 2024-09-10T21:42:16.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.726+0000 7feafcc00640 1 --2- 172.21.15.7:0/3094863317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaf815bc20 0x7feaf81f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.726+0000 7feafcc00640 1 -- 172.21.15.7:0/3094863317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf80b5520 con 0x7feaf815e470 2024-09-10T21:42:16.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.725+0000 7fe30fe00640 1 -- 172.21.15.7:0/3960647260 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe30800ce60 con 0x7fe318161e80 2024-09-10T21:42:16.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.727+0000 7feafcc00640 1 --2- 172.21.15.7:0/3094863317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feaf815e470 0x7feaf81f8730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7feae8006010 tx=0x7feae8003820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.725+0000 7fe31f000640 1 -- 172.21.15.7:0/3960647260 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe318158170 con 0x7fe318161e80 2024-09-10T21:42:16.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.727+0000 7feaefe00640 1 -- 172.21.15.7:0/3094863317 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae803f070 con 0x7feaf815e470 2024-09-10T21:42:16.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.727+0000 7feaefe00640 1 -- 172.21.15.7:0/3094863317 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feae80047a0 con 0x7feaf815e470 2024-09-10T21:42:16.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.727+0000 7feafec00640 1 -- 172.21.15.7:0/3094863317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf8072400 con 0x7feaf815e470 2024-09-10T21:42:16.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.727+0000 7fbd49400640 1 --2- 172.21.15.7:0/2337736543 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbd28077fc0 0x7fbd2807a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.727+0000 7f1e9be00640 1 -- 172.21.15.7:0/3671554452 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1e9c0a0080 con 0x7f1ea40b3060 2024-09-10T21:42:16.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.728+0000 7feaefe00640 1 -- 172.21.15.7:0/3094863317 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae8035030 con 0x7feaf815e470 2024-09-10T21:42:16.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.729+0000 7fe31f000640 1 -- 172.21.15.7:0/3960647260 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe318073080 con 0x7fe318161e80 2024-09-10T21:42:16.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.728+0000 7fbd49400640 1 --2- 172.21.15.7:0/2337736543 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbd28077fc0 0x7fbd2807a480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbd38000fb0 tx=0x7fbd38001580 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.729+0000 7fe30fe00640 1 -- 172.21.15.7:0/3960647260 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe3080035f0 con 0x7fe318161e80 2024-09-10T21:42:16.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.729+0000 7feafec00640 1 -- 172.21.15.7:0/3094863317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf80b5800 con 0x7feaf815e470 2024-09-10T21:42:16.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.730+0000 7feafec00640 1 -- 172.21.15.7:0/3094863317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf806f7f0 con 0x7feaf815e470 2024-09-10T21:42:16.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.731+0000 7fe30fe00640 1 --2- 172.21.15.7:0/3960647260 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe2f8078070 0x7fe2f807a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.731+0000 7fe30fe00640 1 -- 172.21.15.7:0/3960647260 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe30809a510 con 0x7fe318161e80 2024-09-10T21:42:16.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.732+0000 7fe317e00640 1 --2- 172.21.15.7:0/3960647260 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe2f8078070 0x7fe2f807a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.732+0000 7f468cc00640 1 -- 172.21.15.7:0/1138883994 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f468403a030 con 0x7f469006e510 2024-09-10T21:42:16.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.732+0000 7fe317e00640 1 --2- 172.21.15.7:0/3960647260 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe2f8078070 0x7fe2f807a530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe310002190 tx=0x7fe310009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.733+0000 7feaefe00640 1 -- 172.21.15.7:0/3094863317 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7feae8004bd0 con 0x7feaf815e470 2024-09-10T21:42:16.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.735+0000 7fbd3e000640 1 -- 172.21.15.7:0/2337736543 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fbd40062f10 con 0x7fbd2c0062d0 2024-09-10T21:42:16.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.734+0000 7feaefe00640 1 --2- 172.21.15.7:0/3094863317 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fead8077f70 0x7fead807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.735+0000 7feaefe00640 1 -- 172.21.15.7:0/3094863317 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feae80bb290 con 0x7feaf815e470 2024-09-10T21:42:16.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.736+0000 7fe30fe00640 1 -- 172.21.15.7:0/3960647260 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe308062ff0 con 0x7fe318161e80 2024-09-10T21:42:16.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.737+0000 7fb320400640 1 -- 172.21.15.7:0/1757650962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb31815e120 msgr2=0x7fb3180d4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.736+0000 7feaf7e00640 1 --2- 172.21.15.7:0/3094863317 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fead8077f70 0x7fead807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.738+0000 7feaf7e00640 1 --2- 172.21.15.7:0/3094863317 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fead8077f70 0x7fead807a430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feaf00012e0 tx=0x7feaf0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.740+0000 7feaefe00640 1 -- 172.21.15.7:0/3094863317 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7feae8083d80 con 0x7feaf815e470 2024-09-10T21:42:16.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.737+0000 7f468cc00640 1 --2- 172.21.15.7:0/1138883994 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4670078040 0x7f467007a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.737+0000 7f468cc00640 1 -- 172.21.15.7:0/1138883994 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f46840ba2b0 con 0x7f469006e510 2024-09-10T21:42:16.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.737+0000 7f468cc00640 1 -- 172.21.15.7:0/1138883994 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4684049070 con 0x7f469006e510 2024-09-10T21:42:16.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.737+0000 7f468fe00640 1 --2- 172.21.15.7:0/1138883994 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4670078040 0x7f467007a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.738+0000 7f468fe00640 1 --2- 172.21.15.7:0/1138883994 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4670078040 0x7f467007a500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f469015af20 tx=0x7f467c009260 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.737+0000 7fb320400640 1 --2- 172.21.15.7:0/1757650962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb31815e120 0x7fb3180d4a30 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb3180ce740 tx=0x7fb30802a460 comp rx=0 tx=0).stop 2024-09-10T21:42:16.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.738+0000 7fb320400640 1 -- 172.21.15.7:0/1757650962 shutdown_connections 2024-09-10T21:42:16.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.738+0000 7fb320400640 1 --2- 172.21.15.7:0/1757650962 >> v2:172.21.15.133:3300/0 conn(0x7fb3180d10d0 0x7fb3180cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.738+0000 7fb320400640 1 --2- 172.21.15.7:0/1757650962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb31815e120 0x7fb3180d4a30 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.738+0000 7fb320400640 1 -- 172.21.15.7:0/1757650962 >> 172.21.15.7:0/1757650962 conn(0x7fb3180d3f10 msgr2=0x7fb3180d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:16.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.738+0000 7fb320400640 1 -- 172.21.15.7:0/1757650962 shutdown_connections 2024-09-10T21:42:16.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.738+0000 7fb320400640 1 -- 172.21.15.7:0/1757650962 wait complete. 2024-09-10T21:42:16.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.743+0000 7fb320400640 1 Processor -- start 2024-09-10T21:42:16.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.744+0000 7fb320400640 1 -- start start 2024-09-10T21:42:16.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.746+0000 7fb320400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3180d10d0 0x7fb3180c8450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.750+0000 7fb320400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3180d1f90 0x7fb3180a1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.750+0000 7fb320400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb31815e120 0x7fb3181f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.750+0000 7fb320400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb3180b7800 con 0x7fb3180d10d0 2024-09-10T21:42:16.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.750+0000 7fb320400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb3180b7680 con 0x7fb31815e120 2024-09-10T21:42:16.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.750+0000 7fb320400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fb3180c7fc0 con 0x7fb3180d1f90 2024-09-10T21:42:16.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.750+0000 7fb31d000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3180d1f90 0x7fb3180a1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.751+0000 7fb31d000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3180d1f90 0x7fb3180a1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:50272/0 (socket says 172.21.15.7:50272) 2024-09-10T21:42:16.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.751+0000 7fb31d000640 1 -- 172.21.15.7:0/2393027522 learned_addr learned my addr 172.21.15.7:0/2393027522 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:16.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.751+0000 7fb31da00640 1 --2- 172.21.15.7:0/2393027522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3180d10d0 0x7fb3180c8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.751+0000 7fb31e400640 1 --2- 172.21.15.7:0/2393027522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb31815e120 0x7fb3181f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.753+0000 7fb31da00640 1 -- 172.21.15.7:0/2393027522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3180d1f90 msgr2=0x7fb3180a1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.753+0000 7fb31da00640 1 --2- 172.21.15.7:0/2393027522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3180d1f90 0x7fb3180a1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.753+0000 7fb31da00640 1 -- 172.21.15.7:0/2393027522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb31815e120 msgr2=0x7fb3181f4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:16.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.753+0000 7fb31da00640 1 --2- 172.21.15.7:0/2393027522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb31815e120 0x7fb3181f4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:16.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.753+0000 7fb31da00640 1 -- 172.21.15.7:0/2393027522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3181f5a10 con 0x7fb3180d10d0 2024-09-10T21:42:16.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.753+0000 7fb31d000640 1 --2- 172.21.15.7:0/2393027522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3180d1f90 0x7fb3180a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:16.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.754+0000 7fb31da00640 1 --2- 172.21.15.7:0/2393027522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3180d10d0 0x7fb3180c8450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb308031000 tx=0x7fb3080039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.754+0000 7fb315600640 1 -- 172.21.15.7:0/2393027522 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb30803e030 con 0x7fb3180d10d0 2024-09-10T21:42:16.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.754+0000 7fb315600640 1 -- 172.21.15.7:0/2393027522 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3080047a0 con 0x7fb3180d10d0 2024-09-10T21:42:16.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.754+0000 7fb315600640 1 -- 172.21.15.7:0/2393027522 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb308035040 con 0x7fb3180d10d0 2024-09-10T21:42:16.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.755+0000 7fb320400640 1 -- 172.21.15.7:0/2393027522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3181f5cf0 con 0x7fb3180d10d0 2024-09-10T21:42:16.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.755+0000 7fb320400640 1 -- 172.21.15.7:0/2393027522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3180c8990 con 0x7fb3180d10d0 2024-09-10T21:42:16.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.757+0000 7fb320400640 1 -- 172.21.15.7:0/2393027522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3180c7a40 con 0x7fb3180d10d0 2024-09-10T21:42:16.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.764+0000 7fb315600640 1 -- 172.21.15.7:0/2393027522 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb308004b90 con 0x7fb3180d10d0 2024-09-10T21:42:16.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.765+0000 7fb315600640 1 --2- 172.21.15.7:0/2393027522 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb2fc078040 0x7fb2fc07a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:16.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.765+0000 7fb315600640 1 -- 172.21.15.7:0/2393027522 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb3080bafa0 con 0x7fb3180d10d0 2024-09-10T21:42:16.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.767+0000 7fb31d000640 1 --2- 172.21.15.7:0/2393027522 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb2fc078040 0x7fb2fc07a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:16.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.768+0000 7fb31d000640 1 --2- 172.21.15.7:0/2393027522 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb2fc078040 0x7fb2fc07a500 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb3180cabb0 tx=0x7fb30c008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:16.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:16.770+0000 7fb315600640 1 -- 172.21.15.7:0/2393027522 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb308083430 con 0x7fb3180d10d0 2024-09-10T21:42:17.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.152+0000 7fa20d200640 1 -- 172.21.15.7:0/707327189 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fa1e0005850 con 0x7fa1ec004800 2024-09-10T21:42:17.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.153+0000 7fa1fe000640 1 -- 172.21.15.7:0/707327189 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa1f806e460 con 0x7fa1ec004800 2024-09-10T21:42:17.155 INFO:teuthology.orchestra.run.smithi007.stdout:64424509445 2024-09-10T21:42:17.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.158+0000 7fa20d200640 1 -- 172.21.15.7:0/707327189 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa1e8077fc0 msgr2=0x7fa1e807a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:17.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.158+0000 7fa20d200640 1 --2- 172.21.15.7:0/707327189 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa1e8077fc0 0x7fa1e807a480 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa1f4007b50 tx=0x7fa1f400d040 comp rx=0 tx=0).stop 2024-09-10T21:42:17.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.158+0000 7fa20d200640 1 -- 172.21.15.7:0/707327189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1ec004800 msgr2=0x7fa200021800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:17.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.158+0000 7fa20d200640 1 --2- 172.21.15.7:0/707327189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1ec004800 0x7fa200021800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa1f8009a40 tx=0x7fa1f8009f00 comp rx=0 tx=0).stop 2024-09-10T21:42:17.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.159+0000 7fa20d200640 1 -- 172.21.15.7:0/707327189 shutdown_connections 2024-09-10T21:42:17.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.159+0000 7fe313800640 1 -- 172.21.15.7:0/503837256 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe30c073100 con 0x7fe30c0cc520 2024-09-10T21:42:17.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.159+0000 7fa20d200640 1 --2- 172.21.15.7:0/707327189 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa1e8077fc0 0x7fa1e807a480 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.159+0000 7fa20d200640 1 --2- 172.21.15.7:0/707327189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa2000de900 0x7fa200031900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.159+0000 7fa20d200640 1 --2- 172.21.15.7:0/707327189 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa2000de480 0x7fa2000313c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.159+0000 7fa20d200640 1 --2- 172.21.15.7:0/707327189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa1ec004800 0x7fa200021800 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.159+0000 7fa20d200640 1 -- 172.21.15.7:0/707327189 >> 172.21.15.7:0/707327189 conn(0x7fa20001d7a0 msgr2=0x7fa2000540e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:17.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.159+0000 7fe308c00640 1 -- 172.21.15.7:0/503837256 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe2f804a060 con 0x7fe30c0cc520 2024-09-10T21:42:17.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.159+0000 7fa20d200640 1 -- 172.21.15.7:0/707327189 shutdown_connections 2024-09-10T21:42:17.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.159+0000 7fa20d200640 1 -- 172.21.15.7:0/707327189 wait complete. 2024-09-10T21:42:17.162 INFO:teuthology.orchestra.run.smithi007.stdout:60129542149 2024-09-10T21:42:17.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.163+0000 7fe313800640 1 -- 172.21.15.7:0/503837256 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe2ec078040 msgr2=0x7fe2ec07a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:17.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.163+0000 7fe313800640 1 --2- 172.21.15.7:0/503837256 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe2ec078040 0x7fe2ec07a500 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe30c0c63b0 tx=0x7fe304009670 comp rx=0 tx=0).stop 2024-09-10T21:42:17.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.164+0000 7fe313800640 1 -- 172.21.15.7:0/503837256 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30c0cc520 msgr2=0x7fe30c0b0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:17.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.164+0000 7fe313800640 1 --2- 172.21.15.7:0/503837256 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30c0cc520 0x7fe30c0b0960 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe2f8000c00 tx=0x7fe2f8032b90 comp rx=0 tx=0).stop 2024-09-10T21:42:17.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.164+0000 7fe313800640 1 -- 172.21.15.7:0/503837256 shutdown_connections 2024-09-10T21:42:17.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.164+0000 7fe313800640 1 --2- 172.21.15.7:0/503837256 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe2ec078040 0x7fe2ec07a500 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.164+0000 7fe313800640 1 --2- 172.21.15.7:0/503837256 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe30c0cc900 0x7fe30c0b4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.165+0000 7fe313800640 1 --2- 172.21.15.7:0/503837256 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe30c161e80 0x7fe30c0c4a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.165+0000 7fe313800640 1 --2- 172.21.15.7:0/503837256 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe30c0cc520 0x7fe30c0b0960 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.165+0000 7fe313800640 1 -- 172.21.15.7:0/503837256 >> 172.21.15.7:0/503837256 conn(0x7fe30c0b1660 msgr2=0x7fe30c06eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:17.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.165+0000 7fe313800640 1 -- 172.21.15.7:0/503837256 shutdown_connections 2024-09-10T21:42:17.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.165+0000 7fe313800640 1 -- 172.21.15.7:0/503837256 wait complete. 2024-09-10T21:42:17.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.176+0000 7f1eab000640 1 -- 172.21.15.7:0/3671554452 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f1e7c005580 con 0x7f1ea40b3060 2024-09-10T21:42:17.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.178+0000 7f1e9be00640 1 -- 172.21.15.7:0/3671554452 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1ea4150910 con 0x7f1ea40b3060 2024-09-10T21:42:17.182 INFO:teuthology.orchestra.run.smithi007.stdout:64424509447 2024-09-10T21:42:17.182 INFO:teuthology.orchestra.run.smithi007.stdout:64424509446 2024-09-10T21:42:17.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.180+0000 7f4697a00640 1 -- 172.21.15.7:0/1138883994 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f46900caa00 con 0x7f469006e510 2024-09-10T21:42:17.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.180+0000 7f468cc00640 1 -- 172.21.15.7:0/1138883994 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4684082d90 con 0x7f469006e510 2024-09-10T21:42:17.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f1eab000640 1 -- 172.21.15.7:0/3671554452 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1e84078170 msgr2=0x7f1e8407a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:17.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f1eab000640 1 --2- 172.21.15.7:0/3671554452 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1e84078170 0x7f1e8407a630 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1ea40b2ee0 tx=0x7f1e9000b040 comp rx=0 tx=0).stop 2024-09-10T21:42:17.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f1eab000640 1 -- 172.21.15.7:0/3671554452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ea40b3060 msgr2=0x7f1ea41503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:17.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f1eab000640 1 --2- 172.21.15.7:0/3671554452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ea40b3060 0x7f1ea41503d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00be50 tx=0x7f1e9c009f70 comp rx=0 tx=0).stop 2024-09-10T21:42:17.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f1eab000640 1 -- 172.21.15.7:0/3671554452 shutdown_connections 2024-09-10T21:42:17.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f1eab000640 1 --2- 172.21.15.7:0/3671554452 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1e84078170 0x7f1e8407a630 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f1eab000640 1 --2- 172.21.15.7:0/3671554452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ea40d0bb0 0x7f1ea415f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f1eab000640 1 --2- 172.21.15.7:0/3671554452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ea415e060 0x7f1ea41f7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f1eab000640 1 --2- 172.21.15.7:0/3671554452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ea40b3060 0x7f1ea41503d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f1eab000640 1 -- 172.21.15.7:0/3671554452 >> 172.21.15.7:0/3671554452 conn(0x7f1ea409d270 msgr2=0x7f1ea406ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:17.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f1eab000640 1 -- 172.21.15.7:0/3671554452 shutdown_connections 2024-09-10T21:42:17.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.183+0000 7fbd4b200640 1 -- 172.21.15.7:0/2337736543 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fbd441f01b0 con 0x7fbd2c0062d0 2024-09-10T21:42:17.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f4697a00640 1 -- 172.21.15.7:0/1138883994 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4670078040 msgr2=0x7f467007a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:17.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f4697a00640 1 --2- 172.21.15.7:0/1138883994 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4670078040 0x7f467007a500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f469015af20 tx=0x7f467c009260 comp rx=0 tx=0).stop 2024-09-10T21:42:17.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f4697a00640 1 -- 172.21.15.7:0/1138883994 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f469006e510 msgr2=0x7f469015a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:17.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f4697a00640 1 --2- 172.21.15.7:0/1138883994 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f469006e510 0x7f469015a9e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4684000c00 tx=0x7f4684033c20 comp rx=0 tx=0).stop 2024-09-10T21:42:17.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.185+0000 7f1eab000640 1 -- 172.21.15.7:0/3671554452 wait complete. 2024-09-10T21:42:17.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f4697a00640 1 -- 172.21.15.7:0/1138883994 shutdown_connections 2024-09-10T21:42:17.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f4697a00640 1 --2- 172.21.15.7:0/1138883994 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4670078040 0x7f467007a500 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f469015af20 tx=0x7f467c009260 comp rx=0 tx=0).stop 2024-09-10T21:42:17.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f4697a00640 1 --2- 172.21.15.7:0/1138883994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f469006e960 0x7f469015b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f4697a00640 1 --2- 172.21.15.7:0/1138883994 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ff50 0x7f46900c52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f4697a00640 1 --2- 172.21.15.7:0/1138883994 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f469006e510 0x7f469015a9e0 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f4684000c00 tx=0x7f4684033c20 comp rx=0 tx=0).stop 2024-09-10T21:42:17.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.184+0000 7f4697a00640 1 -- 172.21.15.7:0/1138883994 >> 172.21.15.7:0/1138883994 conn(0x7f46900b1660 msgr2=0x7f469015ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:17.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.185+0000 7f4697a00640 1 -- 172.21.15.7:0/1138883994 shutdown_connections 2024-09-10T21:42:17.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.185+0000 7fbd3e000640 1 -- 172.21.15.7:0/2337736543 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbd40067940 con 0x7fbd2c0062d0 2024-09-10T21:42:17.192 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.0 2024-09-10T21:42:17.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.185+0000 7f4697a00640 1 -- 172.21.15.7:0/1138883994 wait complete. 2024-09-10T21:42:17.193 INFO:teuthology.orchestra.run.smithi007.stdout:60129542148 2024-09-10T21:42:17.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.188+0000 7fe31f000640 1 -- 172.21.15.7:0/3960647260 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe318070210 con 0x7fe318161e80 2024-09-10T21:42:17.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.189+0000 7fe30fe00640 1 -- 172.21.15.7:0/3960647260 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe3080628f0 con 0x7fe318161e80 2024-09-10T21:42:17.194 INFO:teuthology.orchestra.run.smithi007.stdout:64424509445 2024-09-10T21:42:17.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.190+0000 7fbd4b200640 1 -- 172.21.15.7:0/2337736543 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbd28077fc0 msgr2=0x7fbd2807a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:17.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.190+0000 7fbd4b200640 1 --2- 172.21.15.7:0/2337736543 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbd28077fc0 0x7fbd2807a480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbd38000fb0 tx=0x7fbd38001580 comp rx=0 tx=0).stop 2024-09-10T21:42:17.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.190+0000 7fbd4b200640 1 -- 172.21.15.7:0/2337736543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd2c0062d0 msgr2=0x7fbd4414e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:17.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.190+0000 7fbd4b200640 1 --2- 172.21.15.7:0/2337736543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd2c0062d0 0x7fbd4414e800 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fbd40010bc0 tx=0x7fbd4000e6c0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.190+0000 7fbd4b200640 1 -- 172.21.15.7:0/2337736543 shutdown_connections 2024-09-10T21:42:17.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.190+0000 7fbd4b200640 1 --2- 172.21.15.7:0/2337736543 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbd28077fc0 0x7fbd2807a480 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.190+0000 7fbd4b200640 1 --2- 172.21.15.7:0/2337736543 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbd44153fd0 0x7fbd441556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.190+0000 7fbd4b200640 1 --2- 172.21.15.7:0/2337736543 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbd44156610 0x7fbd441594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.190+0000 7fbd4b200640 1 --2- 172.21.15.7:0/2337736543 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbd2c0062d0 0x7fbd4414e800 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.190+0000 7fbd4b200640 1 -- 172.21.15.7:0/2337736543 >> 172.21.15.7:0/2337736543 conn(0x7fbd440b16d0 msgr2=0x7fbd4406e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:17.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.190+0000 7fbd4b200640 1 -- 172.21.15.7:0/2337736543 shutdown_connections 2024-09-10T21:42:17.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.190+0000 7fbd4b200640 1 -- 172.21.15.7:0/2337736543 wait complete. 2024-09-10T21:42:17.197 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542149 for osd.3 2024-09-10T21:42:17.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.193+0000 7fe31f000640 1 -- 172.21.15.7:0/3960647260 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe2f8078070 msgr2=0x7fe2f807a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:17.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.193+0000 7fe31f000640 1 --2- 172.21.15.7:0/3960647260 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe2f8078070 0x7fe2f807a530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe310002190 tx=0x7fe310009040 comp rx=0 tx=0).stop 2024-09-10T21:42:17.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.193+0000 7fe31f000640 1 -- 172.21.15.7:0/3960647260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe318161e80 msgr2=0x7fe318204740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:17.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.193+0000 7fe31f000640 1 --2- 172.21.15.7:0/3960647260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe318161e80 0x7fe318204740 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe3180b2480 tx=0x7fe30800edd0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.194+0000 7fe31f000640 1 -- 172.21.15.7:0/3960647260 shutdown_connections 2024-09-10T21:42:17.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.194+0000 7fe31f000640 1 --2- 172.21.15.7:0/3960647260 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe2f8078070 0x7fe2f807a530 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.194+0000 7fe31f000640 1 --2- 172.21.15.7:0/3960647260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe318161e80 0x7fe318204740 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.194+0000 7fe31f000640 1 --2- 172.21.15.7:0/3960647260 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe31815eda0 0x7fe3181fe4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.194+0000 7fe31f000640 1 --2- 172.21.15.7:0/3960647260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3180cc520 0x7fe3180b2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.194+0000 7fe31f000640 1 -- 172.21.15.7:0/3960647260 >> 172.21.15.7:0/3960647260 conn(0x7fe3180b1660 msgr2=0x7fe318073290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:17.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.194+0000 7fe31f000640 1 -- 172.21.15.7:0/3960647260 shutdown_connections 2024-09-10T21:42:17.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.194+0000 7fe31f000640 1 -- 172.21.15.7:0/3960647260 wait complete. 2024-09-10T21:42:17.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.209+0000 7feafec00640 1 -- 172.21.15.7:0/3094863317 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7feaf80b5a90 con 0x7feaf815e470 2024-09-10T21:42:17.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.209+0000 7feaefe00640 1 -- 172.21.15.7:0/3094863317 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7feae8083680 con 0x7feaf815e470 2024-09-10T21:42:17.211 INFO:teuthology.orchestra.run.smithi007.stdout:64424509445 2024-09-10T21:42:17.213 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.5 2024-09-10T21:42:17.214 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509447 for osd.1 2024-09-10T21:42:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.213+0000 7feafec00640 1 -- 172.21.15.7:0/3094863317 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fead8077f70 msgr2=0x7fead807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.213+0000 7feafec00640 1 --2- 172.21.15.7:0/3094863317 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fead8077f70 0x7fead807a430 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7feaf00012e0 tx=0x7feaf0009040 comp rx=0 tx=0).stop 2024-09-10T21:42:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.213+0000 7feafec00640 1 -- 172.21.15.7:0/3094863317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feaf815e470 msgr2=0x7feaf81f8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.213+0000 7feafec00640 1 --2- 172.21.15.7:0/3094863317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feaf815e470 0x7feaf81f8730 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7feae8006010 tx=0x7feae8003820 comp rx=0 tx=0).stop 2024-09-10T21:42:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.213+0000 7feafec00640 1 -- 172.21.15.7:0/3094863317 shutdown_connections 2024-09-10T21:42:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.213+0000 7feafec00640 1 --2- 172.21.15.7:0/3094863317 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fead8077f70 0x7fead807a430 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.213+0000 7feafec00640 1 --2- 172.21.15.7:0/3094863317 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feaf815e470 0x7feaf81f8730 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.213+0000 7feafec00640 1 --2- 172.21.15.7:0/3094863317 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feaf815bc20 0x7feaf81f81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.213+0000 7feafec00640 1 --2- 172.21.15.7:0/3094863317 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feaf80cc520 0x7feaf80c9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.213+0000 7feafec00640 1 -- 172.21.15.7:0/3094863317 >> 172.21.15.7:0/3094863317 conn(0x7feaf80b1660 msgr2=0x7feaf80ca890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:17.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.213+0000 7feafec00640 1 -- 172.21.15.7:0/3094863317 shutdown_connections 2024-09-10T21:42:17.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.213+0000 7feafec00640 1 -- 172.21.15.7:0/3094863317 wait complete. 2024-09-10T21:42:17.218 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.7 2024-09-10T21:42:17.219 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.4 2024-09-10T21:42:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.218+0000 7fb320400640 1 -- 172.21.15.7:0/2393027522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fb3180d2710 con 0x7fb3180d10d0 2024-09-10T21:42:17.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.219+0000 7fb315600640 1 -- 172.21.15.7:0/2393027522 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb308087e60 con 0x7fb3180d10d0 2024-09-10T21:42:17.220 INFO:teuthology.orchestra.run.smithi007.stdout:64424509446 2024-09-10T21:42:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.222+0000 7fb320400640 1 -- 172.21.15.7:0/2393027522 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb2fc078040 msgr2=0x7fb2fc07a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.222+0000 7fb320400640 1 --2- 172.21.15.7:0/2393027522 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb2fc078040 0x7fb2fc07a500 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb3180cabb0 tx=0x7fb30c008040 comp rx=0 tx=0).stop 2024-09-10T21:42:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.222+0000 7fb320400640 1 -- 172.21.15.7:0/2393027522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3180d10d0 msgr2=0x7fb3180c8450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.222+0000 7fb320400640 1 --2- 172.21.15.7:0/2393027522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3180d10d0 0x7fb3180c8450 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb308031000 tx=0x7fb3080039e0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.222+0000 7fb320400640 1 -- 172.21.15.7:0/2393027522 shutdown_connections 2024-09-10T21:42:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.222+0000 7fb320400640 1 --2- 172.21.15.7:0/2393027522 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb2fc078040 0x7fb2fc07a500 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.222+0000 7fb320400640 1 --2- 172.21.15.7:0/2393027522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb31815e120 0x7fb3181f4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.222+0000 7fb320400640 1 --2- 172.21.15.7:0/2393027522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3180d1f90 0x7fb3180a1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.222+0000 7fb320400640 1 --2- 172.21.15.7:0/2393027522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3180d10d0 0x7fb3180c8450 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.222+0000 7fb320400640 1 -- 172.21.15.7:0/2393027522 >> 172.21.15.7:0/2393027522 conn(0x7fb3180d3f10 msgr2=0x7fb3180b87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.222+0000 7fb320400640 1 -- 172.21.15.7:0/2393027522 shutdown_connections 2024-09-10T21:42:17.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:17.222+0000 7fb320400640 1 -- 172.21.15.7:0/2393027522 wait complete. 2024-09-10T21:42:17.235 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.2 2024-09-10T21:42:17.239 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509446 for osd.6 2024-09-10T21:42:18.193 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T21:42:18.198 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T21:42:18.214 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T21:42:18.216 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T21:42:18.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T21:42:18.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T21:42:18.236 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T21:42:18.241 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T21:42:18.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.447+0000 7faa7f200640 1 -- 172.21.15.7:0/1113836387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa8015df40 msgr2=0x7faa800b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.447+0000 7faa7ca00640 1 -- 172.21.15.7:0/1113836387 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78004230 con 0x7faa8015df40 2024-09-10T21:42:18.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.447+0000 7faa7f200640 1 --2- 172.21.15.7:0/1113836387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa8015df40 0x7faa800b3ae0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7faa78005fd0 tx=0x7faa7802a4d0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.449+0000 7faa7f200640 1 -- 172.21.15.7:0/1113836387 shutdown_connections 2024-09-10T21:42:18.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.449+0000 7faa7f200640 1 --2- 172.21.15.7:0/1113836387 >> v2:172.21.15.133:3300/0 conn(0x7faa80070910 0x7faa800b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.449+0000 7faa7f200640 1 --2- 172.21.15.7:0/1113836387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa8015df40 0x7faa800b3ae0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.449+0000 7faa7f200640 1 -- 172.21.15.7:0/1113836387 >> 172.21.15.7:0/1113836387 conn(0x7faa800cfa60 msgr2=0x7faa800cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:18.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.449+0000 7faa7f200640 1 -- 172.21.15.7:0/1113836387 shutdown_connections 2024-09-10T21:42:18.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.450+0000 7faa7f200640 1 -- 172.21.15.7:0/1113836387 wait complete. 2024-09-10T21:42:18.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.452+0000 7faa7f200640 1 Processor -- start 2024-09-10T21:42:18.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.452+0000 7faa7f200640 1 -- start start 2024-09-10T21:42:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.455+0000 7faa7f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa80070910 0x7faa801e9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.455+0000 7faa7f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa800b4560 0x7faa801e70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.456+0000 7faa7de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa80070910 0x7faa801e9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.456+0000 7faa7d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa800b4560 0x7faa801e70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.456+0000 7faa7de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa80070910 0x7faa801e9c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:49108/0 (socket says 172.21.15.7:49108) 2024-09-10T21:42:18.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.456+0000 7faa7de00640 1 -- 172.21.15.7:0/1680297434 learned_addr learned my addr 172.21.15.7:0/1680297434 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:18.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.456+0000 7faa7f200640 1 --2- 172.21.15.7:0/1680297434 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa8015df40 0x7faa800cd830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.456+0000 7faa7f200640 1 -- 172.21.15.7:0/1680297434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faa800735d0 con 0x7faa800b4560 2024-09-10T21:42:18.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.457+0000 7faa7f200640 1 -- 172.21.15.7:0/1680297434 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faa80073450 con 0x7faa80070910 2024-09-10T21:42:18.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.457+0000 7faa7f200640 1 -- 172.21.15.7:0/1680297434 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7faa800adff0 con 0x7faa8015df40 2024-09-10T21:42:18.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.458+0000 7faa7e800640 1 --2- 172.21.15.7:0/1680297434 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa8015df40 0x7faa800cd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.458+0000 7faa7de00640 1 -- 172.21.15.7:0/1680297434 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa8015df40 msgr2=0x7faa800cd830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.458+0000 7faa7de00640 1 --2- 172.21.15.7:0/1680297434 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa8015df40 0x7faa800cd830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.458+0000 7faa7de00640 1 -- 172.21.15.7:0/1680297434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa800b4560 msgr2=0x7faa801e70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.458+0000 7faa7de00640 1 --2- 172.21.15.7:0/1680297434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa800b4560 0x7faa801e70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.459+0000 7faa7de00640 1 -- 172.21.15.7:0/1680297434 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa800cf330 con 0x7faa80070910 2024-09-10T21:42:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.459+0000 7faa7d400640 1 --2- 172.21.15.7:0/1680297434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa800b4560 0x7faa801e70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.460+0000 7faa7de00640 1 --2- 172.21.15.7:0/1680297434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa80070910 0x7faa801e9c70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa78006010 tx=0x7faa780042e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.461+0000 7faa76000640 1 -- 172.21.15.7:0/1680297434 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78041070 con 0x7faa80070910 2024-09-10T21:42:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.461+0000 7faa76000640 1 -- 172.21.15.7:0/1680297434 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa780319c0 con 0x7faa80070910 2024-09-10T21:42:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.461+0000 7faa76000640 1 -- 172.21.15.7:0/1680297434 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa78031e10 con 0x7faa80070910 2024-09-10T21:42:18.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.462+0000 7faa7f200640 1 -- 172.21.15.7:0/1680297434 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa801f53a0 con 0x7faa80070910 2024-09-10T21:42:18.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.462+0000 7faa7f200640 1 -- 172.21.15.7:0/1680297434 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa801e65e0 con 0x7faa80070910 2024-09-10T21:42:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.465+0000 7faa7f200640 1 -- 172.21.15.7:0/1680297434 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa800ae4e0 con 0x7faa80070910 2024-09-10T21:42:18.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.467+0000 7faa76000640 1 -- 172.21.15.7:0/1680297434 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7faa7803b430 con 0x7faa80070910 2024-09-10T21:42:18.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.469+0000 7faa76000640 1 --2- 172.21.15.7:0/1680297434 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7faa60077fa0 0x7faa6007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.469+0000 7faa76000640 1 -- 172.21.15.7:0/1680297434 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7faa780bb030 con 0x7faa80070910 2024-09-10T21:42:18.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.470+0000 7faa7d400640 1 --2- 172.21.15.7:0/1680297434 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7faa60077fa0 0x7faa6007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.471+0000 7faa7d400640 1 --2- 172.21.15.7:0/1680297434 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7faa60077fa0 0x7faa6007a460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faa800cc220 tx=0x7faa7000a800 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:18.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.473+0000 7faa76000640 1 -- 172.21.15.7:0/1680297434 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7faa800cf330 con 0x7faa80070910 2024-09-10T21:42:18.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.525+0000 7efe44e00640 1 -- 172.21.15.7:0/3537540145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe400cfaa0 msgr2=0x7efe4015d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.525+0000 7efe44e00640 1 --2- 172.21.15.7:0/3537540145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe400cfaa0 0x7efe4015d980 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7efe30009830 tx=0x7efe3002a480 comp rx=0 tx=0).stop 2024-09-10T21:42:18.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.525+0000 7efe44e00640 1 -- 172.21.15.7:0/3537540145 shutdown_connections 2024-09-10T21:42:18.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.525+0000 7efe44e00640 1 --2- 172.21.15.7:0/3537540145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe400cfaa0 0x7efe4015d980 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.525+0000 7efe44e00640 1 --2- 172.21.15.7:0/3537540145 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe400d1920 0x7efe400d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.525+0000 7efe44e00640 1 -- 172.21.15.7:0/3537540145 >> 172.21.15.7:0/3537540145 conn(0x7efe400d3d50 msgr2=0x7efe400d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:18.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.526+0000 7efe44e00640 1 -- 172.21.15.7:0/3537540145 shutdown_connections 2024-09-10T21:42:18.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.526+0000 7efe44e00640 1 -- 172.21.15.7:0/3537540145 wait complete. 2024-09-10T21:42:18.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.528+0000 7efe44e00640 1 Processor -- start 2024-09-10T21:42:18.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.529+0000 7efe44e00640 1 -- start start 2024-09-10T21:42:18.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.531+0000 7efe44e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe400cfaa0 0x7efe4015cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.531+0000 7efe44e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe400d1920 0x7efe401fd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.531+0000 7efe44e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe400d2310 0x7efe400b6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.531+0000 7efe44e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe4015e250 con 0x7efe400cfaa0 2024-09-10T21:42:18.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.531+0000 7efe44e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efe4015e0d0 con 0x7efe400d2310 2024-09-10T21:42:18.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.531+0000 7efe44e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7efe400b3410 con 0x7efe400d1920 2024-09-10T21:42:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.532+0000 7fad8ca00640 1 -- 172.21.15.7:0/1403509375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad8815df70 msgr2=0x7fad880b6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.532+0000 7efe3f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe400cfaa0 0x7efe4015cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.532+0000 7efe3f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe400cfaa0 0x7efe4015cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54328/0 (socket says 172.21.15.7:54328) 2024-09-10T21:42:18.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.532+0000 7efe3f400640 1 -- 172.21.15.7:0/3712851096 learned_addr learned my addr 172.21.15.7:0/3712851096 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.532+0000 7fad8ca00640 1 --2- 172.21.15.7:0/1403509375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad8815df70 0x7fad880b6cd0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fad78005fd0 tx=0x7fad7802a540 comp rx=0 tx=0).stop 2024-09-10T21:42:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.532+0000 7fad8ca00640 1 -- 172.21.15.7:0/1403509375 shutdown_connections 2024-09-10T21:42:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.532+0000 7efe3fe00640 1 --2- 172.21.15.7:0/3712851096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe400d2310 0x7efe400b6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.532+0000 7efe3ea00640 1 --2- 172.21.15.7:0/3712851096 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe400d1920 0x7efe401fd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.532+0000 7fad8ca00640 1 --2- 172.21.15.7:0/1403509375 >> v2:172.21.15.133:3300/0 conn(0x7fad880b1120 0x7fad880d26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.532+0000 7fad8ca00640 1 --2- 172.21.15.7:0/1403509375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad8815df70 0x7fad880b6cd0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.532+0000 7fad8ca00640 1 -- 172.21.15.7:0/1403509375 >> 172.21.15.7:0/1403509375 conn(0x7fad880d3e80 msgr2=0x7fad880d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:18.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.533+0000 7efe3f400640 1 -- 172.21.15.7:0/3712851096 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe400d1920 msgr2=0x7efe401fd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.533+0000 7efe3f400640 1 --2- 172.21.15.7:0/3712851096 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe400d1920 0x7efe401fd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.533+0000 7efe3f400640 1 -- 172.21.15.7:0/3712851096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe400d2310 msgr2=0x7efe400b6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.533+0000 7efe3f400640 1 --2- 172.21.15.7:0/3712851096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe400d2310 0x7efe400b6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.533+0000 7efe3f400640 1 -- 172.21.15.7:0/3712851096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe4015b390 con 0x7efe400cfaa0 2024-09-10T21:42:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.533+0000 7efe3f400640 1 --2- 172.21.15.7:0/3712851096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe400cfaa0 0x7efe4015cd10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efe3800ba50 tx=0x7efe3800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.533+0000 7fad8ca00640 1 -- 172.21.15.7:0/1403509375 shutdown_connections 2024-09-10T21:42:18.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.533+0000 7fad8ca00640 1 -- 172.21.15.7:0/1403509375 wait complete. 2024-09-10T21:42:18.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.534+0000 7efe37400640 1 -- 172.21.15.7:0/3712851096 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe38019070 con 0x7efe400cfaa0 2024-09-10T21:42:18.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.534+0000 7efe37400640 1 -- 172.21.15.7:0/3712851096 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe3800cd10 con 0x7efe400cfaa0 2024-09-10T21:42:18.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.534+0000 7efe37400640 1 -- 172.21.15.7:0/3712851096 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe380146a0 con 0x7efe400cfaa0 2024-09-10T21:42:18.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.534+0000 7f3cdb200640 1 -- 172.21.15.7:0/536640456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd40e28c0 msgr2=0x7f3cd40538c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.534+0000 7f3cdb200640 1 --2- 172.21.15.7:0/536640456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd40e28c0 0x7f3cd40538c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3cc80098e0 tx=0x7f3cc802a5f0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.534+0000 7fe1c7000640 1 -- 172.21.15.7:0/2999886964 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1c00b5f50 msgr2=0x7fe1c00b4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.535+0000 7fe1c7000640 1 --2- 172.21.15.7:0/2999886964 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1c00b5f50 0x7fe1c00b4aa0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009830 tx=0x7fe1b802a520 comp rx=0 tx=0).stop 2024-09-10T21:42:18.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.535+0000 7f3cdb200640 1 -- 172.21.15.7:0/536640456 shutdown_connections 2024-09-10T21:42:18.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.535+0000 7f3cdb200640 1 --2- 172.21.15.7:0/536640456 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cd403a370 0x7f3cd403a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.535+0000 7f3cdb200640 1 --2- 172.21.15.7:0/536640456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd40e28c0 0x7f3cd40538c0 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3cc80098e0 tx=0x7f3cc802a5f0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.535+0000 7f3cdb200640 1 -- 172.21.15.7:0/536640456 >> 172.21.15.7:0/536640456 conn(0x7f3cd4055550 msgr2=0x7f3cd4055930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:18.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.535+0000 7fe1c7000640 1 -- 172.21.15.7:0/2999886964 shutdown_connections 2024-09-10T21:42:18.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.535+0000 7fe1c7000640 1 --2- 172.21.15.7:0/2999886964 >> v2:172.21.15.7:3300/0 conn(0x7fe1c00ce8d0 0x7fe1c00ced60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.535+0000 7fe1c7000640 1 --2- 172.21.15.7:0/2999886964 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1c00b5f50 0x7fe1c00b4aa0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.535+0000 7fe1c7000640 1 -- 172.21.15.7:0/2999886964 >> 172.21.15.7:0/2999886964 conn(0x7fe1c00d3dc0 msgr2=0x7fe1c00d41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:18.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.535+0000 7fe1c7000640 1 -- 172.21.15.7:0/2999886964 shutdown_connections 2024-09-10T21:42:18.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.535+0000 7fe1c7000640 1 -- 172.21.15.7:0/2999886964 wait complete. 2024-09-10T21:42:18.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.535+0000 7efe44e00640 1 -- 172.21.15.7:0/3712851096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe400b93b0 con 0x7efe400cfaa0 2024-09-10T21:42:18.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.535+0000 7efe44e00640 1 -- 172.21.15.7:0/3712851096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe4015b630 con 0x7efe400cfaa0 2024-09-10T21:42:18.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.537+0000 7fad8ca00640 1 Processor -- start 2024-09-10T21:42:18.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.537+0000 7fad8ca00640 1 -- start start 2024-09-10T21:42:18.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.538+0000 7fe1c7000640 1 Processor -- start 2024-09-10T21:42:18.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.538+0000 7fe1c7000640 1 -- start start 2024-09-10T21:42:18.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.539+0000 7efe44e00640 1 -- 172.21.15.7:0/3712851096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe400d1180 con 0x7efe400cfaa0 2024-09-10T21:42:18.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.539+0000 7f6e7e600640 1 -- 172.21.15.7:0/1791046527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e700e2bc0 msgr2=0x7f6e70006780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.539+0000 7f6e7e600640 1 --2- 172.21.15.7:0/1791046527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e700e2bc0 0x7f6e70006780 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6e78052150 tx=0x7f6e780a5e40 comp rx=0 tx=0).stop 2024-09-10T21:42:18.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.539+0000 7fad8ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad880b1120 0x7fad880c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.539+0000 7fad8ca00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad880d22b0 0x7fad880c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.539+0000 7fad8ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad8815df70 0x7fad880c99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.539+0000 7fad8ca00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad880b2960 con 0x7fad880b1120 2024-09-10T21:42:18.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.539+0000 7fad8ca00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fad880b27e0 con 0x7fad8815df70 2024-09-10T21:42:18.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.539+0000 7fad8ca00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fad880b2ae0 con 0x7fad880d22b0 2024-09-10T21:42:18.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.539+0000 7f3cdb200640 1 -- 172.21.15.7:0/536640456 shutdown_connections 2024-09-10T21:42:18.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.539+0000 7f3cdb200640 1 -- 172.21.15.7:0/536640456 wait complete. 2024-09-10T21:42:18.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.539+0000 7f6e7e600640 1 -- 172.21.15.7:0/1791046527 shutdown_connections 2024-09-10T21:42:18.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.539+0000 7f6e7e600640 1 --2- 172.21.15.7:0/1791046527 >> v2:172.21.15.133:3300/0 conn(0x7f6e7003a980 0x7f6e700e6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.539+0000 7f6e7e600640 1 --2- 172.21.15.7:0/1791046527 >> v2:172.21.15.7:3300/0 conn(0x7f6e700379f0 0x7f6e7003a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.539+0000 7f6e7e600640 1 --2- 172.21.15.7:0/1791046527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e700e2bc0 0x7f6e70006780 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.539+0000 7f6e7e600640 1 -- 172.21.15.7:0/1791046527 >> 172.21.15.7:0/1791046527 conn(0x7f6e70055580 msgr2=0x7f6e70055960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.540+0000 7fe1c7000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c00b5f50 0x7fe1c00cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.540+0000 7fe1c7000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1c00ce8d0 0x7fe1c006e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.540+0000 7fe1c7000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1c01614c0 0x7fe1c00ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.540+0000 7fe1c7000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c00cf2a0 con 0x7fe1c01614c0 2024-09-10T21:42:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.540+0000 7fe1c7000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c00d2cc0 con 0x7fe1c00b5f50 2024-09-10T21:42:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.540+0000 7fe1c7000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe1c00cf420 con 0x7fe1c00ce8d0 2024-09-10T21:42:18.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.540+0000 7fad87400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad880b1120 0x7fad880c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.540+0000 7f6e7e600640 1 -- 172.21.15.7:0/1791046527 shutdown_connections 2024-09-10T21:42:18.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.540+0000 7fad87e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad8815df70 0x7fad880c99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.540+0000 7f6e7e600640 1 -- 172.21.15.7:0/1791046527 wait complete. 2024-09-10T21:42:18.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.541+0000 7fe1c5200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1c00ce8d0 0x7fe1c006e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.541+0000 7fe1c5200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1c00ce8d0 0x7fe1c006e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:50350/0 (socket says 172.21.15.7:50350) 2024-09-10T21:42:18.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.541+0000 7fe1c5c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c00b5f50 0x7fe1c00cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.541+0000 7fe1c5200640 1 -- 172.21.15.7:0/1487482020 learned_addr learned my addr 172.21.15.7:0/1487482020 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:18.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.540+0000 7fad86a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad880d22b0 0x7fad880c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.541+0000 7fad86a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad880d22b0 0x7fad880c8200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:50346/0 (socket says 172.21.15.7:50346) 2024-09-10T21:42:18.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.541+0000 7fad86a00640 1 -- 172.21.15.7:0/502100468 learned_addr learned my addr 172.21.15.7:0/502100468 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:18.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.541+0000 7fe1c5200640 1 -- 172.21.15.7:0/1487482020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c00b5f50 msgr2=0x7fe1c00cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.541+0000 7fe1c5200640 1 --2- 172.21.15.7:0/1487482020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c00b5f50 0x7fe1c00cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.541+0000 7fe1c5200640 1 -- 172.21.15.7:0/1487482020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1c01614c0 msgr2=0x7fe1c00ccae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.541+0000 7fe1c5200640 1 --2- 172.21.15.7:0/1487482020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1c01614c0 0x7fe1c00ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.541+0000 7fe1c5200640 1 -- 172.21.15.7:0/1487482020 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c014d560 con 0x7fe1c00ce8d0 2024-09-10T21:42:18.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.542+0000 7fe1c5200640 1 --2- 172.21.15.7:0/1487482020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1c00ce8d0 0x7fe1c006e730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8000c00 tx=0x7fe1b8032c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:18.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.542+0000 7fad86a00640 1 -- 172.21.15.7:0/502100468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad8815df70 msgr2=0x7fad880c99f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.542+0000 7fad86a00640 1 --2- 172.21.15.7:0/502100468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad8815df70 0x7fad880c99f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.542+0000 7fad86a00640 1 -- 172.21.15.7:0/502100468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad880b1120 msgr2=0x7fad880c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.542+0000 7fad86a00640 1 --2- 172.21.15.7:0/502100468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad880b1120 0x7fad880c77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.542+0000 7fad86a00640 1 -- 172.21.15.7:0/502100468 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad881f8090 con 0x7fad880d22b0 2024-09-10T21:42:18.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.542+0000 7fad87e00640 1 --2- 172.21.15.7:0/502100468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad8815df70 0x7fad880c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:18.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.543+0000 7f3cdb200640 1 Processor -- start 2024-09-10T21:42:18.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.543+0000 7fe1bd600640 1 -- 172.21.15.7:0/1487482020 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8040070 con 0x7fe1c00ce8d0 2024-09-10T21:42:18.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.543+0000 7f6e7e600640 1 Processor -- start 2024-09-10T21:42:18.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.543+0000 7f3cdb200640 1 -- start start 2024-09-10T21:42:18.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.543+0000 7f0474c00640 1 -- 172.21.15.7:0/654227994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04700d0e70 msgr2=0x7f04700d1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.543+0000 7f0474c00640 1 --2- 172.21.15.7:0/654227994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04700d0e70 0x7f04700d1ee0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f04700c7830 tx=0x7f046802a780 comp rx=0 tx=0).stop 2024-09-10T21:42:18.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.543+0000 7fe1bd600640 1 -- 172.21.15.7:0/1487482020 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1b80038e0 con 0x7fe1c00ce8d0 2024-09-10T21:42:18.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.543+0000 7fe1bd600640 1 -- 172.21.15.7:0/1487482020 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b8005c90 con 0x7fe1c00ce8d0 2024-09-10T21:42:18.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.543+0000 7f0474c00640 1 -- 172.21.15.7:0/654227994 shutdown_connections 2024-09-10T21:42:18.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.543+0000 7f0474c00640 1 --2- 172.21.15.7:0/654227994 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04700d0e70 0x7f04700d1ee0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.543+0000 7f0474c00640 1 --2- 172.21.15.7:0/654227994 >> v2:172.21.15.7:3300/0 conn(0x7f047015dfd0 0x7f04700d4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.544+0000 7f0474c00640 1 -- 172.21.15.7:0/654227994 >> 172.21.15.7:0/654227994 conn(0x7f04700d3d50 msgr2=0x7f04700d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:18.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.543+0000 7f6e7e600640 1 -- start start 2024-09-10T21:42:18.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.544+0000 7fe1c7000640 1 -- 172.21.15.7:0/1487482020 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c00caf80 con 0x7fe1c00ce8d0 2024-09-10T21:42:18.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.544+0000 7fe1c7000640 1 -- 172.21.15.7:0/1487482020 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c015e7e0 con 0x7fe1c00ce8d0 2024-09-10T21:42:18.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.545+0000 7f6e7e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e700379f0 0x7f6e7017c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.545+0000 7f6e7e600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e7003a980 0x7f6e7017a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.545+0000 7f6e7e600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e700e2bc0 0x7f6e700dc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.545+0000 7f6e7e600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6e700e6c50 con 0x7f6e700379f0 2024-09-10T21:42:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.545+0000 7f6e7e600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6e700e6ad0 con 0x7f6e700e2bc0 2024-09-10T21:42:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.545+0000 7f6e7e600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f6e700e6dd0 con 0x7f6e7003a980 2024-09-10T21:42:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.545+0000 7f6e77e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e700e2bc0 0x7f6e700dc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.545+0000 7f6e77e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e700e2bc0 0x7f6e700dc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:49224/0 (socket says 172.21.15.7:49224) 2024-09-10T21:42:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.545+0000 7f6e77e00640 1 -- 172.21.15.7:0/3840713818 learned_addr learned my addr 172.21.15.7:0/3840713818 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.546+0000 7f6e77e00640 1 -- 172.21.15.7:0/3840713818 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e7003a980 msgr2=0x7f6e7017a720 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:42:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.546+0000 7f6e77e00640 1 --2- 172.21.15.7:0/3840713818 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e7003a980 0x7f6e7017a720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.546+0000 7f6e77400640 1 --2- 172.21.15.7:0/3840713818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e700379f0 0x7f6e7017c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.546+0000 7f6e77e00640 1 -- 172.21.15.7:0/3840713818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e700379f0 msgr2=0x7f6e7017c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.546+0000 7f6e77e00640 1 --2- 172.21.15.7:0/3840713818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e700379f0 0x7f6e7017c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.546+0000 7f6e77e00640 1 -- 172.21.15.7:0/3840713818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e700dd1e0 con 0x7f6e700e2bc0 2024-09-10T21:42:18.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.546+0000 7f3cdb200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cd4006b30 0x7f3cd417ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.546+0000 7fe1c7000640 1 -- 172.21.15.7:0/1487482020 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe198005290 con 0x7fe1c00ce8d0 2024-09-10T21:42:18.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.546+0000 7f6e77400640 1 --2- 172.21.15.7:0/3840713818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e700379f0 0x7f6e7017c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:18.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.546+0000 7efe37400640 1 -- 172.21.15.7:0/3712851096 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7efe38026080 con 0x7efe400cfaa0 2024-09-10T21:42:18.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.547+0000 7fad86a00640 1 --2- 172.21.15.7:0/502100468 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad880d22b0 0x7fad880c8200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fad7400c400 tx=0x7fad7400c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:18.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.546+0000 7f3cdb200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cd403a370 0x7f3cd40e5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.546+0000 7f3cdb200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd40e5ca0 0x7f3cd40d6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.546+0000 7f3cdb200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd4057db0 con 0x7f3cd4006b30 2024-09-10T21:42:18.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.546+0000 7f3cdb200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd4057c30 con 0x7f3cd40e5ca0 2024-09-10T21:42:18.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.546+0000 7f3cdb200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd4057f30 con 0x7f3cd403a370 2024-09-10T21:42:18.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.547+0000 7f6e77e00640 1 --2- 172.21.15.7:0/3840713818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e700e2bc0 0x7f6e700dc690 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00e9a0 tx=0x7f6e6c00eda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:18.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.547+0000 7f3cda800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd40e5ca0 0x7f3cd40d6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.547+0000 7f3cda800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd40e5ca0 0x7f3cd40d6550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:49234/0 (socket says 172.21.15.7:49234) 2024-09-10T21:42:18.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.547+0000 7f3cda800640 1 -- 172.21.15.7:0/1843231727 learned_addr learned my addr 172.21.15.7:0/1843231727 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:18.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.547+0000 7f3cd9e00640 1 --2- 172.21.15.7:0/1843231727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cd4006b30 0x7f3cd417ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.549+0000 7efe37400640 1 --2- 172.21.15.7:0/3712851096 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efe20077fc0 0x7efe2007a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.548+0000 7f3cd9400640 1 --2- 172.21.15.7:0/1843231727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cd403a370 0x7f3cd40e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.549+0000 7f3cd9e00640 1 -- 172.21.15.7:0/1843231727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cd403a370 msgr2=0x7f3cd40e5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.549+0000 7f3cd9e00640 1 --2- 172.21.15.7:0/1843231727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cd403a370 0x7f3cd40e5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.549+0000 7f3cd9e00640 1 -- 172.21.15.7:0/1843231727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd40e5ca0 msgr2=0x7f3cd40d6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.549+0000 7f3cd9e00640 1 --2- 172.21.15.7:0/1843231727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd40e5ca0 0x7f3cd40d6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.549+0000 7f3cd9e00640 1 -- 172.21.15.7:0/1843231727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd417d500 con 0x7f3cd4006b30 2024-09-10T21:42:18.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.549+0000 7f3cda800640 1 --2- 172.21.15.7:0/1843231727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd40e5ca0 0x7f3cd40d6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:42:18.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.548+0000 7fe1bd600640 1 -- 172.21.15.7:0/1487482020 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe1b8031770 con 0x7fe1c00ce8d0 2024-09-10T21:42:18.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.547+0000 7f0474c00640 1 -- 172.21.15.7:0/654227994 shutdown_connections 2024-09-10T21:42:18.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.547+0000 7f0474c00640 1 -- 172.21.15.7:0/654227994 wait complete. 2024-09-10T21:42:18.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.547+0000 7f6e6b400640 1 -- 172.21.15.7:0/3840713818 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c00cc60 con 0x7f6e700e2bc0 2024-09-10T21:42:18.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.547+0000 7f6e6b400640 1 -- 172.21.15.7:0/3840713818 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e6c00ce00 con 0x7f6e700e2bc0 2024-09-10T21:42:18.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.547+0000 7f6e6b400640 1 -- 172.21.15.7:0/3840713818 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c005280 con 0x7f6e700e2bc0 2024-09-10T21:42:18.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.548+0000 7f6e7e600640 1 -- 172.21.15.7:0/3840713818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e700e4090 con 0x7f6e700e2bc0 2024-09-10T21:42:18.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.548+0000 7f6e7e600640 1 -- 172.21.15.7:0/3840713818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e700e4930 con 0x7f6e700e2bc0 2024-09-10T21:42:18.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.549+0000 7f6e7e600640 1 -- 172.21.15.7:0/3840713818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e700e4b30 con 0x7f6e700e2bc0 2024-09-10T21:42:18.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.550+0000 7f3cd9e00640 1 --2- 172.21.15.7:0/1843231727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cd4006b30 0x7f3cd417ccd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8031000 tx=0x7f3cc80052e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:18.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.550+0000 7f0474c00640 1 Processor -- start 2024-09-10T21:42:18.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.551+0000 7f0474c00640 1 -- start start 2024-09-10T21:42:18.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.550+0000 7efe3ea00640 1 --2- 172.21.15.7:0/3712851096 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efe20077fc0 0x7efe2007a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.551+0000 7fe1bd600640 1 --2- 172.21.15.7:0/1487482020 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe1a0077ff0 0x7fe1a007a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.551+0000 7efe37400640 1 -- 172.21.15.7:0/3712851096 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7efe3809a210 con 0x7efe400cfaa0 2024-09-10T21:42:18.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.551+0000 7fe1bd600640 1 -- 172.21.15.7:0/1487482020 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe1b80bb8b0 con 0x7fe1c00ce8d0 2024-09-10T21:42:18.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.551+0000 7f3cd2000640 1 -- 172.21.15.7:0/1843231727 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8043bc0 con 0x7f3cd4006b30 2024-09-10T21:42:18.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.551+0000 7f3cd2000640 1 -- 172.21.15.7:0/1843231727 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3cc8043da0 con 0x7f3cd4006b30 2024-09-10T21:42:18.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.551+0000 7f3cd2000640 1 -- 172.21.15.7:0/1843231727 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cc8004290 con 0x7f3cd4006b30 2024-09-10T21:42:18.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.551+0000 7fe1c5c00640 1 --2- 172.21.15.7:0/1487482020 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe1a0077ff0 0x7fe1a007a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.551+0000 7efe3ea00640 1 --2- 172.21.15.7:0/3712851096 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efe20077fc0 0x7efe2007a480 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efe2c005fd0 tx=0x7efe2c00b590 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:18.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.551+0000 7efe37400640 1 -- 172.21.15.7:0/3712851096 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7efe380627a0 con 0x7efe400cfaa0 2024-09-10T21:42:18.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.552+0000 7f3cdb200640 1 -- 172.21.15.7:0/1843231727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd417d7e0 con 0x7f3cd4006b30 2024-09-10T21:42:18.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.552+0000 7f3cdb200640 1 -- 172.21.15.7:0/1843231727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd417d210 con 0x7f3cd4006b30 2024-09-10T21:42:18.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.552+0000 7fe1c5c00640 1 --2- 172.21.15.7:0/1487482020 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe1a0077ff0 0x7fe1a007a4b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac007fd0 tx=0x7fe1ac001370 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:18.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.552+0000 7fad7f400640 1 -- 172.21.15.7:0/502100468 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74017070 con 0x7fad880d22b0 2024-09-10T21:42:18.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.552+0000 7fad7f400640 1 -- 172.21.15.7:0/502100468 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad740035f0 con 0x7fad880d22b0 2024-09-10T21:42:18.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.552+0000 7fad7f400640 1 -- 172.21.15.7:0/502100468 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad74003a80 con 0x7fad880d22b0 2024-09-10T21:42:18.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.553+0000 7fad8ca00640 1 -- 172.21.15.7:0/502100468 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad881f8370 con 0x7fad880d22b0 2024-09-10T21:42:18.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.553+0000 7fad8ca00640 1 -- 172.21.15.7:0/502100468 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fad881f8930 con 0x7fad880d22b0 2024-09-10T21:42:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.553+0000 7f0474c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04700b2b50 0x7f04700b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.553+0000 7f0474c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04700d0e70 0x7f047006fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.553+0000 7f0474c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f047015dfd0 0x7f0470070380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.553+0000 7f0474c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f04700d25a0 con 0x7f047015dfd0 2024-09-10T21:42:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.553+0000 7f0474c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f04700d2420 con 0x7f04700d0e70 2024-09-10T21:42:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.553+0000 7f0474c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f04700d2720 con 0x7f04700b2b50 2024-09-10T21:42:18.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.552+0000 7f6e6b400640 1 -- 172.21.15.7:0/3840713818 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6e6c005420 con 0x7f6e700e2bc0 2024-09-10T21:42:18.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.554+0000 7f046f400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04700b2b50 0x7f04700b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.554+0000 7f046ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04700d0e70 0x7f047006fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.554+0000 7f046f400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04700b2b50 0x7f04700b4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:50386/0 (socket says 172.21.15.7:50386) 2024-09-10T21:42:18.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.554+0000 7f046f400640 1 -- 172.21.15.7:0/4141560308 learned_addr learned my addr 172.21.15.7:0/4141560308 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:18.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.553+0000 7fe1bd600640 1 -- 172.21.15.7:0/1487482020 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe1b804a060 con 0x7fe1c00ce8d0 2024-09-10T21:42:18.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.554+0000 7f3cdb200640 1 -- 172.21.15.7:0/1843231727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd40583f0 con 0x7f3cd4006b30 2024-09-10T21:42:18.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.554+0000 7fad8ca00640 1 -- 172.21.15.7:0/502100468 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad880b9c30 con 0x7fad880d22b0 2024-09-10T21:42:18.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.555+0000 7f046ea00640 1 -- 172.21.15.7:0/4141560308 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04700b2b50 msgr2=0x7f04700b4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.555+0000 7f046ea00640 1 --2- 172.21.15.7:0/4141560308 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04700b2b50 0x7f04700b4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.555+0000 7f046ea00640 1 -- 172.21.15.7:0/4141560308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f047015dfd0 msgr2=0x7f0470070380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.555+0000 7f046ea00640 1 --2- 172.21.15.7:0/4141560308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f047015dfd0 0x7f0470070380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.555+0000 7f046ea00640 1 -- 172.21.15.7:0/4141560308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0470201540 con 0x7f04700d0e70 2024-09-10T21:42:18.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.555+0000 7f046ea00640 1 --2- 172.21.15.7:0/4141560308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04700d0e70 0x7f047006fe40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f045c00c410 tx=0x7f045c00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:18.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.556+0000 7f6e6b400640 1 --2- 172.21.15.7:0/3840713818 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e58077fc0 0x7f6e5807a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.557+0000 7f6e6b400640 1 -- 172.21.15.7:0/3840713818 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6e6c09a760 con 0x7f6e700e2bc0 2024-09-10T21:42:18.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.557+0000 7f6e77400640 1 --2- 172.21.15.7:0/3840713818 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e58077fc0 0x7f6e5807a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.557+0000 7f3cd2000640 1 -- 172.21.15.7:0/1843231727 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3cc80047a0 con 0x7f3cd4006b30 2024-09-10T21:42:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.556+0000 7fad7f400640 1 -- 172.21.15.7:0/502100468 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fad74003c20 con 0x7fad880d22b0 2024-09-10T21:42:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.557+0000 7fad7f400640 1 --2- 172.21.15.7:0/502100468 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fad68077fc0 0x7fad6807a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.558+0000 7fad7f400640 1 -- 172.21.15.7:0/502100468 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fad74099240 con 0x7fad880d22b0 2024-09-10T21:42:18.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.558+0000 7f0467400640 1 -- 172.21.15.7:0/4141560308 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c006890 con 0x7f04700d0e70 2024-09-10T21:42:18.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.558+0000 7f0474c00640 1 -- 172.21.15.7:0/4141560308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0470201820 con 0x7f04700d0e70 2024-09-10T21:42:18.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.558+0000 7f0467400640 1 -- 172.21.15.7:0/4141560308 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f045c006b20 con 0x7f04700d0e70 2024-09-10T21:42:18.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.559+0000 7f6e6b400640 1 -- 172.21.15.7:0/3840713818 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6e700e4b30 con 0x7f6e700e2bc0 2024-09-10T21:42:18.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.559+0000 7f3cd2000640 1 --2- 172.21.15.7:0/1843231727 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3cbc078040 0x7f3cbc07a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.559+0000 7f3cd2000640 1 -- 172.21.15.7:0/1843231727 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3cc80c3540 con 0x7f3cd4006b30 2024-09-10T21:42:18.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.558+0000 7f0467400640 1 -- 172.21.15.7:0/4141560308 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045c006890 con 0x7f04700d0e70 2024-09-10T21:42:18.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.559+0000 7f0474c00640 1 -- 172.21.15.7:0/4141560308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04702011e0 con 0x7f04700d0e70 2024-09-10T21:42:18.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.560+0000 7f3cd9400640 1 --2- 172.21.15.7:0/1843231727 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3cbc078040 0x7f3cbc07a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.560+0000 7f6e77400640 1 --2- 172.21.15.7:0/3840713818 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e58077fc0 0x7f6e5807a480 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6e70006e40 tx=0x7f6e7806b330 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:18.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.560+0000 7fad87400640 1 --2- 172.21.15.7:0/502100468 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fad68077fc0 0x7fad6807a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.561+0000 7f3cd9400640 1 --2- 172.21.15.7:0/1843231727 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3cbc078040 0x7f3cbc07a500 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3cd404f440 tx=0x7f3ccc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:18.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.561+0000 7f0474c00640 1 -- 172.21.15.7:0/4141560308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0448005290 con 0x7f04700d0e70 2024-09-10T21:42:18.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.562+0000 7fad87400640 1 --2- 172.21.15.7:0/502100468 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fad68077fc0 0x7fad6807a480 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fad78033000 tx=0x7fad78004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:18.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.563+0000 7f0467400640 1 -- 172.21.15.7:0/4141560308 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f045c011590 con 0x7f04700d0e70 2024-09-10T21:42:18.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.563+0000 7fad7f400640 1 -- 172.21.15.7:0/502100468 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fad740010c0 con 0x7fad880d22b0 2024-09-10T21:42:18.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.563+0000 7f3d0ce00640 1 -- 172.21.15.7:0/1033334327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d0815e060 msgr2=0x7f3d080d4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.563+0000 7f3d0ce00640 1 --2- 172.21.15.7:0/1033334327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d0815e060 0x7f3d080d4680 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3d080d26d0 tx=0x7f3cf802a370 comp rx=0 tx=0).stop 2024-09-10T21:42:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.564+0000 7f3d0ce00640 1 -- 172.21.15.7:0/1033334327 shutdown_connections 2024-09-10T21:42:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.564+0000 7f3d0ce00640 1 --2- 172.21.15.7:0/1033334327 >> v2:172.21.15.7:3300/0 conn(0x7f3d080d0d80 0x7f3d080b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.564+0000 7f3d0ce00640 1 --2- 172.21.15.7:0/1033334327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d0815e060 0x7f3d080d4680 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.564+0000 7f3d0ce00640 1 -- 172.21.15.7:0/1033334327 >> 172.21.15.7:0/1033334327 conn(0x7f3d0809d270 msgr2=0x7f3d080d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.564+0000 7f3d0ce00640 1 -- 172.21.15.7:0/1033334327 shutdown_connections 2024-09-10T21:42:18.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.564+0000 7f3d0ce00640 1 -- 172.21.15.7:0/1033334327 wait complete. 2024-09-10T21:42:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.564+0000 7f0467400640 1 --2- 172.21.15.7:0/4141560308 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0450078170 0x7f045007a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.564+0000 7f0467400640 1 -- 172.21.15.7:0/4141560308 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f045c099290 con 0x7f04700d0e70 2024-09-10T21:42:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.565+0000 7f046f400640 1 --2- 172.21.15.7:0/4141560308 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0450078170 0x7f045007a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.566+0000 7f046f400640 1 --2- 172.21.15.7:0/4141560308 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0450078170 0x7f045007a630 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0460009c30 tx=0x7f0460005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.566+0000 7f3d0ce00640 1 Processor -- start 2024-09-10T21:42:18.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.567+0000 7f3d0ce00640 1 -- start start 2024-09-10T21:42:18.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.567+0000 7f3cd2000640 1 -- 172.21.15.7:0/1843231727 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3cc8051060 con 0x7f3cd4006b30 2024-09-10T21:42:18.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.569+0000 7f3d0ce00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d080b3060 0x7f3d081fe650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.569+0000 7f0467400640 1 -- 172.21.15.7:0/4141560308 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f045c061cf0 con 0x7f04700d0e70 2024-09-10T21:42:18.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.569+0000 7f3d0ce00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d080d0d80 0x7f3d08201b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.570+0000 7f3d0ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d0815e060 0x7f3d080b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.570+0000 7f3d05e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d080b3060 0x7f3d081fe650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.570+0000 7f3d05e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d080b3060 0x7f3d081fe650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:49260/0 (socket says 172.21.15.7:49260) 2024-09-10T21:42:18.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.570+0000 7f3d05e00640 1 -- 172.21.15.7:0/1619080247 learned_addr learned my addr 172.21.15.7:0/1619080247 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:18.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.570+0000 7f3d0ce00640 1 -- 172.21.15.7:0/1619080247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3d080b4360 con 0x7f3d0815e060 2024-09-10T21:42:18.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.571+0000 7f3d0ce00640 1 -- 172.21.15.7:0/1619080247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3d080b41e0 con 0x7f3d080b3060 2024-09-10T21:42:18.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.571+0000 7f3d0ce00640 1 -- 172.21.15.7:0/1619080247 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f3d080b44e0 con 0x7f3d080d0d80 2024-09-10T21:42:18.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.571+0000 7f3d05400640 1 --2- 172.21.15.7:0/1619080247 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d080d0d80 0x7f3d08201b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.572+0000 7f3d06800640 1 --2- 172.21.15.7:0/1619080247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d0815e060 0x7f3d080b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.572+0000 7f3d05e00640 1 -- 172.21.15.7:0/1619080247 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d080d0d80 msgr2=0x7f3d08201b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.572+0000 7f3d05e00640 1 --2- 172.21.15.7:0/1619080247 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d080d0d80 0x7f3d08201b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.572+0000 7f3d05e00640 1 -- 172.21.15.7:0/1619080247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d0815e060 msgr2=0x7f3d080b1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.573+0000 7f3d05e00640 1 --2- 172.21.15.7:0/1619080247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d0815e060 0x7f3d080b1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.573+0000 7f3d05e00640 1 -- 172.21.15.7:0/1619080247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d080c90e0 con 0x7f3d080b3060 2024-09-10T21:42:18.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.573+0000 7f3d06800640 1 --2- 172.21.15.7:0/1619080247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d0815e060 0x7f3d080b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:18.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.574+0000 7f3d05e00640 1 --2- 172.21.15.7:0/1619080247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d080b3060 0x7f3d081fe650 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8000c00 tx=0x7f3cf8033ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:18.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.576+0000 7f3cfe000640 1 -- 172.21.15.7:0/1619080247 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8037040 con 0x7f3d080b3060 2024-09-10T21:42:18.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.576+0000 7f3cfe000640 1 -- 172.21.15.7:0/1619080247 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3cf8042070 con 0x7f3d080b3060 2024-09-10T21:42:18.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.576+0000 7f3cfe000640 1 -- 172.21.15.7:0/1619080247 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8003850 con 0x7f3d080b3060 2024-09-10T21:42:18.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.576+0000 7f3d0ce00640 1 -- 172.21.15.7:0/1619080247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d080c93c0 con 0x7f3d080b3060 2024-09-10T21:42:18.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.576+0000 7f3d0ce00640 1 -- 172.21.15.7:0/1619080247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d080c9980 con 0x7f3d080b3060 2024-09-10T21:42:18.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.578+0000 7f3d0ce00640 1 -- 172.21.15.7:0/1619080247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ce0005290 con 0x7f3d080b3060 2024-09-10T21:42:18.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.581+0000 7f3cfe000640 1 -- 172.21.15.7:0/1619080247 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3cf803a030 con 0x7f3d080b3060 2024-09-10T21:42:18.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.583+0000 7f3cfe000640 1 --2- 172.21.15.7:0/1619080247 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3ce8077fc0 0x7f3ce807a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:18.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.583+0000 7f3cfe000640 1 -- 172.21.15.7:0/1619080247 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f3cf80ba190 con 0x7f3d080b3060 2024-09-10T21:42:18.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.584+0000 7f3d05400640 1 --2- 172.21.15.7:0/1619080247 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3ce8077fc0 0x7f3ce807a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:18.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.584+0000 7f3cfe000640 1 -- 172.21.15.7:0/1619080247 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3cf804b060 con 0x7f3d080b3060 2024-09-10T21:42:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.586+0000 7f3d05400640 1 --2- 172.21.15.7:0/1619080247 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3ce8077fc0 0x7f3ce807a480 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3d08074220 tx=0x7f3cf4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:18.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.903+0000 7faa7f200640 1 -- 172.21.15.7:0/1680297434 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7faa800aef70 con 0x7faa80070910 2024-09-10T21:42:18.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.904+0000 7faa76000640 1 -- 172.21.15.7:0/1680297434 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7faa78083b30 con 0x7faa80070910 2024-09-10T21:42:18.905 INFO:teuthology.orchestra.run.smithi007.stdout:64424509447 2024-09-10T21:42:18.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.907+0000 7faa7f200640 1 -- 172.21.15.7:0/1680297434 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7faa60077fa0 msgr2=0x7faa6007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.907+0000 7faa7f200640 1 --2- 172.21.15.7:0/1680297434 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7faa60077fa0 0x7faa6007a460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faa800cc220 tx=0x7faa7000a800 comp rx=0 tx=0).stop 2024-09-10T21:42:18.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.908+0000 7faa7f200640 1 -- 172.21.15.7:0/1680297434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa80070910 msgr2=0x7faa801e9c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.908+0000 7faa7f200640 1 --2- 172.21.15.7:0/1680297434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa80070910 0x7faa801e9c70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7faa78006010 tx=0x7faa780042e0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.908+0000 7faa7f200640 1 -- 172.21.15.7:0/1680297434 shutdown_connections 2024-09-10T21:42:18.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.909+0000 7faa7f200640 1 --2- 172.21.15.7:0/1680297434 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7faa60077fa0 0x7faa6007a460 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.909+0000 7faa7f200640 1 --2- 172.21.15.7:0/1680297434 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faa8015df40 0x7faa800cd830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.909+0000 7faa7f200640 1 --2- 172.21.15.7:0/1680297434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faa800b4560 0x7faa801e70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.909+0000 7faa7f200640 1 --2- 172.21.15.7:0/1680297434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faa80070910 0x7faa801e9c70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.909+0000 7faa7f200640 1 -- 172.21.15.7:0/1680297434 >> 172.21.15.7:0/1680297434 conn(0x7faa800cfa60 msgr2=0x7faa8015b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:18.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.909+0000 7faa7f200640 1 -- 172.21.15.7:0/1680297434 shutdown_connections 2024-09-10T21:42:18.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.910+0000 7faa7f200640 1 -- 172.21.15.7:0/1680297434 wait complete. 2024-09-10T21:42:18.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.0 2024-09-10T21:42:18.938 DEBUG:teuthology.parallel:result is None 2024-09-10T21:42:18.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.962+0000 7fe1c7000640 1 -- 172.21.15.7:0/1487482020 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fe198005850 con 0x7fe1c00ce8d0 2024-09-10T21:42:18.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.962+0000 7fe1bd600640 1 -- 172.21.15.7:0/1487482020 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe1b8084520 con 0x7fe1c00ce8d0 2024-09-10T21:42:18.964 INFO:teuthology.orchestra.run.smithi007.stdout:64424509449 2024-09-10T21:42:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.965+0000 7fe1c7000640 1 -- 172.21.15.7:0/1487482020 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe1a0077ff0 msgr2=0x7fe1a007a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.965+0000 7fe1c7000640 1 --2- 172.21.15.7:0/1487482020 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe1a0077ff0 0x7fe1a007a4b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fe1ac007fd0 tx=0x7fe1ac001370 comp rx=0 tx=0).stop 2024-09-10T21:42:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.965+0000 7fe1c7000640 1 -- 172.21.15.7:0/1487482020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1c00ce8d0 msgr2=0x7fe1c006e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:18.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.965+0000 7fe1c7000640 1 --2- 172.21.15.7:0/1487482020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1c00ce8d0 0x7fe1c006e730 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8000c00 tx=0x7fe1b8032c10 comp rx=0 tx=0).stop 2024-09-10T21:42:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.965+0000 7fe1c7000640 1 -- 172.21.15.7:0/1487482020 shutdown_connections 2024-09-10T21:42:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.965+0000 7fe1c7000640 1 --2- 172.21.15.7:0/1487482020 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe1a0077ff0 0x7fe1a007a4b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.965+0000 7fe1c7000640 1 --2- 172.21.15.7:0/1487482020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1c01614c0 0x7fe1c00ccae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.965+0000 7fe1c7000640 1 --2- 172.21.15.7:0/1487482020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1c00ce8d0 0x7fe1c006e730 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.965+0000 7fe1c7000640 1 --2- 172.21.15.7:0/1487482020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1c00b5f50 0x7fe1c00cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.965+0000 7fe1c7000640 1 -- 172.21.15.7:0/1487482020 >> 172.21.15.7:0/1487482020 conn(0x7fe1c00d3dc0 msgr2=0x7fe1c01e67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.965+0000 7fe1c7000640 1 -- 172.21.15.7:0/1487482020 shutdown_connections 2024-09-10T21:42:18.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:18.965+0000 7fe1c7000640 1 -- 172.21.15.7:0/1487482020 wait complete. 2024-09-10T21:42:18.983 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509448 got 64424509449 for osd.1 2024-09-10T21:42:18.983 DEBUG:teuthology.parallel:result is None 2024-09-10T21:42:19.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.003+0000 7efe44e00640 1 -- 172.21.15.7:0/3712851096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7efe400ce180 con 0x7efe400cfaa0 2024-09-10T21:42:19.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.003+0000 7efe37400640 1 -- 172.21.15.7:0/3712851096 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efe380671d0 con 0x7efe400cfaa0 2024-09-10T21:42:19.006 INFO:teuthology.orchestra.run.smithi007.stdout:60129542151 2024-09-10T21:42:19.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.007+0000 7efe44e00640 1 -- 172.21.15.7:0/3712851096 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efe20077fc0 msgr2=0x7efe2007a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.007+0000 7efe44e00640 1 --2- 172.21.15.7:0/3712851096 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efe20077fc0 0x7efe2007a480 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efe2c005fd0 tx=0x7efe2c00b590 comp rx=0 tx=0).stop 2024-09-10T21:42:19.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.007+0000 7efe44e00640 1 -- 172.21.15.7:0/3712851096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe400cfaa0 msgr2=0x7efe4015cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.007+0000 7efe44e00640 1 --2- 172.21.15.7:0/3712851096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe400cfaa0 0x7efe4015cd10 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efe3800ba50 tx=0x7efe3800bf10 comp rx=0 tx=0).stop 2024-09-10T21:42:19.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.007+0000 7efe44e00640 1 -- 172.21.15.7:0/3712851096 shutdown_connections 2024-09-10T21:42:19.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.007+0000 7efe44e00640 1 --2- 172.21.15.7:0/3712851096 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efe20077fc0 0x7efe2007a480 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.007+0000 7efe44e00640 1 --2- 172.21.15.7:0/3712851096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe400d2310 0x7efe400b6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.007+0000 7efe44e00640 1 --2- 172.21.15.7:0/3712851096 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe400d1920 0x7efe401fd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.007+0000 7efe44e00640 1 --2- 172.21.15.7:0/3712851096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe400cfaa0 0x7efe4015cd10 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.007+0000 7efe44e00640 1 -- 172.21.15.7:0/3712851096 >> 172.21.15.7:0/3712851096 conn(0x7efe400d3d50 msgr2=0x7efe400d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:19.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.008+0000 7efe44e00640 1 -- 172.21.15.7:0/3712851096 shutdown_connections 2024-09-10T21:42:19.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.008+0000 7efe44e00640 1 -- 172.21.15.7:0/3712851096 wait complete. 2024-09-10T21:42:19.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.009+0000 7f6e7e600640 1 -- 172.21.15.7:0/3840713818 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f6e700e4260 con 0x7f6e700e2bc0 2024-09-10T21:42:19.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.009+0000 7f6e6b400640 1 -- 172.21.15.7:0/3840713818 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6e6c0633d0 con 0x7f6e700e2bc0 2024-09-10T21:42:19.011 INFO:teuthology.orchestra.run.smithi007.stdout:64424509447 2024-09-10T21:42:19.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.012+0000 7f0474c00640 1 -- 172.21.15.7:0/4141560308 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f0448005580 con 0x7f04700d0e70 2024-09-10T21:42:19.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.013+0000 7f6e7e600640 1 -- 172.21.15.7:0/3840713818 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e58077fc0 msgr2=0x7f6e5807a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.013+0000 7f6e7e600640 1 --2- 172.21.15.7:0/3840713818 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e58077fc0 0x7f6e5807a480 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6e70006e40 tx=0x7f6e7806b330 comp rx=0 tx=0).stop 2024-09-10T21:42:19.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.013+0000 7f6e7e600640 1 -- 172.21.15.7:0/3840713818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e700e2bc0 msgr2=0x7f6e700dc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.013+0000 7f6e7e600640 1 --2- 172.21.15.7:0/3840713818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e700e2bc0 0x7f6e700dc690 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c00e9a0 tx=0x7f6e6c00eda0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.013+0000 7f0467400640 1 -- 172.21.15.7:0/4141560308 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f045c06a410 con 0x7f04700d0e70 2024-09-10T21:42:19.015 INFO:teuthology.orchestra.run.smithi007.stdout:64424509447 2024-09-10T21:42:19.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.013+0000 7f6e7e600640 1 -- 172.21.15.7:0/3840713818 shutdown_connections 2024-09-10T21:42:19.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.013+0000 7f6e7e600640 1 --2- 172.21.15.7:0/3840713818 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e58077fc0 0x7f6e5807a480 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.013+0000 7f6e7e600640 1 --2- 172.21.15.7:0/3840713818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e700e2bc0 0x7f6e700dc690 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.013+0000 7f6e7e600640 1 --2- 172.21.15.7:0/3840713818 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e7003a980 0x7f6e7017a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.013+0000 7f6e7e600640 1 --2- 172.21.15.7:0/3840713818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e700379f0 0x7f6e7017c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.013+0000 7f6e7e600640 1 -- 172.21.15.7:0/3840713818 >> 172.21.15.7:0/3840713818 conn(0x7f6e70055580 msgr2=0x7f6e7000e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:19.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.013+0000 7f6e7e600640 1 -- 172.21.15.7:0/3840713818 shutdown_connections 2024-09-10T21:42:19.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.013+0000 7f6e7e600640 1 -- 172.21.15.7:0/3840713818 wait complete. 2024-09-10T21:42:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.016+0000 7f0474c00640 1 -- 172.21.15.7:0/4141560308 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0450078170 msgr2=0x7f045007a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.016+0000 7f0474c00640 1 --2- 172.21.15.7:0/4141560308 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0450078170 0x7f045007a630 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0460009c30 tx=0x7f0460005f90 comp rx=0 tx=0).stop 2024-09-10T21:42:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.016+0000 7f0474c00640 1 -- 172.21.15.7:0/4141560308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04700d0e70 msgr2=0x7f047006fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.016+0000 7f0474c00640 1 --2- 172.21.15.7:0/4141560308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04700d0e70 0x7f047006fe40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f045c00c410 tx=0x7f045c00c8d0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.017+0000 7f0474c00640 1 -- 172.21.15.7:0/4141560308 shutdown_connections 2024-09-10T21:42:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.017+0000 7f0474c00640 1 --2- 172.21.15.7:0/4141560308 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0450078170 0x7f045007a630 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.017+0000 7f0474c00640 1 --2- 172.21.15.7:0/4141560308 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04700d0e70 0x7f047006fe40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.017+0000 7f0474c00640 1 --2- 172.21.15.7:0/4141560308 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f047015dfd0 0x7f0470070380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.017+0000 7f0474c00640 1 --2- 172.21.15.7:0/4141560308 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04700b2b50 0x7f04700b4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.017+0000 7f0474c00640 1 -- 172.21.15.7:0/4141560308 >> 172.21.15.7:0/4141560308 conn(0x7f04700d3d50 msgr2=0x7f047006d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.017+0000 7f0474c00640 1 -- 172.21.15.7:0/4141560308 shutdown_connections 2024-09-10T21:42:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.017+0000 7f0474c00640 1 -- 172.21.15.7:0/4141560308 wait complete. 2024-09-10T21:42:19.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.019+0000 7fad8ca00640 1 -- 172.21.15.7:0/502100468 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fad880b9950 con 0x7fad880d22b0 2024-09-10T21:42:19.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.020+0000 7fad7f400640 1 -- 172.21.15.7:0/502100468 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fad740617d0 con 0x7fad880d22b0 2024-09-10T21:42:19.022 INFO:teuthology.orchestra.run.smithi007.stdout:60129542150 2024-09-10T21:42:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.021+0000 7f3cdb200640 1 -- 172.21.15.7:0/1843231727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f3cd403a8c0 con 0x7f3cd4006b30 2024-09-10T21:42:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.021+0000 7f3cd2000640 1 -- 172.21.15.7:0/1843231727 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3cc808b9d0 con 0x7f3cd4006b30 2024-09-10T21:42:19.022 INFO:teuthology.orchestra.run.smithi007.stdout:64424509448 2024-09-10T21:42:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7f3cdb200640 1 -- 172.21.15.7:0/1843231727 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3cbc078040 msgr2=0x7f3cbc07a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7f3cdb200640 1 --2- 172.21.15.7:0/1843231727 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3cbc078040 0x7f3cbc07a500 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3cd404f440 tx=0x7f3ccc009040 comp rx=0 tx=0).stop 2024-09-10T21:42:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7f3cdb200640 1 -- 172.21.15.7:0/1843231727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cd4006b30 msgr2=0x7f3cd417ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7f3cdb200640 1 --2- 172.21.15.7:0/1843231727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cd4006b30 0x7f3cd417ccd0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8031000 tx=0x7f3cc80052e0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7fad8ca00640 1 -- 172.21.15.7:0/502100468 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fad68077fc0 msgr2=0x7fad6807a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7fad8ca00640 1 --2- 172.21.15.7:0/502100468 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fad68077fc0 0x7fad6807a480 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fad78033000 tx=0x7fad78004730 comp rx=0 tx=0).stop 2024-09-10T21:42:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7fad8ca00640 1 -- 172.21.15.7:0/502100468 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad880d22b0 msgr2=0x7fad880c8200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7fad8ca00640 1 --2- 172.21.15.7:0/502100468 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad880d22b0 0x7fad880c8200 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fad7400c400 tx=0x7fad7400c8c0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7f3cdb200640 1 -- 172.21.15.7:0/1843231727 shutdown_connections 2024-09-10T21:42:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7f3cdb200640 1 --2- 172.21.15.7:0/1843231727 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3cbc078040 0x7f3cbc07a500 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7f3cdb200640 1 --2- 172.21.15.7:0/1843231727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cd40e5ca0 0x7f3cd40d6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7f3cdb200640 1 --2- 172.21.15.7:0/1843231727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cd403a370 0x7f3cd40e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7f3cdb200640 1 --2- 172.21.15.7:0/1843231727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cd4006b30 0x7f3cd417ccd0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7f3cdb200640 1 -- 172.21.15.7:0/1843231727 >> 172.21.15.7:0/1843231727 conn(0x7f3cd4055550 msgr2=0x7f3cd403c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7fad8ca00640 1 -- 172.21.15.7:0/502100468 shutdown_connections 2024-09-10T21:42:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7fad8ca00640 1 --2- 172.21.15.7:0/502100468 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fad68077fc0 0x7fad6807a480 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7fad8ca00640 1 --2- 172.21.15.7:0/502100468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad8815df70 0x7fad880c99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7fad8ca00640 1 --2- 172.21.15.7:0/502100468 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad880d22b0 0x7fad880c8200 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7fad8ca00640 1 --2- 172.21.15.7:0/502100468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad880b1120 0x7fad880c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.024+0000 7fad8ca00640 1 -- 172.21.15.7:0/502100468 >> 172.21.15.7:0/502100468 conn(0x7fad880d3e80 msgr2=0x7fad880b8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.025+0000 7fad8ca00640 1 -- 172.21.15.7:0/502100468 shutdown_connections 2024-09-10T21:42:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.025+0000 7f3cdb200640 1 -- 172.21.15.7:0/1843231727 shutdown_connections 2024-09-10T21:42:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.025+0000 7fad8ca00640 1 -- 172.21.15.7:0/502100468 wait complete. 2024-09-10T21:42:19.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.025+0000 7f3cdb200640 1 -- 172.21.15.7:0/1843231727 wait complete. 2024-09-10T21:42:19.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.032+0000 7f3d0ce00640 1 -- 172.21.15.7:0/1619080247 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3ce0005850 con 0x7f3d080b3060 2024-09-10T21:42:19.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.033+0000 7f3cfe000640 1 -- 172.21.15.7:0/1619080247 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3cf8082e00 con 0x7f3d080b3060 2024-09-10T21:42:19.036 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.2 2024-09-10T21:42:19.036 INFO:teuthology.orchestra.run.smithi007.stdout:64424509448 2024-09-10T21:42:19.036 DEBUG:teuthology.parallel:result is None 2024-09-10T21:42:19.037 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.3 2024-09-10T21:42:19.037 DEBUG:teuthology.parallel:result is None 2024-09-10T21:42:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.037+0000 7f3d0ce00640 1 -- 172.21.15.7:0/1619080247 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3ce8077fc0 msgr2=0x7f3ce807a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.037+0000 7f3d0ce00640 1 --2- 172.21.15.7:0/1619080247 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3ce8077fc0 0x7f3ce807a480 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3d08074220 tx=0x7f3cf4009040 comp rx=0 tx=0).stop 2024-09-10T21:42:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.037+0000 7f3d0ce00640 1 -- 172.21.15.7:0/1619080247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d080b3060 msgr2=0x7f3d081fe650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.037+0000 7f3d0ce00640 1 --2- 172.21.15.7:0/1619080247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d080b3060 0x7f3d081fe650 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f3cf8000c00 tx=0x7f3cf8033ee0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.037+0000 7f3d0ce00640 1 -- 172.21.15.7:0/1619080247 shutdown_connections 2024-09-10T21:42:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.037+0000 7f3d0ce00640 1 --2- 172.21.15.7:0/1619080247 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3ce8077fc0 0x7f3ce807a480 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.037+0000 7f3d0ce00640 1 --2- 172.21.15.7:0/1619080247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d0815e060 0x7f3d080b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.037+0000 7f3d0ce00640 1 --2- 172.21.15.7:0/1619080247 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d080d0d80 0x7f3d08201b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.037+0000 7f3d0ce00640 1 --2- 172.21.15.7:0/1619080247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d080b3060 0x7f3d081fe650 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.037+0000 7f3d0ce00640 1 -- 172.21.15.7:0/1619080247 >> 172.21.15.7:0/1619080247 conn(0x7f3d0809d270 msgr2=0x7f3d080d3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.038+0000 7f3d0ce00640 1 -- 172.21.15.7:0/1619080247 shutdown_connections 2024-09-10T21:42:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.038+0000 7f3d0ce00640 1 -- 172.21.15.7:0/1619080247 wait complete. 2024-09-10T21:42:19.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.4 2024-09-10T21:42:19.045 DEBUG:teuthology.parallel:result is None 2024-09-10T21:42:19.049 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509448 for osd.5 2024-09-10T21:42:19.049 DEBUG:teuthology.parallel:result is None 2024-09-10T21:42:19.050 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542150 for osd.7 2024-09-10T21:42:19.050 DEBUG:teuthology.parallel:result is None 2024-09-10T21:42:19.056 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509448 for osd.6 2024-09-10T21:42:19.056 DEBUG:teuthology.parallel:result is None 2024-09-10T21:42:19.056 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:42:19.056 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:42:19.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.255+0000 7f6a2a400640 1 -- 172.21.15.7:0/633302591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a24073c70 msgr2=0x7f6a2415fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.255+0000 7f6a2a400640 1 --2- 172.21.15.7:0/633302591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a24073c70 0x7f6a2415fda0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6a2406db70 tx=0x7f6a1c02a8f0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.255+0000 7f6a2a400640 1 -- 172.21.15.7:0/633302591 shutdown_connections 2024-09-10T21:42:19.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.255+0000 7f6a2a400640 1 --2- 172.21.15.7:0/633302591 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a24073c70 0x7f6a2415fda0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.255+0000 7f6a2a400640 1 --2- 172.21.15.7:0/633302591 >> v2:172.21.15.133:3300/0 conn(0x7f6a240cd750 0x7f6a24073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.255+0000 7f6a2a400640 1 --2- 172.21.15.7:0/633302591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a2415bc90 0x7f6a240ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.255+0000 7f6a2a400640 1 -- 172.21.15.7:0/633302591 >> 172.21.15.7:0/633302591 conn(0x7f6a240b16d0 msgr2=0x7f6a240b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:19.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.255+0000 7f6a2a400640 1 -- 172.21.15.7:0/633302591 shutdown_connections 2024-09-10T21:42:19.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.255+0000 7f6a2a400640 1 -- 172.21.15.7:0/633302591 wait complete. 2024-09-10T21:42:19.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.257+0000 7f6a2a400640 1 Processor -- start 2024-09-10T21:42:19.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.257+0000 7f6a2a400640 1 -- start start 2024-09-10T21:42:19.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.258+0000 7f6a2a400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a24073c70 0x7f6a241f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:19.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.258+0000 7f6a2a400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a240cd750 0x7f6a240c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:19.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.258+0000 7f6a2a400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a2415bc90 0x7f6a241f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:19.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.258+0000 7f6a2a400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6a24160670 con 0x7f6a240cd750 2024-09-10T21:42:19.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.258+0000 7f6a2a400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6a241604f0 con 0x7f6a24073c70 2024-09-10T21:42:19.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.258+0000 7f6a2a400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f6a2406fcf0 con 0x7f6a2415bc90 2024-09-10T21:42:19.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.258+0000 7f6a22a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a240cd750 0x7f6a240c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:19.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.258+0000 7f6a22a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a240cd750 0x7f6a240c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54412/0 (socket says 172.21.15.7:54412) 2024-09-10T21:42:19.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.258+0000 7f6a22a00640 1 -- 172.21.15.7:0/1162814600 learned_addr learned my addr 172.21.15.7:0/1162814600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.258+0000 7f6a23400640 1 --2- 172.21.15.7:0/1162814600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a24073c70 0x7f6a241f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.258+0000 7f6a22a00640 1 -- 172.21.15.7:0/1162814600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a2415bc90 msgr2=0x7f6a241f7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.258+0000 7f6a22a00640 1 --2- 172.21.15.7:0/1162814600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a2415bc90 0x7f6a241f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.258+0000 7f6a22a00640 1 -- 172.21.15.7:0/1162814600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a24073c70 msgr2=0x7f6a241f3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.258+0000 7f6a22a00640 1 --2- 172.21.15.7:0/1162814600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a24073c70 0x7f6a241f3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.258+0000 7f6a22a00640 1 -- 172.21.15.7:0/1162814600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a240cb3a0 con 0x7f6a240cd750 2024-09-10T21:42:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.259+0000 7f6a22a00640 1 --2- 172.21.15.7:0/1162814600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a240cd750 0x7f6a240c97e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000e9f0 tx=0x7f6a1000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.259+0000 7f6a1b400640 1 -- 172.21.15.7:0/1162814600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1000cd20 con 0x7f6a240cd750 2024-09-10T21:42:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.259+0000 7f6a1b400640 1 -- 172.21.15.7:0/1162814600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a10010430 con 0x7f6a240cd750 2024-09-10T21:42:19.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.259+0000 7f6a2a400640 1 -- 172.21.15.7:0/1162814600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a240cb680 con 0x7f6a240cd750 2024-09-10T21:42:19.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.259+0000 7f6a1b400640 1 -- 172.21.15.7:0/1162814600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a10010840 con 0x7f6a240cd750 2024-09-10T21:42:19.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.259+0000 7f6a2a400640 1 -- 172.21.15.7:0/1162814600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a241f44a0 con 0x7f6a240cd750 2024-09-10T21:42:19.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.260+0000 7f6a2a400640 1 -- 172.21.15.7:0/1162814600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a24073730 con 0x7f6a240cd750 2024-09-10T21:42:19.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.261+0000 7f6a1b400640 1 -- 172.21.15.7:0/1162814600 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6a100109e0 con 0x7f6a240cd750 2024-09-10T21:42:19.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.262+0000 7f6a1b400640 1 --2- 172.21.15.7:0/1162814600 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6a04077f70 0x7f6a0407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:19.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.262+0000 7f6a1b400640 1 -- 172.21.15.7:0/1162814600 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f6a1009a870 con 0x7f6a240cd750 2024-09-10T21:42:19.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.263+0000 7f6a23400640 1 --2- 172.21.15.7:0/1162814600 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6a04077f70 0x7f6a0407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:19.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.264+0000 7f6a23400640 1 --2- 172.21.15.7:0/1162814600 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6a04077f70 0x7f6a0407a430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009960 tx=0x7f6a14005c90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:19.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.266+0000 7f6a1b400640 1 -- 172.21.15.7:0/1162814600 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6a10063420 con 0x7f6a240cd750 2024-09-10T21:42:19.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.458+0000 7f6a2a400640 1 -- 172.21.15.7:0/1162814600 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6a2415f5c0 con 0x7f6a04077f70 2024-09-10T21:42:19.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.463+0000 7f6a1b400640 1 -- 172.21.15.7:0/1162814600 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251825 (secure 0 0 0) 0x7f6a2415f5c0 con 0x7f6a04077f70 2024-09-10T21:42:19.467 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:42:19.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.467+0000 7f6a2a400640 1 -- 172.21.15.7:0/1162814600 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6a04077f70 msgr2=0x7f6a0407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.467+0000 7f6a2a400640 1 --2- 172.21.15.7:0/1162814600 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6a04077f70 0x7f6a0407a430 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6a14009960 tx=0x7f6a14005c90 comp rx=0 tx=0).stop 2024-09-10T21:42:19.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.467+0000 7f6a2a400640 1 -- 172.21.15.7:0/1162814600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a240cd750 msgr2=0x7f6a240c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.467+0000 7f6a2a400640 1 --2- 172.21.15.7:0/1162814600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a240cd750 0x7f6a240c97e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6a1000e9f0 tx=0x7f6a1000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.467+0000 7f6a2a400640 1 -- 172.21.15.7:0/1162814600 shutdown_connections 2024-09-10T21:42:19.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.467+0000 7f6a2a400640 1 --2- 172.21.15.7:0/1162814600 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6a04077f70 0x7f6a0407a430 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.467+0000 7f6a2a400640 1 --2- 172.21.15.7:0/1162814600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a2415bc90 0x7f6a241f7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.467+0000 7f6a2a400640 1 --2- 172.21.15.7:0/1162814600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a240cd750 0x7f6a240c97e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.467+0000 7f6a2a400640 1 --2- 172.21.15.7:0/1162814600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a24073c70 0x7f6a241f3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.467+0000 7f6a2a400640 1 -- 172.21.15.7:0/1162814600 >> 172.21.15.7:0/1162814600 conn(0x7f6a240b16d0 msgr2=0x7f6a24072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:19.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.467+0000 7f6a2a400640 1 -- 172.21.15.7:0/1162814600 shutdown_connections 2024-09-10T21:42:19.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.467+0000 7f6a2a400640 1 -- 172.21.15.7:0/1162814600 wait complete. 2024-09-10T21:42:19.484 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-09-10T21:42:17.857664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235724,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738996,"statfs":{"total":773094113280,"available":772852731904,"internally_reserved":0,"allocated":2924544,"data_stored":1763052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.183593"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782002+0000","last_change":"2024-09-10T21:42:09.712975+0000","last_active":"2024-09-10T21:42:11.782002+0000","last_peered":"2024-09-10T21:42:11.782002+0000","last_clean":"2024-09-10T21:42:11.782002+0000","last_became_active":"2024-09-10T21:42:09.712242+0000","last_became_peered":"2024-09-10T21:42:09.712242+0000","last_unstale":"2024-09-10T21:42:11.782002+0000","last_undegraded":"2024-09-10T21:42:11.782002+0000","last_fullsized":"2024-09-10T21:42:11.782002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:00.850316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695614+0000","last_change":"2024-09-10T21:42:09.697173+0000","last_active":"2024-09-10T21:42:10.695614+0000","last_peered":"2024-09-10T21:42:10.695614+0000","last_clean":"2024-09-10T21:42:10.695614+0000","last_became_active":"2024-09-10T21:42:09.696835+0000","last_became_peered":"2024-09-10T21:42:09.696835+0000","last_unstale":"2024-09-10T21:42:10.695614+0000","last_undegraded":"2024-09-10T21:42:10.695614+0000","last_fullsized":"2024-09-10T21:42:10.695614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:13.873778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975496+0000","last_change":"2024-09-10T21:42:09.693813+0000","last_active":"2024-09-10T21:42:11.975496+0000","last_peered":"2024-09-10T21:42:11.975496+0000","last_clean":"2024-09-10T21:42:11.975496+0000","last_became_active":"2024-09-10T21:42:09.693499+0000","last_became_peered":"2024-09-10T21:42:09.693499+0000","last_unstale":"2024-09-10T21:42:11.975496+0000","last_undegraded":"2024-09-10T21:42:11.975496+0000","last_fullsized":"2024-09-10T21:42:11.975496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:51.172800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702414+0000","last_change":"2024-09-10T21:42:09.698507+0000","last_active":"2024-09-10T21:42:10.702414+0000","last_peered":"2024-09-10T21:42:10.702414+0000","last_clean":"2024-09-10T21:42:10.702414+0000","last_became_active":"2024-09-10T21:42:09.697845+0000","last_became_peered":"2024-09-10T21:42:09.697845+0000","last_unstale":"2024-09-10T21:42:10.702414+0000","last_undegraded":"2024-09-10T21:42:10.702414+0000","last_fullsized":"2024-09-10T21:42:10.702414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:34.523014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781839+0000","last_change":"2024-09-10T21:42:09.703989+0000","last_active":"2024-09-10T21:42:11.781839+0000","last_peered":"2024-09-10T21:42:11.781839+0000","last_clean":"2024-09-10T21:42:11.781839+0000","last_became_active":"2024-09-10T21:42:09.703641+0000","last_became_peered":"2024-09-10T21:42:09.703641+0000","last_unstale":"2024-09-10T21:42:11.781839+0000","last_undegraded":"2024-09-10T21:42:11.781839+0000","last_fullsized":"2024-09-10T21:42:11.781839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:51.463128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701240+0000","last_change":"2024-09-10T21:42:09.691432+0000","last_active":"2024-09-10T21:42:10.701240+0000","last_peered":"2024-09-10T21:42:10.701240+0000","last_clean":"2024-09-10T21:42:10.701240+0000","last_became_active":"2024-09-10T21:42:09.691029+0000","last_became_peered":"2024-09-10T21:42:09.691029+0000","last_unstale":"2024-09-10T21:42:10.701240+0000","last_undegraded":"2024-09-10T21:42:10.701240+0000","last_fullsized":"2024-09-10T21:42:10.701240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:06.860570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781477+0000","last_change":"2024-09-10T21:42:09.716770+0000","last_active":"2024-09-10T21:42:11.781477+0000","last_peered":"2024-09-10T21:42:11.781477+0000","last_clean":"2024-09-10T21:42:11.781477+0000","last_became_active":"2024-09-10T21:42:09.716490+0000","last_became_peered":"2024-09-10T21:42:09.716490+0000","last_unstale":"2024-09-10T21:42:11.781477+0000","last_undegraded":"2024-09-10T21:42:11.781477+0000","last_fullsized":"2024-09-10T21:42:11.781477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:45.131035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249308+0000","last_change":"2024-09-10T21:42:09.695560+0000","last_active":"2024-09-10T21:42:12.249308+0000","last_peered":"2024-09-10T21:42:12.249308+0000","last_clean":"2024-09-10T21:42:12.249308+0000","last_became_active":"2024-09-10T21:42:09.695252+0000","last_became_peered":"2024-09-10T21:42:09.695252+0000","last_unstale":"2024-09-10T21:42:12.249308+0000","last_undegraded":"2024-09-10T21:42:12.249308+0000","last_fullsized":"2024-09-10T21:42:12.249308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:22:09.178909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782198+0000","last_change":"2024-09-10T21:42:09.713256+0000","last_active":"2024-09-10T21:42:11.782198+0000","last_peered":"2024-09-10T21:42:11.782198+0000","last_clean":"2024-09-10T21:42:11.782198+0000","last_became_active":"2024-09-10T21:42:09.712700+0000","last_became_peered":"2024-09-10T21:42:09.712700+0000","last_unstale":"2024-09-10T21:42:11.782198+0000","last_undegraded":"2024-09-10T21:42:11.782198+0000","last_fullsized":"2024-09-10T21:42:11.782198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:19.953355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685329+0000","last_change":"2024-09-10T21:42:09.702384+0000","last_active":"2024-09-10T21:42:11.685329+0000","last_peered":"2024-09-10T21:42:11.685329+0000","last_clean":"2024-09-10T21:42:11.685329+0000","last_became_active":"2024-09-10T21:42:09.698800+0000","last_became_peered":"2024-09-10T21:42:09.698800+0000","last_unstale":"2024-09-10T21:42:11.685329+0000","last_undegraded":"2024-09-10T21:42:11.685329+0000","last_fullsized":"2024-09-10T21:42:11.685329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:13.873778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685153+0000","last_change":"2024-09-10T21:42:09.704731+0000","last_active":"2024-09-10T21:42:11.685153+0000","last_peered":"2024-09-10T21:42:11.685153+0000","last_clean":"2024-09-10T21:42:11.685153+0000","last_became_active":"2024-09-10T21:42:09.698600+0000","last_became_peered":"2024-09-10T21:42:09.698600+0000","last_unstale":"2024-09-10T21:42:11.685153+0000","last_undegraded":"2024-09-10T21:42:11.685153+0000","last_fullsized":"2024-09-10T21:42:11.685153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:06.304338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701569+0000","last_change":"2024-09-10T21:42:09.710639+0000","last_active":"2024-09-10T21:42:10.701569+0000","last_peered":"2024-09-10T21:42:10.701569+0000","last_clean":"2024-09-10T21:42:10.701569+0000","last_became_active":"2024-09-10T21:42:09.710337+0000","last_became_peered":"2024-09-10T21:42:09.710337+0000","last_unstale":"2024-09-10T21:42:10.701569+0000","last_undegraded":"2024-09-10T21:42:10.701569+0000","last_fullsized":"2024-09-10T21:42:10.701569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:47.171015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249679+0000","last_change":"2024-09-10T21:42:09.706632+0000","last_active":"2024-09-10T21:42:12.249679+0000","last_peered":"2024-09-10T21:42:12.249679+0000","last_clean":"2024-09-10T21:42:12.249679+0000","last_became_active":"2024-09-10T21:42:09.706366+0000","last_became_peered":"2024-09-10T21:42:09.706366+0000","last_unstale":"2024-09-10T21:42:12.249679+0000","last_undegraded":"2024-09-10T21:42:12.249679+0000","last_fullsized":"2024-09-10T21:42:12.249679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:13.070168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.974972+0000","last_change":"2024-09-10T21:42:09.696596+0000","last_active":"2024-09-10T21:42:11.974972+0000","last_peered":"2024-09-10T21:42:11.974972+0000","last_clean":"2024-09-10T21:42:11.974972+0000","last_became_active":"2024-09-10T21:42:09.696292+0000","last_became_peered":"2024-09-10T21:42:09.696292+0000","last_unstale":"2024-09-10T21:42:11.974972+0000","last_undegraded":"2024-09-10T21:42:11.974972+0000","last_fullsized":"2024-09-10T21:42:11.974972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:49.235638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695329+0000","last_change":"2024-09-10T21:42:09.701498+0000","last_active":"2024-09-10T21:42:10.695329+0000","last_peered":"2024-09-10T21:42:10.695329+0000","last_clean":"2024-09-10T21:42:10.695329+0000","last_became_active":"2024-09-10T21:42:09.701149+0000","last_became_peered":"2024-09-10T21:42:09.701149+0000","last_unstale":"2024-09-10T21:42:10.695329+0000","last_undegraded":"2024-09-10T21:42:10.695329+0000","last_fullsized":"2024-09-10T21:42:10.695329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:46.339409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.700962+0000","last_change":"2024-09-10T21:42:09.694844+0000","last_active":"2024-09-10T21:42:10.700962+0000","last_peered":"2024-09-10T21:42:10.700962+0000","last_clean":"2024-09-10T21:42:10.700962+0000","last_became_active":"2024-09-10T21:42:09.694573+0000","last_became_peered":"2024-09-10T21:42:09.694573+0000","last_unstale":"2024-09-10T21:42:10.700962+0000","last_undegraded":"2024-09-10T21:42:10.700962+0000","last_fullsized":"2024-09-10T21:42:10.700962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:13.873778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702591+0000","last_change":"2024-09-10T21:42:09.707330+0000","last_active":"2024-09-10T21:42:10.702591+0000","last_peered":"2024-09-10T21:42:10.702591+0000","last_clean":"2024-09-10T21:42:10.702591+0000","last_became_active":"2024-09-10T21:42:09.707071+0000","last_became_peered":"2024-09-10T21:42:09.707071+0000","last_unstale":"2024-09-10T21:42:10.702591+0000","last_undegraded":"2024-09-10T21:42:10.702591+0000","last_fullsized":"2024-09-10T21:42:10.702591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:33.348954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428727+0000","last_change":"2024-09-10T21:42:09.709362+0000","last_active":"2024-09-10T21:42:11.428727+0000","last_peered":"2024-09-10T21:42:11.428727+0000","last_clean":"2024-09-10T21:42:11.428727+0000","last_became_active":"2024-09-10T21:42:09.707734+0000","last_became_peered":"2024-09-10T21:42:09.707734+0000","last_unstale":"2024-09-10T21:42:11.428727+0000","last_undegraded":"2024-09-10T21:42:11.428727+0000","last_fullsized":"2024-09-10T21:42:11.428727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:54.172897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782132+0000","last_change":"2024-09-10T21:42:09.709392+0000","last_active":"2024-09-10T21:42:11.782132+0000","last_peered":"2024-09-10T21:42:11.782132+0000","last_clean":"2024-09-10T21:42:11.782132+0000","last_became_active":"2024-09-10T21:42:09.708821+0000","last_became_peered":"2024-09-10T21:42:09.708821+0000","last_unstale":"2024-09-10T21:42:11.782132+0000","last_undegraded":"2024-09-10T21:42:11.782132+0000","last_fullsized":"2024-09-10T21:42:11.782132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:52.646979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685221+0000","last_change":"2024-09-10T21:42:09.711112+0000","last_active":"2024-09-10T21:42:11.685221+0000","last_peered":"2024-09-10T21:42:11.685221+0000","last_clean":"2024-09-10T21:42:11.685221+0000","last_became_active":"2024-09-10T21:42:09.710589+0000","last_became_peered":"2024-09-10T21:42:09.710589+0000","last_unstale":"2024-09-10T21:42:11.685221+0000","last_undegraded":"2024-09-10T21:42:11.685221+0000","last_fullsized":"2024-09-10T21:42:11.685221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:51.172800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975107+0000","last_change":"2024-09-10T21:42:09.707660+0000","last_active":"2024-09-10T21:42:11.975107+0000","last_peered":"2024-09-10T21:42:11.975107+0000","last_clean":"2024-09-10T21:42:11.975107+0000","last_became_active":"2024-09-10T21:42:09.707031+0000","last_became_peered":"2024-09-10T21:42:09.707031+0000","last_unstale":"2024-09-10T21:42:11.975107+0000","last_undegraded":"2024-09-10T21:42:11.975107+0000","last_fullsized":"2024-09-10T21:42:11.975107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:13.474744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701445+0000","last_change":"2024-09-10T21:42:09.708730+0000","last_active":"2024-09-10T21:42:10.701445+0000","last_peered":"2024-09-10T21:42:10.701445+0000","last_clean":"2024-09-10T21:42:10.701445+0000","last_became_active":"2024-09-10T21:42:09.708056+0000","last_became_peered":"2024-09-10T21:42:09.708056+0000","last_unstale":"2024-09-10T21:42:10.701445+0000","last_undegraded":"2024-09-10T21:42:10.701445+0000","last_fullsized":"2024-09-10T21:42:10.701445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:58.932679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249421+0000","last_change":"2024-09-10T21:42:09.705245+0000","last_active":"2024-09-10T21:42:12.249421+0000","last_peered":"2024-09-10T21:42:12.249421+0000","last_clean":"2024-09-10T21:42:12.249421+0000","last_became_active":"2024-09-10T21:42:09.699361+0000","last_became_peered":"2024-09-10T21:42:09.699361+0000","last_unstale":"2024-09-10T21:42:12.249421+0000","last_undegraded":"2024-09-10T21:42:12.249421+0000","last_fullsized":"2024-09-10T21:42:12.249421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:22.862332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701128+0000","last_change":"2024-09-10T21:42:09.704790+0000","last_active":"2024-09-10T21:42:10.701128+0000","last_peered":"2024-09-10T21:42:10.701128+0000","last_clean":"2024-09-10T21:42:10.701128+0000","last_became_active":"2024-09-10T21:42:09.699333+0000","last_became_peered":"2024-09-10T21:42:09.699333+0000","last_unstale":"2024-09-10T21:42:10.701128+0000","last_undegraded":"2024-09-10T21:42:10.701128+0000","last_fullsized":"2024-09-10T21:42:10.701128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:48.304330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249975+0000","last_change":"2024-09-10T21:42:09.712518+0000","last_active":"2024-09-10T21:42:12.249975+0000","last_peered":"2024-09-10T21:42:12.249975+0000","last_clean":"2024-09-10T21:42:12.249975+0000","last_became_active":"2024-09-10T21:42:09.711734+0000","last_became_peered":"2024-09-10T21:42:09.711734+0000","last_unstale":"2024-09-10T21:42:12.249975+0000","last_undegraded":"2024-09-10T21:42:12.249975+0000","last_fullsized":"2024-09-10T21:42:12.249975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:31.834184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685338+0000","last_change":"2024-09-10T21:42:09.713125+0000","last_active":"2024-09-10T21:42:11.685338+0000","last_peered":"2024-09-10T21:42:11.685338+0000","last_clean":"2024-09-10T21:42:11.685338+0000","last_became_active":"2024-09-10T21:42:09.711555+0000","last_became_peered":"2024-09-10T21:42:09.711555+0000","last_unstale":"2024-09-10T21:42:11.685338+0000","last_undegraded":"2024-09-10T21:42:11.685338+0000","last_fullsized":"2024-09-10T21:42:11.685338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:48.304330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695578+0000","last_change":"2024-09-10T21:42:09.698227+0000","last_active":"2024-09-10T21:42:10.695578+0000","last_peered":"2024-09-10T21:42:10.695578+0000","last_clean":"2024-09-10T21:42:10.695578+0000","last_became_active":"2024-09-10T21:42:09.695964+0000","last_became_peered":"2024-09-10T21:42:09.695964+0000","last_unstale":"2024-09-10T21:42:10.695578+0000","last_undegraded":"2024-09-10T21:42:10.695578+0000","last_fullsized":"2024-09-10T21:42:10.695578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:51.172800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249699+0000","last_change":"2024-09-10T21:42:09.699013+0000","last_active":"2024-09-10T21:42:12.249699+0000","last_peered":"2024-09-10T21:42:12.249699+0000","last_clean":"2024-09-10T21:42:12.249699+0000","last_became_active":"2024-09-10T21:42:09.698647+0000","last_became_peered":"2024-09-10T21:42:09.698647+0000","last_unstale":"2024-09-10T21:42:12.249699+0000","last_undegraded":"2024-09-10T21:42:12.249699+0000","last_fullsized":"2024-09-10T21:42:12.249699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:07.215352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975129+0000","last_change":"2024-09-10T21:42:09.697577+0000","last_active":"2024-09-10T21:42:11.975129+0000","last_peered":"2024-09-10T21:42:11.975129+0000","last_clean":"2024-09-10T21:42:11.975129+0000","last_became_active":"2024-09-10T21:42:09.696978+0000","last_became_peered":"2024-09-10T21:42:09.696978+0000","last_unstale":"2024-09-10T21:42:11.975129+0000","last_undegraded":"2024-09-10T21:42:11.975129+0000","last_fullsized":"2024-09-10T21:42:11.975129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:37.025454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701665+0000","last_change":"2024-09-10T21:42:09.697210+0000","last_active":"2024-09-10T21:42:10.701665+0000","last_peered":"2024-09-10T21:42:10.701665+0000","last_clean":"2024-09-10T21:42:10.701665+0000","last_became_active":"2024-09-10T21:42:09.696910+0000","last_became_peered":"2024-09-10T21:42:09.696910+0000","last_unstale":"2024-09-10T21:42:10.701665+0000","last_undegraded":"2024-09-10T21:42:10.701665+0000","last_fullsized":"2024-09-10T21:42:10.701665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:17.732763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428577+0000","last_change":"2024-09-10T21:42:09.702664+0000","last_active":"2024-09-10T21:42:11.428577+0000","last_peered":"2024-09-10T21:42:11.428577+0000","last_clean":"2024-09-10T21:42:11.428577+0000","last_became_active":"2024-09-10T21:42:09.700834+0000","last_became_peered":"2024-09-10T21:42:09.700834+0000","last_unstale":"2024-09-10T21:42:11.428577+0000","last_undegraded":"2024-09-10T21:42:11.428577+0000","last_fullsized":"2024-09-10T21:42:11.428577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:35.668254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.684927+0000","last_change":"2024-09-10T21:42:09.707098+0000","last_active":"2024-09-10T21:42:11.684927+0000","last_peered":"2024-09-10T21:42:11.684927+0000","last_clean":"2024-09-10T21:42:11.684927+0000","last_became_active":"2024-09-10T21:42:09.706747+0000","last_became_peered":"2024-09-10T21:42:09.706747+0000","last_unstale":"2024-09-10T21:42:11.684927+0000","last_undegraded":"2024-09-10T21:42:11.684927+0000","last_fullsized":"2024-09-10T21:42:11.684927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:38.013273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701269+0000","last_change":"2024-09-10T21:42:09.694901+0000","last_active":"2024-09-10T21:42:10.701269+0000","last_peered":"2024-09-10T21:42:10.701269+0000","last_clean":"2024-09-10T21:42:10.701269+0000","last_became_active":"2024-09-10T21:42:09.694509+0000","last_became_peered":"2024-09-10T21:42:09.694509+0000","last_unstale":"2024-09-10T21:42:10.701269+0000","last_undegraded":"2024-09-10T21:42:10.701269+0000","last_fullsized":"2024-09-10T21:42:10.701269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:57.099056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249846+0000","last_change":"2024-09-10T21:42:08.690816+0000","last_active":"2024-09-10T21:42:12.249846+0000","last_peered":"2024-09-10T21:42:12.249846+0000","last_clean":"2024-09-10T21:42:12.249846+0000","last_became_active":"2024-09-10T21:42:08.689282+0000","last_became_peered":"2024-09-10T21:42:08.689282+0000","last_unstale":"2024-09-10T21:42:12.249846+0000","last_undegraded":"2024-09-10T21:42:12.249846+0000","last_fullsized":"2024-09-10T21:42:12.249846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:17.328468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975323+0000","last_change":"2024-09-10T21:42:09.707990+0000","last_active":"2024-09-10T21:42:11.975323+0000","last_peered":"2024-09-10T21:42:11.975323+0000","last_clean":"2024-09-10T21:42:11.975323+0000","last_became_active":"2024-09-10T21:42:09.707404+0000","last_became_peered":"2024-09-10T21:42:09.707404+0000","last_unstale":"2024-09-10T21:42:11.975323+0000","last_undegraded":"2024-09-10T21:42:11.975323+0000","last_fullsized":"2024-09-10T21:42:11.975323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:40.556148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.684709+0000","last_change":"2024-09-10T21:42:08.697982+0000","last_active":"2024-09-10T21:42:11.684709+0000","last_peered":"2024-09-10T21:42:11.684709+0000","last_clean":"2024-09-10T21:42:11.684709+0000","last_became_active":"2024-09-10T21:42:08.696642+0000","last_became_peered":"2024-09-10T21:42:08.696642+0000","last_unstale":"2024-09-10T21:42:11.684709+0000","last_undegraded":"2024-09-10T21:42:11.684709+0000","last_fullsized":"2024-09-10T21:42:11.684709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:50.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249358+0000","last_change":"2024-09-10T21:42:08.704347+0000","last_active":"2024-09-10T21:42:12.249358+0000","last_peered":"2024-09-10T21:42:12.249358+0000","last_clean":"2024-09-10T21:42:12.249358+0000","last_became_active":"2024-09-10T21:42:08.701532+0000","last_became_peered":"2024-09-10T21:42:08.701532+0000","last_unstale":"2024-09-10T21:42:12.249358+0000","last_undegraded":"2024-09-10T21:42:12.249358+0000","last_fullsized":"2024-09-10T21:42:12.249358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:05.016544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249383+0000","last_change":"2024-09-10T21:42:09.698844+0000","last_active":"2024-09-10T21:42:12.249383+0000","last_peered":"2024-09-10T21:42:12.249383+0000","last_clean":"2024-09-10T21:42:12.249383+0000","last_became_active":"2024-09-10T21:42:09.698554+0000","last_became_peered":"2024-09-10T21:42:09.698554+0000","last_unstale":"2024-09-10T21:42:12.249383+0000","last_undegraded":"2024-09-10T21:42:12.249383+0000","last_fullsized":"2024-09-10T21:42:12.249383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:34:30.864687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701042+0000","last_change":"2024-09-10T21:42:08.685541+0000","last_active":"2024-09-10T21:42:10.701042+0000","last_peered":"2024-09-10T21:42:10.701042+0000","last_clean":"2024-09-10T21:42:10.701042+0000","last_became_active":"2024-09-10T21:42:08.685254+0000","last_became_peered":"2024-09-10T21:42:08.685254+0000","last_unstale":"2024-09-10T21:42:10.701042+0000","last_undegraded":"2024-09-10T21:42:10.701042+0000","last_fullsized":"2024-09-10T21:42:10.701042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:56.090844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695329+0000","last_change":"2024-09-10T21:42:09.695387+0000","last_active":"2024-09-10T21:42:10.695329+0000","last_peered":"2024-09-10T21:42:10.695329+0000","last_clean":"2024-09-10T21:42:10.695329+0000","last_became_active":"2024-09-10T21:42:09.695065+0000","last_became_peered":"2024-09-10T21:42:09.695065+0000","last_unstale":"2024-09-10T21:42:10.695329+0000","last_undegraded":"2024-09-10T21:42:10.695329+0000","last_fullsized":"2024-09-10T21:42:10.695329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:38.013273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685092+0000","last_change":"2024-09-10T21:42:08.692049+0000","last_active":"2024-09-10T21:42:11.685092+0000","last_peered":"2024-09-10T21:42:11.685092+0000","last_clean":"2024-09-10T21:42:11.685092+0000","last_became_active":"2024-09-10T21:42:08.691746+0000","last_became_peered":"2024-09-10T21:42:08.691746+0000","last_unstale":"2024-09-10T21:42:11.685092+0000","last_undegraded":"2024-09-10T21:42:11.685092+0000","last_fullsized":"2024-09-10T21:42:11.685092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:46.162803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702258+0000","last_change":"2024-09-10T21:42:09.703241+0000","last_active":"2024-09-10T21:42:10.702258+0000","last_peered":"2024-09-10T21:42:10.702258+0000","last_clean":"2024-09-10T21:42:10.702258+0000","last_became_active":"2024-09-10T21:42:09.702971+0000","last_became_peered":"2024-09-10T21:42:09.702971+0000","last_unstale":"2024-09-10T21:42:10.702258+0000","last_undegraded":"2024-09-10T21:42:10.702258+0000","last_fullsized":"2024-09-10T21:42:10.702258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:42.927407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702191+0000","last_change":"2024-09-10T21:42:08.698423+0000","last_active":"2024-09-10T21:42:10.702191+0000","last_peered":"2024-09-10T21:42:10.702191+0000","last_clean":"2024-09-10T21:42:10.702191+0000","last_became_active":"2024-09-10T21:42:08.696721+0000","last_became_peered":"2024-09-10T21:42:08.696721+0000","last_unstale":"2024-09-10T21:42:10.702191+0000","last_undegraded":"2024-09-10T21:42:10.702191+0000","last_fullsized":"2024-09-10T21:42:10.702191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:50.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975587+0000","last_change":"2024-09-10T21:42:09.713357+0000","last_active":"2024-09-10T21:42:11.975587+0000","last_peered":"2024-09-10T21:42:11.975587+0000","last_clean":"2024-09-10T21:42:11.975587+0000","last_became_active":"2024-09-10T21:42:09.713078+0000","last_became_peered":"2024-09-10T21:42:09.713078+0000","last_unstale":"2024-09-10T21:42:11.975587+0000","last_undegraded":"2024-09-10T21:42:11.975587+0000","last_fullsized":"2024-09-10T21:42:11.975587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:52.381281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695294+0000","last_change":"2024-09-10T21:42:08.696594+0000","last_active":"2024-09-10T21:42:10.695294+0000","last_peered":"2024-09-10T21:42:10.695294+0000","last_clean":"2024-09-10T21:42:10.695294+0000","last_became_active":"2024-09-10T21:42:08.696243+0000","last_became_peered":"2024-09-10T21:42:08.696243+0000","last_unstale":"2024-09-10T21:42:10.695294+0000","last_undegraded":"2024-09-10T21:42:10.695294+0000","last_fullsized":"2024-09-10T21:42:10.695294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:46.162803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701779+0000","last_change":"2024-09-10T21:42:09.704846+0000","last_active":"2024-09-10T21:42:10.701779+0000","last_peered":"2024-09-10T21:42:10.701779+0000","last_clean":"2024-09-10T21:42:10.701779+0000","last_became_active":"2024-09-10T21:42:09.704582+0000","last_became_peered":"2024-09-10T21:42:09.704582+0000","last_unstale":"2024-09-10T21:42:10.701779+0000","last_undegraded":"2024-09-10T21:42:10.701779+0000","last_fullsized":"2024-09-10T21:42:10.701779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:34.026318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685136+0000","last_change":"2024-09-10T21:42:08.694242+0000","last_active":"2024-09-10T21:42:11.685136+0000","last_peered":"2024-09-10T21:42:11.685136+0000","last_clean":"2024-09-10T21:42:11.685136+0000","last_became_active":"2024-09-10T21:42:08.693785+0000","last_became_peered":"2024-09-10T21:42:08.693785+0000","last_unstale":"2024-09-10T21:42:11.685136+0000","last_undegraded":"2024-09-10T21:42:11.685136+0000","last_fullsized":"2024-09-10T21:42:11.685136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:56.090844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781904+0000","last_change":"2024-09-10T21:42:09.709096+0000","last_active":"2024-09-10T21:42:11.781904+0000","last_peered":"2024-09-10T21:42:11.781904+0000","last_clean":"2024-09-10T21:42:11.781904+0000","last_became_active":"2024-09-10T21:42:09.708202+0000","last_became_peered":"2024-09-10T21:42:09.708202+0000","last_unstale":"2024-09-10T21:42:11.781904+0000","last_undegraded":"2024-09-10T21:42:11.781904+0000","last_fullsized":"2024-09-10T21:42:11.781904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:12:39.888041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975170+0000","last_change":"2024-09-10T21:42:08.694284+0000","last_active":"2024-09-10T21:42:11.975170+0000","last_peered":"2024-09-10T21:42:11.975170+0000","last_clean":"2024-09-10T21:42:11.975170+0000","last_became_active":"2024-09-10T21:42:08.693910+0000","last_became_peered":"2024-09-10T21:42:08.693910+0000","last_unstale":"2024-09-10T21:42:11.975170+0000","last_undegraded":"2024-09-10T21:42:11.975170+0000","last_fullsized":"2024-09-10T21:42:11.975170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:40.174118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975003+0000","last_change":"2024-09-10T21:42:09.712940+0000","last_active":"2024-09-10T21:42:11.975003+0000","last_peered":"2024-09-10T21:42:11.975003+0000","last_clean":"2024-09-10T21:42:11.975003+0000","last_became_active":"2024-09-10T21:42:09.709015+0000","last_became_peered":"2024-09-10T21:42:09.709015+0000","last_unstale":"2024-09-10T21:42:11.975003+0000","last_undegraded":"2024-09-10T21:42:11.975003+0000","last_fullsized":"2024-09-10T21:42:11.975003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:41.900249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701745+0000","last_change":"2024-09-10T21:42:08.689371+0000","last_active":"2024-09-10T21:42:10.701745+0000","last_peered":"2024-09-10T21:42:10.701745+0000","last_clean":"2024-09-10T21:42:10.701745+0000","last_became_active":"2024-09-10T21:42:08.688880+0000","last_became_peered":"2024-09-10T21:42:08.688880+0000","last_unstale":"2024-09-10T21:42:10.701745+0000","last_undegraded":"2024-09-10T21:42:10.701745+0000","last_fullsized":"2024-09-10T21:42:10.701745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:46.162803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781881+0000","last_change":"2024-09-10T21:42:09.713450+0000","last_active":"2024-09-10T21:42:11.781881+0000","last_peered":"2024-09-10T21:42:11.781881+0000","last_clean":"2024-09-10T21:42:11.781881+0000","last_became_active":"2024-09-10T21:42:09.713199+0000","last_became_peered":"2024-09-10T21:42:09.713199+0000","last_unstale":"2024-09-10T21:42:11.781881+0000","last_undegraded":"2024-09-10T21:42:11.781881+0000","last_fullsized":"2024-09-10T21:42:11.781881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:23.530344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.684649+0000","last_change":"2024-09-10T21:42:08.703573+0000","last_active":"2024-09-10T21:42:11.684649+0000","last_peered":"2024-09-10T21:42:11.684649+0000","last_clean":"2024-09-10T21:42:11.684649+0000","last_became_active":"2024-09-10T21:42:08.701580+0000","last_became_peered":"2024-09-10T21:42:08.701580+0000","last_unstale":"2024-09-10T21:42:11.684649+0000","last_undegraded":"2024-09-10T21:42:11.684649+0000","last_fullsized":"2024-09-10T21:42:11.684649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:16.471687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.684924+0000","last_change":"2024-09-10T21:42:09.700853+0000","last_active":"2024-09-10T21:42:11.684924+0000","last_peered":"2024-09-10T21:42:11.684924+0000","last_clean":"2024-09-10T21:42:11.684924+0000","last_became_active":"2024-09-10T21:42:09.700370+0000","last_became_peered":"2024-09-10T21:42:09.700370+0000","last_unstale":"2024-09-10T21:42:11.684924+0000","last_undegraded":"2024-09-10T21:42:11.684924+0000","last_fullsized":"2024-09-10T21:42:11.684924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:13.700823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428370+0000","last_change":"2024-09-10T21:42:08.697117+0000","last_active":"2024-09-10T21:42:11.428370+0000","last_peered":"2024-09-10T21:42:11.428370+0000","last_clean":"2024-09-10T21:42:11.428370+0000","last_became_active":"2024-09-10T21:42:08.695800+0000","last_became_peered":"2024-09-10T21:42:08.695800+0000","last_unstale":"2024-09-10T21:42:11.428370+0000","last_undegraded":"2024-09-10T21:42:11.428370+0000","last_fullsized":"2024-09-10T21:42:11.428370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:36.934204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781790+0000","last_change":"2024-09-10T21:42:09.707807+0000","last_active":"2024-09-10T21:42:11.781790+0000","last_peered":"2024-09-10T21:42:11.781790+0000","last_clean":"2024-09-10T21:42:11.781790+0000","last_became_active":"2024-09-10T21:42:09.707283+0000","last_became_peered":"2024-09-10T21:42:09.707283+0000","last_unstale":"2024-09-10T21:42:11.781790+0000","last_undegraded":"2024-09-10T21:42:11.781790+0000","last_fullsized":"2024-09-10T21:42:11.781790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:50.040141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701923+0000","last_change":"2024-09-10T21:42:08.693177+0000","last_active":"2024-09-10T21:42:10.701923+0000","last_peered":"2024-09-10T21:42:10.701923+0000","last_clean":"2024-09-10T21:42:10.701923+0000","last_became_active":"2024-09-10T21:42:08.692871+0000","last_became_peered":"2024-09-10T21:42:08.692871+0000","last_unstale":"2024-09-10T21:42:10.701923+0000","last_undegraded":"2024-09-10T21:42:10.701923+0000","last_fullsized":"2024-09-10T21:42:10.701923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:06.841814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781985+0000","last_change":"2024-09-10T21:42:09.710163+0000","last_active":"2024-09-10T21:42:11.781985+0000","last_peered":"2024-09-10T21:42:11.781985+0000","last_clean":"2024-09-10T21:42:11.781985+0000","last_became_active":"2024-09-10T21:42:09.709887+0000","last_became_peered":"2024-09-10T21:42:09.709887+0000","last_unstale":"2024-09-10T21:42:11.781985+0000","last_undegraded":"2024-09-10T21:42:11.781985+0000","last_fullsized":"2024-09-10T21:42:11.781985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:05:52.771916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428604+0000","last_change":"2024-09-10T21:42:08.686678+0000","last_active":"2024-09-10T21:42:11.428604+0000","last_peered":"2024-09-10T21:42:11.428604+0000","last_clean":"2024-09-10T21:42:11.428604+0000","last_became_active":"2024-09-10T21:42:08.686395+0000","last_became_peered":"2024-09-10T21:42:08.686395+0000","last_unstale":"2024-09-10T21:42:11.428604+0000","last_undegraded":"2024-09-10T21:42:11.428604+0000","last_fullsized":"2024-09-10T21:42:11.428604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:50.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782250+0000","last_change":"2024-09-10T21:42:09.714231+0000","last_active":"2024-09-10T21:42:11.782250+0000","last_peered":"2024-09-10T21:42:11.782250+0000","last_clean":"2024-09-10T21:42:11.782250+0000","last_became_active":"2024-09-10T21:42:09.713873+0000","last_became_peered":"2024-09-10T21:42:09.713873+0000","last_unstale":"2024-09-10T21:42:11.782250+0000","last_undegraded":"2024-09-10T21:42:11.782250+0000","last_fullsized":"2024-09-10T21:42:11.782250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:50.638349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695399+0000","last_change":"2024-09-10T21:42:08.694275+0000","last_active":"2024-09-10T21:42:10.695399+0000","last_peered":"2024-09-10T21:42:10.695399+0000","last_clean":"2024-09-10T21:42:10.695399+0000","last_became_active":"2024-09-10T21:42:08.694014+0000","last_became_peered":"2024-09-10T21:42:08.694014+0000","last_unstale":"2024-09-10T21:42:10.695399+0000","last_undegraded":"2024-09-10T21:42:10.695399+0000","last_fullsized":"2024-09-10T21:42:10.695399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:52.021550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702618+0000","last_change":"2024-09-10T21:42:09.698139+0000","last_active":"2024-09-10T21:42:10.702618+0000","last_peered":"2024-09-10T21:42:10.702618+0000","last_clean":"2024-09-10T21:42:10.702618+0000","last_became_active":"2024-09-10T21:42:09.697391+0000","last_became_peered":"2024-09-10T21:42:09.697391+0000","last_unstale":"2024-09-10T21:42:10.702618+0000","last_undegraded":"2024-09-10T21:42:10.702618+0000","last_fullsized":"2024-09-10T21:42:10.702618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:47.497053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428569+0000","last_change":"2024-09-10T21:42:08.691652+0000","last_active":"2024-09-10T21:42:11.428569+0000","last_peered":"2024-09-10T21:42:11.428569+0000","last_clean":"2024-09-10T21:42:11.428569+0000","last_became_active":"2024-09-10T21:42:08.691317+0000","last_became_peered":"2024-09-10T21:42:08.691317+0000","last_unstale":"2024-09-10T21:42:11.428569+0000","last_undegraded":"2024-09-10T21:42:11.428569+0000","last_fullsized":"2024-09-10T21:42:11.428569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:10.198175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781834+0000","last_change":"2024-09-10T21:42:09.720201+0000","last_active":"2024-09-10T21:42:11.781834+0000","last_peered":"2024-09-10T21:42:11.781834+0000","last_clean":"2024-09-10T21:42:11.781834+0000","last_became_active":"2024-09-10T21:42:09.719872+0000","last_became_peered":"2024-09-10T21:42:09.719872+0000","last_unstale":"2024-09-10T21:42:11.781834+0000","last_undegraded":"2024-09-10T21:42:11.781834+0000","last_fullsized":"2024-09-10T21:42:11.781834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:31.222577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781370+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:42:11.781370+0000","last_peered":"2024-09-10T21:42:11.781370+0000","last_clean":"2024-09-10T21:42:11.781370+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:42:11.781370+0000","last_undegraded":"2024-09-10T21:42:11.781370+0000","last_fullsized":"2024-09-10T21:42:11.781370+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701380+0000","last_change":"2024-09-10T21:42:08.698035+0000","last_active":"2024-09-10T21:42:10.701380+0000","last_peered":"2024-09-10T21:42:10.701380+0000","last_clean":"2024-09-10T21:42:10.701380+0000","last_became_active":"2024-09-10T21:42:08.695961+0000","last_became_peered":"2024-09-10T21:42:08.695961+0000","last_unstale":"2024-09-10T21:42:10.701380+0000","last_undegraded":"2024-09-10T21:42:10.701380+0000","last_fullsized":"2024-09-10T21:42:10.701380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:16.471687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249554+0000","last_change":"2024-09-10T21:42:09.692488+0000","last_active":"2024-09-10T21:42:12.249554+0000","last_peered":"2024-09-10T21:42:12.249554+0000","last_clean":"2024-09-10T21:42:12.249554+0000","last_became_active":"2024-09-10T21:42:09.691349+0000","last_became_peered":"2024-09-10T21:42:09.691349+0000","last_unstale":"2024-09-10T21:42:12.249554+0000","last_undegraded":"2024-09-10T21:42:12.249554+0000","last_fullsized":"2024-09-10T21:42:12.249554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:00.574924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781332+0000","last_change":"2024-09-10T21:42:08.706510+0000","last_active":"2024-09-10T21:42:11.781332+0000","last_peered":"2024-09-10T21:42:11.781332+0000","last_clean":"2024-09-10T21:42:11.781332+0000","last_became_active":"2024-09-10T21:42:08.706080+0000","last_became_peered":"2024-09-10T21:42:08.706080+0000","last_unstale":"2024-09-10T21:42:11.781332+0000","last_undegraded":"2024-09-10T21:42:11.781332+0000","last_fullsized":"2024-09-10T21:42:11.781332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:25.172297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975060+0000","last_change":"2024-09-10T21:42:09.698977+0000","last_active":"2024-09-10T21:42:11.975060+0000","last_peered":"2024-09-10T21:42:11.975060+0000","last_clean":"2024-09-10T21:42:11.975060+0000","last_became_active":"2024-09-10T21:42:09.698557+0000","last_became_peered":"2024-09-10T21:42:09.698557+0000","last_unstale":"2024-09-10T21:42:11.975060+0000","last_undegraded":"2024-09-10T21:42:11.975060+0000","last_fullsized":"2024-09-10T21:42:11.975060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:40.541602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428457+0000","last_change":"2024-09-10T21:42:08.689207+0000","last_active":"2024-09-10T21:42:11.428457+0000","last_peered":"2024-09-10T21:42:11.428457+0000","last_clean":"2024-09-10T21:42:11.428457+0000","last_became_active":"2024-09-10T21:42:08.688941+0000","last_became_peered":"2024-09-10T21:42:08.688941+0000","last_unstale":"2024-09-10T21:42:11.428457+0000","last_undegraded":"2024-09-10T21:42:11.428457+0000","last_fullsized":"2024-09-10T21:42:11.428457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:56.090844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695552+0000","last_change":"2024-09-10T21:42:09.688301+0000","last_active":"2024-09-10T21:42:10.695552+0000","last_peered":"2024-09-10T21:42:10.695552+0000","last_clean":"2024-09-10T21:42:10.695552+0000","last_became_active":"2024-09-10T21:42:09.687944+0000","last_became_peered":"2024-09-10T21:42:09.687944+0000","last_unstale":"2024-09-10T21:42:10.695552+0000","last_undegraded":"2024-09-10T21:42:10.695552+0000","last_fullsized":"2024-09-10T21:42:10.695552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:37.025454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701842+0000","last_change":"2024-09-10T21:42:08.698803+0000","last_active":"2024-09-10T21:42:10.701842+0000","last_peered":"2024-09-10T21:42:10.701842+0000","last_clean":"2024-09-10T21:42:10.701842+0000","last_became_active":"2024-09-10T21:42:08.697174+0000","last_became_peered":"2024-09-10T21:42:08.697174+0000","last_unstale":"2024-09-10T21:42:10.701842+0000","last_undegraded":"2024-09-10T21:42:10.701842+0000","last_fullsized":"2024-09-10T21:42:10.701842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:47.619335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702033+0000","last_change":"2024-09-10T21:42:09.688575+0000","last_active":"2024-09-10T21:42:10.702033+0000","last_peered":"2024-09-10T21:42:10.702033+0000","last_clean":"2024-09-10T21:42:10.702033+0000","last_became_active":"2024-09-10T21:42:09.688284+0000","last_became_peered":"2024-09-10T21:42:09.688284+0000","last_unstale":"2024-09-10T21:42:10.702033+0000","last_undegraded":"2024-09-10T21:42:10.702033+0000","last_fullsized":"2024-09-10T21:42:10.702033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:53.029762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695680+0000","last_change":"2024-09-10T21:42:08.698800+0000","last_active":"2024-09-10T21:42:10.695680+0000","last_peered":"2024-09-10T21:42:10.695680+0000","last_clean":"2024-09-10T21:42:10.695680+0000","last_became_active":"2024-09-10T21:42:08.696627+0000","last_became_peered":"2024-09-10T21:42:08.696627+0000","last_unstale":"2024-09-10T21:42:10.695680+0000","last_undegraded":"2024-09-10T21:42:10.695680+0000","last_fullsized":"2024-09-10T21:42:10.695680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:56.090844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701667+0000","last_change":"2024-09-10T21:42:09.700048+0000","last_active":"2024-09-10T21:42:10.701667+0000","last_peered":"2024-09-10T21:42:10.701667+0000","last_clean":"2024-09-10T21:42:10.701667+0000","last_became_active":"2024-09-10T21:42:09.699085+0000","last_became_peered":"2024-09-10T21:42:09.699085+0000","last_unstale":"2024-09-10T21:42:10.701667+0000","last_undegraded":"2024-09-10T21:42:10.701667+0000","last_fullsized":"2024-09-10T21:42:10.701667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:22.835437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782141+0000","last_change":"2024-09-10T21:42:08.704257+0000","last_active":"2024-09-10T21:42:11.782141+0000","last_peered":"2024-09-10T21:42:11.782141+0000","last_clean":"2024-09-10T21:42:11.782141+0000","last_became_active":"2024-09-10T21:42:08.703837+0000","last_became_peered":"2024-09-10T21:42:08.703837+0000","last_unstale":"2024-09-10T21:42:11.782141+0000","last_undegraded":"2024-09-10T21:42:11.782141+0000","last_fullsized":"2024-09-10T21:42:11.782141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:44.076555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695679+0000","last_change":"2024-09-10T21:42:09.695578+0000","last_active":"2024-09-10T21:42:10.695679+0000","last_peered":"2024-09-10T21:42:10.695679+0000","last_clean":"2024-09-10T21:42:10.695679+0000","last_became_active":"2024-09-10T21:42:09.693969+0000","last_became_peered":"2024-09-10T21:42:09.693969+0000","last_unstale":"2024-09-10T21:42:10.695679+0000","last_undegraded":"2024-09-10T21:42:10.695679+0000","last_fullsized":"2024-09-10T21:42:10.695679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:48.304330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695614+0000","last_change":"2024-09-10T21:42:08.699017+0000","last_active":"2024-09-10T21:42:10.695614+0000","last_peered":"2024-09-10T21:42:10.695614+0000","last_clean":"2024-09-10T21:42:10.695614+0000","last_became_active":"2024-09-10T21:42:08.698540+0000","last_became_peered":"2024-09-10T21:42:08.698540+0000","last_unstale":"2024-09-10T21:42:10.695614+0000","last_undegraded":"2024-09-10T21:42:10.695614+0000","last_fullsized":"2024-09-10T21:42:10.695614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:33.018106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.250013+0000","last_change":"2024-09-10T21:42:09.709063+0000","last_active":"2024-09-10T21:42:12.250013+0000","last_peered":"2024-09-10T21:42:12.250013+0000","last_clean":"2024-09-10T21:42:12.250013+0000","last_became_active":"2024-09-10T21:42:09.708726+0000","last_became_peered":"2024-09-10T21:42:09.708726+0000","last_unstale":"2024-09-10T21:42:12.250013+0000","last_undegraded":"2024-09-10T21:42:12.250013+0000","last_fullsized":"2024-09-10T21:42:12.250013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:00.508736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685182+0000","last_change":"2024-09-10T21:42:08.697045+0000","last_active":"2024-09-10T21:42:11.685182+0000","last_peered":"2024-09-10T21:42:11.685182+0000","last_clean":"2024-09-10T21:42:11.685182+0000","last_became_active":"2024-09-10T21:42:08.696483+0000","last_became_peered":"2024-09-10T21:42:08.696483+0000","last_unstale":"2024-09-10T21:42:11.685182+0000","last_undegraded":"2024-09-10T21:42:11.685182+0000","last_fullsized":"2024-09-10T21:42:11.685182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:34.907546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975478+0000","last_change":"2024-09-10T21:42:09.694168+0000","last_active":"2024-09-10T21:42:11.975478+0000","last_peered":"2024-09-10T21:42:11.975478+0000","last_clean":"2024-09-10T21:42:11.975478+0000","last_became_active":"2024-09-10T21:42:09.693779+0000","last_became_peered":"2024-09-10T21:42:09.693779+0000","last_unstale":"2024-09-10T21:42:11.975478+0000","last_undegraded":"2024-09-10T21:42:11.975478+0000","last_fullsized":"2024-09-10T21:42:11.975478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:24:55.155776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781313+0000","last_change":"2024-09-10T21:42:08.697319+0000","last_active":"2024-09-10T21:42:11.781313+0000","last_peered":"2024-09-10T21:42:11.781313+0000","last_clean":"2024-09-10T21:42:11.781313+0000","last_became_active":"2024-09-10T21:42:08.697039+0000","last_became_peered":"2024-09-10T21:42:08.697039+0000","last_unstale":"2024-09-10T21:42:11.781313+0000","last_undegraded":"2024-09-10T21:42:11.781313+0000","last_fullsized":"2024-09-10T21:42:11.781313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:48.264843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701623+0000","last_change":"2024-09-10T21:42:09.689554+0000","last_active":"2024-09-10T21:42:10.701623+0000","last_peered":"2024-09-10T21:42:10.701623+0000","last_clean":"2024-09-10T21:42:10.701623+0000","last_became_active":"2024-09-10T21:42:09.689041+0000","last_became_peered":"2024-09-10T21:42:09.689041+0000","last_unstale":"2024-09-10T21:42:10.701623+0000","last_undegraded":"2024-09-10T21:42:10.701623+0000","last_fullsized":"2024-09-10T21:42:10.701623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:48.627547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695404+0000","last_change":"2024-09-10T21:42:08.700219+0000","last_active":"2024-09-10T21:42:10.695404+0000","last_peered":"2024-09-10T21:42:10.695404+0000","last_clean":"2024-09-10T21:42:10.695404+0000","last_became_active":"2024-09-10T21:42:08.696072+0000","last_became_peered":"2024-09-10T21:42:08.696072+0000","last_unstale":"2024-09-10T21:42:10.695404+0000","last_undegraded":"2024-09-10T21:42:10.695404+0000","last_fullsized":"2024-09-10T21:42:10.695404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:50.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249562+0000","last_change":"2024-09-10T21:42:09.695584+0000","last_active":"2024-09-10T21:42:12.249562+0000","last_peered":"2024-09-10T21:42:12.249562+0000","last_clean":"2024-09-10T21:42:12.249562+0000","last_became_active":"2024-09-10T21:42:09.695169+0000","last_became_peered":"2024-09-10T21:42:09.695169+0000","last_unstale":"2024-09-10T21:42:12.249562+0000","last_undegraded":"2024-09-10T21:42:12.249562+0000","last_fullsized":"2024-09-10T21:42:12.249562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:07.671505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695459+0000","last_change":"2024-09-10T21:42:08.702389+0000","last_active":"2024-09-10T21:42:10.695459+0000","last_peered":"2024-09-10T21:42:10.695459+0000","last_clean":"2024-09-10T21:42:10.695459+0000","last_became_active":"2024-09-10T21:42:08.701730+0000","last_became_peered":"2024-09-10T21:42:08.701730+0000","last_unstale":"2024-09-10T21:42:10.695459+0000","last_undegraded":"2024-09-10T21:42:10.695459+0000","last_fullsized":"2024-09-10T21:42:10.695459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:47.619335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249655+0000","last_change":"2024-09-10T21:42:09.691569+0000","last_active":"2024-09-10T21:42:12.249655+0000","last_peered":"2024-09-10T21:42:12.249655+0000","last_clean":"2024-09-10T21:42:12.249655+0000","last_became_active":"2024-09-10T21:42:09.691270+0000","last_became_peered":"2024-09-10T21:42:09.691270+0000","last_unstale":"2024-09-10T21:42:12.249655+0000","last_undegraded":"2024-09-10T21:42:12.249655+0000","last_fullsized":"2024-09-10T21:42:12.249655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:13.594517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782013+0000","last_change":"2024-09-10T21:42:08.702598+0000","last_active":"2024-09-10T21:42:11.782013+0000","last_peered":"2024-09-10T21:42:11.782013+0000","last_clean":"2024-09-10T21:42:11.782013+0000","last_became_active":"2024-09-10T21:42:08.702352+0000","last_became_peered":"2024-09-10T21:42:08.702352+0000","last_unstale":"2024-09-10T21:42:11.782013+0000","last_undegraded":"2024-09-10T21:42:11.782013+0000","last_fullsized":"2024-09-10T21:42:11.782013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:45.571003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685024+0000","last_change":"2024-09-10T21:42:09.708756+0000","last_active":"2024-09-10T21:42:11.685024+0000","last_peered":"2024-09-10T21:42:11.685024+0000","last_clean":"2024-09-10T21:42:11.685024+0000","last_became_active":"2024-09-10T21:42:09.708458+0000","last_became_peered":"2024-09-10T21:42:09.708458+0000","last_unstale":"2024-09-10T21:42:11.685024+0000","last_undegraded":"2024-09-10T21:42:11.685024+0000","last_fullsized":"2024-09-10T21:42:11.685024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:37.025454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701552+0000","last_change":"2024-09-10T21:42:09.692810+0000","last_active":"2024-09-10T21:42:10.701552+0000","last_peered":"2024-09-10T21:42:10.701552+0000","last_clean":"2024-09-10T21:42:10.701552+0000","last_became_active":"2024-09-10T21:42:09.692537+0000","last_became_peered":"2024-09-10T21:42:09.692537+0000","last_unstale":"2024-09-10T21:42:10.701552+0000","last_undegraded":"2024-09-10T21:42:10.701552+0000","last_fullsized":"2024-09-10T21:42:10.701552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:55.273108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781669+0000","last_change":"2024-09-10T21:42:08.705305+0000","last_active":"2024-09-10T21:42:11.781669+0000","last_peered":"2024-09-10T21:42:11.781669+0000","last_clean":"2024-09-10T21:42:11.781669+0000","last_became_active":"2024-09-10T21:42:08.704991+0000","last_became_peered":"2024-09-10T21:42:08.704991+0000","last_unstale":"2024-09-10T21:42:11.781669+0000","last_undegraded":"2024-09-10T21:42:11.781669+0000","last_fullsized":"2024-09-10T21:42:11.781669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:59.418988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702644+0000","last_change":"2024-09-10T21:42:09.704028+0000","last_active":"2024-09-10T21:42:10.702644+0000","last_peered":"2024-09-10T21:42:10.702644+0000","last_clean":"2024-09-10T21:42:10.702644+0000","last_became_active":"2024-09-10T21:42:09.703608+0000","last_became_peered":"2024-09-10T21:42:09.703608+0000","last_unstale":"2024-09-10T21:42:10.702644+0000","last_undegraded":"2024-09-10T21:42:10.702644+0000","last_fullsized":"2024-09-10T21:42:10.702644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:11.626660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782020+0000","last_change":"2024-09-10T21:42:08.704448+0000","last_active":"2024-09-10T21:42:11.782020+0000","last_peered":"2024-09-10T21:42:11.782020+0000","last_clean":"2024-09-10T21:42:11.782020+0000","last_became_active":"2024-09-10T21:42:08.704156+0000","last_became_peered":"2024-09-10T21:42:08.704156+0000","last_unstale":"2024-09-10T21:42:11.782020+0000","last_undegraded":"2024-09-10T21:42:11.782020+0000","last_fullsized":"2024-09-10T21:42:11.782020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:07.043672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975512+0000","last_change":"2024-09-10T21:42:09.693556+0000","last_active":"2024-09-10T21:42:11.975512+0000","last_peered":"2024-09-10T21:42:11.975512+0000","last_clean":"2024-09-10T21:42:11.975512+0000","last_became_active":"2024-09-10T21:42:09.693330+0000","last_became_peered":"2024-09-10T21:42:09.693330+0000","last_unstale":"2024-09-10T21:42:11.975512+0000","last_undegraded":"2024-09-10T21:42:11.975512+0000","last_fullsized":"2024-09-10T21:42:11.975512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:26:39.608345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249807+0000","last_change":"2024-09-10T21:42:08.699157+0000","last_active":"2024-09-10T21:42:12.249807+0000","last_peered":"2024-09-10T21:42:12.249807+0000","last_clean":"2024-09-10T21:42:12.249807+0000","last_became_active":"2024-09-10T21:42:08.691408+0000","last_became_peered":"2024-09-10T21:42:08.691408+0000","last_unstale":"2024-09-10T21:42:12.249807+0000","last_undegraded":"2024-09-10T21:42:12.249807+0000","last_fullsized":"2024-09-10T21:42:12.249807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:52.464492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249860+0000","last_change":"2024-09-10T21:42:09.704294+0000","last_active":"2024-09-10T21:42:12.249860+0000","last_peered":"2024-09-10T21:42:12.249860+0000","last_clean":"2024-09-10T21:42:12.249860+0000","last_became_active":"2024-09-10T21:42:09.703607+0000","last_became_peered":"2024-09-10T21:42:09.703607+0000","last_unstale":"2024-09-10T21:42:12.249860+0000","last_undegraded":"2024-09-10T21:42:12.249860+0000","last_fullsized":"2024-09-10T21:42:12.249860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:58.276453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975594+0000","last_change":"2024-09-10T21:42:08.702462+0000","last_active":"2024-09-10T21:42:11.975594+0000","last_peered":"2024-09-10T21:42:11.975594+0000","last_clean":"2024-09-10T21:42:11.975594+0000","last_became_active":"2024-09-10T21:42:08.702103+0000","last_became_peered":"2024-09-10T21:42:08.702103+0000","last_unstale":"2024-09-10T21:42:11.975594+0000","last_undegraded":"2024-09-10T21:42:11.975594+0000","last_fullsized":"2024-09-10T21:42:11.975594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:30.979951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":72277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":72313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":72187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":815104,"data_stored":663193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":663137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:42:19.488 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:42:19.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.683+0000 7f442c600640 1 -- 172.21.15.7:0/255344936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44241619e0 msgr2=0x7f442415e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.683+0000 7f442c600640 1 --2- 172.21.15.7:0/255344936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44241619e0 0x7f442415e3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4418005fd0 tx=0x7f441802a8b0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.683+0000 7f442c600640 1 -- 172.21.15.7:0/255344936 shutdown_connections 2024-09-10T21:42:19.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.683+0000 7f442c600640 1 --2- 172.21.15.7:0/255344936 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f44240cc1c0 0x7f44240cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.683+0000 7f442c600640 1 --2- 172.21.15.7:0/255344936 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44241619e0 0x7f442415e3c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.683+0000 7f442c600640 1 -- 172.21.15.7:0/255344936 >> 172.21.15.7:0/255344936 conn(0x7f44240b16d0 msgr2=0x7f44240b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:19.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.683+0000 7f442c600640 1 -- 172.21.15.7:0/255344936 shutdown_connections 2024-09-10T21:42:19.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.683+0000 7f442c600640 1 -- 172.21.15.7:0/255344936 wait complete. 2024-09-10T21:42:19.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.685+0000 7f442c600640 1 Processor -- start 2024-09-10T21:42:19.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.685+0000 7f442c600640 1 -- start start 2024-09-10T21:42:19.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.686+0000 7f442c600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f44240cbd50 0x7f44240c5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:19.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.686+0000 7f442c600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44240cc1c0 0x7f44240b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:19.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.686+0000 7f442c600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44241619e0 0x7f44240b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:19.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.686+0000 7f442c600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f44240ce050 con 0x7f44241619e0 2024-09-10T21:42:19.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.686+0000 7f442c600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f44240cded0 con 0x7f44240cc1c0 2024-09-10T21:42:19.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.686+0000 7f442c600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4424161dc0 con 0x7f44240cbd50 2024-09-10T21:42:19.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.686+0000 7f442a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44241619e0 0x7f44240b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:19.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.686+0000 7f442a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44241619e0 0x7f44240b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54422/0 (socket says 172.21.15.7:54422) 2024-09-10T21:42:19.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.686+0000 7f442a600640 1 -- 172.21.15.7:0/2888369851 learned_addr learned my addr 172.21.15.7:0/2888369851 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:19.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.686+0000 7f4429200640 1 --2- 172.21.15.7:0/2888369851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44240cc1c0 0x7f44240b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:19.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.686+0000 7f4429c00640 1 --2- 172.21.15.7:0/2888369851 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f44240cbd50 0x7f44240c5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:19.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.687+0000 7f442a600640 1 -- 172.21.15.7:0/2888369851 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f44240cbd50 msgr2=0x7f44240c5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.687+0000 7f442a600640 1 --2- 172.21.15.7:0/2888369851 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f44240cbd50 0x7f44240c5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.687+0000 7f442a600640 1 -- 172.21.15.7:0/2888369851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44240cc1c0 msgr2=0x7f44240b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.687+0000 7f442a600640 1 --2- 172.21.15.7:0/2888369851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44240cc1c0 0x7f44240b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.687+0000 7f442a600640 1 -- 172.21.15.7:0/2888369851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44241f91e0 con 0x7f44241619e0 2024-09-10T21:42:19.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.687+0000 7f442a600640 1 --2- 172.21.15.7:0/2888369851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44241619e0 0x7f44240b2f10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f441c00e940 tx=0x7f441c00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:19.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.687+0000 7f4421600640 1 -- 172.21.15.7:0/2888369851 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c00cd00 con 0x7f44241619e0 2024-09-10T21:42:19.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.687+0000 7f4421600640 1 -- 172.21.15.7:0/2888369851 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f441c004020 con 0x7f44241619e0 2024-09-10T21:42:19.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.687+0000 7f442c600640 1 -- 172.21.15.7:0/2888369851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f442414a7f0 con 0x7f44241619e0 2024-09-10T21:42:19.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.687+0000 7f4421600640 1 -- 172.21.15.7:0/2888369851 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f441c00cea0 con 0x7f44241619e0 2024-09-10T21:42:19.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.687+0000 7f442c600640 1 -- 172.21.15.7:0/2888369851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f442414adb0 con 0x7f44241619e0 2024-09-10T21:42:19.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.689+0000 7f442c600640 1 -- 172.21.15.7:0/2888369851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44240cd750 con 0x7f44241619e0 2024-09-10T21:42:19.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.689+0000 7f4421600640 1 -- 172.21.15.7:0/2888369851 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f441c0035f0 con 0x7f44241619e0 2024-09-10T21:42:19.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.690+0000 7f4421600640 1 --2- 172.21.15.7:0/2888369851 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4404077f70 0x7f440407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:19.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.690+0000 7f4421600640 1 -- 172.21.15.7:0/2888369851 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f441c09a520 con 0x7f44241619e0 2024-09-10T21:42:19.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.691+0000 7f4429c00640 1 --2- 172.21.15.7:0/2888369851 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4404077f70 0x7f440407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:19.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.692+0000 7f4429c00640 1 --2- 172.21.15.7:0/2888369851 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4404077f70 0x7f440407a430 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4418034000 tx=0x7f4418004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:19.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.695+0000 7f4421600640 1 -- 172.21.15.7:0/2888369851 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f441c062a60 con 0x7f44241619e0 2024-09-10T21:42:19.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.887+0000 7f442c600640 1 -- 172.21.15.7:0/2888369851 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f44240b5b00 con 0x7f4404077f70 2024-09-10T21:42:19.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.892+0000 7f4421600640 1 -- 172.21.15.7:0/2888369851 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251826 (secure 0 0 0) 0x7f44240b5b00 con 0x7f4404077f70 2024-09-10T21:42:19.896 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:19.897 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:42:19.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.896+0000 7f442c600640 1 -- 172.21.15.7:0/2888369851 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4404077f70 msgr2=0x7f440407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.896+0000 7f442c600640 1 --2- 172.21.15.7:0/2888369851 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4404077f70 0x7f440407a430 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4418034000 tx=0x7f4418004ab0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.896+0000 7f442c600640 1 -- 172.21.15.7:0/2888369851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44241619e0 msgr2=0x7f44240b2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:19.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.896+0000 7f442c600640 1 --2- 172.21.15.7:0/2888369851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44241619e0 0x7f44240b2f10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f441c00e940 tx=0x7f441c00ee00 comp rx=0 tx=0).stop 2024-09-10T21:42:19.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.896+0000 7f442c600640 1 -- 172.21.15.7:0/2888369851 shutdown_connections 2024-09-10T21:42:19.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.896+0000 7f442c600640 1 --2- 172.21.15.7:0/2888369851 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4404077f70 0x7f440407a430 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.896+0000 7f442c600640 1 --2- 172.21.15.7:0/2888369851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f44240cc1c0 0x7f44240b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.896+0000 7f442c600640 1 --2- 172.21.15.7:0/2888369851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f44241619e0 0x7f44240b2f10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.896+0000 7f442c600640 1 --2- 172.21.15.7:0/2888369851 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f44240cbd50 0x7f44240c5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:19.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.896+0000 7f442c600640 1 -- 172.21.15.7:0/2888369851 >> 172.21.15.7:0/2888369851 conn(0x7f44240b16d0 msgr2=0x7f442415b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:19.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.896+0000 7f442c600640 1 -- 172.21.15.7:0/2888369851 shutdown_connections 2024-09-10T21:42:19.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:19.896+0000 7f442c600640 1 -- 172.21.15.7:0/2888369851 wait complete. 2024-09-10T21:42:19.914 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-10T21:42:19.858216+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":235724,"kb_used_data":2856,"kb_used_omap":12,"kb_used_meta":232691,"kb_avail":754738996,"statfs":{"total":773094113280,"available":772852731904,"internally_reserved":0,"allocated":2924544,"data_stored":1763052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":238276145},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002634"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782002+0000","last_change":"2024-09-10T21:42:09.712975+0000","last_active":"2024-09-10T21:42:11.782002+0000","last_peered":"2024-09-10T21:42:11.782002+0000","last_clean":"2024-09-10T21:42:11.782002+0000","last_became_active":"2024-09-10T21:42:09.712242+0000","last_became_peered":"2024-09-10T21:42:09.712242+0000","last_unstale":"2024-09-10T21:42:11.782002+0000","last_undegraded":"2024-09-10T21:42:11.782002+0000","last_fullsized":"2024-09-10T21:42:11.782002+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:00.850316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695614+0000","last_change":"2024-09-10T21:42:09.697173+0000","last_active":"2024-09-10T21:42:10.695614+0000","last_peered":"2024-09-10T21:42:10.695614+0000","last_clean":"2024-09-10T21:42:10.695614+0000","last_became_active":"2024-09-10T21:42:09.696835+0000","last_became_peered":"2024-09-10T21:42:09.696835+0000","last_unstale":"2024-09-10T21:42:10.695614+0000","last_undegraded":"2024-09-10T21:42:10.695614+0000","last_fullsized":"2024-09-10T21:42:10.695614+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:13.873778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975496+0000","last_change":"2024-09-10T21:42:09.693813+0000","last_active":"2024-09-10T21:42:11.975496+0000","last_peered":"2024-09-10T21:42:11.975496+0000","last_clean":"2024-09-10T21:42:11.975496+0000","last_became_active":"2024-09-10T21:42:09.693499+0000","last_became_peered":"2024-09-10T21:42:09.693499+0000","last_unstale":"2024-09-10T21:42:11.975496+0000","last_undegraded":"2024-09-10T21:42:11.975496+0000","last_fullsized":"2024-09-10T21:42:11.975496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:51.172800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702414+0000","last_change":"2024-09-10T21:42:09.698507+0000","last_active":"2024-09-10T21:42:10.702414+0000","last_peered":"2024-09-10T21:42:10.702414+0000","last_clean":"2024-09-10T21:42:10.702414+0000","last_became_active":"2024-09-10T21:42:09.697845+0000","last_became_peered":"2024-09-10T21:42:09.697845+0000","last_unstale":"2024-09-10T21:42:10.702414+0000","last_undegraded":"2024-09-10T21:42:10.702414+0000","last_fullsized":"2024-09-10T21:42:10.702414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:41:34.523014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781839+0000","last_change":"2024-09-10T21:42:09.703989+0000","last_active":"2024-09-10T21:42:11.781839+0000","last_peered":"2024-09-10T21:42:11.781839+0000","last_clean":"2024-09-10T21:42:11.781839+0000","last_became_active":"2024-09-10T21:42:09.703641+0000","last_became_peered":"2024-09-10T21:42:09.703641+0000","last_unstale":"2024-09-10T21:42:11.781839+0000","last_undegraded":"2024-09-10T21:42:11.781839+0000","last_fullsized":"2024-09-10T21:42:11.781839+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:27:51.463128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701240+0000","last_change":"2024-09-10T21:42:09.691432+0000","last_active":"2024-09-10T21:42:10.701240+0000","last_peered":"2024-09-10T21:42:10.701240+0000","last_clean":"2024-09-10T21:42:10.701240+0000","last_became_active":"2024-09-10T21:42:09.691029+0000","last_became_peered":"2024-09-10T21:42:09.691029+0000","last_unstale":"2024-09-10T21:42:10.701240+0000","last_undegraded":"2024-09-10T21:42:10.701240+0000","last_fullsized":"2024-09-10T21:42:10.701240+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:06.860570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781477+0000","last_change":"2024-09-10T21:42:09.716770+0000","last_active":"2024-09-10T21:42:11.781477+0000","last_peered":"2024-09-10T21:42:11.781477+0000","last_clean":"2024-09-10T21:42:11.781477+0000","last_became_active":"2024-09-10T21:42:09.716490+0000","last_became_peered":"2024-09-10T21:42:09.716490+0000","last_unstale":"2024-09-10T21:42:11.781477+0000","last_undegraded":"2024-09-10T21:42:11.781477+0000","last_fullsized":"2024-09-10T21:42:11.781477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:45.131035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249308+0000","last_change":"2024-09-10T21:42:09.695560+0000","last_active":"2024-09-10T21:42:12.249308+0000","last_peered":"2024-09-10T21:42:12.249308+0000","last_clean":"2024-09-10T21:42:12.249308+0000","last_became_active":"2024-09-10T21:42:09.695252+0000","last_became_peered":"2024-09-10T21:42:09.695252+0000","last_unstale":"2024-09-10T21:42:12.249308+0000","last_undegraded":"2024-09-10T21:42:12.249308+0000","last_fullsized":"2024-09-10T21:42:12.249308+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:22:09.178909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782198+0000","last_change":"2024-09-10T21:42:09.713256+0000","last_active":"2024-09-10T21:42:11.782198+0000","last_peered":"2024-09-10T21:42:11.782198+0000","last_clean":"2024-09-10T21:42:11.782198+0000","last_became_active":"2024-09-10T21:42:09.712700+0000","last_became_peered":"2024-09-10T21:42:09.712700+0000","last_unstale":"2024-09-10T21:42:11.782198+0000","last_undegraded":"2024-09-10T21:42:11.782198+0000","last_fullsized":"2024-09-10T21:42:11.782198+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:19.953355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685329+0000","last_change":"2024-09-10T21:42:09.702384+0000","last_active":"2024-09-10T21:42:11.685329+0000","last_peered":"2024-09-10T21:42:11.685329+0000","last_clean":"2024-09-10T21:42:11.685329+0000","last_became_active":"2024-09-10T21:42:09.698800+0000","last_became_peered":"2024-09-10T21:42:09.698800+0000","last_unstale":"2024-09-10T21:42:11.685329+0000","last_undegraded":"2024-09-10T21:42:11.685329+0000","last_fullsized":"2024-09-10T21:42:11.685329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:13.873778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685153+0000","last_change":"2024-09-10T21:42:09.704731+0000","last_active":"2024-09-10T21:42:11.685153+0000","last_peered":"2024-09-10T21:42:11.685153+0000","last_clean":"2024-09-10T21:42:11.685153+0000","last_became_active":"2024-09-10T21:42:09.698600+0000","last_became_peered":"2024-09-10T21:42:09.698600+0000","last_unstale":"2024-09-10T21:42:11.685153+0000","last_undegraded":"2024-09-10T21:42:11.685153+0000","last_fullsized":"2024-09-10T21:42:11.685153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:33:06.304338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701569+0000","last_change":"2024-09-10T21:42:09.710639+0000","last_active":"2024-09-10T21:42:10.701569+0000","last_peered":"2024-09-10T21:42:10.701569+0000","last_clean":"2024-09-10T21:42:10.701569+0000","last_became_active":"2024-09-10T21:42:09.710337+0000","last_became_peered":"2024-09-10T21:42:09.710337+0000","last_unstale":"2024-09-10T21:42:10.701569+0000","last_undegraded":"2024-09-10T21:42:10.701569+0000","last_fullsized":"2024-09-10T21:42:10.701569+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:47.171015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249679+0000","last_change":"2024-09-10T21:42:09.706632+0000","last_active":"2024-09-10T21:42:12.249679+0000","last_peered":"2024-09-10T21:42:12.249679+0000","last_clean":"2024-09-10T21:42:12.249679+0000","last_became_active":"2024-09-10T21:42:09.706366+0000","last_became_peered":"2024-09-10T21:42:09.706366+0000","last_unstale":"2024-09-10T21:42:12.249679+0000","last_undegraded":"2024-09-10T21:42:12.249679+0000","last_fullsized":"2024-09-10T21:42:12.249679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:13.070168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.974972+0000","last_change":"2024-09-10T21:42:09.696596+0000","last_active":"2024-09-10T21:42:11.974972+0000","last_peered":"2024-09-10T21:42:11.974972+0000","last_clean":"2024-09-10T21:42:11.974972+0000","last_became_active":"2024-09-10T21:42:09.696292+0000","last_became_peered":"2024-09-10T21:42:09.696292+0000","last_unstale":"2024-09-10T21:42:11.974972+0000","last_undegraded":"2024-09-10T21:42:11.974972+0000","last_fullsized":"2024-09-10T21:42:11.974972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:49.235638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695329+0000","last_change":"2024-09-10T21:42:09.701498+0000","last_active":"2024-09-10T21:42:10.695329+0000","last_peered":"2024-09-10T21:42:10.695329+0000","last_clean":"2024-09-10T21:42:10.695329+0000","last_became_active":"2024-09-10T21:42:09.701149+0000","last_became_peered":"2024-09-10T21:42:09.701149+0000","last_unstale":"2024-09-10T21:42:10.695329+0000","last_undegraded":"2024-09-10T21:42:10.695329+0000","last_fullsized":"2024-09-10T21:42:10.695329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:46.339409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.700962+0000","last_change":"2024-09-10T21:42:09.694844+0000","last_active":"2024-09-10T21:42:10.700962+0000","last_peered":"2024-09-10T21:42:10.700962+0000","last_clean":"2024-09-10T21:42:10.700962+0000","last_became_active":"2024-09-10T21:42:09.694573+0000","last_became_peered":"2024-09-10T21:42:09.694573+0000","last_unstale":"2024-09-10T21:42:10.700962+0000","last_undegraded":"2024-09-10T21:42:10.700962+0000","last_fullsized":"2024-09-10T21:42:10.700962+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:11:13.873778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702591+0000","last_change":"2024-09-10T21:42:09.707330+0000","last_active":"2024-09-10T21:42:10.702591+0000","last_peered":"2024-09-10T21:42:10.702591+0000","last_clean":"2024-09-10T21:42:10.702591+0000","last_became_active":"2024-09-10T21:42:09.707071+0000","last_became_peered":"2024-09-10T21:42:09.707071+0000","last_unstale":"2024-09-10T21:42:10.702591+0000","last_undegraded":"2024-09-10T21:42:10.702591+0000","last_fullsized":"2024-09-10T21:42:10.702591+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:33.348954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428727+0000","last_change":"2024-09-10T21:42:09.709362+0000","last_active":"2024-09-10T21:42:11.428727+0000","last_peered":"2024-09-10T21:42:11.428727+0000","last_clean":"2024-09-10T21:42:11.428727+0000","last_became_active":"2024-09-10T21:42:09.707734+0000","last_became_peered":"2024-09-10T21:42:09.707734+0000","last_unstale":"2024-09-10T21:42:11.428727+0000","last_undegraded":"2024-09-10T21:42:11.428727+0000","last_fullsized":"2024-09-10T21:42:11.428727+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:54.172897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782132+0000","last_change":"2024-09-10T21:42:09.709392+0000","last_active":"2024-09-10T21:42:11.782132+0000","last_peered":"2024-09-10T21:42:11.782132+0000","last_clean":"2024-09-10T21:42:11.782132+0000","last_became_active":"2024-09-10T21:42:09.708821+0000","last_became_peered":"2024-09-10T21:42:09.708821+0000","last_unstale":"2024-09-10T21:42:11.782132+0000","last_undegraded":"2024-09-10T21:42:11.782132+0000","last_fullsized":"2024-09-10T21:42:11.782132+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:52.646979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685221+0000","last_change":"2024-09-10T21:42:09.711112+0000","last_active":"2024-09-10T21:42:11.685221+0000","last_peered":"2024-09-10T21:42:11.685221+0000","last_clean":"2024-09-10T21:42:11.685221+0000","last_became_active":"2024-09-10T21:42:09.710589+0000","last_became_peered":"2024-09-10T21:42:09.710589+0000","last_unstale":"2024-09-10T21:42:11.685221+0000","last_undegraded":"2024-09-10T21:42:11.685221+0000","last_fullsized":"2024-09-10T21:42:11.685221+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:51.172800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975107+0000","last_change":"2024-09-10T21:42:09.707660+0000","last_active":"2024-09-10T21:42:11.975107+0000","last_peered":"2024-09-10T21:42:11.975107+0000","last_clean":"2024-09-10T21:42:11.975107+0000","last_became_active":"2024-09-10T21:42:09.707031+0000","last_became_peered":"2024-09-10T21:42:09.707031+0000","last_unstale":"2024-09-10T21:42:11.975107+0000","last_undegraded":"2024-09-10T21:42:11.975107+0000","last_fullsized":"2024-09-10T21:42:11.975107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:13.474744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701445+0000","last_change":"2024-09-10T21:42:09.708730+0000","last_active":"2024-09-10T21:42:10.701445+0000","last_peered":"2024-09-10T21:42:10.701445+0000","last_clean":"2024-09-10T21:42:10.701445+0000","last_became_active":"2024-09-10T21:42:09.708056+0000","last_became_peered":"2024-09-10T21:42:09.708056+0000","last_unstale":"2024-09-10T21:42:10.701445+0000","last_undegraded":"2024-09-10T21:42:10.701445+0000","last_fullsized":"2024-09-10T21:42:10.701445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:58.932679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249421+0000","last_change":"2024-09-10T21:42:09.705245+0000","last_active":"2024-09-10T21:42:12.249421+0000","last_peered":"2024-09-10T21:42:12.249421+0000","last_clean":"2024-09-10T21:42:12.249421+0000","last_became_active":"2024-09-10T21:42:09.699361+0000","last_became_peered":"2024-09-10T21:42:09.699361+0000","last_unstale":"2024-09-10T21:42:12.249421+0000","last_undegraded":"2024-09-10T21:42:12.249421+0000","last_fullsized":"2024-09-10T21:42:12.249421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:22.862332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701128+0000","last_change":"2024-09-10T21:42:09.704790+0000","last_active":"2024-09-10T21:42:10.701128+0000","last_peered":"2024-09-10T21:42:10.701128+0000","last_clean":"2024-09-10T21:42:10.701128+0000","last_became_active":"2024-09-10T21:42:09.699333+0000","last_became_peered":"2024-09-10T21:42:09.699333+0000","last_unstale":"2024-09-10T21:42:10.701128+0000","last_undegraded":"2024-09-10T21:42:10.701128+0000","last_fullsized":"2024-09-10T21:42:10.701128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:48.304330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249975+0000","last_change":"2024-09-10T21:42:09.712518+0000","last_active":"2024-09-10T21:42:12.249975+0000","last_peered":"2024-09-10T21:42:12.249975+0000","last_clean":"2024-09-10T21:42:12.249975+0000","last_became_active":"2024-09-10T21:42:09.711734+0000","last_became_peered":"2024-09-10T21:42:09.711734+0000","last_unstale":"2024-09-10T21:42:12.249975+0000","last_undegraded":"2024-09-10T21:42:12.249975+0000","last_fullsized":"2024-09-10T21:42:12.249975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:31.834184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685338+0000","last_change":"2024-09-10T21:42:09.713125+0000","last_active":"2024-09-10T21:42:11.685338+0000","last_peered":"2024-09-10T21:42:11.685338+0000","last_clean":"2024-09-10T21:42:11.685338+0000","last_became_active":"2024-09-10T21:42:09.711555+0000","last_became_peered":"2024-09-10T21:42:09.711555+0000","last_unstale":"2024-09-10T21:42:11.685338+0000","last_undegraded":"2024-09-10T21:42:11.685338+0000","last_fullsized":"2024-09-10T21:42:11.685338+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:48.304330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695578+0000","last_change":"2024-09-10T21:42:09.698227+0000","last_active":"2024-09-10T21:42:10.695578+0000","last_peered":"2024-09-10T21:42:10.695578+0000","last_clean":"2024-09-10T21:42:10.695578+0000","last_became_active":"2024-09-10T21:42:09.695964+0000","last_became_peered":"2024-09-10T21:42:09.695964+0000","last_unstale":"2024-09-10T21:42:10.695578+0000","last_undegraded":"2024-09-10T21:42:10.695578+0000","last_fullsized":"2024-09-10T21:42:10.695578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:51.172800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249699+0000","last_change":"2024-09-10T21:42:09.699013+0000","last_active":"2024-09-10T21:42:12.249699+0000","last_peered":"2024-09-10T21:42:12.249699+0000","last_clean":"2024-09-10T21:42:12.249699+0000","last_became_active":"2024-09-10T21:42:09.698647+0000","last_became_peered":"2024-09-10T21:42:09.698647+0000","last_unstale":"2024-09-10T21:42:12.249699+0000","last_undegraded":"2024-09-10T21:42:12.249699+0000","last_fullsized":"2024-09-10T21:42:12.249699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:33:07.215352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975129+0000","last_change":"2024-09-10T21:42:09.697577+0000","last_active":"2024-09-10T21:42:11.975129+0000","last_peered":"2024-09-10T21:42:11.975129+0000","last_clean":"2024-09-10T21:42:11.975129+0000","last_became_active":"2024-09-10T21:42:09.696978+0000","last_became_peered":"2024-09-10T21:42:09.696978+0000","last_unstale":"2024-09-10T21:42:11.975129+0000","last_undegraded":"2024-09-10T21:42:11.975129+0000","last_fullsized":"2024-09-10T21:42:11.975129+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:37.025454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701665+0000","last_change":"2024-09-10T21:42:09.697210+0000","last_active":"2024-09-10T21:42:10.701665+0000","last_peered":"2024-09-10T21:42:10.701665+0000","last_clean":"2024-09-10T21:42:10.701665+0000","last_became_active":"2024-09-10T21:42:09.696910+0000","last_became_peered":"2024-09-10T21:42:09.696910+0000","last_unstale":"2024-09-10T21:42:10.701665+0000","last_undegraded":"2024-09-10T21:42:10.701665+0000","last_fullsized":"2024-09-10T21:42:10.701665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:17.732763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428577+0000","last_change":"2024-09-10T21:42:09.702664+0000","last_active":"2024-09-10T21:42:11.428577+0000","last_peered":"2024-09-10T21:42:11.428577+0000","last_clean":"2024-09-10T21:42:11.428577+0000","last_became_active":"2024-09-10T21:42:09.700834+0000","last_became_peered":"2024-09-10T21:42:09.700834+0000","last_unstale":"2024-09-10T21:42:11.428577+0000","last_undegraded":"2024-09-10T21:42:11.428577+0000","last_fullsized":"2024-09-10T21:42:11.428577+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:35.668254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.684927+0000","last_change":"2024-09-10T21:42:09.707098+0000","last_active":"2024-09-10T21:42:11.684927+0000","last_peered":"2024-09-10T21:42:11.684927+0000","last_clean":"2024-09-10T21:42:11.684927+0000","last_became_active":"2024-09-10T21:42:09.706747+0000","last_became_peered":"2024-09-10T21:42:09.706747+0000","last_unstale":"2024-09-10T21:42:11.684927+0000","last_undegraded":"2024-09-10T21:42:11.684927+0000","last_fullsized":"2024-09-10T21:42:11.684927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:38.013273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701269+0000","last_change":"2024-09-10T21:42:09.694901+0000","last_active":"2024-09-10T21:42:10.701269+0000","last_peered":"2024-09-10T21:42:10.701269+0000","last_clean":"2024-09-10T21:42:10.701269+0000","last_became_active":"2024-09-10T21:42:09.694509+0000","last_became_peered":"2024-09-10T21:42:09.694509+0000","last_unstale":"2024-09-10T21:42:10.701269+0000","last_undegraded":"2024-09-10T21:42:10.701269+0000","last_fullsized":"2024-09-10T21:42:10.701269+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:57.099056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249846+0000","last_change":"2024-09-10T21:42:08.690816+0000","last_active":"2024-09-10T21:42:12.249846+0000","last_peered":"2024-09-10T21:42:12.249846+0000","last_clean":"2024-09-10T21:42:12.249846+0000","last_became_active":"2024-09-10T21:42:08.689282+0000","last_became_peered":"2024-09-10T21:42:08.689282+0000","last_unstale":"2024-09-10T21:42:12.249846+0000","last_undegraded":"2024-09-10T21:42:12.249846+0000","last_fullsized":"2024-09-10T21:42:12.249846+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:17.328468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975323+0000","last_change":"2024-09-10T21:42:09.707990+0000","last_active":"2024-09-10T21:42:11.975323+0000","last_peered":"2024-09-10T21:42:11.975323+0000","last_clean":"2024-09-10T21:42:11.975323+0000","last_became_active":"2024-09-10T21:42:09.707404+0000","last_became_peered":"2024-09-10T21:42:09.707404+0000","last_unstale":"2024-09-10T21:42:11.975323+0000","last_undegraded":"2024-09-10T21:42:11.975323+0000","last_fullsized":"2024-09-10T21:42:11.975323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:52:40.556148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.684709+0000","last_change":"2024-09-10T21:42:08.697982+0000","last_active":"2024-09-10T21:42:11.684709+0000","last_peered":"2024-09-10T21:42:11.684709+0000","last_clean":"2024-09-10T21:42:11.684709+0000","last_became_active":"2024-09-10T21:42:08.696642+0000","last_became_peered":"2024-09-10T21:42:08.696642+0000","last_unstale":"2024-09-10T21:42:11.684709+0000","last_undegraded":"2024-09-10T21:42:11.684709+0000","last_fullsized":"2024-09-10T21:42:11.684709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:50.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249358+0000","last_change":"2024-09-10T21:42:08.704347+0000","last_active":"2024-09-10T21:42:12.249358+0000","last_peered":"2024-09-10T21:42:12.249358+0000","last_clean":"2024-09-10T21:42:12.249358+0000","last_became_active":"2024-09-10T21:42:08.701532+0000","last_became_peered":"2024-09-10T21:42:08.701532+0000","last_unstale":"2024-09-10T21:42:12.249358+0000","last_undegraded":"2024-09-10T21:42:12.249358+0000","last_fullsized":"2024-09-10T21:42:12.249358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:05.016544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249383+0000","last_change":"2024-09-10T21:42:09.698844+0000","last_active":"2024-09-10T21:42:12.249383+0000","last_peered":"2024-09-10T21:42:12.249383+0000","last_clean":"2024-09-10T21:42:12.249383+0000","last_became_active":"2024-09-10T21:42:09.698554+0000","last_became_peered":"2024-09-10T21:42:09.698554+0000","last_unstale":"2024-09-10T21:42:12.249383+0000","last_undegraded":"2024-09-10T21:42:12.249383+0000","last_fullsized":"2024-09-10T21:42:12.249383+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:34:30.864687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701042+0000","last_change":"2024-09-10T21:42:08.685541+0000","last_active":"2024-09-10T21:42:10.701042+0000","last_peered":"2024-09-10T21:42:10.701042+0000","last_clean":"2024-09-10T21:42:10.701042+0000","last_became_active":"2024-09-10T21:42:08.685254+0000","last_became_peered":"2024-09-10T21:42:08.685254+0000","last_unstale":"2024-09-10T21:42:10.701042+0000","last_undegraded":"2024-09-10T21:42:10.701042+0000","last_fullsized":"2024-09-10T21:42:10.701042+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:56.090844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695329+0000","last_change":"2024-09-10T21:42:09.695387+0000","last_active":"2024-09-10T21:42:10.695329+0000","last_peered":"2024-09-10T21:42:10.695329+0000","last_clean":"2024-09-10T21:42:10.695329+0000","last_became_active":"2024-09-10T21:42:09.695065+0000","last_became_peered":"2024-09-10T21:42:09.695065+0000","last_unstale":"2024-09-10T21:42:10.695329+0000","last_undegraded":"2024-09-10T21:42:10.695329+0000","last_fullsized":"2024-09-10T21:42:10.695329+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:38.013273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685092+0000","last_change":"2024-09-10T21:42:08.692049+0000","last_active":"2024-09-10T21:42:11.685092+0000","last_peered":"2024-09-10T21:42:11.685092+0000","last_clean":"2024-09-10T21:42:11.685092+0000","last_became_active":"2024-09-10T21:42:08.691746+0000","last_became_peered":"2024-09-10T21:42:08.691746+0000","last_unstale":"2024-09-10T21:42:11.685092+0000","last_undegraded":"2024-09-10T21:42:11.685092+0000","last_fullsized":"2024-09-10T21:42:11.685092+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:46.162803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702258+0000","last_change":"2024-09-10T21:42:09.703241+0000","last_active":"2024-09-10T21:42:10.702258+0000","last_peered":"2024-09-10T21:42:10.702258+0000","last_clean":"2024-09-10T21:42:10.702258+0000","last_became_active":"2024-09-10T21:42:09.702971+0000","last_became_peered":"2024-09-10T21:42:09.702971+0000","last_unstale":"2024-09-10T21:42:10.702258+0000","last_undegraded":"2024-09-10T21:42:10.702258+0000","last_fullsized":"2024-09-10T21:42:10.702258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:42.927407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702191+0000","last_change":"2024-09-10T21:42:08.698423+0000","last_active":"2024-09-10T21:42:10.702191+0000","last_peered":"2024-09-10T21:42:10.702191+0000","last_clean":"2024-09-10T21:42:10.702191+0000","last_became_active":"2024-09-10T21:42:08.696721+0000","last_became_peered":"2024-09-10T21:42:08.696721+0000","last_unstale":"2024-09-10T21:42:10.702191+0000","last_undegraded":"2024-09-10T21:42:10.702191+0000","last_fullsized":"2024-09-10T21:42:10.702191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:50.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975587+0000","last_change":"2024-09-10T21:42:09.713357+0000","last_active":"2024-09-10T21:42:11.975587+0000","last_peered":"2024-09-10T21:42:11.975587+0000","last_clean":"2024-09-10T21:42:11.975587+0000","last_became_active":"2024-09-10T21:42:09.713078+0000","last_became_peered":"2024-09-10T21:42:09.713078+0000","last_unstale":"2024-09-10T21:42:11.975587+0000","last_undegraded":"2024-09-10T21:42:11.975587+0000","last_fullsized":"2024-09-10T21:42:11.975587+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:52.381281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695294+0000","last_change":"2024-09-10T21:42:08.696594+0000","last_active":"2024-09-10T21:42:10.695294+0000","last_peered":"2024-09-10T21:42:10.695294+0000","last_clean":"2024-09-10T21:42:10.695294+0000","last_became_active":"2024-09-10T21:42:08.696243+0000","last_became_peered":"2024-09-10T21:42:08.696243+0000","last_unstale":"2024-09-10T21:42:10.695294+0000","last_undegraded":"2024-09-10T21:42:10.695294+0000","last_fullsized":"2024-09-10T21:42:10.695294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:46.162803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701779+0000","last_change":"2024-09-10T21:42:09.704846+0000","last_active":"2024-09-10T21:42:10.701779+0000","last_peered":"2024-09-10T21:42:10.701779+0000","last_clean":"2024-09-10T21:42:10.701779+0000","last_became_active":"2024-09-10T21:42:09.704582+0000","last_became_peered":"2024-09-10T21:42:09.704582+0000","last_unstale":"2024-09-10T21:42:10.701779+0000","last_undegraded":"2024-09-10T21:42:10.701779+0000","last_fullsized":"2024-09-10T21:42:10.701779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:34.026318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":21,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685136+0000","last_change":"2024-09-10T21:42:08.694242+0000","last_active":"2024-09-10T21:42:11.685136+0000","last_peered":"2024-09-10T21:42:11.685136+0000","last_clean":"2024-09-10T21:42:11.685136+0000","last_became_active":"2024-09-10T21:42:08.693785+0000","last_became_peered":"2024-09-10T21:42:08.693785+0000","last_unstale":"2024-09-10T21:42:11.685136+0000","last_undegraded":"2024-09-10T21:42:11.685136+0000","last_fullsized":"2024-09-10T21:42:11.685136+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:56.090844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781904+0000","last_change":"2024-09-10T21:42:09.709096+0000","last_active":"2024-09-10T21:42:11.781904+0000","last_peered":"2024-09-10T21:42:11.781904+0000","last_clean":"2024-09-10T21:42:11.781904+0000","last_became_active":"2024-09-10T21:42:09.708202+0000","last_became_peered":"2024-09-10T21:42:09.708202+0000","last_unstale":"2024-09-10T21:42:11.781904+0000","last_undegraded":"2024-09-10T21:42:11.781904+0000","last_fullsized":"2024-09-10T21:42:11.781904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:12:39.888041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975170+0000","last_change":"2024-09-10T21:42:08.694284+0000","last_active":"2024-09-10T21:42:11.975170+0000","last_peered":"2024-09-10T21:42:11.975170+0000","last_clean":"2024-09-10T21:42:11.975170+0000","last_became_active":"2024-09-10T21:42:08.693910+0000","last_became_peered":"2024-09-10T21:42:08.693910+0000","last_unstale":"2024-09-10T21:42:11.975170+0000","last_undegraded":"2024-09-10T21:42:11.975170+0000","last_fullsized":"2024-09-10T21:42:11.975170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:40.174118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975003+0000","last_change":"2024-09-10T21:42:09.712940+0000","last_active":"2024-09-10T21:42:11.975003+0000","last_peered":"2024-09-10T21:42:11.975003+0000","last_clean":"2024-09-10T21:42:11.975003+0000","last_became_active":"2024-09-10T21:42:09.709015+0000","last_became_peered":"2024-09-10T21:42:09.709015+0000","last_unstale":"2024-09-10T21:42:11.975003+0000","last_undegraded":"2024-09-10T21:42:11.975003+0000","last_fullsized":"2024-09-10T21:42:11.975003+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:43:41.900249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701745+0000","last_change":"2024-09-10T21:42:08.689371+0000","last_active":"2024-09-10T21:42:10.701745+0000","last_peered":"2024-09-10T21:42:10.701745+0000","last_clean":"2024-09-10T21:42:10.701745+0000","last_became_active":"2024-09-10T21:42:08.688880+0000","last_became_peered":"2024-09-10T21:42:08.688880+0000","last_unstale":"2024-09-10T21:42:10.701745+0000","last_undegraded":"2024-09-10T21:42:10.701745+0000","last_fullsized":"2024-09-10T21:42:10.701745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:46.162803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781881+0000","last_change":"2024-09-10T21:42:09.713450+0000","last_active":"2024-09-10T21:42:11.781881+0000","last_peered":"2024-09-10T21:42:11.781881+0000","last_clean":"2024-09-10T21:42:11.781881+0000","last_became_active":"2024-09-10T21:42:09.713199+0000","last_became_peered":"2024-09-10T21:42:09.713199+0000","last_unstale":"2024-09-10T21:42:11.781881+0000","last_undegraded":"2024-09-10T21:42:11.781881+0000","last_fullsized":"2024-09-10T21:42:11.781881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:23.530344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.684649+0000","last_change":"2024-09-10T21:42:08.703573+0000","last_active":"2024-09-10T21:42:11.684649+0000","last_peered":"2024-09-10T21:42:11.684649+0000","last_clean":"2024-09-10T21:42:11.684649+0000","last_became_active":"2024-09-10T21:42:08.701580+0000","last_became_peered":"2024-09-10T21:42:08.701580+0000","last_unstale":"2024-09-10T21:42:11.684649+0000","last_undegraded":"2024-09-10T21:42:11.684649+0000","last_fullsized":"2024-09-10T21:42:11.684649+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:16.471687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.684924+0000","last_change":"2024-09-10T21:42:09.700853+0000","last_active":"2024-09-10T21:42:11.684924+0000","last_peered":"2024-09-10T21:42:11.684924+0000","last_clean":"2024-09-10T21:42:11.684924+0000","last_became_active":"2024-09-10T21:42:09.700370+0000","last_became_peered":"2024-09-10T21:42:09.700370+0000","last_unstale":"2024-09-10T21:42:11.684924+0000","last_undegraded":"2024-09-10T21:42:11.684924+0000","last_fullsized":"2024-09-10T21:42:11.684924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:13.700823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":23,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428370+0000","last_change":"2024-09-10T21:42:08.697117+0000","last_active":"2024-09-10T21:42:11.428370+0000","last_peered":"2024-09-10T21:42:11.428370+0000","last_clean":"2024-09-10T21:42:11.428370+0000","last_became_active":"2024-09-10T21:42:08.695800+0000","last_became_peered":"2024-09-10T21:42:08.695800+0000","last_unstale":"2024-09-10T21:42:11.428370+0000","last_undegraded":"2024-09-10T21:42:11.428370+0000","last_fullsized":"2024-09-10T21:42:11.428370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:20:36.934204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781790+0000","last_change":"2024-09-10T21:42:09.707807+0000","last_active":"2024-09-10T21:42:11.781790+0000","last_peered":"2024-09-10T21:42:11.781790+0000","last_clean":"2024-09-10T21:42:11.781790+0000","last_became_active":"2024-09-10T21:42:09.707283+0000","last_became_peered":"2024-09-10T21:42:09.707283+0000","last_unstale":"2024-09-10T21:42:11.781790+0000","last_undegraded":"2024-09-10T21:42:11.781790+0000","last_fullsized":"2024-09-10T21:42:11.781790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:50.040141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701923+0000","last_change":"2024-09-10T21:42:08.693177+0000","last_active":"2024-09-10T21:42:10.701923+0000","last_peered":"2024-09-10T21:42:10.701923+0000","last_clean":"2024-09-10T21:42:10.701923+0000","last_became_active":"2024-09-10T21:42:08.692871+0000","last_became_peered":"2024-09-10T21:42:08.692871+0000","last_unstale":"2024-09-10T21:42:10.701923+0000","last_undegraded":"2024-09-10T21:42:10.701923+0000","last_fullsized":"2024-09-10T21:42:10.701923+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:06.841814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781985+0000","last_change":"2024-09-10T21:42:09.710163+0000","last_active":"2024-09-10T21:42:11.781985+0000","last_peered":"2024-09-10T21:42:11.781985+0000","last_clean":"2024-09-10T21:42:11.781985+0000","last_became_active":"2024-09-10T21:42:09.709887+0000","last_became_peered":"2024-09-10T21:42:09.709887+0000","last_unstale":"2024-09-10T21:42:11.781985+0000","last_undegraded":"2024-09-10T21:42:11.781985+0000","last_fullsized":"2024-09-10T21:42:11.781985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:05:52.771916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428604+0000","last_change":"2024-09-10T21:42:08.686678+0000","last_active":"2024-09-10T21:42:11.428604+0000","last_peered":"2024-09-10T21:42:11.428604+0000","last_clean":"2024-09-10T21:42:11.428604+0000","last_became_active":"2024-09-10T21:42:08.686395+0000","last_became_peered":"2024-09-10T21:42:08.686395+0000","last_unstale":"2024-09-10T21:42:11.428604+0000","last_undegraded":"2024-09-10T21:42:11.428604+0000","last_fullsized":"2024-09-10T21:42:11.428604+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:50.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782250+0000","last_change":"2024-09-10T21:42:09.714231+0000","last_active":"2024-09-10T21:42:11.782250+0000","last_peered":"2024-09-10T21:42:11.782250+0000","last_clean":"2024-09-10T21:42:11.782250+0000","last_became_active":"2024-09-10T21:42:09.713873+0000","last_became_peered":"2024-09-10T21:42:09.713873+0000","last_unstale":"2024-09-10T21:42:11.782250+0000","last_undegraded":"2024-09-10T21:42:11.782250+0000","last_fullsized":"2024-09-10T21:42:11.782250+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:50.638349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695399+0000","last_change":"2024-09-10T21:42:08.694275+0000","last_active":"2024-09-10T21:42:10.695399+0000","last_peered":"2024-09-10T21:42:10.695399+0000","last_clean":"2024-09-10T21:42:10.695399+0000","last_became_active":"2024-09-10T21:42:08.694014+0000","last_became_peered":"2024-09-10T21:42:08.694014+0000","last_unstale":"2024-09-10T21:42:10.695399+0000","last_undegraded":"2024-09-10T21:42:10.695399+0000","last_fullsized":"2024-09-10T21:42:10.695399+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:52.021550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702618+0000","last_change":"2024-09-10T21:42:09.698139+0000","last_active":"2024-09-10T21:42:10.702618+0000","last_peered":"2024-09-10T21:42:10.702618+0000","last_clean":"2024-09-10T21:42:10.702618+0000","last_became_active":"2024-09-10T21:42:09.697391+0000","last_became_peered":"2024-09-10T21:42:09.697391+0000","last_unstale":"2024-09-10T21:42:10.702618+0000","last_undegraded":"2024-09-10T21:42:10.702618+0000","last_fullsized":"2024-09-10T21:42:10.702618+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:47.497053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428569+0000","last_change":"2024-09-10T21:42:08.691652+0000","last_active":"2024-09-10T21:42:11.428569+0000","last_peered":"2024-09-10T21:42:11.428569+0000","last_clean":"2024-09-10T21:42:11.428569+0000","last_became_active":"2024-09-10T21:42:08.691317+0000","last_became_peered":"2024-09-10T21:42:08.691317+0000","last_unstale":"2024-09-10T21:42:11.428569+0000","last_undegraded":"2024-09-10T21:42:11.428569+0000","last_fullsized":"2024-09-10T21:42:11.428569+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:23:10.198175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781834+0000","last_change":"2024-09-10T21:42:09.720201+0000","last_active":"2024-09-10T21:42:11.781834+0000","last_peered":"2024-09-10T21:42:11.781834+0000","last_clean":"2024-09-10T21:42:11.781834+0000","last_became_active":"2024-09-10T21:42:09.719872+0000","last_became_peered":"2024-09-10T21:42:09.719872+0000","last_unstale":"2024-09-10T21:42:11.781834+0000","last_undegraded":"2024-09-10T21:42:11.781834+0000","last_fullsized":"2024-09-10T21:42:11.781834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:29:31.222577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":111,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781370+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:42:11.781370+0000","last_peered":"2024-09-10T21:42:11.781370+0000","last_clean":"2024-09-10T21:42:11.781370+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:42:11.781370+0000","last_undegraded":"2024-09-10T21:42:11.781370+0000","last_fullsized":"2024-09-10T21:42:11.781370+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701380+0000","last_change":"2024-09-10T21:42:08.698035+0000","last_active":"2024-09-10T21:42:10.701380+0000","last_peered":"2024-09-10T21:42:10.701380+0000","last_clean":"2024-09-10T21:42:10.701380+0000","last_became_active":"2024-09-10T21:42:08.695961+0000","last_became_peered":"2024-09-10T21:42:08.695961+0000","last_unstale":"2024-09-10T21:42:10.701380+0000","last_undegraded":"2024-09-10T21:42:10.701380+0000","last_fullsized":"2024-09-10T21:42:10.701380+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:36:16.471687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249554+0000","last_change":"2024-09-10T21:42:09.692488+0000","last_active":"2024-09-10T21:42:12.249554+0000","last_peered":"2024-09-10T21:42:12.249554+0000","last_clean":"2024-09-10T21:42:12.249554+0000","last_became_active":"2024-09-10T21:42:09.691349+0000","last_became_peered":"2024-09-10T21:42:09.691349+0000","last_unstale":"2024-09-10T21:42:12.249554+0000","last_undegraded":"2024-09-10T21:42:12.249554+0000","last_fullsized":"2024-09-10T21:42:12.249554+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:00.574924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781332+0000","last_change":"2024-09-10T21:42:08.706510+0000","last_active":"2024-09-10T21:42:11.781332+0000","last_peered":"2024-09-10T21:42:11.781332+0000","last_clean":"2024-09-10T21:42:11.781332+0000","last_became_active":"2024-09-10T21:42:08.706080+0000","last_became_peered":"2024-09-10T21:42:08.706080+0000","last_unstale":"2024-09-10T21:42:11.781332+0000","last_undegraded":"2024-09-10T21:42:11.781332+0000","last_fullsized":"2024-09-10T21:42:11.781332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:25.172297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975060+0000","last_change":"2024-09-10T21:42:09.698977+0000","last_active":"2024-09-10T21:42:11.975060+0000","last_peered":"2024-09-10T21:42:11.975060+0000","last_clean":"2024-09-10T21:42:11.975060+0000","last_became_active":"2024-09-10T21:42:09.698557+0000","last_became_peered":"2024-09-10T21:42:09.698557+0000","last_unstale":"2024-09-10T21:42:11.975060+0000","last_undegraded":"2024-09-10T21:42:11.975060+0000","last_fullsized":"2024-09-10T21:42:11.975060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:40.541602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.428457+0000","last_change":"2024-09-10T21:42:08.689207+0000","last_active":"2024-09-10T21:42:11.428457+0000","last_peered":"2024-09-10T21:42:11.428457+0000","last_clean":"2024-09-10T21:42:11.428457+0000","last_became_active":"2024-09-10T21:42:08.688941+0000","last_became_peered":"2024-09-10T21:42:08.688941+0000","last_unstale":"2024-09-10T21:42:11.428457+0000","last_undegraded":"2024-09-10T21:42:11.428457+0000","last_fullsized":"2024-09-10T21:42:11.428457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:56.090844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695552+0000","last_change":"2024-09-10T21:42:09.688301+0000","last_active":"2024-09-10T21:42:10.695552+0000","last_peered":"2024-09-10T21:42:10.695552+0000","last_clean":"2024-09-10T21:42:10.695552+0000","last_became_active":"2024-09-10T21:42:09.687944+0000","last_became_peered":"2024-09-10T21:42:09.687944+0000","last_unstale":"2024-09-10T21:42:10.695552+0000","last_undegraded":"2024-09-10T21:42:10.695552+0000","last_fullsized":"2024-09-10T21:42:10.695552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:37.025454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701842+0000","last_change":"2024-09-10T21:42:08.698803+0000","last_active":"2024-09-10T21:42:10.701842+0000","last_peered":"2024-09-10T21:42:10.701842+0000","last_clean":"2024-09-10T21:42:10.701842+0000","last_became_active":"2024-09-10T21:42:08.697174+0000","last_became_peered":"2024-09-10T21:42:08.697174+0000","last_unstale":"2024-09-10T21:42:10.701842+0000","last_undegraded":"2024-09-10T21:42:10.701842+0000","last_fullsized":"2024-09-10T21:42:10.701842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:47.619335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702033+0000","last_change":"2024-09-10T21:42:09.688575+0000","last_active":"2024-09-10T21:42:10.702033+0000","last_peered":"2024-09-10T21:42:10.702033+0000","last_clean":"2024-09-10T21:42:10.702033+0000","last_became_active":"2024-09-10T21:42:09.688284+0000","last_became_peered":"2024-09-10T21:42:09.688284+0000","last_unstale":"2024-09-10T21:42:10.702033+0000","last_undegraded":"2024-09-10T21:42:10.702033+0000","last_fullsized":"2024-09-10T21:42:10.702033+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:05:53.029762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695680+0000","last_change":"2024-09-10T21:42:08.698800+0000","last_active":"2024-09-10T21:42:10.695680+0000","last_peered":"2024-09-10T21:42:10.695680+0000","last_clean":"2024-09-10T21:42:10.695680+0000","last_became_active":"2024-09-10T21:42:08.696627+0000","last_became_peered":"2024-09-10T21:42:08.696627+0000","last_unstale":"2024-09-10T21:42:10.695680+0000","last_undegraded":"2024-09-10T21:42:10.695680+0000","last_fullsized":"2024-09-10T21:42:10.695680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:56.090844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701667+0000","last_change":"2024-09-10T21:42:09.700048+0000","last_active":"2024-09-10T21:42:10.701667+0000","last_peered":"2024-09-10T21:42:10.701667+0000","last_clean":"2024-09-10T21:42:10.701667+0000","last_became_active":"2024-09-10T21:42:09.699085+0000","last_became_peered":"2024-09-10T21:42:09.699085+0000","last_unstale":"2024-09-10T21:42:10.701667+0000","last_undegraded":"2024-09-10T21:42:10.701667+0000","last_fullsized":"2024-09-10T21:42:10.701667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:22.835437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782141+0000","last_change":"2024-09-10T21:42:08.704257+0000","last_active":"2024-09-10T21:42:11.782141+0000","last_peered":"2024-09-10T21:42:11.782141+0000","last_clean":"2024-09-10T21:42:11.782141+0000","last_became_active":"2024-09-10T21:42:08.703837+0000","last_became_peered":"2024-09-10T21:42:08.703837+0000","last_unstale":"2024-09-10T21:42:11.782141+0000","last_undegraded":"2024-09-10T21:42:11.782141+0000","last_fullsized":"2024-09-10T21:42:11.782141+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:09:44.076555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695679+0000","last_change":"2024-09-10T21:42:09.695578+0000","last_active":"2024-09-10T21:42:10.695679+0000","last_peered":"2024-09-10T21:42:10.695679+0000","last_clean":"2024-09-10T21:42:10.695679+0000","last_became_active":"2024-09-10T21:42:09.693969+0000","last_became_peered":"2024-09-10T21:42:09.693969+0000","last_unstale":"2024-09-10T21:42:10.695679+0000","last_undegraded":"2024-09-10T21:42:10.695679+0000","last_fullsized":"2024-09-10T21:42:10.695679+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:48.304330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695614+0000","last_change":"2024-09-10T21:42:08.699017+0000","last_active":"2024-09-10T21:42:10.695614+0000","last_peered":"2024-09-10T21:42:10.695614+0000","last_clean":"2024-09-10T21:42:10.695614+0000","last_became_active":"2024-09-10T21:42:08.698540+0000","last_became_peered":"2024-09-10T21:42:08.698540+0000","last_unstale":"2024-09-10T21:42:10.695614+0000","last_undegraded":"2024-09-10T21:42:10.695614+0000","last_fullsized":"2024-09-10T21:42:10.695614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:33.018106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.250013+0000","last_change":"2024-09-10T21:42:09.709063+0000","last_active":"2024-09-10T21:42:12.250013+0000","last_peered":"2024-09-10T21:42:12.250013+0000","last_clean":"2024-09-10T21:42:12.250013+0000","last_became_active":"2024-09-10T21:42:09.708726+0000","last_became_peered":"2024-09-10T21:42:09.708726+0000","last_unstale":"2024-09-10T21:42:12.250013+0000","last_undegraded":"2024-09-10T21:42:12.250013+0000","last_fullsized":"2024-09-10T21:42:12.250013+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:00.508736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685182+0000","last_change":"2024-09-10T21:42:08.697045+0000","last_active":"2024-09-10T21:42:11.685182+0000","last_peered":"2024-09-10T21:42:11.685182+0000","last_clean":"2024-09-10T21:42:11.685182+0000","last_became_active":"2024-09-10T21:42:08.696483+0000","last_became_peered":"2024-09-10T21:42:08.696483+0000","last_unstale":"2024-09-10T21:42:11.685182+0000","last_undegraded":"2024-09-10T21:42:11.685182+0000","last_fullsized":"2024-09-10T21:42:11.685182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:34.907546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975478+0000","last_change":"2024-09-10T21:42:09.694168+0000","last_active":"2024-09-10T21:42:11.975478+0000","last_peered":"2024-09-10T21:42:11.975478+0000","last_clean":"2024-09-10T21:42:11.975478+0000","last_became_active":"2024-09-10T21:42:09.693779+0000","last_became_peered":"2024-09-10T21:42:09.693779+0000","last_unstale":"2024-09-10T21:42:11.975478+0000","last_undegraded":"2024-09-10T21:42:11.975478+0000","last_fullsized":"2024-09-10T21:42:11.975478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:24:55.155776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781313+0000","last_change":"2024-09-10T21:42:08.697319+0000","last_active":"2024-09-10T21:42:11.781313+0000","last_peered":"2024-09-10T21:42:11.781313+0000","last_clean":"2024-09-10T21:42:11.781313+0000","last_became_active":"2024-09-10T21:42:08.697039+0000","last_became_peered":"2024-09-10T21:42:08.697039+0000","last_unstale":"2024-09-10T21:42:11.781313+0000","last_undegraded":"2024-09-10T21:42:11.781313+0000","last_fullsized":"2024-09-10T21:42:11.781313+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:15:48.264843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701623+0000","last_change":"2024-09-10T21:42:09.689554+0000","last_active":"2024-09-10T21:42:10.701623+0000","last_peered":"2024-09-10T21:42:10.701623+0000","last_clean":"2024-09-10T21:42:10.701623+0000","last_became_active":"2024-09-10T21:42:09.689041+0000","last_became_peered":"2024-09-10T21:42:09.689041+0000","last_unstale":"2024-09-10T21:42:10.701623+0000","last_undegraded":"2024-09-10T21:42:10.701623+0000","last_fullsized":"2024-09-10T21:42:10.701623+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:48.627547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695404+0000","last_change":"2024-09-10T21:42:08.700219+0000","last_active":"2024-09-10T21:42:10.695404+0000","last_peered":"2024-09-10T21:42:10.695404+0000","last_clean":"2024-09-10T21:42:10.695404+0000","last_became_active":"2024-09-10T21:42:08.696072+0000","last_became_peered":"2024-09-10T21:42:08.696072+0000","last_unstale":"2024-09-10T21:42:10.695404+0000","last_undegraded":"2024-09-10T21:42:10.695404+0000","last_fullsized":"2024-09-10T21:42:10.695404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:45:50.308052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249562+0000","last_change":"2024-09-10T21:42:09.695584+0000","last_active":"2024-09-10T21:42:12.249562+0000","last_peered":"2024-09-10T21:42:12.249562+0000","last_clean":"2024-09-10T21:42:12.249562+0000","last_became_active":"2024-09-10T21:42:09.695169+0000","last_became_peered":"2024-09-10T21:42:09.695169+0000","last_unstale":"2024-09-10T21:42:12.249562+0000","last_undegraded":"2024-09-10T21:42:12.249562+0000","last_fullsized":"2024-09-10T21:42:12.249562+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:07.671505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.695459+0000","last_change":"2024-09-10T21:42:08.702389+0000","last_active":"2024-09-10T21:42:10.695459+0000","last_peered":"2024-09-10T21:42:10.695459+0000","last_clean":"2024-09-10T21:42:10.695459+0000","last_became_active":"2024-09-10T21:42:08.701730+0000","last_became_peered":"2024-09-10T21:42:08.701730+0000","last_unstale":"2024-09-10T21:42:10.695459+0000","last_undegraded":"2024-09-10T21:42:10.695459+0000","last_fullsized":"2024-09-10T21:42:10.695459+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:47.619335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249655+0000","last_change":"2024-09-10T21:42:09.691569+0000","last_active":"2024-09-10T21:42:12.249655+0000","last_peered":"2024-09-10T21:42:12.249655+0000","last_clean":"2024-09-10T21:42:12.249655+0000","last_became_active":"2024-09-10T21:42:09.691270+0000","last_became_peered":"2024-09-10T21:42:09.691270+0000","last_unstale":"2024-09-10T21:42:12.249655+0000","last_undegraded":"2024-09-10T21:42:12.249655+0000","last_fullsized":"2024-09-10T21:42:12.249655+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:13.594517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782013+0000","last_change":"2024-09-10T21:42:08.702598+0000","last_active":"2024-09-10T21:42:11.782013+0000","last_peered":"2024-09-10T21:42:11.782013+0000","last_clean":"2024-09-10T21:42:11.782013+0000","last_became_active":"2024-09-10T21:42:08.702352+0000","last_became_peered":"2024-09-10T21:42:08.702352+0000","last_unstale":"2024-09-10T21:42:11.782013+0000","last_undegraded":"2024-09-10T21:42:11.782013+0000","last_fullsized":"2024-09-10T21:42:11.782013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:10:45.571003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.685024+0000","last_change":"2024-09-10T21:42:09.708756+0000","last_active":"2024-09-10T21:42:11.685024+0000","last_peered":"2024-09-10T21:42:11.685024+0000","last_clean":"2024-09-10T21:42:11.685024+0000","last_became_active":"2024-09-10T21:42:09.708458+0000","last_became_peered":"2024-09-10T21:42:09.708458+0000","last_unstale":"2024-09-10T21:42:11.685024+0000","last_undegraded":"2024-09-10T21:42:11.685024+0000","last_fullsized":"2024-09-10T21:42:11.685024+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:37.025454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.701552+0000","last_change":"2024-09-10T21:42:09.692810+0000","last_active":"2024-09-10T21:42:10.701552+0000","last_peered":"2024-09-10T21:42:10.701552+0000","last_clean":"2024-09-10T21:42:10.701552+0000","last_became_active":"2024-09-10T21:42:09.692537+0000","last_became_peered":"2024-09-10T21:42:09.692537+0000","last_unstale":"2024-09-10T21:42:10.701552+0000","last_undegraded":"2024-09-10T21:42:10.701552+0000","last_fullsized":"2024-09-10T21:42:10.701552+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:55.273108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.781669+0000","last_change":"2024-09-10T21:42:08.705305+0000","last_active":"2024-09-10T21:42:11.781669+0000","last_peered":"2024-09-10T21:42:11.781669+0000","last_clean":"2024-09-10T21:42:11.781669+0000","last_became_active":"2024-09-10T21:42:08.704991+0000","last_became_peered":"2024-09-10T21:42:08.704991+0000","last_unstale":"2024-09-10T21:42:11.781669+0000","last_undegraded":"2024-09-10T21:42:11.781669+0000","last_fullsized":"2024-09-10T21:42:11.781669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:59:59.418988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:10.702644+0000","last_change":"2024-09-10T21:42:09.704028+0000","last_active":"2024-09-10T21:42:10.702644+0000","last_peered":"2024-09-10T21:42:10.702644+0000","last_clean":"2024-09-10T21:42:10.702644+0000","last_became_active":"2024-09-10T21:42:09.703608+0000","last_became_peered":"2024-09-10T21:42:09.703608+0000","last_unstale":"2024-09-10T21:42:10.702644+0000","last_undegraded":"2024-09-10T21:42:10.702644+0000","last_fullsized":"2024-09-10T21:42:10.702644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:11.626660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.782020+0000","last_change":"2024-09-10T21:42:08.704448+0000","last_active":"2024-09-10T21:42:11.782020+0000","last_peered":"2024-09-10T21:42:11.782020+0000","last_clean":"2024-09-10T21:42:11.782020+0000","last_became_active":"2024-09-10T21:42:08.704156+0000","last_became_peered":"2024-09-10T21:42:08.704156+0000","last_unstale":"2024-09-10T21:42:11.782020+0000","last_undegraded":"2024-09-10T21:42:11.782020+0000","last_fullsized":"2024-09-10T21:42:11.782020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:00:07.043672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975512+0000","last_change":"2024-09-10T21:42:09.693556+0000","last_active":"2024-09-10T21:42:11.975512+0000","last_peered":"2024-09-10T21:42:11.975512+0000","last_clean":"2024-09-10T21:42:11.975512+0000","last_became_active":"2024-09-10T21:42:09.693330+0000","last_became_peered":"2024-09-10T21:42:09.693330+0000","last_unstale":"2024-09-10T21:42:11.975512+0000","last_undegraded":"2024-09-10T21:42:11.975512+0000","last_fullsized":"2024-09-10T21:42:11.975512+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:26:39.608345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249807+0000","last_change":"2024-09-10T21:42:08.699157+0000","last_active":"2024-09-10T21:42:12.249807+0000","last_peered":"2024-09-10T21:42:12.249807+0000","last_clean":"2024-09-10T21:42:12.249807+0000","last_became_active":"2024-09-10T21:42:08.691408+0000","last_became_peered":"2024-09-10T21:42:08.691408+0000","last_unstale":"2024-09-10T21:42:12.249807+0000","last_undegraded":"2024-09-10T21:42:12.249807+0000","last_fullsized":"2024-09-10T21:42:12.249807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:52.464492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:12.249860+0000","last_change":"2024-09-10T21:42:09.704294+0000","last_active":"2024-09-10T21:42:12.249860+0000","last_peered":"2024-09-10T21:42:12.249860+0000","last_clean":"2024-09-10T21:42:12.249860+0000","last_became_active":"2024-09-10T21:42:09.703607+0000","last_became_peered":"2024-09-10T21:42:09.703607+0000","last_unstale":"2024-09-10T21:42:12.249860+0000","last_undegraded":"2024-09-10T21:42:12.249860+0000","last_fullsized":"2024-09-10T21:42:12.249860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:08.663512+0000","last_clean_scrub_stamp":"2024-09-10T21:42:08.663512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:58.276453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-10T21:42:11.975594+0000","last_change":"2024-09-10T21:42:08.702462+0000","last_active":"2024-09-10T21:42:11.975594+0000","last_peered":"2024-09-10T21:42:11.975594+0000","last_clean":"2024-09-10T21:42:11.975594+0000","last_became_active":"2024-09-10T21:42:08.702103+0000","last_became_peered":"2024-09-10T21:42:08.702103+0000","last_unstale":"2024-09-10T21:42:11.975594+0000","last_undegraded":"2024-09-10T21:42:11.975594+0000","last_fullsized":"2024-09-10T21:42:11.975594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:07.655300+0000","last_clean_scrub_stamp":"2024-09-10T21:42:07.655300+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:30.979951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509448,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509448,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":212992,"data_stored":72277,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":221184,"data_stored":72313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509447,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31392,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340448,"statfs":{"total":96636764160,"available":96604618752,"internally_reserved":0,"allocated":208896,"data_stored":72187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":217088,"data_stored":72347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509447,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31984,"kb_used_data":796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339856,"statfs":{"total":96636764160,"available":96604012544,"internally_reserved":0,"allocated":815104,"data_stored":663193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344020,"statfs":{"total":96636764160,"available":96608276480,"internally_reserved":0,"allocated":806912,"data_stored":663137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542151,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31404,"kb_used_data":216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340436,"statfs":{"total":96636764160,"available":96604606464,"internally_reserved":0,"allocated":221184,"data_stored":73799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:42:19.918 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:42:19.918 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T21:42:19.918 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-10T21:42:19.918 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:42:20.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.113+0000 7f2f2d600640 1 -- 172.21.15.7:0/2954114573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f0c004800 msgr2=0x7f2f0c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.113+0000 7f2f2d600640 1 --2- 172.21.15.7:0/2954114573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f0c004800 0x7f2f0c004c80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2f1400c310 tx=0x7f2f0c005830 comp rx=0 tx=0).stop 2024-09-10T21:42:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.113+0000 7f2f2d600640 1 -- 172.21.15.7:0/2954114573 shutdown_connections 2024-09-10T21:42:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.113+0000 7f2f2d600640 1 --2- 172.21.15.7:0/2954114573 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f0c004800 0x7f2f0c004c80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.113+0000 7f2f2d600640 1 -- 172.21.15.7:0/2954114573 >> 172.21.15.7:0/2954114573 conn(0x7f2f2809d260 msgr2=0x7f2f280d3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.113+0000 7f2f2d600640 1 -- 172.21.15.7:0/2954114573 shutdown_connections 2024-09-10T21:42:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.113+0000 7f2f2d600640 1 -- 172.21.15.7:0/2954114573 wait complete. 2024-09-10T21:42:20.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.114+0000 7f2f2d600640 1 Processor -- start 2024-09-10T21:42:20.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.115+0000 7f2f2d600640 1 -- start start 2024-09-10T21:42:20.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.115+0000 7f2f2d600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f0c004800 0x7f2f2814cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:20.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.115+0000 7f2f2d600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f280b3460 0x7f2f281f5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:20.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.115+0000 7f2f2d600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281f5640 0x7f2f281f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:20.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.116+0000 7f2f2d600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2f280b3ff0 con 0x7f2f281f5640 2024-09-10T21:42:20.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.116+0000 7f2f2d600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2f280b3e70 con 0x7f2f280b3460 2024-09-10T21:42:20.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.116+0000 7f2f2d600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f2f280b4170 con 0x7f2f0c004800 2024-09-10T21:42:20.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.116+0000 7f2f27400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281f5640 0x7f2f281f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:20.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.116+0000 7f2f27400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281f5640 0x7f2f281f5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54440/0 (socket says 172.21.15.7:54440) 2024-09-10T21:42:20.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.116+0000 7f2f27400640 1 -- 172.21.15.7:0/1380881057 learned_addr learned my addr 172.21.15.7:0/1380881057 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:20.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.116+0000 7f2f26000640 1 --2- 172.21.15.7:0/1380881057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f280b3460 0x7f2f281f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:20.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.116+0000 7f2f26a00640 1 --2- 172.21.15.7:0/1380881057 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f0c004800 0x7f2f2814cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:20.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.116+0000 7f2f27400640 1 -- 172.21.15.7:0/1380881057 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f0c004800 msgr2=0x7f2f2814cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:20.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.116+0000 7f2f27400640 1 --2- 172.21.15.7:0/1380881057 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f0c004800 0x7f2f2814cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.116+0000 7f2f27400640 1 -- 172.21.15.7:0/1380881057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f280b3460 msgr2=0x7f2f281f5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:20.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.116+0000 7f2f27400640 1 --2- 172.21.15.7:0/1380881057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f280b3460 0x7f2f281f5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.116+0000 7f2f27400640 1 -- 172.21.15.7:0/1380881057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f280b3c40 con 0x7f2f281f5640 2024-09-10T21:42:20.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.116+0000 7f2f27400640 1 --2- 172.21.15.7:0/1380881057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281f5640 0x7f2f281f5ab0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2f28161c70 tx=0x7f2f2000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:20.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.117+0000 7f2f1ea00640 1 -- 172.21.15.7:0/1380881057 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20003d80 con 0x7f2f281f5640 2024-09-10T21:42:20.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.117+0000 7f2f1ea00640 1 -- 172.21.15.7:0/1380881057 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f20004b20 con 0x7f2f281f5640 2024-09-10T21:42:20.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.117+0000 7f2f2d600640 1 -- 172.21.15.7:0/1380881057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f281f5ff0 con 0x7f2f281f5640 2024-09-10T21:42:20.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.117+0000 7f2f1ea00640 1 -- 172.21.15.7:0/1380881057 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f20004050 con 0x7f2f281f5640 2024-09-10T21:42:20.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.117+0000 7f2f2d600640 1 -- 172.21.15.7:0/1380881057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f280b78f0 con 0x7f2f281f5640 2024-09-10T21:42:20.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.118+0000 7f2f2d600640 1 -- 172.21.15.7:0/1380881057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f280d0fe0 con 0x7f2f281f5640 2024-09-10T21:42:20.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.119+0000 7f2f1ea00640 1 -- 172.21.15.7:0/1380881057 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2f20022070 con 0x7f2f281f5640 2024-09-10T21:42:20.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.121+0000 7f2f1ea00640 1 --2- 172.21.15.7:0/1380881057 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f08077f70 0x7f2f0807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:20.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.121+0000 7f2f1ea00640 1 -- 172.21.15.7:0/1380881057 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2f2009aee0 con 0x7f2f281f5640 2024-09-10T21:42:20.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.121+0000 7f2f26a00640 1 --2- 172.21.15.7:0/1380881057 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f08077f70 0x7f2f0807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:20.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.122+0000 7f2f26a00640 1 --2- 172.21.15.7:0/1380881057 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f08077f70 0x7f2f0807a430 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2f2806c190 tx=0x7f2f18006b70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:20.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.125+0000 7f2f1ea00640 1 -- 172.21.15.7:0/1380881057 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2f280d0fe0 con 0x7f2f281f5640 2024-09-10T21:42:20.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.357+0000 7f2f2d600640 1 -- 172.21.15.7:0/1380881057 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2f280b4500 con 0x7f2f281f5640 2024-09-10T21:42:20.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.358+0000 7f2f1ea00640 1 -- 172.21.15.7:0/1380881057 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f2f281f5ff0 con 0x7f2f281f5640 2024-09-10T21:42:20.359 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:20.359 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T21:42:20.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.360+0000 7f2f2d600640 1 -- 172.21.15.7:0/1380881057 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f08077f70 msgr2=0x7f2f0807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.360+0000 7f2f2d600640 1 --2- 172.21.15.7:0/1380881057 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f08077f70 0x7f2f0807a430 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2f2806c190 tx=0x7f2f18006b70 comp rx=0 tx=0).stop 2024-09-10T21:42:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.360+0000 7f2f2d600640 1 -- 172.21.15.7:0/1380881057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281f5640 msgr2=0x7f2f281f5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.360+0000 7f2f2d600640 1 --2- 172.21.15.7:0/1380881057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281f5640 0x7f2f281f5ab0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2f28161c70 tx=0x7f2f2000f790 comp rx=0 tx=0).stop 2024-09-10T21:42:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.360+0000 7f2f2d600640 1 -- 172.21.15.7:0/1380881057 shutdown_connections 2024-09-10T21:42:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.360+0000 7f2f2d600640 1 --2- 172.21.15.7:0/1380881057 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f08077f70 0x7f2f0807a430 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.360+0000 7f2f2d600640 1 --2- 172.21.15.7:0/1380881057 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f280b3460 0x7f2f281f5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.360+0000 7f2f2d600640 1 --2- 172.21.15.7:0/1380881057 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f281f5640 0x7f2f281f5ab0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.360+0000 7f2f2d600640 1 --2- 172.21.15.7:0/1380881057 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f0c004800 0x7f2f2814cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.360+0000 7f2f2d600640 1 -- 172.21.15.7:0/1380881057 >> 172.21.15.7:0/1380881057 conn(0x7f2f2809d260 msgr2=0x7f2f2806d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.360+0000 7f2f2d600640 1 -- 172.21.15.7:0/1380881057 shutdown_connections 2024-09-10T21:42:20.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.360+0000 7f2f2d600640 1 -- 172.21.15.7:0/1380881057 wait complete. 2024-09-10T21:42:20.377 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-10T21:42:20.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:42:20.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.536+0000 7f2217200640 1 -- 172.21.15.7:0/2379143471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22100ce860 msgr2=0x7f22100ceca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:20.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.536+0000 7f2217200640 1 --2- 172.21.15.7:0/2379143471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22100ce860 0x7f22100ceca0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f21fc005fd0 tx=0x7f21fc02a5d0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.536+0000 7f2217200640 1 -- 172.21.15.7:0/2379143471 shutdown_connections 2024-09-10T21:42:20.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.536+0000 7f2217200640 1 --2- 172.21.15.7:0/2379143471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22100ce860 0x7f22100ceca0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.536+0000 7f2217200640 1 --2- 172.21.15.7:0/2379143471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f221015aa30 0x7f2210073550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.536+0000 7f2217200640 1 -- 172.21.15.7:0/2379143471 >> 172.21.15.7:0/2379143471 conn(0x7f22100b16d0 msgr2=0x7f22100b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:20.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.536+0000 7f2217200640 1 -- 172.21.15.7:0/2379143471 shutdown_connections 2024-09-10T21:42:20.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.536+0000 7f2217200640 1 -- 172.21.15.7:0/2379143471 wait complete. 2024-09-10T21:42:20.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.538+0000 7f2217200640 1 Processor -- start 2024-09-10T21:42:20.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.538+0000 7f2217200640 1 -- start start 2024-09-10T21:42:20.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.539+0000 7f2217200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22100ce860 0x7f2210159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:20.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.539+0000 7f2217200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f221015aa30 0x7f22100c8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:20.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.539+0000 7f2217200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f221015e650 0x7f22100c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:20.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.539+0000 7f2217200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f221006ee30 con 0x7f221015e650 2024-09-10T21:42:20.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.539+0000 7f2217200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f221006ecb0 con 0x7f221015aa30 2024-09-10T21:42:20.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.539+0000 7f2217200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f22100cbcd0 con 0x7f22100ce860 2024-09-10T21:42:20.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.539+0000 7f2215200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f221015e650 0x7f22100c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:20.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.539+0000 7f2215200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f221015e650 0x7f22100c8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54462/0 (socket says 172.21.15.7:54462) 2024-09-10T21:42:20.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.539+0000 7f2215200640 1 -- 172.21.15.7:0/3096572749 learned_addr learned my addr 172.21.15.7:0/3096572749 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:20.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.539+0000 7f220f400640 1 --2- 172.21.15.7:0/3096572749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f221015aa30 0x7f22100c8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:20.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.539+0000 7f220fe00640 1 --2- 172.21.15.7:0/3096572749 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22100ce860 0x7f2210159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:20.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.539+0000 7f2215200640 1 -- 172.21.15.7:0/3096572749 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22100ce860 msgr2=0x7f2210159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:20.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.539+0000 7f2215200640 1 --2- 172.21.15.7:0/3096572749 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22100ce860 0x7f2210159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.539+0000 7f2215200640 1 -- 172.21.15.7:0/3096572749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f221015aa30 msgr2=0x7f22100c8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:20.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.539+0000 7f2215200640 1 --2- 172.21.15.7:0/3096572749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f221015aa30 0x7f22100c8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.539+0000 7f2215200640 1 -- 172.21.15.7:0/3096572749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22101f7ab0 con 0x7f221015e650 2024-09-10T21:42:20.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.540+0000 7f2215200640 1 --2- 172.21.15.7:0/3096572749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f221015e650 0x7f22100c8c70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f220800c3c0 tx=0x7f220800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:20.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.540+0000 7f2207e00640 1 -- 172.21.15.7:0/3096572749 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208017070 con 0x7f221015e650 2024-09-10T21:42:20.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.540+0000 7f2207e00640 1 -- 172.21.15.7:0/3096572749 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22080053c0 con 0x7f221015e650 2024-09-10T21:42:20.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.540+0000 7f2207e00640 1 -- 172.21.15.7:0/3096572749 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208003850 con 0x7f221015e650 2024-09-10T21:42:20.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.540+0000 7f2217200640 1 -- 172.21.15.7:0/3096572749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22101f7d20 con 0x7f221015e650 2024-09-10T21:42:20.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.540+0000 7f2217200640 1 -- 172.21.15.7:0/3096572749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22101f82e0 con 0x7f221015e650 2024-09-10T21:42:20.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.541+0000 7f2217200640 1 -- 172.21.15.7:0/3096572749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22100ceca0 con 0x7f221015e650 2024-09-10T21:42:20.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.542+0000 7f2207e00640 1 -- 172.21.15.7:0/3096572749 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f22080039f0 con 0x7f221015e650 2024-09-10T21:42:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.543+0000 7f2207e00640 1 --2- 172.21.15.7:0/3096572749 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f21f0077fa0 0x7f21f007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:20.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.543+0000 7f2207e00640 1 -- 172.21.15.7:0/3096572749 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f22080992a0 con 0x7f221015e650 2024-09-10T21:42:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.544+0000 7f220fe00640 1 --2- 172.21.15.7:0/3096572749 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f21f0077fa0 0x7f21f007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:20.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.544+0000 7f220fe00640 1 --2- 172.21.15.7:0/3096572749 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f21f0077fa0 0x7f21f007a460 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2200005fd0 tx=0x7f2200009680 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:20.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.547+0000 7f2207e00640 1 -- 172.21.15.7:0/3096572749 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2208061770 con 0x7f221015e650 2024-09-10T21:42:20.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.777+0000 7f2217200640 1 -- 172.21.15.7:0/3096572749 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22100b1df0 con 0x7f221015e650 2024-09-10T21:42:20.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.778+0000 7f2207e00640 1 -- 172.21.15.7:0/3096572749 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5227 (secure 0 0 0) 0x7f22080661a0 con 0x7f221015e650 2024-09-10T21:42:20.779 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:20.779 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"btime":"2024-09-10T21:42:12:706481+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4342,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3303727910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3303727910},{"type":"v1","addr":"172.21.15.7:6835","nonce":3303727910}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4383,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3193801272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3193801272},{"type":"v1","addr":"172.21.15.7:6834","nonce":3193801272}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4401,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1546148687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1546148687},{"type":"v1","addr":"172.21.15.133:6837","nonce":1546148687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:42:09.121017+0000","modified":"2024-09-10T21:42:12.137567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234179":{"gid":234179,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6836/3769241160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3769241160},{"type":"v1","addr":"172.21.15.133:6836","nonce":3769241160}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234179,"qdb_cluster":[234179]},"id":1}]} 2024-09-10T21:42:20.779 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-10T21:42:20.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.780+0000 7f2217200640 1 -- 172.21.15.7:0/3096572749 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f21f0077fa0 msgr2=0x7f21f007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:20.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.780+0000 7f2217200640 1 --2- 172.21.15.7:0/3096572749 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f21f0077fa0 0x7f21f007a460 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2200005fd0 tx=0x7f2200009680 comp rx=0 tx=0).stop 2024-09-10T21:42:20.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.780+0000 7f2217200640 1 -- 172.21.15.7:0/3096572749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f221015e650 msgr2=0x7f22100c8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:20.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.780+0000 7f2217200640 1 --2- 172.21.15.7:0/3096572749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f221015e650 0x7f22100c8c70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f220800c3c0 tx=0x7f220800c880 comp rx=0 tx=0).stop 2024-09-10T21:42:20.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.780+0000 7f2217200640 1 -- 172.21.15.7:0/3096572749 shutdown_connections 2024-09-10T21:42:20.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.780+0000 7f2217200640 1 --2- 172.21.15.7:0/3096572749 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f21f0077fa0 0x7f21f007a460 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.780+0000 7f2217200640 1 --2- 172.21.15.7:0/3096572749 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f221015aa30 0x7f22100c8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.780+0000 7f2217200640 1 --2- 172.21.15.7:0/3096572749 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f221015e650 0x7f22100c8c70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.780+0000 7f2217200640 1 --2- 172.21.15.7:0/3096572749 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22100ce860 0x7f2210159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.780+0000 7f2217200640 1 -- 172.21.15.7:0/3096572749 >> 172.21.15.7:0/3096572749 conn(0x7f22100b16d0 msgr2=0x7f221015b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:20.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.780+0000 7f2217200640 1 -- 172.21.15.7:0/3096572749 shutdown_connections 2024-09-10T21:42:20.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.780+0000 7f2217200640 1 -- 172.21.15.7:0/3096572749 wait complete. 2024-09-10T21:42:20.797 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:42:20.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.956+0000 7f7e9d600640 1 -- 172.21.15.7:0/2573127236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9815fda0 msgr2=0x7f7e981601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:20.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.956+0000 7f7e9d600640 1 --2- 172.21.15.7:0/2573127236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9815fda0 0x7f7e981601f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7e90005fd0 tx=0x7f7e9002a980 comp rx=0 tx=0).stop 2024-09-10T21:42:20.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.956+0000 7f7e9d600640 1 -- 172.21.15.7:0/2573127236 shutdown_connections 2024-09-10T21:42:20.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.956+0000 7f7e9d600640 1 --2- 172.21.15.7:0/2573127236 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9815fda0 0x7f7e981601f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.956+0000 7f7e9d600640 1 --2- 172.21.15.7:0/2573127236 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e9815bc90 0x7f7e980ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.956+0000 7f7e9d600640 1 -- 172.21.15.7:0/2573127236 >> 172.21.15.7:0/2573127236 conn(0x7f7e980b16d0 msgr2=0x7f7e980b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:20.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.956+0000 7f7e9d600640 1 -- 172.21.15.7:0/2573127236 shutdown_connections 2024-09-10T21:42:20.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.956+0000 7f7e9d600640 1 -- 172.21.15.7:0/2573127236 wait complete. 2024-09-10T21:42:20.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.958+0000 7f7e9d600640 1 Processor -- start 2024-09-10T21:42:20.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.958+0000 7f7e9d600640 1 -- start start 2024-09-10T21:42:20.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.959+0000 7f7e9d600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e980cd750 0x7f7e980725d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:20.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.959+0000 7f7e9d600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e9815bc90 0x7f7e9815e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:20.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.959+0000 7f7e9d600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9815fda0 0x7f7e9815aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.959+0000 7f7e9d600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9806fe70 con 0x7f7e9815fda0 2024-09-10T21:42:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.959+0000 7f7e9d600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7e9806fcf0 con 0x7f7e9815bc90 2024-09-10T21:42:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.959+0000 7f7e9d600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f7e9806fff0 con 0x7f7e980cd750 2024-09-10T21:42:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.959+0000 7f7e97400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9815fda0 0x7f7e9815aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.959+0000 7f7e97400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9815fda0 0x7f7e9815aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54468/0 (socket says 172.21.15.7:54468) 2024-09-10T21:42:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.959+0000 7f7e97400640 1 -- 172.21.15.7:0/2519623603 learned_addr learned my addr 172.21.15.7:0/2519623603 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.959+0000 7f7e8e000640 1 --2- 172.21.15.7:0/2519623603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e9815bc90 0x7f7e9815e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.959+0000 7f7e96a00640 1 --2- 172.21.15.7:0/2519623603 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e980cd750 0x7f7e980725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.959+0000 7f7e97400640 1 -- 172.21.15.7:0/2519623603 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e980cd750 msgr2=0x7f7e980725d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.959+0000 7f7e97400640 1 --2- 172.21.15.7:0/2519623603 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e980cd750 0x7f7e980725d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.959+0000 7f7e97400640 1 -- 172.21.15.7:0/2519623603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e9815bc90 msgr2=0x7f7e9815e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:20.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.959+0000 7f7e97400640 1 --2- 172.21.15.7:0/2519623603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e9815bc90 0x7f7e9815e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.959+0000 7f7e97400640 1 -- 172.21.15.7:0/2519623603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e981f44d0 con 0x7f7e9815fda0 2024-09-10T21:42:20.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.960+0000 7f7e97400640 1 --2- 172.21.15.7:0/2519623603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9815fda0 0x7f7e9815aa10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7e90006010 tx=0x7f7e90009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:20.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.960+0000 7f7e8f400640 1 -- 172.21.15.7:0/2519623603 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90034040 con 0x7f7e9815fda0 2024-09-10T21:42:20.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.960+0000 7f7e8f400640 1 -- 172.21.15.7:0/2519623603 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e90004540 con 0x7f7e9815fda0 2024-09-10T21:42:20.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.960+0000 7f7e8f400640 1 -- 172.21.15.7:0/2519623603 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e90045480 con 0x7f7e9815fda0 2024-09-10T21:42:20.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.960+0000 7f7e9d600640 1 -- 172.21.15.7:0/2519623603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e981f47b0 con 0x7f7e9815fda0 2024-09-10T21:42:20.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.960+0000 7f7e9d600640 1 -- 172.21.15.7:0/2519623603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e981f4180 con 0x7f7e9815fda0 2024-09-10T21:42:20.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.961+0000 7f7e9d600640 1 -- 172.21.15.7:0/2519623603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e98070510 con 0x7f7e9815fda0 2024-09-10T21:42:20.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.962+0000 7f7e8f400640 1 -- 172.21.15.7:0/2519623603 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7e90045620 con 0x7f7e9815fda0 2024-09-10T21:42:20.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.963+0000 7f7e8f400640 1 --2- 172.21.15.7:0/2519623603 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7e74077f70 0x7f7e7407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:20.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.963+0000 7f7e8f400640 1 -- 172.21.15.7:0/2519623603 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7e900bb7a0 con 0x7f7e9815fda0 2024-09-10T21:42:20.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.964+0000 7f7e96a00640 1 --2- 172.21.15.7:0/2519623603 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7e74077f70 0x7f7e7407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:20.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.964+0000 7f7e96a00640 1 --2- 172.21.15.7:0/2519623603 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7e74077f70 0x7f7e7407a430 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7e88005fd0 tx=0x7f7e8800b590 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:20.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:20.967+0000 7f7e8f400640 1 -- 172.21.15.7:0/2519623603 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7e90084210 con 0x7f7e9815fda0 2024-09-10T21:42:21.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.197+0000 7f7e9d600640 1 -- 172.21.15.7:0/2519623603 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7e980008d0 con 0x7f7e9815fda0 2024-09-10T21:42:21.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.198+0000 7f7e8f400640 1 -- 172.21.15.7:0/2519623603 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5227 (secure 0 0 0) 0x7f7e90083b10 con 0x7f7e9815fda0 2024-09-10T21:42:21.199 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:21.199 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"btime":"2024-09-10T21:42:12:706481+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4342,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3303727910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3303727910},{"type":"v1","addr":"172.21.15.7:6835","nonce":3303727910}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4383,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3193801272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3193801272},{"type":"v1","addr":"172.21.15.7:6834","nonce":3193801272}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4401,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1546148687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1546148687},{"type":"v1","addr":"172.21.15.133:6837","nonce":1546148687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:42:09.121017+0000","modified":"2024-09-10T21:42:12.137567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234179":{"gid":234179,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6836/3769241160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3769241160},{"type":"v1","addr":"172.21.15.133:6836","nonce":3769241160}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234179,"qdb_cluster":[234179]},"id":1}]} 2024-09-10T21:42:21.199 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-10T21:42:21.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.200+0000 7f7e9d600640 1 -- 172.21.15.7:0/2519623603 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7e74077f70 msgr2=0x7f7e7407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:21.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.200+0000 7f7e9d600640 1 --2- 172.21.15.7:0/2519623603 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7e74077f70 0x7f7e7407a430 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f7e88005fd0 tx=0x7f7e8800b590 comp rx=0 tx=0).stop 2024-09-10T21:42:21.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.200+0000 7f7e9d600640 1 -- 172.21.15.7:0/2519623603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9815fda0 msgr2=0x7f7e9815aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:21.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.200+0000 7f7e9d600640 1 --2- 172.21.15.7:0/2519623603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9815fda0 0x7f7e9815aa10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f7e90006010 tx=0x7f7e90009360 comp rx=0 tx=0).stop 2024-09-10T21:42:21.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.200+0000 7f7e9d600640 1 -- 172.21.15.7:0/2519623603 shutdown_connections 2024-09-10T21:42:21.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.200+0000 7f7e9d600640 1 --2- 172.21.15.7:0/2519623603 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7e74077f70 0x7f7e7407a430 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.200+0000 7f7e9d600640 1 --2- 172.21.15.7:0/2519623603 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7e9815bc90 0x7f7e9815e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.200+0000 7f7e9d600640 1 --2- 172.21.15.7:0/2519623603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7e9815fda0 0x7f7e9815aa10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.200+0000 7f7e9d600640 1 --2- 172.21.15.7:0/2519623603 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7e980cd750 0x7f7e980725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.200+0000 7f7e9d600640 1 -- 172.21.15.7:0/2519623603 >> 172.21.15.7:0/2519623603 conn(0x7f7e980b16d0 msgr2=0x7f7e980b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:21.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.200+0000 7f7e9d600640 1 -- 172.21.15.7:0/2519623603 shutdown_connections 2024-09-10T21:42:21.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.200+0000 7f7e9d600640 1 -- 172.21.15.7:0/2519623603 wait complete. 2024-09-10T21:42:21.217 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:42:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.377+0000 7f4ca3c00640 1 -- 172.21.15.7:0/1622986034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c9c1619e0 msgr2=0x7f4c9c15e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.377+0000 7f4ca3c00640 1 --2- 172.21.15.7:0/1622986034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c9c1619e0 0x7f4c9c15e3c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0cd6a0 tx=0x7f4c8c02a5f0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.377+0000 7f4ca3c00640 1 -- 172.21.15.7:0/1622986034 shutdown_connections 2024-09-10T21:42:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.377+0000 7f4ca3c00640 1 --2- 172.21.15.7:0/1622986034 >> v2:172.21.15.133:3300/0 conn(0x7f4c9c0cbd50 0x7f4c9c0cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.377+0000 7f4ca3c00640 1 --2- 172.21.15.7:0/1622986034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c9c1619e0 0x7f4c9c15e3c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.377+0000 7f4ca3c00640 1 -- 172.21.15.7:0/1622986034 >> 172.21.15.7:0/1622986034 conn(0x7f4c9c0b16d0 msgr2=0x7f4c9c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.377+0000 7f4ca3c00640 1 -- 172.21.15.7:0/1622986034 shutdown_connections 2024-09-10T21:42:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.377+0000 7f4ca3c00640 1 -- 172.21.15.7:0/1622986034 wait complete. 2024-09-10T21:42:21.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.379+0000 7f4ca3c00640 1 Processor -- start 2024-09-10T21:42:21.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.379+0000 7f4ca3c00640 1 -- start start 2024-09-10T21:42:21.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.380+0000 7f4ca3c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c9c0ae020 0x7f4c9c1f58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:21.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.380+0000 7f4ca3c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c9c0cbd50 0x7f4c9c0b5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:21.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.380+0000 7f4ca3c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c9c1619e0 0x7f4c9c1f8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.380+0000 7f4ca3c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c0ae890 con 0x7f4c9c0cbd50 2024-09-10T21:42:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.380+0000 7f4ca3c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c0ae710 con 0x7f4c9c0ae020 2024-09-10T21:42:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.380+0000 7f4ca3c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4c9c161dc0 con 0x7f4c9c1619e0 2024-09-10T21:42:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.380+0000 7f4c93e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c9c0cbd50 0x7f4c9c0b5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.380+0000 7f4c93e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c9c0cbd50 0x7f4c9c0b5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54492/0 (socket says 172.21.15.7:54492) 2024-09-10T21:42:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.380+0000 7f4c93e00640 1 -- 172.21.15.7:0/1299178699 learned_addr learned my addr 172.21.15.7:0/1299178699 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.380+0000 7f4ca1c00640 1 --2- 172.21.15.7:0/1299178699 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c9c1619e0 0x7f4c9c1f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.380+0000 7f4c93e00640 1 -- 172.21.15.7:0/1299178699 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c9c1619e0 msgr2=0x7f4c9c1f8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.380+0000 7f4ca1200640 1 --2- 172.21.15.7:0/1299178699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c9c0ae020 0x7f4c9c1f58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.380+0000 7f4c93e00640 1 --2- 172.21.15.7:0/1299178699 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c9c1619e0 0x7f4c9c1f8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.381+0000 7f4c93e00640 1 -- 172.21.15.7:0/1299178699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c9c0ae020 msgr2=0x7f4c9c1f58c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.381+0000 7f4c93e00640 1 --2- 172.21.15.7:0/1299178699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c9c0ae020 0x7f4c9c1f58c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.381+0000 7f4c93e00640 1 -- 172.21.15.7:0/1299178699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9c0b58c0 con 0x7f4c9c0cbd50 2024-09-10T21:42:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.381+0000 7f4ca1c00640 1 --2- 172.21.15.7:0/1299178699 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c9c1619e0 0x7f4c9c1f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.381+0000 7f4ca1200640 1 --2- 172.21.15.7:0/1299178699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c9c0ae020 0x7f4c9c1f58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.381+0000 7f4c93e00640 1 --2- 172.21.15.7:0/1299178699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c9c0cbd50 0x7f4c9c0b5380 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400b4a0 tx=0x7f4c8400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.381+0000 7f4c99600640 1 -- 172.21.15.7:0/1299178699 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84014070 con 0x7f4c9c0cbd50 2024-09-10T21:42:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.381+0000 7f4c99600640 1 -- 172.21.15.7:0/1299178699 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c84010700 con 0x7f4c9c0cbd50 2024-09-10T21:42:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.381+0000 7f4c99600640 1 -- 172.21.15.7:0/1299178699 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c84010b50 con 0x7f4c9c0cbd50 2024-09-10T21:42:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.381+0000 7f4ca3c00640 1 -- 172.21.15.7:0/1299178699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c9c0c8b60 con 0x7f4c9c0cbd50 2024-09-10T21:42:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.381+0000 7f4ca3c00640 1 -- 172.21.15.7:0/1299178699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c9c1f8d90 con 0x7f4c9c0cbd50 2024-09-10T21:42:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.382+0000 7f4ca3c00640 1 -- 172.21.15.7:0/1299178699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c9c0cc1c0 con 0x7f4c9c0cbd50 2024-09-10T21:42:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.384+0000 7f4c99600640 1 -- 172.21.15.7:0/1299178699 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4c84017360 con 0x7f4c9c0cbd50 2024-09-10T21:42:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.384+0000 7f4c99600640 1 --2- 172.21.15.7:0/1299178699 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c78077fa0 0x7f4c7807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:21.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.385+0000 7f4c99600640 1 -- 172.21.15.7:0/1299178699 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4c84099ad0 con 0x7f4c9c0cbd50 2024-09-10T21:42:21.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.385+0000 7f4ca1200640 1 --2- 172.21.15.7:0/1299178699 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c78077fa0 0x7f4c7807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:21.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.386+0000 7f4ca1200640 1 --2- 172.21.15.7:0/1299178699 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c78077fa0 0x7f4c7807a460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c033000 tx=0x7f4c8c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:21.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.388+0000 7f4c99600640 1 -- 172.21.15.7:0/1299178699 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4c840625c0 con 0x7f4c9c0cbd50 2024-09-10T21:42:21.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.583+0000 7f4ca3c00640 1 -- 172.21.15.7:0/1299178699 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c9c0b0b40 con 0x7f4c9c0cbd50 2024-09-10T21:42:21.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.584+0000 7f4c99600640 1 -- 172.21.15.7:0/1299178699 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16629 (secure 0 0 0) 0x7f4c84061ec0 con 0x7f4c9c0cbd50 2024-09-10T21:42:21.586 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:21.586 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:42:10.684978+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:42:07.149267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:42:08.077083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:42:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.587+0000 7f4ca3c00640 1 -- 172.21.15.7:0/1299178699 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c78077fa0 msgr2=0x7f4c7807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.587+0000 7f4ca3c00640 1 --2- 172.21.15.7:0/1299178699 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c78077fa0 0x7f4c7807a460 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c033000 tx=0x7f4c8c004730 comp rx=0 tx=0).stop 2024-09-10T21:42:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.587+0000 7f4ca3c00640 1 -- 172.21.15.7:0/1299178699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c9c0cbd50 msgr2=0x7f4c9c0b5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.587+0000 7f4ca3c00640 1 --2- 172.21.15.7:0/1299178699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c9c0cbd50 0x7f4c9c0b5380 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400b4a0 tx=0x7f4c8400b960 comp rx=0 tx=0).stop 2024-09-10T21:42:21.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.587+0000 7f4ca3c00640 1 -- 172.21.15.7:0/1299178699 shutdown_connections 2024-09-10T21:42:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.587+0000 7f4ca3c00640 1 --2- 172.21.15.7:0/1299178699 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c78077fa0 0x7f4c7807a460 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.587+0000 7f4ca3c00640 1 --2- 172.21.15.7:0/1299178699 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c9c1619e0 0x7f4c9c1f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.587+0000 7f4ca3c00640 1 --2- 172.21.15.7:0/1299178699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c9c0cbd50 0x7f4c9c0b5380 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.587+0000 7f4ca3c00640 1 --2- 172.21.15.7:0/1299178699 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c9c0ae020 0x7f4c9c1f58c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.587+0000 7f4ca3c00640 1 -- 172.21.15.7:0/1299178699 >> 172.21.15.7:0/1299178699 conn(0x7f4c9c0b16d0 msgr2=0x7f4c9c1581c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.587+0000 7f4ca3c00640 1 -- 172.21.15.7:0/1299178699 shutdown_connections 2024-09-10T21:42:21.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.587+0000 7f4ca3c00640 1 -- 172.21.15.7:0/1299178699 wait complete. 2024-09-10T21:42:21.605 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:42:21.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.762+0000 7f567be00640 1 -- 172.21.15.7:0/2518879087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f567c0d0e70 msgr2=0x7f567c0d4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:21.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.762+0000 7f567be00640 1 --2- 172.21.15.7:0/2518879087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f567c0d0e70 0x7f567c0d4870 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f5668005fd0 tx=0x7f566802a540 comp rx=0 tx=0).stop 2024-09-10T21:42:21.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.763+0000 7f567be00640 1 -- 172.21.15.7:0/2518879087 shutdown_connections 2024-09-10T21:42:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.763+0000 7f567be00640 1 --2- 172.21.15.7:0/2518879087 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f567c0d0e70 0x7f567c0d4870 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.763+0000 7f567be00640 1 -- 172.21.15.7:0/2518879087 >> 172.21.15.7:0/2518879087 conn(0x7f567c0d3d50 msgr2=0x7f567c0d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.763+0000 7f567be00640 1 -- 172.21.15.7:0/2518879087 shutdown_connections 2024-09-10T21:42:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.763+0000 7f567be00640 1 -- 172.21.15.7:0/2518879087 wait complete. 2024-09-10T21:42:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.764+0000 7f567be00640 1 Processor -- start 2024-09-10T21:42:21.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.764+0000 7f567be00640 1 -- start start 2024-09-10T21:42:21.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.765+0000 7f567be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f567c0b2b50 0x7f567c1f5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:21.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.765+0000 7f567be00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f567c0d0e70 0x7f567c1f1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:21.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.765+0000 7f567be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f567c15dfd0 0x7f567c160300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:21.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.765+0000 7f567be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f567c0d2770 con 0x7f567c0b2b50 2024-09-10T21:42:21.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.765+0000 7f567be00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f567c0d25f0 con 0x7f567c15dfd0 2024-09-10T21:42:21.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.765+0000 7f567be00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f567c0d28f0 con 0x7f567c0d0e70 2024-09-10T21:42:21.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.765+0000 7f567aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f567c0b2b50 0x7f567c1f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:21.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.765+0000 7f567aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f567c0b2b50 0x7f567c1f5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54508/0 (socket says 172.21.15.7:54508) 2024-09-10T21:42:21.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.765+0000 7f567aa00640 1 -- 172.21.15.7:0/3418268882 learned_addr learned my addr 172.21.15.7:0/3418268882 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:21.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.766+0000 7f567b400640 1 --2- 172.21.15.7:0/3418268882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f567c15dfd0 0x7f567c160300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:21.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.766+0000 7f567a000640 1 --2- 172.21.15.7:0/3418268882 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f567c0d0e70 0x7f567c1f1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:21.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.766+0000 7f567aa00640 1 -- 172.21.15.7:0/3418268882 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f567c0d0e70 msgr2=0x7f567c1f1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:21.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.766+0000 7f567aa00640 1 --2- 172.21.15.7:0/3418268882 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f567c0d0e70 0x7f567c1f1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.766+0000 7f567aa00640 1 -- 172.21.15.7:0/3418268882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f567c15dfd0 msgr2=0x7f567c160300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:21.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.766+0000 7f567aa00640 1 --2- 172.21.15.7:0/3418268882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f567c15dfd0 0x7f567c160300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.766+0000 7f567aa00640 1 -- 172.21.15.7:0/3418268882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f567c1f5740 con 0x7f567c0b2b50 2024-09-10T21:42:21.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.766+0000 7f567aa00640 1 --2- 172.21.15.7:0/3418268882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f567c0b2b50 0x7f567c1f5200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f566c00c3d0 tx=0x7f566c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:21.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.766+0000 7f5672a00640 1 -- 172.21.15.7:0/3418268882 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c017070 con 0x7f567c0b2b50 2024-09-10T21:42:21.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.766+0000 7f5672a00640 1 -- 172.21.15.7:0/3418268882 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f566c0053c0 con 0x7f567c0b2b50 2024-09-10T21:42:21.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.766+0000 7f5672a00640 1 -- 172.21.15.7:0/3418268882 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c003850 con 0x7f567c0b2b50 2024-09-10T21:42:21.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.766+0000 7f567be00640 1 -- 172.21.15.7:0/3418268882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f567c1f21c0 con 0x7f567c0b2b50 2024-09-10T21:42:21.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.767+0000 7f567be00640 1 -- 172.21.15.7:0/3418268882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f567c15e820 con 0x7f567c0b2b50 2024-09-10T21:42:21.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.768+0000 7f567be00640 1 -- 172.21.15.7:0/3418268882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f567c0d2dd0 con 0x7f567c0b2b50 2024-09-10T21:42:21.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.769+0000 7f5672a00640 1 -- 172.21.15.7:0/3418268882 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f566c0039f0 con 0x7f567c0b2b50 2024-09-10T21:42:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.770+0000 7f5672a00640 1 --2- 172.21.15.7:0/3418268882 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f565c077f70 0x7f565c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.770+0000 7f5672a00640 1 -- 172.21.15.7:0/3418268882 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f566c099430 con 0x7f567c0b2b50 2024-09-10T21:42:21.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.770+0000 7f567a000640 1 --2- 172.21.15.7:0/3418268882 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f565c077f70 0x7f565c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:21.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.771+0000 7f567a000640 1 --2- 172.21.15.7:0/3418268882 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f565c077f70 0x7f565c07a430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5668033000 tx=0x7f5668004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:21.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:21.774+0000 7f5672a00640 1 -- 172.21.15.7:0/3418268882 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f567c1f21c0 con 0x7f567c0b2b50 2024-09-10T21:42:22.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.008+0000 7f567be00640 1 -- 172.21.15.7:0/3418268882 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f567c0b9cd0 con 0x7f567c0b2b50 2024-09-10T21:42:22.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.008+0000 7f5672a00640 1 -- 172.21.15.7:0/3418268882 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5227 (secure 0 0 0) 0x7f567c1f5740 con 0x7f567c0b2b50 2024-09-10T21:42:22.010 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:22.010 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"btime":"2024-09-10T21:42:12:706481+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4342,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3303727910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3303727910},{"type":"v1","addr":"172.21.15.7:6835","nonce":3303727910}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4383,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3193801272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3193801272},{"type":"v1","addr":"172.21.15.7:6834","nonce":3193801272}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4401,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1546148687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1546148687},{"type":"v1","addr":"172.21.15.133:6837","nonce":1546148687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:42:09.121017+0000","modified":"2024-09-10T21:42:12.137567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234179":{"gid":234179,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6836/3769241160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3769241160},{"type":"v1","addr":"172.21.15.133:6836","nonce":3769241160}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234179,"qdb_cluster":[234179]},"id":1}]} 2024-09-10T21:42:22.010 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-10T21:42:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.011+0000 7f567be00640 1 -- 172.21.15.7:0/3418268882 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f565c077f70 msgr2=0x7f565c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:22.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.011+0000 7f567be00640 1 --2- 172.21.15.7:0/3418268882 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f565c077f70 0x7f565c07a430 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f5668033000 tx=0x7f5668004730 comp rx=0 tx=0).stop 2024-09-10T21:42:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.011+0000 7f567be00640 1 -- 172.21.15.7:0/3418268882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f567c0b2b50 msgr2=0x7f567c1f5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.011+0000 7f567be00640 1 --2- 172.21.15.7:0/3418268882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f567c0b2b50 0x7f567c1f5200 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f566c00c3d0 tx=0x7f566c00c890 comp rx=0 tx=0).stop 2024-09-10T21:42:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.011+0000 7f567be00640 1 -- 172.21.15.7:0/3418268882 shutdown_connections 2024-09-10T21:42:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.011+0000 7f567be00640 1 --2- 172.21.15.7:0/3418268882 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f565c077f70 0x7f565c07a430 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.011+0000 7f567be00640 1 --2- 172.21.15.7:0/3418268882 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f567c15dfd0 0x7f567c160300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.011+0000 7f567be00640 1 --2- 172.21.15.7:0/3418268882 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f567c0d0e70 0x7f567c1f1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.011+0000 7f567be00640 1 --2- 172.21.15.7:0/3418268882 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f567c0b2b50 0x7f567c1f5200 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.011+0000 7f567be00640 1 -- 172.21.15.7:0/3418268882 >> 172.21.15.7:0/3418268882 conn(0x7f567c0d3d50 msgr2=0x7f567c0b3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.011+0000 7f567be00640 1 -- 172.21.15.7:0/3418268882 shutdown_connections 2024-09-10T21:42:22.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.011+0000 7f567be00640 1 -- 172.21.15.7:0/3418268882 wait complete. 2024-09-10T21:42:22.431 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:42:09.121017+0000', 'modified': '2024-09-10T21:42:12.137567+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 234179}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_234179': {'gid': 234179, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6836/3769241160', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 3769241160}, {'type': 'v1', 'addr': '172.21.15.133:6836', 'nonce': 3769241160}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 234179, 'qdb_cluster': [234179]} 2024-09-10T21:42:22.432 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:42:22.432 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T21:42:22.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.593+0000 7f2c97200640 1 -- 172.21.15.7:0/3510572161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c9815dfd0 msgr2=0x7f2c980d4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:22.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.593+0000 7f2c97200640 1 --2- 172.21.15.7:0/3510572161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c9815dfd0 0x7f2c980d4870 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2c98074d80 tx=0x7f2c8802a4d0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.593+0000 7f2c97200640 1 -- 172.21.15.7:0/3510572161 shutdown_connections 2024-09-10T21:42:22.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.593+0000 7f2c97200640 1 --2- 172.21.15.7:0/3510572161 >> v2:172.21.15.133:3300/0 conn(0x7f2c980b2b50 0x7f2c980d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.593+0000 7f2c97200640 1 --2- 172.21.15.7:0/3510572161 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c9815dfd0 0x7f2c980d4870 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.593+0000 7f2c97200640 1 -- 172.21.15.7:0/3510572161 >> 172.21.15.7:0/3510572161 conn(0x7f2c980d3d50 msgr2=0x7f2c980d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.594+0000 7f2c97200640 1 -- 172.21.15.7:0/3510572161 shutdown_connections 2024-09-10T21:42:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.594+0000 7f2c97200640 1 -- 172.21.15.7:0/3510572161 wait complete. 2024-09-10T21:42:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.595+0000 7f2c97200640 1 Processor -- start 2024-09-10T21:42:22.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.595+0000 7f2c97200640 1 -- start start 2024-09-10T21:42:22.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.596+0000 7f2c97200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c980b2b50 0x7f2c9815c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.596+0000 7f2c97200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c980d0ca0 0x7f2c980c9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.596+0000 7f2c97200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c9815dfd0 0x7f2c980ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.596+0000 7f2c97200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2c980b08c0 con 0x7f2c980d0ca0 2024-09-10T21:42:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.596+0000 7f2c97200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2c980d2ef0 con 0x7f2c980b2b50 2024-09-10T21:42:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.596+0000 7f2c97200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f2c980b0a40 con 0x7f2c9815dfd0 2024-09-10T21:42:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.596+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c980d0ca0 0x7f2c980c9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.596+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c980d0ca0 0x7f2c980c9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54518/0 (socket says 172.21.15.7:54518) 2024-09-10T21:42:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.596+0000 7f2c8fe00640 1 -- 172.21.15.7:0/1576371535 learned_addr learned my addr 172.21.15.7:0/1576371535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.596+0000 7f2c95e00640 1 --2- 172.21.15.7:0/1576371535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c980b2b50 0x7f2c9815c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.597+0000 7f2c96800640 1 --2- 172.21.15.7:0/1576371535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c9815dfd0 0x7f2c980ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.597+0000 7f2c8fe00640 1 -- 172.21.15.7:0/1576371535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c9815dfd0 msgr2=0x7f2c980ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.597+0000 7f2c8fe00640 1 --2- 172.21.15.7:0/1576371535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c9815dfd0 0x7f2c980ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.597+0000 7f2c8fe00640 1 -- 172.21.15.7:0/1576371535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c980b2b50 msgr2=0x7f2c9815c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.597+0000 7f2c8fe00640 1 --2- 172.21.15.7:0/1576371535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c980b2b50 0x7f2c9815c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.597+0000 7f2c8fe00640 1 -- 172.21.15.7:0/1576371535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c981f4d40 con 0x7f2c980d0ca0 2024-09-10T21:42:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.597+0000 7f2c96800640 1 --2- 172.21.15.7:0/1576371535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c9815dfd0 0x7f2c980ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.597+0000 7f2c8fe00640 1 --2- 172.21.15.7:0/1576371535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c980d0ca0 0x7f2c980c9c90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000c920 tx=0x7f2c8000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.597+0000 7f2c8e000640 1 -- 172.21.15.7:0/1576371535 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80007c70 con 0x7f2c980d0ca0 2024-09-10T21:42:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.597+0000 7f2c8e000640 1 -- 172.21.15.7:0/1576371535 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c8000f450 con 0x7f2c980d0ca0 2024-09-10T21:42:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.597+0000 7f2c8e000640 1 -- 172.21.15.7:0/1576371535 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8000f8a0 con 0x7f2c980d0ca0 2024-09-10T21:42:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.597+0000 7f2c97200640 1 -- 172.21.15.7:0/1576371535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c981f5020 con 0x7f2c980d0ca0 2024-09-10T21:42:22.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.598+0000 7f2c97200640 1 -- 172.21.15.7:0/1576371535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c981f55e0 con 0x7f2c980d0ca0 2024-09-10T21:42:22.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.600+0000 7f2c8e000640 1 -- 172.21.15.7:0/1576371535 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2c8000fa40 con 0x7f2c980d0ca0 2024-09-10T21:42:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.601+0000 7f2c8e000640 1 --2- 172.21.15.7:0/1576371535 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2c74077f70 0x7f2c7407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.601+0000 7f2c8e000640 1 -- 172.21.15.7:0/1576371535 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2c8009a670 con 0x7f2c980d0ca0 2024-09-10T21:42:22.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.601+0000 7f2c9f835740 1 Processor -- start 2024-09-10T21:42:22.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.601+0000 7f2c9f835740 1 -- start start 2024-09-10T21:42:22.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.601+0000 7f2c95e00640 1 --2- 172.21.15.7:0/1576371535 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2c74077f70 0x7f2c7407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.602+0000 7f2c9f835740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26daaa0 0x55ccf26dae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.602+0000 7f2c9f835740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf2695c10 0x55ccf26d9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.602+0000 7f2c9f835740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf2696190 0x55ccf26d9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.602+0000 7f2c9f835740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ccf2618430 con 0x55ccf2696190 2024-09-10T21:42:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.602+0000 7f2c9f835740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ccf26085d0 con 0x55ccf2695c10 2024-09-10T21:42:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.602+0000 7f2c9f835740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55ccf25f7a00 con 0x55ccf26daaa0 2024-09-10T21:42:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.602+0000 7f2c95e00640 1 --2- 172.21.15.7:0/1576371535 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2c74077f70 0x7f2c7407a430 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2c88033000 tx=0x7f2c88004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.603+0000 7f2c96800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf2696190 0x55ccf26d9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.603+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf2695c10 0x55ccf26d9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:22.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.603+0000 7f2c95e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26daaa0 0x55ccf26dae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.603+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf2695c10 0x55ccf26d9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:49368/0 (socket says 172.21.15.7:49368) 2024-09-10T21:42:22.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.603+0000 7f2c8fe00640 1 -- 172.21.15.7:0/1392995572 learned_addr learned my addr 172.21.15.7:0/1392995572 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.603+0000 7f2c96800640 1 -- 172.21.15.7:0/1392995572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26daaa0 msgr2=0x55ccf26dae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:42:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.603+0000 7f2c96800640 1 --2- 172.21.15.7:0/1392995572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26daaa0 0x55ccf26dae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.603+0000 7f2c96800640 1 -- 172.21.15.7:0/1392995572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf2695c10 msgr2=0x55ccf26d9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:42:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.603+0000 7f2c96800640 1 --2- 172.21.15.7:0/1392995572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf2695c10 0x55ccf26d9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.603+0000 7f2c96800640 1 -- 172.21.15.7:0/1392995572 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccf25c0520 con 0x55ccf2696190 2024-09-10T21:42:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.603+0000 7f2c8fe00640 1 --2- 172.21.15.7:0/1392995572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf2695c10 0x55ccf26d9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:42:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.603+0000 7f2c96800640 1 --2- 172.21.15.7:0/1392995572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf2696190 0x55ccf26d9ac0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2c900043b0 tx=0x7f2c900047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bd876a5514a6126 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.604+0000 7f2c86600640 1 -- 172.21.15.7:0/1392995572 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c9000ea10 con 0x55ccf2696190 2024-09-10T21:42:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.604+0000 7f2c86600640 1 -- 172.21.15.7:0/1392995572 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c9000ebb0 con 0x55ccf2696190 2024-09-10T21:42:22.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.604+0000 7f2c86600640 1 -- 172.21.15.7:0/1392995572 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90004940 con 0x55ccf2696190 2024-09-10T21:42:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.604+0000 7f2c9f835740 1 -- 172.21.15.7:0/1392995572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf2696190 msgr2=0x55ccf26d9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.604+0000 7f2c9f835740 1 --2- 172.21.15.7:0/1392995572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf2696190 0x55ccf26d9ac0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2c900043b0 tx=0x7f2c900047b0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.604+0000 7f2c9f835740 1 -- 172.21.15.7:0/1392995572 shutdown_connections 2024-09-10T21:42:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.604+0000 7f2c9f835740 1 --2- 172.21.15.7:0/1392995572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf2695c10 0x55ccf26d9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.604+0000 7f2c9f835740 1 --2- 172.21.15.7:0/1392995572 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf2696190 0x55ccf26d9ac0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.604+0000 7f2c9f835740 1 --2- 172.21.15.7:0/1392995572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26daaa0 0x55ccf26dae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.604+0000 7f2c9f835740 1 -- 172.21.15.7:0/1392995572 >> 172.21.15.7:0/1392995572 conn(0x55ccf26911e0 msgr2=0x55ccf2694c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.604+0000 7f2c9f835740 1 -- 172.21.15.7:0/1392995572 shutdown_connections 2024-09-10T21:42:22.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.604+0000 7f2c9f835740 1 -- 172.21.15.7:0/1392995572 wait complete. 2024-09-10T21:42:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.605+0000 7f2c9f835740 1 -- start start 2024-09-10T21:42:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.605+0000 7f2c9f835740 1 Processor -- start 2024-09-10T21:42:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.606+0000 7f2c9f835740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf2696190 0x55ccf26ea6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.606+0000 7f2c9f835740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26daaa0 0x55ccf26e9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.606+0000 7f2c9f835740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf26e98d0 0x55ccf26d94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.606+0000 7f2c9f835740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ccf259e330 con 0x55ccf2696190 2024-09-10T21:42:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.606+0000 7f2c9f835740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ccf25fb810 con 0x55ccf26e98d0 2024-09-10T21:42:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.606+0000 7f2c9f835740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55ccf2368620 con 0x55ccf26daaa0 2024-09-10T21:42:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.606+0000 7f2c9f835740 10 client.0 _ll_register_callbacks cb 0x55ccf25a3420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:42:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.606+0000 7f2c95e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf2696190 0x55ccf26ea6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.606+0000 7f2c95e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf2696190 0x55ccf26ea6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54536/0 (socket says 172.21.15.7:54536) 2024-09-10T21:42:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.606+0000 7f2c95e00640 1 -- 172.21.15.7:0/3571769482 learned_addr learned my addr 172.21.15.7:0/3571769482 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.607+0000 7f2c8fe00640 1 --2- 172.21.15.7:0/3571769482 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26daaa0 0x55ccf26e9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.607+0000 7f2c96800640 1 --2- 172.21.15.7:0/3571769482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf26e98d0 0x55ccf26d94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.607+0000 7f2c95e00640 1 -- 172.21.15.7:0/3571769482 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26daaa0 msgr2=0x55ccf26e9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:42:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.607+0000 7f2c95e00640 1 --2- 172.21.15.7:0/3571769482 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26daaa0 0x55ccf26e9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.607+0000 7f2c95e00640 1 -- 172.21.15.7:0/3571769482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf26e98d0 msgr2=0x55ccf26d94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:42:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.607+0000 7f2c95e00640 1 --2- 172.21.15.7:0/3571769482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf26e98d0 0x55ccf26d94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.607+0000 7f2c95e00640 1 -- 172.21.15.7:0/3571769482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccf2359850 con 0x55ccf2696190 2024-09-10T21:42:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.607+0000 7f2c95e00640 1 --2- 172.21.15.7:0/3571769482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf2696190 0x55ccf26ea6c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2c880099c0 tx=0x7f2c88009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488032538ed0979a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:22.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.607+0000 7f2c7aa00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:42:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.607+0000 7f2c7aa00640 1 -- 172.21.15.7:0/3571769482 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8804e030 con 0x55ccf2696190 2024-09-10T21:42:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.607+0000 7f2c7aa00640 1 -- 172.21.15.7:0/3571769482 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2c8803c880 con 0x55ccf2696190 2024-09-10T21:42:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.607+0000 7f2c9f835740 1 -- 172.21.15.7:0/3571769482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c9000ebb0 con 0x55ccf2696190 2024-09-10T21:42:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.608+0000 7f2c7aa00640 1 -- 172.21.15.7:0/3571769482 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f2c88035040 con 0x55ccf2696190 2024-09-10T21:42:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.608+0000 7f2c9f835740 10 client.4671 fetch_fsmap learned FSMap version 8 2024-09-10T21:42:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.608+0000 7f2c9f835740 1 -- 172.21.15.7:0/3571769482 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55ccf23d06b0 con 0x55ccf2696190 2024-09-10T21:42:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.608+0000 7f2c7aa00640 1 -- 172.21.15.7:0/3571769482 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c9000ebb0 con 0x55ccf2696190 2024-09-10T21:42:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.608+0000 7f2c7aa00640 1 -- 172.21.15.7:0/3571769482 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f2c8803a040 con 0x55ccf2696190 2024-09-10T21:42:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.608+0000 7f2c9f835740 10 client.4671 fetch_fsmap finished waiting for FSMap version 8 2024-09-10T21:42:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.608+0000 7f2c9f835740 10 client.4671 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:42:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.608+0000 7f2c9f835740 20 client.4671 populate_metadata read hostname 'smithi007' 2024-09-10T21:42:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.608+0000 7f2c9f835740 1 --2- 172.21.15.7:0/3571769482 >> [v2:172.21.15.133:6834/3769241160,v1:172.21.15.133:6836/3769241160] conn(0x55ccf26a99c0 0x55ccf26ed5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:22.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.608+0000 7f2c9f835740 4 client.4671 mds_command: new command op to 234179 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:42:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.608+0000 7f2c9f835740 1 -- 172.21.15.7:0/3571769482 --> [v2:172.21.15.133:6834/3769241160,v1:172.21.15.133:6836/3769241160] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccf260a520 con 0x55ccf26a99c0 2024-09-10T21:42:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.608+0000 7f2c8fe00640 1 --2- 172.21.15.7:0/3571769482 >> [v2:172.21.15.133:6834/3769241160,v1:172.21.15.133:6836/3769241160] conn(0x55ccf26a99c0 0x55ccf26ed5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.609+0000 7f2c8fe00640 1 --2- 172.21.15.7:0/3571769482 >> [v2:172.21.15.133:6834/3769241160,v1:172.21.15.133:6836/3769241160] conn(0x55ccf26a99c0 0x55ccf26ed5c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80bdb319497ddfd3 server_cookie=621a6103db468262 in_seq=0 out_seq=0 2024-09-10T21:42:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.609+0000 7f2c7aa00640 10 client.4671 ms_handle_connect on v2:172.21.15.133:6834/3769241160 2024-09-10T21:42:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.610+0000 7f2c7aa00640 1 -- 172.21.15.7:0/3571769482 <== mds.0 v2:172.21.15.133:6834/3769241160 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ccf260a520 con 0x55ccf26a99c0 2024-09-10T21:42:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.610+0000 7f2c7aa00640 10 client.4671 handle_command_reply: tid=0 2024-09-10T21:42:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.610+0000 7f2c9f835740 1 client.4671 shutdown 2024-09-10T21:42:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.610+0000 7f2c9f835740 1 -- 172.21.15.7:0/3571769482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf2696190 msgr2=0x55ccf26ea6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.610+0000 7f2c9f835740 1 --2- 172.21.15.7:0/3571769482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf2696190 0x55ccf26ea6c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2c880099c0 tx=0x7f2c88009260 comp rx=0 tx=0).stop 2024-09-10T21:42:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.610+0000 7f2c9f835740 1 -- 172.21.15.7:0/3571769482 shutdown_connections 2024-09-10T21:42:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.610+0000 7f2c9f835740 1 --2- 172.21.15.7:0/3571769482 >> [v2:172.21.15.133:6834/3769241160,v1:172.21.15.133:6836/3769241160] conn(0x55ccf26a99c0 0x55ccf26ed5c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.610+0000 7f2c9f835740 1 --2- 172.21.15.7:0/3571769482 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf26e98d0 0x55ccf26d94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.610+0000 7f2c9f835740 1 --2- 172.21.15.7:0/3571769482 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26daaa0 0x55ccf26e9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.610+0000 7f2c7aa00640 0 client.4671 ms_handle_reset on v2:172.21.15.133:6834/3769241160 2024-09-10T21:42:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.610+0000 7f2c9f835740 1 --2- 172.21.15.7:0/3571769482 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf2696190 0x55ccf26ea6c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.610+0000 7f2c9f835740 1 -- 172.21.15.7:0/3571769482 >> 172.21.15.7:0/3571769482 conn(0x55ccf2695c10 msgr2=0x55ccf26d3680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.610+0000 7f2c9f835740 1 -- 172.21.15.7:0/3571769482 shutdown_connections 2024-09-10T21:42:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.611+0000 7f2c9f835740 1 -- 172.21.15.7:0/3571769482 wait complete. 2024-09-10T21:42:22.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.611+0000 7f2c9f835740 20 client.4671 trim_cache size 0 max 16384 2024-09-10T21:42:22.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.630+0000 7f2c9f835740 1 Processor -- start 2024-09-10T21:42:22.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.630+0000 7f2c9f835740 1 -- start start 2024-09-10T21:42:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.631+0000 7f2c9f835740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf2696190 0x55ccf26abef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.631+0000 7f2c9f835740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26daaa0 0x55ccf26fefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:22.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.631+0000 7f2c9f835740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf26e98d0 0x55ccf26bdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.631+0000 7f2c9f835740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2c982000b0 con 0x55ccf26e98d0 2024-09-10T21:42:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.631+0000 7f2c9f835740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2c981ff3f0 con 0x55ccf2696190 2024-09-10T21:42:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.631+0000 7f2c9f835740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f2c981ffa40 con 0x55ccf26daaa0 2024-09-10T21:42:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.631+0000 7f2c95e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf2696190 0x55ccf26abef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.631+0000 7f2c96800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf26e98d0 0x55ccf26bdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.631+0000 7f2c95e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf2696190 0x55ccf26abef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:49384/0 (socket says 172.21.15.7:49384) 2024-09-10T21:42:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.631+0000 7f2c95e00640 1 -- 172.21.15.7:0/1544824588 learned_addr learned my addr 172.21.15.7:0/1544824588 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.631+0000 7f2c8fe00640 1 --2- 172.21.15.7:0/1544824588 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26daaa0 0x55ccf26fefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.631+0000 7f2c96800640 1 -- 172.21.15.7:0/1544824588 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26daaa0 msgr2=0x55ccf26fefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:42:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.631+0000 7f2c96800640 1 --2- 172.21.15.7:0/1544824588 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26daaa0 0x55ccf26fefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.631+0000 7f2c96800640 1 -- 172.21.15.7:0/1544824588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf2696190 msgr2=0x55ccf26abef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:42:22.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.631+0000 7f2c96800640 1 --2- 172.21.15.7:0/1544824588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf2696190 0x55ccf26abef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.631+0000 7f2c96800640 1 -- 172.21.15.7:0/1544824588 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccf26d9840 con 0x55ccf26e98d0 2024-09-10T21:42:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.632+0000 7f2c96800640 1 --2- 172.21.15.7:0/1544824588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf26e98d0 0x55ccf26bdf40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2c9815e740 tx=0x7f2c90016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e4356f1150ff3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.632+0000 7f2c85c00640 1 -- 172.21.15.7:0/1544824588 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90004b40 con 0x55ccf26e98d0 2024-09-10T21:42:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.632+0000 7f2c85c00640 1 -- 172.21.15.7:0/1544824588 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c90004df0 con 0x55ccf26e98d0 2024-09-10T21:42:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.632+0000 7f2c85c00640 1 -- 172.21.15.7:0/1544824588 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c90005270 con 0x55ccf26e98d0 2024-09-10T21:42:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.632+0000 7f2c9f835740 1 -- 172.21.15.7:0/1544824588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf26e98d0 msgr2=0x55ccf26bdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.632+0000 7f2c9f835740 1 --2- 172.21.15.7:0/1544824588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf26e98d0 0x55ccf26bdf40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2c9815e740 tx=0x7f2c90016040 comp rx=0 tx=0).stop 2024-09-10T21:42:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.632+0000 7f2c9f835740 1 -- 172.21.15.7:0/1544824588 shutdown_connections 2024-09-10T21:42:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.632+0000 7f2c9f835740 1 --2- 172.21.15.7:0/1544824588 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf26e98d0 0x55ccf26bdf40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.632+0000 7f2c9f835740 1 --2- 172.21.15.7:0/1544824588 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26daaa0 0x55ccf26fefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.632+0000 7f2c9f835740 1 --2- 172.21.15.7:0/1544824588 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf2696190 0x55ccf26abef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.632+0000 7f2c9f835740 1 -- 172.21.15.7:0/1544824588 >> 172.21.15.7:0/1544824588 conn(0x55ccf2695c10 msgr2=0x55ccf26f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.633+0000 7f2c9f835740 1 -- 172.21.15.7:0/1544824588 shutdown_connections 2024-09-10T21:42:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.633+0000 7f2c9f835740 1 -- 172.21.15.7:0/1544824588 wait complete. 2024-09-10T21:42:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.633+0000 7f2c9f835740 1 -- start start 2024-09-10T21:42:22.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.633+0000 7f2c9f835740 1 Processor -- start 2024-09-10T21:42:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.634+0000 7f2c9f835740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf2696190 0x55ccf26ee9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.634+0000 7f2c9f835740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf26daaa0 0x55ccf275b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.634+0000 7f2c9f835740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26e98d0 0x55ccf26bdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.634+0000 7f2c9f835740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ccf26a9840 con 0x55ccf26daaa0 2024-09-10T21:42:22.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.634+0000 7f2c9f835740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ccf26ad420 con 0x55ccf2696190 2024-09-10T21:42:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.634+0000 7f2c9f835740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55ccf26a9e00 con 0x55ccf26e98d0 2024-09-10T21:42:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.634+0000 7f2c9f835740 10 client.0 _ll_register_callbacks cb 0x7f2c880036c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:42:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.634+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf26daaa0 0x55ccf275b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.635+0000 7f2c8fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf26daaa0 0x55ccf275b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54562/0 (socket says 172.21.15.7:54562) 2024-09-10T21:42:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.635+0000 7f2c8fe00640 1 -- 172.21.15.7:0/2612348644 learned_addr learned my addr 172.21.15.7:0/2612348644 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.635+0000 7f2c95e00640 1 --2- 172.21.15.7:0/2612348644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf2696190 0x55ccf26ee9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.635+0000 7f2c96800640 1 --2- 172.21.15.7:0/2612348644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26e98d0 0x55ccf26bdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.635+0000 7f2c8fe00640 1 -- 172.21.15.7:0/2612348644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26e98d0 msgr2=0x55ccf26bdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:42:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.635+0000 7f2c8fe00640 1 --2- 172.21.15.7:0/2612348644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26e98d0 0x55ccf26bdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.635+0000 7f2c8fe00640 1 -- 172.21.15.7:0/2612348644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf2696190 msgr2=0x55ccf26ee9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:42:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.635+0000 7f2c8fe00640 1 --2- 172.21.15.7:0/2612348644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf2696190 0x55ccf26ee9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.635+0000 7f2c8fe00640 1 -- 172.21.15.7:0/2612348644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccf26a9180 con 0x55ccf26daaa0 2024-09-10T21:42:22.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.635+0000 7f2c8fe00640 1 --2- 172.21.15.7:0/2612348644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf26daaa0 0x55ccf275b2d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2c80098fd0 tx=0x7f2c8005f150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4d9d8d3011604d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.635+0000 7f2c7aa00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:42:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.635+0000 7f2c7aa00640 1 -- 172.21.15.7:0/2612348644 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8001d040 con 0x55ccf26daaa0 2024-09-10T21:42:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.635+0000 7f2c7aa00640 1 -- 172.21.15.7:0/2612348644 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2c8005ec40 con 0x55ccf26daaa0 2024-09-10T21:42:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.635+0000 7f2c9f835740 1 -- 172.21.15.7:0/2612348644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c90004df0 con 0x55ccf26daaa0 2024-09-10T21:42:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.636+0000 7f2c7aa00640 1 -- 172.21.15.7:0/2612348644 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f2c800674b0 con 0x55ccf26daaa0 2024-09-10T21:42:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.636+0000 7f2c9f835740 10 client.4677 fetch_fsmap learned FSMap version 8 2024-09-10T21:42:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.636+0000 7f2c9f835740 1 -- 172.21.15.7:0/2612348644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x55ccf275ce20 con 0x55ccf26daaa0 2024-09-10T21:42:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.636+0000 7f2c7aa00640 1 -- 172.21.15.7:0/2612348644 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c90004df0 con 0x55ccf26daaa0 2024-09-10T21:42:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.636+0000 7f2c7aa00640 1 -- 172.21.15.7:0/2612348644 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f2c8006ba40 con 0x55ccf26daaa0 2024-09-10T21:42:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.636+0000 7f2c9f835740 10 client.4677 fetch_fsmap finished waiting for FSMap version 8 2024-09-10T21:42:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.636+0000 7f2c9f835740 10 client.4677 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:42:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.636+0000 7f2c9f835740 20 client.4677 populate_metadata read hostname 'smithi007' 2024-09-10T21:42:22.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.636+0000 7f2c9f835740 1 --2- 172.21.15.7:0/2612348644 >> [v2:172.21.15.133:6834/3769241160,v1:172.21.15.133:6836/3769241160] conn(0x55ccf275c9f0 0x55ccf26943f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.636+0000 7f2c9f835740 4 client.4677 mds_command: new command op to 234179 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:42:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.636+0000 7f2c9f835740 1 -- 172.21.15.7:0/2612348644 --> [v2:172.21.15.133:6834/3769241160,v1:172.21.15.133:6836/3769241160] -- command(tid 0: {"prefix": "status"}) -- 0x55ccf2605fd0 con 0x55ccf275c9f0 2024-09-10T21:42:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.636+0000 7f2c95e00640 1 --2- 172.21.15.7:0/2612348644 >> [v2:172.21.15.133:6834/3769241160,v1:172.21.15.133:6836/3769241160] conn(0x55ccf275c9f0 0x55ccf26943f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.637+0000 7f2c95e00640 1 --2- 172.21.15.7:0/2612348644 >> [v2:172.21.15.133:6834/3769241160,v1:172.21.15.133:6836/3769241160] conn(0x55ccf275c9f0 0x55ccf26943f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76a7867c86405771 server_cookie=86f6d507525e7f01 in_seq=0 out_seq=0 2024-09-10T21:42:22.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.637+0000 7f2c7aa00640 10 client.4677 ms_handle_connect on v2:172.21.15.133:6834/3769241160 2024-09-10T21:42:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.637+0000 7f2c7aa00640 1 -- 172.21.15.7:0/2612348644 <== mds.0 v2:172.21.15.133:6834/3769241160 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55ccf2605fd0 con 0x55ccf275c9f0 2024-09-10T21:42:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.637+0000 7f2c7aa00640 10 client.4677 handle_command_reply: tid=0 2024-09-10T21:42:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.637+0000 7f2c9f835740 1 client.4677 shutdown 2024-09-10T21:42:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.638+0000 7f2c9f835740 1 -- 172.21.15.7:0/2612348644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf26daaa0 msgr2=0x55ccf275b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.638+0000 7f2c9f835740 1 --2- 172.21.15.7:0/2612348644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf26daaa0 0x55ccf275b2d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2c80098fd0 tx=0x7f2c8005f150 comp rx=0 tx=0).stop 2024-09-10T21:42:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.638+0000 7f2c9f835740 1 -- 172.21.15.7:0/2612348644 shutdown_connections 2024-09-10T21:42:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.638+0000 7f2c9f835740 1 --2- 172.21.15.7:0/2612348644 >> [v2:172.21.15.133:6834/3769241160,v1:172.21.15.133:6836/3769241160] conn(0x55ccf275c9f0 0x55ccf26943f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.638+0000 7f2c9f835740 1 --2- 172.21.15.7:0/2612348644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ccf26e98d0 0x55ccf26bdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.638+0000 7f2c9f835740 1 --2- 172.21.15.7:0/2612348644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ccf26daaa0 0x55ccf275b2d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.638+0000 7f2c9f835740 1 --2- 172.21.15.7:0/2612348644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ccf2696190 0x55ccf26ee9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.638+0000 7f2c9f835740 1 -- 172.21.15.7:0/2612348644 >> 172.21.15.7:0/2612348644 conn(0x55ccf2695c10 msgr2=0x55ccf26f4c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.638+0000 7f2c7aa00640 0 client.4677 ms_handle_reset on v2:172.21.15.133:6834/3769241160 2024-09-10T21:42:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.638+0000 7f2c9f835740 1 -- 172.21.15.7:0/2612348644 shutdown_connections 2024-09-10T21:42:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.638+0000 7f2c9f835740 1 -- 172.21.15.7:0/2612348644 wait complete. 2024-09-10T21:42:22.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.638+0000 7f2c9f835740 20 client.4677 trim_cache size 0 max 16384 2024-09-10T21:42:22.643 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:42:22.643 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2029451a-3c26-48c7-b874-0597197278ac", 2024-09-10T21:42:22.643 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-10T21:42:22.643 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 234179, 2024-09-10T21:42:22.644 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T21:42:22.644 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T21:42:22.644 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T21:42:22.644 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 12.964056188000001, 2024-09-10T21:42:22.644 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 8, 2024-09-10T21:42:22.644 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 21, 2024-09-10T21:42:22.644 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 21, 2024-09-10T21:42:22.644 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 18.986753527000001 2024-09-10T21:42:22.644 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:42:22.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.638+0000 7f2c97200640 1 -- 172.21.15.7:0/1576371535 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2c74077f70 msgr2=0x7f2c7407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:22.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.638+0000 7f2c97200640 1 --2- 172.21.15.7:0/1576371535 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2c74077f70 0x7f2c7407a430 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2c88033000 tx=0x7f2c88004730 comp rx=0 tx=0).stop 2024-09-10T21:42:22.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.638+0000 7f2c97200640 1 -- 172.21.15.7:0/1576371535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c980d0ca0 msgr2=0x7f2c980c9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:22.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.638+0000 7f2c97200640 1 --2- 172.21.15.7:0/1576371535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c980d0ca0 0x7f2c980c9c90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000c920 tx=0x7f2c8000cde0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.639+0000 7f2c97200640 1 -- 172.21.15.7:0/1576371535 shutdown_connections 2024-09-10T21:42:22.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.639+0000 7f2c97200640 1 --2- 172.21.15.7:0/1576371535 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2c74077f70 0x7f2c7407a430 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.639+0000 7f2c97200640 1 --2- 172.21.15.7:0/1576371535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c9815dfd0 0x7f2c980ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.639+0000 7f2c97200640 1 --2- 172.21.15.7:0/1576371535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c980d0ca0 0x7f2c980c9c90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.639+0000 7f2c97200640 1 --2- 172.21.15.7:0/1576371535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c980b2b50 0x7f2c9815c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:22.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.639+0000 7f2c97200640 1 -- 172.21.15.7:0/1576371535 >> 172.21.15.7:0/1576371535 conn(0x7f2c980d3d50 msgr2=0x7f2c9806d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:22.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.639+0000 7f2c97200640 1 -- 172.21.15.7:0/1576371535 shutdown_connections 2024-09-10T21:42:22.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:22.639+0000 7f2c97200640 1 -- 172.21.15.7:0/1576371535 wait complete. 2024-09-10T21:42:22.655 INFO:teuthology.run_tasks:Running task kclient... 2024-09-10T21:42:22.680 INFO:tasks.kclient:Mounting kernel clients... 2024-09-10T21:42:22.680 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-10T21:42:22.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:22.681 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:42:22.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:22.717 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-09-10T21:42:22.818 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-09-10T21:42:22.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:42:22.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:22.821 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:42:22.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:22.857 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-09-10T21:42:22.956 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-09-10T21:42:22.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:42:22.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:22.959 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-09-10T21:42:22.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:22.999 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2024-09-10T21:42:23.101 INFO:teuthology.orchestra.run.smithi133.stderr:Cannot find device "ceph-brx" 2024-09-10T21:42:23.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:42:23.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:23.104 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-09-10T21:42:23.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:23.139 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2024-09-10T21:42:23.237 INFO:teuthology.orchestra.run.smithi133.stderr:Cannot find device "ceph-brx" 2024-09-10T21:42:23.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:42:23.240 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2024-09-10T21:42:23.240 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T21:42:23.241 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:42:23.241 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:42:23.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.401+0000 7fe9fcc00640 1 -- 172.21.15.7:0/1111104158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f80b4c90 msgr2=0x7fe9f80b2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:23.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.401+0000 7fe9fcc00640 1 --2- 172.21.15.7:0/1111104158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f80b4c90 0x7fe9f80b2a40 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009830 tx=0x7fe9e402a5c0 comp rx=0 tx=0).stop 2024-09-10T21:42:23.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.401+0000 7fe9fcc00640 1 -- 172.21.15.7:0/1111104158 shutdown_connections 2024-09-10T21:42:23.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.401+0000 7fe9fcc00640 1 --2- 172.21.15.7:0/1111104158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f80b4c90 0x7fe9f80b2a40 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:23.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.401+0000 7fe9fcc00640 1 -- 172.21.15.7:0/1111104158 >> 172.21.15.7:0/1111104158 conn(0x7fe9f80b1d70 msgr2=0x7fe9f80cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:23.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.402+0000 7fe9fcc00640 1 -- 172.21.15.7:0/1111104158 shutdown_connections 2024-09-10T21:42:23.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.402+0000 7fe9fcc00640 1 -- 172.21.15.7:0/1111104158 wait complete. 2024-09-10T21:42:23.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.403+0000 7fe9fcc00640 1 Processor -- start 2024-09-10T21:42:23.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.403+0000 7fe9fcc00640 1 -- start start 2024-09-10T21:42:23.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.404+0000 7fe9fcc00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9f80b2f80 0x7fe9f813da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:23.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.404+0000 7fe9fcc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9f80b4c90 0x7fe9f80a9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:23.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.404+0000 7fe9fcc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f8145540 0x7fe9f815acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:23.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.404+0000 7fe9fcc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f80cef80 con 0x7fe9f8145540 2024-09-10T21:42:23.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.404+0000 7fe9fcc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe9f80cee00 con 0x7fe9f80b4c90 2024-09-10T21:42:23.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.404+0000 7fe9fcc00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe9f80cf100 con 0x7fe9f80b2f80 2024-09-10T21:42:23.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.404+0000 7fe9f7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f8145540 0x7fe9f815acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:23.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.404+0000 7fe9f7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f8145540 0x7fe9f815acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54568/0 (socket says 172.21.15.7:54568) 2024-09-10T21:42:23.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.404+0000 7fe9f7e00640 1 -- 172.21.15.7:0/66604514 learned_addr learned my addr 172.21.15.7:0/66604514 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:23.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.405+0000 7fe9f6a00640 1 --2- 172.21.15.7:0/66604514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9f80b4c90 0x7fe9f80a9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:23.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.405+0000 7fe9f7400640 1 --2- 172.21.15.7:0/66604514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9f80b2f80 0x7fe9f813da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:23.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.405+0000 7fe9f7e00640 1 -- 172.21.15.7:0/66604514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9f80b2f80 msgr2=0x7fe9f813da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:23.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.405+0000 7fe9f7e00640 1 --2- 172.21.15.7:0/66604514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9f80b2f80 0x7fe9f813da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:23.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.405+0000 7fe9f7e00640 1 -- 172.21.15.7:0/66604514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9f80b4c90 msgr2=0x7fe9f80a9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:23.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.405+0000 7fe9f7e00640 1 --2- 172.21.15.7:0/66604514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9f80b4c90 0x7fe9f80a9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:23.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.405+0000 7fe9f7e00640 1 -- 172.21.15.7:0/66604514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9f80a9ff0 con 0x7fe9f8145540 2024-09-10T21:42:23.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.405+0000 7fe9f7e00640 1 --2- 172.21.15.7:0/66604514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f8145540 0x7fe9f815acb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000cbc0 tx=0x7fe9f000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:23.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.405+0000 7fe9ef400640 1 -- 172.21.15.7:0/66604514 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0016020 con 0x7fe9f8145540 2024-09-10T21:42:23.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.405+0000 7fe9ef400640 1 -- 172.21.15.7:0/66604514 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9f000ae30 con 0x7fe9f8145540 2024-09-10T21:42:23.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.405+0000 7fe9ef400640 1 -- 172.21.15.7:0/66604514 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0012660 con 0x7fe9f8145540 2024-09-10T21:42:23.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.405+0000 7fe9fcc00640 1 -- 172.21.15.7:0/66604514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9f8073f10 con 0x7fe9f8145540 2024-09-10T21:42:23.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.405+0000 7fe9fcc00640 1 -- 172.21.15.7:0/66604514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9f813e7b0 con 0x7fe9f8145540 2024-09-10T21:42:23.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.406+0000 7fe9fcc00640 1 -- 172.21.15.7:0/66604514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9f80cd9f0 con 0x7fe9f8145540 2024-09-10T21:42:23.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.408+0000 7fe9ef400640 1 -- 172.21.15.7:0/66604514 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe9f001b400 con 0x7fe9f8145540 2024-09-10T21:42:23.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.409+0000 7fe9ef400640 1 --2- 172.21.15.7:0/66604514 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe9d8077fa0 0x7fe9d807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:23.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.409+0000 7fe9ef400640 1 -- 172.21.15.7:0/66604514 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fe9f009a760 con 0x7fe9f8145540 2024-09-10T21:42:23.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.409+0000 7fe9f7400640 1 --2- 172.21.15.7:0/66604514 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe9d8077fa0 0x7fe9d807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:23.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.410+0000 7fe9f7400640 1 --2- 172.21.15.7:0/66604514 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe9d8077fa0 0x7fe9d807a460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8000f60 tx=0x7fe9e8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:23.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.413+0000 7fe9ef400640 1 -- 172.21.15.7:0/66604514 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe9f80cd9f0 con 0x7fe9f8145540 2024-09-10T21:42:23.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.645+0000 7fe9fcc00640 1 -- 172.21.15.7:0/66604514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe9f80cf580 con 0x7fe9f8145540 2024-09-10T21:42:23.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.646+0000 7fe9ef400640 1 -- 172.21.15.7:0/66604514 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fe9f0020020 con 0x7fe9f8145540 2024-09-10T21:42:23.647 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:42:23.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.648+0000 7fe9fcc00640 1 -- 172.21.15.7:0/66604514 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe9d8077fa0 msgr2=0x7fe9d807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:23.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.648+0000 7fe9fcc00640 1 --2- 172.21.15.7:0/66604514 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe9d8077fa0 0x7fe9d807a460 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe9e8000f60 tx=0x7fe9e8009040 comp rx=0 tx=0).stop 2024-09-10T21:42:23.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.648+0000 7fe9fcc00640 1 -- 172.21.15.7:0/66604514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f8145540 msgr2=0x7fe9f815acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:23.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.648+0000 7fe9fcc00640 1 --2- 172.21.15.7:0/66604514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f8145540 0x7fe9f815acb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe9f000cbc0 tx=0x7fe9f000a430 comp rx=0 tx=0).stop 2024-09-10T21:42:23.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.648+0000 7fe9fcc00640 1 -- 172.21.15.7:0/66604514 shutdown_connections 2024-09-10T21:42:23.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.648+0000 7fe9fcc00640 1 --2- 172.21.15.7:0/66604514 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe9d8077fa0 0x7fe9d807a460 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:23.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.648+0000 7fe9fcc00640 1 --2- 172.21.15.7:0/66604514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9f80b4c90 0x7fe9f80a9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:23.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.648+0000 7fe9fcc00640 1 --2- 172.21.15.7:0/66604514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9f8145540 0x7fe9f815acb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:23.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.648+0000 7fe9fcc00640 1 --2- 172.21.15.7:0/66604514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9f80b2f80 0x7fe9f813da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:23.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.648+0000 7fe9fcc00640 1 -- 172.21.15.7:0/66604514 >> 172.21.15.7:0/66604514 conn(0x7fe9f80b1d70 msgr2=0x7fe9f8156150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:23.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.649+0000 7fe9fcc00640 1 -- 172.21.15.7:0/66604514 shutdown_connections 2024-09-10T21:42:23.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23.649+0000 7fe9fcc00640 1 -- 172.21.15.7:0/66604514 wait complete. 2024-09-10T21:42:23.665 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T21:42:23.665 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:42:23.665 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-10T21:42:23.665 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:42:23.665 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:42:23.665 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T21:42:23.666 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T21:42:23.666 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:42:23.666 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:42:23.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:23.666 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-10T21:42:23.701 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:42:23.702 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:42:23.702 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:42:23.702 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:23.702 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T21:42:23.702 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:23.702 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:42:23.702 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T21:42:23.702 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-10T21:42:23.702 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:42:23.702 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T21:42:23.702 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-10T21:42:23.702 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:42:23.702 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-10T21:42:23.703 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-10T21:42:23.703 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T21:42:23.703 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42601sec preferred_lft 42601sec 2024-09-10T21:42:23.703 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-10T21:42:23.703 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:23.703 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:42:23.703 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-10T21:42:23.703 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-10T21:42:23.704 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T21:42:23.704 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:42:23.704 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:42:23.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add name ceph-brx type bridge 2024-09-10T21:42:23.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr flush dev ceph-brx 2024-09-10T21:42:23.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set ceph-brx up 2024-09-10T21:42:23.704 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T21:42:23.704 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:42:23.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:24.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:24.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:24.121 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:42:24.177 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:42:24.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:24.180 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-10T21:42:24.244 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-10T21:42:24.244 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:42:24.244 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T21:42:24.244 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T21:42:24.244 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:42:24.245 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:42:24.245 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:42:24.245 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:42:24.245 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:42:24.245 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:42:24.245 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:42:24.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:24.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:24.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:24.939 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:42:24.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:24.977 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-09-10T21:42:25.053 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:42:25.053 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:42:25.053 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:42:25.053 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T21:42:25.053 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:42:25.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:25.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:25.232 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T21:42:25.233 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:42:25.233 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:42:25.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T21:42:25.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T21:42:25.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T21:42:25.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T21:42:25.233 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T21:42:25.233 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:42:25.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:25.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:25.486 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:42:25.486 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:42:25.486 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-10T21:42:25.486 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T21:42:25.486 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:42:25.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:25 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:25.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:25.654 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:42:25.654 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:42:25.690 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:42:25.690 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:42:25.690 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:42:25.765 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:42:25.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:25.765 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:42:25.924 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:42:25.924 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:42:26.563 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:42:26.563 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:42:26.563 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.133:3300,172.21.15.133:3301" 2024-09-10T21:42:26.564 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@2029451a-3c26-48c7-b874-0597197278ac.cephfs=/ 2024-09-10T21:42:26.564 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.133:3300/172.21.15.133:3301" will pass to kernel 2024-09-10T21:42:26.565 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T21:42:26.565 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:42:26.565 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:42:26.565 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:42:26.565 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:42:26.568 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:42:26.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:26.568 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:42:26.626 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:42:26.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:42:26.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:26.727 INFO:teuthology.orchestra.run.smithi007.stdout:client4689 2024-09-10T21:42:26.727 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:42:26.727 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:42:26.763 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:42:26.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:42:26.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:26.902 INFO:teuthology.orchestra.run.smithi007.stdout:client4689 2024-09-10T21:42:26.902 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:42:26.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:42:26.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:27.002 INFO:teuthology.orchestra.run.smithi007.stdout:client4689 2024-09-10T21:42:27.003 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/2029451a-3c26-48c7-b874-0597197278ac.client4689/status') 2024-09-10T21:42:27.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:27.109 INFO:teuthology.orchestra.run.smithi007.stdout:instance: client.4689 (3)192.168.0.1:0/3550585567 2024-09-10T21:42:27.109 INFO:teuthology.orchestra.run.smithi007.stdout:blocklisted: false 2024-09-10T21:42:27.110 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records in 2024-09-10T21:42:27.110 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records out 2024-09-10T21:42:27.110 INFO:teuthology.orchestra.run.smithi007.stderr:69 bytes copied, 5.9569e-05 s, 1.2 MB/s 2024-09-10T21:42:27.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:27.120 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2024-09-10T21:42:27.120 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T21:42:27.120 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T21:42:27.120 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:42:27.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.279+0000 7fa93da00640 1 -- 172.21.15.7:0/2507120437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9380d1e60 msgr2=0x7fa9380d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.279+0000 7fa93da00640 1 --2- 172.21.15.7:0/2507120437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9380d1e60 0x7fa9380d4900 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa928009830 tx=0x7fa92802a650 comp rx=0 tx=0).stop 2024-09-10T21:42:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.279+0000 7fa93da00640 1 -- 172.21.15.7:0/2507120437 shutdown_connections 2024-09-10T21:42:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.279+0000 7fa93da00640 1 --2- 172.21.15.7:0/2507120437 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9380d1e60 0x7fa9380d4900 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.279+0000 7fa93da00640 1 -- 172.21.15.7:0/2507120437 >> 172.21.15.7:0/2507120437 conn(0x7fa9380d3de0 msgr2=0x7fa9380d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.279+0000 7fa93da00640 1 -- 172.21.15.7:0/2507120437 shutdown_connections 2024-09-10T21:42:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.279+0000 7fa93da00640 1 -- 172.21.15.7:0/2507120437 wait complete. 2024-09-10T21:42:27.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.280+0000 7fa93da00640 1 Processor -- start 2024-09-10T21:42:27.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.281+0000 7fa93da00640 1 -- start start 2024-09-10T21:42:27.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.281+0000 7fa93da00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9380c82d0 0x7fa93815cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:27.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.281+0000 7fa93da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9380d1e60 0x7fa938150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.282+0000 7fa93da00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9380d2a20 0x7fa93814ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.282+0000 7fa93da00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa9380c9220 con 0x7fa9380d1e60 2024-09-10T21:42:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.282+0000 7fa93da00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa9380c90a0 con 0x7fa9380c82d0 2024-09-10T21:42:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.282+0000 7fa93da00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa9380c93a0 con 0x7fa9380d2a20 2024-09-10T21:42:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.282+0000 7fa937400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9380d1e60 0x7fa938150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.282+0000 7fa937400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9380d1e60 0x7fa938150d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45346/0 (socket says 172.21.15.7:45346) 2024-09-10T21:42:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.282+0000 7fa937400640 1 -- 172.21.15.7:0/626851023 learned_addr learned my addr 172.21.15.7:0/626851023 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.282+0000 7fa937e00640 1 --2- 172.21.15.7:0/626851023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9380c82d0 0x7fa93815cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.282+0000 7fa93d000640 1 --2- 172.21.15.7:0/626851023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9380d2a20 0x7fa93814ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.282+0000 7fa937400640 1 -- 172.21.15.7:0/626851023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9380d2a20 msgr2=0x7fa93814ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.282+0000 7fa937400640 1 --2- 172.21.15.7:0/626851023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9380d2a20 0x7fa93814ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:27.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.282+0000 7fa937400640 1 -- 172.21.15.7:0/626851023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9380c82d0 msgr2=0x7fa93815cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.282+0000 7fa937400640 1 --2- 172.21.15.7:0/626851023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9380c82d0 0x7fa93815cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.282+0000 7fa937400640 1 -- 172.21.15.7:0/626851023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa938151450 con 0x7fa9380d1e60 2024-09-10T21:42:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.283+0000 7fa937400640 1 --2- 172.21.15.7:0/626851023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9380d1e60 0x7fa938150d00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa924001eb0 tx=0x7fa92400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.283+0000 7fa92fe00640 1 -- 172.21.15.7:0/626851023 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924011790 con 0x7fa9380d1e60 2024-09-10T21:42:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.283+0000 7fa92fe00640 1 -- 172.21.15.7:0/626851023 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa924017070 con 0x7fa9380d1e60 2024-09-10T21:42:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.283+0000 7fa92fe00640 1 -- 172.21.15.7:0/626851023 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924011ca0 con 0x7fa9380d1e60 2024-09-10T21:42:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.283+0000 7fa93da00640 1 -- 172.21.15.7:0/626851023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa938151730 con 0x7fa9380d1e60 2024-09-10T21:42:27.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.283+0000 7fa93da00640 1 -- 172.21.15.7:0/626851023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9380ccf00 con 0x7fa9380d1e60 2024-09-10T21:42:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.284+0000 7fa93da00640 1 -- 172.21.15.7:0/626851023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9380d4900 con 0x7fa9380d1e60 2024-09-10T21:42:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.285+0000 7fa92fe00640 1 -- 172.21.15.7:0/626851023 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa92401c460 con 0x7fa9380d1e60 2024-09-10T21:42:27.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.286+0000 7fa92fe00640 1 --2- 172.21.15.7:0/626851023 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa918077f70 0x7fa91807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.286+0000 7fa92fe00640 1 -- 172.21.15.7:0/626851023 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa924099b30 con 0x7fa9380d1e60 2024-09-10T21:42:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.287+0000 7fa937e00640 1 --2- 172.21.15.7:0/626851023 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa918077f70 0x7fa91807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:27.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.287+0000 7fa937e00640 1 --2- 172.21.15.7:0/626851023 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa918077f70 0x7fa91807a430 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa928034000 tx=0x7fa928004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:27.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.290+0000 7fa92fe00640 1 -- 172.21.15.7:0/626851023 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fa924062620 con 0x7fa9380d1e60 2024-09-10T21:42:27.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.520+0000 7fa93da00640 1 -- 172.21.15.7:0/626851023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fa9380ce1a0 con 0x7fa9380d1e60 2024-09-10T21:42:27.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.521+0000 7fa92fe00640 1 -- 172.21.15.7:0/626851023 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fa924061f20 con 0x7fa9380d1e60 2024-09-10T21:42:27.522 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:42:27.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.523+0000 7fa93da00640 1 -- 172.21.15.7:0/626851023 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa918077f70 msgr2=0x7fa91807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:27.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.523+0000 7fa93da00640 1 --2- 172.21.15.7:0/626851023 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa918077f70 0x7fa91807a430 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa928034000 tx=0x7fa928004730 comp rx=0 tx=0).stop 2024-09-10T21:42:27.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.523+0000 7fa93da00640 1 -- 172.21.15.7:0/626851023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9380d1e60 msgr2=0x7fa938150d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:27.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.523+0000 7fa93da00640 1 --2- 172.21.15.7:0/626851023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9380d1e60 0x7fa938150d00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa924001eb0 tx=0x7fa92400c890 comp rx=0 tx=0).stop 2024-09-10T21:42:27.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.523+0000 7fa93da00640 1 -- 172.21.15.7:0/626851023 shutdown_connections 2024-09-10T21:42:27.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.523+0000 7fa93da00640 1 --2- 172.21.15.7:0/626851023 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa918077f70 0x7fa91807a430 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:27.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.523+0000 7fa93da00640 1 --2- 172.21.15.7:0/626851023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa9380d2a20 0x7fa93814ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:27.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.523+0000 7fa93da00640 1 --2- 172.21.15.7:0/626851023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa9380d1e60 0x7fa938150d00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:27.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.523+0000 7fa93da00640 1 --2- 172.21.15.7:0/626851023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa9380c82d0 0x7fa93815cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:27.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.523+0000 7fa93da00640 1 -- 172.21.15.7:0/626851023 >> 172.21.15.7:0/626851023 conn(0x7fa9380d3de0 msgr2=0x7fa9380b9100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:27.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.523+0000 7fa93da00640 1 -- 172.21.15.7:0/626851023 shutdown_connections 2024-09-10T21:42:27.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27.523+0000 7fa93da00640 1 -- 172.21.15.7:0/626851023 wait complete. 2024-09-10T21:42:27.540 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T21:42:27.540 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:42:27.540 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-10T21:42:27.540 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-10T21:42:27.540 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-10T21:42:27.540 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T21:42:27.540 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T21:42:27.540 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:42:27.540 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-10T21:42:27.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:27.541 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-10T21:42:27.576 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:42:27.576 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:42:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:42:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T21:42:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:27.577 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:42:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T21:42:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-10T21:42:27.577 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:42:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T21:42:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-10T21:42:27.577 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:42:27.577 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-10T21:42:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-10T21:42:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T21:42:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42597sec preferred_lft 42597sec 2024-09-10T21:42:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-10T21:42:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:27.578 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:42:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-10T21:42:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-10T21:42:27.578 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:42:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether f6:f9:69:5a:86:f5 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:42:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:42:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::3c3d:dbff:fe02:4c15/64 scope link 2024-09-10T21:42:27.578 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:27.579 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:42:27.579 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether f6:f9:69:5a:86:f5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:42:27.579 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f4f9:69ff:fe5a:86f5/64 scope link 2024-09-10T21:42:27.579 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:27.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:27.579 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:42:27.671 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:42:27.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:27.674 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-10T21:42:27.710 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-10T21:42:27.710 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:42:27.710 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T21:42:27.710 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T21:42:27.710 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:42:27.711 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:42:27.711 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:42:27.711 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:42:27.711 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:42:27.711 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:42:27.711 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:42:27.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:27.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:27.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:27.910 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:42:27.945 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:42:27.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:27.946 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list-id 2024-09-10T21:42:28.021 INFO:teuthology.orchestra.run.smithi007.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-10T21:42:28.022 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:42:28.022 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:42:28.022 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:42:28.022 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-10T21:42:28.022 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:42:28.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:28.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:28.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:28.199 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T21:42:28.255 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:42:28.256 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:42:28.256 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:42:28.256 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:28.256 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T21:42:28.256 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:28.256 INFO:teuthology.orchestra.run.smithi007.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:42:28.256 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 36:86:f4:86:1b:ec brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T21:42:28.256 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T21:42:28.256 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:28.256 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::3486:f4ff:fe86:1bec/64 scope link 2024-09-10T21:42:28.256 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:28.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:28.258 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-10T21:42:28.314 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:42:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:42:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:42:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T21:42:28.314 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:28.315 INFO:teuthology.orchestra.run.smithi007.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:42:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 36:86:f4:86:1b:ec brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T21:42:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T21:42:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::3486:f4ff:fe86:1bec/64 scope link 2024-09-10T21:42:28.315 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:28.317 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-10T21:42:28.317 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:42:28.317 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:42:28.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-10T21:42:28.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T21:42:28.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-10T21:42:28.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-10T21:42:28.318 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-10T21:42:28.318 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:42:28.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:28.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:28.535 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:42:28.535 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:42:28.535 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.1 up 2024-09-10T21:42:28.535 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T21:42:28.536 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:42:28.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:28.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:28.703 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:42:28.703 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-10T21:42:28.738 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-10T21:42:28.738 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:42:28.738 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-10T21:42:28.812 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2024-09-10T21:42:28.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:28.813 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:42:28.941 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:42:28.942 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:42:28.970 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:42:28.970 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:42:28.970 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.133:3300,172.21.15.133:3301" 2024-09-10T21:42:28.970 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 2@2029451a-3c26-48c7-b874-0597197278ac.cephfs=/ 2024-09-10T21:42:28.970 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.7:3300/172.21.15.133:3300/172.21.15.133:3301" will pass to kernel 2024-09-10T21:42:28.971 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-09-10T21:42:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:42:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:42:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:42:28.972 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:42:28.975 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:42:28.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:28.975 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-10T21:42:29.030 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:42:29.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:42:29.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:29.120 INFO:teuthology.orchestra.run.smithi007.stdout:client4657 2024-09-10T21:42:29.120 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:42:29.120 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:42:29.156 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:42:29.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:42:29.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:29.286 INFO:teuthology.orchestra.run.smithi007.stdout:client4657 2024-09-10T21:42:29.287 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:42:29.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:42:29.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:29.376 INFO:teuthology.orchestra.run.smithi007.stdout:client4657 2024-09-10T21:42:29.377 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/2029451a-3c26-48c7-b874-0597197278ac.client4657/status') 2024-09-10T21:42:29.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:29.476 INFO:teuthology.orchestra.run.smithi007.stdout:instance: client.4657 (3)172.21.15.7:0/2020182669 2024-09-10T21:42:29.476 INFO:teuthology.orchestra.run.smithi007.stdout:blocklisted: false 2024-09-10T21:42:29.476 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records in 2024-09-10T21:42:29.476 INFO:teuthology.orchestra.run.smithi007.stderr:0+1 records out 2024-09-10T21:42:29.476 INFO:teuthology.orchestra.run.smithi007.stderr:69 bytes copied, 5.7386e-05 s, 1.2 MB/s 2024-09-10T21:42:29.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:29.487 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2024-09-10T21:42:29.487 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T21:42:29.487 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T21:42:29.487 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:42:29.566 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:42:29.570 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:42:29.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.647+0000 7f88da600640 1 -- 172.21.15.7:0/3834475396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88d415e090 msgr2=0x7f88d40d1180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:29.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.647+0000 7f88da600640 1 --2- 172.21.15.7:0/3834475396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88d415e090 0x7f88d40d1180 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f88d40a1e30 tx=0x7f88c402a5f0 comp rx=0 tx=0).stop 2024-09-10T21:42:29.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.647+0000 7f88da600640 1 -- 172.21.15.7:0/3834475396 shutdown_connections 2024-09-10T21:42:29.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.647+0000 7f88da600640 1 --2- 172.21.15.7:0/3834475396 >> v2:172.21.15.133:3300/0 conn(0x7f88d40d1ae0 0x7f88d40d24d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:29.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.647+0000 7f88da600640 1 --2- 172.21.15.7:0/3834475396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88d415e090 0x7f88d40d1180 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:29.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.647+0000 7f88da600640 1 -- 172.21.15.7:0/3834475396 >> 172.21.15.7:0/3834475396 conn(0x7f88d40d3f10 msgr2=0x7f88d40d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:29.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.647+0000 7f88da600640 1 -- 172.21.15.7:0/3834475396 shutdown_connections 2024-09-10T21:42:29.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.647+0000 7f88da600640 1 -- 172.21.15.7:0/3834475396 wait complete. 2024-09-10T21:42:29.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.649+0000 7f88da600640 1 Processor -- start 2024-09-10T21:42:29.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.649+0000 7f88da600640 1 -- start start 2024-09-10T21:42:29.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.650+0000 7f88da600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88d40d1ae0 0x7f88d40cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:29.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.650+0000 7f88da600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88d40d2a10 0x7f88d41574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:29.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.650+0000 7f88da600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88d415e090 0x7f88d41e9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:29.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.650+0000 7f88da600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88d40cf1c0 con 0x7f88d40d1ae0 2024-09-10T21:42:29.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.650+0000 7f88da600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f88d40b5670 con 0x7f88d40d2a10 2024-09-10T21:42:29.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.650+0000 7f88da600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f88d40cf340 con 0x7f88d415e090 2024-09-10T21:42:29.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.650+0000 7f88d3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88d40d1ae0 0x7f88d40cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:29.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.650+0000 7f88d3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88d40d1ae0 0x7f88d40cdc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45366/0 (socket says 172.21.15.7:45366) 2024-09-10T21:42:29.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.650+0000 7f88d3400640 1 -- 172.21.15.7:0/2096467514 learned_addr learned my addr 172.21.15.7:0/2096467514 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:29.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.650+0000 7f88d2a00640 1 --2- 172.21.15.7:0/2096467514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88d40d2a10 0x7f88d41574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:29.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.650+0000 7f88d3e00640 1 --2- 172.21.15.7:0/2096467514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88d415e090 0x7f88d41e9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:29.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.650+0000 7f88d3400640 1 -- 172.21.15.7:0/2096467514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88d415e090 msgr2=0x7f88d41e9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:29.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.650+0000 7f88d3400640 1 --2- 172.21.15.7:0/2096467514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88d415e090 0x7f88d41e9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:29.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.650+0000 7f88d3400640 1 -- 172.21.15.7:0/2096467514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88d40d2a10 msgr2=0x7f88d41574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:29.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.650+0000 7f88d3400640 1 --2- 172.21.15.7:0/2096467514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88d40d2a10 0x7f88d41574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:29.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.650+0000 7f88d3400640 1 -- 172.21.15.7:0/2096467514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d40ccbc0 con 0x7f88d40d1ae0 2024-09-10T21:42:29.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.650+0000 7f88d2a00640 1 --2- 172.21.15.7:0/2096467514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88d40d2a10 0x7f88d41574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:29.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.651+0000 7f88d3400640 1 --2- 172.21.15.7:0/2096467514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88d40d1ae0 0x7f88d40cdc60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f88c4034040 tx=0x7f88c4003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:29.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.651+0000 7f88cb400640 1 -- 172.21.15.7:0/2096467514 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c403e030 con 0x7f88d40d1ae0 2024-09-10T21:42:29.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.651+0000 7f88cb400640 1 -- 172.21.15.7:0/2096467514 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88c4003d10 con 0x7f88d40d1ae0 2024-09-10T21:42:29.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.651+0000 7f88cb400640 1 -- 172.21.15.7:0/2096467514 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4035040 con 0x7f88d40d1ae0 2024-09-10T21:42:29.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.651+0000 7f88da600640 1 -- 172.21.15.7:0/2096467514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d40ccea0 con 0x7f88d40d1ae0 2024-09-10T21:42:29.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.651+0000 7f88da600640 1 -- 172.21.15.7:0/2096467514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d40cd460 con 0x7f88d40d1ae0 2024-09-10T21:42:29.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.652+0000 7f88da600640 1 -- 172.21.15.7:0/2096467514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d40d4a30 con 0x7f88d40d1ae0 2024-09-10T21:42:29.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.653+0000 7f88cb400640 1 -- 172.21.15.7:0/2096467514 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f88c40049c0 con 0x7f88d40d1ae0 2024-09-10T21:42:29.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.655+0000 7f88cb400640 1 --2- 172.21.15.7:0/2096467514 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f88b4077f70 0x7f88b407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:29.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.655+0000 7f88cb400640 1 -- 172.21.15.7:0/2096467514 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f88c40ba6e0 con 0x7f88d40d1ae0 2024-09-10T21:42:29.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.655+0000 7f88d2a00640 1 --2- 172.21.15.7:0/2096467514 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f88b4077f70 0x7f88b407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:29.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.656+0000 7f88d2a00640 1 --2- 172.21.15.7:0/2096467514 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f88b4077f70 0x7f88b407a430 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009c20 tx=0x7f88c0005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:29.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.659+0000 7f88cb400640 1 -- 172.21.15.7:0/2096467514 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f88d40d4a30 con 0x7f88d40d1ae0 2024-09-10T21:42:29.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.898+0000 7f88da600640 1 -- 172.21.15.7:0/2096467514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f88d40cf860 con 0x7f88d40d1ae0 2024-09-10T21:42:29.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.898+0000 7f88cb400640 1 -- 172.21.15.7:0/2096467514 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f88c4083140 con 0x7f88d40d1ae0 2024-09-10T21:42:29.899 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:42:29.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.901+0000 7f88da600640 1 -- 172.21.15.7:0/2096467514 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f88b4077f70 msgr2=0x7f88b407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:29.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.901+0000 7f88da600640 1 --2- 172.21.15.7:0/2096467514 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f88b4077f70 0x7f88b407a430 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f88c0009c20 tx=0x7f88c0005f90 comp rx=0 tx=0).stop 2024-09-10T21:42:29.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.901+0000 7f88da600640 1 -- 172.21.15.7:0/2096467514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88d40d1ae0 msgr2=0x7f88d40cdc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:29.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.901+0000 7f88da600640 1 --2- 172.21.15.7:0/2096467514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88d40d1ae0 0x7f88d40cdc60 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f88c4034040 tx=0x7f88c4003820 comp rx=0 tx=0).stop 2024-09-10T21:42:29.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.901+0000 7f88da600640 1 -- 172.21.15.7:0/2096467514 shutdown_connections 2024-09-10T21:42:29.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.901+0000 7f88da600640 1 --2- 172.21.15.7:0/2096467514 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f88b4077f70 0x7f88b407a430 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:29.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.901+0000 7f88da600640 1 --2- 172.21.15.7:0/2096467514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88d40d2a10 0x7f88d41574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:29.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.901+0000 7f88da600640 1 --2- 172.21.15.7:0/2096467514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88d415e090 0x7f88d41e9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:29.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.901+0000 7f88da600640 1 --2- 172.21.15.7:0/2096467514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88d40d1ae0 0x7f88d40cdc60 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:29.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.901+0000 7f88da600640 1 -- 172.21.15.7:0/2096467514 >> 172.21.15.7:0/2096467514 conn(0x7f88d40d3f10 msgr2=0x7f88d40b26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:29.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.901+0000 7f88da600640 1 -- 172.21.15.7:0/2096467514 shutdown_connections 2024-09-10T21:42:29.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:29.901+0000 7f88da600640 1 -- 172.21.15.7:0/2096467514 wait complete. 2024-09-10T21:42:29.917 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T21:42:29.917 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:42:29.917 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-09-10T21:42:29.918 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T21:42:29.918 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T21:42:29.918 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T21:42:29.918 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T21:42:29.918 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:42:29.918 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:42:29.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:29.918 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-09-10T21:42:29.957 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:42:29.958 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:42:29.958 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:42:29.958 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:29.958 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-09-10T21:42:29.958 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:29.958 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:42:29.958 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:42:29.958 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-09-10T21:42:29.958 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:42:29.958 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:42:29.958 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-09-10T21:42:29.958 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:42:29.959 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:42:29.959 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:42:29.959 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:42:29.959 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T21:42:29.959 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 42571sec preferred_lft 42571sec 2024-09-10T21:42:29.959 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link noprefixroute 2024-09-10T21:42:29.959 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:29.959 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T21:42:29.959 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:42:29.959 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-09-10T21:42:29.960 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add name ceph-brx type bridge 2024-09-10T21:42:29.960 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr flush dev ceph-brx 2024-09-10T21:42:29.960 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set ceph-brx up 2024-09-10T21:42:29.960 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T21:42:29.960 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-09-10T21:42:30.079 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:30.377 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:30.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:30.386 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:42:30.441 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-09-10T21:42:30.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:30.445 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-09-10T21:42:30.490 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-09-10T21:42:30.490 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:42:30.490 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:42:30.490 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:42:30.490 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:42:30.491 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:42:30.491 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-09-10T21:42:30.491 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:42:30.491 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:42:30.491 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:42:30.491 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-09-10T21:42:30.611 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:31.122 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:31.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:31.130 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-09-10T21:42:31.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:31.166 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list-id 2024-09-10T21:42:31.243 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:42:31.243 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-09-10T21:42:31.243 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:42:31.243 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-10T21:42:31.243 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-09-10T21:42:31.360 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:31.412 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:31.420 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-10T21:42:31.421 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:42:31.421 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-09-10T21:42:31.421 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-10T21:42:31.421 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T21:42:31.421 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-10T21:42:31.421 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-10T21:42:31.421 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-10T21:42:31.421 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-09-10T21:42:31.498 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:31.668 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:31.676 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:42:31.676 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-09-10T21:42:31.676 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.0 up 2024-09-10T21:42:31.676 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T21:42:31.676 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-09-10T21:42:31.754 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:31.837 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:31.845 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:42:31.845 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T21:42:31.881 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T21:42:31.881 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:42:31.882 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:42:31.957 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2024-09-10T21:42:31.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:31.957 DEBUG:teuthology.orchestra.run.smithi133:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:42:32.110 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:42:32.110 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:42:32.773 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:42:32.773 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:42:32.773 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.133:3300,172.21.15.133:3301" 2024-09-10T21:42:32.773 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: trying mount with new device syntax: 1@2029451a-3c26-48c7-b874-0597197278ac.cephfs=/ 2024-09-10T21:42:32.773 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.7:3300/172.21.15.133:3300/172.21.15.133:3301" will pass to kernel 2024-09-10T21:42:32.774 INFO:teuthology.orchestra.run.smithi133.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-10T21:42:32.775 INFO:teuthology.orchestra.run.smithi133.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:42:32.775 INFO:teuthology.orchestra.run.smithi133.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:42:32.775 INFO:teuthology.orchestra.run.smithi133.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:42:32.775 INFO:teuthology.orchestra.run.smithi133.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:42:32.778 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:42:32.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:32.778 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T21:42:32.835 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:42:32.914 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:42:32.918 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:32.925 INFO:teuthology.orchestra.run.smithi133.stdout:client4710 2024-09-10T21:42:32.926 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:42:32.926 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:42:32.961 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:42:33.078 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:42:33.082 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:33.090 INFO:teuthology.orchestra.run.smithi133.stdout:client4710 2024-09-10T21:42:33.090 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:42:33.168 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:42:33.172 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:33.179 INFO:teuthology.orchestra.run.smithi133.stdout:client4710 2024-09-10T21:42:33.180 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/2029451a-3c26-48c7-b874-0597197278ac.client4710/status') 2024-09-10T21:42:33.257 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:33.278 INFO:teuthology.orchestra.run.smithi133.stdout:instance: client.4710 (3)172.21.15.133:0/3000452254 2024-09-10T21:42:33.278 INFO:teuthology.orchestra.run.smithi133.stdout:blocklisted: false 2024-09-10T21:42:33.279 INFO:teuthology.orchestra.run.smithi133.stderr:0+1 records in 2024-09-10T21:42:33.279 INFO:teuthology.orchestra.run.smithi133.stderr:0+1 records out 2024-09-10T21:42:33.279 INFO:teuthology.orchestra.run.smithi133.stderr:71 bytes copied, 5.1304e-05 s, 1.4 MB/s 2024-09-10T21:42:33.280 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:33.289 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2024-09-10T21:42:33.289 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T21:42:33.289 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T21:42:33.289 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:42:33.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.459+0000 7f7f80e00640 1 -- 172.21.15.7:0/2440525525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c15bc90 msgr2=0x7f7f7c0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:33.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.459+0000 7f7f80e00640 1 --2- 172.21.15.7:0/2440525525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c15bc90 0x7f7f7c0ccef0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c005fd0 tx=0x7f7f6c02a4b0 comp rx=0 tx=0).stop 2024-09-10T21:42:33.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.459+0000 7f7f80e00640 1 -- 172.21.15.7:0/2440525525 shutdown_connections 2024-09-10T21:42:33.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.459+0000 7f7f80e00640 1 --2- 172.21.15.7:0/2440525525 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f7c0cd750 0x7f7f7c073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:33.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.459+0000 7f7f80e00640 1 --2- 172.21.15.7:0/2440525525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c15bc90 0x7f7f7c0ccef0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:33.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.459+0000 7f7f80e00640 1 -- 172.21.15.7:0/2440525525 >> 172.21.15.7:0/2440525525 conn(0x7f7f7c0b16d0 msgr2=0x7f7f7c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:33.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.459+0000 7f7f80e00640 1 -- 172.21.15.7:0/2440525525 shutdown_connections 2024-09-10T21:42:33.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.459+0000 7f7f80e00640 1 -- 172.21.15.7:0/2440525525 wait complete. 2024-09-10T21:42:33.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.461+0000 7f7f80e00640 1 Processor -- start 2024-09-10T21:42:33.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.461+0000 7f7f80e00640 1 -- start start 2024-09-10T21:42:33.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.462+0000 7f7f80e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f7c073c70 0x7f7f7c1f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:33.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.462+0000 7f7f80e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c0cd750 0x7f7f7c15e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:33.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.462+0000 7f7f80e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c15bc90 0x7f7f7c0b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:33.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.462+0000 7f7f80e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c06fcf0 con 0x7f7f7c0cd750 2024-09-10T21:42:33.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.462+0000 7f7f80e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c1606c0 con 0x7f7f7c15bc90 2024-09-10T21:42:33.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.462+0000 7f7f80e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f7f7c06fe70 con 0x7f7f7c073c70 2024-09-10T21:42:33.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.462+0000 7f7f71400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c0cd750 0x7f7f7c15e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:33.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.463+0000 7f7f79e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f7c073c70 0x7f7f7c1f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:33.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.463+0000 7f7f71400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c0cd750 0x7f7f7c15e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45374/0 (socket says 172.21.15.7:45374) 2024-09-10T21:42:33.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.463+0000 7f7f71400640 1 -- 172.21.15.7:0/2381979600 learned_addr learned my addr 172.21.15.7:0/2381979600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:33.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.463+0000 7f7f7a800640 1 --2- 172.21.15.7:0/2381979600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c15bc90 0x7f7f7c0b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:33.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.463+0000 7f7f71400640 1 -- 172.21.15.7:0/2381979600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f7c073c70 msgr2=0x7f7f7c1f2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:33.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.463+0000 7f7f71400640 1 --2- 172.21.15.7:0/2381979600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f7c073c70 0x7f7f7c1f2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:33.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.463+0000 7f7f71400640 1 -- 172.21.15.7:0/2381979600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c15bc90 msgr2=0x7f7f7c0b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:33.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.463+0000 7f7f71400640 1 --2- 172.21.15.7:0/2381979600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c15bc90 0x7f7f7c0b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:33.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.463+0000 7f7f71400640 1 -- 172.21.15.7:0/2381979600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7c1f58e0 con 0x7f7f7c0cd750 2024-09-10T21:42:33.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.463+0000 7f7f71400640 1 --2- 172.21.15.7:0/2381979600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c0cd750 0x7f7f7c15e6a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400d950 tx=0x7f7f6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:33.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.463+0000 7f7f72a00640 1 -- 172.21.15.7:0/2381979600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f64017030 con 0x7f7f7c0cd750 2024-09-10T21:42:33.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.464+0000 7f7f72a00640 1 -- 172.21.15.7:0/2381979600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f64004110 con 0x7f7f7c0cd750 2024-09-10T21:42:33.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.464+0000 7f7f72a00640 1 -- 172.21.15.7:0/2381979600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f640099d0 con 0x7f7f7c0cd750 2024-09-10T21:42:33.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.464+0000 7f7f80e00640 1 -- 172.21.15.7:0/2381979600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f7c1f5bc0 con 0x7f7f7c0cd750 2024-09-10T21:42:33.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.464+0000 7f7f80e00640 1 -- 172.21.15.7:0/2381979600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f7c1f6180 con 0x7f7f7c0cd750 2024-09-10T21:42:33.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.465+0000 7f7f80e00640 1 -- 172.21.15.7:0/2381979600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f7c073730 con 0x7f7f7c0cd750 2024-09-10T21:42:33.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.466+0000 7f7f72a00640 1 -- 172.21.15.7:0/2381979600 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7f64009b70 con 0x7f7f7c0cd750 2024-09-10T21:42:33.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.467+0000 7f7f72a00640 1 --2- 172.21.15.7:0/2381979600 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f58077f70 0x7f7f5807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:33.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.468+0000 7f7f72a00640 1 -- 172.21.15.7:0/2381979600 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f7f64099d90 con 0x7f7f7c0cd750 2024-09-10T21:42:33.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.468+0000 7f7f79e00640 1 --2- 172.21.15.7:0/2381979600 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f58077f70 0x7f7f5807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:33.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.468+0000 7f7f79e00640 1 --2- 172.21.15.7:0/2381979600 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f58077f70 0x7f7f5807a430 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c033000 tx=0x7f7f6c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:33.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.472+0000 7f7f72a00640 1 -- 172.21.15.7:0/2381979600 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7f64062230 con 0x7f7f7c0cd750 2024-09-10T21:42:33.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.706+0000 7f7f80e00640 1 -- 172.21.15.7:0/2381979600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7f7c069810 con 0x7f7f7c0cd750 2024-09-10T21:42:33.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.706+0000 7f7f72a00640 1 -- 172.21.15.7:0/2381979600 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f7f64066c60 con 0x7f7f7c0cd750 2024-09-10T21:42:33.708 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:42:33.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.709+0000 7f7f80e00640 1 -- 172.21.15.7:0/2381979600 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f58077f70 msgr2=0x7f7f5807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:33.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.709+0000 7f7f80e00640 1 --2- 172.21.15.7:0/2381979600 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f58077f70 0x7f7f5807a430 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c033000 tx=0x7f7f6c004730 comp rx=0 tx=0).stop 2024-09-10T21:42:33.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.709+0000 7f7f80e00640 1 -- 172.21.15.7:0/2381979600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c0cd750 msgr2=0x7f7f7c15e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:33.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.709+0000 7f7f80e00640 1 --2- 172.21.15.7:0/2381979600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c0cd750 0x7f7f7c15e6a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7f6400d950 tx=0x7f7f6400de10 comp rx=0 tx=0).stop 2024-09-10T21:42:33.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.709+0000 7f7f80e00640 1 -- 172.21.15.7:0/2381979600 shutdown_connections 2024-09-10T21:42:33.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.709+0000 7f7f80e00640 1 --2- 172.21.15.7:0/2381979600 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f58077f70 0x7f7f5807a430 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:33.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.709+0000 7f7f80e00640 1 --2- 172.21.15.7:0/2381979600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c15bc90 0x7f7f7c0b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:33.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.709+0000 7f7f80e00640 1 --2- 172.21.15.7:0/2381979600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c0cd750 0x7f7f7c15e6a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:33.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.709+0000 7f7f80e00640 1 --2- 172.21.15.7:0/2381979600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f7c073c70 0x7f7f7c1f2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:33.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.709+0000 7f7f80e00640 1 -- 172.21.15.7:0/2381979600 >> 172.21.15.7:0/2381979600 conn(0x7f7f7c0b16d0 msgr2=0x7f7f7c0b4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:33.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.709+0000 7f7f80e00640 1 -- 172.21.15.7:0/2381979600 shutdown_connections 2024-09-10T21:42:33.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:33.709+0000 7f7f80e00640 1 -- 172.21.15.7:0/2381979600 wait complete. 2024-09-10T21:42:33.726 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T21:42:33.726 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:42:33.726 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi133.front.sepia.ceph.com 2024-09-10T21:42:33.726 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-10T21:42:33.726 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-10T21:42:33.726 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T21:42:33.726 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T21:42:33.726 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:42:33.726 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-10T21:42:33.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:33.727 DEBUG:teuthology.orchestra.run.smithi133:> ip addr 2024-09-10T21:42:33.763 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:42:33.763 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:42:33.763 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:42:33.763 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:33.763 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-09-10T21:42:33.763 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:33.763 INFO:teuthology.orchestra.run.smithi133.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:42:33.763 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b2 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:42:33.764 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f0 2024-09-10T21:42:33.764 INFO:teuthology.orchestra.run.smithi133.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:42:33.764 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:d9:38:b3 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:42:33.764 INFO:teuthology.orchestra.run.smithi133.stdout: altname enp7s0f1 2024-09-10T21:42:33.764 INFO:teuthology.orchestra.run.smithi133.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:42:33.764 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:86 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:42:33.764 INFO:teuthology.orchestra.run.smithi133.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:42:33.764 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 0c:c4:7a:88:80:87 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:42:33.764 INFO:teuthology.orchestra.run.smithi133.stdout: inet 172.21.15.133/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T21:42:33.764 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft 42567sec preferred_lft 42567sec 2024-09-10T21:42:33.765 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::ec4:7aff:fe88:8087/64 scope link noprefixroute 2024-09-10T21:42:33.765 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:33.765 INFO:teuthology.orchestra.run.smithi133.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:42:33.765 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 76:f1:15:91:4c:97 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:42:33.765 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:42:33.765 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:33.765 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::b461:6aff:feeb:707b/64 scope link 2024-09-10T21:42:33.765 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:33.765 INFO:teuthology.orchestra.run.smithi133.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:42:33.765 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 76:f1:15:91:4c:97 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:42:33.765 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::74f1:15ff:fe91:4c97/64 scope link 2024-09-10T21:42:33.765 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:33.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:33.766 DEBUG:teuthology.orchestra.run.smithi133:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:42:33.861 INFO:teuthology.orchestra.run.smithi133.stdout:1 2024-09-10T21:42:33.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:33.864 DEBUG:teuthology.orchestra.run.smithi133:> route 2024-09-10T21:42:33.900 INFO:teuthology.orchestra.run.smithi133.stdout:Kernel IP routing table 2024-09-10T21:42:33.900 INFO:teuthology.orchestra.run.smithi133.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:42:33.900 INFO:teuthology.orchestra.run.smithi133.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T21:42:33.900 INFO:teuthology.orchestra.run.smithi133.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T21:42:33.900 INFO:teuthology.orchestra.run.smithi133.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:42:33.901 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:42:33.901 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-09-10T21:42:33.901 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T21:42:33.901 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T21:42:33.901 DEBUG:teuthology.orchestra.run.smithi133:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T21:42:33.901 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-09-10T21:42:34.018 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:34.088 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:34.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:34.097 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-09-10T21:42:34.132 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T21:42:34.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:34.132 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list-id 2024-09-10T21:42:34.206 INFO:teuthology.orchestra.run.smithi133.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-10T21:42:34.207 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:42:34.207 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-09-10T21:42:34.207 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T21:42:34.207 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-10T21:42:34.207 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-09-10T21:42:34.323 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:34.375 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:34.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:34.383 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T21:42:34.438 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:42:34.438 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:42:34.438 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:42:34.438 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:34.438 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-09-10T21:42:34.438 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:34.438 INFO:teuthology.orchestra.run.smithi133.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:42:34.438 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 6a:4b:40:42:e8:3f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T21:42:34.438 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T21:42:34.439 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:34.439 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::684b:40ff:fe42:e83f/64 scope link 2024-09-10T21:42:34.439 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:34.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:34.441 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-10T21:42:34.496 INFO:teuthology.orchestra.run.smithi133.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:42:34.496 INFO:teuthology.orchestra.run.smithi133.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:42:34.496 INFO:teuthology.orchestra.run.smithi133.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:42:34.496 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:34.496 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 ::1/128 scope host 2024-09-10T21:42:34.496 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:34.497 INFO:teuthology.orchestra.run.smithi133.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:42:34.497 INFO:teuthology.orchestra.run.smithi133.stdout: link/ether 6a:4b:40:42:e8:3f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-10T21:42:34.497 INFO:teuthology.orchestra.run.smithi133.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-10T21:42:34.497 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:34.497 INFO:teuthology.orchestra.run.smithi133.stdout: inet6 fe80::684b:40ff:fe42:e83f/64 scope link 2024-09-10T21:42:34.497 INFO:teuthology.orchestra.run.smithi133.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:42:34.499 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-10T21:42:34.500 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:42:34.500 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-09-10T21:42:34.500 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-10T21:42:34.500 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-10T21:42:34.500 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-10T21:42:34.500 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-10T21:42:34.500 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-10T21:42:34.500 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-09-10T21:42:34.578 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:34.706 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:34.715 DEBUG:teuthology.orchestra.run.smithi133:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:42:34.715 DEBUG:teuthology.orchestra.run.smithi133:> set -e 2024-09-10T21:42:34.715 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set brx.1 up 2024-09-10T21:42:34.715 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link set dev brx.1 master ceph-brx 2024-09-10T21:42:34.715 DEBUG:teuthology.orchestra.run.smithi133:> ') 2024-09-10T21:42:34.793 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:34.873 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:34.881 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:42:34.881 DEBUG:teuthology.orchestra.run.smithi133:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-10T21:42:34.916 INFO:teuthology.orchestra.run.smithi133.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-10T21:42:34.916 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:42:34.917 DEBUG:teuthology.orchestra.run.smithi133:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-10T21:42:34.992 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2024-09-10T21:42:34.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:34.992 DEBUG:teuthology.orchestra.run.smithi133:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:42:35.118 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:42:35.119 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:42:35.146 INFO:teuthology.orchestra.run.smithi133.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:42:35.146 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:42:35.146 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.133:3300,172.21.15.133:3301" 2024-09-10T21:42:35.146 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: trying mount with new device syntax: 3@2029451a-3c26-48c7-b874-0597197278ac.cephfs=/ 2024-09-10T21:42:35.147 INFO:teuthology.orchestra.run.smithi133.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.7:3300/172.21.15.133:3300/172.21.15.133:3301" will pass to kernel 2024-09-10T21:42:35.148 INFO:teuthology.orchestra.run.smithi133.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-09-10T21:42:35.148 INFO:teuthology.orchestra.run.smithi133.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:42:35.148 INFO:teuthology.orchestra.run.smithi133.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:42:35.148 INFO:teuthology.orchestra.run.smithi133.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:42:35.148 INFO:teuthology.orchestra.run.smithi133.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:42:35.151 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:42:35.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:35.151 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-10T21:42:35.207 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:42:35.285 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:42:35.288 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:35.296 INFO:teuthology.orchestra.run.smithi133.stdout:client4672 2024-09-10T21:42:35.297 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:42:35.297 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:42:35.332 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:42:35.449 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:42:35.452 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:35.459 INFO:teuthology.orchestra.run.smithi133.stdout:client4672 2024-09-10T21:42:35.460 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:42:35.537 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:35 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:42:35.539 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:35.547 INFO:teuthology.orchestra.run.smithi133.stdout:client4672 2024-09-10T21:42:35.547 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/2029451a-3c26-48c7-b874-0597197278ac.client4672/status') 2024-09-10T21:42:35.625 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:35 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:42:35.646 INFO:teuthology.orchestra.run.smithi133.stdout:instance: client.4672 (3)192.168.0.2:0/1273324947 2024-09-10T21:42:35.646 INFO:teuthology.orchestra.run.smithi133.stdout:blocklisted: false 2024-09-10T21:42:35.647 INFO:teuthology.orchestra.run.smithi133.stderr:0+1 records in 2024-09-10T21:42:35.647 INFO:teuthology.orchestra.run.smithi133.stderr:0+1 records out 2024-09-10T21:42:35.647 INFO:teuthology.orchestra.run.smithi133.stderr:69 bytes copied, 5.5444e-05 s, 1.2 MB/s 2024-09-10T21:42:35.648 INFO:teuthology.orchestra.run.smithi133.stderr:2024-09-10T21:42:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:42:35.657 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-10T21:42:35.697 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-10T21:42:35.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:42:35.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.868+0000 7fc416400640 1 -- 172.21.15.7:0/2353749393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc41015aa30 msgr2=0x7fc410073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:35.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.868+0000 7fc416400640 1 --2- 172.21.15.7:0/2353749393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc41015aa30 0x7fc410073550 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc4100c3320 tx=0x7fc40002c770 comp rx=0 tx=0).stop 2024-09-10T21:42:35.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.869+0000 7fc416400640 1 -- 172.21.15.7:0/2353749393 shutdown_connections 2024-09-10T21:42:35.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.869+0000 7fc416400640 1 --2- 172.21.15.7:0/2353749393 >> v2:172.21.15.133:3300/0 conn(0x7fc4100ce860 0x7fc4100ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:35.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.869+0000 7fc416400640 1 --2- 172.21.15.7:0/2353749393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc41015aa30 0x7fc410073550 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:35.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.869+0000 7fc416400640 1 -- 172.21.15.7:0/2353749393 >> 172.21.15.7:0/2353749393 conn(0x7fc4100b16d0 msgr2=0x7fc4100b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:35.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.869+0000 7fc416400640 1 -- 172.21.15.7:0/2353749393 shutdown_connections 2024-09-10T21:42:35.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.869+0000 7fc416400640 1 -- 172.21.15.7:0/2353749393 wait complete. 2024-09-10T21:42:35.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.870+0000 7fc416400640 1 Processor -- start 2024-09-10T21:42:35.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.870+0000 7fc416400640 1 -- start start 2024-09-10T21:42:35.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.871+0000 7fc416400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4100ce860 0x7fc410156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:35.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.871+0000 7fc416400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc41015aa30 0x7fc41015b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:35.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.871+0000 7fc416400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc41015e650 0x7fc4101598b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.871+0000 7fc416400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc41006ee30 con 0x7fc4100ce860 2024-09-10T21:42:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.871+0000 7fc416400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc41006ecb0 con 0x7fc41015e650 2024-09-10T21:42:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.871+0000 7fc416400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc4100cbcd0 con 0x7fc41015aa30 2024-09-10T21:42:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.872+0000 7fc40f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4100ce860 0x7fc410156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.872+0000 7fc40f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4100ce860 0x7fc410156030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44152/0 (socket says 172.21.15.7:44152) 2024-09-10T21:42:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.872+0000 7fc40f400640 1 -- 172.21.15.7:0/2880866119 learned_addr learned my addr 172.21.15.7:0/2880866119 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.872+0000 7fc40ea00640 1 --2- 172.21.15.7:0/2880866119 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc41015aa30 0x7fc41015b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.872+0000 7fc40fe00640 1 --2- 172.21.15.7:0/2880866119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc41015e650 0x7fc4101598b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.872+0000 7fc40f400640 1 -- 172.21.15.7:0/2880866119 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc41015aa30 msgr2=0x7fc41015b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.872+0000 7fc40f400640 1 --2- 172.21.15.7:0/2880866119 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc41015aa30 0x7fc41015b100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.872+0000 7fc40f400640 1 -- 172.21.15.7:0/2880866119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc41015e650 msgr2=0x7fc4101598b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:35.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.872+0000 7fc40f400640 1 --2- 172.21.15.7:0/2880866119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc41015e650 0x7fc4101598b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:35.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.872+0000 7fc40f400640 1 -- 172.21.15.7:0/2880866119 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41015e2b0 con 0x7fc4100ce860 2024-09-10T21:42:35.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.872+0000 7fc40f400640 1 --2- 172.21.15.7:0/2880866119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4100ce860 0x7fc410156030 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc400000c00 tx=0x7fc400034ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:35.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.873+0000 7fc407400640 1 -- 172.21.15.7:0/2880866119 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400043020 con 0x7fc4100ce860 2024-09-10T21:42:35.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.873+0000 7fc407400640 1 -- 172.21.15.7:0/2880866119 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc400003700 con 0x7fc4100ce860 2024-09-10T21:42:35.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.873+0000 7fc407400640 1 -- 172.21.15.7:0/2880866119 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc400038040 con 0x7fc4100ce860 2024-09-10T21:42:35.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.873+0000 7fc416400640 1 -- 172.21.15.7:0/2880866119 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc410155550 con 0x7fc4100ce860 2024-09-10T21:42:35.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.873+0000 7fc416400640 1 -- 172.21.15.7:0/2880866119 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc410155b10 con 0x7fc4100ce860 2024-09-10T21:42:35.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.874+0000 7fc416400640 1 -- 172.21.15.7:0/2880866119 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc41015b700 con 0x7fc4100ce860 2024-09-10T21:42:35.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.875+0000 7fc407400640 1 -- 172.21.15.7:0/2880866119 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc400003b50 con 0x7fc4100ce860 2024-09-10T21:42:35.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.876+0000 7fc407400640 1 --2- 172.21.15.7:0/2880866119 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc3f0077f70 0x7fc3f007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:35.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.876+0000 7fc407400640 1 -- 172.21.15.7:0/2880866119 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fc4000bcd70 con 0x7fc4100ce860 2024-09-10T21:42:35.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.876+0000 7fc40ea00640 1 --2- 172.21.15.7:0/2880866119 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc3f0077f70 0x7fc3f007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:35.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.877+0000 7fc40ea00640 1 --2- 172.21.15.7:0/2880866119 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc3f0077f70 0x7fc3f007a430 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc4100ca130 tx=0x7fc3fc00b410 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:35.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:35.880+0000 7fc407400640 1 -- 172.21.15.7:0/2880866119 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc40004d030 con 0x7fc4100ce860 2024-09-10T21:42:36.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.074+0000 7fc416400640 1 -- 172.21.15.7:0/2880866119 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4100ceca0 con 0x7fc4100ce860 2024-09-10T21:42:36.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.075+0000 7fc407400640 1 -- 172.21.15.7:0/2880866119 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16629 (secure 0 0 0) 0x7fc400085210 con 0x7fc4100ce860 2024-09-10T21:42:36.077 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:36.077 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:42:10.684978+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:42:07.149267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:42:08.077083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:42:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.077+0000 7fc416400640 1 -- 172.21.15.7:0/2880866119 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc3f0077f70 msgr2=0x7fc3f007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.077+0000 7fc416400640 1 --2- 172.21.15.7:0/2880866119 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc3f0077f70 0x7fc3f007a430 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc4100ca130 tx=0x7fc3fc00b410 comp rx=0 tx=0).stop 2024-09-10T21:42:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.077+0000 7fc416400640 1 -- 172.21.15.7:0/2880866119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4100ce860 msgr2=0x7fc410156030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.077+0000 7fc416400640 1 --2- 172.21.15.7:0/2880866119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4100ce860 0x7fc410156030 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc400000c00 tx=0x7fc400034ed0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.078+0000 7fc416400640 1 -- 172.21.15.7:0/2880866119 shutdown_connections 2024-09-10T21:42:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.078+0000 7fc416400640 1 --2- 172.21.15.7:0/2880866119 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc3f0077f70 0x7fc3f007a430 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.078+0000 7fc416400640 1 --2- 172.21.15.7:0/2880866119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc41015e650 0x7fc4101598b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.078+0000 7fc416400640 1 --2- 172.21.15.7:0/2880866119 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc41015aa30 0x7fc41015b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.078+0000 7fc416400640 1 --2- 172.21.15.7:0/2880866119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4100ce860 0x7fc410156030 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.078+0000 7fc416400640 1 -- 172.21.15.7:0/2880866119 >> 172.21.15.7:0/2880866119 conn(0x7fc4100b16d0 msgr2=0x7fc4100cdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:36.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.078+0000 7fc416400640 1 -- 172.21.15.7:0/2880866119 shutdown_connections 2024-09-10T21:42:36.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.078+0000 7fc416400640 1 -- 172.21.15.7:0/2880866119 wait complete. 2024-09-10T21:42:36.095 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:41:55.889618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T21:42:07.149267+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T21:42:08.077083+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:42:36.095 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:42:36.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.258+0000 7f448fc00640 1 -- 172.21.15.7:0/899175632 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4488074450 msgr2=0x7f44880ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:36.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.258+0000 7f448fc00640 1 --2- 172.21.15.7:0/899175632 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4488074450 0x7f44880ca270 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4480005fd0 tx=0x7f448002a8b0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.258+0000 7f448fc00640 1 -- 172.21.15.7:0/899175632 shutdown_connections 2024-09-10T21:42:36.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.258+0000 7f448fc00640 1 --2- 172.21.15.7:0/899175632 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4488074450 0x7f44880ca270 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.258+0000 7f448fc00640 1 --2- 172.21.15.7:0/899175632 >> v2:172.21.15.133:3300/0 conn(0x7f448806fcf0 0x7f4488160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.258+0000 7f448fc00640 1 -- 172.21.15.7:0/899175632 >> 172.21.15.7:0/899175632 conn(0x7f44880b16d0 msgr2=0x7f44880b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:36.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.259+0000 7f448fc00640 1 -- 172.21.15.7:0/899175632 shutdown_connections 2024-09-10T21:42:36.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.259+0000 7f448fc00640 1 -- 172.21.15.7:0/899175632 wait complete. 2024-09-10T21:42:36.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.260+0000 7f448fc00640 1 Processor -- start 2024-09-10T21:42:36.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.260+0000 7f448fc00640 1 -- start start 2024-09-10T21:42:36.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.261+0000 7f448fc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f448806fcf0 0x7f44881f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:36.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.261+0000 7f448fc00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4488074450 0x7f44881f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:36.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.261+0000 7f448fc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4488156090 0x7f44880c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:36.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.261+0000 7f448fc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f44880cab40 con 0x7f448806fcf0 2024-09-10T21:42:36.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.261+0000 7f448fc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f44880ca9c0 con 0x7f4488156090 2024-09-10T21:42:36.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.261+0000 7f448fc00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f44880cacc0 con 0x7f4488074450 2024-09-10T21:42:36.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.261+0000 7f448d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f448806fcf0 0x7f44881f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:36.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.262+0000 7f448d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f448806fcf0 0x7f44881f5050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44158/0 (socket says 172.21.15.7:44158) 2024-09-10T21:42:36.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.262+0000 7f448d200640 1 -- 172.21.15.7:0/701872247 learned_addr learned my addr 172.21.15.7:0/701872247 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:36.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.262+0000 7f447fe00640 1 --2- 172.21.15.7:0/701872247 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4488074450 0x7f44881f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:36.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.262+0000 7f448dc00640 1 --2- 172.21.15.7:0/701872247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4488156090 0x7f44880c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:36.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.262+0000 7f448d200640 1 -- 172.21.15.7:0/701872247 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4488074450 msgr2=0x7f44881f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.262+0000 7f448d200640 1 --2- 172.21.15.7:0/701872247 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4488074450 0x7f44881f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.262+0000 7f448d200640 1 -- 172.21.15.7:0/701872247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4488156090 msgr2=0x7f44880c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.262+0000 7f448d200640 1 --2- 172.21.15.7:0/701872247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4488156090 0x7f44880c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.262+0000 7f448d200640 1 -- 172.21.15.7:0/701872247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44881f6010 con 0x7f448806fcf0 2024-09-10T21:42:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.262+0000 7f448d200640 1 --2- 172.21.15.7:0/701872247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f448806fcf0 0x7f44881f5050 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f447800c3e0 tx=0x7f447800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.262+0000 7f4485600640 1 -- 172.21.15.7:0/701872247 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4478011790 con 0x7f448806fcf0 2024-09-10T21:42:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.263+0000 7f4485600640 1 -- 172.21.15.7:0/701872247 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4478017070 con 0x7f448806fcf0 2024-09-10T21:42:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.263+0000 7f448fc00640 1 -- 172.21.15.7:0/701872247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44881f62f0 con 0x7f448806fcf0 2024-09-10T21:42:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.263+0000 7f4485600640 1 -- 172.21.15.7:0/701872247 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4478011ca0 con 0x7f448806fcf0 2024-09-10T21:42:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.263+0000 7f448fc00640 1 -- 172.21.15.7:0/701872247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44880b3cb0 con 0x7f448806fcf0 2024-09-10T21:42:36.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.264+0000 7f448fc00640 1 -- 172.21.15.7:0/701872247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44880cb170 con 0x7f448806fcf0 2024-09-10T21:42:36.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.265+0000 7f4485600640 1 -- 172.21.15.7:0/701872247 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f447801c460 con 0x7f448806fcf0 2024-09-10T21:42:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.266+0000 7f4485600640 1 --2- 172.21.15.7:0/701872247 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4464077f70 0x7f446407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.266+0000 7f4485600640 1 -- 172.21.15.7:0/701872247 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f44780998f0 con 0x7f448806fcf0 2024-09-10T21:42:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.266+0000 7f447fe00640 1 --2- 172.21.15.7:0/701872247 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4464077f70 0x7f446407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:36.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.267+0000 7f447fe00640 1 --2- 172.21.15.7:0/701872247 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4464077f70 0x7f446407a430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f448814ef20 tx=0x7f4470005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:36.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.270+0000 7f4485600640 1 -- 172.21.15.7:0/701872247 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4478061ef0 con 0x7f448806fcf0 2024-09-10T21:42:36.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.464+0000 7f448fc00640 1 -- 172.21.15.7:0/701872247 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f448815ec70 con 0x7f448806fcf0 2024-09-10T21:42:36.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.465+0000 7f4485600640 1 -- 172.21.15.7:0/701872247 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) ==== 93+0+10 (secure 0 0 0) 0x7f4478066920 con 0x7f448806fcf0 2024-09-10T21:42:36.466 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-10T21:42:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.467+0000 7f448fc00640 1 -- 172.21.15.7:0/701872247 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4464077f70 msgr2=0x7f446407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.467+0000 7f448fc00640 1 --2- 172.21.15.7:0/701872247 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4464077f70 0x7f446407a430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f448814ef20 tx=0x7f4470005f90 comp rx=0 tx=0).stop 2024-09-10T21:42:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.467+0000 7f448fc00640 1 -- 172.21.15.7:0/701872247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f448806fcf0 msgr2=0x7f44881f5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.467+0000 7f448fc00640 1 --2- 172.21.15.7:0/701872247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f448806fcf0 0x7f44881f5050 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f447800c3e0 tx=0x7f447800c8a0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.467+0000 7f448fc00640 1 -- 172.21.15.7:0/701872247 shutdown_connections 2024-09-10T21:42:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.467+0000 7f448fc00640 1 --2- 172.21.15.7:0/701872247 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4464077f70 0x7f446407a430 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.467+0000 7f448fc00640 1 --2- 172.21.15.7:0/701872247 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4488156090 0x7f44880c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.467+0000 7f448fc00640 1 --2- 172.21.15.7:0/701872247 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4488074450 0x7f44881f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.467+0000 7f448fc00640 1 --2- 172.21.15.7:0/701872247 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f448806fcf0 0x7f44881f5050 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.467+0000 7f448fc00640 1 -- 172.21.15.7:0/701872247 >> 172.21.15.7:0/701872247 conn(0x7f44880b16d0 msgr2=0x7f44880b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.467+0000 7f448fc00640 1 -- 172.21.15.7:0/701872247 shutdown_connections 2024-09-10T21:42:36.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.467+0000 7f448fc00640 1 -- 172.21.15.7:0/701872247 wait complete. 2024-09-10T21:42:36.484 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T21:42:36.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.645+0000 7f6512400640 1 -- 172.21.15.7:0/608376966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650c074530 msgr2=0x7f650c15cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:36.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.645+0000 7f6512400640 1 --2- 172.21.15.7:0/608376966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650c074530 0x7f650c15cc00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f650c0ca2f0 tx=0x7f650402a890 comp rx=0 tx=0).stop 2024-09-10T21:42:36.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.646+0000 7f6512400640 1 -- 172.21.15.7:0/608376966 shutdown_connections 2024-09-10T21:42:36.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.646+0000 7f6512400640 1 --2- 172.21.15.7:0/608376966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650c074530 0x7f650c15cc00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.646+0000 7f6512400640 1 -- 172.21.15.7:0/608376966 >> 172.21.15.7:0/608376966 conn(0x7f650c0b16d0 msgr2=0x7f650c0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:36.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.646+0000 7f6512400640 1 -- 172.21.15.7:0/608376966 shutdown_connections 2024-09-10T21:42:36.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.646+0000 7f6512400640 1 -- 172.21.15.7:0/608376966 wait complete. 2024-09-10T21:42:36.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.647+0000 7f6512400640 1 Processor -- start 2024-09-10T21:42:36.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.647+0000 7f6512400640 1 -- start start 2024-09-10T21:42:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.648+0000 7f6512400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650c06e580 0x7f650c1f5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.648+0000 7f6512400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f650c074530 0x7f650c1516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.648+0000 7f6512400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f650c159680 0x7f650c154370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.648+0000 7f6512400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f650c0caa10 con 0x7f650c06e580 2024-09-10T21:42:36.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.648+0000 7f6512400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f650c0ca890 con 0x7f650c159680 2024-09-10T21:42:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.648+0000 7f6512400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f650c0cab90 con 0x7f650c074530 2024-09-10T21:42:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.648+0000 7f650b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650c06e580 0x7f650c1f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.649+0000 7f650b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650c06e580 0x7f650c1f5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44178/0 (socket says 172.21.15.7:44178) 2024-09-10T21:42:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.649+0000 7f650b400640 1 -- 172.21.15.7:0/3039726016 learned_addr learned my addr 172.21.15.7:0/3039726016 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.649+0000 7f650be00640 1 --2- 172.21.15.7:0/3039726016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f650c159680 0x7f650c154370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.649+0000 7f6502a00640 1 --2- 172.21.15.7:0/3039726016 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f650c074530 0x7f650c1516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.649+0000 7f650b400640 1 -- 172.21.15.7:0/3039726016 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f650c074530 msgr2=0x7f650c1516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.649+0000 7f650b400640 1 --2- 172.21.15.7:0/3039726016 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f650c074530 0x7f650c1516f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.649+0000 7f650b400640 1 -- 172.21.15.7:0/3039726016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f650c159680 msgr2=0x7f650c154370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.649+0000 7f650b400640 1 --2- 172.21.15.7:0/3039726016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f650c159680 0x7f650c154370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.649+0000 7f650b400640 1 -- 172.21.15.7:0/3039726016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f650c153ac0 con 0x7f650c06e580 2024-09-10T21:42:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.649+0000 7f6502a00640 1 --2- 172.21.15.7:0/3039726016 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f650c074530 0x7f650c1516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:36.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.649+0000 7f650b400640 1 --2- 172.21.15.7:0/3039726016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650c06e580 0x7f650c1f5600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00c3d0 tx=0x7f64fc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:36.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.649+0000 7f6503e00640 1 -- 172.21.15.7:0/3039726016 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc017070 con 0x7f650c06e580 2024-09-10T21:42:36.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.650+0000 7f6503e00640 1 -- 172.21.15.7:0/3039726016 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64fc0053c0 con 0x7f650c06e580 2024-09-10T21:42:36.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.650+0000 7f6512400640 1 -- 172.21.15.7:0/3039726016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f650c153da0 con 0x7f650c06e580 2024-09-10T21:42:36.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.650+0000 7f6503e00640 1 -- 172.21.15.7:0/3039726016 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc003850 con 0x7f650c06e580 2024-09-10T21:42:36.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.650+0000 7f6512400640 1 -- 172.21.15.7:0/3039726016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f650c15a680 con 0x7f650c06e580 2024-09-10T21:42:36.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.651+0000 7f6512400640 1 -- 172.21.15.7:0/3039726016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f650c0cb040 con 0x7f650c06e580 2024-09-10T21:42:36.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.652+0000 7f6503e00640 1 -- 172.21.15.7:0/3039726016 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f64fc0039f0 con 0x7f650c06e580 2024-09-10T21:42:36.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.653+0000 7f6503e00640 1 --2- 172.21.15.7:0/3039726016 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f64e8077f70 0x7f64e807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:36.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.653+0000 7f6503e00640 1 -- 172.21.15.7:0/3039726016 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f64fc099300 con 0x7f650c06e580 2024-09-10T21:42:36.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.653+0000 7f6502a00640 1 --2- 172.21.15.7:0/3039726016 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f64e8077f70 0x7f64e807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:36.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.654+0000 7f6502a00640 1 --2- 172.21.15.7:0/3039726016 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f64e8077f70 0x7f64e807a430 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f650c15f230 tx=0x7f64f4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:36.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.657+0000 7f6503e00640 1 -- 172.21.15.7:0/3039726016 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f64fc0617a0 con 0x7f650c06e580 2024-09-10T21:42:36.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.851+0000 7f6512400640 1 -- 172.21.15.7:0/3039726016 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f650c158db0 con 0x7f650c06e580 2024-09-10T21:42:36.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.852+0000 7f6503e00640 1 -- 172.21.15.7:0/3039726016 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) ==== 104+0+11 (secure 0 0 0) 0x7f64fc0661d0 con 0x7f650c06e580 2024-09-10T21:42:36.853 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 32 2024-09-10T21:42:36.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.854+0000 7f6512400640 1 -- 172.21.15.7:0/3039726016 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f64e8077f70 msgr2=0x7f64e807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:36.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.854+0000 7f6512400640 1 --2- 172.21.15.7:0/3039726016 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f64e8077f70 0x7f64e807a430 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f650c15f230 tx=0x7f64f4009040 comp rx=0 tx=0).stop 2024-09-10T21:42:36.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.854+0000 7f6512400640 1 -- 172.21.15.7:0/3039726016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650c06e580 msgr2=0x7f650c1f5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:36.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.854+0000 7f6512400640 1 --2- 172.21.15.7:0/3039726016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650c06e580 0x7f650c1f5600 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00c3d0 tx=0x7f64fc00c890 comp rx=0 tx=0).stop 2024-09-10T21:42:36.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.854+0000 7f6512400640 1 -- 172.21.15.7:0/3039726016 shutdown_connections 2024-09-10T21:42:36.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.854+0000 7f6512400640 1 --2- 172.21.15.7:0/3039726016 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f64e8077f70 0x7f64e807a430 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.854+0000 7f6512400640 1 --2- 172.21.15.7:0/3039726016 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f650c159680 0x7f650c154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.854+0000 7f6512400640 1 --2- 172.21.15.7:0/3039726016 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f650c074530 0x7f650c1516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.854+0000 7f6512400640 1 --2- 172.21.15.7:0/3039726016 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f650c06e580 0x7f650c1f5600 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:36.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.854+0000 7f6512400640 1 -- 172.21.15.7:0/3039726016 >> 172.21.15.7:0/3039726016 conn(0x7f650c0b16d0 msgr2=0x7f650c15dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:36.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.854+0000 7f6512400640 1 -- 172.21.15.7:0/3039726016 shutdown_connections 2024-09-10T21:42:36.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:36.854+0000 7f6512400640 1 -- 172.21.15.7:0/3039726016 wait complete. 2024-09-10T21:42:36.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T21:42:37.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.035+0000 7f2ae7e00640 1 -- 172.21.15.7:0/2343589739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2acc003c10 msgr2=0x7f2acc004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:37.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.035+0000 7f2ae7e00640 1 --2- 172.21.15.7:0/2343589739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2acc003c10 0x7f2acc004010 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8009b80 tx=0x7f2acc005830 comp rx=0 tx=0).stop 2024-09-10T21:42:37.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.035+0000 7f2ae7400640 1 -- 172.21.15.7:0/2343589739 reap_dead start 2024-09-10T21:42:37.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.035+0000 7f2ae7e00640 1 -- 172.21.15.7:0/2343589739 shutdown_connections 2024-09-10T21:42:37.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.035+0000 7f2ae7e00640 1 -- 172.21.15.7:0/2343589739 >> 172.21.15.7:0/2343589739 conn(0x7f2ae806cd00 msgr2=0x7f2ae806d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:37.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.035+0000 7f2ae7e00640 1 -- 172.21.15.7:0/2343589739 shutdown_connections 2024-09-10T21:42:37.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.035+0000 7f2ae7e00640 1 -- 172.21.15.7:0/2343589739 wait complete. 2024-09-10T21:42:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.037+0000 7f2ae7e00640 1 Processor -- start 2024-09-10T21:42:37.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.037+0000 7f2ae7e00640 1 -- start start 2024-09-10T21:42:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.038+0000 7f2ae7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2acc003c10 0x7f2ae81f4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.038+0000 7f2ae7e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ae8072460 0x7f2ae8146f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:37.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.038+0000 7f2ae7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ae8148f90 0x7f2ae815d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.038+0000 7f2ae7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae80b4d20 con 0x7f2acc003c10 2024-09-10T21:42:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.038+0000 7f2ae7e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae80b4ba0 con 0x7f2ae8148f90 2024-09-10T21:42:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.038+0000 7f2ae7e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f2ae8152fc0 con 0x7f2ae8072460 2024-09-10T21:42:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.038+0000 7f2ae6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2acc003c10 0x7f2ae81f4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.038+0000 7f2ae6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2acc003c10 0x7f2ae81f4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44182/0 (socket says 172.21.15.7:44182) 2024-09-10T21:42:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.038+0000 7f2ae6a00640 1 -- 172.21.15.7:0/1209576318 learned_addr learned my addr 172.21.15.7:0/1209576318 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.038+0000 7f2ae6a00640 1 -- 172.21.15.7:0/1209576318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ae8072460 msgr2=0x7f2ae8146f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.038+0000 7f2ae6a00640 1 --2- 172.21.15.7:0/1209576318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ae8072460 0x7f2ae8146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.038+0000 7f2ae6a00640 1 -- 172.21.15.7:0/1209576318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ae8148f90 msgr2=0x7f2ae815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.038+0000 7f2ae6a00640 1 --2- 172.21.15.7:0/1209576318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ae8148f90 0x7f2ae815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:37.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.038+0000 7f2ae6a00640 1 -- 172.21.15.7:0/1209576318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae81f4a80 con 0x7f2acc003c10 2024-09-10T21:42:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.039+0000 7f2ae6a00640 1 --2- 172.21.15.7:0/1209576318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2acc003c10 0x7f2ae81f4450 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8036040 tx=0x7f2ad8033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:37.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.039+0000 7f2adea00640 1 -- 172.21.15.7:0/1209576318 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8042020 con 0x7f2acc003c10 2024-09-10T21:42:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.039+0000 7f2adea00640 1 -- 172.21.15.7:0/1209576318 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ad8004150 con 0x7f2acc003c10 2024-09-10T21:42:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.039+0000 7f2adea00640 1 -- 172.21.15.7:0/1209576318 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ad8038040 con 0x7f2acc003c10 2024-09-10T21:42:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.039+0000 7f2ae7e00640 1 -- 172.21.15.7:0/1209576318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae81f4d60 con 0x7f2acc003c10 2024-09-10T21:42:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.039+0000 7f2ae7e00640 1 -- 172.21.15.7:0/1209576318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae81f5320 con 0x7f2acc003c10 2024-09-10T21:42:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.040+0000 7f2ae7e00640 1 -- 172.21.15.7:0/1209576318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae80b24f0 con 0x7f2acc003c10 2024-09-10T21:42:37.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.041+0000 7f2adea00640 1 -- 172.21.15.7:0/1209576318 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2ad803b030 con 0x7f2acc003c10 2024-09-10T21:42:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.043+0000 7f2adea00640 1 --2- 172.21.15.7:0/1209576318 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2ac8077f70 0x7f2ac807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.043+0000 7f2adea00640 1 -- 172.21.15.7:0/1209576318 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f2ad80b9bc0 con 0x7f2acc003c10 2024-09-10T21:42:37.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.043+0000 7f2ae6000640 1 --2- 172.21.15.7:0/1209576318 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2ac8077f70 0x7f2ac807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:37.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.044+0000 7f2ae6000640 1 --2- 172.21.15.7:0/1209576318 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2ac8077f70 0x7f2ac807a430 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2ae815dee0 tx=0x7f2ad4006060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:37.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.047+0000 7f2adea00640 1 -- 172.21.15.7:0/1209576318 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2ae80b24f0 con 0x7f2acc003c10 2024-09-10T21:42:37.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.238+0000 7f2ae7e00640 1 -- 172.21.15.7:0/1209576318 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f2ae814dc90 con 0x7f2acc003c10 2024-09-10T21:42:37.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.238+0000 7f2adea00640 1 -- 172.21.15.7:0/1209576318 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) ==== 100+0+11 (secure 0 0 0) 0x7f2ad80826c0 con 0x7f2acc003c10 2024-09-10T21:42:37.240 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 64 2024-09-10T21:42:37.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.241+0000 7f2ae7e00640 1 -- 172.21.15.7:0/1209576318 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2ac8077f70 msgr2=0x7f2ac807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:37.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.241+0000 7f2ae7e00640 1 --2- 172.21.15.7:0/1209576318 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2ac8077f70 0x7f2ac807a430 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2ae815dee0 tx=0x7f2ad4006060 comp rx=0 tx=0).stop 2024-09-10T21:42:37.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.241+0000 7f2ae7e00640 1 -- 172.21.15.7:0/1209576318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2acc003c10 msgr2=0x7f2ae81f4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:37.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.241+0000 7f2ae7e00640 1 --2- 172.21.15.7:0/1209576318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2acc003c10 0x7f2ae81f4450 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2ad8036040 tx=0x7f2ad8033670 comp rx=0 tx=0).stop 2024-09-10T21:42:37.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.241+0000 7f2ae7e00640 1 -- 172.21.15.7:0/1209576318 shutdown_connections 2024-09-10T21:42:37.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.241+0000 7f2ae7e00640 1 --2- 172.21.15.7:0/1209576318 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2ac8077f70 0x7f2ac807a430 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:37.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.241+0000 7f2ae7e00640 1 --2- 172.21.15.7:0/1209576318 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ae8148f90 0x7f2ae815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:37.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.241+0000 7f2ae7e00640 1 --2- 172.21.15.7:0/1209576318 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ae8072460 0x7f2ae8146f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:37.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.241+0000 7f2ae7e00640 1 --2- 172.21.15.7:0/1209576318 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2acc003c10 0x7f2ae81f4450 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:37.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.241+0000 7f2ae7e00640 1 -- 172.21.15.7:0/1209576318 >> 172.21.15.7:0/1209576318 conn(0x7f2ae806cd00 msgr2=0x7f2ae80af970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:37.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.241+0000 7f2ae7e00640 1 -- 172.21.15.7:0/1209576318 shutdown_connections 2024-09-10T21:42:37.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.241+0000 7f2ae7e00640 1 -- 172.21.15.7:0/1209576318 wait complete. 2024-09-10T21:42:37.258 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-09-10T21:42:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.417+0000 7fef0a200640 1 -- 172.21.15.7:0/2129843384 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fef040cbd50 msgr2=0x7fef040cc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.417+0000 7fef0a200640 1 --2- 172.21.15.7:0/2129843384 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fef040cbd50 0x7fef040cc190 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7feef0009830 tx=0x7feef002a510 comp rx=0 tx=0).stop 2024-09-10T21:42:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.417+0000 7fef0a200640 1 -- 172.21.15.7:0/2129843384 shutdown_connections 2024-09-10T21:42:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.417+0000 7fef0a200640 1 --2- 172.21.15.7:0/2129843384 >> v2:172.21.15.133:3300/0 conn(0x7fef040ae020 0x7fef040ae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.417+0000 7fef0a200640 1 --2- 172.21.15.7:0/2129843384 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fef040cbd50 0x7fef040cc190 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.417+0000 7fef0a200640 1 -- 172.21.15.7:0/2129843384 >> 172.21.15.7:0/2129843384 conn(0x7fef040b16d0 msgr2=0x7fef040b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.417+0000 7fef0a200640 1 -- 172.21.15.7:0/2129843384 shutdown_connections 2024-09-10T21:42:37.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.417+0000 7fef0a200640 1 -- 172.21.15.7:0/2129843384 wait complete. 2024-09-10T21:42:37.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.419+0000 7fef0a200640 1 Processor -- start 2024-09-10T21:42:37.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.419+0000 7fef0a200640 1 -- start start 2024-09-10T21:42:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.420+0000 7fef0a200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef040ae020 0x7fef040c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.420+0000 7fef0a200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fef040cbd50 0x7fef041574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.420+0000 7fef0a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef041619e0 0x7fef041f81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.420+0000 7fef0a200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fef040ce100 con 0x7fef041619e0 2024-09-10T21:42:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.420+0000 7fef0a200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fef040cdf80 con 0x7fef040ae020 2024-09-10T21:42:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.420+0000 7fef0a200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fef04161dc0 con 0x7fef040cbd50 2024-09-10T21:42:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.420+0000 7fef03e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef041619e0 0x7fef041f81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.420+0000 7fef03e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef041619e0 0x7fef041f81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44198/0 (socket says 172.21.15.7:44198) 2024-09-10T21:42:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.420+0000 7fef03e00640 1 -- 172.21.15.7:0/221282350 learned_addr learned my addr 172.21.15.7:0/221282350 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:37.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.421+0000 7fef02a00640 1 --2- 172.21.15.7:0/221282350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fef040cbd50 0x7fef041574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.421+0000 7fef03400640 1 --2- 172.21.15.7:0/221282350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef040ae020 0x7fef040c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.421+0000 7fef03e00640 1 -- 172.21.15.7:0/221282350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fef040cbd50 msgr2=0x7fef041574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.421+0000 7fef03e00640 1 --2- 172.21.15.7:0/221282350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fef040cbd50 0x7fef041574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.421+0000 7fef03e00640 1 -- 172.21.15.7:0/221282350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef040ae020 msgr2=0x7fef040c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.421+0000 7fef03e00640 1 --2- 172.21.15.7:0/221282350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef040ae020 0x7fef040c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.421+0000 7fef03e00640 1 -- 172.21.15.7:0/221282350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef040b4c70 con 0x7fef041619e0 2024-09-10T21:42:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.421+0000 7fef03400640 1 --2- 172.21.15.7:0/221282350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef040ae020 0x7fef040c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.421+0000 7fef03e00640 1 --2- 172.21.15.7:0/221282350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef041619e0 0x7fef041f81f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feefc00ed60 tx=0x7feefc00c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.421+0000 7feefb400640 1 -- 172.21.15.7:0/221282350 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc004d20 con 0x7fef041619e0 2024-09-10T21:42:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.421+0000 7feefb400640 1 -- 172.21.15.7:0/221282350 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feefc003970 con 0x7fef041619e0 2024-09-10T21:42:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.421+0000 7fef0a200640 1 -- 172.21.15.7:0/221282350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef040b4f50 con 0x7fef041619e0 2024-09-10T21:42:37.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.421+0000 7feefb400640 1 -- 172.21.15.7:0/221282350 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feefc003dc0 con 0x7fef041619e0 2024-09-10T21:42:37.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.421+0000 7fef0a200640 1 -- 172.21.15.7:0/221282350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef0415fc70 con 0x7fef041619e0 2024-09-10T21:42:37.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.422+0000 7fef0a200640 1 -- 172.21.15.7:0/221282350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef040ae4d0 con 0x7fef041619e0 2024-09-10T21:42:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.424+0000 7feefb400640 1 -- 172.21.15.7:0/221282350 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7feefc00b040 con 0x7fef041619e0 2024-09-10T21:42:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.424+0000 7feefb400640 1 --2- 172.21.15.7:0/221282350 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7feee4077f70 0x7feee407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.425+0000 7feefb400640 1 -- 172.21.15.7:0/221282350 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7feefc09a790 con 0x7fef041619e0 2024-09-10T21:42:37.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.425+0000 7fef03400640 1 --2- 172.21.15.7:0/221282350 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7feee4077f70 0x7feee407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:37.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.426+0000 7fef03400640 1 --2- 172.21.15.7:0/221282350 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7feee4077f70 0x7feee407a430 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feef4001740 tx=0x7feef4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:37.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.428+0000 7feefb400640 1 -- 172.21.15.7:0/221282350 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7feefc063280 con 0x7fef041619e0 2024-09-10T21:42:37.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.699+0000 7fef0a200640 1 -- 172.21.15.7:0/221282350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7fef040b5b00 con 0x7fef041619e0 2024-09-10T21:42:37.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.795+0000 7feefb400640 1 -- 172.21.15.7:0/221282350 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v47) ==== 145+0+0 (secure 0 0 0) 0x7feefc062b80 con 0x7fef041619e0 2024-09-10T21:42:37.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.798+0000 7fef0a200640 1 -- 172.21.15.7:0/221282350 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7feee4077f70 msgr2=0x7feee407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:37.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.798+0000 7fef0a200640 1 --2- 172.21.15.7:0/221282350 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7feee4077f70 0x7feee407a430 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7feef4001740 tx=0x7feef4009040 comp rx=0 tx=0).stop 2024-09-10T21:42:37.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.798+0000 7fef0a200640 1 -- 172.21.15.7:0/221282350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef041619e0 msgr2=0x7fef041f81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:37.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.798+0000 7fef0a200640 1 --2- 172.21.15.7:0/221282350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef041619e0 0x7fef041f81f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7feefc00ed60 tx=0x7feefc00c740 comp rx=0 tx=0).stop 2024-09-10T21:42:37.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.798+0000 7fef0a200640 1 -- 172.21.15.7:0/221282350 shutdown_connections 2024-09-10T21:42:37.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.798+0000 7fef0a200640 1 --2- 172.21.15.7:0/221282350 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7feee4077f70 0x7feee407a430 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:37.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.798+0000 7fef0a200640 1 --2- 172.21.15.7:0/221282350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fef041619e0 0x7fef041f81f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:37.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.798+0000 7fef0a200640 1 --2- 172.21.15.7:0/221282350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fef040cbd50 0x7fef041574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:37.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.798+0000 7fef0a200640 1 --2- 172.21.15.7:0/221282350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fef040ae020 0x7fef040c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:37.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.798+0000 7fef0a200640 1 -- 172.21.15.7:0/221282350 >> 172.21.15.7:0/221282350 conn(0x7fef040b16d0 msgr2=0x7fef0415cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:37.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.798+0000 7fef0a200640 1 -- 172.21.15.7:0/221282350 shutdown_connections 2024-09-10T21:42:37.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.798+0000 7fef0a200640 1 -- 172.21.15.7:0/221282350 wait complete. 2024-09-10T21:42:37.816 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:42:37.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.976+0000 7fcd40e00640 1 -- 172.21.15.7:0/3381453903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd20005210 msgr2=0x7fcd2000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:37.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.976+0000 7fcd40e00640 1 --2- 172.21.15.7:0/3381453903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd20005210 0x7fcd2000b9c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fcd34009ed0 tx=0x7fcd20005830 comp rx=0 tx=0).stop 2024-09-10T21:42:37.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.976+0000 7fcd40e00640 1 -- 172.21.15.7:0/3381453903 shutdown_connections 2024-09-10T21:42:37.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.976+0000 7fcd40e00640 1 --2- 172.21.15.7:0/3381453903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd20005210 0x7fcd2000b9c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:37.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.976+0000 7fcd40e00640 1 -- 172.21.15.7:0/3381453903 >> 172.21.15.7:0/3381453903 conn(0x7fcd3c0b16d0 msgr2=0x7fcd3c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:37.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.976+0000 7fcd40e00640 1 -- 172.21.15.7:0/3381453903 shutdown_connections 2024-09-10T21:42:37.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.976+0000 7fcd40e00640 1 -- 172.21.15.7:0/3381453903 wait complete. 2024-09-10T21:42:37.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.977+0000 7fcd40e00640 1 Processor -- start 2024-09-10T21:42:37.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.978+0000 7fcd40e00640 1 -- start start 2024-09-10T21:42:37.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.978+0000 7fcd40e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd20005210 0x7fcd3c0c9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:37.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.978+0000 7fcd40e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd3c1595d0 0x7fcd3c1f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:37.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.978+0000 7fcd40e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd3c159a50 0x7fcd3c14da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:37.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.978+0000 7fcd40e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c06ee30 con 0x7fcd3c1595d0 2024-09-10T21:42:37.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.979+0000 7fcd40e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c06ecb0 con 0x7fcd20005210 2024-09-10T21:42:37.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.979+0000 7fcd40e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fcd3c073550 con 0x7fcd3c159a50 2024-09-10T21:42:37.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.979+0000 7fcd39400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd3c1595d0 0x7fcd3c1f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:37.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.979+0000 7fcd39400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd3c1595d0 0x7fcd3c1f59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44206/0 (socket says 172.21.15.7:44206) 2024-09-10T21:42:37.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.979+0000 7fcd39400640 1 -- 172.21.15.7:0/1282373453 learned_addr learned my addr 172.21.15.7:0/1282373453 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:37.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.979+0000 7fcd39e00640 1 --2- 172.21.15.7:0/1282373453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd20005210 0x7fcd3c0c9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:37.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.979+0000 7fcd3a800640 1 --2- 172.21.15.7:0/1282373453 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd3c159a50 0x7fcd3c14da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:37.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.979+0000 7fcd39400640 1 -- 172.21.15.7:0/1282373453 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd3c159a50 msgr2=0x7fcd3c14da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:37.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.979+0000 7fcd39400640 1 --2- 172.21.15.7:0/1282373453 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd3c159a50 0x7fcd3c14da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:37.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.979+0000 7fcd39400640 1 -- 172.21.15.7:0/1282373453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd20005210 msgr2=0x7fcd3c0c9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:37.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.979+0000 7fcd39400640 1 --2- 172.21.15.7:0/1282373453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd20005210 0x7fcd3c0c9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:37.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.979+0000 7fcd39400640 1 -- 172.21.15.7:0/1282373453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3c1f64b0 con 0x7fcd3c1595d0 2024-09-10T21:42:37.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.979+0000 7fcd39e00640 1 --2- 172.21.15.7:0/1282373453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd20005210 0x7fcd3c0c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:37.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.979+0000 7fcd3a800640 1 --2- 172.21.15.7:0/1282373453 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd3c159a50 0x7fcd3c14da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:37.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.979+0000 7fcd39400640 1 --2- 172.21.15.7:0/1282373453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd3c1595d0 0x7fcd3c1f59f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcd28000fd0 tx=0x7fcd2800ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:37.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.980+0000 7fcd32000640 1 -- 172.21.15.7:0/1282373453 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28015070 con 0x7fcd3c1595d0 2024-09-10T21:42:37.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.980+0000 7fcd32000640 1 -- 172.21.15.7:0/1282373453 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd28010980 con 0x7fcd3c1595d0 2024-09-10T21:42:37.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.980+0000 7fcd32000640 1 -- 172.21.15.7:0/1282373453 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd28010d70 con 0x7fcd3c1595d0 2024-09-10T21:42:37.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.980+0000 7fcd40e00640 1 -- 172.21.15.7:0/1282373453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3c1f6790 con 0x7fcd3c1595d0 2024-09-10T21:42:37.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.980+0000 7fcd40e00640 1 -- 172.21.15.7:0/1282373453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3c0c9e80 con 0x7fcd3c1595d0 2024-09-10T21:42:37.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.981+0000 7fcd40e00640 1 -- 172.21.15.7:0/1282373453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd3c0c8f30 con 0x7fcd3c1595d0 2024-09-10T21:42:37.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.982+0000 7fcd32000640 1 -- 172.21.15.7:0/1282373453 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fcd28011460 con 0x7fcd3c1595d0 2024-09-10T21:42:37.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.983+0000 7fcd32000640 1 --2- 172.21.15.7:0/1282373453 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcd1c077f70 0x7fcd1c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:37.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.983+0000 7fcd32000640 1 -- 172.21.15.7:0/1282373453 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcd2809a750 con 0x7fcd3c1595d0 2024-09-10T21:42:37.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.983+0000 7fcd39e00640 1 --2- 172.21.15.7:0/1282373453 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcd1c077f70 0x7fcd1c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:37.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.984+0000 7fcd39e00640 1 --2- 172.21.15.7:0/1282373453 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcd1c077f70 0x7fcd1c07a430 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c006fd0 tx=0x7fcd2c006b10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:37.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:37.987+0000 7fcd32000640 1 -- 172.21.15.7:0/1282373453 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fcd28063240 con 0x7fcd3c1595d0 2024-09-10T21:42:38.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.182+0000 7fcd40e00640 1 -- 172.21.15.7:0/1282373453 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fcd3c073ab0 con 0x7fcd3c1595d0 2024-09-10T21:42:38.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.182+0000 7fcd32000640 1 -- 172.21.15.7:0/1282373453 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v27) ==== 59+0+1998 (secure 0 0 0) 0x7fcd28062b40 con 0x7fcd3c1595d0 2024-09-10T21:42:38.184 INFO:teuthology.orchestra.run.smithi007.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:42:38.184 INFO:teuthology.orchestra.run.smithi007.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:42:38.184 INFO:teuthology.orchestra.run.smithi007.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:42:38.184 INFO:teuthology.orchestra.run.smithi007.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:42:38.184 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3770.177547 2024-09-10T21:42:38.184 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3542.062395 2024-09-10T21:42:38.184 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3788.037135 2024-09-10T21:42:38.184 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5808.986321 2024-09-10T21:42:38.185 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4630.109911 2024-09-10T21:42:38.185 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4510.255493 2024-09-10T21:42:38.185 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4792.698527 2024-09-10T21:42:38.185 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6842.370281 2024-09-10T21:42:38.185 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_mds 20/20 2024-09-10T21:42:38.185 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:42:38.185 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_ms 2/2 2024-09-10T21:42:38.185 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_debug_frag true 2024-09-10T21:42:38.185 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:42:38.185 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:42:38.185 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_verify_scatter true 2024-09-10T21:42:38.185 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:42:38.185 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:42:38.186 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:42:38.186 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced client_mount_timeout 600 2024-09-10T21:42:38.186 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced debug_client 20/20 2024-09-10T21:42:38.186 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced debug_ms 1/1 2024-09-10T21:42:38.186 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:42:38.186 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:42:38.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.185+0000 7fcd40e00640 1 -- 172.21.15.7:0/1282373453 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcd1c077f70 msgr2=0x7fcd1c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:38.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.185+0000 7fcd40e00640 1 --2- 172.21.15.7:0/1282373453 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcd1c077f70 0x7fcd1c07a430 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c006fd0 tx=0x7fcd2c006b10 comp rx=0 tx=0).stop 2024-09-10T21:42:38.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.185+0000 7fcd40e00640 1 -- 172.21.15.7:0/1282373453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd3c1595d0 msgr2=0x7fcd3c1f59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:38.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.185+0000 7fcd40e00640 1 --2- 172.21.15.7:0/1282373453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd3c1595d0 0x7fcd3c1f59f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fcd28000fd0 tx=0x7fcd2800ea00 comp rx=0 tx=0).stop 2024-09-10T21:42:38.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.185+0000 7fcd40e00640 1 -- 172.21.15.7:0/1282373453 shutdown_connections 2024-09-10T21:42:38.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.185+0000 7fcd40e00640 1 --2- 172.21.15.7:0/1282373453 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcd1c077f70 0x7fcd1c07a430 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.185+0000 7fcd40e00640 1 --2- 172.21.15.7:0/1282373453 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcd3c159a50 0x7fcd3c14da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.185+0000 7fcd40e00640 1 --2- 172.21.15.7:0/1282373453 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcd3c1595d0 0x7fcd3c1f59f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.185+0000 7fcd40e00640 1 --2- 172.21.15.7:0/1282373453 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcd20005210 0x7fcd3c0c9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.185+0000 7fcd40e00640 1 -- 172.21.15.7:0/1282373453 >> 172.21.15.7:0/1282373453 conn(0x7fcd3c0b16d0 msgr2=0x7fcd3c072c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:38.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.185+0000 7fcd40e00640 1 -- 172.21.15.7:0/1282373453 shutdown_connections 2024-09-10T21:42:38.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.185+0000 7fcd40e00640 1 -- 172.21.15.7:0/1282373453 wait complete. 2024-09-10T21:42:38.202 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:42:38.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.364+0000 7f8073200640 1 -- 172.21.15.7:0/70385711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80740d0700 msgr2=0x7f807415ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:38.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.364+0000 7f8073200640 1 --2- 172.21.15.7:0/70385711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80740d0700 0x7f807415ca80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f80740b4da0 tx=0x7f806402a560 comp rx=0 tx=0).stop 2024-09-10T21:42:38.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.364+0000 7f8073200640 1 -- 172.21.15.7:0/70385711 shutdown_connections 2024-09-10T21:42:38.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.364+0000 7f8073200640 1 --2- 172.21.15.7:0/70385711 >> v2:172.21.15.133:3300/0 conn(0x7f807406d230 0x7f80740b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.364+0000 7f8073200640 1 --2- 172.21.15.7:0/70385711 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f80740d0700 0x7f807415ca80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.364+0000 7f8073200640 1 -- 172.21.15.7:0/70385711 >> 172.21.15.7:0/70385711 conn(0x7f80740b26e0 msgr2=0x7f80740b2af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:38.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.364+0000 7f8073200640 1 -- 172.21.15.7:0/70385711 shutdown_connections 2024-09-10T21:42:38.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.364+0000 7f8073200640 1 -- 172.21.15.7:0/70385711 wait complete. 2024-09-10T21:42:38.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.366+0000 7f8073200640 1 Processor -- start 2024-09-10T21:42:38.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.366+0000 7f8073200640 1 -- start start 2024-09-10T21:42:38.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.367+0000 7f8073200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f807406d230 0x7f80741f87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:38.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.367+0000 7f8073200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80740b5530 0x7f80741e5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:38.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.367+0000 7f8073200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80740d0700 0x7f807415eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:38.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.367+0000 7f8073200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f80740b42b0 con 0x7f807406d230 2024-09-10T21:42:38.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.367+0000 7f8073200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f80740b4130 con 0x7f80740d0700 2024-09-10T21:42:38.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.367+0000 7f8073200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f80740b4430 con 0x7f80740b5530 2024-09-10T21:42:38.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.367+0000 7f8071e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f807406d230 0x7f80741f87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:38.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.367+0000 7f8071e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f807406d230 0x7f80741f87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44218/0 (socket says 172.21.15.7:44218) 2024-09-10T21:42:38.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.367+0000 7f8071e00640 1 -- 172.21.15.7:0/2849433455 learned_addr learned my addr 172.21.15.7:0/2849433455 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:38.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.367+0000 7f8072800640 1 --2- 172.21.15.7:0/2849433455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80740d0700 0x7f807415eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:38.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.367+0000 7f8071400640 1 --2- 172.21.15.7:0/2849433455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80740b5530 0x7f80741e5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:38.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.368+0000 7f8071e00640 1 -- 172.21.15.7:0/2849433455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80740b5530 msgr2=0x7f80741e5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:38.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.368+0000 7f8071e00640 1 --2- 172.21.15.7:0/2849433455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80740b5530 0x7f80741e5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.368+0000 7f8071e00640 1 -- 172.21.15.7:0/2849433455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80740d0700 msgr2=0x7f807415eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:38.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.368+0000 7f8071e00640 1 --2- 172.21.15.7:0/2849433455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80740d0700 0x7f807415eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.368+0000 7f8071e00640 1 -- 172.21.15.7:0/2849433455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80741e5c60 con 0x7f807406d230 2024-09-10T21:42:38.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.368+0000 7f8071400640 1 --2- 172.21.15.7:0/2849433455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80740b5530 0x7f80741e5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:38.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.368+0000 7f8071e00640 1 --2- 172.21.15.7:0/2849433455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f807406d230 0x7f80741f87b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8064000c00 tx=0x7f8064003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:38.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.368+0000 7f806a000640 1 -- 172.21.15.7:0/2849433455 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f806403e030 con 0x7f807406d230 2024-09-10T21:42:38.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.368+0000 7f806a000640 1 -- 172.21.15.7:0/2849433455 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8064003e00 con 0x7f807406d230 2024-09-10T21:42:38.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.368+0000 7f806a000640 1 -- 172.21.15.7:0/2849433455 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8064035040 con 0x7f807406d230 2024-09-10T21:42:38.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.368+0000 7f8073200640 1 -- 172.21.15.7:0/2849433455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80741e5f40 con 0x7f807406d230 2024-09-10T21:42:38.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.368+0000 7f8073200640 1 -- 172.21.15.7:0/2849433455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80741e5950 con 0x7f807406d230 2024-09-10T21:42:38.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.369+0000 7f8073200640 1 -- 172.21.15.7:0/2849433455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80740b4950 con 0x7f807406d230 2024-09-10T21:42:38.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.371+0000 7f806a000640 1 -- 172.21.15.7:0/2849433455 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f80640049c0 con 0x7f807406d230 2024-09-10T21:42:38.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.372+0000 7f806a000640 1 --2- 172.21.15.7:0/2849433455 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8054077f70 0x7f805407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:38.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.372+0000 7f806a000640 1 -- 172.21.15.7:0/2849433455 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f80640ba710 con 0x7f807406d230 2024-09-10T21:42:38.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.373+0000 7f8071400640 1 --2- 172.21.15.7:0/2849433455 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8054077f70 0x7f805407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:38.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.373+0000 7f8071400640 1 --2- 172.21.15.7:0/2849433455 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8054077f70 0x7f805407a430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8060007fd0 tx=0x7f8060009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:38.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.376+0000 7f806a000640 1 -- 172.21.15.7:0/2849433455 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8064082bb0 con 0x7f807406d230 2024-09-10T21:42:38.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.568+0000 7f8073200640 1 -- 172.21.15.7:0/2849433455 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f80740b3370 con 0x7f807406d230 2024-09-10T21:42:38.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.572+0000 7f806a000640 1 -- 172.21.15.7:0/2849433455 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) ==== 122+0+0 (secure 0 0 0) 0x7f80640875e0 con 0x7f807406d230 2024-09-10T21:42:38.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.575+0000 7f8073200640 1 -- 172.21.15.7:0/2849433455 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8054077f70 msgr2=0x7f805407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:38.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.575+0000 7f8073200640 1 --2- 172.21.15.7:0/2849433455 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8054077f70 0x7f805407a430 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8060007fd0 tx=0x7f8060009040 comp rx=0 tx=0).stop 2024-09-10T21:42:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.575+0000 7f8073200640 1 -- 172.21.15.7:0/2849433455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f807406d230 msgr2=0x7f80741f87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.575+0000 7f8073200640 1 --2- 172.21.15.7:0/2849433455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f807406d230 0x7f80741f87b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8064000c00 tx=0x7f8064003910 comp rx=0 tx=0).stop 2024-09-10T21:42:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.575+0000 7f8073200640 1 -- 172.21.15.7:0/2849433455 shutdown_connections 2024-09-10T21:42:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.575+0000 7f8073200640 1 --2- 172.21.15.7:0/2849433455 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8054077f70 0x7f805407a430 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.575+0000 7f8073200640 1 --2- 172.21.15.7:0/2849433455 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f80740d0700 0x7f807415eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.575+0000 7f8073200640 1 --2- 172.21.15.7:0/2849433455 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f80740b5530 0x7f80741e5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.575+0000 7f8073200640 1 --2- 172.21.15.7:0/2849433455 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f807406d230 0x7f80741f87b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.575+0000 7f8073200640 1 -- 172.21.15.7:0/2849433455 >> 172.21.15.7:0/2849433455 conn(0x7f80740b26e0 msgr2=0x7f8074070200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.575+0000 7f8073200640 1 -- 172.21.15.7:0/2849433455 shutdown_connections 2024-09-10T21:42:38.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:38.575+0000 7f8073200640 1 -- 172.21.15.7:0/2849433455 wait complete. 2024-09-10T21:42:38.592 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:42:38.592 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:42:38.692 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:42:38.783 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:42:38.783 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:42:38.783 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:42:38.819 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:42:38.819 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:42:38.895 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:42:38.896 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:42:38.971 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:42:39.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.174+0000 7f9196200640 1 -- 172.21.15.7:0/3731752626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9174004800 msgr2=0x7f9174004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.174+0000 7f9196200640 1 --2- 172.21.15.7:0/3731752626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9174004800 0x7f9174004c80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f917800c310 tx=0x7f9174005830 comp rx=0 tx=0).stop 2024-09-10T21:42:39.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.174+0000 7f9196200640 1 -- 172.21.15.7:0/3731752626 shutdown_connections 2024-09-10T21:42:39.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.174+0000 7f9196200640 1 --2- 172.21.15.7:0/3731752626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9174004800 0x7f9174004c80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.174+0000 7f9196200640 1 -- 172.21.15.7:0/3731752626 >> 172.21.15.7:0/3731752626 conn(0x7f91900b1660 msgr2=0x7f91900b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:39.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.174+0000 7f9196200640 1 -- 172.21.15.7:0/3731752626 shutdown_connections 2024-09-10T21:42:39.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.174+0000 7f9196200640 1 -- 172.21.15.7:0/3731752626 wait complete. 2024-09-10T21:42:39.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.176+0000 7f9196200640 1 Processor -- start 2024-09-10T21:42:39.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.176+0000 7f9196200640 1 -- start start 2024-09-10T21:42:39.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.177+0000 7f9196200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9174004800 0x7f919015e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.177+0000 7f9196200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91901f2df0 0x7f91900c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.177+0000 7f9196200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f919014bc00 0x7f919014c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.177+0000 7f9196200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f919006e510 con 0x7f91901f2df0 2024-09-10T21:42:39.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.177+0000 7f9196200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f919006f930 con 0x7f919014bc00 2024-09-10T21:42:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.177+0000 7f9196200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f919006e690 con 0x7f9174004800 2024-09-10T21:42:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.177+0000 7f9186a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91901f2df0 0x7f91900c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.177+0000 7f9186a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91901f2df0 0x7f91900c6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44224/0 (socket says 172.21.15.7:44224) 2024-09-10T21:42:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.177+0000 7f9186a00640 1 -- 172.21.15.7:0/544660835 learned_addr learned my addr 172.21.15.7:0/544660835 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.177+0000 7f918f400640 1 --2- 172.21.15.7:0/544660835 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9174004800 0x7f919015e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:39.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.177+0000 7f918fe00640 1 --2- 172.21.15.7:0/544660835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f919014bc00 0x7f919014c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:39.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.178+0000 7f9186a00640 1 -- 172.21.15.7:0/544660835 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9174004800 msgr2=0x7f919015e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.178+0000 7f9186a00640 1 --2- 172.21.15.7:0/544660835 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9174004800 0x7f919015e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.178+0000 7f9186a00640 1 -- 172.21.15.7:0/544660835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f919014bc00 msgr2=0x7f919014c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.178+0000 7f9186a00640 1 --2- 172.21.15.7:0/544660835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f919014bc00 0x7f919014c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.178+0000 7f9186a00640 1 -- 172.21.15.7:0/544660835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91900c80d0 con 0x7f91901f2df0 2024-09-10T21:42:39.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.178+0000 7f9186a00640 1 --2- 172.21.15.7:0/544660835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91901f2df0 0x7f91900c6be0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f917802b180 tx=0x7f91780060d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:39.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.178+0000 7f9187e00640 1 -- 172.21.15.7:0/544660835 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178031040 con 0x7f91901f2df0 2024-09-10T21:42:39.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.178+0000 7f9187e00640 1 -- 172.21.15.7:0/544660835 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91780035f0 con 0x7f91901f2df0 2024-09-10T21:42:39.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.178+0000 7f9187e00640 1 -- 172.21.15.7:0/544660835 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178003a20 con 0x7f91901f2df0 2024-09-10T21:42:39.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.178+0000 7f9196200640 1 -- 172.21.15.7:0/544660835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91900c75b0 con 0x7f91901f2df0 2024-09-10T21:42:39.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.178+0000 7f9196200640 1 -- 172.21.15.7:0/544660835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91900c7df0 con 0x7f91901f2df0 2024-09-10T21:42:39.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.179+0000 7f9196200640 1 -- 172.21.15.7:0/544660835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9190073cd0 con 0x7f91901f2df0 2024-09-10T21:42:39.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.180+0000 7f9187e00640 1 -- 172.21.15.7:0/544660835 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9178003bc0 con 0x7f91901f2df0 2024-09-10T21:42:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.181+0000 7f9187e00640 1 --2- 172.21.15.7:0/544660835 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f916c077f70 0x7f916c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.181+0000 7f9187e00640 1 -- 172.21.15.7:0/544660835 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f91780b9bf0 con 0x7f91901f2df0 2024-09-10T21:42:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.182+0000 7f918f400640 1 --2- 172.21.15.7:0/544660835 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f916c077f70 0x7f916c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:39.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.182+0000 7f918f400640 1 --2- 172.21.15.7:0/544660835 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f916c077f70 0x7f916c07a430 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f91800036b0 tx=0x7f9180006b70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:39.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.186+0000 7f9187e00640 1 -- 172.21.15.7:0/544660835 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f91900c75b0 con 0x7f91901f2df0 2024-09-10T21:42:39.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.396+0000 7f9196200640 1 -- 172.21.15.7:0/544660835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f91900c7890 con 0x7f91901f2df0 2024-09-10T21:42:39.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.396+0000 7f9187e00640 1 -- 172.21.15.7:0/544660835 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f9178006240 con 0x7f91901f2df0 2024-09-10T21:42:39.398 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-10T21:42:39.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.398+0000 7f9196200640 1 -- 172.21.15.7:0/544660835 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f916c077f70 msgr2=0x7f916c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.398+0000 7f9196200640 1 --2- 172.21.15.7:0/544660835 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f916c077f70 0x7f916c07a430 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f91800036b0 tx=0x7f9180006b70 comp rx=0 tx=0).stop 2024-09-10T21:42:39.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.398+0000 7f9196200640 1 -- 172.21.15.7:0/544660835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91901f2df0 msgr2=0x7f91900c6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.398+0000 7f9196200640 1 --2- 172.21.15.7:0/544660835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91901f2df0 0x7f91900c6be0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f917802b180 tx=0x7f91780060d0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.399+0000 7f9196200640 1 -- 172.21.15.7:0/544660835 shutdown_connections 2024-09-10T21:42:39.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.399+0000 7f9196200640 1 --2- 172.21.15.7:0/544660835 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f916c077f70 0x7f916c07a430 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.399+0000 7f9196200640 1 --2- 172.21.15.7:0/544660835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f919014bc00 0x7f919014c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.399+0000 7f9196200640 1 --2- 172.21.15.7:0/544660835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91901f2df0 0x7f91900c6be0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.399+0000 7f9196200640 1 --2- 172.21.15.7:0/544660835 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9174004800 0x7f919015e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.399+0000 7f9196200640 1 -- 172.21.15.7:0/544660835 >> 172.21.15.7:0/544660835 conn(0x7f91900b1660 msgr2=0x7f91900b4330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:39.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.399+0000 7f9196200640 1 -- 172.21.15.7:0/544660835 shutdown_connections 2024-09-10T21:42:39.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.399+0000 7f9196200640 1 -- 172.21.15.7:0/544660835 wait complete. 2024-09-10T21:42:39.415 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:42:39.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:39.416 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-10T21:42:39.477 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:42:39.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:39.478 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:42:39.524 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:42:39.524 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:42:39.564 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:42:39.564 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:42:39.643 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:42:39.643 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:42:39.721 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:42:39.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.883+0000 7f35efc00640 1 -- 172.21.15.7:0/2734975812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35e80ccd80 msgr2=0x7f35e80b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.883+0000 7f35efc00640 1 --2- 172.21.15.7:0/2734975812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35e80ccd80 0x7f35e80b1710 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f35e0009be0 tx=0x7f35e002a890 comp rx=0 tx=0).stop 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.883+0000 7f35efc00640 1 -- 172.21.15.7:0/2734975812 shutdown_connections 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.883+0000 7f35efc00640 1 --2- 172.21.15.7:0/2734975812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35e80ccd80 0x7f35e80b1710 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.883+0000 7f35efc00640 1 -- 172.21.15.7:0/2734975812 >> 172.21.15.7:0/2734975812 conn(0x7f35e80b1f00 msgr2=0x7f35e80b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.883+0000 7f35efc00640 1 -- 172.21.15.7:0/2734975812 shutdown_connections 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.883+0000 7f35efc00640 1 -- 172.21.15.7:0/2734975812 wait complete. 2024-09-10T21:42:39.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.885+0000 7f35efc00640 1 Processor -- start 2024-09-10T21:42:39.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.885+0000 7f35efc00640 1 -- start start 2024-09-10T21:42:39.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.886+0000 7f35efc00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35e80688e0 0x7f35e81596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.886+0000 7f35efc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e80ccd80 0x7f35e81f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.886+0000 7f35efc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e815bbf0 0x7f35e8158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.886+0000 7f35efc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8167030 con 0x7f35e80ccd80 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.886+0000 7f35efc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f35e8166eb0 con 0x7f35e815bbf0 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.886+0000 7f35efc00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f35e81671b0 con 0x7f35e80688e0 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.886+0000 7f35e7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e80ccd80 0x7f35e81f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.886+0000 7f35e7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e80ccd80 0x7f35e81f5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44240/0 (socket says 172.21.15.7:44240) 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.886+0000 7f35e7e00640 1 -- 172.21.15.7:0/618051795 learned_addr learned my addr 172.21.15.7:0/618051795 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.886+0000 7f35edc00640 1 --2- 172.21.15.7:0/618051795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e815bbf0 0x7f35e8158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.886+0000 7f35ed200640 1 --2- 172.21.15.7:0/618051795 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35e80688e0 0x7f35e81596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.886+0000 7f35e7e00640 1 -- 172.21.15.7:0/618051795 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35e80688e0 msgr2=0x7f35e81596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.886+0000 7f35e7e00640 1 --2- 172.21.15.7:0/618051795 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35e80688e0 0x7f35e81596f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.886+0000 7f35e7e00640 1 -- 172.21.15.7:0/618051795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e815bbf0 msgr2=0x7f35e8158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.886+0000 7f35e7e00640 1 --2- 172.21.15.7:0/618051795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e815bbf0 0x7f35e8158000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.886+0000 7f35e7e00640 1 -- 172.21.15.7:0/618051795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35e80ca020 con 0x7f35e80ccd80 2024-09-10T21:42:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.886+0000 7f35edc00640 1 --2- 172.21.15.7:0/618051795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e815bbf0 0x7f35e8158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:39.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.887+0000 7f35e7e00640 1 --2- 172.21.15.7:0/618051795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e80ccd80 0x7f35e81f5530 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f35d400c450 tx=0x7f35d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.887+0000 7f35e4c00640 1 -- 172.21.15.7:0/618051795 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d40052d0 con 0x7f35e80ccd80 2024-09-10T21:42:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.887+0000 7f35e4c00640 1 -- 172.21.15.7:0/618051795 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35d4005490 con 0x7f35e80ccd80 2024-09-10T21:42:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.887+0000 7f35e4c00640 1 -- 172.21.15.7:0/618051795 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35d4012890 con 0x7f35e80ccd80 2024-09-10T21:42:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.887+0000 7f35efc00640 1 -- 172.21.15.7:0/618051795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35e80ca300 con 0x7f35e80ccd80 2024-09-10T21:42:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.887+0000 7f35efc00640 1 -- 172.21.15.7:0/618051795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35e815d580 con 0x7f35e80ccd80 2024-09-10T21:42:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.888+0000 7f35efc00640 1 -- 172.21.15.7:0/618051795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35e815bfd0 con 0x7f35e80ccd80 2024-09-10T21:42:39.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.889+0000 7f35e4c00640 1 -- 172.21.15.7:0/618051795 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f35d4012a30 con 0x7f35e80ccd80 2024-09-10T21:42:39.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.890+0000 7f35e4c00640 1 --2- 172.21.15.7:0/618051795 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f35c8077f70 0x7f35c807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.890+0000 7f35e4c00640 1 -- 172.21.15.7:0/618051795 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f35d4099970 con 0x7f35e80ccd80 2024-09-10T21:42:39.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.891+0000 7f35ed200640 1 --2- 172.21.15.7:0/618051795 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f35c8077f70 0x7f35c807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:39.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.891+0000 7f35ed200640 1 --2- 172.21.15.7:0/618051795 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f35c8077f70 0x7f35c807a430 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0015b0 tx=0x7f35dc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:39.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:39.894+0000 7f35e4c00640 1 -- 172.21.15.7:0/618051795 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f35d40623d0 con 0x7f35e80ccd80 2024-09-10T21:42:40.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.100+0000 7f35efc00640 1 -- 172.21.15.7:0/618051795 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f35e80ca7c0 con 0x7f35e80ccd80 2024-09-10T21:42:40.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.101+0000 7f35e4c00640 1 -- 172.21.15.7:0/618051795 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f35d4061cd0 con 0x7f35e80ccd80 2024-09-10T21:42:40.102 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-10T21:42:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.103+0000 7f35efc00640 1 -- 172.21.15.7:0/618051795 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f35c8077f70 msgr2=0x7f35c807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.103+0000 7f35efc00640 1 --2- 172.21.15.7:0/618051795 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f35c8077f70 0x7f35c807a430 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f35dc0015b0 tx=0x7f35dc009040 comp rx=0 tx=0).stop 2024-09-10T21:42:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.103+0000 7f35efc00640 1 -- 172.21.15.7:0/618051795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e80ccd80 msgr2=0x7f35e81f5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.103+0000 7f35efc00640 1 --2- 172.21.15.7:0/618051795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e80ccd80 0x7f35e81f5530 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f35d400c450 tx=0x7f35d400c910 comp rx=0 tx=0).stop 2024-09-10T21:42:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.103+0000 7f35efc00640 1 -- 172.21.15.7:0/618051795 shutdown_connections 2024-09-10T21:42:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.103+0000 7f35efc00640 1 --2- 172.21.15.7:0/618051795 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f35c8077f70 0x7f35c807a430 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.103+0000 7f35efc00640 1 --2- 172.21.15.7:0/618051795 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35e815bbf0 0x7f35e8158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.103+0000 7f35efc00640 1 --2- 172.21.15.7:0/618051795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35e80ccd80 0x7f35e81f5530 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.103+0000 7f35efc00640 1 --2- 172.21.15.7:0/618051795 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35e80688e0 0x7f35e81596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.103+0000 7f35efc00640 1 -- 172.21.15.7:0/618051795 >> 172.21.15.7:0/618051795 conn(0x7f35e80b1f00 msgr2=0x7f35e80b0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:40.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.103+0000 7f35efc00640 1 -- 172.21.15.7:0/618051795 shutdown_connections 2024-09-10T21:42:40.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.103+0000 7f35efc00640 1 -- 172.21.15.7:0/618051795 wait complete. 2024-09-10T21:42:40.120 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-10T21:42:40.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:40.121 DEBUG:teuthology.orchestra.run.smithi133:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-10T21:42:40.186 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-09-10T21:42:40.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:40.187 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-10T21:42:40.234 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:42:40.235 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:42:40.270 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:42:40.270 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:42:40.345 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:42:40.345 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:42:40.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:42:40.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.620+0000 7f4ed3000640 1 -- 172.21.15.7:0/2154282575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4eb4004e00 msgr2=0x7f4eb4005280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.620+0000 7f4ed3000640 1 --2- 172.21.15.7:0/2154282575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4eb4004e00 0x7f4eb4005280 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009ed0 tx=0x7f4eb4005830 comp rx=0 tx=0).stop 2024-09-10T21:42:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.620+0000 7f4ed2600640 1 -- 172.21.15.7:0/2154282575 reap_dead start 2024-09-10T21:42:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.620+0000 7f4ed3000640 1 -- 172.21.15.7:0/2154282575 shutdown_connections 2024-09-10T21:42:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.620+0000 7f4ed3000640 1 -- 172.21.15.7:0/2154282575 >> 172.21.15.7:0/2154282575 conn(0x7f4ecc0d3d50 msgr2=0x7f4ecc0d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.620+0000 7f4ed3000640 1 -- 172.21.15.7:0/2154282575 shutdown_connections 2024-09-10T21:42:40.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.620+0000 7f4ed3000640 1 -- 172.21.15.7:0/2154282575 wait complete. 2024-09-10T21:42:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.622+0000 7f4ed3000640 1 Processor -- start 2024-09-10T21:42:40.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.622+0000 7f4ed3000640 1 -- start start 2024-09-10T21:42:40.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.623+0000 7f4ed3000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4eb4004e00 0x7f4ecc14c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.623+0000 7f4ed3000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ecc0cddf0 0x7f4ecc0c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.623+0000 7f4ed3000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ecc0c93b0 0x7f4ecc0c9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.623+0000 7f4ed3000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc0d49f0 con 0x7f4ecc0c93b0 2024-09-10T21:42:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.623+0000 7f4ed3000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4ecc0d4870 con 0x7f4ecc0cddf0 2024-09-10T21:42:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.623+0000 7f4ed3000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4ecc0d4b70 con 0x7f4eb4004e00 2024-09-10T21:42:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.623+0000 7f4ed2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ecc0c93b0 0x7f4ecc0c9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.623+0000 7f4ed2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ecc0c93b0 0x7f4ecc0c9860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44250/0 (socket says 172.21.15.7:44250) 2024-09-10T21:42:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.623+0000 7f4ed2600640 1 -- 172.21.15.7:0/3491738859 learned_addr learned my addr 172.21.15.7:0/3491738859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.623+0000 7f4ed1c00640 1 --2- 172.21.15.7:0/3491738859 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4eb4004e00 0x7f4ecc14c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.623+0000 7f4ed1200640 1 --2- 172.21.15.7:0/3491738859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ecc0cddf0 0x7f4ecc0c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.623+0000 7f4ed2600640 1 -- 172.21.15.7:0/3491738859 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4eb4004e00 msgr2=0x7f4ecc14c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.623+0000 7f4ed2600640 1 --2- 172.21.15.7:0/3491738859 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4eb4004e00 0x7f4ecc14c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:40.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.623+0000 7f4ed2600640 1 -- 172.21.15.7:0/3491738859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ecc0cddf0 msgr2=0x7f4ecc0c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.623+0000 7f4ed2600640 1 --2- 172.21.15.7:0/3491738859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ecc0cddf0 0x7f4ecc0c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.623+0000 7f4ed2600640 1 -- 172.21.15.7:0/3491738859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ecc09f480 con 0x7f4ecc0c93b0 2024-09-10T21:42:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.623+0000 7f4ed1200640 1 --2- 172.21.15.7:0/3491738859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ecc0cddf0 0x7f4ecc0c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.624+0000 7f4ed2600640 1 --2- 172.21.15.7:0/3491738859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ecc0c93b0 0x7f4ecc0c9860 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009ed0 tx=0x7f4ec4033bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:40.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.624+0000 7f4ec9600640 1 -- 172.21.15.7:0/3491738859 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4043070 con 0x7f4ecc0c93b0 2024-09-10T21:42:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.624+0000 7f4ec9600640 1 -- 172.21.15.7:0/3491738859 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ec4033ce0 con 0x7f4ecc0c93b0 2024-09-10T21:42:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.624+0000 7f4ec9600640 1 -- 172.21.15.7:0/3491738859 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec4038040 con 0x7f4ecc0c93b0 2024-09-10T21:42:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.624+0000 7f4ed3000640 1 -- 172.21.15.7:0/3491738859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ecc09d9d0 con 0x7f4ecc0c93b0 2024-09-10T21:42:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.624+0000 7f4ed3000640 1 -- 172.21.15.7:0/3491738859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ecc14d340 con 0x7f4ecc0c93b0 2024-09-10T21:42:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.625+0000 7f4ed3000640 1 -- 172.21.15.7:0/3491738859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ecc0c73d0 con 0x7f4ecc0c93b0 2024-09-10T21:42:40.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.626+0000 7f4ec9600640 1 -- 172.21.15.7:0/3491738859 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4ec403b030 con 0x7f4ecc0c93b0 2024-09-10T21:42:40.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.627+0000 7f4ec9600640 1 --2- 172.21.15.7:0/3491738859 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4eac077f70 0x7f4eac07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:40.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.627+0000 7f4ec9600640 1 -- 172.21.15.7:0/3491738859 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f4ec40b9b90 con 0x7f4ecc0c93b0 2024-09-10T21:42:40.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.627+0000 7f4ed1c00640 1 --2- 172.21.15.7:0/3491738859 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4eac077f70 0x7f4eac07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:40.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.628+0000 7f4ed1c00640 1 --2- 172.21.15.7:0/3491738859 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4eac077f70 0x7f4eac07a430 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00010e0 tx=0x7f4ec0006b10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:40.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.631+0000 7f4ec9600640 1 -- 172.21.15.7:0/3491738859 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4ecc0c73d0 con 0x7f4ecc0c93b0 2024-09-10T21:42:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.839+0000 7f4ed3000640 1 -- 172.21.15.7:0/3491738859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4ecc0b5380 con 0x7f4ecc0c93b0 2024-09-10T21:42:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.839+0000 7f4ec9600640 1 -- 172.21.15.7:0/3491738859 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f4ecc09f480 con 0x7f4ecc0c93b0 2024-09-10T21:42:40.840 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-10T21:42:40.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.841+0000 7f4ed3000640 1 -- 172.21.15.7:0/3491738859 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4eac077f70 msgr2=0x7f4eac07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:40.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.841+0000 7f4ed3000640 1 --2- 172.21.15.7:0/3491738859 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4eac077f70 0x7f4eac07a430 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4ec00010e0 tx=0x7f4ec0006b10 comp rx=0 tx=0).stop 2024-09-10T21:42:40.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.841+0000 7f4ed3000640 1 -- 172.21.15.7:0/3491738859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ecc0c93b0 msgr2=0x7f4ecc0c9860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:40.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.841+0000 7f4ed3000640 1 --2- 172.21.15.7:0/3491738859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ecc0c93b0 0x7f4ecc0c9860 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009ed0 tx=0x7f4ec4033bf0 comp rx=0 tx=0).stop 2024-09-10T21:42:40.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.842+0000 7f4ed3000640 1 -- 172.21.15.7:0/3491738859 shutdown_connections 2024-09-10T21:42:40.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.842+0000 7f4ed3000640 1 --2- 172.21.15.7:0/3491738859 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4eac077f70 0x7f4eac07a430 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:40.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.842+0000 7f4ed3000640 1 --2- 172.21.15.7:0/3491738859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ecc0cddf0 0x7f4ecc0c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:40.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.842+0000 7f4ed3000640 1 --2- 172.21.15.7:0/3491738859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ecc0c93b0 0x7f4ecc0c9860 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:40.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.842+0000 7f4ed3000640 1 --2- 172.21.15.7:0/3491738859 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4eb4004e00 0x7f4ecc14c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:40.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.842+0000 7f4ed3000640 1 -- 172.21.15.7:0/3491738859 >> 172.21.15.7:0/3491738859 conn(0x7f4ecc0d3d50 msgr2=0x7f4ecc0b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:40.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.842+0000 7f4ed3000640 1 -- 172.21.15.7:0/3491738859 shutdown_connections 2024-09-10T21:42:40.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:40.842+0000 7f4ed3000640 1 -- 172.21.15.7:0/3491738859 wait complete. 2024-09-10T21:42:40.859 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-10T21:42:40.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:40.860 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-10T21:42:40.919 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:42:40.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:40.919 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-10T21:42:40.955 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:42:40.956 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:42:40.995 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:42:40.995 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:42:41.071 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:42:41.071 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:42:41.147 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:42:41.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.308+0000 7f1330a00640 1 -- 172.21.15.7:0/3083395472 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f132c15cbe0 msgr2=0x7f132c073cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:41.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.308+0000 7f1330a00640 1 --2- 172.21.15.7:0/3083395472 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f132c15cbe0 0x7f132c073cd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1314009830 tx=0x7f131402a300 comp rx=0 tx=0).stop 2024-09-10T21:42:41.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.308+0000 7f1330a00640 1 -- 172.21.15.7:0/3083395472 shutdown_connections 2024-09-10T21:42:41.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.308+0000 7f1330a00640 1 --2- 172.21.15.7:0/3083395472 >> v2:172.21.15.133:3300/0 conn(0x7f132c074210 0x7f132c15f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.308+0000 7f1330a00640 1 --2- 172.21.15.7:0/3083395472 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f132c15cbe0 0x7f132c073cd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.308+0000 7f1330a00640 1 -- 172.21.15.7:0/3083395472 >> 172.21.15.7:0/3083395472 conn(0x7f132c0b1660 msgr2=0x7f132c0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:41.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.308+0000 7f1330a00640 1 -- 172.21.15.7:0/3083395472 shutdown_connections 2024-09-10T21:42:41.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.308+0000 7f1330a00640 1 -- 172.21.15.7:0/3083395472 wait complete. 2024-09-10T21:42:41.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.310+0000 7f1330a00640 1 Processor -- start 2024-09-10T21:42:41.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.310+0000 7f1330a00640 1 -- start start 2024-09-10T21:42:41.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.311+0000 7f1330a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f132c06ff50 0x7f132c0c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:41.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.311+0000 7f1330a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f132c074210 0x7f132c0b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:41.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.311+0000 7f1330a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f132c15cbe0 0x7f132c1f44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:41.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.311+0000 7f1330a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f132c06f8a0 con 0x7f132c074210 2024-09-10T21:42:41.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.311+0000 7f1330a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f132c06f720 con 0x7f132c06ff50 2024-09-10T21:42:41.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.311+0000 7f1330a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f132c15fc50 con 0x7f132c15cbe0 2024-09-10T21:42:41.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.311+0000 7f1321400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f132c074210 0x7f132c0b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:41.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.311+0000 7f1321400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f132c074210 0x7f132c0b3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44264/0 (socket says 172.21.15.7:44264) 2024-09-10T21:42:41.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.311+0000 7f1329e00640 1 --2- 172.21.15.7:0/1115002153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f132c06ff50 0x7f132c0c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:41.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.311+0000 7f1321400640 1 -- 172.21.15.7:0/1115002153 learned_addr learned my addr 172.21.15.7:0/1115002153 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:41.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.311+0000 7f132a800640 1 --2- 172.21.15.7:0/1115002153 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f132c15cbe0 0x7f132c1f44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:41.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.312+0000 7f1321400640 1 -- 172.21.15.7:0/1115002153 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f132c15cbe0 msgr2=0x7f132c1f44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:41.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.312+0000 7f1321400640 1 --2- 172.21.15.7:0/1115002153 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f132c15cbe0 0x7f132c1f44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.312+0000 7f1321400640 1 -- 172.21.15.7:0/1115002153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f132c06ff50 msgr2=0x7f132c0c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:41.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.312+0000 7f1321400640 1 --2- 172.21.15.7:0/1115002153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f132c06ff50 0x7f132c0c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.312+0000 7f1321400640 1 -- 172.21.15.7:0/1115002153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132c1f4a30 con 0x7f132c074210 2024-09-10T21:42:41.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.312+0000 7f1329e00640 1 --2- 172.21.15.7:0/1115002153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f132c06ff50 0x7f132c0c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:42:41.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.312+0000 7f132a800640 1 --2- 172.21.15.7:0/1115002153 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f132c15cbe0 0x7f132c1f44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:42:41.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.312+0000 7f1321400640 1 --2- 172.21.15.7:0/1115002153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f132c074210 0x7f132c0b3c80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1314009950 tx=0x7f1314033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:41.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.313+0000 7f1322a00640 1 -- 172.21.15.7:0/1115002153 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131403f020 con 0x7f132c074210 2024-09-10T21:42:41.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.313+0000 7f1322a00640 1 -- 172.21.15.7:0/1115002153 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1314033e10 con 0x7f132c074210 2024-09-10T21:42:41.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.313+0000 7f1322a00640 1 -- 172.21.15.7:0/1115002153 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314037040 con 0x7f132c074210 2024-09-10T21:42:41.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.313+0000 7f1330a00640 1 -- 172.21.15.7:0/1115002153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132c1f4cd0 con 0x7f132c074210 2024-09-10T21:42:41.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.313+0000 7f1330a00640 1 -- 172.21.15.7:0/1115002153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f132c1f5290 con 0x7f132c074210 2024-09-10T21:42:41.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.314+0000 7f1330a00640 1 -- 172.21.15.7:0/1115002153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f132c073cd0 con 0x7f132c074210 2024-09-10T21:42:41.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.315+0000 7f1322a00640 1 -- 172.21.15.7:0/1115002153 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f131403a030 con 0x7f132c074210 2024-09-10T21:42:41.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.316+0000 7f1322a00640 1 --2- 172.21.15.7:0/1115002153 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1308077f70 0x7f130807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:41.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.317+0000 7f1322a00640 1 -- 172.21.15.7:0/1115002153 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f13140b9c40 con 0x7f132c074210 2024-09-10T21:42:41.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.317+0000 7f1329e00640 1 --2- 172.21.15.7:0/1115002153 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1308077f70 0x7f130807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:41.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.318+0000 7f1329e00640 1 --2- 172.21.15.7:0/1115002153 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1308077f70 0x7f130807a430 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f132c0c5d50 tx=0x7f131c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:41.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.321+0000 7f1322a00640 1 -- 172.21.15.7:0/1115002153 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1314049030 con 0x7f132c074210 2024-09-10T21:42:41.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.526+0000 7f1330a00640 1 -- 172.21.15.7:0/1115002153 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f132c0ccab0 con 0x7f132c074210 2024-09-10T21:42:41.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.527+0000 7f1322a00640 1 -- 172.21.15.7:0/1115002153 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) ==== 80+0+0 (secure 0 0 0) 0x7f1314082050 con 0x7f132c074210 2024-09-10T21:42:41.528 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-10T21:42:41.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.530+0000 7f1330a00640 1 -- 172.21.15.7:0/1115002153 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1308077f70 msgr2=0x7f130807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:41.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.530+0000 7f1330a00640 1 --2- 172.21.15.7:0/1115002153 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1308077f70 0x7f130807a430 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f132c0c5d50 tx=0x7f131c009040 comp rx=0 tx=0).stop 2024-09-10T21:42:41.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.530+0000 7f1330a00640 1 -- 172.21.15.7:0/1115002153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f132c074210 msgr2=0x7f132c0b3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:41.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.530+0000 7f1330a00640 1 --2- 172.21.15.7:0/1115002153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f132c074210 0x7f132c0b3c80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1314009950 tx=0x7f1314033c20 comp rx=0 tx=0).stop 2024-09-10T21:42:41.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.530+0000 7f1330a00640 1 -- 172.21.15.7:0/1115002153 shutdown_connections 2024-09-10T21:42:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.530+0000 7f1330a00640 1 --2- 172.21.15.7:0/1115002153 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1308077f70 0x7f130807a430 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.530+0000 7f1330a00640 1 --2- 172.21.15.7:0/1115002153 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f132c15cbe0 0x7f132c1f44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.530+0000 7f1330a00640 1 --2- 172.21.15.7:0/1115002153 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f132c074210 0x7f132c0b3c80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.530+0000 7f1330a00640 1 --2- 172.21.15.7:0/1115002153 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f132c06ff50 0x7f132c0c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.530+0000 7f1330a00640 1 -- 172.21.15.7:0/1115002153 >> 172.21.15.7:0/1115002153 conn(0x7f132c0b1660 msgr2=0x7f132c15d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.530+0000 7f1330a00640 1 -- 172.21.15.7:0/1115002153 shutdown_connections 2024-09-10T21:42:41.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.530+0000 7f1330a00640 1 -- 172.21.15.7:0/1115002153 wait complete. 2024-09-10T21:42:41.547 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-10T21:42:41.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:41.547 DEBUG:teuthology.orchestra.run.smithi133:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-10T21:42:41.607 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi133.front.sepia.ceph.com 2024-09-10T21:42:41.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:42:41.608 DEBUG:teuthology.orchestra.run.smithi133:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-10T21:42:41.643 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:42:41.644 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:42:41.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.806+0000 7fcdf7e00640 1 -- 172.21.15.7:0/863312517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf80b21f0 msgr2=0x7fcdf80b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:41.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.806+0000 7fcdf7e00640 1 --2- 172.21.15.7:0/863312517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf80b21f0 0x7fcdf80b2660 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fcdf808c8d0 tx=0x7fcdf002a820 comp rx=0 tx=0).stop 2024-09-10T21:42:41.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.806+0000 7fcdf7e00640 1 -- 172.21.15.7:0/863312517 shutdown_connections 2024-09-10T21:42:41.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.806+0000 7fcdf7e00640 1 --2- 172.21.15.7:0/863312517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf80b21f0 0x7fcdf80b2660 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.806+0000 7fcdf7e00640 1 --2- 172.21.15.7:0/863312517 >> v2:172.21.15.133:3300/0 conn(0x7fcdf815dff0 0x7fcdf80d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.806+0000 7fcdf7e00640 1 -- 172.21.15.7:0/863312517 >> 172.21.15.7:0/863312517 conn(0x7fcdf80d3de0 msgr2=0x7fcdf80d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:41.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.806+0000 7fcdf7e00640 1 -- 172.21.15.7:0/863312517 shutdown_connections 2024-09-10T21:42:41.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.806+0000 7fcdf7e00640 1 -- 172.21.15.7:0/863312517 wait complete. 2024-09-10T21:42:41.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.808+0000 7fcdf7e00640 1 Processor -- start 2024-09-10T21:42:41.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.808+0000 7fcdf7e00640 1 -- start start 2024-09-10T21:42:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.809+0000 7fcdf7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf80b0970 0x7fcdf8156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.809+0000 7fcdf7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdf80b21f0 0x7fcdf81606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.809+0000 7fcdf7e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcdf815dff0 0x7fcdf80b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.809+0000 7fcdf7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf80cfb60 con 0x7fcdf80b0970 2024-09-10T21:42:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.809+0000 7fcdf7e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcdf80cf9e0 con 0x7fcdf80b21f0 2024-09-10T21:42:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.809+0000 7fcdf7e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fcdf80cfce0 con 0x7fcdf815dff0 2024-09-10T21:42:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.809+0000 7fcdf6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf80b0970 0x7fcdf8156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.809+0000 7fcdf6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf80b0970 0x7fcdf8156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44272/0 (socket says 172.21.15.7:44272) 2024-09-10T21:42:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.809+0000 7fcdf6a00640 1 -- 172.21.15.7:0/3880899761 learned_addr learned my addr 172.21.15.7:0/3880899761 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.809+0000 7fcdf7400640 1 --2- 172.21.15.7:0/3880899761 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcdf815dff0 0x7fcdf80b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.809+0000 7fcdee000640 1 --2- 172.21.15.7:0/3880899761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdf80b21f0 0x7fcdf81606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:41.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.809+0000 7fcdf6a00640 1 -- 172.21.15.7:0/3880899761 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcdf815dff0 msgr2=0x7fcdf80b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:41.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.809+0000 7fcdf6a00640 1 --2- 172.21.15.7:0/3880899761 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcdf815dff0 0x7fcdf80b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.809+0000 7fcdf6a00640 1 -- 172.21.15.7:0/3880899761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdf80b21f0 msgr2=0x7fcdf81606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:41.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.809+0000 7fcdf6a00640 1 --2- 172.21.15.7:0/3880899761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdf80b21f0 0x7fcdf81606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.809+0000 7fcdf6a00640 1 -- 172.21.15.7:0/3880899761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdf8157380 con 0x7fcdf80b0970 2024-09-10T21:42:41.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.810+0000 7fcdf6a00640 1 --2- 172.21.15.7:0/3880899761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf80b0970 0x7fcdf8156e40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcde800e790 tx=0x7fcde800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:41.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.810+0000 7fcdef400640 1 -- 172.21.15.7:0/3880899761 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde800eea0 con 0x7fcdf80b0970 2024-09-10T21:42:41.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.810+0000 7fcdef400640 1 -- 172.21.15.7:0/3880899761 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcde800cdb0 con 0x7fcdf80b0970 2024-09-10T21:42:41.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.810+0000 7fcdef400640 1 -- 172.21.15.7:0/3880899761 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde8010690 con 0x7fcdf80b0970 2024-09-10T21:42:41.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.810+0000 7fcdf7e00640 1 -- 172.21.15.7:0/3880899761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdf80b33b0 con 0x7fcdf80b0970 2024-09-10T21:42:41.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.810+0000 7fcdf7e00640 1 -- 172.21.15.7:0/3880899761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdf80b3930 con 0x7fcdf80b0970 2024-09-10T21:42:41.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.811+0000 7fcdf7e00640 1 -- 172.21.15.7:0/3880899761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdf80d0190 con 0x7fcdf80b0970 2024-09-10T21:42:41.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.812+0000 7fcdef400640 1 -- 172.21.15.7:0/3880899761 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fcde8010830 con 0x7fcdf80b0970 2024-09-10T21:42:41.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.813+0000 7fcdef400640 1 --2- 172.21.15.7:0/3880899761 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcdd4077f70 0x7fcdd407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:41.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.813+0000 7fcdef400640 1 -- 172.21.15.7:0/3880899761 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fcde809a7e0 con 0x7fcdf80b0970 2024-09-10T21:42:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.814+0000 7fcdee000640 1 --2- 172.21.15.7:0/3880899761 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcdd4077f70 0x7fcdd407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:41.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.814+0000 7fcdee000640 1 --2- 172.21.15.7:0/3880899761 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcdd4077f70 0x7fcdd407a430 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcdf815ba10 tx=0x7fcde0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:41.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:41.817+0000 7fcdef400640 1 -- 172.21.15.7:0/3880899761 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fcdf8157380 con 0x7fcdf80b0970 2024-09-10T21:42:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.046+0000 7fcdf7e00640 1 -- 172.21.15.7:0/3880899761 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcdf80b8910 con 0x7fcdf80b0970 2024-09-10T21:42:42.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.047+0000 7fcdef400640 1 -- 172.21.15.7:0/3880899761 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5227 (secure 0 0 0) 0x7fcde8062bf0 con 0x7fcdf80b0970 2024-09-10T21:42:42.048 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:42.048 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"btime":"2024-09-10T21:42:12:706481+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4342,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3303727910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3303727910},{"type":"v1","addr":"172.21.15.7:6835","nonce":3303727910}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4383,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3193801272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3193801272},{"type":"v1","addr":"172.21.15.7:6834","nonce":3193801272}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4401,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1546148687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1546148687},{"type":"v1","addr":"172.21.15.133:6837","nonce":1546148687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:42:09.121017+0000","modified":"2024-09-10T21:42:12.137567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234179":{"gid":234179,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6836/3769241160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3769241160},{"type":"v1","addr":"172.21.15.133:6836","nonce":3769241160}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234179,"qdb_cluster":[234179]},"id":1}]} 2024-09-10T21:42:42.049 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-10T21:42:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.049+0000 7fcdf7e00640 1 -- 172.21.15.7:0/3880899761 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcdd4077f70 msgr2=0x7fcdd407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.049+0000 7fcdf7e00640 1 --2- 172.21.15.7:0/3880899761 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcdd4077f70 0x7fcdd407a430 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fcdf815ba10 tx=0x7fcde0009040 comp rx=0 tx=0).stop 2024-09-10T21:42:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.049+0000 7fcdf7e00640 1 -- 172.21.15.7:0/3880899761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf80b0970 msgr2=0x7fcdf8156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.049+0000 7fcdf7e00640 1 --2- 172.21.15.7:0/3880899761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf80b0970 0x7fcdf8156e40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcde800e790 tx=0x7fcde800ec50 comp rx=0 tx=0).stop 2024-09-10T21:42:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.050+0000 7fcdf7e00640 1 -- 172.21.15.7:0/3880899761 shutdown_connections 2024-09-10T21:42:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.050+0000 7fcdf7e00640 1 --2- 172.21.15.7:0/3880899761 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcdd4077f70 0x7fcdd407a430 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.050+0000 7fcdf7e00640 1 --2- 172.21.15.7:0/3880899761 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcdf80b21f0 0x7fcdf81606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.050+0000 7fcdf7e00640 1 --2- 172.21.15.7:0/3880899761 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcdf815dff0 0x7fcdf80b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.050+0000 7fcdf7e00640 1 --2- 172.21.15.7:0/3880899761 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcdf80b0970 0x7fcdf8156e40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.050+0000 7fcdf7e00640 1 -- 172.21.15.7:0/3880899761 >> 172.21.15.7:0/3880899761 conn(0x7fcdf80d3de0 msgr2=0x7fcdf815ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.050+0000 7fcdf7e00640 1 -- 172.21.15.7:0/3880899761 shutdown_connections 2024-09-10T21:42:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.050+0000 7fcdf7e00640 1 -- 172.21.15.7:0/3880899761 wait complete. 2024-09-10T21:42:42.067 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:42:42.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.226+0000 7fa6f8400640 1 -- 172.21.15.7:0/3874696629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6f015fda0 msgr2=0x7fa6f0160180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:42.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.226+0000 7fa6f8400640 1 --2- 172.21.15.7:0/3874696629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6f015fda0 0x7fa6f0160180 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fa6f006dac0 tx=0x7fa6e802a980 comp rx=0 tx=0).stop 2024-09-10T21:42:42.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.226+0000 7fa6f8400640 1 -- 172.21.15.7:0/3874696629 shutdown_connections 2024-09-10T21:42:42.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.226+0000 7fa6f8400640 1 --2- 172.21.15.7:0/3874696629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6f015fda0 0x7fa6f0160180 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.226+0000 7fa6f8400640 1 --2- 172.21.15.7:0/3874696629 >> v2:172.21.15.133:3300/0 conn(0x7fa6f015bc90 0x7fa6f00ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.226+0000 7fa6f8400640 1 -- 172.21.15.7:0/3874696629 >> 172.21.15.7:0/3874696629 conn(0x7fa6f00b16d0 msgr2=0x7fa6f00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:42.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.226+0000 7fa6f8400640 1 -- 172.21.15.7:0/3874696629 shutdown_connections 2024-09-10T21:42:42.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.226+0000 7fa6f8400640 1 -- 172.21.15.7:0/3874696629 wait complete. 2024-09-10T21:42:42.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.228+0000 7fa6f8400640 1 Processor -- start 2024-09-10T21:42:42.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.228+0000 7fa6f8400640 1 -- start start 2024-09-10T21:42:42.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.229+0000 7fa6f8400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6f00cd750 0x7fa6f015d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:42.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.229+0000 7fa6f8400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6f015bc90 0x7fa6f00c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:42.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.229+0000 7fa6f8400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6f015fda0 0x7fa6f00c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:42.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.229+0000 7fa6f8400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f006fe70 con 0x7fa6f00cd750 2024-09-10T21:42:42.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.229+0000 7fa6f8400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f006fcf0 con 0x7fa6f015bc90 2024-09-10T21:42:42.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.229+0000 7fa6f8400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f006fff0 con 0x7fa6f015fda0 2024-09-10T21:42:42.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.229+0000 7fa6f5a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6f00cd750 0x7fa6f015d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:42.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.229+0000 7fa6f5a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6f00cd750 0x7fa6f015d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44282/0 (socket says 172.21.15.7:44282) 2024-09-10T21:42:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.229+0000 7fa6f5a00640 1 -- 172.21.15.7:0/1591570649 learned_addr learned my addr 172.21.15.7:0/1591570649 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.229+0000 7fa6f5000640 1 --2- 172.21.15.7:0/1591570649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6f015bc90 0x7fa6f00c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.229+0000 7fa6f6400640 1 --2- 172.21.15.7:0/1591570649 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6f015fda0 0x7fa6f00c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.230+0000 7fa6f5a00640 1 -- 172.21.15.7:0/1591570649 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6f015fda0 msgr2=0x7fa6f00c7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.230+0000 7fa6f5a00640 1 --2- 172.21.15.7:0/1591570649 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6f015fda0 0x7fa6f00c7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.230+0000 7fa6f5a00640 1 -- 172.21.15.7:0/1591570649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6f015bc90 msgr2=0x7fa6f00c73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.230+0000 7fa6f5a00640 1 --2- 172.21.15.7:0/1591570649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6f015bc90 0x7fa6f00c73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.230+0000 7fa6f5a00640 1 -- 172.21.15.7:0/1591570649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f01f44b0 con 0x7fa6f00cd750 2024-09-10T21:42:42.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.230+0000 7fa6f5a00640 1 --2- 172.21.15.7:0/1591570649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6f00cd750 0x7fa6f015d8b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400d7a0 tx=0x7fa6e400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:42.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.230+0000 7fa6ed600640 1 -- 172.21.15.7:0/1591570649 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4017070 con 0x7fa6f00cd750 2024-09-10T21:42:42.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.230+0000 7fa6ed600640 1 -- 172.21.15.7:0/1591570649 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6e4009d20 con 0x7fa6f00cd750 2024-09-10T21:42:42.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.230+0000 7fa6f8400640 1 -- 172.21.15.7:0/1591570649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f015e390 con 0x7fa6f00cd750 2024-09-10T21:42:42.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.230+0000 7fa6ed600640 1 -- 172.21.15.7:0/1591570649 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e4005280 con 0x7fa6f00cd750 2024-09-10T21:42:42.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.230+0000 7fa6f8400640 1 -- 172.21.15.7:0/1591570649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f015e950 con 0x7fa6f00cd750 2024-09-10T21:42:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.231+0000 7fa6f8400640 1 -- 172.21.15.7:0/1591570649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f00739b0 con 0x7fa6f00cd750 2024-09-10T21:42:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.233+0000 7fa6ed600640 1 -- 172.21.15.7:0/1591570649 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa6e4005420 con 0x7fa6f00cd750 2024-09-10T21:42:42.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.233+0000 7fa6ed600640 1 --2- 172.21.15.7:0/1591570649 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa6d0077f70 0x7fa6d007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.234+0000 7fa6ed600640 1 -- 172.21.15.7:0/1591570649 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fa6e4099ea0 con 0x7fa6f00cd750 2024-09-10T21:42:42.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.234+0000 7fa6f5000640 1 --2- 172.21.15.7:0/1591570649 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa6d0077f70 0x7fa6d007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:42.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.235+0000 7fa6f5000640 1 --2- 172.21.15.7:0/1591570649 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa6d0077f70 0x7fa6d007a430 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0073040 tx=0x7fa6dc007e30 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:42.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.238+0000 7fa6ed600640 1 -- 172.21.15.7:0/1591570649 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fa6e4062340 con 0x7fa6f00cd750 2024-09-10T21:42:42.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.469+0000 7fa6f8400640 1 -- 172.21.15.7:0/1591570649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6f0070510 con 0x7fa6f00cd750 2024-09-10T21:42:42.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.470+0000 7fa6ed600640 1 -- 172.21.15.7:0/1591570649 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5227 (secure 0 0 0) 0x7fa6e406b0b0 con 0x7fa6f00cd750 2024-09-10T21:42:42.471 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:42.471 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"btime":"2024-09-10T21:42:12:706481+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4342,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3303727910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3303727910},{"type":"v1","addr":"172.21.15.7:6835","nonce":3303727910}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4383,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3193801272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3193801272},{"type":"v1","addr":"172.21.15.7:6834","nonce":3193801272}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4401,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1546148687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1546148687},{"type":"v1","addr":"172.21.15.133:6837","nonce":1546148687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:42:09.121017+0000","modified":"2024-09-10T21:42:12.137567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234179":{"gid":234179,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6836/3769241160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3769241160},{"type":"v1","addr":"172.21.15.133:6836","nonce":3769241160}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234179,"qdb_cluster":[234179]},"id":1}]} 2024-09-10T21:42:42.471 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-10T21:42:42.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.472+0000 7fa6f8400640 1 -- 172.21.15.7:0/1591570649 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa6d0077f70 msgr2=0x7fa6d007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:42.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.472+0000 7fa6f8400640 1 --2- 172.21.15.7:0/1591570649 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa6d0077f70 0x7fa6d007a430 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa6f0073040 tx=0x7fa6dc007e30 comp rx=0 tx=0).stop 2024-09-10T21:42:42.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.472+0000 7fa6f8400640 1 -- 172.21.15.7:0/1591570649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6f00cd750 msgr2=0x7fa6f015d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:42.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.472+0000 7fa6f8400640 1 --2- 172.21.15.7:0/1591570649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6f00cd750 0x7fa6f015d8b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400d7a0 tx=0x7fa6e400dc60 comp rx=0 tx=0).stop 2024-09-10T21:42:42.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.472+0000 7fa6f8400640 1 -- 172.21.15.7:0/1591570649 shutdown_connections 2024-09-10T21:42:42.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.472+0000 7fa6f8400640 1 --2- 172.21.15.7:0/1591570649 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa6d0077f70 0x7fa6d007a430 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.472+0000 7fa6f8400640 1 --2- 172.21.15.7:0/1591570649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa6f015bc90 0x7fa6f00c73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.472+0000 7fa6f8400640 1 --2- 172.21.15.7:0/1591570649 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa6f015fda0 0x7fa6f00c7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.472+0000 7fa6f8400640 1 --2- 172.21.15.7:0/1591570649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa6f00cd750 0x7fa6f015d8b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.472+0000 7fa6f8400640 1 -- 172.21.15.7:0/1591570649 >> 172.21.15.7:0/1591570649 conn(0x7fa6f00b16d0 msgr2=0x7fa6f00b2a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:42.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.472+0000 7fa6f8400640 1 -- 172.21.15.7:0/1591570649 shutdown_connections 2024-09-10T21:42:42.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.472+0000 7fa6f8400640 1 -- 172.21.15.7:0/1591570649 wait complete. 2024-09-10T21:42:42.489 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:42:42.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.648+0000 7f45c4a00640 1 -- 172.21.15.7:0/2765975525 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45c01619e0 msgr2=0x7f45c015e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:42.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.648+0000 7f45c4a00640 1 --2- 172.21.15.7:0/2765975525 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45c01619e0 0x7f45c015e3c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009830 tx=0x7f45b002a520 comp rx=0 tx=0).stop 2024-09-10T21:42:42.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.649+0000 7f45c4a00640 1 -- 172.21.15.7:0/2765975525 shutdown_connections 2024-09-10T21:42:42.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.649+0000 7f45c4a00640 1 --2- 172.21.15.7:0/2765975525 >> v2:172.21.15.133:3300/0 conn(0x7f45c00cbd50 0x7f45c00cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.649+0000 7f45c4a00640 1 --2- 172.21.15.7:0/2765975525 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45c01619e0 0x7f45c015e3c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.649+0000 7f45c4a00640 1 -- 172.21.15.7:0/2765975525 >> 172.21.15.7:0/2765975525 conn(0x7f45c00b16d0 msgr2=0x7f45c00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:42.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.649+0000 7f45c4a00640 1 -- 172.21.15.7:0/2765975525 shutdown_connections 2024-09-10T21:42:42.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.649+0000 7f45c4a00640 1 -- 172.21.15.7:0/2765975525 wait complete. 2024-09-10T21:42:42.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.650+0000 7f45c4a00640 1 Processor -- start 2024-09-10T21:42:42.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.650+0000 7f45c4a00640 1 -- start start 2024-09-10T21:42:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.651+0000 7f45c4a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c00ae020 0x7f45c0152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.651+0000 7f45c4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45c00cbd50 0x7f45c00c7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.651+0000 7f45c4a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45c01619e0 0x7f45c006f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.651+0000 7f45c4a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f45c00ae950 con 0x7f45c00cbd50 2024-09-10T21:42:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.651+0000 7f45c4a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f45c00ae7d0 con 0x7f45c00ae020 2024-09-10T21:42:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.651+0000 7f45c4a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f45c0161dc0 con 0x7f45c01619e0 2024-09-10T21:42:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.652+0000 7f45bd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45c00cbd50 0x7f45c00c7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.652+0000 7f45bd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45c00cbd50 0x7f45c00c7ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44290/0 (socket says 172.21.15.7:44290) 2024-09-10T21:42:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.652+0000 7f45bd400640 1 -- 172.21.15.7:0/3448048476 learned_addr learned my addr 172.21.15.7:0/3448048476 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:42.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.652+0000 7f45be800640 1 --2- 172.21.15.7:0/3448048476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45c01619e0 0x7f45c006f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:42.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.652+0000 7f45bde00640 1 --2- 172.21.15.7:0/3448048476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c00ae020 0x7f45c0152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.652+0000 7f45bd400640 1 -- 172.21.15.7:0/3448048476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45c01619e0 msgr2=0x7f45c006f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.652+0000 7f45bd400640 1 --2- 172.21.15.7:0/3448048476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45c01619e0 0x7f45c006f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.652+0000 7f45bd400640 1 -- 172.21.15.7:0/3448048476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c00ae020 msgr2=0x7f45c0152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.652+0000 7f45bd400640 1 --2- 172.21.15.7:0/3448048476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c00ae020 0x7f45c0152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.652+0000 7f45bd400640 1 -- 172.21.15.7:0/3448048476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45c00b2a70 con 0x7f45c00cbd50 2024-09-10T21:42:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.652+0000 7f45bd400640 1 --2- 172.21.15.7:0/3448048476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45c00cbd50 0x7f45c00c7ee0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00e9c0 tx=0x7f45ac00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.653+0000 7f45b6000640 1 -- 172.21.15.7:0/3448048476 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac00cd80 con 0x7f45c00cbd50 2024-09-10T21:42:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.653+0000 7f45b6000640 1 -- 172.21.15.7:0/3448048476 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45ac004020 con 0x7f45c00cbd50 2024-09-10T21:42:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.653+0000 7f45b6000640 1 -- 172.21.15.7:0/3448048476 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac004450 con 0x7f45c00cbd50 2024-09-10T21:42:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.653+0000 7f45c4a00640 1 -- 172.21.15.7:0/3448048476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45c00b2d10 con 0x7f45c00cbd50 2024-09-10T21:42:42.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.653+0000 7f45c4a00640 1 -- 172.21.15.7:0/3448048476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45c00b32d0 con 0x7f45c00cbd50 2024-09-10T21:42:42.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.654+0000 7f45c4a00640 1 -- 172.21.15.7:0/3448048476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45c00cc190 con 0x7f45c00cbd50 2024-09-10T21:42:42.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.655+0000 7f45b6000640 1 -- 172.21.15.7:0/3448048476 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f45ac0035f0 con 0x7f45c00cbd50 2024-09-10T21:42:42.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.656+0000 7f45b6000640 1 --2- 172.21.15.7:0/3448048476 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f45a0077f70 0x7f45a007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:42.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.656+0000 7f45b6000640 1 -- 172.21.15.7:0/3448048476 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f45ac09a690 con 0x7f45c00cbd50 2024-09-10T21:42:42.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.656+0000 7f45bde00640 1 --2- 172.21.15.7:0/3448048476 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f45a0077f70 0x7f45a007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:42.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.657+0000 7f45bde00640 1 --2- 172.21.15.7:0/3448048476 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f45a0077f70 0x7f45a007a430 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f45b0034000 tx=0x7f45b0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:42.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.660+0000 7f45b6000640 1 -- 172.21.15.7:0/3448048476 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f45ac062b30 con 0x7f45c00cbd50 2024-09-10T21:42:42.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.858+0000 7f45c4a00640 1 -- 172.21.15.7:0/3448048476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45c0072c60 con 0x7f45c00cbd50 2024-09-10T21:42:42.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.859+0000 7f45b6000640 1 -- 172.21.15.7:0/3448048476 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16629 (secure 0 0 0) 0x7f45ac067560 con 0x7f45c00cbd50 2024-09-10T21:42:42.861 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:42.861 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:42:10.684978+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:42:07.149267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:42:08.077083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:42:42.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.862+0000 7f45c4a00640 1 -- 172.21.15.7:0/3448048476 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f45a0077f70 msgr2=0x7f45a007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:42.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.862+0000 7f45c4a00640 1 --2- 172.21.15.7:0/3448048476 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f45a0077f70 0x7f45a007a430 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f45b0034000 tx=0x7f45b0004730 comp rx=0 tx=0).stop 2024-09-10T21:42:42.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.862+0000 7f45c4a00640 1 -- 172.21.15.7:0/3448048476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45c00cbd50 msgr2=0x7f45c00c7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:42.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.862+0000 7f45c4a00640 1 --2- 172.21.15.7:0/3448048476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45c00cbd50 0x7f45c00c7ee0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f45ac00e9c0 tx=0x7f45ac00ee80 comp rx=0 tx=0).stop 2024-09-10T21:42:42.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.862+0000 7f45c4a00640 1 -- 172.21.15.7:0/3448048476 shutdown_connections 2024-09-10T21:42:42.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.862+0000 7f45c4a00640 1 --2- 172.21.15.7:0/3448048476 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f45a0077f70 0x7f45a007a430 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.862+0000 7f45c4a00640 1 --2- 172.21.15.7:0/3448048476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45c01619e0 0x7f45c006f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.862+0000 7f45c4a00640 1 --2- 172.21.15.7:0/3448048476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45c00cbd50 0x7f45c00c7ee0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.862+0000 7f45c4a00640 1 --2- 172.21.15.7:0/3448048476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45c00ae020 0x7f45c0152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:42.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.862+0000 7f45c4a00640 1 -- 172.21.15.7:0/3448048476 >> 172.21.15.7:0/3448048476 conn(0x7f45c00b16d0 msgr2=0x7f45c00afc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:42.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.862+0000 7f45c4a00640 1 -- 172.21.15.7:0/3448048476 shutdown_connections 2024-09-10T21:42:42.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:42.862+0000 7f45c4a00640 1 -- 172.21.15.7:0/3448048476 wait complete. 2024-09-10T21:42:42.880 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:42:42.880 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:42:43.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.039+0000 7f825dc00640 1 -- 172.21.15.7:0/409859768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825815dfd0 msgr2=0x7f82580d4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:43.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.039+0000 7f825dc00640 1 --2- 172.21.15.7:0/409859768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825815dfd0 0x7f82580d4870 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f8248005fd0 tx=0x7f824802a540 comp rx=0 tx=0).stop 2024-09-10T21:42:43.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.040+0000 7f825dc00640 1 -- 172.21.15.7:0/409859768 shutdown_connections 2024-09-10T21:42:43.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.040+0000 7f825dc00640 1 --2- 172.21.15.7:0/409859768 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825815dfd0 0x7f82580d4870 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:43.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.040+0000 7f825dc00640 1 -- 172.21.15.7:0/409859768 >> 172.21.15.7:0/409859768 conn(0x7f82580d3d50 msgr2=0x7f82580d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:43.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.040+0000 7f825dc00640 1 -- 172.21.15.7:0/409859768 shutdown_connections 2024-09-10T21:42:43.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.040+0000 7f825dc00640 1 -- 172.21.15.7:0/409859768 wait complete. 2024-09-10T21:42:43.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.041+0000 7f825dc00640 1 Processor -- start 2024-09-10T21:42:43.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.042+0000 7f825dc00640 1 -- start start 2024-09-10T21:42:43.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.042+0000 7f825dc00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f82580b2b50 0x7f8258152eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:43.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.042+0000 7f825dc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82580d0e70 0x7f82581f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:43.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.042+0000 7f825dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825815dfd0 0x7f82580c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:43.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.042+0000 7f825dc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82580d2770 con 0x7f825815dfd0 2024-09-10T21:42:43.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.042+0000 7f825dc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f82580d25f0 con 0x7f82580d0e70 2024-09-10T21:42:43.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.042+0000 7f825dc00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f82580d28f0 con 0x7f82580b2b50 2024-09-10T21:42:43.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.043+0000 7f825d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825815dfd0 0x7f82580c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:43.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.043+0000 7f825d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825815dfd0 0x7f82580c9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44302/0 (socket says 172.21.15.7:44302) 2024-09-10T21:42:43.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.043+0000 7f8257e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f82580b2b50 0x7f8258152eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:43.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.043+0000 7f825d200640 1 -- 172.21.15.7:0/1546922101 learned_addr learned my addr 172.21.15.7:0/1546922101 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:43.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.043+0000 7f824fe00640 1 --2- 172.21.15.7:0/1546922101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82580d0e70 0x7f82581f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.043+0000 7f825d200640 1 -- 172.21.15.7:0/1546922101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f82580b2b50 msgr2=0x7f8258152eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.043+0000 7f825d200640 1 --2- 172.21.15.7:0/1546922101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f82580b2b50 0x7f8258152eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.043+0000 7f825d200640 1 -- 172.21.15.7:0/1546922101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82580d0e70 msgr2=0x7f82581f8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.043+0000 7f825d200640 1 --2- 172.21.15.7:0/1546922101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82580d0e70 0x7f82581f8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.043+0000 7f825d200640 1 -- 172.21.15.7:0/1546922101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82580cde50 con 0x7f825815dfd0 2024-09-10T21:42:43.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.044+0000 7f825d200640 1 --2- 172.21.15.7:0/1546922101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825815dfd0 0x7f82580c9110 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f825000c7a0 tx=0x7f825000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.044+0000 7f8254c00640 1 -- 172.21.15.7:0/1546922101 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8250003b70 con 0x7f825815dfd0 2024-09-10T21:42:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.044+0000 7f8254c00640 1 -- 172.21.15.7:0/1546922101 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8250003d30 con 0x7f825815dfd0 2024-09-10T21:42:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.044+0000 7f825dc00640 1 -- 172.21.15.7:0/1546922101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82580b8070 con 0x7f825815dfd0 2024-09-10T21:42:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.044+0000 7f8254c00640 1 -- 172.21.15.7:0/1546922101 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825000aab0 con 0x7f825815dfd0 2024-09-10T21:42:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.044+0000 7f825dc00640 1 -- 172.21.15.7:0/1546922101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82580b8630 con 0x7f825815dfd0 2024-09-10T21:42:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.045+0000 7f825dc00640 1 -- 172.21.15.7:0/1546922101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82580d2d50 con 0x7f825815dfd0 2024-09-10T21:42:43.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.046+0000 7f8254c00640 1 -- 172.21.15.7:0/1546922101 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f825000ac50 con 0x7f825815dfd0 2024-09-10T21:42:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.047+0000 7f8254c00640 1 --2- 172.21.15.7:0/1546922101 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8234077f70 0x7f823407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:43.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.047+0000 7f8254c00640 1 -- 172.21.15.7:0/1546922101 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f8250099a60 con 0x7f825815dfd0 2024-09-10T21:42:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.048+0000 7f8257e00640 1 --2- 172.21.15.7:0/1546922101 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8234077f70 0x7f823407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:43.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.048+0000 7f8257e00640 1 --2- 172.21.15.7:0/1546922101 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8234077f70 0x7f823407a430 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8248033000 tx=0x7f8248004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:43.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.052+0000 7f8254c00640 1 -- 172.21.15.7:0/1546922101 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8250061f00 con 0x7f825815dfd0 2024-09-10T21:42:43.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.282+0000 7f825dc00640 1 -- 172.21.15.7:0/1546922101 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82580b9cd0 con 0x7f825815dfd0 2024-09-10T21:42:43.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.282+0000 7f8254c00640 1 -- 172.21.15.7:0/1546922101 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5227 (secure 0 0 0) 0x7f8250066930 con 0x7f825815dfd0 2024-09-10T21:42:43.284 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:43.284 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"btime":"2024-09-10T21:42:12:706481+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4342,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3303727910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3303727910},{"type":"v1","addr":"172.21.15.7:6835","nonce":3303727910}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4383,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3193801272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3193801272},{"type":"v1","addr":"172.21.15.7:6834","nonce":3193801272}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4401,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1546148687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1546148687},{"type":"v1","addr":"172.21.15.133:6837","nonce":1546148687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:42:09.121017+0000","modified":"2024-09-10T21:42:12.137567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234179":{"gid":234179,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6836/3769241160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3769241160},{"type":"v1","addr":"172.21.15.133:6836","nonce":3769241160}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234179,"qdb_cluster":[234179]},"id":1}]} 2024-09-10T21:42:43.284 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-10T21:42:43.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.285+0000 7f825dc00640 1 -- 172.21.15.7:0/1546922101 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8234077f70 msgr2=0x7f823407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:43.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.285+0000 7f825dc00640 1 --2- 172.21.15.7:0/1546922101 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8234077f70 0x7f823407a430 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8248033000 tx=0x7f8248004730 comp rx=0 tx=0).stop 2024-09-10T21:42:43.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.285+0000 7f825dc00640 1 -- 172.21.15.7:0/1546922101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825815dfd0 msgr2=0x7f82580c9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:43.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.285+0000 7f825dc00640 1 --2- 172.21.15.7:0/1546922101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825815dfd0 0x7f82580c9110 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f825000c7a0 tx=0x7f825000cc60 comp rx=0 tx=0).stop 2024-09-10T21:42:43.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.285+0000 7f825dc00640 1 -- 172.21.15.7:0/1546922101 shutdown_connections 2024-09-10T21:42:43.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.285+0000 7f825dc00640 1 --2- 172.21.15.7:0/1546922101 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8234077f70 0x7f823407a430 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:43.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.285+0000 7f825dc00640 1 --2- 172.21.15.7:0/1546922101 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f82580d0e70 0x7f82581f8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:43.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.285+0000 7f825dc00640 1 --2- 172.21.15.7:0/1546922101 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f825815dfd0 0x7f82580c9110 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:43.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.285+0000 7f825dc00640 1 --2- 172.21.15.7:0/1546922101 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f82580b2b50 0x7f8258152eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:43.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.285+0000 7f825dc00640 1 -- 172.21.15.7:0/1546922101 >> 172.21.15.7:0/1546922101 conn(0x7f82580d3d50 msgr2=0x7f82580d03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:43.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.285+0000 7f825dc00640 1 -- 172.21.15.7:0/1546922101 shutdown_connections 2024-09-10T21:42:43.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.285+0000 7f825dc00640 1 -- 172.21.15.7:0/1546922101 wait complete. 2024-09-10T21:42:43.302 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:42:43.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.460+0000 7fb551800640 1 -- 172.21.15.7:0/4107453021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb54c0cbd50 msgr2=0x7fb54c0cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:43.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.460+0000 7fb551800640 1 --2- 172.21.15.7:0/4107453021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb54c0cbd50 0x7fb54c0cc1c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb534005fd0 tx=0x7fb53402a5d0 comp rx=0 tx=0).stop 2024-09-10T21:42:43.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.461+0000 7fb551800640 1 -- 172.21.15.7:0/4107453021 shutdown_connections 2024-09-10T21:42:43.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.461+0000 7fb551800640 1 --2- 172.21.15.7:0/4107453021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb54c0cbd50 0x7fb54c0cc1c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:43.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.461+0000 7fb551800640 1 -- 172.21.15.7:0/4107453021 >> 172.21.15.7:0/4107453021 conn(0x7fb54c0b16d0 msgr2=0x7fb54c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:43.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.461+0000 7fb551800640 1 -- 172.21.15.7:0/4107453021 shutdown_connections 2024-09-10T21:42:43.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.461+0000 7fb551800640 1 -- 172.21.15.7:0/4107453021 wait complete. 2024-09-10T21:42:43.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.462+0000 7fb551800640 1 Processor -- start 2024-09-10T21:42:43.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.462+0000 7fb551800640 1 -- start start 2024-09-10T21:42:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.463+0000 7fb551800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb54c0ae020 0x7fb54c151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.463+0000 7fb551800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb54c0cbd50 0x7fb54c0733b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.463+0000 7fb551800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb54c1619e0 0x7fb54c0738f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.463+0000 7fb551800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c161f40 con 0x7fb54c0cbd50 2024-09-10T21:42:43.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.463+0000 7fb551800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb54c161dc0 con 0x7fb54c0ae020 2024-09-10T21:42:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.463+0000 7fb551800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fb54c1620c0 con 0x7fb54c1619e0 2024-09-10T21:42:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.463+0000 7fb542000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb54c0cbd50 0x7fb54c0733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.464+0000 7fb542000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb54c0cbd50 0x7fb54c0733b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44324/0 (socket says 172.21.15.7:44324) 2024-09-10T21:42:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.464+0000 7fb542000640 1 -- 172.21.15.7:0/2671431930 learned_addr learned my addr 172.21.15.7:0/2671431930 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.464+0000 7fb54b400640 1 --2- 172.21.15.7:0/2671431930 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb54c1619e0 0x7fb54c0738f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.464+0000 7fb54aa00640 1 --2- 172.21.15.7:0/2671431930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb54c0ae020 0x7fb54c151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.464+0000 7fb542000640 1 -- 172.21.15.7:0/2671431930 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb54c1619e0 msgr2=0x7fb54c0738f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.464+0000 7fb542000640 1 --2- 172.21.15.7:0/2671431930 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb54c1619e0 0x7fb54c0738f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.464+0000 7fb542000640 1 -- 172.21.15.7:0/2671431930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb54c0ae020 msgr2=0x7fb54c151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.464+0000 7fb542000640 1 --2- 172.21.15.7:0/2671431930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb54c0ae020 0x7fb54c151fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.464+0000 7fb542000640 1 -- 172.21.15.7:0/2671431930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb54c06f2b0 con 0x7fb54c0cbd50 2024-09-10T21:42:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.464+0000 7fb54aa00640 1 --2- 172.21.15.7:0/2671431930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb54c0ae020 0x7fb54c151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:43.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.464+0000 7fb542000640 1 --2- 172.21.15.7:0/2671431930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb54c0cbd50 0x7fb54c0733b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb534009850 tx=0x7fb5340038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.464+0000 7fb543400640 1 -- 172.21.15.7:0/2671431930 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb53403f070 con 0x7fb54c0cbd50 2024-09-10T21:42:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.465+0000 7fb543400640 1 -- 172.21.15.7:0/2671431930 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5340047a0 con 0x7fb54c0cbd50 2024-09-10T21:42:43.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.465+0000 7fb551800640 1 -- 172.21.15.7:0/2671431930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb54c0c6dd0 con 0x7fb54c0cbd50 2024-09-10T21:42:43.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.465+0000 7fb543400640 1 -- 172.21.15.7:0/2671431930 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb534035030 con 0x7fb54c0cbd50 2024-09-10T21:42:43.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.465+0000 7fb551800640 1 -- 172.21.15.7:0/2671431930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb54c0c7390 con 0x7fb54c0cbd50 2024-09-10T21:42:43.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.466+0000 7fb551800640 1 -- 172.21.15.7:0/2671431930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb54c162560 con 0x7fb54c0cbd50 2024-09-10T21:42:43.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.467+0000 7fb543400640 1 -- 172.21.15.7:0/2671431930 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb534004b90 con 0x7fb54c0cbd50 2024-09-10T21:42:43.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.468+0000 7fb543400640 1 --2- 172.21.15.7:0/2671431930 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb528077f70 0x7fb52807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:43.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.468+0000 7fb543400640 1 -- 172.21.15.7:0/2671431930 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7fb5340badb0 con 0x7fb54c0cbd50 2024-09-10T21:42:43.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.469+0000 7fb54aa00640 1 --2- 172.21.15.7:0/2671431930 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb528077f70 0x7fb52807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:43.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.469+0000 7fb54aa00640 1 --2- 172.21.15.7:0/2671431930 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb528077f70 0x7fb52807a430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb53c007fd0 tx=0x7fb53c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:43.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.472+0000 7fb543400640 1 -- 172.21.15.7:0/2671431930 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb5340bf070 con 0x7fb54c0cbd50 2024-09-10T21:42:43.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.705+0000 7fb551800640 1 -- 172.21.15.7:0/2671431930 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb54c0cc450 con 0x7fb54c0cbd50 2024-09-10T21:42:43.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.705+0000 7fb543400640 1 -- 172.21.15.7:0/2671431930 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5227 (secure 0 0 0) 0x7fb534083250 con 0x7fb54c0cbd50 2024-09-10T21:42:43.707 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:43.707 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":8,"btime":"2024-09-10T21:42:12:706481+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4342,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3303727910","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3303727910},{"type":"v1","addr":"172.21.15.7:6835","nonce":3303727910}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4383,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3193801272","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3193801272},{"type":"v1","addr":"172.21.15.7:6834","nonce":3193801272}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4401,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1546148687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1546148687},{"type":"v1","addr":"172.21.15.133:6837","nonce":1546148687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:42:09.121017+0000","modified":"2024-09-10T21:42:12.137567+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234179},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234179":{"gid":234179,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6836/3769241160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3769241160},{"type":"v1","addr":"172.21.15.133:6836","nonce":3769241160}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234179,"qdb_cluster":[234179]},"id":1}]} 2024-09-10T21:42:43.707 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 8 2024-09-10T21:42:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.707+0000 7fb551800640 1 -- 172.21.15.7:0/2671431930 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb528077f70 msgr2=0x7fb52807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.707+0000 7fb551800640 1 --2- 172.21.15.7:0/2671431930 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb528077f70 0x7fb52807a430 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb53c007fd0 tx=0x7fb53c009040 comp rx=0 tx=0).stop 2024-09-10T21:42:43.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.707+0000 7fb551800640 1 -- 172.21.15.7:0/2671431930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb54c0cbd50 msgr2=0x7fb54c0733b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.708+0000 7fb551800640 1 --2- 172.21.15.7:0/2671431930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb54c0cbd50 0x7fb54c0733b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb534009850 tx=0x7fb5340038a0 comp rx=0 tx=0).stop 2024-09-10T21:42:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.708+0000 7fb551800640 1 -- 172.21.15.7:0/2671431930 shutdown_connections 2024-09-10T21:42:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.708+0000 7fb551800640 1 --2- 172.21.15.7:0/2671431930 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb528077f70 0x7fb52807a430 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.708+0000 7fb551800640 1 --2- 172.21.15.7:0/2671431930 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb54c1619e0 0x7fb54c0738f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.708+0000 7fb551800640 1 --2- 172.21.15.7:0/2671431930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb54c0cbd50 0x7fb54c0733b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.708+0000 7fb551800640 1 --2- 172.21.15.7:0/2671431930 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb54c0ae020 0x7fb54c151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.708+0000 7fb551800640 1 -- 172.21.15.7:0/2671431930 >> 172.21.15.7:0/2671431930 conn(0x7fb54c0b16d0 msgr2=0x7fb54c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.708+0000 7fb551800640 1 -- 172.21.15.7:0/2671431930 shutdown_connections 2024-09-10T21:42:43.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.708+0000 7fb551800640 1 -- 172.21.15.7:0/2671431930 wait complete. 2024-09-10T21:42:43.724 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:42:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.886+0000 7f0f25000640 1 -- 172.21.15.7:0/2429218699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f040040c0 msgr2=0x7f0f04004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.886+0000 7f0f25000640 1 --2- 172.21.15.7:0/2429218699 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f040040c0 0x7f0f04004800 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0f08006f90 tx=0x7f0f04005830 comp rx=0 tx=0).stop 2024-09-10T21:42:43.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.886+0000 7f0f1fe00640 1 -- 172.21.15.7:0/2429218699 reap_dead start 2024-09-10T21:42:43.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.886+0000 7f0f25000640 1 -- 172.21.15.7:0/2429218699 shutdown_connections 2024-09-10T21:42:43.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.886+0000 7f0f25000640 1 -- 172.21.15.7:0/2429218699 >> 172.21.15.7:0/2429218699 conn(0x7f0f200d3d50 msgr2=0x7f0f200d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:43.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.886+0000 7f0f25000640 1 -- 172.21.15.7:0/2429218699 shutdown_connections 2024-09-10T21:42:43.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.886+0000 7f0f25000640 1 -- 172.21.15.7:0/2429218699 wait complete. 2024-09-10T21:42:43.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.887+0000 7f0f25000640 1 Processor -- start 2024-09-10T21:42:43.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.887+0000 7f0f25000640 1 -- start start 2024-09-10T21:42:43.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.888+0000 7f0f25000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f040040c0 0x7f0f200cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:43.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.888+0000 7f0f25000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f201f7e80 0x7f0f2014a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:43.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.888+0000 7f0f25000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0f20149860 0x7f0f20149ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:43.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.888+0000 7f0f25000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0f200d49f0 con 0x7f0f040040c0 2024-09-10T21:42:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.888+0000 7f0f25000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0f200d4870 con 0x7f0f201f7e80 2024-09-10T21:42:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.888+0000 7f0f25000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f0f200d4b70 con 0x7f0f20149860 2024-09-10T21:42:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.889+0000 7f0f1f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f040040c0 0x7f0f200cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.889+0000 7f0f1f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f040040c0 0x7f0f200cb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54808/0 (socket says 172.21.15.7:54808) 2024-09-10T21:42:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.889+0000 7f0f1f400640 1 -- 172.21.15.7:0/546264971 learned_addr learned my addr 172.21.15.7:0/546264971 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.889+0000 7f0f16a00640 1 --2- 172.21.15.7:0/546264971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f201f7e80 0x7f0f2014a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.889+0000 7f0f1fe00640 1 --2- 172.21.15.7:0/546264971 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0f20149860 0x7f0f20149ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.889+0000 7f0f1f400640 1 -- 172.21.15.7:0/546264971 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0f20149860 msgr2=0x7f0f20149ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.889+0000 7f0f1f400640 1 --2- 172.21.15.7:0/546264971 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0f20149860 0x7f0f20149ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:43.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.889+0000 7f0f1f400640 1 -- 172.21.15.7:0/546264971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f201f7e80 msgr2=0x7f0f2014a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.889+0000 7f0f1f400640 1 --2- 172.21.15.7:0/546264971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f201f7e80 0x7f0f2014a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.889+0000 7f0f1f400640 1 -- 172.21.15.7:0/546264971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f201f85e0 con 0x7f0f040040c0 2024-09-10T21:42:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.889+0000 7f0f16a00640 1 --2- 172.21.15.7:0/546264971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f201f7e80 0x7f0f2014a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.889+0000 7f0f1f400640 1 --2- 172.21.15.7:0/546264971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f040040c0 0x7f0f200cb2c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0f10010a70 tx=0x7f0f10010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.890+0000 7f0f17e00640 1 -- 172.21.15.7:0/546264971 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1000eda0 con 0x7f0f040040c0 2024-09-10T21:42:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.890+0000 7f0f17e00640 1 -- 172.21.15.7:0/546264971 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f1001c480 con 0x7f0f040040c0 2024-09-10T21:42:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.890+0000 7f0f17e00640 1 -- 172.21.15.7:0/546264971 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f1001c8d0 con 0x7f0f040040c0 2024-09-10T21:42:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.890+0000 7f0f25000640 1 -- 172.21.15.7:0/546264971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f201f88c0 con 0x7f0f040040c0 2024-09-10T21:42:43.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.890+0000 7f0f25000640 1 -- 172.21.15.7:0/546264971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f200ccbb0 con 0x7f0f040040c0 2024-09-10T21:42:43.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.891+0000 7f0f25000640 1 -- 172.21.15.7:0/546264971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f200d0760 con 0x7f0f040040c0 2024-09-10T21:42:43.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.892+0000 7f0f17e00640 1 -- 172.21.15.7:0/546264971 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0f10024030 con 0x7f0f040040c0 2024-09-10T21:42:43.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.893+0000 7f0f17e00640 1 --2- 172.21.15.7:0/546264971 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0efc077fa0 0x7f0efc07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:43.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.893+0000 7f0f17e00640 1 -- 172.21.15.7:0/546264971 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f0f1009aa80 con 0x7f0f040040c0 2024-09-10T21:42:43.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.893+0000 7f0f16a00640 1 --2- 172.21.15.7:0/546264971 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0efc077fa0 0x7f0efc07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:43.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.894+0000 7f0f16a00640 1 --2- 172.21.15.7:0/546264971 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0efc077fa0 0x7f0efc07a460 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0f08033000 tx=0x7f0f200cfd50 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:43.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:43.897+0000 7f0f17e00640 1 -- 172.21.15.7:0/546264971 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0f1009ce20 con 0x7f0f040040c0 2024-09-10T21:42:44.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.094+0000 7f0f25000640 1 -- 172.21.15.7:0/546264971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f200b9cd0 con 0x7f0f040040c0 2024-09-10T21:42:44.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.095+0000 7f0f17e00640 1 -- 172.21.15.7:0/546264971 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) ==== 74+0+16629 (secure 0 0 0) 0x7f0f10063570 con 0x7f0f040040c0 2024-09-10T21:42:44.097 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:44.097 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":21,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:42:10.684978+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:42:07.149267+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:42:08.077083+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:42:44.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.098+0000 7f0f25000640 1 -- 172.21.15.7:0/546264971 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0efc077fa0 msgr2=0x7f0efc07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:44.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.098+0000 7f0f25000640 1 --2- 172.21.15.7:0/546264971 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0efc077fa0 0x7f0efc07a460 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0f08033000 tx=0x7f0f200cfd50 comp rx=0 tx=0).stop 2024-09-10T21:42:44.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.098+0000 7f0f25000640 1 -- 172.21.15.7:0/546264971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f040040c0 msgr2=0x7f0f200cb2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:44.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.098+0000 7f0f25000640 1 --2- 172.21.15.7:0/546264971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f040040c0 0x7f0f200cb2c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0f10010a70 tx=0x7f0f10010f30 comp rx=0 tx=0).stop 2024-09-10T21:42:44.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.098+0000 7f0f25000640 1 -- 172.21.15.7:0/546264971 shutdown_connections 2024-09-10T21:42:44.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.098+0000 7f0f25000640 1 --2- 172.21.15.7:0/546264971 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0efc077fa0 0x7f0efc07a460 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:44.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.098+0000 7f0f25000640 1 --2- 172.21.15.7:0/546264971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0f201f7e80 0x7f0f2014a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:44.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.098+0000 7f0f25000640 1 --2- 172.21.15.7:0/546264971 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0f20149860 0x7f0f20149ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:44.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.098+0000 7f0f25000640 1 --2- 172.21.15.7:0/546264971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0f040040c0 0x7f0f200cb2c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:44.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.098+0000 7f0f25000640 1 -- 172.21.15.7:0/546264971 >> 172.21.15.7:0/546264971 conn(0x7f0f200d3d50 msgr2=0x7f0f200d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:44.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.098+0000 7f0f25000640 1 -- 172.21.15.7:0/546264971 shutdown_connections 2024-09-10T21:42:44.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.098+0000 7f0f25000640 1 -- 172.21.15.7:0/546264971 wait complete. 2024-09-10T21:42:44.115 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:42:44.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.274+0000 7f0423e00640 1 -- 172.21.15.7:0/63305328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04240cc330 msgr2=0x7f04240ad2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:44.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.274+0000 7f0423e00640 1 --2- 172.21.15.7:0/63305328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04240cc330 0x7f04240ad2f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0424073a60 tx=0x7f041c02a8c0 comp rx=0 tx=0).stop 2024-09-10T21:42:44.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.275+0000 7f0423e00640 1 -- 172.21.15.7:0/63305328 shutdown_connections 2024-09-10T21:42:44.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.275+0000 7f0423e00640 1 --2- 172.21.15.7:0/63305328 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04240cc330 0x7f04240ad2f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:44.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.275+0000 7f0423e00640 1 --2- 172.21.15.7:0/63305328 >> v2:172.21.15.133:3300/0 conn(0x7f0424074c00 0x7f04240ccf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:44.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.275+0000 7f0423e00640 1 -- 172.21.15.7:0/63305328 >> 172.21.15.7:0/63305328 conn(0x7f0424074080 msgr2=0x7f0424074460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:44.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.275+0000 7f0423e00640 1 -- 172.21.15.7:0/63305328 shutdown_connections 2024-09-10T21:42:44.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.275+0000 7f0423e00640 1 -- 172.21.15.7:0/63305328 wait complete. 2024-09-10T21:42:44.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.276+0000 7f0423e00640 1 Processor -- start 2024-09-10T21:42:44.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.276+0000 7f0423e00640 1 -- start start 2024-09-10T21:42:44.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.277+0000 7f0423e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0424074c00 0x7f04241f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:44.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.277+0000 7f0423e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04240cc330 0x7f042415e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:44.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.277+0000 7f0423e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f042415f420 0x7f04240c7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:44.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.277+0000 7f0423e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f04240adcb0 con 0x7f042415f420 2024-09-10T21:42:44.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.277+0000 7f0423e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f04240adb30 con 0x7f04240cc330 2024-09-10T21:42:44.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.277+0000 7f0423e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f04240ade30 con 0x7f0424074c00 2024-09-10T21:42:44.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.278+0000 7f0423400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f042415f420 0x7f04240c7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:44.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.278+0000 7f0423400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f042415f420 0x7f04240c7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54828/0 (socket says 172.21.15.7:54828) 2024-09-10T21:42:44.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.278+0000 7f0423400640 1 -- 172.21.15.7:0/3254356471 learned_addr learned my addr 172.21.15.7:0/3254356471 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:44.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.278+0000 7f0422a00640 1 --2- 172.21.15.7:0/3254356471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0424074c00 0x7f04241f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:44.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.278+0000 7f0422000640 1 --2- 172.21.15.7:0/3254356471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04240cc330 0x7f042415e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:44.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.278+0000 7f0423400640 1 -- 172.21.15.7:0/3254356471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0424074c00 msgr2=0x7f04241f7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:44.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.278+0000 7f0423400640 1 --2- 172.21.15.7:0/3254356471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0424074c00 0x7f04241f7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:44.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.278+0000 7f0423400640 1 -- 172.21.15.7:0/3254356471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04240cc330 msgr2=0x7f042415e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:44.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.278+0000 7f0423400640 1 --2- 172.21.15.7:0/3254356471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04240cc330 0x7f042415e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:44.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.278+0000 7f0423400640 1 -- 172.21.15.7:0/3254356471 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0424151ec0 con 0x7f042415f420 2024-09-10T21:42:44.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.278+0000 7f0423400640 1 --2- 172.21.15.7:0/3254356471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f042415f420 0x7f04240c7390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f041c02aeb0 tx=0x7f041c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:44.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.279+0000 7f041aa00640 1 -- 172.21.15.7:0/3254356471 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c040070 con 0x7f042415f420 2024-09-10T21:42:44.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.279+0000 7f041aa00640 1 -- 172.21.15.7:0/3254356471 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f041c004560 con 0x7f042415f420 2024-09-10T21:42:44.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.279+0000 7f041aa00640 1 -- 172.21.15.7:0/3254356471 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c003e10 con 0x7f042415f420 2024-09-10T21:42:44.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.279+0000 7f0423e00640 1 -- 172.21.15.7:0/3254356471 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042415b920 con 0x7f042415f420 2024-09-10T21:42:44.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.279+0000 7f0423e00640 1 -- 172.21.15.7:0/3254356471 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04240ca3b0 con 0x7f042415f420 2024-09-10T21:42:44.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.280+0000 7f0423e00640 1 -- 172.21.15.7:0/3254356471 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04240ae350 con 0x7f042415f420 2024-09-10T21:42:44.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.281+0000 7f041aa00640 1 -- 172.21.15.7:0/3254356471 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f041c03c420 con 0x7f042415f420 2024-09-10T21:42:44.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.282+0000 7f041aa00640 1 --2- 172.21.15.7:0/3254356471 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0404077fa0 0x7f040407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:44.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.282+0000 7f041aa00640 1 -- 172.21.15.7:0/3254356471 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5709+0+0 (secure 0 0 0) 0x7f041c0bb7a0 con 0x7f042415f420 2024-09-10T21:42:44.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.282+0000 7f0422a00640 1 --2- 172.21.15.7:0/3254356471 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0404077fa0 0x7f040407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:44.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.283+0000 7f0422a00640 1 --2- 172.21.15.7:0/3254356471 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0404077fa0 0x7f040407a460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0414001720 tx=0x7f0414007e90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:44.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.286+0000 7f041aa00640 1 -- 172.21.15.7:0/3254356471 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f041c083c40 con 0x7f042415f420 2024-09-10T21:42:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.516+0000 7f0423e00640 1 -- 172.21.15.7:0/3254356471 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0424068f50 con 0x7f042415f420 2024-09-10T21:42:44.518 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-10T21:42:44.517+0000 7feb3fa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:42:44.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.519+0000 7f041aa00640 1 -- 172.21.15.7:0/3254356471 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7f041c088670 con 0x7f042415f420 2024-09-10T21:42:44.520 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:42:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.521+0000 7f0423e00640 1 -- 172.21.15.7:0/3254356471 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0404077fa0 msgr2=0x7f040407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.521+0000 7f0423e00640 1 --2- 172.21.15.7:0/3254356471 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0404077fa0 0x7f040407a460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0414001720 tx=0x7f0414007e90 comp rx=0 tx=0).stop 2024-09-10T21:42:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.521+0000 7f0423e00640 1 -- 172.21.15.7:0/3254356471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f042415f420 msgr2=0x7f04240c7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:44.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.521+0000 7f0423e00640 1 --2- 172.21.15.7:0/3254356471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f042415f420 0x7f04240c7390 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f041c02aeb0 tx=0x7f041c003c20 comp rx=0 tx=0).stop 2024-09-10T21:42:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.522+0000 7f0423e00640 1 -- 172.21.15.7:0/3254356471 shutdown_connections 2024-09-10T21:42:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.522+0000 7f0423e00640 1 --2- 172.21.15.7:0/3254356471 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0404077fa0 0x7f040407a460 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.522+0000 7f0423e00640 1 --2- 172.21.15.7:0/3254356471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04240cc330 0x7f042415e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:44.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.522+0000 7f0423e00640 1 --2- 172.21.15.7:0/3254356471 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f042415f420 0x7f04240c7390 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:44.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.522+0000 7f0423e00640 1 --2- 172.21.15.7:0/3254356471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0424074c00 0x7f04241f7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:44.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.522+0000 7f0423e00640 1 -- 172.21.15.7:0/3254356471 >> 172.21.15.7:0/3254356471 conn(0x7f0424074080 msgr2=0x7f042406fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:44.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.522+0000 7f0423e00640 1 -- 172.21.15.7:0/3254356471 shutdown_connections 2024-09-10T21:42:44.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.522+0000 7f0423e00640 1 -- 172.21.15.7:0/3254356471 wait complete. 2024-09-10T21:42:44.537 INFO:tasks.ceph.mds.b.smithi133.stderr: -7099> 2024-09-10T21:42:14.653+0000 7fd3eb600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:42:44.538 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:42:44.589 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2024-09-10T21:42:44.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.701+0000 7f7ca7200640 1 -- 172.21.15.7:0/3634970831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7ca815def0 msgr2=0x7f7ca80d05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.701+0000 7f7ca7200640 1 --2- 172.21.15.7:0/3634970831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7ca815def0 0x7f7ca80d05b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7c98009860 tx=0x7f7c9802a510 comp rx=0 tx=0).stop 2024-09-10T21:42:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.701+0000 7f7ca7200640 1 -- 172.21.15.7:0/3634970831 shutdown_connections 2024-09-10T21:42:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.701+0000 7f7ca7200640 1 --2- 172.21.15.7:0/3634970831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7ca815def0 0x7f7ca80d05b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.701+0000 7f7ca7200640 1 -- 172.21.15.7:0/3634970831 >> 172.21.15.7:0/3634970831 conn(0x7f7ca80d3d70 msgr2=0x7f7ca80d4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.701+0000 7f7ca7200640 1 -- 172.21.15.7:0/3634970831 shutdown_connections 2024-09-10T21:42:44.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.701+0000 7f7ca7200640 1 -- 172.21.15.7:0/3634970831 wait complete. 2024-09-10T21:42:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.703+0000 7f7ca7200640 1 Processor -- start 2024-09-10T21:42:44.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.703+0000 7f7ca7200640 1 -- start start 2024-09-10T21:42:44.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.704+0000 7f7ca7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ca80d0af0 0x7f7ca81f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:44.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.704+0000 7f7ca7200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ca80d2a60 0x7f7ca8159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.704+0000 7f7ca7200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7ca815def0 0x7f7ca815aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.704+0000 7f7ca7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca80b4de0 con 0x7f7ca80d0af0 2024-09-10T21:42:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.704+0000 7f7ca7200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7ca80b4c60 con 0x7f7ca80d2a60 2024-09-10T21:42:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.704+0000 7f7ca7200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f7ca80b4f60 con 0x7f7ca815def0 2024-09-10T21:42:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.704+0000 7f7ca5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ca80d0af0 0x7f7ca81f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.704+0000 7f7ca5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ca80d0af0 0x7f7ca81f44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54832/0 (socket says 172.21.15.7:54832) 2024-09-10T21:42:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.704+0000 7f7ca5e00640 1 -- 172.21.15.7:0/230608251 learned_addr learned my addr 172.21.15.7:0/230608251 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.704+0000 7f7ca6800640 1 --2- 172.21.15.7:0/230608251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7ca815def0 0x7f7ca815aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.704+0000 7f7ca5400640 1 --2- 172.21.15.7:0/230608251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ca80d2a60 0x7f7ca8159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.704+0000 7f7ca5e00640 1 -- 172.21.15.7:0/230608251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7ca815def0 msgr2=0x7f7ca815aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.704+0000 7f7ca5e00640 1 --2- 172.21.15.7:0/230608251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7ca815def0 0x7f7ca815aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:44.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.704+0000 7f7ca5e00640 1 -- 172.21.15.7:0/230608251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ca80d2a60 msgr2=0x7f7ca8159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:44.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.704+0000 7f7ca5e00640 1 --2- 172.21.15.7:0/230608251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ca80d2a60 0x7f7ca8159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:44.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.704+0000 7f7ca5e00640 1 -- 172.21.15.7:0/230608251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ca8158890 con 0x7f7ca80d0af0 2024-09-10T21:42:44.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.705+0000 7f7ca5e00640 1 --2- 172.21.15.7:0/230608251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ca80d0af0 0x7f7ca81f44d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7c98009980 tx=0x7f7c98033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:44.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.705+0000 7f7c9e000640 1 -- 172.21.15.7:0/230608251 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98031b50 con 0x7f7ca80d0af0 2024-09-10T21:42:44.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.705+0000 7f7c9e000640 1 -- 172.21.15.7:0/230608251 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c98031de0 con 0x7f7ca80d0af0 2024-09-10T21:42:44.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.705+0000 7f7c9e000640 1 -- 172.21.15.7:0/230608251 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c98005280 con 0x7f7ca80d0af0 2024-09-10T21:42:44.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.705+0000 7f7ca7200640 1 -- 172.21.15.7:0/230608251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ca8158b70 con 0x7f7ca80d0af0 2024-09-10T21:42:44.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.705+0000 7f7ca7200640 1 -- 172.21.15.7:0/230608251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ca8159410 con 0x7f7ca80d0af0 2024-09-10T21:42:44.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.706+0000 7f7ca7200640 1 -- 172.21.15.7:0/230608251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ca80d05b0 con 0x7f7ca80d0af0 2024-09-10T21:42:44.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.707+0000 7f7c9e000640 1 -- 172.21.15.7:0/230608251 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7c98005420 con 0x7f7ca80d0af0 2024-09-10T21:42:44.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.708+0000 7f7c9e000640 1 --2- 172.21.15.7:0/230608251 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7c88077f70 0x7f7c8807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:44.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.708+0000 7f7c9e000640 1 -- 172.21.15.7:0/230608251 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7f7c980bb1b0 con 0x7f7ca80d0af0 2024-09-10T21:42:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.709+0000 7f7ca5400640 1 --2- 172.21.15.7:0/230608251 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7c88077f70 0x7f7c8807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:44.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.709+0000 7f7ca5400640 1 --2- 172.21.15.7:0/230608251 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7c88077f70 0x7f7c8807a430 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7c94007fd0 tx=0x7f7c94009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:44.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.712+0000 7f7c9e000640 1 -- 172.21.15.7:0/230608251 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7c9803f030 con 0x7f7ca80d0af0 2024-09-10T21:42:44.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:44.948+0000 7f7ca7200640 1 -- 172.21.15.7:0/230608251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7ca815c6a0 con 0x7f7ca80d0af0 2024-09-10T21:42:45.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.521+0000 7f7c9e000640 1 -- 172.21.15.7:0/230608251 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f7c98031770 con 0x7f7ca80d0af0 2024-09-10T21:42:45.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.524+0000 7f7ca7200640 1 -- 172.21.15.7:0/230608251 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7c88077f70 msgr2=0x7f7c8807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:45.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.524+0000 7f7ca7200640 1 --2- 172.21.15.7:0/230608251 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7c88077f70 0x7f7c8807a430 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7c94007fd0 tx=0x7f7c94009040 comp rx=0 tx=0).stop 2024-09-10T21:42:45.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.524+0000 7f7ca7200640 1 -- 172.21.15.7:0/230608251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ca80d0af0 msgr2=0x7f7ca81f44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:45.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.524+0000 7f7ca7200640 1 --2- 172.21.15.7:0/230608251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ca80d0af0 0x7f7ca81f44d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7c98009980 tx=0x7f7c98033ba0 comp rx=0 tx=0).stop 2024-09-10T21:42:45.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.524+0000 7f7ca7200640 1 -- 172.21.15.7:0/230608251 shutdown_connections 2024-09-10T21:42:45.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.524+0000 7f7ca7200640 1 --2- 172.21.15.7:0/230608251 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7c88077f70 0x7f7c8807a430 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:45.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.524+0000 7f7ca7200640 1 --2- 172.21.15.7:0/230608251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7ca80d2a60 0x7f7ca8159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:45.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.524+0000 7f7ca7200640 1 --2- 172.21.15.7:0/230608251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7ca815def0 0x7f7ca815aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:45.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.524+0000 7f7ca7200640 1 --2- 172.21.15.7:0/230608251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7ca80d0af0 0x7f7ca81f44d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:45.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.524+0000 7f7ca7200640 1 -- 172.21.15.7:0/230608251 >> 172.21.15.7:0/230608251 conn(0x7f7ca80d3d70 msgr2=0x7f7ca80d4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:45.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.524+0000 7f7ca7200640 1 -- 172.21.15.7:0/230608251 shutdown_connections 2024-09-10T21:42:45.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.524+0000 7f7ca7200640 1 -- 172.21.15.7:0/230608251 wait complete. 2024-09-10T21:42:45.544 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:42:45.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.712+0000 7f7ba4200640 1 -- 172.21.15.7:0/2168599451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c15bce0 msgr2=0x7f7b9c06cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:45.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.712+0000 7f7ba4200640 1 --2- 172.21.15.7:0/2168599451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c15bce0 0x7f7b9c06cb20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f7b94005fd0 tx=0x7f7b9402aa70 comp rx=0 tx=0).stop 2024-09-10T21:42:45.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.712+0000 7f7ba4200640 1 -- 172.21.15.7:0/2168599451 shutdown_connections 2024-09-10T21:42:45.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.712+0000 7f7ba4200640 1 --2- 172.21.15.7:0/2168599451 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c15bce0 0x7f7b9c06cb20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:45.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.712+0000 7f7ba4200640 1 --2- 172.21.15.7:0/2168599451 >> v2:172.21.15.133:3300/0 conn(0x7f7b9c15b900 0x7f7b9c070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:45.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.712+0000 7f7ba4200640 1 -- 172.21.15.7:0/2168599451 >> 172.21.15.7:0/2168599451 conn(0x7f7b9c0b1c10 msgr2=0x7f7b9c0b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:45.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.712+0000 7f7ba4200640 1 -- 172.21.15.7:0/2168599451 shutdown_connections 2024-09-10T21:42:45.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.712+0000 7f7ba4200640 1 -- 172.21.15.7:0/2168599451 wait complete. 2024-09-10T21:42:45.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.713+0000 7f7ba4200640 1 Processor -- start 2024-09-10T21:42:45.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.714+0000 7f7ba4200640 1 -- start start 2024-09-10T21:42:45.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.714+0000 7f7ba4200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b9c0ace80 0x7f7b9c154b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:45.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.714+0000 7f7ba4200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b9c15b900 0x7f7b9c14e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:45.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.715+0000 7f7ba4200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c15bce0 0x7f7b9c1f5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:45.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.715+0000 7f7ba4200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c0aed00 con 0x7f7b9c15bce0 2024-09-10T21:42:45.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.715+0000 7f7ba4200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7b9c0aeb80 con 0x7f7b9c0ace80 2024-09-10T21:42:45.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.715+0000 7f7ba4200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f7b9c0aee80 con 0x7f7b9c15b900 2024-09-10T21:42:45.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.715+0000 7f7ba2200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c15bce0 0x7f7b9c1f5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:45.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.715+0000 7f7ba2200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c15bce0 0x7f7b9c1f5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54838/0 (socket says 172.21.15.7:54838) 2024-09-10T21:42:45.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.715+0000 7f7ba2200640 1 -- 172.21.15.7:0/811210945 learned_addr learned my addr 172.21.15.7:0/811210945 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:45.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.715+0000 7f7ba0e00640 1 --2- 172.21.15.7:0/811210945 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b9c15b900 0x7f7b9c14e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:45.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.715+0000 7f7ba1800640 1 --2- 172.21.15.7:0/811210945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b9c0ace80 0x7f7b9c154b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:45.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.715+0000 7f7ba2200640 1 -- 172.21.15.7:0/811210945 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b9c15b900 msgr2=0x7f7b9c14e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:45.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.715+0000 7f7ba2200640 1 --2- 172.21.15.7:0/811210945 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b9c15b900 0x7f7b9c14e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:45.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.715+0000 7f7ba2200640 1 -- 172.21.15.7:0/811210945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b9c0ace80 msgr2=0x7f7b9c154b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:45.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.715+0000 7f7ba2200640 1 --2- 172.21.15.7:0/811210945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b9c0ace80 0x7f7b9c154b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:45.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.715+0000 7f7ba2200640 1 -- 172.21.15.7:0/811210945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b9c155070 con 0x7f7b9c15bce0 2024-09-10T21:42:45.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.716+0000 7f7ba2200640 1 --2- 172.21.15.7:0/811210945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c15bce0 0x7f7b9c1f5cd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009d90 tx=0x7f7b94003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:45.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.716+0000 7f7b99600640 1 -- 172.21.15.7:0/811210945 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94033050 con 0x7f7b9c15bce0 2024-09-10T21:42:45.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.716+0000 7f7b99600640 1 -- 172.21.15.7:0/811210945 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b94004450 con 0x7f7b9c15bce0 2024-09-10T21:42:45.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.716+0000 7f7ba4200640 1 -- 172.21.15.7:0/811210945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b9c152d70 con 0x7f7b9c15bce0 2024-09-10T21:42:45.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.716+0000 7f7b99600640 1 -- 172.21.15.7:0/811210945 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b94038450 con 0x7f7b9c15bce0 2024-09-10T21:42:45.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.716+0000 7f7ba4200640 1 -- 172.21.15.7:0/811210945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b9c153610 con 0x7f7b9c15bce0 2024-09-10T21:42:45.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.717+0000 7f7ba4200640 1 -- 172.21.15.7:0/811210945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b9c0af330 con 0x7f7b9c15bce0 2024-09-10T21:42:45.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.718+0000 7f7b99600640 1 -- 172.21.15.7:0/811210945 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7b94040070 con 0x7f7b9c15bce0 2024-09-10T21:42:45.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.719+0000 7f7b99600640 1 --2- 172.21.15.7:0/811210945 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b7c077f70 0x7f7b7c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:45.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.719+0000 7f7b99600640 1 -- 172.21.15.7:0/811210945 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5795+0+0 (secure 0 0 0) 0x7f7b940bb100 con 0x7f7b9c15bce0 2024-09-10T21:42:45.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.720+0000 7f7ba1800640 1 --2- 172.21.15.7:0/811210945 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b7c077f70 0x7f7b7c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:45.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.720+0000 7f7ba1800640 1 --2- 172.21.15.7:0/811210945 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b7c077f70 0x7f7b7c07a430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000bbe0 tx=0x7f7b9000b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.723+0000 7f7b99600640 1 -- 172.21.15.7:0/811210945 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7b9c155070 con 0x7f7b9c15bce0 2024-09-10T21:42:45.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:45.918+0000 7f7ba4200640 1 -- 172.21.15.7:0/811210945 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7b9c159270 con 0x7f7b9c15bce0 2024-09-10T21:42:46.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:46.837+0000 7f7b99600640 1 -- 172.21.15.7:0/811210945 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v23) ==== 181+0+0 (secure 0 0 0) 0x7f7b940834b0 con 0x7f7b9c15bce0 2024-09-10T21:42:46.839 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:42:46.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:46.841+0000 7f7ba4200640 1 -- 172.21.15.7:0/811210945 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b7c077f70 msgr2=0x7f7b7c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:46.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:46.841+0000 7f7ba4200640 1 --2- 172.21.15.7:0/811210945 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b7c077f70 0x7f7b7c07a430 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7b9000bbe0 tx=0x7f7b9000b390 comp rx=0 tx=0).stop 2024-09-10T21:42:46.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:46.841+0000 7f7ba4200640 1 -- 172.21.15.7:0/811210945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c15bce0 msgr2=0x7f7b9c1f5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:46.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:46.841+0000 7f7ba4200640 1 --2- 172.21.15.7:0/811210945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c15bce0 0x7f7b9c1f5cd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7b94009d90 tx=0x7f7b94003f10 comp rx=0 tx=0).stop 2024-09-10T21:42:46.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:46.842+0000 7f7ba4200640 1 -- 172.21.15.7:0/811210945 shutdown_connections 2024-09-10T21:42:46.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:46.842+0000 7f7ba4200640 1 --2- 172.21.15.7:0/811210945 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b7c077f70 0x7f7b7c07a430 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:46.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:46.842+0000 7f7ba4200640 1 --2- 172.21.15.7:0/811210945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b9c15bce0 0x7f7b9c1f5cd0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:46.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:46.842+0000 7f7ba4200640 1 --2- 172.21.15.7:0/811210945 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b9c15b900 0x7f7b9c14e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:46.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:46.842+0000 7f7ba4200640 1 --2- 172.21.15.7:0/811210945 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b9c0ace80 0x7f7b9c154b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:46.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:46.842+0000 7f7ba4200640 1 -- 172.21.15.7:0/811210945 >> 172.21.15.7:0/811210945 conn(0x7f7b9c0b1c10 msgr2=0x7f7b9c15f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:46.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:46.842+0000 7f7ba4200640 1 -- 172.21.15.7:0/811210945 shutdown_connections 2024-09-10T21:42:46.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:46.842+0000 7f7ba4200640 1 -- 172.21.15.7:0/811210945 wait complete. 2024-09-10T21:42:46.863 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:42:47.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.031+0000 7fd08d400640 1 -- 172.21.15.7:0/1951145856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0880ace80 msgr2=0x7fd08815f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.031+0000 7fd08d400640 1 --2- 172.21.15.7:0/1951145856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0880ace80 0x7fd08815f730 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd070005fd0 tx=0x7fd07002a540 comp rx=0 tx=0).stop 2024-09-10T21:42:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.032+0000 7fd08d400640 1 -- 172.21.15.7:0/1951145856 shutdown_connections 2024-09-10T21:42:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.032+0000 7fd08d400640 1 --2- 172.21.15.7:0/1951145856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0880ace80 0x7fd08815f730 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.032+0000 7fd08d400640 1 --2- 172.21.15.7:0/1951145856 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd08815b900 0x7fd088070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.032+0000 7fd08d400640 1 -- 172.21.15.7:0/1951145856 >> 172.21.15.7:0/1951145856 conn(0x7fd0880b1c10 msgr2=0x7fd0880b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.032+0000 7fd08d400640 1 -- 172.21.15.7:0/1951145856 shutdown_connections 2024-09-10T21:42:47.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.032+0000 7fd08d400640 1 -- 172.21.15.7:0/1951145856 wait complete. 2024-09-10T21:42:47.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.033+0000 7fd08d400640 1 Processor -- start 2024-09-10T21:42:47.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.033+0000 7fd08d400640 1 -- start start 2024-09-10T21:42:47.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.034+0000 7fd08d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0880ace80 0x7fd08809aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:47.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.034+0000 7fd08d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08815b900 0x7fd088153f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:47.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.034+0000 7fd08d400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd08815bce0 0x7fd088154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:47.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.034+0000 7fd08d400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd0880aed00 con 0x7fd08815b900 2024-09-10T21:42:47.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.034+0000 7fd08d400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd0880aeb80 con 0x7fd0880ace80 2024-09-10T21:42:47.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.034+0000 7fd08d400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fd0880aee80 con 0x7fd08815bce0 2024-09-10T21:42:47.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.035+0000 7fd07de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08815b900 0x7fd088153f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:47.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.035+0000 7fd07de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08815b900 0x7fd088153f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54858/0 (socket says 172.21.15.7:54858) 2024-09-10T21:42:47.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.035+0000 7fd07de00640 1 -- 172.21.15.7:0/3519193067 learned_addr learned my addr 172.21.15.7:0/3519193067 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:47.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.035+0000 7fd086800640 1 --2- 172.21.15.7:0/3519193067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0880ace80 0x7fd08809aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:47.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.035+0000 7fd087200640 1 --2- 172.21.15.7:0/3519193067 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd08815bce0 0x7fd088154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:47.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.035+0000 7fd07de00640 1 -- 172.21.15.7:0/3519193067 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd08815bce0 msgr2=0x7fd088154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:47.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.035+0000 7fd07de00640 1 --2- 172.21.15.7:0/3519193067 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd08815bce0 0x7fd088154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:47.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.035+0000 7fd07de00640 1 -- 172.21.15.7:0/3519193067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0880ace80 msgr2=0x7fd08809aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:47.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.035+0000 7fd07de00640 1 --2- 172.21.15.7:0/3519193067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0880ace80 0x7fd08809aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:47.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.035+0000 7fd07de00640 1 -- 172.21.15.7:0/3519193067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd088074000 con 0x7fd08815b900 2024-09-10T21:42:47.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.035+0000 7fd086800640 1 --2- 172.21.15.7:0/3519193067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0880ace80 0x7fd08809aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:42:47.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.035+0000 7fd07de00640 1 --2- 172.21.15.7:0/3519193067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08815b900 0x7fd088153f00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd070009850 tx=0x7fd0700038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:47.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.036+0000 7fd07f400640 1 -- 172.21.15.7:0/3519193067 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07003f070 con 0x7fd08815b900 2024-09-10T21:42:47.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.036+0000 7fd07f400640 1 -- 172.21.15.7:0/3519193067 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0700047a0 con 0x7fd08815b900 2024-09-10T21:42:47.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.036+0000 7fd07f400640 1 -- 172.21.15.7:0/3519193067 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd070035030 con 0x7fd08815b900 2024-09-10T21:42:47.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.036+0000 7fd08d400640 1 -- 172.21.15.7:0/3519193067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0880742a0 con 0x7fd08815b900 2024-09-10T21:42:47.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.036+0000 7fd08d400640 1 -- 172.21.15.7:0/3519193067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd088072980 con 0x7fd08815b900 2024-09-10T21:42:47.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.037+0000 7fd08d400640 1 -- 172.21.15.7:0/3519193067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0880af330 con 0x7fd08815b900 2024-09-10T21:42:47.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.038+0000 7fd07f400640 1 -- 172.21.15.7:0/3519193067 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd070004b90 con 0x7fd08815b900 2024-09-10T21:42:47.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.039+0000 7fd07f400640 1 --2- 172.21.15.7:0/3519193067 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd064077f70 0x7fd06407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:47.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.039+0000 7fd07f400640 1 -- 172.21.15.7:0/3519193067 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5357+0+0 (secure 0 0 0) 0x7fd0700bac90 con 0x7fd08815b900 2024-09-10T21:42:47.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.040+0000 7fd086800640 1 --2- 172.21.15.7:0/3519193067 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd064077f70 0x7fd06407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:47.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.040+0000 7fd086800640 1 --2- 172.21.15.7:0/3519193067 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd064077f70 0x7fd06407a430 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd0881552d0 tx=0x7fd07800b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:47.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.043+0000 7fd07f400640 1 -- 172.21.15.7:0/3519193067 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd070083290 con 0x7fd08815b900 2024-09-10T21:42:47.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.238+0000 7fd08d400640 1 -- 172.21.15.7:0/3519193067 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fd0880b0a10 con 0x7fd08815b900 2024-09-10T21:42:47.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.840+0000 7fd07f400640 1 -- 172.21.15.7:0/3519193067 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v24) ==== 169+0+0 (secure 0 0 0) 0x7fd070087cc0 con 0x7fd08815b900 2024-09-10T21:42:47.841 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-09-10T21:42:47.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.843+0000 7fd08d400640 1 -- 172.21.15.7:0/3519193067 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd064077f70 msgr2=0x7fd06407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:47.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.843+0000 7fd08d400640 1 --2- 172.21.15.7:0/3519193067 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd064077f70 0x7fd06407a430 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd0881552d0 tx=0x7fd07800b390 comp rx=0 tx=0).stop 2024-09-10T21:42:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.843+0000 7fd08d400640 1 -- 172.21.15.7:0/3519193067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08815b900 msgr2=0x7fd088153f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.843+0000 7fd08d400640 1 --2- 172.21.15.7:0/3519193067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08815b900 0x7fd088153f00 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd070009850 tx=0x7fd0700038a0 comp rx=0 tx=0).stop 2024-09-10T21:42:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.843+0000 7fd08d400640 1 -- 172.21.15.7:0/3519193067 shutdown_connections 2024-09-10T21:42:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.843+0000 7fd08d400640 1 --2- 172.21.15.7:0/3519193067 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd064077f70 0x7fd06407a430 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.843+0000 7fd08d400640 1 --2- 172.21.15.7:0/3519193067 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd08815bce0 0x7fd088154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.843+0000 7fd08d400640 1 --2- 172.21.15.7:0/3519193067 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd08815b900 0x7fd088153f00 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.843+0000 7fd08d400640 1 --2- 172.21.15.7:0/3519193067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0880ace80 0x7fd08809aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.843+0000 7fd08d400640 1 -- 172.21.15.7:0/3519193067 >> 172.21.15.7:0/3519193067 conn(0x7fd0880b1c10 msgr2=0x7fd0880b1210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.843+0000 7fd08d400640 1 -- 172.21.15.7:0/3519193067 shutdown_connections 2024-09-10T21:42:47.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:47.843+0000 7fd08d400640 1 -- 172.21.15.7:0/3519193067 wait complete. 2024-09-10T21:42:47.870 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:42:47.870 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:42:47.871 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:42:47.871 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:42:47.871 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:42:47.871 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:42:47.871 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:42:47.871 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:42:47.872 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:42:47.872 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:42:47.872 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:42:47.872 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:42:47.872 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:42:47.872 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:42:47.873 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:42:47.873 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:42:47.892 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:42:47.892 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:42:47.893 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:42:47.893 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:42:47.894 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:42:47.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:42:47.895 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:42:47.895 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:42:47.936 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:42:47.937 DEBUG:teuthology.parallel:result is None 2024-09-10T21:42:47.938 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:42:47.938 DEBUG:teuthology.parallel:result is None 2024-09-10T21:42:47.939 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:42:47.939 DEBUG:teuthology.parallel:result is None 2024-09-10T21:42:47.940 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:42:47.941 DEBUG:teuthology.parallel:result is None 2024-09-10T21:42:47.941 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:42:48.067 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2024-09-10T21:42:48.068 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2024-09-10T21:42:48.075 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-09-10T21:42:48.075 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-09-10T21:42:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.111+0000 7f1242400640 1 -- 172.21.15.7:0/696778480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f123c0cf650 msgr2=0x7f123c1579b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.111+0000 7f1242400640 1 --2- 172.21.15.7:0/696778480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f123c0cf650 0x7f123c1579b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1234009be0 tx=0x7f123402a920 comp rx=0 tx=0).stop 2024-09-10T21:42:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.111+0000 7f1242400640 1 -- 172.21.15.7:0/696778480 shutdown_connections 2024-09-10T21:42:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.111+0000 7f1242400640 1 --2- 172.21.15.7:0/696778480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f123c0cf650 0x7f123c1579b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.111+0000 7f1242400640 1 -- 172.21.15.7:0/696778480 >> 172.21.15.7:0/696778480 conn(0x7f123c0b1660 msgr2=0x7f123c0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.112+0000 7f1242400640 1 -- 172.21.15.7:0/696778480 shutdown_connections 2024-09-10T21:42:48.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.112+0000 7f1242400640 1 -- 172.21.15.7:0/696778480 wait complete. 2024-09-10T21:42:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.113+0000 7f1242400640 1 Processor -- start 2024-09-10T21:42:48.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.113+0000 7f1242400640 1 -- start start 2024-09-10T21:42:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.114+0000 7f1242400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f123c070600 0x7f123c0c9460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.114+0000 7f1242400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f123c0ccc30 0x7f123c156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.114+0000 7f1242400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f123c0cf650 0x7f123c073f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.114+0000 7f1242400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f123c0af200 con 0x7f123c0ccc30 2024-09-10T21:42:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.114+0000 7f1242400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f123c0af080 con 0x7f123c070600 2024-09-10T21:42:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.114+0000 7f1242400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f123c1650d0 con 0x7f123c0cf650 2024-09-10T21:42:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.114+0000 7f1233e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f123c0ccc30 0x7f123c156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.114+0000 7f1233e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f123c0ccc30 0x7f123c156bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54898/0 (socket says 172.21.15.7:54898) 2024-09-10T21:42:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.114+0000 7f1233e00640 1 -- 172.21.15.7:0/645299521 learned_addr learned my addr 172.21.15.7:0/645299521 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.115+0000 7f123b400640 1 --2- 172.21.15.7:0/645299521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f123c070600 0x7f123c0c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:48.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.115+0000 7f123be00640 1 --2- 172.21.15.7:0/645299521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f123c0cf650 0x7f123c073f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.115+0000 7f1233e00640 1 -- 172.21.15.7:0/645299521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f123c0cf650 msgr2=0x7f123c073f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.115+0000 7f1233e00640 1 --2- 172.21.15.7:0/645299521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f123c0cf650 0x7f123c073f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.115+0000 7f1233e00640 1 -- 172.21.15.7:0/645299521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f123c070600 msgr2=0x7f123c0c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.115+0000 7f1233e00640 1 --2- 172.21.15.7:0/645299521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f123c070600 0x7f123c0c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.115+0000 7f1233e00640 1 -- 172.21.15.7:0/645299521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f123c1f5410 con 0x7f123c0ccc30 2024-09-10T21:42:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.115+0000 7f123b400640 1 --2- 172.21.15.7:0/645299521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f123c070600 0x7f123c0c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.115+0000 7f1233e00640 1 --2- 172.21.15.7:0/645299521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f123c0ccc30 0x7f123c156bc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f122400c490 tx=0x7f122400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.115+0000 7f1233400640 1 -- 172.21.15.7:0/645299521 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224016020 con 0x7f123c0ccc30 2024-09-10T21:42:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.115+0000 7f1233400640 1 -- 172.21.15.7:0/645299521 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1224012920 con 0x7f123c0ccc30 2024-09-10T21:42:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.116+0000 7f1233400640 1 -- 172.21.15.7:0/645299521 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1224012d70 con 0x7f123c0ccc30 2024-09-10T21:42:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.116+0000 7f1242400640 1 -- 172.21.15.7:0/645299521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f123c155e00 con 0x7f123c0ccc30 2024-09-10T21:42:48.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.116+0000 7f1242400640 1 -- 172.21.15.7:0/645299521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f123c1563c0 con 0x7f123c0ccc30 2024-09-10T21:42:48.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.117+0000 7f1242400640 1 -- 172.21.15.7:0/645299521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f123c1568b0 con 0x7f123c0ccc30 2024-09-10T21:42:48.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.118+0000 7f1233400640 1 -- 172.21.15.7:0/645299521 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f122401a3b0 con 0x7f123c0ccc30 2024-09-10T21:42:48.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.119+0000 7f1233400640 1 --2- 172.21.15.7:0/645299521 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1218077f70 0x7f121807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:48.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.119+0000 7f1233400640 1 -- 172.21.15.7:0/645299521 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 4943+0+0 (secure 0 0 0) 0x7f1224099740 con 0x7f123c0ccc30 2024-09-10T21:42:48.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.119+0000 7f123b400640 1 --2- 172.21.15.7:0/645299521 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1218077f70 0x7f121807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:48.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.120+0000 7f123b400640 1 --2- 172.21.15.7:0/645299521 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1218077f70 0x7f121807a430 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f123c15d530 tx=0x7f122c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:48.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.123+0000 7f1233400640 1 -- 172.21.15.7:0/645299521 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1224020070 con 0x7f123c0ccc30 2024-09-10T21:42:48.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.330+0000 7f1242400640 1 -- 172.21.15.7:0/645299521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f123c1579b0 con 0x7f123c0ccc30 2024-09-10T21:42:48.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.866+0000 7f1233400640 1 -- 172.21.15.7:0/645299521 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v26) ==== 97+0+0 (secure 0 0 0) 0x7f12240624a0 con 0x7f123c0ccc30 2024-09-10T21:42:48.867 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-09-10T21:42:48.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.868+0000 7f1242400640 1 -- 172.21.15.7:0/645299521 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1218077f70 msgr2=0x7f121807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:48.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.868+0000 7f1242400640 1 --2- 172.21.15.7:0/645299521 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1218077f70 0x7f121807a430 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f123c15d530 tx=0x7f122c009040 comp rx=0 tx=0).stop 2024-09-10T21:42:48.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.868+0000 7f1242400640 1 -- 172.21.15.7:0/645299521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f123c0ccc30 msgr2=0x7f123c156bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:48.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.868+0000 7f1242400640 1 --2- 172.21.15.7:0/645299521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f123c0ccc30 0x7f123c156bc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f122400c490 tx=0x7f122400c950 comp rx=0 tx=0).stop 2024-09-10T21:42:48.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.869+0000 7f1242400640 1 -- 172.21.15.7:0/645299521 shutdown_connections 2024-09-10T21:42:48.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.869+0000 7f1242400640 1 --2- 172.21.15.7:0/645299521 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1218077f70 0x7f121807a430 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:48.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.869+0000 7f1242400640 1 --2- 172.21.15.7:0/645299521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f123c0cf650 0x7f123c073f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:48.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.869+0000 7f1242400640 1 --2- 172.21.15.7:0/645299521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f123c0ccc30 0x7f123c156bc0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:48.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.869+0000 7f1242400640 1 --2- 172.21.15.7:0/645299521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f123c070600 0x7f123c0c9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:48.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.869+0000 7f1242400640 1 -- 172.21.15.7:0/645299521 >> 172.21.15.7:0/645299521 conn(0x7f123c0b1660 msgr2=0x7f123c1678e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:48.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.869+0000 7f1242400640 1 -- 172.21.15.7:0/645299521 shutdown_connections 2024-09-10T21:42:48.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:48.869+0000 7f1242400640 1 -- 172.21.15.7:0/645299521 wait complete. 2024-09-10T21:42:48.889 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:42:49.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.050+0000 7f1edde00640 1 -- 172.21.15.7:0/879567393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed8070600 msgr2=0x7f1ed806e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:49.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.050+0000 7f1edde00640 1 --2- 172.21.15.7:0/879567393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed8070600 0x7f1ed806e510 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8009860 tx=0x7f1ec802a440 comp rx=0 tx=0).stop 2024-09-10T21:42:49.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.050+0000 7f1edde00640 1 -- 172.21.15.7:0/879567393 shutdown_connections 2024-09-10T21:42:49.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.050+0000 7f1edde00640 1 --2- 172.21.15.7:0/879567393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed8070600 0x7f1ed806e510 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:49.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.050+0000 7f1edde00640 1 -- 172.21.15.7:0/879567393 >> 172.21.15.7:0/879567393 conn(0x7f1ed80b16d0 msgr2=0x7f1ed80b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:49.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.050+0000 7f1edde00640 1 -- 172.21.15.7:0/879567393 shutdown_connections 2024-09-10T21:42:49.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.051+0000 7f1edde00640 1 -- 172.21.15.7:0/879567393 wait complete. 2024-09-10T21:42:49.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.052+0000 7f1edde00640 1 Processor -- start 2024-09-10T21:42:49.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.052+0000 7f1edde00640 1 -- start start 2024-09-10T21:42:49.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.053+0000 7f1edde00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ed8070600 0x7f1ed81f5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:49.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.053+0000 7f1edde00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed80ca360 0x7f1ed80c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:49.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.053+0000 7f1edde00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed815da10 0x7f1ed80b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:49.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.053+0000 7f1edde00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed815b380 con 0x7f1ed815da10 2024-09-10T21:42:49.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.053+0000 7f1edde00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed815b200 con 0x7f1ed80ca360 2024-09-10T21:42:49.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.053+0000 7f1edde00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed80cc560 con 0x7f1ed8070600 2024-09-10T21:42:49.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.053+0000 7f1ed7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed815da10 0x7f1ed80b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:49.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.053+0000 7f1ed7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed815da10 0x7f1ed80b0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54910/0 (socket says 172.21.15.7:54910) 2024-09-10T21:42:49.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.053+0000 7f1ed7400640 1 -- 172.21.15.7:0/3440234029 learned_addr learned my addr 172.21.15.7:0/3440234029 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:49.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.054+0000 7f1ed6a00640 1 --2- 172.21.15.7:0/3440234029 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ed8070600 0x7f1ed81f5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:49.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.054+0000 7f1ece000640 1 --2- 172.21.15.7:0/3440234029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed80ca360 0x7f1ed80c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:49.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.054+0000 7f1ed7400640 1 -- 172.21.15.7:0/3440234029 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ed8070600 msgr2=0x7f1ed81f5510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:49.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.054+0000 7f1ed7400640 1 --2- 172.21.15.7:0/3440234029 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ed8070600 0x7f1ed81f5510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:49.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.054+0000 7f1ed7400640 1 -- 172.21.15.7:0/3440234029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed80ca360 msgr2=0x7f1ed80c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:49.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.054+0000 7f1ed7400640 1 --2- 172.21.15.7:0/3440234029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed80ca360 0x7f1ed80c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:49.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.054+0000 7f1ed7400640 1 -- 172.21.15.7:0/3440234029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed80c61b0 con 0x7f1ed815da10 2024-09-10T21:42:49.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.054+0000 7f1ece000640 1 --2- 172.21.15.7:0/3440234029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed80ca360 0x7f1ed80c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:42:49.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.054+0000 7f1ed7400640 1 --2- 172.21.15.7:0/3440234029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed815da10 0x7f1ed80b0d80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000c8a0 tx=0x7f1ed000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:49.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.054+0000 7f1ecfe00640 1 -- 172.21.15.7:0/3440234029 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0017070 con 0x7f1ed815da10 2024-09-10T21:42:49.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.055+0000 7f1ecfe00640 1 -- 172.21.15.7:0/3440234029 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ed0005070 con 0x7f1ed815da10 2024-09-10T21:42:49.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.055+0000 7f1ecfe00640 1 -- 172.21.15.7:0/3440234029 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed00054c0 con 0x7f1ed815da10 2024-09-10T21:42:49.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.055+0000 7f1edde00640 1 -- 172.21.15.7:0/3440234029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed80c6490 con 0x7f1ed815da10 2024-09-10T21:42:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.055+0000 7f1edde00640 1 -- 172.21.15.7:0/3440234029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed80c8760 con 0x7f1ed815da10 2024-09-10T21:42:49.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.056+0000 7f1edde00640 1 -- 172.21.15.7:0/3440234029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed806e510 con 0x7f1ed815da10 2024-09-10T21:42:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.057+0000 7f1ecfe00640 1 -- 172.21.15.7:0/3440234029 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1ed0005660 con 0x7f1ed815da10 2024-09-10T21:42:49.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.058+0000 7f1ecfe00640 1 --2- 172.21.15.7:0/3440234029 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1eb4077f70 0x7f1eb407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:49.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.058+0000 7f1ecfe00640 1 -- 172.21.15.7:0/3440234029 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f1ed0099420 con 0x7f1ed815da10 2024-09-10T21:42:49.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.058+0000 7f1ed6a00640 1 --2- 172.21.15.7:0/3440234029 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1eb4077f70 0x7f1eb407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:49.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.059+0000 7f1ed6a00640 1 --2- 172.21.15.7:0/3440234029 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1eb4077f70 0x7f1eb407a430 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8034000 tx=0x7f1ec8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:49.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.062+0000 7f1ecfe00640 1 -- 172.21.15.7:0/3440234029 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1ed0061c10 con 0x7f1ed815da10 2024-09-10T21:42:49.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.329+0000 7f1edde00640 1 -- 172.21.15.7:0/3440234029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1ed80cc340 con 0x7f1ed815da10 2024-09-10T21:42:49.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.329+0000 7f1ecfe00640 1 -- 172.21.15.7:0/3440234029 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) ==== 80+0+6645 (secure 0 0 0) 0x7f1ed0066640 con 0x7f1ed815da10 2024-09-10T21:42:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:49.331 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:42:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-09-10T21:42:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-09-10T21:42:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmgRMuLBAA88QT12AXv2qf11IxU20Xbg==", 2024-09-10T21:42:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.331 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:42:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:42:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:42:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:42:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-09-10T21:42:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmRUM2BBAAwcQM6z+hjZI4ZWBlVrYMwQ==", 2024-09-10T21:42:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:42:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:42:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:42:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:42:49.332 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-09-10T21:42:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmnq31NRAAj1eJUAIS986idxppfeXEAA==", 2024-09-10T21:42:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:42:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:42:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:42:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:42:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.333 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-09-10T21:42:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmy43uDRAAzdKjLxZMGROuDABrsJ0zgQ==", 2024-09-10T21:42:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:42:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:42:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:42:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:42:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.334 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-09-10T21:42:49.335 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDwvOBmzfgtOxAAk6K5fU0Uar4VW9nUUIuNdw==", 2024-09-10T21:42:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:42:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:42:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:42:49.336 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-09-10T21:42:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD0vOBmHcRzChAA3z+CaGgu5NkyUmjpqzmKjw==", 2024-09-10T21:42:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:42:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:42:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:42:49.337 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-09-10T21:42:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD3vOBm9sArFBAAGQlyiw6qEcyumEBICWEBKA==", 2024-09-10T21:42:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:42:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:42:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:42:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-09-10T21:42:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD6vOBmkO3rHxAAW0zPydYZRGUKjGh3yIyPCA==", 2024-09-10T21:42:49.338 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:42:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:42:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:42:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-09-10T21:42:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAAveBmlPABMxAAeazVOzc1DhsVAwoHn87QIA==", 2024-09-10T21:42:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:42:49.339 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:42:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:42:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-09-10T21:42:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAEveBmL9qzARAApu8nWIY9HVsDl4yUnfhH9w==", 2024-09-10T21:42:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:42:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:42:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:42:49.340 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-09-10T21:42:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAHveBmJ4ofDRAAEiCFN8nGzVSR3u2gKoT+2Q==", 2024-09-10T21:42:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:42:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:42:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:42:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-09-10T21:42:49.341 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAKveBmoUGEGBAAhXmkUaniuAQrBiYPWr8Tog==", 2024-09-10T21:42:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:42:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:42:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:42:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-09-10T21:42:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBm0aqcFhAAtid2bnzjcpaYAbU7g0JcOA==", 2024-09-10T21:42:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.342 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:42:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:42:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:42:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:42:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-09-10T21:42:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmqf+MIxAAib5hVnbBsw/bJPqsQszJwg==", 2024-09-10T21:42:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:42:49.343 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:42:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:42:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:42:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-09-10T21:42:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmybQhHRAAZUwqnIsihEWnHZWQTYr+uQ==", 2024-09-10T21:42:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:42:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:42:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:42:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:42:49.344 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-09-10T21:42:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmDbrrKRAAUxb3EBfKIJ4FAGHW3QIsLg==", 2024-09-10T21:42:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:42:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:42:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:42:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:42:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.345 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-09-10T21:42:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDtvOBmF+eEOBAA3icvNqI0nWDShE49DZ3bvg==", 2024-09-10T21:42:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:42:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:42:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:42:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:42:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-mds", 2024-09-10T21:42:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAWveBmkt5JMBAAJU4Gflq7CnDEvzMwJjLehA==", 2024-09-10T21:42:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-mds" 2024-09-10T21:42:49.346 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-mgr", 2024-09-10T21:42:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAWveBmcOhJMBAAknR76J2xjaWV8Pu1H3KNrQ==", 2024-09-10T21:42:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-10T21:42:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.347 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-osd", 2024-09-10T21:42:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAWveBmDPJJMBAAn6TrUk/rZH2Xa4sjiTgOOQ==", 2024-09-10T21:42:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-osd" 2024-09-10T21:42:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rbd", 2024-09-10T21:42:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAWveBmRfpJMBAAsob18GGwzklz0rleffrtig==", 2024-09-10T21:42:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-10T21:42:49.348 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.349 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-10T21:42:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAWveBmgwZKMBAAGOQFGd5ugI/TTP+dHgLWkw==", 2024-09-10T21:42:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-10T21:42:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.bootstrap-rgw", 2024-09-10T21:42:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAWveBm5w5KMBAAdQjFV78S3E0fe+j9MYdqCw==", 2024-09-10T21:42:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-10T21:42:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.350 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-09-10T21:42:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmsu6OJRAAhdgDvpeHY/zDKUeUOv6xxA==", 2024-09-10T21:42:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:42:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-10T21:42:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:42:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:42:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-09-10T21:42:49.351 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmKBHLHhAA0W5My19tIhN1v2qev4cBTA==", 2024-09-10T21:42:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:42:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:42:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-10T21:42:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:42:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:49.352 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T21:42:49.352 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:42:49.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.332+0000 7f1edde00640 1 -- 172.21.15.7:0/3440234029 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1eb4077f70 msgr2=0x7f1eb407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:49.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.332+0000 7f1edde00640 1 --2- 172.21.15.7:0/3440234029 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1eb4077f70 0x7f1eb407a430 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8034000 tx=0x7f1ec8004730 comp rx=0 tx=0).stop 2024-09-10T21:42:49.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.332+0000 7f1edde00640 1 -- 172.21.15.7:0/3440234029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed815da10 msgr2=0x7f1ed80b0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.332+0000 7f1edde00640 1 --2- 172.21.15.7:0/3440234029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed815da10 0x7f1ed80b0d80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1ed000c8a0 tx=0x7f1ed000cd60 comp rx=0 tx=0).stop 2024-09-10T21:42:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.332+0000 7f1edde00640 1 -- 172.21.15.7:0/3440234029 shutdown_connections 2024-09-10T21:42:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.332+0000 7f1edde00640 1 --2- 172.21.15.7:0/3440234029 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1eb4077f70 0x7f1eb407a430 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.332+0000 7f1edde00640 1 --2- 172.21.15.7:0/3440234029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed80ca360 0x7f1ed80c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.332+0000 7f1edde00640 1 --2- 172.21.15.7:0/3440234029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed815da10 0x7f1ed80b0d80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.332+0000 7f1edde00640 1 --2- 172.21.15.7:0/3440234029 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ed8070600 0x7f1ed81f5510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.332+0000 7f1edde00640 1 -- 172.21.15.7:0/3440234029 >> 172.21.15.7:0/3440234029 conn(0x7f1ed80b16d0 msgr2=0x7f1ed80b49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.332+0000 7f1edde00640 1 -- 172.21.15.7:0/3440234029 shutdown_connections 2024-09-10T21:42:49.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.332+0000 7f1edde00640 1 -- 172.21.15.7:0/3440234029 wait complete. 2024-09-10T21:42:49.355 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-10T21:42:49.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.515+0000 7fd041800640 1 -- 172.21.15.7:0/3448891789 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd03c15cbe0 msgr2=0x7fd03c073cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:49.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.515+0000 7fd041800640 1 --2- 172.21.15.7:0/3448891789 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd03c15cbe0 0x7fd03c073cd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd024009860 tx=0x7fd02402a480 comp rx=0 tx=0).stop 2024-09-10T21:42:49.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.515+0000 7fd041800640 1 -- 172.21.15.7:0/3448891789 shutdown_connections 2024-09-10T21:42:49.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.515+0000 7fd041800640 1 --2- 172.21.15.7:0/3448891789 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd03c15cbe0 0x7fd03c073cd0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:49.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.515+0000 7fd041800640 1 -- 172.21.15.7:0/3448891789 >> 172.21.15.7:0/3448891789 conn(0x7fd03c0b1660 msgr2=0x7fd03c0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:49.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.515+0000 7fd041800640 1 -- 172.21.15.7:0/3448891789 shutdown_connections 2024-09-10T21:42:49.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.516+0000 7fd041800640 1 -- 172.21.15.7:0/3448891789 wait complete. 2024-09-10T21:42:49.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.517+0000 7fd041800640 1 Processor -- start 2024-09-10T21:42:49.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.517+0000 7fd041800640 1 -- start start 2024-09-10T21:42:49.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.518+0000 7fd041800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd03c06ff50 0x7fd03c0c3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:49.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.518+0000 7fd041800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd03c074210 0x7fd03c14e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:49.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.518+0000 7fd041800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd03c15cbe0 0x7fd03c0c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:49.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.518+0000 7fd041800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c0703e0 con 0x7fd03c15cbe0 2024-09-10T21:42:49.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.518+0000 7fd041800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd03c06f930 con 0x7fd03c074210 2024-09-10T21:42:49.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.518+0000 7fd041800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fd03c15fe20 con 0x7fd03c06ff50 2024-09-10T21:42:49.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.519+0000 7fd03b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd03c15cbe0 0x7fd03c0c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:49.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.519+0000 7fd03b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd03c15cbe0 0x7fd03c0c5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54918/0 (socket says 172.21.15.7:54918) 2024-09-10T21:42:49.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.519+0000 7fd03b400640 1 -- 172.21.15.7:0/2478007802 learned_addr learned my addr 172.21.15.7:0/2478007802 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:49.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.519+0000 7fd032000640 1 --2- 172.21.15.7:0/2478007802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd03c074210 0x7fd03c14e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:49.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.519+0000 7fd03aa00640 1 --2- 172.21.15.7:0/2478007802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd03c06ff50 0x7fd03c0c3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:49.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.519+0000 7fd03b400640 1 -- 172.21.15.7:0/2478007802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd03c06ff50 msgr2=0x7fd03c0c3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:49.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.519+0000 7fd03b400640 1 --2- 172.21.15.7:0/2478007802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd03c06ff50 0x7fd03c0c3f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:49.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.519+0000 7fd03b400640 1 -- 172.21.15.7:0/2478007802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd03c074210 msgr2=0x7fd03c14e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:49.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.519+0000 7fd03b400640 1 --2- 172.21.15.7:0/2478007802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd03c074210 0x7fd03c14e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:49.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.519+0000 7fd03b400640 1 -- 172.21.15.7:0/2478007802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03c0c5ff0 con 0x7fd03c15cbe0 2024-09-10T21:42:49.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.519+0000 7fd03aa00640 1 --2- 172.21.15.7:0/2478007802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd03c06ff50 0x7fd03c0c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:49.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.519+0000 7fd03b400640 1 --2- 172.21.15.7:0/2478007802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd03c15cbe0 0x7fd03c0c5230 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd03400caf0 tx=0x7fd03400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:49.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.520+0000 7fd033400640 1 -- 172.21.15.7:0/2478007802 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034016020 con 0x7fd03c15cbe0 2024-09-10T21:42:49.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.520+0000 7fd033400640 1 -- 172.21.15.7:0/2478007802 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd03400adb0 con 0x7fd03c15cbe0 2024-09-10T21:42:49.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.520+0000 7fd033400640 1 -- 172.21.15.7:0/2478007802 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd034012680 con 0x7fd03c15cbe0 2024-09-10T21:42:49.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.520+0000 7fd041800640 1 -- 172.21.15.7:0/2478007802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd03c0b5840 con 0x7fd03c15cbe0 2024-09-10T21:42:49.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.520+0000 7fd041800640 1 -- 172.21.15.7:0/2478007802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd03c0b5560 con 0x7fd03c15cbe0 2024-09-10T21:42:49.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.521+0000 7fd041800640 1 -- 172.21.15.7:0/2478007802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd03c160270 con 0x7fd03c15cbe0 2024-09-10T21:42:49.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.522+0000 7fd033400640 1 -- 172.21.15.7:0/2478007802 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd034012820 con 0x7fd03c15cbe0 2024-09-10T21:42:49.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.523+0000 7fd033400640 1 --2- 172.21.15.7:0/2478007802 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd018077fa0 0x7fd01807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:49.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.523+0000 7fd033400640 1 -- 172.21.15.7:0/2478007802 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fd03409a2d0 con 0x7fd03c15cbe0 2024-09-10T21:42:49.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.524+0000 7fd03aa00640 1 --2- 172.21.15.7:0/2478007802 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd018077fa0 0x7fd01807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:49.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.524+0000 7fd03aa00640 1 --2- 172.21.15.7:0/2478007802 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd018077fa0 0x7fd01807a460 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd02c001710 tx=0x7fd02c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:49.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.528+0000 7fd033400640 1 -- 172.21.15.7:0/2478007802 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd034020020 con 0x7fd03c15cbe0 2024-09-10T21:42:49.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.797+0000 7fd041800640 1 -- 172.21.15.7:0/2478007802 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7fd03c050fc0 con 0x7fd03c15cbe0 2024-09-10T21:42:49.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.800+0000 7fd033400640 1 -- 172.21.15.7:0/2478007802 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v26) ==== 90+0+0 (secure 0 0 0) 0x7fd03c0c5ff0 con 0x7fd03c15cbe0 2024-09-10T21:42:49.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.802+0000 7fd041800640 1 -- 172.21.15.7:0/2478007802 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd018077fa0 msgr2=0x7fd01807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:49.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.802+0000 7fd041800640 1 --2- 172.21.15.7:0/2478007802 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd018077fa0 0x7fd01807a460 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd02c001710 tx=0x7fd02c009040 comp rx=0 tx=0).stop 2024-09-10T21:42:49.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.802+0000 7fd041800640 1 -- 172.21.15.7:0/2478007802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd03c15cbe0 msgr2=0x7fd03c0c5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:49.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.802+0000 7fd041800640 1 --2- 172.21.15.7:0/2478007802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd03c15cbe0 0x7fd03c0c5230 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd03400caf0 tx=0x7fd03400a060 comp rx=0 tx=0).stop 2024-09-10T21:42:49.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.803+0000 7fd041800640 1 -- 172.21.15.7:0/2478007802 shutdown_connections 2024-09-10T21:42:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.803+0000 7fd041800640 1 --2- 172.21.15.7:0/2478007802 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd018077fa0 0x7fd01807a460 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.803+0000 7fd041800640 1 --2- 172.21.15.7:0/2478007802 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd03c074210 0x7fd03c14e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.803+0000 7fd041800640 1 --2- 172.21.15.7:0/2478007802 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd03c15cbe0 0x7fd03c0c5230 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.803+0000 7fd041800640 1 --2- 172.21.15.7:0/2478007802 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd03c06ff50 0x7fd03c0c3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.803+0000 7fd041800640 1 -- 172.21.15.7:0/2478007802 >> 172.21.15.7:0/2478007802 conn(0x7fd03c0b1660 msgr2=0x7fd03c0b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.803+0000 7fd041800640 1 -- 172.21.15.7:0/2478007802 shutdown_connections 2024-09-10T21:42:49.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.803+0000 7fd041800640 1 -- 172.21.15.7:0/2478007802 wait complete. 2024-09-10T21:42:49.820 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-10T21:42:49.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.982+0000 7f821ae00640 1 -- 172.21.15.7:0/996146581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f821415bc20 msgr2=0x7f82140ad8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:49.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.982+0000 7f821ae00640 1 --2- 172.21.15.7:0/996146581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f821415bc20 0x7f82140ad8f0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8214070440 tx=0x7f820402a560 comp rx=0 tx=0).stop 2024-09-10T21:42:49.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.983+0000 7f821ae00640 1 -- 172.21.15.7:0/996146581 shutdown_connections 2024-09-10T21:42:49.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.983+0000 7f821ae00640 1 --2- 172.21.15.7:0/996146581 >> v2:172.21.15.133:3300/0 conn(0x7f82140cc520 0x7f82140cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:49.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.983+0000 7f821ae00640 1 --2- 172.21.15.7:0/996146581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f821415bc20 0x7f82140ad8f0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:49.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.983+0000 7f821ae00640 1 -- 172.21.15.7:0/996146581 >> 172.21.15.7:0/996146581 conn(0x7f82140b1660 msgr2=0x7f82140b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:49.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.983+0000 7f821ae00640 1 -- 172.21.15.7:0/996146581 shutdown_connections 2024-09-10T21:42:49.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.983+0000 7f821ae00640 1 -- 172.21.15.7:0/996146581 wait complete. 2024-09-10T21:42:49.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.984+0000 7f821ae00640 1 Processor -- start 2024-09-10T21:42:49.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.984+0000 7f821ae00640 1 -- start start 2024-09-10T21:42:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.985+0000 7f821ae00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82140cc520 0x7f8214157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.985+0000 7f821ae00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f821415bc20 0x7f82141600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.985+0000 7f821ae00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821415e470 0x7f82140b2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.985+0000 7f821ae00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f821406f150 con 0x7f82140cc520 2024-09-10T21:42:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.985+0000 7f821ae00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f821406efd0 con 0x7f821415e470 2024-09-10T21:42:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.985+0000 7f821ae00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f821406f2d0 con 0x7f821415bc20 2024-09-10T21:42:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.985+0000 7f8213e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82140cc520 0x7f8214157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.986+0000 7f8213e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82140cc520 0x7f8214157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54942/0 (socket says 172.21.15.7:54942) 2024-09-10T21:42:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.986+0000 7f8213e00640 1 -- 172.21.15.7:0/2197240415 learned_addr learned my addr 172.21.15.7:0/2197240415 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.986+0000 7f8213e00640 1 -- 172.21.15.7:0/2197240415 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f821415bc20 msgr2=0x7f82141600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.986+0000 7f8213e00640 1 --2- 172.21.15.7:0/2197240415 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f821415bc20 0x7f82141600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.986+0000 7f8218e00640 1 --2- 172.21.15.7:0/2197240415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821415e470 0x7f82140b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.986+0000 7f8213e00640 1 -- 172.21.15.7:0/2197240415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821415e470 msgr2=0x7f82140b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.986+0000 7f8213e00640 1 --2- 172.21.15.7:0/2197240415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821415e470 0x7f82140b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.986+0000 7f8213e00640 1 -- 172.21.15.7:0/2197240415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82140b3740 con 0x7f82140cc520 2024-09-10T21:42:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.986+0000 7f8218e00640 1 --2- 172.21.15.7:0/2197240415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821415e470 0x7f82140b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.986+0000 7f8213e00640 1 --2- 172.21.15.7:0/2197240415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82140cc520 0x7f8214157de0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f820402ab50 tx=0x7f82040039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.986+0000 7f820be00640 1 -- 172.21.15.7:0/2197240415 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f820403f070 con 0x7f82140cc520 2024-09-10T21:42:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.987+0000 7f820be00640 1 -- 172.21.15.7:0/2197240415 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8204004890 con 0x7f82140cc520 2024-09-10T21:42:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.987+0000 7f820be00640 1 -- 172.21.15.7:0/2197240415 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8204035030 con 0x7f82140cc520 2024-09-10T21:42:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.987+0000 7f821ae00640 1 -- 172.21.15.7:0/2197240415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82140b39e0 con 0x7f82140cc520 2024-09-10T21:42:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.987+0000 7f821ae00640 1 -- 172.21.15.7:0/2197240415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82140b3fa0 con 0x7f82140cc520 2024-09-10T21:42:49.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.988+0000 7f821ae00640 1 -- 172.21.15.7:0/2197240415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82140c7d40 con 0x7f82140cc520 2024-09-10T21:42:49.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.989+0000 7f820be00640 1 -- 172.21.15.7:0/2197240415 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8204004ce0 con 0x7f82140cc520 2024-09-10T21:42:49.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.990+0000 7f820be00640 1 --2- 172.21.15.7:0/2197240415 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f81f4077f70 0x7f81f407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:49.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.990+0000 7f820be00640 1 -- 172.21.15.7:0/2197240415 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f82040bab80 con 0x7f82140cc520 2024-09-10T21:42:49.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.990+0000 7f8213400640 1 --2- 172.21.15.7:0/2197240415 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f81f4077f70 0x7f81f407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:49.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.991+0000 7f8213400640 1 --2- 172.21.15.7:0/2197240415 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f81f4077f70 0x7f81f407a430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8200000f60 tx=0x7f82000079b0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:49.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:49.994+0000 7f820be00640 1 -- 172.21.15.7:0/2197240415 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f82040839c0 con 0x7f82140cc520 2024-09-10T21:42:50.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.261+0000 7f821ae00640 1 -- 172.21.15.7:0/2197240415 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f82140cdb10 con 0x7f82140cc520 2024-09-10T21:42:50.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.264+0000 7f820be00640 1 -- 172.21.15.7:0/2197240415 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v27) ==== 90+0+0 (secure 0 0 0) 0x7f82040832c0 con 0x7f82140cc520 2024-09-10T21:42:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.266+0000 7f821ae00640 1 -- 172.21.15.7:0/2197240415 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f81f4077f70 msgr2=0x7f81f407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.266+0000 7f821ae00640 1 --2- 172.21.15.7:0/2197240415 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f81f4077f70 0x7f81f407a430 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8200000f60 tx=0x7f82000079b0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.266+0000 7f821ae00640 1 -- 172.21.15.7:0/2197240415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82140cc520 msgr2=0x7f8214157de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.266+0000 7f821ae00640 1 --2- 172.21.15.7:0/2197240415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82140cc520 0x7f8214157de0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f820402ab50 tx=0x7f82040039e0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.267+0000 7f821ae00640 1 -- 172.21.15.7:0/2197240415 shutdown_connections 2024-09-10T21:42:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.267+0000 7f821ae00640 1 --2- 172.21.15.7:0/2197240415 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f81f4077f70 0x7f81f407a430 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.267+0000 7f821ae00640 1 --2- 172.21.15.7:0/2197240415 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f821415e470 0x7f82140b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.267+0000 7f821ae00640 1 --2- 172.21.15.7:0/2197240415 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f821415bc20 0x7f82141600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.267+0000 7f821ae00640 1 --2- 172.21.15.7:0/2197240415 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82140cc520 0x7f8214157de0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.267+0000 7f821ae00640 1 -- 172.21.15.7:0/2197240415 >> 172.21.15.7:0/2197240415 conn(0x7f82140b1660 msgr2=0x7f82140ca960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.267+0000 7f821ae00640 1 -- 172.21.15.7:0/2197240415 shutdown_connections 2024-09-10T21:42:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.267+0000 7f821ae00640 1 -- 172.21.15.7:0/2197240415 wait complete. 2024-09-10T21:42:50.283 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-10T21:42:50.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.442+0000 7f2f67000640 1 -- 172.21.15.7:0/4067496004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f6006e960 msgr2=0x7f2f6015f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:50.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.442+0000 7f2f67000640 1 --2- 172.21.15.7:0/4067496004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f6006e960 0x7f2f6015f710 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2f600b2620 tx=0x7f2f5802a9a0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.443+0000 7f2f67000640 1 -- 172.21.15.7:0/4067496004 shutdown_connections 2024-09-10T21:42:50.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.443+0000 7f2f67000640 1 --2- 172.21.15.7:0/4067496004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f6006e960 0x7f2f6015f710 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.443+0000 7f2f67000640 1 --2- 172.21.15.7:0/4067496004 >> v2:172.21.15.133:3300/0 conn(0x7f2f6006ff50 0x7f2f6015cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.443+0000 7f2f67000640 1 -- 172.21.15.7:0/4067496004 >> 172.21.15.7:0/4067496004 conn(0x7f2f600b1660 msgr2=0x7f2f600b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:50.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.443+0000 7f2f67000640 1 -- 172.21.15.7:0/4067496004 shutdown_connections 2024-09-10T21:42:50.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.443+0000 7f2f67000640 1 -- 172.21.15.7:0/4067496004 wait complete. 2024-09-10T21:42:50.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.444+0000 7f2f67000640 1 Processor -- start 2024-09-10T21:42:50.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.444+0000 7f2f67000640 1 -- start start 2024-09-10T21:42:50.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.445+0000 7f2f67000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f6006e510 0x7f2f601f4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:50.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.445+0000 7f2f67000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f6006e960 0x7f2f600c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:50.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.445+0000 7f2f67000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f6006ff50 0x7f2f601f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:50.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.445+0000 7f2f67000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2f6015fc50 con 0x7f2f6006ff50 2024-09-10T21:42:50.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.445+0000 7f2f67000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2f600745f0 con 0x7f2f6006e960 2024-09-10T21:42:50.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.445+0000 7f2f67000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f2f6015fdd0 con 0x7f2f6006e510 2024-09-10T21:42:50.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.445+0000 7f2f65000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f6006ff50 0x7f2f601f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:50.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.446+0000 7f2f65000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f6006ff50 0x7f2f601f7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54958/0 (socket says 172.21.15.7:54958) 2024-09-10T21:42:50.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.446+0000 7f2f65000640 1 -- 172.21.15.7:0/682409350 learned_addr learned my addr 172.21.15.7:0/682409350 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:50.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.446+0000 7f2f5fe00640 1 --2- 172.21.15.7:0/682409350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f6006e510 0x7f2f601f4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:50.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.446+0000 7f2f65000640 1 -- 172.21.15.7:0/682409350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f6006e510 msgr2=0x7f2f601f4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:50.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.446+0000 7f2f5f400640 1 --2- 172.21.15.7:0/682409350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f6006e960 0x7f2f600c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:50.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.446+0000 7f2f65000640 1 --2- 172.21.15.7:0/682409350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f6006e510 0x7f2f601f4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.446+0000 7f2f65000640 1 -- 172.21.15.7:0/682409350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f6006e960 msgr2=0x7f2f600c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:50.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.446+0000 7f2f65000640 1 --2- 172.21.15.7:0/682409350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f6006e960 0x7f2f600c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.446+0000 7f2f65000640 1 -- 172.21.15.7:0/682409350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f601f5510 con 0x7f2f6006ff50 2024-09-10T21:42:50.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.446+0000 7f2f5fe00640 1 --2- 172.21.15.7:0/682409350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f6006e510 0x7f2f601f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:50.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.446+0000 7f2f5f400640 1 --2- 172.21.15.7:0/682409350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f6006e960 0x7f2f600c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:50.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.446+0000 7f2f65000640 1 --2- 172.21.15.7:0/682409350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f6006ff50 0x7f2f601f7bb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2f58009d60 tx=0x7f2f58003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:50.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.447+0000 7f2f57e00640 1 -- 172.21.15.7:0/682409350 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58047070 con 0x7f2f6006ff50 2024-09-10T21:42:50.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.447+0000 7f2f67000640 1 -- 172.21.15.7:0/682409350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f600c75c0 con 0x7f2f6006ff50 2024-09-10T21:42:50.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.447+0000 7f2f67000640 1 -- 172.21.15.7:0/682409350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f601f8870 con 0x7f2f6006ff50 2024-09-10T21:42:50.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.447+0000 7f2f57e00640 1 -- 172.21.15.7:0/682409350 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f5802f600 con 0x7f2f6006ff50 2024-09-10T21:42:50.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.447+0000 7f2f57e00640 1 -- 172.21.15.7:0/682409350 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58033050 con 0x7f2f6006ff50 2024-09-10T21:42:50.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.448+0000 7f2f67000640 1 -- 172.21.15.7:0/682409350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f601602f0 con 0x7f2f6006ff50 2024-09-10T21:42:50.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.449+0000 7f2f57e00640 1 -- 172.21.15.7:0/682409350 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2f58004490 con 0x7f2f6006ff50 2024-09-10T21:42:50.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.450+0000 7f2f57e00640 1 --2- 172.21.15.7:0/682409350 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f40077f70 0x7f2f4007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:50.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.450+0000 7f2f57e00640 1 -- 172.21.15.7:0/682409350 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f2f580bab60 con 0x7f2f6006ff50 2024-09-10T21:42:50.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.450+0000 7f2f5fe00640 1 --2- 172.21.15.7:0/682409350 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f40077f70 0x7f2f4007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:50.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.451+0000 7f2f5fe00640 1 --2- 172.21.15.7:0/682409350 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f40077f70 0x7f2f4007a430 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2f50005d00 tx=0x7f2f50005db0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:50.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.454+0000 7f2f57e00640 1 -- 172.21.15.7:0/682409350 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2f601f5510 con 0x7f2f6006ff50 2024-09-10T21:42:50.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.720+0000 7f2f67000640 1 -- 172.21.15.7:0/682409350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7f2f600ccb20 con 0x7f2f6006ff50 2024-09-10T21:42:50.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.722+0000 7f2f57e00640 1 -- 172.21.15.7:0/682409350 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v28) ==== 90+0+0 (secure 0 0 0) 0x7f2f5808c0c0 con 0x7f2f6006ff50 2024-09-10T21:42:50.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.725+0000 7f2f67000640 1 -- 172.21.15.7:0/682409350 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f40077f70 msgr2=0x7f2f4007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:50.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.725+0000 7f2f67000640 1 --2- 172.21.15.7:0/682409350 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f40077f70 0x7f2f4007a430 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2f50005d00 tx=0x7f2f50005db0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.725+0000 7f2f67000640 1 -- 172.21.15.7:0/682409350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f6006ff50 msgr2=0x7f2f601f7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.725+0000 7f2f67000640 1 --2- 172.21.15.7:0/682409350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f6006ff50 0x7f2f601f7bb0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2f58009d60 tx=0x7f2f58003ef0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.725+0000 7f2f67000640 1 -- 172.21.15.7:0/682409350 shutdown_connections 2024-09-10T21:42:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.725+0000 7f2f67000640 1 --2- 172.21.15.7:0/682409350 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f40077f70 0x7f2f4007a430 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.725+0000 7f2f67000640 1 --2- 172.21.15.7:0/682409350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f6006e960 0x7f2f600c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.725+0000 7f2f67000640 1 --2- 172.21.15.7:0/682409350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f6006ff50 0x7f2f601f7bb0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.725+0000 7f2f67000640 1 --2- 172.21.15.7:0/682409350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f6006e510 0x7f2f601f4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.725+0000 7f2f67000640 1 -- 172.21.15.7:0/682409350 >> 172.21.15.7:0/682409350 conn(0x7f2f600b1660 msgr2=0x7f2f6015efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.725+0000 7f2f67000640 1 -- 172.21.15.7:0/682409350 shutdown_connections 2024-09-10T21:42:50.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.725+0000 7f2f67000640 1 -- 172.21.15.7:0/682409350 wait complete. 2024-09-10T21:42:50.742 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-10T21:42:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.904+0000 7fddb9600640 1 -- 172.21.15.7:0/2307803610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb40d0e70 msgr2=0x7fddb40d4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.904+0000 7fddb9600640 1 --2- 172.21.15.7:0/2307803610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb40d0e70 0x7fddb40d4870 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fddb40b2a60 tx=0x7fdda002a560 comp rx=0 tx=0).stop 2024-09-10T21:42:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.904+0000 7fddb9600640 1 -- 172.21.15.7:0/2307803610 shutdown_connections 2024-09-10T21:42:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.904+0000 7fddb9600640 1 --2- 172.21.15.7:0/2307803610 >> v2:172.21.15.133:3300/0 conn(0x7fddb40b2b50 0x7fddb40d1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.904+0000 7fddb9600640 1 --2- 172.21.15.7:0/2307803610 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb40d0e70 0x7fddb40d4870 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.904+0000 7fddb9600640 1 -- 172.21.15.7:0/2307803610 >> 172.21.15.7:0/2307803610 conn(0x7fddb40d3d50 msgr2=0x7fddb40d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.904+0000 7fddb9600640 1 -- 172.21.15.7:0/2307803610 shutdown_connections 2024-09-10T21:42:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.904+0000 7fddb9600640 1 -- 172.21.15.7:0/2307803610 wait complete. 2024-09-10T21:42:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.906+0000 7fddb9600640 1 Processor -- start 2024-09-10T21:42:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.906+0000 7fddb9600640 1 -- start start 2024-09-10T21:42:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.907+0000 7fddb9600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddb40b2b50 0x7fddb40c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.907+0000 7fddb9600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb40d0e70 0x7fddb40cd880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.907+0000 7fddb9600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddb415dfd0 0x7fddb41f7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.907+0000 7fddb9600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fddb40d25a0 con 0x7fddb40d0e70 2024-09-10T21:42:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.907+0000 7fddb9600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fddb40d2420 con 0x7fddb40b2b50 2024-09-10T21:42:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.907+0000 7fddb9600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fddb40d2720 con 0x7fddb415dfd0 2024-09-10T21:42:50.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.907+0000 7fddb3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb40d0e70 0x7fddb40cd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:50.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.907+0000 7fddb3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb40d0e70 0x7fddb40cd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54964/0 (socket says 172.21.15.7:54964) 2024-09-10T21:42:50.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.907+0000 7fddb3400640 1 -- 172.21.15.7:0/64628180 learned_addr learned my addr 172.21.15.7:0/64628180 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:50.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.907+0000 7fddb3e00640 1 --2- 172.21.15.7:0/64628180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddb40b2b50 0x7fddb40c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:50.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.907+0000 7fddb8c00640 1 --2- 172.21.15.7:0/64628180 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddb415dfd0 0x7fddb41f7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:50.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.907+0000 7fddb3400640 1 -- 172.21.15.7:0/64628180 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddb415dfd0 msgr2=0x7fddb41f7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:50.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.907+0000 7fddb3400640 1 --2- 172.21.15.7:0/64628180 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddb415dfd0 0x7fddb41f7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.907+0000 7fddb3400640 1 -- 172.21.15.7:0/64628180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddb40b2b50 msgr2=0x7fddb40c8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:50.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.907+0000 7fddb3400640 1 --2- 172.21.15.7:0/64628180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddb40b2b50 0x7fddb40c8260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:50.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.907+0000 7fddb3400640 1 -- 172.21.15.7:0/64628180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddb40c87a0 con 0x7fddb40d0e70 2024-09-10T21:42:50.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.908+0000 7fddb3400640 1 --2- 172.21.15.7:0/64628180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb40d0e70 0x7fddb40cd880 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdda0009850 tx=0x7fdda0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:50.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.908+0000 7fddabe00640 1 -- 172.21.15.7:0/64628180 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda003f070 con 0x7fddb40d0e70 2024-09-10T21:42:50.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.908+0000 7fddabe00640 1 -- 172.21.15.7:0/64628180 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdda00047a0 con 0x7fddb40d0e70 2024-09-10T21:42:50.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.908+0000 7fddabe00640 1 -- 172.21.15.7:0/64628180 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdda0035030 con 0x7fddb40d0e70 2024-09-10T21:42:50.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.908+0000 7fddb9600640 1 -- 172.21.15.7:0/64628180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddb40c8a70 con 0x7fddb40d0e70 2024-09-10T21:42:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.908+0000 7fddb9600640 1 -- 172.21.15.7:0/64628180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddb40c9030 con 0x7fddb40d0e70 2024-09-10T21:42:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.910+0000 7fddb9600640 1 -- 172.21.15.7:0/64628180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddb40d1ee0 con 0x7fddb40d0e70 2024-09-10T21:42:50.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.911+0000 7fddabe00640 1 -- 172.21.15.7:0/64628180 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fdda0004bb0 con 0x7fddb40d0e70 2024-09-10T21:42:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.911+0000 7fddabe00640 1 --2- 172.21.15.7:0/64628180 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdd94077f70 0x7fdd9407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:50.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.912+0000 7fddabe00640 1 -- 172.21.15.7:0/64628180 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fdda00bab40 con 0x7fddb40d0e70 2024-09-10T21:42:50.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.912+0000 7fddb3e00640 1 --2- 172.21.15.7:0/64628180 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdd94077f70 0x7fdd9407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:50.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.913+0000 7fddb3e00640 1 --2- 172.21.15.7:0/64628180 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdd94077f70 0x7fdd9407a430 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdda4001720 tx=0x7fdda4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:50.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:50.916+0000 7fddabe00640 1 -- 172.21.15.7:0/64628180 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fdda0083340 con 0x7fddb40d0e70 2024-09-10T21:42:51.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.181+0000 7fddb9600640 1 -- 172.21.15.7:0/64628180 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7fddb406e030 con 0x7fddb40d0e70 2024-09-10T21:42:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.184+0000 7fddabe00640 1 -- 172.21.15.7:0/64628180 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v29) ==== 90+0+0 (secure 0 0 0) 0x7fdda0087d70 con 0x7fddb40d0e70 2024-09-10T21:42:51.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.186+0000 7fddb9600640 1 -- 172.21.15.7:0/64628180 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdd94077f70 msgr2=0x7fdd9407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:51.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.186+0000 7fddb9600640 1 --2- 172.21.15.7:0/64628180 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdd94077f70 0x7fdd9407a430 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdda4001720 tx=0x7fdda4009040 comp rx=0 tx=0).stop 2024-09-10T21:42:51.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.186+0000 7fddb9600640 1 -- 172.21.15.7:0/64628180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb40d0e70 msgr2=0x7fddb40cd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:51.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.186+0000 7fddb9600640 1 --2- 172.21.15.7:0/64628180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb40d0e70 0x7fddb40cd880 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fdda0009850 tx=0x7fdda0003910 comp rx=0 tx=0).stop 2024-09-10T21:42:51.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.186+0000 7fddb9600640 1 -- 172.21.15.7:0/64628180 shutdown_connections 2024-09-10T21:42:51.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.186+0000 7fddb9600640 1 --2- 172.21.15.7:0/64628180 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdd94077f70 0x7fdd9407a430 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:51.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.186+0000 7fddb9600640 1 --2- 172.21.15.7:0/64628180 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddb415dfd0 0x7fddb41f7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:51.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.186+0000 7fddb9600640 1 --2- 172.21.15.7:0/64628180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddb40d0e70 0x7fddb40cd880 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:51.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.186+0000 7fddb9600640 1 --2- 172.21.15.7:0/64628180 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddb40b2b50 0x7fddb40c8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:51.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.186+0000 7fddb9600640 1 -- 172.21.15.7:0/64628180 >> 172.21.15.7:0/64628180 conn(0x7fddb40d3d50 msgr2=0x7fddb406e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:51.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.186+0000 7fddb9600640 1 -- 172.21.15.7:0/64628180 shutdown_connections 2024-09-10T21:42:51.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.186+0000 7fddb9600640 1 -- 172.21.15.7:0/64628180 wait complete. 2024-09-10T21:42:51.204 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-10T21:42:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.362+0000 7fe778a00640 1 -- 172.21.15.7:0/1962870763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe77415bc20 msgr2=0x7fe7740ad8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.362+0000 7fe778a00640 1 --2- 172.21.15.7:0/1962870763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe77415bc20 0x7fe7740ad8f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fe764009830 tx=0x7fe76402a530 comp rx=0 tx=0).stop 2024-09-10T21:42:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.363+0000 7fe778a00640 1 -- 172.21.15.7:0/1962870763 shutdown_connections 2024-09-10T21:42:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.363+0000 7fe778a00640 1 --2- 172.21.15.7:0/1962870763 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe77415bc20 0x7fe7740ad8f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.363+0000 7fe778a00640 1 -- 172.21.15.7:0/1962870763 >> 172.21.15.7:0/1962870763 conn(0x7fe7740b1660 msgr2=0x7fe7740b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.363+0000 7fe778a00640 1 -- 172.21.15.7:0/1962870763 shutdown_connections 2024-09-10T21:42:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.363+0000 7fe778a00640 1 -- 172.21.15.7:0/1962870763 wait complete. 2024-09-10T21:42:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.364+0000 7fe778a00640 1 Processor -- start 2024-09-10T21:42:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.364+0000 7fe778a00640 1 -- start start 2024-09-10T21:42:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.365+0000 7fe778a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7740cc520 0x7fe774160f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.365+0000 7fe778a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe77415bc20 0x7fe7741f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.365+0000 7fe778a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe77415e470 0x7fe7740b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.365+0000 7fe778a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe7740ccd00 con 0x7fe7740cc520 2024-09-10T21:42:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.365+0000 7fe778a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe7740ccb80 con 0x7fe77415bc20 2024-09-10T21:42:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.365+0000 7fe778a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe7740cce80 con 0x7fe77415e470 2024-09-10T21:42:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.366+0000 7fe771e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7740cc520 0x7fe774160f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.366+0000 7fe771e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7740cc520 0x7fe774160f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54980/0 (socket says 172.21.15.7:54980) 2024-09-10T21:42:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.366+0000 7fe771e00640 1 -- 172.21.15.7:0/2082513417 learned_addr learned my addr 172.21.15.7:0/2082513417 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.366+0000 7fe769400640 1 --2- 172.21.15.7:0/2082513417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe77415bc20 0x7fe7741f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.366+0000 7fe772800640 1 --2- 172.21.15.7:0/2082513417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe77415e470 0x7fe7740b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.366+0000 7fe771e00640 1 -- 172.21.15.7:0/2082513417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe77415e470 msgr2=0x7fe7740b5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.366+0000 7fe771e00640 1 --2- 172.21.15.7:0/2082513417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe77415e470 0x7fe7740b5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.366+0000 7fe771e00640 1 -- 172.21.15.7:0/2082513417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe77415bc20 msgr2=0x7fe7741f53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.366+0000 7fe771e00640 1 --2- 172.21.15.7:0/2082513417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe77415bc20 0x7fe7741f53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.366+0000 7fe771e00640 1 -- 172.21.15.7:0/2082513417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7741615c0 con 0x7fe7740cc520 2024-09-10T21:42:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.366+0000 7fe769400640 1 --2- 172.21.15.7:0/2082513417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe77415bc20 0x7fe7741f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.366+0000 7fe771e00640 1 --2- 172.21.15.7:0/2082513417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7740cc520 0x7fe774160f00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe764009950 tx=0x7fe764032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.366+0000 7fe76aa00640 1 -- 172.21.15.7:0/2082513417 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe764040070 con 0x7fe7740cc520 2024-09-10T21:42:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.367+0000 7fe76aa00640 1 -- 172.21.15.7:0/2082513417 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe764005310 con 0x7fe7740cc520 2024-09-10T21:42:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.367+0000 7fe76aa00640 1 -- 172.21.15.7:0/2082513417 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7640047a0 con 0x7fe7740cc520 2024-09-10T21:42:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.367+0000 7fe778a00640 1 -- 172.21.15.7:0/2082513417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7741618a0 con 0x7fe7740cc520 2024-09-10T21:42:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.367+0000 7fe778a00640 1 -- 172.21.15.7:0/2082513417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7740c8ae0 con 0x7fe7740cc520 2024-09-10T21:42:51.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.368+0000 7fe778a00640 1 -- 172.21.15.7:0/2082513417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe77406f7d0 con 0x7fe7740cc520 2024-09-10T21:42:51.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.369+0000 7fe76aa00640 1 -- 172.21.15.7:0/2082513417 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe764004940 con 0x7fe7740cc520 2024-09-10T21:42:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.370+0000 7fe76aa00640 1 --2- 172.21.15.7:0/2082513417 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe750077f70 0x7fe75007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.371+0000 7fe76aa00640 1 -- 172.21.15.7:0/2082513417 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7fe7640baf00 con 0x7fe7740cc520 2024-09-10T21:42:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.371+0000 7fe769400640 1 --2- 172.21.15.7:0/2082513417 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe750077f70 0x7fe75007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.372+0000 7fe769400640 1 --2- 172.21.15.7:0/2082513417 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe750077f70 0x7fe75007a430 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe75c007fd0 tx=0x7fe75c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:51.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.375+0000 7fe76aa00640 1 -- 172.21.15.7:0/2082513417 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe764083cb0 con 0x7fe7740cc520 2024-09-10T21:42:51.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.640+0000 7fe778a00640 1 -- 172.21.15.7:0/2082513417 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fe7740c9d30 con 0x7fe7740cc520 2024-09-10T21:42:51.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.643+0000 7fe76aa00640 1 -- 172.21.15.7:0/2082513417 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v30) ==== 97+0+0 (secure 0 0 0) 0x7fe7640835b0 con 0x7fe7740cc520 2024-09-10T21:42:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.645+0000 7fe778a00640 1 -- 172.21.15.7:0/2082513417 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe750077f70 msgr2=0x7fe75007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.645+0000 7fe778a00640 1 --2- 172.21.15.7:0/2082513417 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe750077f70 0x7fe75007a430 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe75c007fd0 tx=0x7fe75c009040 comp rx=0 tx=0).stop 2024-09-10T21:42:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.645+0000 7fe778a00640 1 -- 172.21.15.7:0/2082513417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7740cc520 msgr2=0x7fe774160f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.645+0000 7fe778a00640 1 --2- 172.21.15.7:0/2082513417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7740cc520 0x7fe774160f00 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe764009950 tx=0x7fe764032b90 comp rx=0 tx=0).stop 2024-09-10T21:42:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.646+0000 7fe778a00640 1 -- 172.21.15.7:0/2082513417 shutdown_connections 2024-09-10T21:42:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.646+0000 7fe778a00640 1 --2- 172.21.15.7:0/2082513417 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe750077f70 0x7fe75007a430 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:51.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.646+0000 7fe778a00640 1 --2- 172.21.15.7:0/2082513417 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe77415bc20 0x7fe7741f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:51.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.646+0000 7fe778a00640 1 --2- 172.21.15.7:0/2082513417 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe77415e470 0x7fe7740b5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:51.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.646+0000 7fe778a00640 1 --2- 172.21.15.7:0/2082513417 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe7740cc520 0x7fe774160f00 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:51.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.646+0000 7fe778a00640 1 -- 172.21.15.7:0/2082513417 >> 172.21.15.7:0/2082513417 conn(0x7fe7740b1660 msgr2=0x7fe7740aeb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:51.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.646+0000 7fe778a00640 1 -- 172.21.15.7:0/2082513417 shutdown_connections 2024-09-10T21:42:51.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.646+0000 7fe778a00640 1 -- 172.21.15.7:0/2082513417 wait complete. 2024-09-10T21:42:51.665 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-10T21:42:51.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.825+0000 7f280ea00640 1 -- 172.21.15.7:0/1545962978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f280815e040 msgr2=0x7f2808161e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:51.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.825+0000 7f280ea00640 1 --2- 172.21.15.7:0/1545962978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f280815e040 0x7f2808161e00 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f28080b1610 tx=0x7f27fc02a4b0 comp rx=0 tx=0).stop 2024-09-10T21:42:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.825+0000 7f280ea00640 1 -- 172.21.15.7:0/1545962978 shutdown_connections 2024-09-10T21:42:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.825+0000 7f280ea00640 1 --2- 172.21.15.7:0/1545962978 >> v2:172.21.15.133:3300/0 conn(0x7f28080d0d40 0x7f280806dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.825+0000 7f280ea00640 1 --2- 172.21.15.7:0/1545962978 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f280815e040 0x7f2808161e00 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.825+0000 7f280ea00640 1 -- 172.21.15.7:0/1545962978 >> 172.21.15.7:0/1545962978 conn(0x7f28080d1c10 msgr2=0x7f28080d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.825+0000 7f280ea00640 1 -- 172.21.15.7:0/1545962978 shutdown_connections 2024-09-10T21:42:51.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.825+0000 7f280ea00640 1 -- 172.21.15.7:0/1545962978 wait complete. 2024-09-10T21:42:51.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.826+0000 7f280ea00640 1 Processor -- start 2024-09-10T21:42:51.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.827+0000 7f280ea00640 1 -- start start 2024-09-10T21:42:51.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.827+0000 7f280ea00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f28080d0d40 0x7f28081f4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.827+0000 7f280ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f280815e040 0x7f280809f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.827+0000 7f280ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f280815e420 0x7f280809b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.828+0000 7f280ea00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f280806e170 con 0x7f280815e420 2024-09-10T21:42:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.828+0000 7f280ea00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2808072770 con 0x7f280815e040 2024-09-10T21:42:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.828+0000 7f280ea00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f280806e2f0 con 0x7f28080d0d40 2024-09-10T21:42:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.828+0000 7f280e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f280815e420 0x7f280809b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.828+0000 7f280e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f280815e420 0x7f280809b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54996/0 (socket says 172.21.15.7:54996) 2024-09-10T21:42:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.828+0000 7f280e000640 1 -- 172.21.15.7:0/628858730 learned_addr learned my addr 172.21.15.7:0/628858730 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.828+0000 7f280cc00640 1 --2- 172.21.15.7:0/628858730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f280815e040 0x7f280809f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.828+0000 7f280d600640 1 --2- 172.21.15.7:0/628858730 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f28080d0d40 0x7f28081f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.828+0000 7f280e000640 1 -- 172.21.15.7:0/628858730 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f28080d0d40 msgr2=0x7f28081f4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:51.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.828+0000 7f280e000640 1 --2- 172.21.15.7:0/628858730 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f28080d0d40 0x7f28081f4760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:51.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.828+0000 7f280e000640 1 -- 172.21.15.7:0/628858730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f280815e040 msgr2=0x7f280809f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:51.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.828+0000 7f280e000640 1 --2- 172.21.15.7:0/628858730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f280815e040 0x7f280809f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:51.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.828+0000 7f280e000640 1 -- 172.21.15.7:0/628858730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280815cf20 con 0x7f280815e420 2024-09-10T21:42:51.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.828+0000 7f280d600640 1 --2- 172.21.15.7:0/628858730 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f28080d0d40 0x7f28081f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:51.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.829+0000 7f280e000640 1 --2- 172.21.15.7:0/628858730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f280815e420 0x7f280809b9f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f280000eb10 tx=0x7f280000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.829+0000 7f2805600640 1 -- 172.21.15.7:0/628858730 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800004b20 con 0x7f280815e420 2024-09-10T21:42:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.829+0000 7f2805600640 1 -- 172.21.15.7:0/628858730 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2800004ce0 con 0x7f280815e420 2024-09-10T21:42:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.829+0000 7f2805600640 1 -- 172.21.15.7:0/628858730 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800005280 con 0x7f280815e420 2024-09-10T21:42:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.829+0000 7f280ea00640 1 -- 172.21.15.7:0/628858730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280809c0f0 con 0x7f280815e420 2024-09-10T21:42:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.829+0000 7f280ea00640 1 -- 172.21.15.7:0/628858730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f280809c6b0 con 0x7f280815e420 2024-09-10T21:42:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.830+0000 7f280ea00640 1 -- 172.21.15.7:0/628858730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f280806dc30 con 0x7f280815e420 2024-09-10T21:42:51.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.831+0000 7f2805600640 1 -- 172.21.15.7:0/628858730 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2800005420 con 0x7f280815e420 2024-09-10T21:42:51.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.832+0000 7f2805600640 1 --2- 172.21.15.7:0/628858730 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f27e8077fa0 0x7f27e807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:51.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.832+0000 7f2805600640 1 -- 172.21.15.7:0/628858730 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f280009a710 con 0x7f280815e420 2024-09-10T21:42:51.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.833+0000 7f280d600640 1 --2- 172.21.15.7:0/628858730 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f27e8077fa0 0x7f27e807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:51.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.833+0000 7f280d600640 1 --2- 172.21.15.7:0/628858730 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f27e8077fa0 0x7f27e807a460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f27fc034000 tx=0x7f27fc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:51.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:51.836+0000 7f2805600640 1 -- 172.21.15.7:0/628858730 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f28000634e0 con 0x7f280815e420 2024-09-10T21:42:52.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.106+0000 7f280ea00640 1 -- 172.21.15.7:0/628858730 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f28080b7b90 con 0x7f280815e420 2024-09-10T21:42:52.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.109+0000 7f2805600640 1 -- 172.21.15.7:0/628858730 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v31) ==== 90+0+0 (secure 0 0 0) 0x7f2800062de0 con 0x7f280815e420 2024-09-10T21:42:52.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.111+0000 7f280ea00640 1 -- 172.21.15.7:0/628858730 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f27e8077fa0 msgr2=0x7f27e807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:52.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.111+0000 7f280ea00640 1 --2- 172.21.15.7:0/628858730 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f27e8077fa0 0x7f27e807a460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f27fc034000 tx=0x7f27fc004730 comp rx=0 tx=0).stop 2024-09-10T21:42:52.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.111+0000 7f280ea00640 1 -- 172.21.15.7:0/628858730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f280815e420 msgr2=0x7f280809b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:52.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.111+0000 7f280ea00640 1 --2- 172.21.15.7:0/628858730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f280815e420 0x7f280809b9f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f280000eb10 tx=0x7f280000c2d0 comp rx=0 tx=0).stop 2024-09-10T21:42:52.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.112+0000 7f280ea00640 1 -- 172.21.15.7:0/628858730 shutdown_connections 2024-09-10T21:42:52.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.112+0000 7f280ea00640 1 --2- 172.21.15.7:0/628858730 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f27e8077fa0 0x7f27e807a460 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:52.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.112+0000 7f280ea00640 1 --2- 172.21.15.7:0/628858730 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f280815e040 0x7f280809f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:52.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.112+0000 7f280ea00640 1 --2- 172.21.15.7:0/628858730 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f280815e420 0x7f280809b9f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:52.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.112+0000 7f280ea00640 1 --2- 172.21.15.7:0/628858730 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f28080d0d40 0x7f28081f4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:52.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.112+0000 7f280ea00640 1 -- 172.21.15.7:0/628858730 >> 172.21.15.7:0/628858730 conn(0x7f28080d1c10 msgr2=0x7f28080b4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:52.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.112+0000 7f280ea00640 1 -- 172.21.15.7:0/628858730 shutdown_connections 2024-09-10T21:42:52.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.112+0000 7f280ea00640 1 -- 172.21.15.7:0/628858730 wait complete. 2024-09-10T21:42:52.132 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:42:52.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.290+0000 7f7099000640 1 -- 172.21.15.7:0/3933835002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70940cd750 msgr2=0x7f70940ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.290+0000 7f7099000640 1 --2- 172.21.15.7:0/3933835002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70940cd750 0x7f70940ccef0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f709415fa90 tx=0x7f707c02a530 comp rx=0 tx=0).stop 2024-09-10T21:42:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.290+0000 7f7099000640 1 -- 172.21.15.7:0/3933835002 shutdown_connections 2024-09-10T21:42:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.290+0000 7f7099000640 1 --2- 172.21.15.7:0/3933835002 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f709415fda0 0x7f70941601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.290+0000 7f7099000640 1 --2- 172.21.15.7:0/3933835002 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f70940cd750 0x7f70940ccef0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.290+0000 7f7099000640 1 -- 172.21.15.7:0/3933835002 >> 172.21.15.7:0/3933835002 conn(0x7f70940b16d0 msgr2=0x7f70940b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.290+0000 7f7099000640 1 -- 172.21.15.7:0/3933835002 shutdown_connections 2024-09-10T21:42:52.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.290+0000 7f7099000640 1 -- 172.21.15.7:0/3933835002 wait complete. 2024-09-10T21:42:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.292+0000 7f7099000640 1 Processor -- start 2024-09-10T21:42:52.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.292+0000 7f7099000640 1 -- start start 2024-09-10T21:42:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.293+0000 7f7099000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70940cd750 0x7f709415c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:52.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.293+0000 7f7099000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f709415bc90 0x7f70940cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.293+0000 7f7099000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f709415fda0 0x7f70940cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.293+0000 7f7099000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f709406fe70 con 0x7f709415fda0 2024-09-10T21:42:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.293+0000 7f7099000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f709406fcf0 con 0x7f70940cd750 2024-09-10T21:42:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.293+0000 7f7099000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f709406fff0 con 0x7f709415bc90 2024-09-10T21:42:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.293+0000 7f7092800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f709415fda0 0x7f70940cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.293+0000 7f7092800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f709415fda0 0x7f70940cb890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55020/0 (socket says 172.21.15.7:55020) 2024-09-10T21:42:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.293+0000 7f7092800640 1 -- 172.21.15.7:0/2552707987 learned_addr learned my addr 172.21.15.7:0/2552707987 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.293+0000 7f7091e00640 1 --2- 172.21.15.7:0/2552707987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70940cd750 0x7f709415c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.293+0000 7f7089400640 1 --2- 172.21.15.7:0/2552707987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f709415bc90 0x7f70940cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.293+0000 7f7092800640 1 -- 172.21.15.7:0/2552707987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f709415bc90 msgr2=0x7f70940cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:52.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.293+0000 7f7092800640 1 --2- 172.21.15.7:0/2552707987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f709415bc90 0x7f70940cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.293+0000 7f7092800640 1 -- 172.21.15.7:0/2552707987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70940cd750 msgr2=0x7f709415c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.293+0000 7f7092800640 1 --2- 172.21.15.7:0/2552707987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70940cd750 0x7f709415c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.293+0000 7f7092800640 1 -- 172.21.15.7:0/2552707987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f709414cbc0 con 0x7f709415fda0 2024-09-10T21:42:52.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.293+0000 7f7089400640 1 --2- 172.21.15.7:0/2552707987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f709415bc90 0x7f70940cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.294+0000 7f7092800640 1 --2- 172.21.15.7:0/2552707987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f709415fda0 0x7f70940cb890 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f708c00b9e0 tx=0x7f708c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.294+0000 7f708aa00640 1 -- 172.21.15.7:0/2552707987 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c019070 con 0x7f709415fda0 2024-09-10T21:42:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.294+0000 7f708aa00640 1 -- 172.21.15.7:0/2552707987 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f708c00cbb0 con 0x7f709415fda0 2024-09-10T21:42:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.294+0000 7f708aa00640 1 -- 172.21.15.7:0/2552707987 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f708c014440 con 0x7f709415fda0 2024-09-10T21:42:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.294+0000 7f7099000640 1 -- 172.21.15.7:0/2552707987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f709414cea0 con 0x7f709415fda0 2024-09-10T21:42:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.294+0000 7f7099000640 1 -- 172.21.15.7:0/2552707987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f709414d460 con 0x7f709415fda0 2024-09-10T21:42:52.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.295+0000 7f7099000640 1 -- 172.21.15.7:0/2552707987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70940704c0 con 0x7f709415fda0 2024-09-10T21:42:52.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.296+0000 7f708aa00640 1 -- 172.21.15.7:0/2552707987 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f708c01d3b0 con 0x7f709415fda0 2024-09-10T21:42:52.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.297+0000 7f708aa00640 1 --2- 172.21.15.7:0/2552707987 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7070077f70 0x7f707007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:52.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.297+0000 7f708aa00640 1 -- 172.21.15.7:0/2552707987 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f708c099a80 con 0x7f709415fda0 2024-09-10T21:42:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.298+0000 7f7091e00640 1 --2- 172.21.15.7:0/2552707987 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7070077f70 0x7f707007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:52.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.298+0000 7f7091e00640 1 --2- 172.21.15.7:0/2552707987 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7070077f70 0x7f707007a430 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7084001680 tx=0x7f7084009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:52.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.301+0000 7f708aa00640 1 -- 172.21.15.7:0/2552707987 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f708c09be20 con 0x7f709415fda0 2024-09-10T21:42:52.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.496+0000 7f7099000640 1 -- 172.21.15.7:0/2552707987 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7094069810 con 0x7f709415fda0 2024-09-10T21:42:52.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.497+0000 7f708aa00640 1 -- 172.21.15.7:0/2552707987 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v28) ==== 84+0+65 (secure 0 0 0) 0x7f708c062830 con 0x7f709415fda0 2024-09-10T21:42:52.499 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:52.499 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-09-10T21:42:52.499 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:42:52.500 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-09-10T21:42:52.500 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-09-10T21:42:52.500 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:42:52.500 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-09-10T21:42:52.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.499+0000 7f7099000640 1 -- 172.21.15.7:0/2552707987 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7070077f70 msgr2=0x7f707007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:52.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.499+0000 7f7099000640 1 --2- 172.21.15.7:0/2552707987 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7070077f70 0x7f707007a430 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7084001680 tx=0x7f7084009040 comp rx=0 tx=0).stop 2024-09-10T21:42:52.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.499+0000 7f7099000640 1 -- 172.21.15.7:0/2552707987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f709415fda0 msgr2=0x7f70940cb890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:52.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.499+0000 7f7099000640 1 --2- 172.21.15.7:0/2552707987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f709415fda0 0x7f70940cb890 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f708c00b9e0 tx=0x7f708c00bea0 comp rx=0 tx=0).stop 2024-09-10T21:42:52.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.499+0000 7f7099000640 1 -- 172.21.15.7:0/2552707987 shutdown_connections 2024-09-10T21:42:52.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.499+0000 7f7099000640 1 --2- 172.21.15.7:0/2552707987 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7070077f70 0x7f707007a430 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:52.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.499+0000 7f7099000640 1 --2- 172.21.15.7:0/2552707987 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f709415fda0 0x7f70940cb890 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:52.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.499+0000 7f7099000640 1 --2- 172.21.15.7:0/2552707987 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f709415bc90 0x7f70940cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:52.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.499+0000 7f7099000640 1 --2- 172.21.15.7:0/2552707987 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f70940cd750 0x7f709415c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:52.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.499+0000 7f7099000640 1 -- 172.21.15.7:0/2552707987 >> 172.21.15.7:0/2552707987 conn(0x7f70940b16d0 msgr2=0x7f70940b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:52.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.499+0000 7f7099000640 1 -- 172.21.15.7:0/2552707987 shutdown_connections 2024-09-10T21:42:52.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.499+0000 7f7099000640 1 -- 172.21.15.7:0/2552707987 wait complete. 2024-09-10T21:42:52.517 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:42:52.518 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:42:52.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.681+0000 7f4c60c00640 1 -- 172.21.15.7:0/4167804384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c5c0ace80 msgr2=0x7f4c5c070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:52.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.681+0000 7f4c60c00640 1 --2- 172.21.15.7:0/4167804384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c5c0ace80 0x7f4c5c070230 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4c44009860 tx=0x7f4c4402a4b0 comp rx=0 tx=0).stop 2024-09-10T21:42:52.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.681+0000 7f4c60c00640 1 -- 172.21.15.7:0/4167804384 shutdown_connections 2024-09-10T21:42:52.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.681+0000 7f4c60c00640 1 --2- 172.21.15.7:0/4167804384 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c5c15bce0 0x7f4c5c06cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:52.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.681+0000 7f4c60c00640 1 --2- 172.21.15.7:0/4167804384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c5c0ace80 0x7f4c5c070230 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:52.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.681+0000 7f4c60c00640 1 -- 172.21.15.7:0/4167804384 >> 172.21.15.7:0/4167804384 conn(0x7f4c5c0b1c10 msgr2=0x7f4c5c0b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:52.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.681+0000 7f4c60c00640 1 -- 172.21.15.7:0/4167804384 shutdown_connections 2024-09-10T21:42:52.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.681+0000 7f4c60c00640 1 -- 172.21.15.7:0/4167804384 wait complete. 2024-09-10T21:42:52.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.683+0000 7f4c60c00640 1 Processor -- start 2024-09-10T21:42:52.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.683+0000 7f4c60c00640 1 -- start start 2024-09-10T21:42:52.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.684+0000 7f4c60c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c5c0ace80 0x7f4c5c148de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:52.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.684+0000 7f4c60c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c5c15b900 0x7f4c5c147810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.684+0000 7f4c60c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c5c15bce0 0x7f4c5c0b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.684+0000 7f4c60c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c0aed00 con 0x7f4c5c0ace80 2024-09-10T21:42:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.684+0000 7f4c60c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4c5c0aeb80 con 0x7f4c5c15b900 2024-09-10T21:42:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.684+0000 7f4c60c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4c5c0aee80 con 0x7f4c5c15bce0 2024-09-10T21:42:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.684+0000 7f4c59e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c5c0ace80 0x7f4c5c148de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.684+0000 7f4c59e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c5c0ace80 0x7f4c5c148de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55022/0 (socket says 172.21.15.7:55022) 2024-09-10T21:42:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.684+0000 7f4c59e00640 1 -- 172.21.15.7:0/1884563109 learned_addr learned my addr 172.21.15.7:0/1884563109 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.685+0000 7f4c51400640 1 --2- 172.21.15.7:0/1884563109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c5c15b900 0x7f4c5c147810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.685+0000 7f4c5a800640 1 --2- 172.21.15.7:0/1884563109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c5c15bce0 0x7f4c5c0b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.685+0000 7f4c59e00640 1 -- 172.21.15.7:0/1884563109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c5c15bce0 msgr2=0x7f4c5c0b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.685+0000 7f4c59e00640 1 --2- 172.21.15.7:0/1884563109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c5c15bce0 0x7f4c5c0b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.685+0000 7f4c59e00640 1 -- 172.21.15.7:0/1884563109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c5c15b900 msgr2=0x7f4c5c147810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.685+0000 7f4c59e00640 1 --2- 172.21.15.7:0/1884563109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c5c15b900 0x7f4c5c147810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:52.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.685+0000 7f4c59e00640 1 -- 172.21.15.7:0/1884563109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c5c1f5cb0 con 0x7f4c5c0ace80 2024-09-10T21:42:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.685+0000 7f4c51400640 1 --2- 172.21.15.7:0/1884563109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c5c15b900 0x7f4c5c147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.685+0000 7f4c59e00640 1 --2- 172.21.15.7:0/1884563109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c5c0ace80 0x7f4c5c148de0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00c490 tx=0x7f4c4c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.685+0000 7f4c52a00640 1 -- 172.21.15.7:0/1884563109 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c016020 con 0x7f4c5c0ace80 2024-09-10T21:42:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.685+0000 7f4c52a00640 1 -- 172.21.15.7:0/1884563109 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c4c012940 con 0x7f4c5c0ace80 2024-09-10T21:42:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.685+0000 7f4c52a00640 1 -- 172.21.15.7:0/1884563109 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c012d90 con 0x7f4c5c0ace80 2024-09-10T21:42:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.685+0000 7f4c60c00640 1 -- 172.21.15.7:0/1884563109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c5c1f5f90 con 0x7f4c5c0ace80 2024-09-10T21:42:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.686+0000 7f4c60c00640 1 -- 172.21.15.7:0/1884563109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c5c1f6550 con 0x7f4c5c0ace80 2024-09-10T21:42:52.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.687+0000 7f4c60c00640 1 -- 172.21.15.7:0/1884563109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c5c0af3a0 con 0x7f4c5c0ace80 2024-09-10T21:42:52.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.688+0000 7f4c52a00640 1 -- 172.21.15.7:0/1884563109 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4c4c01a3b0 con 0x7f4c5c0ace80 2024-09-10T21:42:52.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.689+0000 7f4c52a00640 1 --2- 172.21.15.7:0/1884563109 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c38077f70 0x7f4c3807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:52.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.689+0000 7f4c52a00640 1 -- 172.21.15.7:0/1884563109 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 4857+0+0 (secure 0 0 0) 0x7f4c4c0997a0 con 0x7f4c5c0ace80 2024-09-10T21:42:52.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.689+0000 7f4c51400640 1 --2- 172.21.15.7:0/1884563109 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c38077f70 0x7f4c3807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:52.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.690+0000 7f4c51400640 1 --2- 172.21.15.7:0/1884563109 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c38077f70 0x7f4c3807a430 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4c44034000 tx=0x7f4c44004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:52.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.693+0000 7f4c52a00640 1 -- 172.21.15.7:0/1884563109 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4c4c020070 con 0x7f4c5c0ace80 2024-09-10T21:42:52.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:52.886+0000 7f4c60c00640 1 -- 172.21.15.7:0/1884563109 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f4c5c15f770 con 0x7f4c5c0ace80 2024-09-10T21:42:53.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:53.884+0000 7f4c52a00640 1 -- 172.21.15.7:0/1884563109 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v29) ==== 138+0+0 (secure 0 0 0) 0x7f4c4c062550 con 0x7f4c5c0ace80 2024-09-10T21:42:53.886 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-09-10T21:42:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:53.887+0000 7f4c60c00640 1 -- 172.21.15.7:0/1884563109 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c38077f70 msgr2=0x7f4c3807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:53.887+0000 7f4c60c00640 1 --2- 172.21.15.7:0/1884563109 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c38077f70 0x7f4c3807a430 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4c44034000 tx=0x7f4c44004730 comp rx=0 tx=0).stop 2024-09-10T21:42:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:53.887+0000 7f4c60c00640 1 -- 172.21.15.7:0/1884563109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c5c0ace80 msgr2=0x7f4c5c148de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:53.887+0000 7f4c60c00640 1 --2- 172.21.15.7:0/1884563109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c5c0ace80 0x7f4c5c148de0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4c4c00c490 tx=0x7f4c4c00c950 comp rx=0 tx=0).stop 2024-09-10T21:42:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:53.887+0000 7f4c60c00640 1 -- 172.21.15.7:0/1884563109 shutdown_connections 2024-09-10T21:42:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:53.887+0000 7f4c60c00640 1 --2- 172.21.15.7:0/1884563109 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c38077f70 0x7f4c3807a430 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:53.887+0000 7f4c60c00640 1 --2- 172.21.15.7:0/1884563109 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c5c15b900 0x7f4c5c147810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:53.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:53.887+0000 7f4c60c00640 1 --2- 172.21.15.7:0/1884563109 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c5c15bce0 0x7f4c5c0b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:53.887+0000 7f4c60c00640 1 --2- 172.21.15.7:0/1884563109 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c5c0ace80 0x7f4c5c148de0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:53.887+0000 7f4c60c00640 1 -- 172.21.15.7:0/1884563109 >> 172.21.15.7:0/1884563109 conn(0x7f4c5c0b1c10 msgr2=0x7f4c5c158ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:53.888+0000 7f4c60c00640 1 -- 172.21.15.7:0/1884563109 shutdown_connections 2024-09-10T21:42:53.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:53.888+0000 7f4c60c00640 1 -- 172.21.15.7:0/1884563109 wait complete. 2024-09-10T21:42:53.912 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:42:54.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.083+0000 7fc484400640 1 -- 172.21.15.7:0/3322410667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc47c06cb20 msgr2=0x7fc47c15ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:54.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.083+0000 7fc484400640 1 --2- 172.21.15.7:0/3322410667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc47c06cb20 0x7fc47c15ffe0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fc474005fd0 tx=0x7fc47402aa80 comp rx=0 tx=0).stop 2024-09-10T21:42:54.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.083+0000 7fc484400640 1 -- 172.21.15.7:0/3322410667 shutdown_connections 2024-09-10T21:42:54.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.083+0000 7fc484400640 1 --2- 172.21.15.7:0/3322410667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc47c06cb20 0x7fc47c15ffe0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:54.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.083+0000 7fc484400640 1 --2- 172.21.15.7:0/3322410667 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc47c0cca40 0x7fc47c073740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:54.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.083+0000 7fc484400640 1 -- 172.21.15.7:0/3322410667 >> 172.21.15.7:0/3322410667 conn(0x7fc47c0b27c0 msgr2=0x7fc47c0b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:54.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.083+0000 7fc484400640 1 -- 172.21.15.7:0/3322410667 shutdown_connections 2024-09-10T21:42:54.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.083+0000 7fc484400640 1 -- 172.21.15.7:0/3322410667 wait complete. 2024-09-10T21:42:54.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.084+0000 7fc484400640 1 Processor -- start 2024-09-10T21:42:54.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.085+0000 7fc484400640 1 -- start start 2024-09-10T21:42:54.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.085+0000 7fc484400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc47c06cb20 0x7fc47c0c5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:54.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.085+0000 7fc484400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc47c0cca40 0x7fc47c0c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:54.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.085+0000 7fc484400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47c15bd30 0x7fc47c157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:54.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.085+0000 7fc484400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c06fa20 con 0x7fc47c06cb20 2024-09-10T21:42:54.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.085+0000 7fc484400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc47c06f8a0 con 0x7fc47c15bd30 2024-09-10T21:42:54.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.085+0000 7fc484400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc47c06fba0 con 0x7fc47c0cca40 2024-09-10T21:42:54.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.086+0000 7fc481a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc47c06cb20 0x7fc47c0c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:54.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.086+0000 7fc481a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc47c06cb20 0x7fc47c0c5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57352/0 (socket says 172.21.15.7:57352) 2024-09-10T21:42:54.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.086+0000 7fc481a00640 1 -- 172.21.15.7:0/458515671 learned_addr learned my addr 172.21.15.7:0/458515671 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:54.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.086+0000 7fc481000640 1 --2- 172.21.15.7:0/458515671 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc47c0cca40 0x7fc47c0c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:54.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.086+0000 7fc482400640 1 --2- 172.21.15.7:0/458515671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47c15bd30 0x7fc47c157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:54.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.086+0000 7fc481a00640 1 -- 172.21.15.7:0/458515671 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc47c0cca40 msgr2=0x7fc47c0c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:54.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.086+0000 7fc481a00640 1 --2- 172.21.15.7:0/458515671 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc47c0cca40 0x7fc47c0c6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:54.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.086+0000 7fc481a00640 1 -- 172.21.15.7:0/458515671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47c15bd30 msgr2=0x7fc47c157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:54.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.086+0000 7fc481a00640 1 --2- 172.21.15.7:0/458515671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47c15bd30 0x7fc47c157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:54.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.086+0000 7fc481a00640 1 -- 172.21.15.7:0/458515671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc47c1f3330 con 0x7fc47c06cb20 2024-09-10T21:42:54.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.086+0000 7fc481a00640 1 --2- 172.21.15.7:0/458515671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc47c06cb20 0x7fc47c0c5940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc47000d950 tx=0x7fc47000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.087+0000 7fc479600640 1 -- 172.21.15.7:0/458515671 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470017070 con 0x7fc47c06cb20 2024-09-10T21:42:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.087+0000 7fc479600640 1 -- 172.21.15.7:0/458515671 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc470009d20 con 0x7fc47c06cb20 2024-09-10T21:42:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.087+0000 7fc484400640 1 -- 172.21.15.7:0/458515671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc47c0c4e60 con 0x7fc47c06cb20 2024-09-10T21:42:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.087+0000 7fc479600640 1 -- 172.21.15.7:0/458515671 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4700106c0 con 0x7fc47c06cb20 2024-09-10T21:42:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.087+0000 7fc484400640 1 -- 172.21.15.7:0/458515671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc47c0c5420 con 0x7fc47c06cb20 2024-09-10T21:42:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.088+0000 7fc484400640 1 -- 172.21.15.7:0/458515671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc47c06ffc0 con 0x7fc47c06cb20 2024-09-10T21:42:54.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.089+0000 7fc479600640 1 -- 172.21.15.7:0/458515671 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc470010860 con 0x7fc47c06cb20 2024-09-10T21:42:54.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.090+0000 7fc479600640 1 --2- 172.21.15.7:0/458515671 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc45c077f70 0x7fc45c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:54.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.090+0000 7fc479600640 1 -- 172.21.15.7:0/458515671 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 5227+0+0 (secure 0 0 0) 0x7fc470099ff0 con 0x7fc47c06cb20 2024-09-10T21:42:54.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.091+0000 7fc481000640 1 --2- 172.21.15.7:0/458515671 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc45c077f70 0x7fc45c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:54.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.091+0000 7fc481000640 1 --2- 172.21.15.7:0/458515671 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc45c077f70 0x7fc45c07a430 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc47c0b3260 tx=0x7fc468009060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:54.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.095+0000 7fc479600640 1 -- 172.21.15.7:0/458515671 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc4700625a0 con 0x7fc47c06cb20 2024-09-10T21:42:54.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.289+0000 7fc484400640 1 -- 172.21.15.7:0/458515671 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc47c0b4f00 con 0x7fc47c06cb20 2024-09-10T21:42:54.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.887+0000 7fc479600640 1 -- 172.21.15.7:0/458515671 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v30) ==== 170+0+0 (secure 0 0 0) 0x7fc470066fd0 con 0x7fc47c06cb20 2024-09-10T21:42:54.889 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-09-10T21:42:54.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.890+0000 7fc484400640 1 -- 172.21.15.7:0/458515671 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc45c077f70 msgr2=0x7fc45c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:54.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.891+0000 7fc484400640 1 --2- 172.21.15.7:0/458515671 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc45c077f70 0x7fc45c07a430 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc47c0b3260 tx=0x7fc468009060 comp rx=0 tx=0).stop 2024-09-10T21:42:54.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.891+0000 7fc484400640 1 -- 172.21.15.7:0/458515671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc47c06cb20 msgr2=0x7fc47c0c5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:54.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.891+0000 7fc484400640 1 --2- 172.21.15.7:0/458515671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc47c06cb20 0x7fc47c0c5940 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fc47000d950 tx=0x7fc47000de10 comp rx=0 tx=0).stop 2024-09-10T21:42:54.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.891+0000 7fc484400640 1 -- 172.21.15.7:0/458515671 shutdown_connections 2024-09-10T21:42:54.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.891+0000 7fc484400640 1 --2- 172.21.15.7:0/458515671 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc45c077f70 0x7fc45c07a430 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:54.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.891+0000 7fc484400640 1 --2- 172.21.15.7:0/458515671 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc47c15bd30 0x7fc47c157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:54.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.891+0000 7fc484400640 1 --2- 172.21.15.7:0/458515671 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc47c0cca40 0x7fc47c0c6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:54.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.891+0000 7fc484400640 1 --2- 172.21.15.7:0/458515671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc47c06cb20 0x7fc47c0c5940 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:54.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.891+0000 7fc484400640 1 -- 172.21.15.7:0/458515671 >> 172.21.15.7:0/458515671 conn(0x7fc47c0b27c0 msgr2=0x7fc47c0b5550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:54.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.891+0000 7fc484400640 1 -- 172.21.15.7:0/458515671 shutdown_connections 2024-09-10T21:42:54.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:54.891+0000 7fc484400640 1 -- 172.21.15.7:0/458515671 wait complete. 2024-09-10T21:42:54.932 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:42:55.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.095+0000 7f61f5e00640 1 -- 172.21.15.7:0/3018768119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f00b0c50 msgr2=0x7f61f006e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:55.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.095+0000 7f61f5e00640 1 --2- 172.21.15.7:0/3018768119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f00b0c50 0x7f61f006e420 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f61e8005fd0 tx=0x7f61e802a9d0 comp rx=0 tx=0).stop 2024-09-10T21:42:55.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.095+0000 7f61f5e00640 1 -- 172.21.15.7:0/3018768119 shutdown_connections 2024-09-10T21:42:55.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.095+0000 7f61f5e00640 1 --2- 172.21.15.7:0/3018768119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f00b0c50 0x7f61f006e420 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:55.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.095+0000 7f61f5e00640 1 -- 172.21.15.7:0/3018768119 >> 172.21.15.7:0/3018768119 conn(0x7f61f00d1db0 msgr2=0x7f61f00d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:55.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.095+0000 7f61f5e00640 1 -- 172.21.15.7:0/3018768119 shutdown_connections 2024-09-10T21:42:55.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.095+0000 7f61f5e00640 1 -- 172.21.15.7:0/3018768119 wait complete. 2024-09-10T21:42:55.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.096+0000 7f61f5e00640 1 Processor -- start 2024-09-10T21:42:55.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.097+0000 7f61f5e00640 1 -- start start 2024-09-10T21:42:55.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.097+0000 7f61f5e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61f00b0c50 0x7f61f015dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:55.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.097+0000 7f61f5e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f00cee20 0x7f61f00b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:55.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.097+0000 7f61f5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f0161760 0x7f61f00b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:55.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.098+0000 7f61f5e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f61f00cf6f0 con 0x7f61f0161760 2024-09-10T21:42:55.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.098+0000 7f61f5e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f61f00cf570 con 0x7f61f00cee20 2024-09-10T21:42:55.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.098+0000 7f61f5e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f61f006e960 con 0x7f61f00b0c50 2024-09-10T21:42:55.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.098+0000 7f61ef400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f0161760 0x7f61f00b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:55.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.098+0000 7f61ef400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f0161760 0x7f61f00b4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57356/0 (socket says 172.21.15.7:57356) 2024-09-10T21:42:55.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.098+0000 7f61ef400640 1 -- 172.21.15.7:0/1884268430 learned_addr learned my addr 172.21.15.7:0/1884268430 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:55.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.098+0000 7f61ee000640 1 --2- 172.21.15.7:0/1884268430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f00cee20 0x7f61f00b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:55.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.098+0000 7f61eea00640 1 --2- 172.21.15.7:0/1884268430 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61f00b0c50 0x7f61f015dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:55.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.098+0000 7f61ef400640 1 -- 172.21.15.7:0/1884268430 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61f00b0c50 msgr2=0x7f61f015dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:55.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.098+0000 7f61ef400640 1 --2- 172.21.15.7:0/1884268430 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61f00b0c50 0x7f61f015dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:55.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.098+0000 7f61ef400640 1 -- 172.21.15.7:0/1884268430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f00cee20 msgr2=0x7f61f00b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:55.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.098+0000 7f61ef400640 1 --2- 172.21.15.7:0/1884268430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f00cee20 0x7f61f00b1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:55.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.098+0000 7f61ef400640 1 -- 172.21.15.7:0/1884268430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61f00b2470 con 0x7f61f0161760 2024-09-10T21:42:55.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.099+0000 7f61ef400640 1 --2- 172.21.15.7:0/1884268430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f0161760 0x7f61f00b4c70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f61e8006010 tx=0x7f61e802f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:55.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.099+0000 7f61e7400640 1 -- 172.21.15.7:0/1884268430 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8047070 con 0x7f61f0161760 2024-09-10T21:42:55.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.099+0000 7f61e7400640 1 -- 172.21.15.7:0/1884268430 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61e8003b80 con 0x7f61f0161760 2024-09-10T21:42:55.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.099+0000 7f61e7400640 1 -- 172.21.15.7:0/1884268430 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e8004040 con 0x7f61f0161760 2024-09-10T21:42:55.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.099+0000 7f61f5e00640 1 -- 172.21.15.7:0/1884268430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61f00b2750 con 0x7f61f0161760 2024-09-10T21:42:55.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.099+0000 7f61f5e00640 1 -- 172.21.15.7:0/1884268430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61f00b2d10 con 0x7f61f0161760 2024-09-10T21:42:55.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.100+0000 7f61f5e00640 1 -- 172.21.15.7:0/1884268430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61f006ee10 con 0x7f61f0161760 2024-09-10T21:42:55.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.102+0000 7f61e7400640 1 -- 172.21.15.7:0/1884268430 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f61e80041e0 con 0x7f61f0161760 2024-09-10T21:42:55.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.102+0000 7f61e7400640 1 --2- 172.21.15.7:0/1884268430 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f61d0077fa0 0x7f61d007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:55.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.103+0000 7f61e7400640 1 -- 172.21.15.7:0/1884268430 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5609+0+0 (secure 0 0 0) 0x7f61e80baa20 con 0x7f61f0161760 2024-09-10T21:42:55.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.103+0000 7f61eea00640 1 --2- 172.21.15.7:0/1884268430 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f61d0077fa0 0x7f61d007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:55.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.104+0000 7f61eea00640 1 --2- 172.21.15.7:0/1884268430 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f61d0077fa0 0x7f61d007a460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f61e00046b0 tx=0x7f61e0007cc0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:55.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.107+0000 7f61e7400640 1 -- 172.21.15.7:0/1884268430 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f61e8033020 con 0x7f61f0161760 2024-09-10T21:42:55.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.335+0000 7f61f5e00640 1 -- 172.21.15.7:0/1884268430 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f61f00b3c20 con 0x7f61f0161760 2024-09-10T21:42:55.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.909+0000 7f61e7400640 1 -- 172.21.15.7:0/1884268430 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f61e8082e90 con 0x7f61f0161760 2024-09-10T21:42:55.915 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 4 and data pool 5 2024-09-10T21:42:55.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.915+0000 7f61f5e00640 1 -- 172.21.15.7:0/1884268430 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f61d0077fa0 msgr2=0x7f61d007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:55.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.915+0000 7f61f5e00640 1 --2- 172.21.15.7:0/1884268430 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f61d0077fa0 0x7f61d007a460 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f61e00046b0 tx=0x7f61e0007cc0 comp rx=0 tx=0).stop 2024-09-10T21:42:55.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.915+0000 7f61f5e00640 1 -- 172.21.15.7:0/1884268430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f0161760 msgr2=0x7f61f00b4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:55.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.915+0000 7f61f5e00640 1 --2- 172.21.15.7:0/1884268430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f0161760 0x7f61f00b4c70 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f61e8006010 tx=0x7f61e802f5f0 comp rx=0 tx=0).stop 2024-09-10T21:42:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.916+0000 7f61f5e00640 1 -- 172.21.15.7:0/1884268430 shutdown_connections 2024-09-10T21:42:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.916+0000 7f61f5e00640 1 --2- 172.21.15.7:0/1884268430 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f61d0077fa0 0x7f61d007a460 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.916+0000 7f61f5e00640 1 --2- 172.21.15.7:0/1884268430 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f61f00cee20 0x7f61f00b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.916+0000 7f61f5e00640 1 --2- 172.21.15.7:0/1884268430 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f61f0161760 0x7f61f00b4c70 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.916+0000 7f61f5e00640 1 --2- 172.21.15.7:0/1884268430 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61f00b0c50 0x7f61f015dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:55.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.916+0000 7f61f5e00640 1 -- 172.21.15.7:0/1884268430 >> 172.21.15.7:0/1884268430 conn(0x7f61f00d1db0 msgr2=0x7f61f0071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:55.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.917+0000 7f61f5e00640 1 -- 172.21.15.7:0/1884268430 shutdown_connections 2024-09-10T21:42:55.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:55.917+0000 7f61f5e00640 1 -- 172.21.15.7:0/1884268430 wait complete. 2024-09-10T21:42:55.953 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:42:56.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.116+0000 7fbf2c200640 1 -- 172.21.15.7:0/87136837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf0c004d40 msgr2=0x7fbf0c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:56.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.116+0000 7fbf2c200640 1 --2- 172.21.15.7:0/87136837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf0c004d40 0x7fbf0c0051c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009f00 tx=0x7fbf0c005830 comp rx=0 tx=0).stop 2024-09-10T21:42:56.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.116+0000 7fbf2a200640 1 -- 172.21.15.7:0/87136837 reap_dead start 2024-09-10T21:42:56.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.117+0000 7fbf2c200640 1 -- 172.21.15.7:0/87136837 shutdown_connections 2024-09-10T21:42:56.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.117+0000 7fbf2c200640 1 -- 172.21.15.7:0/87136837 >> 172.21.15.7:0/87136837 conn(0x7fbf240b16d0 msgr2=0x7fbf240b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:56.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.117+0000 7fbf2c200640 1 -- 172.21.15.7:0/87136837 shutdown_connections 2024-09-10T21:42:56.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.117+0000 7fbf2c200640 1 -- 172.21.15.7:0/87136837 wait complete. 2024-09-10T21:42:56.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.118+0000 7fbf2c200640 1 Processor -- start 2024-09-10T21:42:56.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.119+0000 7fbf2c200640 1 -- start start 2024-09-10T21:42:56.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.119+0000 7fbf2c200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf0c004d40 0x7fbf241f8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:56.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.119+0000 7fbf2c200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf2415b300 0x7fbf240b50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:56.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.120+0000 7fbf2c200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf240b55f0 0x7fbf24072360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:56.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.120+0000 7fbf2c200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbf240cdae0 con 0x7fbf240b55f0 2024-09-10T21:42:56.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.120+0000 7fbf2c200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbf240cd960 con 0x7fbf0c004d40 2024-09-10T21:42:56.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.120+0000 7fbf2c200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fbf240cdc60 con 0x7fbf2415b300 2024-09-10T21:42:56.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.120+0000 7fbf2a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf240b55f0 0x7fbf24072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:56.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.120+0000 7fbf2a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf240b55f0 0x7fbf24072360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57378/0 (socket says 172.21.15.7:57378) 2024-09-10T21:42:56.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.120+0000 7fbf2a200640 1 -- 172.21.15.7:0/2615563332 learned_addr learned my addr 172.21.15.7:0/2615563332 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:56.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.120+0000 7fbf29800640 1 --2- 172.21.15.7:0/2615563332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf0c004d40 0x7fbf241f8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:56.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.120+0000 7fbf28e00640 1 --2- 172.21.15.7:0/2615563332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf2415b300 0x7fbf240b50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:56.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.120+0000 7fbf2a200640 1 -- 172.21.15.7:0/2615563332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf2415b300 msgr2=0x7fbf240b50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:56.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.120+0000 7fbf2a200640 1 --2- 172.21.15.7:0/2615563332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf2415b300 0x7fbf240b50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.120+0000 7fbf2a200640 1 -- 172.21.15.7:0/2615563332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf0c004d40 msgr2=0x7fbf241f8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:56.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.120+0000 7fbf2a200640 1 --2- 172.21.15.7:0/2615563332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf0c004d40 0x7fbf241f8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.120+0000 7fbf2a200640 1 -- 172.21.15.7:0/2615563332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf241f8fc0 con 0x7fbf240b55f0 2024-09-10T21:42:56.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.121+0000 7fbf2a200640 1 --2- 172.21.15.7:0/2615563332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf240b55f0 0x7fbf24072360 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00e380 tx=0x7fbf1c033d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:56.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.121+0000 7fbf21600640 1 -- 172.21.15.7:0/2615563332 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c042070 con 0x7fbf240b55f0 2024-09-10T21:42:56.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.121+0000 7fbf21600640 1 -- 172.21.15.7:0/2615563332 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf1c003c80 con 0x7fbf240b55f0 2024-09-10T21:42:56.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.121+0000 7fbf2c200640 1 -- 172.21.15.7:0/2615563332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2415b880 con 0x7fbf240b55f0 2024-09-10T21:42:56.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.121+0000 7fbf21600640 1 -- 172.21.15.7:0/2615563332 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c038040 con 0x7fbf240b55f0 2024-09-10T21:42:56.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.121+0000 7fbf2c200640 1 -- 172.21.15.7:0/2615563332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf240b48b0 con 0x7fbf240b55f0 2024-09-10T21:42:56.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.123+0000 7fbf2c200640 1 -- 172.21.15.7:0/2615563332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf2415e3c0 con 0x7fbf240b55f0 2024-09-10T21:42:56.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.124+0000 7fbf21600640 1 -- 172.21.15.7:0/2615563332 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fbf1c03b070 con 0x7fbf240b55f0 2024-09-10T21:42:56.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.125+0000 7fbf21600640 1 --2- 172.21.15.7:0/2615563332 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf04077fa0 0x7fbf0407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:56.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.125+0000 7fbf21600640 1 -- 172.21.15.7:0/2615563332 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7fbf1c0ba060 con 0x7fbf240b55f0 2024-09-10T21:42:56.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.125+0000 7fbf29800640 1 --2- 172.21.15.7:0/2615563332 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf04077fa0 0x7fbf0407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:56.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.125+0000 7fbf29800640 1 --2- 172.21.15.7:0/2615563332 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf04077fa0 0x7fbf0407a460 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009c10 tx=0x7fbf18005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:56.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.129+0000 7fbf21600640 1 -- 172.21.15.7:0/2615563332 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fbf1c082470 con 0x7fbf240b55f0 2024-09-10T21:42:56.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.321+0000 7fbf2c200640 1 -- 172.21.15.7:0/2615563332 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf240b5b00 con 0x7fbf240b55f0 2024-09-10T21:42:56.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.322+0000 7fbf21600640 1 -- 172.21.15.7:0/2615563332 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16706 (secure 0 0 0) 0x7fbf1c08b1e0 con 0x7fbf240b55f0 2024-09-10T21:42:56.324 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:56.324 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":31,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:42:55.338137+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:42:52.896912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T21:42:54.300403+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:42:56.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.325+0000 7fbf2c200640 1 -- 172.21.15.7:0/2615563332 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf04077fa0 msgr2=0x7fbf0407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:56.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.325+0000 7fbf2c200640 1 --2- 172.21.15.7:0/2615563332 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf04077fa0 0x7fbf0407a460 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fbf18009c10 tx=0x7fbf18005f90 comp rx=0 tx=0).stop 2024-09-10T21:42:56.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.325+0000 7fbf2c200640 1 -- 172.21.15.7:0/2615563332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf240b55f0 msgr2=0x7fbf24072360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:56.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.325+0000 7fbf2c200640 1 --2- 172.21.15.7:0/2615563332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf240b55f0 0x7fbf24072360 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c00e380 tx=0x7fbf1c033d50 comp rx=0 tx=0).stop 2024-09-10T21:42:56.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.325+0000 7fbf2c200640 1 -- 172.21.15.7:0/2615563332 shutdown_connections 2024-09-10T21:42:56.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.325+0000 7fbf2c200640 1 --2- 172.21.15.7:0/2615563332 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf04077fa0 0x7fbf0407a460 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.325+0000 7fbf2c200640 1 --2- 172.21.15.7:0/2615563332 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf240b55f0 0x7fbf24072360 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.325+0000 7fbf2c200640 1 --2- 172.21.15.7:0/2615563332 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf2415b300 0x7fbf240b50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.325+0000 7fbf2c200640 1 --2- 172.21.15.7:0/2615563332 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf0c004d40 0x7fbf241f8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.325+0000 7fbf2c200640 1 -- 172.21.15.7:0/2615563332 >> 172.21.15.7:0/2615563332 conn(0x7fbf240b16d0 msgr2=0x7fbf2415c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:56.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.325+0000 7fbf2c200640 1 -- 172.21.15.7:0/2615563332 shutdown_connections 2024-09-10T21:42:56.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.325+0000 7fbf2c200640 1 -- 172.21.15.7:0/2615563332 wait complete. 2024-09-10T21:42:56.342 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:42:56.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.501+0000 7f586ec00640 1 -- 172.21.15.7:0/1456487449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586815fda0 msgr2=0x7f5868160180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:56.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.501+0000 7f586ec00640 1 --2- 172.21.15.7:0/1456487449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586815fda0 0x7f5868160180 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f586806dac0 tx=0x7f586002a820 comp rx=0 tx=0).stop 2024-09-10T21:42:56.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.501+0000 7f586ec00640 1 -- 172.21.15.7:0/1456487449 shutdown_connections 2024-09-10T21:42:56.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.501+0000 7f586ec00640 1 --2- 172.21.15.7:0/1456487449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586815fda0 0x7f5868160180 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.501+0000 7f586ec00640 1 --2- 172.21.15.7:0/1456487449 >> v2:172.21.15.133:3300/0 conn(0x7f586815bc90 0x7f58680ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.501+0000 7f586ec00640 1 -- 172.21.15.7:0/1456487449 >> 172.21.15.7:0/1456487449 conn(0x7f58680b16d0 msgr2=0x7f58680b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:56.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.501+0000 7f586ec00640 1 -- 172.21.15.7:0/1456487449 shutdown_connections 2024-09-10T21:42:56.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.501+0000 7f586ec00640 1 -- 172.21.15.7:0/1456487449 wait complete. 2024-09-10T21:42:56.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.503+0000 7f586ec00640 1 Processor -- start 2024-09-10T21:42:56.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.503+0000 7f586ec00640 1 -- start start 2024-09-10T21:42:56.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.504+0000 7f586ec00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58680cd750 0x7f58680c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:56.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.504+0000 7f586ec00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586815bc90 0x7f58681f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:56.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.504+0000 7f586ec00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f586815fda0 0x7f586814acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.504+0000 7f586ec00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f586806fcf0 con 0x7f586815bc90 2024-09-10T21:42:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.504+0000 7f586ec00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5868073ef0 con 0x7f586815fda0 2024-09-10T21:42:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.504+0000 7f586ec00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f586806fe70 con 0x7f58680cd750 2024-09-10T21:42:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.504+0000 7f585f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586815bc90 0x7f58681f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.504+0000 7f585f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586815bc90 0x7f58681f5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57398/0 (socket says 172.21.15.7:57398) 2024-09-10T21:42:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.504+0000 7f585f400640 1 -- 172.21.15.7:0/1376612384 learned_addr learned my addr 172.21.15.7:0/1376612384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.504+0000 7f586cc00640 1 --2- 172.21.15.7:0/1376612384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f586815fda0 0x7f586814acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.504+0000 7f5867e00640 1 --2- 172.21.15.7:0/1376612384 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58680cd750 0x7f58680c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.504+0000 7f585f400640 1 -- 172.21.15.7:0/1376612384 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58680cd750 msgr2=0x7f58680c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.504+0000 7f585f400640 1 --2- 172.21.15.7:0/1376612384 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58680cd750 0x7f58680c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.504+0000 7f585f400640 1 -- 172.21.15.7:0/1376612384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f586815fda0 msgr2=0x7f586814acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.504+0000 7f585f400640 1 --2- 172.21.15.7:0/1376612384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f586815fda0 0x7f586814acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.504+0000 7f585f400640 1 -- 172.21.15.7:0/1376612384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586814be90 con 0x7f586815bc90 2024-09-10T21:42:56.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.504+0000 7f586cc00640 1 --2- 172.21.15.7:0/1376612384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f586815fda0 0x7f586814acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:56.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.505+0000 7f585f400640 1 --2- 172.21.15.7:0/1376612384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586815bc90 0x7f58681f5a00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f585000c500 tx=0x7f585000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:56.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.505+0000 7f5864c00640 1 -- 172.21.15.7:0/1376612384 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5850017070 con 0x7f586815bc90 2024-09-10T21:42:56.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.505+0000 7f5864c00640 1 -- 172.21.15.7:0/1376612384 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58500035f0 con 0x7f586815bc90 2024-09-10T21:42:56.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.505+0000 7f5864c00640 1 -- 172.21.15.7:0/1376612384 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5850003a40 con 0x7f586815bc90 2024-09-10T21:42:56.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.505+0000 7f586ec00640 1 -- 172.21.15.7:0/1376612384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586814c170 con 0x7f586815bc90 2024-09-10T21:42:56.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.505+0000 7f586ec00640 1 -- 172.21.15.7:0/1376612384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f586814c730 con 0x7f586815bc90 2024-09-10T21:42:56.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.506+0000 7f586ec00640 1 -- 172.21.15.7:0/1376612384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58680ccef0 con 0x7f586815bc90 2024-09-10T21:42:56.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.507+0000 7f5864c00640 1 -- 172.21.15.7:0/1376612384 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5850003be0 con 0x7f586815bc90 2024-09-10T21:42:56.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.508+0000 7f5864c00640 1 --2- 172.21.15.7:0/1376612384 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5844077f70 0x7f584407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:56.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.508+0000 7f5864c00640 1 -- 172.21.15.7:0/1376612384 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f58500995c0 con 0x7f586815bc90 2024-09-10T21:42:56.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.509+0000 7f5867e00640 1 --2- 172.21.15.7:0/1376612384 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5844077f70 0x7f584407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:56.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.509+0000 7f5867e00640 1 --2- 172.21.15.7:0/1376612384 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5844077f70 0x7f584407a430 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5858005fd0 tx=0x7f585800b590 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:56.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.513+0000 7f5864c00640 1 -- 172.21.15.7:0/1376612384 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f58500620b0 con 0x7f586815bc90 2024-09-10T21:42:56.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.709+0000 7f586ec00640 1 -- 172.21.15.7:0/1376612384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f58680703b0 con 0x7f586815bc90 2024-09-10T21:42:56.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.710+0000 7f5864c00640 1 -- 172.21.15.7:0/1376612384 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) ==== 74+0+16706 (secure 0 0 0) 0x7f58500619b0 con 0x7f586815bc90 2024-09-10T21:42:56.712 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:56.712 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":31,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:42:55.338137+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:42:52.896912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T21:42:54.300403+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:42:56.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.713+0000 7f586ec00640 1 -- 172.21.15.7:0/1376612384 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5844077f70 msgr2=0x7f584407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.713+0000 7f586ec00640 1 --2- 172.21.15.7:0/1376612384 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5844077f70 0x7f584407a430 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f5858005fd0 tx=0x7f585800b590 comp rx=0 tx=0).stop 2024-09-10T21:42:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.713+0000 7f586ec00640 1 -- 172.21.15.7:0/1376612384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586815bc90 msgr2=0x7f58681f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.713+0000 7f586ec00640 1 --2- 172.21.15.7:0/1376612384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586815bc90 0x7f58681f5a00 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f585000c500 tx=0x7f585000c9c0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.713+0000 7f586ec00640 1 -- 172.21.15.7:0/1376612384 shutdown_connections 2024-09-10T21:42:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.713+0000 7f586ec00640 1 --2- 172.21.15.7:0/1376612384 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5844077f70 0x7f584407a430 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.713+0000 7f586ec00640 1 --2- 172.21.15.7:0/1376612384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f586815fda0 0x7f586814acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.713+0000 7f586ec00640 1 --2- 172.21.15.7:0/1376612384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f586815bc90 0x7f58681f5a00 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.713+0000 7f586ec00640 1 --2- 172.21.15.7:0/1376612384 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58680cd750 0x7f58680c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.713+0000 7f586ec00640 1 -- 172.21.15.7:0/1376612384 >> 172.21.15.7:0/1376612384 conn(0x7f58680b16d0 msgr2=0x7f58680b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.713+0000 7f586ec00640 1 -- 172.21.15.7:0/1376612384 shutdown_connections 2024-09-10T21:42:56.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.713+0000 7f586ec00640 1 -- 172.21.15.7:0/1376612384 wait complete. 2024-09-10T21:42:56.730 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:42:56.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.894+0000 7f3bbd200640 1 -- 172.21.15.7:0/975843226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b9c004800 msgr2=0x7f3b9c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:56.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.894+0000 7f3bbd200640 1 --2- 172.21.15.7:0/975843226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b9c004800 0x7f3b9c004c80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40016c0 tx=0x7f3b9c005830 comp rx=0 tx=0).stop 2024-09-10T21:42:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.894+0000 7f3bbd200640 1 -- 172.21.15.7:0/975843226 shutdown_connections 2024-09-10T21:42:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.894+0000 7f3bbd200640 1 --2- 172.21.15.7:0/975843226 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3b9c004800 0x7f3b9c004c80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.894+0000 7f3bbd200640 1 -- 172.21.15.7:0/975843226 >> 172.21.15.7:0/975843226 conn(0x7f3bb80b27c0 msgr2=0x7f3bb80b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.894+0000 7f3bbd200640 1 -- 172.21.15.7:0/975843226 shutdown_connections 2024-09-10T21:42:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.894+0000 7f3bbd200640 1 -- 172.21.15.7:0/975843226 wait complete. 2024-09-10T21:42:56.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.895+0000 7f3bbd200640 1 Processor -- start 2024-09-10T21:42:56.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.896+0000 7f3bbd200640 1 -- start start 2024-09-10T21:42:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.896+0000 7f3bbd200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b9c004800 0x7f3bb80c7030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.896+0000 7f3bbd200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3bb81f4280 0x7f3bb81f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.896+0000 7f3bbd200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bb81f8450 0x7f3bb80c4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.896+0000 7f3bbd200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb8070da0 con 0x7f3bb81f8450 2024-09-10T21:42:56.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.896+0000 7f3bbd200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb806d010 con 0x7f3b9c004800 2024-09-10T21:42:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.896+0000 7f3bbd200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f3bb8070f20 con 0x7f3bb81f4280 2024-09-10T21:42:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.897+0000 7f3bb6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bb81f8450 0x7f3bb80c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.897+0000 7f3bb6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bb81f8450 0x7f3bb80c4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57418/0 (socket says 172.21.15.7:57418) 2024-09-10T21:42:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.897+0000 7f3bb6800640 1 -- 172.21.15.7:0/720967672 learned_addr learned my addr 172.21.15.7:0/720967672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.897+0000 7f3bb5400640 1 --2- 172.21.15.7:0/720967672 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3bb81f4280 0x7f3bb81f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.897+0000 7f3bb5e00640 1 --2- 172.21.15.7:0/720967672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b9c004800 0x7f3bb80c7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.897+0000 7f3bb6800640 1 -- 172.21.15.7:0/720967672 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3bb81f4280 msgr2=0x7f3bb81f7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.897+0000 7f3bb6800640 1 --2- 172.21.15.7:0/720967672 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3bb81f4280 0x7f3bb81f7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.897+0000 7f3bb6800640 1 -- 172.21.15.7:0/720967672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b9c004800 msgr2=0x7f3bb80c7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.897+0000 7f3bb6800640 1 --2- 172.21.15.7:0/720967672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b9c004800 0x7f3bb80c7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.897+0000 7f3bb6800640 1 -- 172.21.15.7:0/720967672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb80c66a0 con 0x7f3bb81f8450 2024-09-10T21:42:56.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.897+0000 7f3bb5e00640 1 --2- 172.21.15.7:0/720967672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b9c004800 0x7f3bb80c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:56.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.898+0000 7f3bb6800640 1 --2- 172.21.15.7:0/720967672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bb81f8450 0x7f3bb80c4720 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0005fd0 tx=0x7f3bb000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:56.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.898+0000 7f3bae000640 1 -- 172.21.15.7:0/720967672 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0003d80 con 0x7f3bb81f8450 2024-09-10T21:42:56.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.898+0000 7f3bae000640 1 -- 172.21.15.7:0/720967672 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bb0004b20 con 0x7f3bb81f8450 2024-09-10T21:42:56.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.898+0000 7f3bbd200640 1 -- 172.21.15.7:0/720967672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb80c6980 con 0x7f3bb81f8450 2024-09-10T21:42:56.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.898+0000 7f3bae000640 1 -- 172.21.15.7:0/720967672 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0004050 con 0x7f3bb81f8450 2024-09-10T21:42:56.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.898+0000 7f3bbd200640 1 -- 172.21.15.7:0/720967672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb81e68a0 con 0x7f3bb81f8450 2024-09-10T21:42:56.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.899+0000 7f3bbd200640 1 -- 172.21.15.7:0/720967672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb806f690 con 0x7f3bb81f8450 2024-09-10T21:42:56.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.900+0000 7f3bae000640 1 -- 172.21.15.7:0/720967672 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3bb0018070 con 0x7f3bb81f8450 2024-09-10T21:42:56.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.901+0000 7f3bae000640 1 --2- 172.21.15.7:0/720967672 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3b98077fa0 0x7f3b9807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.901+0000 7f3bae000640 1 -- 172.21.15.7:0/720967672 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 5709+0+0 (secure 0 0 0) 0x7f3bb009aab0 con 0x7f3bb81f8450 2024-09-10T21:42:56.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.902+0000 7f3bb5e00640 1 --2- 172.21.15.7:0/720967672 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3b98077fa0 0x7f3b9807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:56.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.903+0000 7f3bb5e00640 1 --2- 172.21.15.7:0/720967672 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3b98077fa0 0x7f3b9807a460 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8000fd0 tx=0x7f3ba80067a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:56.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:56.905+0000 7f3bae000640 1 -- 172.21.15.7:0/720967672 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3bb00635a0 con 0x7f3bb81f8450 2024-09-10T21:42:57.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:57.146+0000 7f3bbd200640 1 -- 172.21.15.7:0/720967672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3bb80c79a0 con 0x7f3bb81f8450 2024-09-10T21:42:57.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:57.924+0000 7f3bae000640 1 -- 172.21.15.7:0/720967672 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f3bb0062ea0 con 0x7f3bb81f8450 2024-09-10T21:42:57.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:57.927+0000 7f3bbd200640 1 -- 172.21.15.7:0/720967672 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3b98077fa0 msgr2=0x7f3b9807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:57.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:57.927+0000 7f3bbd200640 1 --2- 172.21.15.7:0/720967672 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3b98077fa0 0x7f3b9807a460 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8000fd0 tx=0x7f3ba80067a0 comp rx=0 tx=0).stop 2024-09-10T21:42:57.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:57.927+0000 7f3bbd200640 1 -- 172.21.15.7:0/720967672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bb81f8450 msgr2=0x7f3bb80c4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:57.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:57.927+0000 7f3bbd200640 1 --2- 172.21.15.7:0/720967672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bb81f8450 0x7f3bb80c4720 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0005fd0 tx=0x7f3bb000f790 comp rx=0 tx=0).stop 2024-09-10T21:42:57.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:57.928+0000 7f3bbd200640 1 -- 172.21.15.7:0/720967672 shutdown_connections 2024-09-10T21:42:57.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:57.928+0000 7f3bbd200640 1 --2- 172.21.15.7:0/720967672 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3b98077fa0 0x7f3b9807a460 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:57.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:57.928+0000 7f3bbd200640 1 --2- 172.21.15.7:0/720967672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3bb81f8450 0x7f3bb80c4720 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:57.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:57.928+0000 7f3bbd200640 1 --2- 172.21.15.7:0/720967672 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3bb81f4280 0x7f3bb81f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:57.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:57.928+0000 7f3bbd200640 1 --2- 172.21.15.7:0/720967672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3b9c004800 0x7f3bb80c7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:57.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:57.928+0000 7f3bbd200640 1 -- 172.21.15.7:0/720967672 >> 172.21.15.7:0/720967672 conn(0x7f3bb80b27c0 msgr2=0x7f3bb80b1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:57.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:57.928+0000 7f3bbd200640 1 -- 172.21.15.7:0/720967672 shutdown_connections 2024-09-10T21:42:57.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:57.928+0000 7f3bbd200640 1 -- 172.21.15.7:0/720967672 wait complete. 2024-09-10T21:42:57.948 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:42:57.949 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:42:58.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.116+0000 7f84dbc00640 1 -- 172.21.15.7:0/3680040127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d4156090 msgr2=0x7f84d4160100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:58.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.116+0000 7f84dbc00640 1 --2- 172.21.15.7:0/3680040127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d4156090 0x7f84d4160100 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009830 tx=0x7f84c802a510 comp rx=0 tx=0).stop 2024-09-10T21:42:58.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.117+0000 7f84dbc00640 1 -- 172.21.15.7:0/3680040127 shutdown_connections 2024-09-10T21:42:58.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.117+0000 7f84dbc00640 1 --2- 172.21.15.7:0/3680040127 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d4156090 0x7f84d4160100 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:58.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.117+0000 7f84dbc00640 1 -- 172.21.15.7:0/3680040127 >> 172.21.15.7:0/3680040127 conn(0x7f84d40b16d0 msgr2=0x7f84d40b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:58.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.117+0000 7f84dbc00640 1 -- 172.21.15.7:0/3680040127 shutdown_connections 2024-09-10T21:42:58.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.117+0000 7f84dbc00640 1 -- 172.21.15.7:0/3680040127 wait complete. 2024-09-10T21:42:58.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.118+0000 7f84dbc00640 1 Processor -- start 2024-09-10T21:42:58.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.118+0000 7f84dbc00640 1 -- start start 2024-09-10T21:42:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.119+0000 7f84dbc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d406fcf0 0x7f84d41507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.119+0000 7f84dbc00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84d4074450 0x7f84d4152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.119+0000 7f84dbc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84d4156090 0x7f84d4153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.119+0000 7f84dbc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f84d40cad10 con 0x7f84d406fcf0 2024-09-10T21:42:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.119+0000 7f84dbc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f84d40cab90 con 0x7f84d4156090 2024-09-10T21:42:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.119+0000 7f84dbc00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f84d40cae90 con 0x7f84d4074450 2024-09-10T21:42:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.120+0000 7f84d9200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d406fcf0 0x7f84d41507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.120+0000 7f84d9200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d406fcf0 0x7f84d41507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57448/0 (socket says 172.21.15.7:57448) 2024-09-10T21:42:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.120+0000 7f84d9200640 1 -- 172.21.15.7:0/684733888 learned_addr learned my addr 172.21.15.7:0/684733888 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:58.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.120+0000 7f84d9c00640 1 --2- 172.21.15.7:0/684733888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84d4156090 0x7f84d4153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:58.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.120+0000 7f84d3e00640 1 --2- 172.21.15.7:0/684733888 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84d4074450 0x7f84d4152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:58.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.120+0000 7f84d9200640 1 -- 172.21.15.7:0/684733888 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84d4074450 msgr2=0x7f84d4152ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:58.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.120+0000 7f84d9200640 1 --2- 172.21.15.7:0/684733888 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84d4074450 0x7f84d4152ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:58.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.120+0000 7f84d9200640 1 -- 172.21.15.7:0/684733888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84d4156090 msgr2=0x7f84d4153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:58.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.120+0000 7f84d9200640 1 --2- 172.21.15.7:0/684733888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84d4156090 0x7f84d4153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:58.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.120+0000 7f84d9200640 1 -- 172.21.15.7:0/684733888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d40c62a0 con 0x7f84d406fcf0 2024-09-10T21:42:58.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.120+0000 7f84d9200640 1 --2- 172.21.15.7:0/684733888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d406fcf0 0x7f84d41507e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f84c8000c00 tx=0x7f84c8033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:58.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.121+0000 7f84d0c00640 1 -- 172.21.15.7:0/684733888 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c8031d60 con 0x7f84d406fcf0 2024-09-10T21:42:58.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.121+0000 7f84d0c00640 1 -- 172.21.15.7:0/684733888 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84c80330d0 con 0x7f84d406fcf0 2024-09-10T21:42:58.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.121+0000 7f84d0c00640 1 -- 172.21.15.7:0/684733888 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c8033520 con 0x7f84d406fcf0 2024-09-10T21:42:58.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.121+0000 7f84dbc00640 1 -- 172.21.15.7:0/684733888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d40c6580 con 0x7f84d406fcf0 2024-09-10T21:42:58.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.121+0000 7f84dbc00640 1 -- 172.21.15.7:0/684733888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d41524e0 con 0x7f84d406fcf0 2024-09-10T21:42:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.122+0000 7f84dbc00640 1 -- 172.21.15.7:0/684733888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d4160100 con 0x7f84d406fcf0 2024-09-10T21:42:58.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.123+0000 7f84d0c00640 1 -- 172.21.15.7:0/684733888 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f84c80035f0 con 0x7f84d406fcf0 2024-09-10T21:42:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.124+0000 7f84d0c00640 1 --2- 172.21.15.7:0/684733888 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f84b4077f70 0x7f84b407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.124+0000 7f84d0c00640 1 -- 172.21.15.7:0/684733888 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f84c80bb160 con 0x7f84d406fcf0 2024-09-10T21:42:58.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.124+0000 7f84d3e00640 1 --2- 172.21.15.7:0/684733888 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f84b4077f70 0x7f84b407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:58.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.125+0000 7f84d3e00640 1 --2- 172.21.15.7:0/684733888 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f84b4077f70 0x7f84b407a430 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f84d41f5d20 tx=0x7f84c0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:58.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.128+0000 7f84d0c00640 1 -- 172.21.15.7:0/684733888 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f84c803f030 con 0x7f84d406fcf0 2024-09-10T21:42:58.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.360+0000 7f84dbc00640 1 -- 172.21.15.7:0/684733888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f84d415d620 con 0x7f84d406fcf0 2024-09-10T21:42:58.862 INFO:tasks.ceph.mds.b.smithi133.stderr:2024-09-10T21:42:58.861+0000 7fc251400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:42:58.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.931+0000 7f84d0c00640 1 -- 172.21.15.7:0/684733888 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f84c8083bd0 con 0x7f84d406fcf0 2024-09-10T21:42:58.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.934+0000 7f84dbc00640 1 -- 172.21.15.7:0/684733888 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f84b4077f70 msgr2=0x7f84b407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.934+0000 7f84dbc00640 1 --2- 172.21.15.7:0/684733888 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f84b4077f70 0x7f84b407a430 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f84d41f5d20 tx=0x7f84c0009040 comp rx=0 tx=0).stop 2024-09-10T21:42:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.934+0000 7f84dbc00640 1 -- 172.21.15.7:0/684733888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d406fcf0 msgr2=0x7f84d41507e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.934+0000 7f84dbc00640 1 --2- 172.21.15.7:0/684733888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d406fcf0 0x7f84d41507e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f84c8000c00 tx=0x7f84c8033d60 comp rx=0 tx=0).stop 2024-09-10T21:42:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.934+0000 7f84dbc00640 1 -- 172.21.15.7:0/684733888 shutdown_connections 2024-09-10T21:42:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.934+0000 7f84dbc00640 1 --2- 172.21.15.7:0/684733888 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f84b4077f70 0x7f84b407a430 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.934+0000 7f84dbc00640 1 --2- 172.21.15.7:0/684733888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f84d4156090 0x7f84d4153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.934+0000 7f84dbc00640 1 --2- 172.21.15.7:0/684733888 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84d4074450 0x7f84d4152ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.934+0000 7f84dbc00640 1 --2- 172.21.15.7:0/684733888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84d406fcf0 0x7f84d41507e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.934+0000 7f84dbc00640 1 -- 172.21.15.7:0/684733888 >> 172.21.15.7:0/684733888 conn(0x7f84d40b16d0 msgr2=0x7f84d40b50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.934+0000 7f84dbc00640 1 -- 172.21.15.7:0/684733888 shutdown_connections 2024-09-10T21:42:58.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:58.934+0000 7f84dbc00640 1 -- 172.21.15.7:0/684733888 wait complete. 2024-09-10T21:42:58.953 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:42:58.953 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:42:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.114+0000 7f81d8e00640 1 -- 172.21.15.7:0/2278031996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b8004000 msgr2=0x7f81b8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.114+0000 7f81d8e00640 1 --2- 172.21.15.7:0/2278031996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81b8004000 0x7f81b8004800 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f81bc001690 tx=0x7f81b8005830 comp rx=0 tx=0).stop 2024-09-10T21:42:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.114+0000 7f81d2800640 1 -- 172.21.15.7:0/2278031996 reap_dead start 2024-09-10T21:42:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.114+0000 7f81d8e00640 1 -- 172.21.15.7:0/2278031996 shutdown_connections 2024-09-10T21:42:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.114+0000 7f81d8e00640 1 -- 172.21.15.7:0/2278031996 >> 172.21.15.7:0/2278031996 conn(0x7f81d40b16d0 msgr2=0x7f81d40b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.114+0000 7f81d8e00640 1 -- 172.21.15.7:0/2278031996 shutdown_connections 2024-09-10T21:42:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.114+0000 7f81d8e00640 1 -- 172.21.15.7:0/2278031996 wait complete. 2024-09-10T21:42:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.116+0000 7f81d8e00640 1 Processor -- start 2024-09-10T21:42:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.116+0000 7f81d8e00640 1 -- start start 2024-09-10T21:42:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.117+0000 7f81d8e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81b8004000 0x7f81d40b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.117+0000 7f81d8e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81d40ce5c0 0x7f81d415c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.117+0000 7f81d8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81d4150120 0x7f81d40c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.117+0000 7f81d8e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f81d40cc8f0 con 0x7f81d4150120 2024-09-10T21:42:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.117+0000 7f81d8e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f81d40cc770 con 0x7f81d40ce5c0 2024-09-10T21:42:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.117+0000 7f81d8e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f81d40cca70 con 0x7f81b8004000 2024-09-10T21:42:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.117+0000 7f81d2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81d4150120 0x7f81d40c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.117+0000 7f81d2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81d4150120 0x7f81d40c8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57476/0 (socket says 172.21.15.7:57476) 2024-09-10T21:42:59.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.117+0000 7f81d2800640 1 -- 172.21.15.7:0/2188151394 learned_addr learned my addr 172.21.15.7:0/2188151394 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:59.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.117+0000 7f81d1e00640 1 --2- 172.21.15.7:0/2188151394 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81b8004000 0x7f81d40b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:59.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.117+0000 7f81cbe00640 1 --2- 172.21.15.7:0/2188151394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81d40ce5c0 0x7f81d415c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:59.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.118+0000 7f81d2800640 1 -- 172.21.15.7:0/2188151394 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81b8004000 msgr2=0x7f81d40b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:59.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.118+0000 7f81d2800640 1 --2- 172.21.15.7:0/2188151394 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81b8004000 0x7f81d40b0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.118+0000 7f81d2800640 1 -- 172.21.15.7:0/2188151394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81d40ce5c0 msgr2=0x7f81d415c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:59.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.118+0000 7f81d2800640 1 --2- 172.21.15.7:0/2188151394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81d40ce5c0 0x7f81d415c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.118+0000 7f81d2800640 1 -- 172.21.15.7:0/2188151394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d40b1120 con 0x7f81d4150120 2024-09-10T21:42:59.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.118+0000 7f81cbe00640 1 --2- 172.21.15.7:0/2188151394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81d40ce5c0 0x7f81d415c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:59.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.118+0000 7f81d2800640 1 --2- 172.21.15.7:0/2188151394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81d4150120 0x7f81d40c8e20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f81d4074940 tx=0x7f81cc00d7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:59.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.118+0000 7f81ca000640 1 -- 172.21.15.7:0/2188151394 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc00de50 con 0x7f81d4150120 2024-09-10T21:42:59.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.118+0000 7f81ca000640 1 -- 172.21.15.7:0/2188151394 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81cc003b80 con 0x7f81d4150120 2024-09-10T21:42:59.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.118+0000 7f81ca000640 1 -- 172.21.15.7:0/2188151394 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81cc004030 con 0x7f81d4150120 2024-09-10T21:42:59.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.118+0000 7f81d8e00640 1 -- 172.21.15.7:0/2188151394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d4159750 con 0x7f81d4150120 2024-09-10T21:42:59.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.118+0000 7f81d8e00640 1 -- 172.21.15.7:0/2188151394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d4157f60 con 0x7f81d4150120 2024-09-10T21:42:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.119+0000 7f81d8e00640 1 -- 172.21.15.7:0/2188151394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81d406e510 con 0x7f81d4150120 2024-09-10T21:42:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.120+0000 7f81ca000640 1 -- 172.21.15.7:0/2188151394 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f81cc00b040 con 0x7f81d4150120 2024-09-10T21:42:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.121+0000 7f81ca000640 1 --2- 172.21.15.7:0/2188151394 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f81b0077fa0 0x7f81b007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.122+0000 7f81ca000640 1 -- 172.21.15.7:0/2188151394 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f81cc09a490 con 0x7f81d4150120 2024-09-10T21:42:59.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.122+0000 7f81d1e00640 1 --2- 172.21.15.7:0/2188151394 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f81b0077fa0 0x7f81b007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:59.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.123+0000 7f81d1e00640 1 --2- 172.21.15.7:0/2188151394 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f81b0077fa0 0x7f81b007a460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f81c4000fb0 tx=0x7f81c40067a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:59.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.125+0000 7f81ca000640 1 -- 172.21.15.7:0/2188151394 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f81d40b1120 con 0x7f81d4150120 2024-09-10T21:42:59.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.360+0000 7f81d8e00640 1 -- 172.21.15.7:0/2188151394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81d415c160 con 0x7f81d4150120 2024-09-10T21:42:59.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.360+0000 7f81ca000640 1 -- 172.21.15.7:0/2188151394 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5221 (secure 0 0 0) 0x7f81cc062f80 con 0x7f81d4150120 2024-09-10T21:42:59.362 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:59.362 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"btime":"2024-09-10T21:42:58:925871+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/923954733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":923954733},{"type":"v1","addr":"172.21.15.133:6837","nonce":923954733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/13426364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":13426364},{"type":"v1","addr":"172.21.15.7:6835","nonce":13426364}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2275341781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2275341781},{"type":"v1","addr":"172.21.15.7:6834","nonce":2275341781}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:42:55.338104+0000","modified":"2024-09-10T21:42:58.362130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234524":{"gid":234524,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6836/849012248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":849012248},{"type":"v1","addr":"172.21.15.133:6836","nonce":849012248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234524,"qdb_cluster":[234524]},"id":2}]} 2024-09-10T21:42:59.362 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-09-10T21:42:59.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.363+0000 7f81d8e00640 1 -- 172.21.15.7:0/2188151394 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f81b0077fa0 msgr2=0x7f81b007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:59.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.363+0000 7f81d8e00640 1 --2- 172.21.15.7:0/2188151394 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f81b0077fa0 0x7f81b007a460 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f81c4000fb0 tx=0x7f81c40067a0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.363+0000 7f81d8e00640 1 -- 172.21.15.7:0/2188151394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81d4150120 msgr2=0x7f81d40c8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:59.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.363+0000 7f81d8e00640 1 --2- 172.21.15.7:0/2188151394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81d4150120 0x7f81d40c8e20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f81d4074940 tx=0x7f81cc00d7b0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.363+0000 7f81d8e00640 1 -- 172.21.15.7:0/2188151394 shutdown_connections 2024-09-10T21:42:59.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.363+0000 7f81d8e00640 1 --2- 172.21.15.7:0/2188151394 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f81b0077fa0 0x7f81b007a460 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.363+0000 7f81d8e00640 1 --2- 172.21.15.7:0/2188151394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81d40ce5c0 0x7f81d415c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.363+0000 7f81d8e00640 1 --2- 172.21.15.7:0/2188151394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81d4150120 0x7f81d40c8e20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.363+0000 7f81d8e00640 1 --2- 172.21.15.7:0/2188151394 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81b8004000 0x7f81d40b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.363+0000 7f81d8e00640 1 -- 172.21.15.7:0/2188151394 >> 172.21.15.7:0/2188151394 conn(0x7f81d40b16d0 msgr2=0x7f81d41541d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:59.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.363+0000 7f81d8e00640 1 -- 172.21.15.7:0/2188151394 shutdown_connections 2024-09-10T21:42:59.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.363+0000 7f81d8e00640 1 -- 172.21.15.7:0/2188151394 wait complete. 2024-09-10T21:42:59.381 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:42:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.540+0000 7f0b84c00640 1 -- 172.21.15.7:0/2793449517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b80070600 msgr2=0x7f0b8006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.540+0000 7f0b84c00640 1 --2- 172.21.15.7:0/2793449517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b80070600 0x7f0b8006e510 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009830 tx=0x7f0b7002a560 comp rx=0 tx=0).stop 2024-09-10T21:42:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.540+0000 7f0b84c00640 1 -- 172.21.15.7:0/2793449517 shutdown_connections 2024-09-10T21:42:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.540+0000 7f0b84c00640 1 --2- 172.21.15.7:0/2793449517 >> v2:172.21.15.133:3300/0 conn(0x7f0b8015da10 0x7f0b800cf360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.540+0000 7f0b84c00640 1 --2- 172.21.15.7:0/2793449517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b80070600 0x7f0b8006e510 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.540+0000 7f0b84c00640 1 -- 172.21.15.7:0/2793449517 >> 172.21.15.7:0/2793449517 conn(0x7f0b800b16d0 msgr2=0x7f0b800b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.540+0000 7f0b84c00640 1 -- 172.21.15.7:0/2793449517 shutdown_connections 2024-09-10T21:42:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.540+0000 7f0b84c00640 1 -- 172.21.15.7:0/2793449517 wait complete. 2024-09-10T21:42:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.542+0000 7f0b84c00640 1 Processor -- start 2024-09-10T21:42:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.542+0000 7f0b84c00640 1 -- start start 2024-09-10T21:42:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.543+0000 7f0b84c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b80070600 0x7f0b8014e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.543+0000 7f0b84c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b800ca360 0x7f0b80156580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.543+0000 7f0b84c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b8015da10 0x7f0b8015c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.543+0000 7f0b84c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0b8015aff0 con 0x7f0b800ca360 2024-09-10T21:42:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.543+0000 7f0b84c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0b800caa20 con 0x7f0b8015da10 2024-09-10T21:42:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.543+0000 7f0b84c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f0b8015b170 con 0x7f0b80070600 2024-09-10T21:42:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.543+0000 7f0b75400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b800ca360 0x7f0b80156580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.543+0000 7f0b75400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b800ca360 0x7f0b80156580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57498/0 (socket says 172.21.15.7:57498) 2024-09-10T21:42:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.543+0000 7f0b75400640 1 -- 172.21.15.7:0/2231466790 learned_addr learned my addr 172.21.15.7:0/2231466790 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.543+0000 7f0b7de00640 1 --2- 172.21.15.7:0/2231466790 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b80070600 0x7f0b8014e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.543+0000 7f0b7e800640 1 --2- 172.21.15.7:0/2231466790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b8015da10 0x7f0b8015c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.543+0000 7f0b75400640 1 -- 172.21.15.7:0/2231466790 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b80070600 msgr2=0x7f0b8014e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.543+0000 7f0b75400640 1 --2- 172.21.15.7:0/2231466790 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b80070600 0x7f0b8014e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.543+0000 7f0b75400640 1 -- 172.21.15.7:0/2231466790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b8015da10 msgr2=0x7f0b8015c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.543+0000 7f0b75400640 1 --2- 172.21.15.7:0/2231466790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b8015da10 0x7f0b8015c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.544+0000 7f0b75400640 1 -- 172.21.15.7:0/2231466790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b8014f270 con 0x7f0b800ca360 2024-09-10T21:42:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.544+0000 7f0b7e800640 1 --2- 172.21.15.7:0/2231466790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b8015da10 0x7f0b8015c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.544+0000 7f0b75400640 1 --2- 172.21.15.7:0/2231466790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b800ca360 0x7f0b80156580 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800dbc0 tx=0x7f0b6800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.544+0000 7f0b76a00640 1 -- 172.21.15.7:0/2231466790 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6801b070 con 0x7f0b800ca360 2024-09-10T21:42:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.544+0000 7f0b76a00640 1 -- 172.21.15.7:0/2231466790 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b6800ae50 con 0x7f0b800ca360 2024-09-10T21:42:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.544+0000 7f0b76a00640 1 -- 172.21.15.7:0/2231466790 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b6800e9a0 con 0x7f0b800ca360 2024-09-10T21:42:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.544+0000 7f0b84c00640 1 -- 172.21.15.7:0/2231466790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b801f5e80 con 0x7f0b800ca360 2024-09-10T21:42:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.544+0000 7f0b84c00640 1 -- 172.21.15.7:0/2231466790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b801f6440 con 0x7f0b800ca360 2024-09-10T21:42:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.545+0000 7f0b84c00640 1 -- 172.21.15.7:0/2231466790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b8006e510 con 0x7f0b800ca360 2024-09-10T21:42:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.546+0000 7f0b76a00640 1 -- 172.21.15.7:0/2231466790 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0b6800b040 con 0x7f0b800ca360 2024-09-10T21:42:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.547+0000 7f0b76a00640 1 --2- 172.21.15.7:0/2231466790 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0b5c077f70 0x7f0b5c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.547+0000 7f0b76a00640 1 -- 172.21.15.7:0/2231466790 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f0b6809a0b0 con 0x7f0b800ca360 2024-09-10T21:42:59.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.548+0000 7f0b7de00640 1 --2- 172.21.15.7:0/2231466790 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0b5c077f70 0x7f0b5c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:59.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.548+0000 7f0b7de00640 1 --2- 172.21.15.7:0/2231466790 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0b5c077f70 0x7f0b5c07a430 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0b70034000 tx=0x7f0b70004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:59.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.551+0000 7f0b76a00640 1 -- 172.21.15.7:0/2231466790 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0b68062550 con 0x7f0b800ca360 2024-09-10T21:42:59.725 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:42:59.730 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:42:59.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.746+0000 7f0b84c00640 1 -- 172.21.15.7:0/2231466790 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0b80156c40 con 0x7f0b800ca360 2024-09-10T21:42:59.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.747+0000 7f0b76a00640 1 -- 172.21.15.7:0/2231466790 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16693 (secure 0 0 0) 0x7f0b68066f80 con 0x7f0b800ca360 2024-09-10T21:42:59.749 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:42:59.749 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":32,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:42:56.911349+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:42:52.896912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T21:42:54.300403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:42:59.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.749+0000 7f0b84c00640 1 -- 172.21.15.7:0/2231466790 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0b5c077f70 msgr2=0x7f0b5c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:59.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.749+0000 7f0b84c00640 1 --2- 172.21.15.7:0/2231466790 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0b5c077f70 0x7f0b5c07a430 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0b70034000 tx=0x7f0b70004730 comp rx=0 tx=0).stop 2024-09-10T21:42:59.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.749+0000 7f0b84c00640 1 -- 172.21.15.7:0/2231466790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b800ca360 msgr2=0x7f0b80156580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:59.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.749+0000 7f0b84c00640 1 --2- 172.21.15.7:0/2231466790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b800ca360 0x7f0b80156580 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800dbc0 tx=0x7f0b6800a620 comp rx=0 tx=0).stop 2024-09-10T21:42:59.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.750+0000 7f0b84c00640 1 -- 172.21.15.7:0/2231466790 shutdown_connections 2024-09-10T21:42:59.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.750+0000 7f0b84c00640 1 --2- 172.21.15.7:0/2231466790 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0b5c077f70 0x7f0b5c07a430 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.750+0000 7f0b84c00640 1 --2- 172.21.15.7:0/2231466790 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b8015da10 0x7f0b8015c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.750+0000 7f0b84c00640 1 --2- 172.21.15.7:0/2231466790 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b800ca360 0x7f0b80156580 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.750+0000 7f0b84c00640 1 --2- 172.21.15.7:0/2231466790 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b80070600 0x7f0b8014e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.750+0000 7f0b84c00640 1 -- 172.21.15.7:0/2231466790 >> 172.21.15.7:0/2231466790 conn(0x7f0b800b16d0 msgr2=0x7f0b80157ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:59.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.750+0000 7f0b84c00640 1 -- 172.21.15.7:0/2231466790 shutdown_connections 2024-09-10T21:42:59.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.750+0000 7f0b84c00640 1 -- 172.21.15.7:0/2231466790 wait complete. 2024-09-10T21:42:59.767 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:42:59.768 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:42:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.967+0000 7fbe64400640 1 -- 172.21.15.7:0/2038248319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe5c0cd750 msgr2=0x7fbe5c0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.967+0000 7fbe64400640 1 --2- 172.21.15.7:0/2038248319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe5c0cd750 0x7fbe5c0ccef0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c0c3650 tx=0x7fbe4802a4d0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.967+0000 7fbe64400640 1 -- 172.21.15.7:0/2038248319 shutdown_connections 2024-09-10T21:42:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.967+0000 7fbe64400640 1 --2- 172.21.15.7:0/2038248319 >> v2:172.21.15.133:3300/0 conn(0x7fbe5c15fda0 0x7fbe5c160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.967+0000 7fbe64400640 1 --2- 172.21.15.7:0/2038248319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe5c0cd750 0x7fbe5c0ccef0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.967+0000 7fbe64400640 1 -- 172.21.15.7:0/2038248319 >> 172.21.15.7:0/2038248319 conn(0x7fbe5c0b16d0 msgr2=0x7fbe5c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.967+0000 7fbe64400640 1 -- 172.21.15.7:0/2038248319 shutdown_connections 2024-09-10T21:42:59.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.967+0000 7fbe64400640 1 -- 172.21.15.7:0/2038248319 wait complete. 2024-09-10T21:42:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.969+0000 7fbe64400640 1 Processor -- start 2024-09-10T21:42:59.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.969+0000 7fbe64400640 1 -- start start 2024-09-10T21:42:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.970+0000 7fbe64400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe5c0cd750 0x7fbe5c14db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:59.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.970+0000 7fbe64400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe5c15bc90 0x7fbe5c14d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.970+0000 7fbe64400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe5c15fda0 0x7fbe5c0ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.970+0000 7fbe64400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c06fe70 con 0x7fbe5c15fda0 2024-09-10T21:42:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.970+0000 7fbe64400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbe5c06fcf0 con 0x7fbe5c0cd750 2024-09-10T21:42:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.970+0000 7fbe64400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fbe5c06fff0 con 0x7fbe5c15bc90 2024-09-10T21:42:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.970+0000 7fbe62400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe5c15fda0 0x7fbe5c0ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.970+0000 7fbe62400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe5c15fda0 0x7fbe5c0ad8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57518/0 (socket says 172.21.15.7:57518) 2024-09-10T21:42:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.970+0000 7fbe62400640 1 -- 172.21.15.7:0/1883395562 learned_addr learned my addr 172.21.15.7:0/1883395562 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:42:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.970+0000 7fbe61a00640 1 --2- 172.21.15.7:0/1883395562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe5c0cd750 0x7fbe5c14db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.970+0000 7fbe61000640 1 --2- 172.21.15.7:0/1883395562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe5c15bc90 0x7fbe5c14d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:59.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.970+0000 7fbe62400640 1 -- 172.21.15.7:0/1883395562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe5c15bc90 msgr2=0x7fbe5c14d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.970+0000 7fbe62400640 1 --2- 172.21.15.7:0/1883395562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe5c15bc90 0x7fbe5c14d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.970+0000 7fbe62400640 1 -- 172.21.15.7:0/1883395562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe5c0cd750 msgr2=0x7fbe5c14db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.970+0000 7fbe62400640 1 --2- 172.21.15.7:0/1883395562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe5c0cd750 0x7fbe5c14db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.970+0000 7fbe62400640 1 -- 172.21.15.7:0/1883395562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe5c0c47a0 con 0x7fbe5c15fda0 2024-09-10T21:42:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.970+0000 7fbe61000640 1 --2- 172.21.15.7:0/1883395562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe5c15bc90 0x7fbe5c14d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:42:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.971+0000 7fbe62400640 1 --2- 172.21.15.7:0/1883395562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe5c15fda0 0x7fbe5c0ad8f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400e8b0 tx=0x7fbe5400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.971+0000 7fbe59600640 1 -- 172.21.15.7:0/1883395562 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54004d20 con 0x7fbe5c15fda0 2024-09-10T21:42:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.971+0000 7fbe59600640 1 -- 172.21.15.7:0/1883395562 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe5400ca60 con 0x7fbe5c15fda0 2024-09-10T21:42:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.971+0000 7fbe59600640 1 -- 172.21.15.7:0/1883395562 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe54005020 con 0x7fbe5c15fda0 2024-09-10T21:42:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.971+0000 7fbe64400640 1 -- 172.21.15.7:0/1883395562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5c0c4a80 con 0x7fbe5c15fda0 2024-09-10T21:42:59.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.971+0000 7fbe64400640 1 -- 172.21.15.7:0/1883395562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe5c0c5040 con 0x7fbe5c15fda0 2024-09-10T21:42:59.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.972+0000 7fbe64400640 1 -- 172.21.15.7:0/1883395562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe5c06a1f0 con 0x7fbe5c15fda0 2024-09-10T21:42:59.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.973+0000 7fbe59600640 1 -- 172.21.15.7:0/1883395562 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fbe5400f040 con 0x7fbe5c15fda0 2024-09-10T21:42:59.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.974+0000 7fbe59600640 1 --2- 172.21.15.7:0/1883395562 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbe3c077f70 0x7fbe3c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:59.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.974+0000 7fbe59600640 1 -- 172.21.15.7:0/1883395562 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fbe5409a260 con 0x7fbe5c15fda0 2024-09-10T21:42:59.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.974+0000 7fbe61a00640 1 --2- 172.21.15.7:0/1883395562 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbe3c077f70 0x7fbe3c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:59.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.975+0000 7fbe61a00640 1 --2- 172.21.15.7:0/1883395562 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbe3c077f70 0x7fbe3c07a430 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbe50004ea0 tx=0x7fbe50009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:59.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:42:59.978+0000 7fbe59600640 1 -- 172.21.15.7:0/1883395562 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fbe5c06a1f0 con 0x7fbe5c15fda0 2024-09-10T21:43:00.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.169+0000 7fbe64400640 1 -- 172.21.15.7:0/1883395562 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbe5c070510 con 0x7fbe3c077f70 2024-09-10T21:43:00.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.174+0000 7fbe59600640 1 -- 172.21.15.7:0/1883395562 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251934 (secure 0 0 0) 0x7fbe5c070510 con 0x7fbe3c077f70 2024-09-10T21:43:00.179 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:00.181 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:43:00.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.179+0000 7fbe64400640 1 -- 172.21.15.7:0/1883395562 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbe3c077f70 msgr2=0x7fbe3c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:00.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.179+0000 7fbe64400640 1 --2- 172.21.15.7:0/1883395562 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbe3c077f70 0x7fbe3c07a430 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbe50004ea0 tx=0x7fbe50009040 comp rx=0 tx=0).stop 2024-09-10T21:43:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.179+0000 7fbe64400640 1 -- 172.21.15.7:0/1883395562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe5c15fda0 msgr2=0x7fbe5c0ad8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.179+0000 7fbe64400640 1 --2- 172.21.15.7:0/1883395562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe5c15fda0 0x7fbe5c0ad8f0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbe5400e8b0 tx=0x7fbe5400ed70 comp rx=0 tx=0).stop 2024-09-10T21:43:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.179+0000 7fbe64400640 1 -- 172.21.15.7:0/1883395562 shutdown_connections 2024-09-10T21:43:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.179+0000 7fbe64400640 1 --2- 172.21.15.7:0/1883395562 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbe3c077f70 0x7fbe3c07a430 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.179+0000 7fbe64400640 1 --2- 172.21.15.7:0/1883395562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe5c15fda0 0x7fbe5c0ad8f0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.179+0000 7fbe64400640 1 --2- 172.21.15.7:0/1883395562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe5c15bc90 0x7fbe5c14d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.179+0000 7fbe64400640 1 --2- 172.21.15.7:0/1883395562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe5c0cd750 0x7fbe5c14db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.179+0000 7fbe64400640 1 -- 172.21.15.7:0/1883395562 >> 172.21.15.7:0/1883395562 conn(0x7fbe5c0b16d0 msgr2=0x7fbe5c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.179+0000 7fbe64400640 1 -- 172.21.15.7:0/1883395562 shutdown_connections 2024-09-10T21:43:00.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.179+0000 7fbe64400640 1 -- 172.21.15.7:0/1883395562 wait complete. 2024-09-10T21:43:00.195 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-10T21:42:59.866926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240976,"kb_used_data":3928,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733744,"statfs":{"total":773094113280,"available":772847353856,"internally_reserved":0,"allocated":4022272,"data_stored":2243388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.978723"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.925894+0000","last_change":"2024-09-10T21:42:55.919761+0000","last_active":"2024-09-10T21:42:55.925894+0000","last_peered":"2024-09-10T21:42:55.925894+0000","last_clean":"2024-09-10T21:42:55.925894+0000","last_became_active":"2024-09-10T21:42:55.919462+0000","last_became_peered":"2024-09-10T21:42:55.919462+0000","last_unstale":"2024-09-10T21:42:55.925894+0000","last_undegraded":"2024-09-10T21:42:55.925894+0000","last_fullsized":"2024-09-10T21:42:55.925894+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:27.227573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.945243+0000","last_change":"2024-09-10T21:42:55.933968+0000","last_active":"2024-09-10T21:42:55.945243+0000","last_peered":"2024-09-10T21:42:55.945243+0000","last_clean":"2024-09-10T21:42:55.945243+0000","last_became_active":"2024-09-10T21:42:55.931328+0000","last_became_peered":"2024-09-10T21:42:55.931328+0000","last_unstale":"2024-09-10T21:42:55.945243+0000","last_undegraded":"2024-09-10T21:42:55.945243+0000","last_fullsized":"2024-09-10T21:42:55.945243+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:39.175452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925810+0000","last_change":"2024-09-10T21:42:55.923668+0000","last_active":"2024-09-10T21:42:56.925810+0000","last_peered":"2024-09-10T21:42:56.925810+0000","last_clean":"2024-09-10T21:42:56.925810+0000","last_became_active":"2024-09-10T21:42:55.920236+0000","last_became_peered":"2024-09-10T21:42:55.920236+0000","last_unstale":"2024-09-10T21:42:56.925810+0000","last_undegraded":"2024-09-10T21:42:56.925810+0000","last_fullsized":"2024-09-10T21:42:56.925810+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:05.352427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933316+0000","last_change":"2024-09-10T21:42:55.921945+0000","last_active":"2024-09-10T21:42:56.933316+0000","last_peered":"2024-09-10T21:42:56.933316+0000","last_clean":"2024-09-10T21:42:56.933316+0000","last_became_active":"2024-09-10T21:42:55.921643+0000","last_became_peered":"2024-09-10T21:42:55.921643+0000","last_unstale":"2024-09-10T21:42:56.933316+0000","last_undegraded":"2024-09-10T21:42:56.933316+0000","last_fullsized":"2024-09-10T21:42:56.933316+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:39.244361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.919624+0000","last_change":"2024-09-10T21:42:55.932409+0000","last_active":"2024-09-10T21:42:56.919624+0000","last_peered":"2024-09-10T21:42:56.919624+0000","last_clean":"2024-09-10T21:42:56.919624+0000","last_became_active":"2024-09-10T21:42:55.932122+0000","last_became_peered":"2024-09-10T21:42:55.932122+0000","last_unstale":"2024-09-10T21:42:56.919624+0000","last_undegraded":"2024-09-10T21:42:56.919624+0000","last_fullsized":"2024-09-10T21:42:56.919624+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:40.173393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.938048+0000","last_change":"2024-09-10T21:42:55.929536+0000","last_active":"2024-09-10T21:42:55.938048+0000","last_peered":"2024-09-10T21:42:55.938048+0000","last_clean":"2024-09-10T21:42:55.938048+0000","last_became_active":"2024-09-10T21:42:55.929201+0000","last_became_peered":"2024-09-10T21:42:55.929201+0000","last_unstale":"2024-09-10T21:42:55.938048+0000","last_undegraded":"2024-09-10T21:42:55.938048+0000","last_fullsized":"2024-09-10T21:42:55.938048+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:04.496067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.937887+0000","last_change":"2024-09-10T21:42:55.930869+0000","last_active":"2024-09-10T21:42:55.937887+0000","last_peered":"2024-09-10T21:42:55.937887+0000","last_clean":"2024-09-10T21:42:55.937887+0000","last_became_active":"2024-09-10T21:42:55.930615+0000","last_became_peered":"2024-09-10T21:42:55.930615+0000","last_unstale":"2024-09-10T21:42:55.937887+0000","last_undegraded":"2024-09-10T21:42:55.937887+0000","last_fullsized":"2024-09-10T21:42:55.937887+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:04.496067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.927789+0000","last_change":"2024-09-10T21:42:55.923253+0000","last_active":"2024-09-10T21:42:55.927789+0000","last_peered":"2024-09-10T21:42:55.927789+0000","last_clean":"2024-09-10T21:42:55.927789+0000","last_became_active":"2024-09-10T21:42:55.922923+0000","last_became_peered":"2024-09-10T21:42:55.922923+0000","last_unstale":"2024-09-10T21:42:55.927789+0000","last_undegraded":"2024-09-10T21:42:55.927789+0000","last_fullsized":"2024-09-10T21:42:55.927789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:02.328070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.926411+0000","last_change":"2024-09-10T21:42:55.917855+0000","last_active":"2024-09-10T21:42:56.926411+0000","last_peered":"2024-09-10T21:42:56.926411+0000","last_clean":"2024-09-10T21:42:56.926411+0000","last_became_active":"2024-09-10T21:42:55.916101+0000","last_became_peered":"2024-09-10T21:42:55.916101+0000","last_unstale":"2024-09-10T21:42:56.926411+0000","last_undegraded":"2024-09-10T21:42:56.926411+0000","last_fullsized":"2024-09-10T21:42:56.926411+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:48:39.274454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933559+0000","last_change":"2024-09-10T21:42:55.939027+0000","last_active":"2024-09-10T21:42:56.933559+0000","last_peered":"2024-09-10T21:42:56.933559+0000","last_clean":"2024-09-10T21:42:56.933559+0000","last_became_active":"2024-09-10T21:42:55.938713+0000","last_became_peered":"2024-09-10T21:42:55.938713+0000","last_unstale":"2024-09-10T21:42:56.933559+0000","last_undegraded":"2024-09-10T21:42:56.933559+0000","last_fullsized":"2024-09-10T21:42:56.933559+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:48:39.274454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.935092+0000","last_change":"2024-09-10T21:42:55.929796+0000","last_active":"2024-09-10T21:42:55.935092+0000","last_peered":"2024-09-10T21:42:55.935092+0000","last_clean":"2024-09-10T21:42:55.935092+0000","last_became_active":"2024-09-10T21:42:55.929462+0000","last_became_peered":"2024-09-10T21:42:55.929462+0000","last_unstale":"2024-09-10T21:42:55.935092+0000","last_undegraded":"2024-09-10T21:42:55.935092+0000","last_fullsized":"2024-09-10T21:42:55.935092+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:08.093849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.926252+0000","last_change":"2024-09-10T21:42:55.934439+0000","last_active":"2024-09-10T21:42:56.926252+0000","last_peered":"2024-09-10T21:42:56.926252+0000","last_clean":"2024-09-10T21:42:56.926252+0000","last_became_active":"2024-09-10T21:42:55.934173+0000","last_became_peered":"2024-09-10T21:42:55.934173+0000","last_unstale":"2024-09-10T21:42:56.926252+0000","last_undegraded":"2024-09-10T21:42:56.926252+0000","last_fullsized":"2024-09-10T21:42:56.926252+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:00.111962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.919753+0000","last_change":"2024-09-10T21:42:55.917418+0000","last_active":"2024-09-10T21:42:56.919753+0000","last_peered":"2024-09-10T21:42:56.919753+0000","last_clean":"2024-09-10T21:42:56.919753+0000","last_became_active":"2024-09-10T21:42:55.917199+0000","last_became_peered":"2024-09-10T21:42:55.917199+0000","last_unstale":"2024-09-10T21:42:56.919753+0000","last_undegraded":"2024-09-10T21:42:56.919753+0000","last_fullsized":"2024-09-10T21:42:56.919753+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:53.037933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.938080+0000","last_change":"2024-09-10T21:42:55.929521+0000","last_active":"2024-09-10T21:42:55.938080+0000","last_peered":"2024-09-10T21:42:55.938080+0000","last_clean":"2024-09-10T21:42:55.938080+0000","last_became_active":"2024-09-10T21:42:55.929188+0000","last_became_peered":"2024-09-10T21:42:55.929188+0000","last_unstale":"2024-09-10T21:42:55.938080+0000","last_undegraded":"2024-09-10T21:42:55.938080+0000","last_fullsized":"2024-09-10T21:42:55.938080+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:08.093849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.924698+0000","last_change":"2024-09-10T21:42:55.928378+0000","last_active":"2024-09-10T21:42:56.924698+0000","last_peered":"2024-09-10T21:42:56.924698+0000","last_clean":"2024-09-10T21:42:56.924698+0000","last_became_active":"2024-09-10T21:42:55.928035+0000","last_became_peered":"2024-09-10T21:42:55.928035+0000","last_unstale":"2024-09-10T21:42:56.924698+0000","last_undegraded":"2024-09-10T21:42:56.924698+0000","last_fullsized":"2024-09-10T21:42:56.924698+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:34.003809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.943699+0000","last_change":"2024-09-10T21:42:55.938217+0000","last_active":"2024-09-10T21:42:55.943699+0000","last_peered":"2024-09-10T21:42:55.943699+0000","last_clean":"2024-09-10T21:42:55.943699+0000","last_became_active":"2024-09-10T21:42:55.937896+0000","last_became_peered":"2024-09-10T21:42:55.937896+0000","last_unstale":"2024-09-10T21:42:55.943699+0000","last_undegraded":"2024-09-10T21:42:55.943699+0000","last_fullsized":"2024-09-10T21:42:55.943699+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:29.498336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.937963+0000","last_change":"2024-09-10T21:42:55.928076+0000","last_active":"2024-09-10T21:42:55.937963+0000","last_peered":"2024-09-10T21:42:55.937963+0000","last_clean":"2024-09-10T21:42:55.937963+0000","last_became_active":"2024-09-10T21:42:55.923824+0000","last_became_peered":"2024-09-10T21:42:55.923824+0000","last_unstale":"2024-09-10T21:42:55.937963+0000","last_undegraded":"2024-09-10T21:42:55.937963+0000","last_fullsized":"2024-09-10T21:42:55.937963+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:36:39.159143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.945188+0000","last_change":"2024-09-10T21:42:55.931745+0000","last_active":"2024-09-10T21:42:55.945188+0000","last_peered":"2024-09-10T21:42:55.945188+0000","last_clean":"2024-09-10T21:42:55.945188+0000","last_became_active":"2024-09-10T21:42:55.930731+0000","last_became_peered":"2024-09-10T21:42:55.930731+0000","last_unstale":"2024-09-10T21:42:55.945188+0000","last_undegraded":"2024-09-10T21:42:55.945188+0000","last_fullsized":"2024-09-10T21:42:55.945188+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:14.793883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.945560+0000","last_change":"2024-09-10T21:42:55.931885+0000","last_active":"2024-09-10T21:42:55.945560+0000","last_peered":"2024-09-10T21:42:55.945560+0000","last_clean":"2024-09-10T21:42:55.945560+0000","last_became_active":"2024-09-10T21:42:55.931518+0000","last_became_peered":"2024-09-10T21:42:55.931518+0000","last_unstale":"2024-09-10T21:42:55.945560+0000","last_undegraded":"2024-09-10T21:42:55.945560+0000","last_fullsized":"2024-09-10T21:42:55.945560+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:18.552544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.919806+0000","last_change":"2024-09-10T21:42:55.918959+0000","last_active":"2024-09-10T21:42:56.919806+0000","last_peered":"2024-09-10T21:42:56.919806+0000","last_clean":"2024-09-10T21:42:56.919806+0000","last_became_active":"2024-09-10T21:42:55.918643+0000","last_became_peered":"2024-09-10T21:42:55.918643+0000","last_unstale":"2024-09-10T21:42:56.919806+0000","last_undegraded":"2024-09-10T21:42:56.919806+0000","last_fullsized":"2024-09-10T21:42:56.919806+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:24.608749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925132+0000","last_change":"2024-09-10T21:42:55.921167+0000","last_active":"2024-09-10T21:42:56.925132+0000","last_peered":"2024-09-10T21:42:56.925132+0000","last_clean":"2024-09-10T21:42:56.925132+0000","last_became_active":"2024-09-10T21:42:55.920760+0000","last_became_peered":"2024-09-10T21:42:55.920760+0000","last_unstale":"2024-09-10T21:42:56.925132+0000","last_undegraded":"2024-09-10T21:42:56.925132+0000","last_fullsized":"2024-09-10T21:42:56.925132+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:30.389091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925343+0000","last_change":"2024-09-10T21:42:55.932797+0000","last_active":"2024-09-10T21:42:56.925343+0000","last_peered":"2024-09-10T21:42:56.925343+0000","last_clean":"2024-09-10T21:42:56.925343+0000","last_became_active":"2024-09-10T21:42:55.931802+0000","last_became_peered":"2024-09-10T21:42:55.931802+0000","last_unstale":"2024-09-10T21:42:56.925343+0000","last_undegraded":"2024-09-10T21:42:56.925343+0000","last_fullsized":"2024-09-10T21:42:56.925343+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:50.985138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.930561+0000","last_change":"2024-09-10T21:42:55.919179+0000","last_active":"2024-09-10T21:42:56.930561+0000","last_peered":"2024-09-10T21:42:56.930561+0000","last_clean":"2024-09-10T21:42:56.930561+0000","last_became_active":"2024-09-10T21:42:55.918455+0000","last_became_peered":"2024-09-10T21:42:55.918455+0000","last_unstale":"2024-09-10T21:42:56.930561+0000","last_undegraded":"2024-09-10T21:42:56.930561+0000","last_fullsized":"2024-09-10T21:42:56.930561+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:36.719449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.924497+0000","last_change":"2024-09-10T21:42:55.921609+0000","last_active":"2024-09-10T21:42:56.924497+0000","last_peered":"2024-09-10T21:42:56.924497+0000","last_clean":"2024-09-10T21:42:56.924497+0000","last_became_active":"2024-09-10T21:42:55.918479+0000","last_became_peered":"2024-09-10T21:42:55.918479+0000","last_unstale":"2024-09-10T21:42:56.924497+0000","last_undegraded":"2024-09-10T21:42:56.924497+0000","last_fullsized":"2024-09-10T21:42:56.924497+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:47.086671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933777+0000","last_change":"2024-09-10T21:42:55.919209+0000","last_active":"2024-09-10T21:42:56.933777+0000","last_peered":"2024-09-10T21:42:56.933777+0000","last_clean":"2024-09-10T21:42:56.933777+0000","last_became_active":"2024-09-10T21:42:55.917683+0000","last_became_peered":"2024-09-10T21:42:55.917683+0000","last_unstale":"2024-09-10T21:42:56.933777+0000","last_undegraded":"2024-09-10T21:42:56.933777+0000","last_fullsized":"2024-09-10T21:42:56.933777+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:27.675336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.945343+0000","last_change":"2024-09-10T21:42:55.931588+0000","last_active":"2024-09-10T21:42:55.945343+0000","last_peered":"2024-09-10T21:42:55.945343+0000","last_clean":"2024-09-10T21:42:55.945343+0000","last_became_active":"2024-09-10T21:42:55.931042+0000","last_became_peered":"2024-09-10T21:42:55.931042+0000","last_unstale":"2024-09-10T21:42:55.945343+0000","last_undegraded":"2024-09-10T21:42:55.945343+0000","last_fullsized":"2024-09-10T21:42:55.945343+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:58.268834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933739+0000","last_change":"2024-09-10T21:42:55.924810+0000","last_active":"2024-09-10T21:42:56.933739+0000","last_peered":"2024-09-10T21:42:56.933739+0000","last_clean":"2024-09-10T21:42:56.933739+0000","last_became_active":"2024-09-10T21:42:55.922683+0000","last_became_peered":"2024-09-10T21:42:55.922683+0000","last_unstale":"2024-09-10T21:42:56.933739+0000","last_undegraded":"2024-09-10T21:42:56.933739+0000","last_fullsized":"2024-09-10T21:42:56.933739+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:55.359591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.926397+0000","last_change":"2024-09-10T21:42:55.925007+0000","last_active":"2024-09-10T21:42:56.926397+0000","last_peered":"2024-09-10T21:42:56.926397+0000","last_clean":"2024-09-10T21:42:56.926397+0000","last_became_active":"2024-09-10T21:42:55.924748+0000","last_became_peered":"2024-09-10T21:42:55.924748+0000","last_unstale":"2024-09-10T21:42:56.926397+0000","last_undegraded":"2024-09-10T21:42:56.926397+0000","last_fullsized":"2024-09-10T21:42:56.926397+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:11.876440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925246+0000","last_change":"2024-09-10T21:42:55.934668+0000","last_active":"2024-09-10T21:42:56.925246+0000","last_peered":"2024-09-10T21:42:56.925246+0000","last_clean":"2024-09-10T21:42:56.925246+0000","last_became_active":"2024-09-10T21:42:55.934204+0000","last_became_peered":"2024-09-10T21:42:55.934204+0000","last_unstale":"2024-09-10T21:42:56.925246+0000","last_undegraded":"2024-09-10T21:42:56.925246+0000","last_fullsized":"2024-09-10T21:42:56.925246+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:50:01.404871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.922622+0000","last_change":"2024-09-10T21:42:55.918528+0000","last_active":"2024-09-10T21:42:55.922622+0000","last_peered":"2024-09-10T21:42:55.922622+0000","last_clean":"2024-09-10T21:42:55.922622+0000","last_became_active":"2024-09-10T21:42:55.918147+0000","last_became_peered":"2024-09-10T21:42:55.918147+0000","last_unstale":"2024-09-10T21:42:55.922622+0000","last_undegraded":"2024-09-10T21:42:55.922622+0000","last_fullsized":"2024-09-10T21:42:55.922622+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:27.104414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.926077+0000","last_change":"2024-09-10T21:42:55.917548+0000","last_active":"2024-09-10T21:42:55.926077+0000","last_peered":"2024-09-10T21:42:55.926077+0000","last_clean":"2024-09-10T21:42:55.926077+0000","last_became_active":"2024-09-10T21:42:55.913219+0000","last_became_peered":"2024-09-10T21:42:55.913219+0000","last_unstale":"2024-09-10T21:42:55.926077+0000","last_undegraded":"2024-09-10T21:42:55.926077+0000","last_fullsized":"2024-09-10T21:42:55.926077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:17.471225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.927766+0000","last_change":"2024-09-10T21:42:55.922444+0000","last_active":"2024-09-10T21:42:55.927766+0000","last_peered":"2024-09-10T21:42:55.927766+0000","last_clean":"2024-09-10T21:42:55.927766+0000","last_became_active":"2024-09-10T21:42:55.922199+0000","last_became_peered":"2024-09-10T21:42:55.922199+0000","last_unstale":"2024-09-10T21:42:55.927766+0000","last_undegraded":"2024-09-10T21:42:55.927766+0000","last_fullsized":"2024-09-10T21:42:55.927766+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:23.262669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.930883+0000","last_change":"2024-09-10T21:42:55.923894+0000","last_active":"2024-09-10T21:42:56.930883+0000","last_peered":"2024-09-10T21:42:56.930883+0000","last_clean":"2024-09-10T21:42:56.930883+0000","last_became_active":"2024-09-10T21:42:55.923621+0000","last_became_peered":"2024-09-10T21:42:55.923621+0000","last_unstale":"2024-09-10T21:42:56.930883+0000","last_undegraded":"2024-09-10T21:42:56.930883+0000","last_fullsized":"2024-09-10T21:42:56.930883+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:31.127604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.943463+0000","last_change":"2024-09-10T21:42:54.911491+0000","last_active":"2024-09-10T21:42:55.943463+0000","last_peered":"2024-09-10T21:42:55.943463+0000","last_clean":"2024-09-10T21:42:55.943463+0000","last_became_active":"2024-09-10T21:42:54.911178+0000","last_became_peered":"2024-09-10T21:42:54.911178+0000","last_unstale":"2024-09-10T21:42:55.943463+0000","last_undegraded":"2024-09-10T21:42:55.943463+0000","last_fullsized":"2024-09-10T21:42:55.943463+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:34.519141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.929570+0000","last_change":"2024-09-10T21:42:55.923625+0000","last_active":"2024-09-10T21:42:55.929570+0000","last_peered":"2024-09-10T21:42:55.929570+0000","last_clean":"2024-09-10T21:42:55.929570+0000","last_became_active":"2024-09-10T21:42:55.923358+0000","last_became_peered":"2024-09-10T21:42:55.923358+0000","last_unstale":"2024-09-10T21:42:55.929570+0000","last_undegraded":"2024-09-10T21:42:55.929570+0000","last_fullsized":"2024-09-10T21:42:55.929570+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:46.613096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.930904+0000","last_change":"2024-09-10T21:42:54.918303+0000","last_active":"2024-09-10T21:42:56.930904+0000","last_peered":"2024-09-10T21:42:56.930904+0000","last_clean":"2024-09-10T21:42:56.930904+0000","last_became_active":"2024-09-10T21:42:54.916342+0000","last_became_peered":"2024-09-10T21:42:54.916342+0000","last_unstale":"2024-09-10T21:42:56.930904+0000","last_undegraded":"2024-09-10T21:42:56.930904+0000","last_fullsized":"2024-09-10T21:42:56.930904+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:21.011406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925830+0000","last_change":"2024-09-10T21:42:55.929216+0000","last_active":"2024-09-10T21:42:56.925830+0000","last_peered":"2024-09-10T21:42:56.925830+0000","last_clean":"2024-09-10T21:42:56.925830+0000","last_became_active":"2024-09-10T21:42:55.928921+0000","last_became_peered":"2024-09-10T21:42:55.928921+0000","last_unstale":"2024-09-10T21:42:56.925830+0000","last_undegraded":"2024-09-10T21:42:56.925830+0000","last_fullsized":"2024-09-10T21:42:56.925830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:39.244361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.945617+0000","last_change":"2024-09-10T21:42:54.918085+0000","last_active":"2024-09-10T21:42:55.945617+0000","last_peered":"2024-09-10T21:42:55.945617+0000","last_clean":"2024-09-10T21:42:55.945617+0000","last_became_active":"2024-09-10T21:42:54.917769+0000","last_became_peered":"2024-09-10T21:42:54.917769+0000","last_unstale":"2024-09-10T21:42:55.945617+0000","last_undegraded":"2024-09-10T21:42:55.945617+0000","last_fullsized":"2024-09-10T21:42:55.945617+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:01.322169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933867+0000","last_change":"2024-09-10T21:42:55.922965+0000","last_active":"2024-09-10T21:42:56.933867+0000","last_peered":"2024-09-10T21:42:56.933867+0000","last_clean":"2024-09-10T21:42:56.933867+0000","last_became_active":"2024-09-10T21:42:55.922297+0000","last_became_peered":"2024-09-10T21:42:55.922297+0000","last_unstale":"2024-09-10T21:42:56.933867+0000","last_undegraded":"2024-09-10T21:42:56.933867+0000","last_fullsized":"2024-09-10T21:42:56.933867+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:51.881372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.919918+0000","last_change":"2024-09-10T21:42:54.916465+0000","last_active":"2024-09-10T21:42:56.919918+0000","last_peered":"2024-09-10T21:42:56.919918+0000","last_clean":"2024-09-10T21:42:56.919918+0000","last_became_active":"2024-09-10T21:42:54.916138+0000","last_became_peered":"2024-09-10T21:42:54.916138+0000","last_unstale":"2024-09-10T21:42:56.919918+0000","last_undegraded":"2024-09-10T21:42:56.919918+0000","last_fullsized":"2024-09-10T21:42:56.919918+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:25.772752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.935027+0000","last_change":"2024-09-10T21:42:55.927341+0000","last_active":"2024-09-10T21:42:55.935027+0000","last_peered":"2024-09-10T21:42:55.935027+0000","last_clean":"2024-09-10T21:42:55.935027+0000","last_became_active":"2024-09-10T21:42:55.927069+0000","last_became_peered":"2024-09-10T21:42:55.927069+0000","last_unstale":"2024-09-10T21:42:55.935027+0000","last_undegraded":"2024-09-10T21:42:55.935027+0000","last_fullsized":"2024-09-10T21:42:55.935027+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:56.115221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925243+0000","last_change":"2024-09-10T21:42:54.914690+0000","last_active":"2024-09-10T21:42:56.925243+0000","last_peered":"2024-09-10T21:42:56.925243+0000","last_clean":"2024-09-10T21:42:56.925243+0000","last_became_active":"2024-09-10T21:42:54.913846+0000","last_became_peered":"2024-09-10T21:42:54.913846+0000","last_unstale":"2024-09-10T21:42:56.925243+0000","last_undegraded":"2024-09-10T21:42:56.925243+0000","last_fullsized":"2024-09-10T21:42:56.925243+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:23.602849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925696+0000","last_change":"2024-09-10T21:42:55.925685+0000","last_active":"2024-09-10T21:42:56.925696+0000","last_peered":"2024-09-10T21:42:56.925696+0000","last_clean":"2024-09-10T21:42:56.925696+0000","last_became_active":"2024-09-10T21:42:55.924888+0000","last_became_peered":"2024-09-10T21:42:55.924888+0000","last_unstale":"2024-09-10T21:42:56.925696+0000","last_undegraded":"2024-09-10T21:42:56.925696+0000","last_fullsized":"2024-09-10T21:42:56.925696+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:26.729812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925463+0000","last_change":"2024-09-10T21:42:54.913475+0000","last_active":"2024-09-10T21:42:56.925463+0000","last_peered":"2024-09-10T21:42:56.925463+0000","last_clean":"2024-09-10T21:42:56.925463+0000","last_became_active":"2024-09-10T21:42:54.913195+0000","last_became_peered":"2024-09-10T21:42:54.913195+0000","last_unstale":"2024-09-10T21:42:56.925463+0000","last_undegraded":"2024-09-10T21:42:56.925463+0000","last_fullsized":"2024-09-10T21:42:56.925463+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:45.328840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925422+0000","last_change":"2024-09-10T21:42:55.918347+0000","last_active":"2024-09-10T21:42:56.925422+0000","last_peered":"2024-09-10T21:42:56.925422+0000","last_clean":"2024-09-10T21:42:56.925422+0000","last_became_active":"2024-09-10T21:42:55.917738+0000","last_became_peered":"2024-09-10T21:42:55.917738+0000","last_unstale":"2024-09-10T21:42:56.925422+0000","last_undegraded":"2024-09-10T21:42:56.925422+0000","last_fullsized":"2024-09-10T21:42:56.925422+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:31.504136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.915627+0000","last_change":"2024-09-10T21:42:54.921414+0000","last_active":"2024-09-10T21:42:55.915627+0000","last_peered":"2024-09-10T21:42:55.915627+0000","last_clean":"2024-09-10T21:42:55.915627+0000","last_became_active":"2024-09-10T21:42:54.921123+0000","last_became_peered":"2024-09-10T21:42:54.921123+0000","last_unstale":"2024-09-10T21:42:55.915627+0000","last_undegraded":"2024-09-10T21:42:55.915627+0000","last_fullsized":"2024-09-10T21:42:55.915627+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:02.988891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933160+0000","last_change":"2024-09-10T21:42:55.926817+0000","last_active":"2024-09-10T21:42:56.933160+0000","last_peered":"2024-09-10T21:42:56.933160+0000","last_clean":"2024-09-10T21:42:56.933160+0000","last_became_active":"2024-09-10T21:42:55.926516+0000","last_became_peered":"2024-09-10T21:42:55.926516+0000","last_unstale":"2024-09-10T21:42:56.933160+0000","last_undegraded":"2024-09-10T21:42:56.933160+0000","last_fullsized":"2024-09-10T21:42:56.933160+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:31.504136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.924913+0000","last_change":"2024-09-10T21:42:54.903779+0000","last_active":"2024-09-10T21:42:56.924913+0000","last_peered":"2024-09-10T21:42:56.924913+0000","last_clean":"2024-09-10T21:42:56.924913+0000","last_became_active":"2024-09-10T21:42:54.903313+0000","last_became_peered":"2024-09-10T21:42:54.903313+0000","last_unstale":"2024-09-10T21:42:56.924913+0000","last_undegraded":"2024-09-10T21:42:56.924913+0000","last_fullsized":"2024-09-10T21:42:56.924913+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:08.052040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.926743+0000","last_change":"2024-09-10T21:42:55.939246+0000","last_active":"2024-09-10T21:42:56.926743+0000","last_peered":"2024-09-10T21:42:56.926743+0000","last_clean":"2024-09-10T21:42:56.926743+0000","last_became_active":"2024-09-10T21:42:55.938980+0000","last_became_peered":"2024-09-10T21:42:55.938980+0000","last_unstale":"2024-09-10T21:42:56.926743+0000","last_undegraded":"2024-09-10T21:42:56.926743+0000","last_fullsized":"2024-09-10T21:42:56.926743+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:45.531431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.921569+0000","last_change":"2024-09-10T21:42:54.915558+0000","last_active":"2024-09-10T21:42:55.921569+0000","last_peered":"2024-09-10T21:42:55.921569+0000","last_clean":"2024-09-10T21:42:55.921569+0000","last_became_active":"2024-09-10T21:42:54.915231+0000","last_became_peered":"2024-09-10T21:42:54.915231+0000","last_unstale":"2024-09-10T21:42:55.921569+0000","last_undegraded":"2024-09-10T21:42:55.921569+0000","last_fullsized":"2024-09-10T21:42:55.921569+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:55.109320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.945489+0000","last_change":"2024-09-10T21:42:55.931769+0000","last_active":"2024-09-10T21:42:55.945489+0000","last_peered":"2024-09-10T21:42:55.945489+0000","last_clean":"2024-09-10T21:42:55.945489+0000","last_became_active":"2024-09-10T21:42:55.931429+0000","last_became_peered":"2024-09-10T21:42:55.931429+0000","last_unstale":"2024-09-10T21:42:55.945489+0000","last_undegraded":"2024-09-10T21:42:55.945489+0000","last_fullsized":"2024-09-10T21:42:55.945489+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:18.457525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.931020+0000","last_change":"2024-09-10T21:42:54.919588+0000","last_active":"2024-09-10T21:42:56.931020+0000","last_peered":"2024-09-10T21:42:56.931020+0000","last_clean":"2024-09-10T21:42:56.931020+0000","last_became_active":"2024-09-10T21:42:54.918023+0000","last_became_peered":"2024-09-10T21:42:54.918023+0000","last_unstale":"2024-09-10T21:42:56.931020+0000","last_undegraded":"2024-09-10T21:42:56.931020+0000","last_fullsized":"2024-09-10T21:42:56.931020+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:47.410137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.942142+0000","last_change":"2024-09-10T21:42:55.928373+0000","last_active":"2024-09-10T21:42:55.942142+0000","last_peered":"2024-09-10T21:42:55.942142+0000","last_clean":"2024-09-10T21:42:55.942142+0000","last_became_active":"2024-09-10T21:42:55.927931+0000","last_became_peered":"2024-09-10T21:42:55.927931+0000","last_unstale":"2024-09-10T21:42:55.942142+0000","last_undegraded":"2024-09-10T21:42:55.942142+0000","last_fullsized":"2024-09-10T21:42:55.942142+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:07.120861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925927+0000","last_change":"2024-09-10T21:42:54.917326+0000","last_active":"2024-09-10T21:42:56.925927+0000","last_peered":"2024-09-10T21:42:56.925927+0000","last_clean":"2024-09-10T21:42:56.925927+0000","last_became_active":"2024-09-10T21:42:54.916975+0000","last_became_peered":"2024-09-10T21:42:54.916975+0000","last_unstale":"2024-09-10T21:42:56.925927+0000","last_undegraded":"2024-09-10T21:42:56.925927+0000","last_fullsized":"2024-09-10T21:42:56.925927+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:37:29.926328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925569+0000","last_change":"2024-09-10T21:42:55.917321+0000","last_active":"2024-09-10T21:42:56.925569+0000","last_peered":"2024-09-10T21:42:56.925569+0000","last_clean":"2024-09-10T21:42:56.925569+0000","last_became_active":"2024-09-10T21:42:55.917055+0000","last_became_peered":"2024-09-10T21:42:55.917055+0000","last_unstale":"2024-09-10T21:42:56.925569+0000","last_undegraded":"2024-09-10T21:42:56.925569+0000","last_fullsized":"2024-09-10T21:42:56.925569+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:13.755661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.948018+0000","last_change":"2024-09-10T21:42:54.918811+0000","last_active":"2024-09-10T21:42:55.948018+0000","last_peered":"2024-09-10T21:42:55.948018+0000","last_clean":"2024-09-10T21:42:55.948018+0000","last_became_active":"2024-09-10T21:42:54.918542+0000","last_became_peered":"2024-09-10T21:42:54.918542+0000","last_unstale":"2024-09-10T21:42:55.948018+0000","last_undegraded":"2024-09-10T21:42:55.948018+0000","last_fullsized":"2024-09-10T21:42:55.948018+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:22.256768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933205+0000","last_change":"2024-09-10T21:42:55.921692+0000","last_active":"2024-09-10T21:42:56.933205+0000","last_peered":"2024-09-10T21:42:56.933205+0000","last_clean":"2024-09-10T21:42:56.933205+0000","last_became_active":"2024-09-10T21:42:55.921345+0000","last_became_peered":"2024-09-10T21:42:55.921345+0000","last_unstale":"2024-09-10T21:42:56.933205+0000","last_undegraded":"2024-09-10T21:42:56.933205+0000","last_fullsized":"2024-09-10T21:42:56.933205+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:29.212260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925335+0000","last_change":"2024-09-10T21:42:54.903509+0000","last_active":"2024-09-10T21:42:56.925335+0000","last_peered":"2024-09-10T21:42:56.925335+0000","last_clean":"2024-09-10T21:42:56.925335+0000","last_became_active":"2024-09-10T21:42:54.903182+0000","last_became_peered":"2024-09-10T21:42:54.903182+0000","last_unstale":"2024-09-10T21:42:56.925335+0000","last_undegraded":"2024-09-10T21:42:56.925335+0000","last_fullsized":"2024-09-10T21:42:56.925335+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:02.063418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.945218+0000","last_change":"2024-09-10T21:42:55.930485+0000","last_active":"2024-09-10T21:42:55.945218+0000","last_peered":"2024-09-10T21:42:55.945218+0000","last_clean":"2024-09-10T21:42:55.945218+0000","last_became_active":"2024-09-10T21:42:55.929597+0000","last_became_peered":"2024-09-10T21:42:55.929597+0000","last_unstale":"2024-09-10T21:42:55.945218+0000","last_undegraded":"2024-09-10T21:42:55.945218+0000","last_fullsized":"2024-09-10T21:42:55.945218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:22.309511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933305+0000","last_change":"2024-09-10T21:42:54.908462+0000","last_active":"2024-09-10T21:42:56.933305+0000","last_peered":"2024-09-10T21:42:56.933305+0000","last_clean":"2024-09-10T21:42:56.933305+0000","last_became_active":"2024-09-10T21:42:54.906528+0000","last_became_peered":"2024-09-10T21:42:54.906528+0000","last_unstale":"2024-09-10T21:42:56.933305+0000","last_undegraded":"2024-09-10T21:42:56.933305+0000","last_fullsized":"2024-09-10T21:42:56.933305+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:09.539994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.927743+0000","last_change":"2024-09-10T21:42:55.922274+0000","last_active":"2024-09-10T21:42:55.927743+0000","last_peered":"2024-09-10T21:42:55.927743+0000","last_clean":"2024-09-10T21:42:55.927743+0000","last_became_active":"2024-09-10T21:42:55.922011+0000","last_became_peered":"2024-09-10T21:42:55.922011+0000","last_unstale":"2024-09-10T21:42:55.927743+0000","last_undegraded":"2024-09-10T21:42:55.927743+0000","last_fullsized":"2024-09-10T21:42:55.927743+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:24.106475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.916195+0000","last_change":"2024-09-10T21:42:54.915698+0000","last_active":"2024-09-10T21:42:55.916195+0000","last_peered":"2024-09-10T21:42:55.916195+0000","last_clean":"2024-09-10T21:42:55.916195+0000","last_became_active":"2024-09-10T21:42:54.915434+0000","last_became_peered":"2024-09-10T21:42:54.915434+0000","last_unstale":"2024-09-10T21:42:55.916195+0000","last_undegraded":"2024-09-10T21:42:55.916195+0000","last_fullsized":"2024-09-10T21:42:55.916195+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:26.098513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.930456+0000","last_change":"2024-09-10T21:42:55.925709+0000","last_active":"2024-09-10T21:42:55.930456+0000","last_peered":"2024-09-10T21:42:55.930456+0000","last_clean":"2024-09-10T21:42:55.930456+0000","last_became_active":"2024-09-10T21:42:55.922936+0000","last_became_peered":"2024-09-10T21:42:55.922936+0000","last_unstale":"2024-09-10T21:42:55.930456+0000","last_undegraded":"2024-09-10T21:42:55.930456+0000","last_fullsized":"2024-09-10T21:42:55.930456+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:01.214509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.946228+0000","last_change":"2024-09-10T21:42:54.913658+0000","last_active":"2024-09-10T21:42:55.946228+0000","last_peered":"2024-09-10T21:42:55.946228+0000","last_clean":"2024-09-10T21:42:55.946228+0000","last_became_active":"2024-09-10T21:42:54.913386+0000","last_became_peered":"2024-09-10T21:42:54.913386+0000","last_unstale":"2024-09-10T21:42:55.946228+0000","last_undegraded":"2024-09-10T21:42:55.946228+0000","last_fullsized":"2024-09-10T21:42:55.946228+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:24.590339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925754+0000","last_change":"2024-09-10T21:42:55.920787+0000","last_active":"2024-09-10T21:42:56.925754+0000","last_peered":"2024-09-10T21:42:56.925754+0000","last_clean":"2024-09-10T21:42:56.925754+0000","last_became_active":"2024-09-10T21:42:55.919469+0000","last_became_peered":"2024-09-10T21:42:55.919469+0000","last_unstale":"2024-09-10T21:42:56.925754+0000","last_undegraded":"2024-09-10T21:42:56.925754+0000","last_fullsized":"2024-09-10T21:42:56.925754+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:29.212260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933841+0000","last_change":"2024-09-10T21:42:54.913809+0000","last_active":"2024-09-10T21:42:56.933841+0000","last_peered":"2024-09-10T21:42:56.933841+0000","last_clean":"2024-09-10T21:42:56.933841+0000","last_became_active":"2024-09-10T21:42:54.913489+0000","last_became_peered":"2024-09-10T21:42:54.913489+0000","last_unstale":"2024-09-10T21:42:56.933841+0000","last_undegraded":"2024-09-10T21:42:56.933841+0000","last_fullsized":"2024-09-10T21:42:56.933841+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:49.605597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.926710+0000","last_change":"2024-09-10T21:42:55.926751+0000","last_active":"2024-09-10T21:42:56.926710+0000","last_peered":"2024-09-10T21:42:56.926710+0000","last_clean":"2024-09-10T21:42:56.926710+0000","last_became_active":"2024-09-10T21:42:55.926247+0000","last_became_peered":"2024-09-10T21:42:55.926247+0000","last_unstale":"2024-09-10T21:42:56.926710+0000","last_undegraded":"2024-09-10T21:42:56.926710+0000","last_fullsized":"2024-09-10T21:42:56.926710+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:53.704714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925896+0000","last_change":"2024-09-10T21:42:54.906657+0000","last_active":"2024-09-10T21:42:56.925896+0000","last_peered":"2024-09-10T21:42:56.925896+0000","last_clean":"2024-09-10T21:42:56.925896+0000","last_became_active":"2024-09-10T21:42:54.906308+0000","last_became_peered":"2024-09-10T21:42:54.906308+0000","last_unstale":"2024-09-10T21:42:56.925896+0000","last_undegraded":"2024-09-10T21:42:56.925896+0000","last_fullsized":"2024-09-10T21:42:56.925896+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:00.835587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933662+0000","last_change":"2024-09-10T21:42:55.921836+0000","last_active":"2024-09-10T21:42:56.933662+0000","last_peered":"2024-09-10T21:42:56.933662+0000","last_clean":"2024-09-10T21:42:56.933662+0000","last_became_active":"2024-09-10T21:42:55.921559+0000","last_became_peered":"2024-09-10T21:42:55.921559+0000","last_unstale":"2024-09-10T21:42:56.933662+0000","last_undegraded":"2024-09-10T21:42:56.933662+0000","last_fullsized":"2024-09-10T21:42:56.933662+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:00.956802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.926261+0000","last_change":"2024-09-10T21:42:54.910592+0000","last_active":"2024-09-10T21:42:56.926261+0000","last_peered":"2024-09-10T21:42:56.926261+0000","last_clean":"2024-09-10T21:42:56.926261+0000","last_became_active":"2024-09-10T21:42:54.910296+0000","last_became_peered":"2024-09-10T21:42:54.910296+0000","last_unstale":"2024-09-10T21:42:56.926261+0000","last_undegraded":"2024-09-10T21:42:56.926261+0000","last_fullsized":"2024-09-10T21:42:56.926261+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:39.167492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.920025+0000","last_change":"2024-09-10T21:42:55.923971+0000","last_active":"2024-09-10T21:42:56.920025+0000","last_peered":"2024-09-10T21:42:56.920025+0000","last_clean":"2024-09-10T21:42:56.920025+0000","last_became_active":"2024-09-10T21:42:55.923669+0000","last_became_peered":"2024-09-10T21:42:55.923669+0000","last_unstale":"2024-09-10T21:42:56.920025+0000","last_undegraded":"2024-09-10T21:42:56.920025+0000","last_fullsized":"2024-09-10T21:42:56.920025+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:35.525042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.926146+0000","last_change":"2024-09-10T21:42:54.913677+0000","last_active":"2024-09-10T21:42:56.926146+0000","last_peered":"2024-09-10T21:42:56.926146+0000","last_clean":"2024-09-10T21:42:56.926146+0000","last_became_active":"2024-09-10T21:42:54.913385+0000","last_became_peered":"2024-09-10T21:42:54.913385+0000","last_unstale":"2024-09-10T21:42:56.926146+0000","last_undegraded":"2024-09-10T21:42:56.926146+0000","last_fullsized":"2024-09-10T21:42:56.926146+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:49.605597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.919683+0000","last_change":"2024-09-10T21:42:55.916897+0000","last_active":"2024-09-10T21:42:56.919683+0000","last_peered":"2024-09-10T21:42:56.919683+0000","last_clean":"2024-09-10T21:42:56.919683+0000","last_became_active":"2024-09-10T21:42:55.916627+0000","last_became_peered":"2024-09-10T21:42:55.916627+0000","last_unstale":"2024-09-10T21:42:56.919683+0000","last_undegraded":"2024-09-10T21:42:56.919683+0000","last_fullsized":"2024-09-10T21:42:56.919683+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:20.745519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.930707+0000","last_change":"2024-09-10T21:42:54.920488+0000","last_active":"2024-09-10T21:42:56.930707+0000","last_peered":"2024-09-10T21:42:56.930707+0000","last_clean":"2024-09-10T21:42:56.930707+0000","last_became_active":"2024-09-10T21:42:54.920128+0000","last_became_peered":"2024-09-10T21:42:54.920128+0000","last_unstale":"2024-09-10T21:42:56.930707+0000","last_undegraded":"2024-09-10T21:42:56.930707+0000","last_fullsized":"2024-09-10T21:42:56.930707+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:29.144828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.919840+0000","last_change":"2024-09-10T21:42:55.924371+0000","last_active":"2024-09-10T21:42:56.919840+0000","last_peered":"2024-09-10T21:42:56.919840+0000","last_clean":"2024-09-10T21:42:56.919840+0000","last_became_active":"2024-09-10T21:42:55.924105+0000","last_became_peered":"2024-09-10T21:42:55.924105+0000","last_unstale":"2024-09-10T21:42:56.919840+0000","last_undegraded":"2024-09-10T21:42:56.919840+0000","last_fullsized":"2024-09-10T21:42:56.919840+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:03.069319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.907556+0000","last_change":"2024-09-10T21:42:54.918220+0000","last_active":"2024-09-10T21:42:55.907556+0000","last_peered":"2024-09-10T21:42:55.907556+0000","last_clean":"2024-09-10T21:42:55.907556+0000","last_became_active":"2024-09-10T21:42:54.917887+0000","last_became_peered":"2024-09-10T21:42:54.917887+0000","last_unstale":"2024-09-10T21:42:55.907556+0000","last_undegraded":"2024-09-10T21:42:55.907556+0000","last_fullsized":"2024-09-10T21:42:55.907556+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:49.605597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.919680+0000","last_change":"2024-09-10T21:42:55.914548+0000","last_active":"2024-09-10T21:42:56.919680+0000","last_peered":"2024-09-10T21:42:56.919680+0000","last_clean":"2024-09-10T21:42:56.919680+0000","last_became_active":"2024-09-10T21:42:55.913610+0000","last_became_peered":"2024-09-10T21:42:55.913610+0000","last_unstale":"2024-09-10T21:42:56.919680+0000","last_undegraded":"2024-09-10T21:42:56.919680+0000","last_fullsized":"2024-09-10T21:42:56.919680+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:46.334741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925224+0000","last_change":"2024-09-10T21:42:54.909244+0000","last_active":"2024-09-10T21:42:56.925224+0000","last_peered":"2024-09-10T21:42:56.925224+0000","last_clean":"2024-09-10T21:42:56.925224+0000","last_became_active":"2024-09-10T21:42:54.908942+0000","last_became_peered":"2024-09-10T21:42:54.908942+0000","last_unstale":"2024-09-10T21:42:56.925224+0000","last_undegraded":"2024-09-10T21:42:56.925224+0000","last_fullsized":"2024-09-10T21:42:56.925224+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:52.032032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.937914+0000","last_change":"2024-09-10T21:42:55.931798+0000","last_active":"2024-09-10T21:42:55.937914+0000","last_peered":"2024-09-10T21:42:55.937914+0000","last_clean":"2024-09-10T21:42:55.937914+0000","last_became_active":"2024-09-10T21:42:55.931556+0000","last_became_peered":"2024-09-10T21:42:55.931556+0000","last_unstale":"2024-09-10T21:42:55.937914+0000","last_undegraded":"2024-09-10T21:42:55.937914+0000","last_fullsized":"2024-09-10T21:42:55.937914+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:34:32.444806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.924383+0000","last_change":"2024-09-10T21:42:54.915745+0000","last_active":"2024-09-10T21:42:56.924383+0000","last_peered":"2024-09-10T21:42:56.924383+0000","last_clean":"2024-09-10T21:42:56.924383+0000","last_became_active":"2024-09-10T21:42:54.913515+0000","last_became_peered":"2024-09-10T21:42:54.913515+0000","last_unstale":"2024-09-10T21:42:56.924383+0000","last_undegraded":"2024-09-10T21:42:56.924383+0000","last_fullsized":"2024-09-10T21:42:56.924383+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:03.523688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.920067+0000","last_change":"2024-09-10T21:42:55.919268+0000","last_active":"2024-09-10T21:42:56.920067+0000","last_peered":"2024-09-10T21:42:56.920067+0000","last_clean":"2024-09-10T21:42:56.920067+0000","last_became_active":"2024-09-10T21:42:55.919006+0000","last_became_peered":"2024-09-10T21:42:55.919006+0000","last_unstale":"2024-09-10T21:42:56.920067+0000","last_undegraded":"2024-09-10T21:42:56.920067+0000","last_fullsized":"2024-09-10T21:42:56.920067+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:50.611498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933904+0000","last_change":"2024-09-10T21:42:54.913663+0000","last_active":"2024-09-10T21:42:56.933904+0000","last_peered":"2024-09-10T21:42:56.933904+0000","last_clean":"2024-09-10T21:42:56.933904+0000","last_became_active":"2024-09-10T21:42:54.913235+0000","last_became_peered":"2024-09-10T21:42:54.913235+0000","last_unstale":"2024-09-10T21:42:56.933904+0000","last_undegraded":"2024-09-10T21:42:56.933904+0000","last_fullsized":"2024-09-10T21:42:56.933904+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:26.221672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.926886+0000","last_change":"2024-09-10T21:42:55.927903+0000","last_active":"2024-09-10T21:42:56.926886+0000","last_peered":"2024-09-10T21:42:56.926886+0000","last_clean":"2024-09-10T21:42:56.926886+0000","last_became_active":"2024-09-10T21:42:55.927644+0000","last_became_peered":"2024-09-10T21:42:55.927644+0000","last_unstale":"2024-09-10T21:42:56.926886+0000","last_undegraded":"2024-09-10T21:42:56.926886+0000","last_fullsized":"2024-09-10T21:42:56.926886+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:22.149532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.932434+0000","last_change":"2024-09-10T21:42:54.915239+0000","last_active":"2024-09-10T21:42:56.932434+0000","last_peered":"2024-09-10T21:42:56.932434+0000","last_clean":"2024-09-10T21:42:56.932434+0000","last_became_active":"2024-09-10T21:42:54.914940+0000","last_became_peered":"2024-09-10T21:42:54.914940+0000","last_unstale":"2024-09-10T21:42:56.932434+0000","last_undegraded":"2024-09-10T21:42:56.932434+0000","last_fullsized":"2024-09-10T21:42:56.932434+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:29.899175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.930651+0000","last_change":"2024-09-10T21:42:55.922264+0000","last_active":"2024-09-10T21:42:56.930651+0000","last_peered":"2024-09-10T21:42:56.930651+0000","last_clean":"2024-09-10T21:42:56.930651+0000","last_became_active":"2024-09-10T21:42:55.920794+0000","last_became_peered":"2024-09-10T21:42:55.920794+0000","last_unstale":"2024-09-10T21:42:56.930651+0000","last_undegraded":"2024-09-10T21:42:56.930651+0000","last_fullsized":"2024-09-10T21:42:56.930651+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:12.151596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933424+0000","last_change":"2024-09-10T21:42:54.910060+0000","last_active":"2024-09-10T21:42:56.933424+0000","last_peered":"2024-09-10T21:42:56.933424+0000","last_clean":"2024-09-10T21:42:56.933424+0000","last_became_active":"2024-09-10T21:42:54.909577+0000","last_became_peered":"2024-09-10T21:42:54.909577+0000","last_unstale":"2024-09-10T21:42:56.933424+0000","last_undegraded":"2024-09-10T21:42:56.933424+0000","last_fullsized":"2024-09-10T21:42:56.933424+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:16.465324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.927911+0000","last_change":"2024-09-10T21:42:55.920207+0000","last_active":"2024-09-10T21:42:55.927911+0000","last_peered":"2024-09-10T21:42:55.927911+0000","last_clean":"2024-09-10T21:42:55.927911+0000","last_became_active":"2024-09-10T21:42:55.919870+0000","last_became_peered":"2024-09-10T21:42:55.919870+0000","last_unstale":"2024-09-10T21:42:55.927911+0000","last_undegraded":"2024-09-10T21:42:55.927911+0000","last_fullsized":"2024-09-10T21:42:55.927911+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:10.545895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.930841+0000","last_change":"2024-09-10T21:42:54.920494+0000","last_active":"2024-09-10T21:42:56.930841+0000","last_peered":"2024-09-10T21:42:56.930841+0000","last_clean":"2024-09-10T21:42:56.930841+0000","last_became_active":"2024-09-10T21:42:54.920164+0000","last_became_peered":"2024-09-10T21:42:54.920164+0000","last_unstale":"2024-09-10T21:42:56.930841+0000","last_undegraded":"2024-09-10T21:42:56.930841+0000","last_fullsized":"2024-09-10T21:42:56.930841+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:09.939461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.930763+0000","last_change":"2024-09-10T21:42:55.920622+0000","last_active":"2024-09-10T21:42:56.930763+0000","last_peered":"2024-09-10T21:42:56.930763+0000","last_clean":"2024-09-10T21:42:56.930763+0000","last_became_active":"2024-09-10T21:42:55.920352+0000","last_became_peered":"2024-09-10T21:42:55.920352+0000","last_unstale":"2024-09-10T21:42:56.930763+0000","last_undegraded":"2024-09-10T21:42:56.930763+0000","last_fullsized":"2024-09-10T21:42:56.930763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:28.540988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933409+0000","last_change":"2024-09-10T21:42:54.908752+0000","last_active":"2024-09-10T21:42:56.933409+0000","last_peered":"2024-09-10T21:42:56.933409+0000","last_clean":"2024-09-10T21:42:56.933409+0000","last_became_active":"2024-09-10T21:42:54.908287+0000","last_became_peered":"2024-09-10T21:42:54.908287+0000","last_unstale":"2024-09-10T21:42:56.933409+0000","last_undegraded":"2024-09-10T21:42:56.933409+0000","last_fullsized":"2024-09-10T21:42:56.933409+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:45.607195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.945405+0000","last_change":"2024-09-10T21:42:55.929837+0000","last_active":"2024-09-10T21:42:55.945405+0000","last_peered":"2024-09-10T21:42:55.945405+0000","last_clean":"2024-09-10T21:42:55.945405+0000","last_became_active":"2024-09-10T21:42:55.929566+0000","last_became_peered":"2024-09-10T21:42:55.929566+0000","last_unstale":"2024-09-10T21:42:55.945405+0000","last_undegraded":"2024-09-10T21:42:55.945405+0000","last_fullsized":"2024-09-10T21:42:55.945405+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:36:39.159143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925318+0000","last_change":"2024-09-10T21:42:54.913739+0000","last_active":"2024-09-10T21:42:56.925318+0000","last_peered":"2024-09-10T21:42:56.925318+0000","last_clean":"2024-09-10T21:42:56.925318+0000","last_became_active":"2024-09-10T21:42:54.913435+0000","last_became_peered":"2024-09-10T21:42:54.913435+0000","last_unstale":"2024-09-10T21:42:56.925318+0000","last_undegraded":"2024-09-10T21:42:56.925318+0000","last_fullsized":"2024-09-10T21:42:56.925318+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:34.519141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925101+0000","last_change":"2024-09-10T21:42:55.929272+0000","last_active":"2024-09-10T21:42:56.925101+0000","last_peered":"2024-09-10T21:42:56.925101+0000","last_clean":"2024-09-10T21:42:56.925101+0000","last_became_active":"2024-09-10T21:42:55.929034+0000","last_became_peered":"2024-09-10T21:42:55.929034+0000","last_unstale":"2024-09-10T21:42:56.925101+0000","last_undegraded":"2024-09-10T21:42:56.925101+0000","last_fullsized":"2024-09-10T21:42:56.925101+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:05.196124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.916216+0000","last_change":"2024-09-10T21:42:54.921213+0000","last_active":"2024-09-10T21:42:55.916216+0000","last_peered":"2024-09-10T21:42:55.916216+0000","last_clean":"2024-09-10T21:42:55.916216+0000","last_became_active":"2024-09-10T21:42:54.920945+0000","last_became_peered":"2024-09-10T21:42:54.920945+0000","last_unstale":"2024-09-10T21:42:55.916216+0000","last_undegraded":"2024-09-10T21:42:55.916216+0000","last_fullsized":"2024-09-10T21:42:55.916216+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:23.100574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.935061+0000","last_change":"2024-09-10T21:42:55.927589+0000","last_active":"2024-09-10T21:42:55.935061+0000","last_peered":"2024-09-10T21:42:55.935061+0000","last_clean":"2024-09-10T21:42:55.935061+0000","last_became_active":"2024-09-10T21:42:55.927279+0000","last_became_peered":"2024-09-10T21:42:55.927279+0000","last_unstale":"2024-09-10T21:42:55.935061+0000","last_undegraded":"2024-09-10T21:42:55.935061+0000","last_fullsized":"2024-09-10T21:42:55.935061+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:07.120861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.906553+0000","last_change":"2024-09-10T21:42:54.916404+0000","last_active":"2024-09-10T21:42:55.906553+0000","last_peered":"2024-09-10T21:42:55.906553+0000","last_clean":"2024-09-10T21:42:55.906553+0000","last_became_active":"2024-09-10T21:42:54.916085+0000","last_became_peered":"2024-09-10T21:42:54.916085+0000","last_unstale":"2024-09-10T21:42:55.906553+0000","last_undegraded":"2024-09-10T21:42:55.906553+0000","last_fullsized":"2024-09-10T21:42:55.906553+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:02.063418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":127,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.911175+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:42:55.911175+0000","last_peered":"2024-09-10T21:42:55.911175+0000","last_clean":"2024-09-10T21:42:55.911175+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:42:55.911175+0000","last_undegraded":"2024-09-10T21:42:55.911175+0000","last_fullsized":"2024-09-10T21:42:55.911175+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":348160,"data_stored":129695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":348160,"data_stored":129183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":356352,"data_stored":135449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":368640,"data_stored":137013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":725747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":937984,"data_stored":719537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542159,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31540,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":356352,"data_stored":130713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:43:00.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:43:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.395+0000 7f2589e00640 1 -- 172.21.15.7:0/4102229399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2584146c50 msgr2=0x7f25840723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.395+0000 7f2589e00640 1 --2- 172.21.15.7:0/4102229399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2584146c50 0x7f25840723b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2584070410 tx=0x7f257402a490 comp rx=0 tx=0).stop 2024-09-10T21:43:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.395+0000 7f2589e00640 1 -- 172.21.15.7:0/4102229399 shutdown_connections 2024-09-10T21:43:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.395+0000 7f2589e00640 1 --2- 172.21.15.7:0/4102229399 >> v2:172.21.15.133:3300/0 conn(0x7f25841470b0 0x7f25840ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.395+0000 7f2589e00640 1 --2- 172.21.15.7:0/4102229399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2584146c50 0x7f25840723b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.395+0000 7f2589e00640 1 -- 172.21.15.7:0/4102229399 >> 172.21.15.7:0/4102229399 conn(0x7f25840b1d70 msgr2=0x7f25840cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:00.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.395+0000 7f2589e00640 1 -- 172.21.15.7:0/4102229399 shutdown_connections 2024-09-10T21:43:00.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.395+0000 7f2589e00640 1 -- 172.21.15.7:0/4102229399 wait complete. 2024-09-10T21:43:00.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.396+0000 7f2589e00640 1 Processor -- start 2024-09-10T21:43:00.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.397+0000 7f2589e00640 1 -- start start 2024-09-10T21:43:00.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.397+0000 7f2589e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f258406f590 0x7f2584138690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.398+0000 7f2589e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2584146c50 0x7f258413ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.398+0000 7f2589e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25841470b0 0x7f258413b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.398+0000 7f2589e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f25840ade20 con 0x7f2584146c50 2024-09-10T21:43:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.398+0000 7f2589e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2584073210 con 0x7f258406f590 2024-09-10T21:43:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.398+0000 7f2589e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f25840adfa0 con 0x7f25841470b0 2024-09-10T21:43:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.398+0000 7f257be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2584146c50 0x7f258413ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.398+0000 7f257be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2584146c50 0x7f258413ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57538/0 (socket says 172.21.15.7:57538) 2024-09-10T21:43:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.398+0000 7f257be00640 1 -- 172.21.15.7:0/1416581926 learned_addr learned my addr 172.21.15.7:0/1416581926 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.398+0000 7f2588a00640 1 --2- 172.21.15.7:0/1416581926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f258406f590 0x7f2584138690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.398+0000 7f2589400640 1 --2- 172.21.15.7:0/1416581926 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25841470b0 0x7f258413b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.398+0000 7f257be00640 1 -- 172.21.15.7:0/1416581926 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25841470b0 msgr2=0x7f258413b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:00.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.398+0000 7f257be00640 1 --2- 172.21.15.7:0/1416581926 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25841470b0 0x7f258413b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:00.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.398+0000 7f257be00640 1 -- 172.21.15.7:0/1416581926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f258406f590 msgr2=0x7f2584138690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:00.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.398+0000 7f257be00640 1 --2- 172.21.15.7:0/1416581926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f258406f590 0x7f2584138690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:00.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.398+0000 7f257be00640 1 -- 172.21.15.7:0/1416581926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f258413fbe0 con 0x7f2584146c50 2024-09-10T21:43:00.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.398+0000 7f2589400640 1 --2- 172.21.15.7:0/1416581926 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25841470b0 0x7f258413b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:00.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.399+0000 7f257be00640 1 --2- 172.21.15.7:0/1416581926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2584146c50 0x7f258413ad10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f256c00c480 tx=0x7f256c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.399+0000 7f2581600640 1 -- 172.21.15.7:0/1416581926 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c016020 con 0x7f2584146c50 2024-09-10T21:43:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.399+0000 7f2581600640 1 -- 172.21.15.7:0/1416581926 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f256c012920 con 0x7f2584146c50 2024-09-10T21:43:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.399+0000 7f2581600640 1 -- 172.21.15.7:0/1416581926 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f256c012d70 con 0x7f2584146c50 2024-09-10T21:43:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.399+0000 7f2589e00640 1 -- 172.21.15.7:0/1416581926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f258413a230 con 0x7f2584146c50 2024-09-10T21:43:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.399+0000 7f2589e00640 1 -- 172.21.15.7:0/1416581926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f258413a7f0 con 0x7f2584146c50 2024-09-10T21:43:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.400+0000 7f2589e00640 1 -- 172.21.15.7:0/1416581926 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25840ad8e0 con 0x7f2584146c50 2024-09-10T21:43:00.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.401+0000 7f2581600640 1 -- 172.21.15.7:0/1416581926 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f256c01b400 con 0x7f2584146c50 2024-09-10T21:43:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.402+0000 7f2581600640 1 --2- 172.21.15.7:0/1416581926 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2560077f70 0x7f256007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:00.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.402+0000 7f2581600640 1 -- 172.21.15.7:0/1416581926 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f256c099fd0 con 0x7f2584146c50 2024-09-10T21:43:00.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.403+0000 7f2588a00640 1 --2- 172.21.15.7:0/1416581926 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2560077f70 0x7f256007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:00.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.403+0000 7f2588a00640 1 --2- 172.21.15.7:0/1416581926 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2560077f70 0x7f256007a430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2574034000 tx=0x7f2574004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:00.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.406+0000 7f2581600640 1 -- 172.21.15.7:0/1416581926 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f256c020020 con 0x7f2584146c50 2024-09-10T21:43:00.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.599+0000 7f2589e00640 1 -- 172.21.15.7:0/1416581926 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2584161570 con 0x7f2560077f70 2024-09-10T21:43:00.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.604+0000 7f2581600640 1 -- 172.21.15.7:0/1416581926 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251934 (secure 0 0 0) 0x7f2584161570 con 0x7f2560077f70 2024-09-10T21:43:00.608 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:00.610 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:43:00.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.609+0000 7f2589e00640 1 -- 172.21.15.7:0/1416581926 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2560077f70 msgr2=0x7f256007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:00.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.609+0000 7f2589e00640 1 --2- 172.21.15.7:0/1416581926 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2560077f70 0x7f256007a430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2574034000 tx=0x7f2574004730 comp rx=0 tx=0).stop 2024-09-10T21:43:00.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.609+0000 7f2589e00640 1 -- 172.21.15.7:0/1416581926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2584146c50 msgr2=0x7f258413ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:00.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.609+0000 7f2589e00640 1 --2- 172.21.15.7:0/1416581926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2584146c50 0x7f258413ad10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f256c00c480 tx=0x7f256c00c940 comp rx=0 tx=0).stop 2024-09-10T21:43:00.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.609+0000 7f2589e00640 1 -- 172.21.15.7:0/1416581926 shutdown_connections 2024-09-10T21:43:00.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.609+0000 7f2589e00640 1 --2- 172.21.15.7:0/1416581926 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2560077f70 0x7f256007a430 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:00.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.609+0000 7f2589e00640 1 --2- 172.21.15.7:0/1416581926 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25841470b0 0x7f258413b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:00.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.609+0000 7f2589e00640 1 --2- 172.21.15.7:0/1416581926 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2584146c50 0x7f258413ad10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:00.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.609+0000 7f2589e00640 1 --2- 172.21.15.7:0/1416581926 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f258406f590 0x7f2584138690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:00.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.609+0000 7f2589e00640 1 -- 172.21.15.7:0/1416581926 >> 172.21.15.7:0/1416581926 conn(0x7f25840b1d70 msgr2=0x7f25840b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:00.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.609+0000 7f2589e00640 1 -- 172.21.15.7:0/1416581926 shutdown_connections 2024-09-10T21:43:00.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.609+0000 7f2589e00640 1 -- 172.21.15.7:0/1416581926 wait complete. 2024-09-10T21:43:00.625 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2024-09-10T21:42:59.866926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240976,"kb_used_data":3928,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754733744,"statfs":{"total":773094113280,"available":772847353856,"internally_reserved":0,"allocated":4022272,"data_stored":2243388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.978723"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.925894+0000","last_change":"2024-09-10T21:42:55.919761+0000","last_active":"2024-09-10T21:42:55.925894+0000","last_peered":"2024-09-10T21:42:55.925894+0000","last_clean":"2024-09-10T21:42:55.925894+0000","last_became_active":"2024-09-10T21:42:55.919462+0000","last_became_peered":"2024-09-10T21:42:55.919462+0000","last_unstale":"2024-09-10T21:42:55.925894+0000","last_undegraded":"2024-09-10T21:42:55.925894+0000","last_fullsized":"2024-09-10T21:42:55.925894+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:27.227573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.945243+0000","last_change":"2024-09-10T21:42:55.933968+0000","last_active":"2024-09-10T21:42:55.945243+0000","last_peered":"2024-09-10T21:42:55.945243+0000","last_clean":"2024-09-10T21:42:55.945243+0000","last_became_active":"2024-09-10T21:42:55.931328+0000","last_became_peered":"2024-09-10T21:42:55.931328+0000","last_unstale":"2024-09-10T21:42:55.945243+0000","last_undegraded":"2024-09-10T21:42:55.945243+0000","last_fullsized":"2024-09-10T21:42:55.945243+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:39.175452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925810+0000","last_change":"2024-09-10T21:42:55.923668+0000","last_active":"2024-09-10T21:42:56.925810+0000","last_peered":"2024-09-10T21:42:56.925810+0000","last_clean":"2024-09-10T21:42:56.925810+0000","last_became_active":"2024-09-10T21:42:55.920236+0000","last_became_peered":"2024-09-10T21:42:55.920236+0000","last_unstale":"2024-09-10T21:42:56.925810+0000","last_undegraded":"2024-09-10T21:42:56.925810+0000","last_fullsized":"2024-09-10T21:42:56.925810+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:05.352427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933316+0000","last_change":"2024-09-10T21:42:55.921945+0000","last_active":"2024-09-10T21:42:56.933316+0000","last_peered":"2024-09-10T21:42:56.933316+0000","last_clean":"2024-09-10T21:42:56.933316+0000","last_became_active":"2024-09-10T21:42:55.921643+0000","last_became_peered":"2024-09-10T21:42:55.921643+0000","last_unstale":"2024-09-10T21:42:56.933316+0000","last_undegraded":"2024-09-10T21:42:56.933316+0000","last_fullsized":"2024-09-10T21:42:56.933316+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:39.244361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.919624+0000","last_change":"2024-09-10T21:42:55.932409+0000","last_active":"2024-09-10T21:42:56.919624+0000","last_peered":"2024-09-10T21:42:56.919624+0000","last_clean":"2024-09-10T21:42:56.919624+0000","last_became_active":"2024-09-10T21:42:55.932122+0000","last_became_peered":"2024-09-10T21:42:55.932122+0000","last_unstale":"2024-09-10T21:42:56.919624+0000","last_undegraded":"2024-09-10T21:42:56.919624+0000","last_fullsized":"2024-09-10T21:42:56.919624+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:40.173393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.938048+0000","last_change":"2024-09-10T21:42:55.929536+0000","last_active":"2024-09-10T21:42:55.938048+0000","last_peered":"2024-09-10T21:42:55.938048+0000","last_clean":"2024-09-10T21:42:55.938048+0000","last_became_active":"2024-09-10T21:42:55.929201+0000","last_became_peered":"2024-09-10T21:42:55.929201+0000","last_unstale":"2024-09-10T21:42:55.938048+0000","last_undegraded":"2024-09-10T21:42:55.938048+0000","last_fullsized":"2024-09-10T21:42:55.938048+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:04.496067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.937887+0000","last_change":"2024-09-10T21:42:55.930869+0000","last_active":"2024-09-10T21:42:55.937887+0000","last_peered":"2024-09-10T21:42:55.937887+0000","last_clean":"2024-09-10T21:42:55.937887+0000","last_became_active":"2024-09-10T21:42:55.930615+0000","last_became_peered":"2024-09-10T21:42:55.930615+0000","last_unstale":"2024-09-10T21:42:55.937887+0000","last_undegraded":"2024-09-10T21:42:55.937887+0000","last_fullsized":"2024-09-10T21:42:55.937887+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:04.496067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.927789+0000","last_change":"2024-09-10T21:42:55.923253+0000","last_active":"2024-09-10T21:42:55.927789+0000","last_peered":"2024-09-10T21:42:55.927789+0000","last_clean":"2024-09-10T21:42:55.927789+0000","last_became_active":"2024-09-10T21:42:55.922923+0000","last_became_peered":"2024-09-10T21:42:55.922923+0000","last_unstale":"2024-09-10T21:42:55.927789+0000","last_undegraded":"2024-09-10T21:42:55.927789+0000","last_fullsized":"2024-09-10T21:42:55.927789+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:02.328070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.926411+0000","last_change":"2024-09-10T21:42:55.917855+0000","last_active":"2024-09-10T21:42:56.926411+0000","last_peered":"2024-09-10T21:42:56.926411+0000","last_clean":"2024-09-10T21:42:56.926411+0000","last_became_active":"2024-09-10T21:42:55.916101+0000","last_became_peered":"2024-09-10T21:42:55.916101+0000","last_unstale":"2024-09-10T21:42:56.926411+0000","last_undegraded":"2024-09-10T21:42:56.926411+0000","last_fullsized":"2024-09-10T21:42:56.926411+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:48:39.274454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933559+0000","last_change":"2024-09-10T21:42:55.939027+0000","last_active":"2024-09-10T21:42:56.933559+0000","last_peered":"2024-09-10T21:42:56.933559+0000","last_clean":"2024-09-10T21:42:56.933559+0000","last_became_active":"2024-09-10T21:42:55.938713+0000","last_became_peered":"2024-09-10T21:42:55.938713+0000","last_unstale":"2024-09-10T21:42:56.933559+0000","last_undegraded":"2024-09-10T21:42:56.933559+0000","last_fullsized":"2024-09-10T21:42:56.933559+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:48:39.274454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.935092+0000","last_change":"2024-09-10T21:42:55.929796+0000","last_active":"2024-09-10T21:42:55.935092+0000","last_peered":"2024-09-10T21:42:55.935092+0000","last_clean":"2024-09-10T21:42:55.935092+0000","last_became_active":"2024-09-10T21:42:55.929462+0000","last_became_peered":"2024-09-10T21:42:55.929462+0000","last_unstale":"2024-09-10T21:42:55.935092+0000","last_undegraded":"2024-09-10T21:42:55.935092+0000","last_fullsized":"2024-09-10T21:42:55.935092+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:08.093849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.926252+0000","last_change":"2024-09-10T21:42:55.934439+0000","last_active":"2024-09-10T21:42:56.926252+0000","last_peered":"2024-09-10T21:42:56.926252+0000","last_clean":"2024-09-10T21:42:56.926252+0000","last_became_active":"2024-09-10T21:42:55.934173+0000","last_became_peered":"2024-09-10T21:42:55.934173+0000","last_unstale":"2024-09-10T21:42:56.926252+0000","last_undegraded":"2024-09-10T21:42:56.926252+0000","last_fullsized":"2024-09-10T21:42:56.926252+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:00.111962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.919753+0000","last_change":"2024-09-10T21:42:55.917418+0000","last_active":"2024-09-10T21:42:56.919753+0000","last_peered":"2024-09-10T21:42:56.919753+0000","last_clean":"2024-09-10T21:42:56.919753+0000","last_became_active":"2024-09-10T21:42:55.917199+0000","last_became_peered":"2024-09-10T21:42:55.917199+0000","last_unstale":"2024-09-10T21:42:56.919753+0000","last_undegraded":"2024-09-10T21:42:56.919753+0000","last_fullsized":"2024-09-10T21:42:56.919753+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:53.037933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.938080+0000","last_change":"2024-09-10T21:42:55.929521+0000","last_active":"2024-09-10T21:42:55.938080+0000","last_peered":"2024-09-10T21:42:55.938080+0000","last_clean":"2024-09-10T21:42:55.938080+0000","last_became_active":"2024-09-10T21:42:55.929188+0000","last_became_peered":"2024-09-10T21:42:55.929188+0000","last_unstale":"2024-09-10T21:42:55.938080+0000","last_undegraded":"2024-09-10T21:42:55.938080+0000","last_fullsized":"2024-09-10T21:42:55.938080+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:08.093849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.924698+0000","last_change":"2024-09-10T21:42:55.928378+0000","last_active":"2024-09-10T21:42:56.924698+0000","last_peered":"2024-09-10T21:42:56.924698+0000","last_clean":"2024-09-10T21:42:56.924698+0000","last_became_active":"2024-09-10T21:42:55.928035+0000","last_became_peered":"2024-09-10T21:42:55.928035+0000","last_unstale":"2024-09-10T21:42:56.924698+0000","last_undegraded":"2024-09-10T21:42:56.924698+0000","last_fullsized":"2024-09-10T21:42:56.924698+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:34.003809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.943699+0000","last_change":"2024-09-10T21:42:55.938217+0000","last_active":"2024-09-10T21:42:55.943699+0000","last_peered":"2024-09-10T21:42:55.943699+0000","last_clean":"2024-09-10T21:42:55.943699+0000","last_became_active":"2024-09-10T21:42:55.937896+0000","last_became_peered":"2024-09-10T21:42:55.937896+0000","last_unstale":"2024-09-10T21:42:55.943699+0000","last_undegraded":"2024-09-10T21:42:55.943699+0000","last_fullsized":"2024-09-10T21:42:55.943699+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:29.498336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.937963+0000","last_change":"2024-09-10T21:42:55.928076+0000","last_active":"2024-09-10T21:42:55.937963+0000","last_peered":"2024-09-10T21:42:55.937963+0000","last_clean":"2024-09-10T21:42:55.937963+0000","last_became_active":"2024-09-10T21:42:55.923824+0000","last_became_peered":"2024-09-10T21:42:55.923824+0000","last_unstale":"2024-09-10T21:42:55.937963+0000","last_undegraded":"2024-09-10T21:42:55.937963+0000","last_fullsized":"2024-09-10T21:42:55.937963+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:36:39.159143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.945188+0000","last_change":"2024-09-10T21:42:55.931745+0000","last_active":"2024-09-10T21:42:55.945188+0000","last_peered":"2024-09-10T21:42:55.945188+0000","last_clean":"2024-09-10T21:42:55.945188+0000","last_became_active":"2024-09-10T21:42:55.930731+0000","last_became_peered":"2024-09-10T21:42:55.930731+0000","last_unstale":"2024-09-10T21:42:55.945188+0000","last_undegraded":"2024-09-10T21:42:55.945188+0000","last_fullsized":"2024-09-10T21:42:55.945188+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:06:14.793883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.945560+0000","last_change":"2024-09-10T21:42:55.931885+0000","last_active":"2024-09-10T21:42:55.945560+0000","last_peered":"2024-09-10T21:42:55.945560+0000","last_clean":"2024-09-10T21:42:55.945560+0000","last_became_active":"2024-09-10T21:42:55.931518+0000","last_became_peered":"2024-09-10T21:42:55.931518+0000","last_unstale":"2024-09-10T21:42:55.945560+0000","last_undegraded":"2024-09-10T21:42:55.945560+0000","last_fullsized":"2024-09-10T21:42:55.945560+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:45:18.552544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.919806+0000","last_change":"2024-09-10T21:42:55.918959+0000","last_active":"2024-09-10T21:42:56.919806+0000","last_peered":"2024-09-10T21:42:56.919806+0000","last_clean":"2024-09-10T21:42:56.919806+0000","last_became_active":"2024-09-10T21:42:55.918643+0000","last_became_peered":"2024-09-10T21:42:55.918643+0000","last_unstale":"2024-09-10T21:42:56.919806+0000","last_undegraded":"2024-09-10T21:42:56.919806+0000","last_fullsized":"2024-09-10T21:42:56.919806+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:24.608749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925132+0000","last_change":"2024-09-10T21:42:55.921167+0000","last_active":"2024-09-10T21:42:56.925132+0000","last_peered":"2024-09-10T21:42:56.925132+0000","last_clean":"2024-09-10T21:42:56.925132+0000","last_became_active":"2024-09-10T21:42:55.920760+0000","last_became_peered":"2024-09-10T21:42:55.920760+0000","last_unstale":"2024-09-10T21:42:56.925132+0000","last_undegraded":"2024-09-10T21:42:56.925132+0000","last_fullsized":"2024-09-10T21:42:56.925132+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:30.389091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925343+0000","last_change":"2024-09-10T21:42:55.932797+0000","last_active":"2024-09-10T21:42:56.925343+0000","last_peered":"2024-09-10T21:42:56.925343+0000","last_clean":"2024-09-10T21:42:56.925343+0000","last_became_active":"2024-09-10T21:42:55.931802+0000","last_became_peered":"2024-09-10T21:42:55.931802+0000","last_unstale":"2024-09-10T21:42:56.925343+0000","last_undegraded":"2024-09-10T21:42:56.925343+0000","last_fullsized":"2024-09-10T21:42:56.925343+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:50.985138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.930561+0000","last_change":"2024-09-10T21:42:55.919179+0000","last_active":"2024-09-10T21:42:56.930561+0000","last_peered":"2024-09-10T21:42:56.930561+0000","last_clean":"2024-09-10T21:42:56.930561+0000","last_became_active":"2024-09-10T21:42:55.918455+0000","last_became_peered":"2024-09-10T21:42:55.918455+0000","last_unstale":"2024-09-10T21:42:56.930561+0000","last_undegraded":"2024-09-10T21:42:56.930561+0000","last_fullsized":"2024-09-10T21:42:56.930561+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:36.719449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.924497+0000","last_change":"2024-09-10T21:42:55.921609+0000","last_active":"2024-09-10T21:42:56.924497+0000","last_peered":"2024-09-10T21:42:56.924497+0000","last_clean":"2024-09-10T21:42:56.924497+0000","last_became_active":"2024-09-10T21:42:55.918479+0000","last_became_peered":"2024-09-10T21:42:55.918479+0000","last_unstale":"2024-09-10T21:42:56.924497+0000","last_undegraded":"2024-09-10T21:42:56.924497+0000","last_fullsized":"2024-09-10T21:42:56.924497+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:47.086671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933777+0000","last_change":"2024-09-10T21:42:55.919209+0000","last_active":"2024-09-10T21:42:56.933777+0000","last_peered":"2024-09-10T21:42:56.933777+0000","last_clean":"2024-09-10T21:42:56.933777+0000","last_became_active":"2024-09-10T21:42:55.917683+0000","last_became_peered":"2024-09-10T21:42:55.917683+0000","last_unstale":"2024-09-10T21:42:56.933777+0000","last_undegraded":"2024-09-10T21:42:56.933777+0000","last_fullsized":"2024-09-10T21:42:56.933777+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:27.675336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.945343+0000","last_change":"2024-09-10T21:42:55.931588+0000","last_active":"2024-09-10T21:42:55.945343+0000","last_peered":"2024-09-10T21:42:55.945343+0000","last_clean":"2024-09-10T21:42:55.945343+0000","last_became_active":"2024-09-10T21:42:55.931042+0000","last_became_peered":"2024-09-10T21:42:55.931042+0000","last_unstale":"2024-09-10T21:42:55.945343+0000","last_undegraded":"2024-09-10T21:42:55.945343+0000","last_fullsized":"2024-09-10T21:42:55.945343+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:54:58.268834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933739+0000","last_change":"2024-09-10T21:42:55.924810+0000","last_active":"2024-09-10T21:42:56.933739+0000","last_peered":"2024-09-10T21:42:56.933739+0000","last_clean":"2024-09-10T21:42:56.933739+0000","last_became_active":"2024-09-10T21:42:55.922683+0000","last_became_peered":"2024-09-10T21:42:55.922683+0000","last_unstale":"2024-09-10T21:42:56.933739+0000","last_undegraded":"2024-09-10T21:42:56.933739+0000","last_fullsized":"2024-09-10T21:42:56.933739+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:15:55.359591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.926397+0000","last_change":"2024-09-10T21:42:55.925007+0000","last_active":"2024-09-10T21:42:56.926397+0000","last_peered":"2024-09-10T21:42:56.926397+0000","last_clean":"2024-09-10T21:42:56.926397+0000","last_became_active":"2024-09-10T21:42:55.924748+0000","last_became_peered":"2024-09-10T21:42:55.924748+0000","last_unstale":"2024-09-10T21:42:56.926397+0000","last_undegraded":"2024-09-10T21:42:56.926397+0000","last_fullsized":"2024-09-10T21:42:56.926397+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:57:11.876440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925246+0000","last_change":"2024-09-10T21:42:55.934668+0000","last_active":"2024-09-10T21:42:56.925246+0000","last_peered":"2024-09-10T21:42:56.925246+0000","last_clean":"2024-09-10T21:42:56.925246+0000","last_became_active":"2024-09-10T21:42:55.934204+0000","last_became_peered":"2024-09-10T21:42:55.934204+0000","last_unstale":"2024-09-10T21:42:56.925246+0000","last_undegraded":"2024-09-10T21:42:56.925246+0000","last_fullsized":"2024-09-10T21:42:56.925246+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:50:01.404871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.922622+0000","last_change":"2024-09-10T21:42:55.918528+0000","last_active":"2024-09-10T21:42:55.922622+0000","last_peered":"2024-09-10T21:42:55.922622+0000","last_clean":"2024-09-10T21:42:55.922622+0000","last_became_active":"2024-09-10T21:42:55.918147+0000","last_became_peered":"2024-09-10T21:42:55.918147+0000","last_unstale":"2024-09-10T21:42:55.922622+0000","last_undegraded":"2024-09-10T21:42:55.922622+0000","last_fullsized":"2024-09-10T21:42:55.922622+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:27.104414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.926077+0000","last_change":"2024-09-10T21:42:55.917548+0000","last_active":"2024-09-10T21:42:55.926077+0000","last_peered":"2024-09-10T21:42:55.926077+0000","last_clean":"2024-09-10T21:42:55.926077+0000","last_became_active":"2024-09-10T21:42:55.913219+0000","last_became_peered":"2024-09-10T21:42:55.913219+0000","last_unstale":"2024-09-10T21:42:55.926077+0000","last_undegraded":"2024-09-10T21:42:55.926077+0000","last_fullsized":"2024-09-10T21:42:55.926077+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:17.471225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.927766+0000","last_change":"2024-09-10T21:42:55.922444+0000","last_active":"2024-09-10T21:42:55.927766+0000","last_peered":"2024-09-10T21:42:55.927766+0000","last_clean":"2024-09-10T21:42:55.927766+0000","last_became_active":"2024-09-10T21:42:55.922199+0000","last_became_peered":"2024-09-10T21:42:55.922199+0000","last_unstale":"2024-09-10T21:42:55.927766+0000","last_undegraded":"2024-09-10T21:42:55.927766+0000","last_fullsized":"2024-09-10T21:42:55.927766+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:23.262669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.930883+0000","last_change":"2024-09-10T21:42:55.923894+0000","last_active":"2024-09-10T21:42:56.930883+0000","last_peered":"2024-09-10T21:42:56.930883+0000","last_clean":"2024-09-10T21:42:56.930883+0000","last_became_active":"2024-09-10T21:42:55.923621+0000","last_became_peered":"2024-09-10T21:42:55.923621+0000","last_unstale":"2024-09-10T21:42:56.930883+0000","last_undegraded":"2024-09-10T21:42:56.930883+0000","last_fullsized":"2024-09-10T21:42:56.930883+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:58:31.127604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"31'4","reported_seq":17,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.943463+0000","last_change":"2024-09-10T21:42:54.911491+0000","last_active":"2024-09-10T21:42:55.943463+0000","last_peered":"2024-09-10T21:42:55.943463+0000","last_clean":"2024-09-10T21:42:55.943463+0000","last_became_active":"2024-09-10T21:42:54.911178+0000","last_became_peered":"2024-09-10T21:42:54.911178+0000","last_unstale":"2024-09-10T21:42:55.943463+0000","last_undegraded":"2024-09-10T21:42:55.943463+0000","last_fullsized":"2024-09-10T21:42:55.943463+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:34.519141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.929570+0000","last_change":"2024-09-10T21:42:55.923625+0000","last_active":"2024-09-10T21:42:55.929570+0000","last_peered":"2024-09-10T21:42:55.929570+0000","last_clean":"2024-09-10T21:42:55.929570+0000","last_became_active":"2024-09-10T21:42:55.923358+0000","last_became_peered":"2024-09-10T21:42:55.923358+0000","last_unstale":"2024-09-10T21:42:55.929570+0000","last_undegraded":"2024-09-10T21:42:55.929570+0000","last_fullsized":"2024-09-10T21:42:55.929570+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:46.613096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.930904+0000","last_change":"2024-09-10T21:42:54.918303+0000","last_active":"2024-09-10T21:42:56.930904+0000","last_peered":"2024-09-10T21:42:56.930904+0000","last_clean":"2024-09-10T21:42:56.930904+0000","last_became_active":"2024-09-10T21:42:54.916342+0000","last_became_peered":"2024-09-10T21:42:54.916342+0000","last_unstale":"2024-09-10T21:42:56.930904+0000","last_undegraded":"2024-09-10T21:42:56.930904+0000","last_fullsized":"2024-09-10T21:42:56.930904+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:05:21.011406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925830+0000","last_change":"2024-09-10T21:42:55.929216+0000","last_active":"2024-09-10T21:42:56.925830+0000","last_peered":"2024-09-10T21:42:56.925830+0000","last_clean":"2024-09-10T21:42:56.925830+0000","last_became_active":"2024-09-10T21:42:55.928921+0000","last_became_peered":"2024-09-10T21:42:55.928921+0000","last_unstale":"2024-09-10T21:42:56.925830+0000","last_undegraded":"2024-09-10T21:42:56.925830+0000","last_fullsized":"2024-09-10T21:42:56.925830+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:24:39.244361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"31'2","reported_seq":15,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.945617+0000","last_change":"2024-09-10T21:42:54.918085+0000","last_active":"2024-09-10T21:42:55.945617+0000","last_peered":"2024-09-10T21:42:55.945617+0000","last_clean":"2024-09-10T21:42:55.945617+0000","last_became_active":"2024-09-10T21:42:54.917769+0000","last_became_peered":"2024-09-10T21:42:54.917769+0000","last_unstale":"2024-09-10T21:42:55.945617+0000","last_undegraded":"2024-09-10T21:42:55.945617+0000","last_fullsized":"2024-09-10T21:42:55.945617+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:58:01.322169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933867+0000","last_change":"2024-09-10T21:42:55.922965+0000","last_active":"2024-09-10T21:42:56.933867+0000","last_peered":"2024-09-10T21:42:56.933867+0000","last_clean":"2024-09-10T21:42:56.933867+0000","last_became_active":"2024-09-10T21:42:55.922297+0000","last_became_peered":"2024-09-10T21:42:55.922297+0000","last_unstale":"2024-09-10T21:42:56.933867+0000","last_undegraded":"2024-09-10T21:42:56.933867+0000","last_fullsized":"2024-09-10T21:42:56.933867+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:51.881372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.919918+0000","last_change":"2024-09-10T21:42:54.916465+0000","last_active":"2024-09-10T21:42:56.919918+0000","last_peered":"2024-09-10T21:42:56.919918+0000","last_clean":"2024-09-10T21:42:56.919918+0000","last_became_active":"2024-09-10T21:42:54.916138+0000","last_became_peered":"2024-09-10T21:42:54.916138+0000","last_unstale":"2024-09-10T21:42:56.919918+0000","last_undegraded":"2024-09-10T21:42:56.919918+0000","last_fullsized":"2024-09-10T21:42:56.919918+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:25.772752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.935027+0000","last_change":"2024-09-10T21:42:55.927341+0000","last_active":"2024-09-10T21:42:55.935027+0000","last_peered":"2024-09-10T21:42:55.935027+0000","last_clean":"2024-09-10T21:42:55.935027+0000","last_became_active":"2024-09-10T21:42:55.927069+0000","last_became_peered":"2024-09-10T21:42:55.927069+0000","last_unstale":"2024-09-10T21:42:55.935027+0000","last_undegraded":"2024-09-10T21:42:55.935027+0000","last_fullsized":"2024-09-10T21:42:55.935027+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:56.115221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925243+0000","last_change":"2024-09-10T21:42:54.914690+0000","last_active":"2024-09-10T21:42:56.925243+0000","last_peered":"2024-09-10T21:42:56.925243+0000","last_clean":"2024-09-10T21:42:56.925243+0000","last_became_active":"2024-09-10T21:42:54.913846+0000","last_became_peered":"2024-09-10T21:42:54.913846+0000","last_unstale":"2024-09-10T21:42:56.925243+0000","last_undegraded":"2024-09-10T21:42:56.925243+0000","last_fullsized":"2024-09-10T21:42:56.925243+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:23.602849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925696+0000","last_change":"2024-09-10T21:42:55.925685+0000","last_active":"2024-09-10T21:42:56.925696+0000","last_peered":"2024-09-10T21:42:56.925696+0000","last_clean":"2024-09-10T21:42:56.925696+0000","last_became_active":"2024-09-10T21:42:55.924888+0000","last_became_peered":"2024-09-10T21:42:55.924888+0000","last_unstale":"2024-09-10T21:42:56.925696+0000","last_undegraded":"2024-09-10T21:42:56.925696+0000","last_fullsized":"2024-09-10T21:42:56.925696+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:26.729812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925463+0000","last_change":"2024-09-10T21:42:54.913475+0000","last_active":"2024-09-10T21:42:56.925463+0000","last_peered":"2024-09-10T21:42:56.925463+0000","last_clean":"2024-09-10T21:42:56.925463+0000","last_became_active":"2024-09-10T21:42:54.913195+0000","last_became_peered":"2024-09-10T21:42:54.913195+0000","last_unstale":"2024-09-10T21:42:56.925463+0000","last_undegraded":"2024-09-10T21:42:56.925463+0000","last_fullsized":"2024-09-10T21:42:56.925463+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:45.328840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925422+0000","last_change":"2024-09-10T21:42:55.918347+0000","last_active":"2024-09-10T21:42:56.925422+0000","last_peered":"2024-09-10T21:42:56.925422+0000","last_clean":"2024-09-10T21:42:56.925422+0000","last_became_active":"2024-09-10T21:42:55.917738+0000","last_became_peered":"2024-09-10T21:42:55.917738+0000","last_unstale":"2024-09-10T21:42:56.925422+0000","last_undegraded":"2024-09-10T21:42:56.925422+0000","last_fullsized":"2024-09-10T21:42:56.925422+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:31.504136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.915627+0000","last_change":"2024-09-10T21:42:54.921414+0000","last_active":"2024-09-10T21:42:55.915627+0000","last_peered":"2024-09-10T21:42:55.915627+0000","last_clean":"2024-09-10T21:42:55.915627+0000","last_became_active":"2024-09-10T21:42:54.921123+0000","last_became_peered":"2024-09-10T21:42:54.921123+0000","last_unstale":"2024-09-10T21:42:55.915627+0000","last_undegraded":"2024-09-10T21:42:55.915627+0000","last_fullsized":"2024-09-10T21:42:55.915627+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:31:02.988891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933160+0000","last_change":"2024-09-10T21:42:55.926817+0000","last_active":"2024-09-10T21:42:56.933160+0000","last_peered":"2024-09-10T21:42:56.933160+0000","last_clean":"2024-09-10T21:42:56.933160+0000","last_became_active":"2024-09-10T21:42:55.926516+0000","last_became_peered":"2024-09-10T21:42:55.926516+0000","last_unstale":"2024-09-10T21:42:56.933160+0000","last_undegraded":"2024-09-10T21:42:56.933160+0000","last_fullsized":"2024-09-10T21:42:56.933160+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:31.504136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.924913+0000","last_change":"2024-09-10T21:42:54.903779+0000","last_active":"2024-09-10T21:42:56.924913+0000","last_peered":"2024-09-10T21:42:56.924913+0000","last_clean":"2024-09-10T21:42:56.924913+0000","last_became_active":"2024-09-10T21:42:54.903313+0000","last_became_peered":"2024-09-10T21:42:54.903313+0000","last_unstale":"2024-09-10T21:42:56.924913+0000","last_undegraded":"2024-09-10T21:42:56.924913+0000","last_fullsized":"2024-09-10T21:42:56.924913+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:08.052040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.926743+0000","last_change":"2024-09-10T21:42:55.939246+0000","last_active":"2024-09-10T21:42:56.926743+0000","last_peered":"2024-09-10T21:42:56.926743+0000","last_clean":"2024-09-10T21:42:56.926743+0000","last_became_active":"2024-09-10T21:42:55.938980+0000","last_became_peered":"2024-09-10T21:42:55.938980+0000","last_unstale":"2024-09-10T21:42:56.926743+0000","last_undegraded":"2024-09-10T21:42:56.926743+0000","last_fullsized":"2024-09-10T21:42:56.926743+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:45.531431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.921569+0000","last_change":"2024-09-10T21:42:54.915558+0000","last_active":"2024-09-10T21:42:55.921569+0000","last_peered":"2024-09-10T21:42:55.921569+0000","last_clean":"2024-09-10T21:42:55.921569+0000","last_became_active":"2024-09-10T21:42:54.915231+0000","last_became_peered":"2024-09-10T21:42:54.915231+0000","last_unstale":"2024-09-10T21:42:55.921569+0000","last_undegraded":"2024-09-10T21:42:55.921569+0000","last_fullsized":"2024-09-10T21:42:55.921569+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:55.109320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.945489+0000","last_change":"2024-09-10T21:42:55.931769+0000","last_active":"2024-09-10T21:42:55.945489+0000","last_peered":"2024-09-10T21:42:55.945489+0000","last_clean":"2024-09-10T21:42:55.945489+0000","last_became_active":"2024-09-10T21:42:55.931429+0000","last_became_peered":"2024-09-10T21:42:55.931429+0000","last_unstale":"2024-09-10T21:42:55.945489+0000","last_undegraded":"2024-09-10T21:42:55.945489+0000","last_fullsized":"2024-09-10T21:42:55.945489+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:04:18.457525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.931020+0000","last_change":"2024-09-10T21:42:54.919588+0000","last_active":"2024-09-10T21:42:56.931020+0000","last_peered":"2024-09-10T21:42:56.931020+0000","last_clean":"2024-09-10T21:42:56.931020+0000","last_became_active":"2024-09-10T21:42:54.918023+0000","last_became_peered":"2024-09-10T21:42:54.918023+0000","last_unstale":"2024-09-10T21:42:56.931020+0000","last_undegraded":"2024-09-10T21:42:56.931020+0000","last_fullsized":"2024-09-10T21:42:56.931020+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:47.410137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.942142+0000","last_change":"2024-09-10T21:42:55.928373+0000","last_active":"2024-09-10T21:42:55.942142+0000","last_peered":"2024-09-10T21:42:55.942142+0000","last_clean":"2024-09-10T21:42:55.942142+0000","last_became_active":"2024-09-10T21:42:55.927931+0000","last_became_peered":"2024-09-10T21:42:55.927931+0000","last_unstale":"2024-09-10T21:42:55.942142+0000","last_undegraded":"2024-09-10T21:42:55.942142+0000","last_fullsized":"2024-09-10T21:42:55.942142+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:07.120861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"31'3","reported_seq":18,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925927+0000","last_change":"2024-09-10T21:42:54.917326+0000","last_active":"2024-09-10T21:42:56.925927+0000","last_peered":"2024-09-10T21:42:56.925927+0000","last_clean":"2024-09-10T21:42:56.925927+0000","last_became_active":"2024-09-10T21:42:54.916975+0000","last_became_peered":"2024-09-10T21:42:54.916975+0000","last_unstale":"2024-09-10T21:42:56.925927+0000","last_undegraded":"2024-09-10T21:42:56.925927+0000","last_fullsized":"2024-09-10T21:42:56.925927+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:37:29.926328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925569+0000","last_change":"2024-09-10T21:42:55.917321+0000","last_active":"2024-09-10T21:42:56.925569+0000","last_peered":"2024-09-10T21:42:56.925569+0000","last_clean":"2024-09-10T21:42:56.925569+0000","last_became_active":"2024-09-10T21:42:55.917055+0000","last_became_peered":"2024-09-10T21:42:55.917055+0000","last_unstale":"2024-09-10T21:42:56.925569+0000","last_undegraded":"2024-09-10T21:42:56.925569+0000","last_fullsized":"2024-09-10T21:42:56.925569+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:13.755661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"31'6","reported_seq":19,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.948018+0000","last_change":"2024-09-10T21:42:54.918811+0000","last_active":"2024-09-10T21:42:55.948018+0000","last_peered":"2024-09-10T21:42:55.948018+0000","last_clean":"2024-09-10T21:42:55.948018+0000","last_became_active":"2024-09-10T21:42:54.918542+0000","last_became_peered":"2024-09-10T21:42:54.918542+0000","last_unstale":"2024-09-10T21:42:55.948018+0000","last_undegraded":"2024-09-10T21:42:55.948018+0000","last_fullsized":"2024-09-10T21:42:55.948018+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:22.256768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933205+0000","last_change":"2024-09-10T21:42:55.921692+0000","last_active":"2024-09-10T21:42:56.933205+0000","last_peered":"2024-09-10T21:42:56.933205+0000","last_clean":"2024-09-10T21:42:56.933205+0000","last_became_active":"2024-09-10T21:42:55.921345+0000","last_became_peered":"2024-09-10T21:42:55.921345+0000","last_unstale":"2024-09-10T21:42:56.933205+0000","last_undegraded":"2024-09-10T21:42:56.933205+0000","last_fullsized":"2024-09-10T21:42:56.933205+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:29.212260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925335+0000","last_change":"2024-09-10T21:42:54.903509+0000","last_active":"2024-09-10T21:42:56.925335+0000","last_peered":"2024-09-10T21:42:56.925335+0000","last_clean":"2024-09-10T21:42:56.925335+0000","last_became_active":"2024-09-10T21:42:54.903182+0000","last_became_peered":"2024-09-10T21:42:54.903182+0000","last_unstale":"2024-09-10T21:42:56.925335+0000","last_undegraded":"2024-09-10T21:42:56.925335+0000","last_fullsized":"2024-09-10T21:42:56.925335+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:02.063418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.945218+0000","last_change":"2024-09-10T21:42:55.930485+0000","last_active":"2024-09-10T21:42:55.945218+0000","last_peered":"2024-09-10T21:42:55.945218+0000","last_clean":"2024-09-10T21:42:55.945218+0000","last_became_active":"2024-09-10T21:42:55.929597+0000","last_became_peered":"2024-09-10T21:42:55.929597+0000","last_unstale":"2024-09-10T21:42:55.945218+0000","last_undegraded":"2024-09-10T21:42:55.945218+0000","last_fullsized":"2024-09-10T21:42:55.945218+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:22.309511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933305+0000","last_change":"2024-09-10T21:42:54.908462+0000","last_active":"2024-09-10T21:42:56.933305+0000","last_peered":"2024-09-10T21:42:56.933305+0000","last_clean":"2024-09-10T21:42:56.933305+0000","last_became_active":"2024-09-10T21:42:54.906528+0000","last_became_peered":"2024-09-10T21:42:54.906528+0000","last_unstale":"2024-09-10T21:42:56.933305+0000","last_undegraded":"2024-09-10T21:42:56.933305+0000","last_fullsized":"2024-09-10T21:42:56.933305+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:09.539994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.927743+0000","last_change":"2024-09-10T21:42:55.922274+0000","last_active":"2024-09-10T21:42:55.927743+0000","last_peered":"2024-09-10T21:42:55.927743+0000","last_clean":"2024-09-10T21:42:55.927743+0000","last_became_active":"2024-09-10T21:42:55.922011+0000","last_became_peered":"2024-09-10T21:42:55.922011+0000","last_unstale":"2024-09-10T21:42:55.927743+0000","last_undegraded":"2024-09-10T21:42:55.927743+0000","last_fullsized":"2024-09-10T21:42:55.927743+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:24.106475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.916195+0000","last_change":"2024-09-10T21:42:54.915698+0000","last_active":"2024-09-10T21:42:55.916195+0000","last_peered":"2024-09-10T21:42:55.916195+0000","last_clean":"2024-09-10T21:42:55.916195+0000","last_became_active":"2024-09-10T21:42:54.915434+0000","last_became_peered":"2024-09-10T21:42:54.915434+0000","last_unstale":"2024-09-10T21:42:55.916195+0000","last_undegraded":"2024-09-10T21:42:55.916195+0000","last_fullsized":"2024-09-10T21:42:55.916195+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:26.098513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.930456+0000","last_change":"2024-09-10T21:42:55.925709+0000","last_active":"2024-09-10T21:42:55.930456+0000","last_peered":"2024-09-10T21:42:55.930456+0000","last_clean":"2024-09-10T21:42:55.930456+0000","last_became_active":"2024-09-10T21:42:55.922936+0000","last_became_peered":"2024-09-10T21:42:55.922936+0000","last_unstale":"2024-09-10T21:42:55.930456+0000","last_undegraded":"2024-09-10T21:42:55.930456+0000","last_fullsized":"2024-09-10T21:42:55.930456+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:01.214509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"31'8","reported_seq":21,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.946228+0000","last_change":"2024-09-10T21:42:54.913658+0000","last_active":"2024-09-10T21:42:55.946228+0000","last_peered":"2024-09-10T21:42:55.946228+0000","last_clean":"2024-09-10T21:42:55.946228+0000","last_became_active":"2024-09-10T21:42:54.913386+0000","last_became_peered":"2024-09-10T21:42:54.913386+0000","last_unstale":"2024-09-10T21:42:55.946228+0000","last_undegraded":"2024-09-10T21:42:55.946228+0000","last_fullsized":"2024-09-10T21:42:55.946228+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:24.590339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925754+0000","last_change":"2024-09-10T21:42:55.920787+0000","last_active":"2024-09-10T21:42:56.925754+0000","last_peered":"2024-09-10T21:42:56.925754+0000","last_clean":"2024-09-10T21:42:56.925754+0000","last_became_active":"2024-09-10T21:42:55.919469+0000","last_became_peered":"2024-09-10T21:42:55.919469+0000","last_unstale":"2024-09-10T21:42:56.925754+0000","last_undegraded":"2024-09-10T21:42:56.925754+0000","last_fullsized":"2024-09-10T21:42:56.925754+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:29.212260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933841+0000","last_change":"2024-09-10T21:42:54.913809+0000","last_active":"2024-09-10T21:42:56.933841+0000","last_peered":"2024-09-10T21:42:56.933841+0000","last_clean":"2024-09-10T21:42:56.933841+0000","last_became_active":"2024-09-10T21:42:54.913489+0000","last_became_peered":"2024-09-10T21:42:54.913489+0000","last_unstale":"2024-09-10T21:42:56.933841+0000","last_undegraded":"2024-09-10T21:42:56.933841+0000","last_fullsized":"2024-09-10T21:42:56.933841+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:49.605597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.926710+0000","last_change":"2024-09-10T21:42:55.926751+0000","last_active":"2024-09-10T21:42:56.926710+0000","last_peered":"2024-09-10T21:42:56.926710+0000","last_clean":"2024-09-10T21:42:56.926710+0000","last_became_active":"2024-09-10T21:42:55.926247+0000","last_became_peered":"2024-09-10T21:42:55.926247+0000","last_unstale":"2024-09-10T21:42:56.926710+0000","last_undegraded":"2024-09-10T21:42:56.926710+0000","last_fullsized":"2024-09-10T21:42:56.926710+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:53.704714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925896+0000","last_change":"2024-09-10T21:42:54.906657+0000","last_active":"2024-09-10T21:42:56.925896+0000","last_peered":"2024-09-10T21:42:56.925896+0000","last_clean":"2024-09-10T21:42:56.925896+0000","last_became_active":"2024-09-10T21:42:54.906308+0000","last_became_peered":"2024-09-10T21:42:54.906308+0000","last_unstale":"2024-09-10T21:42:56.925896+0000","last_undegraded":"2024-09-10T21:42:56.925896+0000","last_fullsized":"2024-09-10T21:42:56.925896+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:00.835587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933662+0000","last_change":"2024-09-10T21:42:55.921836+0000","last_active":"2024-09-10T21:42:56.933662+0000","last_peered":"2024-09-10T21:42:56.933662+0000","last_clean":"2024-09-10T21:42:56.933662+0000","last_became_active":"2024-09-10T21:42:55.921559+0000","last_became_peered":"2024-09-10T21:42:55.921559+0000","last_unstale":"2024-09-10T21:42:56.933662+0000","last_undegraded":"2024-09-10T21:42:56.933662+0000","last_fullsized":"2024-09-10T21:42:56.933662+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:00.956802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"31'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.926261+0000","last_change":"2024-09-10T21:42:54.910592+0000","last_active":"2024-09-10T21:42:56.926261+0000","last_peered":"2024-09-10T21:42:56.926261+0000","last_clean":"2024-09-10T21:42:56.926261+0000","last_became_active":"2024-09-10T21:42:54.910296+0000","last_became_peered":"2024-09-10T21:42:54.910296+0000","last_unstale":"2024-09-10T21:42:56.926261+0000","last_undegraded":"2024-09-10T21:42:56.926261+0000","last_fullsized":"2024-09-10T21:42:56.926261+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:39.167492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.920025+0000","last_change":"2024-09-10T21:42:55.923971+0000","last_active":"2024-09-10T21:42:56.920025+0000","last_peered":"2024-09-10T21:42:56.920025+0000","last_clean":"2024-09-10T21:42:56.920025+0000","last_became_active":"2024-09-10T21:42:55.923669+0000","last_became_peered":"2024-09-10T21:42:55.923669+0000","last_unstale":"2024-09-10T21:42:56.920025+0000","last_undegraded":"2024-09-10T21:42:56.920025+0000","last_fullsized":"2024-09-10T21:42:56.920025+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:35.525042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.926146+0000","last_change":"2024-09-10T21:42:54.913677+0000","last_active":"2024-09-10T21:42:56.926146+0000","last_peered":"2024-09-10T21:42:56.926146+0000","last_clean":"2024-09-10T21:42:56.926146+0000","last_became_active":"2024-09-10T21:42:54.913385+0000","last_became_peered":"2024-09-10T21:42:54.913385+0000","last_unstale":"2024-09-10T21:42:56.926146+0000","last_undegraded":"2024-09-10T21:42:56.926146+0000","last_fullsized":"2024-09-10T21:42:56.926146+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:49.605597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.919683+0000","last_change":"2024-09-10T21:42:55.916897+0000","last_active":"2024-09-10T21:42:56.919683+0000","last_peered":"2024-09-10T21:42:56.919683+0000","last_clean":"2024-09-10T21:42:56.919683+0000","last_became_active":"2024-09-10T21:42:55.916627+0000","last_became_peered":"2024-09-10T21:42:55.916627+0000","last_unstale":"2024-09-10T21:42:56.919683+0000","last_undegraded":"2024-09-10T21:42:56.919683+0000","last_fullsized":"2024-09-10T21:42:56.919683+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:42:20.745519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.930707+0000","last_change":"2024-09-10T21:42:54.920488+0000","last_active":"2024-09-10T21:42:56.930707+0000","last_peered":"2024-09-10T21:42:56.930707+0000","last_clean":"2024-09-10T21:42:56.930707+0000","last_became_active":"2024-09-10T21:42:54.920128+0000","last_became_peered":"2024-09-10T21:42:54.920128+0000","last_unstale":"2024-09-10T21:42:56.930707+0000","last_undegraded":"2024-09-10T21:42:56.930707+0000","last_fullsized":"2024-09-10T21:42:56.930707+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:29.144828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.919840+0000","last_change":"2024-09-10T21:42:55.924371+0000","last_active":"2024-09-10T21:42:56.919840+0000","last_peered":"2024-09-10T21:42:56.919840+0000","last_clean":"2024-09-10T21:42:56.919840+0000","last_became_active":"2024-09-10T21:42:55.924105+0000","last_became_peered":"2024-09-10T21:42:55.924105+0000","last_unstale":"2024-09-10T21:42:56.919840+0000","last_undegraded":"2024-09-10T21:42:56.919840+0000","last_fullsized":"2024-09-10T21:42:56.919840+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:03.069319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.907556+0000","last_change":"2024-09-10T21:42:54.918220+0000","last_active":"2024-09-10T21:42:55.907556+0000","last_peered":"2024-09-10T21:42:55.907556+0000","last_clean":"2024-09-10T21:42:55.907556+0000","last_became_active":"2024-09-10T21:42:54.917887+0000","last_became_peered":"2024-09-10T21:42:54.917887+0000","last_unstale":"2024-09-10T21:42:55.907556+0000","last_undegraded":"2024-09-10T21:42:55.907556+0000","last_fullsized":"2024-09-10T21:42:55.907556+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:49.605597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.919680+0000","last_change":"2024-09-10T21:42:55.914548+0000","last_active":"2024-09-10T21:42:56.919680+0000","last_peered":"2024-09-10T21:42:56.919680+0000","last_clean":"2024-09-10T21:42:56.919680+0000","last_became_active":"2024-09-10T21:42:55.913610+0000","last_became_peered":"2024-09-10T21:42:55.913610+0000","last_unstale":"2024-09-10T21:42:56.919680+0000","last_undegraded":"2024-09-10T21:42:56.919680+0000","last_fullsized":"2024-09-10T21:42:56.919680+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:52:46.334741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925224+0000","last_change":"2024-09-10T21:42:54.909244+0000","last_active":"2024-09-10T21:42:56.925224+0000","last_peered":"2024-09-10T21:42:56.925224+0000","last_clean":"2024-09-10T21:42:56.925224+0000","last_became_active":"2024-09-10T21:42:54.908942+0000","last_became_peered":"2024-09-10T21:42:54.908942+0000","last_unstale":"2024-09-10T21:42:56.925224+0000","last_undegraded":"2024-09-10T21:42:56.925224+0000","last_fullsized":"2024-09-10T21:42:56.925224+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:52.032032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.937914+0000","last_change":"2024-09-10T21:42:55.931798+0000","last_active":"2024-09-10T21:42:55.937914+0000","last_peered":"2024-09-10T21:42:55.937914+0000","last_clean":"2024-09-10T21:42:55.937914+0000","last_became_active":"2024-09-10T21:42:55.931556+0000","last_became_peered":"2024-09-10T21:42:55.931556+0000","last_unstale":"2024-09-10T21:42:55.937914+0000","last_undegraded":"2024-09-10T21:42:55.937914+0000","last_fullsized":"2024-09-10T21:42:55.937914+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:34:32.444806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.924383+0000","last_change":"2024-09-10T21:42:54.915745+0000","last_active":"2024-09-10T21:42:56.924383+0000","last_peered":"2024-09-10T21:42:56.924383+0000","last_clean":"2024-09-10T21:42:56.924383+0000","last_became_active":"2024-09-10T21:42:54.913515+0000","last_became_peered":"2024-09-10T21:42:54.913515+0000","last_unstale":"2024-09-10T21:42:56.924383+0000","last_undegraded":"2024-09-10T21:42:56.924383+0000","last_fullsized":"2024-09-10T21:42:56.924383+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:03.523688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.920067+0000","last_change":"2024-09-10T21:42:55.919268+0000","last_active":"2024-09-10T21:42:56.920067+0000","last_peered":"2024-09-10T21:42:56.920067+0000","last_clean":"2024-09-10T21:42:56.920067+0000","last_became_active":"2024-09-10T21:42:55.919006+0000","last_became_peered":"2024-09-10T21:42:55.919006+0000","last_unstale":"2024-09-10T21:42:56.920067+0000","last_undegraded":"2024-09-10T21:42:56.920067+0000","last_fullsized":"2024-09-10T21:42:56.920067+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:50.611498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"31'2","reported_seq":17,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933904+0000","last_change":"2024-09-10T21:42:54.913663+0000","last_active":"2024-09-10T21:42:56.933904+0000","last_peered":"2024-09-10T21:42:56.933904+0000","last_clean":"2024-09-10T21:42:56.933904+0000","last_became_active":"2024-09-10T21:42:54.913235+0000","last_became_peered":"2024-09-10T21:42:54.913235+0000","last_unstale":"2024-09-10T21:42:56.933904+0000","last_undegraded":"2024-09-10T21:42:56.933904+0000","last_fullsized":"2024-09-10T21:42:56.933904+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:34:26.221672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.926886+0000","last_change":"2024-09-10T21:42:55.927903+0000","last_active":"2024-09-10T21:42:56.926886+0000","last_peered":"2024-09-10T21:42:56.926886+0000","last_clean":"2024-09-10T21:42:56.926886+0000","last_became_active":"2024-09-10T21:42:55.927644+0000","last_became_peered":"2024-09-10T21:42:55.927644+0000","last_unstale":"2024-09-10T21:42:56.926886+0000","last_undegraded":"2024-09-10T21:42:56.926886+0000","last_fullsized":"2024-09-10T21:42:56.926886+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:22.149532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"32'1","reported_seq":16,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.932434+0000","last_change":"2024-09-10T21:42:54.915239+0000","last_active":"2024-09-10T21:42:56.932434+0000","last_peered":"2024-09-10T21:42:56.932434+0000","last_clean":"2024-09-10T21:42:56.932434+0000","last_became_active":"2024-09-10T21:42:54.914940+0000","last_became_peered":"2024-09-10T21:42:54.914940+0000","last_unstale":"2024-09-10T21:42:56.932434+0000","last_undegraded":"2024-09-10T21:42:56.932434+0000","last_fullsized":"2024-09-10T21:42:56.932434+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:29.899175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.930651+0000","last_change":"2024-09-10T21:42:55.922264+0000","last_active":"2024-09-10T21:42:56.930651+0000","last_peered":"2024-09-10T21:42:56.930651+0000","last_clean":"2024-09-10T21:42:56.930651+0000","last_became_active":"2024-09-10T21:42:55.920794+0000","last_became_peered":"2024-09-10T21:42:55.920794+0000","last_unstale":"2024-09-10T21:42:56.930651+0000","last_undegraded":"2024-09-10T21:42:56.930651+0000","last_fullsized":"2024-09-10T21:42:56.930651+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:55:12.151596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933424+0000","last_change":"2024-09-10T21:42:54.910060+0000","last_active":"2024-09-10T21:42:56.933424+0000","last_peered":"2024-09-10T21:42:56.933424+0000","last_clean":"2024-09-10T21:42:56.933424+0000","last_became_active":"2024-09-10T21:42:54.909577+0000","last_became_peered":"2024-09-10T21:42:54.909577+0000","last_unstale":"2024-09-10T21:42:56.933424+0000","last_undegraded":"2024-09-10T21:42:56.933424+0000","last_fullsized":"2024-09-10T21:42:56.933424+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:16.465324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.927911+0000","last_change":"2024-09-10T21:42:55.920207+0000","last_active":"2024-09-10T21:42:55.927911+0000","last_peered":"2024-09-10T21:42:55.927911+0000","last_clean":"2024-09-10T21:42:55.927911+0000","last_became_active":"2024-09-10T21:42:55.919870+0000","last_became_peered":"2024-09-10T21:42:55.919870+0000","last_unstale":"2024-09-10T21:42:55.927911+0000","last_undegraded":"2024-09-10T21:42:55.927911+0000","last_fullsized":"2024-09-10T21:42:55.927911+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:10.545895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.930841+0000","last_change":"2024-09-10T21:42:54.920494+0000","last_active":"2024-09-10T21:42:56.930841+0000","last_peered":"2024-09-10T21:42:56.930841+0000","last_clean":"2024-09-10T21:42:56.930841+0000","last_became_active":"2024-09-10T21:42:54.920164+0000","last_became_peered":"2024-09-10T21:42:54.920164+0000","last_unstale":"2024-09-10T21:42:56.930841+0000","last_undegraded":"2024-09-10T21:42:56.930841+0000","last_fullsized":"2024-09-10T21:42:56.930841+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:09.939461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.930763+0000","last_change":"2024-09-10T21:42:55.920622+0000","last_active":"2024-09-10T21:42:56.930763+0000","last_peered":"2024-09-10T21:42:56.930763+0000","last_clean":"2024-09-10T21:42:56.930763+0000","last_became_active":"2024-09-10T21:42:55.920352+0000","last_became_peered":"2024-09-10T21:42:55.920352+0000","last_unstale":"2024-09-10T21:42:56.930763+0000","last_undegraded":"2024-09-10T21:42:56.930763+0000","last_fullsized":"2024-09-10T21:42:56.930763+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:28.540988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.933409+0000","last_change":"2024-09-10T21:42:54.908752+0000","last_active":"2024-09-10T21:42:56.933409+0000","last_peered":"2024-09-10T21:42:56.933409+0000","last_clean":"2024-09-10T21:42:56.933409+0000","last_became_active":"2024-09-10T21:42:54.908287+0000","last_became_peered":"2024-09-10T21:42:54.908287+0000","last_unstale":"2024-09-10T21:42:56.933409+0000","last_undegraded":"2024-09-10T21:42:56.933409+0000","last_fullsized":"2024-09-10T21:42:56.933409+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:33:45.607195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.945405+0000","last_change":"2024-09-10T21:42:55.929837+0000","last_active":"2024-09-10T21:42:55.945405+0000","last_peered":"2024-09-10T21:42:55.945405+0000","last_clean":"2024-09-10T21:42:55.945405+0000","last_became_active":"2024-09-10T21:42:55.929566+0000","last_became_peered":"2024-09-10T21:42:55.929566+0000","last_unstale":"2024-09-10T21:42:55.945405+0000","last_undegraded":"2024-09-10T21:42:55.945405+0000","last_fullsized":"2024-09-10T21:42:55.945405+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:36:39.159143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925318+0000","last_change":"2024-09-10T21:42:54.913739+0000","last_active":"2024-09-10T21:42:56.925318+0000","last_peered":"2024-09-10T21:42:56.925318+0000","last_clean":"2024-09-10T21:42:56.925318+0000","last_became_active":"2024-09-10T21:42:54.913435+0000","last_became_peered":"2024-09-10T21:42:54.913435+0000","last_unstale":"2024-09-10T21:42:56.925318+0000","last_undegraded":"2024-09-10T21:42:56.925318+0000","last_fullsized":"2024-09-10T21:42:56.925318+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:22:34.519141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-10T21:42:56.925101+0000","last_change":"2024-09-10T21:42:55.929272+0000","last_active":"2024-09-10T21:42:56.925101+0000","last_peered":"2024-09-10T21:42:56.925101+0000","last_clean":"2024-09-10T21:42:56.925101+0000","last_became_active":"2024-09-10T21:42:55.929034+0000","last_became_peered":"2024-09-10T21:42:55.929034+0000","last_unstale":"2024-09-10T21:42:56.925101+0000","last_undegraded":"2024-09-10T21:42:56.925101+0000","last_fullsized":"2024-09-10T21:42:56.925101+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:05.196124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.916216+0000","last_change":"2024-09-10T21:42:54.921213+0000","last_active":"2024-09-10T21:42:55.916216+0000","last_peered":"2024-09-10T21:42:55.916216+0000","last_clean":"2024-09-10T21:42:55.916216+0000","last_became_active":"2024-09-10T21:42:54.920945+0000","last_became_peered":"2024-09-10T21:42:54.920945+0000","last_unstale":"2024-09-10T21:42:55.916216+0000","last_undegraded":"2024-09-10T21:42:55.916216+0000","last_fullsized":"2024-09-10T21:42:55.916216+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:18:23.100574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.935061+0000","last_change":"2024-09-10T21:42:55.927589+0000","last_active":"2024-09-10T21:42:55.935061+0000","last_peered":"2024-09-10T21:42:55.935061+0000","last_clean":"2024-09-10T21:42:55.935061+0000","last_became_active":"2024-09-10T21:42:55.927279+0000","last_became_peered":"2024-09-10T21:42:55.927279+0000","last_unstale":"2024-09-10T21:42:55.935061+0000","last_undegraded":"2024-09-10T21:42:55.935061+0000","last_fullsized":"2024-09-10T21:42:55.935061+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":30,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:54.886016+0000","last_clean_scrub_stamp":"2024-09-10T21:42:54.886016+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:07.120861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.906553+0000","last_change":"2024-09-10T21:42:54.916404+0000","last_active":"2024-09-10T21:42:55.906553+0000","last_peered":"2024-09-10T21:42:55.906553+0000","last_clean":"2024-09-10T21:42:55.906553+0000","last_became_active":"2024-09-10T21:42:54.916085+0000","last_became_peered":"2024-09-10T21:42:54.916085+0000","last_unstale":"2024-09-10T21:42:55.906553+0000","last_undegraded":"2024-09-10T21:42:55.906553+0000","last_fullsized":"2024-09-10T21:42:55.906553+0000","mapping_epoch":29,"log_start":"0'0","ondisk_log_start":"0'0","created":29,"last_epoch_clean":30,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:42:53.880115+0000","last_clean_scrub_stamp":"2024-09-10T21:42:53.880115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:02.063418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":127,"reported_epoch":31,"state":"active+clean","last_fresh":"2024-09-10T21:42:55.911175+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:42:55.911175+0000","last_peered":"2024-09-10T21:42:55.911175+0000","last_clean":"2024-09-10T21:42:55.911175+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:42:55.911175+0000","last_undegraded":"2024-09-10T21:42:55.911175+0000","last_fullsized":"2024-09-10T21:42:55.911175+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27376,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344464,"statfs":{"total":96636764160,"available":96608731136,"internally_reserved":0,"allocated":348160,"data_stored":129695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509455,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27368,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344472,"statfs":{"total":96636764160,"available":96608739328,"internally_reserved":0,"allocated":348160,"data_stored":129183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509455,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31536,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340304,"statfs":{"total":96636764160,"available":96604471296,"internally_reserved":0,"allocated":356352,"data_stored":135449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509457,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31552,"kb_used_data":360,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340288,"statfs":{"total":96636764160,"available":96604454912,"internally_reserved":0,"allocated":368640,"data_stored":137013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509455,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339728,"statfs":{"total":96636764160,"available":96603881472,"internally_reserved":0,"allocated":946176,"data_stored":725747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542158,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32108,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339732,"statfs":{"total":96636764160,"available":96603885568,"internally_reserved":0,"allocated":937984,"data_stored":719537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542159,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31540,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340300,"statfs":{"total":96636764160,"available":96604467200,"internally_reserved":0,"allocated":356352,"data_stored":130713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:43:00.629 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:43:00.629 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:43:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.834+0000 7faba1e00640 1 -- 172.21.15.7:0/3725841312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab9c0cde00 msgr2=0x7fab9c0ad8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.834+0000 7faba1e00640 1 --2- 172.21.15.7:0/3725841312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab9c0cde00 0x7fab9c0ad8f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fab84009830 tx=0x7fab8402a480 comp rx=0 tx=0).stop 2024-09-10T21:43:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.834+0000 7faba1e00640 1 -- 172.21.15.7:0/3725841312 shutdown_connections 2024-09-10T21:43:00.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.834+0000 7faba1e00640 1 --2- 172.21.15.7:0/3725841312 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab9c158600 0x7fab9c0ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.834+0000 7faba1e00640 1 --2- 172.21.15.7:0/3725841312 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab9c0cde00 0x7fab9c0ad8f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.834+0000 7faba1e00640 1 -- 172.21.15.7:0/3725841312 >> 172.21.15.7:0/3725841312 conn(0x7fab9c0b16d0 msgr2=0x7fab9c0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.834+0000 7faba1e00640 1 -- 172.21.15.7:0/3725841312 shutdown_connections 2024-09-10T21:43:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.834+0000 7faba1e00640 1 -- 172.21.15.7:0/3725841312 wait complete. 2024-09-10T21:43:00.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.835+0000 7faba1e00640 1 Processor -- start 2024-09-10T21:43:00.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.836+0000 7faba1e00640 1 -- start start 2024-09-10T21:43:00.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.836+0000 7faba1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab9c0cde00 0x7fab9c0c4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:00.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.836+0000 7faba1e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab9c158220 0x7fab9c0c5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:00.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.836+0000 7faba1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab9c158600 0x7fab9c156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:00.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.837+0000 7faba1e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c06fa10 con 0x7fab9c0cde00 2024-09-10T21:43:00.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.837+0000 7faba1e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fab9c06f890 con 0x7fab9c158600 2024-09-10T21:43:00.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.837+0000 7faba1e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fab9c06fb90 con 0x7fab9c158220 2024-09-10T21:43:00.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.837+0000 7fab9aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab9c0cde00 0x7fab9c0c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:00.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.837+0000 7fab9aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab9c0cde00 0x7fab9c0c4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57550/0 (socket says 172.21.15.7:57550) 2024-09-10T21:43:00.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.837+0000 7fab9aa00640 1 -- 172.21.15.7:0/216898847 learned_addr learned my addr 172.21.15.7:0/216898847 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:00.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.837+0000 7fab92000640 1 --2- 172.21.15.7:0/216898847 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab9c158220 0x7fab9c0c5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:00.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.837+0000 7fab9b400640 1 --2- 172.21.15.7:0/216898847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab9c158600 0x7fab9c156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:00.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.837+0000 7fab9aa00640 1 -- 172.21.15.7:0/216898847 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab9c158220 msgr2=0x7fab9c0c5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:00.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.837+0000 7fab9aa00640 1 --2- 172.21.15.7:0/216898847 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab9c158220 0x7fab9c0c5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:00.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.837+0000 7fab9aa00640 1 -- 172.21.15.7:0/216898847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab9c158600 msgr2=0x7fab9c156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:00.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.837+0000 7fab9aa00640 1 --2- 172.21.15.7:0/216898847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab9c158600 0x7fab9c156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:00.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.837+0000 7fab9aa00640 1 -- 172.21.15.7:0/216898847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab9c0741c0 con 0x7fab9c0cde00 2024-09-10T21:43:00.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.838+0000 7fab9aa00640 1 --2- 172.21.15.7:0/216898847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab9c0cde00 0x7fab9c0c4300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00c490 tx=0x7fab8c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:00.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.838+0000 7fab93e00640 1 -- 172.21.15.7:0/216898847 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c0052d0 con 0x7fab9c0cde00 2024-09-10T21:43:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.838+0000 7fab93e00640 1 -- 172.21.15.7:0/216898847 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab8c005490 con 0x7fab9c0cde00 2024-09-10T21:43:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.838+0000 7faba1e00640 1 -- 172.21.15.7:0/216898847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab9c0744a0 con 0x7fab9c0cde00 2024-09-10T21:43:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.838+0000 7fab93e00640 1 -- 172.21.15.7:0/216898847 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab8c012870 con 0x7fab9c0cde00 2024-09-10T21:43:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.838+0000 7faba1e00640 1 -- 172.21.15.7:0/216898847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab9c0b2d50 con 0x7fab9c0cde00 2024-09-10T21:43:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.839+0000 7faba1e00640 1 -- 172.21.15.7:0/216898847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab9c0b2f50 con 0x7fab9c0cde00 2024-09-10T21:43:00.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.840+0000 7fab93e00640 1 -- 172.21.15.7:0/216898847 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fab8c012a10 con 0x7fab9c0cde00 2024-09-10T21:43:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.842+0000 7fab93e00640 1 --2- 172.21.15.7:0/216898847 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fab78077fa0 0x7fab7807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.842+0000 7fab93e00640 1 -- 172.21.15.7:0/216898847 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fab8c099750 con 0x7fab9c0cde00 2024-09-10T21:43:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.842+0000 7fab92000640 1 --2- 172.21.15.7:0/216898847 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fab78077fa0 0x7fab7807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:00.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.843+0000 7fab92000640 1 --2- 172.21.15.7:0/216898847 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fab78077fa0 0x7fab7807a460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fab84034000 tx=0x7fab84004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:00.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:00.846+0000 7fab93e00640 1 -- 172.21.15.7:0/216898847 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fab9c0b2f50 con 0x7fab9c0cde00 2024-09-10T21:43:01.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.113+0000 7faba1e00640 1 -- 172.21.15.7:0/216898847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fab9c0ce320 con 0x7fab9c0cde00 2024-09-10T21:43:01.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.116+0000 7fab93e00640 1 -- 172.21.15.7:0/216898847 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v32) ==== 190+0+147 (secure 0 0 0) 0x7fab8c061bf0 con 0x7fab9c0cde00 2024-09-10T21:43:01.118 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-09-10T21:43:01.118 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDvvOBm0aqcFhAAtid2bnzjcpaYAbU7g0JcOA== 2024-09-10T21:43:01.118 INFO:teuthology.orchestra.run.smithi007.stdout: caps mds = "allow" 2024-09-10T21:43:01.118 INFO:teuthology.orchestra.run.smithi007.stdout: caps mon = "allow r" 2024-09-10T21:43:01.118 INFO:teuthology.orchestra.run.smithi007.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:43:01.118 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-09-10T21:43:01.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.119+0000 7faba1e00640 1 -- 172.21.15.7:0/216898847 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fab78077fa0 msgr2=0x7fab7807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:01.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.119+0000 7faba1e00640 1 --2- 172.21.15.7:0/216898847 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fab78077fa0 0x7fab7807a460 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fab84034000 tx=0x7fab84004730 comp rx=0 tx=0).stop 2024-09-10T21:43:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.119+0000 7faba1e00640 1 -- 172.21.15.7:0/216898847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab9c0cde00 msgr2=0x7fab9c0c4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.119+0000 7faba1e00640 1 --2- 172.21.15.7:0/216898847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab9c0cde00 0x7fab9c0c4300 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fab8c00c490 tx=0x7fab8c00c950 comp rx=0 tx=0).stop 2024-09-10T21:43:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.119+0000 7faba1e00640 1 -- 172.21.15.7:0/216898847 shutdown_connections 2024-09-10T21:43:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.119+0000 7faba1e00640 1 --2- 172.21.15.7:0/216898847 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fab78077fa0 0x7fab7807a460 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.119+0000 7faba1e00640 1 --2- 172.21.15.7:0/216898847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab9c158600 0x7fab9c156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.119+0000 7faba1e00640 1 --2- 172.21.15.7:0/216898847 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab9c158220 0x7fab9c0c5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.119+0000 7faba1e00640 1 --2- 172.21.15.7:0/216898847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab9c0cde00 0x7fab9c0c4300 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.119+0000 7faba1e00640 1 -- 172.21.15.7:0/216898847 >> 172.21.15.7:0/216898847 conn(0x7fab9c0b16d0 msgr2=0x7fab9c160900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.119+0000 7faba1e00640 1 -- 172.21.15.7:0/216898847 shutdown_connections 2024-09-10T21:43:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.119+0000 7faba1e00640 1 -- 172.21.15.7:0/216898847 wait complete. 2024-09-10T21:43:01.135 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:43:01.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.294+0000 7fc8fbe00640 1 -- 172.21.15.7:0/566631986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8f40cd750 msgr2=0x7fc8f4073730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:01.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.294+0000 7fc8fbe00640 1 --2- 172.21.15.7:0/566631986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8f40cd750 0x7fc8f4073730 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc8f406dac0 tx=0x7fc8e002a560 comp rx=0 tx=0).stop 2024-09-10T21:43:01.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.294+0000 7fc8fbe00640 1 -- 172.21.15.7:0/566631986 shutdown_connections 2024-09-10T21:43:01.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.294+0000 7fc8fbe00640 1 --2- 172.21.15.7:0/566631986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8f40cd750 0x7fc8f4073730 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:01.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.294+0000 7fc8fbe00640 1 --2- 172.21.15.7:0/566631986 >> v2:172.21.15.133:3300/0 conn(0x7fc8f415bc90 0x7fc8f40ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:01.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.294+0000 7fc8fbe00640 1 -- 172.21.15.7:0/566631986 >> 172.21.15.7:0/566631986 conn(0x7fc8f40b16d0 msgr2=0x7fc8f40b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:01.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.295+0000 7fc8fbe00640 1 -- 172.21.15.7:0/566631986 shutdown_connections 2024-09-10T21:43:01.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.295+0000 7fc8fbe00640 1 -- 172.21.15.7:0/566631986 wait complete. 2024-09-10T21:43:01.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.296+0000 7fc8fbe00640 1 Processor -- start 2024-09-10T21:43:01.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.296+0000 7fc8fbe00640 1 -- start start 2024-09-10T21:43:01.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.297+0000 7fc8fbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8f4073c70 0x7fc8f40ad960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:01.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.297+0000 7fc8fbe00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8f40cd750 0x7fc8f414a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:01.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.297+0000 7fc8fbe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8f415bc90 0x7fc8f414dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:01.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.297+0000 7fc8fbe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f406fcf0 con 0x7fc8f4073c70 2024-09-10T21:43:01.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.297+0000 7fc8fbe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f41606c0 con 0x7fc8f415bc90 2024-09-10T21:43:01.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.297+0000 7fc8fbe00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc8f406fe70 con 0x7fc8f40cd750 2024-09-10T21:43:01.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.297+0000 7fc8f9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8f4073c70 0x7fc8f40ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:01.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.297+0000 7fc8f9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8f4073c70 0x7fc8f40ad960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57566/0 (socket says 172.21.15.7:57566) 2024-09-10T21:43:01.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.297+0000 7fc8f9400640 1 -- 172.21.15.7:0/1651558920 learned_addr learned my addr 172.21.15.7:0/1651558920 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:01.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.297+0000 7fc8f9e00640 1 --2- 172.21.15.7:0/1651558920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8f415bc90 0x7fc8f414dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:01.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.298+0000 7fc8f8a00640 1 --2- 172.21.15.7:0/1651558920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8f40cd750 0x7fc8f414a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:01.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.298+0000 7fc8f9400640 1 -- 172.21.15.7:0/1651558920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8f40cd750 msgr2=0x7fc8f414a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:01.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.298+0000 7fc8f9400640 1 --2- 172.21.15.7:0/1651558920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8f40cd750 0x7fc8f414a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:01.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.298+0000 7fc8f9400640 1 -- 172.21.15.7:0/1651558920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8f415bc90 msgr2=0x7fc8f414dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:01.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.298+0000 7fc8f9400640 1 --2- 172.21.15.7:0/1651558920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8f415bc90 0x7fc8f414dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:01.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.298+0000 7fc8f9400640 1 -- 172.21.15.7:0/1651558920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f415ec70 con 0x7fc8f4073c70 2024-09-10T21:43:01.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.298+0000 7fc8f9e00640 1 --2- 172.21.15.7:0/1651558920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8f415bc90 0x7fc8f414dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:43:01.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.298+0000 7fc8f9400640 1 --2- 172.21.15.7:0/1651558920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8f4073c70 0x7fc8f40ad960 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800d770 tx=0x7fc8e800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:01.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.298+0000 7fc8f1600640 1 -- 172.21.15.7:0/1651558920 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8017070 con 0x7fc8f4073c70 2024-09-10T21:43:01.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.299+0000 7fc8f1600640 1 -- 172.21.15.7:0/1651558920 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8e8009d20 con 0x7fc8f4073c70 2024-09-10T21:43:01.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.299+0000 7fc8fbe00640 1 -- 172.21.15.7:0/1651558920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f414ad70 con 0x7fc8f4073c70 2024-09-10T21:43:01.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.299+0000 7fc8f1600640 1 -- 172.21.15.7:0/1651558920 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e80106c0 con 0x7fc8f4073c70 2024-09-10T21:43:01.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.299+0000 7fc8fbe00640 1 -- 172.21.15.7:0/1651558920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f40af530 con 0x7fc8f4073c70 2024-09-10T21:43:01.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.300+0000 7fc8fbe00640 1 -- 172.21.15.7:0/1651558920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8f40af960 con 0x7fc8f4073c70 2024-09-10T21:43:01.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.301+0000 7fc8f1600640 1 -- 172.21.15.7:0/1651558920 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc8e8010860 con 0x7fc8f4073c70 2024-09-10T21:43:01.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.302+0000 7fc8f1600640 1 --2- 172.21.15.7:0/1651558920 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc8d4077f70 0x7fc8d407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:01.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.302+0000 7fc8f1600640 1 -- 172.21.15.7:0/1651558920 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fc8e809a230 con 0x7fc8f4073c70 2024-09-10T21:43:01.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.302+0000 7fc8f8a00640 1 --2- 172.21.15.7:0/1651558920 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc8d4077f70 0x7fc8d407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:01.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.303+0000 7fc8f8a00640 1 --2- 172.21.15.7:0/1651558920 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc8d4077f70 0x7fc8d407a430 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0033000 tx=0x7fc8e0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:01.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.306+0000 7fc8f1600640 1 -- 172.21.15.7:0/1651558920 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc8e8062650 con 0x7fc8f4073c70 2024-09-10T21:43:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.540+0000 7fc8fbe00640 1 -- 172.21.15.7:0/1651558920 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc8f4073730 con 0x7fc8f4073c70 2024-09-10T21:43:01.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.540+0000 7fc8f1600640 1 -- 172.21.15.7:0/1651558920 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5221 (secure 0 0 0) 0x7fc8e8067080 con 0x7fc8f4073c70 2024-09-10T21:43:01.541 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:01.542 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"btime":"2024-09-10T21:42:58:925871+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/923954733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":923954733},{"type":"v1","addr":"172.21.15.133:6837","nonce":923954733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/13426364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":13426364},{"type":"v1","addr":"172.21.15.7:6835","nonce":13426364}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2275341781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2275341781},{"type":"v1","addr":"172.21.15.7:6834","nonce":2275341781}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:42:55.338104+0000","modified":"2024-09-10T21:42:58.362130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234524":{"gid":234524,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6836/849012248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":849012248},{"type":"v1","addr":"172.21.15.133:6836","nonce":849012248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234524,"qdb_cluster":[234524]},"id":2}]} 2024-09-10T21:43:01.542 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-09-10T21:43:01.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.542+0000 7fc8fbe00640 1 -- 172.21.15.7:0/1651558920 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc8d4077f70 msgr2=0x7fc8d407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:01.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.542+0000 7fc8fbe00640 1 --2- 172.21.15.7:0/1651558920 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc8d4077f70 0x7fc8d407a430 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0033000 tx=0x7fc8e0004730 comp rx=0 tx=0).stop 2024-09-10T21:43:01.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.542+0000 7fc8fbe00640 1 -- 172.21.15.7:0/1651558920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8f4073c70 msgr2=0x7fc8f40ad960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:01.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.542+0000 7fc8fbe00640 1 --2- 172.21.15.7:0/1651558920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8f4073c70 0x7fc8f40ad960 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800d770 tx=0x7fc8e800dc30 comp rx=0 tx=0).stop 2024-09-10T21:43:01.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.542+0000 7fc8fbe00640 1 -- 172.21.15.7:0/1651558920 shutdown_connections 2024-09-10T21:43:01.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.542+0000 7fc8fbe00640 1 --2- 172.21.15.7:0/1651558920 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc8d4077f70 0x7fc8d407a430 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:01.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.542+0000 7fc8fbe00640 1 --2- 172.21.15.7:0/1651558920 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8f415bc90 0x7fc8f414dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:01.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.542+0000 7fc8fbe00640 1 --2- 172.21.15.7:0/1651558920 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8f40cd750 0x7fc8f414a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:01.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.543+0000 7fc8fbe00640 1 --2- 172.21.15.7:0/1651558920 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8f4073c70 0x7fc8f40ad960 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:01.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.543+0000 7fc8fbe00640 1 -- 172.21.15.7:0/1651558920 >> 172.21.15.7:0/1651558920 conn(0x7fc8f40b16d0 msgr2=0x7fc8f415fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:01.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.543+0000 7fc8fbe00640 1 -- 172.21.15.7:0/1651558920 shutdown_connections 2024-09-10T21:43:01.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:01.543+0000 7fc8fbe00640 1 -- 172.21.15.7:0/1651558920 wait complete. 2024-09-10T21:43:01.964 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:42:55.338104+0000', 'modified': '2024-09-10T21:42:58.362130+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 234524}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_234524': {'gid': 234524, 'name': 'b', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6836/849012248', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 849012248}, {'type': 'v1', 'addr': '172.21.15.133:6836', 'nonce': 849012248}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 234524, 'qdb_cluster': [234524]} 2024-09-10T21:43:01.964 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:43:01.964 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T21:43:02.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.124+0000 7f8e58a00640 1 -- 172.21.15.7:0/1497875044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e540743d0 msgr2=0x7f8e540cc9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:02.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.124+0000 7f8e58a00640 1 --2- 172.21.15.7:0/1497875044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e540743d0 0x7f8e540cc9c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009830 tx=0x7f8e3c02a480 comp rx=0 tx=0).stop 2024-09-10T21:43:02.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.125+0000 7f8e58a00640 1 -- 172.21.15.7:0/1497875044 shutdown_connections 2024-09-10T21:43:02.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.125+0000 7f8e58a00640 1 --2- 172.21.15.7:0/1497875044 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e540ccf00 0x7f8e5415c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.125+0000 7f8e58a00640 1 --2- 172.21.15.7:0/1497875044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e540743d0 0x7f8e540cc9c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.125+0000 7f8e58a00640 1 -- 172.21.15.7:0/1497875044 >> 172.21.15.7:0/1497875044 conn(0x7f8e540b2750 msgr2=0x7f8e540b2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:02.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.125+0000 7f8e58a00640 1 -- 172.21.15.7:0/1497875044 shutdown_connections 2024-09-10T21:43:02.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.125+0000 7f8e58a00640 1 -- 172.21.15.7:0/1497875044 wait complete. 2024-09-10T21:43:02.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.126+0000 7f8e58a00640 1 Processor -- start 2024-09-10T21:43:02.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.127+0000 7f8e58a00640 1 -- start start 2024-09-10T21:43:02.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.127+0000 7f8e58a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e5406d3a0 0x7f8e5415aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.127+0000 7f8e58a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e540743d0 0x7f8e540ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.128+0000 7f8e58a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e540ccf00 0x7f8e5415a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.128+0000 7f8e58a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5415c890 con 0x7f8e5406d3a0 2024-09-10T21:43:02.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.128+0000 7f8e58a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8e5415c710 con 0x7f8e540ccf00 2024-09-10T21:43:02.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.128+0000 7f8e58a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f8e5415ca10 con 0x7f8e540743d0 2024-09-10T21:43:02.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.128+0000 7f8e51e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e5406d3a0 0x7f8e5415aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.128+0000 7f8e51e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e5406d3a0 0x7f8e5415aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57596/0 (socket says 172.21.15.7:57596) 2024-09-10T21:43:02.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.128+0000 7f8e51e00640 1 -- 172.21.15.7:0/3545592280 learned_addr learned my addr 172.21.15.7:0/3545592280 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:02.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.128+0000 7f8e49400640 1 --2- 172.21.15.7:0/3545592280 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e540743d0 0x7f8e540ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.128+0000 7f8e52800640 1 --2- 172.21.15.7:0/3545592280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e540ccf00 0x7f8e5415a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.128+0000 7f8e51e00640 1 -- 172.21.15.7:0/3545592280 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e540743d0 msgr2=0x7f8e540ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:02.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.128+0000 7f8e51e00640 1 --2- 172.21.15.7:0/3545592280 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e540743d0 0x7f8e540ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.128+0000 7f8e51e00640 1 -- 172.21.15.7:0/3545592280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e540ccf00 msgr2=0x7f8e5415a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:02.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.128+0000 7f8e51e00640 1 --2- 172.21.15.7:0/3545592280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e540ccf00 0x7f8e5415a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.128+0000 7f8e51e00640 1 -- 172.21.15.7:0/3545592280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e5415b710 con 0x7f8e5406d3a0 2024-09-10T21:43:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.129+0000 7f8e51e00640 1 --2- 172.21.15.7:0/3545592280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e5406d3a0 0x7f8e5415aec0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400c4c0 tx=0x7f8e4400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.129+0000 7f8e4aa00640 1 -- 172.21.15.7:0/3545592280 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44016020 con 0x7f8e5406d3a0 2024-09-10T21:43:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.129+0000 7f8e4aa00640 1 -- 172.21.15.7:0/3545592280 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e44012940 con 0x7f8e5406d3a0 2024-09-10T21:43:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.129+0000 7f8e4aa00640 1 -- 172.21.15.7:0/3545592280 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44012d90 con 0x7f8e5406d3a0 2024-09-10T21:43:02.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.129+0000 7f8e58a00640 1 -- 172.21.15.7:0/3545592280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e540cab00 con 0x7f8e5406d3a0 2024-09-10T21:43:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.129+0000 7f8e58a00640 1 -- 172.21.15.7:0/3545592280 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e540c48f0 con 0x7f8e5406d3a0 2024-09-10T21:43:02.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.131+0000 7f8e4aa00640 1 -- 172.21.15.7:0/3545592280 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8e4401a3b0 con 0x7f8e5406d3a0 2024-09-10T21:43:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.132+0000 7f8e4aa00640 1 --2- 172.21.15.7:0/3545592280 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8e30077fa0 0x7f8e3007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.132+0000 7f8e4aa00640 1 -- 172.21.15.7:0/3545592280 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f8e44099ba0 con 0x7f8e5406d3a0 2024-09-10T21:43:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.132+0000 7f8e49400640 1 --2- 172.21.15.7:0/3545592280 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8e30077fa0 0x7f8e3007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.133+0000 7f8e5b9cc740 1 Processor -- start 2024-09-10T21:43:02.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.133+0000 7f8e5b9cc740 1 -- start start 2024-09-10T21:43:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.133+0000 7f8e49400640 1 --2- 172.21.15.7:0/3545592280 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8e30077fa0 0x7f8e3007a460 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c034000 tx=0x7f8e3c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.134+0000 7f8e5b9cc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d748b0 0x55b595d74c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.134+0000 7f8e5b9cc740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d2fa20 0x55b595d73390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.134+0000 7f8e5b9cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d2ffa0 0x55b595d738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.134+0000 7f8e5b9cc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b595cb2430 con 0x55b595d748b0 2024-09-10T21:43:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.134+0000 7f8e5b9cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b595ca25d0 con 0x55b595d2ffa0 2024-09-10T21:43:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.134+0000 7f8e5b9cc740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55b595c91a00 con 0x55b595d2fa20 2024-09-10T21:43:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.134+0000 7f8e51e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d748b0 0x55b595d74c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.134+0000 7f8e51e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d748b0 0x55b595d74c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57610/0 (socket says 172.21.15.7:57610) 2024-09-10T21:43:02.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.134+0000 7f8e51e00640 1 -- 172.21.15.7:0/3699665440 learned_addr learned my addr 172.21.15.7:0/3699665440 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.134+0000 7f8e52800640 1 --2- 172.21.15.7:0/3699665440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d2ffa0 0x55b595d738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.134+0000 7f8e49400640 1 --2- 172.21.15.7:0/3699665440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d2fa20 0x55b595d73390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.134+0000 7f8e51e00640 1 -- 172.21.15.7:0/3699665440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d2fa20 msgr2=0x55b595d73390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.134+0000 7f8e51e00640 1 --2- 172.21.15.7:0/3699665440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d2fa20 0x55b595d73390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.134+0000 7f8e51e00640 1 -- 172.21.15.7:0/3699665440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d2ffa0 msgr2=0x55b595d738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.134+0000 7f8e51e00640 1 --2- 172.21.15.7:0/3699665440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d2ffa0 0x55b595d738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.134+0000 7f8e51e00640 1 -- 172.21.15.7:0/3699665440 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b595c5a520 con 0x55b595d748b0 2024-09-10T21:43:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.134+0000 7f8e49400640 1 --2- 172.21.15.7:0/3699665440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d2fa20 0x55b595d73390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.135+0000 7f8e51e00640 1 --2- 172.21.15.7:0/3699665440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d748b0 0x55b595d74c90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8e44012fd0 tx=0x7f8e44061b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1c6920600368f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:02.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.135+0000 7f8e42600640 1 -- 172.21.15.7:0/3699665440 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e4406b400 con 0x55b595d748b0 2024-09-10T21:43:02.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.135+0000 7f8e42600640 1 -- 172.21.15.7:0/3699665440 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e44067b70 con 0x55b595d748b0 2024-09-10T21:43:02.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.135+0000 7f8e42600640 1 -- 172.21.15.7:0/3699665440 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44061210 con 0x55b595d748b0 2024-09-10T21:43:02.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.135+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3699665440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d748b0 msgr2=0x55b595d74c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:02.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.135+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/3699665440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d748b0 0x55b595d74c90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8e44012fd0 tx=0x7f8e44061b80 comp rx=0 tx=0).stop 2024-09-10T21:43:02.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.135+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3699665440 shutdown_connections 2024-09-10T21:43:02.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.135+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/3699665440 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d2ffa0 0x55b595d738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.135+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/3699665440 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d2fa20 0x55b595d73390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.136+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/3699665440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d748b0 0x55b595d74c90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.136+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3699665440 >> 172.21.15.7:0/3699665440 conn(0x55b595d2aff0 msgr2=0x55b595d2ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:02.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.136+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3699665440 shutdown_connections 2024-09-10T21:43:02.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.136+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3699665440 wait complete. 2024-09-10T21:43:02.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.136+0000 7f8e5b9cc740 1 -- start start 2024-09-10T21:43:02.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.137+0000 7f8e5b9cc740 1 Processor -- start 2024-09-10T21:43:02.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.137+0000 7f8e5b9cc740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d2ffa0 0x55b595d468b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.138+0000 7f8e5b9cc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d748b0 0x55b595d69d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.138+0000 7f8e5b9cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d710b0 0x55b595d714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.138+0000 7f8e5b9cc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b595c38330 con 0x55b595d748b0 2024-09-10T21:43:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.138+0000 7f8e5b9cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b595c95810 con 0x55b595d710b0 2024-09-10T21:43:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.138+0000 7f8e5b9cc740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55b595a02620 con 0x55b595d2ffa0 2024-09-10T21:43:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.138+0000 7f8e5b9cc740 10 client.0 _ll_register_callbacks cb 0x55b595c3d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:43:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.138+0000 7f8e49400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d748b0 0x55b595d69d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.138+0000 7f8e49400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d748b0 0x55b595d69d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57612/0 (socket says 172.21.15.7:57612) 2024-09-10T21:43:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.138+0000 7f8e49400640 1 -- 172.21.15.7:0/3255138793 learned_addr learned my addr 172.21.15.7:0/3255138793 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.138+0000 7f8e51e00640 1 --2- 172.21.15.7:0/3255138793 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d2ffa0 0x55b595d468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.138+0000 7f8e52800640 1 --2- 172.21.15.7:0/3255138793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d710b0 0x55b595d714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.138+0000 7f8e49400640 1 -- 172.21.15.7:0/3255138793 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d2ffa0 msgr2=0x55b595d468b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.138+0000 7f8e49400640 1 --2- 172.21.15.7:0/3255138793 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d2ffa0 0x55b595d468b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.138+0000 7f8e49400640 1 -- 172.21.15.7:0/3255138793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d710b0 msgr2=0x55b595d714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:02.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.138+0000 7f8e49400640 1 --2- 172.21.15.7:0/3255138793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d710b0 0x55b595d714f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.138+0000 7f8e49400640 1 -- 172.21.15.7:0/3255138793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5959f3850 con 0x55b595d748b0 2024-09-10T21:43:02.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.139+0000 7f8e49400640 1 --2- 172.21.15.7:0/3255138793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d748b0 0x55b595d69d00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009950 tx=0x7f8e3c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf9e8d3c26228a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:02.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.139+0000 7f8e36a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:43:02.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.139+0000 7f8e36a00640 1 -- 172.21.15.7:0/3255138793 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c04a900 con 0x55b595d748b0 2024-09-10T21:43:02.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.139+0000 7f8e36a00640 1 -- 172.21.15.7:0/3255138793 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f8e3c054b20 con 0x55b595d748b0 2024-09-10T21:43:02.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.139+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3255138793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e44067b70 con 0x55b595d748b0 2024-09-10T21:43:02.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.139+0000 7f8e36a00640 1 -- 172.21.15.7:0/3255138793 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f8e3c054d20 con 0x55b595d748b0 2024-09-10T21:43:02.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.139+0000 7f8e5b9cc740 10 client.5061 fetch_fsmap learned FSMap version 17 2024-09-10T21:43:02.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.139+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3255138793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55b595a6a6b0 con 0x55b595d748b0 2024-09-10T21:43:02.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.139+0000 7f8e36a00640 1 -- 172.21.15.7:0/3255138793 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e44067b70 con 0x55b595d748b0 2024-09-10T21:43:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.139+0000 7f8e36a00640 1 -- 172.21.15.7:0/3255138793 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f8e3c04aaa0 con 0x55b595d748b0 2024-09-10T21:43:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.139+0000 7f8e5b9cc740 10 client.5061 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T21:43:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.139+0000 7f8e5b9cc740 10 client.5061 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:43:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.140+0000 7f8e5b9cc740 20 client.5061 populate_metadata read hostname 'smithi007' 2024-09-10T21:43:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.140+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/3255138793 >> [v2:172.21.15.133:6834/849012248,v1:172.21.15.133:6836/849012248] conn(0x55b595d57e90 0x55b595d58270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.140+0000 7f8e5b9cc740 4 client.5061 mds_command: new command op to 234524 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:43:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.140+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3255138793 --> [v2:172.21.15.133:6834/849012248,v1:172.21.15.133:6836/849012248] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b595ca4520 con 0x55b595d57e90 2024-09-10T21:43:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.140+0000 7f8e51e00640 1 --2- 172.21.15.7:0/3255138793 >> [v2:172.21.15.133:6834/849012248,v1:172.21.15.133:6836/849012248] conn(0x55b595d57e90 0x55b595d58270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.140+0000 7f8e51e00640 1 --2- 172.21.15.7:0/3255138793 >> [v2:172.21.15.133:6834/849012248,v1:172.21.15.133:6836/849012248] conn(0x55b595d57e90 0x55b595d58270 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=707b9bffc490d3a0 server_cookie=ece71298b9532eb8 in_seq=0 out_seq=0 2024-09-10T21:43:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.140+0000 7f8e36a00640 10 client.5061 ms_handle_connect on v2:172.21.15.133:6834/849012248 2024-09-10T21:43:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.141+0000 7f8e36a00640 1 -- 172.21.15.7:0/3255138793 <== mds.0 v2:172.21.15.133:6834/849012248 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b595ca4520 con 0x55b595d57e90 2024-09-10T21:43:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.141+0000 7f8e36a00640 10 client.5061 handle_command_reply: tid=0 2024-09-10T21:43:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.141+0000 7f8e5b9cc740 1 client.5061 shutdown 2024-09-10T21:43:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.142+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3255138793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d748b0 msgr2=0x55b595d69d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.142+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/3255138793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d748b0 0x55b595d69d00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009950 tx=0x7f8e3c005f90 comp rx=0 tx=0).stop 2024-09-10T21:43:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.142+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3255138793 shutdown_connections 2024-09-10T21:43:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.142+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/3255138793 >> [v2:172.21.15.133:6834/849012248,v1:172.21.15.133:6836/849012248] conn(0x55b595d57e90 0x55b595d58270 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.142+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/3255138793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d710b0 0x55b595d714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.142+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/3255138793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d748b0 0x55b595d69d00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.142+0000 7f8e36a00640 0 client.5061 ms_handle_reset on v2:172.21.15.133:6834/849012248 2024-09-10T21:43:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.142+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/3255138793 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d2ffa0 0x55b595d468b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.142+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3255138793 >> 172.21.15.7:0/3255138793 conn(0x55b595d2fa20 msgr2=0x55b595d6d3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.142+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3255138793 shutdown_connections 2024-09-10T21:43:02.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.142+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3255138793 wait complete. 2024-09-10T21:43:02.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.142+0000 7f8e5b9cc740 20 client.5061 trim_cache size 0 max 16384 2024-09-10T21:43:02.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.161+0000 7f8e5b9cc740 1 Processor -- start 2024-09-10T21:43:02.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.161+0000 7f8e5b9cc740 1 -- start start 2024-09-10T21:43:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.162+0000 7f8e5b9cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d2ffa0 0x55b595d698f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.162+0000 7f8e5b9cc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d710b0 0x55b595d96e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.162+0000 7f8e5b9cc740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d748b0 0x55b595d973d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.162+0000 7f8e5b9cc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8e54200e50 con 0x55b595d710b0 2024-09-10T21:43:02.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.162+0000 7f8e5b9cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8e540c6310 con 0x55b595d2ffa0 2024-09-10T21:43:02.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.162+0000 7f8e5b9cc740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f8e540c89d0 con 0x55b595d748b0 2024-09-10T21:43:02.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.162+0000 7f8e49400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d710b0 0x55b595d96e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.162+0000 7f8e49400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d710b0 0x55b595d96e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57628/0 (socket says 172.21.15.7:57628) 2024-09-10T21:43:02.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.162+0000 7f8e49400640 1 -- 172.21.15.7:0/4071289830 learned_addr learned my addr 172.21.15.7:0/4071289830 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:02.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.162+0000 7f8e51e00640 1 --2- 172.21.15.7:0/4071289830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d2ffa0 0x55b595d698f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.162+0000 7f8e52800640 1 --2- 172.21.15.7:0/4071289830 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d748b0 0x55b595d973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.163+0000 7f8e49400640 1 -- 172.21.15.7:0/4071289830 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d748b0 msgr2=0x55b595d973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:02.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.163+0000 7f8e49400640 1 --2- 172.21.15.7:0/4071289830 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d748b0 0x55b595d973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.163+0000 7f8e49400640 1 -- 172.21.15.7:0/4071289830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d2ffa0 msgr2=0x55b595d698f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:02.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.163+0000 7f8e49400640 1 --2- 172.21.15.7:0/4071289830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d2ffa0 0x55b595d698f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.163+0000 7f8e49400640 1 -- 172.21.15.7:0/4071289830 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b595d71860 con 0x55b595d710b0 2024-09-10T21:43:02.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.163+0000 7f8e51e00640 1 --2- 172.21.15.7:0/4071289830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d2ffa0 0x55b595d698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:43:02.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.163+0000 7f8e49400640 1 --2- 172.21.15.7:0/4071289830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d710b0 0x55b595d96e90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009830 tx=0x7f8e3c005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecd6132901b349ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:02.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.163+0000 7f8e41c00640 1 -- 172.21.15.7:0/4071289830 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c06c070 con 0x55b595d710b0 2024-09-10T21:43:02.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.163+0000 7f8e41c00640 1 -- 172.21.15.7:0/4071289830 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e3c04e070 con 0x55b595d710b0 2024-09-10T21:43:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.163+0000 7f8e41c00640 1 -- 172.21.15.7:0/4071289830 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c0541b0 con 0x55b595d710b0 2024-09-10T21:43:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.164+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/4071289830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d710b0 msgr2=0x55b595d96e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.164+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/4071289830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d710b0 0x55b595d96e90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009830 tx=0x7f8e3c005a90 comp rx=0 tx=0).stop 2024-09-10T21:43:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.164+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/4071289830 shutdown_connections 2024-09-10T21:43:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.164+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/4071289830 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d748b0 0x55b595d973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.164+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/4071289830 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d710b0 0x55b595d96e90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.164+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/4071289830 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d2ffa0 0x55b595d698f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.164+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/4071289830 >> 172.21.15.7:0/4071289830 conn(0x55b595d2fa20 msgr2=0x55b595d29b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:02.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.164+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/4071289830 shutdown_connections 2024-09-10T21:43:02.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.164+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/4071289830 wait complete. 2024-09-10T21:43:02.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.165+0000 7f8e5b9cc740 1 -- start start 2024-09-10T21:43:02.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.165+0000 7f8e5b9cc740 1 Processor -- start 2024-09-10T21:43:02.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.166+0000 7f8e5b9cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d2ffa0 0x55b595d826f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.166+0000 7f8e5b9cc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d710b0 0x55b595d96e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.166+0000 7f8e5b9cc740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d748b0 0x55b595df3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.166+0000 7f8e5b9cc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b595d73b40 con 0x55b595d710b0 2024-09-10T21:43:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.166+0000 7f8e5b9cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8e540c5cc0 con 0x55b595d2ffa0 2024-09-10T21:43:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.166+0000 7f8e5b9cc740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55b595d58a10 con 0x55b595d748b0 2024-09-10T21:43:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.166+0000 7f8e5b9cc740 10 client.0 _ll_register_callbacks cb 0x7f8e3c0035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:43:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.166+0000 7f8e49400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d710b0 0x55b595d96e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.166+0000 7f8e49400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d710b0 0x55b595d96e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57634/0 (socket says 172.21.15.7:57634) 2024-09-10T21:43:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.166+0000 7f8e49400640 1 -- 172.21.15.7:0/3377820965 learned_addr learned my addr 172.21.15.7:0/3377820965 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.166+0000 7f8e51e00640 1 --2- 172.21.15.7:0/3377820965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d2ffa0 0x55b595d826f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.166+0000 7f8e52800640 1 --2- 172.21.15.7:0/3377820965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d748b0 0x55b595df3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.166+0000 7f8e49400640 1 -- 172.21.15.7:0/3377820965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d748b0 msgr2=0x55b595df3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.166+0000 7f8e49400640 1 --2- 172.21.15.7:0/3377820965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d748b0 0x55b595df3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.166+0000 7f8e49400640 1 -- 172.21.15.7:0/3377820965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d2ffa0 msgr2=0x55b595d826f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.166+0000 7f8e49400640 1 --2- 172.21.15.7:0/3377820965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d2ffa0 0x55b595d826f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.166+0000 7f8e49400640 1 -- 172.21.15.7:0/3377820965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b595d2d4e0 con 0x55b595d710b0 2024-09-10T21:43:02.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.167+0000 7f8e49400640 1 --2- 172.21.15.7:0/3377820965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d710b0 0x55b595d96e50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c066000 tx=0x7f8e3c067680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5087185b68853626 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:02.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.167+0000 7f8e36a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:43:02.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.167+0000 7f8e36a00640 1 -- 172.21.15.7:0/3377820965 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c06c070 con 0x55b595d710b0 2024-09-10T21:43:02.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.167+0000 7f8e36a00640 1 -- 172.21.15.7:0/3377820965 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f8e3c073ac0 con 0x55b595d710b0 2024-09-10T21:43:02.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.167+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3377820965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e3c04e070 con 0x55b595d710b0 2024-09-10T21:43:02.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.167+0000 7f8e36a00640 1 -- 172.21.15.7:0/3377820965 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f8e3c073ce0 con 0x55b595d710b0 2024-09-10T21:43:02.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.167+0000 7f8e5b9cc740 10 client.5067 fetch_fsmap learned FSMap version 17 2024-09-10T21:43:02.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.167+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3377820965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55b595d97390 con 0x55b595d710b0 2024-09-10T21:43:02.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.168+0000 7f8e36a00640 1 -- 172.21.15.7:0/3377820965 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e3c04e070 con 0x55b595d710b0 2024-09-10T21:43:02.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.168+0000 7f8e36a00640 1 -- 172.21.15.7:0/3377820965 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7f8e3c07d020 con 0x55b595d710b0 2024-09-10T21:43:02.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.168+0000 7f8e5b9cc740 10 client.5067 fetch_fsmap finished waiting for FSMap version 17 2024-09-10T21:43:02.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.168+0000 7f8e5b9cc740 10 client.5067 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:43:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.168+0000 7f8e5b9cc740 20 client.5067 populate_metadata read hostname 'smithi007' 2024-09-10T21:43:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.168+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/3377820965 >> [v2:172.21.15.133:6834/849012248,v1:172.21.15.133:6836/849012248] conn(0x55b595d42410 0x55b595d427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.168+0000 7f8e5b9cc740 4 client.5067 mds_command: new command op to 234524 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:43:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.168+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3377820965 --> [v2:172.21.15.133:6834/849012248,v1:172.21.15.133:6836/849012248] -- command(tid 0: {"prefix": "status"}) -- 0x55b595d2e2e0 con 0x55b595d42410 2024-09-10T21:43:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.168+0000 7f8e51e00640 1 --2- 172.21.15.7:0/3377820965 >> [v2:172.21.15.133:6834/849012248,v1:172.21.15.133:6836/849012248] conn(0x55b595d42410 0x55b595d427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.169+0000 7f8e51e00640 1 --2- 172.21.15.7:0/3377820965 >> [v2:172.21.15.133:6834/849012248,v1:172.21.15.133:6836/849012248] conn(0x55b595d42410 0x55b595d427f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3be04a8a94fb6de6 server_cookie=d2543023e0fbbea4 in_seq=0 out_seq=0 2024-09-10T21:43:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.169+0000 7f8e36a00640 10 client.5067 ms_handle_connect on v2:172.21.15.133:6834/849012248 2024-09-10T21:43:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.169+0000 7f8e36a00640 1 -- 172.21.15.7:0/3377820965 <== mds.0 v2:172.21.15.133:6834/849012248 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55b595d2e2e0 con 0x55b595d42410 2024-09-10T21:43:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.169+0000 7f8e36a00640 10 client.5067 handle_command_reply: tid=0 2024-09-10T21:43:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.169+0000 7f8e5b9cc740 1 client.5067 shutdown 2024-09-10T21:43:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.169+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3377820965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d710b0 msgr2=0x55b595d96e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.169+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/3377820965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d710b0 0x55b595d96e50 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c066000 tx=0x7f8e3c067680 comp rx=0 tx=0).stop 2024-09-10T21:43:02.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.169+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3377820965 shutdown_connections 2024-09-10T21:43:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.169+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/3377820965 >> [v2:172.21.15.133:6834/849012248,v1:172.21.15.133:6836/849012248] conn(0x55b595d42410 0x55b595d427f0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.169+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/3377820965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b595d748b0 0x55b595df3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.169+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/3377820965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b595d710b0 0x55b595d96e50 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.169+0000 7f8e5b9cc740 1 --2- 172.21.15.7:0/3377820965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b595d2ffa0 0x55b595d826f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.169+0000 7f8e36a00640 0 client.5067 ms_handle_reset on v2:172.21.15.133:6834/849012248 2024-09-10T21:43:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.169+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3377820965 >> 172.21.15.7:0/3377820965 conn(0x55b595d2fa20 msgr2=0x55b595d727c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.169+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3377820965 shutdown_connections 2024-09-10T21:43:02.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.169+0000 7f8e5b9cc740 1 -- 172.21.15.7:0/3377820965 wait complete. 2024-09-10T21:43:02.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.169+0000 7f8e5b9cc740 20 client.5067 trim_cache size 0 max 16384 2024-09-10T21:43:02.181 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:43:02.181 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2029451a-3c26-48c7-b874-0597197278ac", 2024-09-10T21:43:02.181 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-10T21:43:02.181 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 234524, 2024-09-10T21:43:02.181 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T21:43:02.182 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T21:43:02.182 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T21:43:02.182 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.254174956, 2024-09-10T21:43:02.182 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 17, 2024-09-10T21:43:02.182 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 32, 2024-09-10T21:43:02.182 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 32, 2024-09-10T21:43:02.182 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 14.098734137999999 2024-09-10T21:43:02.182 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:43:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.170+0000 7f8e58a00640 1 -- 172.21.15.7:0/3545592280 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8e30077fa0 msgr2=0x7f8e3007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.170+0000 7f8e58a00640 1 --2- 172.21.15.7:0/3545592280 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8e30077fa0 0x7f8e3007a460 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c034000 tx=0x7f8e3c004730 comp rx=0 tx=0).stop 2024-09-10T21:43:02.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.170+0000 7f8e58a00640 1 -- 172.21.15.7:0/3545592280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e5406d3a0 msgr2=0x7f8e5415aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.170+0000 7f8e58a00640 1 --2- 172.21.15.7:0/3545592280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e5406d3a0 0x7f8e5415aec0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8e4400c4c0 tx=0x7f8e4400c980 comp rx=0 tx=0).stop 2024-09-10T21:43:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.170+0000 7f8e58a00640 1 -- 172.21.15.7:0/3545592280 shutdown_connections 2024-09-10T21:43:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.170+0000 7f8e58a00640 1 --2- 172.21.15.7:0/3545592280 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8e30077fa0 0x7f8e3007a460 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.170+0000 7f8e58a00640 1 --2- 172.21.15.7:0/3545592280 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8e540ccf00 0x7f8e5415a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.170+0000 7f8e58a00640 1 --2- 172.21.15.7:0/3545592280 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8e540743d0 0x7f8e540ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.170+0000 7f8e58a00640 1 --2- 172.21.15.7:0/3545592280 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8e5406d3a0 0x7f8e5415aec0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.170+0000 7f8e58a00640 1 -- 172.21.15.7:0/3545592280 >> 172.21.15.7:0/3545592280 conn(0x7f8e540b2750 msgr2=0x7f8e5415eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:02.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.170+0000 7f8e58a00640 1 -- 172.21.15.7:0/3545592280 shutdown_connections 2024-09-10T21:43:02.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.170+0000 7f8e58a00640 1 -- 172.21.15.7:0/3545592280 wait complete. 2024-09-10T21:43:02.187 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:43:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.346+0000 7f7f83200640 1 -- 172.21.15.7:0/1414367339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c0b2f80 msgr2=0x7f7f7c0ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.346+0000 7f7f83200640 1 --2- 172.21.15.7:0/1414367339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c0b2f80 0x7f7f7c0ce6f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7f78005fd0 tx=0x7f7f7802aa90 comp rx=0 tx=0).stop 2024-09-10T21:43:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.346+0000 7f7f83200640 1 -- 172.21.15.7:0/1414367339 shutdown_connections 2024-09-10T21:43:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.346+0000 7f7f83200640 1 --2- 172.21.15.7:0/1414367339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c0b2f80 0x7f7f7c0ce6f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.346+0000 7f7f83200640 1 --2- 172.21.15.7:0/1414367339 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f7c0b4c90 0x7f7f7c0b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.346+0000 7f7f83200640 1 -- 172.21.15.7:0/1414367339 >> 172.21.15.7:0/1414367339 conn(0x7f7f7c0b1d70 msgr2=0x7f7f7c0cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.346+0000 7f7f83200640 1 -- 172.21.15.7:0/1414367339 shutdown_connections 2024-09-10T21:43:02.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.346+0000 7f7f83200640 1 -- 172.21.15.7:0/1414367339 wait complete. 2024-09-10T21:43:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.348+0000 7f7f83200640 1 Processor -- start 2024-09-10T21:43:02.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.348+0000 7f7f83200640 1 -- start start 2024-09-10T21:43:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.349+0000 7f7f83200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f7c0b2f80 0x7f7f7c1f4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.349+0000 7f7f83200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c0b4c90 0x7f7f7c1431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.349+0000 7f7f83200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c145540 0x7f7f7c072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.349+0000 7f7f83200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c0cedb0 con 0x7f7f7c0b4c90 2024-09-10T21:43:02.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.349+0000 7f7f83200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7f7c0cec30 con 0x7f7f7c145540 2024-09-10T21:43:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.349+0000 7f7f83200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f7f7c0cef30 con 0x7f7f7c0b2f80 2024-09-10T21:43:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.349+0000 7f7f81400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c0b4c90 0x7f7f7c1431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.349+0000 7f7f81400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c0b4c90 0x7f7f7c1431f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57652/0 (socket says 172.21.15.7:57652) 2024-09-10T21:43:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.349+0000 7f7f81400640 1 -- 172.21.15.7:0/3335357683 learned_addr learned my addr 172.21.15.7:0/3335357683 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.349+0000 7f7f82800640 1 --2- 172.21.15.7:0/3335357683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c145540 0x7f7f7c072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.349+0000 7f7f81e00640 1 --2- 172.21.15.7:0/3335357683 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f7c0b2f80 0x7f7f7c1f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.350+0000 7f7f81400640 1 -- 172.21.15.7:0/3335357683 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f7c0b2f80 msgr2=0x7f7f7c1f4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.350+0000 7f7f81400640 1 --2- 172.21.15.7:0/3335357683 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f7c0b2f80 0x7f7f7c1f4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.350+0000 7f7f81400640 1 -- 172.21.15.7:0/3335357683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c145540 msgr2=0x7f7f7c072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:02.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.350+0000 7f7f81400640 1 --2- 172.21.15.7:0/3335357683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c145540 0x7f7f7c072cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.350+0000 7f7f81400640 1 -- 172.21.15.7:0/3335357683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f7c1f4dc0 con 0x7f7f7c0b4c90 2024-09-10T21:43:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.350+0000 7f7f82800640 1 --2- 172.21.15.7:0/3335357683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c145540 0x7f7f7c072cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:43:02.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.350+0000 7f7f81400640 1 --2- 172.21.15.7:0/3335357683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c0b4c90 0x7f7f7c1431f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00e9f0 tx=0x7f7f6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.350+0000 7f7f76000640 1 -- 172.21.15.7:0/3335357683 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c00cd80 con 0x7f7f7c0b4c90 2024-09-10T21:43:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.350+0000 7f7f76000640 1 -- 172.21.15.7:0/3335357683 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f6c010430 con 0x7f7f7c0b4c90 2024-09-10T21:43:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.350+0000 7f7f76000640 1 -- 172.21.15.7:0/3335357683 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c010840 con 0x7f7f7c0b4c90 2024-09-10T21:43:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.350+0000 7f7f83200640 1 -- 172.21.15.7:0/3335357683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f7c143730 con 0x7f7f7c0b4c90 2024-09-10T21:43:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.351+0000 7f7f83200640 1 -- 172.21.15.7:0/3335357683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f7c1f5320 con 0x7f7f7c0b4c90 2024-09-10T21:43:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.351+0000 7f7f83200640 1 -- 172.21.15.7:0/3335357683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f7c15f0a0 con 0x7f7f7c0b4c90 2024-09-10T21:43:02.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.353+0000 7f7f76000640 1 -- 172.21.15.7:0/3335357683 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7f6c0109e0 con 0x7f7f7c0b4c90 2024-09-10T21:43:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.354+0000 7f7f76000640 1 --2- 172.21.15.7:0/3335357683 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f60077f70 0x7f7f6007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.354+0000 7f7f76000640 1 -- 172.21.15.7:0/3335357683 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f7f6c09a8b0 con 0x7f7f7c0b4c90 2024-09-10T21:43:02.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.354+0000 7f7f81e00640 1 --2- 172.21.15.7:0/3335357683 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f60077f70 0x7f7f6007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:02.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.355+0000 7f7f81e00640 1 --2- 172.21.15.7:0/3335357683 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f60077f70 0x7f7f6007a430 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7f70007fd0 tx=0x7f7f70009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:02.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.358+0000 7f7f76000640 1 -- 172.21.15.7:0/3335357683 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7f7c15f0a0 con 0x7f7f7c0b4c90 2024-09-10T21:43:02.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.591+0000 7f7f83200640 1 -- 172.21.15.7:0/3335357683 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f7f7c0ce6f0 con 0x7f7f7c0b4c90 2024-09-10T21:43:02.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.591+0000 7f7f76000640 1 -- 172.21.15.7:0/3335357683 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f7f7c1f4dc0 con 0x7f7f7c0b4c90 2024-09-10T21:43:02.593 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:43:02.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.594+0000 7f7f83200640 1 -- 172.21.15.7:0/3335357683 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f60077f70 msgr2=0x7f7f6007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:02.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.594+0000 7f7f83200640 1 --2- 172.21.15.7:0/3335357683 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f60077f70 0x7f7f6007a430 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7f70007fd0 tx=0x7f7f70009040 comp rx=0 tx=0).stop 2024-09-10T21:43:02.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.594+0000 7f7f83200640 1 -- 172.21.15.7:0/3335357683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c0b4c90 msgr2=0x7f7f7c1431f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:02.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.594+0000 7f7f83200640 1 --2- 172.21.15.7:0/3335357683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c0b4c90 0x7f7f7c1431f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c00e9f0 tx=0x7f7f6c00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.594+0000 7f7f83200640 1 -- 172.21.15.7:0/3335357683 shutdown_connections 2024-09-10T21:43:02.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.594+0000 7f7f83200640 1 --2- 172.21.15.7:0/3335357683 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f60077f70 0x7f7f6007a430 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.594+0000 7f7f83200640 1 --2- 172.21.15.7:0/3335357683 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f7c145540 0x7f7f7c072cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.594+0000 7f7f83200640 1 --2- 172.21.15.7:0/3335357683 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f7c0b4c90 0x7f7f7c1431f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.594+0000 7f7f83200640 1 --2- 172.21.15.7:0/3335357683 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f7c0b2f80 0x7f7f7c1f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:02.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.594+0000 7f7f83200640 1 -- 172.21.15.7:0/3335357683 >> 172.21.15.7:0/3335357683 conn(0x7f7f7c0b1d70 msgr2=0x7f7f7c15c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:02.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.594+0000 7f7f83200640 1 -- 172.21.15.7:0/3335357683 shutdown_connections 2024-09-10T21:43:02.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02.594+0000 7f7f83200640 1 -- 172.21.15.7:0/3335357683 wait complete. 2024-09-10T21:43:02.611 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:43:02.611 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:43:02.611 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-10T21:43:02.611 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:43:02.612 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:43:02.612 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:43:02.612 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:43:02.612 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:43:02.612 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:43:02.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:02.612 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-10T21:43:02.648 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:43:02.648 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:43:02.648 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:43:02.648 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:43:02.648 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T21:43:02.648 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:43:02.648 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:43:02.648 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T21:43:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-10T21:43:02.649 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:43:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T21:43:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-10T21:43:02.649 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:43:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-10T21:43:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-10T21:43:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T21:43:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42562sec preferred_lft 42562sec 2024-09-10T21:43:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-10T21:43:02.649 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:43:02.650 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:43:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-10T21:43:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-10T21:43:02.650 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:43:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff 2024-09-10T21:43:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:43:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:43:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::3c3d:dbff:fe02:4c15/64 scope link 2024-09-10T21:43:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:43:02.650 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:43:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether f6:f9:69:5a:86:f5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:43:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f4f9:69ff:fe5a:86f5/64 scope link 2024-09-10T21:43:02.650 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:43:02.650 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:43:02.651 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:43:02.651 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::433:5cff:fe44:4a5b/64 scope link 2024-09-10T21:43:02.651 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:43:02.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:02.651 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:43:02.741 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:43:02.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:02.745 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-10T21:43:02.780 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-10T21:43:02.781 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:43:02.781 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T21:43:02.781 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T21:43:02.781 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:43:02.782 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:43:02.782 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:43:02.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:43:02.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:43:02.782 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:43:02.782 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:43:02.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:43:02.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:43:02.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:03.048 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:43:03.084 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:43:03.084 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:43:03.084 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:43:03.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:03.084 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-10T21:43:03.180 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:43:03.181 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:43:03.216 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:43:03.216 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:43:03.216 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:43:03.290 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:43:03.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:03.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:43:03.419 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:43:03.419 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:43:03.447 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:43:03.448 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:43:03.448 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.133:3300,172.21.15.133:3301" 2024-09-10T21:43:03.448 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@2029451a-3c26-48c7-b874-0597197278ac.cephfs=/ 2024-09-10T21:43:03.448 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.133:3300/172.21.15.133:3301" will pass to kernel 2024-09-10T21:43:03.450 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T21:43:03.450 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:43:03.450 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:43:03.450 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:43:03.450 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:43:03.453 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:43:03.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:03.453 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:43:03.510 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:43:03.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:43:03.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:43:03.609 INFO:teuthology.orchestra.run.smithi007.stdout:client5076 2024-09-10T21:43:03.609 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:43:03.609 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:43:03.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:03.645 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2024-09-10T21:43:03.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:03.729 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2024-09-10T21:43:03.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:03.807 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2024-09-10T21:43:03.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:03.885 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2024-09-10T21:43:03.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:03.968 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2024-09-10T21:43:04.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:04.046 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-09-10T21:43:04.129 INFO:teuthology.orchestra.run.smithi007.stdout:./ 2024-09-10T21:43:04.129 INFO:teuthology.orchestra.run.smithi007.stdout:./rfile 2024-09-10T21:43:04.129 INFO:teuthology.orchestra.run.smithi007.stdout:./parentdir 2024-09-10T21:43:04.129 INFO:teuthology.orchestra.run.smithi007.stdout:./parentdir/pfile 2024-09-10T21:43:04.129 INFO:teuthology.orchestra.run.smithi007.stdout:./parentdir/childdir 2024-09-10T21:43:04.129 INFO:teuthology.orchestra.run.smithi007.stdout:./parentdir/childdir/cfile 2024-09-10T21:43:04.130 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:43:04.131 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:43:04.131 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:43:04.131 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:43:04.131 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-09-10T21:43:04.131 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:43:04.270 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:43:04.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:43:04.282 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:43:04.282 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:43:04.282 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:43:04.282 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:43:04.282 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2024-09-10T21:43:04.282 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:43:04.422 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627778 2024-09-10T21:43:04.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:43:04.433 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:43:04.434 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:43:04.434 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:43:04.434 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:43:04.434 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2024-09-10T21:43:04.434 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:43:04.573 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627776 2024-09-10T21:43:04.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:43:04.584 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:43:04.585 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:43:04.585 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:43:04.585 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:43:04.585 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2024-09-10T21:43:04.585 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:43:04.725 INFO:teuthology.orchestra.run.smithi007.stdout:1099511628279 2024-09-10T21:43:04.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:43:04.736 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:43:04.737 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:43:04.737 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:43:04.737 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:43:04.737 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2024-09-10T21:43:04.737 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:43:04.876 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627777 2024-09-10T21:43:04.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:43:04.888 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:43:04.888 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:43:04.888 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:43:04.888 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:43:04.888 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2024-09-10T21:43:04.888 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:43:05.029 INFO:teuthology.orchestra.run.smithi007.stdout:1099511628280 2024-09-10T21:43:05.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:43:05.041 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:43:05.041 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:43:05.116 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:43:05.117 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:43:05.192 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:43:05.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.393+0000 7f5d9d800640 1 -- 172.21.15.7:0/3911700315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d98073cf0 msgr2=0x7f5d98074130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:05.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.393+0000 7f5d9d800640 1 --2- 172.21.15.7:0/3911700315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d98073cf0 0x7f5d98074130 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f5d980705a0 tx=0x7f5d8002a530 comp rx=0 tx=0).stop 2024-09-10T21:43:05.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.393+0000 7f5d9d800640 1 -- 172.21.15.7:0/3911700315 shutdown_connections 2024-09-10T21:43:05.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.393+0000 7f5d9d800640 1 --2- 172.21.15.7:0/3911700315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d98073cf0 0x7f5d98074130 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:05.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.393+0000 7f5d9d800640 1 --2- 172.21.15.7:0/3911700315 >> v2:172.21.15.133:3300/0 conn(0x7f5d980cc940 0x7f5d9806fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:05.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.393+0000 7f5d9d800640 1 -- 172.21.15.7:0/3911700315 >> 172.21.15.7:0/3911700315 conn(0x7f5d980b1660 msgr2=0x7f5d980b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:05.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.394+0000 7f5d9d800640 1 -- 172.21.15.7:0/3911700315 shutdown_connections 2024-09-10T21:43:05.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.394+0000 7f5d9d800640 1 -- 172.21.15.7:0/3911700315 wait complete. 2024-09-10T21:43:05.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.395+0000 7f5d9d800640 1 Processor -- start 2024-09-10T21:43:05.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.395+0000 7f5d9d800640 1 -- start start 2024-09-10T21:43:05.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.396+0000 7f5d9d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d98073cf0 0x7f5d981488c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:05.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.396+0000 7f5d9d800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d980cc940 0x7f5d9815c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:05.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.396+0000 7f5d9d800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5d9815e7b0 0x7f5d980c5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:05.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.396+0000 7f5d9d800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9815f490 con 0x7f5d98073cf0 2024-09-10T21:43:05.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.396+0000 7f5d9d800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5d9815f310 con 0x7f5d980cc940 2024-09-10T21:43:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.396+0000 7f5d9d800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f5d980ccd20 con 0x7f5d9815e7b0 2024-09-10T21:43:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.396+0000 7f5d96a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d98073cf0 0x7f5d981488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.396+0000 7f5d96a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d98073cf0 0x7f5d981488c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40950/0 (socket says 172.21.15.7:40950) 2024-09-10T21:43:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.396+0000 7f5d96a00640 1 -- 172.21.15.7:0/3916079631 learned_addr learned my addr 172.21.15.7:0/3916079631 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.397+0000 7f5d8e000640 1 --2- 172.21.15.7:0/3916079631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d980cc940 0x7f5d9815c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.397+0000 7f5d97400640 1 --2- 172.21.15.7:0/3916079631 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5d9815e7b0 0x7f5d980c5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.397+0000 7f5d96a00640 1 -- 172.21.15.7:0/3916079631 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5d9815e7b0 msgr2=0x7f5d980c5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.397+0000 7f5d96a00640 1 --2- 172.21.15.7:0/3916079631 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5d9815e7b0 0x7f5d980c5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.397+0000 7f5d96a00640 1 -- 172.21.15.7:0/3916079631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d980cc940 msgr2=0x7f5d9815c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.397+0000 7f5d96a00640 1 --2- 172.21.15.7:0/3916079631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d980cc940 0x7f5d9815c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.397+0000 7f5d96a00640 1 -- 172.21.15.7:0/3916079631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d980c6470 con 0x7f5d98073cf0 2024-09-10T21:43:05.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.397+0000 7f5d97400640 1 --2- 172.21.15.7:0/3916079631 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5d9815e7b0 0x7f5d980c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:05.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.397+0000 7f5d96a00640 1 --2- 172.21.15.7:0/3916079631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d98073cf0 0x7f5d981488c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800c920 tx=0x7f5d8800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:05.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.397+0000 7f5d8f400640 1 -- 172.21.15.7:0/3916079631 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d88007c70 con 0x7f5d98073cf0 2024-09-10T21:43:05.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.398+0000 7f5d8f400640 1 -- 172.21.15.7:0/3916079631 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d8800f450 con 0x7f5d98073cf0 2024-09-10T21:43:05.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.398+0000 7f5d8f400640 1 -- 172.21.15.7:0/3916079631 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d8800f8a0 con 0x7f5d98073cf0 2024-09-10T21:43:05.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.398+0000 7f5d9d800640 1 -- 172.21.15.7:0/3916079631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d9815c970 con 0x7f5d98073cf0 2024-09-10T21:43:05.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.398+0000 7f5d9d800640 1 -- 172.21.15.7:0/3916079631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d9815cf30 con 0x7f5d98073cf0 2024-09-10T21:43:05.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.399+0000 7f5d9d800640 1 -- 172.21.15.7:0/3916079631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d98074130 con 0x7f5d98073cf0 2024-09-10T21:43:05.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.400+0000 7f5d8f400640 1 -- 172.21.15.7:0/3916079631 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5d88020030 con 0x7f5d98073cf0 2024-09-10T21:43:05.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.401+0000 7f5d8f400640 1 --2- 172.21.15.7:0/3916079631 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5d74077f70 0x7f5d7407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:05.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.401+0000 7f5d8f400640 1 -- 172.21.15.7:0/3916079631 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f5d8809a2e0 con 0x7f5d98073cf0 2024-09-10T21:43:05.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.401+0000 7f5d8e000640 1 --2- 172.21.15.7:0/3916079631 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5d74077f70 0x7f5d7407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:05.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.402+0000 7f5d8e000640 1 --2- 172.21.15.7:0/3916079631 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5d74077f70 0x7f5d7407a430 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5d80033000 tx=0x7f5d80004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:05.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.405+0000 7f5d8f400640 1 -- 172.21.15.7:0/3916079631 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5d88014030 con 0x7f5d98073cf0 2024-09-10T21:43:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.613+0000 7f5d9d800640 1 -- 172.21.15.7:0/3916079631 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5d980aec10 con 0x7f5d98073cf0 2024-09-10T21:43:05.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.613+0000 7f5d8f400640 1 -- 172.21.15.7:0/3916079631 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v32) ==== 80+0+0 (secure 0 0 0) 0x7f5d88062d40 con 0x7f5d98073cf0 2024-09-10T21:43:05.615 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-10T21:43:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.616+0000 7f5d9d800640 1 -- 172.21.15.7:0/3916079631 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5d74077f70 msgr2=0x7f5d7407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.616+0000 7f5d9d800640 1 --2- 172.21.15.7:0/3916079631 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5d74077f70 0x7f5d7407a430 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f5d80033000 tx=0x7f5d80004730 comp rx=0 tx=0).stop 2024-09-10T21:43:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.616+0000 7f5d9d800640 1 -- 172.21.15.7:0/3916079631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d98073cf0 msgr2=0x7f5d981488c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.616+0000 7f5d9d800640 1 --2- 172.21.15.7:0/3916079631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d98073cf0 0x7f5d981488c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5d8800c920 tx=0x7f5d8800cde0 comp rx=0 tx=0).stop 2024-09-10T21:43:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.616+0000 7f5d9d800640 1 -- 172.21.15.7:0/3916079631 shutdown_connections 2024-09-10T21:43:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.616+0000 7f5d9d800640 1 --2- 172.21.15.7:0/3916079631 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5d74077f70 0x7f5d7407a430 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.616+0000 7f5d9d800640 1 --2- 172.21.15.7:0/3916079631 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5d980cc940 0x7f5d9815c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.616+0000 7f5d9d800640 1 --2- 172.21.15.7:0/3916079631 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5d9815e7b0 0x7f5d980c5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.616+0000 7f5d9d800640 1 --2- 172.21.15.7:0/3916079631 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5d98073cf0 0x7f5d981488c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.616+0000 7f5d9d800640 1 -- 172.21.15.7:0/3916079631 >> 172.21.15.7:0/3916079631 conn(0x7f5d980b1660 msgr2=0x7f5d980b3ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.616+0000 7f5d9d800640 1 -- 172.21.15.7:0/3916079631 shutdown_connections 2024-09-10T21:43:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.616+0000 7f5d9d800640 1 -- 172.21.15.7:0/3916079631 wait complete. 2024-09-10T21:43:05.633 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:43:05.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:05.633 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:43:05.734 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:43:05.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:05.734 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:43:05.777 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:43:05.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.976+0000 7f88d1a00640 1 -- 172.21.15.7:0/3135470753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc0b2f80 msgr2=0x7f88cc0ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:05.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.977+0000 7f88d1a00640 1 --2- 172.21.15.7:0/3135470753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc0b2f80 0x7f88cc0ce6f0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f88cc0cdc00 tx=0x7f88c402a8b0 comp rx=0 tx=0).stop 2024-09-10T21:43:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.977+0000 7f88d1a00640 1 -- 172.21.15.7:0/3135470753 shutdown_connections 2024-09-10T21:43:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.977+0000 7f88d1a00640 1 --2- 172.21.15.7:0/3135470753 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc0b2f80 0x7f88cc0ce6f0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.977+0000 7f88d1a00640 1 --2- 172.21.15.7:0/3135470753 >> v2:172.21.15.133:3300/0 conn(0x7f88cc0b4c90 0x7f88cc0b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.977+0000 7f88d1a00640 1 -- 172.21.15.7:0/3135470753 >> 172.21.15.7:0/3135470753 conn(0x7f88cc0b1d70 msgr2=0x7f88cc0cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.977+0000 7f88d1a00640 1 -- 172.21.15.7:0/3135470753 shutdown_connections 2024-09-10T21:43:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.977+0000 7f88d1a00640 1 -- 172.21.15.7:0/3135470753 wait complete. 2024-09-10T21:43:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.978+0000 7f88d1a00640 1 Processor -- start 2024-09-10T21:43:05.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.979+0000 7f88d1a00640 1 -- start start 2024-09-10T21:43:05.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.979+0000 7f88d1a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc0b2f80 0x7f88cc0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:05.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.979+0000 7f88d1a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88cc0b4c90 0x7f88cc1f77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:05.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.979+0000 7f88d1a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88cc145540 0x7f88cc15fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:05.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.979+0000 7f88d1a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc0cedb0 con 0x7f88cc0b2f80 2024-09-10T21:43:05.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.979+0000 7f88d1a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f88cc0cec30 con 0x7f88cc0b4c90 2024-09-10T21:43:05.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.979+0000 7f88d1a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f88cc0cef30 con 0x7f88cc145540 2024-09-10T21:43:05.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.980+0000 7f88cbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc0b2f80 0x7f88cc0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.980+0000 7f88cbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc0b2f80 0x7f88cc0cc1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40970/0 (socket says 172.21.15.7:40970) 2024-09-10T21:43:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.980+0000 7f88cbe00640 1 -- 172.21.15.7:0/3143880183 learned_addr learned my addr 172.21.15.7:0/3143880183 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.980+0000 7f88c3400640 1 --2- 172.21.15.7:0/3143880183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88cc0b4c90 0x7f88cc1f77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.980+0000 7f88d1000640 1 --2- 172.21.15.7:0/3143880183 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88cc145540 0x7f88cc15fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.980+0000 7f88cbe00640 1 -- 172.21.15.7:0/3143880183 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88cc145540 msgr2=0x7f88cc15fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.980+0000 7f88cbe00640 1 --2- 172.21.15.7:0/3143880183 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88cc145540 0x7f88cc15fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.980+0000 7f88cbe00640 1 -- 172.21.15.7:0/3143880183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88cc0b4c90 msgr2=0x7f88cc1f77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.980+0000 7f88cbe00640 1 --2- 172.21.15.7:0/3143880183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88cc0b4c90 0x7f88cc1f77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.980+0000 7f88cbe00640 1 -- 172.21.15.7:0/3143880183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88cc0cc6e0 con 0x7f88cc0b2f80 2024-09-10T21:43:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.980+0000 7f88c3400640 1 --2- 172.21.15.7:0/3143880183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88cc0b4c90 0x7f88cc1f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:43:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.981+0000 7f88cbe00640 1 --2- 172.21.15.7:0/3143880183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc0b2f80 0x7f88cc0cc1a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00c3a0 tx=0x7f88bc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.981+0000 7f88c8c00640 1 -- 172.21.15.7:0/3143880183 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc011790 con 0x7f88cc0b2f80 2024-09-10T21:43:05.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.981+0000 7f88c8c00640 1 -- 172.21.15.7:0/3143880183 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88bc017070 con 0x7f88cc0b2f80 2024-09-10T21:43:05.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.981+0000 7f88c8c00640 1 -- 172.21.15.7:0/3143880183 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88bc011cc0 con 0x7f88cc0b2f80 2024-09-10T21:43:05.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.981+0000 7f88d1a00640 1 -- 172.21.15.7:0/3143880183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88cc139e90 con 0x7f88cc0b2f80 2024-09-10T21:43:05.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.981+0000 7f88d1a00640 1 -- 172.21.15.7:0/3143880183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88cc1f8550 con 0x7f88cc0b2f80 2024-09-10T21:43:05.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.982+0000 7f88d1a00640 1 -- 172.21.15.7:0/3143880183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88cc1f8750 con 0x7f88cc0b2f80 2024-09-10T21:43:05.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.983+0000 7f88c8c00640 1 -- 172.21.15.7:0/3143880183 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f88bc01c460 con 0x7f88cc0b2f80 2024-09-10T21:43:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.984+0000 7f88c8c00640 1 --2- 172.21.15.7:0/3143880183 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f88a8077f70 0x7f88a807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.985+0000 7f88c8c00640 1 -- 172.21.15.7:0/3143880183 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f88bc0999b0 con 0x7f88cc0b2f80 2024-09-10T21:43:05.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.985+0000 7f88c3400640 1 --2- 172.21.15.7:0/3143880183 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f88a8077f70 0x7f88a807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:05.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.986+0000 7f88c3400640 1 --2- 172.21.15.7:0/3143880183 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f88a8077f70 0x7f88a807a430 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f88cc13b3b0 tx=0x7f88b400b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:05.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:05.988+0000 7f88c8c00640 1 -- 172.21.15.7:0/3143880183 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f88bc061e50 con 0x7f88cc0b2f80 2024-09-10T21:43:06.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.218+0000 7f88d1a00640 1 -- 172.21.15.7:0/3143880183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88cc0ce6f0 con 0x7f88cc0b2f80 2024-09-10T21:43:06.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.219+0000 7f88c8c00640 1 -- 172.21.15.7:0/3143880183 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5221 (secure 0 0 0) 0x7f88bc066880 con 0x7f88cc0b2f80 2024-09-10T21:43:06.220 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:06.220 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"btime":"2024-09-10T21:42:58:925871+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/923954733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":923954733},{"type":"v1","addr":"172.21.15.133:6837","nonce":923954733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/13426364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":13426364},{"type":"v1","addr":"172.21.15.7:6835","nonce":13426364}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2275341781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2275341781},{"type":"v1","addr":"172.21.15.7:6834","nonce":2275341781}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:42:55.338104+0000","modified":"2024-09-10T21:42:58.362130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234524":{"gid":234524,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6836/849012248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":849012248},{"type":"v1","addr":"172.21.15.133:6836","nonce":849012248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234524,"qdb_cluster":[234524]},"id":2}]} 2024-09-10T21:43:06.221 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-09-10T21:43:06.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.221+0000 7f88d1a00640 1 -- 172.21.15.7:0/3143880183 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f88a8077f70 msgr2=0x7f88a807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:06.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.221+0000 7f88d1a00640 1 --2- 172.21.15.7:0/3143880183 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f88a8077f70 0x7f88a807a430 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f88cc13b3b0 tx=0x7f88b400b390 comp rx=0 tx=0).stop 2024-09-10T21:43:06.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.221+0000 7f88d1a00640 1 -- 172.21.15.7:0/3143880183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc0b2f80 msgr2=0x7f88cc0cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:06.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.221+0000 7f88d1a00640 1 --2- 172.21.15.7:0/3143880183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc0b2f80 0x7f88cc0cc1a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f88bc00c3a0 tx=0x7f88bc00c860 comp rx=0 tx=0).stop 2024-09-10T21:43:06.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.222+0000 7f88d1a00640 1 -- 172.21.15.7:0/3143880183 shutdown_connections 2024-09-10T21:43:06.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.222+0000 7f88d1a00640 1 --2- 172.21.15.7:0/3143880183 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f88a8077f70 0x7f88a807a430 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:06.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.222+0000 7f88d1a00640 1 --2- 172.21.15.7:0/3143880183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88cc0b4c90 0x7f88cc1f77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:06.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.222+0000 7f88d1a00640 1 --2- 172.21.15.7:0/3143880183 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88cc145540 0x7f88cc15fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:06.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.222+0000 7f88d1a00640 1 --2- 172.21.15.7:0/3143880183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88cc0b2f80 0x7f88cc0cc1a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:06.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.222+0000 7f88d1a00640 1 -- 172.21.15.7:0/3143880183 >> 172.21.15.7:0/3143880183 conn(0x7f88cc0b1d70 msgr2=0x7f88cc0b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:06.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.222+0000 7f88d1a00640 1 -- 172.21.15.7:0/3143880183 shutdown_connections 2024-09-10T21:43:06.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.222+0000 7f88d1a00640 1 -- 172.21.15.7:0/3143880183 wait complete. 2024-09-10T21:43:06.239 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-09-10T21:43:06.431 INFO:teuthology.orchestra.run.smithi133.stdout:{"message":"","return_code":0} 2024-09-10T21:43:06.431 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-10T21:43:06.431 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:43:06.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.591+0000 7f5c3d600640 1 -- 172.21.15.7:0/1157300803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c38146c50 msgr2=0x7f5c380723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.592+0000 7f5c3d600640 1 --2- 172.21.15.7:0/1157300803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c38146c50 0x7f5c380723b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f5c28005fd0 tx=0x7f5c2802a540 comp rx=0 tx=0).stop 2024-09-10T21:43:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.592+0000 7f5c3d600640 1 -- 172.21.15.7:0/1157300803 shutdown_connections 2024-09-10T21:43:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.592+0000 7f5c3d600640 1 --2- 172.21.15.7:0/1157300803 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c381470b0 0x7f5c380ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.592+0000 7f5c3d600640 1 --2- 172.21.15.7:0/1157300803 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c38146c50 0x7f5c380723b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.592+0000 7f5c3d600640 1 -- 172.21.15.7:0/1157300803 >> 172.21.15.7:0/1157300803 conn(0x7f5c380b1d70 msgr2=0x7f5c380cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.592+0000 7f5c3d600640 1 -- 172.21.15.7:0/1157300803 shutdown_connections 2024-09-10T21:43:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.592+0000 7f5c3d600640 1 -- 172.21.15.7:0/1157300803 wait complete. 2024-09-10T21:43:06.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.593+0000 7f5c3d600640 1 Processor -- start 2024-09-10T21:43:06.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.593+0000 7f5c3d600640 1 -- start start 2024-09-10T21:43:06.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.594+0000 7f5c3d600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c3806f590 0x7f5c381f9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:06.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.594+0000 7f5c3d600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c38146c50 0x7f5c380b4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:06.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.594+0000 7f5c3d600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c381470b0 0x7f5c3813de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:06.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.594+0000 7f5c3d600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c380adfa0 con 0x7f5c3806f590 2024-09-10T21:43:06.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.594+0000 7f5c3d600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5c380ade20 con 0x7f5c381470b0 2024-09-10T21:43:06.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.594+0000 7f5c3d600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f5c380ae120 con 0x7f5c38146c50 2024-09-10T21:43:06.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.595+0000 7f5c37e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c3806f590 0x7f5c381f9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:06.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.595+0000 7f5c37400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c38146c50 0x7f5c380b4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:06.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.595+0000 7f5c3cc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c381470b0 0x7f5c3813de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:06.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.595+0000 7f5c37e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c3806f590 0x7f5c381f9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:41000/0 (socket says 172.21.15.7:41000) 2024-09-10T21:43:06.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.595+0000 7f5c37e00640 1 -- 172.21.15.7:0/3326261406 learned_addr learned my addr 172.21.15.7:0/3326261406 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:06.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.595+0000 7f5c3cc00640 1 -- 172.21.15.7:0/3326261406 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c38146c50 msgr2=0x7f5c380b4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.595+0000 7f5c3cc00640 1 --2- 172.21.15.7:0/3326261406 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c38146c50 0x7f5c380b4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.595+0000 7f5c3cc00640 1 -- 172.21.15.7:0/3326261406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c3806f590 msgr2=0x7f5c381f9090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.595+0000 7f5c3cc00640 1 --2- 172.21.15.7:0/3326261406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c3806f590 0x7f5c381f9090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.595+0000 7f5c3cc00640 1 -- 172.21.15.7:0/3326261406 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c3813d3d0 con 0x7f5c381470b0 2024-09-10T21:43:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.595+0000 7f5c37e00640 1 --2- 172.21.15.7:0/3326261406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c3806f590 0x7f5c381f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:43:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.595+0000 7f5c3cc00640 1 --2- 172.21.15.7:0/3326261406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c381470b0 0x7f5c3813de10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5c3000e9b0 tx=0x7f5c3000ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.596+0000 7f5c2fe00640 1 -- 172.21.15.7:0/3326261406 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c3000cd80 con 0x7f5c381470b0 2024-09-10T21:43:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.596+0000 7f5c2fe00640 1 -- 172.21.15.7:0/3326261406 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c30004c10 con 0x7f5c381470b0 2024-09-10T21:43:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.596+0000 7f5c2fe00640 1 -- 172.21.15.7:0/3326261406 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c30005020 con 0x7f5c381470b0 2024-09-10T21:43:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.596+0000 7f5c3d600640 1 -- 172.21.15.7:0/3326261406 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c3813d0f0 con 0x7f5c381470b0 2024-09-10T21:43:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.596+0000 7f5c3d600640 1 -- 172.21.15.7:0/3326261406 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c381f8890 con 0x7f5c381470b0 2024-09-10T21:43:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.597+0000 7f5c3d600640 1 -- 172.21.15.7:0/3326261406 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c381f8d70 con 0x7f5c381470b0 2024-09-10T21:43:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.598+0000 7f5c2fe00640 1 -- 172.21.15.7:0/3326261406 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5c30020030 con 0x7f5c381470b0 2024-09-10T21:43:06.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.599+0000 7f5c2fe00640 1 --2- 172.21.15.7:0/3326261406 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5c18077f70 0x7f5c1807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:06.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.599+0000 7f5c2fe00640 1 -- 172.21.15.7:0/3326261406 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f5c3009a6d0 con 0x7f5c381470b0 2024-09-10T21:43:06.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.599+0000 7f5c37e00640 1 --2- 172.21.15.7:0/3326261406 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5c18077f70 0x7f5c1807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:06.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.600+0000 7f5c37e00640 1 --2- 172.21.15.7:0/3326261406 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5c18077f70 0x7f5c1807a430 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5c280096c0 tx=0x7f5c28004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:06.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.604+0000 7f5c2fe00640 1 -- 172.21.15.7:0/3326261406 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5c381f8d70 con 0x7f5c381470b0 2024-09-10T21:43:06.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.834+0000 7f5c3d600640 1 -- 172.21.15.7:0/3326261406 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c380ae640 con 0x7f5c381470b0 2024-09-10T21:43:06.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.835+0000 7f5c2fe00640 1 -- 172.21.15.7:0/3326261406 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5221 (secure 0 0 0) 0x7f5c30062b70 con 0x7f5c381470b0 2024-09-10T21:43:06.836 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:06.836 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"btime":"2024-09-10T21:42:58:925871+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/923954733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":923954733},{"type":"v1","addr":"172.21.15.133:6837","nonce":923954733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/13426364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":13426364},{"type":"v1","addr":"172.21.15.7:6835","nonce":13426364}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2275341781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2275341781},{"type":"v1","addr":"172.21.15.7:6834","nonce":2275341781}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:42:55.338104+0000","modified":"2024-09-10T21:42:58.362130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234524":{"gid":234524,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6836/849012248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":849012248},{"type":"v1","addr":"172.21.15.133:6836","nonce":849012248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234524,"qdb_cluster":[234524]},"id":2}]} 2024-09-10T21:43:06.837 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-09-10T21:43:06.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.837+0000 7f5c3d600640 1 -- 172.21.15.7:0/3326261406 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5c18077f70 msgr2=0x7f5c1807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:06.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.837+0000 7f5c3d600640 1 --2- 172.21.15.7:0/3326261406 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5c18077f70 0x7f5c1807a430 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5c280096c0 tx=0x7f5c28004730 comp rx=0 tx=0).stop 2024-09-10T21:43:06.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.837+0000 7f5c3d600640 1 -- 172.21.15.7:0/3326261406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c381470b0 msgr2=0x7f5c3813de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:06.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.837+0000 7f5c3d600640 1 --2- 172.21.15.7:0/3326261406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c381470b0 0x7f5c3813de10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5c3000e9b0 tx=0x7f5c3000ee70 comp rx=0 tx=0).stop 2024-09-10T21:43:06.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.837+0000 7f5c3d600640 1 -- 172.21.15.7:0/3326261406 shutdown_connections 2024-09-10T21:43:06.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.837+0000 7f5c3d600640 1 --2- 172.21.15.7:0/3326261406 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5c18077f70 0x7f5c1807a430 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:06.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.837+0000 7f5c3d600640 1 --2- 172.21.15.7:0/3326261406 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c381470b0 0x7f5c3813de10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:06.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.837+0000 7f5c3d600640 1 --2- 172.21.15.7:0/3326261406 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c38146c50 0x7f5c380b4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:06.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.837+0000 7f5c3d600640 1 --2- 172.21.15.7:0/3326261406 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c3806f590 0x7f5c381f9090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:06.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.837+0000 7f5c3d600640 1 -- 172.21.15.7:0/3326261406 >> 172.21.15.7:0/3326261406 conn(0x7f5c380b1d70 msgr2=0x7f5c380b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:06.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.837+0000 7f5c3d600640 1 -- 172.21.15.7:0/3326261406 shutdown_connections 2024-09-10T21:43:06.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:06.837+0000 7f5c3d600640 1 -- 172.21.15.7:0/3326261406 wait complete. 2024-09-10T21:43:06.854 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json tag path /parentdir mytag 2024-09-10T21:43:07.031 INFO:teuthology.orchestra.run.smithi133.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2024-09-10T21:43:07.031 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2024-09-10T21:43:17.033 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2024-09-10T21:43:17.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.117+0000 7fc07751df40 1 Processor -- start 2024-09-10T21:43:17.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.117+0000 7fc07751df40 1 -- start start 2024-09-10T21:43:17.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.118+0000 7fc07751df40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b42d3f8ae0 0x55b42d426dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.118+0000 7fc07751df40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b42d445110 0x55b42d4d23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.118+0000 7fc07751df40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b42d4b5620 0x55b42d42afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.118+0000 7fc07751df40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b42d347c60 con 0x55b42d445110 2024-09-10T21:43:17.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.118+0000 7fc07751df40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b42d3a61d0 con 0x55b42d4b5620 2024-09-10T21:43:17.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.118+0000 7fc07751df40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55b42d4441a0 con 0x55b42d3f8ae0 2024-09-10T21:43:17.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.118+0000 7fc073200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b42d445110 0x55b42d4d23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.118+0000 7fc073200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b42d445110 0x55b42d4d23f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55458/0 (socket says 172.21.15.7:55458) 2024-09-10T21:43:17.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.118+0000 7fc073200640 1 -- 172.21.15.7:0/941834669 learned_addr learned my addr 172.21.15.7:0/941834669 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:17.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.118+0000 7fc073c00640 1 --2- 172.21.15.7:0/941834669 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b42d3f8ae0 0x55b42d426dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.118+0000 7fc074600640 1 --2- 172.21.15.7:0/941834669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b42d4b5620 0x55b42d42afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.119+0000 7fc073200640 1 -- 172.21.15.7:0/941834669 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b42d3f8ae0 msgr2=0x55b42d426dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:17.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.119+0000 7fc073200640 1 --2- 172.21.15.7:0/941834669 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b42d3f8ae0 0x55b42d426dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.119+0000 7fc073200640 1 -- 172.21.15.7:0/941834669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b42d4b5620 msgr2=0x55b42d42afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:17.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.119+0000 7fc073200640 1 --2- 172.21.15.7:0/941834669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b42d4b5620 0x55b42d42afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.119+0000 7fc073200640 1 -- 172.21.15.7:0/941834669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b42d4d15e0 con 0x55b42d445110 2024-09-10T21:43:17.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.119+0000 7fc074600640 1 --2- 172.21.15.7:0/941834669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b42d4b5620 0x55b42d42afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:17.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.119+0000 7fc073200640 1 --2- 172.21.15.7:0/941834669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b42d445110 0x55b42d4d23f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc06000c600 tx=0x7fc06000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646f6c4c0ca1e13c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.119+0000 7fc071e00640 1 -- 172.21.15.7:0/941834669 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06000ab10 con 0x55b42d445110 2024-09-10T21:43:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.119+0000 7fc071e00640 1 -- 172.21.15.7:0/941834669 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc06000acb0 con 0x55b42d445110 2024-09-10T21:43:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.119+0000 7fc071e00640 1 -- 172.21.15.7:0/941834669 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06000ae60 con 0x55b42d445110 2024-09-10T21:43:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.120+0000 7fc07751df40 1 -- 172.21.15.7:0/941834669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b42d445110 msgr2=0x55b42d4d23f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.120+0000 7fc07751df40 1 --2- 172.21.15.7:0/941834669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b42d445110 0x55b42d4d23f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc06000c600 tx=0x7fc06000cac0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.120+0000 7fc07751df40 1 -- 172.21.15.7:0/941834669 shutdown_connections 2024-09-10T21:43:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.120+0000 7fc07751df40 1 --2- 172.21.15.7:0/941834669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b42d4b5620 0x55b42d42afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.120+0000 7fc07751df40 1 --2- 172.21.15.7:0/941834669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b42d445110 0x55b42d4d23f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.120+0000 7fc07751df40 1 --2- 172.21.15.7:0/941834669 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b42d3f8ae0 0x55b42d426dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.120+0000 7fc07751df40 1 -- 172.21.15.7:0/941834669 >> 172.21.15.7:0/941834669 conn(0x55b42d427de0 msgr2=0x55b42d428eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.120+0000 7fc07751df40 1 -- 172.21.15.7:0/941834669 shutdown_connections 2024-09-10T21:43:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.120+0000 7fc07751df40 1 -- 172.21.15.7:0/941834669 wait complete. 2024-09-10T21:43:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.121+0000 7fc07751df40 1 Processor -- start 2024-09-10T21:43:17.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.121+0000 7fc07751df40 1 -- start start 2024-09-10T21:43:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.122+0000 7fc07751df40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b42d3f8ae0 0x55b42d4b14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.122+0000 7fc07751df40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b42d445110 0x55b42d42a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.122+0000 7fc07751df40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b42d4b5620 0x55b42d42a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.122+0000 7fc07751df40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b42d429690 con 0x55b42d3f8ae0 2024-09-10T21:43:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.122+0000 7fc07751df40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b42d4b4a20 con 0x55b42d445110 2024-09-10T21:43:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.122+0000 7fc07751df40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55b42d4b4c70 con 0x55b42d4b5620 2024-09-10T21:43:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.122+0000 7fc073c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b42d3f8ae0 0x55b42d4b14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.122+0000 7fc073c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b42d3f8ae0 0x55b42d4b14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55466/0 (socket says 172.21.15.7:55466) 2024-09-10T21:43:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.122+0000 7fc073c00640 1 -- 172.21.15.7:0/3701724028 learned_addr learned my addr 172.21.15.7:0/3701724028 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.122+0000 7fc074600640 1 --2- 172.21.15.7:0/3701724028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b42d4b5620 0x55b42d42a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.122+0000 7fc073200640 1 --2- 172.21.15.7:0/3701724028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b42d445110 0x55b42d42a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.122+0000 7fc073c00640 1 -- 172.21.15.7:0/3701724028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b42d4b5620 msgr2=0x55b42d42a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.122+0000 7fc073c00640 1 --2- 172.21.15.7:0/3701724028 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b42d4b5620 0x55b42d42a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.122+0000 7fc073c00640 1 -- 172.21.15.7:0/3701724028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b42d445110 msgr2=0x55b42d42a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.122+0000 7fc073c00640 1 --2- 172.21.15.7:0/3701724028 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b42d445110 0x55b42d42a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.122+0000 7fc073c00640 1 -- 172.21.15.7:0/3701724028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b42d4b1a00 con 0x55b42d3f8ae0 2024-09-10T21:43:17.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.123+0000 7fc073c00640 1 --2- 172.21.15.7:0/3701724028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b42d3f8ae0 0x55b42d4b14c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc064046580 tx=0x7fc06404d7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.123+0000 7fc070a00640 1 -- 172.21.15.7:0/3701724028 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064062670 con 0x55b42d3f8ae0 2024-09-10T21:43:17.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.123+0000 7fc070a00640 1 -- 172.21.15.7:0/3701724028 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc064062900 con 0x55b42d3f8ae0 2024-09-10T21:43:17.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.123+0000 7fc07751df40 1 -- 172.21.15.7:0/3701724028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b42d4cb7f0 con 0x55b42d3f8ae0 2024-09-10T21:43:17.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.123+0000 7fc070a00640 1 -- 172.21.15.7:0/3701724028 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc064062d50 con 0x55b42d3f8ae0 2024-09-10T21:43:17.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.123+0000 7fc07751df40 1 -- 172.21.15.7:0/3701724028 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b42d4cbdb0 con 0x55b42d3f8ae0 2024-09-10T21:43:17.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.125+0000 7fc070a00640 1 -- 172.21.15.7:0/3701724028 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc064056470 con 0x55b42d3f8ae0 2024-09-10T21:43:17.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.126+0000 7fc070a00640 1 --2- 172.21.15.7:0/3701724028 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc054076950 0x7fc054078e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.126+0000 7fc070a00640 1 -- 172.21.15.7:0/3701724028 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fc0640dc970 con 0x55b42d3f8ae0 2024-09-10T21:43:17.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.127+0000 7fc073200640 1 --2- 172.21.15.7:0/3701724028 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc054076950 0x7fc054078e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.127+0000 7fc07751df40 1 --2- 172.21.15.7:0/3701724028 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55b42d4cdcb0 0x55b42d4ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.127+0000 7fc07751df40 1 -- 172.21.15.7:0/3701724028 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:1 4.10 4:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55b42d4ce5d0 con 0x55b42d4cdcb0 2024-09-10T21:43:17.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.127+0000 7fc074600640 1 --2- 172.21.15.7:0/3701724028 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55b42d4cdcb0 0x55b42d4ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.127+0000 7fc073200640 1 --2- 172.21.15.7:0/3701724028 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc054076950 0x7fc054078e10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55b42d42bda0 tx=0x7fc06000e040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.128+0000 7fc074600640 1 --2- 172.21.15.7:0/3701724028 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55b42d4cdcb0 0x55b42d4ce090 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.128+0000 7fc074600640 1 -- 172.21.15.7:0/3701724028 <== osd.3 v2:172.21.15.7:6800/2195665006 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7fc06c003040 con 0x55b42d4cdcb0 2024-09-10T21:43:17.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.128+0000 7fc07751df40 1 -- 172.21.15.7:0/3701724028 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55b42d4cdcb0 msgr2=0x55b42d4ce090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.128+0000 7fc07751df40 1 --2- 172.21.15.7:0/3701724028 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55b42d4cdcb0 0x55b42d4ce090 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.128+0000 7fc07751df40 1 -- 172.21.15.7:0/3701724028 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc054076950 msgr2=0x7fc054078e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.128+0000 7fc07751df40 1 --2- 172.21.15.7:0/3701724028 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc054076950 0x7fc054078e10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x55b42d42bda0 tx=0x7fc06000e040 comp rx=0 tx=0).stop 2024-09-10T21:43:17.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.128+0000 7fc07751df40 1 -- 172.21.15.7:0/3701724028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b42d3f8ae0 msgr2=0x55b42d4b14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.128+0000 7fc07751df40 1 --2- 172.21.15.7:0/3701724028 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b42d3f8ae0 0x55b42d4b14c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc064046580 tx=0x7fc06404d7f0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.128+0000 7fc074600640 1 -- 172.21.15.7:0/3701724028 reap_dead start 2024-09-10T21:43:17.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.128+0000 7fc07751df40 1 -- 172.21.15.7:0/3701724028 shutdown_connections 2024-09-10T21:43:17.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.128+0000 7fc07751df40 1 -- 172.21.15.7:0/3701724028 >> 172.21.15.7:0/3701724028 conn(0x55b42d427de0 msgr2=0x55b42d426cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:17.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.128+0000 7fc07751df40 1 -- 172.21.15.7:0/3701724028 shutdown_connections 2024-09-10T21:43:17.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.128+0000 7fc07751df40 1 -- 172.21.15.7:0/3701724028 wait complete. 2024-09-10T21:43:17.135 INFO:teuthology.orchestra.run.smithi007.stdout:mytag 2024-09-10T21:43:17.135 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2024-09-10T21:43:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.213+0000 7f706de11f40 1 Processor -- start 2024-09-10T21:43:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.213+0000 7f706de11f40 1 -- start start 2024-09-10T21:43:17.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.214+0000 7f706de11f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a908f9dcb0 0x55a908fd0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.214+0000 7f706de11f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a908fea2e0 0x55a908f9b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.214+0000 7f706de11f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a90905a620 0x55a909056780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.214+0000 7f706de11f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a908f4b1d0 con 0x55a908f9dcb0 2024-09-10T21:43:17.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.214+0000 7f706de11f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a908f57f90 con 0x55a908fea2e0 2024-09-10T21:43:17.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.214+0000 7f706de11f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55a908eecc60 con 0x55a90905a620 2024-09-10T21:43:17.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.214+0000 7f706b000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a908f9dcb0 0x55a908fd0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.214+0000 7f706b000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a908f9dcb0 0x55a908fd0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55470/0 (socket says 172.21.15.7:55470) 2024-09-10T21:43:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.214+0000 7f706b000640 1 -- 172.21.15.7:0/3129691661 learned_addr learned my addr 172.21.15.7:0/3129691661 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.214+0000 7f706ba00640 1 --2- 172.21.15.7:0/3129691661 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a90905a620 0x55a909056780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.214+0000 7f706a600640 1 --2- 172.21.15.7:0/3129691661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a908fea2e0 0x55a908f9b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.214+0000 7f706b000640 1 -- 172.21.15.7:0/3129691661 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a90905a620 msgr2=0x55a909056780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.214+0000 7f706b000640 1 --2- 172.21.15.7:0/3129691661 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a90905a620 0x55a909056780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.214+0000 7f706b000640 1 -- 172.21.15.7:0/3129691661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a908fea2e0 msgr2=0x55a908f9b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.214+0000 7f706b000640 1 --2- 172.21.15.7:0/3129691661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a908fea2e0 0x55a908f9b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.214+0000 7f706b000640 1 -- 172.21.15.7:0/3129691661 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a908fcdd40 con 0x55a908f9dcb0 2024-09-10T21:43:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.214+0000 7f706a600640 1 --2- 172.21.15.7:0/3129691661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a908fea2e0 0x55a908f9b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:43:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.214+0000 7f706b000640 1 --2- 172.21.15.7:0/3129691661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a908f9dcb0 0x55a908fd0200 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f705c001e40 tx=0x7f705c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3caa40c2a85b42e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.215+0000 7f7069200640 1 -- 172.21.15.7:0/3129691661 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c012070 con 0x55a908f9dcb0 2024-09-10T21:43:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.215+0000 7f7069200640 1 -- 172.21.15.7:0/3129691661 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f705c006050 con 0x55a908f9dcb0 2024-09-10T21:43:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.215+0000 7f7069200640 1 -- 172.21.15.7:0/3129691661 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f705c00ebe0 con 0x55a908f9dcb0 2024-09-10T21:43:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.215+0000 7f706de11f40 1 -- 172.21.15.7:0/3129691661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a908f9dcb0 msgr2=0x55a908fd0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.215+0000 7f706de11f40 1 --2- 172.21.15.7:0/3129691661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a908f9dcb0 0x55a908fd0200 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f705c001e40 tx=0x7f705c001e70 comp rx=0 tx=0).stop 2024-09-10T21:43:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.215+0000 7f706de11f40 1 -- 172.21.15.7:0/3129691661 shutdown_connections 2024-09-10T21:43:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.215+0000 7f706de11f40 1 --2- 172.21.15.7:0/3129691661 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a908fea2e0 0x55a908f9b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.215+0000 7f706de11f40 1 --2- 172.21.15.7:0/3129691661 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a90905a620 0x55a909056780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.215+0000 7f706de11f40 1 --2- 172.21.15.7:0/3129691661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a908f9dcb0 0x55a908fd0200 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.215+0000 7f706de11f40 1 -- 172.21.15.7:0/3129691661 >> 172.21.15.7:0/3129691661 conn(0x55a908fccde0 msgr2=0x55a909073fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.215+0000 7f706de11f40 1 -- 172.21.15.7:0/3129691661 shutdown_connections 2024-09-10T21:43:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.215+0000 7f706de11f40 1 -- 172.21.15.7:0/3129691661 wait complete. 2024-09-10T21:43:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.216+0000 7f706de11f40 1 Processor -- start 2024-09-10T21:43:17.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.216+0000 7f706de11f40 1 -- start start 2024-09-10T21:43:17.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.217+0000 7f706de11f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a908f9dcb0 0x55a90910bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.217+0000 7f706de11f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a908fea2e0 0x55a9090569f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.217+0000 7f706de11f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a90905a620 0x55a90910eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.217+0000 7f706de11f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a908fce000 con 0x55a908fea2e0 2024-09-10T21:43:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.217+0000 7f706de11f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a908fd0740 con 0x55a90905a620 2024-09-10T21:43:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.217+0000 7f706de11f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55a90906b350 con 0x55a908f9dcb0 2024-09-10T21:43:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.217+0000 7f706a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a908fea2e0 0x55a9090569f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.217+0000 7f706a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a908fea2e0 0x55a9090569f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55486/0 (socket says 172.21.15.7:55486) 2024-09-10T21:43:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.217+0000 7f706a600640 1 -- 172.21.15.7:0/2941629682 learned_addr learned my addr 172.21.15.7:0/2941629682 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.218+0000 7f706b000640 1 --2- 172.21.15.7:0/2941629682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a908f9dcb0 0x55a90910bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.218+0000 7f706ba00640 1 --2- 172.21.15.7:0/2941629682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a90905a620 0x55a90910eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.218+0000 7f706a600640 1 -- 172.21.15.7:0/2941629682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a908f9dcb0 msgr2=0x55a90910bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.218+0000 7f706a600640 1 --2- 172.21.15.7:0/2941629682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a908f9dcb0 0x55a90910bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.218+0000 7f706a600640 1 -- 172.21.15.7:0/2941629682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a90905a620 msgr2=0x55a90910eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.218+0000 7f706a600640 1 --2- 172.21.15.7:0/2941629682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a90905a620 0x55a90910eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.218+0000 7f706a600640 1 -- 172.21.15.7:0/2941629682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a909056f30 con 0x55a908fea2e0 2024-09-10T21:43:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.218+0000 7f706a600640 1 --2- 172.21.15.7:0/2941629682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a908fea2e0 0x55a9090569f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f705800ce90 tx=0x7f705800a5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.218+0000 7f7063400640 1 -- 172.21.15.7:0/2941629682 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058010470 con 0x55a908fea2e0 2024-09-10T21:43:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.219+0000 7f7063400640 1 -- 172.21.15.7:0/2941629682 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7058010610 con 0x55a908fea2e0 2024-09-10T21:43:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.219+0000 7f706de11f40 1 -- 172.21.15.7:0/2941629682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a908fe13d0 con 0x55a908fea2e0 2024-09-10T21:43:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.219+0000 7f7063400640 1 -- 172.21.15.7:0/2941629682 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7058010a40 con 0x55a908fea2e0 2024-09-10T21:43:17.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.219+0000 7f706de11f40 1 -- 172.21.15.7:0/2941629682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a908fe1970 con 0x55a908fea2e0 2024-09-10T21:43:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.221+0000 7f7063400640 1 -- 172.21.15.7:0/2941629682 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7058010be0 con 0x55a908fea2e0 2024-09-10T21:43:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.222+0000 7f7063400640 1 --2- 172.21.15.7:0/2941629682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f704c076980 0x7f704c078e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.222+0000 7f7063400640 1 -- 172.21.15.7:0/2941629682 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f7058098ac0 con 0x55a908fea2e0 2024-09-10T21:43:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.222+0000 7f706de11f40 1 --2- 172.21.15.7:0/2941629682 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x55a90910f770 0x55a9090584c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.222+0000 7f706b000640 1 --2- 172.21.15.7:0/2941629682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f704c076980 0x7f704c078e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.222+0000 7f706de11f40 1 -- 172.21.15.7:0/2941629682 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:1 4.7 4:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55a909058a00 con 0x55a90910f770 2024-09-10T21:43:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.222+0000 7f706ba00640 1 --2- 172.21.15.7:0/2941629682 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x55a90910f770 0x55a9090584c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.223+0000 7f706ba00640 1 --2- 172.21.15.7:0/2941629682 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x55a90910f770 0x55a9090584c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.223+0000 7f706b000640 1 --2- 172.21.15.7:0/2941629682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f704c076980 0x7f704c078e40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55a909055e70 tx=0x7f705c01a040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.223+0000 7f706ba00640 1 -- 172.21.15.7:0/2941629682 <== osd.1 v2:172.21.15.7:6824/4175601737 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f706400b040 con 0x55a90910f770 2024-09-10T21:43:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.223+0000 7f706de11f40 1 -- 172.21.15.7:0/2941629682 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x55a90910f770 msgr2=0x55a9090584c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.223+0000 7f706de11f40 1 --2- 172.21.15.7:0/2941629682 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x55a90910f770 0x55a9090584c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.224+0000 7f706de11f40 1 -- 172.21.15.7:0/2941629682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f704c076980 msgr2=0x7f704c078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.224+0000 7f706de11f40 1 --2- 172.21.15.7:0/2941629682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f704c076980 0x7f704c078e40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x55a909055e70 tx=0x7f705c01a040 comp rx=0 tx=0).stop 2024-09-10T21:43:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.224+0000 7f706de11f40 1 -- 172.21.15.7:0/2941629682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a908fea2e0 msgr2=0x55a9090569f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.224+0000 7f706de11f40 1 --2- 172.21.15.7:0/2941629682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a908fea2e0 0x55a9090569f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f705800ce90 tx=0x7f705800a5a0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.224+0000 7f706ba00640 1 -- 172.21.15.7:0/2941629682 reap_dead start 2024-09-10T21:43:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.224+0000 7f706de11f40 1 -- 172.21.15.7:0/2941629682 shutdown_connections 2024-09-10T21:43:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.224+0000 7f706de11f40 1 -- 172.21.15.7:0/2941629682 >> 172.21.15.7:0/2941629682 conn(0x55a908fccde0 msgr2=0x55a909077720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.224+0000 7f706de11f40 1 -- 172.21.15.7:0/2941629682 shutdown_connections 2024-09-10T21:43:17.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.224+0000 7f706de11f40 1 -- 172.21.15.7:0/2941629682 wait complete. 2024-09-10T21:43:17.229 INFO:teuthology.orchestra.run.smithi007.stdout:mytag 2024-09-10T21:43:17.229 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-09-10T21:43:17.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.305+0000 7f3a6f864f40 1 Processor -- start 2024-09-10T21:43:17.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.305+0000 7f3a6f864f40 1 -- start start 2024-09-10T21:43:17.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.305+0000 7f3a6f864f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1deb3ccb0 0x55d1deb6ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.305+0000 7f3a6f864f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d1deb89110 0x55d1deb6f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.305+0000 7f3a6f864f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1debf9620 0x55d1dec0aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.305+0000 7f3a6f864f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d1deaea1d0 con 0x55d1debf9620 2024-09-10T21:43:17.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.305+0000 7f3a6f864f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d1deaf6f90 con 0x55d1deb3ccb0 2024-09-10T21:43:17.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.305+0000 7f3a6f864f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55d1dea8bc60 con 0x55d1deb89110 2024-09-10T21:43:17.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.306+0000 7f3a6d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1debf9620 0x55d1dec0aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.306+0000 7f3a6d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1debf9620 0x55d1dec0aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55504/0 (socket says 172.21.15.7:55504) 2024-09-10T21:43:17.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.306+0000 7f3a6d400640 1 -- 172.21.15.7:0/35414822 learned_addr learned my addr 172.21.15.7:0/35414822 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:17.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.306+0000 7f3a67e00640 1 --2- 172.21.15.7:0/35414822 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d1deb89110 0x55d1deb6f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.306+0000 7f3a6ca00640 1 --2- 172.21.15.7:0/35414822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1deb3ccb0 0x55d1deb6ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.306+0000 7f3a6d400640 1 -- 172.21.15.7:0/35414822 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d1deb89110 msgr2=0x55d1deb6f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:17.309 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.306+0000 7f3a6d400640 1 --2- 172.21.15.7:0/35414822 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d1deb89110 0x55d1deb6f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.306+0000 7f3a6d400640 1 -- 172.21.15.7:0/35414822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1deb3ccb0 msgr2=0x55d1deb6ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:17.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.306+0000 7f3a6d400640 1 --2- 172.21.15.7:0/35414822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1deb3ccb0 0x55d1deb6ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.306+0000 7f3a6d400640 1 -- 172.21.15.7:0/35414822 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1dec0a650 con 0x55d1debf9620 2024-09-10T21:43:17.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.306+0000 7f3a67e00640 1 --2- 172.21.15.7:0/35414822 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d1deb89110 0x55d1deb6f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:17.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.307+0000 7f3a6d400640 1 --2- 172.21.15.7:0/35414822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1debf9620 0x55d1dec0aa20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3a68002630 tx=0x7f3a68002af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d1648ae2fa4688f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.307+0000 7f3a66a00640 1 -- 172.21.15.7:0/35414822 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68016070 con 0x55d1debf9620 2024-09-10T21:43:17.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.307+0000 7f3a66a00640 1 -- 172.21.15.7:0/35414822 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a6800d7a0 con 0x55d1debf9620 2024-09-10T21:43:17.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.307+0000 7f3a66a00640 1 -- 172.21.15.7:0/35414822 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a6800dbf0 con 0x55d1debf9620 2024-09-10T21:43:17.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.307+0000 7f3a6f864f40 1 -- 172.21.15.7:0/35414822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1debf9620 msgr2=0x55d1dec0aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.307+0000 7f3a6f864f40 1 --2- 172.21.15.7:0/35414822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1debf9620 0x55d1dec0aa20 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f3a68002630 tx=0x7f3a68002af0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.307+0000 7f3a6f864f40 1 -- 172.21.15.7:0/35414822 shutdown_connections 2024-09-10T21:43:17.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.307+0000 7f3a6f864f40 1 --2- 172.21.15.7:0/35414822 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1debf9620 0x55d1dec0aa20 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.307+0000 7f3a6f864f40 1 --2- 172.21.15.7:0/35414822 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d1deb89110 0x55d1deb6f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.307+0000 7f3a6f864f40 1 --2- 172.21.15.7:0/35414822 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1deb3ccb0 0x55d1deb6ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.307+0000 7f3a6f864f40 1 -- 172.21.15.7:0/35414822 >> 172.21.15.7:0/35414822 conn(0x55d1deb6bde0 msgr2=0x55d1dec16660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:17.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.307+0000 7f3a6f864f40 1 -- 172.21.15.7:0/35414822 shutdown_connections 2024-09-10T21:43:17.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.307+0000 7f3a6f864f40 1 -- 172.21.15.7:0/35414822 wait complete. 2024-09-10T21:43:17.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.308+0000 7f3a6f864f40 1 Processor -- start 2024-09-10T21:43:17.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.308+0000 7f3a6f864f40 1 -- start start 2024-09-10T21:43:17.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.309+0000 7f3a6f864f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1deb3ccb0 0x55d1debf7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.309+0000 7f3a6f864f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d1deb89110 0x55d1debf6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.309+0000 7f3a6f864f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1debf9620 0x55d1deb6e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.309+0000 7f3a6f864f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d1deb6add0 con 0x55d1debf9620 2024-09-10T21:43:17.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.309+0000 7f3a6f864f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d1deb6ab80 con 0x55d1deb3ccb0 2024-09-10T21:43:17.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.309+0000 7f3a6f864f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55d1deb6d6b0 con 0x55d1deb89110 2024-09-10T21:43:17.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.309+0000 7f3a6d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1debf9620 0x55d1deb6e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.309+0000 7f3a6d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1debf9620 0x55d1deb6e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55506/0 (socket says 172.21.15.7:55506) 2024-09-10T21:43:17.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.309+0000 7f3a6d400640 1 -- 172.21.15.7:0/3122780145 learned_addr learned my addr 172.21.15.7:0/3122780145 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:17.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.310+0000 7f3a67e00640 1 --2- 172.21.15.7:0/3122780145 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d1deb89110 0x55d1debf6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.310+0000 7f3a6ca00640 1 --2- 172.21.15.7:0/3122780145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1deb3ccb0 0x55d1debf7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.310+0000 7f3a6d400640 1 -- 172.21.15.7:0/3122780145 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d1deb89110 msgr2=0x55d1debf6ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.310+0000 7f3a6d400640 1 --2- 172.21.15.7:0/3122780145 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d1deb89110 0x55d1debf6ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.310+0000 7f3a6d400640 1 -- 172.21.15.7:0/3122780145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1deb3ccb0 msgr2=0x55d1debf7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.310+0000 7f3a6d400640 1 --2- 172.21.15.7:0/3122780145 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d1deb3ccb0 0x55d1debf7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.310+0000 7f3a6d400640 1 -- 172.21.15.7:0/3122780145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1deca01e0 con 0x55d1debf9620 2024-09-10T21:43:17.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.310+0000 7f3a6d400640 1 --2- 172.21.15.7:0/3122780145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1debf9620 0x55d1deb6e080 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3a68010000 tx=0x7f3a6800df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.311+0000 7f3a65600640 1 -- 172.21.15.7:0/3122780145 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a68016040 con 0x55d1debf9620 2024-09-10T21:43:17.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.311+0000 7f3a65600640 1 -- 172.21.15.7:0/3122780145 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a68020070 con 0x55d1debf9620 2024-09-10T21:43:17.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.311+0000 7f3a65600640 1 -- 172.21.15.7:0/3122780145 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a680056a0 con 0x55d1debf9620 2024-09-10T21:43:17.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.311+0000 7f3a6f864f40 1 -- 172.21.15.7:0/3122780145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d1dec9f660 con 0x55d1debf9620 2024-09-10T21:43:17.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.311+0000 7f3a6f864f40 1 -- 172.21.15.7:0/3122780145 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1dec9fc20 con 0x55d1debf9620 2024-09-10T21:43:17.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.313+0000 7f3a65600640 1 -- 172.21.15.7:0/3122780145 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3a68005840 con 0x55d1debf9620 2024-09-10T21:43:17.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.314+0000 7f3a65600640 1 --2- 172.21.15.7:0/3122780145 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3a4c076950 0x7f3a4c078e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.314+0000 7f3a65600640 1 -- 172.21.15.7:0/3122780145 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f3a680998e0 con 0x55d1debf9620 2024-09-10T21:43:17.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.314+0000 7f3a6f864f40 1 --2- 172.21.15.7:0/3122780145 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x55d1debf7450 0x55d1dec9e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.314+0000 7f3a6f864f40 1 -- 172.21.15.7:0/3122780145 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:1 5.36 5:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x55d1dec9e730 con 0x55d1debf7450 2024-09-10T21:43:17.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.314+0000 7f3a6ca00640 1 --2- 172.21.15.7:0/3122780145 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3a4c076950 0x7f3a4c078e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.314+0000 7f3a67e00640 1 --2- 172.21.15.7:0/3122780145 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x55d1debf7450 0x55d1dec9e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.315+0000 7f3a6ca00640 1 --2- 172.21.15.7:0/3122780145 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3a4c076950 0x7f3a4c078e10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3a600105b0 tx=0x7f3a600027c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.315+0000 7f3a67e00640 1 --2- 172.21.15.7:0/3122780145 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x55d1debf7450 0x55d1dec9e1f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.315+0000 7f3a67e00640 1 -- 172.21.15.7:0/3122780145 <== osd.1 v2:172.21.15.7:6824/4175601737 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f3a58056470 con 0x55d1debf7450 2024-09-10T21:43:17.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.315+0000 7f3a6f864f40 1 -- 172.21.15.7:0/3122780145 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x55d1debf7450 msgr2=0x55d1dec9e1f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.315+0000 7f3a6f864f40 1 --2- 172.21.15.7:0/3122780145 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x55d1debf7450 0x55d1dec9e1f0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.315+0000 7f3a6f864f40 1 -- 172.21.15.7:0/3122780145 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3a4c076950 msgr2=0x7f3a4c078e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.315+0000 7f3a6f864f40 1 --2- 172.21.15.7:0/3122780145 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3a4c076950 0x7f3a4c078e10 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3a600105b0 tx=0x7f3a600027c0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.315+0000 7f3a6f864f40 1 -- 172.21.15.7:0/3122780145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1debf9620 msgr2=0x55d1deb6e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.315+0000 7f3a6f864f40 1 --2- 172.21.15.7:0/3122780145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d1debf9620 0x55d1deb6e080 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f3a68010000 tx=0x7f3a6800df40 comp rx=0 tx=0).stop 2024-09-10T21:43:17.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.315+0000 7f3a6d400640 1 -- 172.21.15.7:0/3122780145 reap_dead start 2024-09-10T21:43:17.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.315+0000 7f3a6f864f40 1 -- 172.21.15.7:0/3122780145 shutdown_connections 2024-09-10T21:43:17.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.315+0000 7f3a6f864f40 1 -- 172.21.15.7:0/3122780145 >> 172.21.15.7:0/3122780145 conn(0x55d1deb6bde0 msgr2=0x55d1dec0d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:17.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.316+0000 7f3a6f864f40 1 -- 172.21.15.7:0/3122780145 shutdown_connections 2024-09-10T21:43:17.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.316+0000 7f3a6f864f40 1 -- 172.21.15.7:0/3122780145 wait complete. 2024-09-10T21:43:17.321 INFO:teuthology.orchestra.run.smithi007.stdout:mytag 2024-09-10T21:43:17.321 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-09-10T21:43:17.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.397+0000 7f8a19fc8f40 1 Processor -- start 2024-09-10T21:43:17.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.397+0000 7f8a19fc8f40 1 -- start start 2024-09-10T21:43:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.397+0000 7f8a19fc8f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560f13b6fae0 0x560f13ba1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.397+0000 7f8a19fc8f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f13bbc2e0 0x560f13b9ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.397+0000 7f8a19fc8f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f13c2c620 0x560f13c48090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.397+0000 7f8a19fc8f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560f13abec60 con 0x560f13bbc2e0 2024-09-10T21:43:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.397+0000 7f8a19fc8f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560f13b1d1d0 con 0x560f13c2c620 2024-09-10T21:43:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.397+0000 7f8a19fc8f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x560f13bbb150 con 0x560f13b6fae0 2024-09-10T21:43:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.398+0000 7f8a16800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f13bbc2e0 0x560f13b9ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.398+0000 7f8a16800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f13bbc2e0 0x560f13b9ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55508/0 (socket says 172.21.15.7:55508) 2024-09-10T21:43:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.398+0000 7f8a16800640 1 -- 172.21.15.7:0/2146138583 learned_addr learned my addr 172.21.15.7:0/2146138583 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.398+0000 7f8a17200640 1 --2- 172.21.15.7:0/2146138583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560f13b6fae0 0x560f13ba1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.398+0000 7f8a17c00640 1 --2- 172.21.15.7:0/2146138583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f13c2c620 0x560f13c48090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.398+0000 7f8a16800640 1 -- 172.21.15.7:0/2146138583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560f13b6fae0 msgr2=0x560f13ba1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.398+0000 7f8a16800640 1 --2- 172.21.15.7:0/2146138583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560f13b6fae0 0x560f13ba1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.398+0000 7f8a16800640 1 -- 172.21.15.7:0/2146138583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f13c2c620 msgr2=0x560f13c48090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.398+0000 7f8a16800640 1 --2- 172.21.15.7:0/2146138583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f13c2c620 0x560f13c48090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.398+0000 7f8a16800640 1 -- 172.21.15.7:0/2146138583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f13c405b0 con 0x560f13bbc2e0 2024-09-10T21:43:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.399+0000 7f8a16800640 1 --2- 172.21.15.7:0/2146138583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f13bbc2e0 0x560f13b9ff90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8a04001ea0 tx=0x7f8a0400b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a73751ffa8bfc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.399+0000 7f8a15400640 1 -- 172.21.15.7:0/2146138583 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a04012070 con 0x560f13bbc2e0 2024-09-10T21:43:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.399+0000 7f8a15400640 1 -- 172.21.15.7:0/2146138583 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a04006050 con 0x560f13bbc2e0 2024-09-10T21:43:17.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.399+0000 7f8a15400640 1 -- 172.21.15.7:0/2146138583 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0400eba0 con 0x560f13bbc2e0 2024-09-10T21:43:17.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.399+0000 7f8a19fc8f40 1 -- 172.21.15.7:0/2146138583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f13bbc2e0 msgr2=0x560f13b9ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.399+0000 7f8a19fc8f40 1 --2- 172.21.15.7:0/2146138583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f13bbc2e0 0x560f13b9ff90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8a04001ea0 tx=0x7f8a0400b730 comp rx=0 tx=0).stop 2024-09-10T21:43:17.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.399+0000 7f8a19fc8f40 1 -- 172.21.15.7:0/2146138583 shutdown_connections 2024-09-10T21:43:17.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.399+0000 7f8a19fc8f40 1 --2- 172.21.15.7:0/2146138583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f13c2c620 0x560f13c48090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.399+0000 7f8a19fc8f40 1 --2- 172.21.15.7:0/2146138583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f13bbc2e0 0x560f13b9ff90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.399+0000 7f8a19fc8f40 1 --2- 172.21.15.7:0/2146138583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560f13b6fae0 0x560f13ba1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.399+0000 7f8a19fc8f40 1 -- 172.21.15.7:0/2146138583 >> 172.21.15.7:0/2146138583 conn(0x560f13b9ede0 msgr2=0x560f13b9d0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.399+0000 7f8a19fc8f40 1 -- 172.21.15.7:0/2146138583 shutdown_connections 2024-09-10T21:43:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.399+0000 7f8a19fc8f40 1 -- 172.21.15.7:0/2146138583 wait complete. 2024-09-10T21:43:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.400+0000 7f8a19fc8f40 1 Processor -- start 2024-09-10T21:43:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.400+0000 7f8a19fc8f40 1 -- start start 2024-09-10T21:43:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.401+0000 7f8a19fc8f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f13b6fae0 0x560f13b6dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.401+0000 7f8a19fc8f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560f13bbc2e0 0x560f13c2a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.401+0000 7f8a19fc8f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f13c2c620 0x560f13cde4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.401+0000 7f8a19fc8f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560f13c463e0 con 0x560f13b6fae0 2024-09-10T21:43:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.401+0000 7f8a19fc8f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560f13c40820 con 0x560f13c2c620 2024-09-10T21:43:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.401+0000 7f8a19fc8f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x560f13ba2430 con 0x560f13bbc2e0 2024-09-10T21:43:17.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.401+0000 7f8a17200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f13b6fae0 0x560f13b6dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.401+0000 7f8a17200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f13b6fae0 0x560f13b6dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55518/0 (socket says 172.21.15.7:55518) 2024-09-10T21:43:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.401+0000 7f8a17200640 1 -- 172.21.15.7:0/4234185812 learned_addr learned my addr 172.21.15.7:0/4234185812 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.401+0000 7f8a16800640 1 --2- 172.21.15.7:0/4234185812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560f13bbc2e0 0x560f13c2a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.402+0000 7f8a17c00640 1 --2- 172.21.15.7:0/4234185812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f13c2c620 0x560f13cde4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.402+0000 7f8a17200640 1 -- 172.21.15.7:0/4234185812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560f13bbc2e0 msgr2=0x560f13c2a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.402+0000 7f8a17200640 1 --2- 172.21.15.7:0/4234185812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560f13bbc2e0 0x560f13c2a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.402+0000 7f8a17200640 1 -- 172.21.15.7:0/4234185812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f13c2c620 msgr2=0x560f13cde4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.402+0000 7f8a17200640 1 --2- 172.21.15.7:0/4234185812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f13c2c620 0x560f13cde4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.402+0000 7f8a17200640 1 -- 172.21.15.7:0/4234185812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f13c2a780 con 0x560f13b6fae0 2024-09-10T21:43:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.402+0000 7f8a17c00640 1 --2- 172.21.15.7:0/4234185812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560f13c2c620 0x560f13cde4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.402+0000 7f8a17200640 1 --2- 172.21.15.7:0/4234185812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f13b6fae0 0x560f13b6dc50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8a08004230 tx=0x7f8a08004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.402+0000 7f8a0fe00640 1 -- 172.21.15.7:0/4234185812 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08068070 con 0x560f13b6fae0 2024-09-10T21:43:17.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.402+0000 7f8a0fe00640 1 -- 172.21.15.7:0/4234185812 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a08063480 con 0x560f13b6fae0 2024-09-10T21:43:17.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.402+0000 7f8a0fe00640 1 -- 172.21.15.7:0/4234185812 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a08063890 con 0x560f13b6fae0 2024-09-10T21:43:17.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.402+0000 7f8a19fc8f40 1 -- 172.21.15.7:0/4234185812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560f13c3d710 con 0x560f13b6fae0 2024-09-10T21:43:17.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.403+0000 7f8a19fc8f40 1 -- 172.21.15.7:0/4234185812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f13c3d430 con 0x560f13b6fae0 2024-09-10T21:43:17.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.404+0000 7f8a0fe00640 1 -- 172.21.15.7:0/4234185812 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8a08056470 con 0x560f13b6fae0 2024-09-10T21:43:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.405+0000 7f8a0fe00640 1 --2- 172.21.15.7:0/4234185812 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f89f8076980 0x7f89f8078e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.406+0000 7f8a0fe00640 1 -- 172.21.15.7:0/4234185812 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f8a080dc7e0 con 0x560f13b6fae0 2024-09-10T21:43:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.406+0000 7f8a19fc8f40 1 --2- 172.21.15.7:0/4234185812 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x560f13c2aaa0 0x560f13cddf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.406+0000 7f8a16800640 1 --2- 172.21.15.7:0/4234185812 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f89f8076980 0x7f89f8078e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.406+0000 7f8a19fc8f40 1 -- 172.21.15.7:0/4234185812 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:1 5.19 5:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x560f13c45590 con 0x560f13c2aaa0 2024-09-10T21:43:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.406+0000 7f8a17c00640 1 --2- 172.21.15.7:0/4234185812 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x560f13c2aaa0 0x560f13cddf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.407+0000 7f8a16800640 1 --2- 172.21.15.7:0/4234185812 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f89f8076980 0x7f89f8078e40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x560f13c3d940 tx=0x7f8a0401a040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.407+0000 7f8a17c00640 1 --2- 172.21.15.7:0/4234185812 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x560f13c2aaa0 0x560f13cddf20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.407+0000 7f8a17c00640 1 -- 172.21.15.7:0/4234185812 <== osd.1 v2:172.21.15.7:6824/4175601737 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) ==== 164+0+9 (crc 0 0 0) 0x7f8a10003040 con 0x560f13c2aaa0 2024-09-10T21:43:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.407+0000 7f8a19fc8f40 1 -- 172.21.15.7:0/4234185812 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x560f13c2aaa0 msgr2=0x560f13cddf20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.407+0000 7f8a19fc8f40 1 --2- 172.21.15.7:0/4234185812 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x560f13c2aaa0 0x560f13cddf20 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.407+0000 7f8a19fc8f40 1 -- 172.21.15.7:0/4234185812 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f89f8076980 msgr2=0x7f89f8078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.407+0000 7f8a19fc8f40 1 --2- 172.21.15.7:0/4234185812 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f89f8076980 0x7f89f8078e40 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x560f13c3d940 tx=0x7f8a0401a040 comp rx=0 tx=0).stop 2024-09-10T21:43:17.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.407+0000 7f8a19fc8f40 1 -- 172.21.15.7:0/4234185812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f13b6fae0 msgr2=0x560f13b6dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.407+0000 7f8a19fc8f40 1 --2- 172.21.15.7:0/4234185812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560f13b6fae0 0x560f13b6dc50 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8a08004230 tx=0x7f8a08004690 comp rx=0 tx=0).stop 2024-09-10T21:43:17.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.407+0000 7f8a17c00640 1 -- 172.21.15.7:0/4234185812 reap_dead start 2024-09-10T21:43:17.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.407+0000 7f8a19fc8f40 1 -- 172.21.15.7:0/4234185812 shutdown_connections 2024-09-10T21:43:17.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.407+0000 7f8a19fc8f40 1 -- 172.21.15.7:0/4234185812 >> 172.21.15.7:0/4234185812 conn(0x560f13b9ede0 msgr2=0x560f13ba0e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:17.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.407+0000 7f8a19fc8f40 1 -- 172.21.15.7:0/4234185812 shutdown_connections 2024-09-10T21:43:17.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.407+0000 7f8a19fc8f40 1 -- 172.21.15.7:0/4234185812 wait complete. 2024-09-10T21:43:17.412 INFO:teuthology.orchestra.run.smithi007.stdout:mytag 2024-09-10T21:43:17.413 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-09-10T21:43:17.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.488+0000 7f25c244af40 1 Processor -- start 2024-09-10T21:43:17.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.489+0000 7f25c244af40 1 -- start start 2024-09-10T21:43:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.489+0000 7f25c244af40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614f1f2dcb0 0x5614f1ffba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.489+0000 7f25c244af40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5614f1f7a110 0x5614f1f60200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.489+0000 7f25c244af40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614f1fea620 0x5614f1f60740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.489+0000 7f25c244af40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5614f1e7cc60 con 0x5614f1f2dcb0 2024-09-10T21:43:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.489+0000 7f25c244af40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5614f1edb1d0 con 0x5614f1fea620 2024-09-10T21:43:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.489+0000 7f25c244af40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5614f1f79440 con 0x5614f1f7a110 2024-09-10T21:43:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.490+0000 7f25bf600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614f1f2dcb0 0x5614f1ffba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.490+0000 7f25bf600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614f1f2dcb0 0x5614f1ffba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55526/0 (socket says 172.21.15.7:55526) 2024-09-10T21:43:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.490+0000 7f25bf600640 1 -- 172.21.15.7:0/3736914305 learned_addr learned my addr 172.21.15.7:0/3736914305 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.490+0000 7f25c0000640 1 --2- 172.21.15.7:0/3736914305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614f1fea620 0x5614f1f60740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.490+0000 7f25bec00640 1 --2- 172.21.15.7:0/3736914305 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5614f1f7a110 0x5614f1f60200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.490+0000 7f25bf600640 1 -- 172.21.15.7:0/3736914305 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5614f1f7a110 msgr2=0x5614f1f60200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.490+0000 7f25bf600640 1 --2- 172.21.15.7:0/3736914305 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5614f1f7a110 0x5614f1f60200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.490+0000 7f25bf600640 1 -- 172.21.15.7:0/3736914305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614f1fea620 msgr2=0x5614f1f60740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.490+0000 7f25bf600640 1 --2- 172.21.15.7:0/3736914305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614f1fea620 0x5614f1f60740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.490+0000 7f25bf600640 1 -- 172.21.15.7:0/3736914305 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614f1ffe880 con 0x5614f1f2dcb0 2024-09-10T21:43:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.490+0000 7f25bf600640 1 --2- 172.21.15.7:0/3736914305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614f1f2dcb0 0x5614f1ffba20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f25b0001d20 tx=0x7f25b000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a67370352fdc2c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.491+0000 7f25bd800640 1 -- 172.21.15.7:0/3736914305 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b000bd80 con 0x5614f1f2dcb0 2024-09-10T21:43:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.491+0000 7f25bd800640 1 -- 172.21.15.7:0/3736914305 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25b000f400 con 0x5614f1f2dcb0 2024-09-10T21:43:17.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.491+0000 7f25bd800640 1 -- 172.21.15.7:0/3736914305 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b000f890 con 0x5614f1f2dcb0 2024-09-10T21:43:17.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.491+0000 7f25c244af40 1 -- 172.21.15.7:0/3736914305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614f1f2dcb0 msgr2=0x5614f1ffba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.491+0000 7f25c244af40 1 --2- 172.21.15.7:0/3736914305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614f1f2dcb0 0x5614f1ffba20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f25b0001d20 tx=0x7f25b000b5b0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.491+0000 7f25c244af40 1 -- 172.21.15.7:0/3736914305 shutdown_connections 2024-09-10T21:43:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.491+0000 7f25c244af40 1 --2- 172.21.15.7:0/3736914305 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614f1fea620 0x5614f1f60740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.491+0000 7f25c244af40 1 --2- 172.21.15.7:0/3736914305 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5614f1f7a110 0x5614f1f60200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.491+0000 7f25c244af40 1 --2- 172.21.15.7:0/3736914305 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614f1f2dcb0 0x5614f1ffba20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.491+0000 7f25c244af40 1 -- 172.21.15.7:0/3736914305 >> 172.21.15.7:0/3736914305 conn(0x5614f1f5cde0 msgr2=0x5614f1f5b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.491+0000 7f25c244af40 1 -- 172.21.15.7:0/3736914305 shutdown_connections 2024-09-10T21:43:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.491+0000 7f25c244af40 1 -- 172.21.15.7:0/3736914305 wait complete. 2024-09-10T21:43:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.492+0000 7f25c244af40 1 Processor -- start 2024-09-10T21:43:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.492+0000 7f25c244af40 1 -- start start 2024-09-10T21:43:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.493+0000 7f25c244af40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614f1f2dcb0 0x5614f1f5f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.493+0000 7f25c244af40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5614f1f7a110 0x5614f1f5f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.493+0000 7f25c244af40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614f1fea620 0x5614f1fe9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.493+0000 7f25c244af40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5614f1f5e9f0 con 0x5614f1fea620 2024-09-10T21:43:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.493+0000 7f25c244af40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5614f1ffe040 con 0x5614f1f2dcb0 2024-09-10T21:43:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.493+0000 7f25c244af40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5614f1f5e750 con 0x5614f1f7a110 2024-09-10T21:43:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.493+0000 7f25c0000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614f1fea620 0x5614f1fe9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.493+0000 7f25c0000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614f1fea620 0x5614f1fe9130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55542/0 (socket says 172.21.15.7:55542) 2024-09-10T21:43:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.493+0000 7f25c0000640 1 -- 172.21.15.7:0/285819604 learned_addr learned my addr 172.21.15.7:0/285819604 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.493+0000 7f25bf600640 1 --2- 172.21.15.7:0/285819604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614f1f2dcb0 0x5614f1f5f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.493+0000 7f25bec00640 1 --2- 172.21.15.7:0/285819604 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5614f1f7a110 0x5614f1f5f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.494+0000 7f25c0000640 1 -- 172.21.15.7:0/285819604 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5614f1f7a110 msgr2=0x5614f1f5f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.494+0000 7f25c0000640 1 --2- 172.21.15.7:0/285819604 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5614f1f7a110 0x5614f1f5f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.494+0000 7f25c0000640 1 -- 172.21.15.7:0/285819604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614f1f2dcb0 msgr2=0x5614f1f5f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.494+0000 7f25c0000640 1 --2- 172.21.15.7:0/285819604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5614f1f2dcb0 0x5614f1f5f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.494+0000 7f25c0000640 1 -- 172.21.15.7:0/285819604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614f1f5e220 con 0x5614f1fea620 2024-09-10T21:43:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.494+0000 7f25c0000640 1 --2- 172.21.15.7:0/285819604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614f1fea620 0x5614f1fe9130 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f25b800f6f0 tx=0x7f25b800fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.494+0000 7f25b7e00640 1 -- 172.21.15.7:0/285819604 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b800b9d0 con 0x5614f1fea620 2024-09-10T21:43:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.494+0000 7f25b7e00640 1 -- 172.21.15.7:0/285819604 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25b8014070 con 0x5614f1fea620 2024-09-10T21:43:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.494+0000 7f25b7e00640 1 -- 172.21.15.7:0/285819604 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8019470 con 0x5614f1fea620 2024-09-10T21:43:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.494+0000 7f25c244af40 1 -- 172.21.15.7:0/285819604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5614f208cc20 con 0x5614f1fea620 2024-09-10T21:43:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.494+0000 7f25c244af40 1 -- 172.21.15.7:0/285819604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614f208d1e0 con 0x5614f1fea620 2024-09-10T21:43:17.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.497+0000 7f25b7e00640 1 -- 172.21.15.7:0/285819604 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f25b8019610 con 0x5614f1fea620 2024-09-10T21:43:17.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.498+0000 7f25b7e00640 1 --2- 172.21.15.7:0/285819604 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f25a0076980 0x7f25a0078e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.498+0000 7f25b7e00640 1 -- 172.21.15.7:0/285819604 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f25b8099b40 con 0x5614f1fea620 2024-09-10T21:43:17.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.498+0000 7f25c244af40 1 --2- 172.21.15.7:0/285819604 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5614f2002020 0x5614f209b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:17.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.498+0000 7f25c244af40 1 -- 172.21.15.7:0/285819604 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:1 5.12 5:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e32) -- 0x5614f209be40 con 0x5614f2002020 2024-09-10T21:43:17.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.498+0000 7f25bf600640 1 --2- 172.21.15.7:0/285819604 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f25a0076980 0x7f25a0078e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.498+0000 7f25bec00640 1 --2- 172.21.15.7:0/285819604 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5614f2002020 0x5614f209b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:17.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.499+0000 7f25bec00640 1 --2- 172.21.15.7:0/285819604 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5614f2002020 0x5614f209b900 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.499+0000 7f25bf600640 1 --2- 172.21.15.7:0/285819604 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f25a0076980 0x7f25a0078e40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f25b0003040 tx=0x7f25b0000ed0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.499+0000 7f25bec00640 1 -- 172.21.15.7:0/285819604 <== osd.1 v2:172.21.15.7:6824/4175601737 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f25ac056470 con 0x5614f2002020 2024-09-10T21:43:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2024-09-10T21:43:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.500+0000 7f25c244af40 1 -- 172.21.15.7:0/285819604 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5614f2002020 msgr2=0x5614f209b900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.500+0000 7f25c244af40 1 --2- 172.21.15.7:0/285819604 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5614f2002020 0x5614f209b900 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.500+0000 7f25c244af40 1 -- 172.21.15.7:0/285819604 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f25a0076980 msgr2=0x7f25a0078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.500+0000 7f25c244af40 1 --2- 172.21.15.7:0/285819604 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f25a0076980 0x7f25a0078e40 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f25b0003040 tx=0x7f25b0000ed0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.500+0000 7f25c244af40 1 -- 172.21.15.7:0/285819604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614f1fea620 msgr2=0x5614f1fe9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.500+0000 7f25c244af40 1 --2- 172.21.15.7:0/285819604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5614f1fea620 0x5614f1fe9130 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f25b800f6f0 tx=0x7f25b800fbb0 comp rx=0 tx=0).stop 2024-09-10T21:43:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.500+0000 7f25c0000640 1 -- 172.21.15.7:0/285819604 reap_dead start 2024-09-10T21:43:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.500+0000 7f25c244af40 1 -- 172.21.15.7:0/285819604 shutdown_connections 2024-09-10T21:43:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.500+0000 7f25c244af40 1 -- 172.21.15.7:0/285819604 >> 172.21.15.7:0/285819604 conn(0x5614f1f5cde0 msgr2=0x5614f1ffd230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:17.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.500+0000 7f25c244af40 1 -- 172.21.15.7:0/285819604 shutdown_connections 2024-09-10T21:43:17.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.500+0000 7f25c244af40 1 -- 172.21.15.7:0/285819604 wait complete. 2024-09-10T21:43:17.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:43:17.506 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:43:17.506 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:43:17.595 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:43:17.688 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:43:17.688 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:43:17.688 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:43:17.688 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:43:17.724 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:43:17.724 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:43:17.724 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:43:17.724 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:43:17.761 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:43:17.761 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:43:17.761 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:43:17.761 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:43:17.798 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:43:17.798 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:43:17.799 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:43:17.799 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:43:17.837 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:43:17.838 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:43:18.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.998+0000 7fb6ba800640 1 -- 172.21.15.7:0/2919256315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb698003a00 msgr2=0x7fb698003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:18.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.998+0000 7fb6ba800640 1 --2- 172.21.15.7:0/2919256315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb698003a00 0x7fb698003de0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4001740 tx=0x7fb698005830 comp rx=0 tx=0).stop 2024-09-10T21:43:18.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.998+0000 7fb6ba800640 1 -- 172.21.15.7:0/2919256315 shutdown_connections 2024-09-10T21:43:18.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.998+0000 7fb6ba800640 1 --2- 172.21.15.7:0/2919256315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb698003a00 0x7fb698003de0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.998+0000 7fb6ba800640 1 -- 172.21.15.7:0/2919256315 >> 172.21.15.7:0/2919256315 conn(0x7fb6b40b16d0 msgr2=0x7fb6b40b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:18.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.998+0000 7fb6ba800640 1 -- 172.21.15.7:0/2919256315 shutdown_connections 2024-09-10T21:43:18.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:17.998+0000 7fb6ba800640 1 -- 172.21.15.7:0/2919256315 wait complete. 2024-09-10T21:43:18.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.000+0000 7fb6ba800640 1 Processor -- start 2024-09-10T21:43:18.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.000+0000 7fb6ba800640 1 -- start start 2024-09-10T21:43:18.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.001+0000 7fb6ba800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb698003a00 0x7fb6b40b3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.001+0000 7fb6ba800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6b41e6940 0x7fb6b4157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.001+0000 7fb6ba800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6b41f7c90 0x7fb6b41f8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.001+0000 7fb6ba800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b40cd2d0 con 0x7fb6b41e6940 2024-09-10T21:43:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.001+0000 7fb6ba800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b40cd150 con 0x7fb698003a00 2024-09-10T21:43:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.001+0000 7fb6ba800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fb6b4160180 con 0x7fb6b41f7c90 2024-09-10T21:43:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.001+0000 7fb6aaa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6b41e6940 0x7fb6b4157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.001+0000 7fb6aaa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6b41e6940 0x7fb6b4157e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55558/0 (socket says 172.21.15.7:55558) 2024-09-10T21:43:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.001+0000 7fb6aaa00640 1 -- 172.21.15.7:0/4052271373 learned_addr learned my addr 172.21.15.7:0/4052271373 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.001+0000 7fb6b3400640 1 --2- 172.21.15.7:0/4052271373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb698003a00 0x7fb6b40b3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.001+0000 7fb6b3e00640 1 --2- 172.21.15.7:0/4052271373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6b41f7c90 0x7fb6b41f8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:18.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.002+0000 7fb6aaa00640 1 -- 172.21.15.7:0/4052271373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6b41f7c90 msgr2=0x7fb6b41f8110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:18.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.002+0000 7fb6aaa00640 1 --2- 172.21.15.7:0/4052271373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6b41f7c90 0x7fb6b41f8110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.002+0000 7fb6aaa00640 1 -- 172.21.15.7:0/4052271373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb698003a00 msgr2=0x7fb6b40b3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:18.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.002+0000 7fb6aaa00640 1 --2- 172.21.15.7:0/4052271373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb698003a00 0x7fb6b40b3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.002+0000 7fb6aaa00640 1 -- 172.21.15.7:0/4052271373 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b41e8230 con 0x7fb6b41e6940 2024-09-10T21:43:18.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.002+0000 7fb6aaa00640 1 --2- 172.21.15.7:0/4052271373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6b41e6940 0x7fb6b4157e70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb69c011a10 tx=0x7fb69c011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:18.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.002+0000 7fb6abe00640 1 -- 172.21.15.7:0/4052271373 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c01e070 con 0x7fb6b41e6940 2024-09-10T21:43:18.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.002+0000 7fb6abe00640 1 -- 172.21.15.7:0/4052271373 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb69c00fd90 con 0x7fb6b41e6940 2024-09-10T21:43:18.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.002+0000 7fb6ba800640 1 -- 172.21.15.7:0/4052271373 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b41e7190 con 0x7fb6b41e6940 2024-09-10T21:43:18.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.002+0000 7fb6abe00640 1 -- 172.21.15.7:0/4052271373 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69c004290 con 0x7fb6b41e6940 2024-09-10T21:43:18.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.002+0000 7fb6ba800640 1 -- 172.21.15.7:0/4052271373 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b41e7750 con 0x7fb6b41e6940 2024-09-10T21:43:18.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.003+0000 7fb6ba800640 1 -- 172.21.15.7:0/4052271373 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b40736e0 con 0x7fb6b41e6940 2024-09-10T21:43:18.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.004+0000 7fb6abe00640 1 -- 172.21.15.7:0/4052271373 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb69c0035f0 con 0x7fb6b41e6940 2024-09-10T21:43:18.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.005+0000 7fb6abe00640 1 --2- 172.21.15.7:0/4052271373 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb690077fa0 0x7fb69007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:18.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.005+0000 7fb6abe00640 1 -- 172.21.15.7:0/4052271373 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fb69c09a850 con 0x7fb6b41e6940 2024-09-10T21:43:18.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.006+0000 7fb6b3400640 1 --2- 172.21.15.7:0/4052271373 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb690077fa0 0x7fb69007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:18.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.006+0000 7fb6b3400640 1 --2- 172.21.15.7:0/4052271373 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb690077fa0 0x7fb69007a460 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4001710 tx=0x7fb6a4001270 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:18.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.010+0000 7fb6abe00640 1 -- 172.21.15.7:0/4052271373 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb69c062cf0 con 0x7fb6b41e6940 2024-09-10T21:43:18.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.242+0000 7fb6ba800640 1 -- 172.21.15.7:0/4052271373 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6b4069810 con 0x7fb6b41e6940 2024-09-10T21:43:18.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.243+0000 7fb6abe00640 1 -- 172.21.15.7:0/4052271373 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5221 (secure 0 0 0) 0x7fb69c067720 con 0x7fb6b41e6940 2024-09-10T21:43:18.244 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:18.244 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"btime":"2024-09-10T21:42:58:925871+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/923954733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":923954733},{"type":"v1","addr":"172.21.15.133:6837","nonce":923954733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/13426364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":13426364},{"type":"v1","addr":"172.21.15.7:6835","nonce":13426364}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2275341781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2275341781},{"type":"v1","addr":"172.21.15.7:6834","nonce":2275341781}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:42:55.338104+0000","modified":"2024-09-10T21:42:58.362130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234524":{"gid":234524,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6836/849012248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":849012248},{"type":"v1","addr":"172.21.15.133:6836","nonce":849012248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234524,"qdb_cluster":[234524]},"id":2}]} 2024-09-10T21:43:18.245 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-09-10T21:43:18.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.245+0000 7fb6ba800640 1 -- 172.21.15.7:0/4052271373 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb690077fa0 msgr2=0x7fb69007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:18.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.245+0000 7fb6ba800640 1 --2- 172.21.15.7:0/4052271373 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb690077fa0 0x7fb69007a460 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4001710 tx=0x7fb6a4001270 comp rx=0 tx=0).stop 2024-09-10T21:43:18.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.245+0000 7fb6ba800640 1 -- 172.21.15.7:0/4052271373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6b41e6940 msgr2=0x7fb6b4157e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:18.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.245+0000 7fb6ba800640 1 --2- 172.21.15.7:0/4052271373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6b41e6940 0x7fb6b4157e70 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb69c011a10 tx=0x7fb69c011ed0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.245+0000 7fb6ba800640 1 -- 172.21.15.7:0/4052271373 shutdown_connections 2024-09-10T21:43:18.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.245+0000 7fb6ba800640 1 --2- 172.21.15.7:0/4052271373 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb690077fa0 0x7fb69007a460 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.245+0000 7fb6ba800640 1 --2- 172.21.15.7:0/4052271373 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6b41f7c90 0x7fb6b41f8110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.245+0000 7fb6ba800640 1 --2- 172.21.15.7:0/4052271373 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6b41e6940 0x7fb6b4157e70 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.245+0000 7fb6ba800640 1 --2- 172.21.15.7:0/4052271373 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb698003a00 0x7fb6b40b3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.245+0000 7fb6ba800640 1 -- 172.21.15.7:0/4052271373 >> 172.21.15.7:0/4052271373 conn(0x7fb6b40b16d0 msgr2=0x7fb6b4159fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:18.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.246+0000 7fb6ba800640 1 -- 172.21.15.7:0/4052271373 shutdown_connections 2024-09-10T21:43:18.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.246+0000 7fb6ba800640 1 -- 172.21.15.7:0/4052271373 wait complete. 2024-09-10T21:43:18.263 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:43:18.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.424+0000 7f42bb200640 1 -- 172.21.15.7:0/4019110892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f429c004d40 msgr2=0x7f429c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:18.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.424+0000 7f42bb200640 1 --2- 172.21.15.7:0/4019110892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f429c004d40 0x7f429c0051c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f42b000c640 tx=0x7f429c005830 comp rx=0 tx=0).stop 2024-09-10T21:43:18.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.424+0000 7f42ba800640 1 -- 172.21.15.7:0/4019110892 reap_dead start 2024-09-10T21:43:18.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.425+0000 7f42bb200640 1 -- 172.21.15.7:0/4019110892 shutdown_connections 2024-09-10T21:43:18.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.425+0000 7f42bb200640 1 -- 172.21.15.7:0/4019110892 >> 172.21.15.7:0/4019110892 conn(0x7f42b40b1d70 msgr2=0x7f42b40cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:18.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.425+0000 7f42bb200640 1 -- 172.21.15.7:0/4019110892 shutdown_connections 2024-09-10T21:43:18.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.425+0000 7f42bb200640 1 -- 172.21.15.7:0/4019110892 wait complete. 2024-09-10T21:43:18.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.426+0000 7f42bb200640 1 Processor -- start 2024-09-10T21:43:18.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.426+0000 7f42bb200640 1 -- start start 2024-09-10T21:43:18.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.427+0000 7f42bb200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f429c004d40 0x7f42b413e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:18.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.427+0000 7f42bb200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42b4073670 0x7f42b4143380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:18.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.427+0000 7f42bb200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b4073af0 0x7f42b41438c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:18.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.427+0000 7f42bb200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42b40b4700 con 0x7f42b4073670 2024-09-10T21:43:18.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.427+0000 7f42bb200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f42b40b4580 con 0x7f42b4073af0 2024-09-10T21:43:18.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.427+0000 7f42bb200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f42b40b4880 con 0x7f429c004d40 2024-09-10T21:43:18.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.428+0000 7f42b9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42b4073670 0x7f42b4143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:18.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.428+0000 7f42b9e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f429c004d40 0x7f42b413e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:18.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.428+0000 7f42b9e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f429c004d40 0x7f42b413e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:55014/0 (socket says 172.21.15.7:55014) 2024-09-10T21:43:18.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.428+0000 7f42b9e00640 1 -- 172.21.15.7:0/328881891 learned_addr learned my addr 172.21.15.7:0/328881891 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:18.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.428+0000 7f42ba800640 1 --2- 172.21.15.7:0/328881891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b4073af0 0x7f42b41438c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.428+0000 7f42b9400640 1 -- 172.21.15.7:0/328881891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f429c004d40 msgr2=0x7f42b413e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.428+0000 7f42b9400640 1 --2- 172.21.15.7:0/328881891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f429c004d40 0x7f42b413e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.428+0000 7f42b9400640 1 -- 172.21.15.7:0/328881891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b4073af0 msgr2=0x7f42b41438c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.428+0000 7f42b9400640 1 --2- 172.21.15.7:0/328881891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b4073af0 0x7f42b41438c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.428+0000 7f42b9400640 1 -- 172.21.15.7:0/328881891 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42b41f4690 con 0x7f42b4073670 2024-09-10T21:43:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.428+0000 7f42b9400640 1 --2- 172.21.15.7:0/328881891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42b4073670 0x7f42b4143380 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f42b40d0b20 tx=0x7f42a4001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.429+0000 7f42ae000640 1 -- 172.21.15.7:0/328881891 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a4011640 con 0x7f42b4073670 2024-09-10T21:43:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.429+0000 7f42ae000640 1 -- 172.21.15.7:0/328881891 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42a4016020 con 0x7f42b4073670 2024-09-10T21:43:18.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.429+0000 7f42ae000640 1 -- 172.21.15.7:0/328881891 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42a4011a60 con 0x7f42b4073670 2024-09-10T21:43:18.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.429+0000 7f42bb200640 1 -- 172.21.15.7:0/328881891 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42b41f4970 con 0x7f42b4073670 2024-09-10T21:43:18.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.429+0000 7f42bb200640 1 -- 172.21.15.7:0/328881891 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42b41f4f30 con 0x7f42b4073670 2024-09-10T21:43:18.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.430+0000 7f42bb200640 1 -- 172.21.15.7:0/328881891 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42b40ce6f0 con 0x7f42b4073670 2024-09-10T21:43:18.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.431+0000 7f42ae000640 1 -- 172.21.15.7:0/328881891 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f42a4011c00 con 0x7f42b4073670 2024-09-10T21:43:18.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.432+0000 7f42ae000640 1 --2- 172.21.15.7:0/328881891 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4298077fa0 0x7f429807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:18.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.432+0000 7f42ae000640 1 -- 172.21.15.7:0/328881891 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f42a409a820 con 0x7f42b4073670 2024-09-10T21:43:18.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.433+0000 7f42b9e00640 1 --2- 172.21.15.7:0/328881891 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4298077fa0 0x7f429807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:18.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.433+0000 7f42b9e00640 1 --2- 172.21.15.7:0/328881891 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4298077fa0 0x7f429807a460 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f42b413e5c0 tx=0x7f42a8005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:18.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.436+0000 7f42ae000640 1 -- 172.21.15.7:0/328881891 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f42a4020020 con 0x7f42b4073670 2024-09-10T21:43:18.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.667+0000 7f42bb200640 1 -- 172.21.15.7:0/328881891 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42b406ed50 con 0x7f42b4073670 2024-09-10T21:43:18.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.667+0000 7f42ae000640 1 -- 172.21.15.7:0/328881891 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5221 (secure 0 0 0) 0x7f42a4062cd0 con 0x7f42b4073670 2024-09-10T21:43:18.670 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:18.670 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"btime":"2024-09-10T21:42:58:925871+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/923954733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":923954733},{"type":"v1","addr":"172.21.15.133:6837","nonce":923954733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/13426364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":13426364},{"type":"v1","addr":"172.21.15.7:6835","nonce":13426364}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2275341781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2275341781},{"type":"v1","addr":"172.21.15.7:6834","nonce":2275341781}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:42:55.338104+0000","modified":"2024-09-10T21:42:58.362130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234524":{"gid":234524,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6836/849012248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":849012248},{"type":"v1","addr":"172.21.15.133:6836","nonce":849012248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234524,"qdb_cluster":[234524]},"id":2}]} 2024-09-10T21:43:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-09-10T21:43:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.669+0000 7f42bb200640 1 -- 172.21.15.7:0/328881891 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4298077fa0 msgr2=0x7f429807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.669+0000 7f42bb200640 1 --2- 172.21.15.7:0/328881891 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4298077fa0 0x7f429807a460 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f42b413e5c0 tx=0x7f42a8005f90 comp rx=0 tx=0).stop 2024-09-10T21:43:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.669+0000 7f42bb200640 1 -- 172.21.15.7:0/328881891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42b4073670 msgr2=0x7f42b4143380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.669+0000 7f42bb200640 1 --2- 172.21.15.7:0/328881891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42b4073670 0x7f42b4143380 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f42b40d0b20 tx=0x7f42a4001800 comp rx=0 tx=0).stop 2024-09-10T21:43:18.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.669+0000 7f42bb200640 1 -- 172.21.15.7:0/328881891 shutdown_connections 2024-09-10T21:43:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.669+0000 7f42bb200640 1 --2- 172.21.15.7:0/328881891 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4298077fa0 0x7f429807a460 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.669+0000 7f42bb200640 1 --2- 172.21.15.7:0/328881891 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f42b4073af0 0x7f42b41438c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.669+0000 7f42bb200640 1 --2- 172.21.15.7:0/328881891 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42b4073670 0x7f42b4143380 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.669+0000 7f42bb200640 1 --2- 172.21.15.7:0/328881891 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f429c004d40 0x7f42b413e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.669+0000 7f42bb200640 1 -- 172.21.15.7:0/328881891 >> 172.21.15.7:0/328881891 conn(0x7f42b40b1d70 msgr2=0x7f42b406f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.670+0000 7f42bb200640 1 -- 172.21.15.7:0/328881891 shutdown_connections 2024-09-10T21:43:18.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.670+0000 7f42bb200640 1 -- 172.21.15.7:0/328881891 wait complete. 2024-09-10T21:43:18.687 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:43:18.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.845+0000 7f5c84a00640 1 -- 172.21.15.7:0/276854217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8015f0a0 msgr2=0x7f5c800b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:18.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.845+0000 7f5c84a00640 1 --2- 172.21.15.7:0/276854217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8015f0a0 0x7f5c800b2290 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f5c70009830 tx=0x7f5c7002a530 comp rx=0 tx=0).stop 2024-09-10T21:43:18.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.846+0000 7f5c84a00640 1 -- 172.21.15.7:0/276854217 shutdown_connections 2024-09-10T21:43:18.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.846+0000 7f5c84a00640 1 --2- 172.21.15.7:0/276854217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8015f0a0 0x7f5c800b2290 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.846+0000 7f5c84a00640 1 -- 172.21.15.7:0/276854217 >> 172.21.15.7:0/276854217 conn(0x7f5c800d1ba0 msgr2=0x7f5c800d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.846+0000 7f5c84a00640 1 -- 172.21.15.7:0/276854217 shutdown_connections 2024-09-10T21:43:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.846+0000 7f5c84a00640 1 -- 172.21.15.7:0/276854217 wait complete. 2024-09-10T21:43:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.847+0000 7f5c84a00640 1 Processor -- start 2024-09-10T21:43:18.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.847+0000 7f5c84a00640 1 -- start start 2024-09-10T21:43:18.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.848+0000 7f5c84a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c800d2690 0x7f5c800cbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:18.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.848+0000 7f5c84a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c8015f0a0 0x7f5c801f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:18.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.848+0000 7f5c84a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8015f480 0x7f5c800b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:18.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.848+0000 7f5c84a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5c800b0ef0 con 0x7f5c8015f480 2024-09-10T21:43:18.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.848+0000 7f5c84a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5c800b0d70 con 0x7f5c800d2690 2024-09-10T21:43:18.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.848+0000 7f5c84a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f5c800b1070 con 0x7f5c8015f0a0 2024-09-10T21:43:18.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.848+0000 7f5c7fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8015f480 0x7f5c800b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:18.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.849+0000 7f5c7fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8015f480 0x7f5c800b6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55578/0 (socket says 172.21.15.7:55578) 2024-09-10T21:43:18.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.849+0000 7f5c7fe00640 1 -- 172.21.15.7:0/31072932 learned_addr learned my addr 172.21.15.7:0/31072932 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:18.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.849+0000 7f5c7f400640 1 --2- 172.21.15.7:0/31072932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c800d2690 0x7f5c800cbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:18.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.849+0000 7f5c76a00640 1 --2- 172.21.15.7:0/31072932 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c8015f0a0 0x7f5c801f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:18.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.849+0000 7f5c7fe00640 1 -- 172.21.15.7:0/31072932 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c8015f0a0 msgr2=0x7f5c801f80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:18.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.849+0000 7f5c7fe00640 1 --2- 172.21.15.7:0/31072932 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c8015f0a0 0x7f5c801f80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.849+0000 7f5c7fe00640 1 -- 172.21.15.7:0/31072932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c800d2690 msgr2=0x7f5c800cbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:18.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.849+0000 7f5c7fe00640 1 --2- 172.21.15.7:0/31072932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c800d2690 0x7f5c800cbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.849+0000 7f5c7fe00640 1 -- 172.21.15.7:0/31072932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c800c9600 con 0x7f5c8015f480 2024-09-10T21:43:18.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.849+0000 7f5c7fe00640 1 --2- 172.21.15.7:0/31072932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8015f480 0x7f5c800b6e50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800c760 tx=0x7f5c7800cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:18.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.850+0000 7f5c77e00640 1 -- 172.21.15.7:0/31072932 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c78003b70 con 0x7f5c8015f480 2024-09-10T21:43:18.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.850+0000 7f5c77e00640 1 -- 172.21.15.7:0/31072932 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c78003d10 con 0x7f5c8015f480 2024-09-10T21:43:18.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.850+0000 7f5c77e00640 1 -- 172.21.15.7:0/31072932 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c780052d0 con 0x7f5c8015f480 2024-09-10T21:43:18.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.850+0000 7f5c84a00640 1 -- 172.21.15.7:0/31072932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c800caa70 con 0x7f5c8015f480 2024-09-10T21:43:18.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.850+0000 7f5c84a00640 1 -- 172.21.15.7:0/31072932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c800cb030 con 0x7f5c8015f480 2024-09-10T21:43:18.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.851+0000 7f5c84a00640 1 -- 172.21.15.7:0/31072932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c800b1b90 con 0x7f5c8015f480 2024-09-10T21:43:18.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.852+0000 7f5c77e00640 1 -- 172.21.15.7:0/31072932 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5c78005470 con 0x7f5c8015f480 2024-09-10T21:43:18.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.853+0000 7f5c77e00640 1 --2- 172.21.15.7:0/31072932 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5c5c077f70 0x7f5c5c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:18.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.853+0000 7f5c77e00640 1 -- 172.21.15.7:0/31072932 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f5c78099ac0 con 0x7f5c8015f480 2024-09-10T21:43:18.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.854+0000 7f5c7f400640 1 --2- 172.21.15.7:0/31072932 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5c5c077f70 0x7f5c5c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:18.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.854+0000 7f5c7f400640 1 --2- 172.21.15.7:0/31072932 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5c5c077f70 0x7f5c5c07a430 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5c70034000 tx=0x7f5c70004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:18.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:18.857+0000 7f5c77e00640 1 -- 172.21.15.7:0/31072932 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5c800b1b90 con 0x7f5c8015f480 2024-09-10T21:43:19.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.055+0000 7f5c84a00640 1 -- 172.21.15.7:0/31072932 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5c800af320 con 0x7f5c8015f480 2024-09-10T21:43:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.056+0000 7f5c77e00640 1 -- 172.21.15.7:0/31072932 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16693 (secure 0 0 0) 0x7f5c78062520 con 0x7f5c8015f480 2024-09-10T21:43:19.057 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:19.057 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":32,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:42:56.911349+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:42:52.896912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T21:42:54.300403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:43:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.058+0000 7f5c84a00640 1 -- 172.21.15.7:0/31072932 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5c5c077f70 msgr2=0x7f5c5c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.058+0000 7f5c84a00640 1 --2- 172.21.15.7:0/31072932 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5c5c077f70 0x7f5c5c07a430 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5c70034000 tx=0x7f5c70004730 comp rx=0 tx=0).stop 2024-09-10T21:43:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.058+0000 7f5c84a00640 1 -- 172.21.15.7:0/31072932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8015f480 msgr2=0x7f5c800b6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.058+0000 7f5c84a00640 1 --2- 172.21.15.7:0/31072932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8015f480 0x7f5c800b6e50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5c7800c760 tx=0x7f5c7800cb60 comp rx=0 tx=0).stop 2024-09-10T21:43:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.058+0000 7f5c84a00640 1 -- 172.21.15.7:0/31072932 shutdown_connections 2024-09-10T21:43:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.058+0000 7f5c84a00640 1 --2- 172.21.15.7:0/31072932 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5c5c077f70 0x7f5c5c07a430 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.058+0000 7f5c84a00640 1 --2- 172.21.15.7:0/31072932 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5c8015f480 0x7f5c800b6e50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.058+0000 7f5c84a00640 1 --2- 172.21.15.7:0/31072932 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5c8015f0a0 0x7f5c801f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.058+0000 7f5c84a00640 1 --2- 172.21.15.7:0/31072932 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5c800d2690 0x7f5c800cbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.058+0000 7f5c84a00640 1 -- 172.21.15.7:0/31072932 >> 172.21.15.7:0/31072932 conn(0x7f5c800d1ba0 msgr2=0x7f5c800d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:19.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.059+0000 7f5c84a00640 1 -- 172.21.15.7:0/31072932 shutdown_connections 2024-09-10T21:43:19.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.059+0000 7f5c84a00640 1 -- 172.21.15.7:0/31072932 wait complete. 2024-09-10T21:43:19.076 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:43:19.077 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:43:19.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.236+0000 7f9620200640 1 -- 172.21.15.7:0/1240334412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f961815bc90 msgr2=0x7f96180ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:19.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.236+0000 7f9620200640 1 --2- 172.21.15.7:0/1240334412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f961815bc90 0x7f96180ccef0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f96180cf860 tx=0x7f960c02a4d0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.236+0000 7f9620200640 1 -- 172.21.15.7:0/1240334412 shutdown_connections 2024-09-10T21:43:19.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.236+0000 7f9620200640 1 --2- 172.21.15.7:0/1240334412 >> v2:172.21.15.133:3300/0 conn(0x7f96180cd750 0x7f9618073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.236+0000 7f9620200640 1 --2- 172.21.15.7:0/1240334412 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f961815bc90 0x7f96180ccef0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.236+0000 7f9620200640 1 -- 172.21.15.7:0/1240334412 >> 172.21.15.7:0/1240334412 conn(0x7f96180b16d0 msgr2=0x7f96180b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:19.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.236+0000 7f9620200640 1 -- 172.21.15.7:0/1240334412 shutdown_connections 2024-09-10T21:43:19.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.236+0000 7f9620200640 1 -- 172.21.15.7:0/1240334412 wait complete. 2024-09-10T21:43:19.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.237+0000 7f9620200640 1 Processor -- start 2024-09-10T21:43:19.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.238+0000 7f9620200640 1 -- start start 2024-09-10T21:43:19.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.238+0000 7f9620200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9618073c70 0x7f96180c73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:19.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.238+0000 7f9620200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96180cd750 0x7f96181f3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:19.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.238+0000 7f9620200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f961815bc90 0x7f96181efba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:19.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.238+0000 7f9620200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f961806fcf0 con 0x7f9618073c70 2024-09-10T21:43:19.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.238+0000 7f9620200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f96181606c0 con 0x7f961815bc90 2024-09-10T21:43:19.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.238+0000 7f9620200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f961806fe70 con 0x7f96180cd750 2024-09-10T21:43:19.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.239+0000 7f961d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9618073c70 0x7f96180c73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:19.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.239+0000 7f961d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9618073c70 0x7f96180c73c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55594/0 (socket says 172.21.15.7:55594) 2024-09-10T21:43:19.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.239+0000 7f961d800640 1 -- 172.21.15.7:0/3216149781 learned_addr learned my addr 172.21.15.7:0/3216149781 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:19.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.239+0000 7f961e200640 1 --2- 172.21.15.7:0/3216149781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f961815bc90 0x7f96181efba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:19.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.239+0000 7f961ce00640 1 --2- 172.21.15.7:0/3216149781 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96180cd750 0x7f96181f3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:19.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.239+0000 7f961d800640 1 -- 172.21.15.7:0/3216149781 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96180cd750 msgr2=0x7f96181f3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:19.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.239+0000 7f961d800640 1 --2- 172.21.15.7:0/3216149781 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96180cd750 0x7f96181f3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.239+0000 7f961d800640 1 -- 172.21.15.7:0/3216149781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f961815bc90 msgr2=0x7f96181efba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:19.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.239+0000 7f961d800640 1 --2- 172.21.15.7:0/3216149781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f961815bc90 0x7f96181efba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.239+0000 7f961d800640 1 -- 172.21.15.7:0/3216149781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9618157660 con 0x7f9618073c70 2024-09-10T21:43:19.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.239+0000 7f961ce00640 1 --2- 172.21.15.7:0/3216149781 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96180cd750 0x7f96181f3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:19.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.239+0000 7f961d800640 1 --2- 172.21.15.7:0/3216149781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9618073c70 0x7f96180c73c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f960c034040 tx=0x7f960c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:19.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.240+0000 7f9615600640 1 -- 172.21.15.7:0/3216149781 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c03e030 con 0x7f9618073c70 2024-09-10T21:43:19.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.240+0000 7f9615600640 1 -- 172.21.15.7:0/3216149781 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f960c0047a0 con 0x7f9618073c70 2024-09-10T21:43:19.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.240+0000 7f9615600640 1 -- 172.21.15.7:0/3216149781 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c035040 con 0x7f9618073c70 2024-09-10T21:43:19.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.240+0000 7f9620200640 1 -- 172.21.15.7:0/3216149781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96181f00e0 con 0x7f9618073c70 2024-09-10T21:43:19.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.240+0000 7f9620200640 1 -- 172.21.15.7:0/3216149781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9618155f60 con 0x7f9618073c70 2024-09-10T21:43:19.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.241+0000 7f9620200640 1 -- 172.21.15.7:0/3216149781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9618070410 con 0x7f9618073c70 2024-09-10T21:43:19.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.242+0000 7f9615600640 1 -- 172.21.15.7:0/3216149781 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f960c004b70 con 0x7f9618073c70 2024-09-10T21:43:19.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.243+0000 7f9615600640 1 --2- 172.21.15.7:0/3216149781 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f95f8077fa0 0x7f95f807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:19.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.243+0000 7f9615600640 1 -- 172.21.15.7:0/3216149781 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f960c0ba7f0 con 0x7f9618073c70 2024-09-10T21:43:19.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.243+0000 7f961ce00640 1 --2- 172.21.15.7:0/3216149781 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f95f8077fa0 0x7f95f807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:19.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.244+0000 7f961ce00640 1 --2- 172.21.15.7:0/3216149781 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f95f8077fa0 0x7f95f807a460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9604009930 tx=0x7f9604008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:19.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.247+0000 7f9615600640 1 -- 172.21.15.7:0/3216149781 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f960c082c00 con 0x7f9618073c70 2024-09-10T21:43:19.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.478+0000 7f9620200640 1 -- 172.21.15.7:0/3216149781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9618156130 con 0x7f9618073c70 2024-09-10T21:43:19.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.479+0000 7f9615600640 1 -- 172.21.15.7:0/3216149781 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5221 (secure 0 0 0) 0x7f960c087630 con 0x7f9618073c70 2024-09-10T21:43:19.480 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:19.480 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"btime":"2024-09-10T21:42:58:925871+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/923954733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":923954733},{"type":"v1","addr":"172.21.15.133:6837","nonce":923954733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/13426364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":13426364},{"type":"v1","addr":"172.21.15.7:6835","nonce":13426364}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2275341781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2275341781},{"type":"v1","addr":"172.21.15.7:6834","nonce":2275341781}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:42:55.338104+0000","modified":"2024-09-10T21:42:58.362130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234524":{"gid":234524,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6836/849012248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":849012248},{"type":"v1","addr":"172.21.15.133:6836","nonce":849012248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234524,"qdb_cluster":[234524]},"id":2}]} 2024-09-10T21:43:19.480 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-09-10T21:43:19.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.481+0000 7f9620200640 1 -- 172.21.15.7:0/3216149781 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f95f8077fa0 msgr2=0x7f95f807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:19.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.481+0000 7f9620200640 1 --2- 172.21.15.7:0/3216149781 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f95f8077fa0 0x7f95f807a460 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9604009930 tx=0x7f9604008040 comp rx=0 tx=0).stop 2024-09-10T21:43:19.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.481+0000 7f9620200640 1 -- 172.21.15.7:0/3216149781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9618073c70 msgr2=0x7f96180c73c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:19.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.481+0000 7f9620200640 1 --2- 172.21.15.7:0/3216149781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9618073c70 0x7f96180c73c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f960c034040 tx=0x7f960c0039e0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.481+0000 7f9620200640 1 -- 172.21.15.7:0/3216149781 shutdown_connections 2024-09-10T21:43:19.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.481+0000 7f9620200640 1 --2- 172.21.15.7:0/3216149781 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f95f8077fa0 0x7f95f807a460 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.481+0000 7f9620200640 1 --2- 172.21.15.7:0/3216149781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f961815bc90 0x7f96181efba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.481+0000 7f9620200640 1 --2- 172.21.15.7:0/3216149781 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96180cd750 0x7f96181f3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.481+0000 7f9620200640 1 --2- 172.21.15.7:0/3216149781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9618073c70 0x7f96180c73c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.481+0000 7f9620200640 1 -- 172.21.15.7:0/3216149781 >> 172.21.15.7:0/3216149781 conn(0x7f96180b16d0 msgr2=0x7f96180c5de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:19.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.481+0000 7f9620200640 1 -- 172.21.15.7:0/3216149781 shutdown_connections 2024-09-10T21:43:19.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.481+0000 7f9620200640 1 -- 172.21.15.7:0/3216149781 wait complete. 2024-09-10T21:43:19.499 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:43:19.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.658+0000 7f4c87e00640 1 -- 172.21.15.7:0/3029396459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c880d15a0 msgr2=0x7f4c880ccce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:19.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.658+0000 7f4c87e00640 1 --2- 172.21.15.7:0/3029396459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c880d15a0 0x7f4c880ccce0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f4c74009900 tx=0x7f4c7402a4d0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.659+0000 7f4c87e00640 1 -- 172.21.15.7:0/3029396459 shutdown_connections 2024-09-10T21:43:19.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.659+0000 7f4c87e00640 1 --2- 172.21.15.7:0/3029396459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c880d15a0 0x7f4c880ccce0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.659+0000 7f4c87e00640 1 --2- 172.21.15.7:0/3029396459 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c8815bdd0 0x7f4c880cc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.659+0000 7f4c87e00640 1 -- 172.21.15.7:0/3029396459 >> 172.21.15.7:0/3029396459 conn(0x7f4c880d0ab0 msgr2=0x7f4c880d0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:19.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.659+0000 7f4c87e00640 1 -- 172.21.15.7:0/3029396459 shutdown_connections 2024-09-10T21:43:19.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.659+0000 7f4c87e00640 1 -- 172.21.15.7:0/3029396459 wait complete. 2024-09-10T21:43:19.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.660+0000 7f4c87e00640 1 Processor -- start 2024-09-10T21:43:19.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.660+0000 7f4c87e00640 1 -- start start 2024-09-10T21:43:19.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.661+0000 7f4c87e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c8806f590 0x7f4c88158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:19.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.661+0000 7f4c87e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c880d15a0 0x7f4c880b0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:19.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.661+0000 7f4c87e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c8815bdd0 0x7f4c880b0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:19.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.661+0000 7f4c87e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4c880cb280 con 0x7f4c8815bdd0 2024-09-10T21:43:19.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.661+0000 7f4c87e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4c880cb100 con 0x7f4c880d15a0 2024-09-10T21:43:19.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.661+0000 7f4c87e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4c88070e90 con 0x7f4c8806f590 2024-09-10T21:43:19.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.662+0000 7f4c87400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c8815bdd0 0x7f4c880b0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:19.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.662+0000 7f4c87400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c8815bdd0 0x7f4c880b0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55614/0 (socket says 172.21.15.7:55614) 2024-09-10T21:43:19.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.662+0000 7f4c87400640 1 -- 172.21.15.7:0/2464300253 learned_addr learned my addr 172.21.15.7:0/2464300253 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:19.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.662+0000 7f4c86a00640 1 --2- 172.21.15.7:0/2464300253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c8806f590 0x7f4c88158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:19.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.662+0000 7f4c86000640 1 --2- 172.21.15.7:0/2464300253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c880d15a0 0x7f4c880b0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:19.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.662+0000 7f4c87400640 1 -- 172.21.15.7:0/2464300253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c8806f590 msgr2=0x7f4c88158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:19.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.662+0000 7f4c87400640 1 --2- 172.21.15.7:0/2464300253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c8806f590 0x7f4c88158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.662+0000 7f4c87400640 1 -- 172.21.15.7:0/2464300253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c880d15a0 msgr2=0x7f4c880b0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:19.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.662+0000 7f4c87400640 1 --2- 172.21.15.7:0/2464300253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c880d15a0 0x7f4c880b0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.662+0000 7f4c87400640 1 -- 172.21.15.7:0/2464300253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c881f81c0 con 0x7f4c8815bdd0 2024-09-10T21:43:19.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.662+0000 7f4c86000640 1 --2- 172.21.15.7:0/2464300253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c880d15a0 0x7f4c880b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:43:19.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.662+0000 7f4c87400640 1 --2- 172.21.15.7:0/2464300253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c8815bdd0 0x7f4c880b0fa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000c780 tx=0x7f4c8000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:19.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.662+0000 7f4c7ea00640 1 -- 172.21.15.7:0/2464300253 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80017070 con 0x7f4c8815bdd0 2024-09-10T21:43:19.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.663+0000 7f4c7ea00640 1 -- 172.21.15.7:0/2464300253 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c80005070 con 0x7f4c8815bdd0 2024-09-10T21:43:19.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.663+0000 7f4c7ea00640 1 -- 172.21.15.7:0/2464300253 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c800054c0 con 0x7f4c8815bdd0 2024-09-10T21:43:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.663+0000 7f4c87e00640 1 -- 172.21.15.7:0/2464300253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c881f84a0 con 0x7f4c8815bdd0 2024-09-10T21:43:19.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.663+0000 7f4c87e00640 1 -- 172.21.15.7:0/2464300253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c881f8a60 con 0x7f4c8815bdd0 2024-09-10T21:43:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.664+0000 7f4c87e00640 1 -- 172.21.15.7:0/2464300253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c880cc7a0 con 0x7f4c8815bdd0 2024-09-10T21:43:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.665+0000 7f4c7ea00640 1 -- 172.21.15.7:0/2464300253 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4c80005660 con 0x7f4c8815bdd0 2024-09-10T21:43:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.666+0000 7f4c7ea00640 1 --2- 172.21.15.7:0/2464300253 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c68077f70 0x7f4c6807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.666+0000 7f4c7ea00640 1 -- 172.21.15.7:0/2464300253 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f4c80099340 con 0x7f4c8815bdd0 2024-09-10T21:43:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.666+0000 7f4c86a00640 1 --2- 172.21.15.7:0/2464300253 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c68077f70 0x7f4c6807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.667+0000 7f4c86a00640 1 --2- 172.21.15.7:0/2464300253 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c68077f70 0x7f4c6807a430 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800b810 tx=0x7f4c78008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:19.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.670+0000 7f4c7ea00640 1 -- 172.21.15.7:0/2464300253 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4c80061db0 con 0x7f4c8815bdd0 2024-09-10T21:43:19.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.900+0000 7f4c87e00640 1 -- 172.21.15.7:0/2464300253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c88070610 con 0x7f4c8815bdd0 2024-09-10T21:43:19.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.901+0000 7f4c7ea00640 1 -- 172.21.15.7:0/2464300253 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5221 (secure 0 0 0) 0x7f4c800616b0 con 0x7f4c8815bdd0 2024-09-10T21:43:19.902 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:19.902 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":17,"btime":"2024-09-10T21:42:58:925871+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/923954733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":923954733},{"type":"v1","addr":"172.21.15.133:6837","nonce":923954733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/13426364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":13426364},{"type":"v1","addr":"172.21.15.7:6835","nonce":13426364}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2275341781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2275341781},{"type":"v1","addr":"172.21.15.7:6834","nonce":2275341781}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:42:55.338104+0000","modified":"2024-09-10T21:42:58.362130+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234524},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234524":{"gid":234524,"name":"b","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6836/849012248","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":849012248},{"type":"v1","addr":"172.21.15.133:6836","nonce":849012248}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234524,"qdb_cluster":[234524]},"id":2}]} 2024-09-10T21:43:19.903 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 17 2024-09-10T21:43:19.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.903+0000 7f4c87e00640 1 -- 172.21.15.7:0/2464300253 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c68077f70 msgr2=0x7f4c6807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:19.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.903+0000 7f4c87e00640 1 --2- 172.21.15.7:0/2464300253 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c68077f70 0x7f4c6807a430 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f4c7800b810 tx=0x7f4c78008040 comp rx=0 tx=0).stop 2024-09-10T21:43:19.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.903+0000 7f4c87e00640 1 -- 172.21.15.7:0/2464300253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c8815bdd0 msgr2=0x7f4c880b0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:19.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.903+0000 7f4c87e00640 1 --2- 172.21.15.7:0/2464300253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c8815bdd0 0x7f4c880b0fa0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000c780 tx=0x7f4c8000cc40 comp rx=0 tx=0).stop 2024-09-10T21:43:19.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.903+0000 7f4c87e00640 1 -- 172.21.15.7:0/2464300253 shutdown_connections 2024-09-10T21:43:19.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.903+0000 7f4c87e00640 1 --2- 172.21.15.7:0/2464300253 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c68077f70 0x7f4c6807a430 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.903+0000 7f4c87e00640 1 --2- 172.21.15.7:0/2464300253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4c880d15a0 0x7f4c880b0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.903+0000 7f4c87e00640 1 --2- 172.21.15.7:0/2464300253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4c8815bdd0 0x7f4c880b0fa0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.903+0000 7f4c87e00640 1 --2- 172.21.15.7:0/2464300253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4c8806f590 0x7f4c88158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:19.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.903+0000 7f4c87e00640 1 -- 172.21.15.7:0/2464300253 >> 172.21.15.7:0/2464300253 conn(0x7f4c880d0ab0 msgr2=0x7f4c880b5130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:19.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.903+0000 7f4c87e00640 1 -- 172.21.15.7:0/2464300253 shutdown_connections 2024-09-10T21:43:19.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:19.903+0000 7f4c87e00640 1 -- 172.21.15.7:0/2464300253 wait complete. 2024-09-10T21:43:19.921 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:43:20.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.080+0000 7feca6600640 1 -- 172.21.15.7:0/300674515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feca00cbd50 msgr2=0x7feca00cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:20.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.080+0000 7feca6600640 1 --2- 172.21.15.7:0/300674515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feca00cbd50 0x7feca00cc1c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7feca00d0c80 tx=0x7fec8802a560 comp rx=0 tx=0).stop 2024-09-10T21:43:20.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.080+0000 7feca6600640 1 -- 172.21.15.7:0/300674515 shutdown_connections 2024-09-10T21:43:20.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.080+0000 7feca6600640 1 --2- 172.21.15.7:0/300674515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feca00cbd50 0x7feca00cc1c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:20.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.080+0000 7feca6600640 1 --2- 172.21.15.7:0/300674515 >> v2:172.21.15.133:3300/0 conn(0x7feca01619e0 0x7feca015e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:20.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.080+0000 7feca6600640 1 -- 172.21.15.7:0/300674515 >> 172.21.15.7:0/300674515 conn(0x7feca00b16d0 msgr2=0x7feca00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:20.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.080+0000 7feca6600640 1 -- 172.21.15.7:0/300674515 shutdown_connections 2024-09-10T21:43:20.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.080+0000 7feca6600640 1 -- 172.21.15.7:0/300674515 wait complete. 2024-09-10T21:43:20.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.082+0000 7feca6600640 1 Processor -- start 2024-09-10T21:43:20.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.082+0000 7feca6600640 1 -- start start 2024-09-10T21:43:20.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.083+0000 7feca6600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feca00ae020 0x7feca00c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.083+0000 7feca6600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feca00cbd50 0x7feca0149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.083+0000 7feca6600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feca01619e0 0x7feca006eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.083+0000 7feca6600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feca00ae890 con 0x7feca00ae020 2024-09-10T21:43:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.083+0000 7feca6600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feca00ae710 con 0x7feca00cbd50 2024-09-10T21:43:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.083+0000 7feca6600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7feca0161dc0 con 0x7feca01619e0 2024-09-10T21:43:20.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.083+0000 7fec9f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feca00ae020 0x7feca00c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:20.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.083+0000 7fec9f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feca00ae020 0x7feca00c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55622/0 (socket says 172.21.15.7:55622) 2024-09-10T21:43:20.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.083+0000 7fec9f400640 1 -- 172.21.15.7:0/534247585 learned_addr learned my addr 172.21.15.7:0/534247585 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:20.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.083+0000 7fec9fe00640 1 --2- 172.21.15.7:0/534247585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feca01619e0 0x7feca006eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:20.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.084+0000 7fec97e00640 1 --2- 172.21.15.7:0/534247585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feca00cbd50 0x7feca0149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:20.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.084+0000 7fec9f400640 1 -- 172.21.15.7:0/534247585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feca01619e0 msgr2=0x7feca006eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:20.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.084+0000 7fec9f400640 1 --2- 172.21.15.7:0/534247585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feca01619e0 0x7feca006eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:20.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.084+0000 7fec9f400640 1 -- 172.21.15.7:0/534247585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feca00cbd50 msgr2=0x7feca0149f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:20.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.084+0000 7fec9f400640 1 --2- 172.21.15.7:0/534247585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feca00cbd50 0x7feca0149f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:20.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.084+0000 7fec9f400640 1 -- 172.21.15.7:0/534247585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca00c83e0 con 0x7feca00ae020 2024-09-10T21:43:20.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.084+0000 7fec9f400640 1 --2- 172.21.15.7:0/534247585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feca00ae020 0x7feca00c9480 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fec9000d770 tx=0x7fec9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:20.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.084+0000 7fec97400640 1 -- 172.21.15.7:0/534247585 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90017070 con 0x7feca00ae020 2024-09-10T21:43:20.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.084+0000 7fec97400640 1 -- 172.21.15.7:0/534247585 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec90009d20 con 0x7feca00ae020 2024-09-10T21:43:20.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.084+0000 7fec97400640 1 -- 172.21.15.7:0/534247585 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec90005280 con 0x7feca00ae020 2024-09-10T21:43:20.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.084+0000 7feca6600640 1 -- 172.21.15.7:0/534247585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca00c86c0 con 0x7feca00ae020 2024-09-10T21:43:20.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.085+0000 7feca6600640 1 -- 172.21.15.7:0/534247585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca00c8c80 con 0x7feca00ae020 2024-09-10T21:43:20.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.086+0000 7feca6600640 1 -- 172.21.15.7:0/534247585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feca00cc1c0 con 0x7feca00ae020 2024-09-10T21:43:20.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.087+0000 7fec97400640 1 -- 172.21.15.7:0/534247585 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fec90005420 con 0x7feca00ae020 2024-09-10T21:43:20.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.088+0000 7fec97400640 1 --2- 172.21.15.7:0/534247585 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fec7c077f70 0x7fec7c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:20.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.088+0000 7fec97400640 1 -- 172.21.15.7:0/534247585 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7fec90099f40 con 0x7feca00ae020 2024-09-10T21:43:20.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.089+0000 7fec97e00640 1 --2- 172.21.15.7:0/534247585 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fec7c077f70 0x7fec7c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:20.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.089+0000 7fec97e00640 1 --2- 172.21.15.7:0/534247585 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fec7c077f70 0x7fec7c07a430 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fec88033000 tx=0x7fec88004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:20.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.093+0000 7fec97400640 1 -- 172.21.15.7:0/534247585 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fec90062a30 con 0x7feca00ae020 2024-09-10T21:43:20.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.287+0000 7feca6600640 1 -- 172.21.15.7:0/534247585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feca00b5b00 con 0x7feca00ae020 2024-09-10T21:43:20.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.288+0000 7fec97400640 1 -- 172.21.15.7:0/534247585 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v32) ==== 74+0+16693 (secure 0 0 0) 0x7fec9006b150 con 0x7feca00ae020 2024-09-10T21:43:20.290 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:20.290 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":32,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:42:56.911349+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:42:52.896912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"31","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-10T21:42:54.300403+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"32","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:43:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.290+0000 7feca6600640 1 -- 172.21.15.7:0/534247585 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fec7c077f70 msgr2=0x7fec7c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.290+0000 7feca6600640 1 --2- 172.21.15.7:0/534247585 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fec7c077f70 0x7fec7c07a430 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fec88033000 tx=0x7fec88004730 comp rx=0 tx=0).stop 2024-09-10T21:43:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.290+0000 7feca6600640 1 -- 172.21.15.7:0/534247585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feca00ae020 msgr2=0x7feca00c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.290+0000 7feca6600640 1 --2- 172.21.15.7:0/534247585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feca00ae020 0x7feca00c9480 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fec9000d770 tx=0x7fec9000dc30 comp rx=0 tx=0).stop 2024-09-10T21:43:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.290+0000 7feca6600640 1 -- 172.21.15.7:0/534247585 shutdown_connections 2024-09-10T21:43:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.290+0000 7feca6600640 1 --2- 172.21.15.7:0/534247585 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fec7c077f70 0x7fec7c07a430 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.290+0000 7feca6600640 1 --2- 172.21.15.7:0/534247585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feca00cbd50 0x7feca0149f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.290+0000 7feca6600640 1 --2- 172.21.15.7:0/534247585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feca01619e0 0x7feca006eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:20.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.290+0000 7feca6600640 1 --2- 172.21.15.7:0/534247585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feca00ae020 0x7feca00c9480 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.290+0000 7feca6600640 1 -- 172.21.15.7:0/534247585 >> 172.21.15.7:0/534247585 conn(0x7feca00b16d0 msgr2=0x7feca0072360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.290+0000 7feca6600640 1 -- 172.21.15.7:0/534247585 shutdown_connections 2024-09-10T21:43:20.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.290+0000 7feca6600640 1 -- 172.21.15.7:0/534247585 wait complete. 2024-09-10T21:43:20.308 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:43:20.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.468+0000 7f888f200640 1 -- 172.21.15.7:0/1654829501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88900d2a20 msgr2=0x7f88900d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.468+0000 7f888f200640 1 --2- 172.21.15.7:0/1654829501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88900d2a20 0x7f88900d4900 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f887c009830 tx=0x7f887c02a640 comp rx=0 tx=0).stop 2024-09-10T21:43:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.468+0000 7f888f200640 1 -- 172.21.15.7:0/1654829501 shutdown_connections 2024-09-10T21:43:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.468+0000 7f888f200640 1 --2- 172.21.15.7:0/1654829501 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88900d2a20 0x7f88900d4900 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.468+0000 7f888f200640 1 -- 172.21.15.7:0/1654829501 >> 172.21.15.7:0/1654829501 conn(0x7f88900d3de0 msgr2=0x7f88900d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.468+0000 7f888f200640 1 -- 172.21.15.7:0/1654829501 shutdown_connections 2024-09-10T21:43:20.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.468+0000 7f888f200640 1 -- 172.21.15.7:0/1654829501 wait complete. 2024-09-10T21:43:20.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.470+0000 7f888f200640 1 Processor -- start 2024-09-10T21:43:20.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.470+0000 7f888f200640 1 -- start start 2024-09-10T21:43:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.470+0000 7f888f200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88900c82d0 0x7f88901f8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.471+0000 7f888f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88900d1e60 0x7f88901f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.471+0000 7f888f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88900d2a20 0x7f889015d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.471+0000 7f888f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f88900c9220 con 0x7f88900d1e60 2024-09-10T21:43:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.471+0000 7f888f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f88900c90a0 con 0x7f88900d2a20 2024-09-10T21:43:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.471+0000 7f888f200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f88900c93a0 con 0x7f88900c82d0 2024-09-10T21:43:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.471+0000 7f888d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88900d1e60 0x7f88901f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.471+0000 7f888d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88900d1e60 0x7f88901f4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55634/0 (socket says 172.21.15.7:55634) 2024-09-10T21:43:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.471+0000 7f888d400640 1 -- 172.21.15.7:0/1508556908 learned_addr learned my addr 172.21.15.7:0/1508556908 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.471+0000 7f888de00640 1 --2- 172.21.15.7:0/1508556908 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88900c82d0 0x7f88901f8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.471+0000 7f888e800640 1 --2- 172.21.15.7:0/1508556908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88900d2a20 0x7f889015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.471+0000 7f888d400640 1 -- 172.21.15.7:0/1508556908 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88900c82d0 msgr2=0x7f88901f8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.471+0000 7f888d400640 1 --2- 172.21.15.7:0/1508556908 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88900c82d0 0x7f88901f8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.471+0000 7f888d400640 1 -- 172.21.15.7:0/1508556908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88900d2a20 msgr2=0x7f889015d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:20.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.471+0000 7f888d400640 1 --2- 172.21.15.7:0/1508556908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88900d2a20 0x7f889015d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:20.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.471+0000 7f888d400640 1 -- 172.21.15.7:0/1508556908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889015d8a0 con 0x7f88900d1e60 2024-09-10T21:43:20.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.471+0000 7f888e800640 1 --2- 172.21.15.7:0/1508556908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88900d2a20 0x7f889015d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:20.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.472+0000 7f888d400640 1 --2- 172.21.15.7:0/1508556908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88900d1e60 0x7f88901f4bd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f887c000c00 tx=0x7f887c033df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:20.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.472+0000 7f8886000640 1 -- 172.21.15.7:0/1508556908 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c0332e0 con 0x7f88900d1e60 2024-09-10T21:43:20.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.472+0000 7f8886000640 1 -- 172.21.15.7:0/1508556908 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f887c033570 con 0x7f88900d1e60 2024-09-10T21:43:20.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.472+0000 7f8886000640 1 -- 172.21.15.7:0/1508556908 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f887c031a10 con 0x7f88900d1e60 2024-09-10T21:43:20.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.472+0000 7f888f200640 1 -- 172.21.15.7:0/1508556908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88901f5340 con 0x7f88900d1e60 2024-09-10T21:43:20.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.472+0000 7f888f200640 1 -- 172.21.15.7:0/1508556908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88901f5900 con 0x7f88900d1e60 2024-09-10T21:43:20.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.473+0000 7f888f200640 1 -- 172.21.15.7:0/1508556908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88900751b0 con 0x7f88900d1e60 2024-09-10T21:43:20.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.474+0000 7f8886000640 1 -- 172.21.15.7:0/1508556908 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f887c031bb0 con 0x7f88900d1e60 2024-09-10T21:43:20.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.475+0000 7f8886000640 1 --2- 172.21.15.7:0/1508556908 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8870077f70 0x7f887007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:20.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.476+0000 7f8886000640 1 -- 172.21.15.7:0/1508556908 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5709+0+0 (secure 0 0 0) 0x7f887c0bafe0 con 0x7f88900d1e60 2024-09-10T21:43:20.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.476+0000 7f888de00640 1 --2- 172.21.15.7:0/1508556908 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8870077f70 0x7f887007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:20.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.476+0000 7f888de00640 1 --2- 172.21.15.7:0/1508556908 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8870077f70 0x7f887007a430 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8880007fd0 tx=0x7f8880009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:20.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.480+0000 7f8886000640 1 -- 172.21.15.7:0/1508556908 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f887c03f030 con 0x7f88900d1e60 2024-09-10T21:43:20.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.711+0000 7f888f200640 1 -- 172.21.15.7:0/1508556908 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f88900d05d0 con 0x7f88900d1e60 2024-09-10T21:43:20.713 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-10T21:43:20.712+0000 7feb3fa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:43:20.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.714+0000 7f8886000640 1 -- 172.21.15.7:0/1508556908 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) ==== 161+0+0 (secure 0 0 0) 0x7f887c083ad0 con 0x7f88900d1e60 2024-09-10T21:43:20.715 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:43:20.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.717+0000 7f888f200640 1 -- 172.21.15.7:0/1508556908 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8870077f70 msgr2=0x7f887007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:20.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.717+0000 7f888f200640 1 --2- 172.21.15.7:0/1508556908 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8870077f70 0x7f887007a430 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8880007fd0 tx=0x7f8880009040 comp rx=0 tx=0).stop 2024-09-10T21:43:20.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.717+0000 7f888f200640 1 -- 172.21.15.7:0/1508556908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88900d1e60 msgr2=0x7f88901f4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:20.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.717+0000 7f888f200640 1 --2- 172.21.15.7:0/1508556908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88900d1e60 0x7f88901f4bd0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f887c000c00 tx=0x7f887c033df0 comp rx=0 tx=0).stop 2024-09-10T21:43:20.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.717+0000 7f888f200640 1 -- 172.21.15.7:0/1508556908 shutdown_connections 2024-09-10T21:43:20.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.717+0000 7f888f200640 1 --2- 172.21.15.7:0/1508556908 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8870077f70 0x7f887007a430 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:20.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.717+0000 7f888f200640 1 --2- 172.21.15.7:0/1508556908 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f88900d2a20 0x7f889015d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:20.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.717+0000 7f888f200640 1 --2- 172.21.15.7:0/1508556908 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f88900d1e60 0x7f88901f4bd0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:20.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.717+0000 7f888f200640 1 --2- 172.21.15.7:0/1508556908 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f88900c82d0 0x7f88901f8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:20.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.717+0000 7f888f200640 1 -- 172.21.15.7:0/1508556908 >> 172.21.15.7:0/1508556908 conn(0x7f88900d3de0 msgr2=0x7f88900ce9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:20.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.717+0000 7f888f200640 1 -- 172.21.15.7:0/1508556908 shutdown_connections 2024-09-10T21:43:20.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.717+0000 7f888f200640 1 -- 172.21.15.7:0/1508556908 wait complete. 2024-09-10T21:43:20.731 INFO:tasks.ceph.mds.b.smithi133.stderr: -7470> 2024-09-10T21:42:58.861+0000 7fc251400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:43:20.735 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:43:20.783 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2024-09-10T21:43:20.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.896+0000 7fbeee400640 1 -- 172.21.15.7:0/162300337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee8159680 msgr2=0x7fbee815cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:20.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.896+0000 7fbeee400640 1 --2- 172.21.15.7:0/162300337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee8159680 0x7fbee815cc00 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbed8009830 tx=0x7fbed802a480 comp rx=0 tx=0).stop 2024-09-10T21:43:20.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.897+0000 7fbeee400640 1 -- 172.21.15.7:0/162300337 shutdown_connections 2024-09-10T21:43:20.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.897+0000 7fbeee400640 1 --2- 172.21.15.7:0/162300337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee8159680 0x7fbee815cc00 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:20.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.897+0000 7fbeee400640 1 -- 172.21.15.7:0/162300337 >> 172.21.15.7:0/162300337 conn(0x7fbee80b16d0 msgr2=0x7fbee80b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:20.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.897+0000 7fbeee400640 1 -- 172.21.15.7:0/162300337 shutdown_connections 2024-09-10T21:43:20.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.897+0000 7fbeee400640 1 -- 172.21.15.7:0/162300337 wait complete. 2024-09-10T21:43:20.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.898+0000 7fbeee400640 1 Processor -- start 2024-09-10T21:43:20.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.898+0000 7fbeee400640 1 -- start start 2024-09-10T21:43:20.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.899+0000 7fbeee400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbee806e580 0x7fbee814be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:20.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.899+0000 7fbeee400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbee8073e20 0x7fbee81481d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:20.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.899+0000 7fbeee400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee8159680 0x7fbee80cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:20.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.899+0000 7fbeee400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbee80cb180 con 0x7fbee8159680 2024-09-10T21:43:20.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.899+0000 7fbeee400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbee80cb000 con 0x7fbee8073e20 2024-09-10T21:43:20.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.899+0000 7fbeee400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fbee80742d0 con 0x7fbee806e580 2024-09-10T21:43:20.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.900+0000 7fbee7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee8159680 0x7fbee80cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:20.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.900+0000 7fbee7400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbee806e580 0x7fbee814be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:20.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.900+0000 7fbedea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbee8073e20 0x7fbee81481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:20.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.900+0000 7fbee7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee8159680 0x7fbee80cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55642/0 (socket says 172.21.15.7:55642) 2024-09-10T21:43:20.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.900+0000 7fbee7400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbee806e580 0x7fbee814be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:55072/0 (socket says 172.21.15.7:55072) 2024-09-10T21:43:20.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.900+0000 7fbee7e00640 1 -- 172.21.15.7:0/3550264772 learned_addr learned my addr 172.21.15.7:0/3550264772 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:20.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.900+0000 7fbedea00640 1 -- 172.21.15.7:0/3550264772 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbee806e580 msgr2=0x7fbee814be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:20.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.900+0000 7fbedea00640 1 --2- 172.21.15.7:0/3550264772 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbee806e580 0x7fbee814be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:20.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.900+0000 7fbedea00640 1 -- 172.21.15.7:0/3550264772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee8159680 msgr2=0x7fbee80cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:20.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.900+0000 7fbedea00640 1 --2- 172.21.15.7:0/3550264772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee8159680 0x7fbee80cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:20.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.900+0000 7fbedea00640 1 -- 172.21.15.7:0/3550264772 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbee8148710 con 0x7fbee8073e20 2024-09-10T21:43:20.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.900+0000 7fbee7e00640 1 --2- 172.21.15.7:0/3550264772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee8159680 0x7fbee80cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:43:20.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.901+0000 7fbedea00640 1 --2- 172.21.15.7:0/3550264772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbee8073e20 0x7fbee81481d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbed000c750 tx=0x7fbed000cc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:20.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.901+0000 7fbedfe00640 1 -- 172.21.15.7:0/3550264772 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed0016020 con 0x7fbee8073e20 2024-09-10T21:43:20.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.901+0000 7fbedfe00640 1 -- 172.21.15.7:0/3550264772 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbed000a850 con 0x7fbee8073e20 2024-09-10T21:43:20.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.901+0000 7fbedfe00640 1 -- 172.21.15.7:0/3550264772 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbed000ace0 con 0x7fbee8073e20 2024-09-10T21:43:20.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.901+0000 7fbeee400640 1 -- 172.21.15.7:0/3550264772 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbee81522f0 con 0x7fbee8073e20 2024-09-10T21:43:20.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.901+0000 7fbeee400640 1 -- 172.21.15.7:0/3550264772 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbee81528b0 con 0x7fbee8073e20 2024-09-10T21:43:20.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.902+0000 7fbeee400640 1 -- 172.21.15.7:0/3550264772 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbee80747b0 con 0x7fbee8073e20 2024-09-10T21:43:20.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.903+0000 7fbedfe00640 1 -- 172.21.15.7:0/3550264772 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fbed001a3b0 con 0x7fbee8073e20 2024-09-10T21:43:20.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.904+0000 7fbedfe00640 1 --2- 172.21.15.7:0/3550264772 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbec4077fa0 0x7fbec407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:20.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.904+0000 7fbedfe00640 1 -- 172.21.15.7:0/3550264772 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5795+0+0 (secure 0 0 0) 0x7fbed0099ba0 con 0x7fbee8073e20 2024-09-10T21:43:20.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.904+0000 7fbee7400640 1 --2- 172.21.15.7:0/3550264772 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbec4077fa0 0x7fbec407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:20.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.905+0000 7fbee7400640 1 --2- 172.21.15.7:0/3550264772 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbec4077fa0 0x7fbec407a460 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbed8034000 tx=0x7fbed8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:20.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:20.909+0000 7fbedfe00640 1 -- 172.21.15.7:0/3550264772 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fbed0020070 con 0x7fbee8073e20 2024-09-10T21:43:21.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.140+0000 7fbeee400640 1 -- 172.21.15.7:0/3550264772 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fbee815a1e0 con 0x7fbee8073e20 2024-09-10T21:43:21.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.719+0000 7fbedfe00640 1 -- 172.21.15.7:0/3550264772 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) ==== 104+0+0 (secure 0 0 0) 0x7fbee8148710 con 0x7fbee8073e20 2024-09-10T21:43:21.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.722+0000 7fbeee400640 1 -- 172.21.15.7:0/3550264772 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbec4077fa0 msgr2=0x7fbec407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:21.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.722+0000 7fbeee400640 1 --2- 172.21.15.7:0/3550264772 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbec4077fa0 0x7fbec407a460 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbed8034000 tx=0x7fbed8004730 comp rx=0 tx=0).stop 2024-09-10T21:43:21.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.722+0000 7fbeee400640 1 -- 172.21.15.7:0/3550264772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbee8073e20 msgr2=0x7fbee81481d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:21.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.722+0000 7fbeee400640 1 --2- 172.21.15.7:0/3550264772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbee8073e20 0x7fbee81481d0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbed000c750 tx=0x7fbed000cc10 comp rx=0 tx=0).stop 2024-09-10T21:43:21.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.722+0000 7fbeee400640 1 -- 172.21.15.7:0/3550264772 shutdown_connections 2024-09-10T21:43:21.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.722+0000 7fbeee400640 1 --2- 172.21.15.7:0/3550264772 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbec4077fa0 0x7fbec407a460 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:21.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.722+0000 7fbeee400640 1 --2- 172.21.15.7:0/3550264772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbee8073e20 0x7fbee81481d0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:21.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.722+0000 7fbeee400640 1 --2- 172.21.15.7:0/3550264772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee8159680 0x7fbee80cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:21.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.722+0000 7fbeee400640 1 --2- 172.21.15.7:0/3550264772 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbee806e580 0x7fbee814be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:21.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.722+0000 7fbeee400640 1 -- 172.21.15.7:0/3550264772 >> 172.21.15.7:0/3550264772 conn(0x7fbee80b16d0 msgr2=0x7fbee80b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:21.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.722+0000 7fbeee400640 1 -- 172.21.15.7:0/3550264772 shutdown_connections 2024-09-10T21:43:21.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.722+0000 7fbeee400640 1 -- 172.21.15.7:0/3550264772 wait complete. 2024-09-10T21:43:21.741 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:43:21.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.909+0000 7f8721a00640 1 -- 172.21.15.7:0/591045113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700004800 msgr2=0x7f8700004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:21.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.909+0000 7f8721a00640 1 --2- 172.21.15.7:0/591045113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700004800 0x7f8700004c80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f8708009b20 tx=0x7f8700005830 comp rx=0 tx=0).stop 2024-09-10T21:43:21.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.909+0000 7f8721a00640 1 -- 172.21.15.7:0/591045113 shutdown_connections 2024-09-10T21:43:21.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.909+0000 7f8721a00640 1 --2- 172.21.15.7:0/591045113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700004800 0x7f8700004c80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:21.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.909+0000 7f8721a00640 1 -- 172.21.15.7:0/591045113 >> 172.21.15.7:0/591045113 conn(0x7f871c0b1c10 msgr2=0x7f871c0b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:21.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.909+0000 7f8721a00640 1 -- 172.21.15.7:0/591045113 shutdown_connections 2024-09-10T21:43:21.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.909+0000 7f8721a00640 1 -- 172.21.15.7:0/591045113 wait complete. 2024-09-10T21:43:21.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.911+0000 7f8721a00640 1 Processor -- start 2024-09-10T21:43:21.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.911+0000 7f8721a00640 1 -- start start 2024-09-10T21:43:21.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.912+0000 7f8721a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700004800 0x7f871c0733b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:21.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.912+0000 7f8721a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f871c072800 0x7f871c09b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:21.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.912+0000 7f8721a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f871c09b6b0 0x7f871c156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:21.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.912+0000 7f8721a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f871c15f8b0 con 0x7f8700004800 2024-09-10T21:43:21.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.912+0000 7f8721a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f871c15f730 con 0x7f871c072800 2024-09-10T21:43:21.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.912+0000 7f8721a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f871c15fa30 con 0x7f871c09b6b0 2024-09-10T21:43:21.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.912+0000 7f871aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700004800 0x7f871c0733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:21.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.912+0000 7f871aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700004800 0x7f871c0733b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55670/0 (socket says 172.21.15.7:55670) 2024-09-10T21:43:21.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.912+0000 7f871aa00640 1 -- 172.21.15.7:0/562937527 learned_addr learned my addr 172.21.15.7:0/562937527 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:21.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.912+0000 7f871b400640 1 --2- 172.21.15.7:0/562937527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f871c09b6b0 0x7f871c156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:21.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.912+0000 7f871a000640 1 --2- 172.21.15.7:0/562937527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f871c072800 0x7f871c09b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:21.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.912+0000 7f871aa00640 1 -- 172.21.15.7:0/562937527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f871c09b6b0 msgr2=0x7f871c156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:21.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.912+0000 7f871aa00640 1 --2- 172.21.15.7:0/562937527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f871c09b6b0 0x7f871c156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:21.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.912+0000 7f871aa00640 1 -- 172.21.15.7:0/562937527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f871c072800 msgr2=0x7f871c09b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:21.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.912+0000 7f871aa00640 1 --2- 172.21.15.7:0/562937527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f871c072800 0x7f871c09b170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:21.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.912+0000 7f871aa00640 1 -- 172.21.15.7:0/562937527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871c1f5ab0 con 0x7f8700004800 2024-09-10T21:43:21.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.912+0000 7f871a000640 1 --2- 172.21.15.7:0/562937527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f871c072800 0x7f871c09b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:21.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.913+0000 7f871aa00640 1 --2- 172.21.15.7:0/562937527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700004800 0x7f871c0733b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f870c006660 tx=0x7f870c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:21.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.913+0000 7f8712a00640 1 -- 172.21.15.7:0/562937527 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c0117c0 con 0x7f8700004800 2024-09-10T21:43:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.913+0000 7f8712a00640 1 -- 172.21.15.7:0/562937527 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f870c016020 con 0x7f8700004800 2024-09-10T21:43:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.913+0000 7f8721a00640 1 -- 172.21.15.7:0/562937527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871c1f4ea0 con 0x7f8700004800 2024-09-10T21:43:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.913+0000 7f8712a00640 1 -- 172.21.15.7:0/562937527 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870c011c20 con 0x7f8700004800 2024-09-10T21:43:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.913+0000 7f8721a00640 1 -- 172.21.15.7:0/562937527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f871c1f5460 con 0x7f8700004800 2024-09-10T21:43:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.914+0000 7f8721a00640 1 -- 172.21.15.7:0/562937527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f871c06cc00 con 0x7f8700004800 2024-09-10T21:43:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.915+0000 7f8712a00640 1 -- 172.21.15.7:0/562937527 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f870c010420 con 0x7f8700004800 2024-09-10T21:43:21.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.916+0000 7f8712a00640 1 --2- 172.21.15.7:0/562937527 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f86fc077fa0 0x7f86fc07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:21.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.916+0000 7f8712a00640 1 -- 172.21.15.7:0/562937527 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5795+0+0 (secure 0 0 0) 0x7f870c09a5e0 con 0x7f8700004800 2024-09-10T21:43:21.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.917+0000 7f871a000640 1 --2- 172.21.15.7:0/562937527 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f86fc077fa0 0x7f86fc07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:21.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.917+0000 7f871a000640 1 --2- 172.21.15.7:0/562937527 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f86fc077fa0 0x7f86fc07a460 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8708035000 tx=0x7f870802d040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:21.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:21.920+0000 7f8712a00640 1 -- 172.21.15.7:0/562937527 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f870c020030 con 0x7f8700004800 2024-09-10T21:43:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.116+0000 7f8721a00640 1 -- 172.21.15.7:0/562937527 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f871c09da00 con 0x7f8700004800 2024-09-10T21:43:22.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.721+0000 7f8712a00640 1 -- 172.21.15.7:0/562937527 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v34) ==== 181+0+0 (secure 0 0 0) 0x7f870c063070 con 0x7f8700004800 2024-09-10T21:43:22.723 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:43:22.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.725+0000 7f8721a00640 1 -- 172.21.15.7:0/562937527 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f86fc077fa0 msgr2=0x7f86fc07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:22.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.725+0000 7f8721a00640 1 --2- 172.21.15.7:0/562937527 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f86fc077fa0 0x7f86fc07a460 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8708035000 tx=0x7f870802d040 comp rx=0 tx=0).stop 2024-09-10T21:43:22.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.725+0000 7f8721a00640 1 -- 172.21.15.7:0/562937527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700004800 msgr2=0x7f871c0733b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:22.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.725+0000 7f8721a00640 1 --2- 172.21.15.7:0/562937527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700004800 0x7f871c0733b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f870c006660 tx=0x7f870c0039c0 comp rx=0 tx=0).stop 2024-09-10T21:43:22.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.725+0000 7f8721a00640 1 -- 172.21.15.7:0/562937527 shutdown_connections 2024-09-10T21:43:22.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.725+0000 7f8721a00640 1 --2- 172.21.15.7:0/562937527 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f86fc077fa0 0x7f86fc07a460 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:22.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.725+0000 7f8721a00640 1 --2- 172.21.15.7:0/562937527 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f871c072800 0x7f871c09b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:22.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.726+0000 7f8721a00640 1 --2- 172.21.15.7:0/562937527 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f871c09b6b0 0x7f871c156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:22.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.726+0000 7f8721a00640 1 --2- 172.21.15.7:0/562937527 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8700004800 0x7f871c0733b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:22.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.726+0000 7f8721a00640 1 -- 172.21.15.7:0/562937527 >> 172.21.15.7:0/562937527 conn(0x7f871c0b1c10 msgr2=0x7f871c1518f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:22.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.726+0000 7f8721a00640 1 -- 172.21.15.7:0/562937527 shutdown_connections 2024-09-10T21:43:22.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.726+0000 7f8721a00640 1 -- 172.21.15.7:0/562937527 wait complete. 2024-09-10T21:43:22.746 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:43:22.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.915+0000 7f1176400640 1 -- 172.21.15.7:0/3679126221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f117015b000 msgr2=0x7f11700cb9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:22.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.915+0000 7f1176400640 1 --2- 172.21.15.7:0/3679126221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f117015b000 0x7f11700cb9d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f11700cadb0 tx=0x7f116002a890 comp rx=0 tx=0).stop 2024-09-10T21:43:22.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.915+0000 7f1176400640 1 -- 172.21.15.7:0/3679126221 shutdown_connections 2024-09-10T21:43:22.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.915+0000 7f1176400640 1 --2- 172.21.15.7:0/3679126221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f117015b000 0x7f11700cb9d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:22.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.915+0000 7f1176400640 1 -- 172.21.15.7:0/3679126221 >> 172.21.15.7:0/3679126221 conn(0x7f11700b1580 msgr2=0x7f11700b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:22.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.915+0000 7f1176400640 1 -- 172.21.15.7:0/3679126221 shutdown_connections 2024-09-10T21:43:22.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.915+0000 7f1176400640 1 -- 172.21.15.7:0/3679126221 wait complete. 2024-09-10T21:43:22.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.917+0000 7f1176400640 1 Processor -- start 2024-09-10T21:43:22.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.917+0000 7f1176400640 1 -- start start 2024-09-10T21:43:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.918+0000 7f1176400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11700cbf10 0x7f11700c8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.918+0000 7f1176400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11700cdc10 0x7f1170073f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.918+0000 7f1176400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f117015b000 0x7f1170072320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.918+0000 7f1176400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f117006ef20 con 0x7f11700cdc10 2024-09-10T21:43:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.918+0000 7f1176400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f117006eda0 con 0x7f117015b000 2024-09-10T21:43:22.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.918+0000 7f1176400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f117006f0a0 con 0x7f11700cbf10 2024-09-10T21:43:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.918+0000 7f116fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11700cdc10 0x7f1170073f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.918+0000 7f116fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11700cdc10 0x7f1170073f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:55692/0 (socket says 172.21.15.7:55692) 2024-09-10T21:43:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.918+0000 7f116fe00640 1 -- 172.21.15.7:0/747504818 learned_addr learned my addr 172.21.15.7:0/747504818 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.918+0000 7f1175000640 1 --2- 172.21.15.7:0/747504818 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11700cbf10 0x7f11700c8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.918+0000 7f1175a00640 1 --2- 172.21.15.7:0/747504818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f117015b000 0x7f1170072320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.918+0000 7f116fe00640 1 -- 172.21.15.7:0/747504818 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11700cbf10 msgr2=0x7f11700c8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.918+0000 7f116fe00640 1 --2- 172.21.15.7:0/747504818 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11700cbf10 0x7f11700c8980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.918+0000 7f116fe00640 1 -- 172.21.15.7:0/747504818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f117015b000 msgr2=0x7f1170072320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.918+0000 7f116fe00640 1 --2- 172.21.15.7:0/747504818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f117015b000 0x7f1170072320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.918+0000 7f116fe00640 1 -- 172.21.15.7:0/747504818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11700c8ec0 con 0x7f11700cdc10 2024-09-10T21:43:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.918+0000 7f1175000640 1 --2- 172.21.15.7:0/747504818 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11700cbf10 0x7f11700c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:22.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.919+0000 7f116fe00640 1 --2- 172.21.15.7:0/747504818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11700cdc10 0x7f1170073f10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f115c00c400 tx=0x7f115c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:22.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.919+0000 7f116cc00640 1 -- 172.21.15.7:0/747504818 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c011790 con 0x7f11700cdc10 2024-09-10T21:43:22.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.919+0000 7f116cc00640 1 -- 172.21.15.7:0/747504818 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f115c017070 con 0x7f11700cdc10 2024-09-10T21:43:22.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.919+0000 7f116cc00640 1 -- 172.21.15.7:0/747504818 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115c011d00 con 0x7f11700cdc10 2024-09-10T21:43:22.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.919+0000 7f1176400640 1 -- 172.21.15.7:0/747504818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1170074450 con 0x7f11700cdc10 2024-09-10T21:43:22.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.919+0000 7f1176400640 1 -- 172.21.15.7:0/747504818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11701f8fe0 con 0x7f11700cdc10 2024-09-10T21:43:22.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.920+0000 7f1176400640 1 -- 172.21.15.7:0/747504818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11700ccfc0 con 0x7f11700cdc10 2024-09-10T21:43:22.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.921+0000 7f116cc00640 1 -- 172.21.15.7:0/747504818 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f115c01c460 con 0x7f11700cdc10 2024-09-10T21:43:22.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.922+0000 7f116cc00640 1 --2- 172.21.15.7:0/747504818 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1150077f70 0x7f115007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:22.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.922+0000 7f116cc00640 1 -- 172.21.15.7:0/747504818 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5357+0+0 (secure 0 0 0) 0x7f115c099bb0 con 0x7f11700cdc10 2024-09-10T21:43:22.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.923+0000 7f1175000640 1 --2- 172.21.15.7:0/747504818 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1150077f70 0x7f115007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:22.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.923+0000 7f1175000640 1 --2- 172.21.15.7:0/747504818 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1150077f70 0x7f115007a430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1160034000 tx=0x7f1160004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:22.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:22.926+0000 7f116cc00640 1 -- 172.21.15.7:0/747504818 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f115c062770 con 0x7f11700cdc10 2024-09-10T21:43:23.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.121+0000 7f1176400640 1 -- 172.21.15.7:0/747504818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f11700c9c40 con 0x7f11700cdc10 2024-09-10T21:43:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.724+0000 7f116cc00640 1 -- 172.21.15.7:0/747504818 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v35) ==== 169+0+0 (secure 0 0 0) 0x7f115c062070 con 0x7f11700cdc10 2024-09-10T21:43:23.725 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-09-10T21:43:23.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.727+0000 7f1176400640 1 -- 172.21.15.7:0/747504818 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1150077f70 msgr2=0x7f115007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.727+0000 7f1176400640 1 --2- 172.21.15.7:0/747504818 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1150077f70 0x7f115007a430 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f1160034000 tx=0x7f1160004ab0 comp rx=0 tx=0).stop 2024-09-10T21:43:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.727+0000 7f1176400640 1 -- 172.21.15.7:0/747504818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11700cdc10 msgr2=0x7f1170073f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.727+0000 7f1176400640 1 --2- 172.21.15.7:0/747504818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11700cdc10 0x7f1170073f10 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f115c00c400 tx=0x7f115c00c8c0 comp rx=0 tx=0).stop 2024-09-10T21:43:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.727+0000 7f1176400640 1 -- 172.21.15.7:0/747504818 shutdown_connections 2024-09-10T21:43:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.727+0000 7f1176400640 1 --2- 172.21.15.7:0/747504818 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1150077f70 0x7f115007a430 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.727+0000 7f1176400640 1 --2- 172.21.15.7:0/747504818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f117015b000 0x7f1170072320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.727+0000 7f1176400640 1 --2- 172.21.15.7:0/747504818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11700cdc10 0x7f1170073f10 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.727+0000 7f1176400640 1 --2- 172.21.15.7:0/747504818 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11700cbf10 0x7f11700c8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.727+0000 7f1176400640 1 -- 172.21.15.7:0/747504818 >> 172.21.15.7:0/747504818 conn(0x7f11700b1580 msgr2=0x7f11700afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.727+0000 7f1176400640 1 -- 172.21.15.7:0/747504818 shutdown_connections 2024-09-10T21:43:23.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.727+0000 7f1176400640 1 -- 172.21.15.7:0/747504818 wait complete. 2024-09-10T21:43:23.747 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:43:23.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.910+0000 7f8014600640 1 -- 172.21.15.7:0/1823580540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c15cbe0 msgr2=0x7f800c073cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:23.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.910+0000 7f8014600640 1 --2- 172.21.15.7:0/1823580540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c15cbe0 0x7f800c073cd0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f800c073560 tx=0x7f7ff802a420 comp rx=0 tx=0).stop 2024-09-10T21:43:23.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.910+0000 7f8014600640 1 -- 172.21.15.7:0/1823580540 shutdown_connections 2024-09-10T21:43:23.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.910+0000 7f8014600640 1 --2- 172.21.15.7:0/1823580540 >> v2:172.21.15.133:3300/0 conn(0x7f800c074210 0x7f800c15f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:23.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.910+0000 7f8014600640 1 --2- 172.21.15.7:0/1823580540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c15cbe0 0x7f800c073cd0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:23.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.910+0000 7f8014600640 1 -- 172.21.15.7:0/1823580540 >> 172.21.15.7:0/1823580540 conn(0x7f800c0b1660 msgr2=0x7f800c0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:23.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.910+0000 7f8014600640 1 -- 172.21.15.7:0/1823580540 shutdown_connections 2024-09-10T21:43:23.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.910+0000 7f8014600640 1 -- 172.21.15.7:0/1823580540 wait complete. 2024-09-10T21:43:23.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.912+0000 7f8014600640 1 Processor -- start 2024-09-10T21:43:23.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.912+0000 7f8014600640 1 -- start start 2024-09-10T21:43:23.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.913+0000 7f8014600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c06ff50 0x7f800c0ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:23.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.913+0000 7f8014600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f800c074210 0x7f800c0ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:23.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.913+0000 7f8014600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f800c15cbe0 0x7f800c0adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:23.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.913+0000 7f8014600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f800c06f8a0 con 0x7f800c06ff50 2024-09-10T21:43:23.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.913+0000 7f8014600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f800c06f720 con 0x7f800c15cbe0 2024-09-10T21:43:23.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.913+0000 7f8014600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f800c15fc50 con 0x7f800c074210 2024-09-10T21:43:23.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.913+0000 7f8011c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c06ff50 0x7f800c0ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:23.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.913+0000 7f8011c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c06ff50 0x7f800c0ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42202/0 (socket says 172.21.15.7:42202) 2024-09-10T21:43:23.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.913+0000 7f8011c00640 1 -- 172.21.15.7:0/1783604503 learned_addr learned my addr 172.21.15.7:0/1783604503 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:23.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.914+0000 7f8011200640 1 --2- 172.21.15.7:0/1783604503 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f800c074210 0x7f800c0ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:23.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.914+0000 7f8012600640 1 --2- 172.21.15.7:0/1783604503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f800c15cbe0 0x7f800c0adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:23.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.914+0000 7f8011c00640 1 -- 172.21.15.7:0/1783604503 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f800c074210 msgr2=0x7f800c0ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:23.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.914+0000 7f8011c00640 1 --2- 172.21.15.7:0/1783604503 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f800c074210 0x7f800c0ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:23.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.914+0000 7f8011c00640 1 -- 172.21.15.7:0/1783604503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f800c15cbe0 msgr2=0x7f800c0adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:23.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.914+0000 7f8011c00640 1 --2- 172.21.15.7:0/1783604503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f800c15cbe0 0x7f800c0adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:23.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.914+0000 7f8011c00640 1 -- 172.21.15.7:0/1783604503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f800c0c64a0 con 0x7f800c06ff50 2024-09-10T21:43:23.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.914+0000 7f8011c00640 1 --2- 172.21.15.7:0/1783604503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c06ff50 0x7f800c0ca630 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f800000c450 tx=0x7f800000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:23.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.914+0000 7f8009600640 1 -- 172.21.15.7:0/1783604503 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000016020 con 0x7f800c06ff50 2024-09-10T21:43:23.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.914+0000 7f8009600640 1 -- 172.21.15.7:0/1783604503 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8000012920 con 0x7f800c06ff50 2024-09-10T21:43:23.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.914+0000 7f8009600640 1 -- 172.21.15.7:0/1783604503 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8000012d70 con 0x7f800c06ff50 2024-09-10T21:43:23.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.914+0000 7f8014600640 1 -- 172.21.15.7:0/1783604503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f800c0c79a0 con 0x7f800c06ff50 2024-09-10T21:43:23.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.914+0000 7f8014600640 1 -- 172.21.15.7:0/1783604503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f800c0c7f60 con 0x7f800c06ff50 2024-09-10T21:43:23.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.915+0000 7f8014600640 1 -- 172.21.15.7:0/1783604503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f800c0cb510 con 0x7f800c06ff50 2024-09-10T21:43:23.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.917+0000 7f8009600640 1 -- 172.21.15.7:0/1783604503 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f800001b400 con 0x7f800c06ff50 2024-09-10T21:43:23.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.918+0000 7f8009600640 1 --2- 172.21.15.7:0/1783604503 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7fec077f70 0x7f7fec07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:23.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.918+0000 7f8009600640 1 -- 172.21.15.7:0/1783604503 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f8000099b10 con 0x7f800c06ff50 2024-09-10T21:43:23.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.918+0000 7f8011200640 1 --2- 172.21.15.7:0/1783604503 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7fec077f70 0x7f7fec07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:23.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.919+0000 7f8011200640 1 --2- 172.21.15.7:0/1783604503 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7fec077f70 0x7f7fec07a430 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8034000 tx=0x7f7ff8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:23.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:23.922+0000 7f8009600640 1 -- 172.21.15.7:0/1783604503 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f800c0cb510 con 0x7f800c06ff50 2024-09-10T21:43:24.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.116+0000 7f8014600640 1 -- 172.21.15.7:0/1783604503 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f800c15f710 con 0x7f800c06ff50 2024-09-10T21:43:24.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.120+0000 7f8009600640 1 -- 172.21.15.7:0/1783604503 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v29) ==== 71+0+0 (secure 0 0 0) 0x7f8000020030 con 0x7f800c06ff50 2024-09-10T21:43:24.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.123+0000 7f8014600640 1 -- 172.21.15.7:0/1783604503 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7fec077f70 msgr2=0x7f7fec07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.123+0000 7f8014600640 1 --2- 172.21.15.7:0/1783604503 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7fec077f70 0x7f7fec07a430 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f7ff8034000 tx=0x7f7ff8004730 comp rx=0 tx=0).stop 2024-09-10T21:43:24.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.123+0000 7f8014600640 1 -- 172.21.15.7:0/1783604503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c06ff50 msgr2=0x7f800c0ca630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.123+0000 7f8014600640 1 --2- 172.21.15.7:0/1783604503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c06ff50 0x7f800c0ca630 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f800000c450 tx=0x7f800000c910 comp rx=0 tx=0).stop 2024-09-10T21:43:24.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.123+0000 7f8014600640 1 -- 172.21.15.7:0/1783604503 shutdown_connections 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.123+0000 7f8014600640 1 --2- 172.21.15.7:0/1783604503 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7fec077f70 0x7f7fec07a430 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.123+0000 7f8014600640 1 --2- 172.21.15.7:0/1783604503 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f800c15cbe0 0x7f800c0adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.123+0000 7f8014600640 1 --2- 172.21.15.7:0/1783604503 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f800c074210 0x7f800c0ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.123+0000 7f8014600640 1 --2- 172.21.15.7:0/1783604503 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f800c06ff50 0x7f800c0ca630 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.123+0000 7f8014600640 1 -- 172.21.15.7:0/1783604503 >> 172.21.15.7:0/1783604503 conn(0x7f800c0b1660 msgr2=0x7f800c0cc630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.123+0000 7f8014600640 1 -- 172.21.15.7:0/1783604503 shutdown_connections 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.123+0000 7f8014600640 1 -- 172.21.15.7:0/1783604503 wait complete. 2024-09-10T21:43:24.139 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2024-09-10T21:43:24.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.299+0000 7fb6f7200640 1 -- 172.21.15.7:0/2822762030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f00cd750 msgr2=0x7fb6f0073730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.299+0000 7fb6f7200640 1 --2- 172.21.15.7:0/2822762030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f00cd750 0x7fb6f0073730 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fb6f006dac0 tx=0x7fb6d802a4d0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.299+0000 7fb6f7200640 1 -- 172.21.15.7:0/2822762030 shutdown_connections 2024-09-10T21:43:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.299+0000 7fb6f7200640 1 --2- 172.21.15.7:0/2822762030 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6f0073c70 0x7fb6f015fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.299+0000 7fb6f7200640 1 --2- 172.21.15.7:0/2822762030 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f00cd750 0x7fb6f0073730 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.299+0000 7fb6f7200640 1 --2- 172.21.15.7:0/2822762030 >> v2:172.21.15.133:3300/0 conn(0x7fb6f015bc90 0x7fb6f00ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.299+0000 7fb6f7200640 1 -- 172.21.15.7:0/2822762030 >> 172.21.15.7:0/2822762030 conn(0x7fb6f00b16d0 msgr2=0x7fb6f00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.299+0000 7fb6f7200640 1 -- 172.21.15.7:0/2822762030 shutdown_connections 2024-09-10T21:43:24.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.299+0000 7fb6f7200640 1 -- 172.21.15.7:0/2822762030 wait complete. 2024-09-10T21:43:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.300+0000 7fb6f7200640 1 Processor -- start 2024-09-10T21:43:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.301+0000 7fb6f7200640 1 -- start start 2024-09-10T21:43:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.301+0000 7fb6f7200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6f0073c70 0x7fb6f0152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.301+0000 7fb6f7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f00cd750 0x7fb6f015ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.301+0000 7fb6f7200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6f015bc90 0x7fb6f015e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.302+0000 7fb6f7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f0160670 con 0x7fb6f00cd750 2024-09-10T21:43:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.302+0000 7fb6f7200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f01604f0 con 0x7fb6f0073c70 2024-09-10T21:43:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.302+0000 7fb6f7200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f006fcf0 con 0x7fb6f015bc90 2024-09-10T21:43:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.302+0000 7fb6e7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f00cd750 0x7fb6f015ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.302+0000 7fb6e7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f00cd750 0x7fb6f015ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42216/0 (socket says 172.21.15.7:42216) 2024-09-10T21:43:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.302+0000 7fb6e7400640 1 -- 172.21.15.7:0/3916959098 learned_addr learned my addr 172.21.15.7:0/3916959098 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:24.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.302+0000 7fb6efe00640 1 --2- 172.21.15.7:0/3916959098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6f0073c70 0x7fb6f0152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:24.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.302+0000 7fb6f5200640 1 --2- 172.21.15.7:0/3916959098 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6f015bc90 0x7fb6f015e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:24.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.302+0000 7fb6e7400640 1 -- 172.21.15.7:0/3916959098 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6f015bc90 msgr2=0x7fb6f015e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.302+0000 7fb6e7400640 1 --2- 172.21.15.7:0/3916959098 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6f015bc90 0x7fb6f015e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.302+0000 7fb6e7400640 1 -- 172.21.15.7:0/3916959098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6f0073c70 msgr2=0x7fb6f0152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.302+0000 7fb6e7400640 1 --2- 172.21.15.7:0/3916959098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6f0073c70 0x7fb6f0152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.302+0000 7fb6e7400640 1 -- 172.21.15.7:0/3916959098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f015d000 con 0x7fb6f00cd750 2024-09-10T21:43:24.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.303+0000 7fb6e7400640 1 --2- 172.21.15.7:0/3916959098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f00cd750 0x7fb6f015ddf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80099c0 tx=0x7fb6d80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:24.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.303+0000 7fb6ecc00640 1 -- 172.21.15.7:0/3916959098 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d803f070 con 0x7fb6f00cd750 2024-09-10T21:43:24.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.303+0000 7fb6ecc00640 1 -- 172.21.15.7:0/3916959098 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6d80047a0 con 0x7fb6f00cd750 2024-09-10T21:43:24.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.303+0000 7fb6f7200640 1 -- 172.21.15.7:0/3916959098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f015d2e0 con 0x7fb6f00cd750 2024-09-10T21:43:24.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.303+0000 7fb6ecc00640 1 -- 172.21.15.7:0/3916959098 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d8035030 con 0x7fb6f00cd750 2024-09-10T21:43:24.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.303+0000 7fb6f7200640 1 -- 172.21.15.7:0/3916959098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f00c6cf0 con 0x7fb6f00cd750 2024-09-10T21:43:24.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.304+0000 7fb6f7200640 1 -- 172.21.15.7:0/3916959098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f0073730 con 0x7fb6f00cd750 2024-09-10T21:43:24.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.305+0000 7fb6ecc00640 1 -- 172.21.15.7:0/3916959098 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb6d8004bf0 con 0x7fb6f00cd750 2024-09-10T21:43:24.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.306+0000 7fb6ecc00640 1 --2- 172.21.15.7:0/3916959098 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb6cc077f70 0x7fb6cc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.306+0000 7fb6ecc00640 1 -- 172.21.15.7:0/3916959098 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fb6d80ba940 con 0x7fb6f00cd750 2024-09-10T21:43:24.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.307+0000 7fb6efe00640 1 --2- 172.21.15.7:0/3916959098 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb6cc077f70 0x7fb6cc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:24.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.307+0000 7fb6efe00640 1 --2- 172.21.15.7:0/3916959098 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb6cc077f70 0x7fb6cc07a430 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0005fd0 tx=0x7fb6e0009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:24.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.310+0000 7fb6ecc00640 1 -- 172.21.15.7:0/3916959098 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb6d80830e0 con 0x7fb6f00cd750 2024-09-10T21:43:24.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.576+0000 7fb6f7200640 1 -- 172.21.15.7:0/3916959098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) -- 0x7fb6f0069810 con 0x7fb6f00cd750 2024-09-10T21:43:24.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.740+0000 7fb6ecc00640 1 -- 172.21.15.7:0/3916959098 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v86) ==== 142+0+0 (secure 0 0 0) 0x7fb6d8087b10 con 0x7fb6f00cd750 2024-09-10T21:43:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.742+0000 7fb6f7200640 1 -- 172.21.15.7:0/3916959098 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb6cc077f70 msgr2=0x7fb6cc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.742+0000 7fb6f7200640 1 --2- 172.21.15.7:0/3916959098 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb6cc077f70 0x7fb6cc07a430 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb6e0005fd0 tx=0x7fb6e0009500 comp rx=0 tx=0).stop 2024-09-10T21:43:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.742+0000 7fb6f7200640 1 -- 172.21.15.7:0/3916959098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f00cd750 msgr2=0x7fb6f015ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.742+0000 7fb6f7200640 1 --2- 172.21.15.7:0/3916959098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f00cd750 0x7fb6f015ddf0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fb6d80099c0 tx=0x7fb6d80038a0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.742+0000 7fb6f7200640 1 -- 172.21.15.7:0/3916959098 shutdown_connections 2024-09-10T21:43:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.742+0000 7fb6f7200640 1 --2- 172.21.15.7:0/3916959098 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb6cc077f70 0x7fb6cc07a430 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.742+0000 7fb6f7200640 1 --2- 172.21.15.7:0/3916959098 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6f015bc90 0x7fb6f015e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.742+0000 7fb6f7200640 1 --2- 172.21.15.7:0/3916959098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6f00cd750 0x7fb6f015ddf0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.742+0000 7fb6f7200640 1 --2- 172.21.15.7:0/3916959098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6f0073c70 0x7fb6f0152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.742+0000 7fb6f7200640 1 -- 172.21.15.7:0/3916959098 >> 172.21.15.7:0/3916959098 conn(0x7fb6f00b16d0 msgr2=0x7fb6f00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.742+0000 7fb6f7200640 1 -- 172.21.15.7:0/3916959098 shutdown_connections 2024-09-10T21:43:24.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.742+0000 7fb6f7200640 1 -- 172.21.15.7:0/3916959098 wait complete. 2024-09-10T21:43:24.760 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-09-10T21:43:24.760 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2024-09-10T21:43:24.760 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-10T21:43:24.761 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:43:24.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.920+0000 7f2f41200640 1 -- 172.21.15.7:0/2385851343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f3c15bc90 msgr2=0x7f2f3c0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.920+0000 7f2f41200640 1 --2- 172.21.15.7:0/2385851343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f3c15bc90 0x7f2f3c0ccef0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0c3650 tx=0x7f2f2c02a4d0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.920+0000 7f2f41200640 1 -- 172.21.15.7:0/2385851343 shutdown_connections 2024-09-10T21:43:24.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.920+0000 7f2f41200640 1 --2- 172.21.15.7:0/2385851343 >> v2:172.21.15.133:3300/0 conn(0x7f2f3c15fda0 0x7f2f3c160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.920+0000 7f2f41200640 1 --2- 172.21.15.7:0/2385851343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f3c15bc90 0x7f2f3c0ccef0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.920+0000 7f2f41200640 1 -- 172.21.15.7:0/2385851343 >> 172.21.15.7:0/2385851343 conn(0x7f2f3c0b16d0 msgr2=0x7f2f3c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:24.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.920+0000 7f2f41200640 1 -- 172.21.15.7:0/2385851343 shutdown_connections 2024-09-10T21:43:24.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.920+0000 7f2f41200640 1 -- 172.21.15.7:0/2385851343 wait complete. 2024-09-10T21:43:24.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.921+0000 7f2f41200640 1 Processor -- start 2024-09-10T21:43:24.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.922+0000 7f2f41200640 1 -- start start 2024-09-10T21:43:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.922+0000 7f2f41200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f3c0cd750 0x7f2f3c0c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.922+0000 7f2f41200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f3c15bc90 0x7f2f3c0c5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.922+0000 7f2f41200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f3c15fda0 0x7f2f3c15c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.922+0000 7f2f41200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c06fe70 con 0x7f2f3c15bc90 2024-09-10T21:43:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.922+0000 7f2f41200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2f3c06fcf0 con 0x7f2f3c0cd750 2024-09-10T21:43:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.922+0000 7f2f41200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f2f3c06fff0 con 0x7f2f3c15fda0 2024-09-10T21:43:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.923+0000 7f2f39400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f3c15bc90 0x7f2f3c0c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.923+0000 7f2f39400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f3c15bc90 0x7f2f3c0c5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42220/0 (socket says 172.21.15.7:42220) 2024-09-10T21:43:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.923+0000 7f2f39400640 1 -- 172.21.15.7:0/2487845253 learned_addr learned my addr 172.21.15.7:0/2487845253 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.923+0000 7f2f39e00640 1 --2- 172.21.15.7:0/2487845253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f3c0cd750 0x7f2f3c0c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:24.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.923+0000 7f2f3a800640 1 --2- 172.21.15.7:0/2487845253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f3c15fda0 0x7f2f3c15c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.923+0000 7f2f39400640 1 -- 172.21.15.7:0/2487845253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f3c15fda0 msgr2=0x7f2f3c15c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.923+0000 7f2f39400640 1 --2- 172.21.15.7:0/2487845253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f3c15fda0 0x7f2f3c15c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.923+0000 7f2f39400640 1 -- 172.21.15.7:0/2487845253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f3c0cd750 msgr2=0x7f2f3c0c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.923+0000 7f2f39400640 1 --2- 172.21.15.7:0/2487845253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f3c0cd750 0x7f2f3c0c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.923+0000 7f2f39400640 1 -- 172.21.15.7:0/2487845253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f3c0c6240 con 0x7f2f3c15bc90 2024-09-10T21:43:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.923+0000 7f2f39e00640 1 --2- 172.21.15.7:0/2487845253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f3c0cd750 0x7f2f3c0c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.923+0000 7f2f39400640 1 --2- 172.21.15.7:0/2487845253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f3c15bc90 0x7f2f3c0c5310 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800c430 tx=0x7f2f2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.924+0000 7f2f32000640 1 -- 172.21.15.7:0/2487845253 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f28011790 con 0x7f2f3c15bc90 2024-09-10T21:43:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.924+0000 7f2f32000640 1 -- 172.21.15.7:0/2487845253 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f28017070 con 0x7f2f3c15bc90 2024-09-10T21:43:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.924+0000 7f2f32000640 1 -- 172.21.15.7:0/2487845253 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f28011d00 con 0x7f2f3c15bc90 2024-09-10T21:43:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.924+0000 7f2f41200640 1 -- 172.21.15.7:0/2487845253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f3c15aab0 con 0x7f2f3c15bc90 2024-09-10T21:43:24.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.924+0000 7f2f41200640 1 -- 172.21.15.7:0/2487845253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f3c0c8ce0 con 0x7f2f3c15bc90 2024-09-10T21:43:24.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.925+0000 7f2f41200640 1 -- 172.21.15.7:0/2487845253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f3c070510 con 0x7f2f3c15bc90 2024-09-10T21:43:24.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.926+0000 7f2f32000640 1 -- 172.21.15.7:0/2487845253 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2f2801c460 con 0x7f2f3c15bc90 2024-09-10T21:43:24.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.927+0000 7f2f32000640 1 --2- 172.21.15.7:0/2487845253 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f1c077f70 0x7f2f1c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.927+0000 7f2f32000640 1 -- 172.21.15.7:0/2487845253 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f2f28099900 con 0x7f2f3c15bc90 2024-09-10T21:43:24.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.927+0000 7f2f39e00640 1 --2- 172.21.15.7:0/2487845253 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f1c077f70 0x7f2f1c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:24.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.928+0000 7f2f39e00640 1 --2- 172.21.15.7:0/2487845253 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f1c077f70 0x7f2f1c07a430 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c033000 tx=0x7f2f2c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:24.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:24.931+0000 7f2f32000640 1 -- 172.21.15.7:0/2487845253 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2f280626f0 con 0x7f2f3c15bc90 2024-09-10T21:43:25.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.138+0000 7f2f41200640 1 -- 172.21.15.7:0/2487845253 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f3c0b46b0 con 0x7f2f3c15bc90 2024-09-10T21:43:25.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.138+0000 7f2f32000640 1 -- 172.21.15.7:0/2487845253 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+12996 (secure 0 0 0) 0x7f2f28061ff0 con 0x7f2f3c15bc90 2024-09-10T21:43:25.140 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:25.140 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:43:23.722417+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6836/849012248":"2024-09-11T21:43:20.712835+0000","172.21.15.133:6834/849012248":"2024-09-11T21:43:20.712835+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:43:25.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.141+0000 7f2f41200640 1 -- 172.21.15.7:0/2487845253 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f1c077f70 msgr2=0x7f2f1c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:25.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.141+0000 7f2f41200640 1 --2- 172.21.15.7:0/2487845253 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f1c077f70 0x7f2f1c07a430 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c033000 tx=0x7f2f2c004730 comp rx=0 tx=0).stop 2024-09-10T21:43:25.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.141+0000 7f2f41200640 1 -- 172.21.15.7:0/2487845253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f3c15bc90 msgr2=0x7f2f3c0c5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:25.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.141+0000 7f2f41200640 1 --2- 172.21.15.7:0/2487845253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f3c15bc90 0x7f2f3c0c5310 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f2f2800c430 tx=0x7f2f2800c8f0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.141+0000 7f2f41200640 1 -- 172.21.15.7:0/2487845253 shutdown_connections 2024-09-10T21:43:25.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.141+0000 7f2f41200640 1 --2- 172.21.15.7:0/2487845253 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f1c077f70 0x7f2f1c07a430 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.141+0000 7f2f41200640 1 --2- 172.21.15.7:0/2487845253 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f3c15fda0 0x7f2f3c15c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.141+0000 7f2f41200640 1 --2- 172.21.15.7:0/2487845253 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f3c15bc90 0x7f2f3c0c5310 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.141+0000 7f2f41200640 1 --2- 172.21.15.7:0/2487845253 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f3c0cd750 0x7f2f3c0c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.141+0000 7f2f41200640 1 -- 172.21.15.7:0/2487845253 >> 172.21.15.7:0/2487845253 conn(0x7f2f3c0b16d0 msgr2=0x7f2f3c0b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:25.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.141+0000 7f2f41200640 1 -- 172.21.15.7:0/2487845253 shutdown_connections 2024-09-10T21:43:25.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.141+0000 7f2f41200640 1 -- 172.21.15.7:0/2487845253 wait complete. 2024-09-10T21:43:25.158 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:41:55.889618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:43:25.159 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:43:25.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.321+0000 7f2324c00640 1 -- 172.21.15.7:0/1261082122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23200b15d0 msgr2=0x7f23200ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:25.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.321+0000 7f2324c00640 1 --2- 172.21.15.7:0/1261082122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23200b15d0 0x7f23200ce520 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2318005fd0 tx=0x7f231802aaa0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.322+0000 7f2324c00640 1 -- 172.21.15.7:0/1261082122 shutdown_connections 2024-09-10T21:43:25.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.322+0000 7f2324c00640 1 --2- 172.21.15.7:0/1261082122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f23200b15d0 0x7f23200ce520 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.322+0000 7f2324c00640 1 --2- 172.21.15.7:0/1261082122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2320074b70 0x7f23200cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.322+0000 7f2324c00640 1 -- 172.21.15.7:0/1261082122 >> 172.21.15.7:0/1261082122 conn(0x7f2320074080 msgr2=0x7f2320074460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:25.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.322+0000 7f2324c00640 1 -- 172.21.15.7:0/1261082122 shutdown_connections 2024-09-10T21:43:25.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.322+0000 7f2324c00640 1 -- 172.21.15.7:0/1261082122 wait complete. 2024-09-10T21:43:25.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.323+0000 7f2324c00640 1 Processor -- start 2024-09-10T21:43:25.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.323+0000 7f2324c00640 1 -- start start 2024-09-10T21:43:25.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.324+0000 7f2324c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2320074b70 0x7f23200c9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:25.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.324+0000 7f2324c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23200b11f0 0x7f232015c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:25.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.324+0000 7f2324c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23200b15d0 0x7f232015c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:25.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.324+0000 7f2324c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f23200caa50 con 0x7f23200b11f0 2024-09-10T21:43:25.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.324+0000 7f2324c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f23200ca8d0 con 0x7f2320074b70 2024-09-10T21:43:25.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.324+0000 7f2324c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f23200cabd0 con 0x7f23200b15d0 2024-09-10T21:43:25.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.325+0000 7f2317e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23200b11f0 0x7f232015c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:25.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.325+0000 7f2317e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23200b11f0 0x7f232015c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42232/0 (socket says 172.21.15.7:42232) 2024-09-10T21:43:25.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.325+0000 7f2317e00640 1 -- 172.21.15.7:0/3858056716 learned_addr learned my addr 172.21.15.7:0/3858056716 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.325+0000 7f231f400640 1 --2- 172.21.15.7:0/3858056716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2320074b70 0x7f23200c9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.325+0000 7f231fe00640 1 --2- 172.21.15.7:0/3858056716 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23200b15d0 0x7f232015c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.325+0000 7f2317e00640 1 -- 172.21.15.7:0/3858056716 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23200b15d0 msgr2=0x7f232015c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.325+0000 7f2317e00640 1 --2- 172.21.15.7:0/3858056716 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23200b15d0 0x7f232015c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.325+0000 7f2317e00640 1 -- 172.21.15.7:0/3858056716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2320074b70 msgr2=0x7f23200c9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.325+0000 7f2317e00640 1 --2- 172.21.15.7:0/3858056716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2320074b70 0x7f23200c9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.325+0000 7f2317e00640 1 -- 172.21.15.7:0/3858056716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2320146930 con 0x7f23200b11f0 2024-09-10T21:43:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.325+0000 7f231fe00640 1 --2- 172.21.15.7:0/3858056716 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23200b15d0 0x7f232015c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:43:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.325+0000 7f231f400640 1 --2- 172.21.15.7:0/3858056716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2320074b70 0x7f23200c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:43:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.325+0000 7f2317e00640 1 --2- 172.21.15.7:0/3858056716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23200b11f0 0x7f232015c300 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f230800e9a0 tx=0x7f230800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.326+0000 7f2317400640 1 -- 172.21.15.7:0/3858056716 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230800cd80 con 0x7f23200b11f0 2024-09-10T21:43:25.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.326+0000 7f2317400640 1 -- 172.21.15.7:0/3858056716 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2308004020 con 0x7f23200b11f0 2024-09-10T21:43:25.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.326+0000 7f2324c00640 1 -- 172.21.15.7:0/3858056716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2320146c10 con 0x7f23200b11f0 2024-09-10T21:43:25.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.326+0000 7f2317400640 1 -- 172.21.15.7:0/3858056716 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2308004450 con 0x7f23200b11f0 2024-09-10T21:43:25.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.326+0000 7f2324c00640 1 -- 172.21.15.7:0/3858056716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23201471d0 con 0x7f23200b11f0 2024-09-10T21:43:25.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.327+0000 7f2324c00640 1 -- 172.21.15.7:0/3858056716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232015e860 con 0x7f23200b11f0 2024-09-10T21:43:25.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.328+0000 7f2317400640 1 -- 172.21.15.7:0/3858056716 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f23080035f0 con 0x7f23200b11f0 2024-09-10T21:43:25.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.329+0000 7f2317400640 1 --2- 172.21.15.7:0/3858056716 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f22fc077f70 0x7f22fc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:25.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.329+0000 7f2317400640 1 -- 172.21.15.7:0/3858056716 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f230809a560 con 0x7f23200b11f0 2024-09-10T21:43:25.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.329+0000 7f231f400640 1 --2- 172.21.15.7:0/3858056716 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f22fc077f70 0x7f22fc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.330+0000 7f231f400640 1 --2- 172.21.15.7:0/3858056716 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f22fc077f70 0x7f22fc07a430 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2310001740 tx=0x7f2310009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:25.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.333+0000 7f2317400640 1 -- 172.21.15.7:0/3858056716 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f232015e860 con 0x7f23200b11f0 2024-09-10T21:43:25.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.525+0000 7f2324c00640 1 -- 172.21.15.7:0/3858056716 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f23200cd660 con 0x7f23200b11f0 2024-09-10T21:43:25.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.525+0000 7f2317400640 1 -- 172.21.15.7:0/3858056716 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) ==== 93+0+10 (secure 0 0 0) 0x7f23080632d0 con 0x7f23200b11f0 2024-09-10T21:43:25.527 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-10T21:43:25.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.528+0000 7f2324c00640 1 -- 172.21.15.7:0/3858056716 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f22fc077f70 msgr2=0x7f22fc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:25.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.528+0000 7f2324c00640 1 --2- 172.21.15.7:0/3858056716 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f22fc077f70 0x7f22fc07a430 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2310001740 tx=0x7f2310009040 comp rx=0 tx=0).stop 2024-09-10T21:43:25.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.528+0000 7f2324c00640 1 -- 172.21.15.7:0/3858056716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23200b11f0 msgr2=0x7f232015c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:25.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.528+0000 7f2324c00640 1 --2- 172.21.15.7:0/3858056716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23200b11f0 0x7f232015c300 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f230800e9a0 tx=0x7f230800ee60 comp rx=0 tx=0).stop 2024-09-10T21:43:25.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.528+0000 7f2324c00640 1 -- 172.21.15.7:0/3858056716 shutdown_connections 2024-09-10T21:43:25.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.528+0000 7f2324c00640 1 --2- 172.21.15.7:0/3858056716 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f22fc077f70 0x7f22fc07a430 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.528+0000 7f2324c00640 1 --2- 172.21.15.7:0/3858056716 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23200b15d0 0x7f232015c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.528+0000 7f2324c00640 1 --2- 172.21.15.7:0/3858056716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f23200b11f0 0x7f232015c300 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.528+0000 7f2324c00640 1 --2- 172.21.15.7:0/3858056716 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2320074b70 0x7f23200c9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.528+0000 7f2324c00640 1 -- 172.21.15.7:0/3858056716 >> 172.21.15.7:0/3858056716 conn(0x7f2320074080 msgr2=0x7f23200701e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:25.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.528+0000 7f2324c00640 1 -- 172.21.15.7:0/3858056716 shutdown_connections 2024-09-10T21:43:25.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.528+0000 7f2324c00640 1 -- 172.21.15.7:0/3858056716 wait complete. 2024-09-10T21:43:25.545 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-09-10T21:43:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.705+0000 7f0c06800640 1 -- 172.21.15.7:0/599358752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c000cd750 msgr2=0x7f0c000ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.705+0000 7f0c06800640 1 --2- 172.21.15.7:0/599358752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c000cd750 0x7f0c000ccef0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0c0015fa90 tx=0x7f0be802a400 comp rx=0 tx=0).stop 2024-09-10T21:43:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.705+0000 7f0c06800640 1 -- 172.21.15.7:0/599358752 shutdown_connections 2024-09-10T21:43:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.705+0000 7f0c06800640 1 --2- 172.21.15.7:0/599358752 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c0015fda0 0x7f0c001601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.705+0000 7f0c06800640 1 --2- 172.21.15.7:0/599358752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c000cd750 0x7f0c000ccef0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.705+0000 7f0c06800640 1 -- 172.21.15.7:0/599358752 >> 172.21.15.7:0/599358752 conn(0x7f0c000b16d0 msgr2=0x7f0c000b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.705+0000 7f0c06800640 1 -- 172.21.15.7:0/599358752 shutdown_connections 2024-09-10T21:43:25.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.705+0000 7f0c06800640 1 -- 172.21.15.7:0/599358752 wait complete. 2024-09-10T21:43:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.707+0000 7f0c06800640 1 Processor -- start 2024-09-10T21:43:25.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.707+0000 7f0c06800640 1 -- start start 2024-09-10T21:43:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.708+0000 7f0c06800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c000cd750 0x7f0c0015a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.708+0000 7f0c06800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c0015bc90 0x7f0c000c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.708+0000 7f0c06800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0015fda0 0x7f0c000b3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.708+0000 7f0c06800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0006fe70 con 0x7f0c0015fda0 2024-09-10T21:43:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.708+0000 7f0c06800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0c0006fcf0 con 0x7f0c0015bc90 2024-09-10T21:43:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.708+0000 7f0c06800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f0c0006fff0 con 0x7f0c000cd750 2024-09-10T21:43:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.708+0000 7f0bffe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0015fda0 0x7f0c000b3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.708+0000 7f0bffe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0015fda0 0x7f0c000b3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42248/0 (socket says 172.21.15.7:42248) 2024-09-10T21:43:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.708+0000 7f0bffe00640 1 -- 172.21.15.7:0/135515096 learned_addr learned my addr 172.21.15.7:0/135515096 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.708+0000 7f0bf6a00640 1 --2- 172.21.15.7:0/135515096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c0015bc90 0x7f0c000c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.708+0000 7f0bff400640 1 --2- 172.21.15.7:0/135515096 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c000cd750 0x7f0c0015a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:25.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.709+0000 7f0bffe00640 1 -- 172.21.15.7:0/135515096 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c000cd750 msgr2=0x7f0c0015a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.709+0000 7f0bffe00640 1 --2- 172.21.15.7:0/135515096 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c000cd750 0x7f0c0015a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.709+0000 7f0bffe00640 1 -- 172.21.15.7:0/135515096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c0015bc90 msgr2=0x7f0c000c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.709+0000 7f0bffe00640 1 --2- 172.21.15.7:0/135515096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c0015bc90 0x7f0c000c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.709+0000 7f0bffe00640 1 -- 172.21.15.7:0/135515096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c001f7680 con 0x7f0c0015fda0 2024-09-10T21:43:25.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.709+0000 7f0bffe00640 1 --2- 172.21.15.7:0/135515096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0015fda0 0x7f0c000b3fa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800e940 tx=0x7f0bf800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:25.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.709+0000 7f0bf7e00640 1 -- 172.21.15.7:0/135515096 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf800ccc0 con 0x7f0c0015fda0 2024-09-10T21:43:25.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.709+0000 7f0bf7e00640 1 -- 172.21.15.7:0/135515096 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bf800ce60 con 0x7f0c0015fda0 2024-09-10T21:43:25.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.709+0000 7f0c06800640 1 -- 172.21.15.7:0/135515096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c001f7900 con 0x7f0c0015fda0 2024-09-10T21:43:25.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.709+0000 7f0bf7e00640 1 -- 172.21.15.7:0/135515096 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bf8004280 con 0x7f0c0015fda0 2024-09-10T21:43:25.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.709+0000 7f0c06800640 1 -- 172.21.15.7:0/135515096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c001f7ec0 con 0x7f0c0015fda0 2024-09-10T21:43:25.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.710+0000 7f0c06800640 1 -- 172.21.15.7:0/135515096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c000704a0 con 0x7f0c0015fda0 2024-09-10T21:43:25.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.711+0000 7f0bf7e00640 1 -- 172.21.15.7:0/135515096 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0bf80035f0 con 0x7f0c0015fda0 2024-09-10T21:43:25.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.712+0000 7f0bf7e00640 1 --2- 172.21.15.7:0/135515096 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0bdc077f70 0x7f0bdc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:25.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.713+0000 7f0bf7e00640 1 -- 172.21.15.7:0/135515096 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f0bf809a410 con 0x7f0c0015fda0 2024-09-10T21:43:25.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.713+0000 7f0bff400640 1 --2- 172.21.15.7:0/135515096 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0bdc077f70 0x7f0bdc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:25.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.714+0000 7f0bff400640 1 --2- 172.21.15.7:0/135515096 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0bdc077f70 0x7f0bdc07a430 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000e160 tx=0x7f0bf0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:25.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.716+0000 7f0bf7e00640 1 -- 172.21.15.7:0/135515096 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0c001f7680 con 0x7f0c0015fda0 2024-09-10T21:43:25.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:25.984+0000 7f0c06800640 1 -- 172.21.15.7:0/135515096 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7f0c001f8580 con 0x7f0c0015fda0 2024-09-10T21:43:26.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.747+0000 7f0bf7e00640 1 -- 172.21.15.7:0/135515096 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v88) ==== 152+0+0 (secure 0 0 0) 0x7f0bf80630f0 con 0x7f0c0015fda0 2024-09-10T21:43:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.750+0000 7f0c06800640 1 -- 172.21.15.7:0/135515096 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0bdc077f70 msgr2=0x7f0bdc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.750+0000 7f0c06800640 1 --2- 172.21.15.7:0/135515096 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0bdc077f70 0x7f0bdc07a430 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f0bf000e160 tx=0x7f0bf0009040 comp rx=0 tx=0).stop 2024-09-10T21:43:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.750+0000 7f0c06800640 1 -- 172.21.15.7:0/135515096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0015fda0 msgr2=0x7f0c000b3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.750+0000 7f0c06800640 1 --2- 172.21.15.7:0/135515096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0015fda0 0x7f0c000b3fa0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0bf800e940 tx=0x7f0bf800ee00 comp rx=0 tx=0).stop 2024-09-10T21:43:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.750+0000 7f0c06800640 1 -- 172.21.15.7:0/135515096 shutdown_connections 2024-09-10T21:43:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.750+0000 7f0c06800640 1 --2- 172.21.15.7:0/135515096 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0bdc077f70 0x7f0bdc07a430 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.750+0000 7f0c06800640 1 --2- 172.21.15.7:0/135515096 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0c0015bc90 0x7f0c000c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.750+0000 7f0c06800640 1 --2- 172.21.15.7:0/135515096 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0c0015fda0 0x7f0c000b3fa0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.750+0000 7f0c06800640 1 --2- 172.21.15.7:0/135515096 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0c000cd750 0x7f0c0015a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.750+0000 7f0c06800640 1 -- 172.21.15.7:0/135515096 >> 172.21.15.7:0/135515096 conn(0x7f0c000b16d0 msgr2=0x7f0c00072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.750+0000 7f0c06800640 1 -- 172.21.15.7:0/135515096 shutdown_connections 2024-09-10T21:43:26.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.750+0000 7f0c06800640 1 -- 172.21.15.7:0/135515096 wait complete. 2024-09-10T21:43:26.770 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:43:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.933+0000 7f7f71400640 1 -- 172.21.15.7:0/2012457823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c15fda0 msgr2=0x7f7f6c1601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.933+0000 7f7f71400640 1 --2- 172.21.15.7:0/2012457823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c15fda0 0x7f7f6c1601f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f7f64005fd0 tx=0x7f7f6402a780 comp rx=0 tx=0).stop 2024-09-10T21:43:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.934+0000 7f7f71400640 1 -- 172.21.15.7:0/2012457823 shutdown_connections 2024-09-10T21:43:26.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.934+0000 7f7f71400640 1 --2- 172.21.15.7:0/2012457823 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c15fda0 0x7f7f6c1601f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.934+0000 7f7f71400640 1 --2- 172.21.15.7:0/2012457823 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f6c0cd750 0x7f7f6c0ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.934+0000 7f7f71400640 1 -- 172.21.15.7:0/2012457823 >> 172.21.15.7:0/2012457823 conn(0x7f7f6c0b16d0 msgr2=0x7f7f6c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.934+0000 7f7f71400640 1 -- 172.21.15.7:0/2012457823 shutdown_connections 2024-09-10T21:43:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.934+0000 7f7f71400640 1 -- 172.21.15.7:0/2012457823 wait complete. 2024-09-10T21:43:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.935+0000 7f7f71400640 1 Processor -- start 2024-09-10T21:43:26.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.935+0000 7f7f71400640 1 -- start start 2024-09-10T21:43:26.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.936+0000 7f7f71400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c0cd750 0x7f7f6c0b47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:26.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.936+0000 7f7f71400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f6c15bc90 0x7f7f6c0ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:26.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.936+0000 7f7f71400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c15fda0 0x7f7f6c1f6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:26.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.936+0000 7f7f71400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c06fe70 con 0x7f7f6c0cd750 2024-09-10T21:43:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.936+0000 7f7f71400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c06fcf0 con 0x7f7f6c15fda0 2024-09-10T21:43:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.936+0000 7f7f71400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f7f6c06fff0 con 0x7f7f6c15bc90 2024-09-10T21:43:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.937+0000 7f7f6a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c0cd750 0x7f7f6c0b47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.937+0000 7f7f6a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c0cd750 0x7f7f6c0b47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42274/0 (socket says 172.21.15.7:42274) 2024-09-10T21:43:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.937+0000 7f7f6a800640 1 -- 172.21.15.7:0/3140894338 learned_addr learned my addr 172.21.15.7:0/3140894338 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.937+0000 7f7f69e00640 1 --2- 172.21.15.7:0/3140894338 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f6c15bc90 0x7f7f6c0ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.937+0000 7f7f6b200640 1 --2- 172.21.15.7:0/3140894338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c15fda0 0x7f7f6c1f6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.937+0000 7f7f6a800640 1 -- 172.21.15.7:0/3140894338 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f6c15bc90 msgr2=0x7f7f6c0ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.937+0000 7f7f6a800640 1 --2- 172.21.15.7:0/3140894338 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f6c15bc90 0x7f7f6c0ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.937+0000 7f7f6a800640 1 -- 172.21.15.7:0/3140894338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c15fda0 msgr2=0x7f7f6c1f6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.937+0000 7f7f6a800640 1 --2- 172.21.15.7:0/3140894338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c15fda0 0x7f7f6c1f6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.937+0000 7f7f6a800640 1 -- 172.21.15.7:0/3140894338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c0af400 con 0x7f7f6c0cd750 2024-09-10T21:43:26.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.937+0000 7f7f6a800640 1 --2- 172.21.15.7:0/3140894338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c0cd750 0x7f7f6c0b47c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00c480 tx=0x7f7f5c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:26.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.938+0000 7f7f62a00640 1 -- 172.21.15.7:0/3140894338 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c016020 con 0x7f7f6c0cd750 2024-09-10T21:43:26.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.938+0000 7f7f71400640 1 -- 172.21.15.7:0/3140894338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6c0af6e0 con 0x7f7f6c0cd750 2024-09-10T21:43:26.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.938+0000 7f7f62a00640 1 -- 172.21.15.7:0/3140894338 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f5c012920 con 0x7f7f6c0cd750 2024-09-10T21:43:26.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.938+0000 7f7f62a00640 1 -- 172.21.15.7:0/3140894338 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c012d70 con 0x7f7f6c0cd750 2024-09-10T21:43:26.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.938+0000 7f7f71400640 1 -- 172.21.15.7:0/3140894338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6c0affe0 con 0x7f7f6c0cd750 2024-09-10T21:43:26.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.939+0000 7f7f71400640 1 -- 172.21.15.7:0/3140894338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f6c0b1d00 con 0x7f7f6c0cd750 2024-09-10T21:43:26.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.940+0000 7f7f62a00640 1 -- 172.21.15.7:0/3140894338 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7f5c01b400 con 0x7f7f6c0cd750 2024-09-10T21:43:26.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.942+0000 7f7f62a00640 1 --2- 172.21.15.7:0/3140894338 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f4c077fa0 0x7f7f4c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:26.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.942+0000 7f7f62a00640 1 -- 172.21.15.7:0/3140894338 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f7f5c099cd0 con 0x7f7f6c0cd750 2024-09-10T21:43:26.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.942+0000 7f7f69e00640 1 --2- 172.21.15.7:0/3140894338 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f4c077fa0 0x7f7f4c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:26.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.943+0000 7f7f69e00640 1 --2- 172.21.15.7:0/3140894338 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f4c077fa0 0x7f7f4c07a460 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0c3470 tx=0x7f7f58005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:26.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:26.946+0000 7f7f62a00640 1 -- 172.21.15.7:0/3140894338 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7f5c020020 con 0x7f7f6c0cd750 2024-09-10T21:43:27.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.141+0000 7f7f71400640 1 -- 172.21.15.7:0/3140894338 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f7f6c070510 con 0x7f7f6c0cd750 2024-09-10T21:43:27.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.141+0000 7f7f62a00640 1 -- 172.21.15.7:0/3140894338 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v29) ==== 59+0+1998 (secure 0 0 0) 0x7f7f5c0623f0 con 0x7f7f6c0cd750 2024-09-10T21:43:27.143 INFO:teuthology.orchestra.run.smithi007.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:43:27.143 INFO:teuthology.orchestra.run.smithi007.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:43:27.143 INFO:teuthology.orchestra.run.smithi007.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:43:27.143 INFO:teuthology.orchestra.run.smithi007.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:43:27.143 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3770.177547 2024-09-10T21:43:27.143 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3542.062395 2024-09-10T21:43:27.144 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3788.037135 2024-09-10T21:43:27.144 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5808.986321 2024-09-10T21:43:27.144 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4630.109911 2024-09-10T21:43:27.144 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4510.255493 2024-09-10T21:43:27.144 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4792.698527 2024-09-10T21:43:27.144 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6842.370281 2024-09-10T21:43:27.144 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_mds 20/20 2024-09-10T21:43:27.144 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:43:27.144 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_ms 2/2 2024-09-10T21:43:27.145 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_debug_frag true 2024-09-10T21:43:27.145 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:43:27.145 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:43:27.145 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_verify_scatter true 2024-09-10T21:43:27.145 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:43:27.145 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:43:27.145 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:43:27.145 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced client_mount_timeout 600 2024-09-10T21:43:27.146 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced debug_client 20/20 2024-09-10T21:43:27.146 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced debug_ms 1/1 2024-09-10T21:43:27.146 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:43:27.146 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:43:27.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.144+0000 7f7f71400640 1 -- 172.21.15.7:0/3140894338 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f4c077fa0 msgr2=0x7f7f4c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:27.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.144+0000 7f7f71400640 1 --2- 172.21.15.7:0/3140894338 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f4c077fa0 0x7f7f4c07a460 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0c3470 tx=0x7f7f58005f90 comp rx=0 tx=0).stop 2024-09-10T21:43:27.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.144+0000 7f7f71400640 1 -- 172.21.15.7:0/3140894338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c0cd750 msgr2=0x7f7f6c0b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:27.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.144+0000 7f7f71400640 1 --2- 172.21.15.7:0/3140894338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c0cd750 0x7f7f6c0b47c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c00c480 tx=0x7f7f5c00c940 comp rx=0 tx=0).stop 2024-09-10T21:43:27.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.144+0000 7f7f71400640 1 -- 172.21.15.7:0/3140894338 shutdown_connections 2024-09-10T21:43:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.144+0000 7f7f71400640 1 --2- 172.21.15.7:0/3140894338 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f4c077fa0 0x7f7f4c07a460 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.144+0000 7f7f71400640 1 --2- 172.21.15.7:0/3140894338 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c15fda0 0x7f7f6c1f6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.144+0000 7f7f71400640 1 --2- 172.21.15.7:0/3140894338 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f6c15bc90 0x7f7f6c0ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.144+0000 7f7f71400640 1 --2- 172.21.15.7:0/3140894338 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c0cd750 0x7f7f6c0b47c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.144+0000 7f7f71400640 1 -- 172.21.15.7:0/3140894338 >> 172.21.15.7:0/3140894338 conn(0x7f7f6c0b16d0 msgr2=0x7f7f6c0b2a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.144+0000 7f7f71400640 1 -- 172.21.15.7:0/3140894338 shutdown_connections 2024-09-10T21:43:27.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.144+0000 7f7f71400640 1 -- 172.21.15.7:0/3140894338 wait complete. 2024-09-10T21:43:27.161 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:43:27.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.323+0000 7f02be200640 1 -- 172.21.15.7:0/2174025998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b81528b0 msgr2=0x7f02b80b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:27.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.323+0000 7f02be200640 1 --2- 172.21.15.7:0/2174025998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b81528b0 0x7f02b80b4990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f02a8005fd0 tx=0x7f02a802a5d0 comp rx=0 tx=0).stop 2024-09-10T21:43:27.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.323+0000 7f02be200640 1 -- 172.21.15.7:0/2174025998 shutdown_connections 2024-09-10T21:43:27.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.323+0000 7f02be200640 1 --2- 172.21.15.7:0/2174025998 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02b80d0470 0x7f02b80b2ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:27.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.323+0000 7f02be200640 1 --2- 172.21.15.7:0/2174025998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b81528b0 0x7f02b80b4990 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:27.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.323+0000 7f02be200640 1 -- 172.21.15.7:0/2174025998 >> 172.21.15.7:0/2174025998 conn(0x7f02b806cd00 msgr2=0x7f02b806d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:27.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.323+0000 7f02be200640 1 -- 172.21.15.7:0/2174025998 shutdown_connections 2024-09-10T21:43:27.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.323+0000 7f02be200640 1 -- 172.21.15.7:0/2174025998 wait complete. 2024-09-10T21:43:27.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.325+0000 7f02be200640 1 Processor -- start 2024-09-10T21:43:27.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.325+0000 7f02be200640 1 -- start start 2024-09-10T21:43:27.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.326+0000 7f02be200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b8070510 0x7f02b81f84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:27.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.326+0000 7f02be200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b80d0470 0x7f02b815b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:27.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.326+0000 7f02be200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02b81528b0 0x7f02b81e86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:27.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.326+0000 7f02be200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02b80b3380 con 0x7f02b8070510 2024-09-10T21:43:27.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.326+0000 7f02be200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f02b80b3200 con 0x7f02b80d0470 2024-09-10T21:43:27.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.326+0000 7f02be200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f02b80b3500 con 0x7f02b81528b0 2024-09-10T21:43:27.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.326+0000 7f02bce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b8070510 0x7f02b81f84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:27.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.326+0000 7f02bce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b8070510 0x7f02b81f84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42290/0 (socket says 172.21.15.7:42290) 2024-09-10T21:43:27.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.326+0000 7f02bce00640 1 -- 172.21.15.7:0/885520681 learned_addr learned my addr 172.21.15.7:0/885520681 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:27.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.326+0000 7f02bd800640 1 --2- 172.21.15.7:0/885520681 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02b81528b0 0x7f02b81e86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:27.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.326+0000 7f02afe00640 1 --2- 172.21.15.7:0/885520681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b80d0470 0x7f02b815b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:27.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.327+0000 7f02bce00640 1 -- 172.21.15.7:0/885520681 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02b81528b0 msgr2=0x7f02b81e86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:27.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.327+0000 7f02bce00640 1 --2- 172.21.15.7:0/885520681 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02b81528b0 0x7f02b81e86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:27.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.327+0000 7f02bce00640 1 -- 172.21.15.7:0/885520681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b80d0470 msgr2=0x7f02b815b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:27.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.327+0000 7f02bce00640 1 --2- 172.21.15.7:0/885520681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b80d0470 0x7f02b815b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:27.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.327+0000 7f02bce00640 1 -- 172.21.15.7:0/885520681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b81e95b0 con 0x7f02b8070510 2024-09-10T21:43:27.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.327+0000 7f02bce00640 1 --2- 172.21.15.7:0/885520681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b8070510 0x7f02b81f84c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f02a8034040 tx=0x7f02a80038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:27.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.327+0000 7f02b5600640 1 -- 172.21.15.7:0/885520681 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a803e030 con 0x7f02b8070510 2024-09-10T21:43:27.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.327+0000 7f02b5600640 1 -- 172.21.15.7:0/885520681 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02a8003ca0 con 0x7f02b8070510 2024-09-10T21:43:27.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.327+0000 7f02b5600640 1 -- 172.21.15.7:0/885520681 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a8035040 con 0x7f02b8070510 2024-09-10T21:43:27.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.327+0000 7f02be200640 1 -- 172.21.15.7:0/885520681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b815b8c0 con 0x7f02b8070510 2024-09-10T21:43:27.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.327+0000 7f02be200640 1 -- 172.21.15.7:0/885520681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b81f4d60 con 0x7f02b8070510 2024-09-10T21:43:27.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.328+0000 7f02be200640 1 -- 172.21.15.7:0/885520681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b80b3a20 con 0x7f02b8070510 2024-09-10T21:43:27.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.329+0000 7f02b5600640 1 -- 172.21.15.7:0/885520681 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f02a80049e0 con 0x7f02b8070510 2024-09-10T21:43:27.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.330+0000 7f02b5600640 1 --2- 172.21.15.7:0/885520681 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0294077f70 0x7f029407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:27.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.330+0000 7f02b5600640 1 -- 172.21.15.7:0/885520681 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f02a80ba2f0 con 0x7f02b8070510 2024-09-10T21:43:27.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.331+0000 7f02afe00640 1 --2- 172.21.15.7:0/885520681 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0294077f70 0x7f029407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:27.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.332+0000 7f02afe00640 1 --2- 172.21.15.7:0/885520681 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0294077f70 0x7f029407a430 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f02b80739e0 tx=0x7f02a0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:27.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.334+0000 7f02b5600640 1 -- 172.21.15.7:0/885520681 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f02a8083050 con 0x7f02b8070510 2024-09-10T21:43:27.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.529+0000 7f02be200640 1 -- 172.21.15.7:0/885520681 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f02b80b2400 con 0x7f02b8070510 2024-09-10T21:43:27.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.532+0000 7f02b5600640 1 -- 172.21.15.7:0/885520681 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) ==== 122+0+0 (secure 0 0 0) 0x7f02a80ba0d0 con 0x7f02b8070510 2024-09-10T21:43:27.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.535+0000 7f02be200640 1 -- 172.21.15.7:0/885520681 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0294077f70 msgr2=0x7f029407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:27.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.535+0000 7f02be200640 1 --2- 172.21.15.7:0/885520681 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0294077f70 0x7f029407a430 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f02b80739e0 tx=0x7f02a0009040 comp rx=0 tx=0).stop 2024-09-10T21:43:27.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.535+0000 7f02be200640 1 -- 172.21.15.7:0/885520681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b8070510 msgr2=0x7f02b81f84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:27.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.535+0000 7f02be200640 1 --2- 172.21.15.7:0/885520681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b8070510 0x7f02b81f84c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f02a8034040 tx=0x7f02a80038a0 comp rx=0 tx=0).stop 2024-09-10T21:43:27.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.535+0000 7f02be200640 1 -- 172.21.15.7:0/885520681 shutdown_connections 2024-09-10T21:43:27.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.535+0000 7f02be200640 1 --2- 172.21.15.7:0/885520681 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0294077f70 0x7f029407a430 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:27.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.535+0000 7f02be200640 1 --2- 172.21.15.7:0/885520681 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b80d0470 0x7f02b815b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:27.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.535+0000 7f02be200640 1 --2- 172.21.15.7:0/885520681 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02b81528b0 0x7f02b81e86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:27.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.535+0000 7f02be200640 1 --2- 172.21.15.7:0/885520681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b8070510 0x7f02b81f84c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:27.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.535+0000 7f02be200640 1 -- 172.21.15.7:0/885520681 >> 172.21.15.7:0/885520681 conn(0x7f02b806cd00 msgr2=0x7f02b80ce170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:27.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.535+0000 7f02be200640 1 -- 172.21.15.7:0/885520681 shutdown_connections 2024-09-10T21:43:27.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:27.535+0000 7f02be200640 1 -- 172.21.15.7:0/885520681 wait complete. 2024-09-10T21:43:27.553 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:43:27.553 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:43:27.645 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:43:27.741 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:43:27.741 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:43:27.742 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:43:27.779 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:43:27.779 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:43:27.779 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:43:27.817 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:43:27.817 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:43:27.817 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:43:27.854 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:43:27.854 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:43:27.854 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:43:27.893 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:43:27.893 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:43:27.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:43:28.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.053+0000 7fc834800640 1 -- 172.21.15.7:0/4001483935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc82c0cd750 msgr2=0x7fc82c0ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:28.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.053+0000 7fc834800640 1 --2- 172.21.15.7:0/4001483935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc82c0cd750 0x7fc82c0ccf60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc81c009830 tx=0x7fc81c02a510 comp rx=0 tx=0).stop 2024-09-10T21:43:28.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.053+0000 7fc834800640 1 -- 172.21.15.7:0/4001483935 shutdown_connections 2024-09-10T21:43:28.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.053+0000 7fc834800640 1 --2- 172.21.15.7:0/4001483935 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc82c06fcf0 0x7fc82c070120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:28.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.053+0000 7fc834800640 1 --2- 172.21.15.7:0/4001483935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc82c0cd750 0x7fc82c0ccf60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:28.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.053+0000 7fc834800640 1 -- 172.21.15.7:0/4001483935 >> 172.21.15.7:0/4001483935 conn(0x7fc82c0b16d0 msgr2=0x7fc82c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:28.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.053+0000 7fc834800640 1 -- 172.21.15.7:0/4001483935 shutdown_connections 2024-09-10T21:43:28.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.053+0000 7fc834800640 1 -- 172.21.15.7:0/4001483935 wait complete. 2024-09-10T21:43:28.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.055+0000 7fc834800640 1 Processor -- start 2024-09-10T21:43:28.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.055+0000 7fc834800640 1 -- start start 2024-09-10T21:43:28.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.056+0000 7fc834800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc82c06fcf0 0x7fc82c147d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:28.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.056+0000 7fc834800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc82c0cd750 0x7fc82c1f2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:28.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.056+0000 7fc834800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc82c15bc00 0x7fc82c15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:28.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.056+0000 7fc834800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c073e70 con 0x7fc82c06fcf0 2024-09-10T21:43:28.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.056+0000 7fc834800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc82c073cf0 con 0x7fc82c15bc00 2024-09-10T21:43:28.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.056+0000 7fc834800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc82c073ff0 con 0x7fc82c0cd750 2024-09-10T21:43:28.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.056+0000 7fc831e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc82c06fcf0 0x7fc82c147d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:28.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.056+0000 7fc831e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc82c06fcf0 0x7fc82c147d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42310/0 (socket says 172.21.15.7:42310) 2024-09-10T21:43:28.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.056+0000 7fc831e00640 1 -- 172.21.15.7:0/1400730260 learned_addr learned my addr 172.21.15.7:0/1400730260 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:28.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.056+0000 7fc832800640 1 --2- 172.21.15.7:0/1400730260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc82c15bc00 0x7fc82c15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:28.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.056+0000 7fc831400640 1 --2- 172.21.15.7:0/1400730260 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc82c0cd750 0x7fc82c1f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:28.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.057+0000 7fc831e00640 1 -- 172.21.15.7:0/1400730260 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc82c0cd750 msgr2=0x7fc82c1f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:28.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.057+0000 7fc831e00640 1 --2- 172.21.15.7:0/1400730260 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc82c0cd750 0x7fc82c1f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:28.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.057+0000 7fc831e00640 1 -- 172.21.15.7:0/1400730260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc82c15bc00 msgr2=0x7fc82c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:28.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.057+0000 7fc831e00640 1 --2- 172.21.15.7:0/1400730260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc82c15bc00 0x7fc82c15c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:28.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.057+0000 7fc831e00640 1 -- 172.21.15.7:0/1400730260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc82c1482b0 con 0x7fc82c06fcf0 2024-09-10T21:43:28.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.057+0000 7fc831e00640 1 --2- 172.21.15.7:0/1400730260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc82c06fcf0 0x7fc82c147d70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc82000c460 tx=0x7fc82000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:28.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.057+0000 7fc826000640 1 -- 172.21.15.7:0/1400730260 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820016020 con 0x7fc82c06fcf0 2024-09-10T21:43:28.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.057+0000 7fc826000640 1 -- 172.21.15.7:0/1400730260 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc820012940 con 0x7fc82c06fcf0 2024-09-10T21:43:28.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.057+0000 7fc826000640 1 -- 172.21.15.7:0/1400730260 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820012d90 con 0x7fc82c06fcf0 2024-09-10T21:43:28.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.057+0000 7fc834800640 1 -- 172.21.15.7:0/1400730260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc82c15e820 con 0x7fc82c06fcf0 2024-09-10T21:43:28.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.057+0000 7fc834800640 1 -- 172.21.15.7:0/1400730260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc82c15f0c0 con 0x7fc82c06fcf0 2024-09-10T21:43:28.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.058+0000 7fc834800640 1 -- 172.21.15.7:0/1400730260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc82c0745b0 con 0x7fc82c06fcf0 2024-09-10T21:43:28.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.060+0000 7fc826000640 1 -- 172.21.15.7:0/1400730260 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc82001a3b0 con 0x7fc82c06fcf0 2024-09-10T21:43:28.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.060+0000 7fc826000640 1 --2- 172.21.15.7:0/1400730260 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc810077f70 0x7fc81007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:28.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.061+0000 7fc826000640 1 -- 172.21.15.7:0/1400730260 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7fc820099800 con 0x7fc82c06fcf0 2024-09-10T21:43:28.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.061+0000 7fc831400640 1 --2- 172.21.15.7:0/1400730260 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc810077f70 0x7fc81007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:28.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.062+0000 7fc831400640 1 --2- 172.21.15.7:0/1400730260 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc810077f70 0x7fc81007a430 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc81c034000 tx=0x7fc81c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:28.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.065+0000 7fc826000640 1 -- 172.21.15.7:0/1400730260 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc820020030 con 0x7fc82c06fcf0 2024-09-10T21:43:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.298+0000 7fc834800640 1 -- 172.21.15.7:0/1400730260 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc82c0c6770 con 0x7fc82c06fcf0 2024-09-10T21:43:28.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.299+0000 7fc826000640 1 -- 172.21.15.7:0/1400730260 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) ==== 94+0+3902 (secure 0 0 0) 0x7fc82c1482b0 con 0x7fc82c06fcf0 2024-09-10T21:43:28.300 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:28.300 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":19,"btime":"2024-09-10T21:43:21:715026+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/923954733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":923954733},{"type":"v1","addr":"172.21.15.133:6837","nonce":923954733}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4825,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/13426364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":13426364},{"type":"v1","addr":"172.21.15.7:6835","nonce":13426364}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4887,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2275341781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2275341781},{"type":"v1","addr":"172.21.15.7:6834","nonce":2275341781}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":5071,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/1685012639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":1685012639},{"type":"v1","addr":"172.21.15.133:6836","nonce":1685012639}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":19}],"filesystems":[]} 2024-09-10T21:43:28.300 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 19 2024-09-10T21:43:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.301+0000 7fc834800640 1 -- 172.21.15.7:0/1400730260 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc810077f70 msgr2=0x7fc81007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:28.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.301+0000 7fc834800640 1 --2- 172.21.15.7:0/1400730260 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc810077f70 0x7fc81007a430 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc81c034000 tx=0x7fc81c004730 comp rx=0 tx=0).stop 2024-09-10T21:43:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.301+0000 7fc834800640 1 -- 172.21.15.7:0/1400730260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc82c06fcf0 msgr2=0x7fc82c147d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.301+0000 7fc834800640 1 --2- 172.21.15.7:0/1400730260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc82c06fcf0 0x7fc82c147d70 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc82000c460 tx=0x7fc82000c920 comp rx=0 tx=0).stop 2024-09-10T21:43:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.301+0000 7fc834800640 1 -- 172.21.15.7:0/1400730260 shutdown_connections 2024-09-10T21:43:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.301+0000 7fc834800640 1 --2- 172.21.15.7:0/1400730260 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc810077f70 0x7fc81007a430 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.301+0000 7fc834800640 1 --2- 172.21.15.7:0/1400730260 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc82c15bc00 0x7fc82c15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.301+0000 7fc834800640 1 --2- 172.21.15.7:0/1400730260 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc82c0cd750 0x7fc82c1f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.301+0000 7fc834800640 1 --2- 172.21.15.7:0/1400730260 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc82c06fcf0 0x7fc82c147d70 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.301+0000 7fc834800640 1 -- 172.21.15.7:0/1400730260 >> 172.21.15.7:0/1400730260 conn(0x7fc82c0b16d0 msgr2=0x7fc82c159620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.301+0000 7fc834800640 1 -- 172.21.15.7:0/1400730260 shutdown_connections 2024-09-10T21:43:28.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.301+0000 7fc834800640 1 -- 172.21.15.7:0/1400730260 wait complete. 2024-09-10T21:43:28.318 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:43:28.319 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:43:28.319 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:43:28.319 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:43:28.319 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:43:28.319 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:43:28.320 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:43:28.320 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:43:28.320 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:43:28.320 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:43:28.320 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:43:28.320 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:43:28.320 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:43:28.321 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:43:28.321 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:43:28.321 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:43:28.333 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:43:28.333 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:43:28.335 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:43:28.335 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:43:28.338 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:43:28.338 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:43:28.339 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:43:28.339 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:43:28.378 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:43:28.379 DEBUG:teuthology.parallel:result is None 2024-09-10T21:43:28.381 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:43:28.382 DEBUG:teuthology.parallel:result is None 2024-09-10T21:43:28.382 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:43:28.383 DEBUG:teuthology.parallel:result is None 2024-09-10T21:43:28.385 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:43:28.386 DEBUG:teuthology.parallel:result is None 2024-09-10T21:43:28.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:43:28.511 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-09-10T21:43:28.512 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2024-09-10T21:43:28.512 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2024-09-10T21:43:28.513 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-09-10T21:43:28.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.553+0000 7f68b5a00640 1 -- 172.21.15.7:0/2929484535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b015dfd0 msgr2=0x7f68b00d4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.553+0000 7f68b5a00640 1 --2- 172.21.15.7:0/2929484535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b015dfd0 0x7f68b00d4870 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f68a0009830 tx=0x7f68a002a530 comp rx=0 tx=0).stop 2024-09-10T21:43:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.553+0000 7f68b5a00640 1 -- 172.21.15.7:0/2929484535 shutdown_connections 2024-09-10T21:43:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.553+0000 7f68b5a00640 1 --2- 172.21.15.7:0/2929484535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b015dfd0 0x7f68b00d4870 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.553+0000 7f68b5a00640 1 -- 172.21.15.7:0/2929484535 >> 172.21.15.7:0/2929484535 conn(0x7f68b00d3d50 msgr2=0x7f68b00d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.553+0000 7f68b5a00640 1 -- 172.21.15.7:0/2929484535 shutdown_connections 2024-09-10T21:43:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.553+0000 7f68b5a00640 1 -- 172.21.15.7:0/2929484535 wait complete. 2024-09-10T21:43:28.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.554+0000 7f68b5a00640 1 Processor -- start 2024-09-10T21:43:28.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.555+0000 7f68b5a00640 1 -- start start 2024-09-10T21:43:28.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.555+0000 7f68b5a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68b00b2b50 0x7f68b00cd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:28.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.555+0000 7f68b5a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b00d0e70 0x7f68b00c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.556+0000 7f68b5a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b015dfd0 0x7f68b00b8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.556+0000 7f68b5a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68b00d2770 con 0x7f68b00d0e70 2024-09-10T21:43:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.556+0000 7f68b5a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f68b00d25f0 con 0x7f68b015dfd0 2024-09-10T21:43:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.556+0000 7f68b5a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f68b00d28f0 con 0x7f68b00b2b50 2024-09-10T21:43:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.556+0000 7f68a7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b00d0e70 0x7f68b00c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.556+0000 7f68a7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b00d0e70 0x7f68b00c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42368/0 (socket says 172.21.15.7:42368) 2024-09-10T21:43:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.556+0000 7f68a7400640 1 -- 172.21.15.7:0/1432857198 learned_addr learned my addr 172.21.15.7:0/1432857198 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.556+0000 7f68afe00640 1 --2- 172.21.15.7:0/1432857198 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68b00b2b50 0x7f68b00cd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.556+0000 7f68b5000640 1 --2- 172.21.15.7:0/1432857198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b015dfd0 0x7f68b00b8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.556+0000 7f68a7400640 1 -- 172.21.15.7:0/1432857198 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68b00b2b50 msgr2=0x7f68b00cd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.556+0000 7f68a7400640 1 --2- 172.21.15.7:0/1432857198 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68b00b2b50 0x7f68b00cd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.556+0000 7f68a7400640 1 -- 172.21.15.7:0/1432857198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b015dfd0 msgr2=0x7f68b00b8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:28.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.556+0000 7f68a7400640 1 --2- 172.21.15.7:0/1432857198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b015dfd0 0x7f68b00b8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:28.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.556+0000 7f68a7400640 1 -- 172.21.15.7:0/1432857198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b00c8c30 con 0x7f68b00d0e70 2024-09-10T21:43:28.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.557+0000 7f68a7400640 1 --2- 172.21.15.7:0/1432857198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b00d0e70 0x7f68b00c8190 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f689800c450 tx=0x7f689800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:28.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.557+0000 7f68acc00640 1 -- 172.21.15.7:0/1432857198 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68980052d0 con 0x7f68b00d0e70 2024-09-10T21:43:28.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.557+0000 7f68acc00640 1 -- 172.21.15.7:0/1432857198 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6898005470 con 0x7f68b00d0e70 2024-09-10T21:43:28.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.557+0000 7f68b5a00640 1 -- 172.21.15.7:0/1432857198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b01f8000 con 0x7f68b00d0e70 2024-09-10T21:43:28.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.557+0000 7f68acc00640 1 -- 172.21.15.7:0/1432857198 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6898012870 con 0x7f68b00d0e70 2024-09-10T21:43:28.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.557+0000 7f68b5a00640 1 -- 172.21.15.7:0/1432857198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b01f85c0 con 0x7f68b00d0e70 2024-09-10T21:43:28.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.558+0000 7f68b5a00640 1 -- 172.21.15.7:0/1432857198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b00d2d50 con 0x7f68b00d0e70 2024-09-10T21:43:28.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.559+0000 7f68acc00640 1 -- 172.21.15.7:0/1432857198 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f68980035f0 con 0x7f68b00d0e70 2024-09-10T21:43:28.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.560+0000 7f68acc00640 1 --2- 172.21.15.7:0/1432857198 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f688c077fa0 0x7f688c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:28.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.560+0000 7f68acc00640 1 -- 172.21.15.7:0/1432857198 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4943+0+0 (secure 0 0 0) 0x7f6898099410 con 0x7f68b00d0e70 2024-09-10T21:43:28.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.561+0000 7f68afe00640 1 --2- 172.21.15.7:0/1432857198 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f688c077fa0 0x7f688c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:28.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.561+0000 7f68afe00640 1 --2- 172.21.15.7:0/1432857198 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f688c077fa0 0x7f688c07a460 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f68a0034000 tx=0x7f68a0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:28.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.564+0000 7f68acc00640 1 -- 172.21.15.7:0/1432857198 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6898061bb0 con 0x7f68b00d0e70 2024-09-10T21:43:28.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:28.780+0000 7f68b5a00640 1 -- 172.21.15.7:0/1432857198 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f68b00b9cd0 con 0x7f68b00d0e70 2024-09-10T21:43:29.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.781+0000 7f68acc00640 1 -- 172.21.15.7:0/1432857198 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v40) ==== 97+0+0 (secure 0 0 0) 0x7f68980665e0 con 0x7f68b00d0e70 2024-09-10T21:43:29.783 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-09-10T21:43:29.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.784+0000 7f68b5a00640 1 -- 172.21.15.7:0/1432857198 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f688c077fa0 msgr2=0x7f688c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:29.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.784+0000 7f68b5a00640 1 --2- 172.21.15.7:0/1432857198 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f688c077fa0 0x7f688c07a460 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f68a0034000 tx=0x7f68a0004730 comp rx=0 tx=0).stop 2024-09-10T21:43:29.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.784+0000 7f68b5a00640 1 -- 172.21.15.7:0/1432857198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b00d0e70 msgr2=0x7f68b00c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:29.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.784+0000 7f68b5a00640 1 --2- 172.21.15.7:0/1432857198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b00d0e70 0x7f68b00c8190 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f689800c450 tx=0x7f689800c910 comp rx=0 tx=0).stop 2024-09-10T21:43:29.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.784+0000 7f68b5a00640 1 -- 172.21.15.7:0/1432857198 shutdown_connections 2024-09-10T21:43:29.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.784+0000 7f68b5a00640 1 --2- 172.21.15.7:0/1432857198 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f688c077fa0 0x7f688c07a460 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:29.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.784+0000 7f68b5a00640 1 --2- 172.21.15.7:0/1432857198 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b015dfd0 0x7f68b00b8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:29.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.784+0000 7f68b5a00640 1 --2- 172.21.15.7:0/1432857198 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68b00d0e70 0x7f68b00c8190 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:29.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.784+0000 7f68b5a00640 1 --2- 172.21.15.7:0/1432857198 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68b00b2b50 0x7f68b00cd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:29.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.784+0000 7f68b5a00640 1 -- 172.21.15.7:0/1432857198 >> 172.21.15.7:0/1432857198 conn(0x7f68b00d3d50 msgr2=0x7f68b006d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:29.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.784+0000 7f68b5a00640 1 -- 172.21.15.7:0/1432857198 shutdown_connections 2024-09-10T21:43:29.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.785+0000 7f68b5a00640 1 -- 172.21.15.7:0/1432857198 wait complete. 2024-09-10T21:43:29.804 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:43:29.850 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:43:29.854 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:43:29.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.970+0000 7f96b5800640 1 -- 172.21.15.7:0/55164235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96b00b3150 msgr2=0x7f96b0166950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:29.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.970+0000 7f96b5800640 1 --2- 172.21.15.7:0/55164235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96b00b3150 0x7f96b0166950 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f96a8009830 tx=0x7f96a802a5c0 comp rx=0 tx=0).stop 2024-09-10T21:43:29.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.970+0000 7f96b5800640 1 -- 172.21.15.7:0/55164235 shutdown_connections 2024-09-10T21:43:29.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.970+0000 7f96b5800640 1 --2- 172.21.15.7:0/55164235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96b00b3150 0x7f96b0166950 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:29.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.970+0000 7f96b5800640 1 -- 172.21.15.7:0/55164235 >> 172.21.15.7:0/55164235 conn(0x7f96b00b1d70 msgr2=0x7f96b00cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:29.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.970+0000 7f96b5800640 1 -- 172.21.15.7:0/55164235 shutdown_connections 2024-09-10T21:43:29.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.970+0000 7f96b5800640 1 -- 172.21.15.7:0/55164235 wait complete. 2024-09-10T21:43:29.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.972+0000 7f96b5800640 1 Processor -- start 2024-09-10T21:43:29.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.972+0000 7f96b5800640 1 -- start start 2024-09-10T21:43:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.973+0000 7f96b5800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96b00b3150 0x7f96b0144180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:29.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.973+0000 7f96b5800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96b00b4c90 0x7f96b01f5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.973+0000 7f96b5800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96b0145540 0x7f96b0153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.973+0000 7f96b5800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f96b0167010 con 0x7f96b0145540 2024-09-10T21:43:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.973+0000 7f96b5800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f96b0166e90 con 0x7f96b00b3150 2024-09-10T21:43:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.973+0000 7f96b5800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f96b0167190 con 0x7f96b00b4c90 2024-09-10T21:43:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.973+0000 7f96b4e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96b0145540 0x7f96b0153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.973+0000 7f96b4e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96b0145540 0x7f96b0153e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42376/0 (socket says 172.21.15.7:42376) 2024-09-10T21:43:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.973+0000 7f96b4e00640 1 -- 172.21.15.7:0/1185708315 learned_addr learned my addr 172.21.15.7:0/1185708315 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.973+0000 7f96afe00640 1 --2- 172.21.15.7:0/1185708315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96b00b3150 0x7f96b0144180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.973+0000 7f96af400640 1 --2- 172.21.15.7:0/1185708315 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96b00b4c90 0x7f96b01f5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.973+0000 7f96b4e00640 1 -- 172.21.15.7:0/1185708315 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96b00b4c90 msgr2=0x7f96b01f5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.973+0000 7f96b4e00640 1 --2- 172.21.15.7:0/1185708315 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96b00b4c90 0x7f96b01f5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:29.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.973+0000 7f96b4e00640 1 -- 172.21.15.7:0/1185708315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96b00b3150 msgr2=0x7f96b0144180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.973+0000 7f96b4e00640 1 --2- 172.21.15.7:0/1185708315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96b00b3150 0x7f96b0144180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.973+0000 7f96b4e00640 1 -- 172.21.15.7:0/1185708315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96b0153640 con 0x7f96b0145540 2024-09-10T21:43:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.974+0000 7f96b4e00640 1 --2- 172.21.15.7:0/1185708315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96b0145540 0x7f96b0153e00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f96a802abb0 tx=0x7f96a8032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.974+0000 7f96a7e00640 1 -- 172.21.15.7:0/1185708315 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8040070 con 0x7f96b0145540 2024-09-10T21:43:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.974+0000 7f96a7e00640 1 -- 172.21.15.7:0/1185708315 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96a8005310 con 0x7f96b0145540 2024-09-10T21:43:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.974+0000 7f96a7e00640 1 -- 172.21.15.7:0/1185708315 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96a8005c90 con 0x7f96b0145540 2024-09-10T21:43:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.974+0000 7f96b5800640 1 -- 172.21.15.7:0/1185708315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96b01f5fb0 con 0x7f96b0145540 2024-09-10T21:43:29.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.974+0000 7f96b5800640 1 -- 172.21.15.7:0/1185708315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96b01f6560 con 0x7f96b0145540 2024-09-10T21:43:29.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.975+0000 7f96b5800640 1 -- 172.21.15.7:0/1185708315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96b00b2a40 con 0x7f96b0145540 2024-09-10T21:43:29.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.976+0000 7f96a7e00640 1 -- 172.21.15.7:0/1185708315 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f96a80047a0 con 0x7f96b0145540 2024-09-10T21:43:29.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.977+0000 7f96a7e00640 1 --2- 172.21.15.7:0/1185708315 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9690077fa0 0x7f969007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:29.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.977+0000 7f96a7e00640 1 -- 172.21.15.7:0/1185708315 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f96a80bac40 con 0x7f96b0145540 2024-09-10T21:43:29.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.978+0000 7f96afe00640 1 --2- 172.21.15.7:0/1185708315 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9690077fa0 0x7f969007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:29.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.978+0000 7f96afe00640 1 --2- 172.21.15.7:0/1185708315 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9690077fa0 0x7f969007a460 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f96a00012e0 tx=0x7f96a0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:29.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:29.981+0000 7f96a7e00640 1 -- 172.21.15.7:0/1185708315 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f96a8083430 con 0x7f96b0145540 2024-09-10T21:43:30.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.250+0000 7f96b5800640 1 -- 172.21.15.7:0/1185708315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f96b0141420 con 0x7f96b0145540 2024-09-10T21:43:30.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.250+0000 7f96a7e00640 1 -- 172.21.15.7:0/1185708315 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v32) ==== 80+0+5304 (secure 0 0 0) 0x7f96a8087e60 con 0x7f96b0145540 2024-09-10T21:43:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:30.252 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:43:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-09-10T21:43:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-09-10T21:43:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmgRMuLBAA88QT12AXv2qf11IxU20Xbg==", 2024-09-10T21:43:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:43:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:43:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:43:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:43:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:43:30.252 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-09-10T21:43:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmRUM2BBAAwcQM6z+hjZI4ZWBlVrYMwQ==", 2024-09-10T21:43:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:43:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:43:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:43:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:43:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:43:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.253 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-09-10T21:43:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmnq31NRAAj1eJUAIS986idxppfeXEAA==", 2024-09-10T21:43:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:43:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:43:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:43:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:43:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:43:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-09-10T21:43:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmy43uDRAAzdKjLxZMGROuDABrsJ0zgQ==", 2024-09-10T21:43:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:43:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:43:30.254 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:43:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:43:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:43:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-09-10T21:43:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDwvOBmzfgtOxAAk6K5fU0Uar4VW9nUUIuNdw==", 2024-09-10T21:43:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:43:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:43:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:43:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:43:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.255 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-09-10T21:43:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD0vOBmHcRzChAA3z+CaGgu5NkyUmjpqzmKjw==", 2024-09-10T21:43:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:43:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:43:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:43:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:43:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-09-10T21:43:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD3vOBm9sArFBAAGQlyiw6qEcyumEBICWEBKA==", 2024-09-10T21:43:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:43:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:43:30.256 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:43:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:43:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-09-10T21:43:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD6vOBmkO3rHxAAW0zPydYZRGUKjGh3yIyPCA==", 2024-09-10T21:43:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:43:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:43:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:43:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:43:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:30.257 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-09-10T21:43:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAAveBmlPABMxAAeazVOzc1DhsVAwoHn87QIA==", 2024-09-10T21:43:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:43:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:43:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:43:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:43:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-09-10T21:43:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAEveBmL9qzARAApu8nWIY9HVsDl4yUnfhH9w==", 2024-09-10T21:43:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:43:30.258 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:43:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:43:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:43:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-09-10T21:43:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAHveBmJ4ofDRAAEiCFN8nGzVSR3u2gKoT+2Q==", 2024-09-10T21:43:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:43:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:43:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:43:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:43:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.259 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-09-10T21:43:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAKveBmoUGEGBAAhXmkUaniuAQrBiYPWr8Tog==", 2024-09-10T21:43:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:43:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:43:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:43:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:43:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-09-10T21:43:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBm0aqcFhAAtid2bnzjcpaYAbU7g0JcOA==", 2024-09-10T21:43:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:43:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:43:30.260 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-09-10T21:43:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:43:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-09-10T21:43:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmqf+MIxAAib5hVnbBsw/bJPqsQszJwg==", 2024-09-10T21:43:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:43:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:43:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:43:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:43:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:43:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.261 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-09-10T21:43:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmybQhHRAAZUwqnIsihEWnHZWQTYr+uQ==", 2024-09-10T21:43:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:43:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:43:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:43:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:43:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:43:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-09-10T21:43:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmDbrrKRAAUxb3EBfKIJ4FAGHW3QIsLg==", 2024-09-10T21:43:30.262 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:43:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:43:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:43:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:43:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:43:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-09-10T21:43:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDtvOBmF+eEOBAA3icvNqI0nWDShE49DZ3bvg==", 2024-09-10T21:43:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:43:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:43:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:43:30.263 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:43:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:43:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-09-10T21:43:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmsu6OJRAAhdgDvpeHY/zDKUeUOv6xxA==", 2024-09-10T21:43:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:43:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:43:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-10T21:43:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:43:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:30.264 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-09-10T21:43:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmKBHLHhAA0W5My19tIhN1v2qev4cBTA==", 2024-09-10T21:43:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:43:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:43:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-10T21:43:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:43:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.265 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T21:43:30.265 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:43:30.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.253+0000 7f96b5800640 1 -- 172.21.15.7:0/1185708315 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9690077fa0 msgr2=0x7f969007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:30.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.253+0000 7f96b5800640 1 --2- 172.21.15.7:0/1185708315 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9690077fa0 0x7f969007a460 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f96a00012e0 tx=0x7f96a0009040 comp rx=0 tx=0).stop 2024-09-10T21:43:30.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.253+0000 7f96b5800640 1 -- 172.21.15.7:0/1185708315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96b0145540 msgr2=0x7f96b0153e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:30.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.253+0000 7f96b5800640 1 --2- 172.21.15.7:0/1185708315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96b0145540 0x7f96b0153e00 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f96a802abb0 tx=0x7f96a8032c10 comp rx=0 tx=0).stop 2024-09-10T21:43:30.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.253+0000 7f96b5800640 1 -- 172.21.15.7:0/1185708315 shutdown_connections 2024-09-10T21:43:30.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.253+0000 7f96b5800640 1 --2- 172.21.15.7:0/1185708315 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9690077fa0 0x7f969007a460 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:30.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.253+0000 7f96b5800640 1 --2- 172.21.15.7:0/1185708315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96b0145540 0x7f96b0153e00 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:30.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.253+0000 7f96b5800640 1 --2- 172.21.15.7:0/1185708315 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96b00b4c90 0x7f96b01f5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:30.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.253+0000 7f96b5800640 1 --2- 172.21.15.7:0/1185708315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96b00b3150 0x7f96b0144180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:30.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.253+0000 7f96b5800640 1 -- 172.21.15.7:0/1185708315 >> 172.21.15.7:0/1185708315 conn(0x7f96b00b1d70 msgr2=0x7f96b00b44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:30.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.253+0000 7f96b5800640 1 -- 172.21.15.7:0/1185708315 shutdown_connections 2024-09-10T21:43:30.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.253+0000 7f96b5800640 1 -- 172.21.15.7:0/1185708315 wait complete. 2024-09-10T21:43:30.271 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:43:30.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.469+0000 7f462dc00640 1 -- 172.21.15.7:0/561071207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46280cd750 msgr2=0x7f46280ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:30.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.469+0000 7f462dc00640 1 --2- 172.21.15.7:0/561071207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46280cd750 0x7f46280ccef0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f462815fa90 tx=0x7f461002a470 comp rx=0 tx=0).stop 2024-09-10T21:43:30.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.470+0000 7f462dc00640 1 -- 172.21.15.7:0/561071207 shutdown_connections 2024-09-10T21:43:30.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.470+0000 7f462dc00640 1 --2- 172.21.15.7:0/561071207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f462815fda0 0x7f46281601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:30.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.470+0000 7f462dc00640 1 --2- 172.21.15.7:0/561071207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46280cd750 0x7f46280ccef0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:30.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.470+0000 7f462dc00640 1 -- 172.21.15.7:0/561071207 >> 172.21.15.7:0/561071207 conn(0x7f46280b16d0 msgr2=0x7f46280b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:30.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.470+0000 7f462dc00640 1 -- 172.21.15.7:0/561071207 shutdown_connections 2024-09-10T21:43:30.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.470+0000 7f462dc00640 1 -- 172.21.15.7:0/561071207 wait complete. 2024-09-10T21:43:30.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.471+0000 7f462dc00640 1 Processor -- start 2024-09-10T21:43:30.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.472+0000 7f462dc00640 1 -- start start 2024-09-10T21:43:30.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.472+0000 7f462dc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46280cd750 0x7f46280c5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:30.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.472+0000 7f462dc00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f462815bc90 0x7f462815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:30.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.472+0000 7f462dc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f462815fda0 0x7f46281f7a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:30.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.472+0000 7f462dc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f462806fe70 con 0x7f46280cd750 2024-09-10T21:43:30.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.472+0000 7f462dc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f462806fcf0 con 0x7f462815fda0 2024-09-10T21:43:30.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.472+0000 7f462dc00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f462806fff0 con 0x7f462815bc90 2024-09-10T21:43:30.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.473+0000 7f4626a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46280cd750 0x7f46280c5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:30.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.473+0000 7f4626a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46280cd750 0x7f46280c5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42396/0 (socket says 172.21.15.7:42396) 2024-09-10T21:43:30.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.473+0000 7f4626a00640 1 -- 172.21.15.7:0/1826121163 learned_addr learned my addr 172.21.15.7:0/1826121163 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:30.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.473+0000 7f461e000640 1 --2- 172.21.15.7:0/1826121163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f462815bc90 0x7f462815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:30.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.473+0000 7f4627400640 1 --2- 172.21.15.7:0/1826121163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f462815fda0 0x7f46281f7a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:30.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.473+0000 7f4626a00640 1 -- 172.21.15.7:0/1826121163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f462815bc90 msgr2=0x7f462815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:30.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.473+0000 7f4626a00640 1 --2- 172.21.15.7:0/1826121163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f462815bc90 0x7f462815dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:30.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.473+0000 7f4626a00640 1 -- 172.21.15.7:0/1826121163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f462815fda0 msgr2=0x7f46281f7a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:30.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.473+0000 7f4626a00640 1 --2- 172.21.15.7:0/1826121163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f462815fda0 0x7f46281f7a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:30.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.473+0000 7f4626a00640 1 -- 172.21.15.7:0/1826121163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f462815e900 con 0x7f46280cd750 2024-09-10T21:43:30.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.473+0000 7f4627400640 1 --2- 172.21.15.7:0/1826121163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f462815fda0 0x7f46281f7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:30.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.473+0000 7f461e000640 1 --2- 172.21.15.7:0/1826121163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f462815bc90 0x7f462815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:30.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.473+0000 7f4626a00640 1 --2- 172.21.15.7:0/1826121163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46280cd750 0x7f46280c5fe0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f461800c450 tx=0x7f461800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:30.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.474+0000 7f461f400640 1 -- 172.21.15.7:0/1826121163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618016020 con 0x7f46280cd750 2024-09-10T21:43:30.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.474+0000 7f461f400640 1 -- 172.21.15.7:0/1826121163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4618012920 con 0x7f46280cd750 2024-09-10T21:43:30.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.474+0000 7f461f400640 1 -- 172.21.15.7:0/1826121163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618012d70 con 0x7f46280cd750 2024-09-10T21:43:30.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.474+0000 7f462dc00640 1 -- 172.21.15.7:0/1826121163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f462815ebe0 con 0x7f46280cd750 2024-09-10T21:43:30.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.474+0000 7f462dc00640 1 -- 172.21.15.7:0/1826121163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f462815f1a0 con 0x7f46280cd750 2024-09-10T21:43:30.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.475+0000 7f462dc00640 1 -- 172.21.15.7:0/1826121163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46280704a0 con 0x7f46280cd750 2024-09-10T21:43:30.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.476+0000 7f461f400640 1 -- 172.21.15.7:0/1826121163 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f461801b400 con 0x7f46280cd750 2024-09-10T21:43:30.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.477+0000 7f461f400640 1 --2- 172.21.15.7:0/1826121163 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4604077f70 0x7f460407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:30.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.477+0000 7f461f400640 1 -- 172.21.15.7:0/1826121163 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f4618099ac0 con 0x7f46280cd750 2024-09-10T21:43:30.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.478+0000 7f461e000640 1 --2- 172.21.15.7:0/1826121163 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4604077f70 0x7f460407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:30.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.478+0000 7f461e000640 1 --2- 172.21.15.7:0/1826121163 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4604077f70 0x7f460407a430 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4610034000 tx=0x7f4610004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:30.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.481+0000 7f461f400640 1 -- 172.21.15.7:0/1826121163 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4618020030 con 0x7f46280cd750 2024-09-10T21:43:30.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.677+0000 7f462dc00640 1 -- 172.21.15.7:0/1826121163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f46280c7ca0 con 0x7f46280cd750 2024-09-10T21:43:30.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.678+0000 7f461f400640 1 -- 172.21.15.7:0/1826121163 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v40) ==== 84+0+65 (secure 0 0 0) 0x7f4618062870 con 0x7f46280cd750 2024-09-10T21:43:30.679 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:30.680 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-09-10T21:43:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-09-10T21:43:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-09-10T21:43:30.680 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:30.680 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-09-10T21:43:30.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.680+0000 7f462dc00640 1 -- 172.21.15.7:0/1826121163 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4604077f70 msgr2=0x7f460407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.680+0000 7f462dc00640 1 --2- 172.21.15.7:0/1826121163 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4604077f70 0x7f460407a430 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4610034000 tx=0x7f4610004730 comp rx=0 tx=0).stop 2024-09-10T21:43:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.680+0000 7f462dc00640 1 -- 172.21.15.7:0/1826121163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46280cd750 msgr2=0x7f46280c5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.680+0000 7f462dc00640 1 --2- 172.21.15.7:0/1826121163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46280cd750 0x7f46280c5fe0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f461800c450 tx=0x7f461800c910 comp rx=0 tx=0).stop 2024-09-10T21:43:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.680+0000 7f462dc00640 1 -- 172.21.15.7:0/1826121163 shutdown_connections 2024-09-10T21:43:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.680+0000 7f462dc00640 1 --2- 172.21.15.7:0/1826121163 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4604077f70 0x7f460407a430 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.680+0000 7f462dc00640 1 --2- 172.21.15.7:0/1826121163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f462815fda0 0x7f46281f7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.680+0000 7f462dc00640 1 --2- 172.21.15.7:0/1826121163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f462815bc90 0x7f462815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.680+0000 7f462dc00640 1 --2- 172.21.15.7:0/1826121163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46280cd750 0x7f46280c5fe0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.680+0000 7f462dc00640 1 -- 172.21.15.7:0/1826121163 >> 172.21.15.7:0/1826121163 conn(0x7f46280b16d0 msgr2=0x7f46280b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.680+0000 7f462dc00640 1 -- 172.21.15.7:0/1826121163 shutdown_connections 2024-09-10T21:43:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.680+0000 7f462dc00640 1 -- 172.21.15.7:0/1826121163 wait complete. 2024-09-10T21:43:30.698 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:43:30.698 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:43:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.863+0000 7f36b2800640 1 -- 172.21.15.7:0/2799822639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36ac15b900 msgr2=0x7f36ac070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.863+0000 7f36b2800640 1 --2- 172.21.15.7:0/2799822639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36ac15b900 0x7f36ac070230 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f369c009860 tx=0x7f369c02a560 comp rx=0 tx=0).stop 2024-09-10T21:43:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.863+0000 7f36b2800640 1 -- 172.21.15.7:0/2799822639 shutdown_connections 2024-09-10T21:43:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.863+0000 7f36b2800640 1 --2- 172.21.15.7:0/2799822639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36ac15b900 0x7f36ac070230 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.863+0000 7f36b2800640 1 -- 172.21.15.7:0/2799822639 >> 172.21.15.7:0/2799822639 conn(0x7f36ac0b1c10 msgr2=0x7f36ac0b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.863+0000 7f36b2800640 1 -- 172.21.15.7:0/2799822639 shutdown_connections 2024-09-10T21:43:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.863+0000 7f36b2800640 1 -- 172.21.15.7:0/2799822639 wait complete. 2024-09-10T21:43:30.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.865+0000 7f36b2800640 1 Processor -- start 2024-09-10T21:43:30.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.865+0000 7f36b2800640 1 -- start start 2024-09-10T21:43:30.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.866+0000 7f36b2800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36ac0ace80 0x7f36ac1f8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:30.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.866+0000 7f36b2800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36ac15b900 0x7f36ac1f4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:30.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.866+0000 7f36b2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36ac15bce0 0x7f36ac09b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:30.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.866+0000 7f36b2800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac0aef10 con 0x7f36ac15bce0 2024-09-10T21:43:30.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.866+0000 7f36b2800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f36ac0aed90 con 0x7f36ac15b900 2024-09-10T21:43:30.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.866+0000 7f36b2800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f36ac0af090 con 0x7f36ac0ace80 2024-09-10T21:43:30.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.866+0000 7f36abe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36ac15bce0 0x7f36ac09b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:30.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.867+0000 7f36abe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36ac15bce0 0x7f36ac09b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42410/0 (socket says 172.21.15.7:42410) 2024-09-10T21:43:30.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.867+0000 7f36abe00640 1 -- 172.21.15.7:0/4168922787 learned_addr learned my addr 172.21.15.7:0/4168922787 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:30.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.867+0000 7f36ab400640 1 --2- 172.21.15.7:0/4168922787 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36ac0ace80 0x7f36ac1f8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.867+0000 7f36a2a00640 1 --2- 172.21.15.7:0/4168922787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36ac15b900 0x7f36ac1f4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.867+0000 7f36abe00640 1 -- 172.21.15.7:0/4168922787 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36ac0ace80 msgr2=0x7f36ac1f8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.867+0000 7f36abe00640 1 --2- 172.21.15.7:0/4168922787 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36ac0ace80 0x7f36ac1f8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.867+0000 7f36abe00640 1 -- 172.21.15.7:0/4168922787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36ac15b900 msgr2=0x7f36ac1f4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.867+0000 7f36abe00640 1 --2- 172.21.15.7:0/4168922787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36ac15b900 0x7f36ac1f4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.867+0000 7f36abe00640 1 -- 172.21.15.7:0/4168922787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36ac1f50c0 con 0x7f36ac15bce0 2024-09-10T21:43:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.867+0000 7f36abe00640 1 --2- 172.21.15.7:0/4168922787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36ac15bce0 0x7f36ac09b460 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f36a400c700 tx=0x7f36a400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.868+0000 7f36a3e00640 1 -- 172.21.15.7:0/4168922787 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a4003b70 con 0x7f36ac15bce0 2024-09-10T21:43:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.868+0000 7f36a3e00640 1 -- 172.21.15.7:0/4168922787 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36a4003d10 con 0x7f36ac15bce0 2024-09-10T21:43:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.868+0000 7f36a3e00640 1 -- 172.21.15.7:0/4168922787 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36a400aa50 con 0x7f36ac15bce0 2024-09-10T21:43:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.868+0000 7f36b2800640 1 -- 172.21.15.7:0/4168922787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36ac1f5370 con 0x7f36ac15bce0 2024-09-10T21:43:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.868+0000 7f36b2800640 1 -- 172.21.15.7:0/4168922787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36ac1f5930 con 0x7f36ac15bce0 2024-09-10T21:43:30.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.869+0000 7f36b2800640 1 -- 172.21.15.7:0/4168922787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36ac0af540 con 0x7f36ac15bce0 2024-09-10T21:43:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.870+0000 7f36a3e00640 1 -- 172.21.15.7:0/4168922787 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f36a400abf0 con 0x7f36ac15bce0 2024-09-10T21:43:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.871+0000 7f36a3e00640 1 --2- 172.21.15.7:0/4168922787 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3688077fa0 0x7f368807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.871+0000 7f36a3e00640 1 -- 172.21.15.7:0/4168922787 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4857+0+0 (secure 0 0 0) 0x7f36a4099570 con 0x7f36ac15bce0 2024-09-10T21:43:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.871+0000 7f36ab400640 1 --2- 172.21.15.7:0/4168922787 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3688077fa0 0x7f368807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:30.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.872+0000 7f36ab400640 1 --2- 172.21.15.7:0/4168922787 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3688077fa0 0x7f368807a460 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f369c034000 tx=0x7f369c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:30.875+0000 7f36a3e00640 1 -- 172.21.15.7:0/4168922787 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f36a4061e00 con 0x7f36ac15bce0 2024-09-10T21:43:31.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.071+0000 7f36b2800640 1 -- 172.21.15.7:0/4168922787 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f36ac0afb80 con 0x7f36ac15bce0 2024-09-10T21:43:31.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.790+0000 7f36a3e00640 1 -- 172.21.15.7:0/4168922787 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v41) ==== 138+0+0 (secure 0 0 0) 0x7f36a4066830 con 0x7f36ac15bce0 2024-09-10T21:43:31.792 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-09-10T21:43:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.793+0000 7f36b2800640 1 -- 172.21.15.7:0/4168922787 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3688077fa0 msgr2=0x7f368807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.793+0000 7f36b2800640 1 --2- 172.21.15.7:0/4168922787 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3688077fa0 0x7f368807a460 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f369c034000 tx=0x7f369c004730 comp rx=0 tx=0).stop 2024-09-10T21:43:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.793+0000 7f36b2800640 1 -- 172.21.15.7:0/4168922787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36ac15bce0 msgr2=0x7f36ac09b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.793+0000 7f36b2800640 1 --2- 172.21.15.7:0/4168922787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36ac15bce0 0x7f36ac09b460 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f36a400c700 tx=0x7f36a400cbc0 comp rx=0 tx=0).stop 2024-09-10T21:43:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.794+0000 7f36b2800640 1 -- 172.21.15.7:0/4168922787 shutdown_connections 2024-09-10T21:43:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.794+0000 7f36b2800640 1 --2- 172.21.15.7:0/4168922787 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3688077fa0 0x7f368807a460 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.794+0000 7f36b2800640 1 --2- 172.21.15.7:0/4168922787 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f36ac15b900 0x7f36ac1f4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.794+0000 7f36b2800640 1 --2- 172.21.15.7:0/4168922787 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f36ac15bce0 0x7f36ac09b460 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.794+0000 7f36b2800640 1 --2- 172.21.15.7:0/4168922787 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36ac0ace80 0x7f36ac1f8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:31.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.794+0000 7f36b2800640 1 -- 172.21.15.7:0/4168922787 >> 172.21.15.7:0/4168922787 conn(0x7f36ac0b1c10 msgr2=0x7f36ac09cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:31.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.794+0000 7f36b2800640 1 -- 172.21.15.7:0/4168922787 shutdown_connections 2024-09-10T21:43:31.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.794+0000 7f36b2800640 1 -- 172.21.15.7:0/4168922787 wait complete. 2024-09-10T21:43:31.817 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:43:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.987+0000 7f0ad7000640 1 -- 172.21.15.7:0/1395680898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ad0074530 msgr2=0x7f0ad015cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.987+0000 7f0ad7000640 1 --2- 172.21.15.7:0/1395680898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ad0074530 0x7f0ad015cc00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00ca2f0 tx=0x7f0ac802a890 comp rx=0 tx=0).stop 2024-09-10T21:43:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.987+0000 7f0ad7000640 1 -- 172.21.15.7:0/1395680898 shutdown_connections 2024-09-10T21:43:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.987+0000 7f0ad7000640 1 --2- 172.21.15.7:0/1395680898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ad0074530 0x7f0ad015cc00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.987+0000 7f0ad7000640 1 -- 172.21.15.7:0/1395680898 >> 172.21.15.7:0/1395680898 conn(0x7f0ad00b16d0 msgr2=0x7f0ad00b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.987+0000 7f0ad7000640 1 -- 172.21.15.7:0/1395680898 shutdown_connections 2024-09-10T21:43:31.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.987+0000 7f0ad7000640 1 -- 172.21.15.7:0/1395680898 wait complete. 2024-09-10T21:43:31.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.989+0000 7f0ad7000640 1 Processor -- start 2024-09-10T21:43:31.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.989+0000 7f0ad7000640 1 -- start start 2024-09-10T21:43:31.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.990+0000 7f0ad7000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ad006e580 0x7f0ad01f5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:31.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.990+0000 7f0ad7000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad0074530 0x7f0ad014f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:31.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.990+0000 7f0ad7000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ad0159680 0x7f0ad01f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:31.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.990+0000 7f0ad7000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad00caa10 con 0x7f0ad0074530 2024-09-10T21:43:31.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.990+0000 7f0ad7000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0ad00ca890 con 0x7f0ad006e580 2024-09-10T21:43:31.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.990+0000 7f0ad7000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f0ad00cab90 con 0x7f0ad0159680 2024-09-10T21:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.990+0000 7f0ac7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad0074530 0x7f0ad014f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.990+0000 7f0ac7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad0074530 0x7f0ad014f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42424/0 (socket says 172.21.15.7:42424) 2024-09-10T21:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.990+0000 7f0ac7400640 1 -- 172.21.15.7:0/1844984780 learned_addr learned my addr 172.21.15.7:0/1844984780 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.991+0000 7f0ad5000640 1 --2- 172.21.15.7:0/1844984780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ad0159680 0x7f0ad01f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.991+0000 7f0acfe00640 1 --2- 172.21.15.7:0/1844984780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ad006e580 0x7f0ad01f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.991+0000 7f0ac7400640 1 -- 172.21.15.7:0/1844984780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ad0159680 msgr2=0x7f0ad01f7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.991+0000 7f0ac7400640 1 --2- 172.21.15.7:0/1844984780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ad0159680 0x7f0ad01f7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.991+0000 7f0ac7400640 1 -- 172.21.15.7:0/1844984780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ad006e580 msgr2=0x7f0ad01f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.991+0000 7f0ac7400640 1 --2- 172.21.15.7:0/1844984780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ad006e580 0x7f0ad01f5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.991+0000 7f0ac7400640 1 -- 172.21.15.7:0/1844984780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ad01f5940 con 0x7f0ad0074530 2024-09-10T21:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.991+0000 7f0acfe00640 1 --2- 172.21.15.7:0/1844984780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ad006e580 0x7f0ad01f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:43:31.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.991+0000 7f0ac7400640 1 --2- 172.21.15.7:0/1844984780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad0074530 0x7f0ad014f140 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800c730 tx=0x7f0ab800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.991+0000 7f0accc00640 1 -- 172.21.15.7:0/1844984780 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8006a90 con 0x7f0ad0074530 2024-09-10T21:43:31.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.991+0000 7f0accc00640 1 -- 172.21.15.7:0/1844984780 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ab8006c30 con 0x7f0ad0074530 2024-09-10T21:43:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.991+0000 7f0ad7000640 1 -- 172.21.15.7:0/1844984780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ad014f680 con 0x7f0ad0074530 2024-09-10T21:43:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.991+0000 7f0accc00640 1 -- 172.21.15.7:0/1844984780 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8006e60 con 0x7f0ad0074530 2024-09-10T21:43:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.992+0000 7f0ad7000640 1 -- 172.21.15.7:0/1844984780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ad01f8ca0 con 0x7f0ad0074530 2024-09-10T21:43:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.992+0000 7f0ad7000640 1 -- 172.21.15.7:0/1844984780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ad00cb080 con 0x7f0ad0074530 2024-09-10T21:43:31.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.994+0000 7f0accc00640 1 -- 172.21.15.7:0/1844984780 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0ab8007290 con 0x7f0ad0074530 2024-09-10T21:43:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.995+0000 7f0accc00640 1 --2- 172.21.15.7:0/1844984780 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0aac077f70 0x7f0aac07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.995+0000 7f0accc00640 1 -- 172.21.15.7:0/1844984780 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 5227+0+0 (secure 0 0 0) 0x7f0ab80998c0 con 0x7f0ad0074530 2024-09-10T21:43:31.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.995+0000 7f0acfe00640 1 --2- 172.21.15.7:0/1844984780 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0aac077f70 0x7f0aac07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:31.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.996+0000 7f0acfe00640 1 --2- 172.21.15.7:0/1844984780 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0aac077f70 0x7f0aac07a430 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0007fd0 tx=0x7f0ac0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:32.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:31.999+0000 7f0accc00640 1 -- 172.21.15.7:0/1844984780 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0ad0072830 con 0x7f0ad0074530 2024-09-10T21:43:32.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.192+0000 7f0ad7000640 1 -- 172.21.15.7:0/1844984780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0ad006f6b0 con 0x7f0ad0074530 2024-09-10T21:43:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.797+0000 7f0accc00640 1 -- 172.21.15.7:0/1844984780 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v42) ==== 170+0+0 (secure 0 0 0) 0x7f0ab80075b0 con 0x7f0ad0074530 2024-09-10T21:43:32.798 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-09-10T21:43:32.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.800+0000 7f0ad7000640 1 -- 172.21.15.7:0/1844984780 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0aac077f70 msgr2=0x7f0aac07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:32.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.800+0000 7f0ad7000640 1 --2- 172.21.15.7:0/1844984780 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0aac077f70 0x7f0aac07a430 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0ac0007fd0 tx=0x7f0ac0009040 comp rx=0 tx=0).stop 2024-09-10T21:43:32.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.800+0000 7f0ad7000640 1 -- 172.21.15.7:0/1844984780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad0074530 msgr2=0x7f0ad014f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:32.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.800+0000 7f0ad7000640 1 --2- 172.21.15.7:0/1844984780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad0074530 0x7f0ad014f140 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f0ab800c730 tx=0x7f0ab800cbf0 comp rx=0 tx=0).stop 2024-09-10T21:43:32.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.800+0000 7f0ad7000640 1 -- 172.21.15.7:0/1844984780 shutdown_connections 2024-09-10T21:43:32.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.800+0000 7f0ad7000640 1 --2- 172.21.15.7:0/1844984780 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0aac077f70 0x7f0aac07a430 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:32.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.800+0000 7f0ad7000640 1 --2- 172.21.15.7:0/1844984780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ad0159680 0x7f0ad01f7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:32.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.800+0000 7f0ad7000640 1 --2- 172.21.15.7:0/1844984780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ad0074530 0x7f0ad014f140 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:32.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.800+0000 7f0ad7000640 1 --2- 172.21.15.7:0/1844984780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ad006e580 0x7f0ad01f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:32.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.800+0000 7f0ad7000640 1 -- 172.21.15.7:0/1844984780 >> 172.21.15.7:0/1844984780 conn(0x7f0ad00b16d0 msgr2=0x7f0ad00cb290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:32.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.800+0000 7f0ad7000640 1 -- 172.21.15.7:0/1844984780 shutdown_connections 2024-09-10T21:43:32.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.800+0000 7f0ad7000640 1 -- 172.21.15.7:0/1844984780 wait complete. 2024-09-10T21:43:32.833 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:43:33.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.999+0000 7fb906e00640 1 -- 172.21.15.7:0/3144429538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb900070510 msgr2=0x7fb9000cfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:33.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.999+0000 7fb906e00640 1 --2- 172.21.15.7:0/3144429538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb900070510 0x7fb9000cfd60 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec005fd0 tx=0x7fb8ec02a5d0 comp rx=0 tx=0).stop 2024-09-10T21:43:33.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.999+0000 7fb906e00640 1 -- 172.21.15.7:0/3144429538 shutdown_connections 2024-09-10T21:43:33.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.999+0000 7fb906e00640 1 --2- 172.21.15.7:0/3144429538 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb900070510 0x7fb9000cfd60 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:33.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.999+0000 7fb906e00640 1 --2- 172.21.15.7:0/3144429538 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb9001528b0 0x7fb9000b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:33.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.999+0000 7fb906e00640 1 -- 172.21.15.7:0/3144429538 >> 172.21.15.7:0/3144429538 conn(0x7fb90006cd00 msgr2=0x7fb90006d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:33.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.999+0000 7fb906e00640 1 -- 172.21.15.7:0/3144429538 shutdown_connections 2024-09-10T21:43:33.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:32.999+0000 7fb906e00640 1 -- 172.21.15.7:0/3144429538 wait complete. 2024-09-10T21:43:33.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.001+0000 7fb906e00640 1 Processor -- start 2024-09-10T21:43:33.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.001+0000 7fb906e00640 1 -- start start 2024-09-10T21:43:33.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.002+0000 7fb906e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb900070510 0x7fb9001f4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:33.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.002+0000 7fb906e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9001528b0 0x7fb9000afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:33.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.002+0000 7fb906e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb900152c90 0x7fb9001f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:33.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.002+0000 7fb906e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb9000d0630 con 0x7fb900070510 2024-09-10T21:43:33.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.002+0000 7fb906e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb9000d04b0 con 0x7fb9001528b0 2024-09-10T21:43:33.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.002+0000 7fb906e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fb90015fd40 con 0x7fb900152c90 2024-09-10T21:43:33.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.002+0000 7fb905a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb900070510 0x7fb9001f4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:33.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.002+0000 7fb905a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb900070510 0x7fb9001f4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42432/0 (socket says 172.21.15.7:42432) 2024-09-10T21:43:33.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.002+0000 7fb905a00640 1 -- 172.21.15.7:0/1388214059 learned_addr learned my addr 172.21.15.7:0/1388214059 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:33.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.003+0000 7fb905a00640 1 -- 172.21.15.7:0/1388214059 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb900152c90 msgr2=0x7fb9001f5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:33.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.003+0000 7fb905a00640 1 --2- 172.21.15.7:0/1388214059 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb900152c90 0x7fb9001f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:33.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.003+0000 7fb905a00640 1 -- 172.21.15.7:0/1388214059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9001528b0 msgr2=0x7fb9000afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:33.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.003+0000 7fb905a00640 1 --2- 172.21.15.7:0/1388214059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9001528b0 0x7fb9000afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:33.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.003+0000 7fb905a00640 1 -- 172.21.15.7:0/1388214059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9000b0020 con 0x7fb900070510 2024-09-10T21:43:33.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.003+0000 7fb905a00640 1 --2- 172.21.15.7:0/1388214059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb900070510 0x7fb9001f4ad0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400ea60 tx=0x7fb8f400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:33.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.003+0000 7fb8fd600640 1 -- 172.21.15.7:0/1388214059 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f400cde0 con 0x7fb900070510 2024-09-10T21:43:33.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.003+0000 7fb8fd600640 1 -- 172.21.15.7:0/1388214059 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8f4004020 con 0x7fb900070510 2024-09-10T21:43:33.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.003+0000 7fb8fd600640 1 -- 172.21.15.7:0/1388214059 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4004450 con 0x7fb900070510 2024-09-10T21:43:33.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.003+0000 7fb906e00640 1 -- 172.21.15.7:0/1388214059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9000b02c0 con 0x7fb900070510 2024-09-10T21:43:33.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.003+0000 7fb906e00640 1 -- 172.21.15.7:0/1388214059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9000b0880 con 0x7fb900070510 2024-09-10T21:43:33.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.004+0000 7fb906e00640 1 -- 172.21.15.7:0/1388214059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb900160210 con 0x7fb900070510 2024-09-10T21:43:33.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.006+0000 7fb8fd600640 1 -- 172.21.15.7:0/1388214059 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb8f40035f0 con 0x7fb900070510 2024-09-10T21:43:33.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.007+0000 7fb8fd600640 1 --2- 172.21.15.7:0/1388214059 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb8e0077fa0 0x7fb8e007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:33.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.007+0000 7fb8fd600640 1 -- 172.21.15.7:0/1388214059 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 5609+0+0 (secure 0 0 0) 0x7fb8f409a6e0 con 0x7fb900070510 2024-09-10T21:43:33.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.008+0000 7fb905000640 1 --2- 172.21.15.7:0/1388214059 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb8e0077fa0 0x7fb8e007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:33.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.008+0000 7fb905000640 1 --2- 172.21.15.7:0/1388214059 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb8e0077fa0 0x7fb8e007a460 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec033000 tx=0x7fb8ec0046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:33.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.012+0000 7fb8fd600640 1 -- 172.21.15.7:0/1388214059 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb8f4063230 con 0x7fb900070510 2024-09-10T21:43:33.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.244+0000 7fb906e00640 1 -- 172.21.15.7:0/1388214059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb9000ad980 con 0x7fb900070510 2024-09-10T21:43:33.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.799+0000 7fb8fd600640 1 -- 172.21.15.7:0/1388214059 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v22) ==== 172+0+0 (secure 0 0 0) 0x7fb8f4062b30 con 0x7fb900070510 2024-09-10T21:43:33.801 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 6 and data pool 7 2024-09-10T21:43:33.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.804+0000 7fb906e00640 1 -- 172.21.15.7:0/1388214059 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb8e0077fa0 msgr2=0x7fb8e007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:33.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.804+0000 7fb906e00640 1 --2- 172.21.15.7:0/1388214059 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb8e0077fa0 0x7fb8e007a460 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec033000 tx=0x7fb8ec0046c0 comp rx=0 tx=0).stop 2024-09-10T21:43:33.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.804+0000 7fb906e00640 1 -- 172.21.15.7:0/1388214059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb900070510 msgr2=0x7fb9001f4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:33.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.804+0000 7fb906e00640 1 --2- 172.21.15.7:0/1388214059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb900070510 0x7fb9001f4ad0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb8f400ea60 tx=0x7fb8f400ef20 comp rx=0 tx=0).stop 2024-09-10T21:43:33.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.804+0000 7fb906e00640 1 -- 172.21.15.7:0/1388214059 shutdown_connections 2024-09-10T21:43:33.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.804+0000 7fb906e00640 1 --2- 172.21.15.7:0/1388214059 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb8e0077fa0 0x7fb8e007a460 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:33.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.804+0000 7fb906e00640 1 --2- 172.21.15.7:0/1388214059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9001528b0 0x7fb9000afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:33.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.804+0000 7fb906e00640 1 --2- 172.21.15.7:0/1388214059 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb900152c90 0x7fb9001f5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:33.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.804+0000 7fb906e00640 1 --2- 172.21.15.7:0/1388214059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb900070510 0x7fb9001f4ad0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:33.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.804+0000 7fb906e00640 1 -- 172.21.15.7:0/1388214059 >> 172.21.15.7:0/1388214059 conn(0x7fb90006cd00 msgr2=0x7fb9000ae000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:33.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.805+0000 7fb906e00640 1 -- 172.21.15.7:0/1388214059 shutdown_connections 2024-09-10T21:43:33.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:33.806+0000 7fb906e00640 1 -- 172.21.15.7:0/1388214059 wait complete. 2024-09-10T21:43:33.841 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:43:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.005+0000 7f5f04400640 1 -- 172.21.15.7:0/1787631302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee40052e0 msgr2=0x7f5ee400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.005+0000 7f5f04400640 1 --2- 172.21.15.7:0/1787631302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee40052e0 0x7f5ee400b9c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4001d90 tx=0x7f5ee4005830 comp rx=0 tx=0).stop 2024-09-10T21:43:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.005+0000 7f5f04400640 1 -- 172.21.15.7:0/1787631302 shutdown_connections 2024-09-10T21:43:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.005+0000 7f5f04400640 1 --2- 172.21.15.7:0/1787631302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee40052e0 0x7f5ee400b9c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.005+0000 7f5f04400640 1 -- 172.21.15.7:0/1787631302 >> 172.21.15.7:0/1787631302 conn(0x7f5efc0b16d0 msgr2=0x7f5efc0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.005+0000 7f5f04400640 1 -- 172.21.15.7:0/1787631302 shutdown_connections 2024-09-10T21:43:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.005+0000 7f5f04400640 1 -- 172.21.15.7:0/1787631302 wait complete. 2024-09-10T21:43:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.007+0000 7f5f04400640 1 Processor -- start 2024-09-10T21:43:34.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.007+0000 7f5f04400640 1 -- start start 2024-09-10T21:43:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.008+0000 7f5f04400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee40052e0 0x7f5efc1f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.008+0000 7f5f04400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5efc0ad8f0 0x7f5efc0c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.008+0000 7f5f04400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5efc0afb50 0x7f5efc0affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.008+0000 7f5f04400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc0cd330 con 0x7f5ee40052e0 2024-09-10T21:43:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.008+0000 7f5f04400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5efc0cd1b0 con 0x7f5efc0afb50 2024-09-10T21:43:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.008+0000 7f5f04400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f5efc160180 con 0x7f5efc0ad8f0 2024-09-10T21:43:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.008+0000 7f5f01a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee40052e0 0x7f5efc1f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:34.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.008+0000 7f5f01a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee40052e0 0x7f5efc1f4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43968/0 (socket says 172.21.15.7:43968) 2024-09-10T21:43:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.008+0000 7f5f01a00640 1 -- 172.21.15.7:0/2963537772 learned_addr learned my addr 172.21.15.7:0/2963537772 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.008+0000 7f5f01000640 1 --2- 172.21.15.7:0/2963537772 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5efc0ad8f0 0x7f5efc0c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.008+0000 7f5f02400640 1 --2- 172.21.15.7:0/2963537772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5efc0afb50 0x7f5efc0affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.009+0000 7f5f01a00640 1 -- 172.21.15.7:0/2963537772 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5efc0ad8f0 msgr2=0x7f5efc0c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.009+0000 7f5f01a00640 1 --2- 172.21.15.7:0/2963537772 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5efc0ad8f0 0x7f5efc0c6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.009+0000 7f5f01a00640 1 -- 172.21.15.7:0/2963537772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5efc0afb50 msgr2=0x7f5efc0affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.009+0000 7f5f01a00640 1 --2- 172.21.15.7:0/2963537772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5efc0afb50 0x7f5efc0affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.009+0000 7f5f01a00640 1 -- 172.21.15.7:0/2963537772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5efc15f400 con 0x7f5ee40052e0 2024-09-10T21:43:34.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.009+0000 7f5f01a00640 1 --2- 172.21.15.7:0/2963537772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee40052e0 0x7f5efc1f4f80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00117d0 tx=0x7f5ef0011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:34.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.009+0000 7f5ef9600640 1 -- 172.21.15.7:0/2963537772 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0004460 con 0x7f5ee40052e0 2024-09-10T21:43:34.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.009+0000 7f5ef9600640 1 -- 172.21.15.7:0/2963537772 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ef0016070 con 0x7f5ee40052e0 2024-09-10T21:43:34.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.009+0000 7f5ef9600640 1 -- 172.21.15.7:0/2963537772 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ef000fb60 con 0x7f5ee40052e0 2024-09-10T21:43:34.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.009+0000 7f5f04400640 1 -- 172.21.15.7:0/2963537772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5efc0addd0 con 0x7f5ee40052e0 2024-09-10T21:43:34.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.010+0000 7f5f04400640 1 -- 172.21.15.7:0/2963537772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5efc0ae390 con 0x7f5ee40052e0 2024-09-10T21:43:34.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.011+0000 7f5f04400640 1 -- 172.21.15.7:0/2963537772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5efc073730 con 0x7f5ee40052e0 2024-09-10T21:43:34.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.012+0000 7f5ef9600640 1 -- 172.21.15.7:0/2963537772 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5ef000fd00 con 0x7f5ee40052e0 2024-09-10T21:43:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.013+0000 7f5ef9600640 1 --2- 172.21.15.7:0/2963537772 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5edc077f70 0x7f5edc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.013+0000 7f5ef9600640 1 -- 172.21.15.7:0/2963537772 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f5ef009a800 con 0x7f5ee40052e0 2024-09-10T21:43:34.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.013+0000 7f5f01000640 1 --2- 172.21.15.7:0/2963537772 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5edc077f70 0x7f5edc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:34.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.014+0000 7f5f01000640 1 --2- 172.21.15.7:0/2963537772 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5edc077f70 0x7f5edc07a430 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8006bb0 tx=0x7f5ee8006720 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:34.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.017+0000 7f5ef9600640 1 -- 172.21.15.7:0/2963537772 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5ef0062c10 con 0x7f5ee40052e0 2024-09-10T21:43:34.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.211+0000 7f5f04400640 1 -- 172.21.15.7:0/2963537772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5efc0cc1e0 con 0x7f5ee40052e0 2024-09-10T21:43:34.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.212+0000 7f5ef9600640 1 -- 172.21.15.7:0/2963537772 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16650 (secure 0 0 0) 0x7f5ef0067640 con 0x7f5ee40052e0 2024-09-10T21:43:34.214 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:34.214 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":43,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:43:33.245617+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:43:31.081211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T21:43:32.202448+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:43:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.215+0000 7f5f04400640 1 -- 172.21.15.7:0/2963537772 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5edc077f70 msgr2=0x7f5edc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:34.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.215+0000 7f5f04400640 1 --2- 172.21.15.7:0/2963537772 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5edc077f70 0x7f5edc07a430 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8006bb0 tx=0x7f5ee8006720 comp rx=0 tx=0).stop 2024-09-10T21:43:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.215+0000 7f5f04400640 1 -- 172.21.15.7:0/2963537772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee40052e0 msgr2=0x7f5efc1f4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.215+0000 7f5f04400640 1 --2- 172.21.15.7:0/2963537772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee40052e0 0x7f5efc1f4f80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00117d0 tx=0x7f5ef0011c90 comp rx=0 tx=0).stop 2024-09-10T21:43:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.215+0000 7f5f04400640 1 -- 172.21.15.7:0/2963537772 shutdown_connections 2024-09-10T21:43:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.215+0000 7f5f04400640 1 --2- 172.21.15.7:0/2963537772 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5edc077f70 0x7f5edc07a430 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.215+0000 7f5f04400640 1 --2- 172.21.15.7:0/2963537772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5efc0afb50 0x7f5efc0affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.215+0000 7f5f04400640 1 --2- 172.21.15.7:0/2963537772 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5efc0ad8f0 0x7f5efc0c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.215+0000 7f5f04400640 1 --2- 172.21.15.7:0/2963537772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5ee40052e0 0x7f5efc1f4f80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.215+0000 7f5f04400640 1 -- 172.21.15.7:0/2963537772 >> 172.21.15.7:0/2963537772 conn(0x7f5efc0b16d0 msgr2=0x7f5efc0b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.215+0000 7f5f04400640 1 -- 172.21.15.7:0/2963537772 shutdown_connections 2024-09-10T21:43:34.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.215+0000 7f5f04400640 1 -- 172.21.15.7:0/2963537772 wait complete. 2024-09-10T21:43:34.232 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:43:34.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.392+0000 7ff9fdc00640 1 -- 172.21.15.7:0/3757937676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f81604b0 msgr2=0x7ff9f80ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:34.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.392+0000 7ff9fdc00640 1 --2- 172.21.15.7:0/3757937676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f81604b0 0x7ff9f80ccef0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0005fd0 tx=0x7ff9f002a8b0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.392+0000 7ff9fdc00640 1 -- 172.21.15.7:0/3757937676 shutdown_connections 2024-09-10T21:43:34.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.392+0000 7ff9fdc00640 1 --2- 172.21.15.7:0/3757937676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f81604b0 0x7ff9f80ccef0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.392+0000 7ff9fdc00640 1 -- 172.21.15.7:0/3757937676 >> 172.21.15.7:0/3757937676 conn(0x7ff9f80b16d0 msgr2=0x7ff9f80b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:34.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.392+0000 7ff9fdc00640 1 -- 172.21.15.7:0/3757937676 shutdown_connections 2024-09-10T21:43:34.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.392+0000 7ff9fdc00640 1 -- 172.21.15.7:0/3757937676 wait complete. 2024-09-10T21:43:34.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.394+0000 7ff9fdc00640 1 Processor -- start 2024-09-10T21:43:34.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.394+0000 7ff9fdc00640 1 -- start start 2024-09-10T21:43:34.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.395+0000 7ff9fdc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9f80cd750 0x7ff9f80b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:34.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.395+0000 7ff9fdc00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9f815bc90 0x7ff9f80ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:34.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.395+0000 7ff9fdc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f81604b0 0x7ff9f80b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:34.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.395+0000 7ff9fdc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f8070080 con 0x7ff9f80cd750 2024-09-10T21:43:34.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.395+0000 7ff9fdc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f806ff00 con 0x7ff9f81604b0 2024-09-10T21:43:34.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.395+0000 7ff9fdc00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f8070200 con 0x7ff9f815bc90 2024-09-10T21:43:34.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.395+0000 7ff9f6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9f80cd750 0x7ff9f80b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:34.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.395+0000 7ff9f6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9f80cd750 0x7ff9f80b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43974/0 (socket says 172.21.15.7:43974) 2024-09-10T21:43:34.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.395+0000 7ff9f6a00640 1 -- 172.21.15.7:0/38910038 learned_addr learned my addr 172.21.15.7:0/38910038 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:34.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.395+0000 7ff9f7400640 1 --2- 172.21.15.7:0/38910038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f81604b0 0x7ff9f80b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:34.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.395+0000 7ff9f6000640 1 --2- 172.21.15.7:0/38910038 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9f815bc90 0x7ff9f80ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:34.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.395+0000 7ff9f6a00640 1 -- 172.21.15.7:0/38910038 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9f815bc90 msgr2=0x7ff9f80ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:34.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.395+0000 7ff9f6a00640 1 --2- 172.21.15.7:0/38910038 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9f815bc90 0x7ff9f80ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.395+0000 7ff9f6a00640 1 -- 172.21.15.7:0/38910038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f81604b0 msgr2=0x7ff9f80b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:34.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.395+0000 7ff9f6a00640 1 --2- 172.21.15.7:0/38910038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f81604b0 0x7ff9f80b0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.395+0000 7ff9f6a00640 1 -- 172.21.15.7:0/38910038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f81f7fd0 con 0x7ff9f80cd750 2024-09-10T21:43:34.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.395+0000 7ff9f7400640 1 --2- 172.21.15.7:0/38910038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f81604b0 0x7ff9f80b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:43:34.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.396+0000 7ff9f6a00640 1 --2- 172.21.15.7:0/38910038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9f80cd750 0x7ff9f80b4be0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff9e800c4d0 tx=0x7ff9e800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:34.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.396+0000 7ff9eea00640 1 -- 172.21.15.7:0/38910038 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80052d0 con 0x7ff9f80cd750 2024-09-10T21:43:34.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.396+0000 7ff9eea00640 1 -- 172.21.15.7:0/38910038 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9e8005470 con 0x7ff9f80cd750 2024-09-10T21:43:34.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.396+0000 7ff9eea00640 1 -- 172.21.15.7:0/38910038 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8012890 con 0x7ff9f80cd750 2024-09-10T21:43:34.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.396+0000 7ff9fdc00640 1 -- 172.21.15.7:0/38910038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f81f8270 con 0x7ff9f80cd750 2024-09-10T21:43:34.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.396+0000 7ff9fdc00640 1 -- 172.21.15.7:0/38910038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f81f8b10 con 0x7ff9f80cd750 2024-09-10T21:43:34.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.397+0000 7ff9fdc00640 1 -- 172.21.15.7:0/38910038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9f80706b0 con 0x7ff9f80cd750 2024-09-10T21:43:34.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.398+0000 7ff9eea00640 1 -- 172.21.15.7:0/38910038 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff9e80035f0 con 0x7ff9f80cd750 2024-09-10T21:43:34.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.399+0000 7ff9eea00640 1 --2- 172.21.15.7:0/38910038 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff9d8077f70 0x7ff9d807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:34.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.399+0000 7ff9eea00640 1 -- 172.21.15.7:0/38910038 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7ff9e80998a0 con 0x7ff9f80cd750 2024-09-10T21:43:34.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.400+0000 7ff9f6000640 1 --2- 172.21.15.7:0/38910038 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff9d8077f70 0x7ff9d807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:34.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.401+0000 7ff9f6000640 1 --2- 172.21.15.7:0/38910038 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff9d8077f70 0x7ff9d807a430 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4007fd0 tx=0x7ff9e4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:34.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.403+0000 7ff9eea00640 1 -- 172.21.15.7:0/38910038 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff9e8062430 con 0x7ff9f80cd750 2024-09-10T21:43:34.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.600+0000 7ff9fdc00640 1 -- 172.21.15.7:0/38910038 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff9f81f8440 con 0x7ff9f80cd750 2024-09-10T21:43:34.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.601+0000 7ff9eea00640 1 -- 172.21.15.7:0/38910038 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) ==== 74+0+16650 (secure 0 0 0) 0x7ff9e8061d30 con 0x7ff9f80cd750 2024-09-10T21:43:34.602 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:34.602 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":43,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:43:33.245617+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:43:31.081211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T21:43:32.202448+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:43:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.603+0000 7ff9fdc00640 1 -- 172.21.15.7:0/38910038 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff9d8077f70 msgr2=0x7ff9d807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.603+0000 7ff9fdc00640 1 --2- 172.21.15.7:0/38910038 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff9d8077f70 0x7ff9d807a430 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4007fd0 tx=0x7ff9e4009040 comp rx=0 tx=0).stop 2024-09-10T21:43:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.603+0000 7ff9fdc00640 1 -- 172.21.15.7:0/38910038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9f80cd750 msgr2=0x7ff9f80b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.603+0000 7ff9fdc00640 1 --2- 172.21.15.7:0/38910038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9f80cd750 0x7ff9f80b4be0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff9e800c4d0 tx=0x7ff9e800c990 comp rx=0 tx=0).stop 2024-09-10T21:43:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.604+0000 7ff9fdc00640 1 -- 172.21.15.7:0/38910038 shutdown_connections 2024-09-10T21:43:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.604+0000 7ff9fdc00640 1 --2- 172.21.15.7:0/38910038 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff9d8077f70 0x7ff9d807a430 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.604+0000 7ff9fdc00640 1 --2- 172.21.15.7:0/38910038 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff9f81604b0 0x7ff9f80b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.604+0000 7ff9fdc00640 1 --2- 172.21.15.7:0/38910038 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff9f815bc90 0x7ff9f80ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.604+0000 7ff9fdc00640 1 --2- 172.21.15.7:0/38910038 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff9f80cd750 0x7ff9f80b4be0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.604+0000 7ff9fdc00640 1 -- 172.21.15.7:0/38910038 >> 172.21.15.7:0/38910038 conn(0x7ff9f80b16d0 msgr2=0x7ff9f80c5d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.604+0000 7ff9fdc00640 1 -- 172.21.15.7:0/38910038 shutdown_connections 2024-09-10T21:43:34.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.604+0000 7ff9fdc00640 1 -- 172.21.15.7:0/38910038 wait complete. 2024-09-10T21:43:34.621 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:43:34.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.783+0000 7f8931800640 1 -- 172.21.15.7:0/350982813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f892c161b10 msgr2=0x7f892c163f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:34.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.783+0000 7f8931800640 1 --2- 172.21.15.7:0/350982813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f892c161b10 0x7f892c163f00 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f892c0cb130 tx=0x7f892402a8b0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.783+0000 7f8931800640 1 -- 172.21.15.7:0/350982813 shutdown_connections 2024-09-10T21:43:34.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.783+0000 7f8931800640 1 --2- 172.21.15.7:0/350982813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f892c161b10 0x7f892c163f00 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.783+0000 7f8931800640 1 --2- 172.21.15.7:0/350982813 >> v2:172.21.15.133:3300/0 conn(0x7f892c15acf0 0x7f892c154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.783+0000 7f8931800640 1 -- 172.21.15.7:0/350982813 >> 172.21.15.7:0/350982813 conn(0x7f892c0b16d0 msgr2=0x7f892c0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.783+0000 7f8931800640 1 -- 172.21.15.7:0/350982813 shutdown_connections 2024-09-10T21:43:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.783+0000 7f8931800640 1 -- 172.21.15.7:0/350982813 wait complete. 2024-09-10T21:43:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.784+0000 7f8931800640 1 Processor -- start 2024-09-10T21:43:34.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.784+0000 7f8931800640 1 -- start start 2024-09-10T21:43:34.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.785+0000 7f8931800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f892c070670 0x7f892c0ceb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:34.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.785+0000 7f8931800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f892c15acf0 0x7f892c1f7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:34.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.785+0000 7f8931800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f892c161b10 0x7f892c0b3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:34.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.785+0000 7f8931800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f892c0cc840 con 0x7f892c15acf0 2024-09-10T21:43:34.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.785+0000 7f8931800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f892c0cc6c0 con 0x7f892c070670 2024-09-10T21:43:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.785+0000 7f8931800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f892c0cc9c0 con 0x7f892c161b10 2024-09-10T21:43:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.786+0000 7f8923e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f892c15acf0 0x7f892c1f7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.786+0000 7f8923e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f892c15acf0 0x7f892c1f7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44000/0 (socket says 172.21.15.7:44000) 2024-09-10T21:43:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.786+0000 7f8923e00640 1 -- 172.21.15.7:0/1982792239 learned_addr learned my addr 172.21.15.7:0/1982792239 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.786+0000 7f892b400640 1 --2- 172.21.15.7:0/1982792239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f892c161b10 0x7f892c0b3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.786+0000 7f892aa00640 1 --2- 172.21.15.7:0/1982792239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f892c070670 0x7f892c0ceb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.786+0000 7f8923e00640 1 -- 172.21.15.7:0/1982792239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f892c161b10 msgr2=0x7f892c0b3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.786+0000 7f8923e00640 1 --2- 172.21.15.7:0/1982792239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f892c161b10 0x7f892c0b3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.786+0000 7f8923e00640 1 -- 172.21.15.7:0/1982792239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f892c070670 msgr2=0x7f892c0ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.786+0000 7f8923e00640 1 --2- 172.21.15.7:0/1982792239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f892c070670 0x7f892c0ceb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:34.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.786+0000 7f8923e00640 1 -- 172.21.15.7:0/1982792239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f892c0ca340 con 0x7f892c15acf0 2024-09-10T21:43:34.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.786+0000 7f8923e00640 1 --2- 172.21.15.7:0/1982792239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f892c15acf0 0x7f892c1f7680 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f891400ea20 tx=0x7f891400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:34.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.787+0000 7f8922a00640 1 -- 172.21.15.7:0/1982792239 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891400cd40 con 0x7f892c15acf0 2024-09-10T21:43:34.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.787+0000 7f8922a00640 1 -- 172.21.15.7:0/1982792239 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8914016410 con 0x7f892c15acf0 2024-09-10T21:43:34.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.787+0000 7f8922a00640 1 -- 172.21.15.7:0/1982792239 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914016860 con 0x7f892c15acf0 2024-09-10T21:43:34.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.787+0000 7f8931800640 1 -- 172.21.15.7:0/1982792239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f892c0b5570 con 0x7f892c15acf0 2024-09-10T21:43:34.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.787+0000 7f8931800640 1 -- 172.21.15.7:0/1982792239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f892c0ca060 con 0x7f892c15acf0 2024-09-10T21:43:34.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.788+0000 7f8931800640 1 -- 172.21.15.7:0/1982792239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f892c154370 con 0x7f892c15acf0 2024-09-10T21:43:34.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.789+0000 7f8922a00640 1 -- 172.21.15.7:0/1982792239 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f891400b040 con 0x7f892c15acf0 2024-09-10T21:43:34.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.790+0000 7f8922a00640 1 --2- 172.21.15.7:0/1982792239 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8908077fa0 0x7f890807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:34.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.791+0000 7f8922a00640 1 -- 172.21.15.7:0/1982792239 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 5709+0+0 (secure 0 0 0) 0x7f8914099f70 con 0x7f892c15acf0 2024-09-10T21:43:34.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.791+0000 7f892aa00640 1 --2- 172.21.15.7:0/1982792239 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8908077fa0 0x7f890807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:34.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.792+0000 7f892aa00640 1 --2- 172.21.15.7:0/1982792239 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8908077fa0 0x7f890807a460 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f891c007fd0 tx=0x7f891c001410 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:34.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:34.795+0000 7f8922a00640 1 -- 172.21.15.7:0/1982792239 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8914062420 con 0x7f892c15acf0 2024-09-10T21:43:35.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:35.033+0000 7f8931800640 1 -- 172.21.15.7:0/1982792239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f892c152c90 con 0x7f892c15acf0 2024-09-10T21:43:35.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:35.830+0000 7f8922a00640 1 -- 172.21.15.7:0/1982792239 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f8914066e50 con 0x7f892c15acf0 2024-09-10T21:43:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:35.833+0000 7f8931800640 1 -- 172.21.15.7:0/1982792239 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8908077fa0 msgr2=0x7f890807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:35.833+0000 7f8931800640 1 --2- 172.21.15.7:0/1982792239 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8908077fa0 0x7f890807a460 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f891c007fd0 tx=0x7f891c001410 comp rx=0 tx=0).stop 2024-09-10T21:43:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:35.833+0000 7f8931800640 1 -- 172.21.15.7:0/1982792239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f892c15acf0 msgr2=0x7f892c1f7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:35.833+0000 7f8931800640 1 --2- 172.21.15.7:0/1982792239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f892c15acf0 0x7f892c1f7680 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f891400ea20 tx=0x7f891400eee0 comp rx=0 tx=0).stop 2024-09-10T21:43:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:35.834+0000 7f8931800640 1 -- 172.21.15.7:0/1982792239 shutdown_connections 2024-09-10T21:43:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:35.834+0000 7f8931800640 1 --2- 172.21.15.7:0/1982792239 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8908077fa0 0x7f890807a460 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:35.834+0000 7f8931800640 1 --2- 172.21.15.7:0/1982792239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f892c161b10 0x7f892c0b3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:35.834+0000 7f8931800640 1 --2- 172.21.15.7:0/1982792239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f892c15acf0 0x7f892c1f7680 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:35.834+0000 7f8931800640 1 --2- 172.21.15.7:0/1982792239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f892c070670 0x7f892c0ceb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:35.834+0000 7f8931800640 1 -- 172.21.15.7:0/1982792239 >> 172.21.15.7:0/1982792239 conn(0x7f892c0b16d0 msgr2=0x7f892c0cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:35.834+0000 7f8931800640 1 -- 172.21.15.7:0/1982792239 shutdown_connections 2024-09-10T21:43:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:35.834+0000 7f8931800640 1 -- 172.21.15.7:0/1982792239 wait complete. 2024-09-10T21:43:35.854 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:43:35.854 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:43:36.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.022+0000 7f548d600640 1 -- 172.21.15.7:0/4048801081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54880d03d0 msgr2=0x7f548815caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:36.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.022+0000 7f548d600640 1 --2- 172.21.15.7:0/4048801081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54880d03d0 0x7f548815caa0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f5480005fd0 tx=0x7f548002a940 comp rx=0 tx=0).stop 2024-09-10T21:43:36.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.022+0000 7f548d600640 1 -- 172.21.15.7:0/4048801081 shutdown_connections 2024-09-10T21:43:36.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.022+0000 7f548d600640 1 --2- 172.21.15.7:0/4048801081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54880d03d0 0x7f548815caa0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:36.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.022+0000 7f548d600640 1 -- 172.21.15.7:0/4048801081 >> 172.21.15.7:0/4048801081 conn(0x7f548806cd00 msgr2=0x7f548806d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:36.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.022+0000 7f548d600640 1 -- 172.21.15.7:0/4048801081 shutdown_connections 2024-09-10T21:43:36.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.022+0000 7f548d600640 1 -- 172.21.15.7:0/4048801081 wait complete. 2024-09-10T21:43:36.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.024+0000 7f548d600640 1 Processor -- start 2024-09-10T21:43:36.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.024+0000 7f548d600640 1 -- start start 2024-09-10T21:43:36.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.025+0000 7f548d600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54880d03d0 0x7f548814bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:36.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.025+0000 7f548d600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54881529f0 0x7f54881f82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:36.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.025+0000 7f548d600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5488074110 0x7f54881587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:36.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.025+0000 7f548d600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f548806f1a0 con 0x7f54880d03d0 2024-09-10T21:43:36.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.025+0000 7f548d600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f548806f020 con 0x7f54881529f0 2024-09-10T21:43:36.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.025+0000 7f548d600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f548806f320 con 0x7f5488074110 2024-09-10T21:43:36.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.025+0000 7f5487e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54880d03d0 0x7f548814bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:36.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.025+0000 7f5487e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54880d03d0 0x7f548814bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44012/0 (socket says 172.21.15.7:44012) 2024-09-10T21:43:36.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.025+0000 7f5487e00640 1 -- 172.21.15.7:0/2245812331 learned_addr learned my addr 172.21.15.7:0/2245812331 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:36.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.025+0000 7f548cc00640 1 --2- 172.21.15.7:0/2245812331 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5488074110 0x7f54881587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:36.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.025+0000 7f547fe00640 1 --2- 172.21.15.7:0/2245812331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54881529f0 0x7f54881f82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:36.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.026+0000 7f5487e00640 1 -- 172.21.15.7:0/2245812331 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5488074110 msgr2=0x7f54881587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:36.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.026+0000 7f5487e00640 1 --2- 172.21.15.7:0/2245812331 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5488074110 0x7f54881587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:36.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.026+0000 7f5487e00640 1 -- 172.21.15.7:0/2245812331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54881529f0 msgr2=0x7f54881f82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:36.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.026+0000 7f5487e00640 1 --2- 172.21.15.7:0/2245812331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54881529f0 0x7f54881f82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:36.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.026+0000 7f5487e00640 1 -- 172.21.15.7:0/2245812331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5488159020 con 0x7f54880d03d0 2024-09-10T21:43:36.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.026+0000 7f547fe00640 1 --2- 172.21.15.7:0/2245812331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54881529f0 0x7f54881f82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:36.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.026+0000 7f548cc00640 1 --2- 172.21.15.7:0/2245812331 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5488074110 0x7f54881587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:36.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.026+0000 7f5487e00640 1 --2- 172.21.15.7:0/2245812331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54880d03d0 0x7f548814bef0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f547800c430 tx=0x7f547800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:36.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.026+0000 7f5484c00640 1 -- 172.21.15.7:0/2245812331 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478011790 con 0x7f54880d03d0 2024-09-10T21:43:36.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.026+0000 7f5484c00640 1 -- 172.21.15.7:0/2245812331 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5478017070 con 0x7f54880d03d0 2024-09-10T21:43:36.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.026+0000 7f5484c00640 1 -- 172.21.15.7:0/2245812331 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5478011ce0 con 0x7f54880d03d0 2024-09-10T21:43:36.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.026+0000 7f548d600640 1 -- 172.21.15.7:0/2245812331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5488159300 con 0x7f54880d03d0 2024-09-10T21:43:36.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.026+0000 7f548d600640 1 -- 172.21.15.7:0/2245812331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54881f7d90 con 0x7f54880d03d0 2024-09-10T21:43:36.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.028+0000 7f548d600640 1 -- 172.21.15.7:0/2245812331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5488159540 con 0x7f54880d03d0 2024-09-10T21:43:36.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.029+0000 7f5484c00640 1 -- 172.21.15.7:0/2245812331 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f547801c460 con 0x7f54880d03d0 2024-09-10T21:43:36.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.030+0000 7f5484c00640 1 --2- 172.21.15.7:0/2245812331 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5464077f70 0x7f546407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:36.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.030+0000 7f5484c00640 1 -- 172.21.15.7:0/2245812331 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f5478099ac0 con 0x7f54880d03d0 2024-09-10T21:43:36.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.030+0000 7f547fe00640 1 --2- 172.21.15.7:0/2245812331 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5464077f70 0x7f546407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:36.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.031+0000 7f547fe00640 1 --2- 172.21.15.7:0/2245812331 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5464077f70 0x7f546407a430 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5470007fd0 tx=0x7f5470009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:36.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.034+0000 7f5484c00640 1 -- 172.21.15.7:0/2245812331 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5478062650 con 0x7f54880d03d0 2024-09-10T21:43:36.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.264+0000 7f548d600640 1 -- 172.21.15.7:0/2245812331 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f548815caa0 con 0x7f54880d03d0 2024-09-10T21:43:36.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.838+0000 7f5484c00640 1 -- 172.21.15.7:0/2245812331 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f5478061f50 con 0x7f54880d03d0 2024-09-10T21:43:36.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.841+0000 7f548d600640 1 -- 172.21.15.7:0/2245812331 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5464077f70 msgr2=0x7f546407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:36.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.841+0000 7f548d600640 1 --2- 172.21.15.7:0/2245812331 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5464077f70 0x7f546407a430 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f5470007fd0 tx=0x7f5470009040 comp rx=0 tx=0).stop 2024-09-10T21:43:36.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.841+0000 7f548d600640 1 -- 172.21.15.7:0/2245812331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54880d03d0 msgr2=0x7f548814bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:36.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.841+0000 7f548d600640 1 --2- 172.21.15.7:0/2245812331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54880d03d0 0x7f548814bef0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f547800c430 tx=0x7f547800c8f0 comp rx=0 tx=0).stop 2024-09-10T21:43:36.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.841+0000 7f548d600640 1 -- 172.21.15.7:0/2245812331 shutdown_connections 2024-09-10T21:43:36.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.841+0000 7f548d600640 1 --2- 172.21.15.7:0/2245812331 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5464077f70 0x7f546407a430 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:36.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.841+0000 7f548d600640 1 --2- 172.21.15.7:0/2245812331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54881529f0 0x7f54881f82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:36.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.841+0000 7f548d600640 1 --2- 172.21.15.7:0/2245812331 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5488074110 0x7f54881587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:36.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.841+0000 7f548d600640 1 --2- 172.21.15.7:0/2245812331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54880d03d0 0x7f548814bef0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:36.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.841+0000 7f548d600640 1 -- 172.21.15.7:0/2245812331 >> 172.21.15.7:0/2245812331 conn(0x7f548806cd00 msgr2=0x7f54880cf0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:36.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.841+0000 7f548d600640 1 -- 172.21.15.7:0/2245812331 shutdown_connections 2024-09-10T21:43:36.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:36.841+0000 7f548d600640 1 -- 172.21.15.7:0/2245812331 wait complete. 2024-09-10T21:43:36.860 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:43:36.861 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:43:37.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.023+0000 7f46b4400640 1 -- 172.21.15.7:0/2958455849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46ac0cbca0 msgr2=0x7f46ac168d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.023+0000 7f46b4400640 1 --2- 172.21.15.7:0/2958455849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46ac0cbca0 0x7f46ac168d60 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f46a4005fd0 tx=0x7f46a402a8a0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.023+0000 7f46b4400640 1 -- 172.21.15.7:0/2958455849 shutdown_connections 2024-09-10T21:43:37.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.023+0000 7f46b4400640 1 --2- 172.21.15.7:0/2958455849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46ac0cbca0 0x7f46ac168d60 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.023+0000 7f46b4400640 1 -- 172.21.15.7:0/2958455849 >> 172.21.15.7:0/2958455849 conn(0x7f46ac0d0c90 msgr2=0x7f46ac0d1070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:37.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.023+0000 7f46b4400640 1 -- 172.21.15.7:0/2958455849 shutdown_connections 2024-09-10T21:43:37.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.023+0000 7f46b4400640 1 -- 172.21.15.7:0/2958455849 wait complete. 2024-09-10T21:43:37.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.024+0000 7f46b4400640 1 Processor -- start 2024-09-10T21:43:37.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.025+0000 7f46b4400640 1 -- start start 2024-09-10T21:43:37.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.025+0000 7f46b4400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46ac06d230 0x7f46ac153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.025+0000 7f46b4400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46ac0cbca0 0x7f46ac0c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.025+0000 7f46b4400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46ac1613c0 0x7f46ac0c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.025+0000 7f46b4400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac169420 con 0x7f46ac06d230 2024-09-10T21:43:37.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.025+0000 7f46b4400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f46ac1692a0 con 0x7f46ac0cbca0 2024-09-10T21:43:37.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.025+0000 7f46b4400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f46ac1695a0 con 0x7f46ac1613c0 2024-09-10T21:43:37.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.026+0000 7f46b1a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46ac06d230 0x7f46ac153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.026+0000 7f46b1a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46ac06d230 0x7f46ac153400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44044/0 (socket says 172.21.15.7:44044) 2024-09-10T21:43:37.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.026+0000 7f46b1a00640 1 -- 172.21.15.7:0/3638241712 learned_addr learned my addr 172.21.15.7:0/3638241712 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:37.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.026+0000 7f46b2400640 1 --2- 172.21.15.7:0/3638241712 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46ac1613c0 0x7f46ac0c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.026+0000 7f46b1000640 1 --2- 172.21.15.7:0/3638241712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46ac0cbca0 0x7f46ac0c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.026+0000 7f46b1a00640 1 -- 172.21.15.7:0/3638241712 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46ac1613c0 msgr2=0x7f46ac0c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.026+0000 7f46b1a00640 1 --2- 172.21.15.7:0/3638241712 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46ac1613c0 0x7f46ac0c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.026+0000 7f46b1a00640 1 -- 172.21.15.7:0/3638241712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46ac0cbca0 msgr2=0x7f46ac0c81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.026+0000 7f46b1a00640 1 --2- 172.21.15.7:0/3638241712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46ac0cbca0 0x7f46ac0c81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.026+0000 7f46b1a00640 1 -- 172.21.15.7:0/3638241712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ac153940 con 0x7f46ac06d230 2024-09-10T21:43:37.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.026+0000 7f46b1a00640 1 --2- 172.21.15.7:0/3638241712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46ac06d230 0x7f46ac153400 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f46a000c3a0 tx=0x7f46a000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:37.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.026+0000 7f46a9600640 1 -- 172.21.15.7:0/3638241712 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0017070 con 0x7f46ac06d230 2024-09-10T21:43:37.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.027+0000 7f46a9600640 1 -- 172.21.15.7:0/3638241712 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46a00053c0 con 0x7f46ac06d230 2024-09-10T21:43:37.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.027+0000 7f46b4400640 1 -- 172.21.15.7:0/3638241712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ac158600 con 0x7f46ac06d230 2024-09-10T21:43:37.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.027+0000 7f46a9600640 1 -- 172.21.15.7:0/3638241712 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a0003850 con 0x7f46ac06d230 2024-09-10T21:43:37.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.027+0000 7f46b4400640 1 -- 172.21.15.7:0/3638241712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ac0ca680 con 0x7f46ac06d230 2024-09-10T21:43:37.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.028+0000 7f46b4400640 1 -- 172.21.15.7:0/3638241712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ac0cb590 con 0x7f46ac06d230 2024-09-10T21:43:37.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.029+0000 7f46a9600640 1 -- 172.21.15.7:0/3638241712 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f46a00039f0 con 0x7f46ac06d230 2024-09-10T21:43:37.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.030+0000 7f46a9600640 1 --2- 172.21.15.7:0/3638241712 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f468c077fa0 0x7f468c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.030+0000 7f46a9600640 1 -- 172.21.15.7:0/3638241712 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f46a0099440 con 0x7f46ac06d230 2024-09-10T21:43:37.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.030+0000 7f46b1000640 1 --2- 172.21.15.7:0/3638241712 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f468c077fa0 0x7f468c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.031+0000 7f46b1000640 1 --2- 172.21.15.7:0/3638241712 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f468c077fa0 0x7f468c07a460 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f46980046b0 tx=0x7f4698009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:37.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.034+0000 7f46a9600640 1 -- 172.21.15.7:0/3638241712 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f46a0061ea0 con 0x7f46ac06d230 2024-09-10T21:43:37.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.268+0000 7f46b4400640 1 -- 172.21.15.7:0/3638241712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46ac0b6c10 con 0x7f46ac06d230 2024-09-10T21:43:37.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.269+0000 7f46a9600640 1 -- 172.21.15.7:0/3638241712 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5233 (secure 0 0 0) 0x7f46a00617a0 con 0x7f46ac06d230 2024-09-10T21:43:37.270 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:37.271 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"btime":"2024-09-10T21:43:36:831759+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5149,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3851521050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3851521050},{"type":"v1","addr":"172.21.15.133:6836","nonce":3851521050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5289,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2709023497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2709023497},{"type":"v1","addr":"172.21.15.7:6834","nonce":2709023497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/4095876032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":4095876032},{"type":"v1","addr":"172.21.15.7:6835","nonce":4095876032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:43:33.245584+0000","modified":"2024-09-10T21:43:36.266208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234767},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234767":{"gid":234767,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3385628197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3385628197},{"type":"v1","addr":"172.21.15.133:6837","nonce":3385628197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234767,"qdb_cluster":[234767]},"id":3}]} 2024-09-10T21:43:37.271 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-09-10T21:43:37.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.271+0000 7f46b4400640 1 -- 172.21.15.7:0/3638241712 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f468c077fa0 msgr2=0x7f468c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.271+0000 7f46b4400640 1 --2- 172.21.15.7:0/3638241712 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f468c077fa0 0x7f468c07a460 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f46980046b0 tx=0x7f4698009040 comp rx=0 tx=0).stop 2024-09-10T21:43:37.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.271+0000 7f46b4400640 1 -- 172.21.15.7:0/3638241712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46ac06d230 msgr2=0x7f46ac153400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.271+0000 7f46b4400640 1 --2- 172.21.15.7:0/3638241712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46ac06d230 0x7f46ac153400 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f46a000c3a0 tx=0x7f46a000c860 comp rx=0 tx=0).stop 2024-09-10T21:43:37.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.271+0000 7f46b4400640 1 -- 172.21.15.7:0/3638241712 shutdown_connections 2024-09-10T21:43:37.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.271+0000 7f46b4400640 1 --2- 172.21.15.7:0/3638241712 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f468c077fa0 0x7f468c07a460 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.271+0000 7f46b4400640 1 --2- 172.21.15.7:0/3638241712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46ac0cbca0 0x7f46ac0c81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.271+0000 7f46b4400640 1 --2- 172.21.15.7:0/3638241712 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46ac1613c0 0x7f46ac0c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.271+0000 7f46b4400640 1 --2- 172.21.15.7:0/3638241712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46ac06d230 0x7f46ac153400 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.271+0000 7f46b4400640 1 -- 172.21.15.7:0/3638241712 >> 172.21.15.7:0/3638241712 conn(0x7f46ac0d0c90 msgr2=0x7f46ac159560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:37.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.272+0000 7f46b4400640 1 -- 172.21.15.7:0/3638241712 shutdown_connections 2024-09-10T21:43:37.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.272+0000 7f46b4400640 1 -- 172.21.15.7:0/3638241712 wait complete. 2024-09-10T21:43:37.289 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:43:37.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.450+0000 7f6e8ca00640 1 -- 172.21.15.7:0/711652261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e880cd750 msgr2=0x7f6e880ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.450+0000 7f6e8ca00640 1 --2- 172.21.15.7:0/711652261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e880cd750 0x7f6e880ccef0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6e70009860 tx=0x7f6e7002a480 comp rx=0 tx=0).stop 2024-09-10T21:43:37.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.450+0000 7f6e8ca00640 1 -- 172.21.15.7:0/711652261 shutdown_connections 2024-09-10T21:43:37.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.450+0000 7f6e8ca00640 1 --2- 172.21.15.7:0/711652261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e880cd750 0x7f6e880ccef0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.450+0000 7f6e8ca00640 1 -- 172.21.15.7:0/711652261 >> 172.21.15.7:0/711652261 conn(0x7f6e880b16d0 msgr2=0x7f6e880b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:37.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.450+0000 7f6e8ca00640 1 -- 172.21.15.7:0/711652261 shutdown_connections 2024-09-10T21:43:37.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.450+0000 7f6e8ca00640 1 -- 172.21.15.7:0/711652261 wait complete. 2024-09-10T21:43:37.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.451+0000 7f6e8ca00640 1 Processor -- start 2024-09-10T21:43:37.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.451+0000 7f6e8ca00640 1 -- start start 2024-09-10T21:43:37.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.452+0000 7f6e8ca00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e880cd750 0x7f6e881f4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.452+0000 7f6e8ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e8815bc90 0x7f6e8815e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.452+0000 7f6e8ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e8815fda0 0x7f6e880b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.452+0000 7f6e8ca00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6e88070610 con 0x7f6e8815bc90 2024-09-10T21:43:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.452+0000 7f6e8ca00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6e880cf570 con 0x7f6e8815fda0 2024-09-10T21:43:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.452+0000 7f6e8ca00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f6e880ca1d0 con 0x7f6e880cd750 2024-09-10T21:43:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.453+0000 7f6e7fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e8815bc90 0x7f6e8815e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.453+0000 7f6e85e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e880cd750 0x7f6e881f4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.453+0000 7f6e7fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e8815bc90 0x7f6e8815e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44050/0 (socket says 172.21.15.7:44050) 2024-09-10T21:43:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.453+0000 7f6e7fe00640 1 -- 172.21.15.7:0/875476813 learned_addr learned my addr 172.21.15.7:0/875476813 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.453+0000 7f6e86800640 1 --2- 172.21.15.7:0/875476813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e8815fda0 0x7f6e880b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.453+0000 7f6e85e00640 1 -- 172.21.15.7:0/875476813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e8815fda0 msgr2=0x7f6e880b0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.453+0000 7f6e85e00640 1 --2- 172.21.15.7:0/875476813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e8815fda0 0x7f6e880b0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.453+0000 7f6e85e00640 1 -- 172.21.15.7:0/875476813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e8815bc90 msgr2=0x7f6e8815e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.453+0000 7f6e85e00640 1 --2- 172.21.15.7:0/875476813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e8815bc90 0x7f6e8815e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.453+0000 7f6e85e00640 1 -- 172.21.15.7:0/875476813 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8815ed70 con 0x7f6e880cd750 2024-09-10T21:43:37.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.453+0000 7f6e7fe00640 1 --2- 172.21.15.7:0/875476813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e8815bc90 0x7f6e8815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:43:37.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.454+0000 7f6e85e00640 1 --2- 172.21.15.7:0/875476813 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e880cd750 0x7f6e881f4740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e7800c3c0 tx=0x7f6e7800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:37.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.454+0000 7f6e7e000640 1 -- 172.21.15.7:0/875476813 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e78011790 con 0x7f6e880cd750 2024-09-10T21:43:37.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.454+0000 7f6e7e000640 1 -- 172.21.15.7:0/875476813 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e78017070 con 0x7f6e880cd750 2024-09-10T21:43:37.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.454+0000 7f6e7e000640 1 -- 172.21.15.7:0/875476813 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e78011ce0 con 0x7f6e880cd750 2024-09-10T21:43:37.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.454+0000 7f6e8ca00640 1 -- 172.21.15.7:0/875476813 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e8815f050 con 0x7f6e880cd750 2024-09-10T21:43:37.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.454+0000 7f6e8ca00640 1 -- 172.21.15.7:0/875476813 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e880c75c0 con 0x7f6e880cd750 2024-09-10T21:43:37.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.455+0000 7f6e8ca00640 1 -- 172.21.15.7:0/875476813 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e880c77c0 con 0x7f6e880cd750 2024-09-10T21:43:37.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.456+0000 7f6e7e000640 1 -- 172.21.15.7:0/875476813 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6e78005020 con 0x7f6e880cd750 2024-09-10T21:43:37.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.457+0000 7f6e7e000640 1 --2- 172.21.15.7:0/875476813 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e64077f70 0x7f6e6407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.457+0000 7f6e7e000640 1 -- 172.21.15.7:0/875476813 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f6e78099ef0 con 0x7f6e880cd750 2024-09-10T21:43:37.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.458+0000 7f6e7fe00640 1 --2- 172.21.15.7:0/875476813 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e64077f70 0x7f6e6407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.458+0000 7f6e7fe00640 1 --2- 172.21.15.7:0/875476813 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e64077f70 0x7f6e6407a430 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6e700095f0 tx=0x7f6e70004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:37.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.461+0000 7f6e7e000640 1 -- 172.21.15.7:0/875476813 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6e880c77c0 con 0x7f6e880cd750 2024-09-10T21:43:37.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.657+0000 7f6e8ca00640 1 -- 172.21.15.7:0/875476813 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6e88160540 con 0x7f6e880cd750 2024-09-10T21:43:37.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.659+0000 7f6e7e000640 1 -- 172.21.15.7:0/875476813 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) ==== 74+0+16637 (secure 0 0 0) 0x7f6e78062410 con 0x7f6e880cd750 2024-09-10T21:43:37.660 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:37.660 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":44,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:43:34.813460+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:43:31.081211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T21:43:32.202448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:43:37.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.661+0000 7f6e8ca00640 1 -- 172.21.15.7:0/875476813 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e64077f70 msgr2=0x7f6e6407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.661+0000 7f6e8ca00640 1 --2- 172.21.15.7:0/875476813 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e64077f70 0x7f6e6407a430 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6e700095f0 tx=0x7f6e70004730 comp rx=0 tx=0).stop 2024-09-10T21:43:37.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.661+0000 7f6e8ca00640 1 -- 172.21.15.7:0/875476813 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e880cd750 msgr2=0x7f6e881f4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.661+0000 7f6e8ca00640 1 --2- 172.21.15.7:0/875476813 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e880cd750 0x7f6e881f4740 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6e7800c3c0 tx=0x7f6e7800c880 comp rx=0 tx=0).stop 2024-09-10T21:43:37.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.661+0000 7f6e8ca00640 1 -- 172.21.15.7:0/875476813 shutdown_connections 2024-09-10T21:43:37.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.661+0000 7f6e8ca00640 1 --2- 172.21.15.7:0/875476813 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e64077f70 0x7f6e6407a430 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.661+0000 7f6e8ca00640 1 --2- 172.21.15.7:0/875476813 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e8815fda0 0x7f6e880b0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.661+0000 7f6e8ca00640 1 --2- 172.21.15.7:0/875476813 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e8815bc90 0x7f6e8815e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.661+0000 7f6e8ca00640 1 --2- 172.21.15.7:0/875476813 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e880cd750 0x7f6e881f4740 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.661+0000 7f6e8ca00640 1 -- 172.21.15.7:0/875476813 >> 172.21.15.7:0/875476813 conn(0x7f6e880b16d0 msgr2=0x7f6e880c5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:37.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.661+0000 7f6e8ca00640 1 -- 172.21.15.7:0/875476813 shutdown_connections 2024-09-10T21:43:37.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.661+0000 7f6e8ca00640 1 -- 172.21.15.7:0/875476813 wait complete. 2024-09-10T21:43:37.679 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:43:37.679 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:43:37.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.838+0000 7f4fd6000640 1 -- 172.21.15.7:0/3929641115 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fd00b2f80 msgr2=0x7f4fd00ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.838+0000 7f4fd6000640 1 --2- 172.21.15.7:0/3929641115 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fd00b2f80 0x7f4fd00ce6f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8005fd0 tx=0x7f4fc802aaa0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.838+0000 7f4fd6000640 1 -- 172.21.15.7:0/3929641115 shutdown_connections 2024-09-10T21:43:37.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.838+0000 7f4fd6000640 1 --2- 172.21.15.7:0/3929641115 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fd00b2f80 0x7f4fd00ce6f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.838+0000 7f4fd6000640 1 --2- 172.21.15.7:0/3929641115 >> v2:172.21.15.133:3300/0 conn(0x7f4fd00b4c90 0x7f4fd00b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.838+0000 7f4fd6000640 1 -- 172.21.15.7:0/3929641115 >> 172.21.15.7:0/3929641115 conn(0x7f4fd00b1d70 msgr2=0x7f4fd00cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:37.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.838+0000 7f4fd6000640 1 -- 172.21.15.7:0/3929641115 shutdown_connections 2024-09-10T21:43:37.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.838+0000 7f4fd6000640 1 -- 172.21.15.7:0/3929641115 wait complete. 2024-09-10T21:43:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.840+0000 7f4fd6000640 1 Processor -- start 2024-09-10T21:43:37.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.840+0000 7f4fd6000640 1 -- start start 2024-09-10T21:43:37.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.841+0000 7f4fd6000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fd00b2f80 0x7f4fd0135fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.841+0000 7f4fd6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd00b4c90 0x7f4fd00ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.841+0000 7f4fd6000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fd0145540 0x7f4fd0155cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.841+0000 7f4fd6000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd00cefc0 con 0x7f4fd00b4c90 2024-09-10T21:43:37.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.841+0000 7f4fd6000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4fd00cee40 con 0x7f4fd00b2f80 2024-09-10T21:43:37.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.841+0000 7f4fd6000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4fd00cf140 con 0x7f4fd0145540 2024-09-10T21:43:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.841+0000 7f4fcfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd00b4c90 0x7f4fd00ae190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.841+0000 7f4fcfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd00b4c90 0x7f4fd00ae190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44062/0 (socket says 172.21.15.7:44062) 2024-09-10T21:43:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.842+0000 7f4fcfe00640 1 -- 172.21.15.7:0/880835727 learned_addr learned my addr 172.21.15.7:0/880835727 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.842+0000 7f4fd5600640 1 --2- 172.21.15.7:0/880835727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fd0145540 0x7f4fd0155cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.842+0000 7f4fd4c00640 1 --2- 172.21.15.7:0/880835727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fd00b2f80 0x7f4fd0135fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.842+0000 7f4fcfe00640 1 -- 172.21.15.7:0/880835727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fd0145540 msgr2=0x7f4fd0155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.842+0000 7f4fcfe00640 1 --2- 172.21.15.7:0/880835727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fd0145540 0x7f4fd0155cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.842+0000 7f4fcfe00640 1 -- 172.21.15.7:0/880835727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fd00b2f80 msgr2=0x7f4fd0135fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.842+0000 7f4fcfe00640 1 --2- 172.21.15.7:0/880835727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fd00b2f80 0x7f4fd0135fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.842+0000 7f4fcfe00640 1 -- 172.21.15.7:0/880835727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fd01f5550 con 0x7f4fd00b4c90 2024-09-10T21:43:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.842+0000 7f4fd4c00640 1 --2- 172.21.15.7:0/880835727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fd00b2f80 0x7f4fd0135fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:37.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.842+0000 7f4fcfe00640 1 --2- 172.21.15.7:0/880835727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd00b4c90 0x7f4fd00ae190 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00ea60 tx=0x7f4fbc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.842+0000 7f4fccc00640 1 -- 172.21.15.7:0/880835727 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc00cde0 con 0x7f4fd00b4c90 2024-09-10T21:43:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.843+0000 7f4fccc00640 1 -- 172.21.15.7:0/880835727 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fbc004020 con 0x7f4fd00b4c90 2024-09-10T21:43:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.843+0000 7f4fccc00640 1 -- 172.21.15.7:0/880835727 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fbc004450 con 0x7f4fd00b4c90 2024-09-10T21:43:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.843+0000 7f4fd6000640 1 -- 172.21.15.7:0/880835727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fd00a9570 con 0x7f4fd00b4c90 2024-09-10T21:43:37.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.843+0000 7f4fd6000640 1 -- 172.21.15.7:0/880835727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fd00a9b30 con 0x7f4fd00b4c90 2024-09-10T21:43:37.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.844+0000 7f4fd6000640 1 -- 172.21.15.7:0/880835727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fd00cd3f0 con 0x7f4fd00b4c90 2024-09-10T21:43:37.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.845+0000 7f4fccc00640 1 -- 172.21.15.7:0/880835727 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4fbc0035f0 con 0x7f4fd00b4c90 2024-09-10T21:43:37.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.846+0000 7f4fccc00640 1 --2- 172.21.15.7:0/880835727 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4fb0077f70 0x7f4fb007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.846+0000 7f4fccc00640 1 -- 172.21.15.7:0/880835727 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f4fbc09a6c0 con 0x7f4fd00b4c90 2024-09-10T21:43:37.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.846+0000 7f4fd4c00640 1 --2- 172.21.15.7:0/880835727 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4fb0077f70 0x7f4fb007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.847+0000 7f4fd4c00640 1 --2- 172.21.15.7:0/880835727 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4fb0077f70 0x7f4fb007a430 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4007fd0 tx=0x7f4fc4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:37.850+0000 7f4fccc00640 1 -- 172.21.15.7:0/880835727 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4fd00cd3f0 con 0x7f4fd00b4c90 2024-09-10T21:43:38.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.042+0000 7f4fd6000640 1 -- 172.21.15.7:0/880835727 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4fd00cf5f0 con 0x7f4fb0077f70 2024-09-10T21:43:38.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.047+0000 7f4fccc00640 1 -- 172.21.15.7:0/880835727 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7f4fd00cf5f0 con 0x7f4fb0077f70 2024-09-10T21:43:38.051 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:38.053 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:43:38.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.052+0000 7f4fd6000640 1 -- 172.21.15.7:0/880835727 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4fb0077f70 msgr2=0x7f4fb007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:38.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.052+0000 7f4fd6000640 1 --2- 172.21.15.7:0/880835727 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4fb0077f70 0x7f4fb007a430 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4007fd0 tx=0x7f4fc4009040 comp rx=0 tx=0).stop 2024-09-10T21:43:38.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.052+0000 7f4fd6000640 1 -- 172.21.15.7:0/880835727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd00b4c90 msgr2=0x7f4fd00ae190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:38.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.052+0000 7f4fd6000640 1 --2- 172.21.15.7:0/880835727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd00b4c90 0x7f4fd00ae190 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc00ea60 tx=0x7f4fbc00ef20 comp rx=0 tx=0).stop 2024-09-10T21:43:38.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.052+0000 7f4fd6000640 1 -- 172.21.15.7:0/880835727 shutdown_connections 2024-09-10T21:43:38.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.052+0000 7f4fd6000640 1 --2- 172.21.15.7:0/880835727 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4fb0077f70 0x7f4fb007a430 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.052+0000 7f4fd6000640 1 --2- 172.21.15.7:0/880835727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4fd0145540 0x7f4fd0155cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.052+0000 7f4fd6000640 1 --2- 172.21.15.7:0/880835727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4fd00b4c90 0x7f4fd00ae190 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.052+0000 7f4fd6000640 1 --2- 172.21.15.7:0/880835727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4fd00b2f80 0x7f4fd0135fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.052+0000 7f4fd6000640 1 -- 172.21.15.7:0/880835727 >> 172.21.15.7:0/880835727 conn(0x7f4fd00b1d70 msgr2=0x7f4fd015f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:38.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.052+0000 7f4fd6000640 1 -- 172.21.15.7:0/880835727 shutdown_connections 2024-09-10T21:43:38.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.052+0000 7f4fd6000640 1 -- 172.21.15.7:0/880835727 wait complete. 2024-09-10T21:43:38.069 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-09-10T21:43:37.876455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242244,"kb_used_data":5192,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732476,"statfs":{"total":773094113280,"available":772846055424,"internally_reserved":0,"allocated":5316608,"data_stored":2816644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001568"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964362+0000","last_change":"2024-09-10T21:43:33.831659+0000","last_active":"2024-09-10T21:43:34.964362+0000","last_peered":"2024-09-10T21:43:34.964362+0000","last_clean":"2024-09-10T21:43:34.964362+0000","last_became_active":"2024-09-10T21:43:33.831421+0000","last_became_peered":"2024-09-10T21:43:33.831421+0000","last_unstale":"2024-09-10T21:43:34.964362+0000","last_undegraded":"2024-09-10T21:43:34.964362+0000","last_fullsized":"2024-09-10T21:43:34.964362+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:08.088562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.826291+0000","last_change":"2024-09-10T21:43:33.843041+0000","last_active":"2024-09-10T21:43:34.826291+0000","last_peered":"2024-09-10T21:43:34.826291+0000","last_clean":"2024-09-10T21:43:34.826291+0000","last_became_active":"2024-09-10T21:43:33.842756+0000","last_became_peered":"2024-09-10T21:43:33.842756+0000","last_unstale":"2024-09-10T21:43:34.826291+0000","last_undegraded":"2024-09-10T21:43:34.826291+0000","last_fullsized":"2024-09-10T21:43:34.826291+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:17.244798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827421+0000","last_change":"2024-09-10T21:43:33.828575+0000","last_active":"2024-09-10T21:43:34.827421+0000","last_peered":"2024-09-10T21:43:34.827421+0000","last_clean":"2024-09-10T21:43:34.827421+0000","last_became_active":"2024-09-10T21:43:33.826890+0000","last_became_peered":"2024-09-10T21:43:33.826890+0000","last_unstale":"2024-09-10T21:43:34.827421+0000","last_undegraded":"2024-09-10T21:43:34.827421+0000","last_fullsized":"2024-09-10T21:43:34.827421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:36.451245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821590+0000","last_change":"2024-09-10T21:43:33.838502+0000","last_active":"2024-09-10T21:43:34.821590+0000","last_peered":"2024-09-10T21:43:34.821590+0000","last_clean":"2024-09-10T21:43:34.821590+0000","last_became_active":"2024-09-10T21:43:33.838260+0000","last_became_peered":"2024-09-10T21:43:33.838260+0000","last_unstale":"2024-09-10T21:43:34.821590+0000","last_undegraded":"2024-09-10T21:43:34.821590+0000","last_fullsized":"2024-09-10T21:43:34.821590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:42.897275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.825823+0000","last_change":"2024-09-10T21:43:33.822071+0000","last_active":"2024-09-10T21:43:34.825823+0000","last_peered":"2024-09-10T21:43:34.825823+0000","last_clean":"2024-09-10T21:43:34.825823+0000","last_became_active":"2024-09-10T21:43:33.821761+0000","last_became_peered":"2024-09-10T21:43:33.821761+0000","last_unstale":"2024-09-10T21:43:34.825823+0000","last_undegraded":"2024-09-10T21:43:34.825823+0000","last_fullsized":"2024-09-10T21:43:34.825823+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:50.402304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.965038+0000","last_change":"2024-09-10T21:43:33.837822+0000","last_active":"2024-09-10T21:43:34.965038+0000","last_peered":"2024-09-10T21:43:34.965038+0000","last_clean":"2024-09-10T21:43:34.965038+0000","last_became_active":"2024-09-10T21:43:33.837455+0000","last_became_peered":"2024-09-10T21:43:33.837455+0000","last_unstale":"2024-09-10T21:43:34.965038+0000","last_undegraded":"2024-09-10T21:43:34.965038+0000","last_fullsized":"2024-09-10T21:43:34.965038+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:50:32.392416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964409+0000","last_change":"2024-09-10T21:43:33.832834+0000","last_active":"2024-09-10T21:43:34.964409+0000","last_peered":"2024-09-10T21:43:34.964409+0000","last_clean":"2024-09-10T21:43:34.964409+0000","last_became_active":"2024-09-10T21:43:33.830669+0000","last_became_peered":"2024-09-10T21:43:33.830669+0000","last_unstale":"2024-09-10T21:43:34.964409+0000","last_undegraded":"2024-09-10T21:43:34.964409+0000","last_fullsized":"2024-09-10T21:43:34.964409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:17.867453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.820769+0000","last_change":"2024-09-10T21:43:33.844562+0000","last_active":"2024-09-10T21:43:34.820769+0000","last_peered":"2024-09-10T21:43:34.820769+0000","last_clean":"2024-09-10T21:43:34.820769+0000","last_became_active":"2024-09-10T21:43:33.843239+0000","last_became_peered":"2024-09-10T21:43:33.843239+0000","last_unstale":"2024-09-10T21:43:34.820769+0000","last_undegraded":"2024-09-10T21:43:34.820769+0000","last_fullsized":"2024-09-10T21:43:34.820769+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:27.064716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821851+0000","last_change":"2024-09-10T21:43:33.841232+0000","last_active":"2024-09-10T21:43:34.821851+0000","last_peered":"2024-09-10T21:43:34.821851+0000","last_clean":"2024-09-10T21:43:34.821851+0000","last_became_active":"2024-09-10T21:43:33.839497+0000","last_became_peered":"2024-09-10T21:43:33.839497+0000","last_unstale":"2024-09-10T21:43:34.821851+0000","last_undegraded":"2024-09-10T21:43:34.821851+0000","last_fullsized":"2024-09-10T21:43:34.821851+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:46:36.124434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821744+0000","last_change":"2024-09-10T21:43:33.830208+0000","last_active":"2024-09-10T21:43:34.821744+0000","last_peered":"2024-09-10T21:43:34.821744+0000","last_clean":"2024-09-10T21:43:34.821744+0000","last_became_active":"2024-09-10T21:43:33.829759+0000","last_became_peered":"2024-09-10T21:43:33.829759+0000","last_unstale":"2024-09-10T21:43:34.821744+0000","last_undegraded":"2024-09-10T21:43:34.821744+0000","last_fullsized":"2024-09-10T21:43:34.821744+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:35.977807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821888+0000","last_change":"2024-09-10T21:43:33.843016+0000","last_active":"2024-09-10T21:43:34.821888+0000","last_peered":"2024-09-10T21:43:34.821888+0000","last_clean":"2024-09-10T21:43:34.821888+0000","last_became_active":"2024-09-10T21:43:33.839506+0000","last_became_peered":"2024-09-10T21:43:33.839506+0000","last_unstale":"2024-09-10T21:43:34.821888+0000","last_undegraded":"2024-09-10T21:43:34.821888+0000","last_fullsized":"2024-09-10T21:43:34.821888+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:17.244798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.825938+0000","last_change":"2024-09-10T21:43:33.826015+0000","last_active":"2024-09-10T21:43:34.825938+0000","last_peered":"2024-09-10T21:43:34.825938+0000","last_clean":"2024-09-10T21:43:34.825938+0000","last_became_active":"2024-09-10T21:43:33.825736+0000","last_became_peered":"2024-09-10T21:43:33.825736+0000","last_unstale":"2024-09-10T21:43:34.825938+0000","last_undegraded":"2024-09-10T21:43:34.825938+0000","last_fullsized":"2024-09-10T21:43:34.825938+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:35.977807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.963744+0000","last_change":"2024-09-10T21:43:33.824413+0000","last_active":"2024-09-10T21:43:34.963744+0000","last_peered":"2024-09-10T21:43:34.963744+0000","last_clean":"2024-09-10T21:43:34.963744+0000","last_became_active":"2024-09-10T21:43:33.824135+0000","last_became_peered":"2024-09-10T21:43:33.824135+0000","last_unstale":"2024-09-10T21:43:34.963744+0000","last_undegraded":"2024-09-10T21:43:34.963744+0000","last_fullsized":"2024-09-10T21:43:34.963744+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:05.580930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.820932+0000","last_change":"2024-09-10T21:43:33.840904+0000","last_active":"2024-09-10T21:43:34.820932+0000","last_peered":"2024-09-10T21:43:34.820932+0000","last_clean":"2024-09-10T21:43:34.820932+0000","last_became_active":"2024-09-10T21:43:33.839517+0000","last_became_peered":"2024-09-10T21:43:33.839517+0000","last_unstale":"2024-09-10T21:43:34.820932+0000","last_undegraded":"2024-09-10T21:43:34.820932+0000","last_fullsized":"2024-09-10T21:43:34.820932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:15.889480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964561+0000","last_change":"2024-09-10T21:43:33.829606+0000","last_active":"2024-09-10T21:43:34.964561+0000","last_peered":"2024-09-10T21:43:34.964561+0000","last_clean":"2024-09-10T21:43:34.964561+0000","last_became_active":"2024-09-10T21:43:33.828462+0000","last_became_peered":"2024-09-10T21:43:33.828462+0000","last_unstale":"2024-09-10T21:43:34.964561+0000","last_undegraded":"2024-09-10T21:43:34.964561+0000","last_fullsized":"2024-09-10T21:43:34.964561+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:00.271285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964100+0000","last_change":"2024-09-10T21:43:33.843027+0000","last_active":"2024-09-10T21:43:34.964100+0000","last_peered":"2024-09-10T21:43:34.964100+0000","last_clean":"2024-09-10T21:43:34.964100+0000","last_became_active":"2024-09-10T21:43:33.842688+0000","last_became_peered":"2024-09-10T21:43:33.842688+0000","last_unstale":"2024-09-10T21:43:34.964100+0000","last_undegraded":"2024-09-10T21:43:34.964100+0000","last_fullsized":"2024-09-10T21:43:34.964100+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:00.055126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827869+0000","last_change":"2024-09-10T21:43:33.832401+0000","last_active":"2024-09-10T21:43:34.827869+0000","last_peered":"2024-09-10T21:43:34.827869+0000","last_clean":"2024-09-10T21:43:34.827869+0000","last_became_active":"2024-09-10T21:43:33.830076+0000","last_became_peered":"2024-09-10T21:43:33.830076+0000","last_unstale":"2024-09-10T21:43:34.827869+0000","last_undegraded":"2024-09-10T21:43:34.827869+0000","last_fullsized":"2024-09-10T21:43:34.827869+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:48.788649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827135+0000","last_change":"2024-09-10T21:43:33.843130+0000","last_active":"2024-09-10T21:43:34.827135+0000","last_peered":"2024-09-10T21:43:34.827135+0000","last_clean":"2024-09-10T21:43:34.827135+0000","last_became_active":"2024-09-10T21:43:33.842837+0000","last_became_peered":"2024-09-10T21:43:33.842837+0000","last_unstale":"2024-09-10T21:43:34.827135+0000","last_undegraded":"2024-09-10T21:43:34.827135+0000","last_fullsized":"2024-09-10T21:43:34.827135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:35.358907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964359+0000","last_change":"2024-09-10T21:43:33.837854+0000","last_active":"2024-09-10T21:43:34.964359+0000","last_peered":"2024-09-10T21:43:34.964359+0000","last_clean":"2024-09-10T21:43:34.964359+0000","last_became_active":"2024-09-10T21:43:33.837587+0000","last_became_peered":"2024-09-10T21:43:33.837587+0000","last_unstale":"2024-09-10T21:43:34.964359+0000","last_undegraded":"2024-09-10T21:43:34.964359+0000","last_fullsized":"2024-09-10T21:43:34.964359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:12.229844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.963914+0000","last_change":"2024-09-10T21:43:33.824493+0000","last_active":"2024-09-10T21:43:34.963914+0000","last_peered":"2024-09-10T21:43:34.963914+0000","last_clean":"2024-09-10T21:43:34.963914+0000","last_became_active":"2024-09-10T21:43:33.822764+0000","last_became_peered":"2024-09-10T21:43:33.822764+0000","last_unstale":"2024-09-10T21:43:34.963914+0000","last_undegraded":"2024-09-10T21:43:34.963914+0000","last_fullsized":"2024-09-10T21:43:34.963914+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:40.279210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964149+0000","last_change":"2024-09-10T21:43:33.828818+0000","last_active":"2024-09-10T21:43:34.964149+0000","last_peered":"2024-09-10T21:43:34.964149+0000","last_clean":"2024-09-10T21:43:34.964149+0000","last_became_active":"2024-09-10T21:43:33.828499+0000","last_became_peered":"2024-09-10T21:43:33.828499+0000","last_unstale":"2024-09-10T21:43:34.964149+0000","last_undegraded":"2024-09-10T21:43:34.964149+0000","last_fullsized":"2024-09-10T21:43:34.964149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:58.334154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821138+0000","last_change":"2024-09-10T21:43:33.836218+0000","last_active":"2024-09-10T21:43:34.821138+0000","last_peered":"2024-09-10T21:43:34.821138+0000","last_clean":"2024-09-10T21:43:34.821138+0000","last_became_active":"2024-09-10T21:43:33.829991+0000","last_became_peered":"2024-09-10T21:43:33.829991+0000","last_unstale":"2024-09-10T21:43:34.821138+0000","last_undegraded":"2024-09-10T21:43:34.821138+0000","last_fullsized":"2024-09-10T21:43:34.821138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:21.399612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964869+0000","last_change":"2024-09-10T21:43:33.835908+0000","last_active":"2024-09-10T21:43:34.964869+0000","last_peered":"2024-09-10T21:43:34.964869+0000","last_clean":"2024-09-10T21:43:34.964869+0000","last_became_active":"2024-09-10T21:43:33.835629+0000","last_became_peered":"2024-09-10T21:43:33.835629+0000","last_unstale":"2024-09-10T21:43:34.964869+0000","last_undegraded":"2024-09-10T21:43:34.964869+0000","last_fullsized":"2024-09-10T21:43:34.964869+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:39.640769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.965050+0000","last_change":"2024-09-10T21:43:33.836915+0000","last_active":"2024-09-10T21:43:34.965050+0000","last_peered":"2024-09-10T21:43:34.965050+0000","last_clean":"2024-09-10T21:43:34.965050+0000","last_became_active":"2024-09-10T21:43:33.836660+0000","last_became_peered":"2024-09-10T21:43:33.836660+0000","last_unstale":"2024-09-10T21:43:34.965050+0000","last_undegraded":"2024-09-10T21:43:34.965050+0000","last_fullsized":"2024-09-10T21:43:34.965050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:55.896275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964723+0000","last_change":"2024-09-10T21:43:33.829375+0000","last_active":"2024-09-10T21:43:34.964723+0000","last_peered":"2024-09-10T21:43:34.964723+0000","last_clean":"2024-09-10T21:43:34.964723+0000","last_became_active":"2024-09-10T21:43:33.829136+0000","last_became_peered":"2024-09-10T21:43:33.829136+0000","last_unstale":"2024-09-10T21:43:34.964723+0000","last_undegraded":"2024-09-10T21:43:34.964723+0000","last_fullsized":"2024-09-10T21:43:34.964723+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:18.430684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964206+0000","last_change":"2024-09-10T21:43:33.822204+0000","last_active":"2024-09-10T21:43:34.964206+0000","last_peered":"2024-09-10T21:43:34.964206+0000","last_clean":"2024-09-10T21:43:34.964206+0000","last_became_active":"2024-09-10T21:43:33.821839+0000","last_became_peered":"2024-09-10T21:43:33.821839+0000","last_unstale":"2024-09-10T21:43:34.964206+0000","last_undegraded":"2024-09-10T21:43:34.964206+0000","last_fullsized":"2024-09-10T21:43:34.964206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:27.707468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827503+0000","last_change":"2024-09-10T21:43:33.830564+0000","last_active":"2024-09-10T21:43:34.827503+0000","last_peered":"2024-09-10T21:43:34.827503+0000","last_clean":"2024-09-10T21:43:34.827503+0000","last_became_active":"2024-09-10T21:43:33.830317+0000","last_became_peered":"2024-09-10T21:43:33.830317+0000","last_unstale":"2024-09-10T21:43:34.827503+0000","last_undegraded":"2024-09-10T21:43:34.827503+0000","last_fullsized":"2024-09-10T21:43:34.827503+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:07.224662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.825957+0000","last_change":"2024-09-10T21:43:33.820801+0000","last_active":"2024-09-10T21:43:34.825957+0000","last_peered":"2024-09-10T21:43:34.825957+0000","last_clean":"2024-09-10T21:43:34.825957+0000","last_became_active":"2024-09-10T21:43:33.819618+0000","last_became_peered":"2024-09-10T21:43:33.819618+0000","last_unstale":"2024-09-10T21:43:34.825957+0000","last_undegraded":"2024-09-10T21:43:34.825957+0000","last_fullsized":"2024-09-10T21:43:34.825957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:43.414568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964969+0000","last_change":"2024-09-10T21:43:33.826710+0000","last_active":"2024-09-10T21:43:34.964969+0000","last_peered":"2024-09-10T21:43:34.964969+0000","last_clean":"2024-09-10T21:43:34.964969+0000","last_became_active":"2024-09-10T21:43:33.826304+0000","last_became_peered":"2024-09-10T21:43:33.826304+0000","last_unstale":"2024-09-10T21:43:34.964969+0000","last_undegraded":"2024-09-10T21:43:34.964969+0000","last_fullsized":"2024-09-10T21:43:34.964969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:17.312273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.826947+0000","last_change":"2024-09-10T21:43:33.827874+0000","last_active":"2024-09-10T21:43:34.826947+0000","last_peered":"2024-09-10T21:43:34.826947+0000","last_clean":"2024-09-10T21:43:34.826947+0000","last_became_active":"2024-09-10T21:43:33.827192+0000","last_became_peered":"2024-09-10T21:43:33.827192+0000","last_unstale":"2024-09-10T21:43:34.826947+0000","last_undegraded":"2024-09-10T21:43:34.826947+0000","last_fullsized":"2024-09-10T21:43:34.826947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:17.449957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964734+0000","last_change":"2024-09-10T21:43:33.831562+0000","last_active":"2024-09-10T21:43:34.964734+0000","last_peered":"2024-09-10T21:43:34.964734+0000","last_clean":"2024-09-10T21:43:34.964734+0000","last_became_active":"2024-09-10T21:43:33.831280+0000","last_became_peered":"2024-09-10T21:43:33.831280+0000","last_unstale":"2024-09-10T21:43:34.964734+0000","last_undegraded":"2024-09-10T21:43:34.964734+0000","last_fullsized":"2024-09-10T21:43:34.964734+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:07.409474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.963822+0000","last_change":"2024-09-10T21:43:33.816463+0000","last_active":"2024-09-10T21:43:34.963822+0000","last_peered":"2024-09-10T21:43:34.963822+0000","last_clean":"2024-09-10T21:43:34.963822+0000","last_became_active":"2024-09-10T21:43:33.815454+0000","last_became_peered":"2024-09-10T21:43:33.815454+0000","last_unstale":"2024-09-10T21:43:34.963822+0000","last_undegraded":"2024-09-10T21:43:34.963822+0000","last_fullsized":"2024-09-10T21:43:34.963822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:29.786967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964247+0000","last_change":"2024-09-10T21:43:33.817533+0000","last_active":"2024-09-10T21:43:34.964247+0000","last_peered":"2024-09-10T21:43:34.964247+0000","last_clean":"2024-09-10T21:43:34.964247+0000","last_became_active":"2024-09-10T21:43:33.817259+0000","last_became_peered":"2024-09-10T21:43:33.817259+0000","last_unstale":"2024-09-10T21:43:34.964247+0000","last_undegraded":"2024-09-10T21:43:34.964247+0000","last_fullsized":"2024-09-10T21:43:34.964247+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:30.337797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"43'4","reported_seq":19,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.965415+0000","last_change":"2024-09-10T21:43:32.829030+0000","last_active":"2024-09-10T21:43:34.965415+0000","last_peered":"2024-09-10T21:43:34.965415+0000","last_clean":"2024-09-10T21:43:34.965415+0000","last_became_active":"2024-09-10T21:43:32.825541+0000","last_became_peered":"2024-09-10T21:43:32.825541+0000","last_unstale":"2024-09-10T21:43:34.965415+0000","last_undegraded":"2024-09-10T21:43:34.965415+0000","last_fullsized":"2024-09-10T21:43:34.965415+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:02.003048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964741+0000","last_change":"2024-09-10T21:43:33.823433+0000","last_active":"2024-09-10T21:43:34.964741+0000","last_peered":"2024-09-10T21:43:34.964741+0000","last_clean":"2024-09-10T21:43:34.964741+0000","last_became_active":"2024-09-10T21:43:33.823166+0000","last_became_peered":"2024-09-10T21:43:33.823166+0000","last_unstale":"2024-09-10T21:43:34.964741+0000","last_undegraded":"2024-09-10T21:43:34.964741+0000","last_fullsized":"2024-09-10T21:43:34.964741+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:40.031274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827729+0000","last_change":"2024-09-10T21:43:32.815373+0000","last_active":"2024-09-10T21:43:34.827729+0000","last_peered":"2024-09-10T21:43:34.827729+0000","last_clean":"2024-09-10T21:43:34.827729+0000","last_became_active":"2024-09-10T21:43:32.815094+0000","last_became_peered":"2024-09-10T21:43:32.815094+0000","last_unstale":"2024-09-10T21:43:34.827729+0000","last_undegraded":"2024-09-10T21:43:34.827729+0000","last_fullsized":"2024-09-10T21:43:34.827729+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:41.170819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.826187+0000","last_change":"2024-09-10T21:43:33.831504+0000","last_active":"2024-09-10T21:43:34.826187+0000","last_peered":"2024-09-10T21:43:34.826187+0000","last_clean":"2024-09-10T21:43:34.826187+0000","last_became_active":"2024-09-10T21:43:33.831189+0000","last_became_peered":"2024-09-10T21:43:33.831189+0000","last_unstale":"2024-09-10T21:43:34.826187+0000","last_undegraded":"2024-09-10T21:43:34.826187+0000","last_fullsized":"2024-09-10T21:43:34.826187+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:46:36.124434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964731+0000","last_change":"2024-09-10T21:43:32.823634+0000","last_active":"2024-09-10T21:43:34.964731+0000","last_peered":"2024-09-10T21:43:34.964731+0000","last_clean":"2024-09-10T21:43:34.964731+0000","last_became_active":"2024-09-10T21:43:32.823280+0000","last_became_peered":"2024-09-10T21:43:32.823280+0000","last_unstale":"2024-09-10T21:43:34.964731+0000","last_undegraded":"2024-09-10T21:43:34.964731+0000","last_fullsized":"2024-09-10T21:43:34.964731+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:45.959240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964674+0000","last_change":"2024-09-10T21:43:33.822347+0000","last_active":"2024-09-10T21:43:34.964674+0000","last_peered":"2024-09-10T21:43:34.964674+0000","last_clean":"2024-09-10T21:43:34.964674+0000","last_became_active":"2024-09-10T21:43:33.822094+0000","last_became_peered":"2024-09-10T21:43:33.822094+0000","last_unstale":"2024-09-10T21:43:34.964674+0000","last_undegraded":"2024-09-10T21:43:34.964674+0000","last_fullsized":"2024-09-10T21:43:34.964674+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:55.896275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964194+0000","last_change":"2024-09-10T21:43:32.823580+0000","last_active":"2024-09-10T21:43:34.964194+0000","last_peered":"2024-09-10T21:43:34.964194+0000","last_clean":"2024-09-10T21:43:34.964194+0000","last_became_active":"2024-09-10T21:43:32.823332+0000","last_became_peered":"2024-09-10T21:43:32.823332+0000","last_unstale":"2024-09-10T21:43:34.964194+0000","last_undegraded":"2024-09-10T21:43:34.964194+0000","last_fullsized":"2024-09-10T21:43:34.964194+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:51.112408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827062+0000","last_change":"2024-09-10T21:43:33.828469+0000","last_active":"2024-09-10T21:43:34.827062+0000","last_peered":"2024-09-10T21:43:34.827062+0000","last_clean":"2024-09-10T21:43:34.827062+0000","last_became_active":"2024-09-10T21:43:33.827687+0000","last_became_peered":"2024-09-10T21:43:33.827687+0000","last_unstale":"2024-09-10T21:43:34.827062+0000","last_undegraded":"2024-09-10T21:43:34.827062+0000","last_fullsized":"2024-09-10T21:43:34.827062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:37.120232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964416+0000","last_change":"2024-09-10T21:43:32.817120+0000","last_active":"2024-09-10T21:43:34.964416+0000","last_peered":"2024-09-10T21:43:34.964416+0000","last_clean":"2024-09-10T21:43:34.964416+0000","last_became_active":"2024-09-10T21:43:32.816459+0000","last_became_peered":"2024-09-10T21:43:32.816459+0000","last_unstale":"2024-09-10T21:43:34.964416+0000","last_undegraded":"2024-09-10T21:43:34.964416+0000","last_fullsized":"2024-09-10T21:43:34.964416+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:08.385157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821229+0000","last_change":"2024-09-10T21:43:33.831618+0000","last_active":"2024-09-10T21:43:34.821229+0000","last_peered":"2024-09-10T21:43:34.821229+0000","last_clean":"2024-09-10T21:43:34.821229+0000","last_became_active":"2024-09-10T21:43:33.831264+0000","last_became_peered":"2024-09-10T21:43:33.831264+0000","last_unstale":"2024-09-10T21:43:34.821229+0000","last_undegraded":"2024-09-10T21:43:34.821229+0000","last_fullsized":"2024-09-10T21:43:34.821229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:34.427724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827099+0000","last_change":"2024-09-10T21:43:32.817363+0000","last_active":"2024-09-10T21:43:34.827099+0000","last_peered":"2024-09-10T21:43:34.827099+0000","last_clean":"2024-09-10T21:43:34.827099+0000","last_became_active":"2024-09-10T21:43:32.817101+0000","last_became_peered":"2024-09-10T21:43:32.817101+0000","last_unstale":"2024-09-10T21:43:34.827099+0000","last_undegraded":"2024-09-10T21:43:34.827099+0000","last_fullsized":"2024-09-10T21:43:34.827099+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:50.374958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.820877+0000","last_change":"2024-09-10T21:43:33.842005+0000","last_active":"2024-09-10T21:43:34.820877+0000","last_peered":"2024-09-10T21:43:34.820877+0000","last_clean":"2024-09-10T21:43:34.820877+0000","last_became_active":"2024-09-10T21:43:33.841511+0000","last_became_peered":"2024-09-10T21:43:33.841511+0000","last_unstale":"2024-09-10T21:43:34.820877+0000","last_undegraded":"2024-09-10T21:43:34.820877+0000","last_fullsized":"2024-09-10T21:43:34.820877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:19.879570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.825531+0000","last_change":"2024-09-10T21:43:32.821084+0000","last_active":"2024-09-10T21:43:34.825531+0000","last_peered":"2024-09-10T21:43:34.825531+0000","last_clean":"2024-09-10T21:43:34.825531+0000","last_became_active":"2024-09-10T21:43:32.820785+0000","last_became_peered":"2024-09-10T21:43:32.820785+0000","last_unstale":"2024-09-10T21:43:34.825531+0000","last_undegraded":"2024-09-10T21:43:34.825531+0000","last_fullsized":"2024-09-10T21:43:34.825531+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:17.522305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.965027+0000","last_change":"2024-09-10T21:43:33.835492+0000","last_active":"2024-09-10T21:43:34.965027+0000","last_peered":"2024-09-10T21:43:34.965027+0000","last_clean":"2024-09-10T21:43:34.965027+0000","last_became_active":"2024-09-10T21:43:33.835226+0000","last_became_peered":"2024-09-10T21:43:33.835226+0000","last_unstale":"2024-09-10T21:43:34.965027+0000","last_undegraded":"2024-09-10T21:43:34.965027+0000","last_fullsized":"2024-09-10T21:43:34.965027+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:18.972165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.820940+0000","last_change":"2024-09-10T21:43:32.813119+0000","last_active":"2024-09-10T21:43:34.820940+0000","last_peered":"2024-09-10T21:43:34.820940+0000","last_clean":"2024-09-10T21:43:34.820940+0000","last_became_active":"2024-09-10T21:43:32.812830+0000","last_became_peered":"2024-09-10T21:43:32.812830+0000","last_unstale":"2024-09-10T21:43:34.820940+0000","last_undegraded":"2024-09-10T21:43:34.820940+0000","last_fullsized":"2024-09-10T21:43:34.820940+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:10:54.275836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827851+0000","last_change":"2024-09-10T21:43:33.819851+0000","last_active":"2024-09-10T21:43:34.827851+0000","last_peered":"2024-09-10T21:43:34.827851+0000","last_clean":"2024-09-10T21:43:34.827851+0000","last_became_active":"2024-09-10T21:43:33.819541+0000","last_became_peered":"2024-09-10T21:43:33.819541+0000","last_unstale":"2024-09-10T21:43:34.827851+0000","last_undegraded":"2024-09-10T21:43:34.827851+0000","last_fullsized":"2024-09-10T21:43:34.827851+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:01.398499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827898+0000","last_change":"2024-09-10T21:43:32.819165+0000","last_active":"2024-09-10T21:43:34.827898+0000","last_peered":"2024-09-10T21:43:34.827898+0000","last_clean":"2024-09-10T21:43:34.827898+0000","last_became_active":"2024-09-10T21:43:32.818823+0000","last_became_peered":"2024-09-10T21:43:32.818823+0000","last_unstale":"2024-09-10T21:43:34.827898+0000","last_undegraded":"2024-09-10T21:43:34.827898+0000","last_fullsized":"2024-09-10T21:43:34.827898+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:00.693410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964627+0000","last_change":"2024-09-10T21:43:33.835882+0000","last_active":"2024-09-10T21:43:34.964627+0000","last_peered":"2024-09-10T21:43:34.964627+0000","last_clean":"2024-09-10T21:43:34.964627+0000","last_became_active":"2024-09-10T21:43:33.834725+0000","last_became_peered":"2024-09-10T21:43:33.834725+0000","last_unstale":"2024-09-10T21:43:34.964627+0000","last_undegraded":"2024-09-10T21:43:34.964627+0000","last_fullsized":"2024-09-10T21:43:34.964627+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:00.315371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821913+0000","last_change":"2024-09-10T21:43:32.824178+0000","last_active":"2024-09-10T21:43:34.821913+0000","last_peered":"2024-09-10T21:43:34.821913+0000","last_clean":"2024-09-10T21:43:34.821913+0000","last_became_active":"2024-09-10T21:43:32.823711+0000","last_became_peered":"2024-09-10T21:43:32.823711+0000","last_unstale":"2024-09-10T21:43:34.821913+0000","last_undegraded":"2024-09-10T21:43:34.821913+0000","last_fullsized":"2024-09-10T21:43:34.821913+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:24.084153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964852+0000","last_change":"2024-09-10T21:43:33.832073+0000","last_active":"2024-09-10T21:43:34.964852+0000","last_peered":"2024-09-10T21:43:34.964852+0000","last_clean":"2024-09-10T21:43:34.964852+0000","last_became_active":"2024-09-10T21:43:33.831889+0000","last_became_peered":"2024-09-10T21:43:33.831889+0000","last_unstale":"2024-09-10T21:43:34.964852+0000","last_undegraded":"2024-09-10T21:43:34.964852+0000","last_fullsized":"2024-09-10T21:43:34.964852+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:47.971631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"43'3","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.965058+0000","last_change":"2024-09-10T21:43:32.819667+0000","last_active":"2024-09-10T21:43:34.965058+0000","last_peered":"2024-09-10T21:43:34.965058+0000","last_clean":"2024-09-10T21:43:34.965058+0000","last_became_active":"2024-09-10T21:43:32.818260+0000","last_became_peered":"2024-09-10T21:43:32.818260+0000","last_unstale":"2024-09-10T21:43:34.965058+0000","last_undegraded":"2024-09-10T21:43:34.965058+0000","last_fullsized":"2024-09-10T21:43:34.965058+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:08.385157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964793+0000","last_change":"2024-09-10T21:43:33.831800+0000","last_active":"2024-09-10T21:43:34.964793+0000","last_peered":"2024-09-10T21:43:34.964793+0000","last_clean":"2024-09-10T21:43:34.964793+0000","last_became_active":"2024-09-10T21:43:33.831451+0000","last_became_peered":"2024-09-10T21:43:33.831451+0000","last_unstale":"2024-09-10T21:43:34.964793+0000","last_undegraded":"2024-09-10T21:43:34.964793+0000","last_fullsized":"2024-09-10T21:43:34.964793+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:47:55.976861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":21,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.963972+0000","last_change":"2024-09-10T21:43:32.814419+0000","last_active":"2024-09-10T21:43:34.963972+0000","last_peered":"2024-09-10T21:43:34.963972+0000","last_clean":"2024-09-10T21:43:34.963972+0000","last_became_active":"2024-09-10T21:43:32.814065+0000","last_became_peered":"2024-09-10T21:43:32.814065+0000","last_unstale":"2024-09-10T21:43:34.963972+0000","last_undegraded":"2024-09-10T21:43:34.963972+0000","last_fullsized":"2024-09-10T21:43:34.963972+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:27.883432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964000+0000","last_change":"2024-09-10T21:43:33.831862+0000","last_active":"2024-09-10T21:43:34.964000+0000","last_peered":"2024-09-10T21:43:34.964000+0000","last_clean":"2024-09-10T21:43:34.964000+0000","last_became_active":"2024-09-10T21:43:33.831478+0000","last_became_peered":"2024-09-10T21:43:33.831478+0000","last_unstale":"2024-09-10T21:43:34.964000+0000","last_undegraded":"2024-09-10T21:43:34.964000+0000","last_fullsized":"2024-09-10T21:43:34.964000+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:42.178119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964521+0000","last_change":"2024-09-10T21:43:32.817172+0000","last_active":"2024-09-10T21:43:34.964521+0000","last_peered":"2024-09-10T21:43:34.964521+0000","last_clean":"2024-09-10T21:43:34.964521+0000","last_became_active":"2024-09-10T21:43:32.816835+0000","last_became_peered":"2024-09-10T21:43:32.816835+0000","last_unstale":"2024-09-10T21:43:34.964521+0000","last_undegraded":"2024-09-10T21:43:34.964521+0000","last_fullsized":"2024-09-10T21:43:34.964521+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:03.941341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821330+0000","last_change":"2024-09-10T21:43:33.831474+0000","last_active":"2024-09-10T21:43:34.821330+0000","last_peered":"2024-09-10T21:43:34.821330+0000","last_clean":"2024-09-10T21:43:34.821330+0000","last_became_active":"2024-09-10T21:43:33.831188+0000","last_became_peered":"2024-09-10T21:43:33.831188+0000","last_unstale":"2024-09-10T21:43:34.821330+0000","last_undegraded":"2024-09-10T21:43:34.821330+0000","last_fullsized":"2024-09-10T21:43:34.821330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:08.638143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964446+0000","last_change":"2024-09-10T21:43:32.817318+0000","last_active":"2024-09-10T21:43:34.964446+0000","last_peered":"2024-09-10T21:43:34.964446+0000","last_clean":"2024-09-10T21:43:34.964446+0000","last_became_active":"2024-09-10T21:43:32.817011+0000","last_became_peered":"2024-09-10T21:43:32.817011+0000","last_unstale":"2024-09-10T21:43:34.964446+0000","last_undegraded":"2024-09-10T21:43:34.964446+0000","last_fullsized":"2024-09-10T21:43:34.964446+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:48.994627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964344+0000","last_change":"2024-09-10T21:43:33.819355+0000","last_active":"2024-09-10T21:43:34.964344+0000","last_peered":"2024-09-10T21:43:34.964344+0000","last_clean":"2024-09-10T21:43:34.964344+0000","last_became_active":"2024-09-10T21:43:33.818615+0000","last_became_peered":"2024-09-10T21:43:33.818615+0000","last_unstale":"2024-09-10T21:43:34.964344+0000","last_undegraded":"2024-09-10T21:43:34.964344+0000","last_fullsized":"2024-09-10T21:43:34.964344+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:25.929016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964874+0000","last_change":"2024-09-10T21:43:32.823174+0000","last_active":"2024-09-10T21:43:34.964874+0000","last_peered":"2024-09-10T21:43:34.964874+0000","last_clean":"2024-09-10T21:43:34.964874+0000","last_became_active":"2024-09-10T21:43:32.822859+0000","last_became_peered":"2024-09-10T21:43:32.822859+0000","last_unstale":"2024-09-10T21:43:34.964874+0000","last_undegraded":"2024-09-10T21:43:34.964874+0000","last_fullsized":"2024-09-10T21:43:34.964874+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:10.409867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964413+0000","last_change":"2024-09-10T21:43:33.828433+0000","last_active":"2024-09-10T21:43:34.964413+0000","last_peered":"2024-09-10T21:43:34.964413+0000","last_clean":"2024-09-10T21:43:34.964413+0000","last_became_active":"2024-09-10T21:43:33.828120+0000","last_became_peered":"2024-09-10T21:43:33.828120+0000","last_unstale":"2024-09-10T21:43:34.964413+0000","last_undegraded":"2024-09-10T21:43:34.964413+0000","last_fullsized":"2024-09-10T21:43:34.964413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:43.855252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"43'8","reported_seq":23,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821046+0000","last_change":"2024-09-10T21:43:32.820418+0000","last_active":"2024-09-10T21:43:34.821046+0000","last_peered":"2024-09-10T21:43:34.821046+0000","last_clean":"2024-09-10T21:43:34.821046+0000","last_became_active":"2024-09-10T21:43:32.820072+0000","last_became_peered":"2024-09-10T21:43:32.820072+0000","last_unstale":"2024-09-10T21:43:34.821046+0000","last_undegraded":"2024-09-10T21:43:34.821046+0000","last_fullsized":"2024-09-10T21:43:34.821046+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:22.429726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964410+0000","last_change":"2024-09-10T21:43:33.823613+0000","last_active":"2024-09-10T21:43:34.964410+0000","last_peered":"2024-09-10T21:43:34.964410+0000","last_clean":"2024-09-10T21:43:34.964410+0000","last_became_active":"2024-09-10T21:43:33.823284+0000","last_became_peered":"2024-09-10T21:43:33.823284+0000","last_unstale":"2024-09-10T21:43:34.964410+0000","last_undegraded":"2024-09-10T21:43:34.964410+0000","last_fullsized":"2024-09-10T21:43:34.964410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:14.278979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.963676+0000","last_change":"2024-09-10T21:43:32.826242+0000","last_active":"2024-09-10T21:43:34.963676+0000","last_peered":"2024-09-10T21:43:34.963676+0000","last_clean":"2024-09-10T21:43:34.963676+0000","last_became_active":"2024-09-10T21:43:32.825897+0000","last_became_peered":"2024-09-10T21:43:32.825897+0000","last_unstale":"2024-09-10T21:43:34.963676+0000","last_undegraded":"2024-09-10T21:43:34.963676+0000","last_fullsized":"2024-09-10T21:43:34.963676+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:50:38.303165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.822002+0000","last_change":"2024-09-10T21:43:33.821430+0000","last_active":"2024-09-10T21:43:34.822002+0000","last_peered":"2024-09-10T21:43:34.822002+0000","last_clean":"2024-09-10T21:43:34.822002+0000","last_became_active":"2024-09-10T21:43:33.821135+0000","last_became_peered":"2024-09-10T21:43:33.821135+0000","last_unstale":"2024-09-10T21:43:34.822002+0000","last_undegraded":"2024-09-10T21:43:34.822002+0000","last_fullsized":"2024-09-10T21:43:34.822002+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:55.740837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827881+0000","last_change":"2024-09-10T21:43:32.818125+0000","last_active":"2024-09-10T21:43:34.827881+0000","last_peered":"2024-09-10T21:43:34.827881+0000","last_clean":"2024-09-10T21:43:34.827881+0000","last_became_active":"2024-09-10T21:43:32.817854+0000","last_became_peered":"2024-09-10T21:43:32.817854+0000","last_unstale":"2024-09-10T21:43:34.827881+0000","last_undegraded":"2024-09-10T21:43:34.827881+0000","last_fullsized":"2024-09-10T21:43:34.827881+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:39.015707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964395+0000","last_change":"2024-09-10T21:43:33.836217+0000","last_active":"2024-09-10T21:43:34.964395+0000","last_peered":"2024-09-10T21:43:34.964395+0000","last_clean":"2024-09-10T21:43:34.964395+0000","last_became_active":"2024-09-10T21:43:33.834378+0000","last_became_peered":"2024-09-10T21:43:33.834378+0000","last_unstale":"2024-09-10T21:43:34.964395+0000","last_undegraded":"2024-09-10T21:43:34.964395+0000","last_fullsized":"2024-09-10T21:43:34.964395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:46.966540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.826108+0000","last_change":"2024-09-10T21:43:32.819913+0000","last_active":"2024-09-10T21:43:34.826108+0000","last_peered":"2024-09-10T21:43:34.826108+0000","last_clean":"2024-09-10T21:43:34.826108+0000","last_became_active":"2024-09-10T21:43:32.819608+0000","last_became_peered":"2024-09-10T21:43:32.819608+0000","last_unstale":"2024-09-10T21:43:34.826108+0000","last_undegraded":"2024-09-10T21:43:34.826108+0000","last_fullsized":"2024-09-10T21:43:34.826108+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:26.057416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964489+0000","last_change":"2024-09-10T21:43:33.820074+0000","last_active":"2024-09-10T21:43:34.964489+0000","last_peered":"2024-09-10T21:43:34.964489+0000","last_clean":"2024-09-10T21:43:34.964489+0000","last_became_active":"2024-09-10T21:43:33.819680+0000","last_became_peered":"2024-09-10T21:43:33.819680+0000","last_unstale":"2024-09-10T21:43:34.964489+0000","last_undegraded":"2024-09-10T21:43:34.964489+0000","last_fullsized":"2024-09-10T21:43:34.964489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:00.459533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964733+0000","last_change":"2024-09-10T21:43:32.828718+0000","last_active":"2024-09-10T21:43:34.964733+0000","last_peered":"2024-09-10T21:43:34.964733+0000","last_clean":"2024-09-10T21:43:34.964733+0000","last_became_active":"2024-09-10T21:43:32.825482+0000","last_became_peered":"2024-09-10T21:43:32.825482+0000","last_unstale":"2024-09-10T21:43:34.964733+0000","last_undegraded":"2024-09-10T21:43:34.964733+0000","last_fullsized":"2024-09-10T21:43:34.964733+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:12.601680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.965136+0000","last_change":"2024-09-10T21:43:33.826535+0000","last_active":"2024-09-10T21:43:34.965136+0000","last_peered":"2024-09-10T21:43:34.965136+0000","last_clean":"2024-09-10T21:43:34.965136+0000","last_became_active":"2024-09-10T21:43:33.822084+0000","last_became_peered":"2024-09-10T21:43:33.822084+0000","last_unstale":"2024-09-10T21:43:34.965136+0000","last_undegraded":"2024-09-10T21:43:34.965136+0000","last_fullsized":"2024-09-10T21:43:34.965136+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:35.773434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964733+0000","last_change":"2024-09-10T21:43:32.816870+0000","last_active":"2024-09-10T21:43:34.964733+0000","last_peered":"2024-09-10T21:43:34.964733+0000","last_clean":"2024-09-10T21:43:34.964733+0000","last_became_active":"2024-09-10T21:43:32.815750+0000","last_became_peered":"2024-09-10T21:43:32.815750+0000","last_unstale":"2024-09-10T21:43:34.964733+0000","last_undegraded":"2024-09-10T21:43:34.964733+0000","last_fullsized":"2024-09-10T21:43:34.964733+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:10.023479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964885+0000","last_change":"2024-09-10T21:43:33.821301+0000","last_active":"2024-09-10T21:43:34.964885+0000","last_peered":"2024-09-10T21:43:34.964885+0000","last_clean":"2024-09-10T21:43:34.964885+0000","last_became_active":"2024-09-10T21:43:33.819882+0000","last_became_peered":"2024-09-10T21:43:33.819882+0000","last_unstale":"2024-09-10T21:43:34.964885+0000","last_undegraded":"2024-09-10T21:43:34.964885+0000","last_fullsized":"2024-09-10T21:43:34.964885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:18.972165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.825633+0000","last_change":"2024-09-10T21:43:32.818173+0000","last_active":"2024-09-10T21:43:34.825633+0000","last_peered":"2024-09-10T21:43:34.825633+0000","last_clean":"2024-09-10T21:43:34.825633+0000","last_became_active":"2024-09-10T21:43:32.817738+0000","last_became_peered":"2024-09-10T21:43:32.817738+0000","last_unstale":"2024-09-10T21:43:34.825633+0000","last_undegraded":"2024-09-10T21:43:34.825633+0000","last_fullsized":"2024-09-10T21:43:34.825633+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:52.178433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.963974+0000","last_change":"2024-09-10T21:43:33.827483+0000","last_active":"2024-09-10T21:43:34.963974+0000","last_peered":"2024-09-10T21:43:34.963974+0000","last_clean":"2024-09-10T21:43:34.963974+0000","last_became_active":"2024-09-10T21:43:33.827206+0000","last_became_peered":"2024-09-10T21:43:33.827206+0000","last_unstale":"2024-09-10T21:43:34.963974+0000","last_undegraded":"2024-09-10T21:43:34.963974+0000","last_fullsized":"2024-09-10T21:43:34.963974+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:52.873313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821096+0000","last_change":"2024-09-10T21:43:32.824562+0000","last_active":"2024-09-10T21:43:34.821096+0000","last_peered":"2024-09-10T21:43:34.821096+0000","last_clean":"2024-09-10T21:43:34.821096+0000","last_became_active":"2024-09-10T21:43:32.824235+0000","last_became_peered":"2024-09-10T21:43:32.824235+0000","last_unstale":"2024-09-10T21:43:34.821096+0000","last_undegraded":"2024-09-10T21:43:34.821096+0000","last_fullsized":"2024-09-10T21:43:34.821096+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:13.271679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.820852+0000","last_change":"2024-09-10T21:43:33.836357+0000","last_active":"2024-09-10T21:43:34.820852+0000","last_peered":"2024-09-10T21:43:34.820852+0000","last_clean":"2024-09-10T21:43:34.820852+0000","last_became_active":"2024-09-10T21:43:33.827975+0000","last_became_peered":"2024-09-10T21:43:33.827975+0000","last_unstale":"2024-09-10T21:43:34.820852+0000","last_undegraded":"2024-09-10T21:43:34.820852+0000","last_fullsized":"2024-09-10T21:43:34.820852+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:59.993285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.963907+0000","last_change":"2024-09-10T21:43:32.813754+0000","last_active":"2024-09-10T21:43:34.963907+0000","last_peered":"2024-09-10T21:43:34.963907+0000","last_clean":"2024-09-10T21:43:34.963907+0000","last_became_active":"2024-09-10T21:43:32.813450+0000","last_became_peered":"2024-09-10T21:43:32.813450+0000","last_unstale":"2024-09-10T21:43:34.963907+0000","last_undegraded":"2024-09-10T21:43:34.963907+0000","last_fullsized":"2024-09-10T21:43:34.963907+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:48.244430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964869+0000","last_change":"2024-09-10T21:43:33.841793+0000","last_active":"2024-09-10T21:43:34.964869+0000","last_peered":"2024-09-10T21:43:34.964869+0000","last_clean":"2024-09-10T21:43:34.964869+0000","last_became_active":"2024-09-10T21:43:33.841442+0000","last_became_peered":"2024-09-10T21:43:33.841442+0000","last_unstale":"2024-09-10T21:43:34.964869+0000","last_undegraded":"2024-09-10T21:43:34.964869+0000","last_fullsized":"2024-09-10T21:43:34.964869+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:47.128369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"44'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964585+0000","last_change":"2024-09-10T21:43:32.831569+0000","last_active":"2024-09-10T21:43:34.964585+0000","last_peered":"2024-09-10T21:43:34.964585+0000","last_clean":"2024-09-10T21:43:34.964585+0000","last_became_active":"2024-09-10T21:43:32.831126+0000","last_became_peered":"2024-09-10T21:43:32.831126+0000","last_unstale":"2024-09-10T21:43:34.964585+0000","last_undegraded":"2024-09-10T21:43:34.964585+0000","last_fullsized":"2024-09-10T21:43:34.964585+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:27.926888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821686+0000","last_change":"2024-09-10T21:43:33.825712+0000","last_active":"2024-09-10T21:43:34.821686+0000","last_peered":"2024-09-10T21:43:34.821686+0000","last_clean":"2024-09-10T21:43:34.821686+0000","last_became_active":"2024-09-10T21:43:33.825389+0000","last_became_peered":"2024-09-10T21:43:33.825389+0000","last_unstale":"2024-09-10T21:43:34.821686+0000","last_undegraded":"2024-09-10T21:43:34.821686+0000","last_fullsized":"2024-09-10T21:43:34.821686+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:50.402304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.965143+0000","last_change":"2024-09-10T21:43:32.820317+0000","last_active":"2024-09-10T21:43:34.965143+0000","last_peered":"2024-09-10T21:43:34.965143+0000","last_clean":"2024-09-10T21:43:34.965143+0000","last_became_active":"2024-09-10T21:43:32.819927+0000","last_became_peered":"2024-09-10T21:43:32.819927+0000","last_unstale":"2024-09-10T21:43:34.965143+0000","last_undegraded":"2024-09-10T21:43:34.965143+0000","last_fullsized":"2024-09-10T21:43:34.965143+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:00.262739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964996+0000","last_change":"2024-09-10T21:43:33.831421+0000","last_active":"2024-09-10T21:43:34.964996+0000","last_peered":"2024-09-10T21:43:34.964996+0000","last_clean":"2024-09-10T21:43:34.964996+0000","last_became_active":"2024-09-10T21:43:33.831058+0000","last_became_peered":"2024-09-10T21:43:33.831058+0000","last_unstale":"2024-09-10T21:43:34.964996+0000","last_undegraded":"2024-09-10T21:43:34.964996+0000","last_fullsized":"2024-09-10T21:43:34.964996+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:12.562223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964153+0000","last_change":"2024-09-10T21:43:32.823413+0000","last_active":"2024-09-10T21:43:34.964153+0000","last_peered":"2024-09-10T21:43:34.964153+0000","last_clean":"2024-09-10T21:43:34.964153+0000","last_became_active":"2024-09-10T21:43:32.823092+0000","last_became_peered":"2024-09-10T21:43:32.823092+0000","last_unstale":"2024-09-10T21:43:34.964153+0000","last_undegraded":"2024-09-10T21:43:34.964153+0000","last_fullsized":"2024-09-10T21:43:34.964153+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:16.304972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.965233+0000","last_change":"2024-09-10T21:43:33.830477+0000","last_active":"2024-09-10T21:43:34.965233+0000","last_peered":"2024-09-10T21:43:34.965233+0000","last_clean":"2024-09-10T21:43:34.965233+0000","last_became_active":"2024-09-10T21:43:33.830230+0000","last_became_peered":"2024-09-10T21:43:33.830230+0000","last_unstale":"2024-09-10T21:43:34.965233+0000","last_undegraded":"2024-09-10T21:43:34.965233+0000","last_fullsized":"2024-09-10T21:43:34.965233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:12.102372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964664+0000","last_change":"2024-09-10T21:43:32.823868+0000","last_active":"2024-09-10T21:43:34.964664+0000","last_peered":"2024-09-10T21:43:34.964664+0000","last_clean":"2024-09-10T21:43:34.964664+0000","last_became_active":"2024-09-10T21:43:32.822574+0000","last_became_peered":"2024-09-10T21:43:32.822574+0000","last_unstale":"2024-09-10T21:43:34.964664+0000","last_undegraded":"2024-09-10T21:43:34.964664+0000","last_fullsized":"2024-09-10T21:43:34.964664+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:10.409867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964356+0000","last_change":"2024-09-10T21:43:33.831711+0000","last_active":"2024-09-10T21:43:34.964356+0000","last_peered":"2024-09-10T21:43:34.964356+0000","last_clean":"2024-09-10T21:43:34.964356+0000","last_became_active":"2024-09-10T21:43:33.831419+0000","last_became_peered":"2024-09-10T21:43:33.831419+0000","last_unstale":"2024-09-10T21:43:34.964356+0000","last_undegraded":"2024-09-10T21:43:34.964356+0000","last_fullsized":"2024-09-10T21:43:34.964356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:44.041220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827184+0000","last_change":"2024-09-10T21:43:32.823691+0000","last_active":"2024-09-10T21:43:34.827184+0000","last_peered":"2024-09-10T21:43:34.827184+0000","last_clean":"2024-09-10T21:43:34.827184+0000","last_became_active":"2024-09-10T21:43:32.822876+0000","last_became_peered":"2024-09-10T21:43:32.822876+0000","last_unstale":"2024-09-10T21:43:34.827184+0000","last_undegraded":"2024-09-10T21:43:34.827184+0000","last_fullsized":"2024-09-10T21:43:34.827184+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:59.047826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.825484+0000","last_change":"2024-09-10T21:43:33.832025+0000","last_active":"2024-09-10T21:43:34.825484+0000","last_peered":"2024-09-10T21:43:34.825484+0000","last_clean":"2024-09-10T21:43:34.825484+0000","last_became_active":"2024-09-10T21:43:33.831833+0000","last_became_peered":"2024-09-10T21:43:33.831833+0000","last_unstale":"2024-09-10T21:43:34.825484+0000","last_undegraded":"2024-09-10T21:43:34.825484+0000","last_fullsized":"2024-09-10T21:43:34.825484+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:42.897275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.963839+0000","last_change":"2024-09-10T21:43:32.826447+0000","last_active":"2024-09-10T21:43:34.963839+0000","last_peered":"2024-09-10T21:43:34.963839+0000","last_clean":"2024-09-10T21:43:34.963839+0000","last_became_active":"2024-09-10T21:43:32.825986+0000","last_became_peered":"2024-09-10T21:43:32.825986+0000","last_unstale":"2024-09-10T21:43:34.963839+0000","last_undegraded":"2024-09-10T21:43:34.963839+0000","last_fullsized":"2024-09-10T21:43:34.963839+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:11.222544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827015+0000","last_change":"2024-09-10T21:43:33.817751+0000","last_active":"2024-09-10T21:43:34.827015+0000","last_peered":"2024-09-10T21:43:34.827015+0000","last_clean":"2024-09-10T21:43:34.827015+0000","last_became_active":"2024-09-10T21:43:33.816625+0000","last_became_peered":"2024-09-10T21:43:33.816625+0000","last_unstale":"2024-09-10T21:43:34.827015+0000","last_undegraded":"2024-09-10T21:43:34.827015+0000","last_fullsized":"2024-09-10T21:43:34.827015+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:09:52.004431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821074+0000","last_change":"2024-09-10T21:43:32.815348+0000","last_active":"2024-09-10T21:43:34.821074+0000","last_peered":"2024-09-10T21:43:34.821074+0000","last_clean":"2024-09-10T21:43:34.821074+0000","last_became_active":"2024-09-10T21:43:32.815088+0000","last_became_peered":"2024-09-10T21:43:32.815088+0000","last_unstale":"2024-09-10T21:43:34.821074+0000","last_undegraded":"2024-09-10T21:43:34.821074+0000","last_fullsized":"2024-09-10T21:43:34.821074+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:36:06.844596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821777+0000","last_change":"2024-09-10T21:43:33.830830+0000","last_active":"2024-09-10T21:43:34.821777+0000","last_peered":"2024-09-10T21:43:34.821777+0000","last_clean":"2024-09-10T21:43:34.821777+0000","last_became_active":"2024-09-10T21:43:33.830345+0000","last_became_peered":"2024-09-10T21:43:33.830345+0000","last_unstale":"2024-09-10T21:43:34.821777+0000","last_undegraded":"2024-09-10T21:43:34.821777+0000","last_fullsized":"2024-09-10T21:43:34.821777+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:43.414568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.826468+0000","last_change":"2024-09-10T21:43:32.831158+0000","last_active":"2024-09-10T21:43:34.826468+0000","last_peered":"2024-09-10T21:43:34.826468+0000","last_clean":"2024-09-10T21:43:34.826468+0000","last_became_active":"2024-09-10T21:43:32.830920+0000","last_became_peered":"2024-09-10T21:43:32.830920+0000","last_unstale":"2024-09-10T21:43:34.826468+0000","last_undegraded":"2024-09-10T21:43:34.826468+0000","last_fullsized":"2024-09-10T21:43:34.826468+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:07.088072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":151,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964020+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:43:34.964020+0000","last_peered":"2024-09-10T21:43:34.964020+0000","last_clean":"2024-09-10T21:43:34.964020+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:43:34.964020+0000","last_undegraded":"2024-09-10T21:43:34.964020+0000","last_fullsized":"2024-09-10T21:43:34.964020+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509464,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":516096,"data_stored":204558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":203956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":520192,"data_stored":205498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509463,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":524288,"data_stored":205520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":512000,"data_stored":203956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1105920,"data_stored":794288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542166,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1105920,"data_stored":794344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542167,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31696,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340144,"statfs":{"total":96636764160,"available":96604307456,"internally_reserved":0,"allocated":520192,"data_stored":204524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:43:38.074 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:43:38.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.269+0000 7fbf6b400640 1 -- 172.21.15.7:0/3443609397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6415fda0 msgr2=0x7fbf641601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:38.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.269+0000 7fbf6b400640 1 --2- 172.21.15.7:0/3443609397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6415fda0 0x7fbf641601f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c005fd0 tx=0x7fbf5c02a980 comp rx=0 tx=0).stop 2024-09-10T21:43:38.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.269+0000 7fbf6b400640 1 -- 172.21.15.7:0/3443609397 shutdown_connections 2024-09-10T21:43:38.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.269+0000 7fbf6b400640 1 --2- 172.21.15.7:0/3443609397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6415fda0 0x7fbf641601f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.269+0000 7fbf6b400640 1 --2- 172.21.15.7:0/3443609397 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf640cd750 0x7fbf640ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.269+0000 7fbf6b400640 1 -- 172.21.15.7:0/3443609397 >> 172.21.15.7:0/3443609397 conn(0x7fbf640b16d0 msgr2=0x7fbf640b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:38.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.269+0000 7fbf6b400640 1 -- 172.21.15.7:0/3443609397 shutdown_connections 2024-09-10T21:43:38.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.269+0000 7fbf6b400640 1 -- 172.21.15.7:0/3443609397 wait complete. 2024-09-10T21:43:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.271+0000 7fbf6b400640 1 Processor -- start 2024-09-10T21:43:38.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.271+0000 7fbf6b400640 1 -- start start 2024-09-10T21:43:38.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.272+0000 7fbf6b400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf640cd750 0x7fbf6415e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:38.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.272+0000 7fbf6b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6415bc90 0x7fbf64159cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:38.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.272+0000 7fbf6b400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf6415fda0 0x7fbf6415a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:38.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.272+0000 7fbf6b400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6406fe70 con 0x7fbf6415bc90 2024-09-10T21:43:38.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.272+0000 7fbf6b400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbf6406fcf0 con 0x7fbf6415fda0 2024-09-10T21:43:38.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.272+0000 7fbf6b400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fbf6406fff0 con 0x7fbf640cd750 2024-09-10T21:43:38.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.272+0000 7fbf5be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6415bc90 0x7fbf64159cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:38.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.272+0000 7fbf5be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6415bc90 0x7fbf64159cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44078/0 (socket says 172.21.15.7:44078) 2024-09-10T21:43:38.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.272+0000 7fbf5be00640 1 -- 172.21.15.7:0/1660153125 learned_addr learned my addr 172.21.15.7:0/1660153125 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:38.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.272+0000 7fbf69400640 1 --2- 172.21.15.7:0/1660153125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf6415fda0 0x7fbf6415a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:38.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.272+0000 7fbf68a00640 1 --2- 172.21.15.7:0/1660153125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf640cd750 0x7fbf6415e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:38.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.273+0000 7fbf5be00640 1 -- 172.21.15.7:0/1660153125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf640cd750 msgr2=0x7fbf6415e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:38.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.273+0000 7fbf5be00640 1 --2- 172.21.15.7:0/1660153125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf640cd750 0x7fbf6415e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.273+0000 7fbf5be00640 1 -- 172.21.15.7:0/1660153125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf6415fda0 msgr2=0x7fbf6415a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:38.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.273+0000 7fbf5be00640 1 --2- 172.21.15.7:0/1660153125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf6415fda0 0x7fbf6415a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.273+0000 7fbf5be00640 1 -- 172.21.15.7:0/1660153125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf641f4780 con 0x7fbf6415bc90 2024-09-10T21:43:38.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.273+0000 7fbf68a00640 1 --2- 172.21.15.7:0/1660153125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf640cd750 0x7fbf6415e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:38.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.273+0000 7fbf5be00640 1 --2- 172.21.15.7:0/1660153125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6415bc90 0x7fbf64159cb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00d950 tx=0x7fbf4c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:38.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.273+0000 7fbf61600640 1 -- 172.21.15.7:0/1660153125 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c017070 con 0x7fbf6415bc90 2024-09-10T21:43:38.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.273+0000 7fbf61600640 1 -- 172.21.15.7:0/1660153125 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf4c009d20 con 0x7fbf6415bc90 2024-09-10T21:43:38.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.273+0000 7fbf6b400640 1 -- 172.21.15.7:0/1660153125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf640c6c50 con 0x7fbf6415bc90 2024-09-10T21:43:38.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.273+0000 7fbf61600640 1 -- 172.21.15.7:0/1660153125 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4c0106c0 con 0x7fbf6415bc90 2024-09-10T21:43:38.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.273+0000 7fbf6b400640 1 -- 172.21.15.7:0/1660153125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf641f4fe0 con 0x7fbf6415bc90 2024-09-10T21:43:38.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.274+0000 7fbf6b400640 1 -- 172.21.15.7:0/1660153125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf641f4b30 con 0x7fbf6415bc90 2024-09-10T21:43:38.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.275+0000 7fbf61600640 1 -- 172.21.15.7:0/1660153125 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fbf4c010860 con 0x7fbf6415bc90 2024-09-10T21:43:38.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.276+0000 7fbf61600640 1 --2- 172.21.15.7:0/1660153125 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf40077f70 0x7fbf4007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:38.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.277+0000 7fbf61600640 1 -- 172.21.15.7:0/1660153125 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fbf4c09a1e0 con 0x7fbf6415bc90 2024-09-10T21:43:38.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.277+0000 7fbf68a00640 1 --2- 172.21.15.7:0/1660153125 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf40077f70 0x7fbf4007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:38.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.278+0000 7fbf68a00640 1 --2- 172.21.15.7:0/1660153125 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf40077f70 0x7fbf4007a430 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbf54007fd0 tx=0x7fbf54009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:38.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.281+0000 7fbf61600640 1 -- 172.21.15.7:0/1660153125 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fbf4c0625f0 con 0x7fbf6415bc90 2024-09-10T21:43:38.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.470+0000 7fbf6b400640 1 -- 172.21.15.7:0/1660153125 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf64070510 con 0x7fbf40077f70 2024-09-10T21:43:38.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.475+0000 7fbf61600640 1 -- 172.21.15.7:0/1660153125 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251948 (secure 0 0 0) 0x7fbf64070510 con 0x7fbf40077f70 2024-09-10T21:43:38.479 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:38.481 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:43:38.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.479+0000 7fbf6b400640 1 -- 172.21.15.7:0/1660153125 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf40077f70 msgr2=0x7fbf4007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:38.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.479+0000 7fbf6b400640 1 --2- 172.21.15.7:0/1660153125 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf40077f70 0x7fbf4007a430 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbf54007fd0 tx=0x7fbf54009040 comp rx=0 tx=0).stop 2024-09-10T21:43:38.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.479+0000 7fbf6b400640 1 -- 172.21.15.7:0/1660153125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6415bc90 msgr2=0x7fbf64159cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:38.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.479+0000 7fbf6b400640 1 --2- 172.21.15.7:0/1660153125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6415bc90 0x7fbf64159cb0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00d950 tx=0x7fbf4c00de10 comp rx=0 tx=0).stop 2024-09-10T21:43:38.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.479+0000 7fbf6b400640 1 -- 172.21.15.7:0/1660153125 shutdown_connections 2024-09-10T21:43:38.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.479+0000 7fbf6b400640 1 --2- 172.21.15.7:0/1660153125 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf40077f70 0x7fbf4007a430 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.479+0000 7fbf6b400640 1 --2- 172.21.15.7:0/1660153125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf6415fda0 0x7fbf6415a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.479+0000 7fbf6b400640 1 --2- 172.21.15.7:0/1660153125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf6415bc90 0x7fbf64159cb0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.479+0000 7fbf6b400640 1 --2- 172.21.15.7:0/1660153125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf640cd750 0x7fbf6415e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.479+0000 7fbf6b400640 1 -- 172.21.15.7:0/1660153125 >> 172.21.15.7:0/1660153125 conn(0x7fbf640b16d0 msgr2=0x7fbf640c6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:38.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.479+0000 7fbf6b400640 1 -- 172.21.15.7:0/1660153125 shutdown_connections 2024-09-10T21:43:38.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.479+0000 7fbf6b400640 1 -- 172.21.15.7:0/1660153125 wait complete. 2024-09-10T21:43:38.496 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":88,"stamp":"2024-09-10T21:43:37.876455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":242244,"kb_used_data":5192,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754732476,"statfs":{"total":773094113280,"available":772846055424,"internally_reserved":0,"allocated":5316608,"data_stored":2816644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001568"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964362+0000","last_change":"2024-09-10T21:43:33.831659+0000","last_active":"2024-09-10T21:43:34.964362+0000","last_peered":"2024-09-10T21:43:34.964362+0000","last_clean":"2024-09-10T21:43:34.964362+0000","last_became_active":"2024-09-10T21:43:33.831421+0000","last_became_peered":"2024-09-10T21:43:33.831421+0000","last_unstale":"2024-09-10T21:43:34.964362+0000","last_undegraded":"2024-09-10T21:43:34.964362+0000","last_fullsized":"2024-09-10T21:43:34.964362+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:00:08.088562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.826291+0000","last_change":"2024-09-10T21:43:33.843041+0000","last_active":"2024-09-10T21:43:34.826291+0000","last_peered":"2024-09-10T21:43:34.826291+0000","last_clean":"2024-09-10T21:43:34.826291+0000","last_became_active":"2024-09-10T21:43:33.842756+0000","last_became_peered":"2024-09-10T21:43:33.842756+0000","last_unstale":"2024-09-10T21:43:34.826291+0000","last_undegraded":"2024-09-10T21:43:34.826291+0000","last_fullsized":"2024-09-10T21:43:34.826291+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:17.244798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827421+0000","last_change":"2024-09-10T21:43:33.828575+0000","last_active":"2024-09-10T21:43:34.827421+0000","last_peered":"2024-09-10T21:43:34.827421+0000","last_clean":"2024-09-10T21:43:34.827421+0000","last_became_active":"2024-09-10T21:43:33.826890+0000","last_became_peered":"2024-09-10T21:43:33.826890+0000","last_unstale":"2024-09-10T21:43:34.827421+0000","last_undegraded":"2024-09-10T21:43:34.827421+0000","last_fullsized":"2024-09-10T21:43:34.827421+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:36.451245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821590+0000","last_change":"2024-09-10T21:43:33.838502+0000","last_active":"2024-09-10T21:43:34.821590+0000","last_peered":"2024-09-10T21:43:34.821590+0000","last_clean":"2024-09-10T21:43:34.821590+0000","last_became_active":"2024-09-10T21:43:33.838260+0000","last_became_peered":"2024-09-10T21:43:33.838260+0000","last_unstale":"2024-09-10T21:43:34.821590+0000","last_undegraded":"2024-09-10T21:43:34.821590+0000","last_fullsized":"2024-09-10T21:43:34.821590+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:42.897275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.825823+0000","last_change":"2024-09-10T21:43:33.822071+0000","last_active":"2024-09-10T21:43:34.825823+0000","last_peered":"2024-09-10T21:43:34.825823+0000","last_clean":"2024-09-10T21:43:34.825823+0000","last_became_active":"2024-09-10T21:43:33.821761+0000","last_became_peered":"2024-09-10T21:43:33.821761+0000","last_unstale":"2024-09-10T21:43:34.825823+0000","last_undegraded":"2024-09-10T21:43:34.825823+0000","last_fullsized":"2024-09-10T21:43:34.825823+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:50.402304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.965038+0000","last_change":"2024-09-10T21:43:33.837822+0000","last_active":"2024-09-10T21:43:34.965038+0000","last_peered":"2024-09-10T21:43:34.965038+0000","last_clean":"2024-09-10T21:43:34.965038+0000","last_became_active":"2024-09-10T21:43:33.837455+0000","last_became_peered":"2024-09-10T21:43:33.837455+0000","last_unstale":"2024-09-10T21:43:34.965038+0000","last_undegraded":"2024-09-10T21:43:34.965038+0000","last_fullsized":"2024-09-10T21:43:34.965038+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:50:32.392416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964409+0000","last_change":"2024-09-10T21:43:33.832834+0000","last_active":"2024-09-10T21:43:34.964409+0000","last_peered":"2024-09-10T21:43:34.964409+0000","last_clean":"2024-09-10T21:43:34.964409+0000","last_became_active":"2024-09-10T21:43:33.830669+0000","last_became_peered":"2024-09-10T21:43:33.830669+0000","last_unstale":"2024-09-10T21:43:34.964409+0000","last_undegraded":"2024-09-10T21:43:34.964409+0000","last_fullsized":"2024-09-10T21:43:34.964409+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:17.867453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.820769+0000","last_change":"2024-09-10T21:43:33.844562+0000","last_active":"2024-09-10T21:43:34.820769+0000","last_peered":"2024-09-10T21:43:34.820769+0000","last_clean":"2024-09-10T21:43:34.820769+0000","last_became_active":"2024-09-10T21:43:33.843239+0000","last_became_peered":"2024-09-10T21:43:33.843239+0000","last_unstale":"2024-09-10T21:43:34.820769+0000","last_undegraded":"2024-09-10T21:43:34.820769+0000","last_fullsized":"2024-09-10T21:43:34.820769+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:27.064716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821851+0000","last_change":"2024-09-10T21:43:33.841232+0000","last_active":"2024-09-10T21:43:34.821851+0000","last_peered":"2024-09-10T21:43:34.821851+0000","last_clean":"2024-09-10T21:43:34.821851+0000","last_became_active":"2024-09-10T21:43:33.839497+0000","last_became_peered":"2024-09-10T21:43:33.839497+0000","last_unstale":"2024-09-10T21:43:34.821851+0000","last_undegraded":"2024-09-10T21:43:34.821851+0000","last_fullsized":"2024-09-10T21:43:34.821851+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:46:36.124434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821744+0000","last_change":"2024-09-10T21:43:33.830208+0000","last_active":"2024-09-10T21:43:34.821744+0000","last_peered":"2024-09-10T21:43:34.821744+0000","last_clean":"2024-09-10T21:43:34.821744+0000","last_became_active":"2024-09-10T21:43:33.829759+0000","last_became_peered":"2024-09-10T21:43:33.829759+0000","last_unstale":"2024-09-10T21:43:34.821744+0000","last_undegraded":"2024-09-10T21:43:34.821744+0000","last_fullsized":"2024-09-10T21:43:34.821744+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:35.977807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821888+0000","last_change":"2024-09-10T21:43:33.843016+0000","last_active":"2024-09-10T21:43:34.821888+0000","last_peered":"2024-09-10T21:43:34.821888+0000","last_clean":"2024-09-10T21:43:34.821888+0000","last_became_active":"2024-09-10T21:43:33.839506+0000","last_became_peered":"2024-09-10T21:43:33.839506+0000","last_unstale":"2024-09-10T21:43:34.821888+0000","last_undegraded":"2024-09-10T21:43:34.821888+0000","last_fullsized":"2024-09-10T21:43:34.821888+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:17.244798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.825938+0000","last_change":"2024-09-10T21:43:33.826015+0000","last_active":"2024-09-10T21:43:34.825938+0000","last_peered":"2024-09-10T21:43:34.825938+0000","last_clean":"2024-09-10T21:43:34.825938+0000","last_became_active":"2024-09-10T21:43:33.825736+0000","last_became_peered":"2024-09-10T21:43:33.825736+0000","last_unstale":"2024-09-10T21:43:34.825938+0000","last_undegraded":"2024-09-10T21:43:34.825938+0000","last_fullsized":"2024-09-10T21:43:34.825938+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:35.977807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.963744+0000","last_change":"2024-09-10T21:43:33.824413+0000","last_active":"2024-09-10T21:43:34.963744+0000","last_peered":"2024-09-10T21:43:34.963744+0000","last_clean":"2024-09-10T21:43:34.963744+0000","last_became_active":"2024-09-10T21:43:33.824135+0000","last_became_peered":"2024-09-10T21:43:33.824135+0000","last_unstale":"2024-09-10T21:43:34.963744+0000","last_undegraded":"2024-09-10T21:43:34.963744+0000","last_fullsized":"2024-09-10T21:43:34.963744+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:05.580930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.820932+0000","last_change":"2024-09-10T21:43:33.840904+0000","last_active":"2024-09-10T21:43:34.820932+0000","last_peered":"2024-09-10T21:43:34.820932+0000","last_clean":"2024-09-10T21:43:34.820932+0000","last_became_active":"2024-09-10T21:43:33.839517+0000","last_became_peered":"2024-09-10T21:43:33.839517+0000","last_unstale":"2024-09-10T21:43:34.820932+0000","last_undegraded":"2024-09-10T21:43:34.820932+0000","last_fullsized":"2024-09-10T21:43:34.820932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:49:15.889480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964561+0000","last_change":"2024-09-10T21:43:33.829606+0000","last_active":"2024-09-10T21:43:34.964561+0000","last_peered":"2024-09-10T21:43:34.964561+0000","last_clean":"2024-09-10T21:43:34.964561+0000","last_became_active":"2024-09-10T21:43:33.828462+0000","last_became_peered":"2024-09-10T21:43:33.828462+0000","last_unstale":"2024-09-10T21:43:34.964561+0000","last_undegraded":"2024-09-10T21:43:34.964561+0000","last_fullsized":"2024-09-10T21:43:34.964561+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:00.271285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964100+0000","last_change":"2024-09-10T21:43:33.843027+0000","last_active":"2024-09-10T21:43:34.964100+0000","last_peered":"2024-09-10T21:43:34.964100+0000","last_clean":"2024-09-10T21:43:34.964100+0000","last_became_active":"2024-09-10T21:43:33.842688+0000","last_became_peered":"2024-09-10T21:43:33.842688+0000","last_unstale":"2024-09-10T21:43:34.964100+0000","last_undegraded":"2024-09-10T21:43:34.964100+0000","last_fullsized":"2024-09-10T21:43:34.964100+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:14:00.055126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827869+0000","last_change":"2024-09-10T21:43:33.832401+0000","last_active":"2024-09-10T21:43:34.827869+0000","last_peered":"2024-09-10T21:43:34.827869+0000","last_clean":"2024-09-10T21:43:34.827869+0000","last_became_active":"2024-09-10T21:43:33.830076+0000","last_became_peered":"2024-09-10T21:43:33.830076+0000","last_unstale":"2024-09-10T21:43:34.827869+0000","last_undegraded":"2024-09-10T21:43:34.827869+0000","last_fullsized":"2024-09-10T21:43:34.827869+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:55:48.788649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827135+0000","last_change":"2024-09-10T21:43:33.843130+0000","last_active":"2024-09-10T21:43:34.827135+0000","last_peered":"2024-09-10T21:43:34.827135+0000","last_clean":"2024-09-10T21:43:34.827135+0000","last_became_active":"2024-09-10T21:43:33.842837+0000","last_became_peered":"2024-09-10T21:43:33.842837+0000","last_unstale":"2024-09-10T21:43:34.827135+0000","last_undegraded":"2024-09-10T21:43:34.827135+0000","last_fullsized":"2024-09-10T21:43:34.827135+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:35.358907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964359+0000","last_change":"2024-09-10T21:43:33.837854+0000","last_active":"2024-09-10T21:43:34.964359+0000","last_peered":"2024-09-10T21:43:34.964359+0000","last_clean":"2024-09-10T21:43:34.964359+0000","last_became_active":"2024-09-10T21:43:33.837587+0000","last_became_peered":"2024-09-10T21:43:33.837587+0000","last_unstale":"2024-09-10T21:43:34.964359+0000","last_undegraded":"2024-09-10T21:43:34.964359+0000","last_fullsized":"2024-09-10T21:43:34.964359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:12.229844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.963914+0000","last_change":"2024-09-10T21:43:33.824493+0000","last_active":"2024-09-10T21:43:34.963914+0000","last_peered":"2024-09-10T21:43:34.963914+0000","last_clean":"2024-09-10T21:43:34.963914+0000","last_became_active":"2024-09-10T21:43:33.822764+0000","last_became_peered":"2024-09-10T21:43:33.822764+0000","last_unstale":"2024-09-10T21:43:34.963914+0000","last_undegraded":"2024-09-10T21:43:34.963914+0000","last_fullsized":"2024-09-10T21:43:34.963914+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:40.279210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964149+0000","last_change":"2024-09-10T21:43:33.828818+0000","last_active":"2024-09-10T21:43:34.964149+0000","last_peered":"2024-09-10T21:43:34.964149+0000","last_clean":"2024-09-10T21:43:34.964149+0000","last_became_active":"2024-09-10T21:43:33.828499+0000","last_became_peered":"2024-09-10T21:43:33.828499+0000","last_unstale":"2024-09-10T21:43:34.964149+0000","last_undegraded":"2024-09-10T21:43:34.964149+0000","last_fullsized":"2024-09-10T21:43:34.964149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:19:58.334154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821138+0000","last_change":"2024-09-10T21:43:33.836218+0000","last_active":"2024-09-10T21:43:34.821138+0000","last_peered":"2024-09-10T21:43:34.821138+0000","last_clean":"2024-09-10T21:43:34.821138+0000","last_became_active":"2024-09-10T21:43:33.829991+0000","last_became_peered":"2024-09-10T21:43:33.829991+0000","last_unstale":"2024-09-10T21:43:34.821138+0000","last_undegraded":"2024-09-10T21:43:34.821138+0000","last_fullsized":"2024-09-10T21:43:34.821138+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:21.399612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964869+0000","last_change":"2024-09-10T21:43:33.835908+0000","last_active":"2024-09-10T21:43:34.964869+0000","last_peered":"2024-09-10T21:43:34.964869+0000","last_clean":"2024-09-10T21:43:34.964869+0000","last_became_active":"2024-09-10T21:43:33.835629+0000","last_became_peered":"2024-09-10T21:43:33.835629+0000","last_unstale":"2024-09-10T21:43:34.964869+0000","last_undegraded":"2024-09-10T21:43:34.964869+0000","last_fullsized":"2024-09-10T21:43:34.964869+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:39.640769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.965050+0000","last_change":"2024-09-10T21:43:33.836915+0000","last_active":"2024-09-10T21:43:34.965050+0000","last_peered":"2024-09-10T21:43:34.965050+0000","last_clean":"2024-09-10T21:43:34.965050+0000","last_became_active":"2024-09-10T21:43:33.836660+0000","last_became_peered":"2024-09-10T21:43:33.836660+0000","last_unstale":"2024-09-10T21:43:34.965050+0000","last_undegraded":"2024-09-10T21:43:34.965050+0000","last_fullsized":"2024-09-10T21:43:34.965050+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:55.896275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964723+0000","last_change":"2024-09-10T21:43:33.829375+0000","last_active":"2024-09-10T21:43:34.964723+0000","last_peered":"2024-09-10T21:43:34.964723+0000","last_clean":"2024-09-10T21:43:34.964723+0000","last_became_active":"2024-09-10T21:43:33.829136+0000","last_became_peered":"2024-09-10T21:43:33.829136+0000","last_unstale":"2024-09-10T21:43:34.964723+0000","last_undegraded":"2024-09-10T21:43:34.964723+0000","last_fullsized":"2024-09-10T21:43:34.964723+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:18.430684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964206+0000","last_change":"2024-09-10T21:43:33.822204+0000","last_active":"2024-09-10T21:43:34.964206+0000","last_peered":"2024-09-10T21:43:34.964206+0000","last_clean":"2024-09-10T21:43:34.964206+0000","last_became_active":"2024-09-10T21:43:33.821839+0000","last_became_peered":"2024-09-10T21:43:33.821839+0000","last_unstale":"2024-09-10T21:43:34.964206+0000","last_undegraded":"2024-09-10T21:43:34.964206+0000","last_fullsized":"2024-09-10T21:43:34.964206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:27.707468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827503+0000","last_change":"2024-09-10T21:43:33.830564+0000","last_active":"2024-09-10T21:43:34.827503+0000","last_peered":"2024-09-10T21:43:34.827503+0000","last_clean":"2024-09-10T21:43:34.827503+0000","last_became_active":"2024-09-10T21:43:33.830317+0000","last_became_peered":"2024-09-10T21:43:33.830317+0000","last_unstale":"2024-09-10T21:43:34.827503+0000","last_undegraded":"2024-09-10T21:43:34.827503+0000","last_fullsized":"2024-09-10T21:43:34.827503+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:07.224662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.825957+0000","last_change":"2024-09-10T21:43:33.820801+0000","last_active":"2024-09-10T21:43:34.825957+0000","last_peered":"2024-09-10T21:43:34.825957+0000","last_clean":"2024-09-10T21:43:34.825957+0000","last_became_active":"2024-09-10T21:43:33.819618+0000","last_became_peered":"2024-09-10T21:43:33.819618+0000","last_unstale":"2024-09-10T21:43:34.825957+0000","last_undegraded":"2024-09-10T21:43:34.825957+0000","last_fullsized":"2024-09-10T21:43:34.825957+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:43.414568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964969+0000","last_change":"2024-09-10T21:43:33.826710+0000","last_active":"2024-09-10T21:43:34.964969+0000","last_peered":"2024-09-10T21:43:34.964969+0000","last_clean":"2024-09-10T21:43:34.964969+0000","last_became_active":"2024-09-10T21:43:33.826304+0000","last_became_peered":"2024-09-10T21:43:33.826304+0000","last_unstale":"2024-09-10T21:43:34.964969+0000","last_undegraded":"2024-09-10T21:43:34.964969+0000","last_fullsized":"2024-09-10T21:43:34.964969+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:17.312273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.826947+0000","last_change":"2024-09-10T21:43:33.827874+0000","last_active":"2024-09-10T21:43:34.826947+0000","last_peered":"2024-09-10T21:43:34.826947+0000","last_clean":"2024-09-10T21:43:34.826947+0000","last_became_active":"2024-09-10T21:43:33.827192+0000","last_became_peered":"2024-09-10T21:43:33.827192+0000","last_unstale":"2024-09-10T21:43:34.826947+0000","last_undegraded":"2024-09-10T21:43:34.826947+0000","last_fullsized":"2024-09-10T21:43:34.826947+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:25:17.449957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964734+0000","last_change":"2024-09-10T21:43:33.831562+0000","last_active":"2024-09-10T21:43:34.964734+0000","last_peered":"2024-09-10T21:43:34.964734+0000","last_clean":"2024-09-10T21:43:34.964734+0000","last_became_active":"2024-09-10T21:43:33.831280+0000","last_became_peered":"2024-09-10T21:43:33.831280+0000","last_unstale":"2024-09-10T21:43:34.964734+0000","last_undegraded":"2024-09-10T21:43:34.964734+0000","last_fullsized":"2024-09-10T21:43:34.964734+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:07.409474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.963822+0000","last_change":"2024-09-10T21:43:33.816463+0000","last_active":"2024-09-10T21:43:34.963822+0000","last_peered":"2024-09-10T21:43:34.963822+0000","last_clean":"2024-09-10T21:43:34.963822+0000","last_became_active":"2024-09-10T21:43:33.815454+0000","last_became_peered":"2024-09-10T21:43:33.815454+0000","last_unstale":"2024-09-10T21:43:34.963822+0000","last_undegraded":"2024-09-10T21:43:34.963822+0000","last_fullsized":"2024-09-10T21:43:34.963822+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:29.786967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964247+0000","last_change":"2024-09-10T21:43:33.817533+0000","last_active":"2024-09-10T21:43:34.964247+0000","last_peered":"2024-09-10T21:43:34.964247+0000","last_clean":"2024-09-10T21:43:34.964247+0000","last_became_active":"2024-09-10T21:43:33.817259+0000","last_became_peered":"2024-09-10T21:43:33.817259+0000","last_unstale":"2024-09-10T21:43:34.964247+0000","last_undegraded":"2024-09-10T21:43:34.964247+0000","last_fullsized":"2024-09-10T21:43:34.964247+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:30.337797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"43'4","reported_seq":19,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.965415+0000","last_change":"2024-09-10T21:43:32.829030+0000","last_active":"2024-09-10T21:43:34.965415+0000","last_peered":"2024-09-10T21:43:34.965415+0000","last_clean":"2024-09-10T21:43:34.965415+0000","last_became_active":"2024-09-10T21:43:32.825541+0000","last_became_peered":"2024-09-10T21:43:32.825541+0000","last_unstale":"2024-09-10T21:43:34.965415+0000","last_undegraded":"2024-09-10T21:43:34.965415+0000","last_fullsized":"2024-09-10T21:43:34.965415+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:07:02.003048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964741+0000","last_change":"2024-09-10T21:43:33.823433+0000","last_active":"2024-09-10T21:43:34.964741+0000","last_peered":"2024-09-10T21:43:34.964741+0000","last_clean":"2024-09-10T21:43:34.964741+0000","last_became_active":"2024-09-10T21:43:33.823166+0000","last_became_peered":"2024-09-10T21:43:33.823166+0000","last_unstale":"2024-09-10T21:43:34.964741+0000","last_undegraded":"2024-09-10T21:43:34.964741+0000","last_fullsized":"2024-09-10T21:43:34.964741+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:40.031274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827729+0000","last_change":"2024-09-10T21:43:32.815373+0000","last_active":"2024-09-10T21:43:34.827729+0000","last_peered":"2024-09-10T21:43:34.827729+0000","last_clean":"2024-09-10T21:43:34.827729+0000","last_became_active":"2024-09-10T21:43:32.815094+0000","last_became_peered":"2024-09-10T21:43:32.815094+0000","last_unstale":"2024-09-10T21:43:34.827729+0000","last_undegraded":"2024-09-10T21:43:34.827729+0000","last_fullsized":"2024-09-10T21:43:34.827729+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:41.170819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.826187+0000","last_change":"2024-09-10T21:43:33.831504+0000","last_active":"2024-09-10T21:43:34.826187+0000","last_peered":"2024-09-10T21:43:34.826187+0000","last_clean":"2024-09-10T21:43:34.826187+0000","last_became_active":"2024-09-10T21:43:33.831189+0000","last_became_peered":"2024-09-10T21:43:33.831189+0000","last_unstale":"2024-09-10T21:43:34.826187+0000","last_undegraded":"2024-09-10T21:43:34.826187+0000","last_fullsized":"2024-09-10T21:43:34.826187+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:46:36.124434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964731+0000","last_change":"2024-09-10T21:43:32.823634+0000","last_active":"2024-09-10T21:43:34.964731+0000","last_peered":"2024-09-10T21:43:34.964731+0000","last_clean":"2024-09-10T21:43:34.964731+0000","last_became_active":"2024-09-10T21:43:32.823280+0000","last_became_peered":"2024-09-10T21:43:32.823280+0000","last_unstale":"2024-09-10T21:43:34.964731+0000","last_undegraded":"2024-09-10T21:43:34.964731+0000","last_fullsized":"2024-09-10T21:43:34.964731+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:45.959240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964674+0000","last_change":"2024-09-10T21:43:33.822347+0000","last_active":"2024-09-10T21:43:34.964674+0000","last_peered":"2024-09-10T21:43:34.964674+0000","last_clean":"2024-09-10T21:43:34.964674+0000","last_became_active":"2024-09-10T21:43:33.822094+0000","last_became_peered":"2024-09-10T21:43:33.822094+0000","last_unstale":"2024-09-10T21:43:34.964674+0000","last_undegraded":"2024-09-10T21:43:34.964674+0000","last_fullsized":"2024-09-10T21:43:34.964674+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:55.896275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964194+0000","last_change":"2024-09-10T21:43:32.823580+0000","last_active":"2024-09-10T21:43:34.964194+0000","last_peered":"2024-09-10T21:43:34.964194+0000","last_clean":"2024-09-10T21:43:34.964194+0000","last_became_active":"2024-09-10T21:43:32.823332+0000","last_became_peered":"2024-09-10T21:43:32.823332+0000","last_unstale":"2024-09-10T21:43:34.964194+0000","last_undegraded":"2024-09-10T21:43:34.964194+0000","last_fullsized":"2024-09-10T21:43:34.964194+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:47:51.112408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827062+0000","last_change":"2024-09-10T21:43:33.828469+0000","last_active":"2024-09-10T21:43:34.827062+0000","last_peered":"2024-09-10T21:43:34.827062+0000","last_clean":"2024-09-10T21:43:34.827062+0000","last_became_active":"2024-09-10T21:43:33.827687+0000","last_became_peered":"2024-09-10T21:43:33.827687+0000","last_unstale":"2024-09-10T21:43:34.827062+0000","last_undegraded":"2024-09-10T21:43:34.827062+0000","last_fullsized":"2024-09-10T21:43:34.827062+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:37.120232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964416+0000","last_change":"2024-09-10T21:43:32.817120+0000","last_active":"2024-09-10T21:43:34.964416+0000","last_peered":"2024-09-10T21:43:34.964416+0000","last_clean":"2024-09-10T21:43:34.964416+0000","last_became_active":"2024-09-10T21:43:32.816459+0000","last_became_peered":"2024-09-10T21:43:32.816459+0000","last_unstale":"2024-09-10T21:43:34.964416+0000","last_undegraded":"2024-09-10T21:43:34.964416+0000","last_fullsized":"2024-09-10T21:43:34.964416+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:08.385157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821229+0000","last_change":"2024-09-10T21:43:33.831618+0000","last_active":"2024-09-10T21:43:34.821229+0000","last_peered":"2024-09-10T21:43:34.821229+0000","last_clean":"2024-09-10T21:43:34.821229+0000","last_became_active":"2024-09-10T21:43:33.831264+0000","last_became_peered":"2024-09-10T21:43:33.831264+0000","last_unstale":"2024-09-10T21:43:34.821229+0000","last_undegraded":"2024-09-10T21:43:34.821229+0000","last_fullsized":"2024-09-10T21:43:34.821229+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:34.427724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827099+0000","last_change":"2024-09-10T21:43:32.817363+0000","last_active":"2024-09-10T21:43:34.827099+0000","last_peered":"2024-09-10T21:43:34.827099+0000","last_clean":"2024-09-10T21:43:34.827099+0000","last_became_active":"2024-09-10T21:43:32.817101+0000","last_became_peered":"2024-09-10T21:43:32.817101+0000","last_unstale":"2024-09-10T21:43:34.827099+0000","last_undegraded":"2024-09-10T21:43:34.827099+0000","last_fullsized":"2024-09-10T21:43:34.827099+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:50.374958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.820877+0000","last_change":"2024-09-10T21:43:33.842005+0000","last_active":"2024-09-10T21:43:34.820877+0000","last_peered":"2024-09-10T21:43:34.820877+0000","last_clean":"2024-09-10T21:43:34.820877+0000","last_became_active":"2024-09-10T21:43:33.841511+0000","last_became_peered":"2024-09-10T21:43:33.841511+0000","last_unstale":"2024-09-10T21:43:34.820877+0000","last_undegraded":"2024-09-10T21:43:34.820877+0000","last_fullsized":"2024-09-10T21:43:34.820877+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:19.879570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.825531+0000","last_change":"2024-09-10T21:43:32.821084+0000","last_active":"2024-09-10T21:43:34.825531+0000","last_peered":"2024-09-10T21:43:34.825531+0000","last_clean":"2024-09-10T21:43:34.825531+0000","last_became_active":"2024-09-10T21:43:32.820785+0000","last_became_peered":"2024-09-10T21:43:32.820785+0000","last_unstale":"2024-09-10T21:43:34.825531+0000","last_undegraded":"2024-09-10T21:43:34.825531+0000","last_fullsized":"2024-09-10T21:43:34.825531+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:14:17.522305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.965027+0000","last_change":"2024-09-10T21:43:33.835492+0000","last_active":"2024-09-10T21:43:34.965027+0000","last_peered":"2024-09-10T21:43:34.965027+0000","last_clean":"2024-09-10T21:43:34.965027+0000","last_became_active":"2024-09-10T21:43:33.835226+0000","last_became_peered":"2024-09-10T21:43:33.835226+0000","last_unstale":"2024-09-10T21:43:34.965027+0000","last_undegraded":"2024-09-10T21:43:34.965027+0000","last_fullsized":"2024-09-10T21:43:34.965027+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:18.972165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.820940+0000","last_change":"2024-09-10T21:43:32.813119+0000","last_active":"2024-09-10T21:43:34.820940+0000","last_peered":"2024-09-10T21:43:34.820940+0000","last_clean":"2024-09-10T21:43:34.820940+0000","last_became_active":"2024-09-10T21:43:32.812830+0000","last_became_peered":"2024-09-10T21:43:32.812830+0000","last_unstale":"2024-09-10T21:43:34.820940+0000","last_undegraded":"2024-09-10T21:43:34.820940+0000","last_fullsized":"2024-09-10T21:43:34.820940+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:10:54.275836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827851+0000","last_change":"2024-09-10T21:43:33.819851+0000","last_active":"2024-09-10T21:43:34.827851+0000","last_peered":"2024-09-10T21:43:34.827851+0000","last_clean":"2024-09-10T21:43:34.827851+0000","last_became_active":"2024-09-10T21:43:33.819541+0000","last_became_peered":"2024-09-10T21:43:33.819541+0000","last_unstale":"2024-09-10T21:43:34.827851+0000","last_undegraded":"2024-09-10T21:43:34.827851+0000","last_fullsized":"2024-09-10T21:43:34.827851+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:01.398499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827898+0000","last_change":"2024-09-10T21:43:32.819165+0000","last_active":"2024-09-10T21:43:34.827898+0000","last_peered":"2024-09-10T21:43:34.827898+0000","last_clean":"2024-09-10T21:43:34.827898+0000","last_became_active":"2024-09-10T21:43:32.818823+0000","last_became_peered":"2024-09-10T21:43:32.818823+0000","last_unstale":"2024-09-10T21:43:34.827898+0000","last_undegraded":"2024-09-10T21:43:34.827898+0000","last_fullsized":"2024-09-10T21:43:34.827898+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:00.693410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964627+0000","last_change":"2024-09-10T21:43:33.835882+0000","last_active":"2024-09-10T21:43:34.964627+0000","last_peered":"2024-09-10T21:43:34.964627+0000","last_clean":"2024-09-10T21:43:34.964627+0000","last_became_active":"2024-09-10T21:43:33.834725+0000","last_became_peered":"2024-09-10T21:43:33.834725+0000","last_unstale":"2024-09-10T21:43:34.964627+0000","last_undegraded":"2024-09-10T21:43:34.964627+0000","last_fullsized":"2024-09-10T21:43:34.964627+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:00.315371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821913+0000","last_change":"2024-09-10T21:43:32.824178+0000","last_active":"2024-09-10T21:43:34.821913+0000","last_peered":"2024-09-10T21:43:34.821913+0000","last_clean":"2024-09-10T21:43:34.821913+0000","last_became_active":"2024-09-10T21:43:32.823711+0000","last_became_peered":"2024-09-10T21:43:32.823711+0000","last_unstale":"2024-09-10T21:43:34.821913+0000","last_undegraded":"2024-09-10T21:43:34.821913+0000","last_fullsized":"2024-09-10T21:43:34.821913+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:24.084153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964852+0000","last_change":"2024-09-10T21:43:33.832073+0000","last_active":"2024-09-10T21:43:34.964852+0000","last_peered":"2024-09-10T21:43:34.964852+0000","last_clean":"2024-09-10T21:43:34.964852+0000","last_became_active":"2024-09-10T21:43:33.831889+0000","last_became_peered":"2024-09-10T21:43:33.831889+0000","last_unstale":"2024-09-10T21:43:34.964852+0000","last_undegraded":"2024-09-10T21:43:34.964852+0000","last_fullsized":"2024-09-10T21:43:34.964852+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:47.971631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"43'3","reported_seq":18,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.965058+0000","last_change":"2024-09-10T21:43:32.819667+0000","last_active":"2024-09-10T21:43:34.965058+0000","last_peered":"2024-09-10T21:43:34.965058+0000","last_clean":"2024-09-10T21:43:34.965058+0000","last_became_active":"2024-09-10T21:43:32.818260+0000","last_became_peered":"2024-09-10T21:43:32.818260+0000","last_unstale":"2024-09-10T21:43:34.965058+0000","last_undegraded":"2024-09-10T21:43:34.965058+0000","last_fullsized":"2024-09-10T21:43:34.965058+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:08.385157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964793+0000","last_change":"2024-09-10T21:43:33.831800+0000","last_active":"2024-09-10T21:43:34.964793+0000","last_peered":"2024-09-10T21:43:34.964793+0000","last_clean":"2024-09-10T21:43:34.964793+0000","last_became_active":"2024-09-10T21:43:33.831451+0000","last_became_peered":"2024-09-10T21:43:33.831451+0000","last_unstale":"2024-09-10T21:43:34.964793+0000","last_undegraded":"2024-09-10T21:43:34.964793+0000","last_fullsized":"2024-09-10T21:43:34.964793+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:47:55.976861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"43'6","reported_seq":21,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.963972+0000","last_change":"2024-09-10T21:43:32.814419+0000","last_active":"2024-09-10T21:43:34.963972+0000","last_peered":"2024-09-10T21:43:34.963972+0000","last_clean":"2024-09-10T21:43:34.963972+0000","last_became_active":"2024-09-10T21:43:32.814065+0000","last_became_peered":"2024-09-10T21:43:32.814065+0000","last_unstale":"2024-09-10T21:43:34.963972+0000","last_undegraded":"2024-09-10T21:43:34.963972+0000","last_fullsized":"2024-09-10T21:43:34.963972+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:27.883432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964000+0000","last_change":"2024-09-10T21:43:33.831862+0000","last_active":"2024-09-10T21:43:34.964000+0000","last_peered":"2024-09-10T21:43:34.964000+0000","last_clean":"2024-09-10T21:43:34.964000+0000","last_became_active":"2024-09-10T21:43:33.831478+0000","last_became_peered":"2024-09-10T21:43:33.831478+0000","last_unstale":"2024-09-10T21:43:34.964000+0000","last_undegraded":"2024-09-10T21:43:34.964000+0000","last_fullsized":"2024-09-10T21:43:34.964000+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:12:42.178119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964521+0000","last_change":"2024-09-10T21:43:32.817172+0000","last_active":"2024-09-10T21:43:34.964521+0000","last_peered":"2024-09-10T21:43:34.964521+0000","last_clean":"2024-09-10T21:43:34.964521+0000","last_became_active":"2024-09-10T21:43:32.816835+0000","last_became_peered":"2024-09-10T21:43:32.816835+0000","last_unstale":"2024-09-10T21:43:34.964521+0000","last_undegraded":"2024-09-10T21:43:34.964521+0000","last_fullsized":"2024-09-10T21:43:34.964521+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:37:03.941341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821330+0000","last_change":"2024-09-10T21:43:33.831474+0000","last_active":"2024-09-10T21:43:34.821330+0000","last_peered":"2024-09-10T21:43:34.821330+0000","last_clean":"2024-09-10T21:43:34.821330+0000","last_became_active":"2024-09-10T21:43:33.831188+0000","last_became_peered":"2024-09-10T21:43:33.831188+0000","last_unstale":"2024-09-10T21:43:34.821330+0000","last_undegraded":"2024-09-10T21:43:34.821330+0000","last_fullsized":"2024-09-10T21:43:34.821330+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:08.638143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964446+0000","last_change":"2024-09-10T21:43:32.817318+0000","last_active":"2024-09-10T21:43:34.964446+0000","last_peered":"2024-09-10T21:43:34.964446+0000","last_clean":"2024-09-10T21:43:34.964446+0000","last_became_active":"2024-09-10T21:43:32.817011+0000","last_became_peered":"2024-09-10T21:43:32.817011+0000","last_unstale":"2024-09-10T21:43:34.964446+0000","last_undegraded":"2024-09-10T21:43:34.964446+0000","last_fullsized":"2024-09-10T21:43:34.964446+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:48.994627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964344+0000","last_change":"2024-09-10T21:43:33.819355+0000","last_active":"2024-09-10T21:43:34.964344+0000","last_peered":"2024-09-10T21:43:34.964344+0000","last_clean":"2024-09-10T21:43:34.964344+0000","last_became_active":"2024-09-10T21:43:33.818615+0000","last_became_peered":"2024-09-10T21:43:33.818615+0000","last_unstale":"2024-09-10T21:43:34.964344+0000","last_undegraded":"2024-09-10T21:43:34.964344+0000","last_fullsized":"2024-09-10T21:43:34.964344+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:25.929016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964874+0000","last_change":"2024-09-10T21:43:32.823174+0000","last_active":"2024-09-10T21:43:34.964874+0000","last_peered":"2024-09-10T21:43:34.964874+0000","last_clean":"2024-09-10T21:43:34.964874+0000","last_became_active":"2024-09-10T21:43:32.822859+0000","last_became_peered":"2024-09-10T21:43:32.822859+0000","last_unstale":"2024-09-10T21:43:34.964874+0000","last_undegraded":"2024-09-10T21:43:34.964874+0000","last_fullsized":"2024-09-10T21:43:34.964874+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:10.409867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964413+0000","last_change":"2024-09-10T21:43:33.828433+0000","last_active":"2024-09-10T21:43:34.964413+0000","last_peered":"2024-09-10T21:43:34.964413+0000","last_clean":"2024-09-10T21:43:34.964413+0000","last_became_active":"2024-09-10T21:43:33.828120+0000","last_became_peered":"2024-09-10T21:43:33.828120+0000","last_unstale":"2024-09-10T21:43:34.964413+0000","last_undegraded":"2024-09-10T21:43:34.964413+0000","last_fullsized":"2024-09-10T21:43:34.964413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:07:43.855252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"43'8","reported_seq":23,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821046+0000","last_change":"2024-09-10T21:43:32.820418+0000","last_active":"2024-09-10T21:43:34.821046+0000","last_peered":"2024-09-10T21:43:34.821046+0000","last_clean":"2024-09-10T21:43:34.821046+0000","last_became_active":"2024-09-10T21:43:32.820072+0000","last_became_peered":"2024-09-10T21:43:32.820072+0000","last_unstale":"2024-09-10T21:43:34.821046+0000","last_undegraded":"2024-09-10T21:43:34.821046+0000","last_fullsized":"2024-09-10T21:43:34.821046+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:22.429726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964410+0000","last_change":"2024-09-10T21:43:33.823613+0000","last_active":"2024-09-10T21:43:34.964410+0000","last_peered":"2024-09-10T21:43:34.964410+0000","last_clean":"2024-09-10T21:43:34.964410+0000","last_became_active":"2024-09-10T21:43:33.823284+0000","last_became_peered":"2024-09-10T21:43:33.823284+0000","last_unstale":"2024-09-10T21:43:34.964410+0000","last_undegraded":"2024-09-10T21:43:34.964410+0000","last_fullsized":"2024-09-10T21:43:34.964410+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:14.278979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.963676+0000","last_change":"2024-09-10T21:43:32.826242+0000","last_active":"2024-09-10T21:43:34.963676+0000","last_peered":"2024-09-10T21:43:34.963676+0000","last_clean":"2024-09-10T21:43:34.963676+0000","last_became_active":"2024-09-10T21:43:32.825897+0000","last_became_peered":"2024-09-10T21:43:32.825897+0000","last_unstale":"2024-09-10T21:43:34.963676+0000","last_undegraded":"2024-09-10T21:43:34.963676+0000","last_fullsized":"2024-09-10T21:43:34.963676+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:50:38.303165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.822002+0000","last_change":"2024-09-10T21:43:33.821430+0000","last_active":"2024-09-10T21:43:34.822002+0000","last_peered":"2024-09-10T21:43:34.822002+0000","last_clean":"2024-09-10T21:43:34.822002+0000","last_became_active":"2024-09-10T21:43:33.821135+0000","last_became_peered":"2024-09-10T21:43:33.821135+0000","last_unstale":"2024-09-10T21:43:34.822002+0000","last_undegraded":"2024-09-10T21:43:34.822002+0000","last_fullsized":"2024-09-10T21:43:34.822002+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:55.740837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827881+0000","last_change":"2024-09-10T21:43:32.818125+0000","last_active":"2024-09-10T21:43:34.827881+0000","last_peered":"2024-09-10T21:43:34.827881+0000","last_clean":"2024-09-10T21:43:34.827881+0000","last_became_active":"2024-09-10T21:43:32.817854+0000","last_became_peered":"2024-09-10T21:43:32.817854+0000","last_unstale":"2024-09-10T21:43:34.827881+0000","last_undegraded":"2024-09-10T21:43:34.827881+0000","last_fullsized":"2024-09-10T21:43:34.827881+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:39.015707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964395+0000","last_change":"2024-09-10T21:43:33.836217+0000","last_active":"2024-09-10T21:43:34.964395+0000","last_peered":"2024-09-10T21:43:34.964395+0000","last_clean":"2024-09-10T21:43:34.964395+0000","last_became_active":"2024-09-10T21:43:33.834378+0000","last_became_peered":"2024-09-10T21:43:33.834378+0000","last_unstale":"2024-09-10T21:43:34.964395+0000","last_undegraded":"2024-09-10T21:43:34.964395+0000","last_fullsized":"2024-09-10T21:43:34.964395+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:46.966540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"43'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.826108+0000","last_change":"2024-09-10T21:43:32.819913+0000","last_active":"2024-09-10T21:43:34.826108+0000","last_peered":"2024-09-10T21:43:34.826108+0000","last_clean":"2024-09-10T21:43:34.826108+0000","last_became_active":"2024-09-10T21:43:32.819608+0000","last_became_peered":"2024-09-10T21:43:32.819608+0000","last_unstale":"2024-09-10T21:43:34.826108+0000","last_undegraded":"2024-09-10T21:43:34.826108+0000","last_fullsized":"2024-09-10T21:43:34.826108+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:26.057416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964489+0000","last_change":"2024-09-10T21:43:33.820074+0000","last_active":"2024-09-10T21:43:34.964489+0000","last_peered":"2024-09-10T21:43:34.964489+0000","last_clean":"2024-09-10T21:43:34.964489+0000","last_became_active":"2024-09-10T21:43:33.819680+0000","last_became_peered":"2024-09-10T21:43:33.819680+0000","last_unstale":"2024-09-10T21:43:34.964489+0000","last_undegraded":"2024-09-10T21:43:34.964489+0000","last_fullsized":"2024-09-10T21:43:34.964489+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:11:00.459533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964733+0000","last_change":"2024-09-10T21:43:32.828718+0000","last_active":"2024-09-10T21:43:34.964733+0000","last_peered":"2024-09-10T21:43:34.964733+0000","last_clean":"2024-09-10T21:43:34.964733+0000","last_became_active":"2024-09-10T21:43:32.825482+0000","last_became_peered":"2024-09-10T21:43:32.825482+0000","last_unstale":"2024-09-10T21:43:34.964733+0000","last_undegraded":"2024-09-10T21:43:34.964733+0000","last_fullsized":"2024-09-10T21:43:34.964733+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:12.601680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.965136+0000","last_change":"2024-09-10T21:43:33.826535+0000","last_active":"2024-09-10T21:43:34.965136+0000","last_peered":"2024-09-10T21:43:34.965136+0000","last_clean":"2024-09-10T21:43:34.965136+0000","last_became_active":"2024-09-10T21:43:33.822084+0000","last_became_peered":"2024-09-10T21:43:33.822084+0000","last_unstale":"2024-09-10T21:43:34.965136+0000","last_undegraded":"2024-09-10T21:43:34.965136+0000","last_fullsized":"2024-09-10T21:43:34.965136+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:35.773434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964733+0000","last_change":"2024-09-10T21:43:32.816870+0000","last_active":"2024-09-10T21:43:34.964733+0000","last_peered":"2024-09-10T21:43:34.964733+0000","last_clean":"2024-09-10T21:43:34.964733+0000","last_became_active":"2024-09-10T21:43:32.815750+0000","last_became_peered":"2024-09-10T21:43:32.815750+0000","last_unstale":"2024-09-10T21:43:34.964733+0000","last_undegraded":"2024-09-10T21:43:34.964733+0000","last_fullsized":"2024-09-10T21:43:34.964733+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:10.023479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964885+0000","last_change":"2024-09-10T21:43:33.821301+0000","last_active":"2024-09-10T21:43:34.964885+0000","last_peered":"2024-09-10T21:43:34.964885+0000","last_clean":"2024-09-10T21:43:34.964885+0000","last_became_active":"2024-09-10T21:43:33.819882+0000","last_became_peered":"2024-09-10T21:43:33.819882+0000","last_unstale":"2024-09-10T21:43:34.964885+0000","last_undegraded":"2024-09-10T21:43:34.964885+0000","last_fullsized":"2024-09-10T21:43:34.964885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:18.972165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.825633+0000","last_change":"2024-09-10T21:43:32.818173+0000","last_active":"2024-09-10T21:43:34.825633+0000","last_peered":"2024-09-10T21:43:34.825633+0000","last_clean":"2024-09-10T21:43:34.825633+0000","last_became_active":"2024-09-10T21:43:32.817738+0000","last_became_peered":"2024-09-10T21:43:32.817738+0000","last_unstale":"2024-09-10T21:43:34.825633+0000","last_undegraded":"2024-09-10T21:43:34.825633+0000","last_fullsized":"2024-09-10T21:43:34.825633+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:52.178433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.963974+0000","last_change":"2024-09-10T21:43:33.827483+0000","last_active":"2024-09-10T21:43:34.963974+0000","last_peered":"2024-09-10T21:43:34.963974+0000","last_clean":"2024-09-10T21:43:34.963974+0000","last_became_active":"2024-09-10T21:43:33.827206+0000","last_became_peered":"2024-09-10T21:43:33.827206+0000","last_unstale":"2024-09-10T21:43:34.963974+0000","last_undegraded":"2024-09-10T21:43:34.963974+0000","last_fullsized":"2024-09-10T21:43:34.963974+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:31:52.873313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821096+0000","last_change":"2024-09-10T21:43:32.824562+0000","last_active":"2024-09-10T21:43:34.821096+0000","last_peered":"2024-09-10T21:43:34.821096+0000","last_clean":"2024-09-10T21:43:34.821096+0000","last_became_active":"2024-09-10T21:43:32.824235+0000","last_became_peered":"2024-09-10T21:43:32.824235+0000","last_unstale":"2024-09-10T21:43:34.821096+0000","last_undegraded":"2024-09-10T21:43:34.821096+0000","last_fullsized":"2024-09-10T21:43:34.821096+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:13.271679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.820852+0000","last_change":"2024-09-10T21:43:33.836357+0000","last_active":"2024-09-10T21:43:34.820852+0000","last_peered":"2024-09-10T21:43:34.820852+0000","last_clean":"2024-09-10T21:43:34.820852+0000","last_became_active":"2024-09-10T21:43:33.827975+0000","last_became_peered":"2024-09-10T21:43:33.827975+0000","last_unstale":"2024-09-10T21:43:34.820852+0000","last_undegraded":"2024-09-10T21:43:34.820852+0000","last_fullsized":"2024-09-10T21:43:34.820852+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:59.993285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.963907+0000","last_change":"2024-09-10T21:43:32.813754+0000","last_active":"2024-09-10T21:43:34.963907+0000","last_peered":"2024-09-10T21:43:34.963907+0000","last_clean":"2024-09-10T21:43:34.963907+0000","last_became_active":"2024-09-10T21:43:32.813450+0000","last_became_peered":"2024-09-10T21:43:32.813450+0000","last_unstale":"2024-09-10T21:43:34.963907+0000","last_undegraded":"2024-09-10T21:43:34.963907+0000","last_fullsized":"2024-09-10T21:43:34.963907+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:02:48.244430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964869+0000","last_change":"2024-09-10T21:43:33.841793+0000","last_active":"2024-09-10T21:43:34.964869+0000","last_peered":"2024-09-10T21:43:34.964869+0000","last_clean":"2024-09-10T21:43:34.964869+0000","last_became_active":"2024-09-10T21:43:33.841442+0000","last_became_peered":"2024-09-10T21:43:33.841442+0000","last_unstale":"2024-09-10T21:43:34.964869+0000","last_undegraded":"2024-09-10T21:43:34.964869+0000","last_fullsized":"2024-09-10T21:43:34.964869+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:47.128369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"44'1","reported_seq":16,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964585+0000","last_change":"2024-09-10T21:43:32.831569+0000","last_active":"2024-09-10T21:43:34.964585+0000","last_peered":"2024-09-10T21:43:34.964585+0000","last_clean":"2024-09-10T21:43:34.964585+0000","last_became_active":"2024-09-10T21:43:32.831126+0000","last_became_peered":"2024-09-10T21:43:32.831126+0000","last_unstale":"2024-09-10T21:43:34.964585+0000","last_undegraded":"2024-09-10T21:43:34.964585+0000","last_fullsized":"2024-09-10T21:43:34.964585+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:27.926888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821686+0000","last_change":"2024-09-10T21:43:33.825712+0000","last_active":"2024-09-10T21:43:34.821686+0000","last_peered":"2024-09-10T21:43:34.821686+0000","last_clean":"2024-09-10T21:43:34.821686+0000","last_became_active":"2024-09-10T21:43:33.825389+0000","last_became_peered":"2024-09-10T21:43:33.825389+0000","last_unstale":"2024-09-10T21:43:34.821686+0000","last_undegraded":"2024-09-10T21:43:34.821686+0000","last_fullsized":"2024-09-10T21:43:34.821686+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:50.402304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.965143+0000","last_change":"2024-09-10T21:43:32.820317+0000","last_active":"2024-09-10T21:43:34.965143+0000","last_peered":"2024-09-10T21:43:34.965143+0000","last_clean":"2024-09-10T21:43:34.965143+0000","last_became_active":"2024-09-10T21:43:32.819927+0000","last_became_peered":"2024-09-10T21:43:32.819927+0000","last_unstale":"2024-09-10T21:43:34.965143+0000","last_undegraded":"2024-09-10T21:43:34.965143+0000","last_fullsized":"2024-09-10T21:43:34.965143+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:00.262739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964996+0000","last_change":"2024-09-10T21:43:33.831421+0000","last_active":"2024-09-10T21:43:34.964996+0000","last_peered":"2024-09-10T21:43:34.964996+0000","last_clean":"2024-09-10T21:43:34.964996+0000","last_became_active":"2024-09-10T21:43:33.831058+0000","last_became_peered":"2024-09-10T21:43:33.831058+0000","last_unstale":"2024-09-10T21:43:34.964996+0000","last_undegraded":"2024-09-10T21:43:34.964996+0000","last_fullsized":"2024-09-10T21:43:34.964996+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:20:12.562223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964153+0000","last_change":"2024-09-10T21:43:32.823413+0000","last_active":"2024-09-10T21:43:34.964153+0000","last_peered":"2024-09-10T21:43:34.964153+0000","last_clean":"2024-09-10T21:43:34.964153+0000","last_became_active":"2024-09-10T21:43:32.823092+0000","last_became_peered":"2024-09-10T21:43:32.823092+0000","last_unstale":"2024-09-10T21:43:34.964153+0000","last_undegraded":"2024-09-10T21:43:34.964153+0000","last_fullsized":"2024-09-10T21:43:34.964153+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:16.304972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.965233+0000","last_change":"2024-09-10T21:43:33.830477+0000","last_active":"2024-09-10T21:43:34.965233+0000","last_peered":"2024-09-10T21:43:34.965233+0000","last_clean":"2024-09-10T21:43:34.965233+0000","last_became_active":"2024-09-10T21:43:33.830230+0000","last_became_peered":"2024-09-10T21:43:33.830230+0000","last_unstale":"2024-09-10T21:43:34.965233+0000","last_undegraded":"2024-09-10T21:43:34.965233+0000","last_fullsized":"2024-09-10T21:43:34.965233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:54:12.102372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964664+0000","last_change":"2024-09-10T21:43:32.823868+0000","last_active":"2024-09-10T21:43:34.964664+0000","last_peered":"2024-09-10T21:43:34.964664+0000","last_clean":"2024-09-10T21:43:34.964664+0000","last_became_active":"2024-09-10T21:43:32.822574+0000","last_became_peered":"2024-09-10T21:43:32.822574+0000","last_unstale":"2024-09-10T21:43:34.964664+0000","last_undegraded":"2024-09-10T21:43:34.964664+0000","last_fullsized":"2024-09-10T21:43:34.964664+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:10.409867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964356+0000","last_change":"2024-09-10T21:43:33.831711+0000","last_active":"2024-09-10T21:43:34.964356+0000","last_peered":"2024-09-10T21:43:34.964356+0000","last_clean":"2024-09-10T21:43:34.964356+0000","last_became_active":"2024-09-10T21:43:33.831419+0000","last_became_peered":"2024-09-10T21:43:33.831419+0000","last_unstale":"2024-09-10T21:43:34.964356+0000","last_undegraded":"2024-09-10T21:43:34.964356+0000","last_fullsized":"2024-09-10T21:43:34.964356+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:44.041220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827184+0000","last_change":"2024-09-10T21:43:32.823691+0000","last_active":"2024-09-10T21:43:34.827184+0000","last_peered":"2024-09-10T21:43:34.827184+0000","last_clean":"2024-09-10T21:43:34.827184+0000","last_became_active":"2024-09-10T21:43:32.822876+0000","last_became_peered":"2024-09-10T21:43:32.822876+0000","last_unstale":"2024-09-10T21:43:34.827184+0000","last_undegraded":"2024-09-10T21:43:34.827184+0000","last_fullsized":"2024-09-10T21:43:34.827184+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:13:59.047826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.825484+0000","last_change":"2024-09-10T21:43:33.832025+0000","last_active":"2024-09-10T21:43:34.825484+0000","last_peered":"2024-09-10T21:43:34.825484+0000","last_clean":"2024-09-10T21:43:34.825484+0000","last_became_active":"2024-09-10T21:43:33.831833+0000","last_became_peered":"2024-09-10T21:43:33.831833+0000","last_unstale":"2024-09-10T21:43:34.825484+0000","last_undegraded":"2024-09-10T21:43:34.825484+0000","last_fullsized":"2024-09-10T21:43:34.825484+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:19:42.897275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.963839+0000","last_change":"2024-09-10T21:43:32.826447+0000","last_active":"2024-09-10T21:43:34.963839+0000","last_peered":"2024-09-10T21:43:34.963839+0000","last_clean":"2024-09-10T21:43:34.963839+0000","last_became_active":"2024-09-10T21:43:32.825986+0000","last_became_peered":"2024-09-10T21:43:32.825986+0000","last_unstale":"2024-09-10T21:43:34.963839+0000","last_undegraded":"2024-09-10T21:43:34.963839+0000","last_fullsized":"2024-09-10T21:43:34.963839+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:01:11.222544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.827015+0000","last_change":"2024-09-10T21:43:33.817751+0000","last_active":"2024-09-10T21:43:34.827015+0000","last_peered":"2024-09-10T21:43:34.827015+0000","last_clean":"2024-09-10T21:43:34.827015+0000","last_became_active":"2024-09-10T21:43:33.816625+0000","last_became_peered":"2024-09-10T21:43:33.816625+0000","last_unstale":"2024-09-10T21:43:34.827015+0000","last_undegraded":"2024-09-10T21:43:34.827015+0000","last_fullsized":"2024-09-10T21:43:34.827015+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:09:52.004431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":15,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821074+0000","last_change":"2024-09-10T21:43:32.815348+0000","last_active":"2024-09-10T21:43:34.821074+0000","last_peered":"2024-09-10T21:43:34.821074+0000","last_clean":"2024-09-10T21:43:34.821074+0000","last_became_active":"2024-09-10T21:43:32.815088+0000","last_became_peered":"2024-09-10T21:43:32.815088+0000","last_unstale":"2024-09-10T21:43:34.821074+0000","last_undegraded":"2024-09-10T21:43:34.821074+0000","last_fullsized":"2024-09-10T21:43:34.821074+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:36:06.844596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.821777+0000","last_change":"2024-09-10T21:43:33.830830+0000","last_active":"2024-09-10T21:43:34.821777+0000","last_peered":"2024-09-10T21:43:34.821777+0000","last_clean":"2024-09-10T21:43:34.821777+0000","last_became_active":"2024-09-10T21:43:33.830345+0000","last_became_peered":"2024-09-10T21:43:33.830345+0000","last_unstale":"2024-09-10T21:43:34.821777+0000","last_undegraded":"2024-09-10T21:43:34.821777+0000","last_fullsized":"2024-09-10T21:43:34.821777+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":42,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:32.791610+0000","last_clean_scrub_stamp":"2024-09-10T21:43:32.791610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:43.414568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"43'2","reported_seq":17,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.826468+0000","last_change":"2024-09-10T21:43:32.831158+0000","last_active":"2024-09-10T21:43:34.826468+0000","last_peered":"2024-09-10T21:43:34.826468+0000","last_clean":"2024-09-10T21:43:34.826468+0000","last_became_active":"2024-09-10T21:43:32.830920+0000","last_became_peered":"2024-09-10T21:43:32.830920+0000","last_unstale":"2024-09-10T21:43:34.826468+0000","last_undegraded":"2024-09-10T21:43:34.826468+0000","last_fullsized":"2024-09-10T21:43:34.826468+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":41,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:43:31.784310+0000","last_clean_scrub_stamp":"2024-09-10T21:43:31.784310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:52:07.088072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":151,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-09-10T21:43:34.964020+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:43:34.964020+0000","last_peered":"2024-09-10T21:43:34.964020+0000","last_clean":"2024-09-10T21:43:34.964020+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:43:34.964020+0000","last_undegraded":"2024-09-10T21:43:34.964020+0000","last_fullsized":"2024-09-10T21:43:34.964020+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509464,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27540,"kb_used_data":504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344300,"statfs":{"total":96636764160,"available":96608563200,"internally_reserved":0,"allocated":516096,"data_stored":204558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27528,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344312,"statfs":{"total":96636764160,"available":96608575488,"internally_reserved":0,"allocated":512000,"data_stored":203956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509463,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27536,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344304,"statfs":{"total":96636764160,"available":96608567296,"internally_reserved":0,"allocated":520192,"data_stored":205498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509463,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31708,"kb_used_data":512,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340132,"statfs":{"total":96636764160,"available":96604295168,"internally_reserved":0,"allocated":524288,"data_stored":205520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509465,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31688,"kb_used_data":500,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340152,"statfs":{"total":96636764160,"available":96604315648,"internally_reserved":0,"allocated":512000,"data_stored":203956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509463,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32276,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339564,"statfs":{"total":96636764160,"available":96603713536,"internally_reserved":0,"allocated":1105920,"data_stored":794288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542166,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32272,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339568,"statfs":{"total":96636764160,"available":96603717632,"internally_reserved":0,"allocated":1105920,"data_stored":794344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542167,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31696,"kb_used_data":508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340144,"statfs":{"total":96636764160,"available":96604307456,"internally_reserved":0,"allocated":520192,"data_stored":204524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:43:38.499 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:43:38.500 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:43:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.703+0000 7fe4e9800640 1 -- 172.21.15.7:0/1709607878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4e415e160 msgr2=0x7fe4e40cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.703+0000 7fe4e9800640 1 --2- 172.21.15.7:0/1709607878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4e415e160 0x7fe4e40cf170 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fe4e4070a70 tx=0x7fe4d402a5f0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.703+0000 7fe4e9800640 1 -- 172.21.15.7:0/1709607878 shutdown_connections 2024-09-10T21:43:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.703+0000 7fe4e9800640 1 --2- 172.21.15.7:0/1709607878 >> v2:172.21.15.133:3300/0 conn(0x7fe4e406fda0 0x7fe4e40cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.703+0000 7fe4e9800640 1 --2- 172.21.15.7:0/1709607878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4e415e160 0x7fe4e40cf170 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.703+0000 7fe4e9800640 1 -- 172.21.15.7:0/1709607878 >> 172.21.15.7:0/1709607878 conn(0x7fe4e40b16d0 msgr2=0x7fe4e40b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.703+0000 7fe4e9800640 1 -- 172.21.15.7:0/1709607878 shutdown_connections 2024-09-10T21:43:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.703+0000 7fe4e9800640 1 -- 172.21.15.7:0/1709607878 wait complete. 2024-09-10T21:43:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.704+0000 7fe4e9800640 1 Processor -- start 2024-09-10T21:43:38.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.705+0000 7fe4e9800640 1 -- start start 2024-09-10T21:43:38.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.705+0000 7fe4e9800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4e406fda0 0x7fe4e4154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:38.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.705+0000 7fe4e9800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4e4074420 0x7fe4e40b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:38.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.705+0000 7fe4e9800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4e415e160 0x7fe4e40c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:38.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.705+0000 7fe4e9800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e40ce310 con 0x7fe4e415e160 2024-09-10T21:43:38.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.705+0000 7fe4e9800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe4e40ce190 con 0x7fe4e406fda0 2024-09-10T21:43:38.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.705+0000 7fe4e9800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe4e40ce490 con 0x7fe4e4074420 2024-09-10T21:43:38.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.706+0000 7fe4e3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4e415e160 0x7fe4e40c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:38.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.706+0000 7fe4e3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4e415e160 0x7fe4e40c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44098/0 (socket says 172.21.15.7:44098) 2024-09-10T21:43:38.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.706+0000 7fe4e3400640 1 -- 172.21.15.7:0/2087668540 learned_addr learned my addr 172.21.15.7:0/2087668540 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:38.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.706+0000 7fe4e2a00640 1 --2- 172.21.15.7:0/2087668540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4e406fda0 0x7fe4e4154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:38.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.706+0000 7fe4e2000640 1 --2- 172.21.15.7:0/2087668540 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4e4074420 0x7fe4e40b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.706+0000 7fe4e3400640 1 -- 172.21.15.7:0/2087668540 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4e4074420 msgr2=0x7fe4e40b0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.706+0000 7fe4e3400640 1 --2- 172.21.15.7:0/2087668540 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4e4074420 0x7fe4e40b0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.706+0000 7fe4e3400640 1 -- 172.21.15.7:0/2087668540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4e406fda0 msgr2=0x7fe4e4154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.706+0000 7fe4e3400640 1 --2- 172.21.15.7:0/2087668540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4e406fda0 0x7fe4e4154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.706+0000 7fe4e3400640 1 -- 172.21.15.7:0/2087668540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4e41f7f60 con 0x7fe4e415e160 2024-09-10T21:43:38.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.706+0000 7fe4e3400640 1 --2- 172.21.15.7:0/2087668540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4e415e160 0x7fe4e40c8dc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00da70 tx=0x7fe4dc00df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:38.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.707+0000 7fe4daa00640 1 -- 172.21.15.7:0/2087668540 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc017070 con 0x7fe4e415e160 2024-09-10T21:43:38.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.707+0000 7fe4daa00640 1 -- 172.21.15.7:0/2087668540 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4dc003970 con 0x7fe4e415e160 2024-09-10T21:43:38.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.707+0000 7fe4e9800640 1 -- 172.21.15.7:0/2087668540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4e41f8240 con 0x7fe4e415e160 2024-09-10T21:43:38.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.707+0000 7fe4daa00640 1 -- 172.21.15.7:0/2087668540 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc00be60 con 0x7fe4e415e160 2024-09-10T21:43:38.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.707+0000 7fe4e9800640 1 -- 172.21.15.7:0/2087668540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4e41f8800 con 0x7fe4e415e160 2024-09-10T21:43:38.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.708+0000 7fe4e9800640 1 -- 172.21.15.7:0/2087668540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4e40cf2a0 con 0x7fe4e415e160 2024-09-10T21:43:38.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.709+0000 7fe4daa00640 1 -- 172.21.15.7:0/2087668540 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe4dc010460 con 0x7fe4e415e160 2024-09-10T21:43:38.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.710+0000 7fe4daa00640 1 --2- 172.21.15.7:0/2087668540 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe4c4077f70 0x7fe4c407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:38.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.710+0000 7fe4daa00640 1 -- 172.21.15.7:0/2087668540 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7fe4dc09a0f0 con 0x7fe4e415e160 2024-09-10T21:43:38.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.711+0000 7fe4e2a00640 1 --2- 172.21.15.7:0/2087668540 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe4c4077f70 0x7fe4c407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:38.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.711+0000 7fe4e2a00640 1 --2- 172.21.15.7:0/2087668540 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe4c4077f70 0x7fe4c407a430 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4033000 tx=0x7fe4d4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:38.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.715+0000 7fe4daa00640 1 -- 172.21.15.7:0/2087668540 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe4e40cf2a0 con 0x7fe4e415e160 2024-09-10T21:43:38.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.986+0000 7fe4e9800640 1 -- 172.21.15.7:0/2087668540 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe4e41f8a00 con 0x7fe4e415e160 2024-09-10T21:43:38.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.989+0000 7fe4daa00640 1 -- 172.21.15.7:0/2087668540 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v33) ==== 190+0+147 (secure 0 0 0) 0x7fe4e41f8a00 con 0x7fe4e415e160 2024-09-10T21:43:38.990 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-09-10T21:43:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDvvOBm0aqcFhAAtid2bnzjcpaYAbU7g0JcOA== 2024-09-10T21:43:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: caps mds = "allow" 2024-09-10T21:43:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: caps mon = "allow r" 2024-09-10T21:43:38.990 INFO:teuthology.orchestra.run.smithi007.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:43:38.991 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-09-10T21:43:38.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.992+0000 7fe4e9800640 1 -- 172.21.15.7:0/2087668540 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe4c4077f70 msgr2=0x7fe4c407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:38.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.992+0000 7fe4e9800640 1 --2- 172.21.15.7:0/2087668540 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe4c4077f70 0x7fe4c407a430 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe4d4033000 tx=0x7fe4d4004730 comp rx=0 tx=0).stop 2024-09-10T21:43:38.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.992+0000 7fe4e9800640 1 -- 172.21.15.7:0/2087668540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4e415e160 msgr2=0x7fe4e40c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:38.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.992+0000 7fe4e9800640 1 --2- 172.21.15.7:0/2087668540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4e415e160 0x7fe4e40c8dc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc00da70 tx=0x7fe4dc00df30 comp rx=0 tx=0).stop 2024-09-10T21:43:38.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.992+0000 7fe4e9800640 1 -- 172.21.15.7:0/2087668540 shutdown_connections 2024-09-10T21:43:38.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.992+0000 7fe4e9800640 1 --2- 172.21.15.7:0/2087668540 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe4c4077f70 0x7fe4c407a430 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.992+0000 7fe4e9800640 1 --2- 172.21.15.7:0/2087668540 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe4e415e160 0x7fe4e40c8dc0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.992+0000 7fe4e9800640 1 --2- 172.21.15.7:0/2087668540 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe4e4074420 0x7fe4e40b0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.992+0000 7fe4e9800640 1 --2- 172.21.15.7:0/2087668540 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe4e406fda0 0x7fe4e4154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.992+0000 7fe4e9800640 1 -- 172.21.15.7:0/2087668540 >> 172.21.15.7:0/2087668540 conn(0x7fe4e40b16d0 msgr2=0x7fe4e415bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:38.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.992+0000 7fe4e9800640 1 -- 172.21.15.7:0/2087668540 shutdown_connections 2024-09-10T21:43:38.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:38.992+0000 7fe4e9800640 1 -- 172.21.15.7:0/2087668540 wait complete. 2024-09-10T21:43:39.009 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:43:39.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.167+0000 7f52d7400640 1 -- 172.21.15.7:0/1316105546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52d015bc90 msgr2=0x7f52d00ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:39.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.167+0000 7f52d7400640 1 --2- 172.21.15.7:0/1316105546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52d015bc90 0x7f52d00ccef0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009830 tx=0x7f52c402a380 comp rx=0 tx=0).stop 2024-09-10T21:43:39.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.167+0000 7f52d7400640 1 -- 172.21.15.7:0/1316105546 shutdown_connections 2024-09-10T21:43:39.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.167+0000 7f52d7400640 1 --2- 172.21.15.7:0/1316105546 >> v2:172.21.15.133:3300/0 conn(0x7f52d015fda0 0x7f52d01601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:39.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.167+0000 7f52d7400640 1 --2- 172.21.15.7:0/1316105546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52d015bc90 0x7f52d00ccef0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:39.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.167+0000 7f52d7400640 1 -- 172.21.15.7:0/1316105546 >> 172.21.15.7:0/1316105546 conn(0x7f52d00b16d0 msgr2=0x7f52d00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:39.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.167+0000 7f52d7400640 1 -- 172.21.15.7:0/1316105546 shutdown_connections 2024-09-10T21:43:39.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.167+0000 7f52d7400640 1 -- 172.21.15.7:0/1316105546 wait complete. 2024-09-10T21:43:39.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.169+0000 7f52d7400640 1 Processor -- start 2024-09-10T21:43:39.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.169+0000 7f52d7400640 1 -- start start 2024-09-10T21:43:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.170+0000 7f52d7400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52d00cd750 0x7f52d01f85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.170+0000 7f52d7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d015bc90 0x7f52d00c6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.170+0000 7f52d7400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d015fda0 0x7f52d00af280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.170+0000 7f52d7400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f52d006fe70 con 0x7f52d015bc90 2024-09-10T21:43:39.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.170+0000 7f52d7400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f52d006fcf0 con 0x7f52d015fda0 2024-09-10T21:43:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.170+0000 7f52d7400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f52d006fff0 con 0x7f52d00cd750 2024-09-10T21:43:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.170+0000 7f52cfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d015bc90 0x7f52d00c6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.171+0000 7f52cfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d015bc90 0x7f52d00c6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44108/0 (socket says 172.21.15.7:44108) 2024-09-10T21:43:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.171+0000 7f52cfe00640 1 -- 172.21.15.7:0/751562397 learned_addr learned my addr 172.21.15.7:0/751562397 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.171+0000 7f52d4a00640 1 --2- 172.21.15.7:0/751562397 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52d00cd750 0x7f52d01f85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.171+0000 7f52d5400640 1 --2- 172.21.15.7:0/751562397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d015fda0 0x7f52d00af280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.171+0000 7f52cfe00640 1 -- 172.21.15.7:0/751562397 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52d00cd750 msgr2=0x7f52d01f85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.171+0000 7f52cfe00640 1 --2- 172.21.15.7:0/751562397 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52d00cd750 0x7f52d01f85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.171+0000 7f52cfe00640 1 -- 172.21.15.7:0/751562397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d015fda0 msgr2=0x7f52d00af280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.171+0000 7f52cfe00640 1 --2- 172.21.15.7:0/751562397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d015fda0 0x7f52d00af280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.171+0000 7f52cfe00640 1 -- 172.21.15.7:0/751562397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52d00c6540 con 0x7f52d015bc90 2024-09-10T21:43:39.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.171+0000 7f52d4a00640 1 --2- 172.21.15.7:0/751562397 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52d00cd750 0x7f52d01f85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:43:39.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.171+0000 7f52cfe00640 1 --2- 172.21.15.7:0/751562397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d015bc90 0x7f52d00c6000 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00c700 tx=0x7f52bc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:39.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.172+0000 7f52ccc00640 1 -- 172.21.15.7:0/751562397 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc00a840 con 0x7f52d015bc90 2024-09-10T21:43:39.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.172+0000 7f52ccc00640 1 -- 172.21.15.7:0/751562397 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52bc017070 con 0x7f52d015bc90 2024-09-10T21:43:39.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.172+0000 7f52d7400640 1 -- 172.21.15.7:0/751562397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52d00cc110 con 0x7f52d015bc90 2024-09-10T21:43:39.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.172+0000 7f52ccc00640 1 -- 172.21.15.7:0/751562397 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc00ad90 con 0x7f52d015bc90 2024-09-10T21:43:39.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.172+0000 7f52d7400640 1 -- 172.21.15.7:0/751562397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52d01f47b0 con 0x7f52d015bc90 2024-09-10T21:43:39.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.173+0000 7f52d7400640 1 -- 172.21.15.7:0/751562397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52d01f45a0 con 0x7f52d015bc90 2024-09-10T21:43:39.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.174+0000 7f52ccc00640 1 -- 172.21.15.7:0/751562397 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f52bc01c460 con 0x7f52d015bc90 2024-09-10T21:43:39.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.175+0000 7f52ccc00640 1 --2- 172.21.15.7:0/751562397 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f52b0077f70 0x7f52b007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:39.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.175+0000 7f52ccc00640 1 -- 172.21.15.7:0/751562397 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f52bc099c00 con 0x7f52d015bc90 2024-09-10T21:43:39.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.176+0000 7f52d4a00640 1 --2- 172.21.15.7:0/751562397 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f52b0077f70 0x7f52b007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:39.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.176+0000 7f52d4a00640 1 --2- 172.21.15.7:0/751562397 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f52b0077f70 0x7f52b007a430 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f52c4034000 tx=0x7f52c4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:39.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.179+0000 7f52ccc00640 1 -- 172.21.15.7:0/751562397 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f52bc062010 con 0x7f52d015bc90 2024-09-10T21:43:39.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.408+0000 7f52d7400640 1 -- 172.21.15.7:0/751562397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52d0070510 con 0x7f52d015bc90 2024-09-10T21:43:39.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.409+0000 7f52ccc00640 1 -- 172.21.15.7:0/751562397 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5233 (secure 0 0 0) 0x7f52bc066a40 con 0x7f52d015bc90 2024-09-10T21:43:39.410 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:39.410 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"btime":"2024-09-10T21:43:36:831759+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5149,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3851521050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3851521050},{"type":"v1","addr":"172.21.15.133:6836","nonce":3851521050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5289,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2709023497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2709023497},{"type":"v1","addr":"172.21.15.7:6834","nonce":2709023497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/4095876032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":4095876032},{"type":"v1","addr":"172.21.15.7:6835","nonce":4095876032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:43:33.245584+0000","modified":"2024-09-10T21:43:36.266208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234767},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234767":{"gid":234767,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3385628197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3385628197},{"type":"v1","addr":"172.21.15.133:6837","nonce":3385628197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234767,"qdb_cluster":[234767]},"id":3}]} 2024-09-10T21:43:39.410 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-09-10T21:43:39.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.411+0000 7f52d7400640 1 -- 172.21.15.7:0/751562397 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f52b0077f70 msgr2=0x7f52b007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:39.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.411+0000 7f52d7400640 1 --2- 172.21.15.7:0/751562397 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f52b0077f70 0x7f52b007a430 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f52c4034000 tx=0x7f52c4004730 comp rx=0 tx=0).stop 2024-09-10T21:43:39.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.411+0000 7f52d7400640 1 -- 172.21.15.7:0/751562397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d015bc90 msgr2=0x7f52d00c6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:39.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.411+0000 7f52d7400640 1 --2- 172.21.15.7:0/751562397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d015bc90 0x7f52d00c6000 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f52bc00c700 tx=0x7f52bc00cbc0 comp rx=0 tx=0).stop 2024-09-10T21:43:39.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.411+0000 7f52d7400640 1 -- 172.21.15.7:0/751562397 shutdown_connections 2024-09-10T21:43:39.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.411+0000 7f52d7400640 1 --2- 172.21.15.7:0/751562397 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f52b0077f70 0x7f52b007a430 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:39.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.411+0000 7f52d7400640 1 --2- 172.21.15.7:0/751562397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52d015fda0 0x7f52d00af280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:39.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.411+0000 7f52d7400640 1 --2- 172.21.15.7:0/751562397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52d015bc90 0x7f52d00c6000 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:39.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.411+0000 7f52d7400640 1 --2- 172.21.15.7:0/751562397 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52d00cd750 0x7f52d01f85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:39.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.411+0000 7f52d7400640 1 -- 172.21.15.7:0/751562397 >> 172.21.15.7:0/751562397 conn(0x7f52d00b16d0 msgr2=0x7f52d0072360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:39.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.411+0000 7f52d7400640 1 -- 172.21.15.7:0/751562397 shutdown_connections 2024-09-10T21:43:39.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.411+0000 7f52d7400640 1 -- 172.21.15.7:0/751562397 wait complete. 2024-09-10T21:43:39.774 INFO:tasks.ceph.mds.d.smithi133.stderr:2024-09-10T21:43:39.773+0000 7f352ac00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:43:39.831 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:43:33.245584+0000', 'modified': '2024-09-10T21:43:36.266208+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 234767}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_234767': {'gid': 234767, 'name': 'd', 'rank': 0, 'incarnation': 22, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6837/3385628197', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 3385628197}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 3385628197}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 234767, 'qdb_cluster': [234767]} 2024-09-10T21:43:39.831 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:43:39.832 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T21:43:39.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.997+0000 7f6962e00640 1 -- 172.21.15.7:0/1955879892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f695c0cfa60 msgr2=0x7f695c0ccbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.997+0000 7f6962e00640 1 --2- 172.21.15.7:0/1955879892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f695c0cfa60 0x7f695c0ccbb0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6954005fd0 tx=0x7f695402aaa0 comp rx=0 tx=0).stop 2024-09-10T21:43:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.997+0000 7f6962e00640 1 -- 172.21.15.7:0/1955879892 shutdown_connections 2024-09-10T21:43:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.997+0000 7f6962e00640 1 --2- 172.21.15.7:0/1955879892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f695c0cfa60 0x7f695c0ccbb0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.997+0000 7f6962e00640 1 --2- 172.21.15.7:0/1955879892 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f695c0d0650 0x7f695c0cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.997+0000 7f6962e00640 1 -- 172.21.15.7:0/1955879892 >> 172.21.15.7:0/1955879892 conn(0x7f695c0d1b60 msgr2=0x7f695c0d1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.997+0000 7f6962e00640 1 -- 172.21.15.7:0/1955879892 shutdown_connections 2024-09-10T21:43:39.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.997+0000 7f6962e00640 1 -- 172.21.15.7:0/1955879892 wait complete. 2024-09-10T21:43:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.998+0000 7f6962e00640 1 Processor -- start 2024-09-10T21:43:40.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.999+0000 7f6962e00640 1 -- start start 2024-09-10T21:43:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.999+0000 7f6962e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f695c0cfa60 0x7f695c149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.999+0000 7f6962e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f695c0d0650 0x7f695c0b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.999+0000 7f6962e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f695c15f2f0 0x7f695c0b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.999+0000 7f6962e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f695c0cd0f0 con 0x7f695c0d0650 2024-09-10T21:43:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.999+0000 7f6962e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f695c0b23e0 con 0x7f695c15f2f0 2024-09-10T21:43:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:39.999+0000 7f6962e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f695c0cd270 con 0x7f695c0cfa60 2024-09-10T21:43:40.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.000+0000 7f6961000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f695c0d0650 0x7f695c0b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.000+0000 7f6961000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f695c0d0650 0x7f695c0b35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44140/0 (socket says 172.21.15.7:44140) 2024-09-10T21:43:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.000+0000 7f6961000640 1 -- 172.21.15.7:0/3131538290 learned_addr learned my addr 172.21.15.7:0/3131538290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.000+0000 7f6961a00640 1 --2- 172.21.15.7:0/3131538290 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f695c0cfa60 0x7f695c149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.000+0000 7f6962400640 1 --2- 172.21.15.7:0/3131538290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f695c15f2f0 0x7f695c0b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.000+0000 7f6961000640 1 -- 172.21.15.7:0/3131538290 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f695c0cfa60 msgr2=0x7f695c149690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.000+0000 7f6961000640 1 --2- 172.21.15.7:0/3131538290 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f695c0cfa60 0x7f695c149690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.000+0000 7f6961000640 1 -- 172.21.15.7:0/3131538290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f695c15f2f0 msgr2=0x7f695c0b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.000+0000 7f6961000640 1 --2- 172.21.15.7:0/3131538290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f695c15f2f0 0x7f695c0b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.000+0000 7f6961000640 1 -- 172.21.15.7:0/3131538290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695c1f2960 con 0x7f695c0d0650 2024-09-10T21:43:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.001+0000 7f6961000640 1 --2- 172.21.15.7:0/3131538290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f695c0d0650 0x7f695c0b35e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f694800d770 tx=0x7f694800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.001+0000 7f6959600640 1 -- 172.21.15.7:0/3131538290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948017070 con 0x7f695c0d0650 2024-09-10T21:43:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.001+0000 7f6959600640 1 -- 172.21.15.7:0/3131538290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6948009cc0 con 0x7f695c0d0650 2024-09-10T21:43:40.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.001+0000 7f6959600640 1 -- 172.21.15.7:0/3131538290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948005280 con 0x7f695c0d0650 2024-09-10T21:43:40.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.001+0000 7f6962e00640 1 -- 172.21.15.7:0/3131538290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f695c1f2c20 con 0x7f695c0d0650 2024-09-10T21:43:40.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.001+0000 7f6962e00640 1 -- 172.21.15.7:0/3131538290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f695c1f31e0 con 0x7f695c0d0650 2024-09-10T21:43:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.003+0000 7f6959600640 1 -- 172.21.15.7:0/3131538290 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6948005420 con 0x7f695c0d0650 2024-09-10T21:43:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.004+0000 7f6959600640 1 --2- 172.21.15.7:0/3131538290 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f693c077f70 0x7f693c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.004+0000 7f6959600640 1 -- 172.21.15.7:0/3131538290 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f6948099e60 con 0x7f695c0d0650 2024-09-10T21:43:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.004+0000 7f6967259740 1 Processor -- start 2024-09-10T21:43:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.005+0000 7f6967259740 1 -- start start 2024-09-10T21:43:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.005+0000 7f6961a00640 1 --2- 172.21.15.7:0/3131538290 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f693c077f70 0x7f693c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.005+0000 7f6967259740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88434aa0 0x55fd88434e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.005+0000 7f6967259740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fd883ea110 0x55fd88415800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.005+0000 7f6967259740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd883e91f0 0x55fd88415d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.005+0000 7f6967259740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fd88372430 con 0x55fd88434aa0 2024-09-10T21:43:40.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.005+0000 7f6967259740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fd883625d0 con 0x55fd883e91f0 2024-09-10T21:43:40.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.005+0000 7f6967259740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55fd88351a00 con 0x55fd883ea110 2024-09-10T21:43:40.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.006+0000 7f6961a00640 1 --2- 172.21.15.7:0/3131538290 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f693c077f70 0x7f693c07a430 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f695c0c5370 tx=0x7f6950009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:40.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.006+0000 7f6961a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88434aa0 0x55fd88434e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:40.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.006+0000 7f6961a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88434aa0 0x55fd88434e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44146/0 (socket says 172.21.15.7:44146) 2024-09-10T21:43:40.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.006+0000 7f6961a00640 1 -- 172.21.15.7:0/3116308918 learned_addr learned my addr 172.21.15.7:0/3116308918 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:40.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.006+0000 7f6961a00640 1 -- 172.21.15.7:0/3116308918 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fd883ea110 msgr2=0x55fd88415800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:40.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.006+0000 7f6961a00640 1 --2- 172.21.15.7:0/3116308918 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fd883ea110 0x55fd88415800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.006+0000 7f6961a00640 1 -- 172.21.15.7:0/3116308918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd883e91f0 msgr2=0x55fd88415d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:40.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.006+0000 7f6961a00640 1 --2- 172.21.15.7:0/3116308918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd883e91f0 0x55fd88415d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.006+0000 7f6961a00640 1 -- 172.21.15.7:0/3116308918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd8831a520 con 0x55fd88434aa0 2024-09-10T21:43:40.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.006+0000 7f6961a00640 1 --2- 172.21.15.7:0/3116308918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88434aa0 0x55fd88434e80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f695000f930 tx=0x7f6950012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09ee78d9e6fbde2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:40.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.007+0000 7f694e000640 1 -- 172.21.15.7:0/3116308918 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950025070 con 0x55fd88434aa0 2024-09-10T21:43:40.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.007+0000 7f694e000640 1 -- 172.21.15.7:0/3116308918 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f695001b050 con 0x55fd88434aa0 2024-09-10T21:43:40.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.007+0000 7f694e000640 1 -- 172.21.15.7:0/3116308918 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6950014da0 con 0x55fd88434aa0 2024-09-10T21:43:40.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.007+0000 7f6967259740 1 -- 172.21.15.7:0/3116308918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88434aa0 msgr2=0x55fd88434e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:40.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.007+0000 7f6967259740 1 --2- 172.21.15.7:0/3116308918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88434aa0 0x55fd88434e80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f695000f930 tx=0x7f6950012000 comp rx=0 tx=0).stop 2024-09-10T21:43:40.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.007+0000 7f6967259740 1 -- 172.21.15.7:0/3116308918 shutdown_connections 2024-09-10T21:43:40.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.007+0000 7f6967259740 1 --2- 172.21.15.7:0/3116308918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd883e91f0 0x55fd88415d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.007+0000 7f6967259740 1 --2- 172.21.15.7:0/3116308918 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fd883ea110 0x55fd88415800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.007+0000 7f6967259740 1 --2- 172.21.15.7:0/3116308918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88434aa0 0x55fd88434e80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.007+0000 7f6967259740 1 -- 172.21.15.7:0/3116308918 >> 172.21.15.7:0/3116308918 conn(0x55fd883eb1e0 msgr2=0x55fd883eec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.008+0000 7f6967259740 1 -- 172.21.15.7:0/3116308918 shutdown_connections 2024-09-10T21:43:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.008+0000 7f6967259740 1 -- 172.21.15.7:0/3116308918 wait complete. 2024-09-10T21:43:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.008+0000 7f6967259740 1 -- start start 2024-09-10T21:43:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.009+0000 7f6967259740 1 Processor -- start 2024-09-10T21:43:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.009+0000 7f6967259740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fd883ea110 0x55fd884108e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.009+0000 7f6967259740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd88434aa0 0x55fd88433780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.010+0000 7f6967259740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88416430 0x55fd883fce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.010+0000 7f6967259740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fd882f8330 con 0x55fd88416430 2024-09-10T21:43:40.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.010+0000 7f6967259740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fd88355810 con 0x55fd88434aa0 2024-09-10T21:43:40.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.010+0000 7f6967259740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55fd880c2620 con 0x55fd883ea110 2024-09-10T21:43:40.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.010+0000 7f6967259740 10 client.0 _ll_register_callbacks cb 0x55fd882fd420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:43:40.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.010+0000 7f6962400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88416430 0x55fd883fce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:40.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.010+0000 7f6962400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88416430 0x55fd883fce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44158/0 (socket says 172.21.15.7:44158) 2024-09-10T21:43:40.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.010+0000 7f6962400640 1 -- 172.21.15.7:0/1525927902 learned_addr learned my addr 172.21.15.7:0/1525927902 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:40.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.010+0000 7f6961a00640 1 --2- 172.21.15.7:0/1525927902 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fd883ea110 0x55fd884108e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:40.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.010+0000 7f6961000640 1 --2- 172.21.15.7:0/1525927902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd88434aa0 0x55fd88433780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:40.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.010+0000 7f6962400640 1 -- 172.21.15.7:0/1525927902 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fd883ea110 msgr2=0x55fd884108e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:40.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.010+0000 7f6962400640 1 --2- 172.21.15.7:0/1525927902 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fd883ea110 0x55fd884108e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.010+0000 7f6962400640 1 -- 172.21.15.7:0/1525927902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd88434aa0 msgr2=0x55fd88433780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:40.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.010+0000 7f6962400640 1 --2- 172.21.15.7:0/1525927902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd88434aa0 0x55fd88433780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.010+0000 7f6962400640 1 -- 172.21.15.7:0/1525927902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd880b3850 con 0x55fd88416430 2024-09-10T21:43:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.011+0000 7f6962400640 1 --2- 172.21.15.7:0/1525927902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88416430 0x55fd883fce60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6954009d60 tx=0x7f6954003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e21be942a4722eca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.011+0000 7f6942a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:43:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.011+0000 7f6942a00640 1 -- 172.21.15.7:0/1525927902 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695402ebb0 con 0x55fd88416430 2024-09-10T21:43:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.011+0000 7f6942a00640 1 -- 172.21.15.7:0/1525927902 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f695402ed50 con 0x55fd88416430 2024-09-10T21:43:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.011+0000 7f6967259740 1 -- 172.21.15.7:0/1525927902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f695001b050 con 0x55fd88416430 2024-09-10T21:43:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.011+0000 7f6942a00640 1 -- 172.21.15.7:0/1525927902 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f695404d050 con 0x55fd88416430 2024-09-10T21:43:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.011+0000 7f6967259740 10 client.5421 fetch_fsmap learned FSMap version 26 2024-09-10T21:43:40.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.011+0000 7f6967259740 1 -- 172.21.15.7:0/1525927902 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55fd8812a6b0 con 0x55fd88416430 2024-09-10T21:43:40.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.011+0000 7f6942a00640 1 -- 172.21.15.7:0/1525927902 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f695001b050 con 0x55fd88416430 2024-09-10T21:43:40.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.011+0000 7f6942a00640 1 -- 172.21.15.7:0/1525927902 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f6954050020 con 0x55fd88416430 2024-09-10T21:43:40.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.011+0000 7f6967259740 10 client.5421 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T21:43:40.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.011+0000 7f6967259740 10 client.5421 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:43:40.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.011+0000 7f6967259740 20 client.5421 populate_metadata read hostname 'smithi007' 2024-09-10T21:43:40.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.011+0000 7f6967259740 1 --2- 172.21.15.7:0/1525927902 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55fd88418590 0x55fd88418970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.011+0000 7f6967259740 4 client.5421 mds_command: new command op to 234767 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:43:40.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.011+0000 7f6967259740 1 -- 172.21.15.7:0/1525927902 --> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fd88364520 con 0x55fd88418590 2024-09-10T21:43:40.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.012+0000 7f6961a00640 1 --2- 172.21.15.7:0/1525927902 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55fd88418590 0x55fd88418970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:40.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.013+0000 7f6961a00640 1 --2- 172.21.15.7:0/1525927902 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55fd88418590 0x55fd88418970 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1c2fef22ad64b7b server_cookie=89721f7d211c29 in_seq=0 out_seq=0 2024-09-10T21:43:40.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.013+0000 7f6942a00640 10 client.5421 ms_handle_connect on v2:172.21.15.133:6834/3385628197 2024-09-10T21:43:40.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.014+0000 7f6942a00640 1 -- 172.21.15.7:0/1525927902 <== mds.0 v2:172.21.15.133:6834/3385628197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fd88364520 con 0x55fd88418590 2024-09-10T21:43:40.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.014+0000 7f6942a00640 10 client.5421 handle_command_reply: tid=0 2024-09-10T21:43:40.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.014+0000 7f6967259740 1 client.5421 shutdown 2024-09-10T21:43:40.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.014+0000 7f6967259740 1 -- 172.21.15.7:0/1525927902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88416430 msgr2=0x55fd883fce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:40.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.014+0000 7f6967259740 1 --2- 172.21.15.7:0/1525927902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88416430 0x55fd883fce60 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f6954009d60 tx=0x7f6954003e70 comp rx=0 tx=0).stop 2024-09-10T21:43:40.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.014+0000 7f6967259740 1 -- 172.21.15.7:0/1525927902 shutdown_connections 2024-09-10T21:43:40.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.014+0000 7f6967259740 1 --2- 172.21.15.7:0/1525927902 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55fd88418590 0x55fd88418970 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.014+0000 7f6967259740 1 --2- 172.21.15.7:0/1525927902 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd88434aa0 0x55fd88433780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.014+0000 7f6967259740 1 --2- 172.21.15.7:0/1525927902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88416430 0x55fd883fce60 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.014+0000 7f6942a00640 0 client.5421 ms_handle_reset on v2:172.21.15.133:6834/3385628197 2024-09-10T21:43:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.014+0000 7f6967259740 1 --2- 172.21.15.7:0/1525927902 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fd883ea110 0x55fd884108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.014+0000 7f6967259740 1 -- 172.21.15.7:0/1525927902 >> 172.21.15.7:0/1525927902 conn(0x55fd883e91f0 msgr2=0x55fd883ee6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.014+0000 7f6967259740 1 -- 172.21.15.7:0/1525927902 shutdown_connections 2024-09-10T21:43:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.014+0000 7f6967259740 1 -- 172.21.15.7:0/1525927902 wait complete. 2024-09-10T21:43:40.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.014+0000 7f6967259740 20 client.5421 trim_cache size 0 max 16384 2024-09-10T21:43:40.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.033+0000 7f6967259740 1 Processor -- start 2024-09-10T21:43:40.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.033+0000 7f6967259740 1 -- start start 2024-09-10T21:43:40.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.034+0000 7f6967259740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fd883ea110 0x55fd884340b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.034+0000 7f6967259740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd88416430 0x55fd8845bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.034+0000 7f6967259740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88434aa0 0x55fd8845c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.034+0000 7f6967259740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f695c06fbb0 con 0x55fd88434aa0 2024-09-10T21:43:40.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.034+0000 7f6967259740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f695c06ef10 con 0x55fd88416430 2024-09-10T21:43:40.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.034+0000 7f6967259740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f695c06f560 con 0x55fd883ea110 2024-09-10T21:43:40.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.034+0000 7f6962400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88434aa0 0x55fd8845c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:40.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.034+0000 7f6962400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88434aa0 0x55fd8845c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44170/0 (socket says 172.21.15.7:44170) 2024-09-10T21:43:40.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.034+0000 7f6962400640 1 -- 172.21.15.7:0/1606025356 learned_addr learned my addr 172.21.15.7:0/1606025356 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:40.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.035+0000 7f6962400640 1 -- 172.21.15.7:0/1606025356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fd883ea110 msgr2=0x55fd884340b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:40.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.035+0000 7f6962400640 1 --2- 172.21.15.7:0/1606025356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fd883ea110 0x55fd884340b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.035+0000 7f6962400640 1 -- 172.21.15.7:0/1606025356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd88416430 msgr2=0x55fd8845bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:40.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.035+0000 7f6962400640 1 --2- 172.21.15.7:0/1606025356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd88416430 0x55fd8845bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.035+0000 7f6962400640 1 -- 172.21.15.7:0/1606025356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fd88431f60 con 0x55fd88434aa0 2024-09-10T21:43:40.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.035+0000 7f6962400640 1 --2- 172.21.15.7:0/1606025356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88434aa0 0x55fd8845c520 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6954002230 tx=0x7f695402b030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a7c70e495cadfa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:40.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.035+0000 7f694d600640 1 -- 172.21.15.7:0/1606025356 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6954055020 con 0x55fd88434aa0 2024-09-10T21:43:40.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.035+0000 7f694d600640 1 -- 172.21.15.7:0/1606025356 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6954003fe0 con 0x55fd88434aa0 2024-09-10T21:43:40.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.035+0000 7f694d600640 1 -- 172.21.15.7:0/1606025356 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695402ec10 con 0x55fd88434aa0 2024-09-10T21:43:40.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.036+0000 7f6967259740 1 -- 172.21.15.7:0/1606025356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88434aa0 msgr2=0x55fd8845c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:40.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.036+0000 7f6967259740 1 --2- 172.21.15.7:0/1606025356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88434aa0 0x55fd8845c520 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f6954002230 tx=0x7f695402b030 comp rx=0 tx=0).stop 2024-09-10T21:43:40.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.036+0000 7f6967259740 1 -- 172.21.15.7:0/1606025356 shutdown_connections 2024-09-10T21:43:40.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.036+0000 7f6967259740 1 --2- 172.21.15.7:0/1606025356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd88416430 0x55fd8845bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.036+0000 7f6967259740 1 --2- 172.21.15.7:0/1606025356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88434aa0 0x55fd8845c520 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.036+0000 7f6967259740 1 --2- 172.21.15.7:0/1606025356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fd883ea110 0x55fd884340b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.036+0000 7f6967259740 1 -- 172.21.15.7:0/1606025356 >> 172.21.15.7:0/1606025356 conn(0x55fd883e91f0 msgr2=0x55fd8842d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:40.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.036+0000 7f6967259740 1 -- 172.21.15.7:0/1606025356 shutdown_connections 2024-09-10T21:43:40.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.036+0000 7f6967259740 1 -- 172.21.15.7:0/1606025356 wait complete. 2024-09-10T21:43:40.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.037+0000 7f6967259740 1 -- start start 2024-09-10T21:43:40.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.037+0000 7f6967259740 1 Processor -- start 2024-09-10T21:43:40.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.038+0000 7f6967259740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fd883ea110 0x55fd8844a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.038+0000 7f6967259740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88416430 0x55fd88411d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.038+0000 7f6967259740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd88434aa0 0x55fd88412290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.038+0000 7f6967259740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fd88411130 con 0x55fd88416430 2024-09-10T21:43:40.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.038+0000 7f6967259740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fd884195f0 con 0x55fd88434aa0 2024-09-10T21:43:40.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.038+0000 7f6967259740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55fd883ff3c0 con 0x55fd883ea110 2024-09-10T21:43:40.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.038+0000 7f6967259740 10 client.0 _ll_register_callbacks cb 0x7f6954003d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:43:40.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.038+0000 7f6961000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88416430 0x55fd88411d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:40.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.038+0000 7f6961000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88416430 0x55fd88411d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44180/0 (socket says 172.21.15.7:44180) 2024-09-10T21:43:40.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.038+0000 7f6961000640 1 -- 172.21.15.7:0/611035744 learned_addr learned my addr 172.21.15.7:0/611035744 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:40.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.038+0000 7f6961a00640 1 --2- 172.21.15.7:0/611035744 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fd883ea110 0x55fd8844a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:40.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.038+0000 7f6962400640 1 --2- 172.21.15.7:0/611035744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd88434aa0 0x55fd88412290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:40.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.038+0000 7f6961000640 1 -- 172.21.15.7:0/611035744 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fd883ea110 msgr2=0x55fd8844a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:40.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.038+0000 7f6961000640 1 --2- 172.21.15.7:0/611035744 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fd883ea110 0x55fd8844a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.038+0000 7f6961000640 1 -- 172.21.15.7:0/611035744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd88434aa0 msgr2=0x55fd88412290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:40.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.038+0000 7f6961000640 1 --2- 172.21.15.7:0/611035744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd88434aa0 0x55fd88412290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.038+0000 7f6961000640 1 -- 172.21.15.7:0/611035744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fd883e9ea0 con 0x55fd88416430 2024-09-10T21:43:40.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.038+0000 7f6961a00640 1 --2- 172.21.15.7:0/611035744 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fd883ea110 0x55fd8844a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:40.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.038+0000 7f6962400640 1 --2- 172.21.15.7:0/611035744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd88434aa0 0x55fd88412290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:40.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.039+0000 7f6961000640 1 --2- 172.21.15.7:0/611035744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88416430 0x55fd88411d50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f694805f570 tx=0x7f6948062140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a206cf5654ebee server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:40.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.039+0000 7f6942a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:43:40.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.039+0000 7f6942a00640 1 -- 172.21.15.7:0/611035744 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948020040 con 0x55fd88416430 2024-09-10T21:43:40.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.039+0000 7f6942a00640 1 -- 172.21.15.7:0/611035744 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f6948061760 con 0x55fd88416430 2024-09-10T21:43:40.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.039+0000 7f6967259740 1 -- 172.21.15.7:0/611035744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6954003fe0 con 0x55fd88416430 2024-09-10T21:43:40.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.039+0000 7f6942a00640 1 -- 172.21.15.7:0/611035744 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f694806b090 con 0x55fd88416430 2024-09-10T21:43:40.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.039+0000 7f6967259740 10 client.5427 fetch_fsmap learned FSMap version 26 2024-09-10T21:43:40.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.039+0000 7f6967259740 1 -- 172.21.15.7:0/611035744 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55fd884ba110 con 0x55fd88416430 2024-09-10T21:43:40.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.039+0000 7f6942a00640 1 -- 172.21.15.7:0/611035744 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6954003fe0 con 0x55fd88416430 2024-09-10T21:43:40.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.039+0000 7f6942a00640 1 -- 172.21.15.7:0/611035744 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f694805dc50 con 0x55fd88416430 2024-09-10T21:43:40.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.039+0000 7f6967259740 10 client.5427 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T21:43:40.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.040+0000 7f6967259740 10 client.5427 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:43:40.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.040+0000 7f6967259740 20 client.5427 populate_metadata read hostname 'smithi007' 2024-09-10T21:43:40.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.040+0000 7f6967259740 1 --2- 172.21.15.7:0/611035744 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55fd884b9b50 0x55fd8841bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.040+0000 7f6967259740 4 client.5427 mds_command: new command op to 234767 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:43:40.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.040+0000 7f6967259740 1 -- 172.21.15.7:0/611035744 --> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] -- command(tid 0: {"prefix": "status"}) -- 0x55fd8835ffd0 con 0x55fd884b9b50 2024-09-10T21:43:40.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.040+0000 7f6961a00640 1 --2- 172.21.15.7:0/611035744 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55fd884b9b50 0x55fd8841bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:40.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.041+0000 7f6961a00640 1 --2- 172.21.15.7:0/611035744 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55fd884b9b50 0x55fd8841bbb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9c2ba8dd7c362c8 server_cookie=321b41846e68747f in_seq=0 out_seq=0 2024-09-10T21:43:40.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.041+0000 7f6942a00640 10 client.5427 ms_handle_connect on v2:172.21.15.133:6834/3385628197 2024-09-10T21:43:40.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.041+0000 7f6942a00640 1 -- 172.21.15.7:0/611035744 <== mds.0 v2:172.21.15.133:6834/3385628197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55fd8835ffd0 con 0x55fd884b9b50 2024-09-10T21:43:40.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.041+0000 7f6942a00640 10 client.5427 handle_command_reply: tid=0 2024-09-10T21:43:40.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.041+0000 7f6967259740 1 client.5427 shutdown 2024-09-10T21:43:40.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.041+0000 7f6967259740 1 -- 172.21.15.7:0/611035744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88416430 msgr2=0x55fd88411d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:40.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.041+0000 7f6967259740 1 --2- 172.21.15.7:0/611035744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88416430 0x55fd88411d50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f694805f570 tx=0x7f6948062140 comp rx=0 tx=0).stop 2024-09-10T21:43:40.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.041+0000 7f6967259740 1 -- 172.21.15.7:0/611035744 shutdown_connections 2024-09-10T21:43:40.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.041+0000 7f6967259740 1 --2- 172.21.15.7:0/611035744 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55fd884b9b50 0x55fd8841bbb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.041+0000 7f6967259740 1 --2- 172.21.15.7:0/611035744 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fd88434aa0 0x55fd88412290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.041+0000 7f6967259740 1 --2- 172.21.15.7:0/611035744 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fd88416430 0x55fd88411d50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.041+0000 7f6967259740 1 --2- 172.21.15.7:0/611035744 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fd883ea110 0x55fd8844a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.041+0000 7f6942a00640 0 client.5427 ms_handle_reset on v2:172.21.15.133:6834/3385628197 2024-09-10T21:43:40.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.041+0000 7f6967259740 1 -- 172.21.15.7:0/611035744 >> 172.21.15.7:0/611035744 conn(0x55fd883e91f0 msgr2=0x55fd883e95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:40.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.041+0000 7f6967259740 1 -- 172.21.15.7:0/611035744 shutdown_connections 2024-09-10T21:43:40.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.041+0000 7f6967259740 1 -- 172.21.15.7:0/611035744 wait complete. 2024-09-10T21:43:40.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.041+0000 7f6967259740 20 client.5427 trim_cache size 0 max 16384 2024-09-10T21:43:40.047 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:43:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2029451a-3c26-48c7-b874-0597197278ac", 2024-09-10T21:43:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-10T21:43:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 234767, 2024-09-10T21:43:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T21:43:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T21:43:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T21:43:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.2399513139999998, 2024-09-10T21:43:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 26, 2024-09-10T21:43:40.047 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 44, 2024-09-10T21:43:40.048 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 44, 2024-09-10T21:43:40.048 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 11.525562729000001 2024-09-10T21:43:40.048 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:43:40.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.042+0000 7f6962e00640 1 -- 172.21.15.7:0/3131538290 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f693c077f70 msgr2=0x7f693c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:40.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.042+0000 7f6962e00640 1 --2- 172.21.15.7:0/3131538290 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f693c077f70 0x7f693c07a430 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f695c0c5370 tx=0x7f6950009040 comp rx=0 tx=0).stop 2024-09-10T21:43:40.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.042+0000 7f6962e00640 1 -- 172.21.15.7:0/3131538290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f695c0d0650 msgr2=0x7f695c0b35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:40.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.042+0000 7f6962e00640 1 --2- 172.21.15.7:0/3131538290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f695c0d0650 0x7f695c0b35e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f694800d770 tx=0x7f694800dc30 comp rx=0 tx=0).stop 2024-09-10T21:43:40.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.042+0000 7f6962e00640 1 -- 172.21.15.7:0/3131538290 shutdown_connections 2024-09-10T21:43:40.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.042+0000 7f6962e00640 1 --2- 172.21.15.7:0/3131538290 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f693c077f70 0x7f693c07a430 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.042+0000 7f6962e00640 1 --2- 172.21.15.7:0/3131538290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f695c15f2f0 0x7f695c0b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.042+0000 7f6962e00640 1 --2- 172.21.15.7:0/3131538290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f695c0d0650 0x7f695c0b35e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.042+0000 7f6962e00640 1 --2- 172.21.15.7:0/3131538290 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f695c0cfa60 0x7f695c149690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.042+0000 7f6962e00640 1 -- 172.21.15.7:0/3131538290 >> 172.21.15.7:0/3131538290 conn(0x7f695c0d1b60 msgr2=0x7f695c0d1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:40.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.042+0000 7f6962e00640 1 -- 172.21.15.7:0/3131538290 shutdown_connections 2024-09-10T21:43:40.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.042+0000 7f6962e00640 1 -- 172.21.15.7:0/3131538290 wait complete. 2024-09-10T21:43:40.058 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:43:40.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.217+0000 7f3a01000640 1 -- 172.21.15.7:0/4158869272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e0003aa0 msgr2=0x7f39e0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:40.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.217+0000 7f3a01000640 1 --2- 172.21.15.7:0/4158869272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e0003aa0 0x7f39e0003ec0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f39ec006f90 tx=0x7f39e0005830 comp rx=0 tx=0).stop 2024-09-10T21:43:40.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.217+0000 7f3a01000640 1 -- 172.21.15.7:0/4158869272 shutdown_connections 2024-09-10T21:43:40.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.217+0000 7f3a01000640 1 --2- 172.21.15.7:0/4158869272 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e0003aa0 0x7f39e0003ec0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.217+0000 7f3a01000640 1 -- 172.21.15.7:0/4158869272 >> 172.21.15.7:0/4158869272 conn(0x7f39fc0b16d0 msgr2=0x7f39fc0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:40.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.217+0000 7f3a01000640 1 -- 172.21.15.7:0/4158869272 shutdown_connections 2024-09-10T21:43:40.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.217+0000 7f3a01000640 1 -- 172.21.15.7:0/4158869272 wait complete. 2024-09-10T21:43:40.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.219+0000 7f3a01000640 1 Processor -- start 2024-09-10T21:43:40.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.219+0000 7f3a01000640 1 -- start start 2024-09-10T21:43:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.220+0000 7f3a01000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39e0003aa0 0x7f39fc1f77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.220+0000 7f3a01000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39fc0c36b0 0x7f39fc0c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.220+0000 7f3a01000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39fc0b2a40 0x7f39fc0b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.220+0000 7f3a01000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc0cd2d0 con 0x7f39fc0b2a40 2024-09-10T21:43:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.220+0000 7f3a01000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f39fc0cd150 con 0x7f39fc0c36b0 2024-09-10T21:43:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.220+0000 7f3a01000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f39fc160180 con 0x7f39e0003aa0 2024-09-10T21:43:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.220+0000 7f39fa800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39fc0b2a40 0x7f39fc0b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.220+0000 7f39fa800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39fc0b2a40 0x7f39fc0b2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44188/0 (socket says 172.21.15.7:44188) 2024-09-10T21:43:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.220+0000 7f39fa800640 1 -- 172.21.15.7:0/2034410105 learned_addr learned my addr 172.21.15.7:0/2034410105 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.220+0000 7f39f9e00640 1 --2- 172.21.15.7:0/2034410105 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39e0003aa0 0x7f39fc1f77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.220+0000 7f39f1400640 1 --2- 172.21.15.7:0/2034410105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39fc0c36b0 0x7f39fc0c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.221+0000 7f39fa800640 1 -- 172.21.15.7:0/2034410105 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39e0003aa0 msgr2=0x7f39fc1f77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.221+0000 7f39fa800640 1 --2- 172.21.15.7:0/2034410105 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39e0003aa0 0x7f39fc1f77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.221+0000 7f39fa800640 1 -- 172.21.15.7:0/2034410105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39fc0c36b0 msgr2=0x7f39fc0c8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.221+0000 7f39fa800640 1 --2- 172.21.15.7:0/2034410105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39fc0c36b0 0x7f39fc0c8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.221+0000 7f39fa800640 1 -- 172.21.15.7:0/2034410105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39fc0c3b30 con 0x7f39fc0b2a40 2024-09-10T21:43:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.221+0000 7f39fa800640 1 --2- 172.21.15.7:0/2034410105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39fc0b2a40 0x7f39fc0b2ec0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f39f40020c0 tx=0x7f39f4006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.221+0000 7f39f2a00640 1 -- 172.21.15.7:0/2034410105 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4015020 con 0x7f39fc0b2a40 2024-09-10T21:43:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.221+0000 7f39f2a00640 1 -- 172.21.15.7:0/2034410105 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39f4011840 con 0x7f39fc0b2a40 2024-09-10T21:43:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.221+0000 7f39f2a00640 1 -- 172.21.15.7:0/2034410105 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f4011c10 con 0x7f39fc0b2a40 2024-09-10T21:43:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.221+0000 7f3a01000640 1 -- 172.21.15.7:0/2034410105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39fc0c3db0 con 0x7f39fc0b2a40 2024-09-10T21:43:40.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.221+0000 7f3a01000640 1 -- 172.21.15.7:0/2034410105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39fc0c4370 con 0x7f39fc0b2a40 2024-09-10T21:43:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.222+0000 7f3a01000640 1 -- 172.21.15.7:0/2034410105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39fc073730 con 0x7f39fc0b2a40 2024-09-10T21:43:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.224+0000 7f39f2a00640 1 -- 172.21.15.7:0/2034410105 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f39f401a400 con 0x7f39fc0b2a40 2024-09-10T21:43:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.224+0000 7f39f2a00640 1 --2- 172.21.15.7:0/2034410105 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f39d8077f70 0x7f39d807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:40.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.225+0000 7f39f2a00640 1 -- 172.21.15.7:0/2034410105 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f39f409aa60 con 0x7f39fc0b2a40 2024-09-10T21:43:40.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.225+0000 7f39f9e00640 1 --2- 172.21.15.7:0/2034410105 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f39d8077f70 0x7f39d807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:40.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.226+0000 7f39f9e00640 1 --2- 172.21.15.7:0/2034410105 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f39d8077f70 0x7f39d807a430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f39ec033000 tx=0x7f39fc0cde90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:40.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.228+0000 7f39f2a00640 1 -- 172.21.15.7:0/2034410105 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f39f401f020 con 0x7f39fc0b2a40 2024-09-10T21:43:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.458+0000 7f3a01000640 1 -- 172.21.15.7:0/2034410105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f39fc0b0d70 con 0x7f39fc0b2a40 2024-09-10T21:43:40.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.459+0000 7f39f2a00640 1 -- 172.21.15.7:0/2034410105 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f39fc0c3b30 con 0x7f39fc0b2a40 2024-09-10T21:43:40.460 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:43:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.461+0000 7f3a01000640 1 -- 172.21.15.7:0/2034410105 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f39d8077f70 msgr2=0x7f39d807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.461+0000 7f3a01000640 1 --2- 172.21.15.7:0/2034410105 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f39d8077f70 0x7f39d807a430 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f39ec033000 tx=0x7f39fc0cde90 comp rx=0 tx=0).stop 2024-09-10T21:43:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.461+0000 7f3a01000640 1 -- 172.21.15.7:0/2034410105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39fc0b2a40 msgr2=0x7f39fc0b2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.461+0000 7f3a01000640 1 --2- 172.21.15.7:0/2034410105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39fc0b2a40 0x7f39fc0b2ec0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f39f40020c0 tx=0x7f39f4006430 comp rx=0 tx=0).stop 2024-09-10T21:43:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.462+0000 7f3a01000640 1 -- 172.21.15.7:0/2034410105 shutdown_connections 2024-09-10T21:43:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.462+0000 7f3a01000640 1 --2- 172.21.15.7:0/2034410105 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f39d8077f70 0x7f39d807a430 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.462+0000 7f3a01000640 1 --2- 172.21.15.7:0/2034410105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39fc0c36b0 0x7f39fc0c8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.462+0000 7f3a01000640 1 --2- 172.21.15.7:0/2034410105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39fc0b2a40 0x7f39fc0b2ec0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.462+0000 7f3a01000640 1 --2- 172.21.15.7:0/2034410105 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39e0003aa0 0x7f39fc1f77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.462+0000 7f3a01000640 1 -- 172.21.15.7:0/2034410105 >> 172.21.15.7:0/2034410105 conn(0x7f39fc0b16d0 msgr2=0x7f39fc0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.462+0000 7f3a01000640 1 -- 172.21.15.7:0/2034410105 shutdown_connections 2024-09-10T21:43:40.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40.462+0000 7f3a01000640 1 -- 172.21.15.7:0/2034410105 wait complete. 2024-09-10T21:43:40.479 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:43:40.479 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:43:40.479 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-10T21:43:40.479 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:43:40.479 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:43:40.480 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:43:40.480 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:43:40.480 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:43:40.480 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:43:40.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:40.480 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-10T21:43:40.516 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:43:40.516 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:43:40.516 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:43:40.516 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:43:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T21:43:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:43:40.517 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:43:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T21:43:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-10T21:43:40.517 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:43:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T21:43:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-10T21:43:40.517 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:43:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-10T21:43:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-10T21:43:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T21:43:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42524sec preferred_lft 42524sec 2024-09-10T21:43:40.517 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-10T21:43:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:43:40.518 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:43:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-10T21:43:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-10T21:43:40.518 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:43:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff 2024-09-10T21:43:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:43:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:43:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::3c3d:dbff:fe02:4c15/64 scope link 2024-09-10T21:43:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:43:40.518 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:43:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether f6:f9:69:5a:86:f5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:43:40.518 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f4f9:69ff:fe5a:86f5/64 scope link 2024-09-10T21:43:40.519 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:43:40.519 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:43:40.519 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:43:40.519 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::433:5cff:fe44:4a5b/64 scope link 2024-09-10T21:43:40.519 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:43:40.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:40.519 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:43:40.611 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:43:40.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:40.615 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-10T21:43:40.650 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-10T21:43:40.651 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:43:40.651 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T21:43:40.651 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T21:43:40.651 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:43:40.651 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:43:40.651 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:43:40.651 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:43:40.651 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:43:40.651 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:43:40.652 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:43:40.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:43:40.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:43:40.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:40.852 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:43:40.887 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:43:40.887 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:43:40.888 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:43:40.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:40.888 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-10T21:43:40.984 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:43:40.985 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:43:41.020 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:43:41.020 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:43:41.020 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:43:41.098 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:43:41.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:41.098 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:43:41.226 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:43:41.227 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:43:41.256 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:43:41.256 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:43:41.256 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.133:3300,172.21.15.133:3301" 2024-09-10T21:43:41.256 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@2029451a-3c26-48c7-b874-0597197278ac.cephfs=/ 2024-09-10T21:43:41.256 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.133:3300/172.21.15.133:3301" will pass to kernel 2024-09-10T21:43:41.257 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T21:43:41.258 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:43:41.258 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:43:41.258 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:43:41.258 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:43:41.261 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:43:41.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:41.261 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:43:41.318 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:43:41.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:43:41.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:43:41.417 INFO:teuthology.orchestra.run.smithi007.stdout:client234848 2024-09-10T21:43:41.417 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:43:41.417 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:43:41.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:41.453 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2024-09-10T21:43:41.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:41.536 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2024-09-10T21:43:41.615 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:43:41.615 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:43:41.615 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:43:41.615 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:43:41.615 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2024-09-10T21:43:41.615 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:43:41.755 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627777 2024-09-10T21:43:41.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:43:41.767 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:43:41.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.969+0000 7f113c400640 1 -- 172.21.15.7:0/881804302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1134161640 msgr2=0x7f11340ccc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.969+0000 7f113c400640 1 --2- 172.21.15.7:0/881804302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1134161640 0x7f11340ccc80 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f1128005fd0 tx=0x7f112802a4a0 comp rx=0 tx=0).stop 2024-09-10T21:43:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.969+0000 7f113c400640 1 -- 172.21.15.7:0/881804302 shutdown_connections 2024-09-10T21:43:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.969+0000 7f113c400640 1 --2- 172.21.15.7:0/881804302 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11340ce6a0 0x7f11340ceb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.969+0000 7f113c400640 1 --2- 172.21.15.7:0/881804302 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1134161640 0x7f11340ccc80 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.969+0000 7f113c400640 1 -- 172.21.15.7:0/881804302 >> 172.21.15.7:0/881804302 conn(0x7f11340d1d20 msgr2=0x7f11340d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.969+0000 7f113c400640 1 -- 172.21.15.7:0/881804302 shutdown_connections 2024-09-10T21:43:41.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.969+0000 7f113c400640 1 -- 172.21.15.7:0/881804302 wait complete. 2024-09-10T21:43:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.970+0000 7f113c400640 1 Processor -- start 2024-09-10T21:43:41.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.971+0000 7f113c400640 1 -- start start 2024-09-10T21:43:41.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.971+0000 7f113c400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11340cdc00 0x7f113415dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:41.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.971+0000 7f113c400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11340ce6a0 0x7f11340cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:41.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.972+0000 7f113c400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1134161640 0x7f11340cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:41.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.972+0000 7f113c400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f113406d970 con 0x7f11340ce6a0 2024-09-10T21:43:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.972+0000 7f113c400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f113406daf0 con 0x7f11340cdc00 2024-09-10T21:43:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.972+0000 7f113c400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f113406dc70 con 0x7f1134161640 2024-09-10T21:43:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.972+0000 7f1139000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11340ce6a0 0x7f11340cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.972+0000 7f1139000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11340ce6a0 0x7f11340cb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44214/0 (socket says 172.21.15.7:44214) 2024-09-10T21:43:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.972+0000 7f1139000640 1 -- 172.21.15.7:0/2608045320 learned_addr learned my addr 172.21.15.7:0/2608045320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.972+0000 7f1139a00640 1 --2- 172.21.15.7:0/2608045320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11340cdc00 0x7f113415dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.972+0000 7f113a400640 1 --2- 172.21.15.7:0/2608045320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1134161640 0x7f11340cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.972+0000 7f1139000640 1 -- 172.21.15.7:0/2608045320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1134161640 msgr2=0x7f11340cb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.972+0000 7f1139000640 1 --2- 172.21.15.7:0/2608045320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1134161640 0x7f11340cb5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.972+0000 7f1139000640 1 -- 172.21.15.7:0/2608045320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11340cdc00 msgr2=0x7f113415dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.972+0000 7f1139000640 1 --2- 172.21.15.7:0/2608045320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11340cdc00 0x7f113415dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:41.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.972+0000 7f1139000640 1 -- 172.21.15.7:0/2608045320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113415f1d0 con 0x7f11340ce6a0 2024-09-10T21:43:41.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.972+0000 7f1139a00640 1 --2- 172.21.15.7:0/2608045320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11340cdc00 0x7f113415dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:41.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.973+0000 7f1139000640 1 --2- 172.21.15.7:0/2608045320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11340ce6a0 0x7f11340cb070 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f112000c460 tx=0x7f112000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:41.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.973+0000 7f1131600640 1 -- 172.21.15.7:0/2608045320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1120016020 con 0x7f11340ce6a0 2024-09-10T21:43:41.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.973+0000 7f1131600640 1 -- 172.21.15.7:0/2608045320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1120012960 con 0x7f11340ce6a0 2024-09-10T21:43:41.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.973+0000 7f1131600640 1 -- 172.21.15.7:0/2608045320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1120012db0 con 0x7f11340ce6a0 2024-09-10T21:43:41.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.973+0000 7f113c400640 1 -- 172.21.15.7:0/2608045320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f113415f4b0 con 0x7f11340ce6a0 2024-09-10T21:43:41.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.973+0000 7f113c400640 1 -- 172.21.15.7:0/2608045320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f113415fa70 con 0x7f11340ce6a0 2024-09-10T21:43:41.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.974+0000 7f113c400640 1 -- 172.21.15.7:0/2608045320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f113406e200 con 0x7f11340ce6a0 2024-09-10T21:43:41.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.975+0000 7f1131600640 1 -- 172.21.15.7:0/2608045320 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f112001b400 con 0x7f11340ce6a0 2024-09-10T21:43:41.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.976+0000 7f1131600640 1 --2- 172.21.15.7:0/2608045320 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1114077f70 0x7f111407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:41.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.976+0000 7f1131600640 1 -- 172.21.15.7:0/2608045320 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f112009a140 con 0x7f11340ce6a0 2024-09-10T21:43:41.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.977+0000 7f1139a00640 1 --2- 172.21.15.7:0/2608045320 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1114077f70 0x7f111407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:41.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.978+0000 7f1139a00640 1 --2- 172.21.15.7:0/2608045320 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1114077f70 0x7f111407a430 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1128033000 tx=0x7f1128004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:41.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:41.980+0000 7f1131600640 1 -- 172.21.15.7:0/2608045320 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1120020020 con 0x7f11340ce6a0 2024-09-10T21:43:42.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.212+0000 7f113c400640 1 -- 172.21.15.7:0/2608045320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11340cf6b0 con 0x7f11340ce6a0 2024-09-10T21:43:42.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.212+0000 7f1131600640 1 -- 172.21.15.7:0/2608045320 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5233 (secure 0 0 0) 0x7f1120062ba0 con 0x7f11340ce6a0 2024-09-10T21:43:42.214 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:42.214 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"btime":"2024-09-10T21:43:36:831759+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5149,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3851521050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3851521050},{"type":"v1","addr":"172.21.15.133:6836","nonce":3851521050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5289,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2709023497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2709023497},{"type":"v1","addr":"172.21.15.7:6834","nonce":2709023497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/4095876032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":4095876032},{"type":"v1","addr":"172.21.15.7:6835","nonce":4095876032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:43:33.245584+0000","modified":"2024-09-10T21:43:36.266208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234767},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234767":{"gid":234767,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3385628197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3385628197},{"type":"v1","addr":"172.21.15.133:6837","nonce":3385628197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234767,"qdb_cluster":[234767]},"id":3}]} 2024-09-10T21:43:42.214 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-09-10T21:43:42.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.215+0000 7f113c400640 1 -- 172.21.15.7:0/2608045320 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1114077f70 msgr2=0x7f111407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.215+0000 7f113c400640 1 --2- 172.21.15.7:0/2608045320 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1114077f70 0x7f111407a430 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1128033000 tx=0x7f1128004730 comp rx=0 tx=0).stop 2024-09-10T21:43:42.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.215+0000 7f113c400640 1 -- 172.21.15.7:0/2608045320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11340ce6a0 msgr2=0x7f11340cb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.215+0000 7f113c400640 1 --2- 172.21.15.7:0/2608045320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11340ce6a0 0x7f11340cb070 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f112000c460 tx=0x7f112000c920 comp rx=0 tx=0).stop 2024-09-10T21:43:42.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.215+0000 7f113c400640 1 -- 172.21.15.7:0/2608045320 shutdown_connections 2024-09-10T21:43:42.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.215+0000 7f113c400640 1 --2- 172.21.15.7:0/2608045320 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1114077f70 0x7f111407a430 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.215+0000 7f113c400640 1 --2- 172.21.15.7:0/2608045320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1134161640 0x7f11340cb5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.215+0000 7f113c400640 1 --2- 172.21.15.7:0/2608045320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11340ce6a0 0x7f11340cb070 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.215+0000 7f113c400640 1 --2- 172.21.15.7:0/2608045320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11340cdc00 0x7f113415dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.215+0000 7f113c400640 1 -- 172.21.15.7:0/2608045320 >> 172.21.15.7:0/2608045320 conn(0x7f11340d1d20 msgr2=0x7f1134071f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:42.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.215+0000 7f113c400640 1 -- 172.21.15.7:0/2608045320 shutdown_connections 2024-09-10T21:43:42.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.215+0000 7f113c400640 1 -- 172.21.15.7:0/2608045320 wait complete. 2024-09-10T21:43:42.233 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-09-10T21:43:42.425 INFO:teuthology.orchestra.run.smithi133.stdout:{"message":"","return_code":0} 2024-09-10T21:43:42.425 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-10T21:43:42.426 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-09-10T21:43:42.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.501+0000 7f9d04418f40 1 Processor -- start 2024-09-10T21:43:42.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.501+0000 7f9d04418f40 1 -- start start 2024-09-10T21:43:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.502+0000 7f9d04418f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd3938ae0 0x564cd3966920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.502+0000 7f9d04418f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564cd39852e0 0x564cd3a10de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.502+0000 7f9d04418f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd39f5620 0x564cd396a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.502+0000 7f9d04418f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564cd38e61d0 con 0x564cd3938ae0 2024-09-10T21:43:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.502+0000 7f9d04418f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564cd38f2f90 con 0x564cd39f5620 2024-09-10T21:43:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.502+0000 7f9d04418f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x564cd3887c60 con 0x564cd39852e0 2024-09-10T21:43:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.502+0000 7f9d01600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd3938ae0 0x564cd3966920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.502+0000 7f9d01600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd3938ae0 0x564cd3966920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44230/0 (socket says 172.21.15.7:44230) 2024-09-10T21:43:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.502+0000 7f9d01600640 1 -- 172.21.15.7:0/2144021225 learned_addr learned my addr 172.21.15.7:0/2144021225 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:42.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.503+0000 7f9d02000640 1 --2- 172.21.15.7:0/2144021225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd39f5620 0x564cd396a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.503+0000 7f9d00c00640 1 --2- 172.21.15.7:0/2144021225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564cd39852e0 0x564cd3a10de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.503+0000 7f9d01600640 1 -- 172.21.15.7:0/2144021225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564cd39852e0 msgr2=0x564cd3a10de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.503+0000 7f9d01600640 1 --2- 172.21.15.7:0/2144021225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564cd39852e0 0x564cd3a10de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.503+0000 7f9d01600640 1 -- 172.21.15.7:0/2144021225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd39f5620 msgr2=0x564cd396a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.503+0000 7f9d01600640 1 --2- 172.21.15.7:0/2144021225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd39f5620 0x564cd396a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.503+0000 7f9d01600640 1 -- 172.21.15.7:0/2144021225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cd3a09790 con 0x564cd3938ae0 2024-09-10T21:43:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.503+0000 7f9d02000640 1 --2- 172.21.15.7:0/2144021225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd39f5620 0x564cd396a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:43:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.503+0000 7f9d00c00640 1 --2- 172.21.15.7:0/2144021225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564cd39852e0 0x564cd3a10de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:43:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.503+0000 7f9d01600640 1 --2- 172.21.15.7:0/2144021225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd3938ae0 0x564cd3966920 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4004e90 tx=0x7f9cf4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e164217764cb9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.503+0000 7f9cfb400640 1 -- 172.21.15.7:0/2144021225 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf404da60 con 0x564cd3938ae0 2024-09-10T21:43:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.503+0000 7f9cfb400640 1 -- 172.21.15.7:0/2144021225 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cf4056580 con 0x564cd3938ae0 2024-09-10T21:43:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.503+0000 7f9cfb400640 1 -- 172.21.15.7:0/2144021225 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf40569d0 con 0x564cd3938ae0 2024-09-10T21:43:42.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.504+0000 7f9d04418f40 1 -- 172.21.15.7:0/2144021225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd3938ae0 msgr2=0x564cd3966920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.504+0000 7f9d04418f40 1 --2- 172.21.15.7:0/2144021225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd3938ae0 0x564cd3966920 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4004e90 tx=0x7f9cf4002fb0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.504+0000 7f9d04418f40 1 -- 172.21.15.7:0/2144021225 shutdown_connections 2024-09-10T21:43:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.504+0000 7f9d04418f40 1 --2- 172.21.15.7:0/2144021225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd39f5620 0x564cd396a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.504+0000 7f9d04418f40 1 --2- 172.21.15.7:0/2144021225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564cd39852e0 0x564cd3a10de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.504+0000 7f9d04418f40 1 --2- 172.21.15.7:0/2144021225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd3938ae0 0x564cd3966920 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.504+0000 7f9d04418f40 1 -- 172.21.15.7:0/2144021225 >> 172.21.15.7:0/2144021225 conn(0x564cd3967de0 msgr2=0x564cd3a126b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.504+0000 7f9d04418f40 1 -- 172.21.15.7:0/2144021225 shutdown_connections 2024-09-10T21:43:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.504+0000 7f9d04418f40 1 -- 172.21.15.7:0/2144021225 wait complete. 2024-09-10T21:43:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.505+0000 7f9d04418f40 1 Processor -- start 2024-09-10T21:43:42.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.505+0000 7f9d04418f40 1 -- start start 2024-09-10T21:43:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.505+0000 7f9d04418f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564cd3938ae0 0x564cd3a991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.505+0000 7f9d04418f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd39852e0 0x564cd3a10da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.506+0000 7f9d04418f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd39f5620 0x564cd3a112e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.506+0000 7f9d04418f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564cd396bad0 con 0x564cd39852e0 2024-09-10T21:43:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.506+0000 7f9d04418f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564cd396b7f0 con 0x564cd39f5620 2024-09-10T21:43:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.506+0000 7f9d04418f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x564cd3a062e0 con 0x564cd3938ae0 2024-09-10T21:43:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.506+0000 7f9d00c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd39852e0 0x564cd3a10da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.506+0000 7f9d00c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd39852e0 0x564cd3a10da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44234/0 (socket says 172.21.15.7:44234) 2024-09-10T21:43:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.506+0000 7f9d00c00640 1 -- 172.21.15.7:0/291768449 learned_addr learned my addr 172.21.15.7:0/291768449 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.506+0000 7f9d01600640 1 --2- 172.21.15.7:0/291768449 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564cd3938ae0 0x564cd3a991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.506+0000 7f9d02000640 1 --2- 172.21.15.7:0/291768449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd39f5620 0x564cd3a112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.506+0000 7f9d00c00640 1 -- 172.21.15.7:0/291768449 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564cd3938ae0 msgr2=0x564cd3a991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.506+0000 7f9d00c00640 1 --2- 172.21.15.7:0/291768449 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564cd3938ae0 0x564cd3a991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.506+0000 7f9d00c00640 1 -- 172.21.15.7:0/291768449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd39f5620 msgr2=0x564cd3a112e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.506+0000 7f9d00c00640 1 --2- 172.21.15.7:0/291768449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564cd39f5620 0x564cd3a112e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.506+0000 7f9d00c00640 1 -- 172.21.15.7:0/291768449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cd396ad30 con 0x564cd39852e0 2024-09-10T21:43:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.506+0000 7f9d01600640 1 --2- 172.21.15.7:0/291768449 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564cd3938ae0 0x564cd3a991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.506+0000 7f9d00c00640 1 --2- 172.21.15.7:0/291768449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd39852e0 0x564cd3a10da0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000cf80 tx=0x7f9cf000a720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.507+0000 7f9cfa000640 1 -- 172.21.15.7:0/291768449 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000e800 con 0x564cd39852e0 2024-09-10T21:43:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.507+0000 7f9cfa000640 1 -- 172.21.15.7:0/291768449 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cf0006050 con 0x564cd39852e0 2024-09-10T21:43:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.507+0000 7f9cfa000640 1 -- 172.21.15.7:0/291768449 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000ec20 con 0x564cd39852e0 2024-09-10T21:43:42.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.507+0000 7f9d04418f40 1 -- 172.21.15.7:0/291768449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564cd396a770 con 0x564cd39852e0 2024-09-10T21:43:42.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.507+0000 7f9d04418f40 1 -- 172.21.15.7:0/291768449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cd3a99730 con 0x564cd39852e0 2024-09-10T21:43:42.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.509+0000 7f9cfa000640 1 -- 172.21.15.7:0/291768449 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9cf001f440 con 0x564cd39852e0 2024-09-10T21:43:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.510+0000 7f9cfa000640 1 --2- 172.21.15.7:0/291768449 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9ce4076950 0x7f9ce4078e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.510+0000 7f9cfa000640 1 -- 172.21.15.7:0/291768449 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f9cf0098e10 con 0x564cd39852e0 2024-09-10T21:43:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.510+0000 7f9d04418f40 1 --2- 172.21.15.7:0/291768449 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x564cd3a12800 0x564cd3a0ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.510+0000 7f9d04418f40 1 -- 172.21.15.7:0/291768449 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e44) -- 0x564cd3a08c80 con 0x564cd3a12800 2024-09-10T21:43:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.510+0000 7f9d01600640 1 --2- 172.21.15.7:0/291768449 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9ce4076950 0x7f9ce4078e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.510+0000 7f9d02000640 1 --2- 172.21.15.7:0/291768449 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x564cd3a12800 0x564cd3a0ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.511+0000 7f9d02000640 1 --2- 172.21.15.7:0/291768449 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x564cd3a12800 0x564cd3a0ab80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.511+0000 7f9d01600640 1 --2- 172.21.15.7:0/291768449 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9ce4076950 0x7f9ce4078e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4000c00 tx=0x7f9cf4004880 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.511+0000 7f9d02000640 1 -- 172.21.15.7:0/291768449 <== osd.6 v2:172.21.15.133:6810/3365981160 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7f9cfc001500 con 0x564cd3a12800 2024-09-10T21:43:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.512+0000 7f9d04418f40 1 -- 172.21.15.7:0/291768449 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x564cd3a12800 msgr2=0x564cd3a0ab80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.512+0000 7f9d04418f40 1 --2- 172.21.15.7:0/291768449 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x564cd3a12800 0x564cd3a0ab80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.512+0000 7f9d04418f40 1 -- 172.21.15.7:0/291768449 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9ce4076950 msgr2=0x7f9ce4078e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.512+0000 7f9d04418f40 1 --2- 172.21.15.7:0/291768449 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9ce4076950 0x7f9ce4078e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f9cf4000c00 tx=0x7f9cf4004880 comp rx=0 tx=0).stop 2024-09-10T21:43:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.512+0000 7f9d04418f40 1 -- 172.21.15.7:0/291768449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd39852e0 msgr2=0x564cd3a10da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.512+0000 7f9d04418f40 1 --2- 172.21.15.7:0/291768449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564cd39852e0 0x564cd3a10da0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9cf000cf80 tx=0x7f9cf000a720 comp rx=0 tx=0).stop 2024-09-10T21:43:42.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.512+0000 7f9d02000640 1 -- 172.21.15.7:0/291768449 reap_dead start 2024-09-10T21:43:42.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.512+0000 7f9d04418f40 1 -- 172.21.15.7:0/291768449 shutdown_connections 2024-09-10T21:43:42.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.512+0000 7f9d04418f40 1 -- 172.21.15.7:0/291768449 >> 172.21.15.7:0/291768449 conn(0x564cd3967de0 msgr2=0x564cd39862d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:42.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.512+0000 7f9d04418f40 1 -- 172.21.15.7:0/291768449 shutdown_connections 2024-09-10T21:43:42.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.512+0000 7f9d04418f40 1 -- 172.21.15.7:0/291768449 wait complete. 2024-09-10T21:43:42.517 INFO:teuthology.orchestra.run.smithi007.stdout:Yalphaparent_a 2024-09-10T21:43:42.517 DEBUG:teuthology.orchestra.run.smithi007:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-09-10T21:43:42.651 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:43:42.652 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627777, 2024-09-10T21:43:42.652 INFO:teuthology.orchestra.run.smithi007.stdout: "ancestors": [ 2024-09-10T21:43:42.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:42.652 INFO:teuthology.orchestra.run.smithi007.stdout: "dirino": 1099511627776, 2024-09-10T21:43:42.652 INFO:teuthology.orchestra.run.smithi007.stdout: "dname": "alpha", 2024-09-10T21:43:42.652 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 2 2024-09-10T21:43:42.652 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:42.652 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:43:42.652 INFO:teuthology.orchestra.run.smithi007.stdout: "dirino": 1, 2024-09-10T21:43:42.652 INFO:teuthology.orchestra.run.smithi007.stdout: "dname": "parent_a", 2024-09-10T21:43:42.652 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 4 2024-09-10T21:43:42.653 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:42.653 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T21:43:42.653 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 7, 2024-09-10T21:43:42.653 INFO:teuthology.orchestra.run.smithi007.stdout: "old_pools": [] 2024-09-10T21:43:42.653 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:43:42.653 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:42.658 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2024-09-10T21:43:42.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.735+0000 7f0d131c5f40 1 Processor -- start 2024-09-10T21:43:42.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.736+0000 7f0d131c5f40 1 -- start start 2024-09-10T21:43:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.736+0000 7f0d131c5f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df301e3920 0x55df3027e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.736+0000 7f0d131c5f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df301fcda0 0x55df302880e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.736+0000 7f0d131c5f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55df30284b90 0x55df30281e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.736+0000 7f0d131c5f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55df300ffc60 con 0x55df301fcda0 2024-09-10T21:43:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.736+0000 7f0d131c5f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55df3015e1d0 con 0x55df301e3920 2024-09-10T21:43:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.736+0000 7f0d131c5f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55df301ae910 con 0x55df30284b90 2024-09-10T21:43:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.737+0000 7f0d0fa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df301fcda0 0x55df302880e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.737+0000 7f0d0fa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df301fcda0 0x55df302880e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44248/0 (socket says 172.21.15.7:44248) 2024-09-10T21:43:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.737+0000 7f0d0fa00640 1 -- 172.21.15.7:0/3834257361 learned_addr learned my addr 172.21.15.7:0/3834257361 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.737+0000 7f0d10e00640 1 --2- 172.21.15.7:0/3834257361 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55df30284b90 0x55df30281e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.737+0000 7f0d10400640 1 --2- 172.21.15.7:0/3834257361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df301e3920 0x55df3027e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.737+0000 7f0d0fa00640 1 -- 172.21.15.7:0/3834257361 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55df30284b90 msgr2=0x55df30281e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.737+0000 7f0d0fa00640 1 --2- 172.21.15.7:0/3834257361 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55df30284b90 0x55df30281e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.737+0000 7f0d0fa00640 1 -- 172.21.15.7:0/3834257361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df301e3920 msgr2=0x55df3027e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.737+0000 7f0d0fa00640 1 --2- 172.21.15.7:0/3834257361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df301e3920 0x55df3027e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.737+0000 7f0d0fa00640 1 -- 172.21.15.7:0/3834257361 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df30290430 con 0x55df301fcda0 2024-09-10T21:43:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.737+0000 7f0d0fa00640 1 --2- 172.21.15.7:0/3834257361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df301fcda0 0x55df302880e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc025f40 tx=0x7f0cfc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6553dffa35dd5c72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.738+0000 7f0d0e600640 1 -- 172.21.15.7:0/3834257361 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc04d9a0 con 0x55df301fcda0 2024-09-10T21:43:42.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.738+0000 7f0d0e600640 1 -- 172.21.15.7:0/3834257361 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cfc056670 con 0x55df301fcda0 2024-09-10T21:43:42.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.738+0000 7f0d0e600640 1 -- 172.21.15.7:0/3834257361 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc056ac0 con 0x55df301fcda0 2024-09-10T21:43:42.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.738+0000 7f0d131c5f40 1 -- 172.21.15.7:0/3834257361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df301fcda0 msgr2=0x55df302880e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.738+0000 7f0d131c5f40 1 --2- 172.21.15.7:0/3834257361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df301fcda0 0x55df302880e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc025f40 tx=0x7f0cfc002fb0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.738+0000 7f0d131c5f40 1 -- 172.21.15.7:0/3834257361 shutdown_connections 2024-09-10T21:43:42.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.738+0000 7f0d131c5f40 1 --2- 172.21.15.7:0/3834257361 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55df30284b90 0x55df30281e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.738+0000 7f0d131c5f40 1 --2- 172.21.15.7:0/3834257361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df301fcda0 0x55df302880e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.738+0000 7f0d131c5f40 1 --2- 172.21.15.7:0/3834257361 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df301e3920 0x55df3027e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.738+0000 7f0d131c5f40 1 -- 172.21.15.7:0/3834257361 >> 172.21.15.7:0/3834257361 conn(0x55df301fddf0 msgr2=0x55df301dd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.738+0000 7f0d131c5f40 1 -- 172.21.15.7:0/3834257361 shutdown_connections 2024-09-10T21:43:42.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.738+0000 7f0d131c5f40 1 -- 172.21.15.7:0/3834257361 wait complete. 2024-09-10T21:43:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.739+0000 7f0d131c5f40 1 Processor -- start 2024-09-10T21:43:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.739+0000 7f0d131c5f40 1 -- start start 2024-09-10T21:43:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.740+0000 7f0d131c5f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df301e3920 0x55df3028b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.740+0000 7f0d131c5f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55df301fcda0 0x55df302804e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.740+0000 7f0d131c5f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df30284b90 0x55df3027cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.740+0000 7f0d131c5f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55df30281a00 con 0x55df30284b90 2024-09-10T21:43:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.740+0000 7f0d131c5f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55df301dee80 con 0x55df301e3920 2024-09-10T21:43:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.740+0000 7f0d131c5f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55df301df0d0 con 0x55df301fcda0 2024-09-10T21:43:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.740+0000 7f0d10e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df30284b90 0x55df3027cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.740+0000 7f0d10e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df30284b90 0x55df3027cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44250/0 (socket says 172.21.15.7:44250) 2024-09-10T21:43:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.740+0000 7f0d10e00640 1 -- 172.21.15.7:0/174009762 learned_addr learned my addr 172.21.15.7:0/174009762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.741+0000 7f0d10400640 1 --2- 172.21.15.7:0/174009762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df301e3920 0x55df3028b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.741+0000 7f0d0fa00640 1 --2- 172.21.15.7:0/174009762 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55df301fcda0 0x55df302804e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.741+0000 7f0d10e00640 1 -- 172.21.15.7:0/174009762 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55df301fcda0 msgr2=0x55df302804e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.741+0000 7f0d10e00640 1 --2- 172.21.15.7:0/174009762 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55df301fcda0 0x55df302804e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.741+0000 7f0d10e00640 1 -- 172.21.15.7:0/174009762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df301e3920 msgr2=0x55df3028b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.741+0000 7f0d10e00640 1 --2- 172.21.15.7:0/174009762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55df301e3920 0x55df3028b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.741+0000 7f0d10e00640 1 -- 172.21.15.7:0/174009762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df30280be0 con 0x55df30284b90 2024-09-10T21:43:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.741+0000 7f0d0fa00640 1 --2- 172.21.15.7:0/174009762 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55df301fcda0 0x55df302804e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:43:42.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.741+0000 7f0d10e00640 1 --2- 172.21.15.7:0/174009762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df30284b90 0x55df3027cda0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800d640 tx=0x7f0d0800db00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.741+0000 7f0d0d200640 1 -- 172.21.15.7:0/174009762 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d08010970 con 0x55df30284b90 2024-09-10T21:43:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.741+0000 7f0d0d200640 1 -- 172.21.15.7:0/174009762 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d08014070 con 0x55df30284b90 2024-09-10T21:43:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.741+0000 7f0d0d200640 1 -- 172.21.15.7:0/174009762 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d08019470 con 0x55df30284b90 2024-09-10T21:43:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.741+0000 7f0d131c5f40 1 -- 172.21.15.7:0/174009762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55df30280ec0 con 0x55df30284b90 2024-09-10T21:43:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.741+0000 7f0d131c5f40 1 -- 172.21.15.7:0/174009762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df3027d2e0 con 0x55df30284b90 2024-09-10T21:43:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.743+0000 7f0d0d200640 1 -- 172.21.15.7:0/174009762 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0d0800c440 con 0x55df30284b90 2024-09-10T21:43:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.744+0000 7f0d0d200640 1 --2- 172.21.15.7:0/174009762 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0cf0076980 0x7f0cf0078e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.744+0000 7f0d0d200640 1 -- 172.21.15.7:0/174009762 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f0d080997c0 con 0x55df30284b90 2024-09-10T21:43:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.745+0000 7f0d131c5f40 1 --2- 172.21.15.7:0/174009762 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x55df302823a0 0x55df30321a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.745+0000 7f0d131c5f40 1 -- 172.21.15.7:0/174009762 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e44) -- 0x55df30321fc0 con 0x55df302823a0 2024-09-10T21:43:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.745+0000 7f0d10400640 1 --2- 172.21.15.7:0/174009762 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0cf0076980 0x7f0cf0078e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.745+0000 7f0d0fa00640 1 --2- 172.21.15.7:0/174009762 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x55df302823a0 0x55df30321a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.746+0000 7f0d10400640 1 --2- 172.21.15.7:0/174009762 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0cf0076980 0x7f0cf0078e40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0d00010410 tx=0x7f0d00001730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:42.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.746+0000 7f0d0fa00640 1 --2- 172.21.15.7:0/174009762 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x55df302823a0 0x55df30321a80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:42.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.747+0000 7f0d0fa00640 1 -- 172.21.15.7:0/174009762 <== osd.6 v2:172.21.15.133:6810/3365981160 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v44'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f0cfc05f030 con 0x55df302823a0 2024-09-10T21:43:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.747+0000 7f0d131c5f40 1 -- 172.21.15.7:0/174009762 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x55df302823a0 msgr2=0x55df30321a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.747+0000 7f0d131c5f40 1 --2- 172.21.15.7:0/174009762 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x55df302823a0 0x55df30321a80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.748+0000 7f0d131c5f40 1 -- 172.21.15.7:0/174009762 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0cf0076980 msgr2=0x7f0cf0078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.748+0000 7f0d131c5f40 1 --2- 172.21.15.7:0/174009762 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0cf0076980 0x7f0cf0078e40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0d00010410 tx=0x7f0d00001730 comp rx=0 tx=0).stop 2024-09-10T21:43:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.748+0000 7f0d131c5f40 1 -- 172.21.15.7:0/174009762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df30284b90 msgr2=0x55df3027cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.748+0000 7f0d131c5f40 1 --2- 172.21.15.7:0/174009762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55df30284b90 0x55df3027cda0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f0d0800d640 tx=0x7f0d0800db00 comp rx=0 tx=0).stop 2024-09-10T21:43:42.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.748+0000 7f0d10e00640 1 -- 172.21.15.7:0/174009762 reap_dead start 2024-09-10T21:43:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.748+0000 7f0d131c5f40 1 -- 172.21.15.7:0/174009762 shutdown_connections 2024-09-10T21:43:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.748+0000 7f0d131c5f40 1 -- 172.21.15.7:0/174009762 >> 172.21.15.7:0/174009762 conn(0x55df301fddf0 msgr2=0x55df301dd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.748+0000 7f0d131c5f40 1 -- 172.21.15.7:0/174009762 shutdown_connections 2024-09-10T21:43:42.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.748+0000 7f0d131c5f40 1 -- 172.21.15.7:0/174009762 wait complete. 2024-09-10T21:43:42.761 DEBUG:teuthology.orchestra.run.smithi007:> sudo daemon-helper kill ceph --cluster ceph -w 2024-09-10T21:43:42.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:42.766 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2024-09-10T21:43:42.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.931+0000 7f04ef200640 1 -- 172.21.15.7:0/1286608911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04f015bc10 msgr2=0x7f04f006fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.931+0000 7f04ef200640 1 --2- 172.21.15.7:0/1286608911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04f015bc10 0x7f04f006fad0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f04dc005fd0 tx=0x7f04dc02a5d0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.931+0000 7f04ef200640 1 -- 172.21.15.7:0/1286608911 shutdown_connections 2024-09-10T21:43:42.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.931+0000 7f04ef200640 1 --2- 172.21.15.7:0/1286608911 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04f015bc10 0x7f04f006fad0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.931+0000 7f04ef200640 1 --2- 172.21.15.7:0/1286608911 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04f0154e30 0x7f04f006f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.931+0000 7f04ef200640 1 -- 172.21.15.7:0/1286608911 >> 172.21.15.7:0/1286608911 conn(0x7f04f00b1580 msgr2=0x7f04f00b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:42.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.931+0000 7f04ef200640 1 -- 172.21.15.7:0/1286608911 shutdown_connections 2024-09-10T21:43:42.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.931+0000 7f04ef200640 1 -- 172.21.15.7:0/1286608911 wait complete. 2024-09-10T21:43:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.933+0000 7f04ef200640 1 Processor -- start 2024-09-10T21:43:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.933+0000 7f04ef200640 1 -- start start 2024-09-10T21:43:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.934+0000 7f04ef200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04f0154e30 0x7f04f015dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.934+0000 7f04ef200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04f015bc10 0x7f04f00b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.934+0000 7f04ef200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04f015e510 0x7f04f00b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.934+0000 7f04ef200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f04f00ae9f0 con 0x7f04f0154e30 2024-09-10T21:43:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.934+0000 7f04ef200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f04f00ae870 con 0x7f04f015e510 2024-09-10T21:43:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.934+0000 7f04ef200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f04f00aeb70 con 0x7f04f015bc10 2024-09-10T21:43:42.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.934+0000 7f04ede00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04f0154e30 0x7f04f015dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.935+0000 7f04ede00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04f0154e30 0x7f04f015dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44262/0 (socket says 172.21.15.7:44262) 2024-09-10T21:43:42.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.935+0000 7f04ede00640 1 -- 172.21.15.7:0/1311215424 learned_addr learned my addr 172.21.15.7:0/1311215424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:42.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.935+0000 7f04ed400640 1 --2- 172.21.15.7:0/1311215424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04f015bc10 0x7f04f00b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.935+0000 7f04ee800640 1 --2- 172.21.15.7:0/1311215424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04f015e510 0x7f04f00b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.935+0000 7f04ede00640 1 -- 172.21.15.7:0/1311215424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04f015bc10 msgr2=0x7f04f00b2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.935+0000 7f04ede00640 1 --2- 172.21.15.7:0/1311215424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04f015bc10 0x7f04f00b2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.935+0000 7f04ede00640 1 -- 172.21.15.7:0/1311215424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04f015e510 msgr2=0x7f04f00b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.935+0000 7f04ede00640 1 --2- 172.21.15.7:0/1311215424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04f015e510 0x7f04f00b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.935+0000 7f04ede00640 1 -- 172.21.15.7:0/1311215424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04f00afa40 con 0x7f04f0154e30 2024-09-10T21:43:42.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.935+0000 7f04ede00640 1 --2- 172.21.15.7:0/1311215424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04f0154e30 0x7f04f015dc60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f04e000ea60 tx=0x7f04e000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:42.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.936+0000 7f04e6000640 1 -- 172.21.15.7:0/1311215424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e000cde0 con 0x7f04f0154e30 2024-09-10T21:43:42.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.936+0000 7f04e6000640 1 -- 172.21.15.7:0/1311215424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04e0004020 con 0x7f04f0154e30 2024-09-10T21:43:42.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.936+0000 7f04ef200640 1 -- 172.21.15.7:0/1311215424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04f00afd20 con 0x7f04f0154e30 2024-09-10T21:43:42.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.936+0000 7f04e6000640 1 -- 172.21.15.7:0/1311215424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0004450 con 0x7f04f0154e30 2024-09-10T21:43:42.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.936+0000 7f04ef200640 1 -- 172.21.15.7:0/1311215424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04f00b02e0 con 0x7f04f0154e30 2024-09-10T21:43:42.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.939+0000 7f04e6000640 1 -- 172.21.15.7:0/1311215424 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f04e00035f0 con 0x7f04f0154e30 2024-09-10T21:43:42.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.940+0000 7f04f7a24740 1 Processor -- start 2024-09-10T21:43:42.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.940+0000 7f04f7a24740 1 -- start start 2024-09-10T21:43:42.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.940+0000 7f04e6000640 1 --2- 172.21.15.7:0/1311215424 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f04d0077f70 0x7f04d007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.941+0000 7f04e6000640 1 -- 172.21.15.7:0/1311215424 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f04e009a610 con 0x7f04f0154e30 2024-09-10T21:43:42.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.941+0000 7f04f7a24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee39cf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.941+0000 7f04f7a24740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee398a070 0x55eee39cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.941+0000 7f04f7a24740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee398a5f0 0x55eee39cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.941+0000 7f04f7a24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55eee38fbf40 con 0x55eee39cef00 2024-09-10T21:43:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.941+0000 7f04f7a24740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55eee38eb370 con 0x55eee398a070 2024-09-10T21:43:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.941+0000 7f04f7a24740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55eee38ef180 con 0x55eee398a5f0 2024-09-10T21:43:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.941+0000 7f04ede00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee39cf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.941+0000 7f04ede00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee39cf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44274/0 (socket says 172.21.15.7:44274) 2024-09-10T21:43:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.941+0000 7f04ede00640 1 -- 172.21.15.7:0/734748801 learned_addr learned my addr 172.21.15.7:0/734748801 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.941+0000 7f04ee800640 1 --2- 172.21.15.7:0/1311215424 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f04d0077f70 0x7f04d007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.941+0000 7f04ed400640 1 --2- 172.21.15.7:0/734748801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee398a070 0x55eee39cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.941+0000 7f04ee800640 1 --2- 172.21.15.7:0/734748801 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee398a5f0 0x55eee39cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.941+0000 7f04ede00640 1 -- 172.21.15.7:0/734748801 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee398a5f0 msgr2=0x55eee39cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.941+0000 7f04ede00640 1 --2- 172.21.15.7:0/734748801 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee398a5f0 0x55eee39cdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.941+0000 7f04ede00640 1 -- 172.21.15.7:0/734748801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee398a070 msgr2=0x55eee39cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.941+0000 7f04ede00640 1 --2- 172.21.15.7:0/734748801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee398a070 0x55eee39cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.941+0000 7f04ede00640 1 -- 172.21.15.7:0/734748801 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee35dd470 con 0x55eee39cef00 2024-09-10T21:43:42.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.941+0000 7f04ee800640 1 --2- 172.21.15.7:0/734748801 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee398a5f0 0x55eee39cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:42.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.942+0000 7f04ede00640 1 --2- 172.21.15.7:0/734748801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee39cf2e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f04e0062a40 tx=0x7f04e005f230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545489e122ff5b23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:42.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.942+0000 7f04ee800640 1 --2- 172.21.15.7:0/1311215424 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f04d0077f70 0x7f04d007a430 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f04e8007ea0 tx=0x7f04e8007df0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:42.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.942+0000 7f04da600640 1 -- 172.21.15.7:0/734748801 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e0088070 con 0x55eee39cef00 2024-09-10T21:43:42.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.942+0000 7f04da600640 1 -- 172.21.15.7:0/734748801 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04e0062ab0 con 0x55eee39cef00 2024-09-10T21:43:42.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.942+0000 7f04da600640 1 -- 172.21.15.7:0/734748801 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e00674e0 con 0x55eee39cef00 2024-09-10T21:43:42.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.943+0000 7f04f7a24740 1 -- 172.21.15.7:0/734748801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 msgr2=0x55eee39cf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.943+0000 7f04f7a24740 1 --2- 172.21.15.7:0/734748801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee39cf2e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f04e0062a40 tx=0x7f04e005f230 comp rx=0 tx=0).stop 2024-09-10T21:43:42.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.943+0000 7f04f7a24740 1 -- 172.21.15.7:0/734748801 shutdown_connections 2024-09-10T21:43:42.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.943+0000 7f04f7a24740 1 --2- 172.21.15.7:0/734748801 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee398a070 0x55eee39cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.943+0000 7f04f7a24740 1 --2- 172.21.15.7:0/734748801 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee398a5f0 0x55eee39cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.943+0000 7f04f7a24740 1 --2- 172.21.15.7:0/734748801 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee39cf2e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.943+0000 7f04f7a24740 1 -- 172.21.15.7:0/734748801 >> 172.21.15.7:0/734748801 conn(0x55eee3985640 msgr2=0x55eee3989060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.943+0000 7f04f7a24740 1 -- 172.21.15.7:0/734748801 shutdown_connections 2024-09-10T21:43:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.943+0000 7f04f7a24740 1 -- 172.21.15.7:0/734748801 wait complete. 2024-09-10T21:43:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.944+0000 7f04f7a24740 1 -- start start 2024-09-10T21:43:42.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.944+0000 7f04f7a24740 1 Processor -- start 2024-09-10T21:43:42.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.946+0000 7f04f7a24740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee398a5f0 0x55eee39e1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.946+0000 7f04f7a24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee39c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.946+0000 7f04f7a24740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee39ad840 0x55eee39de250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.946+0000 7f04f7a24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55eee365b7c0 con 0x55eee39cef00 2024-09-10T21:43:42.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.946+0000 7f04f7a24740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55eee3890ae0 con 0x55eee39ad840 2024-09-10T21:43:42.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.946+0000 7f04f7a24740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55eee3984060 con 0x55eee398a5f0 2024-09-10T21:43:42.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.946+0000 7f04f7a24740 10 client.0 _ll_register_callbacks cb 0x55eee38a05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:43:42.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.946+0000 7f04ed400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee39c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.946+0000 7f04ed400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee39c4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44292/0 (socket says 172.21.15.7:44292) 2024-09-10T21:43:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.946+0000 7f04ed400640 1 -- 172.21.15.7:0/526570270 learned_addr learned my addr 172.21.15.7:0/526570270 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.946+0000 7f04ede00640 1 --2- 172.21.15.7:0/526570270 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee398a5f0 0x55eee39e1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.946+0000 7f04ed400640 1 -- 172.21.15.7:0/526570270 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee398a5f0 msgr2=0x55eee39e1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.946+0000 7f04ee800640 1 --2- 172.21.15.7:0/526570270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee39ad840 0x55eee39de250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.946+0000 7f04ed400640 1 --2- 172.21.15.7:0/526570270 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee398a5f0 0x55eee39e1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.946+0000 7f04ed400640 1 -- 172.21.15.7:0/526570270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee39ad840 msgr2=0x55eee39de250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.946+0000 7f04ed400640 1 --2- 172.21.15.7:0/526570270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee39ad840 0x55eee39de250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.946+0000 7f04ed400640 1 -- 172.21.15.7:0/526570270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eee36c17e0 con 0x55eee39cef00 2024-09-10T21:43:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.947+0000 7f04ede00640 1 --2- 172.21.15.7:0/526570270 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee398a5f0 0x55eee39e1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:43:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.947+0000 7f04ee800640 1 --2- 172.21.15.7:0/526570270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee39ad840 0x55eee39de250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:43:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.947+0000 7f04ed400640 1 --2- 172.21.15.7:0/526570270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee39c4c70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f04dc000c00 tx=0x7f04dc0041e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a888dc94c1c5c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:42.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.947+0000 7f04cea00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:43:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.947+0000 7f04cea00640 1 -- 172.21.15.7:0/526570270 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04dc035040 con 0x55eee39cef00 2024-09-10T21:43:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.947+0000 7f04f7a24740 1 -- 172.21.15.7:0/526570270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04e0062ab0 con 0x55eee39cef00 2024-09-10T21:43:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.947+0000 7f04cea00640 1 -- 172.21.15.7:0/526570270 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f04dc005220 con 0x55eee39cef00 2024-09-10T21:43:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.947+0000 7f04cea00640 1 -- 172.21.15.7:0/526570270 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f04dc04d050 con 0x55eee39cef00 2024-09-10T21:43:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.947+0000 7f04f7a24740 10 client.5472 fetch_fsmap learned FSMap version 26 2024-09-10T21:43:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.947+0000 7f04f7a24740 1 -- 172.21.15.7:0/526570270 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55eee3721f70 con 0x55eee39cef00 2024-09-10T21:43:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.948+0000 7f04cea00640 1 -- 172.21.15.7:0/526570270 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04e0062ab0 con 0x55eee39cef00 2024-09-10T21:43:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.948+0000 7f04cea00640 1 -- 172.21.15.7:0/526570270 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f04dc050020 con 0x55eee39cef00 2024-09-10T21:43:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.948+0000 7f04f7a24740 10 client.5472 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T21:43:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.948+0000 7f04f7a24740 10 client.5472 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-09-10T21:43:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.948+0000 7f04f7a24740 20 client.5472 populate_metadata read hostname 'smithi007' 2024-09-10T21:43:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.948+0000 7f04f7a24740 1 --2- 172.21.15.7:0/526570270 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55eee39de790 0x55eee39ca3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.948+0000 7f04f7a24740 4 client.5472 mds_command: new command op to 234767 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:43:42.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.948+0000 7f04f7a24740 1 -- 172.21.15.7:0/526570270 --> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eee38fde90 con 0x55eee39de790 2024-09-10T21:43:42.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.948+0000 7f04ede00640 1 --2- 172.21.15.7:0/526570270 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55eee39de790 0x55eee39ca3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.949+0000 7f04ede00640 1 --2- 172.21.15.7:0/526570270 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55eee39de790 0x55eee39ca3b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=394acf91d7d87445 server_cookie=996962c6460489c9 in_seq=0 out_seq=0 2024-09-10T21:43:42.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.949+0000 7f04cea00640 10 client.5472 ms_handle_connect on v2:172.21.15.133:6834/3385628197 2024-09-10T21:43:42.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.950+0000 7f04cea00640 1 -- 172.21.15.7:0/526570270 <== mds.0 v2:172.21.15.133:6834/3385628197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55eee38fde90 con 0x55eee39de790 2024-09-10T21:43:42.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.950+0000 7f04cea00640 10 client.5472 handle_command_reply: tid=0 2024-09-10T21:43:42.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.950+0000 7f04f7a24740 1 client.5472 shutdown 2024-09-10T21:43:42.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.950+0000 7f04f7a24740 1 -- 172.21.15.7:0/526570270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 msgr2=0x55eee39c4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.950+0000 7f04f7a24740 1 --2- 172.21.15.7:0/526570270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee39c4c70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f04dc000c00 tx=0x7f04dc0041e0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.950+0000 7f04f7a24740 1 -- 172.21.15.7:0/526570270 shutdown_connections 2024-09-10T21:43:42.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.950+0000 7f04f7a24740 1 --2- 172.21.15.7:0/526570270 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55eee39de790 0x55eee39ca3b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.950+0000 7f04f7a24740 1 --2- 172.21.15.7:0/526570270 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee39ad840 0x55eee39de250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.950+0000 7f04f7a24740 1 --2- 172.21.15.7:0/526570270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee39c4c70 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.950+0000 7f04f7a24740 1 --2- 172.21.15.7:0/526570270 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee398a5f0 0x55eee39e1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.950+0000 7f04f7a24740 1 -- 172.21.15.7:0/526570270 >> 172.21.15.7:0/526570270 conn(0x55eee398a070 msgr2=0x55eee3982490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:42.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.950+0000 7f04cea00640 0 client.5472 ms_handle_reset on v2:172.21.15.133:6834/3385628197 2024-09-10T21:43:42.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.950+0000 7f04f7a24740 1 -- 172.21.15.7:0/526570270 shutdown_connections 2024-09-10T21:43:42.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.950+0000 7f04f7a24740 1 -- 172.21.15.7:0/526570270 wait complete. 2024-09-10T21:43:42.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.951+0000 7f04f7a24740 20 client.5472 trim_cache size 0 max 16384 2024-09-10T21:43:42.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.958+0000 7f418ec00640 1 -- 172.21.15.7:0/3149111657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41880744c0 msgr2=0x7f418815c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.958+0000 7f418ec00640 1 --2- 172.21.15.7:0/3149111657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41880744c0 0x7f418815c0f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f4180005fd0 tx=0x7f418002a820 comp rx=0 tx=0).stop 2024-09-10T21:43:42.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.958+0000 7f418ec00640 1 -- 172.21.15.7:0/3149111657 shutdown_connections 2024-09-10T21:43:42.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.958+0000 7f418ec00640 1 --2- 172.21.15.7:0/3149111657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41880744c0 0x7f418815c0f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.958+0000 7f418ec00640 1 --2- 172.21.15.7:0/3149111657 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4188152bc0 0x7f418806fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.958+0000 7f418ec00640 1 -- 172.21.15.7:0/3149111657 >> 172.21.15.7:0/3149111657 conn(0x7f41880b1740 msgr2=0x7f41880b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:42.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.958+0000 7f418ec00640 1 -- 172.21.15.7:0/3149111657 shutdown_connections 2024-09-10T21:43:42.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.958+0000 7f418ec00640 1 -- 172.21.15.7:0/3149111657 wait complete. 2024-09-10T21:43:42.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.961+0000 7f418ec00640 1 Processor -- start 2024-09-10T21:43:42.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.961+0000 7f418ec00640 1 -- start start 2024-09-10T21:43:42.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.963+0000 7f418ec00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4188070070 0x7f41881efc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.963+0000 7f418ec00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41880744c0 0x7f41880c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.963+0000 7f418ec00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4188152bc0 0x7f41880ca680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.963+0000 7f418ec00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f418815c630 con 0x7f4188070070 2024-09-10T21:43:42.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.963+0000 7f418ec00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f41880cbce0 con 0x7f4188152bc0 2024-09-10T21:43:42.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.963+0000 7f418ec00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f418815c7b0 con 0x7f41880744c0 2024-09-10T21:43:42.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.963+0000 7f4187e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4188070070 0x7f41881efc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.963+0000 7f4187e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4188070070 0x7f41881efc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44296/0 (socket says 172.21.15.7:44296) 2024-09-10T21:43:42.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.963+0000 7f4187e00640 1 -- 172.21.15.7:0/4040576676 learned_addr learned my addr 172.21.15.7:0/4040576676 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:42.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.963+0000 7f4187400640 1 --2- 172.21.15.7:0/4040576676 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41880744c0 0x7f41880c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.963+0000 7f418cc00640 1 --2- 172.21.15.7:0/4040576676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4188152bc0 0x7f41880ca680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.964+0000 7f4187e00640 1 -- 172.21.15.7:0/4040576676 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41880744c0 msgr2=0x7f41880c7780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.964+0000 7f4187e00640 1 --2- 172.21.15.7:0/4040576676 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41880744c0 0x7f41880c7780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.964+0000 7f4187e00640 1 -- 172.21.15.7:0/4040576676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4188152bc0 msgr2=0x7f41880ca680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.964+0000 7f4187e00640 1 --2- 172.21.15.7:0/4040576676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4188152bc0 0x7f41880ca680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.964+0000 7f4187e00640 1 -- 172.21.15.7:0/4040576676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4188150920 con 0x7f4188070070 2024-09-10T21:43:42.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.964+0000 7f4187e00640 1 --2- 172.21.15.7:0/4040576676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4188070070 0x7f41881efc40 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f417800e790 tx=0x7f417800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:42.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.964+0000 7f417fe00640 1 -- 172.21.15.7:0/4040576676 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417800eea0 con 0x7f4188070070 2024-09-10T21:43:42.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.964+0000 7f417fe00640 1 -- 172.21.15.7:0/4040576676 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f417800cdb0 con 0x7f4188070070 2024-09-10T21:43:42.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.965+0000 7f417fe00640 1 -- 172.21.15.7:0/4040576676 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4178010690 con 0x7f4188070070 2024-09-10T21:43:42.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.965+0000 7f418ec00640 1 -- 172.21.15.7:0/4040576676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41880aa230 con 0x7f4188070070 2024-09-10T21:43:42.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.965+0000 7f418ec00640 1 -- 172.21.15.7:0/4040576676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41880c6ca0 con 0x7f4188070070 2024-09-10T21:43:42.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.965+0000 7f418ec00640 1 -- 172.21.15.7:0/4040576676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f418815c0f0 con 0x7f4188070070 2024-09-10T21:43:42.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.967+0000 7f417fe00640 1 -- 172.21.15.7:0/4040576676 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4178010830 con 0x7f4188070070 2024-09-10T21:43:42.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.969+0000 7f417fe00640 1 --2- 172.21.15.7:0/4040576676 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4168077f70 0x7f416807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.969+0000 7f417fe00640 1 -- 172.21.15.7:0/4040576676 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f417809b2d0 con 0x7f4188070070 2024-09-10T21:43:42.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.969+0000 7f417fe00640 1 -- 172.21.15.7:0/4040576676 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+509 (secure 0 0 0) 0x7f4188150920 con 0x7f4188070070 2024-09-10T21:43:42.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.969+0000 7f4187400640 1 --2- 172.21.15.7:0/4040576676 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4168077f70 0x7f416807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.969+0000 7f418ec00640 1 -- 172.21.15.7:0/4040576676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7f41880b1ea0 con 0x7f4188070070 2024-09-10T21:43:42.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.970+0000 7f4187400640 1 --2- 172.21.15.7:0/4040576676 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4168077f70 0x7f416807a430 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f41881f6c90 tx=0x7f4174007e90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:42.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.975+0000 7f04f7a24740 1 Processor -- start 2024-09-10T21:43:42.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.975+0000 7f04f7a24740 1 -- start start 2024-09-10T21:43:42.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.975+0000 7f04f7a24740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee398a5f0 0x55eee39e0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.975+0000 7f04f7a24740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee39ad840 0x55eee3982490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.975+0000 7f04f7a24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee39829d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.975+0000 7f04f7a24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f04f0200da0 con 0x55eee39cef00 2024-09-10T21:43:42.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.975+0000 7f04f7a24740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f04f01ffda0 con 0x55eee398a5f0 2024-09-10T21:43:42.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.975+0000 7f04f7a24740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f04f01ff100 con 0x55eee39ad840 2024-09-10T21:43:42.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.976+0000 7f04ee800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee39829d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.976+0000 7f04ee800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee39829d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44300/0 (socket says 172.21.15.7:44300) 2024-09-10T21:43:42.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.976+0000 7f04ee800640 1 -- 172.21.15.7:0/2073658360 learned_addr learned my addr 172.21.15.7:0/2073658360 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:42.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.976+0000 7f04ed400640 1 --2- 172.21.15.7:0/2073658360 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee39ad840 0x55eee3982490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.976+0000 7f04ede00640 1 --2- 172.21.15.7:0/2073658360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee398a5f0 0x55eee39e0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.976+0000 7f04ee800640 1 -- 172.21.15.7:0/2073658360 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee39ad840 msgr2=0x55eee3982490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:42.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.976+0000 7f04ee800640 1 --2- 172.21.15.7:0/2073658360 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee39ad840 0x55eee3982490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.976+0000 7f04ee800640 1 -- 172.21.15.7:0/2073658360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee398a5f0 msgr2=0x55eee39e0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:42.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.976+0000 7f04ee800640 1 --2- 172.21.15.7:0/2073658360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee398a5f0 0x55eee39e0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.976+0000 7f04ee800640 1 -- 172.21.15.7:0/2073658360 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee362a720 con 0x55eee39cef00 2024-09-10T21:43:42.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.976+0000 7f04ede00640 1 --2- 172.21.15.7:0/2073658360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee398a5f0 0x55eee39e0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:43:42.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.977+0000 7f04ee800640 1 --2- 172.21.15.7:0/2073658360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee39829d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f04e8007d50 tx=0x7f04e80015b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d0057fa8f4e2ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:42.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.977+0000 7f04d9c00640 1 -- 172.21.15.7:0/2073658360 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8010040 con 0x55eee39cef00 2024-09-10T21:43:42.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.977+0000 7f04d9c00640 1 -- 172.21.15.7:0/2073658360 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04e8001860 con 0x55eee39cef00 2024-09-10T21:43:42.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.977+0000 7f04d9c00640 1 -- 172.21.15.7:0/2073658360 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e80057b0 con 0x55eee39cef00 2024-09-10T21:43:42.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.978+0000 7f04f7a24740 1 -- 172.21.15.7:0/2073658360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 msgr2=0x55eee39829d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.978+0000 7f04f7a24740 1 --2- 172.21.15.7:0/2073658360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee39829d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f04e8007d50 tx=0x7f04e80015b0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.978+0000 7f04f7a24740 1 -- 172.21.15.7:0/2073658360 shutdown_connections 2024-09-10T21:43:42.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.978+0000 7f04f7a24740 1 --2- 172.21.15.7:0/2073658360 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee39829d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.978+0000 7f04f7a24740 1 --2- 172.21.15.7:0/2073658360 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee39ad840 0x55eee3982490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.978+0000 7f04f7a24740 1 --2- 172.21.15.7:0/2073658360 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee398a5f0 0x55eee39e0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.978+0000 7f04f7a24740 1 -- 172.21.15.7:0/2073658360 >> 172.21.15.7:0/2073658360 conn(0x55eee398a070 msgr2=0x55eee39e1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:42.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.978+0000 7f04f7a24740 1 -- 172.21.15.7:0/2073658360 shutdown_connections 2024-09-10T21:43:42.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.978+0000 7f04f7a24740 1 -- 172.21.15.7:0/2073658360 wait complete. 2024-09-10T21:43:42.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.978+0000 7f04f7a24740 1 -- start start 2024-09-10T21:43:42.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.979+0000 7f04f7a24740 1 Processor -- start 2024-09-10T21:43:42.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.979+0000 7f04f7a24740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee398a5f0 0x55eee3982490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.979+0000 7f04f7a24740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee39ad840 0x55eee3a4fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.979+0000 7f04f7a24740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee3986ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.980+0000 7f04f7a24740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55eee3988530 con 0x55eee39cef00 2024-09-10T21:43:42.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.980+0000 7f04f7a24740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f04f01ff750 con 0x55eee398a5f0 2024-09-10T21:43:42.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.980+0000 7f04f7a24740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55eee39acae0 con 0x55eee39ad840 2024-09-10T21:43:42.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.980+0000 7f04f7a24740 10 client.0 _ll_register_callbacks cb 0x7f04dc009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:43:42.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.980+0000 7f04ee800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee3986ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.980+0000 7f04ee800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee3986ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44310/0 (socket says 172.21.15.7:44310) 2024-09-10T21:43:42.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.980+0000 7f04ee800640 1 -- 172.21.15.7:0/1001107703 learned_addr learned my addr 172.21.15.7:0/1001107703 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:42.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.980+0000 7f04ede00640 1 --2- 172.21.15.7:0/1001107703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee398a5f0 0x55eee3982490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.980+0000 7f04ed400640 1 --2- 172.21.15.7:0/1001107703 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee39ad840 0x55eee3a4fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.980+0000 7f04ee800640 1 -- 172.21.15.7:0/1001107703 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee39ad840 msgr2=0x55eee3a4fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:42.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.980+0000 7f04ee800640 1 --2- 172.21.15.7:0/1001107703 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee39ad840 0x55eee3a4fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.980+0000 7f04ee800640 1 -- 172.21.15.7:0/1001107703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee398a5f0 msgr2=0x55eee3982490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:42.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.980+0000 7f04ee800640 1 --2- 172.21.15.7:0/1001107703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee398a5f0 0x55eee3982490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.980+0000 7f04ee800640 1 -- 172.21.15.7:0/1001107703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eee3a52b90 con 0x55eee39cef00 2024-09-10T21:43:42.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.980+0000 7f04ed400640 1 --2- 172.21.15.7:0/1001107703 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee39ad840 0x55eee3a4fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:43:42.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.980+0000 7f04ede00640 1 --2- 172.21.15.7:0/1001107703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee398a5f0 0x55eee3982490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:43:42.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.980+0000 7f04ee800640 1 --2- 172.21.15.7:0/1001107703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee3986ac0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f04e801a000 tx=0x7f04e8019700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48fe2baba9d2b38b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:42.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.980+0000 7f04cea00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:43:42.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.981+0000 7f04cea00640 1 -- 172.21.15.7:0/1001107703 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04e8019b00 con 0x55eee39cef00 2024-09-10T21:43:42.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.981+0000 7f04cea00640 1 -- 172.21.15.7:0/1001107703 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f04e8019ca0 con 0x55eee39cef00 2024-09-10T21:43:42.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.981+0000 7f04f7a24740 1 -- 172.21.15.7:0/1001107703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04e8001860 con 0x55eee39cef00 2024-09-10T21:43:42.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.981+0000 7f04cea00640 1 -- 172.21.15.7:0/1001107703 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f04e8034050 con 0x55eee39cef00 2024-09-10T21:43:42.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.981+0000 7f04f7a24740 10 client.5481 fetch_fsmap learned FSMap version 26 2024-09-10T21:43:42.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.981+0000 7f04f7a24740 1 -- 172.21.15.7:0/1001107703 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55eee3a4f570 con 0x55eee39cef00 2024-09-10T21:43:42.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.981+0000 7f04cea00640 1 -- 172.21.15.7:0/1001107703 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04e8001860 con 0x55eee39cef00 2024-09-10T21:43:42.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.981+0000 7f04cea00640 1 -- 172.21.15.7:0/1001107703 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f04e8037020 con 0x55eee39cef00 2024-09-10T21:43:42.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.981+0000 7f04f7a24740 10 client.5481 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T21:43:42.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.981+0000 7f04f7a24740 10 client.5481 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-09-10T21:43:42.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.981+0000 7f04f7a24740 20 client.5481 populate_metadata read hostname 'smithi007' 2024-09-10T21:43:42.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.981+0000 7f04f7a24740 1 --2- 172.21.15.7:0/1001107703 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55eee3987000 0x55eee39c8210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.981+0000 7f04f7a24740 4 client.5481 mds_command: new command op to 234767 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-09-10T21:43:42.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.981+0000 7f04f7a24740 1 -- 172.21.15.7:0/1001107703 --> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55eee38f9940 con 0x55eee3987000 2024-09-10T21:43:42.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.982+0000 7f04ede00640 1 --2- 172.21.15.7:0/1001107703 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55eee3987000 0x55eee39c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.982+0000 7f04ede00640 1 --2- 172.21.15.7:0/1001107703 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55eee3987000 0x55eee39c8210 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81400997a16d9c6a server_cookie=6b05a324521e4a6d in_seq=0 out_seq=0 2024-09-10T21:43:42.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.982+0000 7f04cea00640 10 client.5481 ms_handle_connect on v2:172.21.15.133:6834/3385628197 2024-09-10T21:43:42.989 INFO:tasks.ceph.mds.d.smithi133.stderr:2024-09-10T21:43:42.984+0000 7f3525c00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 pv4 ap=4 DIRTYPARENT s=0 n(v0 1=1+0)->n(v0 rc2024-09-10T21:43:41.612224+0000 1=1+0) (iauth excl) (ifile lock->excl w=1) (ixattr excl) (iquiesce lock w=1) cr={234848=0-4194304@1} caps={234848=pAsxLsXsxFcb/pAsxXsxFxwb@3},l=234848 | ptrwaiter=1 request=1 lock=3 caps=1 dirtyparent=1 dirty=0 waiter=1 authpin=1 scrubqueue=0 0x55cf8593cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-10T21:43:42.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.983+0000 7f04cea00640 1 -- 172.21.15.7:0/1001107703 <== mds.0 v2:172.21.15.133:6834/3385628197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55eee38f9940 con 0x55eee3987000 2024-09-10T21:43:42.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.983+0000 7f04cea00640 10 client.5481 handle_command_reply: tid=0 2024-09-10T21:43:42.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.983+0000 7f04f7a24740 1 client.5481 shutdown 2024-09-10T21:43:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.983+0000 7f04f7a24740 1 -- 172.21.15.7:0/1001107703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 msgr2=0x55eee3986ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.983+0000 7f04f7a24740 1 --2- 172.21.15.7:0/1001107703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee3986ac0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f04e801a000 tx=0x7f04e8019700 comp rx=0 tx=0).stop 2024-09-10T21:43:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.983+0000 7f04f7a24740 1 -- 172.21.15.7:0/1001107703 shutdown_connections 2024-09-10T21:43:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.983+0000 7f04f7a24740 1 --2- 172.21.15.7:0/1001107703 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55eee3987000 0x55eee39c8210 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.983+0000 7f04f7a24740 1 --2- 172.21.15.7:0/1001107703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55eee39cef00 0x55eee3986ac0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.983+0000 7f04f7a24740 1 --2- 172.21.15.7:0/1001107703 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55eee39ad840 0x55eee3a4fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.983+0000 7f04f7a24740 1 --2- 172.21.15.7:0/1001107703 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55eee398a5f0 0x55eee3982490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.983+0000 7f04f7a24740 1 -- 172.21.15.7:0/1001107703 >> 172.21.15.7:0/1001107703 conn(0x55eee398a070 msgr2=0x55eee39c4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.983+0000 7f04cea00640 0 client.5481 ms_handle_reset on v2:172.21.15.133:6834/3385628197 2024-09-10T21:43:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.984+0000 7f04f7a24740 1 -- 172.21.15.7:0/1001107703 shutdown_connections 2024-09-10T21:43:42.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.984+0000 7f04f7a24740 1 -- 172.21.15.7:0/1001107703 wait complete. 2024-09-10T21:43:42.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.984+0000 7f04f7a24740 20 client.5481 trim_cache size 0 max 16384 2024-09-10T21:43:42.992 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:43:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-09-10T21:43:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "0c8f9dbc-d094-4ce0-98e2-5f3ee7a2ce53", 2024-09-10T21:43:42.992 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-09-10T21:43:42.992 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:43:42.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.984+0000 7f04ef200640 1 -- 172.21.15.7:0/1311215424 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f04d0077f70 msgr2=0x7f04d007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.984+0000 7f04ef200640 1 --2- 172.21.15.7:0/1311215424 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f04d0077f70 0x7f04d007a430 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f04e8007ea0 tx=0x7f04e8007df0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.984+0000 7f04ef200640 1 -- 172.21.15.7:0/1311215424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04f0154e30 msgr2=0x7f04f015dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.984+0000 7f04ef200640 1 --2- 172.21.15.7:0/1311215424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04f0154e30 0x7f04f015dc60 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f04e000ea60 tx=0x7f04e000ef20 comp rx=0 tx=0).stop 2024-09-10T21:43:42.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.985+0000 7f04ef200640 1 -- 172.21.15.7:0/1311215424 shutdown_connections 2024-09-10T21:43:42.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.985+0000 7f04ef200640 1 --2- 172.21.15.7:0/1311215424 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f04d0077f70 0x7f04d007a430 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.985+0000 7f04ef200640 1 --2- 172.21.15.7:0/1311215424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04f015e510 0x7f04f00b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.985+0000 7f04ef200640 1 --2- 172.21.15.7:0/1311215424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f04f015bc10 0x7f04f00b2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.985+0000 7f04ef200640 1 --2- 172.21.15.7:0/1311215424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f04f0154e30 0x7f04f015dc60 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.985+0000 7f04ef200640 1 -- 172.21.15.7:0/1311215424 >> 172.21.15.7:0/1311215424 conn(0x7f04f00b1580 msgr2=0x7f04f015e930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:42.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.985+0000 7f04ef200640 1 -- 172.21.15.7:0/1311215424 shutdown_connections 2024-09-10T21:43:42.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:42.985+0000 7f04ef200640 1 -- 172.21.15.7:0/1311215424 wait complete. 2024-09-10T21:43:43.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:43:43.001 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-09-10T21:43:43.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.161+0000 7f336e000640 1 -- 172.21.15.7:0/1680071491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3368155fb0 msgr2=0x7f33681600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:43.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.161+0000 7f336e000640 1 --2- 172.21.15.7:0/1680071491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3368155fb0 0x7f33681600d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f33680cd510 tx=0x7f335802a560 comp rx=0 tx=0).stop 2024-09-10T21:43:43.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.161+0000 7f336e000640 1 -- 172.21.15.7:0/1680071491 shutdown_connections 2024-09-10T21:43:43.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.161+0000 7f336e000640 1 --2- 172.21.15.7:0/1680071491 >> v2:172.21.15.133:3300/0 conn(0x7f3368074450 0x7f33680ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.161+0000 7f336e000640 1 --2- 172.21.15.7:0/1680071491 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3368155fb0 0x7f33681600d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.161+0000 7f336e000640 1 -- 172.21.15.7:0/1680071491 >> 172.21.15.7:0/1680071491 conn(0x7f33680b16d0 msgr2=0x7f33680b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:43.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.161+0000 7f336e000640 1 -- 172.21.15.7:0/1680071491 shutdown_connections 2024-09-10T21:43:43.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.161+0000 7f336e000640 1 -- 172.21.15.7:0/1680071491 wait complete. 2024-09-10T21:43:43.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.163+0000 7f336e000640 1 Processor -- start 2024-09-10T21:43:43.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.163+0000 7f336e000640 1 -- start start 2024-09-10T21:43:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.164+0000 7f336e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336806fcf0 0x7f336814b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.164+0000 7f336e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3368074450 0x7f33680b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.164+0000 7f336e000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3368155fb0 0x7f33680c33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.164+0000 7f336e000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f33680ca930 con 0x7f336806fcf0 2024-09-10T21:43:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.164+0000 7f336e000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f33680ca7b0 con 0x7f3368074450 2024-09-10T21:43:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.164+0000 7f336e000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f33680caab0 con 0x7f3368155fb0 2024-09-10T21:43:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.164+0000 7f3367400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336806fcf0 0x7f336814b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.164+0000 7f3367400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336806fcf0 0x7f336814b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44340/0 (socket says 172.21.15.7:44340) 2024-09-10T21:43:43.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.164+0000 7f3367400640 1 -- 172.21.15.7:0/1528346179 learned_addr learned my addr 172.21.15.7:0/1528346179 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.164+0000 7f335ea00640 1 --2- 172.21.15.7:0/1528346179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3368074450 0x7f33680b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.164+0000 7f3367e00640 1 --2- 172.21.15.7:0/1528346179 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3368155fb0 0x7f33680c33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.165+0000 7f3367400640 1 -- 172.21.15.7:0/1528346179 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3368155fb0 msgr2=0x7f33680c33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.165+0000 7f3367400640 1 --2- 172.21.15.7:0/1528346179 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3368155fb0 0x7f33680c33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.165+0000 7f3367400640 1 -- 172.21.15.7:0/1528346179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3368074450 msgr2=0x7f33680b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.165+0000 7f3367400640 1 --2- 172.21.15.7:0/1528346179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3368074450 0x7f33680b33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.165+0000 7f3367400640 1 -- 172.21.15.7:0/1528346179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f336814b740 con 0x7f336806fcf0 2024-09-10T21:43:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.165+0000 7f3367e00640 1 --2- 172.21.15.7:0/1528346179 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3368155fb0 0x7f33680c33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:43.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.165+0000 7f3367400640 1 --2- 172.21.15.7:0/1528346179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336806fcf0 0x7f336814b200 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3358034040 tx=0x7f3358003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:43.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.165+0000 7f335fe00640 1 -- 172.21.15.7:0/1528346179 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335803e030 con 0x7f336806fcf0 2024-09-10T21:43:43.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.165+0000 7f335fe00640 1 -- 172.21.15.7:0/1528346179 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3358003e00 con 0x7f336806fcf0 2024-09-10T21:43:43.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.165+0000 7f335fe00640 1 -- 172.21.15.7:0/1528346179 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3358035040 con 0x7f336806fcf0 2024-09-10T21:43:43.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.165+0000 7f336e000640 1 -- 172.21.15.7:0/1528346179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33680b3920 con 0x7f336806fcf0 2024-09-10T21:43:43.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.165+0000 7f336e000640 1 -- 172.21.15.7:0/1528346179 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33680c7070 con 0x7f336806fcf0 2024-09-10T21:43:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.168+0000 7f335fe00640 1 -- 172.21.15.7:0/1528346179 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f33580049c0 con 0x7f336806fcf0 2024-09-10T21:43:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.169+0000 7f3371024740 1 Processor -- start 2024-09-10T21:43:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.169+0000 7f335fe00640 1 --2- 172.21.15.7:0/1528346179 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3344077f70 0x7f334407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.169+0000 7f335fe00640 1 -- 172.21.15.7:0/1528346179 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f33580ba770 con 0x7f336806fcf0 2024-09-10T21:43:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.169+0000 7f3371024740 1 -- start start 2024-09-10T21:43:43.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.169+0000 7f3367e00640 1 --2- 172.21.15.7:0/1528346179 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3344077f70 0x7f334407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:43.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.170+0000 7f3371024740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65eae130 0x560d65eae510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:43.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.170+0000 7f3371024740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e692a0 0x560d65eacc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:43.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.170+0000 7f3371024740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65e69820 0x560d65ead150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.170+0000 7f3371024740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560d65dc81c0 con 0x560d65eae130 2024-09-10T21:43:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.170+0000 7f3371024740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560d65ddb870 con 0x560d65e692a0 2024-09-10T21:43:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.170+0000 7f3371024740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x560d65dcaca0 con 0x560d65e69820 2024-09-10T21:43:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.170+0000 7f3367400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65eae130 0x560d65eae510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.170+0000 7f3367e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65e69820 0x560d65ead150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.170+0000 7f335ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e692a0 0x560d65eacc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.170+0000 7f3367400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65eae130 0x560d65eae510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44348/0 (socket says 172.21.15.7:44348) 2024-09-10T21:43:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.170+0000 7f3367400640 1 -- 172.21.15.7:0/1086648546 learned_addr learned my addr 172.21.15.7:0/1086648546 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.170+0000 7f3367e00640 1 --2- 172.21.15.7:0/1528346179 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3344077f70 0x7f334407a430 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3360005fd0 tx=0x7f33600094a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.170+0000 7f3367400640 1 -- 172.21.15.7:0/1086648546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65e69820 msgr2=0x560d65ead150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.170+0000 7f3367400640 1 --2- 172.21.15.7:0/1086648546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65e69820 0x560d65ead150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.170+0000 7f3367400640 1 -- 172.21.15.7:0/1086648546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e692a0 msgr2=0x560d65eacc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.170+0000 7f3367400640 1 --2- 172.21.15.7:0/1086648546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e692a0 0x560d65eacc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.170+0000 7f3367400640 1 -- 172.21.15.7:0/1086648546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d65ba37d0 con 0x560d65eae130 2024-09-10T21:43:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.171+0000 7f3367400640 1 --2- 172.21.15.7:0/1086648546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65eae130 0x560d65eae510 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3358082b10 tx=0x7f335807f300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=710cbb95b1ad75eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.171+0000 7f3357400640 1 -- 172.21.15.7:0/1086648546 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3358048040 con 0x560d65eae130 2024-09-10T21:43:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.171+0000 7f3357400640 1 -- 172.21.15.7:0/1086648546 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33580875b0 con 0x560d65eae130 2024-09-10T21:43:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.171+0000 7f3357400640 1 -- 172.21.15.7:0/1086648546 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335808bfd0 con 0x560d65eae130 2024-09-10T21:43:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.171+0000 7f3371024740 1 -- 172.21.15.7:0/1086648546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65eae130 msgr2=0x560d65eae510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.171+0000 7f3371024740 1 --2- 172.21.15.7:0/1086648546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65eae130 0x560d65eae510 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f3358082b10 tx=0x7f335807f300 comp rx=0 tx=0).stop 2024-09-10T21:43:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.172+0000 7f3371024740 1 -- 172.21.15.7:0/1086648546 shutdown_connections 2024-09-10T21:43:43.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.172+0000 7f3371024740 1 --2- 172.21.15.7:0/1086648546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e692a0 0x560d65eacc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.172+0000 7f3371024740 1 --2- 172.21.15.7:0/1086648546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65e69820 0x560d65ead150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.172+0000 7f3371024740 1 --2- 172.21.15.7:0/1086648546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65eae130 0x560d65eae510 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.172+0000 7f3371024740 1 -- 172.21.15.7:0/1086648546 >> 172.21.15.7:0/1086648546 conn(0x560d65e64870 msgr2=0x560d65e68290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:43.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.172+0000 7f3371024740 1 -- 172.21.15.7:0/1086648546 shutdown_connections 2024-09-10T21:43:43.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.172+0000 7f3371024740 1 -- 172.21.15.7:0/1086648546 wait complete. 2024-09-10T21:43:43.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.172+0000 7f3371024740 1 -- start start 2024-09-10T21:43:43.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.173+0000 7f3371024740 1 Processor -- start 2024-09-10T21:43:43.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.174+0000 7f3371024740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e69820 0x560d65e7d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:43.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.174+0000 7f3371024740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65eae130 0x560d65eacdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:43.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.174+0000 7f3371024740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65e7cea0 0x560d65e91c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:43.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.174+0000 7f3371024740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560d65d71410 con 0x560d65e7cea0 2024-09-10T21:43:43.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.174+0000 7f3371024740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560d65dceab0 con 0x560d65e69820 2024-09-10T21:43:43.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.174+0000 7f3371024740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x560d65b3b740 con 0x560d65eae130 2024-09-10T21:43:43.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.174+0000 7f3371024740 10 client.0 _ll_register_callbacks cb 0x560d65d4e360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:43:43.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.174+0000 7f3367e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65e7cea0 0x560d65e91c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:43.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.174+0000 7f3367e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65e7cea0 0x560d65e91c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44354/0 (socket says 172.21.15.7:44354) 2024-09-10T21:43:43.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.174+0000 7f3367e00640 1 -- 172.21.15.7:0/3778615069 learned_addr learned my addr 172.21.15.7:0/3778615069 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:43.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.174+0000 7f335ea00640 1 --2- 172.21.15.7:0/3778615069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65eae130 0x560d65eacdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:43.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.174+0000 7f3367400640 1 --2- 172.21.15.7:0/3778615069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e69820 0x560d65e7d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:43.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.174+0000 7f3367e00640 1 -- 172.21.15.7:0/3778615069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65eae130 msgr2=0x560d65eacdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:43.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.174+0000 7f3367e00640 1 --2- 172.21.15.7:0/3778615069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65eae130 0x560d65eacdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.174+0000 7f3367e00640 1 -- 172.21.15.7:0/3778615069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e69820 msgr2=0x560d65e7d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:43.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.174+0000 7f3367e00640 1 --2- 172.21.15.7:0/3778615069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e69820 0x560d65e7d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.174+0000 7f3367e00640 1 -- 172.21.15.7:0/3778615069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d65b2c970 con 0x560d65e7cea0 2024-09-10T21:43:43.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.175+0000 7f3367e00640 1 --2- 172.21.15.7:0/3778615069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65e7cea0 0x560d65e91c30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3360012950 tx=0x7f3360004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c547ab16b36b7b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:43.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.175+0000 7f334be00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:43:43.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.175+0000 7f334be00640 1 -- 172.21.15.7:0/3778615069 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360011d70 con 0x560d65e7cea0 2024-09-10T21:43:43.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.175+0000 7f3371024740 1 -- 172.21.15.7:0/3778615069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33580875b0 con 0x560d65e7cea0 2024-09-10T21:43:43.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.175+0000 7f334be00640 1 -- 172.21.15.7:0/3778615069 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f3360029470 con 0x560d65e7cea0 2024-09-10T21:43:43.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.175+0000 7f334be00640 1 -- 172.21.15.7:0/3778615069 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3360031050 con 0x560d65e7cea0 2024-09-10T21:43:43.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.175+0000 7f3371024740 10 client.5493 fetch_fsmap learned FSMap version 26 2024-09-10T21:43:43.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.175+0000 7f3371024740 1 -- 172.21.15.7:0/3778615069 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x560d65b0a6a0 con 0x560d65e7cea0 2024-09-10T21:43:43.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.175+0000 7f334be00640 1 -- 172.21.15.7:0/3778615069 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33580875b0 con 0x560d65e7cea0 2024-09-10T21:43:43.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.176+0000 7f334be00640 1 -- 172.21.15.7:0/3778615069 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f3360034020 con 0x560d65e7cea0 2024-09-10T21:43:43.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.176+0000 7f3371024740 10 client.5493 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T21:43:43.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.176+0000 7f3371024740 10 client.5493 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-09-10T21:43:43.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.176+0000 7f3371024740 20 client.5493 populate_metadata read hostname 'smithi007' 2024-09-10T21:43:43.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.176+0000 7f3371024740 1 --2- 172.21.15.7:0/3778615069 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x560d65e92170 0x560d65ec06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:43.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.176+0000 7f3371024740 4 client.5493 mds_command: new command op to 234767 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:43:43.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.176+0000 7f3371024740 1 -- 172.21.15.7:0/3778615069 --> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d65ddd7c0 con 0x560d65e92170 2024-09-10T21:43:43.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.176+0000 7f3367400640 1 --2- 172.21.15.7:0/3778615069 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x560d65e92170 0x560d65ec06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:43.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.177+0000 7f3367400640 1 --2- 172.21.15.7:0/3778615069 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x560d65e92170 0x560d65ec06a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=802d59c4420dcf6 server_cookie=a2c8768f22a02680 in_seq=0 out_seq=0 2024-09-10T21:43:43.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.177+0000 7f334be00640 10 client.5493 ms_handle_connect on v2:172.21.15.133:6834/3385628197 2024-09-10T21:43:43.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.177+0000 7f334be00640 1 -- 172.21.15.7:0/3778615069 <== mds.0 v2:172.21.15.133:6834/3385628197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560d65ddd7c0 con 0x560d65e92170 2024-09-10T21:43:43.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.177+0000 7f334be00640 10 client.5493 handle_command_reply: tid=0 2024-09-10T21:43:43.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.177+0000 7f3371024740 1 client.5493 shutdown 2024-09-10T21:43:43.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.178+0000 7f3371024740 1 -- 172.21.15.7:0/3778615069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65e7cea0 msgr2=0x560d65e91c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:43.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.178+0000 7f3371024740 1 --2- 172.21.15.7:0/3778615069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65e7cea0 0x560d65e91c30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f3360012950 tx=0x7f3360004a40 comp rx=0 tx=0).stop 2024-09-10T21:43:43.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.178+0000 7f3371024740 1 -- 172.21.15.7:0/3778615069 shutdown_connections 2024-09-10T21:43:43.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.178+0000 7f3371024740 1 --2- 172.21.15.7:0/3778615069 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x560d65e92170 0x560d65ec06a0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.178+0000 7f3371024740 1 --2- 172.21.15.7:0/3778615069 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65e7cea0 0x560d65e91c30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.178+0000 7f3371024740 1 --2- 172.21.15.7:0/3778615069 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65eae130 0x560d65eacdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.178+0000 7f334be00640 0 client.5493 ms_handle_reset on v2:172.21.15.133:6834/3385628197 2024-09-10T21:43:43.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.178+0000 7f3371024740 1 --2- 172.21.15.7:0/3778615069 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e69820 0x560d65e7d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.178+0000 7f3371024740 1 -- 172.21.15.7:0/3778615069 >> 172.21.15.7:0/3778615069 conn(0x560d65e692a0 msgr2=0x560d65e63440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:43.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.178+0000 7f3371024740 1 -- 172.21.15.7:0/3778615069 shutdown_connections 2024-09-10T21:43:43.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.178+0000 7f3371024740 1 -- 172.21.15.7:0/3778615069 wait complete. 2024-09-10T21:43:43.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.178+0000 7f3371024740 20 client.5493 trim_cache size 0 max 16384 2024-09-10T21:43:43.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.196+0000 7f3371024740 1 Processor -- start 2024-09-10T21:43:43.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.196+0000 7f3371024740 1 -- start start 2024-09-10T21:43:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.197+0000 7f3371024740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65e69820 0x560d65e7d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.197+0000 7f3371024740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e7cea0 0x560d65e7d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.197+0000 7f3371024740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65eae130 0x560d65e63320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.197+0000 7f3371024740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3368201230 con 0x560d65eae130 2024-09-10T21:43:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.197+0000 7f3371024740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3368200270 con 0x560d65e7cea0 2024-09-10T21:43:43.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.197+0000 7f3371024740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f33681f8a30 con 0x560d65e69820 2024-09-10T21:43:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.198+0000 7f3367e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65eae130 0x560d65e63320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.198+0000 7f3367e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65eae130 0x560d65e63320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44368/0 (socket says 172.21.15.7:44368) 2024-09-10T21:43:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.198+0000 7f3367e00640 1 -- 172.21.15.7:0/4202242108 learned_addr learned my addr 172.21.15.7:0/4202242108 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.198+0000 7f335ea00640 1 --2- 172.21.15.7:0/4202242108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e7cea0 0x560d65e7d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.198+0000 7f3367400640 1 --2- 172.21.15.7:0/4202242108 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65e69820 0x560d65e7d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.198+0000 7f3367e00640 1 -- 172.21.15.7:0/4202242108 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65e69820 msgr2=0x560d65e7d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.198+0000 7f3367e00640 1 --2- 172.21.15.7:0/4202242108 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65e69820 0x560d65e7d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.198+0000 7f3367e00640 1 -- 172.21.15.7:0/4202242108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e7cea0 msgr2=0x560d65e7d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.198+0000 7f3367e00640 1 --2- 172.21.15.7:0/4202242108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e7cea0 0x560d65e7d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.198+0000 7f3367e00640 1 -- 172.21.15.7:0/4202242108 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d65e83c90 con 0x560d65eae130 2024-09-10T21:43:43.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.198+0000 7f3367e00640 1 --2- 172.21.15.7:0/4202242108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65eae130 0x560d65e63320 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3360009540 tx=0x7f33600296c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f6d962d89bf6dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:43.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.198+0000 7f3356a00640 1 -- 172.21.15.7:0/4202242108 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360029470 con 0x560d65eae130 2024-09-10T21:43:43.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.199+0000 7f3356a00640 1 -- 172.21.15.7:0/4202242108 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3360039020 con 0x560d65eae130 2024-09-10T21:43:43.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.199+0000 7f3356a00640 1 -- 172.21.15.7:0/4202242108 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3360011b70 con 0x560d65eae130 2024-09-10T21:43:43.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.199+0000 7f3371024740 1 -- 172.21.15.7:0/4202242108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65eae130 msgr2=0x560d65e63320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:43.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.199+0000 7f3371024740 1 --2- 172.21.15.7:0/4202242108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65eae130 0x560d65e63320 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3360009540 tx=0x7f33600296c0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.199+0000 7f3371024740 1 -- 172.21.15.7:0/4202242108 shutdown_connections 2024-09-10T21:43:43.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.199+0000 7f3371024740 1 --2- 172.21.15.7:0/4202242108 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e7cea0 0x560d65e7d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.199+0000 7f3371024740 1 --2- 172.21.15.7:0/4202242108 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65eae130 0x560d65e63320 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.199+0000 7f3371024740 1 --2- 172.21.15.7:0/4202242108 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65e69820 0x560d65e7d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.199+0000 7f3371024740 1 -- 172.21.15.7:0/4202242108 >> 172.21.15.7:0/4202242108 conn(0x560d65e692a0 msgr2=0x560d65e8dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:43.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.199+0000 7f3371024740 1 -- 172.21.15.7:0/4202242108 shutdown_connections 2024-09-10T21:43:43.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.199+0000 7f3371024740 1 -- 172.21.15.7:0/4202242108 wait complete. 2024-09-10T21:43:43.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.200+0000 7f3371024740 1 -- start start 2024-09-10T21:43:43.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.200+0000 7f3371024740 1 Processor -- start 2024-09-10T21:43:43.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.201+0000 7f3371024740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65e69820 0x560d65f2b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:43.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.201+0000 7f3371024740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e7cea0 0x560d65e7d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:43.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.201+0000 7f3371024740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65eae130 0x560d65e7d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:43.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.201+0000 7f3371024740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560d65ea72a0 con 0x560d65e69820 2024-09-10T21:43:43.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.201+0000 7f3371024740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f33680c8760 con 0x560d65e7cea0 2024-09-10T21:43:43.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.201+0000 7f3371024740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x560d65e7f0d0 con 0x560d65eae130 2024-09-10T21:43:43.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.201+0000 7f3371024740 10 client.0 _ll_register_callbacks cb 0x7f3360009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:43:43.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.201+0000 7f3367400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65e69820 0x560d65f2b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:43.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.201+0000 7f3367400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65e69820 0x560d65f2b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44380/0 (socket says 172.21.15.7:44380) 2024-09-10T21:43:43.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.201+0000 7f3367400640 1 -- 172.21.15.7:0/4154209048 learned_addr learned my addr 172.21.15.7:0/4154209048 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:43:43.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.202+0000 7f3367400640 1 -- 172.21.15.7:0/4154209048 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65eae130 msgr2=0x560d65e7d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:43.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.202+0000 7f3367e00640 1 --2- 172.21.15.7:0/4154209048 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65eae130 0x560d65e7d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:43.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.202+0000 7f335ea00640 1 --2- 172.21.15.7:0/4154209048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e7cea0 0x560d65e7d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:43.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.202+0000 7f3367400640 1 --2- 172.21.15.7:0/4154209048 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65eae130 0x560d65e7d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.202+0000 7f3367400640 1 -- 172.21.15.7:0/4154209048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e7cea0 msgr2=0x560d65e7d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:43:43.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.202+0000 7f3367400640 1 --2- 172.21.15.7:0/4154209048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e7cea0 0x560d65e7d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.202+0000 7f3367400640 1 -- 172.21.15.7:0/4154209048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d65f2b570 con 0x560d65e69820 2024-09-10T21:43:43.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.202+0000 7f335ea00640 1 --2- 172.21.15.7:0/4154209048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e7cea0 0x560d65e7d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:43:43.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.202+0000 7f3367e00640 1 --2- 172.21.15.7:0/4154209048 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65eae130 0x560d65e7d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:43:43.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.202+0000 7f3367400640 1 --2- 172.21.15.7:0/4154209048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65e69820 0x560d65f2b030 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3358005fd0 tx=0x7f3358085040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=438940b941ccac59 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:43.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.202+0000 7f334be00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:43:43.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.202+0000 7f334be00640 1 -- 172.21.15.7:0/4154209048 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3358048040 con 0x560d65e69820 2024-09-10T21:43:43.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.202+0000 7f334be00640 1 -- 172.21.15.7:0/4154209048 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(44..44 src has 1..44) ==== 5709+0+0 (secure 0 0 0) 0x7f335808bb40 con 0x560d65e69820 2024-09-10T21:43:43.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.202+0000 7f3371024740 1 -- 172.21.15.7:0/4154209048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3360039020 con 0x560d65e69820 2024-09-10T21:43:43.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.202+0000 7f334be00640 1 -- 172.21.15.7:0/4154209048 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f33580875b0 con 0x560d65e69820 2024-09-10T21:43:43.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.202+0000 7f3371024740 10 client.5499 fetch_fsmap learned FSMap version 26 2024-09-10T21:43:43.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.202+0000 7f3371024740 1 -- 172.21.15.7:0/4154209048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x560d65e66f60 con 0x560d65e69820 2024-09-10T21:43:43.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.203+0000 7f334be00640 1 -- 172.21.15.7:0/4154209048 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3360039020 con 0x560d65e69820 2024-09-10T21:43:43.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.203+0000 7f334be00640 1 -- 172.21.15.7:0/4154209048 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f33580870b0 con 0x560d65e69820 2024-09-10T21:43:43.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.203+0000 7f3371024740 10 client.5499 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T21:43:43.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.203+0000 7f3371024740 10 client.5499 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-09-10T21:43:43.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.203+0000 7f3371024740 20 client.5499 populate_metadata read hostname 'smithi007' 2024-09-10T21:43:43.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.203+0000 7f3371024740 1 --2- 172.21.15.7:0/4154209048 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x560d65eaae90 0x560d65eb0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:43.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.203+0000 7f3371024740 4 client.5499 mds_command: new command op to 234767 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:43:43.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.203+0000 7f3371024740 1 -- 172.21.15.7:0/4154209048 --> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560d65dd9270 con 0x560d65eaae90 2024-09-10T21:43:43.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.203+0000 7f335ea00640 1 --2- 172.21.15.7:0/4154209048 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x560d65eaae90 0x560d65eb0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:43.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.204+0000 7f335ea00640 1 --2- 172.21.15.7:0/4154209048 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x560d65eaae90 0x560d65eb0390 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7307eb3a4dd3312 server_cookie=49ac35f26b6401c4 in_seq=0 out_seq=0 2024-09-10T21:43:43.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.204+0000 7f334be00640 10 client.5499 ms_handle_connect on v2:172.21.15.133:6834/3385628197 2024-09-10T21:43:43.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.204+0000 7f334be00640 1 -- 172.21.15.7:0/4154209048 <== mds.0 v2:172.21.15.133:6834/3385628197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+571 (crc 0 0 0) 0x560d65dd9270 con 0x560d65eaae90 2024-09-10T21:43:43.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.204+0000 7f334be00640 10 client.5499 handle_command_reply: tid=0 2024-09-10T21:43:43.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.204+0000 7f3371024740 1 client.5499 shutdown 2024-09-10T21:43:43.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.204+0000 7f3371024740 1 -- 172.21.15.7:0/4154209048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65e69820 msgr2=0x560d65f2b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:43.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.204+0000 7f3371024740 1 --2- 172.21.15.7:0/4154209048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65e69820 0x560d65f2b030 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f3358005fd0 tx=0x7f3358085040 comp rx=0 tx=0).stop 2024-09-10T21:43:43.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.204+0000 7f3371024740 1 -- 172.21.15.7:0/4154209048 shutdown_connections 2024-09-10T21:43:43.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.204+0000 7f3371024740 1 --2- 172.21.15.7:0/4154209048 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x560d65eaae90 0x560d65eb0390 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.204+0000 7f3371024740 1 --2- 172.21.15.7:0/4154209048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d65e7cea0 0x560d65e7d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.204+0000 7f3371024740 1 --2- 172.21.15.7:0/4154209048 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d65eae130 0x560d65e7d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.204+0000 7f3371024740 1 --2- 172.21.15.7:0/4154209048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d65e69820 0x560d65f2b030 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.204+0000 7f334be00640 0 client.5499 ms_handle_reset on v2:172.21.15.133:6834/3385628197 2024-09-10T21:43:43.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.204+0000 7f3371024740 1 -- 172.21.15.7:0/4154209048 >> 172.21.15.7:0/4154209048 conn(0x560d65e692a0 msgr2=0x560d65e8e080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:43.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.204+0000 7f3371024740 1 -- 172.21.15.7:0/4154209048 shutdown_connections 2024-09-10T21:43:43.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.204+0000 7f3371024740 1 -- 172.21.15.7:0/4154209048 wait complete. 2024-09-10T21:43:43.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.204+0000 7f3371024740 20 client.5499 trim_cache size 0 max 16384 2024-09-10T21:43:43.210 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:43:43.211 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:43:43.211 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-09-10T21:43:43.211 INFO:teuthology.orchestra.run.smithi007.stdout: "0c8f9dbc-d094-4ce0-98e2-5f3ee7a2ce53": { 2024-09-10T21:43:43.211 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-09-10T21:43:43.211 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "0c8f9dbc-d094-4ce0-98e2-5f3ee7a2ce53", 2024-09-10T21:43:43.211 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,repair" 2024-09-10T21:43:43.211 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:43.211 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:43:43.211 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:43:43.211 INFO:teuthology.orchestra.run.smithi007.stdout: "0c8f9dbc-d094-4ce0-98e2-5f3ee7a2ce53": { 2024-09-10T21:43:43.211 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:43:42.983271+0000", 2024-09-10T21:43:43.211 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-09-10T21:43:43.212 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:43:43.212 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:43:43.212 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:43:43.212 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:43:43.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:43.212 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:43:43.212 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:43:43.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.205+0000 7f336e000640 1 -- 172.21.15.7:0/1528346179 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3344077f70 msgr2=0x7f334407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:43.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.205+0000 7f336e000640 1 --2- 172.21.15.7:0/1528346179 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3344077f70 0x7f334407a430 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3360005fd0 tx=0x7f33600094a0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.205+0000 7f336e000640 1 -- 172.21.15.7:0/1528346179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336806fcf0 msgr2=0x7f336814b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:43.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.205+0000 7f336e000640 1 --2- 172.21.15.7:0/1528346179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336806fcf0 0x7f336814b200 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3358034040 tx=0x7f3358003910 comp rx=0 tx=0).stop 2024-09-10T21:43:43.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.205+0000 7f336e000640 1 -- 172.21.15.7:0/1528346179 shutdown_connections 2024-09-10T21:43:43.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.205+0000 7f336e000640 1 --2- 172.21.15.7:0/1528346179 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3344077f70 0x7f334407a430 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.205+0000 7f336e000640 1 --2- 172.21.15.7:0/1528346179 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3368074450 0x7f33680b33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.205+0000 7f336e000640 1 --2- 172.21.15.7:0/1528346179 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3368155fb0 0x7f33680c33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.205+0000 7f336e000640 1 --2- 172.21.15.7:0/1528346179 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f336806fcf0 0x7f336814b200 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:43.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.205+0000 7f336e000640 1 -- 172.21.15.7:0/1528346179 >> 172.21.15.7:0/1528346179 conn(0x7f33680b16d0 msgr2=0x7f33680caeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:43.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.205+0000 7f336e000640 1 -- 172.21.15.7:0/1528346179 shutdown_connections 2024-09-10T21:43:43.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.205+0000 7f336e000640 1 -- 172.21.15.7:0/1528346179 wait complete. 2024-09-10T21:43:43.222 INFO:tasks.cephfs.filesystem:scrub status for tag:0c8f9dbc-d094-4ce0-98e2-5f3ee7a2ce53 - {'path': '/', 'tag': '0c8f9dbc-d094-4ce0-98e2-5f3ee7a2ce53', 'options': 'recursive,repair'} 2024-09-10T21:43:43.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:43.858+0000 7f417fe00640 1 -- 172.21.15.7:0/4040576676 <== mon.0 v2:172.21.15.7:3300/0 7 ==== log(6 entries from seq 1 at 2024-09-10T21:43:42.983803+0000) ==== 1078+0+0 (secure 0 0 0) 0x7f417809b0b0 con 0x7f4188070070 2024-09-10T21:43:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: cluster: 2024-09-10T21:43:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: id: 2029451a-3c26-48c7-b874-0597197278ac 2024-09-10T21:43:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: health: HEALTH_OK 2024-09-10T21:43:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: services: 2024-09-10T21:43:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: mon: 3 daemons, quorum a,b,c (age 118s) 2024-09-10T21:43:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: mgr: x(active, since 115s), standbys: y 2024-09-10T21:43:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: mds: 1/1 daemons up, 3 standby 2024-09-10T21:43:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: osd: 8 osds: 8 up (since 106s), 8 in (since 115s) 2024-09-10T21:43:43.862 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: data: 2024-09-10T21:43:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: volumes: 1/1 healthy 2024-09-10T21:43:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: pools: 3 pools, 97 pgs 2024-09-10T21:43:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: objects: 24 objects, 579 KiB 2024-09-10T21:43:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: usage: 237 MiB used, 720 GiB / 720 GiB avail 2024-09-10T21:43:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: pgs: 97 active+clean 2024-09-10T21:43:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: io: 2024-09-10T21:43:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: client: 1.7 KiB/s wr, 0 op/s rd, 5 op/s wr 2024-09-10T21:43:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:43.863 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:43:43.863 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:43:42.983803+0000 mds.d [INF] scrub queued for path: / 2024-09-10T21:43:43.864 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:43:42.983806+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-09-10T21:43:43.864 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:43:42.983810+0000 mds.d [INF] scrub summary: active paths [/] 2024-09-10T21:43:43.864 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:43:42.985062+0000 mds.d [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2024-09-10T21:43:43.864 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:43:42.985071+0000 mds.d [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2024-09-10T21:43:43.864 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:43:42.986921+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2024-09-10T21:43:44.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:44.864+0000 7f417fe00640 1 -- 172.21.15.7:0/4040576676 <== mon.0 v2:172.21.15.7:3300/0 8 ==== log(1 entries from seq 7 at 2024-09-10T21:43:43.520060+0000) ==== 189+0+0 (secure 0 0 0) 0x7f4178099870 con 0x7f4188070070 2024-09-10T21:43:44.866 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:43:43.520060+0000 mds.d [INF] scrub summary: idle 2024-09-10T21:43:47.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:47.881+0000 7f417fe00640 1 -- 172.21.15.7:0/4040576676 <== mon.0 v2:172.21.15.7:3300/0 9 ==== log(4 entries from seq 115 at 2024-09-10T21:43:47.872173+0000) ==== 1118+0+0 (secure 0 0 0) 0x7f417805f950 con 0x7f4188070070 2024-09-10T21:43:48.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:43:48.884+0000 7f417fe00640 1 -- 172.21.15.7:0/4040576676 <== mon.0 v2:172.21.15.7:3300/0 10 ==== log(2 entries from seq 406 at 2024-09-10T21:43:47.875494+0000) ==== 558+0+0 (secure 0 0 0) 0x7f4178063940 con 0x7f4188070070 2024-09-10T21:43:59.944 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:43:59.949 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:44:13.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:44:13.223 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.3:0 scrub status 2024-09-10T21:44:13.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.397+0000 7f8d72c00640 1 -- 172.21.15.7:0/2518582145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d6c0cea70 msgr2=0x7f8d6c0b1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:13.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.397+0000 7f8d72c00640 1 --2- 172.21.15.7:0/2518582145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d6c0cea70 0x7f8d6c0b1570 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0738a0 tx=0x7f8d6402a820 comp rx=0 tx=0).stop 2024-09-10T21:44:13.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.397+0000 7f8d72c00640 1 -- 172.21.15.7:0/2518582145 shutdown_connections 2024-09-10T21:44:13.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.397+0000 7f8d72c00640 1 --2- 172.21.15.7:0/2518582145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d6c0cea70 0x7f8d6c0b1570 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.397+0000 7f8d72c00640 1 --2- 172.21.15.7:0/2518582145 >> v2:172.21.15.133:3300/0 conn(0x7f8d6c0ce690 0x7f8d6c0cf9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.397+0000 7f8d72c00640 1 -- 172.21.15.7:0/2518582145 >> 172.21.15.7:0/2518582145 conn(0x7f8d6c0d1c10 msgr2=0x7f8d6c0d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:13.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.397+0000 7f8d72c00640 1 -- 172.21.15.7:0/2518582145 shutdown_connections 2024-09-10T21:44:13.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.397+0000 7f8d72c00640 1 -- 172.21.15.7:0/2518582145 wait complete. 2024-09-10T21:44:13.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.398+0000 7f8d72c00640 1 Processor -- start 2024-09-10T21:44:13.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.399+0000 7f8d72c00640 1 -- start start 2024-09-10T21:44:13.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.399+0000 7f8d72c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d6c0b0c70 0x7f8d6c1f2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.399+0000 7f8d72c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d6c0ce690 0x7f8d6c0caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.400+0000 7f8d72c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d6c0cea70 0x7f8d6c0c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.400+0000 7f8d72c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c0cf570 con 0x7f8d6c0ce690 2024-09-10T21:44:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.400+0000 7f8d72c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c0d2940 con 0x7f8d6c0b0c70 2024-09-10T21:44:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.400+0000 7f8d72c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f8d6c0d0640 con 0x7f8d6c0cea70 2024-09-10T21:44:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.400+0000 7f8d70e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d6c0ce690 0x7f8d6c0caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.400+0000 7f8d70e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d6c0ce690 0x7f8d6c0caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59572/0 (socket says 172.21.15.7:59572) 2024-09-10T21:44:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.400+0000 7f8d70e00640 1 -- 172.21.15.7:0/3513653165 learned_addr learned my addr 172.21.15.7:0/3513653165 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.400+0000 7f8d71800640 1 --2- 172.21.15.7:0/3513653165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d6c0b0c70 0x7f8d6c1f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.400+0000 7f8d72200640 1 --2- 172.21.15.7:0/3513653165 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d6c0cea70 0x7f8d6c0c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.400+0000 7f8d70e00640 1 -- 172.21.15.7:0/3513653165 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d6c0cea70 msgr2=0x7f8d6c0c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.400+0000 7f8d70e00640 1 --2- 172.21.15.7:0/3513653165 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d6c0cea70 0x7f8d6c0c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.400+0000 7f8d70e00640 1 -- 172.21.15.7:0/3513653165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d6c0b0c70 msgr2=0x7f8d6c1f2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:13.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.400+0000 7f8d70e00640 1 --2- 172.21.15.7:0/3513653165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d6c0b0c70 0x7f8d6c1f2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.400+0000 7f8d70e00640 1 -- 172.21.15.7:0/3513653165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d6c0c92f0 con 0x7f8d6c0ce690 2024-09-10T21:44:13.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.400+0000 7f8d72200640 1 --2- 172.21.15.7:0/3513653165 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d6c0cea70 0x7f8d6c0c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:13.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.401+0000 7f8d70e00640 1 --2- 172.21.15.7:0/3513653165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d6c0ce690 0x7f8d6c0caad0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800c430 tx=0x7f8d5800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:13.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.401+0000 7f8d69600640 1 -- 172.21.15.7:0/3513653165 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58017070 con 0x7f8d6c0ce690 2024-09-10T21:44:13.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.401+0000 7f8d69600640 1 -- 172.21.15.7:0/3513653165 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d580035f0 con 0x7f8d6c0ce690 2024-09-10T21:44:13.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.401+0000 7f8d69600640 1 -- 172.21.15.7:0/3513653165 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58003ac0 con 0x7f8d6c0ce690 2024-09-10T21:44:13.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.401+0000 7f8d72c00640 1 -- 172.21.15.7:0/3513653165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d6c0c9860 con 0x7f8d6c0ce690 2024-09-10T21:44:13.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.401+0000 7f8d72c00640 1 -- 172.21.15.7:0/3513653165 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d6c0cbc90 con 0x7f8d6c0ce690 2024-09-10T21:44:13.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.403+0000 7f8d69600640 1 -- 172.21.15.7:0/3513653165 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8d58003c60 con 0x7f8d6c0ce690 2024-09-10T21:44:13.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.404+0000 7f8d69600640 1 --2- 172.21.15.7:0/3513653165 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8d4c077f70 0x7f8d4c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.404+0000 7f8d69600640 1 -- 172.21.15.7:0/3513653165 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f8d58099450 con 0x7f8d6c0ce690 2024-09-10T21:44:13.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.405+0000 7f8d771e4740 1 Processor -- start 2024-09-10T21:44:13.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.405+0000 7f8d771e4740 1 -- start start 2024-09-10T21:44:13.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.405+0000 7f8d71800640 1 --2- 172.21.15.7:0/3513653165 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8d4c077f70 0x7f8d4c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.405+0000 7f8d71800640 1 --2- 172.21.15.7:0/3513653165 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8d4c077f70 0x7f8d4c07a430 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0b03d0 tx=0x7f8d6000b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:13.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.405+0000 7f8d771e4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e292f2e0 0x55a0e292f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.405+0000 7f8d771e4740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a0e28ea450 0x55a0e292ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.406+0000 7f8d771e4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e28ea9d0 0x55a0e292e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.406+0000 7f8d771e4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a0e28491c0 con 0x55a0e28ea9d0 2024-09-10T21:44:13.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.406+0000 7f8d771e4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a0e285c870 con 0x55a0e292f2e0 2024-09-10T21:44:13.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.406+0000 7f8d771e4740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55a0e284bca0 con 0x55a0e28ea450 2024-09-10T21:44:13.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.406+0000 7f8d72200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e28ea9d0 0x55a0e292e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.406+0000 7f8d71800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e292f2e0 0x55a0e292f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.406+0000 7f8d72200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e28ea9d0 0x55a0e292e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59580/0 (socket says 172.21.15.7:59580) 2024-09-10T21:44:13.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.406+0000 7f8d71800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e292f2e0 0x55a0e292f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:46040/0 (socket says 172.21.15.7:46040) 2024-09-10T21:44:13.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.406+0000 7f8d72200640 1 -- 172.21.15.7:0/833670672 learned_addr learned my addr 172.21.15.7:0/833670672 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:13.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.406+0000 7f8d70e00640 1 --2- 172.21.15.7:0/833670672 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a0e28ea450 0x55a0e292ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.406+0000 7f8d72200640 1 -- 172.21.15.7:0/833670672 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a0e28ea450 msgr2=0x55a0e292ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:13.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.406+0000 7f8d72200640 1 --2- 172.21.15.7:0/833670672 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a0e28ea450 0x55a0e292ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.406+0000 7f8d72200640 1 -- 172.21.15.7:0/833670672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e292f2e0 msgr2=0x55a0e292f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:13.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.406+0000 7f8d72200640 1 --2- 172.21.15.7:0/833670672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e292f2e0 0x55a0e292f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.406+0000 7f8d72200640 1 -- 172.21.15.7:0/833670672 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0e26247d0 con 0x55a0e28ea9d0 2024-09-10T21:44:13.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.407+0000 7f8d72200640 1 --2- 172.21.15.7:0/833670672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e28ea9d0 0x55a0e292e300 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009ca0 tx=0x7f8d64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b541fc9f2c0bf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:13.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.407+0000 7f8d5e000640 1 -- 172.21.15.7:0/833670672 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64037040 con 0x55a0e28ea9d0 2024-09-10T21:44:13.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.407+0000 7f8d5e000640 1 -- 172.21.15.7:0/833670672 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d64003ae0 con 0x55a0e28ea9d0 2024-09-10T21:44:13.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.407+0000 7f8d5e000640 1 -- 172.21.15.7:0/833670672 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d64004040 con 0x55a0e28ea9d0 2024-09-10T21:44:13.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.408+0000 7f8d771e4740 1 -- 172.21.15.7:0/833670672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e28ea9d0 msgr2=0x55a0e292e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:13.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.408+0000 7f8d771e4740 1 --2- 172.21.15.7:0/833670672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e28ea9d0 0x55a0e292e300 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f8d64009ca0 tx=0x7f8d64004a40 comp rx=0 tx=0).stop 2024-09-10T21:44:13.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.408+0000 7f8d771e4740 1 -- 172.21.15.7:0/833670672 shutdown_connections 2024-09-10T21:44:13.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.408+0000 7f8d771e4740 1 --2- 172.21.15.7:0/833670672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e28ea9d0 0x55a0e292e300 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.408+0000 7f8d771e4740 1 --2- 172.21.15.7:0/833670672 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a0e28ea450 0x55a0e292ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.408+0000 7f8d771e4740 1 --2- 172.21.15.7:0/833670672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e292f2e0 0x55a0e292f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.408+0000 7f8d771e4740 1 -- 172.21.15.7:0/833670672 >> 172.21.15.7:0/833670672 conn(0x55a0e28e5a20 msgr2=0x55a0e28e9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.408+0000 7f8d771e4740 1 -- 172.21.15.7:0/833670672 shutdown_connections 2024-09-10T21:44:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.408+0000 7f8d771e4740 1 -- 172.21.15.7:0/833670672 wait complete. 2024-09-10T21:44:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.408+0000 7f8d771e4740 1 -- start start 2024-09-10T21:44:13.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.409+0000 7f8d771e4740 1 Processor -- start 2024-09-10T21:44:13.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d771e4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e28ea9d0 0x55a0e2927c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d771e4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 0x55a0e292e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d771e4740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a0e2941840 0x55a0e2941ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d771e4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a0e27f2410 con 0x55a0e292f2e0 2024-09-10T21:44:13.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d771e4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a0e284fab0 con 0x55a0e28ea9d0 2024-09-10T21:44:13.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d771e4740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55a0e25bc740 con 0x55a0e2941840 2024-09-10T21:44:13.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d771e4740 10 client.0 _ll_register_callbacks cb 0x55a0e27cf360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:44:13.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d70e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 0x55a0e292e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d70e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 0x55a0e292e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59594/0 (socket says 172.21.15.7:59594) 2024-09-10T21:44:13.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d70e00640 1 -- 172.21.15.7:0/927439370 learned_addr learned my addr 172.21.15.7:0/927439370 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:13.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d71800640 1 --2- 172.21.15.7:0/927439370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e28ea9d0 0x55a0e2927c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d72200640 1 --2- 172.21.15.7:0/927439370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a0e2941840 0x55a0e2941ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d70e00640 1 -- 172.21.15.7:0/927439370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a0e2941840 msgr2=0x55a0e2941ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:13.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d70e00640 1 --2- 172.21.15.7:0/927439370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a0e2941840 0x55a0e2941ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d70e00640 1 -- 172.21.15.7:0/927439370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e28ea9d0 msgr2=0x55a0e2927c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:13.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d70e00640 1 --2- 172.21.15.7:0/927439370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e28ea9d0 0x55a0e2927c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d70e00640 1 -- 172.21.15.7:0/927439370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0e25ad970 con 0x55a0e292f2e0 2024-09-10T21:44:13.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d71800640 1 --2- 172.21.15.7:0/927439370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e28ea9d0 0x55a0e2927c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:13.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d70e00640 1 --2- 172.21.15.7:0/927439370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 0x55a0e292e480 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8d580617b0 tx=0x7f8d58061e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74cf9c660139017 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:13.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.410+0000 7f8d52a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:44:13.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.411+0000 7f8d52a00640 1 -- 172.21.15.7:0/927439370 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58068650 con 0x55a0e292f2e0 2024-09-10T21:44:13.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.411+0000 7f8d52a00640 1 -- 172.21.15.7:0/927439370 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f8d5805d170 con 0x55a0e292f2e0 2024-09-10T21:44:13.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.411+0000 7f8d771e4740 1 -- 172.21.15.7:0/927439370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d64003ae0 con 0x55a0e292f2e0 2024-09-10T21:44:13.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.411+0000 7f8d52a00640 1 -- 172.21.15.7:0/927439370 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8d58061820 con 0x55a0e292f2e0 2024-09-10T21:44:13.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.411+0000 7f8d771e4740 10 client.5514 fetch_fsmap learned FSMap version 26 2024-09-10T21:44:13.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.411+0000 7f8d771e4740 1 -- 172.21.15.7:0/927439370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55a0e258b6a0 con 0x55a0e292f2e0 2024-09-10T21:44:13.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.411+0000 7f8d52a00640 1 -- 172.21.15.7:0/927439370 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d64003ae0 con 0x55a0e292f2e0 2024-09-10T21:44:13.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.411+0000 7f8d52a00640 1 -- 172.21.15.7:0/927439370 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f8d58079030 con 0x55a0e292f2e0 2024-09-10T21:44:13.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.411+0000 7f8d771e4740 10 client.5514 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T21:44:13.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.411+0000 7f8d771e4740 10 client.5514 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-09-10T21:44:13.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.411+0000 7f8d771e4740 20 client.5514 populate_metadata read hostname 'smithi007' 2024-09-10T21:44:13.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.411+0000 7f8d771e4740 1 --2- 172.21.15.7:0/927439370 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55a0e293f8e0 0x55a0e293fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.411+0000 7f8d771e4740 4 client.5514 mds_command: new command op to 234767 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:44:13.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.411+0000 7f8d771e4740 1 -- 172.21.15.7:0/927439370 --> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0e285e7c0 con 0x55a0e293f8e0 2024-09-10T21:44:13.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.412+0000 7f8d71800640 1 --2- 172.21.15.7:0/927439370 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55a0e293f8e0 0x55a0e293fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.412+0000 7f8d71800640 1 --2- 172.21.15.7:0/927439370 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55a0e293f8e0 0x55a0e293fcc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e3ffae45be2ca6a server_cookie=20ac6268a8309085 in_seq=0 out_seq=0 2024-09-10T21:44:13.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.412+0000 7f8d52a00640 10 client.5514 ms_handle_connect on v2:172.21.15.133:6834/3385628197 2024-09-10T21:44:13.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.413+0000 7f8d52a00640 1 -- 172.21.15.7:0/927439370 <== mds.0 v2:172.21.15.133:6834/3385628197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a0e285e7c0 con 0x55a0e293f8e0 2024-09-10T21:44:13.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.413+0000 7f8d52a00640 10 client.5514 handle_command_reply: tid=0 2024-09-10T21:44:13.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.414+0000 7f8d771e4740 1 client.5514 shutdown 2024-09-10T21:44:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.414+0000 7f8d771e4740 1 -- 172.21.15.7:0/927439370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 msgr2=0x55a0e292e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.414+0000 7f8d771e4740 1 --2- 172.21.15.7:0/927439370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 0x55a0e292e480 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f8d580617b0 tx=0x7f8d58061e40 comp rx=0 tx=0).stop 2024-09-10T21:44:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.414+0000 7f8d771e4740 1 -- 172.21.15.7:0/927439370 shutdown_connections 2024-09-10T21:44:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.414+0000 7f8d771e4740 1 --2- 172.21.15.7:0/927439370 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55a0e293f8e0 0x55a0e293fcc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.414+0000 7f8d771e4740 1 --2- 172.21.15.7:0/927439370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a0e2941840 0x55a0e2941ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.414+0000 7f8d771e4740 1 --2- 172.21.15.7:0/927439370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 0x55a0e292e480 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.414+0000 7f8d771e4740 1 --2- 172.21.15.7:0/927439370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e28ea9d0 0x55a0e2927c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.414+0000 7f8d52a00640 0 client.5514 ms_handle_reset on v2:172.21.15.133:6834/3385628197 2024-09-10T21:44:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.414+0000 7f8d771e4740 1 -- 172.21.15.7:0/927439370 >> 172.21.15.7:0/927439370 conn(0x55a0e28ea450 msgr2=0x55a0e28e45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.414+0000 7f8d771e4740 1 -- 172.21.15.7:0/927439370 shutdown_connections 2024-09-10T21:44:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.414+0000 7f8d771e4740 1 -- 172.21.15.7:0/927439370 wait complete. 2024-09-10T21:44:13.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.414+0000 7f8d771e4740 20 client.5514 trim_cache size 0 max 16384 2024-09-10T21:44:13.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.432+0000 7f8d771e4740 1 Processor -- start 2024-09-10T21:44:13.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.433+0000 7f8d771e4740 1 -- start start 2024-09-10T21:44:13.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.433+0000 7f8d771e4740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a0e28ea9d0 0x55a0e28e4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.433+0000 7f8d771e4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 0x55a0e28e4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.433+0000 7f8d771e4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e2941840 0x55a0e2923df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.433+0000 7f8d771e4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c0c6520 con 0x55a0e292f2e0 2024-09-10T21:44:13.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.433+0000 7f8d771e4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c06fc30 con 0x55a0e2941840 2024-09-10T21:44:13.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.433+0000 7f8d771e4740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f8d6c06ef90 con 0x55a0e28ea9d0 2024-09-10T21:44:13.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.434+0000 7f8d70e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 0x55a0e28e4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.434+0000 7f8d70e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 0x55a0e28e4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59600/0 (socket says 172.21.15.7:59600) 2024-09-10T21:44:13.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.434+0000 7f8d70e00640 1 -- 172.21.15.7:0/3165386587 learned_addr learned my addr 172.21.15.7:0/3165386587 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:13.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.434+0000 7f8d70e00640 1 -- 172.21.15.7:0/3165386587 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a0e28ea9d0 msgr2=0x55a0e28e4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:13.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.434+0000 7f8d70e00640 1 --2- 172.21.15.7:0/3165386587 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a0e28ea9d0 0x55a0e28e4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.434+0000 7f8d72200640 1 --2- 172.21.15.7:0/3165386587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e2941840 0x55a0e2923df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.434+0000 7f8d70e00640 1 -- 172.21.15.7:0/3165386587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e2941840 msgr2=0x55a0e2923df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:13.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.434+0000 7f8d70e00640 1 --2- 172.21.15.7:0/3165386587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e2941840 0x55a0e2923df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.434+0000 7f8d70e00640 1 -- 172.21.15.7:0/3165386587 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0e2882d50 con 0x55a0e292f2e0 2024-09-10T21:44:13.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.434+0000 7f8d72200640 1 --2- 172.21.15.7:0/3165386587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e2941840 0x55a0e2923df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:44:13.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.434+0000 7f8d70e00640 1 --2- 172.21.15.7:0/3165386587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 0x55a0e28e4aa0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8d58000fb0 tx=0x7f8d58061680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69bb3ebe613d5344 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:13.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.435+0000 7f8d5d600640 1 -- 172.21.15.7:0/3165386587 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58068650 con 0x55a0e292f2e0 2024-09-10T21:44:13.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.435+0000 7f8d5d600640 1 -- 172.21.15.7:0/3165386587 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d5807b080 con 0x55a0e292f2e0 2024-09-10T21:44:13.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.435+0000 7f8d5d600640 1 -- 172.21.15.7:0/3165386587 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d5805d170 con 0x55a0e292f2e0 2024-09-10T21:44:13.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.435+0000 7f8d771e4740 1 -- 172.21.15.7:0/3165386587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 msgr2=0x55a0e28e4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:13.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.435+0000 7f8d771e4740 1 --2- 172.21.15.7:0/3165386587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 0x55a0e28e4aa0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f8d58000fb0 tx=0x7f8d58061680 comp rx=0 tx=0).stop 2024-09-10T21:44:13.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.435+0000 7f8d771e4740 1 -- 172.21.15.7:0/3165386587 shutdown_connections 2024-09-10T21:44:13.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.435+0000 7f8d771e4740 1 --2- 172.21.15.7:0/3165386587 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e2941840 0x55a0e2923df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.435+0000 7f8d771e4740 1 --2- 172.21.15.7:0/3165386587 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 0x55a0e28e4aa0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.435+0000 7f8d771e4740 1 --2- 172.21.15.7:0/3165386587 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a0e28ea9d0 0x55a0e28e4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.435+0000 7f8d771e4740 1 -- 172.21.15.7:0/3165386587 >> 172.21.15.7:0/3165386587 conn(0x55a0e28ea450 msgr2=0x55a0e28e8f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:13.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.435+0000 7f8d771e4740 1 -- 172.21.15.7:0/3165386587 shutdown_connections 2024-09-10T21:44:13.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.435+0000 7f8d771e4740 1 -- 172.21.15.7:0/3165386587 wait complete. 2024-09-10T21:44:13.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.436+0000 7f8d771e4740 1 -- start start 2024-09-10T21:44:13.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.436+0000 7f8d771e4740 1 Processor -- start 2024-09-10T21:44:13.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.437+0000 7f8d771e4740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e28ea9d0 0x55a0e29310c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.437+0000 7f8d771e4740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 0x55a0e29a8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.437+0000 7f8d771e4740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a0e2941840 0x55a0e29af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.437+0000 7f8d771e4740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8d6c06f5e0 con 0x55a0e292f2e0 2024-09-10T21:44:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.437+0000 7f8d771e4740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a0e28e3620 con 0x55a0e28ea9d0 2024-09-10T21:44:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.437+0000 7f8d771e4740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55a0e2931a30 con 0x55a0e2941840 2024-09-10T21:44:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.437+0000 7f8d771e4740 10 client.0 _ll_register_callbacks cb 0x7f8d5805d790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:44:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.437+0000 7f8d70e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 0x55a0e29a8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.437+0000 7f8d70e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 0x55a0e29a8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59602/0 (socket says 172.21.15.7:59602) 2024-09-10T21:44:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.437+0000 7f8d70e00640 1 -- 172.21.15.7:0/2976237012 learned_addr learned my addr 172.21.15.7:0/2976237012 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.438+0000 7f8d71800640 1 --2- 172.21.15.7:0/2976237012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e28ea9d0 0x55a0e29310c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.438+0000 7f8d72200640 1 --2- 172.21.15.7:0/2976237012 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a0e2941840 0x55a0e29af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.438+0000 7f8d70e00640 1 -- 172.21.15.7:0/2976237012 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a0e2941840 msgr2=0x55a0e29af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.438+0000 7f8d70e00640 1 --2- 172.21.15.7:0/2976237012 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a0e2941840 0x55a0e29af5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.438+0000 7f8d70e00640 1 -- 172.21.15.7:0/2976237012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e28ea9d0 msgr2=0x55a0e29310c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.438+0000 7f8d70e00640 1 --2- 172.21.15.7:0/2976237012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e28ea9d0 0x55a0e29310c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.438+0000 7f8d70e00640 1 -- 172.21.15.7:0/2976237012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0e2913b70 con 0x55a0e292f2e0 2024-09-10T21:44:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.438+0000 7f8d70e00640 1 --2- 172.21.15.7:0/2976237012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 0x55a0e29a8bc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8d5805d8d0 tx=0x7f8d5805d6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3965ffffc97cbe2d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.438+0000 7f8d52a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:44:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.438+0000 7f8d52a00640 1 -- 172.21.15.7:0/2976237012 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d58080060 con 0x55a0e292f2e0 2024-09-10T21:44:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.438+0000 7f8d52a00640 1 -- 172.21.15.7:0/2976237012 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f8d5805d170 con 0x55a0e292f2e0 2024-09-10T21:44:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.438+0000 7f8d771e4740 1 -- 172.21.15.7:0/2976237012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d5807b080 con 0x55a0e292f2e0 2024-09-10T21:44:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.438+0000 7f8d52a00640 1 -- 172.21.15.7:0/2976237012 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f8d58068650 con 0x55a0e292f2e0 2024-09-10T21:44:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.438+0000 7f8d771e4740 10 client.5520 fetch_fsmap learned FSMap version 26 2024-09-10T21:44:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.439+0000 7f8d771e4740 1 -- 172.21.15.7:0/2976237012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55a0e29af110 con 0x55a0e292f2e0 2024-09-10T21:44:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.439+0000 7f8d52a00640 1 -- 172.21.15.7:0/2976237012 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8d5807b080 con 0x55a0e292f2e0 2024-09-10T21:44:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.439+0000 7f8d52a00640 1 -- 172.21.15.7:0/2976237012 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f8d58079030 con 0x55a0e292f2e0 2024-09-10T21:44:13.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.439+0000 7f8d771e4740 10 client.5520 fetch_fsmap finished waiting for FSMap version 26 2024-09-10T21:44:13.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.439+0000 7f8d771e4740 10 client.5520 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-09-10T21:44:13.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.439+0000 7f8d771e4740 20 client.5520 populate_metadata read hostname 'smithi007' 2024-09-10T21:44:13.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.439+0000 7f8d771e4740 1 --2- 172.21.15.7:0/2976237012 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55a0e2911fe0 0x55a0e2950cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.439+0000 7f8d771e4740 4 client.5520 mds_command: new command op to 234767 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:44:13.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.439+0000 7f8d771e4740 1 -- 172.21.15.7:0/2976237012 --> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a0e2929940 con 0x55a0e2911fe0 2024-09-10T21:44:13.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.439+0000 7f8d71800640 1 --2- 172.21.15.7:0/2976237012 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55a0e2911fe0 0x55a0e2950cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.440+0000 7f8d71800640 1 --2- 172.21.15.7:0/2976237012 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55a0e2911fe0 0x55a0e2950cf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=862d7cb10ff6f357 server_cookie=c658f9ca5d8dee55 in_seq=0 out_seq=0 2024-09-10T21:44:13.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.440+0000 7f8d52a00640 10 client.5520 ms_handle_connect on v2:172.21.15.133:6834/3385628197 2024-09-10T21:44:13.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.440+0000 7f8d52a00640 1 -- 172.21.15.7:0/2976237012 <== mds.0 v2:172.21.15.133:6834/3385628197 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x55a0e2929940 con 0x55a0e2911fe0 2024-09-10T21:44:13.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.440+0000 7f8d52a00640 10 client.5520 handle_command_reply: tid=0 2024-09-10T21:44:13.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.440+0000 7f8d771e4740 1 client.5520 shutdown 2024-09-10T21:44:13.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.440+0000 7f8d771e4740 1 -- 172.21.15.7:0/2976237012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 msgr2=0x55a0e29a8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:13.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.440+0000 7f8d771e4740 1 --2- 172.21.15.7:0/2976237012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 0x55a0e29a8bc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f8d5805d8d0 tx=0x7f8d5805d6c0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.440+0000 7f8d771e4740 1 -- 172.21.15.7:0/2976237012 shutdown_connections 2024-09-10T21:44:13.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.440+0000 7f8d771e4740 1 --2- 172.21.15.7:0/2976237012 >> [v2:172.21.15.133:6834/3385628197,v1:172.21.15.133:6837/3385628197] conn(0x55a0e2911fe0 0x55a0e2950cf0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.440+0000 7f8d771e4740 1 --2- 172.21.15.7:0/2976237012 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a0e2941840 0x55a0e29af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.440+0000 7f8d771e4740 1 --2- 172.21.15.7:0/2976237012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a0e292f2e0 0x55a0e29a8bc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.440+0000 7f8d771e4740 1 --2- 172.21.15.7:0/2976237012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a0e28ea9d0 0x55a0e29310c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.440+0000 7f8d52a00640 0 client.5520 ms_handle_reset on v2:172.21.15.133:6834/3385628197 2024-09-10T21:44:13.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.440+0000 7f8d771e4740 1 -- 172.21.15.7:0/2976237012 >> 172.21.15.7:0/2976237012 conn(0x55a0e28ea450 msgr2=0x55a0e28e8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:13.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.440+0000 7f8d771e4740 1 -- 172.21.15.7:0/2976237012 shutdown_connections 2024-09-10T21:44:13.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.440+0000 7f8d771e4740 1 -- 172.21.15.7:0/2976237012 wait complete. 2024-09-10T21:44:13.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.441+0000 7f8d771e4740 20 client.5520 trim_cache size 0 max 16384 2024-09-10T21:44:13.446 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:44:13.447 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-09-10T21:44:13.447 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {}, 2024-09-10T21:44:13.447 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:44:13.447 INFO:teuthology.orchestra.run.smithi007.stdout: "0c8f9dbc-d094-4ce0-98e2-5f3ee7a2ce53": { 2024-09-10T21:44:13.447 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:43:42.983271+0000", 2024-09-10T21:44:13.447 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-09-10T21:44:13.447 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:44:13.447 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:44:13.447 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:44:13.447 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:44:13.447 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:13.448 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:13.448 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:44:13.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.441+0000 7f8d72c00640 1 -- 172.21.15.7:0/3513653165 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8d4c077f70 msgr2=0x7f8d4c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:13.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.441+0000 7f8d72c00640 1 --2- 172.21.15.7:0/3513653165 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8d4c077f70 0x7f8d4c07a430 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8d6c0b03d0 tx=0x7f8d6000b390 comp rx=0 tx=0).stop 2024-09-10T21:44:13.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.441+0000 7f8d72c00640 1 -- 172.21.15.7:0/3513653165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d6c0ce690 msgr2=0x7f8d6c0caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:13.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.441+0000 7f8d72c00640 1 --2- 172.21.15.7:0/3513653165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d6c0ce690 0x7f8d6c0caad0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8d5800c430 tx=0x7f8d5800c8f0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.441+0000 7f8d72c00640 1 -- 172.21.15.7:0/3513653165 shutdown_connections 2024-09-10T21:44:13.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.441+0000 7f8d72c00640 1 --2- 172.21.15.7:0/3513653165 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8d4c077f70 0x7f8d4c07a430 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.441+0000 7f8d72c00640 1 --2- 172.21.15.7:0/3513653165 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8d6c0cea70 0x7f8d6c0c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.441+0000 7f8d72c00640 1 --2- 172.21.15.7:0/3513653165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8d6c0ce690 0x7f8d6c0caad0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.441+0000 7f8d72c00640 1 --2- 172.21.15.7:0/3513653165 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8d6c0b0c70 0x7f8d6c1f2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.441+0000 7f8d72c00640 1 -- 172.21.15.7:0/3513653165 >> 172.21.15.7:0/3513653165 conn(0x7f8d6c0d1c10 msgr2=0x7f8d6c070680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:13.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.441+0000 7f8d72c00640 1 -- 172.21.15.7:0/3513653165 shutdown_connections 2024-09-10T21:44:13.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.441+0000 7f8d72c00640 1 -- 172.21.15.7:0/3513653165 wait complete. 2024-09-10T21:44:13.457 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:44:13.558 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:44:13.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.718+0000 7f79d6600640 1 -- 172.21.15.7:0/2634639122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79d00cdfc0 msgr2=0x7f79d00cefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:13.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.718+0000 7f79d6600640 1 --2- 172.21.15.7:0/2634639122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79d00cdfc0 0x7f79d00cefa0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f79b8005fd0 tx=0x7f79b802a540 comp rx=0 tx=0).stop 2024-09-10T21:44:13.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.718+0000 7f79d6600640 1 -- 172.21.15.7:0/2634639122 shutdown_connections 2024-09-10T21:44:13.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.719+0000 7f79d6600640 1 --2- 172.21.15.7:0/2634639122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79d00cdfc0 0x7f79d00cefa0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.719+0000 7f79d6600640 1 --2- 172.21.15.7:0/2634639122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79d00ce680 0x7f79d00cea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.719+0000 7f79d6600640 1 -- 172.21.15.7:0/2634639122 >> 172.21.15.7:0/2634639122 conn(0x7f79d00d1c70 msgr2=0x7f79d00d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:13.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.719+0000 7f79d6600640 1 -- 172.21.15.7:0/2634639122 shutdown_connections 2024-09-10T21:44:13.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.719+0000 7f79d6600640 1 -- 172.21.15.7:0/2634639122 wait complete. 2024-09-10T21:44:13.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.720+0000 7f79d6600640 1 Processor -- start 2024-09-10T21:44:13.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.720+0000 7f79d6600640 1 -- start start 2024-09-10T21:44:13.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.721+0000 7f79d6600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79d00cdfc0 0x7f79d01f8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.721+0000 7f79d6600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79d00ce680 0x7f79d00c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.721+0000 7f79d6600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79d00cfa50 0x7f79d015d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.721+0000 7f79d6600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f79d00d0610 con 0x7f79d00cfa50 2024-09-10T21:44:13.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.721+0000 7f79d6600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f79d00cd200 con 0x7f79d00cdfc0 2024-09-10T21:44:13.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.721+0000 7f79d6600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f79d00d0790 con 0x7f79d00ce680 2024-09-10T21:44:13.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.721+0000 7f79d5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79d00cfa50 0x7f79d015d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.722+0000 7f79d5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79d00cfa50 0x7f79d015d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59612/0 (socket says 172.21.15.7:59612) 2024-09-10T21:44:13.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.722+0000 7f79d5c00640 1 -- 172.21.15.7:0/1160561571 learned_addr learned my addr 172.21.15.7:0/1160561571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:13.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.722+0000 7f79d5200640 1 --2- 172.21.15.7:0/1160561571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79d00cdfc0 0x7f79d01f8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.722+0000 7f79c7e00640 1 --2- 172.21.15.7:0/1160561571 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79d00ce680 0x7f79d00c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.722+0000 7f79d5c00640 1 -- 172.21.15.7:0/1160561571 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79d00ce680 msgr2=0x7f79d00c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:13.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.722+0000 7f79d5c00640 1 --2- 172.21.15.7:0/1160561571 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79d00ce680 0x7f79d00c70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.722+0000 7f79d5c00640 1 -- 172.21.15.7:0/1160561571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79d00cdfc0 msgr2=0x7f79d01f8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:13.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.722+0000 7f79d5c00640 1 --2- 172.21.15.7:0/1160561571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79d00cdfc0 0x7f79d01f8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.722+0000 7f79d5c00640 1 -- 172.21.15.7:0/1160561571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79d00c7630 con 0x7f79d00cfa50 2024-09-10T21:44:13.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.722+0000 7f79d5c00640 1 --2- 172.21.15.7:0/1160561571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79d00cfa50 0x7f79d015d6e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f79c800c6b0 tx=0x7f79c800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:13.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.723+0000 7f79cd600640 1 -- 172.21.15.7:0/1160561571 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c8003b70 con 0x7f79d00cfa50 2024-09-10T21:44:13.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.723+0000 7f79cd600640 1 -- 172.21.15.7:0/1160561571 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79c8003d30 con 0x7f79d00cfa50 2024-09-10T21:44:13.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.723+0000 7f79cd600640 1 -- 172.21.15.7:0/1160561571 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c8005330 con 0x7f79d00cfa50 2024-09-10T21:44:13.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.723+0000 7f79d6600640 1 -- 172.21.15.7:0/1160561571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79d00c7900 con 0x7f79d00cfa50 2024-09-10T21:44:13.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.723+0000 7f79d6600640 1 -- 172.21.15.7:0/1160561571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79d00c7ec0 con 0x7f79d00cfa50 2024-09-10T21:44:13.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.724+0000 7f79d6600640 1 -- 172.21.15.7:0/1160561571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79d00cf2d0 con 0x7f79d00cfa50 2024-09-10T21:44:13.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.725+0000 7f79cd600640 1 -- 172.21.15.7:0/1160561571 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f79c80054d0 con 0x7f79d00cfa50 2024-09-10T21:44:13.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.726+0000 7f79cd600640 1 --2- 172.21.15.7:0/1160561571 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f79ac077f70 0x7f79ac07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:13.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.726+0000 7f79cd600640 1 -- 172.21.15.7:0/1160561571 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f79c8099b50 con 0x7f79d00cfa50 2024-09-10T21:44:13.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.726+0000 7f79d5200640 1 --2- 172.21.15.7:0/1160561571 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f79ac077f70 0x7f79ac07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:13.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.727+0000 7f79d5200640 1 --2- 172.21.15.7:0/1160561571 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f79ac077f70 0x7f79ac07a430 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f79c0005fd0 tx=0x7f79c0005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:13.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.730+0000 7f79cd600640 1 -- 172.21.15.7:0/1160561571 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f79d00cf2d0 con 0x7f79d00cfa50 2024-09-10T21:44:13.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.968+0000 7f79d6600640 1 -- 172.21.15.7:0/1160561571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79d00ce460 con 0x7f79d00cfa50 2024-09-10T21:44:13.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.968+0000 7f79cd600640 1 -- 172.21.15.7:0/1160561571 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5233 (secure 0 0 0) 0x7f79c80057f0 con 0x7f79d00cfa50 2024-09-10T21:44:13.969 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:13.970 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"btime":"2024-09-10T21:43:36:831759+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5149,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3851521050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3851521050},{"type":"v1","addr":"172.21.15.133:6836","nonce":3851521050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5289,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2709023497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2709023497},{"type":"v1","addr":"172.21.15.7:6834","nonce":2709023497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/4095876032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":4095876032},{"type":"v1","addr":"172.21.15.7:6835","nonce":4095876032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:43:33.245584+0000","modified":"2024-09-10T21:43:36.266208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234767},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234767":{"gid":234767,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3385628197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3385628197},{"type":"v1","addr":"172.21.15.133:6837","nonce":3385628197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234767,"qdb_cluster":[234767]},"id":3}]} 2024-09-10T21:44:13.970 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-09-10T21:44:13.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.970+0000 7f79d6600640 1 -- 172.21.15.7:0/1160561571 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f79ac077f70 msgr2=0x7f79ac07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:13.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.970+0000 7f79d6600640 1 --2- 172.21.15.7:0/1160561571 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f79ac077f70 0x7f79ac07a430 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f79c0005fd0 tx=0x7f79c0005850 comp rx=0 tx=0).stop 2024-09-10T21:44:13.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.970+0000 7f79d6600640 1 -- 172.21.15.7:0/1160561571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79d00cfa50 msgr2=0x7f79d015d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:13.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.970+0000 7f79d6600640 1 --2- 172.21.15.7:0/1160561571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79d00cfa50 0x7f79d015d6e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f79c800c6b0 tx=0x7f79c800cb70 comp rx=0 tx=0).stop 2024-09-10T21:44:13.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.971+0000 7f79d6600640 1 -- 172.21.15.7:0/1160561571 shutdown_connections 2024-09-10T21:44:13.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.971+0000 7f79d6600640 1 --2- 172.21.15.7:0/1160561571 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f79ac077f70 0x7f79ac07a430 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.971+0000 7f79d6600640 1 --2- 172.21.15.7:0/1160561571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f79d00cfa50 0x7f79d015d6e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.971+0000 7f79d6600640 1 --2- 172.21.15.7:0/1160561571 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f79d00ce680 0x7f79d00c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.971+0000 7f79d6600640 1 --2- 172.21.15.7:0/1160561571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f79d00cdfc0 0x7f79d01f8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:13.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.971+0000 7f79d6600640 1 -- 172.21.15.7:0/1160561571 >> 172.21.15.7:0/1160561571 conn(0x7f79d00d1c70 msgr2=0x7f79d0071950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:13.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.971+0000 7f79d6600640 1 -- 172.21.15.7:0/1160561571 shutdown_connections 2024-09-10T21:44:13.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:13.971+0000 7f79d6600640 1 -- 172.21.15.7:0/1160561571 wait complete. 2024-09-10T21:44:13.989 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-09-10T21:44:14.180 INFO:teuthology.orchestra.run.smithi133.stdout:{"message":"","return_code":0} 2024-09-10T21:44:14.180 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-10T21:44:14.180 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-09-10T21:44:14.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.257+0000 7fb798fb8f40 1 Processor -- start 2024-09-10T21:44:14.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.257+0000 7fb798fb8f40 1 -- start start 2024-09-10T21:44:14.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.258+0000 7fb798fb8f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43edc8ae0 0x55f43ee11f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.258+0000 7fb798fb8f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f43ee15110 0x55f43ee7f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.258+0000 7fb798fb8f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f43ee85620 0x55f43ee9efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.258+0000 7fb798fb8f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f43ed17c60 con 0x55f43ee85620 2024-09-10T21:44:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.258+0000 7fb798fb8f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f43ed761d0 con 0x55f43edc8ae0 2024-09-10T21:44:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.258+0000 7fb798fb8f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55f43ee15520 con 0x55f43ee15110 2024-09-10T21:44:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.258+0000 7fb796c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f43ee85620 0x55f43ee9efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.258+0000 7fb796c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f43ee85620 0x55f43ee9efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43074/0 (socket says 172.21.15.7:43074) 2024-09-10T21:44:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.258+0000 7fb796c00640 1 -- 172.21.15.7:0/3854110881 learned_addr learned my addr 172.21.15.7:0/3854110881 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.259+0000 7fb796200640 1 --2- 172.21.15.7:0/3854110881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43edc8ae0 0x55f43ee11f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.259+0000 7fb795800640 1 --2- 172.21.15.7:0/3854110881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f43ee15110 0x55f43ee7f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.259+0000 7fb796c00640 1 -- 172.21.15.7:0/3854110881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f43ee15110 msgr2=0x55f43ee7f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.259+0000 7fb796c00640 1 --2- 172.21.15.7:0/3854110881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f43ee15110 0x55f43ee7f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:14.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.259+0000 7fb796c00640 1 -- 172.21.15.7:0/3854110881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43edc8ae0 msgr2=0x55f43ee11f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:14.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.259+0000 7fb796c00640 1 --2- 172.21.15.7:0/3854110881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43edc8ae0 0x55f43ee11f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:14.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.259+0000 7fb796c00640 1 -- 172.21.15.7:0/3854110881 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f43ee7fa20 con 0x55f43ee85620 2024-09-10T21:44:14.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.259+0000 7fb795800640 1 --2- 172.21.15.7:0/3854110881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f43ee15110 0x55f43ee7f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:44:14.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.259+0000 7fb796200640 1 --2- 172.21.15.7:0/3854110881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43edc8ae0 0x55f43ee11f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:14.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.259+0000 7fb796c00640 1 --2- 172.21.15.7:0/3854110881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f43ee85620 0x55f43ee9efd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb790001660 tx=0x7fb790001b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7baf81720ffa8811 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:14.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.259+0000 7fb78fe00640 1 -- 172.21.15.7:0/3854110881 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790016070 con 0x55f43ee85620 2024-09-10T21:44:14.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.259+0000 7fb78fe00640 1 -- 172.21.15.7:0/3854110881 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb79000f800 con 0x55f43ee85620 2024-09-10T21:44:14.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.259+0000 7fb78fe00640 1 -- 172.21.15.7:0/3854110881 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79000fc10 con 0x55f43ee85620 2024-09-10T21:44:14.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.260+0000 7fb798fb8f40 1 -- 172.21.15.7:0/3854110881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f43ee85620 msgr2=0x55f43ee9efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:14.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.260+0000 7fb798fb8f40 1 --2- 172.21.15.7:0/3854110881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f43ee85620 0x55f43ee9efd0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb790001660 tx=0x7fb790001b20 comp rx=0 tx=0).stop 2024-09-10T21:44:14.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.260+0000 7fb798fb8f40 1 -- 172.21.15.7:0/3854110881 shutdown_connections 2024-09-10T21:44:14.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.260+0000 7fb798fb8f40 1 --2- 172.21.15.7:0/3854110881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f43ee85620 0x55f43ee9efd0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:14.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.260+0000 7fb798fb8f40 1 --2- 172.21.15.7:0/3854110881 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f43ee15110 0x55f43ee7f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:14.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.260+0000 7fb798fb8f40 1 --2- 172.21.15.7:0/3854110881 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43edc8ae0 0x55f43ee11f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:14.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.260+0000 7fb798fb8f40 1 -- 172.21.15.7:0/3854110881 >> 172.21.15.7:0/3854110881 conn(0x55f43edf7de0 msgr2=0x55f43edf70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:14.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.260+0000 7fb798fb8f40 1 -- 172.21.15.7:0/3854110881 shutdown_connections 2024-09-10T21:44:14.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.260+0000 7fb798fb8f40 1 -- 172.21.15.7:0/3854110881 wait complete. 2024-09-10T21:44:14.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.261+0000 7fb798fb8f40 1 Processor -- start 2024-09-10T21:44:14.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.261+0000 7fb798fb8f40 1 -- start start 2024-09-10T21:44:14.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.262+0000 7fb798fb8f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43edc8ae0 0x55f43ee83400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:14.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.262+0000 7fb798fb8f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f43ee15110 0x55f43ef39e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.262+0000 7fb798fb8f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f43ee85620 0x55f43ef3a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.262+0000 7fb798fb8f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f43ee7fd00 con 0x55f43ee85620 2024-09-10T21:44:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.262+0000 7fb798fb8f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f43ee0b120 con 0x55f43edc8ae0 2024-09-10T21:44:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.262+0000 7fb798fb8f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55f43ee0adb0 con 0x55f43ee15110 2024-09-10T21:44:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.262+0000 7fb796c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f43ee85620 0x55f43ef3a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.262+0000 7fb796c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f43ee85620 0x55f43ef3a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43078/0 (socket says 172.21.15.7:43078) 2024-09-10T21:44:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.262+0000 7fb796c00640 1 -- 172.21.15.7:0/4131121188 learned_addr learned my addr 172.21.15.7:0/4131121188 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.262+0000 7fb796c00640 1 -- 172.21.15.7:0/4131121188 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f43ee15110 msgr2=0x55f43ef39e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.262+0000 7fb796c00640 1 --2- 172.21.15.7:0/4131121188 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f43ee15110 0x55f43ef39e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.262+0000 7fb796c00640 1 -- 172.21.15.7:0/4131121188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43edc8ae0 msgr2=0x55f43ee83400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.262+0000 7fb796c00640 1 --2- 172.21.15.7:0/4131121188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f43edc8ae0 0x55f43ee83400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:14.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.262+0000 7fb796c00640 1 -- 172.21.15.7:0/4131121188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f43edfb410 con 0x55f43ee85620 2024-09-10T21:44:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.263+0000 7fb796c00640 1 --2- 172.21.15.7:0/4131121188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f43ee85620 0x55f43ef3a360 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb790003000 tx=0x7fb79000f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.263+0000 7fb78ea00640 1 -- 172.21.15.7:0/4131121188 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790021020 con 0x55f43ee85620 2024-09-10T21:44:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.263+0000 7fb78ea00640 1 -- 172.21.15.7:0/4131121188 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb790017040 con 0x55f43ee85620 2024-09-10T21:44:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.263+0000 7fb78ea00640 1 -- 172.21.15.7:0/4131121188 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790016070 con 0x55f43ee85620 2024-09-10T21:44:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.263+0000 7fb798fb8f40 1 -- 172.21.15.7:0/4131121188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f43ee9fb30 con 0x55f43ee85620 2024-09-10T21:44:14.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.263+0000 7fb798fb8f40 1 -- 172.21.15.7:0/4131121188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f43ee82950 con 0x55f43ee85620 2024-09-10T21:44:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.265+0000 7fb78ea00640 1 -- 172.21.15.7:0/4131121188 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb79001d6e0 con 0x55f43ee85620 2024-09-10T21:44:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.266+0000 7fb78ea00640 1 --2- 172.21.15.7:0/4131121188 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb778076980 0x7fb778078e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.266+0000 7fb78ea00640 1 -- 172.21.15.7:0/4131121188 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fb790099db0 con 0x55f43ee85620 2024-09-10T21:44:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.266+0000 7fb798fb8f40 1 --2- 172.21.15.7:0/4131121188 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x55f43ee83940 0x55f43ee0de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.266+0000 7fb798fb8f40 1 -- 172.21.15.7:0/4131121188 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:1 7.7 7:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46) -- 0x55f43ee7f430 con 0x55f43ee83940 2024-09-10T21:44:14.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.266+0000 7fb796200640 1 --2- 172.21.15.7:0/4131121188 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb778076980 0x7fb778078e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.267+0000 7fb795800640 1 --2- 172.21.15.7:0/4131121188 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x55f43ee83940 0x55f43ee0de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.267+0000 7fb796200640 1 --2- 172.21.15.7:0/4131121188 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb778076980 0x7fb778078e40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb788055000 tx=0x7fb788002fb0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:14.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.267+0000 7fb795800640 1 --2- 172.21.15.7:0/4131121188 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x55f43ee83940 0x55f43ee0de40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.268+0000 7fb795800640 1 -- 172.21.15.7:0/4131121188 <== osd.6 v2:172.21.15.133:6810/3365981160 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x7fb78400c890 con 0x55f43ee83940 2024-09-10T21:44:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.268+0000 7fb798fb8f40 1 -- 172.21.15.7:0/4131121188 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x55f43ee83940 msgr2=0x55f43ee0de40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.268+0000 7fb798fb8f40 1 --2- 172.21.15.7:0/4131121188 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x55f43ee83940 0x55f43ee0de40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.268+0000 7fb798fb8f40 1 -- 172.21.15.7:0/4131121188 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb778076980 msgr2=0x7fb778078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.268+0000 7fb798fb8f40 1 --2- 172.21.15.7:0/4131121188 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb778076980 0x7fb778078e40 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb788055000 tx=0x7fb788002fb0 comp rx=0 tx=0).stop 2024-09-10T21:44:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.268+0000 7fb798fb8f40 1 -- 172.21.15.7:0/4131121188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f43ee85620 msgr2=0x55f43ef3a360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.268+0000 7fb798fb8f40 1 --2- 172.21.15.7:0/4131121188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f43ee85620 0x55f43ef3a360 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fb790003000 tx=0x7fb79000f810 comp rx=0 tx=0).stop 2024-09-10T21:44:14.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.268+0000 7fb796c00640 1 -- 172.21.15.7:0/4131121188 reap_dead start 2024-09-10T21:44:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.268+0000 7fb798fb8f40 1 -- 172.21.15.7:0/4131121188 shutdown_connections 2024-09-10T21:44:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.268+0000 7fb798fb8f40 1 -- 172.21.15.7:0/4131121188 >> 172.21.15.7:0/4131121188 conn(0x55f43edf7de0 msgr2=0x55f43edf6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.268+0000 7fb798fb8f40 1 -- 172.21.15.7:0/4131121188 shutdown_connections 2024-09-10T21:44:14.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.268+0000 7fb798fb8f40 1 -- 172.21.15.7:0/4131121188 wait complete. 2024-09-10T21:44:14.273 INFO:teuthology.orchestra.run.smithi007.stdout:Yalphaparent_a 2024-09-10T21:44:14.274 DEBUG:teuthology.orchestra.run.smithi007:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-09-10T21:44:14.355 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:44:14.355 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627777, 2024-09-10T21:44:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: "ancestors": [ 2024-09-10T21:44:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: "dirino": 1099511627776, 2024-09-10T21:44:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: "dname": "alpha", 2024-09-10T21:44:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 4 2024-09-10T21:44:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: "dirino": 1, 2024-09-10T21:44:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: "dname": "parent_a", 2024-09-10T21:44:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 7 2024-09-10T21:44:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T21:44:14.356 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 7, 2024-09-10T21:44:14.357 INFO:teuthology.orchestra.run.smithi007.stdout: "old_pools": [] 2024-09-10T21:44:14.357 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:44:14.357 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:14.362 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:44:14.363 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:44:14.489 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:44:14.581 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:44:14.582 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:44:14.582 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:44:14.582 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:44:14.617 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:44:14.617 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:44:14.694 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:44:14.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.894+0000 7f5137200640 1 -- 172.21.15.7:0/1695151752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51380d1e60 msgr2=0x7f51380d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:14.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.894+0000 7f5137200640 1 --2- 172.21.15.7:0/1695151752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51380d1e60 0x7f51380d4900 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5128009830 tx=0x7f512802a5a0 comp rx=0 tx=0).stop 2024-09-10T21:44:14.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.894+0000 7f5137200640 1 -- 172.21.15.7:0/1695151752 shutdown_connections 2024-09-10T21:44:14.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.894+0000 7f5137200640 1 --2- 172.21.15.7:0/1695151752 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51380d2a20 0x7f51380d2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:14.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.894+0000 7f5137200640 1 --2- 172.21.15.7:0/1695151752 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51380d1e60 0x7f51380d4900 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:14.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.894+0000 7f5137200640 1 -- 172.21.15.7:0/1695151752 >> 172.21.15.7:0/1695151752 conn(0x7f51380d3de0 msgr2=0x7f51380d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:14.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.894+0000 7f5137200640 1 -- 172.21.15.7:0/1695151752 shutdown_connections 2024-09-10T21:44:14.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.894+0000 7f5137200640 1 -- 172.21.15.7:0/1695151752 wait complete. 2024-09-10T21:44:14.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.895+0000 7f5137200640 1 Processor -- start 2024-09-10T21:44:14.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.896+0000 7f5137200640 1 -- start start 2024-09-10T21:44:14.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.896+0000 7f5137200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51380d07f0 0x7f513814bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.896+0000 7f5137200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51380d1e60 0x7f513814c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.897+0000 7f5137200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51380d2a20 0x7f51381f8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.897+0000 7f5137200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f51380d1500 con 0x7f51380d1e60 2024-09-10T21:44:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.897+0000 7f5137200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f51380d1380 con 0x7f51380d2a20 2024-09-10T21:44:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.897+0000 7f5137200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f51380d1680 con 0x7f51380d07f0 2024-09-10T21:44:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.897+0000 7f5135400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51380d1e60 0x7f513814c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.897+0000 7f5135400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51380d1e60 0x7f513814c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43090/0 (socket says 172.21.15.7:43090) 2024-09-10T21:44:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.897+0000 7f5135400640 1 -- 172.21.15.7:0/490108212 learned_addr learned my addr 172.21.15.7:0/490108212 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.897+0000 7f5136800640 1 --2- 172.21.15.7:0/490108212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51380d2a20 0x7f51381f8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.897+0000 7f5135e00640 1 --2- 172.21.15.7:0/490108212 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51380d07f0 0x7f513814bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.897+0000 7f5135400640 1 -- 172.21.15.7:0/490108212 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51380d07f0 msgr2=0x7f513814bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:14.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.897+0000 7f5135400640 1 --2- 172.21.15.7:0/490108212 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51380d07f0 0x7f513814bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.897+0000 7f5135400640 1 -- 172.21.15.7:0/490108212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51380d2a20 msgr2=0x7f51381f8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.897+0000 7f5135400640 1 --2- 172.21.15.7:0/490108212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51380d2a20 0x7f51381f8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.897+0000 7f5135400640 1 -- 172.21.15.7:0/490108212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f513815a1e0 con 0x7f51380d1e60 2024-09-10T21:44:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.897+0000 7f5136800640 1 --2- 172.21.15.7:0/490108212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51380d2a20 0x7f51381f8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:14.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.898+0000 7f5135400640 1 --2- 172.21.15.7:0/490108212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51380d1e60 0x7f513814c330 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f512400e910 tx=0x7f512400edd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.898+0000 7f512e000640 1 -- 172.21.15.7:0/490108212 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f512400cd80 con 0x7f51380d1e60 2024-09-10T21:44:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.898+0000 7f512e000640 1 -- 172.21.15.7:0/490108212 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5124010430 con 0x7f51380d1e60 2024-09-10T21:44:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.898+0000 7f5137200640 1 -- 172.21.15.7:0/490108212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f513815a4c0 con 0x7f51380d1e60 2024-09-10T21:44:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.898+0000 7f512e000640 1 -- 172.21.15.7:0/490108212 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5124010860 con 0x7f51380d1e60 2024-09-10T21:44:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.898+0000 7f5137200640 1 -- 172.21.15.7:0/490108212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5138154800 con 0x7f51380d1e60 2024-09-10T21:44:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.899+0000 7f5137200640 1 -- 172.21.15.7:0/490108212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51380c94f0 con 0x7f51380d1e60 2024-09-10T21:44:14.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.900+0000 7f512e000640 1 -- 172.21.15.7:0/490108212 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5124010a00 con 0x7f51380d1e60 2024-09-10T21:44:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.902+0000 7f512e000640 1 --2- 172.21.15.7:0/490108212 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5118077f70 0x7f511807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.902+0000 7f512e000640 1 -- 172.21.15.7:0/490108212 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f512409a920 con 0x7f51380d1e60 2024-09-10T21:44:14.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.902+0000 7f5135e00640 1 --2- 172.21.15.7:0/490108212 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5118077f70 0x7f511807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:14.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.903+0000 7f5135e00640 1 --2- 172.21.15.7:0/490108212 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5118077f70 0x7f511807a430 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5128034000 tx=0x7f5128004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:14.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:14.905+0000 7f512e000640 1 -- 172.21.15.7:0/490108212 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f51240633d0 con 0x7f51380d1e60 2024-09-10T21:44:15.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.112+0000 7f5137200640 1 -- 172.21.15.7:0/490108212 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f51380ca0d0 con 0x7f51380d1e60 2024-09-10T21:44:15.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.112+0000 7f512e000640 1 -- 172.21.15.7:0/490108212 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v46) ==== 80+0+0 (secure 0 0 0) 0x7f5124062cd0 con 0x7f51380d1e60 2024-09-10T21:44:15.113 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-10T21:44:15.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.115+0000 7f5137200640 1 -- 172.21.15.7:0/490108212 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5118077f70 msgr2=0x7f511807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:15.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.115+0000 7f5137200640 1 --2- 172.21.15.7:0/490108212 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5118077f70 0x7f511807a430 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f5128034000 tx=0x7f5128004730 comp rx=0 tx=0).stop 2024-09-10T21:44:15.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.115+0000 7f5137200640 1 -- 172.21.15.7:0/490108212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51380d1e60 msgr2=0x7f513814c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:15.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.115+0000 7f5137200640 1 --2- 172.21.15.7:0/490108212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51380d1e60 0x7f513814c330 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f512400e910 tx=0x7f512400edd0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.115+0000 7f5137200640 1 -- 172.21.15.7:0/490108212 shutdown_connections 2024-09-10T21:44:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.115+0000 7f5137200640 1 --2- 172.21.15.7:0/490108212 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5118077f70 0x7f511807a430 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.115+0000 7f5137200640 1 --2- 172.21.15.7:0/490108212 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51380d2a20 0x7f51381f8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.115+0000 7f5137200640 1 --2- 172.21.15.7:0/490108212 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51380d1e60 0x7f513814c330 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.115+0000 7f5137200640 1 --2- 172.21.15.7:0/490108212 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51380d07f0 0x7f513814bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.115+0000 7f5137200640 1 -- 172.21.15.7:0/490108212 >> 172.21.15.7:0/490108212 conn(0x7f51380d3de0 msgr2=0x7f51380d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.115+0000 7f5137200640 1 -- 172.21.15.7:0/490108212 shutdown_connections 2024-09-10T21:44:15.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.115+0000 7f5137200640 1 -- 172.21.15.7:0/490108212 wait complete. 2024-09-10T21:44:15.133 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:44:15.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:44:15.133 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:44:15.232 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:44:15.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:44:15.233 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:44:15.275 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:44:15.275 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:44:15.275 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:44:15.311 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:44:15.311 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:44:15.311 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:44:15.311 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:44:15.351 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:44:15.351 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:44:15.351 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:44:15.352 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:44:15.386 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:44:15.386 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:44:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.549+0000 7fbf30e00640 1 -- 172.21.15.7:0/436010165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf2c15bc90 msgr2=0x7fbf2c0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.549+0000 7fbf30e00640 1 --2- 172.21.15.7:0/436010165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf2c15bc90 0x7fbf2c0ccef0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c005fd0 tx=0x7fbf1c02a410 comp rx=0 tx=0).stop 2024-09-10T21:44:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.549+0000 7fbf30e00640 1 -- 172.21.15.7:0/436010165 shutdown_connections 2024-09-10T21:44:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.549+0000 7fbf30e00640 1 --2- 172.21.15.7:0/436010165 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf2c0cd750 0x7fbf2c073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.549+0000 7fbf30e00640 1 --2- 172.21.15.7:0/436010165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf2c15bc90 0x7fbf2c0ccef0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.549+0000 7fbf30e00640 1 -- 172.21.15.7:0/436010165 >> 172.21.15.7:0/436010165 conn(0x7fbf2c0b16d0 msgr2=0x7fbf2c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.549+0000 7fbf30e00640 1 -- 172.21.15.7:0/436010165 shutdown_connections 2024-09-10T21:44:15.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.549+0000 7fbf30e00640 1 -- 172.21.15.7:0/436010165 wait complete. 2024-09-10T21:44:15.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.551+0000 7fbf30e00640 1 Processor -- start 2024-09-10T21:44:15.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.551+0000 7fbf30e00640 1 -- start start 2024-09-10T21:44:15.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.552+0000 7fbf30e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf2c073c70 0x7fbf2c157670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:15.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.552+0000 7fbf30e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf2c0cd750 0x7fbf2c0c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:15.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.552+0000 7fbf30e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf2c15bc90 0x7fbf2c0c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.552+0000 7fbf30e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c06fcf0 con 0x7fbf2c15bc90 2024-09-10T21:44:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.552+0000 7fbf30e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbf2c1606c0 con 0x7fbf2c0cd750 2024-09-10T21:44:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.552+0000 7fbf30e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fbf2c06fe70 con 0x7fbf2c073c70 2024-09-10T21:44:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.552+0000 7fbf2a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf2c15bc90 0x7fbf2c0c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.552+0000 7fbf2a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf2c15bc90 0x7fbf2c0c7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43098/0 (socket says 172.21.15.7:43098) 2024-09-10T21:44:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.552+0000 7fbf21400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf2c0cd750 0x7fbf2c0c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.552+0000 7fbf2a800640 1 -- 172.21.15.7:0/965356777 learned_addr learned my addr 172.21.15.7:0/965356777 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.552+0000 7fbf29e00640 1 --2- 172.21.15.7:0/965356777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf2c073c70 0x7fbf2c157670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.553+0000 7fbf2a800640 1 -- 172.21.15.7:0/965356777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf2c073c70 msgr2=0x7fbf2c157670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.553+0000 7fbf2a800640 1 --2- 172.21.15.7:0/965356777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf2c073c70 0x7fbf2c157670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.553+0000 7fbf2a800640 1 -- 172.21.15.7:0/965356777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf2c0cd750 msgr2=0x7fbf2c0c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.553+0000 7fbf2a800640 1 --2- 172.21.15.7:0/965356777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf2c0cd750 0x7fbf2c0c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.553+0000 7fbf2a800640 1 -- 172.21.15.7:0/965356777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf2c1f2690 con 0x7fbf2c15bc90 2024-09-10T21:44:15.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.553+0000 7fbf2a800640 1 --2- 172.21.15.7:0/965356777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf2c15bc90 0x7fbf2c0c7190 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400c780 tx=0x7fbf2400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:15.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.553+0000 7fbf22a00640 1 -- 172.21.15.7:0/965356777 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf24016020 con 0x7fbf2c15bc90 2024-09-10T21:44:15.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.553+0000 7fbf22a00640 1 -- 172.21.15.7:0/965356777 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf2400a8f0 con 0x7fbf2c15bc90 2024-09-10T21:44:15.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.553+0000 7fbf22a00640 1 -- 172.21.15.7:0/965356777 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf2400adc0 con 0x7fbf2c15bc90 2024-09-10T21:44:15.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.553+0000 7fbf30e00640 1 -- 172.21.15.7:0/965356777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf2c0c9af0 con 0x7fbf2c15bc90 2024-09-10T21:44:15.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.553+0000 7fbf30e00640 1 -- 172.21.15.7:0/965356777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf2c15c260 con 0x7fbf2c15bc90 2024-09-10T21:44:15.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.554+0000 7fbf30e00640 1 -- 172.21.15.7:0/965356777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf2c073730 con 0x7fbf2c15bc90 2024-09-10T21:44:15.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.556+0000 7fbf22a00640 1 -- 172.21.15.7:0/965356777 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fbf2401b400 con 0x7fbf2c15bc90 2024-09-10T21:44:15.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.556+0000 7fbf22a00640 1 --2- 172.21.15.7:0/965356777 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf08077f70 0x7fbf0807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:15.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.557+0000 7fbf22a00640 1 -- 172.21.15.7:0/965356777 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fbf2409a210 con 0x7fbf2c15bc90 2024-09-10T21:44:15.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.557+0000 7fbf29e00640 1 --2- 172.21.15.7:0/965356777 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf08077f70 0x7fbf0807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:15.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.558+0000 7fbf29e00640 1 --2- 172.21.15.7:0/965356777 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf08077f70 0x7fbf0807a430 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c033000 tx=0x7fbf1c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:15.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.561+0000 7fbf22a00640 1 -- 172.21.15.7:0/965356777 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fbf24020020 con 0x7fbf2c15bc90 2024-09-10T21:44:15.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.792+0000 7fbf30e00640 1 -- 172.21.15.7:0/965356777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbf2c15b560 con 0x7fbf2c15bc90 2024-09-10T21:44:15.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.792+0000 7fbf22a00640 1 -- 172.21.15.7:0/965356777 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5233 (secure 0 0 0) 0x7fbf2406b3e0 con 0x7fbf2c15bc90 2024-09-10T21:44:15.794 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:15.794 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"btime":"2024-09-10T21:43:36:831759+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5149,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3851521050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3851521050},{"type":"v1","addr":"172.21.15.133:6836","nonce":3851521050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5289,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2709023497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2709023497},{"type":"v1","addr":"172.21.15.7:6834","nonce":2709023497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/4095876032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":4095876032},{"type":"v1","addr":"172.21.15.7:6835","nonce":4095876032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:43:33.245584+0000","modified":"2024-09-10T21:43:36.266208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234767},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234767":{"gid":234767,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3385628197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3385628197},{"type":"v1","addr":"172.21.15.133:6837","nonce":3385628197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234767,"qdb_cluster":[234767]},"id":3}]} 2024-09-10T21:44:15.794 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-09-10T21:44:15.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.795+0000 7fbf30e00640 1 -- 172.21.15.7:0/965356777 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf08077f70 msgr2=0x7fbf0807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:15.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.795+0000 7fbf30e00640 1 --2- 172.21.15.7:0/965356777 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf08077f70 0x7fbf0807a430 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c033000 tx=0x7fbf1c004730 comp rx=0 tx=0).stop 2024-09-10T21:44:15.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.795+0000 7fbf30e00640 1 -- 172.21.15.7:0/965356777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf2c15bc90 msgr2=0x7fbf2c0c7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:15.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.795+0000 7fbf30e00640 1 --2- 172.21.15.7:0/965356777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf2c15bc90 0x7fbf2c0c7190 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400c780 tx=0x7fbf2400cc40 comp rx=0 tx=0).stop 2024-09-10T21:44:15.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.795+0000 7fbf30e00640 1 -- 172.21.15.7:0/965356777 shutdown_connections 2024-09-10T21:44:15.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.795+0000 7fbf30e00640 1 --2- 172.21.15.7:0/965356777 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf08077f70 0x7fbf0807a430 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.795+0000 7fbf30e00640 1 --2- 172.21.15.7:0/965356777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf2c0cd750 0x7fbf2c0c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.795+0000 7fbf30e00640 1 --2- 172.21.15.7:0/965356777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf2c15bc90 0x7fbf2c0c7190 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.795+0000 7fbf30e00640 1 --2- 172.21.15.7:0/965356777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf2c073c70 0x7fbf2c157670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.795+0000 7fbf30e00640 1 -- 172.21.15.7:0/965356777 >> 172.21.15.7:0/965356777 conn(0x7fbf2c0b16d0 msgr2=0x7fbf2c15fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:15.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.795+0000 7fbf30e00640 1 -- 172.21.15.7:0/965356777 shutdown_connections 2024-09-10T21:44:15.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.795+0000 7fbf30e00640 1 -- 172.21.15.7:0/965356777 wait complete. 2024-09-10T21:44:15.812 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:44:15.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.974+0000 7efe74000640 1 -- 172.21.15.7:0/3572592001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe6c15bc90 msgr2=0x7efe6c0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:15.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.974+0000 7efe74000640 1 --2- 172.21.15.7:0/3572592001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe6c15bc90 0x7efe6c0ccef0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7efe6c0cf860 tx=0x7efe6002a4d0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.974+0000 7efe74000640 1 -- 172.21.15.7:0/3572592001 shutdown_connections 2024-09-10T21:44:15.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.974+0000 7efe74000640 1 --2- 172.21.15.7:0/3572592001 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe6c073c70 0x7efe6c15fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.974+0000 7efe74000640 1 --2- 172.21.15.7:0/3572592001 >> v2:172.21.15.133:3300/0 conn(0x7efe6c0cd750 0x7efe6c073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.974+0000 7efe74000640 1 --2- 172.21.15.7:0/3572592001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe6c15bc90 0x7efe6c0ccef0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.974+0000 7efe74000640 1 -- 172.21.15.7:0/3572592001 >> 172.21.15.7:0/3572592001 conn(0x7efe6c0b16d0 msgr2=0x7efe6c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:15.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.974+0000 7efe74000640 1 -- 172.21.15.7:0/3572592001 shutdown_connections 2024-09-10T21:44:15.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.974+0000 7efe74000640 1 -- 172.21.15.7:0/3572592001 wait complete. 2024-09-10T21:44:15.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.975+0000 7efe74000640 1 Processor -- start 2024-09-10T21:44:15.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.976+0000 7efe74000640 1 -- start start 2024-09-10T21:44:15.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.977+0000 7efe74000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe6c073c70 0x7efe6c15ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:15.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.977+0000 7efe74000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe6c0cd750 0x7efe6c15d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:15.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.977+0000 7efe74000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe6c15bc90 0x7efe6c072b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.977+0000 7efe74000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c160670 con 0x7efe6c15bc90 2024-09-10T21:44:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.977+0000 7efe74000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efe6c1604f0 con 0x7efe6c0cd750 2024-09-10T21:44:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.977+0000 7efe74000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7efe6c06fcf0 con 0x7efe6c073c70 2024-09-10T21:44:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.977+0000 7efe72000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe6c15bc90 0x7efe6c072b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:15.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.977+0000 7efe72000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe6c15bc90 0x7efe6c072b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43114/0 (socket says 172.21.15.7:43114) 2024-09-10T21:44:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.977+0000 7efe72000640 1 -- 172.21.15.7:0/2071424474 learned_addr learned my addr 172.21.15.7:0/2071424474 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.977+0000 7efe71600640 1 --2- 172.21.15.7:0/2071424474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe6c073c70 0x7efe6c15ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.977+0000 7efe70c00640 1 --2- 172.21.15.7:0/2071424474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe6c0cd750 0x7efe6c15d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.977+0000 7efe72000640 1 -- 172.21.15.7:0/2071424474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe6c073c70 msgr2=0x7efe6c15ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.977+0000 7efe72000640 1 --2- 172.21.15.7:0/2071424474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe6c073c70 0x7efe6c15ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.977+0000 7efe72000640 1 -- 172.21.15.7:0/2071424474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe6c0cd750 msgr2=0x7efe6c15d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.977+0000 7efe72000640 1 --2- 172.21.15.7:0/2071424474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe6c0cd750 0x7efe6c15d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.977+0000 7efe72000640 1 -- 172.21.15.7:0/2071424474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe6c1f80a0 con 0x7efe6c15bc90 2024-09-10T21:44:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.978+0000 7efe72000640 1 --2- 172.21.15.7:0/2071424474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe6c15bc90 0x7efe6c072b20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7efe6400ea90 tx=0x7efe6400ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.978+0000 7efe69600640 1 -- 172.21.15.7:0/2071424474 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6400cea0 con 0x7efe6c15bc90 2024-09-10T21:44:15.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.978+0000 7efe69600640 1 -- 172.21.15.7:0/2071424474 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe64004b20 con 0x7efe6c15bc90 2024-09-10T21:44:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.978+0000 7efe69600640 1 -- 172.21.15.7:0/2071424474 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64005020 con 0x7efe6c15bc90 2024-09-10T21:44:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.978+0000 7efe74000640 1 -- 172.21.15.7:0/2071424474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6c1f8380 con 0x7efe6c15bc90 2024-09-10T21:44:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.978+0000 7efe74000640 1 -- 172.21.15.7:0/2071424474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe6c1f8940 con 0x7efe6c15bc90 2024-09-10T21:44:15.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.979+0000 7efe74000640 1 -- 172.21.15.7:0/2071424474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe6c073730 con 0x7efe6c15bc90 2024-09-10T21:44:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.980+0000 7efe69600640 1 -- 172.21.15.7:0/2071424474 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7efe640051c0 con 0x7efe6c15bc90 2024-09-10T21:44:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.982+0000 7efe69600640 1 --2- 172.21.15.7:0/2071424474 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efe4c077f70 0x7efe4c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.982+0000 7efe69600640 1 -- 172.21.15.7:0/2071424474 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7efe6409a860 con 0x7efe6c15bc90 2024-09-10T21:44:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.982+0000 7efe71600640 1 --2- 172.21.15.7:0/2071424474 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efe4c077f70 0x7efe4c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:15.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.983+0000 7efe71600640 1 --2- 172.21.15.7:0/2071424474 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efe4c077f70 0x7efe4c07a430 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efe60033000 tx=0x7efe60004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:15.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:15.986+0000 7efe69600640 1 -- 172.21.15.7:0/2071424474 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7efe64062cc0 con 0x7efe6c15bc90 2024-09-10T21:44:16.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.217+0000 7efe74000640 1 -- 172.21.15.7:0/2071424474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe6c0c6630 con 0x7efe6c15bc90 2024-09-10T21:44:16.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.218+0000 7efe69600640 1 -- 172.21.15.7:0/2071424474 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5233 (secure 0 0 0) 0x7efe640676f0 con 0x7efe6c15bc90 2024-09-10T21:44:16.219 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:16.219 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"btime":"2024-09-10T21:43:36:831759+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5149,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3851521050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3851521050},{"type":"v1","addr":"172.21.15.133:6836","nonce":3851521050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5289,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2709023497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2709023497},{"type":"v1","addr":"172.21.15.7:6834","nonce":2709023497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/4095876032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":4095876032},{"type":"v1","addr":"172.21.15.7:6835","nonce":4095876032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:43:33.245584+0000","modified":"2024-09-10T21:43:36.266208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234767},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234767":{"gid":234767,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3385628197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3385628197},{"type":"v1","addr":"172.21.15.133:6837","nonce":3385628197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234767,"qdb_cluster":[234767]},"id":3}]} 2024-09-10T21:44:16.219 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-09-10T21:44:16.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.220+0000 7efe74000640 1 -- 172.21.15.7:0/2071424474 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efe4c077f70 msgr2=0x7efe4c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.220+0000 7efe74000640 1 --2- 172.21.15.7:0/2071424474 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efe4c077f70 0x7efe4c07a430 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efe60033000 tx=0x7efe60004730 comp rx=0 tx=0).stop 2024-09-10T21:44:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.220+0000 7efe74000640 1 -- 172.21.15.7:0/2071424474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe6c15bc90 msgr2=0x7efe6c072b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.220+0000 7efe74000640 1 --2- 172.21.15.7:0/2071424474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe6c15bc90 0x7efe6c072b20 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7efe6400ea90 tx=0x7efe6400ef50 comp rx=0 tx=0).stop 2024-09-10T21:44:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.220+0000 7efe74000640 1 -- 172.21.15.7:0/2071424474 shutdown_connections 2024-09-10T21:44:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.220+0000 7efe74000640 1 --2- 172.21.15.7:0/2071424474 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efe4c077f70 0x7efe4c07a430 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.220+0000 7efe74000640 1 --2- 172.21.15.7:0/2071424474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efe6c0cd750 0x7efe6c15d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.220+0000 7efe74000640 1 --2- 172.21.15.7:0/2071424474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efe6c15bc90 0x7efe6c072b20 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.220+0000 7efe74000640 1 --2- 172.21.15.7:0/2071424474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efe6c073c70 0x7efe6c15ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.220+0000 7efe74000640 1 -- 172.21.15.7:0/2071424474 >> 172.21.15.7:0/2071424474 conn(0x7efe6c0b16d0 msgr2=0x7efe6c0b4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.220+0000 7efe74000640 1 -- 172.21.15.7:0/2071424474 shutdown_connections 2024-09-10T21:44:16.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.220+0000 7efe74000640 1 -- 172.21.15.7:0/2071424474 wait complete. 2024-09-10T21:44:16.237 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:44:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.396+0000 7f31bca00640 1 -- 172.21.15.7:0/2288859557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b80b2b50 msgr2=0x7f31b80d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.396+0000 7f31bca00640 1 --2- 172.21.15.7:0/2288859557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b80b2b50 0x7f31b80d0760 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f31a0005fd0 tx=0x7f31a002a540 comp rx=0 tx=0).stop 2024-09-10T21:44:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.396+0000 7f31bca00640 1 -- 172.21.15.7:0/2288859557 shutdown_connections 2024-09-10T21:44:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.396+0000 7f31bca00640 1 --2- 172.21.15.7:0/2288859557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b80b2b50 0x7f31b80d0760 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.396+0000 7f31bca00640 1 --2- 172.21.15.7:0/2288859557 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31b815dfd0 0x7f31b80d4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.396+0000 7f31bca00640 1 -- 172.21.15.7:0/2288859557 >> 172.21.15.7:0/2288859557 conn(0x7f31b80d3d50 msgr2=0x7f31b80d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.396+0000 7f31bca00640 1 -- 172.21.15.7:0/2288859557 shutdown_connections 2024-09-10T21:44:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.396+0000 7f31bca00640 1 -- 172.21.15.7:0/2288859557 wait complete. 2024-09-10T21:44:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.397+0000 7f31bca00640 1 Processor -- start 2024-09-10T21:44:16.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.398+0000 7f31bca00640 1 -- start start 2024-09-10T21:44:16.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.398+0000 7f31bca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b80b2b50 0x7f31b8147b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:16.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.399+0000 7f31bca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b80d0ca0 0x7f31b80cae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:16.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.399+0000 7f31bca00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31b815dfd0 0x7f31b80cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:16.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.399+0000 7f31bca00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f31b80b08c0 con 0x7f31b80b2b50 2024-09-10T21:44:16.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.399+0000 7f31bca00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f31b80d2ef0 con 0x7f31b80d0ca0 2024-09-10T21:44:16.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.399+0000 7f31bca00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f31b80b0a40 con 0x7f31b815dfd0 2024-09-10T21:44:16.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.399+0000 7f31b7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b80b2b50 0x7f31b8147b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:16.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.399+0000 7f31b7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b80b2b50 0x7f31b8147b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43140/0 (socket says 172.21.15.7:43140) 2024-09-10T21:44:16.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.399+0000 7f31b7400640 1 -- 172.21.15.7:0/1057708903 learned_addr learned my addr 172.21.15.7:0/1057708903 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:16.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.399+0000 7f31aea00640 1 --2- 172.21.15.7:0/1057708903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b80d0ca0 0x7f31b80cae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:16.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.399+0000 7f31b7e00640 1 --2- 172.21.15.7:0/1057708903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31b815dfd0 0x7f31b80cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:16.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.399+0000 7f31b7400640 1 -- 172.21.15.7:0/1057708903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31b815dfd0 msgr2=0x7f31b80cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:16.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.399+0000 7f31b7400640 1 --2- 172.21.15.7:0/1057708903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31b815dfd0 0x7f31b80cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:16.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.399+0000 7f31b7400640 1 -- 172.21.15.7:0/1057708903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b80d0ca0 msgr2=0x7f31b80cae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:16.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.399+0000 7f31b7400640 1 --2- 172.21.15.7:0/1057708903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b80d0ca0 0x7f31b80cae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:16.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.399+0000 7f31b7400640 1 -- 172.21.15.7:0/1057708903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31b81f4c20 con 0x7f31b80b2b50 2024-09-10T21:44:16.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.399+0000 7f31aea00640 1 --2- 172.21.15.7:0/1057708903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b80d0ca0 0x7f31b80cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:16.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.400+0000 7f31b7400640 1 --2- 172.21.15.7:0/1057708903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b80b2b50 0x7f31b8147b70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f31a800e790 tx=0x7f31a800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:16.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.400+0000 7f31afe00640 1 -- 172.21.15.7:0/1057708903 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a800eea0 con 0x7f31b80b2b50 2024-09-10T21:44:16.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.400+0000 7f31afe00640 1 -- 172.21.15.7:0/1057708903 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31a800cdb0 con 0x7f31b80b2b50 2024-09-10T21:44:16.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.400+0000 7f31afe00640 1 -- 172.21.15.7:0/1057708903 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8010690 con 0x7f31b80b2b50 2024-09-10T21:44:16.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.400+0000 7f31bca00640 1 -- 172.21.15.7:0/1057708903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31b80cdd60 con 0x7f31b80b2b50 2024-09-10T21:44:16.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.400+0000 7f31bca00640 1 -- 172.21.15.7:0/1057708903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31b80b7fe0 con 0x7f31b80b2b50 2024-09-10T21:44:16.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.401+0000 7f31bca00640 1 -- 172.21.15.7:0/1057708903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31b80d1610 con 0x7f31b80b2b50 2024-09-10T21:44:16.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.402+0000 7f31afe00640 1 -- 172.21.15.7:0/1057708903 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f31a8010830 con 0x7f31b80b2b50 2024-09-10T21:44:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.403+0000 7f31afe00640 1 --2- 172.21.15.7:0/1057708903 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3194077fa0 0x7f319407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.403+0000 7f31afe00640 1 -- 172.21.15.7:0/1057708903 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f31a809a6e0 con 0x7f31b80b2b50 2024-09-10T21:44:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.404+0000 7f31aea00640 1 --2- 172.21.15.7:0/1057708903 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3194077fa0 0x7f319407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.405+0000 7f31aea00640 1 --2- 172.21.15.7:0/1057708903 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3194077fa0 0x7f319407a460 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f31a0033000 tx=0x7f31a0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:16.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.408+0000 7f31afe00640 1 -- 172.21.15.7:0/1057708903 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f31b80d1610 con 0x7f31b80b2b50 2024-09-10T21:44:16.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.604+0000 7f31bca00640 1 -- 172.21.15.7:0/1057708903 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f31b80d0760 con 0x7f31b80b2b50 2024-09-10T21:44:16.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.605+0000 7f31afe00640 1 -- 172.21.15.7:0/1057708903 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16733 (secure 0 0 0) 0x7f31a8062b40 con 0x7f31b80b2b50 2024-09-10T21:44:16.607 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:16.607 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":46,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:43:48.882487+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:43:31.081211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T21:43:32.202448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.12","mappings":[{"from":0,"to":5}]},{"pgid":"7.3b","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:44:16.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.608+0000 7f31bca00640 1 -- 172.21.15.7:0/1057708903 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3194077fa0 msgr2=0x7f319407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:16.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.608+0000 7f31bca00640 1 --2- 172.21.15.7:0/1057708903 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3194077fa0 0x7f319407a460 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f31a0033000 tx=0x7f31a0004730 comp rx=0 tx=0).stop 2024-09-10T21:44:16.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.608+0000 7f31bca00640 1 -- 172.21.15.7:0/1057708903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b80b2b50 msgr2=0x7f31b8147b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:16.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.608+0000 7f31bca00640 1 --2- 172.21.15.7:0/1057708903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b80b2b50 0x7f31b8147b70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f31a800e790 tx=0x7f31a800ec50 comp rx=0 tx=0).stop 2024-09-10T21:44:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.608+0000 7f31bca00640 1 -- 172.21.15.7:0/1057708903 shutdown_connections 2024-09-10T21:44:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.608+0000 7f31bca00640 1 --2- 172.21.15.7:0/1057708903 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3194077fa0 0x7f319407a460 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.608+0000 7f31bca00640 1 --2- 172.21.15.7:0/1057708903 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f31b80d0ca0 0x7f31b80cae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.608+0000 7f31bca00640 1 --2- 172.21.15.7:0/1057708903 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31b815dfd0 0x7f31b80cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.608+0000 7f31bca00640 1 --2- 172.21.15.7:0/1057708903 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31b80b2b50 0x7f31b8147b70 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.608+0000 7f31bca00640 1 -- 172.21.15.7:0/1057708903 >> 172.21.15.7:0/1057708903 conn(0x7f31b80d3d50 msgr2=0x7f31b80d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.608+0000 7f31bca00640 1 -- 172.21.15.7:0/1057708903 shutdown_connections 2024-09-10T21:44:16.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.608+0000 7f31bca00640 1 -- 172.21.15.7:0/1057708903 wait complete. 2024-09-10T21:44:16.625 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:44:16.626 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:44:16.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.785+0000 7efc42c00640 1 -- 172.21.15.7:0/3852723864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc3c15bc90 msgr2=0x7efc3c0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:16.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.785+0000 7efc42c00640 1 --2- 172.21.15.7:0/3852723864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc3c15bc90 0x7efc3c0ccef0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0cf860 tx=0x7efc2c02a4d0 comp rx=0 tx=0).stop 2024-09-10T21:44:16.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.785+0000 7efc42c00640 1 -- 172.21.15.7:0/3852723864 shutdown_connections 2024-09-10T21:44:16.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.785+0000 7efc42c00640 1 --2- 172.21.15.7:0/3852723864 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efc3c073c70 0x7efc3c15fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:16.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.785+0000 7efc42c00640 1 --2- 172.21.15.7:0/3852723864 >> v2:172.21.15.133:3300/0 conn(0x7efc3c0cd750 0x7efc3c073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:16.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.785+0000 7efc42c00640 1 --2- 172.21.15.7:0/3852723864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc3c15bc90 0x7efc3c0ccef0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:16.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.785+0000 7efc42c00640 1 -- 172.21.15.7:0/3852723864 >> 172.21.15.7:0/3852723864 conn(0x7efc3c0b16d0 msgr2=0x7efc3c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:16.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.785+0000 7efc42c00640 1 -- 172.21.15.7:0/3852723864 shutdown_connections 2024-09-10T21:44:16.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.785+0000 7efc42c00640 1 -- 172.21.15.7:0/3852723864 wait complete. 2024-09-10T21:44:16.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.787+0000 7efc42c00640 1 Processor -- start 2024-09-10T21:44:16.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.787+0000 7efc42c00640 1 -- start start 2024-09-10T21:44:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.788+0000 7efc42c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc3c073c70 0x7efc3c15f100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.788+0000 7efc42c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc3c0cd750 0x7efc3c157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.788+0000 7efc42c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efc3c15bc90 0x7efc3c0ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.788+0000 7efc42c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c160670 con 0x7efc3c073c70 2024-09-10T21:44:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.788+0000 7efc42c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c1604f0 con 0x7efc3c0cd750 2024-09-10T21:44:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.788+0000 7efc42c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7efc3c06fcf0 con 0x7efc3c15bc90 2024-09-10T21:44:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.788+0000 7efc3be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc3c073c70 0x7efc3c15f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.788+0000 7efc3be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc3c073c70 0x7efc3c15f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43150/0 (socket says 172.21.15.7:43150) 2024-09-10T21:44:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.788+0000 7efc3be00640 1 -- 172.21.15.7:0/4159495971 learned_addr learned my addr 172.21.15.7:0/4159495971 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:16.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.788+0000 7efc33400640 1 --2- 172.21.15.7:0/4159495971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc3c0cd750 0x7efc3c157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.788+0000 7efc40c00640 1 --2- 172.21.15.7:0/4159495971 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efc3c15bc90 0x7efc3c0ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.789+0000 7efc3be00640 1 -- 172.21.15.7:0/4159495971 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efc3c15bc90 msgr2=0x7efc3c0ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.789+0000 7efc3be00640 1 --2- 172.21.15.7:0/4159495971 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efc3c15bc90 0x7efc3c0ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.789+0000 7efc3be00640 1 -- 172.21.15.7:0/4159495971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc3c0cd750 msgr2=0x7efc3c157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.789+0000 7efc3be00640 1 --2- 172.21.15.7:0/4159495971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc3c0cd750 0x7efc3c157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:16.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.789+0000 7efc3be00640 1 -- 172.21.15.7:0/4159495971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3c15d780 con 0x7efc3c073c70 2024-09-10T21:44:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.789+0000 7efc3be00640 1 --2- 172.21.15.7:0/4159495971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc3c073c70 0x7efc3c15f100 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7efc2c034040 tx=0x7efc2c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.789+0000 7efc38c00640 1 -- 172.21.15.7:0/4159495971 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c03e030 con 0x7efc3c073c70 2024-09-10T21:44:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.789+0000 7efc38c00640 1 -- 172.21.15.7:0/4159495971 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc2c0047a0 con 0x7efc3c073c70 2024-09-10T21:44:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.789+0000 7efc38c00640 1 -- 172.21.15.7:0/4159495971 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c035040 con 0x7efc3c073c70 2024-09-10T21:44:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.789+0000 7efc42c00640 1 -- 172.21.15.7:0/4159495971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3c15da60 con 0x7efc3c073c70 2024-09-10T21:44:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.789+0000 7efc42c00640 1 -- 172.21.15.7:0/4159495971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc3c15e020 con 0x7efc3c073c70 2024-09-10T21:44:16.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.790+0000 7efc42c00640 1 -- 172.21.15.7:0/4159495971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc3c0d0950 con 0x7efc3c073c70 2024-09-10T21:44:16.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.792+0000 7efc38c00640 1 -- 172.21.15.7:0/4159495971 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7efc2c004bf0 con 0x7efc3c073c70 2024-09-10T21:44:16.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.793+0000 7efc38c00640 1 --2- 172.21.15.7:0/4159495971 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efc18077f70 0x7efc1807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:16.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.793+0000 7efc38c00640 1 -- 172.21.15.7:0/4159495971 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7efc2c0ba700 con 0x7efc3c073c70 2024-09-10T21:44:16.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.793+0000 7efc33400640 1 --2- 172.21.15.7:0/4159495971 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efc18077f70 0x7efc1807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:16.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.794+0000 7efc33400640 1 --2- 172.21.15.7:0/4159495971 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efc18077f70 0x7efc1807a430 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0ccbc0 tx=0x7efc2400a2d0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:16.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:16.797+0000 7efc38c00640 1 -- 172.21.15.7:0/4159495971 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7efc3c0d0950 con 0x7efc3c073c70 2024-09-10T21:44:17.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.030+0000 7efc42c00640 1 -- 172.21.15.7:0/4159495971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc3c073730 con 0x7efc3c073c70 2024-09-10T21:44:17.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.031+0000 7efc38c00640 1 -- 172.21.15.7:0/4159495971 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5233 (secure 0 0 0) 0x7efc2c082b60 con 0x7efc3c073c70 2024-09-10T21:44:17.032 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:17.032 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"btime":"2024-09-10T21:43:36:831759+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5149,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3851521050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3851521050},{"type":"v1","addr":"172.21.15.133:6836","nonce":3851521050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5289,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2709023497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2709023497},{"type":"v1","addr":"172.21.15.7:6834","nonce":2709023497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/4095876032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":4095876032},{"type":"v1","addr":"172.21.15.7:6835","nonce":4095876032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:43:33.245584+0000","modified":"2024-09-10T21:43:36.266208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234767},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234767":{"gid":234767,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3385628197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3385628197},{"type":"v1","addr":"172.21.15.133:6837","nonce":3385628197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234767,"qdb_cluster":[234767]},"id":3}]} 2024-09-10T21:44:17.033 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-09-10T21:44:17.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.033+0000 7efc42c00640 1 -- 172.21.15.7:0/4159495971 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efc18077f70 msgr2=0x7efc1807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:17.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.033+0000 7efc42c00640 1 --2- 172.21.15.7:0/4159495971 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efc18077f70 0x7efc1807a430 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efc3c0ccbc0 tx=0x7efc2400a2d0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.033+0000 7efc42c00640 1 -- 172.21.15.7:0/4159495971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc3c073c70 msgr2=0x7efc3c15f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:17.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.033+0000 7efc42c00640 1 --2- 172.21.15.7:0/4159495971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc3c073c70 0x7efc3c15f100 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7efc2c034040 tx=0x7efc2c0039e0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.033+0000 7efc42c00640 1 -- 172.21.15.7:0/4159495971 shutdown_connections 2024-09-10T21:44:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.033+0000 7efc42c00640 1 --2- 172.21.15.7:0/4159495971 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efc18077f70 0x7efc1807a430 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.033+0000 7efc42c00640 1 --2- 172.21.15.7:0/4159495971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efc3c0cd750 0x7efc3c157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.033+0000 7efc42c00640 1 --2- 172.21.15.7:0/4159495971 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efc3c15bc90 0x7efc3c0ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.033+0000 7efc42c00640 1 --2- 172.21.15.7:0/4159495971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efc3c073c70 0x7efc3c15f100 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.033+0000 7efc42c00640 1 -- 172.21.15.7:0/4159495971 >> 172.21.15.7:0/4159495971 conn(0x7efc3c0b16d0 msgr2=0x7efc3c072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.033+0000 7efc42c00640 1 -- 172.21.15.7:0/4159495971 shutdown_connections 2024-09-10T21:44:17.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.033+0000 7efc42c00640 1 -- 172.21.15.7:0/4159495971 wait complete. 2024-09-10T21:44:17.051 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:44:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.211+0000 7fe83be00640 1 -- 172.21.15.7:0/4262330799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe83c0d2690 msgr2=0x7fe83c15f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.211+0000 7fe83be00640 1 --2- 172.21.15.7:0/4262330799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe83c0d2690 0x7fe83c15f910 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fe83c069850 tx=0x7fe82402a560 comp rx=0 tx=0).stop 2024-09-10T21:44:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.212+0000 7fe83be00640 1 -- 172.21.15.7:0/4262330799 shutdown_connections 2024-09-10T21:44:17.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.212+0000 7fe83be00640 1 --2- 172.21.15.7:0/4262330799 >> v2:172.21.15.133:3300/0 conn(0x7fe83c15f480 0x7fe83c15fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.212+0000 7fe83be00640 1 --2- 172.21.15.7:0/4262330799 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe83c0d2690 0x7fe83c15f910 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.212+0000 7fe83be00640 1 --2- 172.21.15.7:0/4262330799 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe83c15f0a0 0x7fe83c0b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.212+0000 7fe83be00640 1 -- 172.21.15.7:0/4262330799 >> 172.21.15.7:0/4262330799 conn(0x7fe83c0d1ba0 msgr2=0x7fe83c0d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.212+0000 7fe83be00640 1 -- 172.21.15.7:0/4262330799 shutdown_connections 2024-09-10T21:44:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.212+0000 7fe83be00640 1 -- 172.21.15.7:0/4262330799 wait complete. 2024-09-10T21:44:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.213+0000 7fe83be00640 1 Processor -- start 2024-09-10T21:44:17.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.213+0000 7fe83be00640 1 -- start start 2024-09-10T21:44:17.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.214+0000 7fe83be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe83c0d2690 0x7fe83c0b5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:17.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.214+0000 7fe83be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe83c15f0a0 0x7fe83c0b7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:17.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.214+0000 7fe83be00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe83c15f480 0x7fe83c0cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:17.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.214+0000 7fe83be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c0ae970 con 0x7fe83c0d2690 2024-09-10T21:44:17.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.214+0000 7fe83be00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe83c0b1000 con 0x7fe83c15f0a0 2024-09-10T21:44:17.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.214+0000 7fe83be00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe83c0aeaf0 con 0x7fe83c15f480 2024-09-10T21:44:17.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.215+0000 7fe83aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe83c0d2690 0x7fe83c0b5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:17.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.215+0000 7fe83aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe83c0d2690 0x7fe83c0b5f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43172/0 (socket says 172.21.15.7:43172) 2024-09-10T21:44:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.215+0000 7fe83aa00640 1 -- 172.21.15.7:0/1316288396 learned_addr learned my addr 172.21.15.7:0/1316288396 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.215+0000 7fe833e00640 1 --2- 172.21.15.7:0/1316288396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe83c15f0a0 0x7fe83c0b7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.215+0000 7fe83b400640 1 --2- 172.21.15.7:0/1316288396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe83c15f480 0x7fe83c0cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.215+0000 7fe83aa00640 1 -- 172.21.15.7:0/1316288396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe83c15f480 msgr2=0x7fe83c0cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.215+0000 7fe83aa00640 1 --2- 172.21.15.7:0/1316288396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe83c15f480 0x7fe83c0cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.215+0000 7fe83aa00640 1 -- 172.21.15.7:0/1316288396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe83c15f0a0 msgr2=0x7fe83c0b7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.215+0000 7fe83aa00640 1 --2- 172.21.15.7:0/1316288396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe83c15f0a0 0x7fe83c0b7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.215+0000 7fe83aa00640 1 -- 172.21.15.7:0/1316288396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe83c0b7900 con 0x7fe83c0d2690 2024-09-10T21:44:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.216+0000 7fe83aa00640 1 --2- 172.21.15.7:0/1316288396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe83c0d2690 0x7fe83c0b5f10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00d780 tx=0x7fe82c00dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.216+0000 7fe832a00640 1 -- 172.21.15.7:0/1316288396 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c017070 con 0x7fe83c0d2690 2024-09-10T21:44:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.216+0000 7fe832a00640 1 -- 172.21.15.7:0/1316288396 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe82c00de90 con 0x7fe83c0d2690 2024-09-10T21:44:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.216+0000 7fe83be00640 1 -- 172.21.15.7:0/1316288396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe83c0c8d10 con 0x7fe83c0d2690 2024-09-10T21:44:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.216+0000 7fe832a00640 1 -- 172.21.15.7:0/1316288396 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe82c005020 con 0x7fe83c0d2690 2024-09-10T21:44:17.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.216+0000 7fe83be00640 1 -- 172.21.15.7:0/1316288396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe83c0c92d0 con 0x7fe83c0d2690 2024-09-10T21:44:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.217+0000 7fe83be00640 1 -- 172.21.15.7:0/1316288396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe83c069650 con 0x7fe83c0d2690 2024-09-10T21:44:17.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.218+0000 7fe832a00640 1 -- 172.21.15.7:0/1316288396 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe82c0051c0 con 0x7fe83c0d2690 2024-09-10T21:44:17.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.219+0000 7fe832a00640 1 --2- 172.21.15.7:0/1316288396 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe818077f70 0x7fe81807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:17.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.219+0000 7fe832a00640 1 -- 172.21.15.7:0/1316288396 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7fe82c099e40 con 0x7fe83c0d2690 2024-09-10T21:44:17.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.219+0000 7fe833e00640 1 --2- 172.21.15.7:0/1316288396 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe818077f70 0x7fe81807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:17.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.220+0000 7fe833e00640 1 --2- 172.21.15.7:0/1316288396 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe818077f70 0x7fe81807a430 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe824033000 tx=0x7fe824004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:17.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.223+0000 7fe832a00640 1 -- 172.21.15.7:0/1316288396 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe83c069650 con 0x7fe83c0d2690 2024-09-10T21:44:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.455+0000 7fe83be00640 1 -- 172.21.15.7:0/1316288396 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe83c0b2290 con 0x7fe83c0d2690 2024-09-10T21:44:17.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.455+0000 7fe832a00640 1 -- 172.21.15.7:0/1316288396 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5233 (secure 0 0 0) 0x7fe82c0622a0 con 0x7fe83c0d2690 2024-09-10T21:44:17.457 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:17.457 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":26,"btime":"2024-09-10T21:43:36:831759+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5149,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3851521050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3851521050},{"type":"v1","addr":"172.21.15.133:6836","nonce":3851521050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5289,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2709023497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2709023497},{"type":"v1","addr":"172.21.15.7:6834","nonce":2709023497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/4095876032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":4095876032},{"type":"v1","addr":"172.21.15.7:6835","nonce":4095876032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:43:33.245584+0000","modified":"2024-09-10T21:43:36.266208+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":234767},"failed":[],"damaged":[],"stopped":[],"info":{"gid_234767":{"gid":234767,"name":"d","rank":0,"incarnation":22,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3385628197","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3385628197},{"type":"v1","addr":"172.21.15.133:6837","nonce":3385628197}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":234767,"qdb_cluster":[234767]},"id":3}]} 2024-09-10T21:44:17.457 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 26 2024-09-10T21:44:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.458+0000 7fe83be00640 1 -- 172.21.15.7:0/1316288396 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe818077f70 msgr2=0x7fe81807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.458+0000 7fe83be00640 1 --2- 172.21.15.7:0/1316288396 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe818077f70 0x7fe81807a430 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe824033000 tx=0x7fe824004730 comp rx=0 tx=0).stop 2024-09-10T21:44:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.458+0000 7fe83be00640 1 -- 172.21.15.7:0/1316288396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe83c0d2690 msgr2=0x7fe83c0b5f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.458+0000 7fe83be00640 1 --2- 172.21.15.7:0/1316288396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe83c0d2690 0x7fe83c0b5f10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fe82c00d780 tx=0x7fe82c00dc40 comp rx=0 tx=0).stop 2024-09-10T21:44:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.458+0000 7fe83be00640 1 -- 172.21.15.7:0/1316288396 shutdown_connections 2024-09-10T21:44:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.458+0000 7fe83be00640 1 --2- 172.21.15.7:0/1316288396 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe818077f70 0x7fe81807a430 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.458+0000 7fe83be00640 1 --2- 172.21.15.7:0/1316288396 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe83c15f0a0 0x7fe83c0b7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.458+0000 7fe83be00640 1 --2- 172.21.15.7:0/1316288396 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe83c15f480 0x7fe83c0cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.458+0000 7fe83be00640 1 --2- 172.21.15.7:0/1316288396 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe83c0d2690 0x7fe83c0b5f10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.458+0000 7fe83be00640 1 -- 172.21.15.7:0/1316288396 >> 172.21.15.7:0/1316288396 conn(0x7fe83c0d1ba0 msgr2=0x7fe83c070ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.458+0000 7fe83be00640 1 -- 172.21.15.7:0/1316288396 shutdown_connections 2024-09-10T21:44:17.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.458+0000 7fe83be00640 1 -- 172.21.15.7:0/1316288396 wait complete. 2024-09-10T21:44:17.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:44:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.635+0000 7f3065200640 1 -- 172.21.15.7:0/1897926962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30600cca30 msgr2=0x7f30600afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.635+0000 7f3065200640 1 --2- 172.21.15.7:0/1897926962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30600cca30 0x7f30600afac0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3048009830 tx=0x7f304802a410 comp rx=0 tx=0).stop 2024-09-10T21:44:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.635+0000 7f3065200640 1 -- 172.21.15.7:0/1897926962 shutdown_connections 2024-09-10T21:44:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.635+0000 7f3065200640 1 --2- 172.21.15.7:0/1897926962 >> v2:172.21.15.133:3300/0 conn(0x7f30600748e0 0x7f30600b0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.635+0000 7f3065200640 1 --2- 172.21.15.7:0/1897926962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30600cca30 0x7f30600afac0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.635+0000 7f3065200640 1 -- 172.21.15.7:0/1897926962 >> 172.21.15.7:0/1897926962 conn(0x7f30600d0430 msgr2=0x7f30600d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.635+0000 7f3065200640 1 -- 172.21.15.7:0/1897926962 shutdown_connections 2024-09-10T21:44:17.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.636+0000 7f3065200640 1 -- 172.21.15.7:0/1897926962 wait complete. 2024-09-10T21:44:17.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.637+0000 7f3065200640 1 Processor -- start 2024-09-10T21:44:17.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.637+0000 7f3065200640 1 -- start start 2024-09-10T21:44:17.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.638+0000 7f3065200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30600748e0 0x7f306009a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:17.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.638+0000 7f3065200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30600cca30 0x7f3060150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:17.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.638+0000 7f3065200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f306015e4c0 0x7f30600c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:17.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.638+0000 7f3065200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f30600af900 con 0x7f306015e4c0 2024-09-10T21:44:17.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.638+0000 7f3065200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f30600af780 con 0x7f30600748e0 2024-09-10T21:44:17.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.638+0000 7f3065200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f30600b0540 con 0x7f30600cca30 2024-09-10T21:44:17.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.638+0000 7f305e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f306015e4c0 0x7f30600c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:17.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.638+0000 7f305e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f306015e4c0 0x7f30600c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43178/0 (socket says 172.21.15.7:43178) 2024-09-10T21:44:17.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.638+0000 7f305e800640 1 -- 172.21.15.7:0/672145290 learned_addr learned my addr 172.21.15.7:0/672145290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:17.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.639+0000 7f305de00640 1 --2- 172.21.15.7:0/672145290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30600748e0 0x7f306009a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:17.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.639+0000 7f3055400640 1 --2- 172.21.15.7:0/672145290 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30600cca30 0x7f3060150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:17.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.639+0000 7f305e800640 1 -- 172.21.15.7:0/672145290 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30600cca30 msgr2=0x7f3060150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:17.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.639+0000 7f305e800640 1 --2- 172.21.15.7:0/672145290 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30600cca30 0x7f3060150e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.639+0000 7f305e800640 1 -- 172.21.15.7:0/672145290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30600748e0 msgr2=0x7f306009a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:17.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.639+0000 7f305e800640 1 --2- 172.21.15.7:0/672145290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30600748e0 0x7f306009a2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.639+0000 7f305e800640 1 -- 172.21.15.7:0/672145290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30601f2e10 con 0x7f306015e4c0 2024-09-10T21:44:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.639+0000 7f305e800640 1 --2- 172.21.15.7:0/672145290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f306015e4c0 0x7f30600c9e00 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f305800eb40 tx=0x7f305800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.639+0000 7f3056a00640 1 -- 172.21.15.7:0/672145290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3058004b20 con 0x7f306015e4c0 2024-09-10T21:44:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.639+0000 7f3056a00640 1 -- 172.21.15.7:0/672145290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3058004db0 con 0x7f306015e4c0 2024-09-10T21:44:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.639+0000 7f3056a00640 1 -- 172.21.15.7:0/672145290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3058005280 con 0x7f306015e4c0 2024-09-10T21:44:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.640+0000 7f3065200640 1 -- 172.21.15.7:0/672145290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30601f6750 con 0x7f306015e4c0 2024-09-10T21:44:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.640+0000 7f3065200640 1 -- 172.21.15.7:0/672145290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30601f6d10 con 0x7f306015e4c0 2024-09-10T21:44:17.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.641+0000 7f3065200640 1 -- 172.21.15.7:0/672145290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30600b0a60 con 0x7f306015e4c0 2024-09-10T21:44:17.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.642+0000 7f3056a00640 1 -- 172.21.15.7:0/672145290 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3058005420 con 0x7f306015e4c0 2024-09-10T21:44:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.643+0000 7f3056a00640 1 --2- 172.21.15.7:0/672145290 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f303c077f70 0x7f303c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:17.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.643+0000 7f3056a00640 1 -- 172.21.15.7:0/672145290 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f305809ab40 con 0x7f306015e4c0 2024-09-10T21:44:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.644+0000 7f305de00640 1 --2- 172.21.15.7:0/672145290 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f303c077f70 0x7f303c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:17.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.645+0000 7f305de00640 1 --2- 172.21.15.7:0/672145290 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f303c077f70 0x7f303c07a430 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3050007960 tx=0x7f3050001290 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:17.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.648+0000 7f3056a00640 1 -- 172.21.15.7:0/672145290 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3058062fa0 con 0x7f306015e4c0 2024-09-10T21:44:17.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.844+0000 7f3065200640 1 -- 172.21.15.7:0/672145290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f30600b1670 con 0x7f306015e4c0 2024-09-10T21:44:17.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.845+0000 7f3056a00640 1 -- 172.21.15.7:0/672145290 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16733 (secure 0 0 0) 0x7f30580679d0 con 0x7f306015e4c0 2024-09-10T21:44:17.846 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:17.846 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":46,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:43:48.882487+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:43:31.081211+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-10T21:43:32.202448+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.12","mappings":[{"from":0,"to":5}]},{"pgid":"7.3b","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:44:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.847+0000 7f3065200640 1 -- 172.21.15.7:0/672145290 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f303c077f70 msgr2=0x7f303c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:17.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.847+0000 7f3065200640 1 --2- 172.21.15.7:0/672145290 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f303c077f70 0x7f303c07a430 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3050007960 tx=0x7f3050001290 comp rx=0 tx=0).stop 2024-09-10T21:44:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.847+0000 7f3065200640 1 -- 172.21.15.7:0/672145290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f306015e4c0 msgr2=0x7f30600c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.847+0000 7f3065200640 1 --2- 172.21.15.7:0/672145290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f306015e4c0 0x7f30600c9e00 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f305800eb40 tx=0x7f305800c2d0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.847+0000 7f3065200640 1 -- 172.21.15.7:0/672145290 shutdown_connections 2024-09-10T21:44:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.847+0000 7f3065200640 1 --2- 172.21.15.7:0/672145290 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f303c077f70 0x7f303c07a430 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.847+0000 7f3065200640 1 --2- 172.21.15.7:0/672145290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f306015e4c0 0x7f30600c9e00 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.847+0000 7f3065200640 1 --2- 172.21.15.7:0/672145290 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30600cca30 0x7f3060150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.847+0000 7f3065200640 1 --2- 172.21.15.7:0/672145290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30600748e0 0x7f306009a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.847+0000 7f3065200640 1 -- 172.21.15.7:0/672145290 >> 172.21.15.7:0/672145290 conn(0x7f30600d0430 msgr2=0x7f30600cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.847+0000 7f3065200640 1 -- 172.21.15.7:0/672145290 shutdown_connections 2024-09-10T21:44:17.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:17.847+0000 7f3065200640 1 -- 172.21.15.7:0/672145290 wait complete. 2024-09-10T21:44:17.866 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:44:18.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.027+0000 7f0056400640 1 -- 172.21.15.7:0/3694082130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0050074210 msgr2=0x7f005015f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:18.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.027+0000 7f0056400640 1 --2- 172.21.15.7:0/3694082130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0050074210 0x7f005015f710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f0048005fd0 tx=0x7f004802a780 comp rx=0 tx=0).stop 2024-09-10T21:44:18.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.027+0000 7f0056400640 1 -- 172.21.15.7:0/3694082130 shutdown_connections 2024-09-10T21:44:18.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.027+0000 7f0056400640 1 --2- 172.21.15.7:0/3694082130 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0050074210 0x7f005015f710 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:18.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.027+0000 7f0056400640 1 --2- 172.21.15.7:0/3694082130 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f005015cbe0 0x7f0050073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:18.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.027+0000 7f0056400640 1 -- 172.21.15.7:0/3694082130 >> 172.21.15.7:0/3694082130 conn(0x7f00500b1660 msgr2=0x7f00500b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:18.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.027+0000 7f0056400640 1 -- 172.21.15.7:0/3694082130 shutdown_connections 2024-09-10T21:44:18.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.027+0000 7f0056400640 1 -- 172.21.15.7:0/3694082130 wait complete. 2024-09-10T21:44:18.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.028+0000 7f0056400640 1 Processor -- start 2024-09-10T21:44:18.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.029+0000 7f0056400640 1 -- start start 2024-09-10T21:44:18.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.030+0000 7f0056400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f005006ff50 0x7f0050148fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:18.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.030+0000 7f0056400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0050074210 0x7f00500b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:18.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.030+0000 7f0056400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005015cbe0 0x7f00500b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:18.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.030+0000 7f0056400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f005006f8a0 con 0x7f005015cbe0 2024-09-10T21:44:18.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.030+0000 7f0056400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f005006f720 con 0x7f0050074210 2024-09-10T21:44:18.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.030+0000 7f0056400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f005015fc50 con 0x7f005006ff50 2024-09-10T21:44:18.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.030+0000 7f004fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005015cbe0 0x7f00500b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:18.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.030+0000 7f004fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005015cbe0 0x7f00500b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43184/0 (socket says 172.21.15.7:43184) 2024-09-10T21:44:18.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.030+0000 7f004fe00640 1 -- 172.21.15.7:0/4265548388 learned_addr learned my addr 172.21.15.7:0/4265548388 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:18.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.030+0000 7f004f400640 1 --2- 172.21.15.7:0/4265548388 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f005006ff50 0x7f0050148fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:18.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.030+0000 7f0046a00640 1 --2- 172.21.15.7:0/4265548388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0050074210 0x7f00500b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:18.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.030+0000 7f004fe00640 1 -- 172.21.15.7:0/4265548388 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f005006ff50 msgr2=0x7f0050148fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:18.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.030+0000 7f004fe00640 1 --2- 172.21.15.7:0/4265548388 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f005006ff50 0x7f0050148fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:18.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.030+0000 7f004fe00640 1 -- 172.21.15.7:0/4265548388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0050074210 msgr2=0x7f00500b4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:18.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.030+0000 7f004fe00640 1 --2- 172.21.15.7:0/4265548388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0050074210 0x7f00500b4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:18.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.030+0000 7f004fe00640 1 -- 172.21.15.7:0/4265548388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0050153390 con 0x7f005015cbe0 2024-09-10T21:44:18.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.031+0000 7f004fe00640 1 --2- 172.21.15.7:0/4265548388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005015cbe0 0x7f00500b48d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0048032fd0 tx=0x7f0048003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:18.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.031+0000 7f0047e00640 1 -- 172.21.15.7:0/4265548388 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048040070 con 0x7f005015cbe0 2024-09-10T21:44:18.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.031+0000 7f0047e00640 1 -- 172.21.15.7:0/4265548388 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0048004330 con 0x7f005015cbe0 2024-09-10T21:44:18.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.031+0000 7f0056400640 1 -- 172.21.15.7:0/4265548388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00500b3750 con 0x7f005015cbe0 2024-09-10T21:44:18.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.031+0000 7f0047e00640 1 -- 172.21.15.7:0/4265548388 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0048004740 con 0x7f005015cbe0 2024-09-10T21:44:18.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.031+0000 7f0056400640 1 -- 172.21.15.7:0/4265548388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00500c49d0 con 0x7f005015cbe0 2024-09-10T21:44:18.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.032+0000 7f0056400640 1 -- 172.21.15.7:0/4265548388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f005015f710 con 0x7f005015cbe0 2024-09-10T21:44:18.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.033+0000 7f0047e00640 1 -- 172.21.15.7:0/4265548388 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0048032740 con 0x7f005015cbe0 2024-09-10T21:44:18.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.034+0000 7f0047e00640 1 --2- 172.21.15.7:0/4265548388 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f002c077f70 0x7f002c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:18.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.034+0000 7f0047e00640 1 -- 172.21.15.7:0/4265548388 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5767+0+0 (secure 0 0 0) 0x7f00480bb120 con 0x7f005015cbe0 2024-09-10T21:44:18.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.035+0000 7f004f400640 1 --2- 172.21.15.7:0/4265548388 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f002c077f70 0x7f002c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:18.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.035+0000 7f004f400640 1 --2- 172.21.15.7:0/4265548388 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f002c077f70 0x7f002c07a430 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0040007fd0 tx=0x7f0040001410 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:18.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.039+0000 7f0047e00640 1 -- 172.21.15.7:0/4265548388 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0050153390 con 0x7f005015cbe0 2024-09-10T21:44:18.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.268+0000 7f0056400640 1 -- 172.21.15.7:0/4265548388 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f005006e920 con 0x7f005015cbe0 2024-09-10T21:44:18.269 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-10T21:44:18.268+0000 7feb3fa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:44:18.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.270+0000 7f0047e00640 1 -- 172.21.15.7:0/4265548388 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7f0048083500 con 0x7f005015cbe0 2024-09-10T21:44:18.272 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:44:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.273+0000 7f0056400640 1 -- 172.21.15.7:0/4265548388 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f002c077f70 msgr2=0x7f002c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.273+0000 7f0056400640 1 --2- 172.21.15.7:0/4265548388 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f002c077f70 0x7f002c07a430 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0040007fd0 tx=0x7f0040001410 comp rx=0 tx=0).stop 2024-09-10T21:44:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.273+0000 7f0056400640 1 -- 172.21.15.7:0/4265548388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005015cbe0 msgr2=0x7f00500b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.273+0000 7f0056400640 1 --2- 172.21.15.7:0/4265548388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005015cbe0 0x7f00500b48d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0048032fd0 tx=0x7f0048003ba0 comp rx=0 tx=0).stop 2024-09-10T21:44:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.273+0000 7f0056400640 1 -- 172.21.15.7:0/4265548388 shutdown_connections 2024-09-10T21:44:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.273+0000 7f0056400640 1 --2- 172.21.15.7:0/4265548388 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f002c077f70 0x7f002c07a430 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.273+0000 7f0056400640 1 --2- 172.21.15.7:0/4265548388 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0050074210 0x7f00500b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.273+0000 7f0056400640 1 --2- 172.21.15.7:0/4265548388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f005015cbe0 0x7f00500b48d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.273+0000 7f0056400640 1 --2- 172.21.15.7:0/4265548388 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f005006ff50 0x7f0050148fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.273+0000 7f0056400640 1 -- 172.21.15.7:0/4265548388 >> 172.21.15.7:0/4265548388 conn(0x7f00500b1660 msgr2=0x7f00500b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:18.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.274+0000 7f0056400640 1 -- 172.21.15.7:0/4265548388 shutdown_connections 2024-09-10T21:44:18.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.274+0000 7f0056400640 1 -- 172.21.15.7:0/4265548388 wait complete. 2024-09-10T21:44:18.289 INFO:tasks.ceph.mds.d.smithi133.stderr: -6920> 2024-09-10T21:43:39.773+0000 7f352ac00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:44:18.291 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:44:18.293 INFO:tasks.ceph.mds.d.smithi133.stderr: -3818> 2024-09-10T21:43:42.984+0000 7f3525c00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 pv4 ap=4 DIRTYPARENT s=0 n(v0 1=1+0)->n(v0 rc2024-09-10T21:43:41.612224+0000 1=1+0) (iauth excl) (ifile lock->excl w=1) (ixattr excl) (iquiesce lock w=1) cr={234848=0-4194304@1} caps={234848=pAsxLsXsxFcb/pAsxXsxFxwb@3},l=234848 | ptrwaiter=1 request=1 lock=3 caps=1 dirtyparent=1 dirty=0 waiter=1 authpin=1 scrubqueue=0 0x55cf8593cc00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-10T21:44:18.341 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2024-09-10T21:44:18.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.453+0000 7fa061000640 1 -- 172.21.15.7:0/720051413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa05c0cfd80 msgr2=0x7fa05c0d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.453+0000 7fa061000640 1 --2- 172.21.15.7:0/720051413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa05c0cfd80 0x7fa05c0d01f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa05c06dae0 tx=0x7fa04402a5f0 comp rx=0 tx=0).stop 2024-09-10T21:44:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.454+0000 7fa061000640 1 -- 172.21.15.7:0/720051413 shutdown_connections 2024-09-10T21:44:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.454+0000 7fa061000640 1 --2- 172.21.15.7:0/720051413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa05c0cfd80 0x7fa05c0d01f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.454+0000 7fa061000640 1 --2- 172.21.15.7:0/720051413 >> v2:172.21.15.133:3300/0 conn(0x7fa05c158500 0x7fa05c06e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.454+0000 7fa061000640 1 -- 172.21.15.7:0/720051413 >> 172.21.15.7:0/720051413 conn(0x7fa05c06cd00 msgr2=0x7fa05c06d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.454+0000 7fa061000640 1 -- 172.21.15.7:0/720051413 shutdown_connections 2024-09-10T21:44:18.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.454+0000 7fa061000640 1 -- 172.21.15.7:0/720051413 wait complete. 2024-09-10T21:44:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.455+0000 7fa061000640 1 Processor -- start 2024-09-10T21:44:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.455+0000 7fa061000640 1 -- start start 2024-09-10T21:44:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.456+0000 7fa061000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa05c0ad8e0 0x7fa05c1f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.456+0000 7fa061000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa05c0cfd80 0x7fa05c15c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.456+0000 7fa061000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa05c158500 0x7fa05c15a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:18.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.456+0000 7fa061000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c06eb80 con 0x7fa05c0ad8e0 2024-09-10T21:44:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.456+0000 7fa061000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa05c06ea00 con 0x7fa05c0cfd80 2024-09-10T21:44:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.456+0000 7fa061000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa05c0b3e00 con 0x7fa05c158500 2024-09-10T21:44:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.456+0000 7fa05b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa05c0ad8e0 0x7fa05c1f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.457+0000 7fa05b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa05c0ad8e0 0x7fa05c1f4600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43200/0 (socket says 172.21.15.7:43200) 2024-09-10T21:44:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.457+0000 7fa05b400640 1 -- 172.21.15.7:0/774813644 learned_addr learned my addr 172.21.15.7:0/774813644 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.457+0000 7fa05be00640 1 --2- 172.21.15.7:0/774813644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa05c158500 0x7fa05c15a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.457+0000 7fa052a00640 1 --2- 172.21.15.7:0/774813644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa05c0cfd80 0x7fa05c15c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.457+0000 7fa05b400640 1 -- 172.21.15.7:0/774813644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa05c158500 msgr2=0x7fa05c15a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:18.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.457+0000 7fa05b400640 1 --2- 172.21.15.7:0/774813644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa05c158500 0x7fa05c15a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.457+0000 7fa05b400640 1 -- 172.21.15.7:0/774813644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa05c0cfd80 msgr2=0x7fa05c15c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.457+0000 7fa05b400640 1 --2- 172.21.15.7:0/774813644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa05c0cfd80 0x7fa05c15c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.457+0000 7fa05b400640 1 -- 172.21.15.7:0/774813644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa05c1f4e80 con 0x7fa05c0ad8e0 2024-09-10T21:44:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.457+0000 7fa05b400640 1 --2- 172.21.15.7:0/774813644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa05c0ad8e0 0x7fa05c1f4600 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00e9f0 tx=0x7fa04c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.458+0000 7fa053e00640 1 -- 172.21.15.7:0/774813644 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c00cd80 con 0x7fa05c0ad8e0 2024-09-10T21:44:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.458+0000 7fa053e00640 1 -- 172.21.15.7:0/774813644 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa04c010430 con 0x7fa05c0ad8e0 2024-09-10T21:44:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.458+0000 7fa053e00640 1 -- 172.21.15.7:0/774813644 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c010840 con 0x7fa05c0ad8e0 2024-09-10T21:44:18.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.458+0000 7fa061000640 1 -- 172.21.15.7:0/774813644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa05c1f5160 con 0x7fa05c0ad8e0 2024-09-10T21:44:18.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.458+0000 7fa061000640 1 -- 172.21.15.7:0/774813644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa05c159c80 con 0x7fa05c0ad8e0 2024-09-10T21:44:18.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.459+0000 7fa061000640 1 -- 172.21.15.7:0/774813644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa05c0700f0 con 0x7fa05c0ad8e0 2024-09-10T21:44:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.460+0000 7fa053e00640 1 -- 172.21.15.7:0/774813644 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa04c0109e0 con 0x7fa05c0ad8e0 2024-09-10T21:44:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.461+0000 7fa053e00640 1 --2- 172.21.15.7:0/774813644 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa038077f70 0x7fa03807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:18.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.461+0000 7fa053e00640 1 -- 172.21.15.7:0/774813644 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7fa04c09a8b0 con 0x7fa05c0ad8e0 2024-09-10T21:44:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.461+0000 7fa052a00640 1 --2- 172.21.15.7:0/774813644 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa038077f70 0x7fa03807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:18.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.462+0000 7fa052a00640 1 --2- 172.21.15.7:0/774813644 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa038077f70 0x7fa03807a430 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa044033000 tx=0x7fa044004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:18.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.465+0000 7fa053e00640 1 -- 172.21.15.7:0/774813644 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fa05c0700f0 con 0x7fa05c0ad8e0 2024-09-10T21:44:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:18.697+0000 7fa061000640 1 -- 172.21.15.7:0/774813644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa05c0ae570 con 0x7fa05c0ad8e0 2024-09-10T21:44:19.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.272+0000 7fa053e00640 1 -- 172.21.15.7:0/774813644 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7fa04c062d90 con 0x7fa05c0ad8e0 2024-09-10T21:44:19.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.275+0000 7fa061000640 1 -- 172.21.15.7:0/774813644 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa038077f70 msgr2=0x7fa03807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:19.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.275+0000 7fa061000640 1 --2- 172.21.15.7:0/774813644 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa038077f70 0x7fa03807a430 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa044033000 tx=0x7fa044004730 comp rx=0 tx=0).stop 2024-09-10T21:44:19.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.275+0000 7fa061000640 1 -- 172.21.15.7:0/774813644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa05c0ad8e0 msgr2=0x7fa05c1f4600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:19.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.275+0000 7fa061000640 1 --2- 172.21.15.7:0/774813644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa05c0ad8e0 0x7fa05c1f4600 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00e9f0 tx=0x7fa04c00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:44:19.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.275+0000 7fa061000640 1 -- 172.21.15.7:0/774813644 shutdown_connections 2024-09-10T21:44:19.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.275+0000 7fa061000640 1 --2- 172.21.15.7:0/774813644 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa038077f70 0x7fa03807a430 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:19.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.275+0000 7fa061000640 1 --2- 172.21.15.7:0/774813644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa05c0cfd80 0x7fa05c15c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:19.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.275+0000 7fa061000640 1 --2- 172.21.15.7:0/774813644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa05c158500 0x7fa05c15a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:19.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.275+0000 7fa061000640 1 --2- 172.21.15.7:0/774813644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa05c0ad8e0 0x7fa05c1f4600 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:19.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.276+0000 7fa061000640 1 -- 172.21.15.7:0/774813644 >> 172.21.15.7:0/774813644 conn(0x7fa05c06cd00 msgr2=0x7fa05c06d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:19.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.276+0000 7fa061000640 1 -- 172.21.15.7:0/774813644 shutdown_connections 2024-09-10T21:44:19.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.276+0000 7fa061000640 1 -- 172.21.15.7:0/774813644 wait complete. 2024-09-10T21:44:19.295 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:44:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.460+0000 7fc06b000640 1 -- 172.21.15.7:0/2718760475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc064158300 msgr2=0x7fc0640b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.460+0000 7fc06b000640 1 --2- 172.21.15.7:0/2718760475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc064158300 0x7fc0640b4a30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fc064070720 tx=0x7fc05402a4a0 comp rx=0 tx=0).stop 2024-09-10T21:44:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.460+0000 7fc06b000640 1 -- 172.21.15.7:0/2718760475 shutdown_connections 2024-09-10T21:44:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.460+0000 7fc06b000640 1 --2- 172.21.15.7:0/2718760475 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc06406d230 0x7fc064073810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.460+0000 7fc06b000640 1 --2- 172.21.15.7:0/2718760475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc064158300 0x7fc0640b4a30 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.460+0000 7fc06b000640 1 -- 172.21.15.7:0/2718760475 >> 172.21.15.7:0/2718760475 conn(0x7fc0640b1c10 msgr2=0x7fc0640b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.460+0000 7fc06b000640 1 -- 172.21.15.7:0/2718760475 shutdown_connections 2024-09-10T21:44:19.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.460+0000 7fc06b000640 1 -- 172.21.15.7:0/2718760475 wait complete. 2024-09-10T21:44:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.462+0000 7fc06b000640 1 Processor -- start 2024-09-10T21:44:19.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.462+0000 7fc06b000640 1 -- start start 2024-09-10T21:44:19.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.463+0000 7fc06b000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc06406d230 0x7fc06409c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:19.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.463+0000 7fc06b000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc064158300 0x7fc0640cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:19.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.463+0000 7fc06b000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc06415edd0 0x7fc06414e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:19.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.463+0000 7fc06b000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc064073ed0 con 0x7fc06406d230 2024-09-10T21:44:19.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.463+0000 7fc06b000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc064073d50 con 0x7fc06415edd0 2024-09-10T21:44:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.463+0000 7fc06b000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc064074050 con 0x7fc064158300 2024-09-10T21:44:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.463+0000 7fc063e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc06406d230 0x7fc06409c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.464+0000 7fc063e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc06406d230 0x7fc06409c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43224/0 (socket says 172.21.15.7:43224) 2024-09-10T21:44:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.464+0000 7fc063e00640 1 -- 172.21.15.7:0/2421166183 learned_addr learned my addr 172.21.15.7:0/2421166183 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.464+0000 7fc069000640 1 --2- 172.21.15.7:0/2421166183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc06415edd0 0x7fc06414e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.464+0000 7fc05b400640 1 --2- 172.21.15.7:0/2421166183 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc064158300 0x7fc0640cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.464+0000 7fc063e00640 1 -- 172.21.15.7:0/2421166183 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc064158300 msgr2=0x7fc0640cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.464+0000 7fc063e00640 1 --2- 172.21.15.7:0/2421166183 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc064158300 0x7fc0640cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.464+0000 7fc063e00640 1 -- 172.21.15.7:0/2421166183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc06415edd0 msgr2=0x7fc06414e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.464+0000 7fc063e00640 1 --2- 172.21.15.7:0/2421166183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc06415edd0 0x7fc06414e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.464+0000 7fc063e00640 1 -- 172.21.15.7:0/2421166183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc06409cca0 con 0x7fc06406d230 2024-09-10T21:44:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.464+0000 7fc05b400640 1 --2- 172.21.15.7:0/2421166183 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc064158300 0x7fc0640cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:19.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.464+0000 7fc063e00640 1 --2- 172.21.15.7:0/2421166183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc06406d230 0x7fc06409c6a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc054009850 tx=0x7fc054003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.465+0000 7fc060c00640 1 -- 172.21.15.7:0/2421166183 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc05403f070 con 0x7fc06406d230 2024-09-10T21:44:19.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.465+0000 7fc06b000640 1 -- 172.21.15.7:0/2421166183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc06409cf80 con 0x7fc06406d230 2024-09-10T21:44:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.465+0000 7fc06b000640 1 -- 172.21.15.7:0/2421166183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc06409d540 con 0x7fc06406d230 2024-09-10T21:44:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.465+0000 7fc060c00640 1 -- 172.21.15.7:0/2421166183 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0540047a0 con 0x7fc06406d230 2024-09-10T21:44:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.465+0000 7fc060c00640 1 -- 172.21.15.7:0/2421166183 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc054049060 con 0x7fc06406d230 2024-09-10T21:44:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.466+0000 7fc06b000640 1 -- 172.21.15.7:0/2421166183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc03c005290 con 0x7fc06406d230 2024-09-10T21:44:19.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.467+0000 7fc060c00640 1 -- 172.21.15.7:0/2421166183 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc054004b90 con 0x7fc06406d230 2024-09-10T21:44:19.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.468+0000 7fc060c00640 1 --2- 172.21.15.7:0/2421166183 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc040077f70 0x7fc04007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:19.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.468+0000 7fc060c00640 1 -- 172.21.15.7:0/2421166183 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5853+0+0 (secure 0 0 0) 0x7fc0540bae90 con 0x7fc06406d230 2024-09-10T21:44:19.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.468+0000 7fc05b400640 1 --2- 172.21.15.7:0/2421166183 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc040077f70 0x7fc04007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:19.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.469+0000 7fc05b400640 1 --2- 172.21.15.7:0/2421166183 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc040077f70 0x7fc04007a430 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc04c006fd0 tx=0x7fc04c008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:19.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.472+0000 7fc060c00640 1 -- 172.21.15.7:0/2421166183 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc054083220 con 0x7fc06406d230 2024-09-10T21:44:19.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:19.667+0000 7fc06b000640 1 -- 172.21.15.7:0/2421166183 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc03c005580 con 0x7fc06406d230 2024-09-10T21:44:20.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.282+0000 7fc060c00640 1 -- 172.21.15.7:0/2421166183 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v48) ==== 181+0+0 (secure 0 0 0) 0x7fc054087c50 con 0x7fc06406d230 2024-09-10T21:44:20.283 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:44:20.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.285+0000 7fc06b000640 1 -- 172.21.15.7:0/2421166183 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc040077f70 msgr2=0x7fc04007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:20.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.285+0000 7fc06b000640 1 --2- 172.21.15.7:0/2421166183 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc040077f70 0x7fc04007a430 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc04c006fd0 tx=0x7fc04c008040 comp rx=0 tx=0).stop 2024-09-10T21:44:20.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.285+0000 7fc06b000640 1 -- 172.21.15.7:0/2421166183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc06406d230 msgr2=0x7fc06409c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.285+0000 7fc06b000640 1 --2- 172.21.15.7:0/2421166183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc06406d230 0x7fc06409c6a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fc054009850 tx=0x7fc054003910 comp rx=0 tx=0).stop 2024-09-10T21:44:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.285+0000 7fc06b000640 1 -- 172.21.15.7:0/2421166183 shutdown_connections 2024-09-10T21:44:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.285+0000 7fc06b000640 1 --2- 172.21.15.7:0/2421166183 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc040077f70 0x7fc04007a430 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.285+0000 7fc06b000640 1 --2- 172.21.15.7:0/2421166183 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc06415edd0 0x7fc06414e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.285+0000 7fc06b000640 1 --2- 172.21.15.7:0/2421166183 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc064158300 0x7fc0640cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.285+0000 7fc06b000640 1 --2- 172.21.15.7:0/2421166183 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc06406d230 0x7fc06409c6a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.285+0000 7fc06b000640 1 -- 172.21.15.7:0/2421166183 >> 172.21.15.7:0/2421166183 conn(0x7fc0640b1c10 msgr2=0x7fc0640ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:20.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.286+0000 7fc06b000640 1 -- 172.21.15.7:0/2421166183 shutdown_connections 2024-09-10T21:44:20.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.286+0000 7fc06b000640 1 -- 172.21.15.7:0/2421166183 wait complete. 2024-09-10T21:44:20.305 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:44:20.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.470+0000 7f9cf7e00640 1 -- 172.21.15.7:0/470489312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cf0072e40 msgr2=0x7f9cf00732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:20.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.470+0000 7f9cf7e00640 1 --2- 172.21.15.7:0/470489312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cf0072e40 0x7f9cf00732d0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9cf00cc1c0 tx=0x7f9ce802aac0 comp rx=0 tx=0).stop 2024-09-10T21:44:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.470+0000 7f9cf7e00640 1 -- 172.21.15.7:0/470489312 shutdown_connections 2024-09-10T21:44:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.470+0000 7f9cf7e00640 1 --2- 172.21.15.7:0/470489312 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cf0072e40 0x7f9cf00732d0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.470+0000 7f9cf7e00640 1 --2- 172.21.15.7:0/470489312 >> v2:172.21.15.133:3300/0 conn(0x7f9cf0070080 0x7f9cf015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.470+0000 7f9cf7e00640 1 -- 172.21.15.7:0/470489312 >> 172.21.15.7:0/470489312 conn(0x7f9cf00b1c10 msgr2=0x7f9cf00b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.470+0000 7f9cf7e00640 1 -- 172.21.15.7:0/470489312 shutdown_connections 2024-09-10T21:44:20.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.470+0000 7f9cf7e00640 1 -- 172.21.15.7:0/470489312 wait complete. 2024-09-10T21:44:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.472+0000 7f9cf7e00640 1 Processor -- start 2024-09-10T21:44:20.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.472+0000 7f9cf7e00640 1 -- start start 2024-09-10T21:44:20.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.473+0000 7f9cf7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf0070080 0x7f9cf014cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:20.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.473+0000 7f9cf7e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf0072e40 0x7f9cf009c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:20.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.473+0000 7f9cf7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cf0151a10 0x7f9cf009cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:20.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.473+0000 7f9cf7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf0073990 con 0x7f9cf0151a10 2024-09-10T21:44:20.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.473+0000 7f9cf7e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9cf0073810 con 0x7f9cf0070080 2024-09-10T21:44:20.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.473+0000 7f9cf7e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f9cf0073b10 con 0x7f9cf0072e40 2024-09-10T21:44:20.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.473+0000 7f9cf5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cf0151a10 0x7f9cf009cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:20.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.473+0000 7f9cf5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cf0151a10 0x7f9cf009cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43248/0 (socket says 172.21.15.7:43248) 2024-09-10T21:44:20.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.473+0000 7f9cf5e00640 1 -- 172.21.15.7:0/1919511637 learned_addr learned my addr 172.21.15.7:0/1919511637 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:20.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.473+0000 7f9cf4a00640 1 --2- 172.21.15.7:0/1919511637 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf0072e40 0x7f9cf009c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:20.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.473+0000 7f9cf5e00640 1 -- 172.21.15.7:0/1919511637 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf0072e40 msgr2=0x7f9cf009c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:20.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.473+0000 7f9cf5e00640 1 --2- 172.21.15.7:0/1919511637 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf0072e40 0x7f9cf009c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:20.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.473+0000 7f9cf5e00640 1 -- 172.21.15.7:0/1919511637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf0070080 msgr2=0x7f9cf014cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:20.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.473+0000 7f9cf5400640 1 --2- 172.21.15.7:0/1919511637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf0070080 0x7f9cf014cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:20.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.473+0000 7f9cf5e00640 1 --2- 172.21.15.7:0/1919511637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf0070080 0x7f9cf014cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:20.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.473+0000 7f9cf5e00640 1 -- 172.21.15.7:0/1919511637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cf009b660 con 0x7f9cf0151a10 2024-09-10T21:44:20.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.473+0000 7f9cf5400640 1 --2- 172.21.15.7:0/1919511637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf0070080 0x7f9cf014cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:44:20.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.474+0000 7f9cf5e00640 1 --2- 172.21.15.7:0/1919511637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cf0151a10 0x7f9cf009cba0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009d60 tx=0x7f9ce8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:20.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.474+0000 7f9ced600640 1 -- 172.21.15.7:0/1919511637 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8044070 con 0x7f9cf0151a10 2024-09-10T21:44:20.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.474+0000 7f9ced600640 1 -- 172.21.15.7:0/1919511637 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ce8004250 con 0x7f9cf0151a10 2024-09-10T21:44:20.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.474+0000 7f9ced600640 1 -- 172.21.15.7:0/1919511637 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce8004680 con 0x7f9cf0151a10 2024-09-10T21:44:20.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.474+0000 7f9cf7e00640 1 -- 172.21.15.7:0/1919511637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cf009b940 con 0x7f9cf0151a10 2024-09-10T21:44:20.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.474+0000 7f9cf7e00640 1 -- 172.21.15.7:0/1919511637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cf014d520 con 0x7f9cf0151a10 2024-09-10T21:44:20.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.475+0000 7f9cf7e00640 1 -- 172.21.15.7:0/1919511637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cf015ef50 con 0x7f9cf0151a10 2024-09-10T21:44:20.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.476+0000 7f9ced600640 1 -- 172.21.15.7:0/1919511637 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9ce802c840 con 0x7f9cf0151a10 2024-09-10T21:44:20.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.477+0000 7f9ced600640 1 --2- 172.21.15.7:0/1919511637 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9cd0077f70 0x7f9cd007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:20.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.477+0000 7f9ced600640 1 -- 172.21.15.7:0/1919511637 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5415+0+0 (secure 0 0 0) 0x7f9ce80ba380 con 0x7f9cf0151a10 2024-09-10T21:44:20.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.478+0000 7f9cf5400640 1 --2- 172.21.15.7:0/1919511637 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9cd0077f70 0x7f9cd007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:20.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.478+0000 7f9cf5400640 1 --2- 172.21.15.7:0/1919511637 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9cd0077f70 0x7f9cd007a430 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4000f60 tx=0x7f9ce4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:20.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.482+0000 7f9ced600640 1 -- 172.21.15.7:0/1919511637 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9ce8032030 con 0x7f9cf0151a10 2024-09-10T21:44:20.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:20.686+0000 7f9cf7e00640 1 -- 172.21.15.7:0/1919511637 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9cf00b2190 con 0x7f9cf0151a10 2024-09-10T21:44:21.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.289+0000 7f9ced600640 1 -- 172.21.15.7:0/1919511637 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v49) ==== 169+0+0 (secure 0 0 0) 0x7f9ce8004820 con 0x7f9cf0151a10 2024-09-10T21:44:21.290 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-09-10T21:44:21.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.292+0000 7f9cf7e00640 1 -- 172.21.15.7:0/1919511637 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9cd0077f70 msgr2=0x7f9cd007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.292+0000 7f9cf7e00640 1 --2- 172.21.15.7:0/1919511637 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9cd0077f70 0x7f9cd007a430 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4000f60 tx=0x7f9ce4009040 comp rx=0 tx=0).stop 2024-09-10T21:44:21.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.292+0000 7f9cf7e00640 1 -- 172.21.15.7:0/1919511637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cf0151a10 msgr2=0x7f9cf009cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.292+0000 7f9cf7e00640 1 --2- 172.21.15.7:0/1919511637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cf0151a10 0x7f9cf009cba0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009d60 tx=0x7f9ce8003f10 comp rx=0 tx=0).stop 2024-09-10T21:44:21.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.293+0000 7f9cf7e00640 1 -- 172.21.15.7:0/1919511637 shutdown_connections 2024-09-10T21:44:21.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.293+0000 7f9cf7e00640 1 --2- 172.21.15.7:0/1919511637 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9cd0077f70 0x7f9cd007a430 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.293+0000 7f9cf7e00640 1 --2- 172.21.15.7:0/1919511637 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9cf0151a10 0x7f9cf009cba0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.293+0000 7f9cf7e00640 1 --2- 172.21.15.7:0/1919511637 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9cf0072e40 0x7f9cf009c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.293+0000 7f9cf7e00640 1 --2- 172.21.15.7:0/1919511637 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9cf0070080 0x7f9cf014cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.293+0000 7f9cf7e00640 1 -- 172.21.15.7:0/1919511637 >> 172.21.15.7:0/1919511637 conn(0x7f9cf00b1c10 msgr2=0x7f9cf006f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:21.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.293+0000 7f9cf7e00640 1 -- 172.21.15.7:0/1919511637 shutdown_connections 2024-09-10T21:44:21.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.293+0000 7f9cf7e00640 1 -- 172.21.15.7:0/1919511637 wait complete. 2024-09-10T21:44:21.313 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:44:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.476+0000 7f86b7400640 1 -- 172.21.15.7:0/2801493228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b015cbe0 msgr2=0x7f86b0073cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.476+0000 7f86b7400640 1 --2- 172.21.15.7:0/2801493228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b015cbe0 0x7f86b0073cd0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f86b0073560 tx=0x7f869c02a410 comp rx=0 tx=0).stop 2024-09-10T21:44:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.476+0000 7f86b7400640 1 -- 172.21.15.7:0/2801493228 shutdown_connections 2024-09-10T21:44:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.476+0000 7f86b7400640 1 --2- 172.21.15.7:0/2801493228 >> v2:172.21.15.133:3300/0 conn(0x7f86b0074210 0x7f86b015f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.476+0000 7f86b7400640 1 --2- 172.21.15.7:0/2801493228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b015cbe0 0x7f86b0073cd0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.476+0000 7f86b7400640 1 -- 172.21.15.7:0/2801493228 >> 172.21.15.7:0/2801493228 conn(0x7f86b00b1660 msgr2=0x7f86b00b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:21.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.476+0000 7f86b7400640 1 -- 172.21.15.7:0/2801493228 shutdown_connections 2024-09-10T21:44:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.476+0000 7f86b7400640 1 -- 172.21.15.7:0/2801493228 wait complete. 2024-09-10T21:44:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.478+0000 7f86b7400640 1 Processor -- start 2024-09-10T21:44:21.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.478+0000 7f86b7400640 1 -- start start 2024-09-10T21:44:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.479+0000 7f86b7400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86b006ff50 0x7f86b00b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.479+0000 7f86b7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b0074210 0x7f86b0153750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.479+0000 7f86b7400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86b015cbe0 0x7f86b00b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.479+0000 7f86b7400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f86b006f8a0 con 0x7f86b0074210 2024-09-10T21:44:21.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.479+0000 7f86b7400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f86b006f720 con 0x7f86b015cbe0 2024-09-10T21:44:21.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.479+0000 7f86b7400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f86b015fc50 con 0x7f86b006ff50 2024-09-10T21:44:21.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.479+0000 7f86afe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b0074210 0x7f86b0153750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.479+0000 7f86afe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b0074210 0x7f86b0153750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43260/0 (socket says 172.21.15.7:43260) 2024-09-10T21:44:21.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.479+0000 7f86afe00640 1 -- 172.21.15.7:0/3942917104 learned_addr learned my addr 172.21.15.7:0/3942917104 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:21.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.480+0000 7f86b4a00640 1 --2- 172.21.15.7:0/3942917104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86b006ff50 0x7f86b00b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.480+0000 7f86b5400640 1 --2- 172.21.15.7:0/3942917104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86b015cbe0 0x7f86b00b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.480+0000 7f86afe00640 1 -- 172.21.15.7:0/3942917104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86b006ff50 msgr2=0x7f86b00b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.480+0000 7f86afe00640 1 --2- 172.21.15.7:0/3942917104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86b006ff50 0x7f86b00b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.480+0000 7f86afe00640 1 -- 172.21.15.7:0/3942917104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86b015cbe0 msgr2=0x7f86b00b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.480+0000 7f86afe00640 1 --2- 172.21.15.7:0/3942917104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86b015cbe0 0x7f86b00b09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.480+0000 7f86afe00640 1 -- 172.21.15.7:0/3942917104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86b00c4900 con 0x7f86b0074210 2024-09-10T21:44:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.480+0000 7f86b4a00640 1 --2- 172.21.15.7:0/3942917104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86b006ff50 0x7f86b00b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.480+0000 7f86b5400640 1 --2- 172.21.15.7:0/3942917104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86b015cbe0 0x7f86b00b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.480+0000 7f86afe00640 1 --2- 172.21.15.7:0/3942917104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b0074210 0x7f86b0153750 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f869c009780 tx=0x7f869c033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.480+0000 7f86acc00640 1 -- 172.21.15.7:0/3942917104 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c031d60 con 0x7f86b0074210 2024-09-10T21:44:21.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.480+0000 7f86acc00640 1 -- 172.21.15.7:0/3942917104 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f869c005020 con 0x7f86b0074210 2024-09-10T21:44:21.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.480+0000 7f86acc00640 1 -- 172.21.15.7:0/3942917104 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c005470 con 0x7f86b0074210 2024-09-10T21:44:21.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.480+0000 7f86b7400640 1 -- 172.21.15.7:0/3942917104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86b00c4340 con 0x7f86b0074210 2024-09-10T21:44:21.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.481+0000 7f86b7400640 1 -- 172.21.15.7:0/3942917104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86b00c50b0 con 0x7f86b0074210 2024-09-10T21:44:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.482+0000 7f86b7400640 1 -- 172.21.15.7:0/3942917104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86b015f710 con 0x7f86b0074210 2024-09-10T21:44:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.483+0000 7f86acc00640 1 -- 172.21.15.7:0/3942917104 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f869c0047a0 con 0x7f86b0074210 2024-09-10T21:44:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.484+0000 7f86acc00640 1 --2- 172.21.15.7:0/3942917104 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8690077f70 0x7f869007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.484+0000 7f86acc00640 1 -- 172.21.15.7:0/3942917104 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f869c0bacd0 con 0x7f86b0074210 2024-09-10T21:44:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.484+0000 7f86b4a00640 1 --2- 172.21.15.7:0/3942917104 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8690077f70 0x7f869007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.485+0000 7f86b4a00640 1 --2- 172.21.15.7:0/3942917104 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8690077f70 0x7f869007a430 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f86a4001740 tx=0x7f86a4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:21.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.488+0000 7f86acc00640 1 -- 172.21.15.7:0/3942917104 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f869c03f030 con 0x7f86b0074210 2024-09-10T21:44:21.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.683+0000 7f86b7400640 1 -- 172.21.15.7:0/3942917104 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f86b00c52f0 con 0x7f86b0074210 2024-09-10T21:44:21.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.686+0000 7f86acc00640 1 -- 172.21.15.7:0/3942917104 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v31) ==== 71+0+0 (secure 0 0 0) 0x7f869c083a30 con 0x7f86b0074210 2024-09-10T21:44:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.689+0000 7f86b7400640 1 -- 172.21.15.7:0/3942917104 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8690077f70 msgr2=0x7f869007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.689+0000 7f86b7400640 1 --2- 172.21.15.7:0/3942917104 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8690077f70 0x7f869007a430 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f86a4001740 tx=0x7f86a4009040 comp rx=0 tx=0).stop 2024-09-10T21:44:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.689+0000 7f86b7400640 1 -- 172.21.15.7:0/3942917104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b0074210 msgr2=0x7f86b0153750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.689+0000 7f86b7400640 1 --2- 172.21.15.7:0/3942917104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b0074210 0x7f86b0153750 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f869c009780 tx=0x7f869c033d60 comp rx=0 tx=0).stop 2024-09-10T21:44:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.689+0000 7f86b7400640 1 -- 172.21.15.7:0/3942917104 shutdown_connections 2024-09-10T21:44:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.689+0000 7f86b7400640 1 --2- 172.21.15.7:0/3942917104 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8690077f70 0x7f869007a430 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.689+0000 7f86b7400640 1 --2- 172.21.15.7:0/3942917104 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f86b015cbe0 0x7f86b00b09d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.689+0000 7f86b7400640 1 --2- 172.21.15.7:0/3942917104 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86b0074210 0x7f86b0153750 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.689+0000 7f86b7400640 1 --2- 172.21.15.7:0/3942917104 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86b006ff50 0x7f86b00b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.689+0000 7f86b7400640 1 -- 172.21.15.7:0/3942917104 >> 172.21.15.7:0/3942917104 conn(0x7f86b00b1660 msgr2=0x7f86b00afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.689+0000 7f86b7400640 1 -- 172.21.15.7:0/3942917104 shutdown_connections 2024-09-10T21:44:21.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.689+0000 7f86b7400640 1 -- 172.21.15.7:0/3942917104 wait complete. 2024-09-10T21:44:21.706 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2024-09-10T21:44:21.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.874+0000 7fb391800640 1 -- 172.21.15.7:0/980060954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb38c1604b0 msgr2=0x7fb38c0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.874+0000 7fb391800640 1 --2- 172.21.15.7:0/980060954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb38c1604b0 0x7fb38c0ccef0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb384005fd0 tx=0x7fb38402a820 comp rx=0 tx=0).stop 2024-09-10T21:44:21.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.874+0000 7fb391800640 1 -- 172.21.15.7:0/980060954 shutdown_connections 2024-09-10T21:44:21.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.874+0000 7fb391800640 1 --2- 172.21.15.7:0/980060954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb38c1604b0 0x7fb38c0ccef0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.874+0000 7fb391800640 1 -- 172.21.15.7:0/980060954 >> 172.21.15.7:0/980060954 conn(0x7fb38c0b16d0 msgr2=0x7fb38c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:21.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.874+0000 7fb391800640 1 -- 172.21.15.7:0/980060954 shutdown_connections 2024-09-10T21:44:21.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.874+0000 7fb391800640 1 -- 172.21.15.7:0/980060954 wait complete. 2024-09-10T21:44:21.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.875+0000 7fb391800640 1 Processor -- start 2024-09-10T21:44:21.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.876+0000 7fb391800640 1 -- start start 2024-09-10T21:44:21.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.876+0000 7fb391800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb38c0cd750 0x7fb38c14c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.876+0000 7fb391800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb38c15bc90 0x7fb38c0c6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.876+0000 7fb391800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb38c1604b0 0x7fb38c14d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.876+0000 7fb391800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb38c070080 con 0x7fb38c15bc90 2024-09-10T21:44:21.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.876+0000 7fb391800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb38c06ff00 con 0x7fb38c1604b0 2024-09-10T21:44:21.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.876+0000 7fb391800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fb38c070200 con 0x7fb38c0cd750 2024-09-10T21:44:21.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.877+0000 7fb38a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb38c15bc90 0x7fb38c0c6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.877+0000 7fb38a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb38c15bc90 0x7fb38c0c6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43284/0 (socket says 172.21.15.7:43284) 2024-09-10T21:44:21.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.877+0000 7fb38a000640 1 -- 172.21.15.7:0/2638708507 learned_addr learned my addr 172.21.15.7:0/2638708507 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:21.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.877+0000 7fb38b400640 1 --2- 172.21.15.7:0/2638708507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb38c1604b0 0x7fb38c14d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.877+0000 7fb38aa00640 1 --2- 172.21.15.7:0/2638708507 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb38c0cd750 0x7fb38c14c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.877+0000 7fb38a000640 1 -- 172.21.15.7:0/2638708507 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb38c0cd750 msgr2=0x7fb38c14c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.877+0000 7fb38a000640 1 --2- 172.21.15.7:0/2638708507 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb38c0cd750 0x7fb38c14c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.877+0000 7fb38a000640 1 -- 172.21.15.7:0/2638708507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb38c1604b0 msgr2=0x7fb38c14d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.877+0000 7fb38a000640 1 --2- 172.21.15.7:0/2638708507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb38c1604b0 0x7fb38c14d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.877+0000 7fb38a000640 1 -- 172.21.15.7:0/2638708507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb38c0c36b0 con 0x7fb38c15bc90 2024-09-10T21:44:21.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.877+0000 7fb38a000640 1 --2- 172.21.15.7:0/2638708507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb38c15bc90 0x7fb38c0c6160 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb37800c480 tx=0x7fb37800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:21.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.878+0000 7fb382a00640 1 -- 172.21.15.7:0/2638708507 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378016020 con 0x7fb38c15bc90 2024-09-10T21:44:21.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.878+0000 7fb382a00640 1 -- 172.21.15.7:0/2638708507 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb378012920 con 0x7fb38c15bc90 2024-09-10T21:44:21.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.878+0000 7fb382a00640 1 -- 172.21.15.7:0/2638708507 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378012d70 con 0x7fb38c15bc90 2024-09-10T21:44:21.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.878+0000 7fb391800640 1 -- 172.21.15.7:0/2638708507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb38c0c3970 con 0x7fb38c15bc90 2024-09-10T21:44:21.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.878+0000 7fb391800640 1 -- 172.21.15.7:0/2638708507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb38c0c3f30 con 0x7fb38c15bc90 2024-09-10T21:44:21.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.879+0000 7fb391800640 1 -- 172.21.15.7:0/2638708507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb38c0c43d0 con 0x7fb38c15bc90 2024-09-10T21:44:21.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.880+0000 7fb382a00640 1 -- 172.21.15.7:0/2638708507 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb37801b400 con 0x7fb38c15bc90 2024-09-10T21:44:21.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.881+0000 7fb382a00640 1 --2- 172.21.15.7:0/2638708507 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb36c077f70 0x7fb36c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.882+0000 7fb382a00640 1 -- 172.21.15.7:0/2638708507 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7fb378099c30 con 0x7fb38c15bc90 2024-09-10T21:44:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.882+0000 7fb38aa00640 1 --2- 172.21.15.7:0/2638708507 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb36c077f70 0x7fb36c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.883+0000 7fb38aa00640 1 --2- 172.21.15.7:0/2638708507 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb36c077f70 0x7fb36c07a430 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb37c004f60 tx=0x7fb37c007e30 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:21.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:21.886+0000 7fb382a00640 1 -- 172.21.15.7:0/2638708507 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb378020020 con 0x7fb38c15bc90 2024-09-10T21:44:22.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.159+0000 7fb391800640 1 -- 172.21.15.7:0/2638708507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) -- 0x7fb38c073a00 con 0x7fb38c15bc90 2024-09-10T21:44:22.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.291+0000 7fb382a00640 1 -- 172.21.15.7:0/2638708507 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v130) ==== 149+0+0 (secure 0 0 0) 0x7fb3780623d0 con 0x7fb38c15bc90 2024-09-10T21:44:22.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.294+0000 7fb391800640 1 -- 172.21.15.7:0/2638708507 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb36c077f70 msgr2=0x7fb36c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:22.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.294+0000 7fb391800640 1 --2- 172.21.15.7:0/2638708507 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb36c077f70 0x7fb36c07a430 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb37c004f60 tx=0x7fb37c007e30 comp rx=0 tx=0).stop 2024-09-10T21:44:22.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.294+0000 7fb391800640 1 -- 172.21.15.7:0/2638708507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb38c15bc90 msgr2=0x7fb38c0c6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:22.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.294+0000 7fb391800640 1 --2- 172.21.15.7:0/2638708507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb38c15bc90 0x7fb38c0c6160 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb37800c480 tx=0x7fb37800c940 comp rx=0 tx=0).stop 2024-09-10T21:44:22.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.294+0000 7fb391800640 1 -- 172.21.15.7:0/2638708507 shutdown_connections 2024-09-10T21:44:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.294+0000 7fb391800640 1 --2- 172.21.15.7:0/2638708507 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb36c077f70 0x7fb36c07a430 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.294+0000 7fb391800640 1 --2- 172.21.15.7:0/2638708507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb38c1604b0 0x7fb38c14d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.294+0000 7fb391800640 1 --2- 172.21.15.7:0/2638708507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb38c15bc90 0x7fb38c0c6160 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.294+0000 7fb391800640 1 --2- 172.21.15.7:0/2638708507 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb38c0cd750 0x7fb38c14c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.294+0000 7fb391800640 1 -- 172.21.15.7:0/2638708507 >> 172.21.15.7:0/2638708507 conn(0x7fb38c0b16d0 msgr2=0x7fb38c0b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.294+0000 7fb391800640 1 -- 172.21.15.7:0/2638708507 shutdown_connections 2024-09-10T21:44:22.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.294+0000 7fb391800640 1 -- 172.21.15.7:0/2638708507 wait complete. 2024-09-10T21:44:22.311 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2024-09-10T21:44:22.311 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-09-10T21:44:22.311 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-10T21:44:22.312 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:44:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.470+0000 7f4723200640 1 -- 172.21.15.7:0/2901744299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f472406f590 msgr2=0x7f47240728f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.470+0000 7f4723200640 1 --2- 172.21.15.7:0/2901744299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f472406f590 0x7f47240728f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f470c005fd0 tx=0x7f470c02a540 comp rx=0 tx=0).stop 2024-09-10T21:44:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.471+0000 7f4723200640 1 -- 172.21.15.7:0/2901744299 shutdown_connections 2024-09-10T21:44:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.471+0000 7f4723200640 1 --2- 172.21.15.7:0/2901744299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f472406f590 0x7f47240728f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.471+0000 7f4723200640 1 --2- 172.21.15.7:0/2901744299 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4724146c50 0x7f47240723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.471+0000 7f4723200640 1 -- 172.21.15.7:0/2901744299 >> 172.21.15.7:0/2901744299 conn(0x7f47240b1d70 msgr2=0x7f47240cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.471+0000 7f4723200640 1 -- 172.21.15.7:0/2901744299 shutdown_connections 2024-09-10T21:44:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.471+0000 7f4723200640 1 -- 172.21.15.7:0/2901744299 wait complete. 2024-09-10T21:44:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.472+0000 7f4723200640 1 Processor -- start 2024-09-10T21:44:22.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.472+0000 7f4723200640 1 -- start start 2024-09-10T21:44:22.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.473+0000 7f4723200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f472406f590 0x7f4724150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:22.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.473+0000 7f4723200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4724146c50 0x7f47241f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:22.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.473+0000 7f4723200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47241470b0 0x7f47240c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:22.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.473+0000 7f4723200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4724161f10 con 0x7f472406f590 2024-09-10T21:44:22.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.473+0000 7f4723200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4724073200 con 0x7f4724146c50 2024-09-10T21:44:22.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.473+0000 7f4723200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4724162090 con 0x7f47241470b0 2024-09-10T21:44:22.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.473+0000 7f4721e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f472406f590 0x7f4724150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:22.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.473+0000 7f4721e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f472406f590 0x7f4724150150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43292/0 (socket says 172.21.15.7:43292) 2024-09-10T21:44:22.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.473+0000 7f4721e00640 1 -- 172.21.15.7:0/3092084186 learned_addr learned my addr 172.21.15.7:0/3092084186 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:22.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.474+0000 7f471be00640 1 --2- 172.21.15.7:0/3092084186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4724146c50 0x7f47241f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:22.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.474+0000 7f4722800640 1 --2- 172.21.15.7:0/3092084186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47241470b0 0x7f47240c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:22.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.474+0000 7f4721e00640 1 -- 172.21.15.7:0/3092084186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47241470b0 msgr2=0x7f47240c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:22.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.474+0000 7f4721e00640 1 --2- 172.21.15.7:0/3092084186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47241470b0 0x7f47240c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.474+0000 7f4721e00640 1 -- 172.21.15.7:0/3092084186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4724146c50 msgr2=0x7f47241f4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:22.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.474+0000 7f4721e00640 1 --2- 172.21.15.7:0/3092084186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4724146c50 0x7f47241f4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.474+0000 7f4721e00640 1 -- 172.21.15.7:0/3092084186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47241f5610 con 0x7f472406f590 2024-09-10T21:44:22.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.474+0000 7f471be00640 1 --2- 172.21.15.7:0/3092084186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4724146c50 0x7f47241f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:44:22.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.474+0000 7f4722800640 1 --2- 172.21.15.7:0/3092084186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47241470b0 0x7f47240c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:44:22.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.474+0000 7f4721e00640 1 --2- 172.21.15.7:0/3092084186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f472406f590 0x7f4724150150 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f471400e9f0 tx=0x7f471400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:22.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.474+0000 7f471a000640 1 -- 172.21.15.7:0/3092084186 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471400cd20 con 0x7f472406f590 2024-09-10T21:44:22.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.474+0000 7f471a000640 1 -- 172.21.15.7:0/3092084186 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4714010430 con 0x7f472406f590 2024-09-10T21:44:22.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.475+0000 7f471a000640 1 -- 172.21.15.7:0/3092084186 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714010840 con 0x7f472406f590 2024-09-10T21:44:22.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.475+0000 7f4723200640 1 -- 172.21.15.7:0/3092084186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47241f58f0 con 0x7f472406f590 2024-09-10T21:44:22.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.475+0000 7f4723200640 1 -- 172.21.15.7:0/3092084186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4724140350 con 0x7f472406f590 2024-09-10T21:44:22.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.476+0000 7f4723200640 1 -- 172.21.15.7:0/3092084186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4724162540 con 0x7f472406f590 2024-09-10T21:44:22.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.477+0000 7f471a000640 1 -- 172.21.15.7:0/3092084186 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f47140109e0 con 0x7f472406f590 2024-09-10T21:44:22.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.478+0000 7f471a000640 1 --2- 172.21.15.7:0/3092084186 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4700077fa0 0x7f470007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:22.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.478+0000 7f471a000640 1 -- 172.21.15.7:0/3092084186 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f471409a590 con 0x7f472406f590 2024-09-10T21:44:22.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.478+0000 7f471be00640 1 --2- 172.21.15.7:0/3092084186 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4700077fa0 0x7f470007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:22.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.479+0000 7f471be00640 1 --2- 172.21.15.7:0/3092084186 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4700077fa0 0x7f470007a460 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f470c033000 tx=0x7f470c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:22.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.482+0000 7f471a000640 1 -- 172.21.15.7:0/3092084186 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4714063440 con 0x7f472406f590 2024-09-10T21:44:22.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.679+0000 7f4723200640 1 -- 172.21.15.7:0/3092084186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4724142e80 con 0x7f472406f590 2024-09-10T21:44:22.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.680+0000 7f471a000640 1 -- 172.21.15.7:0/3092084186 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+12998 (secure 0 0 0) 0x7f4714062d40 con 0x7f472406f590 2024-09-10T21:44:22.681 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:22.681 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":49,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:44:21.283364+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6837/3385628197":"2024-09-11T21:44:18.269351+0000","172.21.15.133:6834/3385628197":"2024-09-11T21:44:18.269351+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:44:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.682+0000 7f4723200640 1 -- 172.21.15.7:0/3092084186 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4700077fa0 msgr2=0x7f470007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.682+0000 7f4723200640 1 --2- 172.21.15.7:0/3092084186 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4700077fa0 0x7f470007a460 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f470c033000 tx=0x7f470c004730 comp rx=0 tx=0).stop 2024-09-10T21:44:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.682+0000 7f4723200640 1 -- 172.21.15.7:0/3092084186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f472406f590 msgr2=0x7f4724150150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.682+0000 7f4723200640 1 --2- 172.21.15.7:0/3092084186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f472406f590 0x7f4724150150 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f471400e9f0 tx=0x7f471400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.682+0000 7f4723200640 1 -- 172.21.15.7:0/3092084186 shutdown_connections 2024-09-10T21:44:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.682+0000 7f4723200640 1 --2- 172.21.15.7:0/3092084186 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4700077fa0 0x7f470007a460 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.682+0000 7f4723200640 1 --2- 172.21.15.7:0/3092084186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4724146c50 0x7f47241f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.682+0000 7f4723200640 1 --2- 172.21.15.7:0/3092084186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47241470b0 0x7f47240c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.682+0000 7f4723200640 1 --2- 172.21.15.7:0/3092084186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f472406f590 0x7f4724150150 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.682+0000 7f4723200640 1 -- 172.21.15.7:0/3092084186 >> 172.21.15.7:0/3092084186 conn(0x7f47240b1d70 msgr2=0x7f4724141fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:22.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.683+0000 7f4723200640 1 -- 172.21.15.7:0/3092084186 shutdown_connections 2024-09-10T21:44:22.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.683+0000 7f4723200640 1 -- 172.21.15.7:0/3092084186 wait complete. 2024-09-10T21:44:22.700 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:41:55.889618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:44:22.701 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:44:22.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.863+0000 7f0959800640 1 -- 172.21.15.7:0/1360019035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f095415bcd0 msgr2=0x7f095406cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:22.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.863+0000 7f0959800640 1 --2- 172.21.15.7:0/1360019035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f095415bcd0 0x7f095406cb20 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f095406ee60 tx=0x7f094402a440 comp rx=0 tx=0).stop 2024-09-10T21:44:22.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.863+0000 7f0959800640 1 -- 172.21.15.7:0/1360019035 shutdown_connections 2024-09-10T21:44:22.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.863+0000 7f0959800640 1 --2- 172.21.15.7:0/1360019035 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09540734c0 0x7f0954073930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.863+0000 7f0959800640 1 --2- 172.21.15.7:0/1360019035 >> v2:172.21.15.133:3300/0 conn(0x7f09540b0190 0x7f095406d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.863+0000 7f0959800640 1 --2- 172.21.15.7:0/1360019035 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f095415bcd0 0x7f095406cb20 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.863+0000 7f0959800640 1 -- 172.21.15.7:0/1360019035 >> 172.21.15.7:0/1360019035 conn(0x7f09540d0b40 msgr2=0x7f09540d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:22.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.863+0000 7f0959800640 1 -- 172.21.15.7:0/1360019035 shutdown_connections 2024-09-10T21:44:22.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.864+0000 7f0959800640 1 -- 172.21.15.7:0/1360019035 wait complete. 2024-09-10T21:44:22.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.865+0000 7f0959800640 1 Processor -- start 2024-09-10T21:44:22.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.865+0000 7f0959800640 1 -- start start 2024-09-10T21:44:22.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.866+0000 7f0959800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09540734c0 0x7f09540ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:22.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.866+0000 7f0959800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09540b0190 0x7f09541f6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.866+0000 7f0959800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f095415bcd0 0x7f0954152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.866+0000 7f0959800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0954074770 con 0x7f095415bcd0 2024-09-10T21:44:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.866+0000 7f0959800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f09540745f0 con 0x7f09540734c0 2024-09-10T21:44:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.866+0000 7f0959800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f09540ce710 con 0x7f09540b0190 2024-09-10T21:44:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.866+0000 7f0958e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f095415bcd0 0x7f0954152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.866+0000 7f0958e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f095415bcd0 0x7f0954152310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43316/0 (socket says 172.21.15.7:43316) 2024-09-10T21:44:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.866+0000 7f0958e00640 1 -- 172.21.15.7:0/2812524948 learned_addr learned my addr 172.21.15.7:0/2812524948 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.866+0000 7f094b400640 1 --2- 172.21.15.7:0/2812524948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09540b0190 0x7f09541f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.867+0000 7f0953e00640 1 --2- 172.21.15.7:0/2812524948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09540734c0 0x7f09540ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.867+0000 7f0958e00640 1 -- 172.21.15.7:0/2812524948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09540b0190 msgr2=0x7f09541f6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.867+0000 7f0958e00640 1 --2- 172.21.15.7:0/2812524948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09540b0190 0x7f09541f6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.867+0000 7f0958e00640 1 -- 172.21.15.7:0/2812524948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09540734c0 msgr2=0x7f09540ca7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:22.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.867+0000 7f0958e00640 1 --2- 172.21.15.7:0/2812524948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09540734c0 0x7f09540ca7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.867+0000 7f0958e00640 1 -- 172.21.15.7:0/2812524948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09541559a0 con 0x7f095415bcd0 2024-09-10T21:44:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.867+0000 7f0958e00640 1 --2- 172.21.15.7:0/2812524948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f095415bcd0 0x7f0954152310 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f094c00e9b0 tx=0x7f094c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.867+0000 7f0950c00640 1 -- 172.21.15.7:0/2812524948 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c004d20 con 0x7f095415bcd0 2024-09-10T21:44:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.867+0000 7f0950c00640 1 -- 172.21.15.7:0/2812524948 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f094c00cb20 con 0x7f095415bcd0 2024-09-10T21:44:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.867+0000 7f0950c00640 1 -- 172.21.15.7:0/2812524948 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f094c005020 con 0x7f095415bcd0 2024-09-10T21:44:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.867+0000 7f0959800640 1 -- 172.21.15.7:0/2812524948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0954155c80 con 0x7f095415bcd0 2024-09-10T21:44:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.868+0000 7f0959800640 1 -- 172.21.15.7:0/2812524948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09541553f0 con 0x7f095415bcd0 2024-09-10T21:44:22.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.869+0000 7f0959800640 1 -- 172.21.15.7:0/2812524948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09540afa50 con 0x7f095415bcd0 2024-09-10T21:44:22.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.870+0000 7f0950c00640 1 -- 172.21.15.7:0/2812524948 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f094c00f040 con 0x7f095415bcd0 2024-09-10T21:44:22.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.871+0000 7f0950c00640 1 --2- 172.21.15.7:0/2812524948 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0930077f70 0x7f093007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:22.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.871+0000 7f0950c00640 1 -- 172.21.15.7:0/2812524948 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f094c099fc0 con 0x7f095415bcd0 2024-09-10T21:44:22.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.871+0000 7f0953e00640 1 --2- 172.21.15.7:0/2812524948 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0930077f70 0x7f093007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:22.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.872+0000 7f0953e00640 1 --2- 172.21.15.7:0/2812524948 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0930077f70 0x7f093007a430 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0944033000 tx=0x7f0944004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:22.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:22.875+0000 7f0950c00640 1 -- 172.21.15.7:0/2812524948 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f094c016030 con 0x7f095415bcd0 2024-09-10T21:44:23.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.068+0000 7f0959800640 1 -- 172.21.15.7:0/2812524948 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f095406cb20 con 0x7f095415bcd0 2024-09-10T21:44:23.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.068+0000 7f0950c00640 1 -- 172.21.15.7:0/2812524948 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v49) ==== 93+0+10 (secure 0 0 0) 0x7f094c062760 con 0x7f095415bcd0 2024-09-10T21:44:23.069 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-10T21:44:23.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.070+0000 7f0959800640 1 -- 172.21.15.7:0/2812524948 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0930077f70 msgr2=0x7f093007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:23.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.070+0000 7f0959800640 1 --2- 172.21.15.7:0/2812524948 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0930077f70 0x7f093007a430 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0944033000 tx=0x7f0944004730 comp rx=0 tx=0).stop 2024-09-10T21:44:23.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.070+0000 7f0959800640 1 -- 172.21.15.7:0/2812524948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f095415bcd0 msgr2=0x7f0954152310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:23.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.070+0000 7f0959800640 1 --2- 172.21.15.7:0/2812524948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f095415bcd0 0x7f0954152310 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f094c00e9b0 tx=0x7f094c00ee70 comp rx=0 tx=0).stop 2024-09-10T21:44:23.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.070+0000 7f0959800640 1 -- 172.21.15.7:0/2812524948 shutdown_connections 2024-09-10T21:44:23.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.070+0000 7f0959800640 1 --2- 172.21.15.7:0/2812524948 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0930077f70 0x7f093007a430 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:23.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.070+0000 7f0959800640 1 --2- 172.21.15.7:0/2812524948 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f095415bcd0 0x7f0954152310 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:23.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.070+0000 7f0959800640 1 --2- 172.21.15.7:0/2812524948 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09540b0190 0x7f09541f6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:23.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.070+0000 7f0959800640 1 --2- 172.21.15.7:0/2812524948 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09540734c0 0x7f09540ca7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:23.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.070+0000 7f0959800640 1 -- 172.21.15.7:0/2812524948 >> 172.21.15.7:0/2812524948 conn(0x7f09540d0b40 msgr2=0x7f09540b5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:23.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.071+0000 7f0959800640 1 -- 172.21.15.7:0/2812524948 shutdown_connections 2024-09-10T21:44:23.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.071+0000 7f0959800640 1 -- 172.21.15.7:0/2812524948 wait complete. 2024-09-10T21:44:23.088 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-09-10T21:44:23.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.248+0000 7fdfe1200640 1 -- 172.21.15.7:0/2058489348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfdc0af600 msgr2=0x7fdfdc0caf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:23.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.248+0000 7fdfe1200640 1 --2- 172.21.15.7:0/2058489348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfdc0af600 0x7fdfdc0caf90 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc005fd0 tx=0x7fdfcc02a540 comp rx=0 tx=0).stop 2024-09-10T21:44:23.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.248+0000 7fdfe1200640 1 -- 172.21.15.7:0/2058489348 shutdown_connections 2024-09-10T21:44:23.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.248+0000 7fdfe1200640 1 --2- 172.21.15.7:0/2058489348 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdfdc0b1cc0 0x7fdfdc0b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:23.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.248+0000 7fdfe1200640 1 --2- 172.21.15.7:0/2058489348 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfdc0af600 0x7fdfdc0caf90 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:23.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.248+0000 7fdfe1200640 1 -- 172.21.15.7:0/2058489348 >> 172.21.15.7:0/2058489348 conn(0x7fdfdc0cccc0 msgr2=0x7fdfdc0cd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:23.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.248+0000 7fdfe1200640 1 -- 172.21.15.7:0/2058489348 shutdown_connections 2024-09-10T21:44:23.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.248+0000 7fdfe1200640 1 -- 172.21.15.7:0/2058489348 wait complete. 2024-09-10T21:44:23.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.250+0000 7fdfe1200640 1 Processor -- start 2024-09-10T21:44:23.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.250+0000 7fdfe1200640 1 -- start start 2024-09-10T21:44:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.251+0000 7fdfe1200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdfdc06eb30 0x7fdfdc1f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.251+0000 7fdfe1200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfdc0af600 0x7fdfdc0c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.251+0000 7fdfe1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfdc0b1cc0 0x7fdfdc0c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.251+0000 7fdfe1200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc068af0 con 0x7fdfdc0b1cc0 2024-09-10T21:44:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.251+0000 7fdfe1200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdfdc0afaf0 con 0x7fdfdc0af600 2024-09-10T21:44:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.251+0000 7fdfe1200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fdfdc068c70 con 0x7fdfdc06eb30 2024-09-10T21:44:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.251+0000 7fdfdbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfdc0b1cc0 0x7fdfdc0c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.251+0000 7fdfdbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfdc0b1cc0 0x7fdfdc0c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43334/0 (socket says 172.21.15.7:43334) 2024-09-10T21:44:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.251+0000 7fdfdbe00640 1 -- 172.21.15.7:0/3691364048 learned_addr learned my addr 172.21.15.7:0/3691364048 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.251+0000 7fdfdb400640 1 --2- 172.21.15.7:0/3691364048 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdfdc06eb30 0x7fdfdc1f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.251+0000 7fdfd3e00640 1 --2- 172.21.15.7:0/3691364048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfdc0af600 0x7fdfdc0c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.251+0000 7fdfdbe00640 1 -- 172.21.15.7:0/3691364048 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdfdc06eb30 msgr2=0x7fdfdc1f5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:23.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.251+0000 7fdfdbe00640 1 --2- 172.21.15.7:0/3691364048 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdfdc06eb30 0x7fdfdc1f5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:23.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.251+0000 7fdfdbe00640 1 -- 172.21.15.7:0/3691364048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfdc0af600 msgr2=0x7fdfdc0c98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:23.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.251+0000 7fdfdbe00640 1 --2- 172.21.15.7:0/3691364048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfdc0af600 0x7fdfdc0c98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:23.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.252+0000 7fdfdbe00640 1 -- 172.21.15.7:0/3691364048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfdc0c3ea0 con 0x7fdfdc0b1cc0 2024-09-10T21:44:23.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.252+0000 7fdfdbe00640 1 --2- 172.21.15.7:0/3691364048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfdc0b1cc0 0x7fdfdc0c9e00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400c6e0 tx=0x7fdfd400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:23.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.252+0000 7fdfd3400640 1 -- 172.21.15.7:0/3691364048 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4003b70 con 0x7fdfdc0b1cc0 2024-09-10T21:44:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.252+0000 7fdfd3400640 1 -- 172.21.15.7:0/3691364048 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfd4003d30 con 0x7fdfdc0b1cc0 2024-09-10T21:44:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.252+0000 7fdfe1200640 1 -- 172.21.15.7:0/3691364048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfdc0c4180 con 0x7fdfdc0b1cc0 2024-09-10T21:44:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.252+0000 7fdfd3400640 1 -- 172.21.15.7:0/3691364048 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfd4005330 con 0x7fdfdc0b1cc0 2024-09-10T21:44:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.252+0000 7fdfe1200640 1 -- 172.21.15.7:0/3691364048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfdc0c4740 con 0x7fdfdc0b1cc0 2024-09-10T21:44:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.253+0000 7fdfe1200640 1 -- 172.21.15.7:0/3691364048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfdc073720 con 0x7fdfdc0b1cc0 2024-09-10T21:44:23.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.255+0000 7fdfd3400640 1 -- 172.21.15.7:0/3691364048 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fdfd40054d0 con 0x7fdfdc0b1cc0 2024-09-10T21:44:23.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.255+0000 7fdfd3400640 1 --2- 172.21.15.7:0/3691364048 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdfb8077f70 0x7fdfb807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:23.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.256+0000 7fdfd3400640 1 -- 172.21.15.7:0/3691364048 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7fdfd40998d0 con 0x7fdfdc0b1cc0 2024-09-10T21:44:23.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.256+0000 7fdfdb400640 1 --2- 172.21.15.7:0/3691364048 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdfb8077f70 0x7fdfb807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:23.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.257+0000 7fdfdb400640 1 --2- 172.21.15.7:0/3691364048 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdfb8077f70 0x7fdfb807a430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc033000 tx=0x7fdfcc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:23.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.260+0000 7fdfd3400640 1 -- 172.21.15.7:0/3691364048 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fdfdc073720 con 0x7fdfdc0b1cc0 2024-09-10T21:44:23.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:23.525+0000 7fdfe1200640 1 -- 172.21.15.7:0/3691364048 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7fdfdc0caf90 con 0x7fdfdc0b1cc0 2024-09-10T21:44:24.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.298+0000 7fdfd3400640 1 -- 172.21.15.7:0/3691364048 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v132) ==== 172+0+0 (secure 0 0 0) 0x7fdfd40057f0 con 0x7fdfdc0b1cc0 2024-09-10T21:44:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.301+0000 7fdfe1200640 1 -- 172.21.15.7:0/3691364048 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdfb8077f70 msgr2=0x7fdfb807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.301+0000 7fdfe1200640 1 --2- 172.21.15.7:0/3691364048 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdfb8077f70 0x7fdfb807a430 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc033000 tx=0x7fdfcc004730 comp rx=0 tx=0).stop 2024-09-10T21:44:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.301+0000 7fdfe1200640 1 -- 172.21.15.7:0/3691364048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfdc0b1cc0 msgr2=0x7fdfdc0c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:24.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.301+0000 7fdfe1200640 1 --2- 172.21.15.7:0/3691364048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfdc0b1cc0 0x7fdfdc0c9e00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fdfd400c6e0 tx=0x7fdfd400cba0 comp rx=0 tx=0).stop 2024-09-10T21:44:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.301+0000 7fdfe1200640 1 -- 172.21.15.7:0/3691364048 shutdown_connections 2024-09-10T21:44:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.301+0000 7fdfe1200640 1 --2- 172.21.15.7:0/3691364048 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdfb8077f70 0x7fdfb807a430 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.301+0000 7fdfe1200640 1 --2- 172.21.15.7:0/3691364048 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdfdc0af600 0x7fdfdc0c98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.301+0000 7fdfe1200640 1 --2- 172.21.15.7:0/3691364048 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdfdc0b1cc0 0x7fdfdc0c9e00 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.301+0000 7fdfe1200640 1 --2- 172.21.15.7:0/3691364048 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdfdc06eb30 0x7fdfdc1f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.301+0000 7fdfe1200640 1 -- 172.21.15.7:0/3691364048 >> 172.21.15.7:0/3691364048 conn(0x7fdfdc0cccc0 msgr2=0x7fdfdc072c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.301+0000 7fdfe1200640 1 -- 172.21.15.7:0/3691364048 shutdown_connections 2024-09-10T21:44:24.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.301+0000 7fdfe1200640 1 -- 172.21.15.7:0/3691364048 wait complete. 2024-09-10T21:44:24.320 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:44:24.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.484+0000 7f4049200640 1 -- 172.21.15.7:0/4067953264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40440cd750 msgr2=0x7f40440ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:24.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.484+0000 7f4049200640 1 --2- 172.21.15.7:0/4067953264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40440cd750 0x7f40440ccef0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f40440c3650 tx=0x7f402c02a580 comp rx=0 tx=0).stop 2024-09-10T21:44:24.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.484+0000 7f4049200640 1 -- 172.21.15.7:0/4067953264 shutdown_connections 2024-09-10T21:44:24.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.484+0000 7f4049200640 1 --2- 172.21.15.7:0/4067953264 >> v2:172.21.15.133:3300/0 conn(0x7f404415fda0 0x7f4044160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:24.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.484+0000 7f4049200640 1 --2- 172.21.15.7:0/4067953264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40440cd750 0x7f40440ccef0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:24.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.484+0000 7f4049200640 1 -- 172.21.15.7:0/4067953264 >> 172.21.15.7:0/4067953264 conn(0x7f40440b16d0 msgr2=0x7f40440b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:24.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.484+0000 7f4049200640 1 -- 172.21.15.7:0/4067953264 shutdown_connections 2024-09-10T21:44:24.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.484+0000 7f4049200640 1 -- 172.21.15.7:0/4067953264 wait complete. 2024-09-10T21:44:24.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.486+0000 7f4049200640 1 Processor -- start 2024-09-10T21:44:24.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.486+0000 7f4049200640 1 -- start start 2024-09-10T21:44:24.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.487+0000 7f4049200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40440cd750 0x7f404414dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:24.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.487+0000 7f4049200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f404415bc90 0x7f404415eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:24.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.487+0000 7f4049200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f404415fda0 0x7f404414d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:24.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.487+0000 7f4049200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f404406fe70 con 0x7f404415bc90 2024-09-10T21:44:24.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.487+0000 7f4049200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f404406fcf0 con 0x7f404415fda0 2024-09-10T21:44:24.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.487+0000 7f4049200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f404406fff0 con 0x7f40440cd750 2024-09-10T21:44:24.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.487+0000 7f4039400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f404415bc90 0x7f404415eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:24.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.487+0000 7f4039400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f404415bc90 0x7f404415eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36786/0 (socket says 172.21.15.7:36786) 2024-09-10T21:44:24.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.487+0000 7f4039400640 1 -- 172.21.15.7:0/2974215955 learned_addr learned my addr 172.21.15.7:0/2974215955 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:24.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.487+0000 7f4041e00640 1 --2- 172.21.15.7:0/2974215955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40440cd750 0x7f404414dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:24.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.487+0000 7f4042800640 1 --2- 172.21.15.7:0/2974215955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f404415fda0 0x7f404414d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:24.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.488+0000 7f4039400640 1 -- 172.21.15.7:0/2974215955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40440cd750 msgr2=0x7f404414dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:24.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.488+0000 7f4039400640 1 --2- 172.21.15.7:0/2974215955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40440cd750 0x7f404414dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:24.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.488+0000 7f4039400640 1 -- 172.21.15.7:0/2974215955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f404415fda0 msgr2=0x7f404414d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:24.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.488+0000 7f4039400640 1 --2- 172.21.15.7:0/2974215955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f404415fda0 0x7f404414d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:24.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.488+0000 7f4039400640 1 -- 172.21.15.7:0/2974215955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404414dca0 con 0x7f404415bc90 2024-09-10T21:44:24.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.488+0000 7f4039400640 1 --2- 172.21.15.7:0/2974215955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f404415bc90 0x7f404415eec0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f402c000c00 tx=0x7f402c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:24.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.488+0000 7f403aa00640 1 -- 172.21.15.7:0/2974215955 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c03f070 con 0x7f404415bc90 2024-09-10T21:44:24.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.488+0000 7f403aa00640 1 -- 172.21.15.7:0/2974215955 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f402c0047a0 con 0x7f404415bc90 2024-09-10T21:44:24.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.488+0000 7f4049200640 1 -- 172.21.15.7:0/2974215955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40440adcb0 con 0x7f404415bc90 2024-09-10T21:44:24.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.488+0000 7f403aa00640 1 -- 172.21.15.7:0/2974215955 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402c035030 con 0x7f404415bc90 2024-09-10T21:44:24.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.489+0000 7f4049200640 1 -- 172.21.15.7:0/2974215955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40440af980 con 0x7f404415bc90 2024-09-10T21:44:24.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.490+0000 7f4049200640 1 -- 172.21.15.7:0/2974215955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4044070510 con 0x7f404415bc90 2024-09-10T21:44:24.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.491+0000 7f403aa00640 1 -- 172.21.15.7:0/2974215955 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f402c004bb0 con 0x7f404415bc90 2024-09-10T21:44:24.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.492+0000 7f403aa00640 1 --2- 172.21.15.7:0/2974215955 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4020077fa0 0x7f402007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:24.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.492+0000 7f403aa00640 1 -- 172.21.15.7:0/2974215955 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f402c0baac0 con 0x7f404415bc90 2024-09-10T21:44:24.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.492+0000 7f4041e00640 1 --2- 172.21.15.7:0/2974215955 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4020077fa0 0x7f402007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:24.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.493+0000 7f4041e00640 1 --2- 172.21.15.7:0/2974215955 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4020077fa0 0x7f402007a460 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f40340017c0 tx=0x7f4034009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:24.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.496+0000 7f403aa00640 1 -- 172.21.15.7:0/2974215955 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f402c0831d0 con 0x7f404415bc90 2024-09-10T21:44:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.693+0000 7f4049200640 1 -- 172.21.15.7:0/2974215955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f4044069810 con 0x7f404415bc90 2024-09-10T21:44:24.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.693+0000 7f403aa00640 1 -- 172.21.15.7:0/2974215955 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+1998 (secure 0 0 0) 0x7f402c087c00 con 0x7f404415bc90 2024-09-10T21:44:24.695 INFO:teuthology.orchestra.run.smithi007.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:44:24.695 INFO:teuthology.orchestra.run.smithi007.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:44:24.695 INFO:teuthology.orchestra.run.smithi007.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:44:24.695 INFO:teuthology.orchestra.run.smithi007.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:44:24.695 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3770.177547 2024-09-10T21:44:24.695 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3542.062395 2024-09-10T21:44:24.695 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3788.037135 2024-09-10T21:44:24.695 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5808.986321 2024-09-10T21:44:24.695 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4630.109911 2024-09-10T21:44:24.695 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4510.255493 2024-09-10T21:44:24.695 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4792.698527 2024-09-10T21:44:24.695 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6842.370281 2024-09-10T21:44:24.696 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_mds 20/20 2024-09-10T21:44:24.696 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:44:24.696 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_ms 2/2 2024-09-10T21:44:24.696 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_debug_frag true 2024-09-10T21:44:24.696 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:44:24.696 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:44:24.696 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_verify_scatter true 2024-09-10T21:44:24.696 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:44:24.696 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:44:24.696 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:44:24.696 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced client_mount_timeout 600 2024-09-10T21:44:24.696 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced debug_client 20/20 2024-09-10T21:44:24.696 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced debug_ms 1/1 2024-09-10T21:44:24.696 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:44:24.697 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:44:24.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.696+0000 7f4049200640 1 -- 172.21.15.7:0/2974215955 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4020077fa0 msgr2=0x7f402007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:24.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.696+0000 7f4049200640 1 --2- 172.21.15.7:0/2974215955 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4020077fa0 0x7f402007a460 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f40340017c0 tx=0x7f4034009040 comp rx=0 tx=0).stop 2024-09-10T21:44:24.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.696+0000 7f4049200640 1 -- 172.21.15.7:0/2974215955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f404415bc90 msgr2=0x7f404415eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:24.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.696+0000 7f4049200640 1 --2- 172.21.15.7:0/2974215955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f404415bc90 0x7f404415eec0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f402c000c00 tx=0x7f402c003820 comp rx=0 tx=0).stop 2024-09-10T21:44:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.696+0000 7f4049200640 1 -- 172.21.15.7:0/2974215955 shutdown_connections 2024-09-10T21:44:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.696+0000 7f4049200640 1 --2- 172.21.15.7:0/2974215955 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4020077fa0 0x7f402007a460 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.696+0000 7f4049200640 1 --2- 172.21.15.7:0/2974215955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f404415fda0 0x7f404414d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.696+0000 7f4049200640 1 --2- 172.21.15.7:0/2974215955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f404415bc90 0x7f404415eec0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.696+0000 7f4049200640 1 --2- 172.21.15.7:0/2974215955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40440cd750 0x7f404414dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.696+0000 7f4049200640 1 -- 172.21.15.7:0/2974215955 >> 172.21.15.7:0/2974215955 conn(0x7f40440b16d0 msgr2=0x7f40440b2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.696+0000 7f4049200640 1 -- 172.21.15.7:0/2974215955 shutdown_connections 2024-09-10T21:44:24.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.696+0000 7f4049200640 1 -- 172.21.15.7:0/2974215955 wait complete. 2024-09-10T21:44:24.713 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:44:24.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.876+0000 7f069e600640 1 -- 172.21.15.7:0/1837748968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f067c004d40 msgr2=0x7f067c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:24.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.876+0000 7f069e600640 1 --2- 172.21.15.7:0/1837748968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f067c004d40 0x7f067c0051c0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f069000c610 tx=0x7f067c005830 comp rx=0 tx=0).stop 2024-09-10T21:44:24.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.876+0000 7f0697e00640 1 -- 172.21.15.7:0/1837748968 reap_dead start 2024-09-10T21:44:24.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.876+0000 7f069e600640 1 -- 172.21.15.7:0/1837748968 shutdown_connections 2024-09-10T21:44:24.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.876+0000 7f069e600640 1 -- 172.21.15.7:0/1837748968 >> 172.21.15.7:0/1837748968 conn(0x7f06980b16d0 msgr2=0x7f06980b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:24.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.876+0000 7f069e600640 1 -- 172.21.15.7:0/1837748968 shutdown_connections 2024-09-10T21:44:24.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.876+0000 7f069e600640 1 -- 172.21.15.7:0/1837748968 wait complete. 2024-09-10T21:44:24.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.878+0000 7f069e600640 1 Processor -- start 2024-09-10T21:44:24.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.878+0000 7f069e600640 1 -- start start 2024-09-10T21:44:24.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.879+0000 7f069e600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f067c004d40 0x7f069814e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:24.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.879+0000 7f069e600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06980ce400 0x7f06980c7830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:24.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.879+0000 7f069e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06980c5820 0x7f06980c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:24.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.879+0000 7f069e600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0698073d40 con 0x7f06980c5820 2024-09-10T21:44:24.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.879+0000 7f069e600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0698074690 con 0x7f06980ce400 2024-09-10T21:44:24.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.879+0000 7f069e600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f0698073ec0 con 0x7f067c004d40 2024-09-10T21:44:24.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.879+0000 7f0697e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06980c5820 0x7f06980c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:24.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.879+0000 7f0697e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06980c5820 0x7f06980c5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36804/0 (socket says 172.21.15.7:36804) 2024-09-10T21:44:24.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.879+0000 7f0697e00640 1 -- 172.21.15.7:0/3843979746 learned_addr learned my addr 172.21.15.7:0/3843979746 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:24.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.880+0000 7f068ea00640 1 --2- 172.21.15.7:0/3843979746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06980ce400 0x7f06980c7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:24.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.880+0000 7f0697400640 1 --2- 172.21.15.7:0/3843979746 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f067c004d40 0x7f069814e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:24.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.880+0000 7f0697e00640 1 -- 172.21.15.7:0/3843979746 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f067c004d40 msgr2=0x7f069814e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:24.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.880+0000 7f0697e00640 1 --2- 172.21.15.7:0/3843979746 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f067c004d40 0x7f069814e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:24.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.880+0000 7f0697e00640 1 -- 172.21.15.7:0/3843979746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06980ce400 msgr2=0x7f06980c7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:24.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.880+0000 7f0697e00640 1 --2- 172.21.15.7:0/3843979746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06980ce400 0x7f06980c7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:24.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.880+0000 7f0697e00640 1 -- 172.21.15.7:0/3843979746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06980c8130 con 0x7f06980c5820 2024-09-10T21:44:24.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.880+0000 7f068ea00640 1 --2- 172.21.15.7:0/3843979746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06980ce400 0x7f06980c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:44:24.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.880+0000 7f0697400640 1 --2- 172.21.15.7:0/3843979746 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f067c004d40 0x7f069814e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:44:24.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.880+0000 7f0697e00640 1 --2- 172.21.15.7:0/3843979746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06980c5820 0x7f06980c5ca0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f069000c700 tx=0x7f0690003bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:24.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.880+0000 7f068fe00640 1 -- 172.21.15.7:0/3843979746 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0690039070 con 0x7f06980c5820 2024-09-10T21:44:24.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.880+0000 7f068fe00640 1 -- 172.21.15.7:0/3843979746 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f069002ba30 con 0x7f06980c5820 2024-09-10T21:44:24.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.880+0000 7f069e600640 1 -- 172.21.15.7:0/3843979746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06980cebc0 con 0x7f06980c5820 2024-09-10T21:44:24.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.880+0000 7f068fe00640 1 -- 172.21.15.7:0/3843979746 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0690034680 con 0x7f06980c5820 2024-09-10T21:44:24.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.881+0000 7f069e600640 1 -- 172.21.15.7:0/3843979746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f069814e700 con 0x7f06980c5820 2024-09-10T21:44:24.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.882+0000 7f069e600640 1 -- 172.21.15.7:0/3843979746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06980c61e0 con 0x7f06980c5820 2024-09-10T21:44:24.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.883+0000 7f068fe00640 1 -- 172.21.15.7:0/3843979746 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0690047060 con 0x7f06980c5820 2024-09-10T21:44:24.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.883+0000 7f068fe00640 1 --2- 172.21.15.7:0/3843979746 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0674077fa0 0x7f067407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:24.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.884+0000 7f068fe00640 1 -- 172.21.15.7:0/3843979746 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f06900b97f0 con 0x7f06980c5820 2024-09-10T21:44:24.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.884+0000 7f0697400640 1 --2- 172.21.15.7:0/3843979746 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0674077fa0 0x7f067407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:24.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.885+0000 7f0697400640 1 --2- 172.21.15.7:0/3843979746 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0674077fa0 0x7f067407a460 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0688009b40 tx=0x7f0688005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:24.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:24.888+0000 7f068fe00640 1 -- 172.21.15.7:0/3843979746 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f06900824c0 con 0x7f06980c5820 2024-09-10T21:44:25.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.083+0000 7f069e600640 1 -- 172.21.15.7:0/3843979746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f06980c9e30 con 0x7f06980c5820 2024-09-10T21:44:25.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.087+0000 7f068fe00640 1 -- 172.21.15.7:0/3843979746 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f0690081dc0 con 0x7f06980c5820 2024-09-10T21:44:25.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.090+0000 7f069e600640 1 -- 172.21.15.7:0/3843979746 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0674077fa0 msgr2=0x7f067407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:25.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.090+0000 7f069e600640 1 --2- 172.21.15.7:0/3843979746 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0674077fa0 0x7f067407a460 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0688009b40 tx=0x7f0688005f90 comp rx=0 tx=0).stop 2024-09-10T21:44:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.090+0000 7f069e600640 1 -- 172.21.15.7:0/3843979746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06980c5820 msgr2=0x7f06980c5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.090+0000 7f069e600640 1 --2- 172.21.15.7:0/3843979746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06980c5820 0x7f06980c5ca0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f069000c700 tx=0x7f0690003bc0 comp rx=0 tx=0).stop 2024-09-10T21:44:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.090+0000 7f069e600640 1 -- 172.21.15.7:0/3843979746 shutdown_connections 2024-09-10T21:44:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.090+0000 7f069e600640 1 --2- 172.21.15.7:0/3843979746 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0674077fa0 0x7f067407a460 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.090+0000 7f069e600640 1 --2- 172.21.15.7:0/3843979746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06980ce400 0x7f06980c7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.090+0000 7f069e600640 1 --2- 172.21.15.7:0/3843979746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06980c5820 0x7f06980c5ca0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.090+0000 7f069e600640 1 --2- 172.21.15.7:0/3843979746 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f067c004d40 0x7f069814e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.090+0000 7f069e600640 1 -- 172.21.15.7:0/3843979746 >> 172.21.15.7:0/3843979746 conn(0x7f06980b16d0 msgr2=0x7f069806f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.090+0000 7f069e600640 1 -- 172.21.15.7:0/3843979746 shutdown_connections 2024-09-10T21:44:25.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.090+0000 7f069e600640 1 -- 172.21.15.7:0/3843979746 wait complete. 2024-09-10T21:44:25.107 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:44:25.107 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:44:25.208 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:44:25.300 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:44:25.300 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:44:25.300 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:44:25.336 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:44:25.336 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:44:25.336 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:44:25.373 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:44:25.373 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:44:25.374 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:44:25.411 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:44:25.411 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:44:25.411 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:44:25.449 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:44:25.449 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:44:25.450 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:44:25.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.609+0000 7f1f35800640 1 -- 172.21.15.7:0/4135229327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f3015bc00 msgr2=0x7f1f300ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:25.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.609+0000 7f1f35800640 1 --2- 172.21.15.7:0/4135229327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f3015bc00 0x7f1f300ccf60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009830 tx=0x7f1f2002a480 comp rx=0 tx=0).stop 2024-09-10T21:44:25.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.610+0000 7f1f35800640 1 -- 172.21.15.7:0/4135229327 shutdown_connections 2024-09-10T21:44:25.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.610+0000 7f1f35800640 1 --2- 172.21.15.7:0/4135229327 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f300cd750 0x7f1f300b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:25.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.610+0000 7f1f35800640 1 --2- 172.21.15.7:0/4135229327 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f3015bc00 0x7f1f300ccf60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:25.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.610+0000 7f1f35800640 1 -- 172.21.15.7:0/4135229327 >> 172.21.15.7:0/4135229327 conn(0x7f1f300b16d0 msgr2=0x7f1f300b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:25.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.610+0000 7f1f35800640 1 -- 172.21.15.7:0/4135229327 shutdown_connections 2024-09-10T21:44:25.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.610+0000 7f1f35800640 1 -- 172.21.15.7:0/4135229327 wait complete. 2024-09-10T21:44:25.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.611+0000 7f1f35800640 1 Processor -- start 2024-09-10T21:44:25.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.611+0000 7f1f35800640 1 -- start start 2024-09-10T21:44:25.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.612+0000 7f1f35800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f3006fcf0 0x7f1f301e8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:25.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.612+0000 7f1f35800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f300cd750 0x7f1f300c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:25.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.612+0000 7f1f35800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f3015bc00 0x7f1f30152970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:25.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.612+0000 7f1f35800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1f30073e70 con 0x7f1f3015bc00 2024-09-10T21:44:25.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.612+0000 7f1f35800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1f30073cf0 con 0x7f1f3006fcf0 2024-09-10T21:44:25.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.612+0000 7f1f35800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f1f30073ff0 con 0x7f1f300cd750 2024-09-10T21:44:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.613+0000 7f1f2f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f3015bc00 0x7f1f30152970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.613+0000 7f1f26000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f300cd750 0x7f1f300c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.613+0000 7f1f2f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f3015bc00 0x7f1f30152970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36820/0 (socket says 172.21.15.7:36820) 2024-09-10T21:44:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.613+0000 7f1f2f400640 1 -- 172.21.15.7:0/1431355558 learned_addr learned my addr 172.21.15.7:0/1431355558 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.613+0000 7f1f2ea00640 1 --2- 172.21.15.7:0/1431355558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f3006fcf0 0x7f1f301e8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.613+0000 7f1f2f400640 1 -- 172.21.15.7:0/1431355558 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f300cd750 msgr2=0x7f1f300c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.613+0000 7f1f2f400640 1 --2- 172.21.15.7:0/1431355558 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f300cd750 0x7f1f300c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.613+0000 7f1f2f400640 1 -- 172.21.15.7:0/1431355558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f3006fcf0 msgr2=0x7f1f301e8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.613+0000 7f1f2f400640 1 --2- 172.21.15.7:0/1431355558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f3006fcf0 0x7f1f301e8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.613+0000 7f1f2f400640 1 -- 172.21.15.7:0/1431355558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f300c7ca0 con 0x7f1f3015bc00 2024-09-10T21:44:25.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.613+0000 7f1f2f400640 1 --2- 172.21.15.7:0/1431355558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f3015bc00 0x7f1f30152970 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800cb40 tx=0x7f1f2800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.614+0000 7f1f27400640 1 -- 172.21.15.7:0/1431355558 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f28016020 con 0x7f1f3015bc00 2024-09-10T21:44:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.614+0000 7f1f27400640 1 -- 172.21.15.7:0/1431355558 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f2800ad50 con 0x7f1f3015bc00 2024-09-10T21:44:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.614+0000 7f1f27400640 1 -- 172.21.15.7:0/1431355558 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f28012680 con 0x7f1f3015bc00 2024-09-10T21:44:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.614+0000 7f1f35800640 1 -- 172.21.15.7:0/1431355558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f300c7f80 con 0x7f1f3015bc00 2024-09-10T21:44:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.614+0000 7f1f35800640 1 -- 172.21.15.7:0/1431355558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f300c8540 con 0x7f1f3015bc00 2024-09-10T21:44:25.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.615+0000 7f1f35800640 1 -- 172.21.15.7:0/1431355558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f30074480 con 0x7f1f3015bc00 2024-09-10T21:44:25.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.616+0000 7f1f27400640 1 -- 172.21.15.7:0/1431355558 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1f28012820 con 0x7f1f3015bc00 2024-09-10T21:44:25.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.617+0000 7f1f27400640 1 --2- 172.21.15.7:0/1431355558 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1f0c077f70 0x7f1f0c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:25.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.617+0000 7f1f27400640 1 -- 172.21.15.7:0/1431355558 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f1f2809a2d0 con 0x7f1f3015bc00 2024-09-10T21:44:25.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.618+0000 7f1f2ea00640 1 --2- 172.21.15.7:0/1431355558 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1f0c077f70 0x7f1f0c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:25.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.618+0000 7f1f2ea00640 1 --2- 172.21.15.7:0/1431355558 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1f0c077f70 0x7f1f0c07a430 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1f20034000 tx=0x7f1f20004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:25.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.621+0000 7f1f27400640 1 -- 172.21.15.7:0/1431355558 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1f28020020 con 0x7f1f3015bc00 2024-09-10T21:44:25.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.854+0000 7f1f35800640 1 -- 172.21.15.7:0/1431355558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f300cd280 con 0x7f1f3015bc00 2024-09-10T21:44:25.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.855+0000 7f1f27400640 1 -- 172.21.15.7:0/1431355558 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+3908 (secure 0 0 0) 0x7f1f28062a70 con 0x7f1f3015bc00 2024-09-10T21:44:25.856 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:25.856 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":28,"btime":"2024-09-10T21:44:19:271640+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5149,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/3851521050","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3851521050},{"type":"v1","addr":"172.21.15.133:6836","nonce":3851521050}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5289,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2709023497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2709023497},{"type":"v1","addr":"172.21.15.7:6834","nonce":2709023497}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20},{"gid":5298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/4095876032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":4095876032},{"type":"v1","addr":"172.21.15.7:6835","nonce":4095876032}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5407,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/824324140","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":824324140},{"type":"v1","addr":"172.21.15.133:6837","nonce":824324140}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28}],"filesystems":[]} 2024-09-10T21:44:25.856 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 28 2024-09-10T21:44:25.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.857+0000 7f1f35800640 1 -- 172.21.15.7:0/1431355558 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1f0c077f70 msgr2=0x7f1f0c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:25.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.857+0000 7f1f35800640 1 --2- 172.21.15.7:0/1431355558 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1f0c077f70 0x7f1f0c07a430 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1f20034000 tx=0x7f1f20004730 comp rx=0 tx=0).stop 2024-09-10T21:44:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.857+0000 7f1f35800640 1 -- 172.21.15.7:0/1431355558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f3015bc00 msgr2=0x7f1f30152970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.857+0000 7f1f35800640 1 --2- 172.21.15.7:0/1431355558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f3015bc00 0x7f1f30152970 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f1f2800cb40 tx=0x7f1f2800a060 comp rx=0 tx=0).stop 2024-09-10T21:44:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.857+0000 7f1f35800640 1 -- 172.21.15.7:0/1431355558 shutdown_connections 2024-09-10T21:44:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.857+0000 7f1f35800640 1 --2- 172.21.15.7:0/1431355558 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1f0c077f70 0x7f1f0c07a430 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.857+0000 7f1f35800640 1 --2- 172.21.15.7:0/1431355558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1f3015bc00 0x7f1f30152970 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.857+0000 7f1f35800640 1 --2- 172.21.15.7:0/1431355558 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1f300cd750 0x7f1f300c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.857+0000 7f1f35800640 1 --2- 172.21.15.7:0/1431355558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1f3006fcf0 0x7f1f301e8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.857+0000 7f1f35800640 1 -- 172.21.15.7:0/1431355558 >> 172.21.15.7:0/1431355558 conn(0x7f1f300b16d0 msgr2=0x7f1f300b0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.857+0000 7f1f35800640 1 -- 172.21.15.7:0/1431355558 shutdown_connections 2024-09-10T21:44:25.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:25.858+0000 7f1f35800640 1 -- 172.21.15.7:0/1431355558 wait complete. 2024-09-10T21:44:25.875 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:44:25.875 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:44:25.875 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:44:25.875 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:44:25.875 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:44:25.875 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:44:25.876 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:44:25.876 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:44:25.876 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:44:25.876 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:44:25.876 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:44:25.876 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:44:25.876 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:44:25.876 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:44:25.877 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:44:25.877 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:44:25.892 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:44:25.892 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:44:25.893 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:44:25.893 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:44:25.894 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:44:25.894 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:44:25.896 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:44:25.896 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:44:25.937 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:44:25.937 DEBUG:teuthology.parallel:result is None 2024-09-10T21:44:25.940 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:44:25.940 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:44:25.941 DEBUG:teuthology.parallel:result is None 2024-09-10T21:44:25.941 DEBUG:teuthology.parallel:result is None 2024-09-10T21:44:25.943 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:44:25.943 DEBUG:teuthology.parallel:result is None 2024-09-10T21:44:25.943 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:44:26.068 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2024-09-10T21:44:26.068 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2024-09-10T21:44:26.072 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-09-10T21:44:26.072 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-09-10T21:44:26.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.111+0000 7f2e35800640 1 -- 172.21.15.7:0/1191937599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e300ccc30 msgr2=0x7f2e301639b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:26.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.111+0000 7f2e35800640 1 --2- 172.21.15.7:0/1191937599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e300ccc30 0x7f2e301639b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f2e300cc4f0 tx=0x7f2e2802aac0 comp rx=0 tx=0).stop 2024-09-10T21:44:26.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.111+0000 7f2e35800640 1 -- 172.21.15.7:0/1191937599 shutdown_connections 2024-09-10T21:44:26.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.111+0000 7f2e35800640 1 --2- 172.21.15.7:0/1191937599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e300ccc30 0x7f2e301639b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:26.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.111+0000 7f2e35800640 1 --2- 172.21.15.7:0/1191937599 >> v2:172.21.15.133:3300/0 conn(0x7f2e30070600 0x7f2e301579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:26.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.111+0000 7f2e35800640 1 -- 172.21.15.7:0/1191937599 >> 172.21.15.7:0/1191937599 conn(0x7f2e300b1660 msgr2=0x7f2e300b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:26.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.111+0000 7f2e35800640 1 -- 172.21.15.7:0/1191937599 shutdown_connections 2024-09-10T21:44:26.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.111+0000 7f2e35800640 1 -- 172.21.15.7:0/1191937599 wait complete. 2024-09-10T21:44:26.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.113+0000 7f2e35800640 1 Processor -- start 2024-09-10T21:44:26.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.113+0000 7f2e35800640 1 -- start start 2024-09-10T21:44:26.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.114+0000 7f2e35800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e30070600 0x7f2e30072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:26.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.114+0000 7f2e35800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2e300ccc30 0x7f2e30150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:26.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.114+0000 7f2e35800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e300cf650 0x7f2e300735f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:26.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.114+0000 7f2e35800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2e300af090 con 0x7f2e300cf650 2024-09-10T21:44:26.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.114+0000 7f2e35800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2e300aef10 con 0x7f2e30070600 2024-09-10T21:44:26.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.114+0000 7f2e35800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f2e300af210 con 0x7f2e300ccc30 2024-09-10T21:44:26.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.114+0000 7f2e2f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e300cf650 0x7f2e300735f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:26.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.114+0000 7f2e26000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2e300ccc30 0x7f2e30150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:26.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.114+0000 7f2e2ea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e30070600 0x7f2e30072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:26.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.114+0000 7f2e2f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e300cf650 0x7f2e300735f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36884/0 (socket says 172.21.15.7:36884) 2024-09-10T21:44:26.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.114+0000 7f2e2f400640 1 -- 172.21.15.7:0/2088891916 learned_addr learned my addr 172.21.15.7:0/2088891916 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:26.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.114+0000 7f2e2f400640 1 -- 172.21.15.7:0/2088891916 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2e300ccc30 msgr2=0x7f2e30150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:26.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.114+0000 7f2e2f400640 1 --2- 172.21.15.7:0/2088891916 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2e300ccc30 0x7f2e30150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:26.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.114+0000 7f2e2f400640 1 -- 172.21.15.7:0/2088891916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e30070600 msgr2=0x7f2e30072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:26.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.114+0000 7f2e2f400640 1 --2- 172.21.15.7:0/2088891916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e30070600 0x7f2e30072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:26.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.114+0000 7f2e2f400640 1 -- 172.21.15.7:0/2088891916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e30073fd0 con 0x7f2e300cf650 2024-09-10T21:44:26.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.114+0000 7f2e2ea00640 1 --2- 172.21.15.7:0/2088891916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e30070600 0x7f2e30072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:44:26.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.115+0000 7f2e2f400640 1 --2- 172.21.15.7:0/2088891916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e300cf650 0x7f2e300735f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2e28009d60 tx=0x7f2e28003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:26.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.115+0000 7f2e27400640 1 -- 172.21.15.7:0/2088891916 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28044070 con 0x7f2e300cf650 2024-09-10T21:44:26.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.115+0000 7f2e27400640 1 -- 172.21.15.7:0/2088891916 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e28004040 con 0x7f2e300cf650 2024-09-10T21:44:26.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.115+0000 7f2e27400640 1 -- 172.21.15.7:0/2088891916 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e28004490 con 0x7f2e300cf650 2024-09-10T21:44:26.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.115+0000 7f2e35800640 1 -- 172.21.15.7:0/2088891916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e300742b0 con 0x7f2e300cf650 2024-09-10T21:44:26.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.115+0000 7f2e35800640 1 -- 172.21.15.7:0/2088891916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e30072640 con 0x7f2e300cf650 2024-09-10T21:44:26.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.116+0000 7f2e35800640 1 -- 172.21.15.7:0/2088891916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e301579b0 con 0x7f2e300cf650 2024-09-10T21:44:26.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.117+0000 7f2e27400640 1 -- 172.21.15.7:0/2088891916 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2e28004630 con 0x7f2e300cf650 2024-09-10T21:44:26.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.118+0000 7f2e27400640 1 --2- 172.21.15.7:0/2088891916 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2e0c077fa0 0x7f2e0c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:26.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.118+0000 7f2e27400640 1 -- 172.21.15.7:0/2088891916 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 4943+0+0 (secure 0 0 0) 0x7f2e280b9f40 con 0x7f2e300cf650 2024-09-10T21:44:26.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.119+0000 7f2e2ea00640 1 --2- 172.21.15.7:0/2088891916 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2e0c077fa0 0x7f2e0c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:26.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.119+0000 7f2e2ea00640 1 --2- 172.21.15.7:0/2088891916 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2e0c077fa0 0x7f2e0c07a460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2e20009fc0 tx=0x7f2e20005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:26.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.123+0000 7f2e27400640 1 -- 172.21.15.7:0/2088891916 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2e28032030 con 0x7f2e300cf650 2024-09-10T21:44:26.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:26.334+0000 7f2e35800640 1 -- 172.21.15.7:0/2088891916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f2e300b1d80 con 0x7f2e300cf650 2024-09-10T21:44:27.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.337+0000 7f2e27400640 1 -- 172.21.15.7:0/2088891916 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) ==== 97+0+0 (secure 0 0 0) 0x7f2e28082650 con 0x7f2e300cf650 2024-09-10T21:44:27.338 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-09-10T21:44:27.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.340+0000 7f2e35800640 1 -- 172.21.15.7:0/2088891916 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2e0c077fa0 msgr2=0x7f2e0c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:27.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.340+0000 7f2e35800640 1 --2- 172.21.15.7:0/2088891916 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2e0c077fa0 0x7f2e0c07a460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f2e20009fc0 tx=0x7f2e20005f90 comp rx=0 tx=0).stop 2024-09-10T21:44:27.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.340+0000 7f2e35800640 1 -- 172.21.15.7:0/2088891916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e300cf650 msgr2=0x7f2e300735f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:27.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.340+0000 7f2e35800640 1 --2- 172.21.15.7:0/2088891916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e300cf650 0x7f2e300735f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f2e28009d60 tx=0x7f2e28003ba0 comp rx=0 tx=0).stop 2024-09-10T21:44:27.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.340+0000 7f2e35800640 1 -- 172.21.15.7:0/2088891916 shutdown_connections 2024-09-10T21:44:27.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.340+0000 7f2e35800640 1 --2- 172.21.15.7:0/2088891916 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2e0c077fa0 0x7f2e0c07a460 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:27.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.340+0000 7f2e35800640 1 --2- 172.21.15.7:0/2088891916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2e300cf650 0x7f2e300735f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:27.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.340+0000 7f2e35800640 1 --2- 172.21.15.7:0/2088891916 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2e300ccc30 0x7f2e30150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:27.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.340+0000 7f2e35800640 1 --2- 172.21.15.7:0/2088891916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2e30070600 0x7f2e30072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:27.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.340+0000 7f2e35800640 1 -- 172.21.15.7:0/2088891916 >> 172.21.15.7:0/2088891916 conn(0x7f2e300b1660 msgr2=0x7f2e300cfaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:27.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.340+0000 7f2e35800640 1 -- 172.21.15.7:0/2088891916 shutdown_connections 2024-09-10T21:44:27.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.340+0000 7f2e35800640 1 -- 172.21.15.7:0/2088891916 wait complete. 2024-09-10T21:44:27.360 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:44:27.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.525+0000 7fc8ea800640 1 -- 172.21.15.7:0/627313701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8e41604b0 msgr2=0x7fc8e40ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:27.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.525+0000 7fc8ea800640 1 --2- 172.21.15.7:0/627313701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8e41604b0 0x7fc8e40ccef0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc005fd0 tx=0x7fc8dc02a8b0 comp rx=0 tx=0).stop 2024-09-10T21:44:27.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.525+0000 7fc8ea800640 1 -- 172.21.15.7:0/627313701 shutdown_connections 2024-09-10T21:44:27.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.525+0000 7fc8ea800640 1 --2- 172.21.15.7:0/627313701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8e41604b0 0x7fc8e40ccef0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:27.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.525+0000 7fc8ea800640 1 -- 172.21.15.7:0/627313701 >> 172.21.15.7:0/627313701 conn(0x7fc8e40b16d0 msgr2=0x7fc8e40b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.525+0000 7fc8ea800640 1 -- 172.21.15.7:0/627313701 shutdown_connections 2024-09-10T21:44:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.525+0000 7fc8ea800640 1 -- 172.21.15.7:0/627313701 wait complete. 2024-09-10T21:44:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.526+0000 7fc8ea800640 1 Processor -- start 2024-09-10T21:44:27.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.526+0000 7fc8ea800640 1 -- start start 2024-09-10T21:44:27.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.527+0000 7fc8ea800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8e40cd750 0x7fc8e4146440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:27.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.527+0000 7fc8ea800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8e415bc90 0x7fc8e4157360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:27.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.527+0000 7fc8ea800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8e41604b0 0x7fc8e415a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:27.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.527+0000 7fc8ea800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e4070080 con 0x7fc8e40cd750 2024-09-10T21:44:27.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.527+0000 7fc8ea800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e406ff00 con 0x7fc8e415bc90 2024-09-10T21:44:27.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.527+0000 7fc8ea800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e4070200 con 0x7fc8e41604b0 2024-09-10T21:44:27.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.528+0000 7fc8e3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8e40cd750 0x7fc8e4146440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.528+0000 7fc8e3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8e40cd750 0x7fc8e4146440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36906/0 (socket says 172.21.15.7:36906) 2024-09-10T21:44:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.528+0000 7fc8e3400640 1 -- 172.21.15.7:0/1251959844 learned_addr learned my addr 172.21.15.7:0/1251959844 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.528+0000 7fc8dbe00640 1 --2- 172.21.15.7:0/1251959844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8e415bc90 0x7fc8e4157360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.528+0000 7fc8e3e00640 1 --2- 172.21.15.7:0/1251959844 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8e41604b0 0x7fc8e415a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.528+0000 7fc8e3400640 1 -- 172.21.15.7:0/1251959844 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8e41604b0 msgr2=0x7fc8e415a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.528+0000 7fc8e3400640 1 --2- 172.21.15.7:0/1251959844 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8e41604b0 0x7fc8e415a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.528+0000 7fc8e3400640 1 -- 172.21.15.7:0/1251959844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8e415bc90 msgr2=0x7fc8e4157360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.528+0000 7fc8e3400640 1 --2- 172.21.15.7:0/1251959844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8e415bc90 0x7fc8e4157360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.528+0000 7fc8e3400640 1 -- 172.21.15.7:0/1251959844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e41578a0 con 0x7fc8e40cd750 2024-09-10T21:44:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.528+0000 7fc8e3400640 1 --2- 172.21.15.7:0/1251959844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8e40cd750 0x7fc8e4146440 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400c450 tx=0x7fc8d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:27.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.529+0000 7fc8db400640 1 -- 172.21.15.7:0/1251959844 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40052d0 con 0x7fc8e40cd750 2024-09-10T21:44:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.529+0000 7fc8db400640 1 -- 172.21.15.7:0/1251959844 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8d4005470 con 0x7fc8e40cd750 2024-09-10T21:44:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.529+0000 7fc8db400640 1 -- 172.21.15.7:0/1251959844 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4012870 con 0x7fc8e40cd750 2024-09-10T21:44:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.529+0000 7fc8ea800640 1 -- 172.21.15.7:0/1251959844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e40c97c0 con 0x7fc8e40cd750 2024-09-10T21:44:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.529+0000 7fc8ea800640 1 -- 172.21.15.7:0/1251959844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e40c9d70 con 0x7fc8e40cd750 2024-09-10T21:44:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.530+0000 7fc8ea800640 1 -- 172.21.15.7:0/1251959844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e40706b0 con 0x7fc8e40cd750 2024-09-10T21:44:27.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.531+0000 7fc8db400640 1 -- 172.21.15.7:0/1251959844 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc8d40035f0 con 0x7fc8e40cd750 2024-09-10T21:44:27.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.532+0000 7fc8db400640 1 --2- 172.21.15.7:0/1251959844 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc8c0077f70 0x7fc8c007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:27.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.532+0000 7fc8db400640 1 -- 172.21.15.7:0/1251959844 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7fc8d4099510 con 0x7fc8e40cd750 2024-09-10T21:44:27.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.532+0000 7fc8dbe00640 1 --2- 172.21.15.7:0/1251959844 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc8c0077f70 0x7fc8c007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:27.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.533+0000 7fc8dbe00640 1 --2- 172.21.15.7:0/1251959844 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc8c0077f70 0x7fc8c007a430 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc8e406adf0 tx=0x7fc8cc007e90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:27.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.536+0000 7fc8db400640 1 -- 172.21.15.7:0/1251959844 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc8d4061c70 con 0x7fc8e40cd750 2024-09-10T21:44:27.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.851+0000 7fc8ea800640 1 -- 172.21.15.7:0/1251959844 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fc8e40b4590 con 0x7fc8e40cd750 2024-09-10T21:44:27.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.851+0000 7fc8db400640 1 -- 172.21.15.7:0/1251959844 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v33) ==== 80+0+5304 (secure 0 0 0) 0x7fc8d40666a0 con 0x7fc8e40cd750 2024-09-10T21:44:27.853 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:27.853 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:44:27.853 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-09-10T21:44:27.853 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:27.853 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-09-10T21:44:27.853 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmgRMuLBAA88QT12AXv2qf11IxU20Xbg==", 2024-09-10T21:44:27.853 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:44:27.853 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:44:27.853 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:44:27.854 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:44:27.854 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:44:27.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:27.854 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:27.854 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-09-10T21:44:27.854 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmRUM2BBAAwcQM6z+hjZI4ZWBlVrYMwQ==", 2024-09-10T21:44:27.854 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:44:27.854 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:44:27.854 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:44:27.854 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:44:27.854 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:44:27.854 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.854 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:27.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:27.855 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-09-10T21:44:27.855 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmnq31NRAAj1eJUAIS986idxppfeXEAA==", 2024-09-10T21:44:27.855 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:44:27.855 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:44:27.855 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:44:27.855 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:44:27.855 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:44:27.855 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.855 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:27.855 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:27.855 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-09-10T21:44:27.855 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmy43uDRAAzdKjLxZMGROuDABrsJ0zgQ==", 2024-09-10T21:44:27.856 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:44:27.856 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:44:27.856 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:44:27.856 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:44:27.856 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:44:27.856 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.856 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:27.856 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:27.856 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-09-10T21:44:27.856 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDwvOBmzfgtOxAAk6K5fU0Uar4VW9nUUIuNdw==", 2024-09-10T21:44:27.856 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:44:27.857 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:44:27.857 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:44:27.857 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:44:27.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.857 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:27.857 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:27.857 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-09-10T21:44:27.857 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD0vOBmHcRzChAA3z+CaGgu5NkyUmjpqzmKjw==", 2024-09-10T21:44:27.857 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:44:27.857 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:44:27.857 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:44:27.857 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:44:27.857 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:27.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:27.858 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-09-10T21:44:27.858 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD3vOBm9sArFBAAGQlyiw6qEcyumEBICWEBKA==", 2024-09-10T21:44:27.858 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:44:27.858 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:44:27.858 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:44:27.858 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:44:27.858 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.858 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:27.858 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:27.858 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-09-10T21:44:27.859 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD6vOBmkO3rHxAAW0zPydYZRGUKjGh3yIyPCA==", 2024-09-10T21:44:27.859 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:44:27.859 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:44:27.859 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:44:27.859 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:44:27.859 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.859 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:27.859 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:27.859 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-09-10T21:44:27.859 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAAveBmlPABMxAAeazVOzc1DhsVAwoHn87QIA==", 2024-09-10T21:44:27.859 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:44:27.859 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:44:27.859 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:44:27.860 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:44:27.860 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:27.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:27.860 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-09-10T21:44:27.860 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAEveBmL9qzARAApu8nWIY9HVsDl4yUnfhH9w==", 2024-09-10T21:44:27.860 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:44:27.860 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:44:27.860 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:44:27.860 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:44:27.860 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.860 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:27.860 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:27.861 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-09-10T21:44:27.861 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAHveBmJ4ofDRAAEiCFN8nGzVSR3u2gKoT+2Q==", 2024-09-10T21:44:27.861 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:44:27.861 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:44:27.861 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:44:27.862 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:44:27.862 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.862 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:27.862 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:27.862 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-09-10T21:44:27.862 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAKveBmoUGEGBAAhXmkUaniuAQrBiYPWr8Tog==", 2024-09-10T21:44:27.862 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:44:27.862 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:44:27.862 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:44:27.862 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:44:27.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:27.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:27.863 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-09-10T21:44:27.863 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBm0aqcFhAAtid2bnzjcpaYAbU7g0JcOA==", 2024-09-10T21:44:27.863 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:44:27.863 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:44:27.863 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-09-10T21:44:27.863 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:44:27.863 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.863 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:27.863 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:27.863 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-09-10T21:44:27.863 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmqf+MIxAAib5hVnbBsw/bJPqsQszJwg==", 2024-09-10T21:44:27.864 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:44:27.864 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:44:27.864 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:44:27.864 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:44:27.864 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:44:27.864 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.864 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:27.864 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:27.864 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-09-10T21:44:27.864 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmybQhHRAAZUwqnIsihEWnHZWQTYr+uQ==", 2024-09-10T21:44:27.864 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:44:27.864 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:44:27.864 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:44:27.864 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:44:27.865 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:44:27.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.865 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:27.865 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:27.865 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-09-10T21:44:27.865 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmDbrrKRAAUxb3EBfKIJ4FAGHW3QIsLg==", 2024-09-10T21:44:27.865 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:44:27.865 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:44:27.865 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:44:27.865 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:44:27.865 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:44:27.865 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:27.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:27.866 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-09-10T21:44:27.866 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDtvOBmF+eEOBAA3icvNqI0nWDShE49DZ3bvg==", 2024-09-10T21:44:27.866 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:44:27.866 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:44:27.866 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:44:27.866 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:44:27.866 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:44:27.866 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.866 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:27.866 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:27.867 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-09-10T21:44:27.867 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmsu6OJRAAhdgDvpeHY/zDKUeUOv6xxA==", 2024-09-10T21:44:27.867 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:44:27.867 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:44:27.867 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-10T21:44:27.867 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:44:27.867 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.867 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:27.867 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:27.867 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-09-10T21:44:27.868 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmKBHLHhAA0W5My19tIhN1v2qev4cBTA==", 2024-09-10T21:44:27.868 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:44:27.868 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:44:27.868 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-10T21:44:27.868 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:44:27.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.868 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:27.868 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T21:44:27.868 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:44:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.854+0000 7fc8ea800640 1 -- 172.21.15.7:0/1251959844 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc8c0077f70 msgr2=0x7fc8c007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.854+0000 7fc8ea800640 1 --2- 172.21.15.7:0/1251959844 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc8c0077f70 0x7fc8c007a430 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc8e406adf0 tx=0x7fc8cc007e90 comp rx=0 tx=0).stop 2024-09-10T21:44:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.854+0000 7fc8ea800640 1 -- 172.21.15.7:0/1251959844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8e40cd750 msgr2=0x7fc8e4146440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:27.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.854+0000 7fc8ea800640 1 --2- 172.21.15.7:0/1251959844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8e40cd750 0x7fc8e4146440 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fc8d400c450 tx=0x7fc8d400c910 comp rx=0 tx=0).stop 2024-09-10T21:44:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.854+0000 7fc8ea800640 1 -- 172.21.15.7:0/1251959844 shutdown_connections 2024-09-10T21:44:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.854+0000 7fc8ea800640 1 --2- 172.21.15.7:0/1251959844 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc8c0077f70 0x7fc8c007a430 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.854+0000 7fc8ea800640 1 --2- 172.21.15.7:0/1251959844 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8e415bc90 0x7fc8e4157360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.854+0000 7fc8ea800640 1 --2- 172.21.15.7:0/1251959844 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8e41604b0 0x7fc8e415a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.854+0000 7fc8ea800640 1 --2- 172.21.15.7:0/1251959844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8e40cd750 0x7fc8e4146440 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.854+0000 7fc8ea800640 1 -- 172.21.15.7:0/1251959844 >> 172.21.15.7:0/1251959844 conn(0x7fc8e40b16d0 msgr2=0x7fc8e40b4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.854+0000 7fc8ea800640 1 -- 172.21.15.7:0/1251959844 shutdown_connections 2024-09-10T21:44:27.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:27.854+0000 7fc8ea800640 1 -- 172.21.15.7:0/1251959844 wait complete. 2024-09-10T21:44:27.871 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:44:28.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.030+0000 7f4cb1e00640 1 -- 172.21.15.7:0/3886315351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0cbd50 msgr2=0x7f4cac0cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:28.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.030+0000 7f4cb1e00640 1 --2- 172.21.15.7:0/3886315351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0cbd50 0x7f4cac0cc1c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009830 tx=0x7f4c9402a510 comp rx=0 tx=0).stop 2024-09-10T21:44:28.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.030+0000 7f4cb1e00640 1 -- 172.21.15.7:0/3886315351 shutdown_connections 2024-09-10T21:44:28.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.031+0000 7f4cb1e00640 1 --2- 172.21.15.7:0/3886315351 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0cbd50 0x7f4cac0cc1c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:28.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.031+0000 7f4cb1e00640 1 -- 172.21.15.7:0/3886315351 >> 172.21.15.7:0/3886315351 conn(0x7f4cac0b16d0 msgr2=0x7f4cac0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:28.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.031+0000 7f4cb1e00640 1 -- 172.21.15.7:0/3886315351 shutdown_connections 2024-09-10T21:44:28.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.031+0000 7f4cb1e00640 1 -- 172.21.15.7:0/3886315351 wait complete. 2024-09-10T21:44:28.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.032+0000 7f4cb1e00640 1 Processor -- start 2024-09-10T21:44:28.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.032+0000 7f4cb1e00640 1 -- start start 2024-09-10T21:44:28.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.033+0000 7f4cb1e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cac0ae020 0x7f4cac1f76d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:28.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.033+0000 7f4cb1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0cbd50 0x7f4cac0b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:28.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.033+0000 7f4cb1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cac1619e0 0x7f4cac073280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:28.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.033+0000 7f4cb1e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac161f40 con 0x7f4cac0cbd50 2024-09-10T21:44:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.033+0000 7f4cb1e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac161dc0 con 0x7f4cac1619e0 2024-09-10T21:44:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.033+0000 7f4cb1e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4cac1620c0 con 0x7f4cac0ae020 2024-09-10T21:44:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.034+0000 7f4ca2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0cbd50 0x7f4cac0b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.034+0000 7f4ca2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0cbd50 0x7f4cac0b3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36930/0 (socket says 172.21.15.7:36930) 2024-09-10T21:44:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.034+0000 7f4ca2000640 1 -- 172.21.15.7:0/2932140119 learned_addr learned my addr 172.21.15.7:0/2932140119 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.034+0000 7f4cab400640 1 --2- 172.21.15.7:0/2932140119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cac1619e0 0x7f4cac073280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.034+0000 7f4caaa00640 1 --2- 172.21.15.7:0/2932140119 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cac0ae020 0x7f4cac1f76d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.034+0000 7f4ca2000640 1 -- 172.21.15.7:0/2932140119 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cac0ae020 msgr2=0x7f4cac1f76d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.034+0000 7f4ca2000640 1 --2- 172.21.15.7:0/2932140119 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cac0ae020 0x7f4cac1f76d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.034+0000 7f4ca2000640 1 -- 172.21.15.7:0/2932140119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cac1619e0 msgr2=0x7f4cac073280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.034+0000 7f4ca2000640 1 --2- 172.21.15.7:0/2932140119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cac1619e0 0x7f4cac073280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.034+0000 7f4ca2000640 1 -- 172.21.15.7:0/2932140119 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cac0b4a20 con 0x7f4cac0cbd50 2024-09-10T21:44:28.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.034+0000 7f4ca2000640 1 --2- 172.21.15.7:0/2932140119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0cbd50 0x7f4cac0b3f60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009950 tx=0x7f4c94032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:28.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.035+0000 7f4ca3e00640 1 -- 172.21.15.7:0/2932140119 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c94040070 con 0x7f4cac0cbd50 2024-09-10T21:44:28.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.035+0000 7f4ca3e00640 1 -- 172.21.15.7:0/2932140119 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4c94003af0 con 0x7f4cac0cbd50 2024-09-10T21:44:28.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.035+0000 7f4cb1e00640 1 -- 172.21.15.7:0/2932140119 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cac0b4d00 con 0x7f4cac0cbd50 2024-09-10T21:44:28.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.035+0000 7f4ca3e00640 1 -- 172.21.15.7:0/2932140119 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c94045460 con 0x7f4cac0cbd50 2024-09-10T21:44:28.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.035+0000 7f4cb1e00640 1 -- 172.21.15.7:0/2932140119 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cac0b52c0 con 0x7f4cac0cbd50 2024-09-10T21:44:28.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.036+0000 7f4cb1e00640 1 -- 172.21.15.7:0/2932140119 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cac1625a0 con 0x7f4cac0cbd50 2024-09-10T21:44:28.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.037+0000 7f4ca3e00640 1 -- 172.21.15.7:0/2932140119 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4c9404d030 con 0x7f4cac0cbd50 2024-09-10T21:44:28.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.038+0000 7f4ca3e00640 1 --2- 172.21.15.7:0/2932140119 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c88077f70 0x7f4c8807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:28.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.038+0000 7f4ca3e00640 1 -- 172.21.15.7:0/2932140119 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f4c940bb040 con 0x7f4cac0cbd50 2024-09-10T21:44:28.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.039+0000 7f4caaa00640 1 --2- 172.21.15.7:0/2932140119 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c88077f70 0x7f4c8807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:28.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.039+0000 7f4caaa00640 1 --2- 172.21.15.7:0/2932140119 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c88077f70 0x7f4c8807a430 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c000f60 tx=0x7f4c9c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:28.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.042+0000 7f4ca3e00640 1 -- 172.21.15.7:0/2932140119 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4c94083e80 con 0x7f4cac0cbd50 2024-09-10T21:44:28.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.239+0000 7f4cb1e00640 1 -- 172.21.15.7:0/2932140119 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4cac0cb850 con 0x7f4cac0cbd50 2024-09-10T21:44:28.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.240+0000 7f4ca3e00640 1 -- 172.21.15.7:0/2932140119 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v54) ==== 84+0+65 (secure 0 0 0) 0x7f4c94083780 con 0x7f4cac0cbd50 2024-09-10T21:44:28.242 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:28.242 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-09-10T21:44:28.242 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:28.242 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-09-10T21:44:28.242 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-09-10T21:44:28.242 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:28.243 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-09-10T21:44:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.242+0000 7f4cb1e00640 1 -- 172.21.15.7:0/2932140119 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c88077f70 msgr2=0x7f4c8807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.242+0000 7f4cb1e00640 1 --2- 172.21.15.7:0/2932140119 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c88077f70 0x7f4c8807a430 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c000f60 tx=0x7f4c9c009040 comp rx=0 tx=0).stop 2024-09-10T21:44:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.242+0000 7f4cb1e00640 1 -- 172.21.15.7:0/2932140119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0cbd50 msgr2=0x7f4cac0b3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:28.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.242+0000 7f4cb1e00640 1 --2- 172.21.15.7:0/2932140119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0cbd50 0x7f4cac0b3f60 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f4c94009950 tx=0x7f4c94032d50 comp rx=0 tx=0).stop 2024-09-10T21:44:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.242+0000 7f4cb1e00640 1 -- 172.21.15.7:0/2932140119 shutdown_connections 2024-09-10T21:44:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.242+0000 7f4cb1e00640 1 --2- 172.21.15.7:0/2932140119 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4c88077f70 0x7f4c8807a430 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.242+0000 7f4cb1e00640 1 --2- 172.21.15.7:0/2932140119 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4cac1619e0 0x7f4cac073280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.242+0000 7f4cb1e00640 1 --2- 172.21.15.7:0/2932140119 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4cac0cbd50 0x7f4cac0b3f60 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.242+0000 7f4cb1e00640 1 --2- 172.21.15.7:0/2932140119 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4cac0ae020 0x7f4cac1f76d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.242+0000 7f4cb1e00640 1 -- 172.21.15.7:0/2932140119 >> 172.21.15.7:0/2932140119 conn(0x7f4cac0b16d0 msgr2=0x7f4cac1606c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.242+0000 7f4cb1e00640 1 -- 172.21.15.7:0/2932140119 shutdown_connections 2024-09-10T21:44:28.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.242+0000 7f4cb1e00640 1 -- 172.21.15.7:0/2932140119 wait complete. 2024-09-10T21:44:28.260 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:44:28.260 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:44:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.425+0000 7f2ad6c00640 1 -- 172.21.15.7:0/1198720088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ad006d230 msgr2=0x7f2ad00b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.425+0000 7f2ad6c00640 1 --2- 172.21.15.7:0/1198720088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ad006d230 0x7f2ad00b4a30 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8005fd0 tx=0x7f2ab802a540 comp rx=0 tx=0).stop 2024-09-10T21:44:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.425+0000 7f2ad6c00640 1 -- 172.21.15.7:0/1198720088 shutdown_connections 2024-09-10T21:44:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.425+0000 7f2ad6c00640 1 --2- 172.21.15.7:0/1198720088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ad006d230 0x7f2ad00b4a30 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.425+0000 7f2ad6c00640 1 -- 172.21.15.7:0/1198720088 >> 172.21.15.7:0/1198720088 conn(0x7f2ad00b1c10 msgr2=0x7f2ad00b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.426+0000 7f2ad6c00640 1 -- 172.21.15.7:0/1198720088 shutdown_connections 2024-09-10T21:44:28.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.426+0000 7f2ad6c00640 1 -- 172.21.15.7:0/1198720088 wait complete. 2024-09-10T21:44:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.427+0000 7f2ad6c00640 1 Processor -- start 2024-09-10T21:44:28.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.427+0000 7f2ad6c00640 1 -- start start 2024-09-10T21:44:28.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.428+0000 7f2ad6c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ad006d230 0x7f2ad0147600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.428+0000 7f2ad6c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ad0158300 0x7f2ad0149260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.428+0000 7f2ad6c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ad015edd0 0x7f2ad01f8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.428+0000 7f2ad6c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad00740a0 con 0x7f2ad006d230 2024-09-10T21:44:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.428+0000 7f2ad6c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad0073f20 con 0x7f2ad015edd0 2024-09-10T21:44:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.428+0000 7f2ad6c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f2ad0074220 con 0x7f2ad0158300 2024-09-10T21:44:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.428+0000 7f2acfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ad006d230 0x7f2ad0147600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.429+0000 7f2acfe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ad006d230 0x7f2ad0147600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36948/0 (socket says 172.21.15.7:36948) 2024-09-10T21:44:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.429+0000 7f2acfe00640 1 -- 172.21.15.7:0/1152923791 learned_addr learned my addr 172.21.15.7:0/1152923791 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.429+0000 7f2ad4c00640 1 --2- 172.21.15.7:0/1152923791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ad015edd0 0x7f2ad01f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:28.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.429+0000 7f2ac7400640 1 --2- 172.21.15.7:0/1152923791 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ad0158300 0x7f2ad0149260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.429+0000 7f2acfe00640 1 -- 172.21.15.7:0/1152923791 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ad0158300 msgr2=0x7f2ad0149260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.429+0000 7f2acfe00640 1 --2- 172.21.15.7:0/1152923791 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ad0158300 0x7f2ad0149260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.429+0000 7f2acfe00640 1 -- 172.21.15.7:0/1152923791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ad015edd0 msgr2=0x7f2ad01f8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.429+0000 7f2acfe00640 1 --2- 172.21.15.7:0/1152923791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ad015edd0 0x7f2ad01f8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.429+0000 7f2acfe00640 1 -- 172.21.15.7:0/1152923791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad01f8cc0 con 0x7f2ad006d230 2024-09-10T21:44:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.429+0000 7f2ac7400640 1 --2- 172.21.15.7:0/1152923791 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ad0158300 0x7f2ad0149260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.429+0000 7f2ad4c00640 1 --2- 172.21.15.7:0/1152923791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ad015edd0 0x7f2ad01f8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.429+0000 7f2acfe00640 1 --2- 172.21.15.7:0/1152923791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ad006d230 0x7f2ad0147600 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000c480 tx=0x7f2ac000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.429+0000 7f2accc00640 1 -- 172.21.15.7:0/1152923791 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00052d0 con 0x7f2ad006d230 2024-09-10T21:44:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.430+0000 7f2accc00640 1 -- 172.21.15.7:0/1152923791 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ac0005470 con 0x7f2ad006d230 2024-09-10T21:44:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.430+0000 7f2ad6c00640 1 -- 172.21.15.7:0/1152923791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad01f8fa0 con 0x7f2ad006d230 2024-09-10T21:44:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.430+0000 7f2accc00640 1 -- 172.21.15.7:0/1152923791 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0012870 con 0x7f2ad006d230 2024-09-10T21:44:28.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.430+0000 7f2ad6c00640 1 -- 172.21.15.7:0/1152923791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad00cab90 con 0x7f2ad006d230 2024-09-10T21:44:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.431+0000 7f2ad6c00640 1 -- 172.21.15.7:0/1152923791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ad0072f10 con 0x7f2ad006d230 2024-09-10T21:44:28.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.432+0000 7f2accc00640 1 -- 172.21.15.7:0/1152923791 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2ac00035f0 con 0x7f2ad006d230 2024-09-10T21:44:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.433+0000 7f2accc00640 1 --2- 172.21.15.7:0/1152923791 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2aac077f70 0x7f2aac07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.433+0000 7f2accc00640 1 -- 172.21.15.7:0/1152923791 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 4857+0+0 (secure 0 0 0) 0x7f2ac0099670 con 0x7f2ad006d230 2024-09-10T21:44:28.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.433+0000 7f2ac7400640 1 --2- 172.21.15.7:0/1152923791 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2aac077f70 0x7f2aac07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:28.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.434+0000 7f2ac7400640 1 --2- 172.21.15.7:0/1152923791 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2aac077f70 0x7f2aac07a430 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8033000 tx=0x7f2ab8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:28.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.437+0000 7f2accc00640 1 -- 172.21.15.7:0/1152923791 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2ad0072f10 con 0x7f2ad006d230 2024-09-10T21:44:28.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:28.632+0000 7f2ad6c00640 1 -- 172.21.15.7:0/1152923791 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2ad00b4a30 con 0x7f2ad006d230 2024-09-10T21:44:29.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.349+0000 7f2accc00640 1 -- 172.21.15.7:0/1152923791 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v55) ==== 138+0+0 (secure 0 0 0) 0x7f2ac00624b0 con 0x7f2ad006d230 2024-09-10T21:44:29.350 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-09-10T21:44:29.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.352+0000 7f2ad6c00640 1 -- 172.21.15.7:0/1152923791 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2aac077f70 msgr2=0x7f2aac07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:29.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.352+0000 7f2ad6c00640 1 --2- 172.21.15.7:0/1152923791 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2aac077f70 0x7f2aac07a430 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8033000 tx=0x7f2ab8004730 comp rx=0 tx=0).stop 2024-09-10T21:44:29.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.352+0000 7f2ad6c00640 1 -- 172.21.15.7:0/1152923791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ad006d230 msgr2=0x7f2ad0147600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.352+0000 7f2ad6c00640 1 --2- 172.21.15.7:0/1152923791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ad006d230 0x7f2ad0147600 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000c480 tx=0x7f2ac000c940 comp rx=0 tx=0).stop 2024-09-10T21:44:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.352+0000 7f2ad6c00640 1 -- 172.21.15.7:0/1152923791 shutdown_connections 2024-09-10T21:44:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.352+0000 7f2ad6c00640 1 --2- 172.21.15.7:0/1152923791 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2aac077f70 0x7f2aac07a430 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.352+0000 7f2ad6c00640 1 --2- 172.21.15.7:0/1152923791 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ad015edd0 0x7f2ad01f8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.352+0000 7f2ad6c00640 1 --2- 172.21.15.7:0/1152923791 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ad0158300 0x7f2ad0149260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.352+0000 7f2ad6c00640 1 --2- 172.21.15.7:0/1152923791 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ad006d230 0x7f2ad0147600 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.352+0000 7f2ad6c00640 1 -- 172.21.15.7:0/1152923791 >> 172.21.15.7:0/1152923791 conn(0x7f2ad00b1c10 msgr2=0x7f2ad0072800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.353+0000 7f2ad6c00640 1 -- 172.21.15.7:0/1152923791 shutdown_connections 2024-09-10T21:44:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.353+0000 7f2ad6c00640 1 -- 172.21.15.7:0/1152923791 wait complete. 2024-09-10T21:44:29.375 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:44:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.545+0000 7fc2daa00640 1 -- 172.21.15.7:0/1294997950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2b8005910 msgr2=0x7fc2b8005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.545+0000 7fc2daa00640 1 --2- 172.21.15.7:0/1294997950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2b8005910 0x7fc2b8005d90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc009b20 tx=0x7fc2b8006940 comp rx=0 tx=0).stop 2024-09-10T21:44:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.545+0000 7fc2daa00640 1 -- 172.21.15.7:0/1294997950 shutdown_connections 2024-09-10T21:44:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.545+0000 7fc2daa00640 1 --2- 172.21.15.7:0/1294997950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2b8005910 0x7fc2b8005d90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:29.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.545+0000 7fc2daa00640 1 -- 172.21.15.7:0/1294997950 >> 172.21.15.7:0/1294997950 conn(0x7fc2d40b16d0 msgr2=0x7fc2d40b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.546+0000 7fc2daa00640 1 -- 172.21.15.7:0/1294997950 shutdown_connections 2024-09-10T21:44:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.546+0000 7fc2daa00640 1 -- 172.21.15.7:0/1294997950 wait complete. 2024-09-10T21:44:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.547+0000 7fc2daa00640 1 Processor -- start 2024-09-10T21:44:29.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.547+0000 7fc2daa00640 1 -- start start 2024-09-10T21:44:29.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.548+0000 7fc2daa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2b8005910 0x7fc2d40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:29.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.548+0000 7fc2daa00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2d4150d10 0x7fc2d40b2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:29.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.548+0000 7fc2daa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2d41507c0 0x7fc2d41ebdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:29.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.548+0000 7fc2daa00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d40ca5e0 con 0x7fc2d41507c0 2024-09-10T21:44:29.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.548+0000 7fc2daa00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc2d40ca460 con 0x7fc2b8005910 2024-09-10T21:44:29.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.548+0000 7fc2daa00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc2d40ca760 con 0x7fc2d4150d10 2024-09-10T21:44:29.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.549+0000 7fc2d8a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2d41507c0 0x7fc2d41ebdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:29.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.549+0000 7fc2d8a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2d41507c0 0x7fc2d41ebdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36962/0 (socket says 172.21.15.7:36962) 2024-09-10T21:44:29.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.549+0000 7fc2d8a00640 1 -- 172.21.15.7:0/3068299600 learned_addr learned my addr 172.21.15.7:0/3068299600 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.549+0000 7fc2d3e00640 1 --2- 172.21.15.7:0/3068299600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2b8005910 0x7fc2d40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:29.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.549+0000 7fc2cb400640 1 --2- 172.21.15.7:0/3068299600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2d4150d10 0x7fc2d40b2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.549+0000 7fc2d8a00640 1 -- 172.21.15.7:0/3068299600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2d4150d10 msgr2=0x7fc2d40b2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.549+0000 7fc2d8a00640 1 --2- 172.21.15.7:0/3068299600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2d4150d10 0x7fc2d40b2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.549+0000 7fc2d8a00640 1 -- 172.21.15.7:0/3068299600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2b8005910 msgr2=0x7fc2d40b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.549+0000 7fc2d8a00640 1 --2- 172.21.15.7:0/3068299600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2b8005910 0x7fc2d40b4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.549+0000 7fc2d8a00640 1 -- 172.21.15.7:0/3068299600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2d40b5350 con 0x7fc2d41507c0 2024-09-10T21:44:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.549+0000 7fc2d8a00640 1 --2- 172.21.15.7:0/3068299600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2d41507c0 0x7fc2d41ebdd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4143ff0 tx=0x7fc2cc0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.550+0000 7fc2d0c00640 1 -- 172.21.15.7:0/3068299600 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc011870 con 0x7fc2d41507c0 2024-09-10T21:44:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.550+0000 7fc2d0c00640 1 -- 172.21.15.7:0/3068299600 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2cc017070 con 0x7fc2d41507c0 2024-09-10T21:44:29.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.550+0000 7fc2d0c00640 1 -- 172.21.15.7:0/3068299600 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2cc011d60 con 0x7fc2d41507c0 2024-09-10T21:44:29.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.550+0000 7fc2daa00640 1 -- 172.21.15.7:0/3068299600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2d413e810 con 0x7fc2d41507c0 2024-09-10T21:44:29.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.550+0000 7fc2daa00640 1 -- 172.21.15.7:0/3068299600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2d413edd0 con 0x7fc2d41507c0 2024-09-10T21:44:29.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.551+0000 7fc2daa00640 1 -- 172.21.15.7:0/3068299600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2d40cc820 con 0x7fc2d41507c0 2024-09-10T21:44:29.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.552+0000 7fc2d0c00640 1 -- 172.21.15.7:0/3068299600 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc2cc01c460 con 0x7fc2d41507c0 2024-09-10T21:44:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.553+0000 7fc2d0c00640 1 --2- 172.21.15.7:0/3068299600 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc2b0077fa0 0x7fc2b007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.553+0000 7fc2d0c00640 1 -- 172.21.15.7:0/3068299600 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5227+0+0 (secure 0 0 0) 0x7fc2cc09ad90 con 0x7fc2d41507c0 2024-09-10T21:44:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.554+0000 7fc2d3e00640 1 --2- 172.21.15.7:0/3068299600 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc2b0077fa0 0x7fc2b007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:29.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.554+0000 7fc2d3e00640 1 --2- 172.21.15.7:0/3068299600 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc2b0077fa0 0x7fc2b007a460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4001240 tx=0x7fc2c4006040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:29.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.556+0000 7fc2d0c00640 1 -- 172.21.15.7:0/3068299600 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc2d40b5350 con 0x7fc2d41507c0 2024-09-10T21:44:29.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:29.755+0000 7fc2daa00640 1 -- 172.21.15.7:0/3068299600 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fc2d4159eb0 con 0x7fc2d41507c0 2024-09-10T21:44:30.083 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:44:30.088 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:44:30.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.355+0000 7fc2d0c00640 1 -- 172.21.15.7:0/3068299600 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v56) ==== 170+0+0 (secure 0 0 0) 0x7fc2cc06c170 con 0x7fc2d41507c0 2024-09-10T21:44:30.357 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-09-10T21:44:30.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.358+0000 7fc2daa00640 1 -- 172.21.15.7:0/3068299600 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc2b0077fa0 msgr2=0x7fc2b007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:30.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.358+0000 7fc2daa00640 1 --2- 172.21.15.7:0/3068299600 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc2b0077fa0 0x7fc2b007a460 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fc2c4001240 tx=0x7fc2c4006040 comp rx=0 tx=0).stop 2024-09-10T21:44:30.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.358+0000 7fc2daa00640 1 -- 172.21.15.7:0/3068299600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2d41507c0 msgr2=0x7fc2d41ebdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:30.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.358+0000 7fc2daa00640 1 --2- 172.21.15.7:0/3068299600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2d41507c0 0x7fc2d41ebdd0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc2d4143ff0 tx=0x7fc2cc0019b0 comp rx=0 tx=0).stop 2024-09-10T21:44:30.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.358+0000 7fc2daa00640 1 -- 172.21.15.7:0/3068299600 shutdown_connections 2024-09-10T21:44:30.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.358+0000 7fc2daa00640 1 --2- 172.21.15.7:0/3068299600 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc2b0077fa0 0x7fc2b007a460 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:30.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.358+0000 7fc2daa00640 1 --2- 172.21.15.7:0/3068299600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2d41507c0 0x7fc2d41ebdd0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:30.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.358+0000 7fc2daa00640 1 --2- 172.21.15.7:0/3068299600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2d4150d10 0x7fc2d40b2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:30.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.358+0000 7fc2daa00640 1 --2- 172.21.15.7:0/3068299600 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2b8005910 0x7fc2d40b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:30.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.358+0000 7fc2daa00640 1 -- 172.21.15.7:0/3068299600 >> 172.21.15.7:0/3068299600 conn(0x7fc2d40b16d0 msgr2=0x7fc2d4158450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:30.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.358+0000 7fc2daa00640 1 -- 172.21.15.7:0/3068299600 shutdown_connections 2024-09-10T21:44:30.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.359+0000 7fc2daa00640 1 -- 172.21.15.7:0/3068299600 wait complete. 2024-09-10T21:44:30.394 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:44:30.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.556+0000 7f7c23000640 1 -- 172.21.15.7:0/3764160573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c1c0cf080 msgr2=0x7f7c1c0cfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:30.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.556+0000 7f7c23000640 1 --2- 172.21.15.7:0/3764160573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c1c0cf080 0x7f7c1c0cfe60 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7c1c0cd280 tx=0x7f7c1402a920 comp rx=0 tx=0).stop 2024-09-10T21:44:30.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.556+0000 7f7c23000640 1 -- 172.21.15.7:0/3764160573 shutdown_connections 2024-09-10T21:44:30.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.556+0000 7f7c23000640 1 --2- 172.21.15.7:0/3764160573 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c1c0cf080 0x7f7c1c0cfe60 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:30.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.556+0000 7f7c23000640 1 -- 172.21.15.7:0/3764160573 >> 172.21.15.7:0/3764160573 conn(0x7f7c1c0d1b80 msgr2=0x7f7c1c0d1f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:30.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.556+0000 7f7c23000640 1 -- 172.21.15.7:0/3764160573 shutdown_connections 2024-09-10T21:44:30.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.556+0000 7f7c23000640 1 -- 172.21.15.7:0/3764160573 wait complete. 2024-09-10T21:44:30.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.558+0000 7f7c23000640 1 Processor -- start 2024-09-10T21:44:30.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.558+0000 7f7c23000640 1 -- start start 2024-09-10T21:44:30.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.559+0000 7f7c23000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c1c0b2760 0x7f7c1c0c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:30.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.559+0000 7f7c23000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c1c0ce590 0x7f7c1c14d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:30.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.559+0000 7f7c23000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c1c0cf080 0x7f7c1c0b3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:30.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.559+0000 7f7c23000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c0d0730 con 0x7f7c1c0ce590 2024-09-10T21:44:30.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.559+0000 7f7c23000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1c0d05b0 con 0x7f7c1c0cf080 2024-09-10T21:44:30.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.559+0000 7f7c23000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f7c1c0d08b0 con 0x7f7c1c0b2760 2024-09-10T21:44:30.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.559+0000 7f7c21200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c1c0ce590 0x7f7c1c14d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:30.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.559+0000 7f7c21200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c1c0ce590 0x7f7c1c14d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36968/0 (socket says 172.21.15.7:36968) 2024-09-10T21:44:30.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.559+0000 7f7c21200640 1 -- 172.21.15.7:0/1144790466 learned_addr learned my addr 172.21.15.7:0/1144790466 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.560+0000 7f7c22600640 1 --2- 172.21.15.7:0/1144790466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c1c0cf080 0x7f7c1c0b3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.560+0000 7f7c21c00640 1 --2- 172.21.15.7:0/1144790466 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c1c0b2760 0x7f7c1c0c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.560+0000 7f7c21200640 1 -- 172.21.15.7:0/1144790466 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c1c0b2760 msgr2=0x7f7c1c0c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.560+0000 7f7c21200640 1 --2- 172.21.15.7:0/1144790466 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c1c0b2760 0x7f7c1c0c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.560+0000 7f7c21200640 1 -- 172.21.15.7:0/1144790466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c1c0cf080 msgr2=0x7f7c1c0b3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.560+0000 7f7c21200640 1 --2- 172.21.15.7:0/1144790466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c1c0cf080 0x7f7c1c0b3670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.560+0000 7f7c21200640 1 -- 172.21.15.7:0/1144790466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1c1f59b0 con 0x7f7c1c0ce590 2024-09-10T21:44:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.560+0000 7f7c21c00640 1 --2- 172.21.15.7:0/1144790466 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c1c0b2760 0x7f7c1c0c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:44:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.560+0000 7f7c21200640 1 --2- 172.21.15.7:0/1144790466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c1c0ce590 0x7f7c1c14d060 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800c500 tx=0x7f7c0800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.560+0000 7f7c19600640 1 -- 172.21.15.7:0/1144790466 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08016020 con 0x7f7c1c0ce590 2024-09-10T21:44:30.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.560+0000 7f7c19600640 1 -- 172.21.15.7:0/1144790466 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c08012940 con 0x7f7c1c0ce590 2024-09-10T21:44:30.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.560+0000 7f7c19600640 1 -- 172.21.15.7:0/1144790466 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c08012d90 con 0x7f7c1c0ce590 2024-09-10T21:44:30.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.560+0000 7f7c23000640 1 -- 172.21.15.7:0/1144790466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c1c1f56d0 con 0x7f7c1c0ce590 2024-09-10T21:44:30.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.561+0000 7f7c23000640 1 -- 172.21.15.7:0/1144790466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c1c0c6af0 con 0x7f7c1c0ce590 2024-09-10T21:44:30.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.561+0000 7f7c23000640 1 -- 172.21.15.7:0/1144790466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c1c0d1210 con 0x7f7c1c0ce590 2024-09-10T21:44:30.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.563+0000 7f7c19600640 1 -- 172.21.15.7:0/1144790466 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7c0801a3b0 con 0x7f7c1c0ce590 2024-09-10T21:44:30.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.564+0000 7f7c19600640 1 --2- 172.21.15.7:0/1144790466 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7bfc077f70 0x7f7bfc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:30.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.564+0000 7f7c19600640 1 -- 172.21.15.7:0/1144790466 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5609+0+0 (secure 0 0 0) 0x7f7c08099d50 con 0x7f7c1c0ce590 2024-09-10T21:44:30.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.564+0000 7f7c21c00640 1 --2- 172.21.15.7:0/1144790466 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7bfc077f70 0x7f7bfc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:30.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.565+0000 7f7c21c00640 1 --2- 172.21.15.7:0/1144790466 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7bfc077f70 0x7f7bfc07a430 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7c10000c00 tx=0x7f7c10009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:30.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.568+0000 7f7c19600640 1 -- 172.21.15.7:0/1144790466 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7c1c0d1210 con 0x7f7c1c0ce590 2024-09-10T21:44:30.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:30.803+0000 7f7c23000640 1 -- 172.21.15.7:0/1144790466 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7c1c0ce970 con 0x7f7c1c0ce590 2024-09-10T21:44:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.358+0000 7f7c19600640 1 -- 172.21.15.7:0/1144790466 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v31) ==== 172+0+0 (secure 0 0 0) 0x7f7c08020030 con 0x7f7c1c0ce590 2024-09-10T21:44:31.359 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 8 and data pool 9 2024-09-10T21:44:31.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.361+0000 7f7c23000640 1 -- 172.21.15.7:0/1144790466 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7bfc077f70 msgr2=0x7f7bfc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:31.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.361+0000 7f7c23000640 1 --2- 172.21.15.7:0/1144790466 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7bfc077f70 0x7f7bfc07a430 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f7c10000c00 tx=0x7f7c10009040 comp rx=0 tx=0).stop 2024-09-10T21:44:31.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.361+0000 7f7c23000640 1 -- 172.21.15.7:0/1144790466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c1c0ce590 msgr2=0x7f7c1c14d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:31.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.361+0000 7f7c23000640 1 --2- 172.21.15.7:0/1144790466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c1c0ce590 0x7f7c1c14d060 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f7c0800c500 tx=0x7f7c0800c9c0 comp rx=0 tx=0).stop 2024-09-10T21:44:31.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.361+0000 7f7c23000640 1 -- 172.21.15.7:0/1144790466 shutdown_connections 2024-09-10T21:44:31.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.361+0000 7f7c23000640 1 --2- 172.21.15.7:0/1144790466 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7bfc077f70 0x7f7bfc07a430 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:31.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.361+0000 7f7c23000640 1 --2- 172.21.15.7:0/1144790466 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7c1c0cf080 0x7f7c1c0b3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:31.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.361+0000 7f7c23000640 1 --2- 172.21.15.7:0/1144790466 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7c1c0ce590 0x7f7c1c14d060 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:31.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.361+0000 7f7c23000640 1 --2- 172.21.15.7:0/1144790466 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7c1c0b2760 0x7f7c1c0c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:31.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.361+0000 7f7c23000640 1 -- 172.21.15.7:0/1144790466 >> 172.21.15.7:0/1144790466 conn(0x7f7c1c0d1b80 msgr2=0x7f7c1c0b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:31.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.362+0000 7f7c23000640 1 -- 172.21.15.7:0/1144790466 shutdown_connections 2024-09-10T21:44:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.363+0000 7f7c23000640 1 -- 172.21.15.7:0/1144790466 wait complete. 2024-09-10T21:44:31.401 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:44:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.567+0000 7f4d60200640 1 -- 172.21.15.7:0/3288167686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d40004d40 msgr2=0x7f4d400051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.567+0000 7f4d60200640 1 --2- 172.21.15.7:0/3288167686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d40004d40 0x7f4d400051c0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4d50005fd0 tx=0x7f4d40005830 comp rx=0 tx=0).stop 2024-09-10T21:44:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.567+0000 7f4d5e200640 1 -- 172.21.15.7:0/3288167686 reap_dead start 2024-09-10T21:44:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.567+0000 7f4d60200640 1 -- 172.21.15.7:0/3288167686 shutdown_connections 2024-09-10T21:44:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.567+0000 7f4d60200640 1 -- 172.21.15.7:0/3288167686 >> 172.21.15.7:0/3288167686 conn(0x7f4d580b16d0 msgr2=0x7f4d580b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:31.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.568+0000 7f4d60200640 1 -- 172.21.15.7:0/3288167686 shutdown_connections 2024-09-10T21:44:31.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.568+0000 7f4d60200640 1 -- 172.21.15.7:0/3288167686 wait complete. 2024-09-10T21:44:31.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.569+0000 7f4d60200640 1 Processor -- start 2024-09-10T21:44:31.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.569+0000 7f4d60200640 1 -- start start 2024-09-10T21:44:31.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.570+0000 7f4d60200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d40004d40 0x7f4d58146e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:31.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.570+0000 7f4d60200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d58153600 0x7f4d580b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:31.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.570+0000 7f4d60200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d580c8480 0x7f4d580c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:31.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.570+0000 7f4d60200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4d580cdd50 con 0x7f4d580c8480 2024-09-10T21:44:31.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.570+0000 7f4d60200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4d580cdbd0 con 0x7f4d40004d40 2024-09-10T21:44:31.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.570+0000 7f4d60200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4d580cded0 con 0x7f4d58153600 2024-09-10T21:44:31.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.570+0000 7f4d5e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d580c8480 0x7f4d580c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:31.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.571+0000 7f4d5e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d580c8480 0x7f4d580c8910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36990/0 (socket says 172.21.15.7:36990) 2024-09-10T21:44:31.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.571+0000 7f4d5e200640 1 -- 172.21.15.7:0/3697197384 learned_addr learned my addr 172.21.15.7:0/3697197384 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.571+0000 7f4d5d800640 1 --2- 172.21.15.7:0/3697197384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d40004d40 0x7f4d58146e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.571+0000 7f4d5ce00640 1 --2- 172.21.15.7:0/3697197384 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d58153600 0x7f4d580b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.571+0000 7f4d5e200640 1 -- 172.21.15.7:0/3697197384 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d58153600 msgr2=0x7f4d580b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.571+0000 7f4d5e200640 1 --2- 172.21.15.7:0/3697197384 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d58153600 0x7f4d580b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.571+0000 7f4d5e200640 1 -- 172.21.15.7:0/3697197384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d40004d40 msgr2=0x7f4d58146e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.571+0000 7f4d5e200640 1 --2- 172.21.15.7:0/3697197384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d40004d40 0x7f4d58146e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:31.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.571+0000 7f4d5e200640 1 -- 172.21.15.7:0/3697197384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d580c91e0 con 0x7f4d580c8480 2024-09-10T21:44:31.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.571+0000 7f4d5e200640 1 --2- 172.21.15.7:0/3697197384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d580c8480 0x7f4d580c8910 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4d50036040 tx=0x7f4d50033bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:31.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.571+0000 7f4d55600640 1 -- 172.21.15.7:0/3697197384 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d50038040 con 0x7f4d580c8480 2024-09-10T21:44:31.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.572+0000 7f4d55600640 1 -- 172.21.15.7:0/3697197384 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d50003970 con 0x7f4d580c8480 2024-09-10T21:44:31.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.572+0000 7f4d55600640 1 -- 172.21.15.7:0/3697197384 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d50009ad0 con 0x7f4d580c8480 2024-09-10T21:44:31.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.572+0000 7f4d60200640 1 -- 172.21.15.7:0/3697197384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d580c94c0 con 0x7f4d580c8480 2024-09-10T21:44:31.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.572+0000 7f4d60200640 1 -- 172.21.15.7:0/3697197384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d580c9a80 con 0x7f4d580c8480 2024-09-10T21:44:31.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.573+0000 7f4d60200640 1 -- 172.21.15.7:0/3697197384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d580ae020 con 0x7f4d580c8480 2024-09-10T21:44:31.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.574+0000 7f4d55600640 1 -- 172.21.15.7:0/3697197384 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4d5003b070 con 0x7f4d580c8480 2024-09-10T21:44:31.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.575+0000 7f4d55600640 1 --2- 172.21.15.7:0/3697197384 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4d38077f70 0x7f4d3807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:31.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.575+0000 7f4d55600640 1 -- 172.21.15.7:0/3697197384 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f4d500ba2c0 con 0x7f4d580c8480 2024-09-10T21:44:31.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.575+0000 7f4d5d800640 1 --2- 172.21.15.7:0/3697197384 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4d38077f70 0x7f4d3807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:31.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.576+0000 7f4d5d800640 1 --2- 172.21.15.7:0/3697197384 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4d38077f70 0x7f4d3807a430 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c006c50 tx=0x7f4d4c001c20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:31.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.579+0000 7f4d55600640 1 -- 172.21.15.7:0/3697197384 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4d5004a020 con 0x7f4d580c8480 2024-09-10T21:44:31.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.775+0000 7f4d60200640 1 -- 172.21.15.7:0/3697197384 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4d58069810 con 0x7f4d580c8480 2024-09-10T21:44:31.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.776+0000 7f4d55600640 1 -- 172.21.15.7:0/3697197384 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16702 (secure 0 0 0) 0x7f4d50082760 con 0x7f4d580c8480 2024-09-10T21:44:31.778 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:31.778 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":57,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:44:30.804615+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:44:28.642474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T21:44:29.765081+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:44:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.778+0000 7f4d60200640 1 -- 172.21.15.7:0/3697197384 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4d38077f70 msgr2=0x7f4d3807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.778+0000 7f4d60200640 1 --2- 172.21.15.7:0/3697197384 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4d38077f70 0x7f4d3807a430 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4d4c006c50 tx=0x7f4d4c001c20 comp rx=0 tx=0).stop 2024-09-10T21:44:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.778+0000 7f4d60200640 1 -- 172.21.15.7:0/3697197384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d580c8480 msgr2=0x7f4d580c8910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.778+0000 7f4d60200640 1 --2- 172.21.15.7:0/3697197384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d580c8480 0x7f4d580c8910 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4d50036040 tx=0x7f4d50033bc0 comp rx=0 tx=0).stop 2024-09-10T21:44:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.779+0000 7f4d60200640 1 -- 172.21.15.7:0/3697197384 shutdown_connections 2024-09-10T21:44:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.779+0000 7f4d60200640 1 --2- 172.21.15.7:0/3697197384 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4d38077f70 0x7f4d3807a430 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.779+0000 7f4d60200640 1 --2- 172.21.15.7:0/3697197384 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d580c8480 0x7f4d580c8910 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.779+0000 7f4d60200640 1 --2- 172.21.15.7:0/3697197384 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d58153600 0x7f4d580b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.779+0000 7f4d60200640 1 --2- 172.21.15.7:0/3697197384 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d40004d40 0x7f4d58146e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:31.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.779+0000 7f4d60200640 1 -- 172.21.15.7:0/3697197384 >> 172.21.15.7:0/3697197384 conn(0x7f4d580b16d0 msgr2=0x7f4d5815b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.779+0000 7f4d60200640 1 -- 172.21.15.7:0/3697197384 shutdown_connections 2024-09-10T21:44:31.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.779+0000 7f4d60200640 1 -- 172.21.15.7:0/3697197384 wait complete. 2024-09-10T21:44:31.797 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:44:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.957+0000 7f45b1c00640 1 -- 172.21.15.7:0/161531747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ac0cd750 msgr2=0x7f45ac0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.957+0000 7f45b1c00640 1 --2- 172.21.15.7:0/161531747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ac0cd750 0x7f45ac0ccef0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f45ac0c3650 tx=0x7f459402a560 comp rx=0 tx=0).stop 2024-09-10T21:44:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.957+0000 7f45b1c00640 1 -- 172.21.15.7:0/161531747 shutdown_connections 2024-09-10T21:44:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.957+0000 7f45b1c00640 1 --2- 172.21.15.7:0/161531747 >> v2:172.21.15.133:3300/0 conn(0x7f45ac15fda0 0x7f45ac160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.957+0000 7f45b1c00640 1 --2- 172.21.15.7:0/161531747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ac0cd750 0x7f45ac0ccef0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.957+0000 7f45b1c00640 1 -- 172.21.15.7:0/161531747 >> 172.21.15.7:0/161531747 conn(0x7f45ac0b16d0 msgr2=0x7f45ac0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.957+0000 7f45b1c00640 1 -- 172.21.15.7:0/161531747 shutdown_connections 2024-09-10T21:44:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.957+0000 7f45b1c00640 1 -- 172.21.15.7:0/161531747 wait complete. 2024-09-10T21:44:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.959+0000 7f45b1c00640 1 Processor -- start 2024-09-10T21:44:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.959+0000 7f45b1c00640 1 -- start start 2024-09-10T21:44:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.960+0000 7f45b1c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ac0cd750 0x7f45ac15ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.960+0000 7f45b1c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac15bc90 0x7f45ac0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.960+0000 7f45b1c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45ac15fda0 0x7f45ac1f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.960+0000 7f45b1c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac06fe70 con 0x7f45ac0cd750 2024-09-10T21:44:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.960+0000 7f45b1c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f45ac06fcf0 con 0x7f45ac15bc90 2024-09-10T21:44:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.960+0000 7f45b1c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f45ac06fff0 con 0x7f45ac15fda0 2024-09-10T21:44:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.960+0000 7f45aaa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ac0cd750 0x7f45ac15ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.960+0000 7f45aaa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ac0cd750 0x7f45ac15ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37006/0 (socket says 172.21.15.7:37006) 2024-09-10T21:44:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.961+0000 7f45aaa00640 1 -- 172.21.15.7:0/4109837893 learned_addr learned my addr 172.21.15.7:0/4109837893 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.961+0000 7f45a2000640 1 --2- 172.21.15.7:0/4109837893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac15bc90 0x7f45ac0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.961+0000 7f45ab400640 1 --2- 172.21.15.7:0/4109837893 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45ac15fda0 0x7f45ac1f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.961+0000 7f45aaa00640 1 -- 172.21.15.7:0/4109837893 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45ac15fda0 msgr2=0x7f45ac1f7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.961+0000 7f45aaa00640 1 --2- 172.21.15.7:0/4109837893 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45ac15fda0 0x7f45ac1f7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.961+0000 7f45aaa00640 1 -- 172.21.15.7:0/4109837893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac15bc90 msgr2=0x7f45ac0c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.961+0000 7f45aaa00640 1 --2- 172.21.15.7:0/4109837893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac15bc90 0x7f45ac0c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.961+0000 7f45aaa00640 1 -- 172.21.15.7:0/4109837893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45ac0c8fb0 con 0x7f45ac0cd750 2024-09-10T21:44:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.961+0000 7f45aaa00640 1 --2- 172.21.15.7:0/4109837893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ac0cd750 0x7f45ac15ed40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f459c00c500 tx=0x7f459c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.961+0000 7f45a3400640 1 -- 172.21.15.7:0/4109837893 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c017070 con 0x7f45ac0cd750 2024-09-10T21:44:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.962+0000 7f45a3400640 1 -- 172.21.15.7:0/4109837893 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f459c0035f0 con 0x7f45ac0cd750 2024-09-10T21:44:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.962+0000 7f45a3400640 1 -- 172.21.15.7:0/4109837893 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f459c003a40 con 0x7f45ac0cd750 2024-09-10T21:44:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.962+0000 7f45b1c00640 1 -- 172.21.15.7:0/4109837893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45ac0c9290 con 0x7f45ac0cd750 2024-09-10T21:44:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.962+0000 7f45b1c00640 1 -- 172.21.15.7:0/4109837893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45ac0af560 con 0x7f45ac0cd750 2024-09-10T21:44:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.963+0000 7f45b1c00640 1 -- 172.21.15.7:0/4109837893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45ac070510 con 0x7f45ac0cd750 2024-09-10T21:44:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.964+0000 7f45a3400640 1 -- 172.21.15.7:0/4109837893 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f459c003be0 con 0x7f45ac0cd750 2024-09-10T21:44:31.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.965+0000 7f45a3400640 1 --2- 172.21.15.7:0/4109837893 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4588077f70 0x7f458807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:31.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.965+0000 7f45a3400640 1 -- 172.21.15.7:0/4109837893 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f459c099510 con 0x7f45ac0cd750 2024-09-10T21:44:31.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.965+0000 7f45a2000640 1 --2- 172.21.15.7:0/4109837893 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4588077f70 0x7f458807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.966+0000 7f45a2000640 1 --2- 172.21.15.7:0/4109837893 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4588077f70 0x7f458807a430 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4594033000 tx=0x7f4594004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:31.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:31.969+0000 7f45a3400640 1 -- 172.21.15.7:0/4109837893 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f459c0619b0 con 0x7f45ac0cd750 2024-09-10T21:44:32.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.162+0000 7f45b1c00640 1 -- 172.21.15.7:0/4109837893 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45ac0cc1e0 con 0x7f45ac0cd750 2024-09-10T21:44:32.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.163+0000 7f45a3400640 1 -- 172.21.15.7:0/4109837893 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) ==== 74+0+16702 (secure 0 0 0) 0x7f459c0663e0 con 0x7f45ac0cd750 2024-09-10T21:44:32.165 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:32.165 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":57,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:44:30.804615+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:44:28.642474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T21:44:29.765081+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:44:32.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.165+0000 7f45b1c00640 1 -- 172.21.15.7:0/4109837893 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4588077f70 msgr2=0x7f458807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:32.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.165+0000 7f45b1c00640 1 --2- 172.21.15.7:0/4109837893 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4588077f70 0x7f458807a430 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4594033000 tx=0x7f4594004730 comp rx=0 tx=0).stop 2024-09-10T21:44:32.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.165+0000 7f45b1c00640 1 -- 172.21.15.7:0/4109837893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ac0cd750 msgr2=0x7f45ac15ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:32.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.165+0000 7f45b1c00640 1 --2- 172.21.15.7:0/4109837893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ac0cd750 0x7f45ac15ed40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f459c00c500 tx=0x7f459c00c9c0 comp rx=0 tx=0).stop 2024-09-10T21:44:32.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.166+0000 7f45b1c00640 1 -- 172.21.15.7:0/4109837893 shutdown_connections 2024-09-10T21:44:32.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.166+0000 7f45b1c00640 1 --2- 172.21.15.7:0/4109837893 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4588077f70 0x7f458807a430 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:32.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.166+0000 7f45b1c00640 1 --2- 172.21.15.7:0/4109837893 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45ac15bc90 0x7f45ac0c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:32.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.166+0000 7f45b1c00640 1 --2- 172.21.15.7:0/4109837893 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45ac15fda0 0x7f45ac1f7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:32.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.166+0000 7f45b1c00640 1 --2- 172.21.15.7:0/4109837893 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45ac0cd750 0x7f45ac15ed40 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:32.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.166+0000 7f45b1c00640 1 -- 172.21.15.7:0/4109837893 >> 172.21.15.7:0/4109837893 conn(0x7f45ac0b16d0 msgr2=0x7f45ac0b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:32.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.166+0000 7f45b1c00640 1 -- 172.21.15.7:0/4109837893 shutdown_connections 2024-09-10T21:44:32.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.166+0000 7f45b1c00640 1 -- 172.21.15.7:0/4109837893 wait complete. 2024-09-10T21:44:32.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:44:32.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.345+0000 7f286ae00640 1 -- 172.21.15.7:0/1364211656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28640b42f0 msgr2=0x7f286415f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:32.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.345+0000 7f286ae00640 1 --2- 172.21.15.7:0/1364211656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28640b42f0 0x7f286415f020 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f286415bd10 tx=0x7f285c02a920 comp rx=0 tx=0).stop 2024-09-10T21:44:32.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.345+0000 7f286ae00640 1 -- 172.21.15.7:0/1364211656 shutdown_connections 2024-09-10T21:44:32.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.345+0000 7f286ae00640 1 --2- 172.21.15.7:0/1364211656 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28640b42f0 0x7f286415f020 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:32.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.345+0000 7f286ae00640 1 -- 172.21.15.7:0/1364211656 >> 172.21.15.7:0/1364211656 conn(0x7f286406cd00 msgr2=0x7f286406d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:32.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.346+0000 7f286ae00640 1 -- 172.21.15.7:0/1364211656 shutdown_connections 2024-09-10T21:44:32.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.346+0000 7f286ae00640 1 -- 172.21.15.7:0/1364211656 wait complete. 2024-09-10T21:44:32.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.347+0000 7f286ae00640 1 Processor -- start 2024-09-10T21:44:32.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.347+0000 7f286ae00640 1 -- start start 2024-09-10T21:44:32.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.348+0000 7f286ae00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28640b3f10 0x7f2864150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:32.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.348+0000 7f286ae00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f28640b42f0 0x7f28641472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:32.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.348+0000 7f286ae00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28640d04b0 0x7f2864163fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:32.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.348+0000 7f286ae00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2864165d00 con 0x7f28640b3f10 2024-09-10T21:44:32.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.348+0000 7f286ae00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2864165b80 con 0x7f28640d04b0 2024-09-10T21:44:32.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.348+0000 7f286ae00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f2864165e80 con 0x7f28640b42f0 2024-09-10T21:44:32.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.348+0000 7f2869a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28640b3f10 0x7f2864150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:32.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.348+0000 7f2869a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28640b3f10 0x7f2864150e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37008/0 (socket says 172.21.15.7:37008) 2024-09-10T21:44:32.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.348+0000 7f2869a00640 1 -- 172.21.15.7:0/2926782263 learned_addr learned my addr 172.21.15.7:0/2926782263 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:32.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.349+0000 7f286a400640 1 --2- 172.21.15.7:0/2926782263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28640d04b0 0x7f2864163fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:32.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.349+0000 7f2869000640 1 --2- 172.21.15.7:0/2926782263 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f28640b42f0 0x7f28641472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:32.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.349+0000 7f2869a00640 1 -- 172.21.15.7:0/2926782263 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f28640b42f0 msgr2=0x7f28641472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:32.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.349+0000 7f2869a00640 1 --2- 172.21.15.7:0/2926782263 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f28640b42f0 0x7f28641472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:32.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.349+0000 7f2869a00640 1 -- 172.21.15.7:0/2926782263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28640d04b0 msgr2=0x7f2864163fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:32.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.349+0000 7f2869a00640 1 --2- 172.21.15.7:0/2926782263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28640d04b0 0x7f2864163fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:32.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.349+0000 7f2869a00640 1 -- 172.21.15.7:0/2926782263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f286414dd90 con 0x7f28640b3f10 2024-09-10T21:44:32.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.349+0000 7f2869a00640 1 --2- 172.21.15.7:0/2926782263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28640b3f10 0x7f2864150e50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f285800c4c0 tx=0x7f285800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:32.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.349+0000 7f2861600640 1 -- 172.21.15.7:0/2926782263 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858016020 con 0x7f28640b3f10 2024-09-10T21:44:32.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.349+0000 7f286ae00640 1 -- 172.21.15.7:0/2926782263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28641f7fa0 con 0x7f28640b3f10 2024-09-10T21:44:32.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.349+0000 7f2861600640 1 -- 172.21.15.7:0/2926782263 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2858012a00 con 0x7f28640b3f10 2024-09-10T21:44:32.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.349+0000 7f2861600640 1 -- 172.21.15.7:0/2926782263 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858012e50 con 0x7f28640b3f10 2024-09-10T21:44:32.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.350+0000 7f286ae00640 1 -- 172.21.15.7:0/2926782263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f286414e000 con 0x7f28640b3f10 2024-09-10T21:44:32.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.351+0000 7f286ae00640 1 -- 172.21.15.7:0/2926782263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28640cfda0 con 0x7f28640b3f10 2024-09-10T21:44:32.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.352+0000 7f2861600640 1 -- 172.21.15.7:0/2926782263 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f285801a3b0 con 0x7f28640b3f10 2024-09-10T21:44:32.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.353+0000 7f2861600640 1 --2- 172.21.15.7:0/2926782263 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2844078140 0x7f284407a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:32.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.353+0000 7f2861600640 1 -- 172.21.15.7:0/2926782263 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5709+0+0 (secure 0 0 0) 0x7f2858099e20 con 0x7f28640b3f10 2024-09-10T21:44:32.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.353+0000 7f2869000640 1 --2- 172.21.15.7:0/2926782263 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2844078140 0x7f284407a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:32.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.354+0000 7f2869000640 1 --2- 172.21.15.7:0/2926782263 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2844078140 0x7f284407a600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2850007fd0 tx=0x7f2850009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:32.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.357+0000 7f2861600640 1 -- 172.21.15.7:0/2926782263 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2858020030 con 0x7f28640b3f10 2024-09-10T21:44:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:32.594+0000 7f286ae00640 1 -- 172.21.15.7:0/2926782263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f28640ce5f0 con 0x7f28640b3f10 2024-09-10T21:44:33.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.388+0000 7f2861600640 1 -- 172.21.15.7:0/2926782263 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v34) ==== 118+0+0 (secure 0 0 0) 0x7f28641f7fa0 con 0x7f28640b3f10 2024-09-10T21:44:33.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.391+0000 7f286ae00640 1 -- 172.21.15.7:0/2926782263 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2844078140 msgr2=0x7f284407a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:33.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.391+0000 7f286ae00640 1 --2- 172.21.15.7:0/2926782263 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2844078140 0x7f284407a600 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2850007fd0 tx=0x7f2850009040 comp rx=0 tx=0).stop 2024-09-10T21:44:33.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.391+0000 7f286ae00640 1 -- 172.21.15.7:0/2926782263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28640b3f10 msgr2=0x7f2864150e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:33.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.391+0000 7f286ae00640 1 --2- 172.21.15.7:0/2926782263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28640b3f10 0x7f2864150e50 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f285800c4c0 tx=0x7f285800c980 comp rx=0 tx=0).stop 2024-09-10T21:44:33.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.392+0000 7f286ae00640 1 -- 172.21.15.7:0/2926782263 shutdown_connections 2024-09-10T21:44:33.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.392+0000 7f286ae00640 1 --2- 172.21.15.7:0/2926782263 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2844078140 0x7f284407a600 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:33.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.392+0000 7f286ae00640 1 --2- 172.21.15.7:0/2926782263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f28640d04b0 0x7f2864163fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:33.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.392+0000 7f286ae00640 1 --2- 172.21.15.7:0/2926782263 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f28640b42f0 0x7f28641472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:33.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.392+0000 7f286ae00640 1 --2- 172.21.15.7:0/2926782263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f28640b3f10 0x7f2864150e50 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:33.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.392+0000 7f286ae00640 1 -- 172.21.15.7:0/2926782263 >> 172.21.15.7:0/2926782263 conn(0x7f286406cd00 msgr2=0x7f286406d0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:33.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.392+0000 7f286ae00640 1 -- 172.21.15.7:0/2926782263 shutdown_connections 2024-09-10T21:44:33.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.392+0000 7f286ae00640 1 -- 172.21.15.7:0/2926782263 wait complete. 2024-09-10T21:44:33.410 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:44:33.411 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:44:33.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.575+0000 7fd6eba00640 1 -- 172.21.15.7:0/767341731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6e4070670 msgr2=0x7fd6e4154370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:33.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.575+0000 7fd6eba00640 1 --2- 172.21.15.7:0/767341731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6e4070670 0x7fd6e4154370 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fd6e40cb520 tx=0x7fd6d402a560 comp rx=0 tx=0).stop 2024-09-10T21:44:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.575+0000 7fd6eba00640 1 -- 172.21.15.7:0/767341731 shutdown_connections 2024-09-10T21:44:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.575+0000 7fd6eba00640 1 --2- 172.21.15.7:0/767341731 >> v2:172.21.15.133:3300/0 conn(0x7fd6e4161b10 0x7fd6e4163f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.575+0000 7fd6eba00640 1 --2- 172.21.15.7:0/767341731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6e4070670 0x7fd6e4154370 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.575+0000 7fd6eba00640 1 -- 172.21.15.7:0/767341731 >> 172.21.15.7:0/767341731 conn(0x7fd6e40b16d0 msgr2=0x7fd6e40b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.576+0000 7fd6eba00640 1 -- 172.21.15.7:0/767341731 shutdown_connections 2024-09-10T21:44:33.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.576+0000 7fd6eba00640 1 -- 172.21.15.7:0/767341731 wait complete. 2024-09-10T21:44:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.577+0000 7fd6eba00640 1 Processor -- start 2024-09-10T21:44:33.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.577+0000 7fd6eba00640 1 -- start start 2024-09-10T21:44:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.578+0000 7fd6eba00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6e4070670 0x7fd6e41f5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.578+0000 7fd6eba00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6e415a5e0 0x7fd6e40c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.578+0000 7fd6eba00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6e4161b10 0x7fd6e4158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.578+0000 7fd6eba00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e40cc840 con 0x7fd6e4161b10 2024-09-10T21:44:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.578+0000 7fd6eba00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e40cc6c0 con 0x7fd6e415a5e0 2024-09-10T21:44:33.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.578+0000 7fd6eba00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fd6e40cc9c0 con 0x7fd6e4070670 2024-09-10T21:44:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.579+0000 7fd6e9a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6e4161b10 0x7fd6e4158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.579+0000 7fd6e9a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6e4161b10 0x7fd6e4158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37016/0 (socket says 172.21.15.7:37016) 2024-09-10T21:44:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.579+0000 7fd6e9a00640 1 -- 172.21.15.7:0/804745604 learned_addr learned my addr 172.21.15.7:0/804745604 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.579+0000 7fd6dbe00640 1 --2- 172.21.15.7:0/804745604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6e415a5e0 0x7fd6e40c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.579+0000 7fd6e9000640 1 --2- 172.21.15.7:0/804745604 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6e4070670 0x7fd6e41f5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.579+0000 7fd6e9a00640 1 -- 172.21.15.7:0/804745604 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6e4070670 msgr2=0x7fd6e41f5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.579+0000 7fd6e9a00640 1 --2- 172.21.15.7:0/804745604 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6e4070670 0x7fd6e41f5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:33.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.579+0000 7fd6e9a00640 1 -- 172.21.15.7:0/804745604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6e415a5e0 msgr2=0x7fd6e40c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.579+0000 7fd6e9a00640 1 --2- 172.21.15.7:0/804745604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6e415a5e0 0x7fd6e40c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.579+0000 7fd6e9a00640 1 -- 172.21.15.7:0/804745604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e40c6600 con 0x7fd6e4161b10 2024-09-10T21:44:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.580+0000 7fd6e9a00640 1 --2- 172.21.15.7:0/804745604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6e4161b10 0x7fd6e4158e80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00cab0 tx=0x7fd6dc00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.580+0000 7fd6e1600640 1 -- 172.21.15.7:0/804745604 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc007d90 con 0x7fd6e4161b10 2024-09-10T21:44:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.580+0000 7fd6e1600640 1 -- 172.21.15.7:0/804745604 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6dc00f540 con 0x7fd6e4161b10 2024-09-10T21:44:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.580+0000 7fd6eba00640 1 -- 172.21.15.7:0/804745604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e4141d50 con 0x7fd6e4161b10 2024-09-10T21:44:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.580+0000 7fd6e1600640 1 -- 172.21.15.7:0/804745604 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc00f990 con 0x7fd6e4161b10 2024-09-10T21:44:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.580+0000 7fd6eba00640 1 -- 172.21.15.7:0/804745604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e4142310 con 0x7fd6e4161b10 2024-09-10T21:44:33.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.581+0000 7fd6eba00640 1 -- 172.21.15.7:0/804745604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e4154370 con 0x7fd6e4161b10 2024-09-10T21:44:33.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.582+0000 7fd6e1600640 1 -- 172.21.15.7:0/804745604 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd6dc00fb30 con 0x7fd6e4161b10 2024-09-10T21:44:33.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.583+0000 7fd6e1600640 1 --2- 172.21.15.7:0/804745604 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd6c0077f70 0x7fd6c007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:33.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.583+0000 7fd6e1600640 1 -- 172.21.15.7:0/804745604 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fd6dc09a7d0 con 0x7fd6e4161b10 2024-09-10T21:44:33.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.583+0000 7fd6e9000640 1 --2- 172.21.15.7:0/804745604 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd6c0077f70 0x7fd6c007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:33.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.584+0000 7fd6e9000640 1 --2- 172.21.15.7:0/804745604 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd6c0077f70 0x7fd6c007a430 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4033000 tx=0x7fd6d4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:33.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.587+0000 7fd6e1600640 1 -- 172.21.15.7:0/804745604 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd6dc014070 con 0x7fd6e4161b10 2024-09-10T21:44:33.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:33.820+0000 7fd6eba00640 1 -- 172.21.15.7:0/804745604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fd6e415a0e0 con 0x7fd6e4161b10 2024-09-10T21:44:34.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.394+0000 7fd6e1600640 1 -- 172.21.15.7:0/804745604 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v35) ==== 122+0+0 (secure 0 0 0) 0x7fd6dc062be0 con 0x7fd6e4161b10 2024-09-10T21:44:34.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.397+0000 7fd6eba00640 1 -- 172.21.15.7:0/804745604 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd6c0077f70 msgr2=0x7fd6c007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.397+0000 7fd6eba00640 1 --2- 172.21.15.7:0/804745604 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd6c0077f70 0x7fd6c007a430 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4033000 tx=0x7fd6d4004730 comp rx=0 tx=0).stop 2024-09-10T21:44:34.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.397+0000 7fd6eba00640 1 -- 172.21.15.7:0/804745604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6e4161b10 msgr2=0x7fd6e4158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.397+0000 7fd6eba00640 1 --2- 172.21.15.7:0/804745604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6e4161b10 0x7fd6e4158e80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc00cab0 tx=0x7fd6dc00cf70 comp rx=0 tx=0).stop 2024-09-10T21:44:34.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.397+0000 7fd6eba00640 1 -- 172.21.15.7:0/804745604 shutdown_connections 2024-09-10T21:44:34.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.397+0000 7fd6eba00640 1 --2- 172.21.15.7:0/804745604 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd6c0077f70 0x7fd6c007a430 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.397+0000 7fd6eba00640 1 --2- 172.21.15.7:0/804745604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6e415a5e0 0x7fd6e40c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.397+0000 7fd6eba00640 1 --2- 172.21.15.7:0/804745604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6e4161b10 0x7fd6e4158e80 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.397+0000 7fd6eba00640 1 --2- 172.21.15.7:0/804745604 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6e4070670 0x7fd6e41f5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.397+0000 7fd6eba00640 1 -- 172.21.15.7:0/804745604 >> 172.21.15.7:0/804745604 conn(0x7fd6e40b16d0 msgr2=0x7fd6e40b54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:34.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.398+0000 7fd6eba00640 1 -- 172.21.15.7:0/804745604 shutdown_connections 2024-09-10T21:44:34.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.398+0000 7fd6eba00640 1 -- 172.21.15.7:0/804745604 wait complete. 2024-09-10T21:44:34.418 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:44:34.418 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:44:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.583+0000 7f69f0c00640 1 -- 172.21.15.7:0/3021197716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69d0003a30 msgr2=0x7f69d0003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.583+0000 7f69f0c00640 1 --2- 172.21.15.7:0/3021197716 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69d0003a30 0x7f69d0003e50 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f69dc0016c0 tx=0x7f69d0005830 comp rx=0 tx=0).stop 2024-09-10T21:44:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.583+0000 7f69ebe00640 1 -- 172.21.15.7:0/3021197716 reap_dead start 2024-09-10T21:44:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.583+0000 7f69f0c00640 1 -- 172.21.15.7:0/3021197716 shutdown_connections 2024-09-10T21:44:34.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.583+0000 7f69f0c00640 1 -- 172.21.15.7:0/3021197716 >> 172.21.15.7:0/3021197716 conn(0x7f69ec0d3d50 msgr2=0x7f69ec0d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:34.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.583+0000 7f69f0c00640 1 -- 172.21.15.7:0/3021197716 shutdown_connections 2024-09-10T21:44:34.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.583+0000 7f69f0c00640 1 -- 172.21.15.7:0/3021197716 wait complete. 2024-09-10T21:44:34.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.584+0000 7f69f0c00640 1 Processor -- start 2024-09-10T21:44:34.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.585+0000 7f69f0c00640 1 -- start start 2024-09-10T21:44:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.586+0000 7f69f0c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69d0003a30 0x7f69ec14cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.586+0000 7f69f0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69ec0b9440 0x7f69ec0ca720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.586+0000 7f69f0c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69ec1f8280 0x7f69ec1f8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.586+0000 7f69f0c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f69ec0d4c00 con 0x7f69ec0b9440 2024-09-10T21:44:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.586+0000 7f69f0c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f69ec0d4a80 con 0x7f69ec1f8280 2024-09-10T21:44:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.586+0000 7f69f0c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f69ec0b0a00 con 0x7f69d0003a30 2024-09-10T21:44:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.586+0000 7f69eaa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69ec0b9440 0x7f69ec0ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.586+0000 7f69eaa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69ec0b9440 0x7f69ec0ca720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43446/0 (socket says 172.21.15.7:43446) 2024-09-10T21:44:34.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.586+0000 7f69eaa00640 1 -- 172.21.15.7:0/4228042670 learned_addr learned my addr 172.21.15.7:0/4228042670 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:34.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.586+0000 7f69ebe00640 1 --2- 172.21.15.7:0/4228042670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69ec1f8280 0x7f69ec1f8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.586+0000 7f69eb400640 1 --2- 172.21.15.7:0/4228042670 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69d0003a30 0x7f69ec14cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.586+0000 7f69eaa00640 1 -- 172.21.15.7:0/4228042670 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69d0003a30 msgr2=0x7f69ec14cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.586+0000 7f69eaa00640 1 --2- 172.21.15.7:0/4228042670 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69d0003a30 0x7f69ec14cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.586+0000 7f69eaa00640 1 -- 172.21.15.7:0/4228042670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69ec1f8280 msgr2=0x7f69ec1f8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.586+0000 7f69eaa00640 1 --2- 172.21.15.7:0/4228042670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69ec1f8280 0x7f69ec1f8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.586+0000 7f69eaa00640 1 -- 172.21.15.7:0/4228042670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69ec0b8680 con 0x7f69ec0b9440 2024-09-10T21:44:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.587+0000 7f69eaa00640 1 --2- 172.21.15.7:0/4228042670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69ec0b9440 0x7f69ec0ca720 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f69d800e570 tx=0x7f69d800ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.587+0000 7f69e3400640 1 -- 172.21.15.7:0/4228042670 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d8015070 con 0x7f69ec0b9440 2024-09-10T21:44:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.587+0000 7f69e3400640 1 -- 172.21.15.7:0/4228042670 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69d800c9e0 con 0x7f69ec0b9440 2024-09-10T21:44:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.587+0000 7f69e3400640 1 -- 172.21.15.7:0/4228042670 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69d800cdd0 con 0x7f69ec0b9440 2024-09-10T21:44:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.587+0000 7f69f0c00640 1 -- 172.21.15.7:0/4228042670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69ec0b8960 con 0x7f69ec0b9440 2024-09-10T21:44:34.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.587+0000 7f69f0c00640 1 -- 172.21.15.7:0/4228042670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69ec15c970 con 0x7f69ec0b9440 2024-09-10T21:44:34.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.588+0000 7f69f0c00640 1 -- 172.21.15.7:0/4228042670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69ec0d1ee0 con 0x7f69ec0b9440 2024-09-10T21:44:34.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.589+0000 7f69e3400640 1 -- 172.21.15.7:0/4228042670 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f69d8011460 con 0x7f69ec0b9440 2024-09-10T21:44:34.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.590+0000 7f69e3400640 1 --2- 172.21.15.7:0/4228042670 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f69cc077f70 0x7f69cc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:34.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.590+0000 7f69e3400640 1 -- 172.21.15.7:0/4228042670 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f69d809a4b0 con 0x7f69ec0b9440 2024-09-10T21:44:34.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.591+0000 7f69eb400640 1 --2- 172.21.15.7:0/4228042670 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f69cc077f70 0x7f69cc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:34.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.591+0000 7f69eb400640 1 --2- 172.21.15.7:0/4228042670 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f69cc077f70 0x7f69cc07a430 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f69dc032000 tx=0x7f69dc001190 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:34.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.594+0000 7f69e3400640 1 -- 172.21.15.7:0/4228042670 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f69d8062f10 con 0x7f69ec0b9440 2024-09-10T21:44:34.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.829+0000 7f69f0c00640 1 -- 172.21.15.7:0/4228042670 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69ec0b9cd0 con 0x7f69ec0b9440 2024-09-10T21:44:34.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.829+0000 7f69e3400640 1 -- 172.21.15.7:0/4228042670 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5227 (secure 0 0 0) 0x7f69d8062810 con 0x7f69ec0b9440 2024-09-10T21:44:34.831 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:34.831 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"btime":"2024-09-10T21:44:34:389735+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5473,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/2853074495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2853074495},{"type":"v1","addr":"172.21.15.133:6836","nonce":2853074495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3498706462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3498706462},{"type":"v1","addr":"172.21.15.133:6837","nonce":3498706462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/521190705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":521190705},{"type":"v1","addr":"172.21.15.7:6835","nonce":521190705}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:44:30.804578+0000","modified":"2024-09-10T21:44:33.821692+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235016},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235016":{"gid":235016,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6834/364473218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":364473218},{"type":"v1","addr":"172.21.15.7:6834","nonce":364473218}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235016,"qdb_cluster":[235016]},"id":4}]} 2024-09-10T21:44:34.831 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 35 2024-09-10T21:44:34.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.832+0000 7f69f0c00640 1 -- 172.21.15.7:0/4228042670 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f69cc077f70 msgr2=0x7f69cc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.832+0000 7f69f0c00640 1 --2- 172.21.15.7:0/4228042670 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f69cc077f70 0x7f69cc07a430 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f69dc032000 tx=0x7f69dc001190 comp rx=0 tx=0).stop 2024-09-10T21:44:34.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.832+0000 7f69f0c00640 1 -- 172.21.15.7:0/4228042670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69ec0b9440 msgr2=0x7f69ec0ca720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.832+0000 7f69f0c00640 1 --2- 172.21.15.7:0/4228042670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69ec0b9440 0x7f69ec0ca720 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f69d800e570 tx=0x7f69d800ea30 comp rx=0 tx=0).stop 2024-09-10T21:44:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.832+0000 7f69f0c00640 1 -- 172.21.15.7:0/4228042670 shutdown_connections 2024-09-10T21:44:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.832+0000 7f69f0c00640 1 --2- 172.21.15.7:0/4228042670 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f69cc077f70 0x7f69cc07a430 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.832+0000 7f69f0c00640 1 --2- 172.21.15.7:0/4228042670 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f69ec1f8280 0x7f69ec1f8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.832+0000 7f69f0c00640 1 --2- 172.21.15.7:0/4228042670 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f69ec0b9440 0x7f69ec0ca720 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.832+0000 7f69f0c00640 1 --2- 172.21.15.7:0/4228042670 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f69d0003a30 0x7f69ec14cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.832+0000 7f69f0c00640 1 -- 172.21.15.7:0/4228042670 >> 172.21.15.7:0/4228042670 conn(0x7f69ec0d3d50 msgr2=0x7f69ec0b1aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.832+0000 7f69f0c00640 1 -- 172.21.15.7:0/4228042670 shutdown_connections 2024-09-10T21:44:34.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:34.832+0000 7f69f0c00640 1 -- 172.21.15.7:0/4228042670 wait complete. 2024-09-10T21:44:34.849 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:44:35.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.008+0000 7f1058c00640 1 -- 172.21.15.7:0/1597961736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f105415bc90 msgr2=0x7f10540ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.008+0000 7f1058c00640 1 --2- 172.21.15.7:0/1597961736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f105415bc90 0x7f10540ccef0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1044009830 tx=0x7f104402a3f0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.008+0000 7f1058c00640 1 -- 172.21.15.7:0/1597961736 shutdown_connections 2024-09-10T21:44:35.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.008+0000 7f1058c00640 1 --2- 172.21.15.7:0/1597961736 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f10540cd750 0x7f1054073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.008+0000 7f1058c00640 1 --2- 172.21.15.7:0/1597961736 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f105415bc90 0x7f10540ccef0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.008+0000 7f1058c00640 1 -- 172.21.15.7:0/1597961736 >> 172.21.15.7:0/1597961736 conn(0x7f10540b16d0 msgr2=0x7f10540b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:35.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.008+0000 7f1058c00640 1 -- 172.21.15.7:0/1597961736 shutdown_connections 2024-09-10T21:44:35.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.008+0000 7f1058c00640 1 -- 172.21.15.7:0/1597961736 wait complete. 2024-09-10T21:44:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.010+0000 7f1058c00640 1 Processor -- start 2024-09-10T21:44:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.010+0000 7f1058c00640 1 -- start start 2024-09-10T21:44:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.011+0000 7f1058c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1054073c70 0x7f10540c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.011+0000 7f1058c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10540cd750 0x7f10541f5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.011+0000 7f1058c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f105415bc90 0x7f10540c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.011+0000 7f1058c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f105406fcf0 con 0x7f1054073c70 2024-09-10T21:44:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.011+0000 7f1058c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f10541606c0 con 0x7f10540cd750 2024-09-10T21:44:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.011+0000 7f1058c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f105406fe70 con 0x7f105415bc90 2024-09-10T21:44:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.011+0000 7f1051e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1054073c70 0x7f10540c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:35.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.011+0000 7f1051e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1054073c70 0x7f10540c8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43458/0 (socket says 172.21.15.7:43458) 2024-09-10T21:44:35.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.011+0000 7f1051e00640 1 -- 172.21.15.7:0/2402601837 learned_addr learned my addr 172.21.15.7:0/2402601837 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:35.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.012+0000 7f1051e00640 1 -- 172.21.15.7:0/2402601837 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f105415bc90 msgr2=0x7f10540c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.012+0000 7f1051e00640 1 --2- 172.21.15.7:0/2402601837 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f105415bc90 0x7f10540c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.012+0000 7f1051e00640 1 -- 172.21.15.7:0/2402601837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10540cd750 msgr2=0x7f10541f5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.012+0000 7f1051e00640 1 --2- 172.21.15.7:0/2402601837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10540cd750 0x7f10541f5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.012+0000 7f1051e00640 1 -- 172.21.15.7:0/2402601837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10540c71a0 con 0x7f1054073c70 2024-09-10T21:44:35.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.012+0000 7f1051e00640 1 --2- 172.21.15.7:0/2402601837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1054073c70 0x7f10540c8240 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1044009950 tx=0x7f1044033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:35.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.012+0000 7f104a000640 1 -- 172.21.15.7:0/2402601837 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044005c90 con 0x7f1054073c70 2024-09-10T21:44:35.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.012+0000 7f104a000640 1 -- 172.21.15.7:0/2402601837 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1044031e50 con 0x7f1054073c70 2024-09-10T21:44:35.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.012+0000 7f104a000640 1 -- 172.21.15.7:0/2402601837 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1044005260 con 0x7f1054073c70 2024-09-10T21:44:35.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.012+0000 7f1058c00640 1 -- 172.21.15.7:0/2402601837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10540c7480 con 0x7f1054073c70 2024-09-10T21:44:35.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.012+0000 7f1058c00640 1 -- 172.21.15.7:0/2402601837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10540c7a40 con 0x7f1054073c70 2024-09-10T21:44:35.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.014+0000 7f1058c00640 1 -- 172.21.15.7:0/2402601837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1054073730 con 0x7f1054073c70 2024-09-10T21:44:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.015+0000 7f104a000640 1 -- 172.21.15.7:0/2402601837 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1044005400 con 0x7f1054073c70 2024-09-10T21:44:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.016+0000 7f104a000640 1 --2- 172.21.15.7:0/2402601837 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1034077f70 0x7f103407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.016+0000 7f104a000640 1 -- 172.21.15.7:0/2402601837 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f10440bad60 con 0x7f1054073c70 2024-09-10T21:44:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.016+0000 7f1051400640 1 --2- 172.21.15.7:0/2402601837 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1034077f70 0x7f103407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:35.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.017+0000 7f1051400640 1 --2- 172.21.15.7:0/2402601837 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1034077f70 0x7f103407a430 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f10400058c0 tx=0x7f1040005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:35.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.020+0000 7f104a000640 1 -- 172.21.15.7:0/2402601837 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f104403f030 con 0x7f1054073c70 2024-09-10T21:44:35.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.214+0000 7f1058c00640 1 -- 172.21.15.7:0/2402601837 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10540b1fa0 con 0x7f1054073c70 2024-09-10T21:44:35.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.216+0000 7f104a000640 1 -- 172.21.15.7:0/2402601837 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16689 (secure 0 0 0) 0x7f10440837c0 con 0x7f1054073c70 2024-09-10T21:44:35.217 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:35.217 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":58,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:44:32.373354+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:44:28.642474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T21:44:29.765081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:44:35.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.218+0000 7f1058c00640 1 -- 172.21.15.7:0/2402601837 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1034077f70 msgr2=0x7f103407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.218+0000 7f1058c00640 1 --2- 172.21.15.7:0/2402601837 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1034077f70 0x7f103407a430 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f10400058c0 tx=0x7f1040005850 comp rx=0 tx=0).stop 2024-09-10T21:44:35.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.218+0000 7f1058c00640 1 -- 172.21.15.7:0/2402601837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1054073c70 msgr2=0x7f10540c8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.218+0000 7f1058c00640 1 --2- 172.21.15.7:0/2402601837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1054073c70 0x7f10540c8240 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1044009950 tx=0x7f1044033c20 comp rx=0 tx=0).stop 2024-09-10T21:44:35.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.218+0000 7f1058c00640 1 -- 172.21.15.7:0/2402601837 shutdown_connections 2024-09-10T21:44:35.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.218+0000 7f1058c00640 1 --2- 172.21.15.7:0/2402601837 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1034077f70 0x7f103407a430 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.218+0000 7f1058c00640 1 --2- 172.21.15.7:0/2402601837 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f10540cd750 0x7f10541f5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.218+0000 7f1058c00640 1 --2- 172.21.15.7:0/2402601837 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f105415bc90 0x7f10540c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.218+0000 7f1058c00640 1 --2- 172.21.15.7:0/2402601837 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1054073c70 0x7f10540c8240 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.218+0000 7f1058c00640 1 -- 172.21.15.7:0/2402601837 >> 172.21.15.7:0/2402601837 conn(0x7f10540b16d0 msgr2=0x7f105415b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:35.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.218+0000 7f1058c00640 1 -- 172.21.15.7:0/2402601837 shutdown_connections 2024-09-10T21:44:35.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.218+0000 7f1058c00640 1 -- 172.21.15.7:0/2402601837 wait complete. 2024-09-10T21:44:35.235 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:44:35.235 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:44:35.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.398+0000 7f3124a00640 1 -- 172.21.15.7:0/2116846757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3104004d40 msgr2=0x7f31040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.398+0000 7f3124a00640 1 --2- 172.21.15.7:0/2116846757 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3104004d40 0x7f31040051c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f3118009f20 tx=0x7f3104005830 comp rx=0 tx=0).stop 2024-09-10T21:44:35.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.398+0000 7f311e800640 1 -- 172.21.15.7:0/2116846757 reap_dead start 2024-09-10T21:44:35.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.398+0000 7f3124a00640 1 -- 172.21.15.7:0/2116846757 shutdown_connections 2024-09-10T21:44:35.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.398+0000 7f3124a00640 1 -- 172.21.15.7:0/2116846757 >> 172.21.15.7:0/2116846757 conn(0x7f31200b16d0 msgr2=0x7f31200b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:35.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.398+0000 7f3124a00640 1 -- 172.21.15.7:0/2116846757 shutdown_connections 2024-09-10T21:44:35.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.398+0000 7f3124a00640 1 -- 172.21.15.7:0/2116846757 wait complete. 2024-09-10T21:44:35.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.400+0000 7f3124a00640 1 Processor -- start 2024-09-10T21:44:35.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.400+0000 7f3124a00640 1 -- start start 2024-09-10T21:44:35.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.401+0000 7f3124a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3104004d40 0x7f31201f2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.401+0000 7f3124a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31200a94e0 0x7f31200a9940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.401+0000 7f3124a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31201f5d80 0x7f31201f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.401+0000 7f3124a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f31200cd070 con 0x7f31201f5d80 2024-09-10T21:44:35.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.401+0000 7f3124a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f31200ccef0 con 0x7f3104004d40 2024-09-10T21:44:35.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.401+0000 7f3124a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f31200cd1f0 con 0x7f31200a94e0 2024-09-10T21:44:35.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.401+0000 7f311e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31201f5d80 0x7f31201f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:35.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.401+0000 7f311e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31201f5d80 0x7f31201f61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43478/0 (socket says 172.21.15.7:43478) 2024-09-10T21:44:35.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.401+0000 7f311e800640 1 -- 172.21.15.7:0/701935695 learned_addr learned my addr 172.21.15.7:0/701935695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:35.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.401+0000 7f311de00640 1 --2- 172.21.15.7:0/701935695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3104004d40 0x7f31201f2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:35.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.401+0000 7f3117e00640 1 --2- 172.21.15.7:0/701935695 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31200a94e0 0x7f31200a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:35.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.401+0000 7f311e800640 1 -- 172.21.15.7:0/701935695 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31200a94e0 msgr2=0x7f31200a9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.401+0000 7f311e800640 1 --2- 172.21.15.7:0/701935695 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31200a94e0 0x7f31200a9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.401+0000 7f311e800640 1 -- 172.21.15.7:0/701935695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3104004d40 msgr2=0x7f31201f2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.401+0000 7f311e800640 1 --2- 172.21.15.7:0/701935695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3104004d40 0x7f31201f2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.402+0000 7f311e800640 1 -- 172.21.15.7:0/701935695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31200c7fd0 con 0x7f31201f5d80 2024-09-10T21:44:35.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.402+0000 7f3117e00640 1 --2- 172.21.15.7:0/701935695 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31200a94e0 0x7f31200a9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:35.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.402+0000 7f311e800640 1 --2- 172.21.15.7:0/701935695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31201f5d80 0x7f31201f61e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3118009f20 tx=0x7f3118003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:35.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.402+0000 7f3116000640 1 -- 172.21.15.7:0/701935695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118031040 con 0x7f31201f5d80 2024-09-10T21:44:35.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.402+0000 7f3116000640 1 -- 172.21.15.7:0/701935695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3118004b20 con 0x7f31201f5d80 2024-09-10T21:44:35.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.402+0000 7f3116000640 1 -- 172.21.15.7:0/701935695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3118035610 con 0x7f31201f5d80 2024-09-10T21:44:35.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.402+0000 7f3124a00640 1 -- 172.21.15.7:0/701935695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31201f2e20 con 0x7f31201f5d80 2024-09-10T21:44:35.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.402+0000 7f3124a00640 1 -- 172.21.15.7:0/701935695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31201f33e0 con 0x7f31201f5d80 2024-09-10T21:44:35.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.404+0000 7f3124a00640 1 -- 172.21.15.7:0/701935695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31201601f0 con 0x7f31201f5d80 2024-09-10T21:44:35.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.404+0000 7f3116000640 1 -- 172.21.15.7:0/701935695 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f311803e070 con 0x7f31201f5d80 2024-09-10T21:44:35.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.405+0000 7f3116000640 1 --2- 172.21.15.7:0/701935695 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f30fc077f70 0x7f30fc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.405+0000 7f3116000640 1 -- 172.21.15.7:0/701935695 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f31180ba070 con 0x7f31201f5d80 2024-09-10T21:44:35.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.406+0000 7f311de00640 1 --2- 172.21.15.7:0/701935695 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f30fc077f70 0x7f30fc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:35.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.406+0000 7f311de00640 1 --2- 172.21.15.7:0/701935695 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f30fc077f70 0x7f30fc07a430 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3110006cd0 tx=0x7f3110000f50 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:35.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.410+0000 7f3116000640 1 -- 172.21.15.7:0/701935695 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3118082520 con 0x7f31201f5d80 2024-09-10T21:44:35.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.599+0000 7f3124a00640 1 -- 172.21.15.7:0/701935695 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f31200b1d90 con 0x7f30fc077f70 2024-09-10T21:44:35.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.605+0000 7f3116000640 1 -- 172.21.15.7:0/701935695 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251941 (secure 0 0 0) 0x7f31200b1d90 con 0x7f30fc077f70 2024-09-10T21:44:35.609 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:35.611 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:44:35.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.609+0000 7f3124a00640 1 -- 172.21.15.7:0/701935695 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f30fc077f70 msgr2=0x7f30fc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.609+0000 7f3124a00640 1 --2- 172.21.15.7:0/701935695 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f30fc077f70 0x7f30fc07a430 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f3110006cd0 tx=0x7f3110000f50 comp rx=0 tx=0).stop 2024-09-10T21:44:35.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.609+0000 7f3124a00640 1 -- 172.21.15.7:0/701935695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31201f5d80 msgr2=0x7f31201f61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.609+0000 7f3124a00640 1 --2- 172.21.15.7:0/701935695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31201f5d80 0x7f31201f61e0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3118009f20 tx=0x7f3118003ba0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.609+0000 7f3124a00640 1 -- 172.21.15.7:0/701935695 shutdown_connections 2024-09-10T21:44:35.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.609+0000 7f3124a00640 1 --2- 172.21.15.7:0/701935695 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f30fc077f70 0x7f30fc07a430 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.609+0000 7f3124a00640 1 --2- 172.21.15.7:0/701935695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f31201f5d80 0x7f31201f61e0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.609+0000 7f3124a00640 1 --2- 172.21.15.7:0/701935695 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f31200a94e0 0x7f31200a9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.609+0000 7f3124a00640 1 --2- 172.21.15.7:0/701935695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3104004d40 0x7f31201f2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.609+0000 7f3124a00640 1 -- 172.21.15.7:0/701935695 >> 172.21.15.7:0/701935695 conn(0x7f31200b16d0 msgr2=0x7f31200b45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:35.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.609+0000 7f3124a00640 1 -- 172.21.15.7:0/701935695 shutdown_connections 2024-09-10T21:44:35.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.609+0000 7f3124a00640 1 -- 172.21.15.7:0/701935695 wait complete. 2024-09-10T21:44:35.626 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":130,"stamp":"2024-09-10T21:44:33.891539+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247700,"kb_used_data":6476,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727020,"statfs":{"total":773094113280,"available":772840468480,"internally_reserved":0,"allocated":6631424,"data_stored":3404573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.536131"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.403629+0000","last_change":"2024-09-10T21:44:31.399608+0000","last_active":"2024-09-10T21:44:31.403629+0000","last_peered":"2024-09-10T21:44:31.403629+0000","last_clean":"2024-09-10T21:44:31.403629+0000","last_became_active":"2024-09-10T21:44:31.398957+0000","last_became_peered":"2024-09-10T21:44:31.398957+0000","last_unstale":"2024-09-10T21:44:31.403629+0000","last_undegraded":"2024-09-10T21:44:31.403629+0000","last_fullsized":"2024-09-10T21:44:31.403629+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:27.896951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402741+0000","last_change":"2024-09-10T21:44:31.397977+0000","last_active":"2024-09-10T21:44:31.402741+0000","last_peered":"2024-09-10T21:44:31.402741+0000","last_clean":"2024-09-10T21:44:31.402741+0000","last_became_active":"2024-09-10T21:44:31.396697+0000","last_became_peered":"2024-09-10T21:44:31.396697+0000","last_unstale":"2024-09-10T21:44:31.402741+0000","last_undegraded":"2024-09-10T21:44:31.402741+0000","last_fullsized":"2024-09-10T21:44:31.402741+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:10.772875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.383181+0000","last_change":"2024-09-10T21:44:31.379067+0000","last_active":"2024-09-10T21:44:31.383181+0000","last_peered":"2024-09-10T21:44:31.383181+0000","last_clean":"2024-09-10T21:44:31.383181+0000","last_became_active":"2024-09-10T21:44:31.378736+0000","last_became_peered":"2024-09-10T21:44:31.378736+0000","last_unstale":"2024-09-10T21:44:31.383181+0000","last_undegraded":"2024-09-10T21:44:31.383181+0000","last_fullsized":"2024-09-10T21:44:31.383181+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:12.839190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385464+0000","last_change":"2024-09-10T21:44:31.392160+0000","last_active":"2024-09-10T21:44:32.385464+0000","last_peered":"2024-09-10T21:44:32.385464+0000","last_clean":"2024-09-10T21:44:32.385464+0000","last_became_active":"2024-09-10T21:44:31.391547+0000","last_became_peered":"2024-09-10T21:44:31.391547+0000","last_unstale":"2024-09-10T21:44:32.385464+0000","last_undegraded":"2024-09-10T21:44:32.385464+0000","last_fullsized":"2024-09-10T21:44:32.385464+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:12.308573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.398687+0000","last_change":"2024-09-10T21:44:31.385084+0000","last_active":"2024-09-10T21:44:31.398687+0000","last_peered":"2024-09-10T21:44:31.398687+0000","last_clean":"2024-09-10T21:44:31.398687+0000","last_became_active":"2024-09-10T21:44:31.383699+0000","last_became_peered":"2024-09-10T21:44:31.383699+0000","last_unstale":"2024-09-10T21:44:31.398687+0000","last_undegraded":"2024-09-10T21:44:31.398687+0000","last_fullsized":"2024-09-10T21:44:31.398687+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:37.590428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.405725+0000","last_change":"2024-09-10T21:44:31.395955+0000","last_active":"2024-09-10T21:44:31.405725+0000","last_peered":"2024-09-10T21:44:31.405725+0000","last_clean":"2024-09-10T21:44:31.405725+0000","last_became_active":"2024-09-10T21:44:31.391364+0000","last_became_peered":"2024-09-10T21:44:31.391364+0000","last_unstale":"2024-09-10T21:44:31.405725+0000","last_undegraded":"2024-09-10T21:44:31.405725+0000","last_fullsized":"2024-09-10T21:44:31.405725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:30.274226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385693+0000","last_change":"2024-09-10T21:44:31.387224+0000","last_active":"2024-09-10T21:44:32.385693+0000","last_peered":"2024-09-10T21:44:32.385693+0000","last_clean":"2024-09-10T21:44:32.385693+0000","last_became_active":"2024-09-10T21:44:31.386953+0000","last_became_peered":"2024-09-10T21:44:31.386953+0000","last_unstale":"2024-09-10T21:44:32.385693+0000","last_undegraded":"2024-09-10T21:44:32.385693+0000","last_fullsized":"2024-09-10T21:44:32.385693+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:17.893343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.389027+0000","last_change":"2024-09-10T21:44:31.379853+0000","last_active":"2024-09-10T21:44:31.389027+0000","last_peered":"2024-09-10T21:44:31.389027+0000","last_clean":"2024-09-10T21:44:31.389027+0000","last_became_active":"2024-09-10T21:44:31.379562+0000","last_became_peered":"2024-09-10T21:44:31.379562+0000","last_unstale":"2024-09-10T21:44:31.389027+0000","last_undegraded":"2024-09-10T21:44:31.389027+0000","last_fullsized":"2024-09-10T21:44:31.389027+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:47.953031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.387147+0000","last_change":"2024-09-10T21:44:31.394796+0000","last_active":"2024-09-10T21:44:32.387147+0000","last_peered":"2024-09-10T21:44:32.387147+0000","last_clean":"2024-09-10T21:44:32.387147+0000","last_became_active":"2024-09-10T21:44:31.394570+0000","last_became_peered":"2024-09-10T21:44:31.394570+0000","last_unstale":"2024-09-10T21:44:32.387147+0000","last_undegraded":"2024-09-10T21:44:32.387147+0000","last_fullsized":"2024-09-10T21:44:32.387147+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:54.134546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.401866+0000","last_change":"2024-09-10T21:44:31.399373+0000","last_active":"2024-09-10T21:44:31.401866+0000","last_peered":"2024-09-10T21:44:31.401866+0000","last_clean":"2024-09-10T21:44:31.401866+0000","last_became_active":"2024-09-10T21:44:31.399148+0000","last_became_peered":"2024-09-10T21:44:31.399148+0000","last_unstale":"2024-09-10T21:44:31.401866+0000","last_undegraded":"2024-09-10T21:44:31.401866+0000","last_fullsized":"2024-09-10T21:44:31.401866+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:31.171030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.400029+0000","last_change":"2024-09-10T21:44:31.394283+0000","last_active":"2024-09-10T21:44:31.400029+0000","last_peered":"2024-09-10T21:44:31.400029+0000","last_clean":"2024-09-10T21:44:31.400029+0000","last_became_active":"2024-09-10T21:44:31.393985+0000","last_became_peered":"2024-09-10T21:44:31.393985+0000","last_unstale":"2024-09-10T21:44:31.400029+0000","last_undegraded":"2024-09-10T21:44:31.400029+0000","last_fullsized":"2024-09-10T21:44:31.400029+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:13.290552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385752+0000","last_change":"2024-09-10T21:44:31.379418+0000","last_active":"2024-09-10T21:44:32.385752+0000","last_peered":"2024-09-10T21:44:32.385752+0000","last_clean":"2024-09-10T21:44:32.385752+0000","last_became_active":"2024-09-10T21:44:31.377166+0000","last_became_peered":"2024-09-10T21:44:31.377166+0000","last_unstale":"2024-09-10T21:44:32.385752+0000","last_undegraded":"2024-09-10T21:44:32.385752+0000","last_fullsized":"2024-09-10T21:44:32.385752+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:44.243658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.619900+0000","last_change":"2024-09-10T21:44:31.392208+0000","last_active":"2024-09-10T21:44:32.619900+0000","last_peered":"2024-09-10T21:44:32.619900+0000","last_clean":"2024-09-10T21:44:32.619900+0000","last_became_active":"2024-09-10T21:44:31.391908+0000","last_became_peered":"2024-09-10T21:44:31.391908+0000","last_unstale":"2024-09-10T21:44:32.619900+0000","last_undegraded":"2024-09-10T21:44:32.619900+0000","last_fullsized":"2024-09-10T21:44:32.619900+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:06.505235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.386331+0000","last_change":"2024-09-10T21:44:31.385438+0000","last_active":"2024-09-10T21:44:32.386331+0000","last_peered":"2024-09-10T21:44:32.386331+0000","last_clean":"2024-09-10T21:44:32.386331+0000","last_became_active":"2024-09-10T21:44:31.385171+0000","last_became_peered":"2024-09-10T21:44:31.385171+0000","last_unstale":"2024-09-10T21:44:32.386331+0000","last_undegraded":"2024-09-10T21:44:32.386331+0000","last_fullsized":"2024-09-10T21:44:32.386331+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:36.018237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385953+0000","last_change":"2024-09-10T21:44:31.382606+0000","last_active":"2024-09-10T21:44:32.385953+0000","last_peered":"2024-09-10T21:44:32.385953+0000","last_clean":"2024-09-10T21:44:32.385953+0000","last_became_active":"2024-09-10T21:44:31.382173+0000","last_became_peered":"2024-09-10T21:44:31.382173+0000","last_unstale":"2024-09-10T21:44:32.385953+0000","last_undegraded":"2024-09-10T21:44:32.385953+0000","last_fullsized":"2024-09-10T21:44:32.385953+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:43.033843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.395769+0000","last_change":"2024-09-10T21:44:31.383272+0000","last_active":"2024-09-10T21:44:31.395769+0000","last_peered":"2024-09-10T21:44:31.395769+0000","last_clean":"2024-09-10T21:44:31.395769+0000","last_became_active":"2024-09-10T21:44:31.382803+0000","last_became_peered":"2024-09-10T21:44:31.382803+0000","last_unstale":"2024-09-10T21:44:31.395769+0000","last_undegraded":"2024-09-10T21:44:31.395769+0000","last_fullsized":"2024-09-10T21:44:31.395769+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:40.861104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.392267+0000","last_change":"2024-09-10T21:44:31.381958+0000","last_active":"2024-09-10T21:44:31.392267+0000","last_peered":"2024-09-10T21:44:31.392267+0000","last_clean":"2024-09-10T21:44:31.392267+0000","last_became_active":"2024-09-10T21:44:31.381540+0000","last_became_peered":"2024-09-10T21:44:31.381540+0000","last_unstale":"2024-09-10T21:44:31.392267+0000","last_undegraded":"2024-09-10T21:44:31.392267+0000","last_fullsized":"2024-09-10T21:44:31.392267+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:35.874957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.397453+0000","last_change":"2024-09-10T21:44:31.394512+0000","last_active":"2024-09-10T21:44:31.397453+0000","last_peered":"2024-09-10T21:44:31.397453+0000","last_clean":"2024-09-10T21:44:31.397453+0000","last_became_active":"2024-09-10T21:44:31.394250+0000","last_became_peered":"2024-09-10T21:44:31.394250+0000","last_unstale":"2024-09-10T21:44:31.397453+0000","last_undegraded":"2024-09-10T21:44:31.397453+0000","last_fullsized":"2024-09-10T21:44:31.397453+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:26.744558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.404955+0000","last_change":"2024-09-10T21:44:31.400156+0000","last_active":"2024-09-10T21:44:31.404955+0000","last_peered":"2024-09-10T21:44:31.404955+0000","last_clean":"2024-09-10T21:44:31.404955+0000","last_became_active":"2024-09-10T21:44:31.397539+0000","last_became_peered":"2024-09-10T21:44:31.397539+0000","last_unstale":"2024-09-10T21:44:31.404955+0000","last_undegraded":"2024-09-10T21:44:31.404955+0000","last_fullsized":"2024-09-10T21:44:31.404955+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:16.531319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.398582+0000","last_change":"2024-09-10T21:44:31.392546+0000","last_active":"2024-09-10T21:44:31.398582+0000","last_peered":"2024-09-10T21:44:31.398582+0000","last_clean":"2024-09-10T21:44:31.398582+0000","last_became_active":"2024-09-10T21:44:31.392282+0000","last_became_peered":"2024-09-10T21:44:31.392282+0000","last_unstale":"2024-09-10T21:44:31.398582+0000","last_undegraded":"2024-09-10T21:44:31.398582+0000","last_fullsized":"2024-09-10T21:44:31.398582+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:55.796826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.397884+0000","last_change":"2024-09-10T21:44:31.392174+0000","last_active":"2024-09-10T21:44:31.397884+0000","last_peered":"2024-09-10T21:44:31.397884+0000","last_clean":"2024-09-10T21:44:31.397884+0000","last_became_active":"2024-09-10T21:44:31.390826+0000","last_became_peered":"2024-09-10T21:44:31.390826+0000","last_unstale":"2024-09-10T21:44:31.397884+0000","last_undegraded":"2024-09-10T21:44:31.397884+0000","last_fullsized":"2024-09-10T21:44:31.397884+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:35.777298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.401319+0000","last_change":"2024-09-10T21:44:31.396513+0000","last_active":"2024-09-10T21:44:31.401319+0000","last_peered":"2024-09-10T21:44:31.401319+0000","last_clean":"2024-09-10T21:44:31.401319+0000","last_became_active":"2024-09-10T21:44:31.396270+0000","last_became_peered":"2024-09-10T21:44:31.396270+0000","last_unstale":"2024-09-10T21:44:31.401319+0000","last_undegraded":"2024-09-10T21:44:31.401319+0000","last_fullsized":"2024-09-10T21:44:31.401319+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:45.937921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402540+0000","last_change":"2024-09-10T21:44:31.397864+0000","last_active":"2024-09-10T21:44:31.402540+0000","last_peered":"2024-09-10T21:44:31.402540+0000","last_clean":"2024-09-10T21:44:31.402540+0000","last_became_active":"2024-09-10T21:44:31.397544+0000","last_became_peered":"2024-09-10T21:44:31.397544+0000","last_unstale":"2024-09-10T21:44:31.402540+0000","last_undegraded":"2024-09-10T21:44:31.402540+0000","last_fullsized":"2024-09-10T21:44:31.402540+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:01:01.598080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.619779+0000","last_change":"2024-09-10T21:44:31.391322+0000","last_active":"2024-09-10T21:44:32.619779+0000","last_peered":"2024-09-10T21:44:32.619779+0000","last_clean":"2024-09-10T21:44:32.619779+0000","last_became_active":"2024-09-10T21:44:31.389549+0000","last_became_peered":"2024-09-10T21:44:31.389549+0000","last_unstale":"2024-09-10T21:44:32.619779+0000","last_undegraded":"2024-09-10T21:44:32.619779+0000","last_fullsized":"2024-09-10T21:44:32.619779+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:31.171030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.403654+0000","last_change":"2024-09-10T21:44:31.399930+0000","last_active":"2024-09-10T21:44:31.403654+0000","last_peered":"2024-09-10T21:44:31.403654+0000","last_clean":"2024-09-10T21:44:31.403654+0000","last_became_active":"2024-09-10T21:44:31.399357+0000","last_became_peered":"2024-09-10T21:44:31.399357+0000","last_unstale":"2024-09-10T21:44:31.403654+0000","last_undegraded":"2024-09-10T21:44:31.403654+0000","last_fullsized":"2024-09-10T21:44:31.403654+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:53.455429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385655+0000","last_change":"2024-09-10T21:44:31.385008+0000","last_active":"2024-09-10T21:44:32.385655+0000","last_peered":"2024-09-10T21:44:32.385655+0000","last_clean":"2024-09-10T21:44:32.385655+0000","last_became_active":"2024-09-10T21:44:31.384750+0000","last_became_peered":"2024-09-10T21:44:31.384750+0000","last_unstale":"2024-09-10T21:44:32.385655+0000","last_undegraded":"2024-09-10T21:44:32.385655+0000","last_fullsized":"2024-09-10T21:44:32.385655+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:22.863091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.403100+0000","last_change":"2024-09-10T21:44:31.394310+0000","last_active":"2024-09-10T21:44:31.403100+0000","last_peered":"2024-09-10T21:44:31.403100+0000","last_clean":"2024-09-10T21:44:31.403100+0000","last_became_active":"2024-09-10T21:44:31.394029+0000","last_became_peered":"2024-09-10T21:44:31.394029+0000","last_unstale":"2024-09-10T21:44:31.403100+0000","last_undegraded":"2024-09-10T21:44:31.403100+0000","last_fullsized":"2024-09-10T21:44:31.403100+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:16.952137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402325+0000","last_change":"2024-09-10T21:44:31.395667+0000","last_active":"2024-09-10T21:44:31.402325+0000","last_peered":"2024-09-10T21:44:31.402325+0000","last_clean":"2024-09-10T21:44:31.402325+0000","last_became_active":"2024-09-10T21:44:31.390131+0000","last_became_peered":"2024-09-10T21:44:31.390131+0000","last_unstale":"2024-09-10T21:44:31.402325+0000","last_undegraded":"2024-09-10T21:44:31.402325+0000","last_fullsized":"2024-09-10T21:44:31.402325+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:37.199923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.398612+0000","last_change":"2024-09-10T21:44:31.390024+0000","last_active":"2024-09-10T21:44:31.398612+0000","last_peered":"2024-09-10T21:44:31.398612+0000","last_clean":"2024-09-10T21:44:31.398612+0000","last_became_active":"2024-09-10T21:44:31.389718+0000","last_became_peered":"2024-09-10T21:44:31.389718+0000","last_unstale":"2024-09-10T21:44:31.398612+0000","last_undegraded":"2024-09-10T21:44:31.398612+0000","last_fullsized":"2024-09-10T21:44:31.398612+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:26.016478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.383038+0000","last_change":"2024-09-10T21:44:31.380107+0000","last_active":"2024-09-10T21:44:31.383038+0000","last_peered":"2024-09-10T21:44:31.383038+0000","last_clean":"2024-09-10T21:44:31.383038+0000","last_became_active":"2024-09-10T21:44:31.379814+0000","last_became_peered":"2024-09-10T21:44:31.379814+0000","last_unstale":"2024-09-10T21:44:31.383038+0000","last_undegraded":"2024-09-10T21:44:31.383038+0000","last_fullsized":"2024-09-10T21:44:31.383038+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:18.902765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.391857+0000","last_change":"2024-09-10T21:44:31.381086+0000","last_active":"2024-09-10T21:44:31.391857+0000","last_peered":"2024-09-10T21:44:31.391857+0000","last_clean":"2024-09-10T21:44:31.391857+0000","last_became_active":"2024-09-10T21:44:31.380668+0000","last_became_peered":"2024-09-10T21:44:31.380668+0000","last_unstale":"2024-09-10T21:44:31.391857+0000","last_undegraded":"2024-09-10T21:44:31.391857+0000","last_fullsized":"2024-09-10T21:44:31.391857+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:15.989838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.403238+0000","last_change":"2024-09-10T21:44:31.393582+0000","last_active":"2024-09-10T21:44:31.403238+0000","last_peered":"2024-09-10T21:44:31.403238+0000","last_clean":"2024-09-10T21:44:31.403238+0000","last_became_active":"2024-09-10T21:44:31.390027+0000","last_became_peered":"2024-09-10T21:44:31.390027+0000","last_unstale":"2024-09-10T21:44:31.403238+0000","last_undegraded":"2024-09-10T21:44:31.403238+0000","last_fullsized":"2024-09-10T21:44:31.403238+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:22.058929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.406848+0000","last_change":"2024-09-10T21:44:31.398001+0000","last_active":"2024-09-10T21:44:31.406848+0000","last_peered":"2024-09-10T21:44:31.406848+0000","last_clean":"2024-09-10T21:44:31.406848+0000","last_became_active":"2024-09-10T21:44:31.391899+0000","last_became_peered":"2024-09-10T21:44:31.391899+0000","last_unstale":"2024-09-10T21:44:31.406848+0000","last_undegraded":"2024-09-10T21:44:31.406848+0000","last_fullsized":"2024-09-10T21:44:31.406848+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:12.308573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402797+0000","last_change":"2024-09-10T21:44:30.379105+0000","last_active":"2024-09-10T21:44:31.402797+0000","last_peered":"2024-09-10T21:44:31.402797+0000","last_clean":"2024-09-10T21:44:31.402797+0000","last_became_active":"2024-09-10T21:44:30.376451+0000","last_became_peered":"2024-09-10T21:44:30.376451+0000","last_unstale":"2024-09-10T21:44:31.402797+0000","last_undegraded":"2024-09-10T21:44:31.402797+0000","last_fullsized":"2024-09-10T21:44:31.402797+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:42.022890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.391608+0000","last_change":"2024-09-10T21:44:31.390579+0000","last_active":"2024-09-10T21:44:31.391608+0000","last_peered":"2024-09-10T21:44:31.391608+0000","last_clean":"2024-09-10T21:44:31.391608+0000","last_became_active":"2024-09-10T21:44:31.386874+0000","last_became_peered":"2024-09-10T21:44:31.386874+0000","last_unstale":"2024-09-10T21:44:31.391608+0000","last_undegraded":"2024-09-10T21:44:31.391608+0000","last_fullsized":"2024-09-10T21:44:31.391608+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:53.299991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.401671+0000","last_change":"2024-09-10T21:44:30.376223+0000","last_active":"2024-09-10T21:44:31.401671+0000","last_peered":"2024-09-10T21:44:31.401671+0000","last_clean":"2024-09-10T21:44:31.401671+0000","last_became_active":"2024-09-10T21:44:30.375951+0000","last_became_peered":"2024-09-10T21:44:30.375951+0000","last_unstale":"2024-09-10T21:44:31.401671+0000","last_undegraded":"2024-09-10T21:44:31.401671+0000","last_fullsized":"2024-09-10T21:44:31.401671+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:04.622616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402010+0000","last_change":"2024-09-10T21:44:31.393057+0000","last_active":"2024-09-10T21:44:31.402010+0000","last_peered":"2024-09-10T21:44:31.402010+0000","last_clean":"2024-09-10T21:44:31.402010+0000","last_became_active":"2024-09-10T21:44:31.389891+0000","last_became_peered":"2024-09-10T21:44:31.389891+0000","last_unstale":"2024-09-10T21:44:31.402010+0000","last_undegraded":"2024-09-10T21:44:31.402010+0000","last_fullsized":"2024-09-10T21:44:31.402010+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:01.862089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.408592+0000","last_change":"2024-09-10T21:44:30.374526+0000","last_active":"2024-09-10T21:44:31.408592+0000","last_peered":"2024-09-10T21:44:31.408592+0000","last_clean":"2024-09-10T21:44:31.408592+0000","last_became_active":"2024-09-10T21:44:30.374191+0000","last_became_peered":"2024-09-10T21:44:30.374191+0000","last_unstale":"2024-09-10T21:44:31.408592+0000","last_undegraded":"2024-09-10T21:44:31.408592+0000","last_fullsized":"2024-09-10T21:44:31.408592+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:16.882390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402727+0000","last_change":"2024-09-10T21:44:31.393655+0000","last_active":"2024-09-10T21:44:31.402727+0000","last_peered":"2024-09-10T21:44:31.402727+0000","last_clean":"2024-09-10T21:44:31.402727+0000","last_became_active":"2024-09-10T21:44:31.388571+0000","last_became_peered":"2024-09-10T21:44:31.388571+0000","last_unstale":"2024-09-10T21:44:31.402727+0000","last_undegraded":"2024-09-10T21:44:31.402727+0000","last_fullsized":"2024-09-10T21:44:31.402727+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:50.031278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385875+0000","last_change":"2024-09-10T21:44:30.376094+0000","last_active":"2024-09-10T21:44:32.385875+0000","last_peered":"2024-09-10T21:44:32.385875+0000","last_clean":"2024-09-10T21:44:32.385875+0000","last_became_active":"2024-09-10T21:44:30.375807+0000","last_became_peered":"2024-09-10T21:44:30.375807+0000","last_unstale":"2024-09-10T21:44:32.385875+0000","last_undegraded":"2024-09-10T21:44:32.385875+0000","last_fullsized":"2024-09-10T21:44:32.385875+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:29.278084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.401293+0000","last_change":"2024-09-10T21:44:31.395804+0000","last_active":"2024-09-10T21:44:31.401293+0000","last_peered":"2024-09-10T21:44:31.401293+0000","last_clean":"2024-09-10T21:44:31.401293+0000","last_became_active":"2024-09-10T21:44:31.395535+0000","last_became_peered":"2024-09-10T21:44:31.395535+0000","last_unstale":"2024-09-10T21:44:31.401293+0000","last_undegraded":"2024-09-10T21:44:31.401293+0000","last_fullsized":"2024-09-10T21:44:31.401293+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:34.062126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.386663+0000","last_change":"2024-09-10T21:44:30.380848+0000","last_active":"2024-09-10T21:44:32.386663+0000","last_peered":"2024-09-10T21:44:32.386663+0000","last_clean":"2024-09-10T21:44:32.386663+0000","last_became_active":"2024-09-10T21:44:30.380572+0000","last_became_peered":"2024-09-10T21:44:30.380572+0000","last_unstale":"2024-09-10T21:44:32.386663+0000","last_undegraded":"2024-09-10T21:44:32.386663+0000","last_fullsized":"2024-09-10T21:44:32.386663+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:37:33.245055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.398630+0000","last_change":"2024-09-10T21:44:31.394405+0000","last_active":"2024-09-10T21:44:31.398630+0000","last_peered":"2024-09-10T21:44:31.398630+0000","last_clean":"2024-09-10T21:44:31.398630+0000","last_became_active":"2024-09-10T21:44:31.394142+0000","last_became_peered":"2024-09-10T21:44:31.394142+0000","last_unstale":"2024-09-10T21:44:31.398630+0000","last_undegraded":"2024-09-10T21:44:31.398630+0000","last_fullsized":"2024-09-10T21:44:31.398630+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:04.427921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402707+0000","last_change":"2024-09-10T21:44:30.380087+0000","last_active":"2024-09-10T21:44:31.402707+0000","last_peered":"2024-09-10T21:44:31.402707+0000","last_clean":"2024-09-10T21:44:31.402707+0000","last_became_active":"2024-09-10T21:44:30.378757+0000","last_became_peered":"2024-09-10T21:44:30.378757+0000","last_unstale":"2024-09-10T21:44:31.402707+0000","last_undegraded":"2024-09-10T21:44:31.402707+0000","last_fullsized":"2024-09-10T21:44:31.402707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:45:14.854663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.619707+0000","last_change":"2024-09-10T21:44:31.375815+0000","last_active":"2024-09-10T21:44:32.619707+0000","last_peered":"2024-09-10T21:44:32.619707+0000","last_clean":"2024-09-10T21:44:32.619707+0000","last_became_active":"2024-09-10T21:44:31.375142+0000","last_became_peered":"2024-09-10T21:44:31.375142+0000","last_unstale":"2024-09-10T21:44:32.619707+0000","last_undegraded":"2024-09-10T21:44:32.619707+0000","last_fullsized":"2024-09-10T21:44:32.619707+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:32.072203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385116+0000","last_change":"2024-09-10T21:44:30.375706+0000","last_active":"2024-09-10T21:44:32.385116+0000","last_peered":"2024-09-10T21:44:32.385116+0000","last_clean":"2024-09-10T21:44:32.385116+0000","last_became_active":"2024-09-10T21:44:30.375436+0000","last_became_peered":"2024-09-10T21:44:30.375436+0000","last_unstale":"2024-09-10T21:44:32.385116+0000","last_undegraded":"2024-09-10T21:44:32.385116+0000","last_fullsized":"2024-09-10T21:44:32.385116+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:43.769901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.406643+0000","last_change":"2024-09-10T21:44:31.403557+0000","last_active":"2024-09-10T21:44:31.406643+0000","last_peered":"2024-09-10T21:44:31.406643+0000","last_clean":"2024-09-10T21:44:31.406643+0000","last_became_active":"2024-09-10T21:44:31.401444+0000","last_became_peered":"2024-09-10T21:44:31.401444+0000","last_unstale":"2024-09-10T21:44:31.406643+0000","last_undegraded":"2024-09-10T21:44:31.406643+0000","last_fullsized":"2024-09-10T21:44:31.406643+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:37:13.559247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385484+0000","last_change":"2024-09-10T21:44:30.373702+0000","last_active":"2024-09-10T21:44:32.385484+0000","last_peered":"2024-09-10T21:44:32.385484+0000","last_clean":"2024-09-10T21:44:32.385484+0000","last_became_active":"2024-09-10T21:44:30.373437+0000","last_became_peered":"2024-09-10T21:44:30.373437+0000","last_unstale":"2024-09-10T21:44:32.385484+0000","last_undegraded":"2024-09-10T21:44:32.385484+0000","last_fullsized":"2024-09-10T21:44:32.385484+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:21.917058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385598+0000","last_change":"2024-09-10T21:44:31.386834+0000","last_active":"2024-09-10T21:44:32.385598+0000","last_peered":"2024-09-10T21:44:32.385598+0000","last_clean":"2024-09-10T21:44:32.385598+0000","last_became_active":"2024-09-10T21:44:31.386245+0000","last_became_peered":"2024-09-10T21:44:31.386245+0000","last_unstale":"2024-09-10T21:44:32.385598+0000","last_undegraded":"2024-09-10T21:44:32.385598+0000","last_fullsized":"2024-09-10T21:44:32.385598+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:45:15.865616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.372075+0000","last_change":"2024-09-10T21:44:30.384555+0000","last_active":"2024-09-10T21:44:31.372075+0000","last_peered":"2024-09-10T21:44:31.372075+0000","last_clean":"2024-09-10T21:44:31.372075+0000","last_became_active":"2024-09-10T21:44:30.384278+0000","last_became_peered":"2024-09-10T21:44:30.384278+0000","last_unstale":"2024-09-10T21:44:31.372075+0000","last_undegraded":"2024-09-10T21:44:31.372075+0000","last_fullsized":"2024-09-10T21:44:31.372075+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:59.701329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.398052+0000","last_change":"2024-09-10T21:44:31.396217+0000","last_active":"2024-09-10T21:44:31.398052+0000","last_peered":"2024-09-10T21:44:31.398052+0000","last_clean":"2024-09-10T21:44:31.398052+0000","last_became_active":"2024-09-10T21:44:31.392774+0000","last_became_peered":"2024-09-10T21:44:31.392774+0000","last_unstale":"2024-09-10T21:44:31.398052+0000","last_undegraded":"2024-09-10T21:44:31.398052+0000","last_fullsized":"2024-09-10T21:44:31.398052+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:20.337480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.371370+0000","last_change":"2024-09-10T21:44:30.383267+0000","last_active":"2024-09-10T21:44:31.371370+0000","last_peered":"2024-09-10T21:44:31.371370+0000","last_clean":"2024-09-10T21:44:31.371370+0000","last_became_active":"2024-09-10T21:44:30.382042+0000","last_became_peered":"2024-09-10T21:44:30.382042+0000","last_unstale":"2024-09-10T21:44:31.371370+0000","last_undegraded":"2024-09-10T21:44:31.371370+0000","last_fullsized":"2024-09-10T21:44:31.371370+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:12:36.873002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.390496+0000","last_change":"2024-09-10T21:44:31.388118+0000","last_active":"2024-09-10T21:44:31.390496+0000","last_peered":"2024-09-10T21:44:31.390496+0000","last_clean":"2024-09-10T21:44:31.390496+0000","last_became_active":"2024-09-10T21:44:31.387812+0000","last_became_peered":"2024-09-10T21:44:31.387812+0000","last_unstale":"2024-09-10T21:44:31.390496+0000","last_undegraded":"2024-09-10T21:44:31.390496+0000","last_fullsized":"2024-09-10T21:44:31.390496+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:14.803952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.395594+0000","last_change":"2024-09-10T21:44:30.378538+0000","last_active":"2024-09-10T21:44:31.395594+0000","last_peered":"2024-09-10T21:44:31.395594+0000","last_clean":"2024-09-10T21:44:31.395594+0000","last_became_active":"2024-09-10T21:44:30.378249+0000","last_became_peered":"2024-09-10T21:44:30.378249+0000","last_unstale":"2024-09-10T21:44:31.395594+0000","last_undegraded":"2024-09-10T21:44:31.395594+0000","last_fullsized":"2024-09-10T21:44:31.395594+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:17.140823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.393898+0000","last_change":"2024-09-10T21:44:31.387175+0000","last_active":"2024-09-10T21:44:31.393898+0000","last_peered":"2024-09-10T21:44:31.393898+0000","last_clean":"2024-09-10T21:44:31.393898+0000","last_became_active":"2024-09-10T21:44:31.386133+0000","last_became_peered":"2024-09-10T21:44:31.386133+0000","last_unstale":"2024-09-10T21:44:31.393898+0000","last_undegraded":"2024-09-10T21:44:31.393898+0000","last_fullsized":"2024-09-10T21:44:31.393898+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:35.642650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":21,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.387075+0000","last_change":"2024-09-10T21:44:30.371690+0000","last_active":"2024-09-10T21:44:32.387075+0000","last_peered":"2024-09-10T21:44:32.387075+0000","last_clean":"2024-09-10T21:44:32.387075+0000","last_became_active":"2024-09-10T21:44:30.371413+0000","last_became_peered":"2024-09-10T21:44:30.371413+0000","last_unstale":"2024-09-10T21:44:32.387075+0000","last_undegraded":"2024-09-10T21:44:32.387075+0000","last_fullsized":"2024-09-10T21:44:32.387075+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:05.494281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385860+0000","last_change":"2024-09-10T21:44:31.387479+0000","last_active":"2024-09-10T21:44:32.385860+0000","last_peered":"2024-09-10T21:44:32.385860+0000","last_clean":"2024-09-10T21:44:32.385860+0000","last_became_active":"2024-09-10T21:44:31.386309+0000","last_became_peered":"2024-09-10T21:44:31.386309+0000","last_unstale":"2024-09-10T21:44:32.385860+0000","last_undegraded":"2024-09-10T21:44:32.385860+0000","last_fullsized":"2024-09-10T21:44:32.385860+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:49.049473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.404334+0000","last_change":"2024-09-10T21:44:30.384321+0000","last_active":"2024-09-10T21:44:31.404334+0000","last_peered":"2024-09-10T21:44:31.404334+0000","last_clean":"2024-09-10T21:44:31.404334+0000","last_became_active":"2024-09-10T21:44:30.384057+0000","last_became_peered":"2024-09-10T21:44:30.384057+0000","last_unstale":"2024-09-10T21:44:31.404334+0000","last_undegraded":"2024-09-10T21:44:31.404334+0000","last_fullsized":"2024-09-10T21:44:31.404334+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:32.514436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.385396+0000","last_change":"2024-09-10T21:44:31.384148+0000","last_active":"2024-09-10T21:44:31.385396+0000","last_peered":"2024-09-10T21:44:31.385396+0000","last_clean":"2024-09-10T21:44:31.385396+0000","last_became_active":"2024-09-10T21:44:31.383857+0000","last_became_peered":"2024-09-10T21:44:31.383857+0000","last_unstale":"2024-09-10T21:44:31.385396+0000","last_undegraded":"2024-09-10T21:44:31.385396+0000","last_fullsized":"2024-09-10T21:44:31.385396+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:50.665870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.619629+0000","last_change":"2024-09-10T21:44:30.376207+0000","last_active":"2024-09-10T21:44:32.619629+0000","last_peered":"2024-09-10T21:44:32.619629+0000","last_clean":"2024-09-10T21:44:32.619629+0000","last_became_active":"2024-09-10T21:44:30.375912+0000","last_became_peered":"2024-09-10T21:44:30.375912+0000","last_unstale":"2024-09-10T21:44:32.619629+0000","last_undegraded":"2024-09-10T21:44:32.619629+0000","last_fullsized":"2024-09-10T21:44:32.619629+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:36:50.546531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385360+0000","last_change":"2024-09-10T21:44:31.386029+0000","last_active":"2024-09-10T21:44:32.385360+0000","last_peered":"2024-09-10T21:44:32.385360+0000","last_clean":"2024-09-10T21:44:32.385360+0000","last_became_active":"2024-09-10T21:44:31.384174+0000","last_became_peered":"2024-09-10T21:44:31.384174+0000","last_unstale":"2024-09-10T21:44:32.385360+0000","last_undegraded":"2024-09-10T21:44:32.385360+0000","last_fullsized":"2024-09-10T21:44:32.385360+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:21.831628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385227+0000","last_change":"2024-09-10T21:44:30.376392+0000","last_active":"2024-09-10T21:44:32.385227+0000","last_peered":"2024-09-10T21:44:32.385227+0000","last_clean":"2024-09-10T21:44:32.385227+0000","last_became_active":"2024-09-10T21:44:30.376100+0000","last_became_peered":"2024-09-10T21:44:30.376100+0000","last_unstale":"2024-09-10T21:44:32.385227+0000","last_undegraded":"2024-09-10T21:44:32.385227+0000","last_fullsized":"2024-09-10T21:44:32.385227+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:51.402154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402050+0000","last_change":"2024-09-10T21:44:31.397163+0000","last_active":"2024-09-10T21:44:31.402050+0000","last_peered":"2024-09-10T21:44:31.402050+0000","last_clean":"2024-09-10T21:44:31.402050+0000","last_became_active":"2024-09-10T21:44:31.396846+0000","last_became_peered":"2024-09-10T21:44:31.396846+0000","last_unstale":"2024-09-10T21:44:31.402050+0000","last_undegraded":"2024-09-10T21:44:31.402050+0000","last_fullsized":"2024-09-10T21:44:31.402050+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:29.828043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":21,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.399807+0000","last_change":"2024-09-10T21:44:30.378527+0000","last_active":"2024-09-10T21:44:31.399807+0000","last_peered":"2024-09-10T21:44:31.399807+0000","last_clean":"2024-09-10T21:44:31.399807+0000","last_became_active":"2024-09-10T21:44:30.377364+0000","last_became_peered":"2024-09-10T21:44:30.377364+0000","last_unstale":"2024-09-10T21:44:31.399807+0000","last_undegraded":"2024-09-10T21:44:31.399807+0000","last_fullsized":"2024-09-10T21:44:31.399807+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:18.682249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.383192+0000","last_change":"2024-09-10T21:44:31.378653+0000","last_active":"2024-09-10T21:44:31.383192+0000","last_peered":"2024-09-10T21:44:31.383192+0000","last_clean":"2024-09-10T21:44:31.383192+0000","last_became_active":"2024-09-10T21:44:31.378328+0000","last_became_peered":"2024-09-10T21:44:31.378328+0000","last_unstale":"2024-09-10T21:44:31.383192+0000","last_undegraded":"2024-09-10T21:44:31.383192+0000","last_fullsized":"2024-09-10T21:44:31.383192+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:13.123398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.405775+0000","last_change":"2024-09-10T21:44:30.380762+0000","last_active":"2024-09-10T21:44:31.405775+0000","last_peered":"2024-09-10T21:44:31.405775+0000","last_clean":"2024-09-10T21:44:31.405775+0000","last_became_active":"2024-09-10T21:44:30.380524+0000","last_became_peered":"2024-09-10T21:44:30.380524+0000","last_unstale":"2024-09-10T21:44:31.405775+0000","last_undegraded":"2024-09-10T21:44:31.405775+0000","last_fullsized":"2024-09-10T21:44:31.405775+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:06.795703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385892+0000","last_change":"2024-09-10T21:44:31.393563+0000","last_active":"2024-09-10T21:44:32.385892+0000","last_peered":"2024-09-10T21:44:32.385892+0000","last_clean":"2024-09-10T21:44:32.385892+0000","last_became_active":"2024-09-10T21:44:31.392646+0000","last_became_peered":"2024-09-10T21:44:31.392646+0000","last_unstale":"2024-09-10T21:44:32.385892+0000","last_undegraded":"2024-09-10T21:44:32.385892+0000","last_fullsized":"2024-09-10T21:44:32.385892+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:36.451092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.406795+0000","last_change":"2024-09-10T21:44:30.384439+0000","last_active":"2024-09-10T21:44:31.406795+0000","last_peered":"2024-09-10T21:44:31.406795+0000","last_clean":"2024-09-10T21:44:31.406795+0000","last_became_active":"2024-09-10T21:44:30.384166+0000","last_became_peered":"2024-09-10T21:44:30.384166+0000","last_unstale":"2024-09-10T21:44:31.406795+0000","last_undegraded":"2024-09-10T21:44:31.406795+0000","last_fullsized":"2024-09-10T21:44:31.406795+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:35.007284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.398553+0000","last_change":"2024-09-10T21:44:31.390661+0000","last_active":"2024-09-10T21:44:31.398553+0000","last_peered":"2024-09-10T21:44:31.398553+0000","last_clean":"2024-09-10T21:44:31.398553+0000","last_became_active":"2024-09-10T21:44:31.385105+0000","last_became_peered":"2024-09-10T21:44:31.385105+0000","last_unstale":"2024-09-10T21:44:31.398553+0000","last_undegraded":"2024-09-10T21:44:31.398553+0000","last_fullsized":"2024-09-10T21:44:31.398553+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:21.472705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385374+0000","last_change":"2024-09-10T21:44:30.379029+0000","last_active":"2024-09-10T21:44:32.385374+0000","last_peered":"2024-09-10T21:44:32.385374+0000","last_clean":"2024-09-10T21:44:32.385374+0000","last_became_active":"2024-09-10T21:44:30.378729+0000","last_became_peered":"2024-09-10T21:44:30.378729+0000","last_unstale":"2024-09-10T21:44:32.385374+0000","last_undegraded":"2024-09-10T21:44:32.385374+0000","last_fullsized":"2024-09-10T21:44:32.385374+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:42.843155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.395530+0000","last_change":"2024-09-10T21:44:31.388055+0000","last_active":"2024-09-10T21:44:31.395530+0000","last_peered":"2024-09-10T21:44:31.395530+0000","last_clean":"2024-09-10T21:44:31.395530+0000","last_became_active":"2024-09-10T21:44:31.386823+0000","last_became_peered":"2024-09-10T21:44:31.386823+0000","last_unstale":"2024-09-10T21:44:31.395530+0000","last_undegraded":"2024-09-10T21:44:31.395530+0000","last_fullsized":"2024-09-10T21:44:31.395530+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:05.840706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.373395+0000","last_change":"2024-09-10T21:44:30.373250+0000","last_active":"2024-09-10T21:44:31.373395+0000","last_peered":"2024-09-10T21:44:31.373395+0000","last_clean":"2024-09-10T21:44:31.373395+0000","last_became_active":"2024-09-10T21:44:30.372938+0000","last_became_peered":"2024-09-10T21:44:30.372938+0000","last_unstale":"2024-09-10T21:44:31.373395+0000","last_undegraded":"2024-09-10T21:44:31.373395+0000","last_fullsized":"2024-09-10T21:44:31.373395+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:15.879793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.386101+0000","last_change":"2024-09-10T21:44:31.389808+0000","last_active":"2024-09-10T21:44:32.386101+0000","last_peered":"2024-09-10T21:44:32.386101+0000","last_clean":"2024-09-10T21:44:32.386101+0000","last_became_active":"2024-09-10T21:44:31.386631+0000","last_became_peered":"2024-09-10T21:44:31.386631+0000","last_unstale":"2024-09-10T21:44:32.386101+0000","last_undegraded":"2024-09-10T21:44:32.386101+0000","last_fullsized":"2024-09-10T21:44:32.386101+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:45.635509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.389477+0000","last_change":"2024-09-10T21:44:30.378706+0000","last_active":"2024-09-10T21:44:31.389477+0000","last_peered":"2024-09-10T21:44:31.389477+0000","last_clean":"2024-09-10T21:44:31.389477+0000","last_became_active":"2024-09-10T21:44:30.378432+0000","last_became_peered":"2024-09-10T21:44:30.378432+0000","last_unstale":"2024-09-10T21:44:31.389477+0000","last_undegraded":"2024-09-10T21:44:31.389477+0000","last_fullsized":"2024-09-10T21:44:31.389477+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:25:49.249958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402818+0000","last_change":"2024-09-10T21:44:31.393342+0000","last_active":"2024-09-10T21:44:31.402818+0000","last_peered":"2024-09-10T21:44:31.402818+0000","last_clean":"2024-09-10T21:44:31.402818+0000","last_became_active":"2024-09-10T21:44:31.389027+0000","last_became_peered":"2024-09-10T21:44:31.389027+0000","last_unstale":"2024-09-10T21:44:31.402818+0000","last_undegraded":"2024-09-10T21:44:31.402818+0000","last_fullsized":"2024-09-10T21:44:31.402818+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:01.195088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.382302+0000","last_change":"2024-09-10T21:44:30.380686+0000","last_active":"2024-09-10T21:44:31.382302+0000","last_peered":"2024-09-10T21:44:31.382302+0000","last_clean":"2024-09-10T21:44:31.382302+0000","last_became_active":"2024-09-10T21:44:30.380445+0000","last_became_peered":"2024-09-10T21:44:30.380445+0000","last_unstale":"2024-09-10T21:44:31.382302+0000","last_undegraded":"2024-09-10T21:44:31.382302+0000","last_fullsized":"2024-09-10T21:44:31.382302+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:32.526008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.392444+0000","last_change":"2024-09-10T21:44:31.384342+0000","last_active":"2024-09-10T21:44:31.392444+0000","last_peered":"2024-09-10T21:44:31.392444+0000","last_clean":"2024-09-10T21:44:31.392444+0000","last_became_active":"2024-09-10T21:44:31.384071+0000","last_became_peered":"2024-09-10T21:44:31.384071+0000","last_unstale":"2024-09-10T21:44:31.392444+0000","last_undegraded":"2024-09-10T21:44:31.392444+0000","last_fullsized":"2024-09-10T21:44:31.392444+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:55.051489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.387249+0000","last_change":"2024-09-10T21:44:30.383485+0000","last_active":"2024-09-10T21:44:32.387249+0000","last_peered":"2024-09-10T21:44:32.387249+0000","last_clean":"2024-09-10T21:44:32.387249+0000","last_became_active":"2024-09-10T21:44:30.383194+0000","last_became_peered":"2024-09-10T21:44:30.383194+0000","last_unstale":"2024-09-10T21:44:32.387249+0000","last_undegraded":"2024-09-10T21:44:32.387249+0000","last_fullsized":"2024-09-10T21:44:32.387249+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:57.794431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385800+0000","last_change":"2024-09-10T21:44:31.380739+0000","last_active":"2024-09-10T21:44:32.385800+0000","last_peered":"2024-09-10T21:44:32.385800+0000","last_clean":"2024-09-10T21:44:32.385800+0000","last_became_active":"2024-09-10T21:44:31.380320+0000","last_became_peered":"2024-09-10T21:44:31.380320+0000","last_unstale":"2024-09-10T21:44:32.385800+0000","last_undegraded":"2024-09-10T21:44:32.385800+0000","last_fullsized":"2024-09-10T21:44:32.385800+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:21.913494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402520+0000","last_change":"2024-09-10T21:44:30.384908+0000","last_active":"2024-09-10T21:44:31.402520+0000","last_peered":"2024-09-10T21:44:31.402520+0000","last_clean":"2024-09-10T21:44:31.402520+0000","last_became_active":"2024-09-10T21:44:30.383035+0000","last_became_peered":"2024-09-10T21:44:30.383035+0000","last_unstale":"2024-09-10T21:44:31.402520+0000","last_undegraded":"2024-09-10T21:44:31.402520+0000","last_fullsized":"2024-09-10T21:44:31.402520+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:39.962769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.386100+0000","last_change":"2024-09-10T21:44:31.381428+0000","last_active":"2024-09-10T21:44:32.386100+0000","last_peered":"2024-09-10T21:44:32.386100+0000","last_clean":"2024-09-10T21:44:32.386100+0000","last_became_active":"2024-09-10T21:44:31.381108+0000","last_became_peered":"2024-09-10T21:44:31.381108+0000","last_unstale":"2024-09-10T21:44:32.386100+0000","last_undegraded":"2024-09-10T21:44:32.386100+0000","last_fullsized":"2024-09-10T21:44:32.386100+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:46.961432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.373392+0000","last_change":"2024-09-10T21:44:30.378892+0000","last_active":"2024-09-10T21:44:31.373392+0000","last_peered":"2024-09-10T21:44:31.373392+0000","last_clean":"2024-09-10T21:44:31.373392+0000","last_became_active":"2024-09-10T21:44:30.378624+0000","last_became_peered":"2024-09-10T21:44:30.378624+0000","last_unstale":"2024-09-10T21:44:31.373392+0000","last_undegraded":"2024-09-10T21:44:31.373392+0000","last_fullsized":"2024-09-10T21:44:31.373392+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:53.123593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.398755+0000","last_change":"2024-09-10T21:44:31.394359+0000","last_active":"2024-09-10T21:44:31.398755+0000","last_peered":"2024-09-10T21:44:31.398755+0000","last_clean":"2024-09-10T21:44:31.398755+0000","last_became_active":"2024-09-10T21:44:31.394059+0000","last_became_peered":"2024-09-10T21:44:31.394059+0000","last_unstale":"2024-09-10T21:44:31.398755+0000","last_undegraded":"2024-09-10T21:44:31.398755+0000","last_fullsized":"2024-09-10T21:44:31.398755+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:04.870549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.619763+0000","last_change":"2024-09-10T21:44:30.374777+0000","last_active":"2024-09-10T21:44:32.619763+0000","last_peered":"2024-09-10T21:44:32.619763+0000","last_clean":"2024-09-10T21:44:32.619763+0000","last_became_active":"2024-09-10T21:44:30.374504+0000","last_became_peered":"2024-09-10T21:44:30.374504+0000","last_unstale":"2024-09-10T21:44:32.619763+0000","last_undegraded":"2024-09-10T21:44:32.619763+0000","last_fullsized":"2024-09-10T21:44:32.619763+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:07.630691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.399912+0000","last_change":"2024-09-10T21:44:31.393032+0000","last_active":"2024-09-10T21:44:31.399912+0000","last_peered":"2024-09-10T21:44:31.399912+0000","last_clean":"2024-09-10T21:44:31.399912+0000","last_became_active":"2024-09-10T21:44:31.388203+0000","last_became_peered":"2024-09-10T21:44:31.388203+0000","last_unstale":"2024-09-10T21:44:31.399912+0000","last_undegraded":"2024-09-10T21:44:31.399912+0000","last_fullsized":"2024-09-10T21:44:31.399912+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:41.208914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385983+0000","last_change":"2024-09-10T21:44:30.381228+0000","last_active":"2024-09-10T21:44:32.385983+0000","last_peered":"2024-09-10T21:44:32.385983+0000","last_clean":"2024-09-10T21:44:32.385983+0000","last_became_active":"2024-09-10T21:44:30.380019+0000","last_became_peered":"2024-09-10T21:44:30.380019+0000","last_unstale":"2024-09-10T21:44:32.385983+0000","last_undegraded":"2024-09-10T21:44:32.385983+0000","last_fullsized":"2024-09-10T21:44:32.385983+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:15.879793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.401884+0000","last_change":"2024-09-10T21:44:31.396905+0000","last_active":"2024-09-10T21:44:31.401884+0000","last_peered":"2024-09-10T21:44:31.401884+0000","last_clean":"2024-09-10T21:44:31.401884+0000","last_became_active":"2024-09-10T21:44:31.395553+0000","last_became_peered":"2024-09-10T21:44:31.395553+0000","last_unstale":"2024-09-10T21:44:31.401884+0000","last_undegraded":"2024-09-10T21:44:31.401884+0000","last_fullsized":"2024-09-10T21:44:31.401884+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:56.264971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.380804+0000","last_change":"2024-09-10T21:44:30.381814+0000","last_active":"2024-09-10T21:44:31.380804+0000","last_peered":"2024-09-10T21:44:31.380804+0000","last_clean":"2024-09-10T21:44:31.380804+0000","last_became_active":"2024-09-10T21:44:30.379714+0000","last_became_peered":"2024-09-10T21:44:30.379714+0000","last_unstale":"2024-09-10T21:44:31.380804+0000","last_undegraded":"2024-09-10T21:44:31.380804+0000","last_fullsized":"2024-09-10T21:44:31.380804+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:39.445476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402067+0000","last_change":"2024-09-10T21:44:31.396981+0000","last_active":"2024-09-10T21:44:31.402067+0000","last_peered":"2024-09-10T21:44:31.402067+0000","last_clean":"2024-09-10T21:44:31.402067+0000","last_became_active":"2024-09-10T21:44:31.396679+0000","last_became_peered":"2024-09-10T21:44:31.396679+0000","last_unstale":"2024-09-10T21:44:31.402067+0000","last_undegraded":"2024-09-10T21:44:31.402067+0000","last_fullsized":"2024-09-10T21:44:31.402067+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:05.332185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.386436+0000","last_change":"2024-09-10T21:44:30.372026+0000","last_active":"2024-09-10T21:44:32.386436+0000","last_peered":"2024-09-10T21:44:32.386436+0000","last_clean":"2024-09-10T21:44:32.386436+0000","last_became_active":"2024-09-10T21:44:30.371748+0000","last_became_peered":"2024-09-10T21:44:30.371748+0000","last_unstale":"2024-09-10T21:44:32.386436+0000","last_undegraded":"2024-09-10T21:44:32.386436+0000","last_fullsized":"2024-09-10T21:44:32.386436+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:10.212300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.387345+0000","last_change":"2024-09-10T21:44:31.385405+0000","last_active":"2024-09-10T21:44:31.387345+0000","last_peered":"2024-09-10T21:44:31.387345+0000","last_clean":"2024-09-10T21:44:31.387345+0000","last_became_active":"2024-09-10T21:44:31.384838+0000","last_became_peered":"2024-09-10T21:44:31.384838+0000","last_unstale":"2024-09-10T21:44:31.387345+0000","last_undegraded":"2024-09-10T21:44:31.387345+0000","last_fullsized":"2024-09-10T21:44:31.387345+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:33.683588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.381148+0000","last_change":"2024-09-10T21:44:30.373885+0000","last_active":"2024-09-10T21:44:31.381148+0000","last_peered":"2024-09-10T21:44:31.381148+0000","last_clean":"2024-09-10T21:44:31.381148+0000","last_became_active":"2024-09-10T21:44:30.373603+0000","last_became_peered":"2024-09-10T21:44:30.373603+0000","last_unstale":"2024-09-10T21:44:31.381148+0000","last_undegraded":"2024-09-10T21:44:31.381148+0000","last_fullsized":"2024-09-10T21:44:31.381148+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:46.950505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.398321+0000","last_change":"2024-09-10T21:44:31.387375+0000","last_active":"2024-09-10T21:44:31.398321+0000","last_peered":"2024-09-10T21:44:31.398321+0000","last_clean":"2024-09-10T21:44:31.398321+0000","last_became_active":"2024-09-10T21:44:31.385735+0000","last_became_peered":"2024-09-10T21:44:31.385735+0000","last_unstale":"2024-09-10T21:44:31.398321+0000","last_undegraded":"2024-09-10T21:44:31.398321+0000","last_fullsized":"2024-09-10T21:44:31.398321+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:29.763778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.370538+0000","last_change":"2024-09-10T21:44:30.378015+0000","last_active":"2024-09-10T21:44:31.370538+0000","last_peered":"2024-09-10T21:44:31.370538+0000","last_clean":"2024-09-10T21:44:31.370538+0000","last_became_active":"2024-09-10T21:44:30.377465+0000","last_became_peered":"2024-09-10T21:44:30.377465+0000","last_unstale":"2024-09-10T21:44:31.370538+0000","last_undegraded":"2024-09-10T21:44:31.370538+0000","last_fullsized":"2024-09-10T21:44:31.370538+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:56.422703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.398567+0000","last_change":"2024-09-10T21:44:31.393215+0000","last_active":"2024-09-10T21:44:31.398567+0000","last_peered":"2024-09-10T21:44:31.398567+0000","last_clean":"2024-09-10T21:44:31.398567+0000","last_became_active":"2024-09-10T21:44:31.392771+0000","last_became_peered":"2024-09-10T21:44:31.392771+0000","last_unstale":"2024-09-10T21:44:31.398567+0000","last_undegraded":"2024-09-10T21:44:31.398567+0000","last_fullsized":"2024-09-10T21:44:31.398567+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:18.404850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.619552+0000","last_change":"2024-09-10T21:44:30.372724+0000","last_active":"2024-09-10T21:44:32.619552+0000","last_peered":"2024-09-10T21:44:32.619552+0000","last_clean":"2024-09-10T21:44:32.619552+0000","last_became_active":"2024-09-10T21:44:30.372476+0000","last_became_peered":"2024-09-10T21:44:30.372476+0000","last_unstale":"2024-09-10T21:44:32.619552+0000","last_undegraded":"2024-09-10T21:44:32.619552+0000","last_fullsized":"2024-09-10T21:44:32.619552+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:04.578355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":175,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.369773+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:44:31.369773+0000","last_peered":"2024-09-10T21:44:31.369773+0000","last_clean":"2024-09-10T21:44:31.369773+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:44:31.369773+0000","last_undegraded":"2024-09-10T21:44:31.369773+0000","last_fullsized":"2024-09-10T21:44:31.369773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509475,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":675840,"data_stored":278251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509475,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":684032,"data_stored":278903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509474,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":688128,"data_stored":279863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509474,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":684032,"data_stored":278307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":679936,"data_stored":278341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509474,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1286144,"data_stored":870813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542177,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1257472,"data_stored":861844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542178,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":675840,"data_stored":278251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:44:35.631 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:44:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.826+0000 7f8649a00640 1 -- 172.21.15.7:0/3496485335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f864415bc90 msgr2=0x7f86440ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.826+0000 7f8649a00640 1 --2- 172.21.15.7:0/3496485335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f864415bc90 0x7f86440ccef0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f86440c3650 tx=0x7f863402a980 comp rx=0 tx=0).stop 2024-09-10T21:44:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.826+0000 7f8649a00640 1 -- 172.21.15.7:0/3496485335 shutdown_connections 2024-09-10T21:44:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.826+0000 7f8649a00640 1 --2- 172.21.15.7:0/3496485335 >> v2:172.21.15.133:3300/0 conn(0x7f864415fda0 0x7f8644160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.826+0000 7f8649a00640 1 --2- 172.21.15.7:0/3496485335 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f864415bc90 0x7f86440ccef0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.826+0000 7f8649a00640 1 -- 172.21.15.7:0/3496485335 >> 172.21.15.7:0/3496485335 conn(0x7f86440b16d0 msgr2=0x7f86440b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.826+0000 7f8649a00640 1 -- 172.21.15.7:0/3496485335 shutdown_connections 2024-09-10T21:44:35.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.826+0000 7f8649a00640 1 -- 172.21.15.7:0/3496485335 wait complete. 2024-09-10T21:44:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.828+0000 7f8649a00640 1 Processor -- start 2024-09-10T21:44:35.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.828+0000 7f8649a00640 1 -- start start 2024-09-10T21:44:35.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.829+0000 7f8649a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86440cd750 0x7f864415ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.829+0000 7f8649a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f864415bc90 0x7f864415cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.829+0000 7f8649a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f864415fda0 0x7f8644155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.829+0000 7f8649a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f864406fe70 con 0x7f86440cd750 2024-09-10T21:44:35.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.829+0000 7f8649a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f864406fcf0 con 0x7f864415fda0 2024-09-10T21:44:35.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.829+0000 7f8649a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f864406fff0 con 0x7f864415bc90 2024-09-10T21:44:35.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.829+0000 7f8642a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86440cd750 0x7f864415ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:35.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.829+0000 7f8642a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86440cd750 0x7f864415ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43510/0 (socket says 172.21.15.7:43510) 2024-09-10T21:44:35.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.829+0000 7f8642a00640 1 -- 172.21.15.7:0/1795441772 learned_addr learned my addr 172.21.15.7:0/1795441772 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:35.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.829+0000 7f8643400640 1 --2- 172.21.15.7:0/1795441772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f864415fda0 0x7f8644155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.829+0000 7f863a000640 1 --2- 172.21.15.7:0/1795441772 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f864415bc90 0x7f864415cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.830+0000 7f8642a00640 1 -- 172.21.15.7:0/1795441772 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f864415bc90 msgr2=0x7f864415cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.830+0000 7f8642a00640 1 --2- 172.21.15.7:0/1795441772 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f864415bc90 0x7f864415cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.830+0000 7f8642a00640 1 -- 172.21.15.7:0/1795441772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f864415fda0 msgr2=0x7f8644155ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.830+0000 7f8642a00640 1 --2- 172.21.15.7:0/1795441772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f864415fda0 0x7f8644155ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.830+0000 7f8642a00640 1 -- 172.21.15.7:0/1795441772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86440c7660 con 0x7f86440cd750 2024-09-10T21:44:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.830+0000 7f8643400640 1 --2- 172.21.15.7:0/1795441772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f864415fda0 0x7f8644155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:35.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.830+0000 7f8642a00640 1 --2- 172.21.15.7:0/1795441772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86440cd750 0x7f864415ed80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8634009bf0 tx=0x7f8634003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:35.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.830+0000 7f863b400640 1 -- 172.21.15.7:0/1795441772 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634034050 con 0x7f86440cd750 2024-09-10T21:44:35.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.830+0000 7f863b400640 1 -- 172.21.15.7:0/1795441772 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86340044d0 con 0x7f86440cd750 2024-09-10T21:44:35.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.830+0000 7f863b400640 1 -- 172.21.15.7:0/1795441772 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8634043440 con 0x7f86440cd750 2024-09-10T21:44:35.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.830+0000 7f8649a00640 1 -- 172.21.15.7:0/1795441772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86440c7900 con 0x7f86440cd750 2024-09-10T21:44:35.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.830+0000 7f8649a00640 1 -- 172.21.15.7:0/1795441772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86440c7ec0 con 0x7f86440cd750 2024-09-10T21:44:35.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.831+0000 7f8649a00640 1 -- 172.21.15.7:0/1795441772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f864415f4a0 con 0x7f86440cd750 2024-09-10T21:44:35.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.833+0000 7f863b400640 1 -- 172.21.15.7:0/1795441772 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8634041070 con 0x7f86440cd750 2024-09-10T21:44:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.834+0000 7f863b400640 1 --2- 172.21.15.7:0/1795441772 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8620077f70 0x7f862007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.834+0000 7f863b400640 1 -- 172.21.15.7:0/1795441772 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f86340bb2b0 con 0x7f86440cd750 2024-09-10T21:44:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.835+0000 7f863a000640 1 --2- 172.21.15.7:0/1795441772 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8620077f70 0x7f862007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:35.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.835+0000 7f863a000640 1 --2- 172.21.15.7:0/1795441772 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8620077f70 0x7f862007a430 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f86440b3430 tx=0x7f862c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:35.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:35.838+0000 7f863b400640 1 -- 172.21.15.7:0/1795441772 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f864415f4a0 con 0x7f86440cd750 2024-09-10T21:44:36.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.031+0000 7f8649a00640 1 -- 172.21.15.7:0/1795441772 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f86440b35a0 con 0x7f8620077f70 2024-09-10T21:44:36.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.036+0000 7f863b400640 1 -- 172.21.15.7:0/1795441772 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251941 (secure 0 0 0) 0x7f86440b35a0 con 0x7f8620077f70 2024-09-10T21:44:36.040 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:36.041 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:44:36.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.040+0000 7f8649a00640 1 -- 172.21.15.7:0/1795441772 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8620077f70 msgr2=0x7f862007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:36.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.040+0000 7f8649a00640 1 --2- 172.21.15.7:0/1795441772 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8620077f70 0x7f862007a430 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f86440b3430 tx=0x7f862c009040 comp rx=0 tx=0).stop 2024-09-10T21:44:36.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.041+0000 7f8649a00640 1 -- 172.21.15.7:0/1795441772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86440cd750 msgr2=0x7f864415ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:36.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.041+0000 7f8649a00640 1 --2- 172.21.15.7:0/1795441772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86440cd750 0x7f864415ed80 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f8634009bf0 tx=0x7f8634003f90 comp rx=0 tx=0).stop 2024-09-10T21:44:36.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.041+0000 7f8649a00640 1 -- 172.21.15.7:0/1795441772 shutdown_connections 2024-09-10T21:44:36.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.041+0000 7f8649a00640 1 --2- 172.21.15.7:0/1795441772 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8620077f70 0x7f862007a430 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.041+0000 7f8649a00640 1 --2- 172.21.15.7:0/1795441772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f864415fda0 0x7f8644155ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.041+0000 7f8649a00640 1 --2- 172.21.15.7:0/1795441772 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f864415bc90 0x7f864415cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.041+0000 7f8649a00640 1 --2- 172.21.15.7:0/1795441772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86440cd750 0x7f864415ed80 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.041+0000 7f8649a00640 1 -- 172.21.15.7:0/1795441772 >> 172.21.15.7:0/1795441772 conn(0x7f86440b16d0 msgr2=0x7f86440b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:36.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.041+0000 7f8649a00640 1 -- 172.21.15.7:0/1795441772 shutdown_connections 2024-09-10T21:44:36.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.041+0000 7f8649a00640 1 -- 172.21.15.7:0/1795441772 wait complete. 2024-09-10T21:44:36.058 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-10T21:44:35.892217+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":132,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":247700,"kb_used_data":6476,"kb_used_omap":12,"kb_used_meta":241011,"kb_avail":754727020,"statfs":{"total":773094113280,"available":772840468480,"internally_reserved":0,"allocated":6631424,"data_stored":3404573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":246795825},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001681"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.403629+0000","last_change":"2024-09-10T21:44:31.399608+0000","last_active":"2024-09-10T21:44:31.403629+0000","last_peered":"2024-09-10T21:44:31.403629+0000","last_clean":"2024-09-10T21:44:31.403629+0000","last_became_active":"2024-09-10T21:44:31.398957+0000","last_became_peered":"2024-09-10T21:44:31.398957+0000","last_unstale":"2024-09-10T21:44:31.403629+0000","last_undegraded":"2024-09-10T21:44:31.403629+0000","last_fullsized":"2024-09-10T21:44:31.403629+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:27.896951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402741+0000","last_change":"2024-09-10T21:44:31.397977+0000","last_active":"2024-09-10T21:44:31.402741+0000","last_peered":"2024-09-10T21:44:31.402741+0000","last_clean":"2024-09-10T21:44:31.402741+0000","last_became_active":"2024-09-10T21:44:31.396697+0000","last_became_peered":"2024-09-10T21:44:31.396697+0000","last_unstale":"2024-09-10T21:44:31.402741+0000","last_undegraded":"2024-09-10T21:44:31.402741+0000","last_fullsized":"2024-09-10T21:44:31.402741+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:44:10.772875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.383181+0000","last_change":"2024-09-10T21:44:31.379067+0000","last_active":"2024-09-10T21:44:31.383181+0000","last_peered":"2024-09-10T21:44:31.383181+0000","last_clean":"2024-09-10T21:44:31.383181+0000","last_became_active":"2024-09-10T21:44:31.378736+0000","last_became_peered":"2024-09-10T21:44:31.378736+0000","last_unstale":"2024-09-10T21:44:31.383181+0000","last_undegraded":"2024-09-10T21:44:31.383181+0000","last_fullsized":"2024-09-10T21:44:31.383181+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:12.839190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385464+0000","last_change":"2024-09-10T21:44:31.392160+0000","last_active":"2024-09-10T21:44:32.385464+0000","last_peered":"2024-09-10T21:44:32.385464+0000","last_clean":"2024-09-10T21:44:32.385464+0000","last_became_active":"2024-09-10T21:44:31.391547+0000","last_became_peered":"2024-09-10T21:44:31.391547+0000","last_unstale":"2024-09-10T21:44:32.385464+0000","last_undegraded":"2024-09-10T21:44:32.385464+0000","last_fullsized":"2024-09-10T21:44:32.385464+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:12.308573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.398687+0000","last_change":"2024-09-10T21:44:31.385084+0000","last_active":"2024-09-10T21:44:31.398687+0000","last_peered":"2024-09-10T21:44:31.398687+0000","last_clean":"2024-09-10T21:44:31.398687+0000","last_became_active":"2024-09-10T21:44:31.383699+0000","last_became_peered":"2024-09-10T21:44:31.383699+0000","last_unstale":"2024-09-10T21:44:31.398687+0000","last_undegraded":"2024-09-10T21:44:31.398687+0000","last_fullsized":"2024-09-10T21:44:31.398687+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:37.590428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.405725+0000","last_change":"2024-09-10T21:44:31.395955+0000","last_active":"2024-09-10T21:44:31.405725+0000","last_peered":"2024-09-10T21:44:31.405725+0000","last_clean":"2024-09-10T21:44:31.405725+0000","last_became_active":"2024-09-10T21:44:31.391364+0000","last_became_peered":"2024-09-10T21:44:31.391364+0000","last_unstale":"2024-09-10T21:44:31.405725+0000","last_undegraded":"2024-09-10T21:44:31.405725+0000","last_fullsized":"2024-09-10T21:44:31.405725+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:30.274226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385693+0000","last_change":"2024-09-10T21:44:31.387224+0000","last_active":"2024-09-10T21:44:32.385693+0000","last_peered":"2024-09-10T21:44:32.385693+0000","last_clean":"2024-09-10T21:44:32.385693+0000","last_became_active":"2024-09-10T21:44:31.386953+0000","last_became_peered":"2024-09-10T21:44:31.386953+0000","last_unstale":"2024-09-10T21:44:32.385693+0000","last_undegraded":"2024-09-10T21:44:32.385693+0000","last_fullsized":"2024-09-10T21:44:32.385693+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:17.893343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.389027+0000","last_change":"2024-09-10T21:44:31.379853+0000","last_active":"2024-09-10T21:44:31.389027+0000","last_peered":"2024-09-10T21:44:31.389027+0000","last_clean":"2024-09-10T21:44:31.389027+0000","last_became_active":"2024-09-10T21:44:31.379562+0000","last_became_peered":"2024-09-10T21:44:31.379562+0000","last_unstale":"2024-09-10T21:44:31.389027+0000","last_undegraded":"2024-09-10T21:44:31.389027+0000","last_fullsized":"2024-09-10T21:44:31.389027+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:47.953031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.387147+0000","last_change":"2024-09-10T21:44:31.394796+0000","last_active":"2024-09-10T21:44:32.387147+0000","last_peered":"2024-09-10T21:44:32.387147+0000","last_clean":"2024-09-10T21:44:32.387147+0000","last_became_active":"2024-09-10T21:44:31.394570+0000","last_became_peered":"2024-09-10T21:44:31.394570+0000","last_unstale":"2024-09-10T21:44:32.387147+0000","last_undegraded":"2024-09-10T21:44:32.387147+0000","last_fullsized":"2024-09-10T21:44:32.387147+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:54.134546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.401866+0000","last_change":"2024-09-10T21:44:31.399373+0000","last_active":"2024-09-10T21:44:31.401866+0000","last_peered":"2024-09-10T21:44:31.401866+0000","last_clean":"2024-09-10T21:44:31.401866+0000","last_became_active":"2024-09-10T21:44:31.399148+0000","last_became_peered":"2024-09-10T21:44:31.399148+0000","last_unstale":"2024-09-10T21:44:31.401866+0000","last_undegraded":"2024-09-10T21:44:31.401866+0000","last_fullsized":"2024-09-10T21:44:31.401866+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:31.171030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.400029+0000","last_change":"2024-09-10T21:44:31.394283+0000","last_active":"2024-09-10T21:44:31.400029+0000","last_peered":"2024-09-10T21:44:31.400029+0000","last_clean":"2024-09-10T21:44:31.400029+0000","last_became_active":"2024-09-10T21:44:31.393985+0000","last_became_peered":"2024-09-10T21:44:31.393985+0000","last_unstale":"2024-09-10T21:44:31.400029+0000","last_undegraded":"2024-09-10T21:44:31.400029+0000","last_fullsized":"2024-09-10T21:44:31.400029+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:03:13.290552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385752+0000","last_change":"2024-09-10T21:44:31.379418+0000","last_active":"2024-09-10T21:44:32.385752+0000","last_peered":"2024-09-10T21:44:32.385752+0000","last_clean":"2024-09-10T21:44:32.385752+0000","last_became_active":"2024-09-10T21:44:31.377166+0000","last_became_peered":"2024-09-10T21:44:31.377166+0000","last_unstale":"2024-09-10T21:44:32.385752+0000","last_undegraded":"2024-09-10T21:44:32.385752+0000","last_fullsized":"2024-09-10T21:44:32.385752+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:29:44.243658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.619900+0000","last_change":"2024-09-10T21:44:31.392208+0000","last_active":"2024-09-10T21:44:32.619900+0000","last_peered":"2024-09-10T21:44:32.619900+0000","last_clean":"2024-09-10T21:44:32.619900+0000","last_became_active":"2024-09-10T21:44:31.391908+0000","last_became_peered":"2024-09-10T21:44:31.391908+0000","last_unstale":"2024-09-10T21:44:32.619900+0000","last_undegraded":"2024-09-10T21:44:32.619900+0000","last_fullsized":"2024-09-10T21:44:32.619900+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:06.505235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.386331+0000","last_change":"2024-09-10T21:44:31.385438+0000","last_active":"2024-09-10T21:44:32.386331+0000","last_peered":"2024-09-10T21:44:32.386331+0000","last_clean":"2024-09-10T21:44:32.386331+0000","last_became_active":"2024-09-10T21:44:31.385171+0000","last_became_peered":"2024-09-10T21:44:31.385171+0000","last_unstale":"2024-09-10T21:44:32.386331+0000","last_undegraded":"2024-09-10T21:44:32.386331+0000","last_fullsized":"2024-09-10T21:44:32.386331+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:36.018237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385953+0000","last_change":"2024-09-10T21:44:31.382606+0000","last_active":"2024-09-10T21:44:32.385953+0000","last_peered":"2024-09-10T21:44:32.385953+0000","last_clean":"2024-09-10T21:44:32.385953+0000","last_became_active":"2024-09-10T21:44:31.382173+0000","last_became_peered":"2024-09-10T21:44:31.382173+0000","last_unstale":"2024-09-10T21:44:32.385953+0000","last_undegraded":"2024-09-10T21:44:32.385953+0000","last_fullsized":"2024-09-10T21:44:32.385953+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:43.033843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.395769+0000","last_change":"2024-09-10T21:44:31.383272+0000","last_active":"2024-09-10T21:44:31.395769+0000","last_peered":"2024-09-10T21:44:31.395769+0000","last_clean":"2024-09-10T21:44:31.395769+0000","last_became_active":"2024-09-10T21:44:31.382803+0000","last_became_peered":"2024-09-10T21:44:31.382803+0000","last_unstale":"2024-09-10T21:44:31.395769+0000","last_undegraded":"2024-09-10T21:44:31.395769+0000","last_fullsized":"2024-09-10T21:44:31.395769+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:38:40.861104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.392267+0000","last_change":"2024-09-10T21:44:31.381958+0000","last_active":"2024-09-10T21:44:31.392267+0000","last_peered":"2024-09-10T21:44:31.392267+0000","last_clean":"2024-09-10T21:44:31.392267+0000","last_became_active":"2024-09-10T21:44:31.381540+0000","last_became_peered":"2024-09-10T21:44:31.381540+0000","last_unstale":"2024-09-10T21:44:31.392267+0000","last_undegraded":"2024-09-10T21:44:31.392267+0000","last_fullsized":"2024-09-10T21:44:31.392267+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:31:35.874957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.397453+0000","last_change":"2024-09-10T21:44:31.394512+0000","last_active":"2024-09-10T21:44:31.397453+0000","last_peered":"2024-09-10T21:44:31.397453+0000","last_clean":"2024-09-10T21:44:31.397453+0000","last_became_active":"2024-09-10T21:44:31.394250+0000","last_became_peered":"2024-09-10T21:44:31.394250+0000","last_unstale":"2024-09-10T21:44:31.397453+0000","last_undegraded":"2024-09-10T21:44:31.397453+0000","last_fullsized":"2024-09-10T21:44:31.397453+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:26.744558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.404955+0000","last_change":"2024-09-10T21:44:31.400156+0000","last_active":"2024-09-10T21:44:31.404955+0000","last_peered":"2024-09-10T21:44:31.404955+0000","last_clean":"2024-09-10T21:44:31.404955+0000","last_became_active":"2024-09-10T21:44:31.397539+0000","last_became_peered":"2024-09-10T21:44:31.397539+0000","last_unstale":"2024-09-10T21:44:31.404955+0000","last_undegraded":"2024-09-10T21:44:31.404955+0000","last_fullsized":"2024-09-10T21:44:31.404955+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:13:16.531319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.398582+0000","last_change":"2024-09-10T21:44:31.392546+0000","last_active":"2024-09-10T21:44:31.398582+0000","last_peered":"2024-09-10T21:44:31.398582+0000","last_clean":"2024-09-10T21:44:31.398582+0000","last_became_active":"2024-09-10T21:44:31.392282+0000","last_became_peered":"2024-09-10T21:44:31.392282+0000","last_unstale":"2024-09-10T21:44:31.398582+0000","last_undegraded":"2024-09-10T21:44:31.398582+0000","last_fullsized":"2024-09-10T21:44:31.398582+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:55.796826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.397884+0000","last_change":"2024-09-10T21:44:31.392174+0000","last_active":"2024-09-10T21:44:31.397884+0000","last_peered":"2024-09-10T21:44:31.397884+0000","last_clean":"2024-09-10T21:44:31.397884+0000","last_became_active":"2024-09-10T21:44:31.390826+0000","last_became_peered":"2024-09-10T21:44:31.390826+0000","last_unstale":"2024-09-10T21:44:31.397884+0000","last_undegraded":"2024-09-10T21:44:31.397884+0000","last_fullsized":"2024-09-10T21:44:31.397884+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:13:35.777298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.401319+0000","last_change":"2024-09-10T21:44:31.396513+0000","last_active":"2024-09-10T21:44:31.401319+0000","last_peered":"2024-09-10T21:44:31.401319+0000","last_clean":"2024-09-10T21:44:31.401319+0000","last_became_active":"2024-09-10T21:44:31.396270+0000","last_became_peered":"2024-09-10T21:44:31.396270+0000","last_unstale":"2024-09-10T21:44:31.401319+0000","last_undegraded":"2024-09-10T21:44:31.401319+0000","last_fullsized":"2024-09-10T21:44:31.401319+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:01:45.937921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402540+0000","last_change":"2024-09-10T21:44:31.397864+0000","last_active":"2024-09-10T21:44:31.402540+0000","last_peered":"2024-09-10T21:44:31.402540+0000","last_clean":"2024-09-10T21:44:31.402540+0000","last_became_active":"2024-09-10T21:44:31.397544+0000","last_became_peered":"2024-09-10T21:44:31.397544+0000","last_unstale":"2024-09-10T21:44:31.402540+0000","last_undegraded":"2024-09-10T21:44:31.402540+0000","last_fullsized":"2024-09-10T21:44:31.402540+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:01:01.598080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.619779+0000","last_change":"2024-09-10T21:44:31.391322+0000","last_active":"2024-09-10T21:44:32.619779+0000","last_peered":"2024-09-10T21:44:32.619779+0000","last_clean":"2024-09-10T21:44:32.619779+0000","last_became_active":"2024-09-10T21:44:31.389549+0000","last_became_peered":"2024-09-10T21:44:31.389549+0000","last_unstale":"2024-09-10T21:44:32.619779+0000","last_undegraded":"2024-09-10T21:44:32.619779+0000","last_fullsized":"2024-09-10T21:44:32.619779+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:31.171030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.403654+0000","last_change":"2024-09-10T21:44:31.399930+0000","last_active":"2024-09-10T21:44:31.403654+0000","last_peered":"2024-09-10T21:44:31.403654+0000","last_clean":"2024-09-10T21:44:31.403654+0000","last_became_active":"2024-09-10T21:44:31.399357+0000","last_became_peered":"2024-09-10T21:44:31.399357+0000","last_unstale":"2024-09-10T21:44:31.403654+0000","last_undegraded":"2024-09-10T21:44:31.403654+0000","last_fullsized":"2024-09-10T21:44:31.403654+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:24:53.455429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385655+0000","last_change":"2024-09-10T21:44:31.385008+0000","last_active":"2024-09-10T21:44:32.385655+0000","last_peered":"2024-09-10T21:44:32.385655+0000","last_clean":"2024-09-10T21:44:32.385655+0000","last_became_active":"2024-09-10T21:44:31.384750+0000","last_became_peered":"2024-09-10T21:44:31.384750+0000","last_unstale":"2024-09-10T21:44:32.385655+0000","last_undegraded":"2024-09-10T21:44:32.385655+0000","last_fullsized":"2024-09-10T21:44:32.385655+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:22.863091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.403100+0000","last_change":"2024-09-10T21:44:31.394310+0000","last_active":"2024-09-10T21:44:31.403100+0000","last_peered":"2024-09-10T21:44:31.403100+0000","last_clean":"2024-09-10T21:44:31.403100+0000","last_became_active":"2024-09-10T21:44:31.394029+0000","last_became_peered":"2024-09-10T21:44:31.394029+0000","last_unstale":"2024-09-10T21:44:31.403100+0000","last_undegraded":"2024-09-10T21:44:31.403100+0000","last_fullsized":"2024-09-10T21:44:31.403100+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:38:16.952137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402325+0000","last_change":"2024-09-10T21:44:31.395667+0000","last_active":"2024-09-10T21:44:31.402325+0000","last_peered":"2024-09-10T21:44:31.402325+0000","last_clean":"2024-09-10T21:44:31.402325+0000","last_became_active":"2024-09-10T21:44:31.390131+0000","last_became_peered":"2024-09-10T21:44:31.390131+0000","last_unstale":"2024-09-10T21:44:31.402325+0000","last_undegraded":"2024-09-10T21:44:31.402325+0000","last_fullsized":"2024-09-10T21:44:31.402325+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:37.199923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.398612+0000","last_change":"2024-09-10T21:44:31.390024+0000","last_active":"2024-09-10T21:44:31.398612+0000","last_peered":"2024-09-10T21:44:31.398612+0000","last_clean":"2024-09-10T21:44:31.398612+0000","last_became_active":"2024-09-10T21:44:31.389718+0000","last_became_peered":"2024-09-10T21:44:31.389718+0000","last_unstale":"2024-09-10T21:44:31.398612+0000","last_undegraded":"2024-09-10T21:44:31.398612+0000","last_fullsized":"2024-09-10T21:44:31.398612+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:30:26.016478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.383038+0000","last_change":"2024-09-10T21:44:31.380107+0000","last_active":"2024-09-10T21:44:31.383038+0000","last_peered":"2024-09-10T21:44:31.383038+0000","last_clean":"2024-09-10T21:44:31.383038+0000","last_became_active":"2024-09-10T21:44:31.379814+0000","last_became_peered":"2024-09-10T21:44:31.379814+0000","last_unstale":"2024-09-10T21:44:31.383038+0000","last_undegraded":"2024-09-10T21:44:31.383038+0000","last_fullsized":"2024-09-10T21:44:31.383038+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:04:18.902765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.391857+0000","last_change":"2024-09-10T21:44:31.381086+0000","last_active":"2024-09-10T21:44:31.391857+0000","last_peered":"2024-09-10T21:44:31.391857+0000","last_clean":"2024-09-10T21:44:31.391857+0000","last_became_active":"2024-09-10T21:44:31.380668+0000","last_became_peered":"2024-09-10T21:44:31.380668+0000","last_unstale":"2024-09-10T21:44:31.391857+0000","last_undegraded":"2024-09-10T21:44:31.391857+0000","last_fullsized":"2024-09-10T21:44:31.391857+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:15.989838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.403238+0000","last_change":"2024-09-10T21:44:31.393582+0000","last_active":"2024-09-10T21:44:31.403238+0000","last_peered":"2024-09-10T21:44:31.403238+0000","last_clean":"2024-09-10T21:44:31.403238+0000","last_became_active":"2024-09-10T21:44:31.390027+0000","last_became_peered":"2024-09-10T21:44:31.390027+0000","last_unstale":"2024-09-10T21:44:31.403238+0000","last_undegraded":"2024-09-10T21:44:31.403238+0000","last_fullsized":"2024-09-10T21:44:31.403238+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:22.058929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.406848+0000","last_change":"2024-09-10T21:44:31.398001+0000","last_active":"2024-09-10T21:44:31.406848+0000","last_peered":"2024-09-10T21:44:31.406848+0000","last_clean":"2024-09-10T21:44:31.406848+0000","last_became_active":"2024-09-10T21:44:31.391899+0000","last_became_peered":"2024-09-10T21:44:31.391899+0000","last_unstale":"2024-09-10T21:44:31.406848+0000","last_undegraded":"2024-09-10T21:44:31.406848+0000","last_fullsized":"2024-09-10T21:44:31.406848+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:12.308573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"57'4","reported_seq":17,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402797+0000","last_change":"2024-09-10T21:44:30.379105+0000","last_active":"2024-09-10T21:44:31.402797+0000","last_peered":"2024-09-10T21:44:31.402797+0000","last_clean":"2024-09-10T21:44:31.402797+0000","last_became_active":"2024-09-10T21:44:30.376451+0000","last_became_peered":"2024-09-10T21:44:30.376451+0000","last_unstale":"2024-09-10T21:44:31.402797+0000","last_undegraded":"2024-09-10T21:44:31.402797+0000","last_fullsized":"2024-09-10T21:44:31.402797+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:26:42.022890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.391608+0000","last_change":"2024-09-10T21:44:31.390579+0000","last_active":"2024-09-10T21:44:31.391608+0000","last_peered":"2024-09-10T21:44:31.391608+0000","last_clean":"2024-09-10T21:44:31.391608+0000","last_became_active":"2024-09-10T21:44:31.386874+0000","last_became_peered":"2024-09-10T21:44:31.386874+0000","last_unstale":"2024-09-10T21:44:31.391608+0000","last_undegraded":"2024-09-10T21:44:31.391608+0000","last_fullsized":"2024-09-10T21:44:31.391608+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:44:53.299991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.401671+0000","last_change":"2024-09-10T21:44:30.376223+0000","last_active":"2024-09-10T21:44:31.401671+0000","last_peered":"2024-09-10T21:44:31.401671+0000","last_clean":"2024-09-10T21:44:31.401671+0000","last_became_active":"2024-09-10T21:44:30.375951+0000","last_became_peered":"2024-09-10T21:44:30.375951+0000","last_unstale":"2024-09-10T21:44:31.401671+0000","last_undegraded":"2024-09-10T21:44:31.401671+0000","last_fullsized":"2024-09-10T21:44:31.401671+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:04.622616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402010+0000","last_change":"2024-09-10T21:44:31.393057+0000","last_active":"2024-09-10T21:44:31.402010+0000","last_peered":"2024-09-10T21:44:31.402010+0000","last_clean":"2024-09-10T21:44:31.402010+0000","last_became_active":"2024-09-10T21:44:31.389891+0000","last_became_peered":"2024-09-10T21:44:31.389891+0000","last_unstale":"2024-09-10T21:44:31.402010+0000","last_undegraded":"2024-09-10T21:44:31.402010+0000","last_fullsized":"2024-09-10T21:44:31.402010+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:15:01.862089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.408592+0000","last_change":"2024-09-10T21:44:30.374526+0000","last_active":"2024-09-10T21:44:31.408592+0000","last_peered":"2024-09-10T21:44:31.408592+0000","last_clean":"2024-09-10T21:44:31.408592+0000","last_became_active":"2024-09-10T21:44:30.374191+0000","last_became_peered":"2024-09-10T21:44:30.374191+0000","last_unstale":"2024-09-10T21:44:31.408592+0000","last_undegraded":"2024-09-10T21:44:31.408592+0000","last_fullsized":"2024-09-10T21:44:31.408592+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:16.882390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402727+0000","last_change":"2024-09-10T21:44:31.393655+0000","last_active":"2024-09-10T21:44:31.402727+0000","last_peered":"2024-09-10T21:44:31.402727+0000","last_clean":"2024-09-10T21:44:31.402727+0000","last_became_active":"2024-09-10T21:44:31.388571+0000","last_became_peered":"2024-09-10T21:44:31.388571+0000","last_unstale":"2024-09-10T21:44:31.402727+0000","last_undegraded":"2024-09-10T21:44:31.402727+0000","last_fullsized":"2024-09-10T21:44:31.402727+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:55:50.031278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"57'2","reported_seq":17,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385875+0000","last_change":"2024-09-10T21:44:30.376094+0000","last_active":"2024-09-10T21:44:32.385875+0000","last_peered":"2024-09-10T21:44:32.385875+0000","last_clean":"2024-09-10T21:44:32.385875+0000","last_became_active":"2024-09-10T21:44:30.375807+0000","last_became_peered":"2024-09-10T21:44:30.375807+0000","last_unstale":"2024-09-10T21:44:32.385875+0000","last_undegraded":"2024-09-10T21:44:32.385875+0000","last_fullsized":"2024-09-10T21:44:32.385875+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:29.278084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.401293+0000","last_change":"2024-09-10T21:44:31.395804+0000","last_active":"2024-09-10T21:44:31.401293+0000","last_peered":"2024-09-10T21:44:31.401293+0000","last_clean":"2024-09-10T21:44:31.401293+0000","last_became_active":"2024-09-10T21:44:31.395535+0000","last_became_peered":"2024-09-10T21:44:31.395535+0000","last_unstale":"2024-09-10T21:44:31.401293+0000","last_undegraded":"2024-09-10T21:44:31.401293+0000","last_fullsized":"2024-09-10T21:44:31.401293+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:06:34.062126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.386663+0000","last_change":"2024-09-10T21:44:30.380848+0000","last_active":"2024-09-10T21:44:32.386663+0000","last_peered":"2024-09-10T21:44:32.386663+0000","last_clean":"2024-09-10T21:44:32.386663+0000","last_became_active":"2024-09-10T21:44:30.380572+0000","last_became_peered":"2024-09-10T21:44:30.380572+0000","last_unstale":"2024-09-10T21:44:32.386663+0000","last_undegraded":"2024-09-10T21:44:32.386663+0000","last_fullsized":"2024-09-10T21:44:32.386663+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:37:33.245055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.398630+0000","last_change":"2024-09-10T21:44:31.394405+0000","last_active":"2024-09-10T21:44:31.398630+0000","last_peered":"2024-09-10T21:44:31.398630+0000","last_clean":"2024-09-10T21:44:31.398630+0000","last_became_active":"2024-09-10T21:44:31.394142+0000","last_became_peered":"2024-09-10T21:44:31.394142+0000","last_unstale":"2024-09-10T21:44:31.398630+0000","last_undegraded":"2024-09-10T21:44:31.398630+0000","last_fullsized":"2024-09-10T21:44:31.398630+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:04.427921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402707+0000","last_change":"2024-09-10T21:44:30.380087+0000","last_active":"2024-09-10T21:44:31.402707+0000","last_peered":"2024-09-10T21:44:31.402707+0000","last_clean":"2024-09-10T21:44:31.402707+0000","last_became_active":"2024-09-10T21:44:30.378757+0000","last_became_peered":"2024-09-10T21:44:30.378757+0000","last_unstale":"2024-09-10T21:44:31.402707+0000","last_undegraded":"2024-09-10T21:44:31.402707+0000","last_fullsized":"2024-09-10T21:44:31.402707+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:45:14.854663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.619707+0000","last_change":"2024-09-10T21:44:31.375815+0000","last_active":"2024-09-10T21:44:32.619707+0000","last_peered":"2024-09-10T21:44:32.619707+0000","last_clean":"2024-09-10T21:44:32.619707+0000","last_became_active":"2024-09-10T21:44:31.375142+0000","last_became_peered":"2024-09-10T21:44:31.375142+0000","last_unstale":"2024-09-10T21:44:32.619707+0000","last_undegraded":"2024-09-10T21:44:32.619707+0000","last_fullsized":"2024-09-10T21:44:32.619707+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:32.072203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385116+0000","last_change":"2024-09-10T21:44:30.375706+0000","last_active":"2024-09-10T21:44:32.385116+0000","last_peered":"2024-09-10T21:44:32.385116+0000","last_clean":"2024-09-10T21:44:32.385116+0000","last_became_active":"2024-09-10T21:44:30.375436+0000","last_became_peered":"2024-09-10T21:44:30.375436+0000","last_unstale":"2024-09-10T21:44:32.385116+0000","last_undegraded":"2024-09-10T21:44:32.385116+0000","last_fullsized":"2024-09-10T21:44:32.385116+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:43.769901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.406643+0000","last_change":"2024-09-10T21:44:31.403557+0000","last_active":"2024-09-10T21:44:31.406643+0000","last_peered":"2024-09-10T21:44:31.406643+0000","last_clean":"2024-09-10T21:44:31.406643+0000","last_became_active":"2024-09-10T21:44:31.401444+0000","last_became_peered":"2024-09-10T21:44:31.401444+0000","last_unstale":"2024-09-10T21:44:31.406643+0000","last_undegraded":"2024-09-10T21:44:31.406643+0000","last_fullsized":"2024-09-10T21:44:31.406643+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:37:13.559247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385484+0000","last_change":"2024-09-10T21:44:30.373702+0000","last_active":"2024-09-10T21:44:32.385484+0000","last_peered":"2024-09-10T21:44:32.385484+0000","last_clean":"2024-09-10T21:44:32.385484+0000","last_became_active":"2024-09-10T21:44:30.373437+0000","last_became_peered":"2024-09-10T21:44:30.373437+0000","last_unstale":"2024-09-10T21:44:32.385484+0000","last_undegraded":"2024-09-10T21:44:32.385484+0000","last_fullsized":"2024-09-10T21:44:32.385484+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:21.917058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385598+0000","last_change":"2024-09-10T21:44:31.386834+0000","last_active":"2024-09-10T21:44:32.385598+0000","last_peered":"2024-09-10T21:44:32.385598+0000","last_clean":"2024-09-10T21:44:32.385598+0000","last_became_active":"2024-09-10T21:44:31.386245+0000","last_became_peered":"2024-09-10T21:44:31.386245+0000","last_unstale":"2024-09-10T21:44:32.385598+0000","last_undegraded":"2024-09-10T21:44:32.385598+0000","last_fullsized":"2024-09-10T21:44:32.385598+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:45:15.865616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.372075+0000","last_change":"2024-09-10T21:44:30.384555+0000","last_active":"2024-09-10T21:44:31.372075+0000","last_peered":"2024-09-10T21:44:31.372075+0000","last_clean":"2024-09-10T21:44:31.372075+0000","last_became_active":"2024-09-10T21:44:30.384278+0000","last_became_peered":"2024-09-10T21:44:30.384278+0000","last_unstale":"2024-09-10T21:44:31.372075+0000","last_undegraded":"2024-09-10T21:44:31.372075+0000","last_fullsized":"2024-09-10T21:44:31.372075+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:17:59.701329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.398052+0000","last_change":"2024-09-10T21:44:31.396217+0000","last_active":"2024-09-10T21:44:31.398052+0000","last_peered":"2024-09-10T21:44:31.398052+0000","last_clean":"2024-09-10T21:44:31.398052+0000","last_became_active":"2024-09-10T21:44:31.392774+0000","last_became_peered":"2024-09-10T21:44:31.392774+0000","last_unstale":"2024-09-10T21:44:31.398052+0000","last_undegraded":"2024-09-10T21:44:31.398052+0000","last_fullsized":"2024-09-10T21:44:31.398052+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:20.337480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.371370+0000","last_change":"2024-09-10T21:44:30.383267+0000","last_active":"2024-09-10T21:44:31.371370+0000","last_peered":"2024-09-10T21:44:31.371370+0000","last_clean":"2024-09-10T21:44:31.371370+0000","last_became_active":"2024-09-10T21:44:30.382042+0000","last_became_peered":"2024-09-10T21:44:30.382042+0000","last_unstale":"2024-09-10T21:44:31.371370+0000","last_undegraded":"2024-09-10T21:44:31.371370+0000","last_fullsized":"2024-09-10T21:44:31.371370+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:12:36.873002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.390496+0000","last_change":"2024-09-10T21:44:31.388118+0000","last_active":"2024-09-10T21:44:31.390496+0000","last_peered":"2024-09-10T21:44:31.390496+0000","last_clean":"2024-09-10T21:44:31.390496+0000","last_became_active":"2024-09-10T21:44:31.387812+0000","last_became_peered":"2024-09-10T21:44:31.387812+0000","last_unstale":"2024-09-10T21:44:31.390496+0000","last_undegraded":"2024-09-10T21:44:31.390496+0000","last_fullsized":"2024-09-10T21:44:31.390496+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:40:14.803952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"57'3","reported_seq":16,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.395594+0000","last_change":"2024-09-10T21:44:30.378538+0000","last_active":"2024-09-10T21:44:31.395594+0000","last_peered":"2024-09-10T21:44:31.395594+0000","last_clean":"2024-09-10T21:44:31.395594+0000","last_became_active":"2024-09-10T21:44:30.378249+0000","last_became_peered":"2024-09-10T21:44:30.378249+0000","last_unstale":"2024-09-10T21:44:31.395594+0000","last_undegraded":"2024-09-10T21:44:31.395594+0000","last_fullsized":"2024-09-10T21:44:31.395594+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:17.140823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.393898+0000","last_change":"2024-09-10T21:44:31.387175+0000","last_active":"2024-09-10T21:44:31.393898+0000","last_peered":"2024-09-10T21:44:31.393898+0000","last_clean":"2024-09-10T21:44:31.393898+0000","last_became_active":"2024-09-10T21:44:31.386133+0000","last_became_peered":"2024-09-10T21:44:31.386133+0000","last_unstale":"2024-09-10T21:44:31.393898+0000","last_undegraded":"2024-09-10T21:44:31.393898+0000","last_fullsized":"2024-09-10T21:44:31.393898+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:35.642650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"57'6","reported_seq":21,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.387075+0000","last_change":"2024-09-10T21:44:30.371690+0000","last_active":"2024-09-10T21:44:32.387075+0000","last_peered":"2024-09-10T21:44:32.387075+0000","last_clean":"2024-09-10T21:44:32.387075+0000","last_became_active":"2024-09-10T21:44:30.371413+0000","last_became_peered":"2024-09-10T21:44:30.371413+0000","last_unstale":"2024-09-10T21:44:32.387075+0000","last_undegraded":"2024-09-10T21:44:32.387075+0000","last_fullsized":"2024-09-10T21:44:32.387075+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:48:05.494281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385860+0000","last_change":"2024-09-10T21:44:31.387479+0000","last_active":"2024-09-10T21:44:32.385860+0000","last_peered":"2024-09-10T21:44:32.385860+0000","last_clean":"2024-09-10T21:44:32.385860+0000","last_became_active":"2024-09-10T21:44:31.386309+0000","last_became_peered":"2024-09-10T21:44:31.386309+0000","last_unstale":"2024-09-10T21:44:32.385860+0000","last_undegraded":"2024-09-10T21:44:32.385860+0000","last_fullsized":"2024-09-10T21:44:32.385860+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:56:49.049473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.404334+0000","last_change":"2024-09-10T21:44:30.384321+0000","last_active":"2024-09-10T21:44:31.404334+0000","last_peered":"2024-09-10T21:44:31.404334+0000","last_clean":"2024-09-10T21:44:31.404334+0000","last_became_active":"2024-09-10T21:44:30.384057+0000","last_became_peered":"2024-09-10T21:44:30.384057+0000","last_unstale":"2024-09-10T21:44:31.404334+0000","last_undegraded":"2024-09-10T21:44:31.404334+0000","last_fullsized":"2024-09-10T21:44:31.404334+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:32.514436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.385396+0000","last_change":"2024-09-10T21:44:31.384148+0000","last_active":"2024-09-10T21:44:31.385396+0000","last_peered":"2024-09-10T21:44:31.385396+0000","last_clean":"2024-09-10T21:44:31.385396+0000","last_became_active":"2024-09-10T21:44:31.383857+0000","last_became_peered":"2024-09-10T21:44:31.383857+0000","last_unstale":"2024-09-10T21:44:31.385396+0000","last_undegraded":"2024-09-10T21:44:31.385396+0000","last_fullsized":"2024-09-10T21:44:31.385396+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:50.665870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.619629+0000","last_change":"2024-09-10T21:44:30.376207+0000","last_active":"2024-09-10T21:44:32.619629+0000","last_peered":"2024-09-10T21:44:32.619629+0000","last_clean":"2024-09-10T21:44:32.619629+0000","last_became_active":"2024-09-10T21:44:30.375912+0000","last_became_peered":"2024-09-10T21:44:30.375912+0000","last_unstale":"2024-09-10T21:44:32.619629+0000","last_undegraded":"2024-09-10T21:44:32.619629+0000","last_fullsized":"2024-09-10T21:44:32.619629+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:36:50.546531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385360+0000","last_change":"2024-09-10T21:44:31.386029+0000","last_active":"2024-09-10T21:44:32.385360+0000","last_peered":"2024-09-10T21:44:32.385360+0000","last_clean":"2024-09-10T21:44:32.385360+0000","last_became_active":"2024-09-10T21:44:31.384174+0000","last_became_peered":"2024-09-10T21:44:31.384174+0000","last_unstale":"2024-09-10T21:44:32.385360+0000","last_undegraded":"2024-09-10T21:44:32.385360+0000","last_fullsized":"2024-09-10T21:44:32.385360+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:21.831628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385227+0000","last_change":"2024-09-10T21:44:30.376392+0000","last_active":"2024-09-10T21:44:32.385227+0000","last_peered":"2024-09-10T21:44:32.385227+0000","last_clean":"2024-09-10T21:44:32.385227+0000","last_became_active":"2024-09-10T21:44:30.376100+0000","last_became_peered":"2024-09-10T21:44:30.376100+0000","last_unstale":"2024-09-10T21:44:32.385227+0000","last_undegraded":"2024-09-10T21:44:32.385227+0000","last_fullsized":"2024-09-10T21:44:32.385227+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:16:51.402154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402050+0000","last_change":"2024-09-10T21:44:31.397163+0000","last_active":"2024-09-10T21:44:31.402050+0000","last_peered":"2024-09-10T21:44:31.402050+0000","last_clean":"2024-09-10T21:44:31.402050+0000","last_became_active":"2024-09-10T21:44:31.396846+0000","last_became_peered":"2024-09-10T21:44:31.396846+0000","last_unstale":"2024-09-10T21:44:31.402050+0000","last_undegraded":"2024-09-10T21:44:31.402050+0000","last_fullsized":"2024-09-10T21:44:31.402050+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:29.828043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"57'8","reported_seq":21,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.399807+0000","last_change":"2024-09-10T21:44:30.378527+0000","last_active":"2024-09-10T21:44:31.399807+0000","last_peered":"2024-09-10T21:44:31.399807+0000","last_clean":"2024-09-10T21:44:31.399807+0000","last_became_active":"2024-09-10T21:44:30.377364+0000","last_became_peered":"2024-09-10T21:44:30.377364+0000","last_unstale":"2024-09-10T21:44:31.399807+0000","last_undegraded":"2024-09-10T21:44:31.399807+0000","last_fullsized":"2024-09-10T21:44:31.399807+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:32:18.682249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.383192+0000","last_change":"2024-09-10T21:44:31.378653+0000","last_active":"2024-09-10T21:44:31.383192+0000","last_peered":"2024-09-10T21:44:31.383192+0000","last_clean":"2024-09-10T21:44:31.383192+0000","last_became_active":"2024-09-10T21:44:31.378328+0000","last_became_peered":"2024-09-10T21:44:31.378328+0000","last_unstale":"2024-09-10T21:44:31.383192+0000","last_undegraded":"2024-09-10T21:44:31.383192+0000","last_fullsized":"2024-09-10T21:44:31.383192+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:13.123398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"57'1","reported_seq":14,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.405775+0000","last_change":"2024-09-10T21:44:30.380762+0000","last_active":"2024-09-10T21:44:31.405775+0000","last_peered":"2024-09-10T21:44:31.405775+0000","last_clean":"2024-09-10T21:44:31.405775+0000","last_became_active":"2024-09-10T21:44:30.380524+0000","last_became_peered":"2024-09-10T21:44:30.380524+0000","last_unstale":"2024-09-10T21:44:31.405775+0000","last_undegraded":"2024-09-10T21:44:31.405775+0000","last_fullsized":"2024-09-10T21:44:31.405775+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:06.795703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385892+0000","last_change":"2024-09-10T21:44:31.393563+0000","last_active":"2024-09-10T21:44:32.385892+0000","last_peered":"2024-09-10T21:44:32.385892+0000","last_clean":"2024-09-10T21:44:32.385892+0000","last_became_active":"2024-09-10T21:44:31.392646+0000","last_became_peered":"2024-09-10T21:44:31.392646+0000","last_unstale":"2024-09-10T21:44:32.385892+0000","last_undegraded":"2024-09-10T21:44:32.385892+0000","last_fullsized":"2024-09-10T21:44:32.385892+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:36.451092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.406795+0000","last_change":"2024-09-10T21:44:30.384439+0000","last_active":"2024-09-10T21:44:31.406795+0000","last_peered":"2024-09-10T21:44:31.406795+0000","last_clean":"2024-09-10T21:44:31.406795+0000","last_became_active":"2024-09-10T21:44:30.384166+0000","last_became_peered":"2024-09-10T21:44:30.384166+0000","last_unstale":"2024-09-10T21:44:31.406795+0000","last_undegraded":"2024-09-10T21:44:31.406795+0000","last_fullsized":"2024-09-10T21:44:31.406795+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:41:35.007284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.398553+0000","last_change":"2024-09-10T21:44:31.390661+0000","last_active":"2024-09-10T21:44:31.398553+0000","last_peered":"2024-09-10T21:44:31.398553+0000","last_clean":"2024-09-10T21:44:31.398553+0000","last_became_active":"2024-09-10T21:44:31.385105+0000","last_became_peered":"2024-09-10T21:44:31.385105+0000","last_unstale":"2024-09-10T21:44:31.398553+0000","last_undegraded":"2024-09-10T21:44:31.398553+0000","last_fullsized":"2024-09-10T21:44:31.398553+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:31:21.472705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"57'1","reported_seq":16,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385374+0000","last_change":"2024-09-10T21:44:30.379029+0000","last_active":"2024-09-10T21:44:32.385374+0000","last_peered":"2024-09-10T21:44:32.385374+0000","last_clean":"2024-09-10T21:44:32.385374+0000","last_became_active":"2024-09-10T21:44:30.378729+0000","last_became_peered":"2024-09-10T21:44:30.378729+0000","last_unstale":"2024-09-10T21:44:32.385374+0000","last_undegraded":"2024-09-10T21:44:32.385374+0000","last_fullsized":"2024-09-10T21:44:32.385374+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:42.843155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.395530+0000","last_change":"2024-09-10T21:44:31.388055+0000","last_active":"2024-09-10T21:44:31.395530+0000","last_peered":"2024-09-10T21:44:31.395530+0000","last_clean":"2024-09-10T21:44:31.395530+0000","last_became_active":"2024-09-10T21:44:31.386823+0000","last_became_peered":"2024-09-10T21:44:31.386823+0000","last_unstale":"2024-09-10T21:44:31.395530+0000","last_undegraded":"2024-09-10T21:44:31.395530+0000","last_fullsized":"2024-09-10T21:44:31.395530+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:05.840706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.373395+0000","last_change":"2024-09-10T21:44:30.373250+0000","last_active":"2024-09-10T21:44:31.373395+0000","last_peered":"2024-09-10T21:44:31.373395+0000","last_clean":"2024-09-10T21:44:31.373395+0000","last_became_active":"2024-09-10T21:44:30.372938+0000","last_became_peered":"2024-09-10T21:44:30.372938+0000","last_unstale":"2024-09-10T21:44:31.373395+0000","last_undegraded":"2024-09-10T21:44:31.373395+0000","last_fullsized":"2024-09-10T21:44:31.373395+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:15.879793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.386101+0000","last_change":"2024-09-10T21:44:31.389808+0000","last_active":"2024-09-10T21:44:32.386101+0000","last_peered":"2024-09-10T21:44:32.386101+0000","last_clean":"2024-09-10T21:44:32.386101+0000","last_became_active":"2024-09-10T21:44:31.386631+0000","last_became_peered":"2024-09-10T21:44:31.386631+0000","last_unstale":"2024-09-10T21:44:32.386101+0000","last_undegraded":"2024-09-10T21:44:32.386101+0000","last_fullsized":"2024-09-10T21:44:32.386101+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:41:45.635509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.389477+0000","last_change":"2024-09-10T21:44:30.378706+0000","last_active":"2024-09-10T21:44:31.389477+0000","last_peered":"2024-09-10T21:44:31.389477+0000","last_clean":"2024-09-10T21:44:31.389477+0000","last_became_active":"2024-09-10T21:44:30.378432+0000","last_became_peered":"2024-09-10T21:44:30.378432+0000","last_unstale":"2024-09-10T21:44:31.389477+0000","last_undegraded":"2024-09-10T21:44:31.389477+0000","last_fullsized":"2024-09-10T21:44:31.389477+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:25:49.249958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402818+0000","last_change":"2024-09-10T21:44:31.393342+0000","last_active":"2024-09-10T21:44:31.402818+0000","last_peered":"2024-09-10T21:44:31.402818+0000","last_clean":"2024-09-10T21:44:31.402818+0000","last_became_active":"2024-09-10T21:44:31.389027+0000","last_became_peered":"2024-09-10T21:44:31.389027+0000","last_unstale":"2024-09-10T21:44:31.402818+0000","last_undegraded":"2024-09-10T21:44:31.402818+0000","last_fullsized":"2024-09-10T21:44:31.402818+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:01.195088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.382302+0000","last_change":"2024-09-10T21:44:30.380686+0000","last_active":"2024-09-10T21:44:31.382302+0000","last_peered":"2024-09-10T21:44:31.382302+0000","last_clean":"2024-09-10T21:44:31.382302+0000","last_became_active":"2024-09-10T21:44:30.380445+0000","last_became_peered":"2024-09-10T21:44:30.380445+0000","last_unstale":"2024-09-10T21:44:31.382302+0000","last_undegraded":"2024-09-10T21:44:31.382302+0000","last_fullsized":"2024-09-10T21:44:31.382302+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:32.526008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.392444+0000","last_change":"2024-09-10T21:44:31.384342+0000","last_active":"2024-09-10T21:44:31.392444+0000","last_peered":"2024-09-10T21:44:31.392444+0000","last_clean":"2024-09-10T21:44:31.392444+0000","last_became_active":"2024-09-10T21:44:31.384071+0000","last_became_peered":"2024-09-10T21:44:31.384071+0000","last_unstale":"2024-09-10T21:44:31.392444+0000","last_undegraded":"2024-09-10T21:44:31.392444+0000","last_fullsized":"2024-09-10T21:44:31.392444+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:55.051489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.387249+0000","last_change":"2024-09-10T21:44:30.383485+0000","last_active":"2024-09-10T21:44:32.387249+0000","last_peered":"2024-09-10T21:44:32.387249+0000","last_clean":"2024-09-10T21:44:32.387249+0000","last_became_active":"2024-09-10T21:44:30.383194+0000","last_became_peered":"2024-09-10T21:44:30.383194+0000","last_unstale":"2024-09-10T21:44:32.387249+0000","last_undegraded":"2024-09-10T21:44:32.387249+0000","last_fullsized":"2024-09-10T21:44:32.387249+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:57.794431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385800+0000","last_change":"2024-09-10T21:44:31.380739+0000","last_active":"2024-09-10T21:44:32.385800+0000","last_peered":"2024-09-10T21:44:32.385800+0000","last_clean":"2024-09-10T21:44:32.385800+0000","last_became_active":"2024-09-10T21:44:31.380320+0000","last_became_peered":"2024-09-10T21:44:31.380320+0000","last_unstale":"2024-09-10T21:44:32.385800+0000","last_undegraded":"2024-09-10T21:44:32.385800+0000","last_fullsized":"2024-09-10T21:44:32.385800+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:21.913494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"57'2","reported_seq":15,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402520+0000","last_change":"2024-09-10T21:44:30.384908+0000","last_active":"2024-09-10T21:44:31.402520+0000","last_peered":"2024-09-10T21:44:31.402520+0000","last_clean":"2024-09-10T21:44:31.402520+0000","last_became_active":"2024-09-10T21:44:30.383035+0000","last_became_peered":"2024-09-10T21:44:30.383035+0000","last_unstale":"2024-09-10T21:44:31.402520+0000","last_undegraded":"2024-09-10T21:44:31.402520+0000","last_fullsized":"2024-09-10T21:44:31.402520+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:39.962769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.386100+0000","last_change":"2024-09-10T21:44:31.381428+0000","last_active":"2024-09-10T21:44:32.386100+0000","last_peered":"2024-09-10T21:44:32.386100+0000","last_clean":"2024-09-10T21:44:32.386100+0000","last_became_active":"2024-09-10T21:44:31.381108+0000","last_became_peered":"2024-09-10T21:44:31.381108+0000","last_unstale":"2024-09-10T21:44:32.386100+0000","last_undegraded":"2024-09-10T21:44:32.386100+0000","last_fullsized":"2024-09-10T21:44:32.386100+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:43:46.961432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.373392+0000","last_change":"2024-09-10T21:44:30.378892+0000","last_active":"2024-09-10T21:44:31.373392+0000","last_peered":"2024-09-10T21:44:31.373392+0000","last_clean":"2024-09-10T21:44:31.373392+0000","last_became_active":"2024-09-10T21:44:30.378624+0000","last_became_peered":"2024-09-10T21:44:30.378624+0000","last_unstale":"2024-09-10T21:44:31.373392+0000","last_undegraded":"2024-09-10T21:44:31.373392+0000","last_fullsized":"2024-09-10T21:44:31.373392+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:53.123593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.398755+0000","last_change":"2024-09-10T21:44:31.394359+0000","last_active":"2024-09-10T21:44:31.398755+0000","last_peered":"2024-09-10T21:44:31.398755+0000","last_clean":"2024-09-10T21:44:31.398755+0000","last_became_active":"2024-09-10T21:44:31.394059+0000","last_became_peered":"2024-09-10T21:44:31.394059+0000","last_unstale":"2024-09-10T21:44:31.398755+0000","last_undegraded":"2024-09-10T21:44:31.398755+0000","last_fullsized":"2024-09-10T21:44:31.398755+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:04.870549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.619763+0000","last_change":"2024-09-10T21:44:30.374777+0000","last_active":"2024-09-10T21:44:32.619763+0000","last_peered":"2024-09-10T21:44:32.619763+0000","last_clean":"2024-09-10T21:44:32.619763+0000","last_became_active":"2024-09-10T21:44:30.374504+0000","last_became_peered":"2024-09-10T21:44:30.374504+0000","last_unstale":"2024-09-10T21:44:32.619763+0000","last_undegraded":"2024-09-10T21:44:32.619763+0000","last_fullsized":"2024-09-10T21:44:32.619763+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:07.630691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.399912+0000","last_change":"2024-09-10T21:44:31.393032+0000","last_active":"2024-09-10T21:44:31.399912+0000","last_peered":"2024-09-10T21:44:31.399912+0000","last_clean":"2024-09-10T21:44:31.399912+0000","last_became_active":"2024-09-10T21:44:31.388203+0000","last_became_peered":"2024-09-10T21:44:31.388203+0000","last_unstale":"2024-09-10T21:44:31.399912+0000","last_undegraded":"2024-09-10T21:44:31.399912+0000","last_fullsized":"2024-09-10T21:44:31.399912+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:10:41.208914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.385983+0000","last_change":"2024-09-10T21:44:30.381228+0000","last_active":"2024-09-10T21:44:32.385983+0000","last_peered":"2024-09-10T21:44:32.385983+0000","last_clean":"2024-09-10T21:44:32.385983+0000","last_became_active":"2024-09-10T21:44:30.380019+0000","last_became_peered":"2024-09-10T21:44:30.380019+0000","last_unstale":"2024-09-10T21:44:32.385983+0000","last_undegraded":"2024-09-10T21:44:32.385983+0000","last_fullsized":"2024-09-10T21:44:32.385983+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:07:15.879793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.401884+0000","last_change":"2024-09-10T21:44:31.396905+0000","last_active":"2024-09-10T21:44:31.401884+0000","last_peered":"2024-09-10T21:44:31.401884+0000","last_clean":"2024-09-10T21:44:31.401884+0000","last_became_active":"2024-09-10T21:44:31.395553+0000","last_became_peered":"2024-09-10T21:44:31.395553+0000","last_unstale":"2024-09-10T21:44:31.401884+0000","last_undegraded":"2024-09-10T21:44:31.401884+0000","last_fullsized":"2024-09-10T21:44:31.401884+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:56.264971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.380804+0000","last_change":"2024-09-10T21:44:30.381814+0000","last_active":"2024-09-10T21:44:31.380804+0000","last_peered":"2024-09-10T21:44:31.380804+0000","last_clean":"2024-09-10T21:44:31.380804+0000","last_became_active":"2024-09-10T21:44:30.379714+0000","last_became_peered":"2024-09-10T21:44:30.379714+0000","last_unstale":"2024-09-10T21:44:31.380804+0000","last_undegraded":"2024-09-10T21:44:31.380804+0000","last_fullsized":"2024-09-10T21:44:31.380804+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:20:39.445476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.402067+0000","last_change":"2024-09-10T21:44:31.396981+0000","last_active":"2024-09-10T21:44:31.402067+0000","last_peered":"2024-09-10T21:44:31.402067+0000","last_clean":"2024-09-10T21:44:31.402067+0000","last_became_active":"2024-09-10T21:44:31.396679+0000","last_became_peered":"2024-09-10T21:44:31.396679+0000","last_unstale":"2024-09-10T21:44:31.402067+0000","last_undegraded":"2024-09-10T21:44:31.402067+0000","last_fullsized":"2024-09-10T21:44:31.402067+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:05.332185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.386436+0000","last_change":"2024-09-10T21:44:30.372026+0000","last_active":"2024-09-10T21:44:32.386436+0000","last_peered":"2024-09-10T21:44:32.386436+0000","last_clean":"2024-09-10T21:44:32.386436+0000","last_became_active":"2024-09-10T21:44:30.371748+0000","last_became_peered":"2024-09-10T21:44:30.371748+0000","last_unstale":"2024-09-10T21:44:32.386436+0000","last_undegraded":"2024-09-10T21:44:32.386436+0000","last_fullsized":"2024-09-10T21:44:32.386436+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:10.212300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.387345+0000","last_change":"2024-09-10T21:44:31.385405+0000","last_active":"2024-09-10T21:44:31.387345+0000","last_peered":"2024-09-10T21:44:31.387345+0000","last_clean":"2024-09-10T21:44:31.387345+0000","last_became_active":"2024-09-10T21:44:31.384838+0000","last_became_peered":"2024-09-10T21:44:31.384838+0000","last_unstale":"2024-09-10T21:44:31.387345+0000","last_undegraded":"2024-09-10T21:44:31.387345+0000","last_fullsized":"2024-09-10T21:44:31.387345+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:33.683588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.381148+0000","last_change":"2024-09-10T21:44:30.373885+0000","last_active":"2024-09-10T21:44:31.381148+0000","last_peered":"2024-09-10T21:44:31.381148+0000","last_clean":"2024-09-10T21:44:31.381148+0000","last_became_active":"2024-09-10T21:44:30.373603+0000","last_became_peered":"2024-09-10T21:44:30.373603+0000","last_unstale":"2024-09-10T21:44:31.381148+0000","last_undegraded":"2024-09-10T21:44:31.381148+0000","last_fullsized":"2024-09-10T21:44:31.381148+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:42:46.950505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.398321+0000","last_change":"2024-09-10T21:44:31.387375+0000","last_active":"2024-09-10T21:44:31.398321+0000","last_peered":"2024-09-10T21:44:31.398321+0000","last_clean":"2024-09-10T21:44:31.398321+0000","last_became_active":"2024-09-10T21:44:31.385735+0000","last_became_peered":"2024-09-10T21:44:31.385735+0000","last_unstale":"2024-09-10T21:44:31.398321+0000","last_undegraded":"2024-09-10T21:44:31.398321+0000","last_fullsized":"2024-09-10T21:44:31.398321+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:53:29.763778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":13,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.370538+0000","last_change":"2024-09-10T21:44:30.378015+0000","last_active":"2024-09-10T21:44:31.370538+0000","last_peered":"2024-09-10T21:44:31.370538+0000","last_clean":"2024-09-10T21:44:31.370538+0000","last_became_active":"2024-09-10T21:44:30.377465+0000","last_became_peered":"2024-09-10T21:44:30.377465+0000","last_unstale":"2024-09-10T21:44:31.370538+0000","last_undegraded":"2024-09-10T21:44:31.370538+0000","last_fullsized":"2024-09-10T21:44:31.370538+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:57:56.422703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.398567+0000","last_change":"2024-09-10T21:44:31.393215+0000","last_active":"2024-09-10T21:44:31.398567+0000","last_peered":"2024-09-10T21:44:31.398567+0000","last_clean":"2024-09-10T21:44:31.398567+0000","last_became_active":"2024-09-10T21:44:31.392771+0000","last_became_peered":"2024-09-10T21:44:31.392771+0000","last_unstale":"2024-09-10T21:44:31.398567+0000","last_undegraded":"2024-09-10T21:44:31.398567+0000","last_fullsized":"2024-09-10T21:44:31.398567+0000","mapping_epoch":56,"log_start":"0'0","ondisk_log_start":"0'0","created":56,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:30.350764+0000","last_clean_scrub_stamp":"2024-09-10T21:44:30.350764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:18.404850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-10T21:44:32.619552+0000","last_change":"2024-09-10T21:44:30.372724+0000","last_active":"2024-09-10T21:44:32.619552+0000","last_peered":"2024-09-10T21:44:32.619552+0000","last_clean":"2024-09-10T21:44:32.619552+0000","last_became_active":"2024-09-10T21:44:30.372476+0000","last_became_peered":"2024-09-10T21:44:30.372476+0000","last_unstale":"2024-09-10T21:44:32.619552+0000","last_undegraded":"2024-09-10T21:44:32.619552+0000","last_fullsized":"2024-09-10T21:44:32.619552+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:44:29.339811+0000","last_clean_scrub_stamp":"2024-09-10T21:44:29.339811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:28:04.578355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":175,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-10T21:44:31.369773+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:44:31.369773+0000","last_peered":"2024-09-10T21:44:31.369773+0000","last_clean":"2024-09-10T21:44:31.369773+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:44:31.369773+0000","last_undegraded":"2024-09-10T21:44:31.369773+0000","last_fullsized":"2024-09-10T21:44:31.369773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509475,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27692,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344148,"statfs":{"total":96636764160,"available":96608407552,"internally_reserved":0,"allocated":675840,"data_stored":278251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509475,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27704,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344136,"statfs":{"total":96636764160,"available":96608395264,"internally_reserved":0,"allocated":684032,"data_stored":278903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509474,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31868,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339972,"statfs":{"total":96636764160,"available":96604131328,"internally_reserved":0,"allocated":688128,"data_stored":279863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509474,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31864,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339976,"statfs":{"total":96636764160,"available":96604135424,"internally_reserved":0,"allocated":684032,"data_stored":278307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509476,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31860,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339980,"statfs":{"total":96636764160,"available":96604139520,"internally_reserved":0,"allocated":679936,"data_stored":278341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509474,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1286144,"data_stored":870813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542177,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1257472,"data_stored":861844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542178,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31848,"kb_used_data":660,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339992,"statfs":{"total":96636764160,"available":96604151808,"internally_reserved":0,"allocated":675840,"data_stored":278251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:44:36.062 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:44:36.063 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:44:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.266+0000 7ff8c1c00640 1 -- 172.21.15.7:0/2948563793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8a0003a00 msgr2=0x7ff8a0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.266+0000 7ff8c1c00640 1 --2- 172.21.15.7:0/2948563793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8a0003a00 0x7ff8a0003e20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac009b50 tx=0x7ff8a0005830 comp rx=0 tx=0).stop 2024-09-10T21:44:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.266+0000 7ff8bb400640 1 -- 172.21.15.7:0/2948563793 reap_dead start 2024-09-10T21:44:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.266+0000 7ff8c1c00640 1 -- 172.21.15.7:0/2948563793 shutdown_connections 2024-09-10T21:44:36.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.266+0000 7ff8c1c00640 1 -- 172.21.15.7:0/2948563793 >> 172.21.15.7:0/2948563793 conn(0x7ff8bc0b16d0 msgr2=0x7ff8bc0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:36.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.266+0000 7ff8c1c00640 1 -- 172.21.15.7:0/2948563793 shutdown_connections 2024-09-10T21:44:36.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.266+0000 7ff8c1c00640 1 -- 172.21.15.7:0/2948563793 wait complete. 2024-09-10T21:44:36.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.268+0000 7ff8c1c00640 1 Processor -- start 2024-09-10T21:44:36.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.268+0000 7ff8c1c00640 1 -- start start 2024-09-10T21:44:36.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.268+0000 7ff8c1c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff8a0003a00 0x7ff8bc0c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:36.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.269+0000 7ff8c1c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc1f16e0 0x7ff8bc1f1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:36.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.269+0000 7ff8c1c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8bc156110 0x7ff8bc156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:36.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.269+0000 7ff8c1c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc0cf2f0 con 0x7ff8bc1f16e0 2024-09-10T21:44:36.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.269+0000 7ff8c1c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff8bc0cf170 con 0x7ff8bc156110 2024-09-10T21:44:36.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.269+0000 7ff8c1c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ff8bc0cf470 con 0x7ff8a0003a00 2024-09-10T21:44:36.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.269+0000 7ff8b2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc1f16e0 0x7ff8bc1f1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:36.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.269+0000 7ff8b2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc1f16e0 0x7ff8bc1f1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43522/0 (socket says 172.21.15.7:43522) 2024-09-10T21:44:36.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.269+0000 7ff8b2000640 1 -- 172.21.15.7:0/2671254370 learned_addr learned my addr 172.21.15.7:0/2671254370 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:36.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.269+0000 7ff8baa00640 1 --2- 172.21.15.7:0/2671254370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff8a0003a00 0x7ff8bc0c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:36.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.269+0000 7ff8bb400640 1 --2- 172.21.15.7:0/2671254370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8bc156110 0x7ff8bc156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:36.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.269+0000 7ff8b2000640 1 -- 172.21.15.7:0/2671254370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff8a0003a00 msgr2=0x7ff8bc0c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:36.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.269+0000 7ff8b2000640 1 --2- 172.21.15.7:0/2671254370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff8a0003a00 0x7ff8bc0c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.269+0000 7ff8b2000640 1 -- 172.21.15.7:0/2671254370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8bc156110 msgr2=0x7ff8bc156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:36.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.269+0000 7ff8b2000640 1 --2- 172.21.15.7:0/2671254370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8bc156110 0x7ff8bc156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.269+0000 7ff8b2000640 1 -- 172.21.15.7:0/2671254370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8bc156a90 con 0x7ff8bc1f16e0 2024-09-10T21:44:36.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.270+0000 7ff8b2000640 1 --2- 172.21.15.7:0/2671254370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc1f16e0 0x7ff8bc1f1b60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4011ae0 tx=0x7ff8a4011ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:36.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.270+0000 7ff8b3400640 1 -- 172.21.15.7:0/2671254370 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a400fbb0 con 0x7ff8bc1f16e0 2024-09-10T21:44:36.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.270+0000 7ff8b3400640 1 -- 172.21.15.7:0/2671254370 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8a400fd50 con 0x7ff8bc1f16e0 2024-09-10T21:44:36.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.270+0000 7ff8b3400640 1 -- 172.21.15.7:0/2671254370 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8a4004280 con 0x7ff8bc1f16e0 2024-09-10T21:44:36.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.270+0000 7ff8c1c00640 1 -- 172.21.15.7:0/2671254370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8bc156d60 con 0x7ff8bc1f16e0 2024-09-10T21:44:36.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.270+0000 7ff8c1c00640 1 -- 172.21.15.7:0/2671254370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8bc157320 con 0x7ff8bc1f16e0 2024-09-10T21:44:36.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.271+0000 7ff8c1c00640 1 -- 172.21.15.7:0/2671254370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8bc06f690 con 0x7ff8bc1f16e0 2024-09-10T21:44:36.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.272+0000 7ff8b3400640 1 -- 172.21.15.7:0/2671254370 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff8a40035f0 con 0x7ff8bc1f16e0 2024-09-10T21:44:36.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.273+0000 7ff8b3400640 1 --2- 172.21.15.7:0/2671254370 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff898077fa0 0x7ff89807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:36.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.273+0000 7ff8b3400640 1 -- 172.21.15.7:0/2671254370 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff8a409a710 con 0x7ff8bc1f16e0 2024-09-10T21:44:36.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.274+0000 7ff8baa00640 1 --2- 172.21.15.7:0/2671254370 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff898077fa0 0x7ff89807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:36.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.274+0000 7ff8baa00640 1 --2- 172.21.15.7:0/2671254370 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff898077fa0 0x7ff89807a460 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac035000 tx=0x7ff8ac02d040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:36.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.277+0000 7ff8b3400640 1 -- 172.21.15.7:0/2671254370 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff8a4063170 con 0x7ff8bc1f16e0 2024-09-10T21:44:36.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.545+0000 7ff8c1c00640 1 -- 172.21.15.7:0/2671254370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7ff8bc0cb230 con 0x7ff8bc1f16e0 2024-09-10T21:44:36.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.547+0000 7ff8b3400640 1 -- 172.21.15.7:0/2671254370 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v34) ==== 190+0+147 (secure 0 0 0) 0x7ff8a4062a70 con 0x7ff8bc1f16e0 2024-09-10T21:44:36.549 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-09-10T21:44:36.549 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDvvOBm0aqcFhAAtid2bnzjcpaYAbU7g0JcOA== 2024-09-10T21:44:36.549 INFO:teuthology.orchestra.run.smithi007.stdout: caps mds = "allow" 2024-09-10T21:44:36.549 INFO:teuthology.orchestra.run.smithi007.stdout: caps mon = "allow r" 2024-09-10T21:44:36.549 INFO:teuthology.orchestra.run.smithi007.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:44:36.549 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-09-10T21:44:36.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.550+0000 7ff8c1c00640 1 -- 172.21.15.7:0/2671254370 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff898077fa0 msgr2=0x7ff89807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:36.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.550+0000 7ff8c1c00640 1 --2- 172.21.15.7:0/2671254370 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff898077fa0 0x7ff89807a460 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7ff8ac035000 tx=0x7ff8ac02d040 comp rx=0 tx=0).stop 2024-09-10T21:44:36.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.550+0000 7ff8c1c00640 1 -- 172.21.15.7:0/2671254370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc1f16e0 msgr2=0x7ff8bc1f1b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:36.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.550+0000 7ff8c1c00640 1 --2- 172.21.15.7:0/2671254370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc1f16e0 0x7ff8bc1f1b60 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4011ae0 tx=0x7ff8a4011ee0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.550+0000 7ff8c1c00640 1 -- 172.21.15.7:0/2671254370 shutdown_connections 2024-09-10T21:44:36.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.550+0000 7ff8c1c00640 1 --2- 172.21.15.7:0/2671254370 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff898077fa0 0x7ff89807a460 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.550+0000 7ff8c1c00640 1 --2- 172.21.15.7:0/2671254370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8bc156110 0x7ff8bc156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.550+0000 7ff8c1c00640 1 --2- 172.21.15.7:0/2671254370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8bc1f16e0 0x7ff8bc1f1b60 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.550+0000 7ff8c1c00640 1 --2- 172.21.15.7:0/2671254370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff8a0003a00 0x7ff8bc0c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.550+0000 7ff8c1c00640 1 -- 172.21.15.7:0/2671254370 >> 172.21.15.7:0/2671254370 conn(0x7ff8bc0b16d0 msgr2=0x7ff8bc0737e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:36.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.550+0000 7ff8c1c00640 1 -- 172.21.15.7:0/2671254370 shutdown_connections 2024-09-10T21:44:36.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.550+0000 7ff8c1c00640 1 -- 172.21.15.7:0/2671254370 wait complete. 2024-09-10T21:44:36.566 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:44:36.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.725+0000 7f17a6600640 1 -- 172.21.15.7:0/3242906799 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17a015bc90 msgr2=0x7f17a00ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:36.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.725+0000 7f17a6600640 1 --2- 172.21.15.7:0/3242906799 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17a015bc90 0x7f17a00ccef0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f1790009860 tx=0x7f179002a3d0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.725+0000 7f17a6600640 1 -- 172.21.15.7:0/3242906799 shutdown_connections 2024-09-10T21:44:36.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.725+0000 7f17a6600640 1 --2- 172.21.15.7:0/3242906799 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17a015bc90 0x7f17a00ccef0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.725+0000 7f17a6600640 1 -- 172.21.15.7:0/3242906799 >> 172.21.15.7:0/3242906799 conn(0x7f17a00b16d0 msgr2=0x7f17a00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:36.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.725+0000 7f17a6600640 1 -- 172.21.15.7:0/3242906799 shutdown_connections 2024-09-10T21:44:36.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.725+0000 7f17a6600640 1 -- 172.21.15.7:0/3242906799 wait complete. 2024-09-10T21:44:36.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.727+0000 7f17a6600640 1 Processor -- start 2024-09-10T21:44:36.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.727+0000 7f17a6600640 1 -- start start 2024-09-10T21:44:36.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.728+0000 7f17a6600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17a00cd750 0x7f17a00af6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:36.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.728+0000 7f17a6600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17a015bc90 0x7f17a014d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:36.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.728+0000 7f17a6600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17a015fda0 0x7f17a014dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:36.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.728+0000 7f17a6600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f17a0070610 con 0x7f17a00cd750 2024-09-10T21:44:36.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.728+0000 7f17a6600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f17a00cf570 con 0x7f17a015fda0 2024-09-10T21:44:36.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.728+0000 7f17a6600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f17a00b00c0 con 0x7f17a015bc90 2024-09-10T21:44:36.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.728+0000 7f179f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17a00cd750 0x7f17a00af6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:36.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.728+0000 7f179f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17a00cd750 0x7f17a00af6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43528/0 (socket says 172.21.15.7:43528) 2024-09-10T21:44:36.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.728+0000 7f179f400640 1 -- 172.21.15.7:0/323852648 learned_addr learned my addr 172.21.15.7:0/323852648 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:36.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.728+0000 7f179fe00640 1 --2- 172.21.15.7:0/323852648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17a015fda0 0x7f17a014dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:36.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.728+0000 7f1797e00640 1 --2- 172.21.15.7:0/323852648 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17a015bc90 0x7f17a014d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:36.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.729+0000 7f179f400640 1 -- 172.21.15.7:0/323852648 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17a015bc90 msgr2=0x7f17a014d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:36.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.729+0000 7f179f400640 1 --2- 172.21.15.7:0/323852648 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17a015bc90 0x7f17a014d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.729+0000 7f179f400640 1 -- 172.21.15.7:0/323852648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17a015fda0 msgr2=0x7f17a014dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:36.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.729+0000 7f179f400640 1 --2- 172.21.15.7:0/323852648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17a015fda0 0x7f17a014dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.729+0000 7f179f400640 1 -- 172.21.15.7:0/323852648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17a00b0600 con 0x7f17a00cd750 2024-09-10T21:44:36.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.729+0000 7f179f400640 1 --2- 172.21.15.7:0/323852648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17a00cd750 0x7f17a00af6f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1790009980 tx=0x7f1790032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:36.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.729+0000 7f1797400640 1 -- 172.21.15.7:0/323852648 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790040070 con 0x7f17a00cd750 2024-09-10T21:44:36.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.729+0000 7f1797400640 1 -- 172.21.15.7:0/323852648 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1790005310 con 0x7f17a00cd750 2024-09-10T21:44:36.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.729+0000 7f1797400640 1 -- 172.21.15.7:0/323852648 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1790005c90 con 0x7f17a00cd750 2024-09-10T21:44:36.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.730+0000 7f17a6600640 1 -- 172.21.15.7:0/323852648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17a00b08e0 con 0x7f17a00cd750 2024-09-10T21:44:36.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.730+0000 7f17a6600640 1 -- 172.21.15.7:0/323852648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17a00afc30 con 0x7f17a00cd750 2024-09-10T21:44:36.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.731+0000 7f17a6600640 1 -- 172.21.15.7:0/323852648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17a0072eb0 con 0x7f17a00cd750 2024-09-10T21:44:36.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.732+0000 7f1797400640 1 -- 172.21.15.7:0/323852648 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f17900049b0 con 0x7f17a00cd750 2024-09-10T21:44:36.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.733+0000 7f1797400640 1 --2- 172.21.15.7:0/323852648 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f177c077f70 0x7f177c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:36.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.733+0000 7f1797400640 1 -- 172.21.15.7:0/323852648 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f17900bb0d0 con 0x7f17a00cd750 2024-09-10T21:44:36.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.733+0000 7f1797e00640 1 --2- 172.21.15.7:0/323852648 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f177c077f70 0x7f177c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:36.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.734+0000 7f1797e00640 1 --2- 172.21.15.7:0/323852648 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f177c077f70 0x7f177c07a430 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1788007fd0 tx=0x7f1788009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:36.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.737+0000 7f1797400640 1 -- 172.21.15.7:0/323852648 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f17a0072eb0 con 0x7f17a00cd750 2024-09-10T21:44:36.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.967+0000 7f17a6600640 1 -- 172.21.15.7:0/323852648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f17a00c9370 con 0x7f17a00cd750 2024-09-10T21:44:36.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.968+0000 7f1797400640 1 -- 172.21.15.7:0/323852648 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5227 (secure 0 0 0) 0x7f17a00c9370 con 0x7f17a00cd750 2024-09-10T21:44:36.969 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:36.970 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"btime":"2024-09-10T21:44:34:389735+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5473,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/2853074495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2853074495},{"type":"v1","addr":"172.21.15.133:6836","nonce":2853074495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3498706462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3498706462},{"type":"v1","addr":"172.21.15.133:6837","nonce":3498706462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/521190705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":521190705},{"type":"v1","addr":"172.21.15.7:6835","nonce":521190705}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:44:30.804578+0000","modified":"2024-09-10T21:44:33.821692+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235016},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235016":{"gid":235016,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6834/364473218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":364473218},{"type":"v1","addr":"172.21.15.7:6834","nonce":364473218}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235016,"qdb_cluster":[235016]},"id":4}]} 2024-09-10T21:44:36.970 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 35 2024-09-10T21:44:36.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.970+0000 7f17a6600640 1 -- 172.21.15.7:0/323852648 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f177c077f70 msgr2=0x7f177c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:36.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.970+0000 7f17a6600640 1 --2- 172.21.15.7:0/323852648 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f177c077f70 0x7f177c07a430 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f1788007fd0 tx=0x7f1788009040 comp rx=0 tx=0).stop 2024-09-10T21:44:36.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.971+0000 7f17a6600640 1 -- 172.21.15.7:0/323852648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17a00cd750 msgr2=0x7f17a00af6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:36.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.971+0000 7f17a6600640 1 --2- 172.21.15.7:0/323852648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17a00cd750 0x7f17a00af6f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1790009980 tx=0x7f1790032c10 comp rx=0 tx=0).stop 2024-09-10T21:44:36.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.971+0000 7f17a6600640 1 -- 172.21.15.7:0/323852648 shutdown_connections 2024-09-10T21:44:36.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.971+0000 7f17a6600640 1 --2- 172.21.15.7:0/323852648 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f177c077f70 0x7f177c07a430 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.971+0000 7f17a6600640 1 --2- 172.21.15.7:0/323852648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f17a015fda0 0x7f17a014dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.971+0000 7f17a6600640 1 --2- 172.21.15.7:0/323852648 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f17a015bc90 0x7f17a014d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.971+0000 7f17a6600640 1 --2- 172.21.15.7:0/323852648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f17a00cd750 0x7f17a00af6f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.971+0000 7f17a6600640 1 -- 172.21.15.7:0/323852648 >> 172.21.15.7:0/323852648 conn(0x7f17a00b16d0 msgr2=0x7f17a00c8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:36.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.971+0000 7f17a6600640 1 -- 172.21.15.7:0/323852648 shutdown_connections 2024-09-10T21:44:36.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:36.971+0000 7f17a6600640 1 -- 172.21.15.7:0/323852648 wait complete. 2024-09-10T21:44:37.324 INFO:tasks.ceph.mds.a.smithi007.stderr:2024-09-10T21:44:37.323+0000 7fe3f5e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:44:37.393 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 35, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:44:30.804578+0000', 'modified': '2024-09-10T21:44:33.821692+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 235016}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_235016': {'gid': 235016, 'name': 'a', 'rank': 0, 'incarnation': 31, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6834/364473218', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6833', 'nonce': 364473218}, {'type': 'v1', 'addr': '172.21.15.7:6834', 'nonce': 364473218}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 235016, 'qdb_cluster': [235016]} 2024-09-10T21:44:37.396 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:44:37.396 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T21:44:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.562+0000 7fed63e00640 1 -- 172.21.15.7:0/1072712751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5c06ff50 msgr2=0x7fed5c15cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.562+0000 7fed63e00640 1 --2- 172.21.15.7:0/1072712751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5c06ff50 0x7fed5c15cbe0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fed5c073840 tx=0x7fed5002a440 comp rx=0 tx=0).stop 2024-09-10T21:44:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.562+0000 7fed63e00640 1 -- 172.21.15.7:0/1072712751 shutdown_connections 2024-09-10T21:44:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.562+0000 7fed63e00640 1 --2- 172.21.15.7:0/1072712751 >> v2:172.21.15.133:3300/0 conn(0x7fed5c06e510 0x7fed5c073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.562+0000 7fed63e00640 1 --2- 172.21.15.7:0/1072712751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5c06ff50 0x7fed5c15cbe0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.562+0000 7fed63e00640 1 -- 172.21.15.7:0/1072712751 >> 172.21.15.7:0/1072712751 conn(0x7fed5c0b1660 msgr2=0x7fed5c0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.562+0000 7fed63e00640 1 -- 172.21.15.7:0/1072712751 shutdown_connections 2024-09-10T21:44:37.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.562+0000 7fed63e00640 1 -- 172.21.15.7:0/1072712751 wait complete. 2024-09-10T21:44:37.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.564+0000 7fed63e00640 1 Processor -- start 2024-09-10T21:44:37.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.564+0000 7fed63e00640 1 -- start start 2024-09-10T21:44:37.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.565+0000 7fed63e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5c06e510 0x7fed5c1f4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.565+0000 7fed63e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed5c06e960 0x7fed5c14aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.565+0000 7fed63e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed5c06ff50 0x7fed5c0c5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.565+0000 7fed63e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c0745a0 con 0x7fed5c06e510 2024-09-10T21:44:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.565+0000 7fed63e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c074420 con 0x7fed5c06e960 2024-09-10T21:44:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.565+0000 7fed63e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fed5c074720 con 0x7fed5c06ff50 2024-09-10T21:44:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.565+0000 7fed61400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5c06e510 0x7fed5c1f4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.565+0000 7fed61400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5c06e510 0x7fed5c1f4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43548/0 (socket says 172.21.15.7:43548) 2024-09-10T21:44:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.565+0000 7fed61400640 1 -- 172.21.15.7:0/566078720 learned_addr learned my addr 172.21.15.7:0/566078720 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.566+0000 7fed60a00640 1 --2- 172.21.15.7:0/566078720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed5c06e960 0x7fed5c14aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.566+0000 7fed61e00640 1 --2- 172.21.15.7:0/566078720 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed5c06ff50 0x7fed5c0c5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.566+0000 7fed61400640 1 -- 172.21.15.7:0/566078720 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed5c06ff50 msgr2=0x7fed5c0c5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:37.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.566+0000 7fed61400640 1 --2- 172.21.15.7:0/566078720 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed5c06ff50 0x7fed5c0c5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.566+0000 7fed61400640 1 -- 172.21.15.7:0/566078720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed5c06e960 msgr2=0x7fed5c14aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:37.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.566+0000 7fed61400640 1 --2- 172.21.15.7:0/566078720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed5c06e960 0x7fed5c14aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.566+0000 7fed61400640 1 -- 172.21.15.7:0/566078720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed5c0cd880 con 0x7fed5c06e510 2024-09-10T21:44:37.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.566+0000 7fed61400640 1 --2- 172.21.15.7:0/566078720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5c06e510 0x7fed5c1f4b30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fed50000c00 tx=0x7fed500039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:37.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.566+0000 7fed59600640 1 -- 172.21.15.7:0/566078720 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5003e030 con 0x7fed5c06e510 2024-09-10T21:44:37.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.566+0000 7fed59600640 1 -- 172.21.15.7:0/566078720 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed500047a0 con 0x7fed5c06e510 2024-09-10T21:44:37.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.566+0000 7fed59600640 1 -- 172.21.15.7:0/566078720 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50035040 con 0x7fed5c06e510 2024-09-10T21:44:37.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.566+0000 7fed63e00640 1 -- 172.21.15.7:0/566078720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed5c0cdb60 con 0x7fed5c06e510 2024-09-10T21:44:37.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.567+0000 7fed63e00640 1 -- 172.21.15.7:0/566078720 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed5c14b410 con 0x7fed5c06e510 2024-09-10T21:44:37.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.569+0000 7fed59600640 1 -- 172.21.15.7:0/566078720 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fed50004b70 con 0x7fed5c06e510 2024-09-10T21:44:37.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.570+0000 7fed59600640 1 --2- 172.21.15.7:0/566078720 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fed3c077f70 0x7fed3c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.570+0000 7fed59600640 1 -- 172.21.15.7:0/566078720 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fed500ba850 con 0x7fed5c06e510 2024-09-10T21:44:37.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.570+0000 7fed66de1740 1 Processor -- start 2024-09-10T21:44:37.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.570+0000 7fed60a00640 1 --2- 172.21.15.7:0/566078720 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fed3c077f70 0x7fed3c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.570+0000 7fed66de1740 1 -- start start 2024-09-10T21:44:37.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.571+0000 7fed60a00640 1 --2- 172.21.15.7:0/566078720 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fed3c077f70 0x7fed3c07a430 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0cf7b0 tx=0x7fed48008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:37.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.571+0000 7fed66de1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b96358b0 0x5616b9635c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.571+0000 7fed66de1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b95f0a20 0x5616b9634390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.571+0000 7fed66de1740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b95f0fa0 0x5616b96348d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.571+0000 7fed66de1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5616b9573430 con 0x5616b95f0a20 2024-09-10T21:44:37.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.571+0000 7fed66de1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5616b95635d0 con 0x5616b96358b0 2024-09-10T21:44:37.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.571+0000 7fed66de1740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5616b9552a00 con 0x5616b95f0fa0 2024-09-10T21:44:37.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.571+0000 7fed60a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b95f0a20 0x5616b9634390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.572+0000 7fed60a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b95f0a20 0x5616b9634390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43558/0 (socket says 172.21.15.7:43558) 2024-09-10T21:44:37.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.572+0000 7fed61400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b96358b0 0x5616b9635c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.572+0000 7fed60a00640 1 -- 172.21.15.7:0/2585033365 learned_addr learned my addr 172.21.15.7:0/2585033365 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:37.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.572+0000 7fed61e00640 1 --2- 172.21.15.7:0/2585033365 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b95f0fa0 0x5616b96348d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.572+0000 7fed60a00640 1 -- 172.21.15.7:0/2585033365 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b95f0fa0 msgr2=0x5616b96348d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:37.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.572+0000 7fed60a00640 1 --2- 172.21.15.7:0/2585033365 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b95f0fa0 0x5616b96348d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.572+0000 7fed60a00640 1 -- 172.21.15.7:0/2585033365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b96358b0 msgr2=0x5616b9635c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:37.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.572+0000 7fed60a00640 1 --2- 172.21.15.7:0/2585033365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b96358b0 0x5616b9635c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.572+0000 7fed60a00640 1 -- 172.21.15.7:0/2585033365 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b951b520 con 0x5616b95f0a20 2024-09-10T21:44:37.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.572+0000 7fed60a00640 1 --2- 172.21.15.7:0/2585033365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b95f0a20 0x5616b9634390 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fed4800d7b0 tx=0x7fed48010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73006867f38d2445 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:37.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.573+0000 7fed4dc00640 1 -- 172.21.15.7:0/2585033365 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48024070 con 0x5616b95f0a20 2024-09-10T21:44:37.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.573+0000 7fed4dc00640 1 -- 172.21.15.7:0/2585033365 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed480149a0 con 0x5616b95f0a20 2024-09-10T21:44:37.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.573+0000 7fed4dc00640 1 -- 172.21.15.7:0/2585033365 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48014d50 con 0x5616b95f0a20 2024-09-10T21:44:37.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.573+0000 7fed66de1740 1 -- 172.21.15.7:0/2585033365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b95f0a20 msgr2=0x5616b9634390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:37.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.573+0000 7fed66de1740 1 --2- 172.21.15.7:0/2585033365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b95f0a20 0x5616b9634390 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fed4800d7b0 tx=0x7fed48010000 comp rx=0 tx=0).stop 2024-09-10T21:44:37.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.573+0000 7fed66de1740 1 -- 172.21.15.7:0/2585033365 shutdown_connections 2024-09-10T21:44:37.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.573+0000 7fed66de1740 1 --2- 172.21.15.7:0/2585033365 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b95f0fa0 0x5616b96348d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.573+0000 7fed66de1740 1 --2- 172.21.15.7:0/2585033365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b95f0a20 0x5616b9634390 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.573+0000 7fed66de1740 1 --2- 172.21.15.7:0/2585033365 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b96358b0 0x5616b9635c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.573+0000 7fed66de1740 1 -- 172.21.15.7:0/2585033365 >> 172.21.15.7:0/2585033365 conn(0x5616b95ebff0 msgr2=0x5616b95efa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:37.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.573+0000 7fed66de1740 1 -- 172.21.15.7:0/2585033365 shutdown_connections 2024-09-10T21:44:37.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.573+0000 7fed66de1740 1 -- 172.21.15.7:0/2585033365 wait complete. 2024-09-10T21:44:37.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.574+0000 7fed66de1740 1 -- start start 2024-09-10T21:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.574+0000 7fed66de1740 1 Processor -- start 2024-09-10T21:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.575+0000 7fed66de1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b95f0fa0 0x5616b96198e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.575+0000 7fed66de1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 0x5616b9644790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.575+0000 7fed66de1740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b96073b0 0x5616b962f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.575+0000 7fed66de1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5616b94f9330 con 0x5616b96358b0 2024-09-10T21:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.575+0000 7fed66de1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5616b9556810 con 0x5616b95f0fa0 2024-09-10T21:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.575+0000 7fed66de1740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5616b92c3620 con 0x5616b96073b0 2024-09-10T21:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.575+0000 7fed66de1740 10 client.0 _ll_register_callbacks cb 0x5616b94fe420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.576+0000 7fed60a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 0x5616b9644790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.576+0000 7fed61400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b95f0fa0 0x5616b96198e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.576+0000 7fed60a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 0x5616b9644790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43566/0 (socket says 172.21.15.7:43566) 2024-09-10T21:44:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.576+0000 7fed60a00640 1 -- 172.21.15.7:0/2635382899 learned_addr learned my addr 172.21.15.7:0/2635382899 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.576+0000 7fed61400640 1 --2- 172.21.15.7:0/2635382899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b95f0fa0 0x5616b96198e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:57812/0 (socket says 172.21.15.7:57812) 2024-09-10T21:44:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.576+0000 7fed61e00640 1 --2- 172.21.15.7:0/2635382899 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b96073b0 0x5616b962f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.576+0000 7fed60a00640 1 -- 172.21.15.7:0/2635382899 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b96073b0 msgr2=0x5616b962f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.576+0000 7fed60a00640 1 --2- 172.21.15.7:0/2635382899 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b96073b0 0x5616b962f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.576+0000 7fed60a00640 1 -- 172.21.15.7:0/2635382899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b95f0fa0 msgr2=0x5616b96198e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.576+0000 7fed60a00640 1 --2- 172.21.15.7:0/2635382899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b95f0fa0 0x5616b96198e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.576+0000 7fed60a00640 1 -- 172.21.15.7:0/2635382899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b92b4850 con 0x5616b96358b0 2024-09-10T21:44:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.576+0000 7fed60a00640 1 --2- 172.21.15.7:0/2635382899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 0x5616b9644790 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fed48022040 tx=0x7fed48014e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a69e6055352b16e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.576+0000 7fed42000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:44:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.576+0000 7fed42000640 1 -- 172.21.15.7:0/2635382899 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4802d060 con 0x5616b96358b0 2024-09-10T21:44:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.577+0000 7fed66de1740 1 -- 172.21.15.7:0/2635382899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed480149a0 con 0x5616b96358b0 2024-09-10T21:44:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.577+0000 7fed42000640 1 -- 172.21.15.7:0/2635382899 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fed4801f740 con 0x5616b96358b0 2024-09-10T21:44:37.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.577+0000 7fed42000640 1 -- 172.21.15.7:0/2635382899 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fed48033050 con 0x5616b96358b0 2024-09-10T21:44:37.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.577+0000 7fed66de1740 10 client.5814 fetch_fsmap learned FSMap version 35 2024-09-10T21:44:37.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.577+0000 7fed66de1740 1 -- 172.21.15.7:0/2635382899 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x5616b932b6b0 con 0x5616b96358b0 2024-09-10T21:44:37.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.577+0000 7fed42000640 1 -- 172.21.15.7:0/2635382899 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed480149a0 con 0x5616b96358b0 2024-09-10T21:44:37.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.577+0000 7fed42000640 1 -- 172.21.15.7:0/2635382899 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7fed48036020 con 0x5616b96358b0 2024-09-10T21:44:37.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.577+0000 7fed66de1740 10 client.5814 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T21:44:37.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.577+0000 7fed66de1740 10 client.5814 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:44:37.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.577+0000 7fed66de1740 20 client.5814 populate_metadata read hostname 'smithi007' 2024-09-10T21:44:37.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.577+0000 7fed66de1740 1 --2- 172.21.15.7:0/2635382899 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x5616b9632910 0x5616b9632cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.577+0000 7fed66de1740 4 client.5814 mds_command: new command op to 235016 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:44:37.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.577+0000 7fed66de1740 1 -- 172.21.15.7:0/2635382899 --> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616b9565520 con 0x5616b9632910 2024-09-10T21:44:37.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.578+0000 7fed61400640 1 --2- 172.21.15.7:0/2635382899 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x5616b9632910 0x5616b9632cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.578+0000 7fed61400640 1 --2- 172.21.15.7:0/2635382899 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x5616b9632910 0x5616b9632cf0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e550a6748ea665f server_cookie=bf26d43e1fd826d5 in_seq=0 out_seq=0 2024-09-10T21:44:37.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.578+0000 7fed42000640 10 client.5814 ms_handle_connect on v2:172.21.15.7:6833/364473218 2024-09-10T21:44:37.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.579+0000 7fed42000640 1 -- 172.21.15.7:0/2635382899 <== mds.0 v2:172.21.15.7:6833/364473218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5616b9565520 con 0x5616b9632910 2024-09-10T21:44:37.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.579+0000 7fed42000640 10 client.5814 handle_command_reply: tid=0 2024-09-10T21:44:37.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.579+0000 7fed66de1740 1 client.5814 shutdown 2024-09-10T21:44:37.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.579+0000 7fed66de1740 1 -- 172.21.15.7:0/2635382899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 msgr2=0x5616b9644790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:37.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.579+0000 7fed66de1740 1 --2- 172.21.15.7:0/2635382899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 0x5616b9644790 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fed48022040 tx=0x7fed48014e40 comp rx=0 tx=0).stop 2024-09-10T21:44:37.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.579+0000 7fed66de1740 1 -- 172.21.15.7:0/2635382899 shutdown_connections 2024-09-10T21:44:37.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.579+0000 7fed66de1740 1 --2- 172.21.15.7:0/2635382899 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x5616b9632910 0x5616b9632cf0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.579+0000 7fed66de1740 1 --2- 172.21.15.7:0/2635382899 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b96073b0 0x5616b962f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.579+0000 7fed42000640 0 client.5814 ms_handle_reset on v2:172.21.15.7:6833/364473218 2024-09-10T21:44:37.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.579+0000 7fed66de1740 1 --2- 172.21.15.7:0/2635382899 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 0x5616b9644790 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.579+0000 7fed66de1740 1 --2- 172.21.15.7:0/2635382899 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b95f0fa0 0x5616b96198e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.579+0000 7fed66de1740 1 -- 172.21.15.7:0/2635382899 >> 172.21.15.7:0/2635382899 conn(0x5616b95f0a20 msgr2=0x5616b95eabc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:37.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.580+0000 7fed66de1740 1 -- 172.21.15.7:0/2635382899 shutdown_connections 2024-09-10T21:44:37.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.580+0000 7fed66de1740 1 -- 172.21.15.7:0/2635382899 wait complete. 2024-09-10T21:44:37.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.580+0000 7fed66de1740 20 client.5814 trim_cache size 0 max 16384 2024-09-10T21:44:37.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.599+0000 7fed66de1740 1 Processor -- start 2024-09-10T21:44:37.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.599+0000 7fed66de1740 1 -- start start 2024-09-10T21:44:37.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.600+0000 7fed66de1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b95f0fa0 0x5616b95ead20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.600+0000 7fed66de1740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b96073b0 0x5616b9640a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.600+0000 7fed66de1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 0x5616b96afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.600+0000 7fed66de1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c200320 con 0x5616b96358b0 2024-09-10T21:44:37.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.600+0000 7fed66de1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fed5c0b3450 con 0x5616b95f0fa0 2024-09-10T21:44:37.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.600+0000 7fed66de1740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fed5c0b3aa0 con 0x5616b96073b0 2024-09-10T21:44:37.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.600+0000 7fed61e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 0x5616b96afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.600+0000 7fed61e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 0x5616b96afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43570/0 (socket says 172.21.15.7:43570) 2024-09-10T21:44:37.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.600+0000 7fed61e00640 1 -- 172.21.15.7:0/2136671209 learned_addr learned my addr 172.21.15.7:0/2136671209 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:37.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.601+0000 7fed60a00640 1 --2- 172.21.15.7:0/2136671209 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b96073b0 0x5616b9640a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.601+0000 7fed61400640 1 --2- 172.21.15.7:0/2136671209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b95f0fa0 0x5616b95ead20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.601+0000 7fed61e00640 1 -- 172.21.15.7:0/2136671209 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b96073b0 msgr2=0x5616b9640a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:37.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.601+0000 7fed61e00640 1 --2- 172.21.15.7:0/2136671209 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b96073b0 0x5616b9640a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.601+0000 7fed61e00640 1 -- 172.21.15.7:0/2136671209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b95f0fa0 msgr2=0x5616b95ead20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:37.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.601+0000 7fed61e00640 1 --2- 172.21.15.7:0/2136671209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b95f0fa0 0x5616b95ead20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.601+0000 7fed61e00640 1 -- 172.21.15.7:0/2136671209 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616b9604420 con 0x5616b96358b0 2024-09-10T21:44:37.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.601+0000 7fed61e00640 1 --2- 172.21.15.7:0/2136671209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 0x5616b96afdf0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fed54001d20 tx=0x7fed540021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6ffe88139fda24 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:37.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.601+0000 7fed4d200640 1 -- 172.21.15.7:0/2136671209 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed54006d20 con 0x5616b96358b0 2024-09-10T21:44:37.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.601+0000 7fed4d200640 1 -- 172.21.15.7:0/2136671209 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed5401b020 con 0x5616b96358b0 2024-09-10T21:44:37.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.601+0000 7fed4d200640 1 -- 172.21.15.7:0/2136671209 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed54014a20 con 0x5616b96358b0 2024-09-10T21:44:37.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.602+0000 7fed66de1740 1 -- 172.21.15.7:0/2136671209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 msgr2=0x5616b96afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:37.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.602+0000 7fed66de1740 1 --2- 172.21.15.7:0/2136671209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 0x5616b96afdf0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fed54001d20 tx=0x7fed540021e0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.602+0000 7fed66de1740 1 -- 172.21.15.7:0/2136671209 shutdown_connections 2024-09-10T21:44:37.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.602+0000 7fed66de1740 1 --2- 172.21.15.7:0/2136671209 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 0x5616b96afdf0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.602+0000 7fed66de1740 1 --2- 172.21.15.7:0/2136671209 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b96073b0 0x5616b9640a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.602+0000 7fed66de1740 1 --2- 172.21.15.7:0/2136671209 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b95f0fa0 0x5616b95ead20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.602+0000 7fed66de1740 1 -- 172.21.15.7:0/2136671209 >> 172.21.15.7:0/2136671209 conn(0x5616b95f0a20 msgr2=0x5616b9634e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:37.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.602+0000 7fed66de1740 1 -- 172.21.15.7:0/2136671209 shutdown_connections 2024-09-10T21:44:37.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.602+0000 7fed66de1740 1 -- 172.21.15.7:0/2136671209 wait complete. 2024-09-10T21:44:37.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.603+0000 7fed66de1740 1 -- start start 2024-09-10T21:44:37.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.603+0000 7fed66de1740 1 Processor -- start 2024-09-10T21:44:37.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.604+0000 7fed66de1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b95f0fa0 0x5616b9656aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.604+0000 7fed66de1740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b96073b0 0x5616b9640a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.604+0000 7fed66de1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 0x5616b9640fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.604+0000 7fed66de1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5616b9607b70 con 0x5616b96358b0 2024-09-10T21:44:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.604+0000 7fed66de1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5616b9645390 con 0x5616b95f0fa0 2024-09-10T21:44:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.604+0000 7fed66de1740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5616b962f930 con 0x5616b96073b0 2024-09-10T21:44:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.604+0000 7fed66de1740 10 client.0 _ll_register_callbacks cb 0x7fed5c0752a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:44:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.604+0000 7fed61e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 0x5616b9640fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.604+0000 7fed61e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 0x5616b9640fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43580/0 (socket says 172.21.15.7:43580) 2024-09-10T21:44:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.604+0000 7fed61e00640 1 -- 172.21.15.7:0/1696695769 learned_addr learned my addr 172.21.15.7:0/1696695769 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.604+0000 7fed61400640 1 --2- 172.21.15.7:0/1696695769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b95f0fa0 0x5616b9656aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.604+0000 7fed60a00640 1 --2- 172.21.15.7:0/1696695769 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b96073b0 0x5616b9640a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.604+0000 7fed61e00640 1 -- 172.21.15.7:0/1696695769 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b96073b0 msgr2=0x5616b9640a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.605+0000 7fed61e00640 1 --2- 172.21.15.7:0/1696695769 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b96073b0 0x5616b9640a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.605+0000 7fed61e00640 1 -- 172.21.15.7:0/1696695769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b95f0fa0 msgr2=0x5616b9656aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.605+0000 7fed61e00640 1 --2- 172.21.15.7:0/1696695769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b95f0fa0 0x5616b9656aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.605+0000 7fed61e00640 1 -- 172.21.15.7:0/1696695769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616b9656fe0 con 0x5616b96358b0 2024-09-10T21:44:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.605+0000 7fed60a00640 1 --2- 172.21.15.7:0/1696695769 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b96073b0 0x5616b9640a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:44:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.605+0000 7fed61e00640 1 --2- 172.21.15.7:0/1696695769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 0x5616b9640fc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fed54019040 tx=0x7fed54004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c04a7cb1b4888e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.605+0000 7fed42000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:44:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.605+0000 7fed42000640 1 -- 172.21.15.7:0/1696695769 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed54024070 con 0x5616b96358b0 2024-09-10T21:44:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.605+0000 7fed42000640 1 -- 172.21.15.7:0/1696695769 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fed54005310 con 0x5616b96358b0 2024-09-10T21:44:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.605+0000 7fed66de1740 1 -- 172.21.15.7:0/1696695769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed5401b020 con 0x5616b96358b0 2024-09-10T21:44:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.605+0000 7fed42000640 1 -- 172.21.15.7:0/1696695769 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fed5402f050 con 0x5616b96358b0 2024-09-10T21:44:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.605+0000 7fed66de1740 10 client.5820 fetch_fsmap learned FSMap version 35 2024-09-10T21:44:37.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.605+0000 7fed66de1740 1 -- 172.21.15.7:0/1696695769 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x5616b9659280 con 0x5616b96358b0 2024-09-10T21:44:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.606+0000 7fed42000640 1 -- 172.21.15.7:0/1696695769 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed5401b020 con 0x5616b96358b0 2024-09-10T21:44:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.606+0000 7fed42000640 1 -- 172.21.15.7:0/1696695769 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7fed54032020 con 0x5616b96358b0 2024-09-10T21:44:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.606+0000 7fed66de1740 10 client.5820 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T21:44:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.606+0000 7fed66de1740 10 client.5820 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:44:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.606+0000 7fed66de1740 20 client.5820 populate_metadata read hostname 'smithi007' 2024-09-10T21:44:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.606+0000 7fed66de1740 1 --2- 172.21.15.7:0/1696695769 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x5616b969b200 0x5616b9698880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.606+0000 7fed66de1740 4 client.5820 mds_command: new command op to 235016 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:44:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.606+0000 7fed66de1740 1 -- 172.21.15.7:0/1696695769 --> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] -- command(tid 0: {"prefix": "status"}) -- 0x5616b9560fd0 con 0x5616b969b200 2024-09-10T21:44:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.606+0000 7fed61400640 1 --2- 172.21.15.7:0/1696695769 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x5616b969b200 0x5616b9698880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.607+0000 7fed61400640 1 --2- 172.21.15.7:0/1696695769 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x5616b969b200 0x5616b9698880 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f27c5674b832a7b0 server_cookie=da7f852e6a06bb8e in_seq=0 out_seq=0 2024-09-10T21:44:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.607+0000 7fed42000640 10 client.5820 ms_handle_connect on v2:172.21.15.7:6833/364473218 2024-09-10T21:44:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.607+0000 7fed42000640 1 -- 172.21.15.7:0/1696695769 <== mds.0 v2:172.21.15.7:6833/364473218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5616b9560fd0 con 0x5616b969b200 2024-09-10T21:44:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.607+0000 7fed42000640 10 client.5820 handle_command_reply: tid=0 2024-09-10T21:44:37.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.607+0000 7fed66de1740 1 client.5820 shutdown 2024-09-10T21:44:37.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.607+0000 7fed66de1740 1 -- 172.21.15.7:0/1696695769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 msgr2=0x5616b9640fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:37.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.607+0000 7fed66de1740 1 --2- 172.21.15.7:0/1696695769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 0x5616b9640fc0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fed54019040 tx=0x7fed54004bb0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.607+0000 7fed66de1740 1 -- 172.21.15.7:0/1696695769 shutdown_connections 2024-09-10T21:44:37.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.607+0000 7fed66de1740 1 --2- 172.21.15.7:0/1696695769 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x5616b969b200 0x5616b9698880 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.607+0000 7fed66de1740 1 --2- 172.21.15.7:0/1696695769 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5616b96358b0 0x5616b9640fc0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.607+0000 7fed66de1740 1 --2- 172.21.15.7:0/1696695769 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5616b96073b0 0x5616b9640a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.607+0000 7fed66de1740 1 --2- 172.21.15.7:0/1696695769 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5616b95f0fa0 0x5616b9656aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.607+0000 7fed42000640 0 client.5820 ms_handle_reset on v2:172.21.15.7:6833/364473218 2024-09-10T21:44:37.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.607+0000 7fed66de1740 1 -- 172.21.15.7:0/1696695769 >> 172.21.15.7:0/1696695769 conn(0x5616b95f0a20 msgr2=0x5616b95ead20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:37.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.607+0000 7fed66de1740 1 -- 172.21.15.7:0/1696695769 shutdown_connections 2024-09-10T21:44:37.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.607+0000 7fed66de1740 1 -- 172.21.15.7:0/1696695769 wait complete. 2024-09-10T21:44:37.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.607+0000 7fed66de1740 20 client.5820 trim_cache size 0 max 16384 2024-09-10T21:44:37.613 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:44:37.613 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2029451a-3c26-48c7-b874-0597197278ac", 2024-09-10T21:44:37.613 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-10T21:44:37.613 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 235016, 2024-09-10T21:44:37.613 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T21:44:37.613 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T21:44:37.613 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T21:44:37.613 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.2470594840000002, 2024-09-10T21:44:37.613 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 35, 2024-09-10T21:44:37.613 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 58, 2024-09-10T21:44:37.614 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 58, 2024-09-10T21:44:37.614 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 11.53178967 2024-09-10T21:44:37.614 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:44:37.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.608+0000 7fed63e00640 1 -- 172.21.15.7:0/566078720 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fed3c077f70 msgr2=0x7fed3c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:37.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.608+0000 7fed63e00640 1 --2- 172.21.15.7:0/566078720 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fed3c077f70 0x7fed3c07a430 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fed5c0cf7b0 tx=0x7fed48008040 comp rx=0 tx=0).stop 2024-09-10T21:44:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.608+0000 7fed63e00640 1 -- 172.21.15.7:0/566078720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5c06e510 msgr2=0x7fed5c1f4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.608+0000 7fed63e00640 1 --2- 172.21.15.7:0/566078720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5c06e510 0x7fed5c1f4b30 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fed50000c00 tx=0x7fed500039e0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.608+0000 7fed63e00640 1 -- 172.21.15.7:0/566078720 shutdown_connections 2024-09-10T21:44:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.608+0000 7fed63e00640 1 --2- 172.21.15.7:0/566078720 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fed3c077f70 0x7fed3c07a430 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.608+0000 7fed63e00640 1 --2- 172.21.15.7:0/566078720 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fed5c06e960 0x7fed5c14aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.608+0000 7fed63e00640 1 --2- 172.21.15.7:0/566078720 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fed5c06ff50 0x7fed5c0c5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.608+0000 7fed63e00640 1 --2- 172.21.15.7:0/566078720 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fed5c06e510 0x7fed5c1f4b30 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.608+0000 7fed63e00640 1 -- 172.21.15.7:0/566078720 >> 172.21.15.7:0/566078720 conn(0x7fed5c0b1660 msgr2=0x7fed5c15d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.608+0000 7fed63e00640 1 -- 172.21.15.7:0/566078720 shutdown_connections 2024-09-10T21:44:37.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.608+0000 7fed63e00640 1 -- 172.21.15.7:0/566078720 wait complete. 2024-09-10T21:44:37.625 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:44:37.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.785+0000 7f74f0200640 1 -- 172.21.15.7:0/1271338694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e815bc90 msgr2=0x7f74e80ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.785+0000 7f74f0200640 1 --2- 172.21.15.7:0/1271338694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e815bc90 0x7f74e80ccef0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f74e80b5560 tx=0x7f74dc02a440 comp rx=0 tx=0).stop 2024-09-10T21:44:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.785+0000 7f74f0200640 1 -- 172.21.15.7:0/1271338694 shutdown_connections 2024-09-10T21:44:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.785+0000 7f74f0200640 1 --2- 172.21.15.7:0/1271338694 >> v2:172.21.15.133:3300/0 conn(0x7f74e8073c70 0x7f74e815fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.785+0000 7f74f0200640 1 --2- 172.21.15.7:0/1271338694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74e80cd750 0x7f74e8073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.785+0000 7f74f0200640 1 --2- 172.21.15.7:0/1271338694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e815bc90 0x7f74e80ccef0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.785+0000 7f74f0200640 1 -- 172.21.15.7:0/1271338694 >> 172.21.15.7:0/1271338694 conn(0x7f74e80b16d0 msgr2=0x7f74e80b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.785+0000 7f74f0200640 1 -- 172.21.15.7:0/1271338694 shutdown_connections 2024-09-10T21:44:37.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.785+0000 7f74f0200640 1 -- 172.21.15.7:0/1271338694 wait complete. 2024-09-10T21:44:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.787+0000 7f74f0200640 1 Processor -- start 2024-09-10T21:44:37.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.787+0000 7f74f0200640 1 -- start start 2024-09-10T21:44:37.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.788+0000 7f74f0200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74e8073c70 0x7f74e815a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.788+0000 7f74f0200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74e80cd750 0x7f74e8156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.788+0000 7f74f0200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e815bc90 0x7f74e81f6000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.788+0000 7f74f0200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f74e8160670 con 0x7f74e815bc90 2024-09-10T21:44:37.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.788+0000 7f74f0200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f74e81604f0 con 0x7f74e80cd750 2024-09-10T21:44:37.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.788+0000 7f74f0200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f74e806fcf0 con 0x7f74e8073c70 2024-09-10T21:44:37.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.788+0000 7f74ee200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e815bc90 0x7f74e81f6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.788+0000 7f74ed800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74e8073c70 0x7f74e815a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.788+0000 7f74ee200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e815bc90 0x7f74e81f6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43596/0 (socket says 172.21.15.7:43596) 2024-09-10T21:44:37.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.788+0000 7f74ed800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74e8073c70 0x7f74e815a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:59640/0 (socket says 172.21.15.7:59640) 2024-09-10T21:44:37.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.788+0000 7f74ee200640 1 -- 172.21.15.7:0/2618369798 learned_addr learned my addr 172.21.15.7:0/2618369798 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:37.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.788+0000 7f74ece00640 1 --2- 172.21.15.7:0/2618369798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74e80cd750 0x7f74e8156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.788+0000 7f74ee200640 1 -- 172.21.15.7:0/2618369798 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74e8073c70 msgr2=0x7f74e815a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:37.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.788+0000 7f74ee200640 1 --2- 172.21.15.7:0/2618369798 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74e8073c70 0x7f74e815a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.789+0000 7f74ee200640 1 -- 172.21.15.7:0/2618369798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74e80cd750 msgr2=0x7f74e8156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:37.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.789+0000 7f74ee200640 1 --2- 172.21.15.7:0/2618369798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74e80cd750 0x7f74e8156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:37.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.789+0000 7f74ee200640 1 -- 172.21.15.7:0/2618369798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e81f6bd0 con 0x7f74e815bc90 2024-09-10T21:44:37.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.789+0000 7f74ee200640 1 --2- 172.21.15.7:0/2618369798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e815bc90 0x7f74e81f6000 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f74e000eab0 tx=0x7f74e0009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.789+0000 7f74e5600640 1 -- 172.21.15.7:0/2618369798 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e000ce40 con 0x7f74e815bc90 2024-09-10T21:44:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.789+0000 7f74e5600640 1 -- 172.21.15.7:0/2618369798 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74e0004c10 con 0x7f74e815bc90 2024-09-10T21:44:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.789+0000 7f74e5600640 1 -- 172.21.15.7:0/2618369798 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e0005020 con 0x7f74e815bc90 2024-09-10T21:44:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.789+0000 7f74f0200640 1 -- 172.21.15.7:0/2618369798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74e81f6e70 con 0x7f74e815bc90 2024-09-10T21:44:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.789+0000 7f74f0200640 1 -- 172.21.15.7:0/2618369798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74e815ec00 con 0x7f74e815bc90 2024-09-10T21:44:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.790+0000 7f74f0200640 1 -- 172.21.15.7:0/2618369798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74e80b4500 con 0x7f74e815bc90 2024-09-10T21:44:37.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.791+0000 7f74e5600640 1 -- 172.21.15.7:0/2618369798 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f74e00051c0 con 0x7f74e815bc90 2024-09-10T21:44:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.792+0000 7f74e5600640 1 --2- 172.21.15.7:0/2618369798 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f74c8077f70 0x7f74c807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:37.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.792+0000 7f74e5600640 1 -- 172.21.15.7:0/2618369798 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f74e009aa80 con 0x7f74e815bc90 2024-09-10T21:44:37.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.793+0000 7f74ed800640 1 --2- 172.21.15.7:0/2618369798 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f74c8077f70 0x7f74c807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:37.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.793+0000 7f74ed800640 1 --2- 172.21.15.7:0/2618369798 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f74c8077f70 0x7f74c807a430 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f74dc033000 tx=0x7f74dc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:37.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:37.796+0000 7f74e5600640 1 -- 172.21.15.7:0/2618369798 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f74e80b4500 con 0x7f74e815bc90 2024-09-10T21:44:38.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:38.027+0000 7f74f0200640 1 -- 172.21.15.7:0/2618369798 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f74e8073a90 con 0x7f74e815bc90 2024-09-10T21:44:38.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:38.028+0000 7f74e5600640 1 -- 172.21.15.7:0/2618369798 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v35) ==== 53+0+73 (secure 0 0 0) 0x7f74e8073a90 con 0x7f74e815bc90 2024-09-10T21:44:38.029 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:44:38.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:38.030+0000 7f74f0200640 1 -- 172.21.15.7:0/2618369798 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f74c8077f70 msgr2=0x7f74c807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:38.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:38.030+0000 7f74f0200640 1 --2- 172.21.15.7:0/2618369798 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f74c8077f70 0x7f74c807a430 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f74dc033000 tx=0x7f74dc004730 comp rx=0 tx=0).stop 2024-09-10T21:44:38.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:38.030+0000 7f74f0200640 1 -- 172.21.15.7:0/2618369798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e815bc90 msgr2=0x7f74e81f6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:38.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:38.030+0000 7f74f0200640 1 --2- 172.21.15.7:0/2618369798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e815bc90 0x7f74e81f6000 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f74e000eab0 tx=0x7f74e0009f80 comp rx=0 tx=0).stop 2024-09-10T21:44:38.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:38.030+0000 7f74f0200640 1 -- 172.21.15.7:0/2618369798 shutdown_connections 2024-09-10T21:44:38.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:38.030+0000 7f74f0200640 1 --2- 172.21.15.7:0/2618369798 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f74c8077f70 0x7f74c807a430 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:38.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:38.030+0000 7f74f0200640 1 --2- 172.21.15.7:0/2618369798 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74e80cd750 0x7f74e8156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:38.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:38.030+0000 7f74f0200640 1 --2- 172.21.15.7:0/2618369798 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74e815bc90 0x7f74e81f6000 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:38.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:38.030+0000 7f74f0200640 1 --2- 172.21.15.7:0/2618369798 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74e8073c70 0x7f74e815a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:38.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:38.030+0000 7f74f0200640 1 -- 172.21.15.7:0/2618369798 >> 172.21.15.7:0/2618369798 conn(0x7f74e80b16d0 msgr2=0x7f74e80b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:38.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:38.030+0000 7f74f0200640 1 -- 172.21.15.7:0/2618369798 shutdown_connections 2024-09-10T21:44:38.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:38.030+0000 7f74f0200640 1 -- 172.21.15.7:0/2618369798 wait complete. 2024-09-10T21:44:38.047 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:44:38.048 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:44:38.048 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-10T21:44:38.048 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:44:38.048 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:44:38.048 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:44:38.048 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:44:38.048 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:44:38.048 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:44:38.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:44:38.049 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-10T21:44:38.085 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:44:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:44:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:44:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:44:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T21:44:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:44:38.085 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:44:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T21:44:38.085 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-10T21:44:38.085 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:44:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T21:44:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-10T21:44:38.086 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:44:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-10T21:44:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-10T21:44:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T21:44:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42467sec preferred_lft 42467sec 2024-09-10T21:44:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-10T21:44:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:44:38.086 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:44:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-10T21:44:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-10T21:44:38.086 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:44:38.086 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff 2024-09-10T21:44:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:44:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:44:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::3c3d:dbff:fe02:4c15/64 scope link 2024-09-10T21:44:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:44:38.087 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:44:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether f6:f9:69:5a:86:f5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:44:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f4f9:69ff:fe5a:86f5/64 scope link 2024-09-10T21:44:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:44:38.087 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:44:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:44:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::433:5cff:fe44:4a5b/64 scope link 2024-09-10T21:44:38.087 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:44:38.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:44:38.088 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:44:38.178 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:44:38.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:44:38.181 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-10T21:44:38.217 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-10T21:44:38.217 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:44:38.217 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T21:44:38.217 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T21:44:38.217 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:44:38.218 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:44:38.218 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:44:38.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:44:38.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:44:38.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:44:38.218 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:44:38.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:38 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:44:38.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:44:38.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:44:38.418 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:44:38.454 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:44:38.454 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:44:38.454 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:44:38.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:44:38.454 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-10T21:44:38.551 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:44:38.551 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:44:38.586 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:44:38.587 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:44:38.587 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:44:38.661 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:44:38.661 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:44:38.662 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:44:38.791 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:44:38.791 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:44:38.821 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:44:38.821 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:44:38.821 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.133:3300,172.21.15.133:3301" 2024-09-10T21:44:38.822 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@2029451a-3c26-48c7-b874-0597197278ac.cephfs=/ 2024-09-10T21:44:38.822 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.133:3300/172.21.15.133:3301" will pass to kernel 2024-09-10T21:44:38.824 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T21:44:38.824 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:44:38.824 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:44:38.824 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:44:38.824 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:44:38.826 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:44:38.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:44:38.826 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:44:38.883 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:44:38.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:44:38.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:44:38.982 INFO:teuthology.orchestra.run.smithi007.stdout:client5596 2024-09-10T21:44:38.982 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:44:38.982 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:44:39.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:44:39.018 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2024-09-10T21:44:39.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:44:39.101 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2024-09-10T21:44:39.178 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:44:39.178 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:44:39.178 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:44:39.179 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:44:39.179 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2024-09-10T21:44:39.179 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:44:39.319 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627777 2024-09-10T21:44:39.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:44:39.331 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:44:39.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.535+0000 7f4889800640 1 -- 172.21.15.7:0/3589460289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48840b4ac0 msgr2=0x7f48840b2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:39.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.535+0000 7f4889800640 1 --2- 172.21.15.7:0/3589460289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48840b4ac0 0x7f48840b2a40 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f48840706a0 tx=0x7f486c02a560 comp rx=0 tx=0).stop 2024-09-10T21:44:39.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.535+0000 7f4889800640 1 -- 172.21.15.7:0/3589460289 shutdown_connections 2024-09-10T21:44:39.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.535+0000 7f4889800640 1 --2- 172.21.15.7:0/3589460289 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48840b2f80 0x7f488415f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:39.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.535+0000 7f4889800640 1 --2- 172.21.15.7:0/3589460289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48840b4ac0 0x7f48840b2a40 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:39.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.535+0000 7f4889800640 1 --2- 172.21.15.7:0/3589460289 >> v2:172.21.15.133:3300/0 conn(0x7f4884145540 0x7f48840b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:39.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.535+0000 7f4889800640 1 -- 172.21.15.7:0/3589460289 >> 172.21.15.7:0/3589460289 conn(0x7f48840b1d70 msgr2=0x7f48840cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:39.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.535+0000 7f4889800640 1 -- 172.21.15.7:0/3589460289 shutdown_connections 2024-09-10T21:44:39.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.535+0000 7f4889800640 1 -- 172.21.15.7:0/3589460289 wait complete. 2024-09-10T21:44:39.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.536+0000 7f4889800640 1 Processor -- start 2024-09-10T21:44:39.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.537+0000 7f4889800640 1 -- start start 2024-09-10T21:44:39.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.537+0000 7f4889800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48840b2f80 0x7f4884157ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:39.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.537+0000 7f4889800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48840b4ac0 0x7f4884130d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:39.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.537+0000 7f4889800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4884145540 0x7f48841f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:39.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.537+0000 7f4889800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f488415fd10 con 0x7f4884145540 2024-09-10T21:44:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.537+0000 7f4889800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4884145a30 con 0x7f48840b2f80 2024-09-10T21:44:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.537+0000 7f4889800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f488415fe90 con 0x7f48840b4ac0 2024-09-10T21:44:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.538+0000 7f4888e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4884145540 0x7f48841f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.538+0000 7f4883e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48840b2f80 0x7f4884157ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.538+0000 7f487b400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48840b4ac0 0x7f4884130d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.538+0000 7f487b400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48840b4ac0 0x7f4884130d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:59658/0 (socket says 172.21.15.7:59658) 2024-09-10T21:44:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.538+0000 7f487b400640 1 -- 172.21.15.7:0/4100203667 learned_addr learned my addr 172.21.15.7:0/4100203667 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.538+0000 7f4883e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48840b2f80 0x7f4884157ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:57862/0 (socket says 172.21.15.7:57862) 2024-09-10T21:44:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.538+0000 7f4888e00640 1 -- 172.21.15.7:0/4100203667 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48840b4ac0 msgr2=0x7f4884130d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.538+0000 7f4888e00640 1 --2- 172.21.15.7:0/4100203667 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48840b4ac0 0x7f4884130d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.538+0000 7f4888e00640 1 -- 172.21.15.7:0/4100203667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48840b2f80 msgr2=0x7f4884157ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.538+0000 7f4888e00640 1 --2- 172.21.15.7:0/4100203667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48840b2f80 0x7f4884157ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:39.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.538+0000 7f4888e00640 1 -- 172.21.15.7:0/4100203667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488415bc60 con 0x7f4884145540 2024-09-10T21:44:39.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.539+0000 7f4888e00640 1 --2- 172.21.15.7:0/4100203667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4884145540 0x7f48841f16d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f487c00e960 tx=0x7f487c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:39.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.539+0000 7f4880c00640 1 -- 172.21.15.7:0/4100203667 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c004d20 con 0x7f4884145540 2024-09-10T21:44:39.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.539+0000 7f4880c00640 1 -- 172.21.15.7:0/4100203667 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f487c00cac0 con 0x7f4884145540 2024-09-10T21:44:39.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.539+0000 7f4880c00640 1 -- 172.21.15.7:0/4100203667 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c005020 con 0x7f4884145540 2024-09-10T21:44:39.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.539+0000 7f4889800640 1 -- 172.21.15.7:0/4100203667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488415b980 con 0x7f4884145540 2024-09-10T21:44:39.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.539+0000 7f4889800640 1 -- 172.21.15.7:0/4100203667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48841572b0 con 0x7f4884145540 2024-09-10T21:44:39.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.540+0000 7f4889800640 1 -- 172.21.15.7:0/4100203667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4884160340 con 0x7f4884145540 2024-09-10T21:44:39.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.541+0000 7f4880c00640 1 -- 172.21.15.7:0/4100203667 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f487c00f040 con 0x7f4884145540 2024-09-10T21:44:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.542+0000 7f4880c00640 1 --2- 172.21.15.7:0/4100203667 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4860077f70 0x7f486007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.542+0000 7f4880c00640 1 -- 172.21.15.7:0/4100203667 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f487c09a280 con 0x7f4884145540 2024-09-10T21:44:39.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.542+0000 7f4883e00640 1 --2- 172.21.15.7:0/4100203667 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4860077f70 0x7f486007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:39.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.543+0000 7f4883e00640 1 --2- 172.21.15.7:0/4100203667 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4860077f70 0x7f486007a430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f487400bb50 tx=0x7f487400b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:39.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.547+0000 7f4880c00640 1 -- 172.21.15.7:0/4100203667 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f488415bc60 con 0x7f4884145540 2024-09-10T21:44:39.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.778+0000 7f4889800640 1 -- 172.21.15.7:0/4100203667 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48840d07c0 con 0x7f4884145540 2024-09-10T21:44:39.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.779+0000 7f4880c00640 1 -- 172.21.15.7:0/4100203667 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 35 v35) ==== 94+0+5227 (secure 0 0 0) 0x7f487c016030 con 0x7f4884145540 2024-09-10T21:44:39.780 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:39.781 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":35,"btime":"2024-09-10T21:44:34:389735+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5473,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/2853074495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2853074495},{"type":"v1","addr":"172.21.15.133:6836","nonce":2853074495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3498706462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3498706462},{"type":"v1","addr":"172.21.15.133:6837","nonce":3498706462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/521190705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":521190705},{"type":"v1","addr":"172.21.15.7:6835","nonce":521190705}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:44:30.804578+0000","modified":"2024-09-10T21:44:33.821692+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235016},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235016":{"gid":235016,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6834/364473218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":364473218},{"type":"v1","addr":"172.21.15.7:6834","nonce":364473218}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235016,"qdb_cluster":[235016]},"id":4}]} 2024-09-10T21:44:39.781 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 35 2024-09-10T21:44:39.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.781+0000 7f4889800640 1 -- 172.21.15.7:0/4100203667 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4860077f70 msgr2=0x7f486007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:39.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.781+0000 7f4889800640 1 --2- 172.21.15.7:0/4100203667 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4860077f70 0x7f486007a430 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f487400bb50 tx=0x7f487400b2e0 comp rx=0 tx=0).stop 2024-09-10T21:44:39.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.781+0000 7f4889800640 1 -- 172.21.15.7:0/4100203667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4884145540 msgr2=0x7f48841f16d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:39.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.781+0000 7f4889800640 1 --2- 172.21.15.7:0/4100203667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4884145540 0x7f48841f16d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f487c00e960 tx=0x7f487c00ee20 comp rx=0 tx=0).stop 2024-09-10T21:44:39.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.781+0000 7f4889800640 1 -- 172.21.15.7:0/4100203667 shutdown_connections 2024-09-10T21:44:39.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.781+0000 7f4889800640 1 --2- 172.21.15.7:0/4100203667 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4860077f70 0x7f486007a430 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:39.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.781+0000 7f4889800640 1 --2- 172.21.15.7:0/4100203667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4884145540 0x7f48841f16d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:39.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.781+0000 7f4889800640 1 --2- 172.21.15.7:0/4100203667 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48840b4ac0 0x7f4884130d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:39.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.781+0000 7f4889800640 1 --2- 172.21.15.7:0/4100203667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48840b2f80 0x7f4884157ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:39.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.781+0000 7f4889800640 1 -- 172.21.15.7:0/4100203667 >> 172.21.15.7:0/4100203667 conn(0x7f48840b1d70 msgr2=0x7f48840afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:39.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.781+0000 7f4889800640 1 -- 172.21.15.7:0/4100203667 shutdown_connections 2024-09-10T21:44:39.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:39.781+0000 7f4889800640 1 -- 172.21.15.7:0/4100203667 wait complete. 2024-09-10T21:44:39.799 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-09-10T21:44:39.984 INFO:teuthology.orchestra.run.smithi007.stdout:{"message":"","return_code":0} 2024-09-10T21:44:39.985 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-10T21:44:39.985 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-09-10T21:44:40.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.060+0000 7f86ae705f40 1 Processor -- start 2024-09-10T21:44:40.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.060+0000 7f86ae705f40 1 -- start start 2024-09-10T21:44:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.061+0000 7f86ae705f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1a5134ae0 0x55c1a5161f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.061+0000 7f86ae705f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1a5181110 0x55c1a520e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.061+0000 7f86ae705f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c1a51f1620 0x55c1a5167190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.061+0000 7f86ae705f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c1a51801d0 con 0x55c1a5134ae0 2024-09-10T21:44:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.061+0000 7f86ae705f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c1a5083c60 con 0x55c1a5181110 2024-09-10T21:44:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.061+0000 7f86ae705f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55c1a5181520 con 0x55c1a51f1620 2024-09-10T21:44:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.062+0000 7f86ab800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1a5134ae0 0x55c1a5161f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.062+0000 7f86ab800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1a5134ae0 0x55c1a5161f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43640/0 (socket says 172.21.15.7:43640) 2024-09-10T21:44:40.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.062+0000 7f86ab800640 1 -- 172.21.15.7:0/2814119585 learned_addr learned my addr 172.21.15.7:0/2814119585 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:40.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.062+0000 7f86aae00640 1 --2- 172.21.15.7:0/2814119585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1a5181110 0x55c1a520e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.062+0000 7f86ac200640 1 --2- 172.21.15.7:0/2814119585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c1a51f1620 0x55c1a5167190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.062+0000 7f86ab800640 1 -- 172.21.15.7:0/2814119585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c1a51f1620 msgr2=0x55c1a5167190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.062+0000 7f86ab800640 1 --2- 172.21.15.7:0/2814119585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c1a51f1620 0x55c1a5167190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.062+0000 7f86ab800640 1 -- 172.21.15.7:0/2814119585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1a5181110 msgr2=0x55c1a520e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.062+0000 7f86ab800640 1 --2- 172.21.15.7:0/2814119585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1a5181110 0x55c1a520e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.062+0000 7f86ab800640 1 -- 172.21.15.7:0/2814119585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1a505bf10 con 0x55c1a5134ae0 2024-09-10T21:44:40.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.062+0000 7f86aae00640 1 --2- 172.21.15.7:0/2814119585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1a5181110 0x55c1a520e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:40.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.062+0000 7f86ab800640 1 --2- 172.21.15.7:0/2814119585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1a5134ae0 0x55c1a5161f40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f869c025e80 tx=0x7f869c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f323312cca5a098b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.062+0000 7f86a9a00640 1 -- 172.21.15.7:0/2814119585 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c04d8e0 con 0x55c1a5134ae0 2024-09-10T21:44:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.062+0000 7f86a9a00640 1 -- 172.21.15.7:0/2814119585 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f869c04dd70 con 0x55c1a5134ae0 2024-09-10T21:44:40.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.063+0000 7f86a9a00640 1 -- 172.21.15.7:0/2814119585 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f869c0566d0 con 0x55c1a5134ae0 2024-09-10T21:44:40.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.063+0000 7f86ae705f40 1 -- 172.21.15.7:0/2814119585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1a5134ae0 msgr2=0x55c1a5161f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.063+0000 7f86ae705f40 1 --2- 172.21.15.7:0/2814119585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1a5134ae0 0x55c1a5161f40 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f869c025e80 tx=0x7f869c002fb0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.063+0000 7f86ae705f40 1 -- 172.21.15.7:0/2814119585 shutdown_connections 2024-09-10T21:44:40.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.063+0000 7f86ae705f40 1 --2- 172.21.15.7:0/2814119585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1a5181110 0x55c1a520e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.063+0000 7f86ae705f40 1 --2- 172.21.15.7:0/2814119585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c1a51f1620 0x55c1a5167190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.063+0000 7f86ae705f40 1 --2- 172.21.15.7:0/2814119585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1a5134ae0 0x55c1a5161f40 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.063+0000 7f86ae705f40 1 -- 172.21.15.7:0/2814119585 >> 172.21.15.7:0/2814119585 conn(0x55c1a5163de0 msgr2=0x55c1a5164eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:40.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.063+0000 7f86ae705f40 1 -- 172.21.15.7:0/2814119585 shutdown_connections 2024-09-10T21:44:40.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.063+0000 7f86ae705f40 1 -- 172.21.15.7:0/2814119585 wait complete. 2024-09-10T21:44:40.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.064+0000 7f86ae705f40 1 Processor -- start 2024-09-10T21:44:40.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.064+0000 7f86ae705f40 1 -- start start 2024-09-10T21:44:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.065+0000 7f86ae705f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c1a5134ae0 0x55c1a52984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.065+0000 7f86ae705f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1a5181110 0x55c1a51ed5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.065+0000 7f86ae705f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1a51f1620 0x55c1a5167050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.065+0000 7f86ae705f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c1a5205d40 con 0x55c1a51f1620 2024-09-10T21:44:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.065+0000 7f86ae705f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c1a5166c50 con 0x55c1a5181110 2024-09-10T21:44:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.065+0000 7f86ae705f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55c1a51636b0 con 0x55c1a5134ae0 2024-09-10T21:44:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.065+0000 7f86ac200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1a51f1620 0x55c1a5167050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.065+0000 7f86ac200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1a51f1620 0x55c1a5167050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43648/0 (socket says 172.21.15.7:43648) 2024-09-10T21:44:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.065+0000 7f86ac200640 1 -- 172.21.15.7:0/398672323 learned_addr learned my addr 172.21.15.7:0/398672323 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.065+0000 7f86ab800640 1 --2- 172.21.15.7:0/398672323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c1a5134ae0 0x55c1a52984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.065+0000 7f86aae00640 1 --2- 172.21.15.7:0/398672323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1a5181110 0x55c1a51ed5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.066+0000 7f86ac200640 1 -- 172.21.15.7:0/398672323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c1a5134ae0 msgr2=0x55c1a52984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.066+0000 7f86ac200640 1 --2- 172.21.15.7:0/398672323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c1a5134ae0 0x55c1a52984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.066+0000 7f86ac200640 1 -- 172.21.15.7:0/398672323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1a5181110 msgr2=0x55c1a51ed5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.066+0000 7f86ac200640 1 --2- 172.21.15.7:0/398672323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c1a5181110 0x55c1a51ed5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.066+0000 7f86ac200640 1 -- 172.21.15.7:0/398672323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1a5298a10 con 0x55c1a51f1620 2024-09-10T21:44:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.066+0000 7f86ac200640 1 --2- 172.21.15.7:0/398672323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1a51f1620 0x55c1a5167050 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f86a4004530 tx=0x7f86a400fb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.066+0000 7f86a3e00640 1 -- 172.21.15.7:0/398672323 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4011920 con 0x55c1a51f1620 2024-09-10T21:44:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.066+0000 7f86a3e00640 1 -- 172.21.15.7:0/398672323 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86a4011ae0 con 0x55c1a51f1620 2024-09-10T21:44:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.066+0000 7f86ae705f40 1 -- 172.21.15.7:0/398672323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c1a5297d10 con 0x55c1a51f1620 2024-09-10T21:44:40.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.066+0000 7f86a3e00640 1 -- 172.21.15.7:0/398672323 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4010440 con 0x55c1a51f1620 2024-09-10T21:44:40.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.066+0000 7f86ae705f40 1 -- 172.21.15.7:0/398672323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1a51668a0 con 0x55c1a51f1620 2024-09-10T21:44:40.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.069+0000 7f86a3e00640 1 -- 172.21.15.7:0/398672323 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f86a400e040 con 0x55c1a51f1620 2024-09-10T21:44:40.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.069+0000 7f86a3e00640 1 --2- 172.21.15.7:0/398672323 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f868c076950 0x7f868c078e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.070+0000 7f86a3e00640 1 -- 172.21.15.7:0/398672323 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f86a4099660 con 0x55c1a51f1620 2024-09-10T21:44:40.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.070+0000 7f86ab800640 1 --2- 172.21.15.7:0/398672323 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f868c076950 0x7f868c078e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.070+0000 7f86ae705f40 1 --2- 172.21.15.7:0/398672323 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x55c1a5167590 0x55c1a5294c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.070+0000 7f86ae705f40 1 -- 172.21.15.7:0/398672323 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e58) -- 0x55c1a5295170 con 0x55c1a5167590 2024-09-10T21:44:40.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.070+0000 7f86aae00640 1 --2- 172.21.15.7:0/398672323 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x55c1a5167590 0x55c1a5294c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.071+0000 7f86ab800640 1 --2- 172.21.15.7:0/398672323 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f868c076950 0x7f868c078e10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f869c025e80 tx=0x7f869c004940 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.071+0000 7f86aae00640 1 --2- 172.21.15.7:0/398672323 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x55c1a5167590 0x55c1a5294c30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.071+0000 7f86aae00640 1 -- 172.21.15.7:0/398672323 <== osd.5 v2:172.21.15.133:6818/3838498382 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) ==== 164+0+94 (crc 0 0 0) 0x7f869800c650 con 0x55c1a5167590 2024-09-10T21:44:40.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.072+0000 7f86ae705f40 1 -- 172.21.15.7:0/398672323 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x55c1a5167590 msgr2=0x55c1a5294c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.072+0000 7f86ae705f40 1 --2- 172.21.15.7:0/398672323 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x55c1a5167590 0x55c1a5294c30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.072+0000 7f86ae705f40 1 -- 172.21.15.7:0/398672323 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f868c076950 msgr2=0x7f868c078e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.072+0000 7f86ae705f40 1 --2- 172.21.15.7:0/398672323 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f868c076950 0x7f868c078e10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f869c025e80 tx=0x7f869c004940 comp rx=0 tx=0).stop 2024-09-10T21:44:40.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.072+0000 7f86ae705f40 1 -- 172.21.15.7:0/398672323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1a51f1620 msgr2=0x55c1a5167050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.072+0000 7f86ae705f40 1 --2- 172.21.15.7:0/398672323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c1a51f1620 0x55c1a5167050 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f86a4004530 tx=0x7f86a400fb00 comp rx=0 tx=0).stop 2024-09-10T21:44:40.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.072+0000 7f86ac200640 1 -- 172.21.15.7:0/398672323 reap_dead start 2024-09-10T21:44:40.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.072+0000 7f86ae705f40 1 -- 172.21.15.7:0/398672323 shutdown_connections 2024-09-10T21:44:40.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.072+0000 7f86ae705f40 1 -- 172.21.15.7:0/398672323 >> 172.21.15.7:0/398672323 conn(0x55c1a5163de0 msgr2=0x55c1a5204a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:40.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.072+0000 7f86ae705f40 1 -- 172.21.15.7:0/398672323 shutdown_connections 2024-09-10T21:44:40.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.072+0000 7f86ae705f40 1 -- 172.21.15.7:0/398672323 wait complete. 2024-09-10T21:44:40.079 INFO:teuthology.orchestra.run.smithi007.stdout:Xfiledir_test 2024-09-10T21:44:40.079 DEBUG:teuthology.orchestra.run.smithi007:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-09-10T21:44:40.161 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:44:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627777, 2024-09-10T21:44:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: "ancestors": [ 2024-09-10T21:44:40.161 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "dirino": 1099511627776, 2024-09-10T21:44:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "dname": "file", 2024-09-10T21:44:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 2 2024-09-10T21:44:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:44:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "dirino": 1, 2024-09-10T21:44:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "dname": "dir_test", 2024-09-10T21:44:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 4 2024-09-10T21:44:40.162 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T21:44:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "pool": 9, 2024-09-10T21:44:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: "old_pools": [] 2024-09-10T21:44:40.163 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:44:40.163 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:44:40.167 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2024-09-10T21:44:40.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.284+0000 7f46c51d5f40 1 Processor -- start 2024-09-10T21:44:40.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.284+0000 7f46c51d5f40 1 -- start start 2024-09-10T21:44:40.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.285+0000 7f46c51d5f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617ec07f210 0x5617ec1214d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.285+0000 7f46c51d5f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617ec098da0 0x5617ec1240a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.285+0000 7f46c51d5f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617ec120b90 0x5617ec07afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.285+0000 7f46c51d5f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5617ebffa1d0 con 0x5617ec098da0 2024-09-10T21:44:40.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.285+0000 7f46c51d5f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5617ec006f90 con 0x5617ec120b90 2024-09-10T21:44:40.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.285+0000 7f46c51d5f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5617ebf9bc60 con 0x5617ec07f210 2024-09-10T21:44:40.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.285+0000 7f46bbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617ec098da0 0x5617ec1240a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.285+0000 7f46bbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617ec098da0 0x5617ec1240a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43654/0 (socket says 172.21.15.7:43654) 2024-09-10T21:44:40.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.285+0000 7f46bbe00640 1 -- 172.21.15.7:0/3782749760 learned_addr learned my addr 172.21.15.7:0/3782749760 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:40.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.285+0000 7f46c2400640 1 --2- 172.21.15.7:0/3782749760 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617ec07f210 0x5617ec1214d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.285+0000 7f46c2e00640 1 --2- 172.21.15.7:0/3782749760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617ec120b90 0x5617ec07afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.286+0000 7f46bbe00640 1 -- 172.21.15.7:0/3782749760 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617ec07f210 msgr2=0x5617ec1214d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.286+0000 7f46bbe00640 1 --2- 172.21.15.7:0/3782749760 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617ec07f210 0x5617ec1214d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.286+0000 7f46bbe00640 1 -- 172.21.15.7:0/3782749760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617ec120b90 msgr2=0x5617ec07afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.286+0000 7f46bbe00640 1 --2- 172.21.15.7:0/3782749760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617ec120b90 0x5617ec07afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.286+0000 7f46bbe00640 1 -- 172.21.15.7:0/3782749760 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617ec125450 con 0x5617ec098da0 2024-09-10T21:44:40.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.286+0000 7f46bbe00640 1 --2- 172.21.15.7:0/3782749760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617ec098da0 0x5617ec1240a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00b350 tx=0x7f46ac00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ea74c0e787467a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.286+0000 7f46c1000640 1 -- 172.21.15.7:0/3782749760 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac012070 con 0x5617ec098da0 2024-09-10T21:44:40.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.286+0000 7f46c1000640 1 -- 172.21.15.7:0/3782749760 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46ac006050 con 0x5617ec098da0 2024-09-10T21:44:40.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.286+0000 7f46c1000640 1 -- 172.21.15.7:0/3782749760 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46ac00dc00 con 0x5617ec098da0 2024-09-10T21:44:40.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.287+0000 7f46c51d5f40 1 -- 172.21.15.7:0/3782749760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617ec098da0 msgr2=0x5617ec1240a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.287+0000 7f46c51d5f40 1 --2- 172.21.15.7:0/3782749760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617ec098da0 0x5617ec1240a0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f46ac00b350 tx=0x7f46ac00b810 comp rx=0 tx=0).stop 2024-09-10T21:44:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.287+0000 7f46c51d5f40 1 -- 172.21.15.7:0/3782749760 shutdown_connections 2024-09-10T21:44:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.287+0000 7f46c51d5f40 1 --2- 172.21.15.7:0/3782749760 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617ec120b90 0x5617ec07afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.287+0000 7f46c51d5f40 1 --2- 172.21.15.7:0/3782749760 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617ec098da0 0x5617ec1240a0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.287+0000 7f46c51d5f40 1 --2- 172.21.15.7:0/3782749760 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617ec07f210 0x5617ec1214d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.287+0000 7f46c51d5f40 1 -- 172.21.15.7:0/3782749760 >> 172.21.15.7:0/3782749760 conn(0x5617ec099df0 msgr2=0x5617ec122e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.287+0000 7f46c51d5f40 1 -- 172.21.15.7:0/3782749760 shutdown_connections 2024-09-10T21:44:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.287+0000 7f46c51d5f40 1 -- 172.21.15.7:0/3782749760 wait complete. 2024-09-10T21:44:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.288+0000 7f46c51d5f40 1 Processor -- start 2024-09-10T21:44:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.288+0000 7f46c51d5f40 1 -- start start 2024-09-10T21:44:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.289+0000 7f46c51d5f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617ec07f210 0x5617ec07af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.289+0000 7f46c51d5f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617ec098da0 0x5617ec110720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.289+0000 7f46c51d5f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617ec120b90 0x5617ec11e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.289+0000 7f46c51d5f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5617ec1259b0 con 0x5617ec07f210 2024-09-10T21:44:40.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.289+0000 7f46c51d5f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5617ec07a9e0 con 0x5617ec098da0 2024-09-10T21:44:40.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.289+0000 7f46c51d5f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5617ec07a3a0 con 0x5617ec120b90 2024-09-10T21:44:40.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.289+0000 7f46c2400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617ec07f210 0x5617ec07af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.289+0000 7f46c2400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617ec07f210 0x5617ec07af00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43668/0 (socket says 172.21.15.7:43668) 2024-09-10T21:44:40.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.289+0000 7f46c2400640 1 -- 172.21.15.7:0/2358580134 learned_addr learned my addr 172.21.15.7:0/2358580134 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:40.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.289+0000 7f46bbe00640 1 --2- 172.21.15.7:0/2358580134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617ec098da0 0x5617ec110720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.289+0000 7f46c2e00640 1 --2- 172.21.15.7:0/2358580134 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617ec120b90 0x5617ec11e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.289+0000 7f46c2400640 1 -- 172.21.15.7:0/2358580134 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617ec120b90 msgr2=0x5617ec11e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.289+0000 7f46c2400640 1 --2- 172.21.15.7:0/2358580134 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617ec120b90 0x5617ec11e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.289+0000 7f46c2400640 1 -- 172.21.15.7:0/2358580134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617ec098da0 msgr2=0x5617ec110720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.289+0000 7f46c2400640 1 --2- 172.21.15.7:0/2358580134 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617ec098da0 0x5617ec110720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.289+0000 7f46c2400640 1 -- 172.21.15.7:0/2358580134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617ec07b770 con 0x5617ec07f210 2024-09-10T21:44:40.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.290+0000 7f46c2400640 1 --2- 172.21.15.7:0/2358580134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617ec07f210 0x5617ec07af00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f46b40464c0 tx=0x7f46b404ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.290+0000 7f46baa00640 1 -- 172.21.15.7:0/2358580134 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b4004580 con 0x5617ec07f210 2024-09-10T21:44:40.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.290+0000 7f46baa00640 1 -- 172.21.15.7:0/2358580134 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46b4067070 con 0x5617ec07f210 2024-09-10T21:44:40.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.290+0000 7f46c51d5f40 1 -- 172.21.15.7:0/2358580134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5617ec1182b0 con 0x5617ec07f210 2024-09-10T21:44:40.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.290+0000 7f46baa00640 1 -- 172.21.15.7:0/2358580134 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b40566b0 con 0x5617ec07f210 2024-09-10T21:44:40.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.290+0000 7f46c51d5f40 1 -- 172.21.15.7:0/2358580134 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617ec118870 con 0x5617ec07f210 2024-09-10T21:44:40.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.292+0000 7f46baa00640 1 -- 172.21.15.7:0/2358580134 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f46b4056850 con 0x5617ec07f210 2024-09-10T21:44:40.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.293+0000 7f46baa00640 1 --2- 172.21.15.7:0/2358580134 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f46a0076950 0x7f46a0078e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.293+0000 7f46baa00640 1 -- 172.21.15.7:0/2358580134 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f46b40dcbe0 con 0x5617ec07f210 2024-09-10T21:44:40.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.293+0000 7f46c51d5f40 1 --2- 172.21.15.7:0/2358580134 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5617ec079890 0x5617ec07dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.293+0000 7f46c51d5f40 1 -- 172.21.15.7:0/2358580134 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e58) -- 0x5617ec110c60 con 0x5617ec079890 2024-09-10T21:44:40.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.294+0000 7f46bbe00640 1 --2- 172.21.15.7:0/2358580134 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f46a0076950 0x7f46a0078e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.294+0000 7f46c2e00640 1 --2- 172.21.15.7:0/2358580134 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5617ec079890 0x5617ec07dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.294+0000 7f46bbe00640 1 --2- 172.21.15.7:0/2358580134 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f46a0076950 0x7f46a0078e10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x5617ec09a840 tx=0x7f46ac019040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.294+0000 7f46c2e00640 1 --2- 172.21.15.7:0/2358580134 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5617ec079890 0x5617ec07dc00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.296+0000 7f46c2e00640 1 -- 172.21.15.7:0/2358580134 <== osd.5 v2:172.21.15.133:6818/3838498382 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v58'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f46bc0038a0 con 0x5617ec079890 2024-09-10T21:44:40.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.296+0000 7f46c51d5f40 1 -- 172.21.15.7:0/2358580134 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5617ec079890 msgr2=0x5617ec07dc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.296+0000 7f46c51d5f40 1 --2- 172.21.15.7:0/2358580134 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5617ec079890 0x5617ec07dc00 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.296+0000 7f46c51d5f40 1 -- 172.21.15.7:0/2358580134 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f46a0076950 msgr2=0x7f46a0078e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.296+0000 7f46c51d5f40 1 --2- 172.21.15.7:0/2358580134 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f46a0076950 0x7f46a0078e10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x5617ec09a840 tx=0x7f46ac019040 comp rx=0 tx=0).stop 2024-09-10T21:44:40.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.296+0000 7f46c51d5f40 1 -- 172.21.15.7:0/2358580134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617ec07f210 msgr2=0x5617ec07af00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.296+0000 7f46c51d5f40 1 --2- 172.21.15.7:0/2358580134 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617ec07f210 0x5617ec07af00 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f46b40464c0 tx=0x7f46b404ec50 comp rx=0 tx=0).stop 2024-09-10T21:44:40.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.296+0000 7f46c2e00640 1 -- 172.21.15.7:0/2358580134 reap_dead start 2024-09-10T21:44:40.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.296+0000 7f46c51d5f40 1 -- 172.21.15.7:0/2358580134 shutdown_connections 2024-09-10T21:44:40.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.296+0000 7f46c51d5f40 1 -- 172.21.15.7:0/2358580134 >> 172.21.15.7:0/2358580134 conn(0x5617ec099df0 msgr2=0x5617ec1214d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:40.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.297+0000 7f46c51d5f40 1 -- 172.21.15.7:0/2358580134 shutdown_connections 2024-09-10T21:44:40.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.297+0000 7f46c51d5f40 1 -- 172.21.15.7:0/2358580134 wait complete. 2024-09-10T21:44:40.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:44:40.302 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2024-09-10T21:44:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.466+0000 7fc483200640 1 -- 172.21.15.7:0/845750509 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4840cf0d0 msgr2=0x7fc4840cf560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.466+0000 7fc483200640 1 --2- 172.21.15.7:0/845750509 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4840cf0d0 0x7fc4840cf560 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc47c005fd0 tx=0x7fc47c02aaa0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.466+0000 7fc483200640 1 -- 172.21.15.7:0/845750509 shutdown_connections 2024-09-10T21:44:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.466+0000 7fc483200640 1 --2- 172.21.15.7:0/845750509 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4840cf0d0 0x7fc4840cf560 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.466+0000 7fc483200640 1 --2- 172.21.15.7:0/845750509 >> v2:172.21.15.133:3300/0 conn(0x7fc48415dbf0 0x7fc4840d4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.466+0000 7fc483200640 1 -- 172.21.15.7:0/845750509 >> 172.21.15.7:0/845750509 conn(0x7fc4840d3d70 msgr2=0x7fc4840d4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.466+0000 7fc483200640 1 -- 172.21.15.7:0/845750509 shutdown_connections 2024-09-10T21:44:40.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.466+0000 7fc483200640 1 -- 172.21.15.7:0/845750509 wait complete. 2024-09-10T21:44:40.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.468+0000 7fc483200640 1 Processor -- start 2024-09-10T21:44:40.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.468+0000 7fc483200640 1 -- start start 2024-09-10T21:44:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.469+0000 7fc483200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4840cf0d0 0x7fc48414d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.469+0000 7fc483200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc48415dbf0 0x7fc4840cd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.469+0000 7fc483200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc48415dfd0 0x7fc4840b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.469+0000 7fc483200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc484162000 con 0x7fc4840cf0d0 2024-09-10T21:44:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.469+0000 7fc483200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc484161e80 con 0x7fc48415dbf0 2024-09-10T21:44:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.469+0000 7fc483200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc484162180 con 0x7fc48415dfd0 2024-09-10T21:44:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.469+0000 7fc481e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4840cf0d0 0x7fc48414d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.469+0000 7fc481e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4840cf0d0 0x7fc48414d660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43680/0 (socket says 172.21.15.7:43680) 2024-09-10T21:44:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.469+0000 7fc481e00640 1 -- 172.21.15.7:0/726596172 learned_addr learned my addr 172.21.15.7:0/726596172 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.469+0000 7fc481400640 1 --2- 172.21.15.7:0/726596172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc48415dbf0 0x7fc4840cd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.469+0000 7fc482800640 1 --2- 172.21.15.7:0/726596172 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc48415dfd0 0x7fc4840b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.470+0000 7fc481e00640 1 -- 172.21.15.7:0/726596172 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc48415dfd0 msgr2=0x7fc4840b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.470+0000 7fc481e00640 1 --2- 172.21.15.7:0/726596172 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc48415dfd0 0x7fc4840b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.470+0000 7fc481e00640 1 -- 172.21.15.7:0/726596172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc48415dbf0 msgr2=0x7fc4840cd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.470+0000 7fc481e00640 1 --2- 172.21.15.7:0/726596172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc48415dbf0 0x7fc4840cd1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.470+0000 7fc481e00640 1 -- 172.21.15.7:0/726596172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4840cdbb0 con 0x7fc4840cf0d0 2024-09-10T21:44:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.470+0000 7fc481400640 1 --2- 172.21.15.7:0/726596172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc48415dbf0 0x7fc4840cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:44:40.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.470+0000 7fc481e00640 1 --2- 172.21.15.7:0/726596172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4840cf0d0 0x7fc48414d660 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc47400ea20 tx=0x7fc47400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.470+0000 7fc47a000640 1 -- 172.21.15.7:0/726596172 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47400cd20 con 0x7fc4840cf0d0 2024-09-10T21:44:40.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.470+0000 7fc47a000640 1 -- 172.21.15.7:0/726596172 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc474010430 con 0x7fc4840cf0d0 2024-09-10T21:44:40.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.470+0000 7fc47a000640 1 -- 172.21.15.7:0/726596172 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc474010840 con 0x7fc4840cf0d0 2024-09-10T21:44:40.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.470+0000 7fc483200640 1 -- 172.21.15.7:0/726596172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48414dff0 con 0x7fc4840cf0d0 2024-09-10T21:44:40.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.470+0000 7fc483200640 1 -- 172.21.15.7:0/726596172 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc48414e5b0 con 0x7fc4840cf0d0 2024-09-10T21:44:40.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.473+0000 7fc47a000640 1 -- 172.21.15.7:0/726596172 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc4740109e0 con 0x7fc4840cf0d0 2024-09-10T21:44:40.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.474+0000 7fc47a000640 1 --2- 172.21.15.7:0/726596172 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc464077f70 0x7fc46407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.474+0000 7fc47a000640 1 -- 172.21.15.7:0/726596172 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fc47409a860 con 0x7fc4840cf0d0 2024-09-10T21:44:40.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.475+0000 7fc48b800740 1 Processor -- start 2024-09-10T21:44:40.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.475+0000 7fc481400640 1 --2- 172.21.15.7:0/726596172 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc464077f70 0x7fc46407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.475+0000 7fc48b800740 1 -- start start 2024-09-10T21:44:40.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.475+0000 7fc481400640 1 --2- 172.21.15.7:0/726596172 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc464077f70 0x7fc46407a430 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc470007fd0 tx=0x7fc470009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.476+0000 7fc48b800740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804557c20 0x563804558000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.476+0000 7fc48b800740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804512d90 0x563804556700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.476+0000 7fc48b800740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804513310 0x563804556c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.476+0000 7fc48b800740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563804484780 con 0x563804513310 2024-09-10T21:44:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.476+0000 7fc48b800740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563804473bb0 con 0x563804512d90 2024-09-10T21:44:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.476+0000 7fc48b800740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5638044779c0 con 0x563804557c20 2024-09-10T21:44:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.476+0000 7fc482800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804513310 0x563804556c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.476+0000 7fc482800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804513310 0x563804556c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43686/0 (socket says 172.21.15.7:43686) 2024-09-10T21:44:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.476+0000 7fc482800640 1 -- 172.21.15.7:0/2259006786 learned_addr learned my addr 172.21.15.7:0/2259006786 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.476+0000 7fc481e00640 1 --2- 172.21.15.7:0/2259006786 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804557c20 0x563804558000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.476+0000 7fc481400640 1 --2- 172.21.15.7:0/2259006786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804512d90 0x563804556700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.476+0000 7fc482800640 1 -- 172.21.15.7:0/2259006786 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804557c20 msgr2=0x563804558000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.476+0000 7fc482800640 1 --2- 172.21.15.7:0/2259006786 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804557c20 0x563804558000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.476+0000 7fc482800640 1 -- 172.21.15.7:0/2259006786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804512d90 msgr2=0x563804556700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.476+0000 7fc482800640 1 --2- 172.21.15.7:0/2259006786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804512d90 0x563804556700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.476+0000 7fc482800640 1 -- 172.21.15.7:0/2259006786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56380424e7b0 con 0x563804513310 2024-09-10T21:44:40.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.477+0000 7fc482800640 1 --2- 172.21.15.7:0/2259006786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804513310 0x563804556c40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009bf0 tx=0x7fc47c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b4b1aeeb2b6091 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.477+0000 7fc46e600640 1 -- 172.21.15.7:0/2259006786 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c003d80 con 0x563804513310 2024-09-10T21:44:40.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.477+0000 7fc46e600640 1 -- 172.21.15.7:0/2259006786 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc47c044070 con 0x563804513310 2024-09-10T21:44:40.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.477+0000 7fc46e600640 1 -- 172.21.15.7:0/2259006786 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c02e8a0 con 0x563804513310 2024-09-10T21:44:40.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.478+0000 7fc48b800740 1 -- 172.21.15.7:0/2259006786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804513310 msgr2=0x563804556c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.478+0000 7fc48b800740 1 --2- 172.21.15.7:0/2259006786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804513310 0x563804556c40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009bf0 tx=0x7fc47c004ab0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.478+0000 7fc48b800740 1 -- 172.21.15.7:0/2259006786 shutdown_connections 2024-09-10T21:44:40.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.478+0000 7fc48b800740 1 --2- 172.21.15.7:0/2259006786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804512d90 0x563804556700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.478+0000 7fc48b800740 1 --2- 172.21.15.7:0/2259006786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804513310 0x563804556c40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.478+0000 7fc48b800740 1 --2- 172.21.15.7:0/2259006786 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804557c20 0x563804558000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.478+0000 7fc48b800740 1 -- 172.21.15.7:0/2259006786 >> 172.21.15.7:0/2259006786 conn(0x56380450e360 msgr2=0x563804511d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:40.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.478+0000 7fc48b800740 1 -- 172.21.15.7:0/2259006786 shutdown_connections 2024-09-10T21:44:40.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.478+0000 7fc48b800740 1 -- 172.21.15.7:0/2259006786 wait complete. 2024-09-10T21:44:40.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.478+0000 7fc48b800740 1 -- start start 2024-09-10T21:44:40.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.479+0000 7fc48b800740 1 Processor -- start 2024-09-10T21:44:40.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.480+0000 7fc48b800740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804513310 0x563804569fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.480+0000 7fc48b800740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804557c20 0x563804529710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.480+0000 7fc48b800740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804553b40 0x563804526d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.480+0000 7fc48b800740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5638041e46f0 con 0x563804557c20 2024-09-10T21:44:40.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.480+0000 7fc48b800740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563804419590 con 0x563804553b40 2024-09-10T21:44:40.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.480+0000 7fc48b800740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x56380450cd80 con 0x563804513310 2024-09-10T21:44:40.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.480+0000 7fc48b800740 10 client.0 _ll_register_callbacks cb 0x56380441f5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:44:40.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.480+0000 7fc481400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804557c20 0x563804529710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.480+0000 7fc481400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804557c20 0x563804529710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43700/0 (socket says 172.21.15.7:43700) 2024-09-10T21:44:40.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.480+0000 7fc481400640 1 -- 172.21.15.7:0/1761458985 learned_addr learned my addr 172.21.15.7:0/1761458985 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:40.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.480+0000 7fc481e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804513310 0x563804569fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.480+0000 7fc482800640 1 --2- 172.21.15.7:0/1761458985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804553b40 0x563804526d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.480+0000 7fc481400640 1 -- 172.21.15.7:0/1761458985 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804513310 msgr2=0x563804569fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.480+0000 7fc481400640 1 --2- 172.21.15.7:0/1761458985 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804513310 0x563804569fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.480+0000 7fc481400640 1 -- 172.21.15.7:0/1761458985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804553b40 msgr2=0x563804526d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.480+0000 7fc481400640 1 --2- 172.21.15.7:0/1761458985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804553b40 0x563804526d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.480+0000 7fc481400640 1 -- 172.21.15.7:0/1761458985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638042aef40 con 0x563804557c20 2024-09-10T21:44:40.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.481+0000 7fc481400640 1 --2- 172.21.15.7:0/1761458985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804557c20 0x563804529710 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc47000de30 tx=0x7fc470001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a8a1b11f7c7bebb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.481+0000 7fc462a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:44:40.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.481+0000 7fc462a00640 1 -- 172.21.15.7:0/1761458985 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc470024030 con 0x563804557c20 2024-09-10T21:44:40.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.481+0000 7fc48b800740 1 -- 172.21.15.7:0/1761458985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc47c044070 con 0x563804557c20 2024-09-10T21:44:40.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.481+0000 7fc462a00640 1 -- 172.21.15.7:0/1761458985 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fc47001d040 con 0x563804557c20 2024-09-10T21:44:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.481+0000 7fc462a00640 1 -- 172.21.15.7:0/1761458985 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc47002d050 con 0x563804557c20 2024-09-10T21:44:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.481+0000 7fc48b800740 10 client.5859 fetch_fsmap learned FSMap version 35 2024-09-10T21:44:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.481+0000 7fc48b800740 1 -- 172.21.15.7:0/1761458985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x5638041b3650 con 0x563804557c20 2024-09-10T21:44:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.482+0000 7fc462a00640 1 -- 172.21.15.7:0/1761458985 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc47c044070 con 0x563804557c20 2024-09-10T21:44:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.482+0000 7fc462a00640 1 -- 172.21.15.7:0/1761458985 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7fc470007020 con 0x563804557c20 2024-09-10T21:44:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.482+0000 7fc48b800740 10 client.5859 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T21:44:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.482+0000 7fc48b800740 10 client.5859 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-10T21:44:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.482+0000 7fc48b800740 20 client.5859 populate_metadata read hostname 'smithi007' 2024-09-10T21:44:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.482+0000 7fc48b800740 1 --2- 172.21.15.7:0/1761458985 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x56380455a350 0x56380454ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.482+0000 7fc48b800740 4 client.5859 mds_command: new command op to 235016 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:44:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.482+0000 7fc48b800740 1 -- 172.21.15.7:0/1761458985 --> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5638044866d0 con 0x56380455a350 2024-09-10T21:44:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.482+0000 7fc481e00640 1 --2- 172.21.15.7:0/1761458985 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x56380455a350 0x56380454ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.483+0000 7fc481e00640 1 --2- 172.21.15.7:0/1761458985 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x56380455a350 0x56380454ec80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cef5e023a4f53b server_cookie=5ef69481a146db96 in_seq=0 out_seq=0 2024-09-10T21:44:40.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.483+0000 7fc462a00640 10 client.5859 ms_handle_connect on v2:172.21.15.7:6833/364473218 2024-09-10T21:44:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.483+0000 7fc462a00640 1 -- 172.21.15.7:0/1761458985 <== mds.0 v2:172.21.15.7:6833/364473218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5638044866d0 con 0x56380455a350 2024-09-10T21:44:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.483+0000 7fc462a00640 10 client.5859 handle_command_reply: tid=0 2024-09-10T21:44:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.483+0000 7fc48b800740 1 client.5859 shutdown 2024-09-10T21:44:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.484+0000 7fc48b800740 1 -- 172.21.15.7:0/1761458985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804557c20 msgr2=0x563804529710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.484+0000 7fc48b800740 1 --2- 172.21.15.7:0/1761458985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804557c20 0x563804529710 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc47000de30 tx=0x7fc470001120 comp rx=0 tx=0).stop 2024-09-10T21:44:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.484+0000 7fc48b800740 1 -- 172.21.15.7:0/1761458985 shutdown_connections 2024-09-10T21:44:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.484+0000 7fc48b800740 1 --2- 172.21.15.7:0/1761458985 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x56380455a350 0x56380454ec80 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.484+0000 7fc48b800740 1 --2- 172.21.15.7:0/1761458985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804553b40 0x563804526d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.484+0000 7fc48b800740 1 --2- 172.21.15.7:0/1761458985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804557c20 0x563804529710 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.484+0000 7fc48b800740 1 --2- 172.21.15.7:0/1761458985 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804513310 0x563804569fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.484+0000 7fc462a00640 0 client.5859 ms_handle_reset on v2:172.21.15.7:6833/364473218 2024-09-10T21:44:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.484+0000 7fc48b800740 1 -- 172.21.15.7:0/1761458985 >> 172.21.15.7:0/1761458985 conn(0x563804512d90 msgr2=0x56380450bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:40.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.484+0000 7fc48b800740 1 -- 172.21.15.7:0/1761458985 shutdown_connections 2024-09-10T21:44:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.484+0000 7fc48b800740 1 -- 172.21.15.7:0/1761458985 wait complete. 2024-09-10T21:44:40.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.484+0000 7fc48b800740 20 client.5859 trim_cache size 0 max 16384 2024-09-10T21:44:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.503+0000 7fc48b800740 1 Processor -- start 2024-09-10T21:44:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.503+0000 7fc48b800740 1 -- start start 2024-09-10T21:44:40.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.503+0000 7fc48b800740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804513310 0x56380452c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.504+0000 7fc48b800740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804553b40 0x56380450b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.504+0000 7fc48b800740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804557c20 0x56380450ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.504+0000 7fc48b800740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc484200f70 con 0x563804557c20 2024-09-10T21:44:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.504+0000 7fc48b800740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841fffd0 con 0x563804553b40 2024-09-10T21:44:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.504+0000 7fc48b800740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc4840cbaa0 con 0x563804513310 2024-09-10T21:44:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.504+0000 7fc482800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804557c20 0x56380450ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.504+0000 7fc482800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804557c20 0x56380450ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43710/0 (socket says 172.21.15.7:43710) 2024-09-10T21:44:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.504+0000 7fc482800640 1 -- 172.21.15.7:0/1251129691 learned_addr learned my addr 172.21.15.7:0/1251129691 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.504+0000 7fc481e00640 1 --2- 172.21.15.7:0/1251129691 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804513310 0x56380452c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.504+0000 7fc482800640 1 -- 172.21.15.7:0/1251129691 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804513310 msgr2=0x56380452c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.504+0000 7fc481400640 1 --2- 172.21.15.7:0/1251129691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804553b40 0x56380450b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.504+0000 7fc482800640 1 --2- 172.21.15.7:0/1251129691 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804513310 0x56380452c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.504+0000 7fc482800640 1 -- 172.21.15.7:0/1251129691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804553b40 msgr2=0x56380450b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.504+0000 7fc482800640 1 --2- 172.21.15.7:0/1251129691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804553b40 0x56380450b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.504+0000 7fc482800640 1 -- 172.21.15.7:0/1251129691 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563804512580 con 0x563804557c20 2024-09-10T21:44:40.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.504+0000 7fc481e00640 1 --2- 172.21.15.7:0/1251129691 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804513310 0x56380452c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:44:40.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.504+0000 7fc481400640 1 --2- 172.21.15.7:0/1251129691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804553b40 0x56380450b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:44:40.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.505+0000 7fc482800640 1 --2- 172.21.15.7:0/1251129691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804557c20 0x56380450ba00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009d90 tx=0x7fc47c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6adf801344e0d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.505+0000 7fc46dc00640 1 -- 172.21.15.7:0/1251129691 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c02e860 con 0x563804557c20 2024-09-10T21:44:40.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.505+0000 7fc46dc00640 1 -- 172.21.15.7:0/1251129691 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc47c04d070 con 0x563804557c20 2024-09-10T21:44:40.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.505+0000 7fc46dc00640 1 -- 172.21.15.7:0/1251129691 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47c02edb0 con 0x563804557c20 2024-09-10T21:44:40.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.505+0000 7fc48b800740 1 -- 172.21.15.7:0/1251129691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804557c20 msgr2=0x56380450ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.505+0000 7fc48b800740 1 --2- 172.21.15.7:0/1251129691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804557c20 0x56380450ba00 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc47c009d90 tx=0x7fc47c004a40 comp rx=0 tx=0).stop 2024-09-10T21:44:40.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.505+0000 7fc48b800740 1 -- 172.21.15.7:0/1251129691 shutdown_connections 2024-09-10T21:44:40.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.505+0000 7fc48b800740 1 --2- 172.21.15.7:0/1251129691 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804553b40 0x56380450b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.505+0000 7fc48b800740 1 --2- 172.21.15.7:0/1251129691 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804557c20 0x56380450ba00 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.505+0000 7fc48b800740 1 --2- 172.21.15.7:0/1251129691 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804513310 0x56380452c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.505+0000 7fc48b800740 1 -- 172.21.15.7:0/1251129691 >> 172.21.15.7:0/1251129691 conn(0x563804512d90 msgr2=0x563804538c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:40.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.506+0000 7fc48b800740 1 -- 172.21.15.7:0/1251129691 shutdown_connections 2024-09-10T21:44:40.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.506+0000 7fc48b800740 1 -- 172.21.15.7:0/1251129691 wait complete. 2024-09-10T21:44:40.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.506+0000 7fc48b800740 1 -- start start 2024-09-10T21:44:40.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.506+0000 7fc48b800740 1 Processor -- start 2024-09-10T21:44:40.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.507+0000 7fc48b800740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804513310 0x5638045500d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.507+0000 7fc48b800740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804553b40 0x5638045d7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.507+0000 7fc48b800740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804557c20 0x56380450cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.507+0000 7fc48b800740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5638045128c0 con 0x563804513310 2024-09-10T21:44:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.507+0000 7fc48b800740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc4841f5150 con 0x563804553b40 2024-09-10T21:44:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.507+0000 7fc48b800740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5638045d9350 con 0x563804557c20 2024-09-10T21:44:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.507+0000 7fc48b800740 10 client.0 _ll_register_callbacks cb 0x7fc47000da50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:44:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.508+0000 7fc481e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804513310 0x5638045500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.508+0000 7fc481e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804513310 0x5638045500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43712/0 (socket says 172.21.15.7:43712) 2024-09-10T21:44:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.508+0000 7fc481e00640 1 -- 172.21.15.7:0/1497553857 learned_addr learned my addr 172.21.15.7:0/1497553857 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.508+0000 7fc481400640 1 --2- 172.21.15.7:0/1497553857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804553b40 0x5638045d7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.508+0000 7fc482800640 1 --2- 172.21.15.7:0/1497553857 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804557c20 0x56380450cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.508+0000 7fc481e00640 1 -- 172.21.15.7:0/1497553857 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804557c20 msgr2=0x56380450cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.508+0000 7fc481e00640 1 --2- 172.21.15.7:0/1497553857 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804557c20 0x56380450cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.508+0000 7fc481e00640 1 -- 172.21.15.7:0/1497553857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804553b40 msgr2=0x5638045d7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.508+0000 7fc481e00640 1 --2- 172.21.15.7:0/1497553857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804553b40 0x5638045d7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.508+0000 7fc481e00640 1 -- 172.21.15.7:0/1497553857 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563804565e80 con 0x563804513310 2024-09-10T21:44:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.508+0000 7fc482800640 1 --2- 172.21.15.7:0/1497553857 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804557c20 0x56380450cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.508+0000 7fc481e00640 1 --2- 172.21.15.7:0/1497553857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804513310 0x5638045500d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc474098d80 tx=0x7fc47405ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=315bd8c44479ec3b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.508+0000 7fc462a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:44:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.509+0000 7fc462a00640 1 -- 172.21.15.7:0/1497553857 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc47406c150 con 0x563804513310 2024-09-10T21:44:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.509+0000 7fc462a00640 1 -- 172.21.15.7:0/1497553857 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fc474061eb0 con 0x563804513310 2024-09-10T21:44:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.509+0000 7fc48b800740 1 -- 172.21.15.7:0/1497553857 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc47c04d070 con 0x563804513310 2024-09-10T21:44:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.509+0000 7fc462a00640 1 -- 172.21.15.7:0/1497553857 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fc474098050 con 0x563804513310 2024-09-10T21:44:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.509+0000 7fc48b800740 10 client.5865 fetch_fsmap learned FSMap version 35 2024-09-10T21:44:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.509+0000 7fc48b800740 1 -- 172.21.15.7:0/1497553857 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x563804565650 con 0x563804513310 2024-09-10T21:44:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.509+0000 7fc462a00640 1 -- 172.21.15.7:0/1497553857 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc47c04d070 con 0x563804513310 2024-09-10T21:44:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.509+0000 7fc462a00640 1 -- 172.21.15.7:0/1497553857 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7fc47405e5a0 con 0x563804513310 2024-09-10T21:44:40.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.509+0000 7fc48b800740 10 client.5865 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T21:44:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.509+0000 7fc48b800740 10 client.5865 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-10T21:44:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.509+0000 7fc48b800740 20 client.5865 populate_metadata read hostname 'smithi007' 2024-09-10T21:44:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.509+0000 7fc48b800740 1 --2- 172.21.15.7:0/1497553857 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x56380450d310 0x563804525780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.509+0000 7fc48b800740 4 client.5865 mds_command: new command op to 235016 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2024-09-10T21:44:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.509+0000 7fc48b800740 1 -- 172.21.15.7:0/1497553857 --> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) -- 0x563804482180 con 0x56380450d310 2024-09-10T21:44:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.509+0000 7fc481400640 1 --2- 172.21.15.7:0/1497553857 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x56380450d310 0x563804525780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.510+0000 7fc481400640 1 --2- 172.21.15.7:0/1497553857 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x56380450d310 0x563804525780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cec3877cd2dd85b6 server_cookie=e97a35ea86ee56e9 in_seq=0 out_seq=0 2024-09-10T21:44:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.510+0000 7fc462a00640 10 client.5865 ms_handle_connect on v2:172.21.15.7:6833/364473218 2024-09-10T21:44:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.511+0000 7fc462a00640 1 -- 172.21.15.7:0/1497553857 <== mds.0 v2:172.21.15.7:6833/364473218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563804482180 con 0x56380450d310 2024-09-10T21:44:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.511+0000 7fc462a00640 10 client.5865 handle_command_reply: tid=0 2024-09-10T21:44:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.511+0000 7fc48b800740 1 client.5865 shutdown 2024-09-10T21:44:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.511+0000 7fc48b800740 1 -- 172.21.15.7:0/1497553857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804513310 msgr2=0x5638045500d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.511+0000 7fc48b800740 1 --2- 172.21.15.7:0/1497553857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804513310 0x5638045500d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc474098d80 tx=0x7fc47405ec20 comp rx=0 tx=0).stop 2024-09-10T21:44:40.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.511+0000 7fc48b800740 1 -- 172.21.15.7:0/1497553857 shutdown_connections 2024-09-10T21:44:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.511+0000 7fc48b800740 1 --2- 172.21.15.7:0/1497553857 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x56380450d310 0x563804525780 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.511+0000 7fc48b800740 1 --2- 172.21.15.7:0/1497553857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563804553b40 0x5638045d7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.511+0000 7fc48b800740 1 --2- 172.21.15.7:0/1497553857 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563804557c20 0x56380450cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.511+0000 7fc48b800740 1 --2- 172.21.15.7:0/1497553857 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563804513310 0x5638045500d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.511+0000 7fc462a00640 0 client.5865 ms_handle_reset on v2:172.21.15.7:6833/364473218 2024-09-10T21:44:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.511+0000 7fc48b800740 1 -- 172.21.15.7:0/1497553857 >> 172.21.15.7:0/1497553857 conn(0x563804512d90 msgr2=0x56380457aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.512+0000 7fc48b800740 1 -- 172.21.15.7:0/1497553857 shutdown_connections 2024-09-10T21:44:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.512+0000 7fc48b800740 1 -- 172.21.15.7:0/1497553857 wait complete. 2024-09-10T21:44:40.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.512+0000 7fc48b800740 20 client.5865 trim_cache size 0 max 16384 2024-09-10T21:44:40.518 INFO:tasks.ceph.mds.a.smithi007.stderr:2024-09-10T21:44:40.512+0000 7fe3f0e00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 pv4 ap=4 s=0 n(v0 1=1+0)->n(v0 rc2024-09-10T21:44:39.176338+0000 1=1+0) (iauth excl) (ifile lock->excl w=1) (ixattr excl) (iquiesce lock w=1) cr={5596=0-4194304@1} caps={5596=pAsxLsXsxFcb/pAsxXsxFxwb@3},l=5596 | ptrwaiter=1 request=1 lock=3 caps=1 dirtyparent=0 dirty=0 waiter=1 authpin=1 scrubqueue=0 0x5608cb1c6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-10T21:44:40.519 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:44:40.519 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-09-10T21:44:40.519 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "cabd7ddd-fca9-48c5-a921-5e626bb8f21c", 2024-09-10T21:44:40.519 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-09-10T21:44:40.519 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:44:40.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.513+0000 7fc483200640 1 -- 172.21.15.7:0/726596172 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc464077f70 msgr2=0x7fc46407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.513+0000 7fc483200640 1 --2- 172.21.15.7:0/726596172 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc464077f70 0x7fc46407a430 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc470007fd0 tx=0x7fc470009040 comp rx=0 tx=0).stop 2024-09-10T21:44:40.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.513+0000 7fc483200640 1 -- 172.21.15.7:0/726596172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4840cf0d0 msgr2=0x7fc48414d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.513+0000 7fc483200640 1 --2- 172.21.15.7:0/726596172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4840cf0d0 0x7fc48414d660 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc47400ea20 tx=0x7fc47400eee0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.513+0000 7fc483200640 1 -- 172.21.15.7:0/726596172 shutdown_connections 2024-09-10T21:44:40.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.513+0000 7fc483200640 1 --2- 172.21.15.7:0/726596172 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc464077f70 0x7fc46407a430 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.513+0000 7fc483200640 1 --2- 172.21.15.7:0/726596172 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc48415dbf0 0x7fc4840cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.513+0000 7fc483200640 1 --2- 172.21.15.7:0/726596172 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc48415dfd0 0x7fc4840b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.513+0000 7fc483200640 1 --2- 172.21.15.7:0/726596172 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc4840cf0d0 0x7fc48414d660 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.513+0000 7fc483200640 1 -- 172.21.15.7:0/726596172 >> 172.21.15.7:0/726596172 conn(0x7fc4840d3d70 msgr2=0x7fc4840b73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:40.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.513+0000 7fc483200640 1 -- 172.21.15.7:0/726596172 shutdown_connections 2024-09-10T21:44:40.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.513+0000 7fc483200640 1 -- 172.21.15.7:0/726596172 wait complete. 2024-09-10T21:44:40.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:44:40.529 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-09-10T21:44:40.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.690+0000 7f8eb6400640 1 -- 172.21.15.7:0/1900228500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8eb0074450 msgr2=0x7f8eb00ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.690+0000 7f8eb6400640 1 --2- 172.21.15.7:0/1900228500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8eb0074450 0x7f8eb00ca270 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8eb00708e0 tx=0x7f8ea802aa30 comp rx=0 tx=0).stop 2024-09-10T21:44:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.691+0000 7f8eb6400640 1 -- 172.21.15.7:0/1900228500 shutdown_connections 2024-09-10T21:44:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.691+0000 7f8eb6400640 1 --2- 172.21.15.7:0/1900228500 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8eb0074450 0x7f8eb00ca270 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.691+0000 7f8eb6400640 1 --2- 172.21.15.7:0/1900228500 >> v2:172.21.15.133:3300/0 conn(0x7f8eb0155fb0 0x7f8eb01600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.691+0000 7f8eb6400640 1 -- 172.21.15.7:0/1900228500 >> 172.21.15.7:0/1900228500 conn(0x7f8eb00b16d0 msgr2=0x7f8eb00b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.691+0000 7f8eb6400640 1 -- 172.21.15.7:0/1900228500 shutdown_connections 2024-09-10T21:44:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.691+0000 7f8eb6400640 1 -- 172.21.15.7:0/1900228500 wait complete. 2024-09-10T21:44:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.692+0000 7f8eb6400640 1 Processor -- start 2024-09-10T21:44:40.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.692+0000 7f8eb6400640 1 -- start start 2024-09-10T21:44:40.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.693+0000 7f8eb6400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8eb006fcf0 0x7f8eb0154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.693+0000 7f8eb6400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8eb0074450 0x7f8eb0153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.693+0000 7f8eb6400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb0155fb0 0x7f8eb00b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.693+0000 7f8eb6400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb00ca930 con 0x7f8eb006fcf0 2024-09-10T21:44:40.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.693+0000 7f8eb6400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb00ca7b0 con 0x7f8eb0155fb0 2024-09-10T21:44:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.693+0000 7f8eb6400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb00caab0 con 0x7f8eb0074450 2024-09-10T21:44:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.693+0000 7f8eaf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8eb006fcf0 0x7f8eb0154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.694+0000 7f8eaf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8eb006fcf0 0x7f8eb0154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43716/0 (socket says 172.21.15.7:43716) 2024-09-10T21:44:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.694+0000 7f8eaf400640 1 -- 172.21.15.7:0/3548941924 learned_addr learned my addr 172.21.15.7:0/3548941924 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.694+0000 7f8eafe00640 1 --2- 172.21.15.7:0/3548941924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb0155fb0 0x7f8eb00b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.694+0000 7f8eaea00640 1 --2- 172.21.15.7:0/3548941924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8eb0074450 0x7f8eb0153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.694+0000 7f8eaf400640 1 -- 172.21.15.7:0/3548941924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8eb0074450 msgr2=0x7f8eb0153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.694+0000 7f8eaf400640 1 --2- 172.21.15.7:0/3548941924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8eb0074450 0x7f8eb0153760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.694+0000 7f8eaf400640 1 -- 172.21.15.7:0/3548941924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb0155fb0 msgr2=0x7f8eb00b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.694+0000 7f8eaf400640 1 --2- 172.21.15.7:0/3548941924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb0155fb0 0x7f8eb00b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.694+0000 7f8eaf400640 1 -- 172.21.15.7:0/3548941924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8eb00b3ff0 con 0x7f8eb006fcf0 2024-09-10T21:44:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.694+0000 7f8eaea00640 1 --2- 172.21.15.7:0/3548941924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8eb0074450 0x7f8eb0153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:40.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.694+0000 7f8eaf400640 1 --2- 172.21.15.7:0/3548941924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8eb006fcf0 0x7f8eb0154310 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000d740 tx=0x7f8ea000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.695+0000 7f8ea7400640 1 -- 172.21.15.7:0/3548941924 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea0017070 con 0x7f8eb006fcf0 2024-09-10T21:44:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.695+0000 7f8ea7400640 1 -- 172.21.15.7:0/3548941924 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ea0009cc0 con 0x7f8eb006fcf0 2024-09-10T21:44:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.695+0000 7f8ea7400640 1 -- 172.21.15.7:0/3548941924 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea00106c0 con 0x7f8eb006fcf0 2024-09-10T21:44:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.695+0000 7f8eb6400640 1 -- 172.21.15.7:0/3548941924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8eb00b42d0 con 0x7f8eb006fcf0 2024-09-10T21:44:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.695+0000 7f8eb6400640 1 -- 172.21.15.7:0/3548941924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8eb00b4890 con 0x7f8eb006fcf0 2024-09-10T21:44:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.697+0000 7f8ea7400640 1 -- 172.21.15.7:0/3548941924 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8ea0010860 con 0x7f8eb006fcf0 2024-09-10T21:44:40.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.698+0000 7f8ea7400640 1 --2- 172.21.15.7:0/3548941924 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8e90077f70 0x7f8e9007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.698+0000 7f8eb9234740 1 Processor -- start 2024-09-10T21:44:40.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.698+0000 7f8ea7400640 1 -- 172.21.15.7:0/3548941924 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8ea009a160 con 0x7f8eb006fcf0 2024-09-10T21:44:40.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.698+0000 7f8eb9234740 1 -- start start 2024-09-10T21:44:40.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.699+0000 7f8eaea00640 1 --2- 172.21.15.7:0/3548941924 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8e90077f70 0x7f8e9007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.699+0000 7f8eb9234740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557701729130 0x557701729510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.699+0000 7f8eb9234740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577016e42a0 0x557701727c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.699+0000 7f8eaea00640 1 --2- 172.21.15.7:0/3548941924 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8e90077f70 0x7f8e9007a430 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c007fd0 tx=0x7f8e9c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.699+0000 7f8eb9234740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577016e4820 0x557701728150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.699+0000 7f8eb9234740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5577016431c0 con 0x5577016e42a0 2024-09-10T21:44:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.699+0000 7f8eb9234740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557701656870 con 0x5577016e4820 2024-09-10T21:44:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.699+0000 7f8eb9234740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x557701645ca0 con 0x557701729130 2024-09-10T21:44:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.699+0000 7f8eaea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577016e42a0 0x557701727c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.700+0000 7f8eaea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577016e42a0 0x557701727c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43726/0 (socket says 172.21.15.7:43726) 2024-09-10T21:44:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.700+0000 7f8eaea00640 1 -- 172.21.15.7:0/258623092 learned_addr learned my addr 172.21.15.7:0/258623092 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.700+0000 7f8eaf400640 1 --2- 172.21.15.7:0/258623092 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557701729130 0x557701729510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.700+0000 7f8eafe00640 1 --2- 172.21.15.7:0/258623092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577016e4820 0x557701728150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.700+0000 7f8eaea00640 1 -- 172.21.15.7:0/258623092 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557701729130 msgr2=0x557701729510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.700+0000 7f8eaea00640 1 --2- 172.21.15.7:0/258623092 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557701729130 0x557701729510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.700+0000 7f8eaea00640 1 -- 172.21.15.7:0/258623092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577016e4820 msgr2=0x557701728150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.700+0000 7f8eaea00640 1 --2- 172.21.15.7:0/258623092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577016e4820 0x557701728150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.700+0000 7f8eaea00640 1 -- 172.21.15.7:0/258623092 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55770141e7d0 con 0x5577016e42a0 2024-09-10T21:44:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.700+0000 7f8eaea00640 1 --2- 172.21.15.7:0/258623092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577016e42a0 0x557701727c10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c005120 tx=0x7f8e9c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c95df1f748209c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.700+0000 7f8e9be00640 1 -- 172.21.15.7:0/258623092 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c0035f0 con 0x5577016e42a0 2024-09-10T21:44:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.700+0000 7f8e9be00640 1 -- 172.21.15.7:0/258623092 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e9c003880 con 0x5577016e42a0 2024-09-10T21:44:40.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.700+0000 7f8e9be00640 1 -- 172.21.15.7:0/258623092 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c003cb0 con 0x5577016e42a0 2024-09-10T21:44:40.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.701+0000 7f8eb9234740 1 -- 172.21.15.7:0/258623092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577016e42a0 msgr2=0x557701727c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.701+0000 7f8eb9234740 1 --2- 172.21.15.7:0/258623092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577016e42a0 0x557701727c10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c005120 tx=0x7f8e9c011000 comp rx=0 tx=0).stop 2024-09-10T21:44:40.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.701+0000 7f8eb9234740 1 -- 172.21.15.7:0/258623092 shutdown_connections 2024-09-10T21:44:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.701+0000 7f8eb9234740 1 --2- 172.21.15.7:0/258623092 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577016e4820 0x557701728150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.701+0000 7f8eb9234740 1 --2- 172.21.15.7:0/258623092 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577016e42a0 0x557701727c10 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.701+0000 7f8eb9234740 1 --2- 172.21.15.7:0/258623092 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x557701729130 0x557701729510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.701+0000 7f8eb9234740 1 -- 172.21.15.7:0/258623092 >> 172.21.15.7:0/258623092 conn(0x5577016df870 msgr2=0x5577016e3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.701+0000 7f8eb9234740 1 -- 172.21.15.7:0/258623092 shutdown_connections 2024-09-10T21:44:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.701+0000 7f8eb9234740 1 -- 172.21.15.7:0/258623092 wait complete. 2024-09-10T21:44:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.702+0000 7f8eb9234740 1 -- start start 2024-09-10T21:44:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.702+0000 7f8eb9234740 1 Processor -- start 2024-09-10T21:44:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.703+0000 7f8eb9234740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5577016e4820 0x557701738360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.703+0000 7f8eb9234740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557701729130 0x55770171e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.703+0000 7f8eb9234740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577017388a0 0x55770171e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.703+0000 7f8eb9234740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5577015ec410 con 0x5577017388a0 2024-09-10T21:44:40.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.703+0000 7f8eb9234740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x557701649ab0 con 0x557701729130 2024-09-10T21:44:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.703+0000 7f8eb9234740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5577013b6740 con 0x5577016e4820 2024-09-10T21:44:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.703+0000 7f8eb9234740 10 client.0 _ll_register_callbacks cb 0x5577015c9360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:44:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.703+0000 7f8eafe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577017388a0 0x55770171e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.704+0000 7f8eafe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577017388a0 0x55770171e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43732/0 (socket says 172.21.15.7:43732) 2024-09-10T21:44:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.704+0000 7f8eafe00640 1 -- 172.21.15.7:0/4251392949 learned_addr learned my addr 172.21.15.7:0/4251392949 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.704+0000 7f8eaea00640 1 --2- 172.21.15.7:0/4251392949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557701729130 0x55770171e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.704+0000 7f8eaf400640 1 --2- 172.21.15.7:0/4251392949 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5577016e4820 0x557701738360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.704+0000 7f8eafe00640 1 -- 172.21.15.7:0/4251392949 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5577016e4820 msgr2=0x557701738360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.704+0000 7f8eafe00640 1 --2- 172.21.15.7:0/4251392949 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5577016e4820 0x557701738360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.704+0000 7f8eafe00640 1 -- 172.21.15.7:0/4251392949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557701729130 msgr2=0x55770171e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.704+0000 7f8eafe00640 1 --2- 172.21.15.7:0/4251392949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557701729130 0x55770171e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.704+0000 7f8eafe00640 1 -- 172.21.15.7:0/4251392949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577013a7970 con 0x5577017388a0 2024-09-10T21:44:40.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.704+0000 7f8eaf400640 1 --2- 172.21.15.7:0/4251392949 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5577016e4820 0x557701738360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.704+0000 7f8eafe00640 1 --2- 172.21.15.7:0/4251392949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577017388a0 0x55770171e890 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009d60 tx=0x7f8ea802f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd5c95a91d633ec server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.704+0000 7f8e98c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:44:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.704+0000 7f8e98c00640 1 -- 172.21.15.7:0/4251392949 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ea8033050 con 0x5577017388a0 2024-09-10T21:44:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.704+0000 7f8e98c00640 1 -- 172.21.15.7:0/4251392949 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8ea8004240 con 0x5577017388a0 2024-09-10T21:44:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.704+0000 7f8eb9234740 1 -- 172.21.15.7:0/4251392949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e9c003880 con 0x5577017388a0 2024-09-10T21:44:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.705+0000 7f8e98c00640 1 -- 172.21.15.7:0/4251392949 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8ea8050050 con 0x5577017388a0 2024-09-10T21:44:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.705+0000 7f8eb9234740 10 client.5880 fetch_fsmap learned FSMap version 35 2024-09-10T21:44:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.705+0000 7f8eb9234740 1 -- 172.21.15.7:0/4251392949 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x5577013856a0 con 0x5577017388a0 2024-09-10T21:44:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.705+0000 7f8e98c00640 1 -- 172.21.15.7:0/4251392949 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e9c003880 con 0x5577017388a0 2024-09-10T21:44:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.705+0000 7f8e98c00640 1 -- 172.21.15.7:0/4251392949 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7f8ea8053020 con 0x5577017388a0 2024-09-10T21:44:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.705+0000 7f8eb9234740 10 client.5880 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T21:44:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.705+0000 7f8eb9234740 10 client.5880 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-10T21:44:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.705+0000 7f8eb9234740 20 client.5880 populate_metadata read hostname 'smithi007' 2024-09-10T21:44:40.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.705+0000 7f8eb9234740 1 --2- 172.21.15.7:0/4251392949 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x557701722910 0x55770170c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.705+0000 7f8eb9234740 4 client.5880 mds_command: new command op to 235016 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:44:40.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.705+0000 7f8eb9234740 1 -- 172.21.15.7:0/4251392949 --> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577016587c0 con 0x557701722910 2024-09-10T21:44:40.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.705+0000 7f8eaf400640 1 --2- 172.21.15.7:0/4251392949 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x557701722910 0x55770170c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.706+0000 7f8e98c00640 10 client.5880 ms_handle_connect on v2:172.21.15.7:6833/364473218 2024-09-10T21:44:40.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.706+0000 7f8eaf400640 1 --2- 172.21.15.7:0/4251392949 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x557701722910 0x55770170c510 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=712ce643b0d0d658 server_cookie=c3cfa8e05d74ca17 in_seq=0 out_seq=0 2024-09-10T21:44:40.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.707+0000 7f8e98c00640 1 -- 172.21.15.7:0/4251392949 <== mds.0 v2:172.21.15.7:6833/364473218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5577016587c0 con 0x557701722910 2024-09-10T21:44:40.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.707+0000 7f8e98c00640 10 client.5880 handle_command_reply: tid=0 2024-09-10T21:44:40.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.707+0000 7f8eb9234740 1 client.5880 shutdown 2024-09-10T21:44:40.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.707+0000 7f8eb9234740 1 -- 172.21.15.7:0/4251392949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577017388a0 msgr2=0x55770171e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.707+0000 7f8eb9234740 1 --2- 172.21.15.7:0/4251392949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577017388a0 0x55770171e890 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8009d60 tx=0x7f8ea802f8c0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.707+0000 7f8eb9234740 1 -- 172.21.15.7:0/4251392949 shutdown_connections 2024-09-10T21:44:40.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.707+0000 7f8eb9234740 1 --2- 172.21.15.7:0/4251392949 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x557701722910 0x55770170c510 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.707+0000 7f8eb9234740 1 --2- 172.21.15.7:0/4251392949 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x557701729130 0x55770171e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.707+0000 7f8eb9234740 1 --2- 172.21.15.7:0/4251392949 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5577017388a0 0x55770171e890 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.707+0000 7f8eb9234740 1 --2- 172.21.15.7:0/4251392949 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5577016e4820 0x557701738360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.707+0000 7f8e98c00640 0 client.5880 ms_handle_reset on v2:172.21.15.7:6833/364473218 2024-09-10T21:44:40.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.707+0000 7f8eb9234740 1 -- 172.21.15.7:0/4251392949 >> 172.21.15.7:0/4251392949 conn(0x5577016e42a0 msgr2=0x5577016dcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:40.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.707+0000 7f8eb9234740 1 -- 172.21.15.7:0/4251392949 shutdown_connections 2024-09-10T21:44:40.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.707+0000 7f8eb9234740 1 -- 172.21.15.7:0/4251392949 wait complete. 2024-09-10T21:44:40.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.707+0000 7f8eb9234740 20 client.5880 trim_cache size 0 max 16384 2024-09-10T21:44:40.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.725+0000 7f8eb9234740 1 Processor -- start 2024-09-10T21:44:40.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.726+0000 7f8eb9234740 1 -- start start 2024-09-10T21:44:40.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.726+0000 7f8eb9234740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577016e4820 0x557701705850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.726+0000 7f8eb9234740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557701729130 0x5577017051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.726+0000 7f8eb9234740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5577017388a0 0x557701737d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.726+0000 7f8eb9234740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb02010a0 con 0x557701729130 2024-09-10T21:44:40.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.726+0000 7f8eb9234740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb02000e0 con 0x5577016e4820 2024-09-10T21:44:40.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.726+0000 7f8eb9234740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f8eb00b3790 con 0x5577017388a0 2024-09-10T21:44:40.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.727+0000 7f8eaea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557701729130 0x5577017051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.727+0000 7f8eaea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557701729130 0x5577017051d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43736/0 (socket says 172.21.15.7:43736) 2024-09-10T21:44:40.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.727+0000 7f8eaea00640 1 -- 172.21.15.7:0/2459269531 learned_addr learned my addr 172.21.15.7:0/2459269531 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:40.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.727+0000 7f8eaf400640 1 --2- 172.21.15.7:0/2459269531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577016e4820 0x557701705850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.727+0000 7f8eafe00640 1 --2- 172.21.15.7:0/2459269531 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5577017388a0 0x557701737d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.727+0000 7f8eaea00640 1 -- 172.21.15.7:0/2459269531 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5577017388a0 msgr2=0x557701737d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.727+0000 7f8eaea00640 1 --2- 172.21.15.7:0/2459269531 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5577017388a0 0x557701737d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.727+0000 7f8eaea00640 1 -- 172.21.15.7:0/2459269531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577016e4820 msgr2=0x557701705850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.727+0000 7f8eaea00640 1 --2- 172.21.15.7:0/2459269531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577016e4820 0x557701705850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.727+0000 7f8eaea00640 1 -- 172.21.15.7:0/2459269531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557701728b40 con 0x557701729130 2024-09-10T21:44:40.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.727+0000 7f8eaea00640 1 --2- 172.21.15.7:0/2459269531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557701729130 0x5577017051d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c010fd0 tx=0x7f8e9c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a580fd9ad938ce1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.728+0000 7f8e9b400640 1 -- 172.21.15.7:0/2459269531 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c003a80 con 0x557701729130 2024-09-10T21:44:40.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.728+0000 7f8e9b400640 1 -- 172.21.15.7:0/2459269531 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e9c003c20 con 0x557701729130 2024-09-10T21:44:40.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.728+0000 7f8e9b400640 1 -- 172.21.15.7:0/2459269531 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c021450 con 0x557701729130 2024-09-10T21:44:40.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.728+0000 7f8eb9234740 1 -- 172.21.15.7:0/2459269531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557701729130 msgr2=0x5577017051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.728+0000 7f8eb9234740 1 --2- 172.21.15.7:0/2459269531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557701729130 0x5577017051d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c010fd0 tx=0x7f8e9c000bc0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.728+0000 7f8eb9234740 1 -- 172.21.15.7:0/2459269531 shutdown_connections 2024-09-10T21:44:40.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.728+0000 7f8eb9234740 1 --2- 172.21.15.7:0/2459269531 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5577017388a0 0x557701737d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.728+0000 7f8eb9234740 1 --2- 172.21.15.7:0/2459269531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557701729130 0x5577017051d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.728+0000 7f8eb9234740 1 --2- 172.21.15.7:0/2459269531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577016e4820 0x557701705850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.728+0000 7f8eb9234740 1 -- 172.21.15.7:0/2459269531 >> 172.21.15.7:0/2459269531 conn(0x5577016e42a0 msgr2=0x557701728330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.728+0000 7f8eb9234740 1 -- 172.21.15.7:0/2459269531 shutdown_connections 2024-09-10T21:44:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.728+0000 7f8eb9234740 1 -- 172.21.15.7:0/2459269531 wait complete. 2024-09-10T21:44:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.729+0000 7f8eb9234740 1 -- start start 2024-09-10T21:44:40.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.729+0000 7f8eb9234740 1 Processor -- start 2024-09-10T21:44:40.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.730+0000 7f8eb9234740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5577016e4820 0x55770174bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.730+0000 7f8eb9234740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557701729130 0x5577017a8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.730+0000 7f8eb9234740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577017388a0 0x5577017ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.730+0000 7f8eb9234740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8eb01f4de0 con 0x557701729130 2024-09-10T21:44:40.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.730+0000 7f8eb9234740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55770160cee0 con 0x5577017388a0 2024-09-10T21:44:40.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.730+0000 7f8eb9234740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55770174c470 con 0x5577016e4820 2024-09-10T21:44:40.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.730+0000 7f8eb9234740 10 client.0 _ll_register_callbacks cb 0x7f8ea8003de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:44:40.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.730+0000 7f8eaea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557701729130 0x5577017a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.730+0000 7f8eaea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557701729130 0x5577017a8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43752/0 (socket says 172.21.15.7:43752) 2024-09-10T21:44:40.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.730+0000 7f8eaea00640 1 -- 172.21.15.7:0/196027668 learned_addr learned my addr 172.21.15.7:0/196027668 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:44:40.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.731+0000 7f8eaf400640 1 --2- 172.21.15.7:0/196027668 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5577016e4820 0x55770174bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.731+0000 7f8eafe00640 1 --2- 172.21.15.7:0/196027668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577017388a0 0x5577017ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.731+0000 7f8eaea00640 1 -- 172.21.15.7:0/196027668 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5577016e4820 msgr2=0x55770174bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.731+0000 7f8eaea00640 1 --2- 172.21.15.7:0/196027668 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5577016e4820 0x55770174bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.731+0000 7f8eaea00640 1 -- 172.21.15.7:0/196027668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577017388a0 msgr2=0x5577017ac9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:44:40.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.731+0000 7f8eaea00640 1 --2- 172.21.15.7:0/196027668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577017388a0 0x5577017ac9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.731+0000 7f8eaea00640 1 -- 172.21.15.7:0/196027668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55770171e140 con 0x557701729130 2024-09-10T21:44:40.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.731+0000 7f8eafe00640 1 --2- 172.21.15.7:0/196027668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577017388a0 0x5577017ac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.731+0000 7f8eaea00640 1 --2- 172.21.15.7:0/196027668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557701729130 0x5577017a8750 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c001570 tx=0x7f8e9c003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b8b740032fad2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.731+0000 7f8e98c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:44:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.731+0000 7f8e98c00640 1 -- 172.21.15.7:0/196027668 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c003a70 con 0x557701729130 2024-09-10T21:44:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.731+0000 7f8e98c00640 1 -- 172.21.15.7:0/196027668 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f8e9c021660 con 0x557701729130 2024-09-10T21:44:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.731+0000 7f8eb9234740 1 -- 172.21.15.7:0/196027668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e9c003c20 con 0x557701729130 2024-09-10T21:44:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.732+0000 7f8e98c00640 1 -- 172.21.15.7:0/196027668 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f8e9c03d050 con 0x557701729130 2024-09-10T21:44:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.732+0000 7f8eb9234740 10 client.5886 fetch_fsmap learned FSMap version 35 2024-09-10T21:44:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.732+0000 7f8eb9234740 1 -- 172.21.15.7:0/196027668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=2+}) -- 0x55770174d710 con 0x557701729130 2024-09-10T21:44:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.732+0000 7f8e98c00640 1 -- 172.21.15.7:0/196027668 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e9c003c20 con 0x557701729130 2024-09-10T21:44:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.732+0000 7f8e98c00640 1 -- 172.21.15.7:0/196027668 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 35) ==== 3502+0+0 (secure 0 0 0) 0x7f8e9c040020 con 0x557701729130 2024-09-10T21:44:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.732+0000 7f8eb9234740 10 client.5886 fetch_fsmap finished waiting for FSMap version 35 2024-09-10T21:44:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.732+0000 7f8eb9234740 10 client.5886 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-10T21:44:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.732+0000 7f8eb9234740 20 client.5886 populate_metadata read hostname 'smithi007' 2024-09-10T21:44:40.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.732+0000 7f8eb9234740 1 --2- 172.21.15.7:0/196027668 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x557701721c20 0x5577017acf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.732+0000 7f8eb9234740 4 client.5886 mds_command: new command op to 235016 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:44:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.732+0000 7f8eb9234740 1 -- 172.21.15.7:0/196027668 --> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557701654270 con 0x557701721c20 2024-09-10T21:44:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.732+0000 7f8eaf400640 1 --2- 172.21.15.7:0/196027668 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x557701721c20 0x5577017acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.733+0000 7f8eaf400640 1 --2- 172.21.15.7:0/196027668 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x557701721c20 0x5577017acf10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce3853f96f4456a4 server_cookie=3156d73bb33e5f9b in_seq=0 out_seq=0 2024-09-10T21:44:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.733+0000 7f8e98c00640 10 client.5886 ms_handle_connect on v2:172.21.15.7:6833/364473218 2024-09-10T21:44:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.733+0000 7f8e98c00640 1 -- 172.21.15.7:0/196027668 <== mds.0 v2:172.21.15.7:6833/364473218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+564 (crc 0 0 0) 0x557701654270 con 0x557701721c20 2024-09-10T21:44:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.733+0000 7f8e98c00640 10 client.5886 handle_command_reply: tid=0 2024-09-10T21:44:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.733+0000 7f8eb9234740 1 client.5886 shutdown 2024-09-10T21:44:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.733+0000 7f8eb9234740 1 -- 172.21.15.7:0/196027668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557701729130 msgr2=0x5577017a8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.733+0000 7f8eb9234740 1 --2- 172.21.15.7:0/196027668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557701729130 0x5577017a8750 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c001570 tx=0x7f8e9c003880 comp rx=0 tx=0).stop 2024-09-10T21:44:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.733+0000 7f8eb9234740 1 -- 172.21.15.7:0/196027668 shutdown_connections 2024-09-10T21:44:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.733+0000 7f8eb9234740 1 --2- 172.21.15.7:0/196027668 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x557701721c20 0x5577017acf10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.733+0000 7f8eb9234740 1 --2- 172.21.15.7:0/196027668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5577017388a0 0x5577017ac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.733+0000 7f8eb9234740 1 --2- 172.21.15.7:0/196027668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x557701729130 0x5577017a8750 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.733+0000 7f8eb9234740 1 --2- 172.21.15.7:0/196027668 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5577016e4820 0x55770174bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.733+0000 7f8eb9234740 1 -- 172.21.15.7:0/196027668 >> 172.21.15.7:0/196027668 conn(0x5577016e42a0 msgr2=0x55770170d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.733+0000 7f8e98c00640 0 client.5886 ms_handle_reset on v2:172.21.15.7:6833/364473218 2024-09-10T21:44:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.733+0000 7f8eb9234740 1 -- 172.21.15.7:0/196027668 shutdown_connections 2024-09-10T21:44:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.733+0000 7f8eb9234740 1 -- 172.21.15.7:0/196027668 wait complete. 2024-09-10T21:44:40.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.734+0000 7f8eb9234740 20 client.5886 trim_cache size 0 max 16384 2024-09-10T21:44:40.739 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:44:40.739 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:44:40.739 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-09-10T21:44:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "cabd7ddd-fca9-48c5-a921-5e626bb8f21c": { 2024-09-10T21:44:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-09-10T21:44:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "cabd7ddd-fca9-48c5-a921-5e626bb8f21c", 2024-09-10T21:44:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive" 2024-09-10T21:44:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:44:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:44:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "cabd7ddd-fca9-48c5-a921-5e626bb8f21c": { 2024-09-10T21:44:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:44:40.510317+0000", 2024-09-10T21:44:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-09-10T21:44:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:44:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:44:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:44:40.740 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:44:40.741 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:40.741 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:44:40.741 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:44:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.734+0000 7f8eb6400640 1 -- 172.21.15.7:0/3548941924 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8e90077f70 msgr2=0x7f8e9007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.734+0000 7f8eb6400640 1 --2- 172.21.15.7:0/3548941924 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8e90077f70 0x7f8e9007a430 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c007fd0 tx=0x7f8e9c009040 comp rx=0 tx=0).stop 2024-09-10T21:44:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.734+0000 7f8eb6400640 1 -- 172.21.15.7:0/3548941924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8eb006fcf0 msgr2=0x7f8eb0154310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.734+0000 7f8eb6400640 1 --2- 172.21.15.7:0/3548941924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8eb006fcf0 0x7f8eb0154310 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f8ea000d740 tx=0x7f8ea000dc00 comp rx=0 tx=0).stop 2024-09-10T21:44:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.734+0000 7f8eb6400640 1 -- 172.21.15.7:0/3548941924 shutdown_connections 2024-09-10T21:44:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.734+0000 7f8eb6400640 1 --2- 172.21.15.7:0/3548941924 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8e90077f70 0x7f8e9007a430 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.734+0000 7f8eb6400640 1 --2- 172.21.15.7:0/3548941924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8eb0155fb0 0x7f8eb00b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.734+0000 7f8eb6400640 1 --2- 172.21.15.7:0/3548941924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8eb0074450 0x7f8eb0153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.734+0000 7f8eb6400640 1 --2- 172.21.15.7:0/3548941924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8eb006fcf0 0x7f8eb0154310 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.734+0000 7f8eb6400640 1 -- 172.21.15.7:0/3548941924 >> 172.21.15.7:0/3548941924 conn(0x7f8eb00b16d0 msgr2=0x7f8eb015b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.735+0000 7f8eb6400640 1 -- 172.21.15.7:0/3548941924 shutdown_connections 2024-09-10T21:44:40.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:44:40.735+0000 7f8eb6400640 1 -- 172.21.15.7:0/3548941924 wait complete. 2024-09-10T21:44:40.752 INFO:tasks.cephfs.filesystem:scrub status for tag:cabd7ddd-fca9-48c5-a921-5e626bb8f21c - {'path': '/', 'tag': 'cabd7ddd-fca9-48c5-a921-5e626bb8f21c', 'options': 'recursive'} 2024-09-10T21:44:43.429 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-10T21:44:43.428+0000 7feb42c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-10T21:45:00.183 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:45:00.188 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:45:10.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:45:10.753 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-09-10T21:45:10.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.926+0000 7ffbed800640 1 -- 172.21.15.7:0/902270323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe8159680 msgr2=0x7ffbe815cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:10.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.926+0000 7ffbed800640 1 --2- 172.21.15.7:0/902270323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe8159680 0x7ffbe815cc00 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009830 tx=0x7ffbd802a480 comp rx=0 tx=0).stop 2024-09-10T21:45:10.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.926+0000 7ffbed800640 1 -- 172.21.15.7:0/902270323 shutdown_connections 2024-09-10T21:45:10.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.926+0000 7ffbed800640 1 --2- 172.21.15.7:0/902270323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe8159680 0x7ffbe815cc00 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.926+0000 7ffbed800640 1 -- 172.21.15.7:0/902270323 >> 172.21.15.7:0/902270323 conn(0x7ffbe80b16d0 msgr2=0x7ffbe80b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:10.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.926+0000 7ffbed800640 1 -- 172.21.15.7:0/902270323 shutdown_connections 2024-09-10T21:45:10.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.926+0000 7ffbed800640 1 -- 172.21.15.7:0/902270323 wait complete. 2024-09-10T21:45:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.928+0000 7ffbed800640 1 Processor -- start 2024-09-10T21:45:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.928+0000 7ffbed800640 1 -- start start 2024-09-10T21:45:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.929+0000 7ffbed800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbe806e580 0x7ffbe814c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.929+0000 7ffbed800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe8073e20 0x7ffbe815e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.929+0000 7ffbed800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbe8159680 0x7ffbe815ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.929+0000 7ffbed800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe80cb0f0 con 0x7ffbe8073e20 2024-09-10T21:45:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.929+0000 7ffbed800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe80caf70 con 0x7ffbe806e580 2024-09-10T21:45:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.929+0000 7ffbed800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ffbe80742d0 con 0x7ffbe8159680 2024-09-10T21:45:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.930+0000 7ffbde000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe8073e20 0x7ffbe815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.930+0000 7ffbde000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe8073e20 0x7ffbe815e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42012/0 (socket says 172.21.15.7:42012) 2024-09-10T21:45:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.930+0000 7ffbde000640 1 -- 172.21.15.7:0/632568188 learned_addr learned my addr 172.21.15.7:0/632568188 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.930+0000 7ffbe6a00640 1 --2- 172.21.15.7:0/632568188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbe806e580 0x7ffbe814c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.930+0000 7ffbe7400640 1 --2- 172.21.15.7:0/632568188 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbe8159680 0x7ffbe815ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.930+0000 7ffbde000640 1 -- 172.21.15.7:0/632568188 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbe8159680 msgr2=0x7ffbe815ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.930+0000 7ffbde000640 1 --2- 172.21.15.7:0/632568188 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbe8159680 0x7ffbe815ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.930+0000 7ffbde000640 1 -- 172.21.15.7:0/632568188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbe806e580 msgr2=0x7ffbe814c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.930+0000 7ffbde000640 1 --2- 172.21.15.7:0/632568188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbe806e580 0x7ffbe814c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.930+0000 7ffbde000640 1 -- 172.21.15.7:0/632568188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbe81f91b0 con 0x7ffbe8073e20 2024-09-10T21:45:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.930+0000 7ffbe7400640 1 --2- 172.21.15.7:0/632568188 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbe8159680 0x7ffbe815ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:45:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.930+0000 7ffbde000640 1 --2- 172.21.15.7:0/632568188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe8073e20 0x7ffbe815e530 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000c420 tx=0x7ffbd000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.931+0000 7ffbdf400640 1 -- 172.21.15.7:0/632568188 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0011790 con 0x7ffbe8073e20 2024-09-10T21:45:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.931+0000 7ffbdf400640 1 -- 172.21.15.7:0/632568188 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbd0017070 con 0x7ffbe8073e20 2024-09-10T21:45:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.931+0000 7ffbdf400640 1 -- 172.21.15.7:0/632568188 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0011c80 con 0x7ffbe8073e20 2024-09-10T21:45:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.931+0000 7ffbed800640 1 -- 172.21.15.7:0/632568188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbe80b50c0 con 0x7ffbe8073e20 2024-09-10T21:45:10.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.931+0000 7ffbed800640 1 -- 172.21.15.7:0/632568188 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbe80b5680 con 0x7ffbe8073e20 2024-09-10T21:45:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.933+0000 7ffbdf400640 1 -- 172.21.15.7:0/632568188 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ffbd001c460 con 0x7ffbe8073e20 2024-09-10T21:45:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.934+0000 7ffbdf400640 1 --2- 172.21.15.7:0/632568188 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ffbc4077fa0 0x7ffbc407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.934+0000 7ffbdf400640 1 -- 172.21.15.7:0/632568188 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ffbd0099ae0 con 0x7ffbe8073e20 2024-09-10T21:45:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.935+0000 7ffbf07c1740 1 Processor -- start 2024-09-10T21:45:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.935+0000 7ffbe6a00640 1 --2- 172.21.15.7:0/632568188 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ffbc4077fa0 0x7ffbc407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.935+0000 7ffbf07c1740 1 -- start start 2024-09-10T21:45:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.936+0000 7ffbf07c1740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563608997130 0x563608997510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.936+0000 7ffbe6a00640 1 --2- 172.21.15.7:0/632568188 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ffbc4077fa0 0x7ffbc407a460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8034000 tx=0x7ffbd8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.936+0000 7ffbf07c1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636089522a0 0x563608995c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.936+0000 7ffbf07c1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563608952820 0x563608996150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.936+0000 7ffbf07c1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5636088b11c0 con 0x5636089522a0 2024-09-10T21:45:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.936+0000 7ffbf07c1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5636088c4870 con 0x563608952820 2024-09-10T21:45:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.936+0000 7ffbf07c1740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5636088b3ca0 con 0x563608997130 2024-09-10T21:45:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.936+0000 7ffbde000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636089522a0 0x563608995c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.936+0000 7ffbde000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636089522a0 0x563608995c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42028/0 (socket says 172.21.15.7:42028) 2024-09-10T21:45:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.936+0000 7ffbde000640 1 -- 172.21.15.7:0/641132017 learned_addr learned my addr 172.21.15.7:0/641132017 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.936+0000 7ffbe6a00640 1 --2- 172.21.15.7:0/641132017 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563608997130 0x563608997510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.936+0000 7ffbe7400640 1 --2- 172.21.15.7:0/641132017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563608952820 0x563608996150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.936+0000 7ffbde000640 1 -- 172.21.15.7:0/641132017 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563608997130 msgr2=0x563608997510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.936+0000 7ffbde000640 1 --2- 172.21.15.7:0/641132017 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563608997130 0x563608997510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.936+0000 7ffbde000640 1 -- 172.21.15.7:0/641132017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563608952820 msgr2=0x563608996150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.936+0000 7ffbde000640 1 --2- 172.21.15.7:0/641132017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563608952820 0x563608996150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.936+0000 7ffbde000640 1 -- 172.21.15.7:0/641132017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56360868c7d0 con 0x5636089522a0 2024-09-10T21:45:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.937+0000 7ffbde000640 1 --2- 172.21.15.7:0/641132017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636089522a0 0x563608995c10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ffbd005f140 tx=0x7ffbd00619e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56328721c5c2d4d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.937+0000 7ffbd7400640 1 -- 172.21.15.7:0/641132017 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0061ed0 con 0x5636089522a0 2024-09-10T21:45:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.937+0000 7ffbd7400640 1 -- 172.21.15.7:0/641132017 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbd006b3d0 con 0x5636089522a0 2024-09-10T21:45:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.937+0000 7ffbd7400640 1 -- 172.21.15.7:0/641132017 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0067b40 con 0x5636089522a0 2024-09-10T21:45:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.937+0000 7ffbf07c1740 1 -- 172.21.15.7:0/641132017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636089522a0 msgr2=0x563608995c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.937+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/641132017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636089522a0 0x563608995c10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ffbd005f140 tx=0x7ffbd00619e0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.937+0000 7ffbf07c1740 1 -- 172.21.15.7:0/641132017 shutdown_connections 2024-09-10T21:45:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.937+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/641132017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563608952820 0x563608996150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.938+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/641132017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5636089522a0 0x563608995c10 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.938+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/641132017 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563608997130 0x563608997510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.938+0000 7ffbf07c1740 1 -- 172.21.15.7:0/641132017 >> 172.21.15.7:0/641132017 conn(0x56360894d870 msgr2=0x563608951290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.938+0000 7ffbf07c1740 1 -- 172.21.15.7:0/641132017 shutdown_connections 2024-09-10T21:45:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.938+0000 7ffbf07c1740 1 -- 172.21.15.7:0/641132017 wait complete. 2024-09-10T21:45:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.938+0000 7ffbf07c1740 1 -- start start 2024-09-10T21:45:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.939+0000 7ffbf07c1740 1 Processor -- start 2024-09-10T21:45:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.940+0000 7ffbf07c1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608952820 0x56360894aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.940+0000 7ffbf07c1740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563608997130 0x5636089957e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.940+0000 7ffbf07c1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56360894af50 0x563608995d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.940+0000 7ffbf07c1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56360885a410 con 0x563608952820 2024-09-10T21:45:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.940+0000 7ffbf07c1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5636088b7ab0 con 0x56360894af50 2024-09-10T21:45:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.940+0000 7ffbf07c1740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x563608624740 con 0x563608997130 2024-09-10T21:45:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.940+0000 7ffbf07c1740 10 client.0 _ll_register_callbacks cb 0x563608837360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:45:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.940+0000 7ffbe6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608952820 0x56360894aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.940+0000 7ffbe6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608952820 0x56360894aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42036/0 (socket says 172.21.15.7:42036) 2024-09-10T21:45:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.940+0000 7ffbe6a00640 1 -- 172.21.15.7:0/2603047046 learned_addr learned my addr 172.21.15.7:0/2603047046 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.940+0000 7ffbe7400640 1 --2- 172.21.15.7:0/2603047046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56360894af50 0x563608995d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.940+0000 7ffbde000640 1 --2- 172.21.15.7:0/2603047046 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563608997130 0x5636089957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.940+0000 7ffbe6a00640 1 -- 172.21.15.7:0/2603047046 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563608997130 msgr2=0x5636089957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.940+0000 7ffbe6a00640 1 --2- 172.21.15.7:0/2603047046 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563608997130 0x5636089957e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.940+0000 7ffbe6a00640 1 -- 172.21.15.7:0/2603047046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56360894af50 msgr2=0x563608995d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.940+0000 7ffbe6a00640 1 --2- 172.21.15.7:0/2603047046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56360894af50 0x563608995d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.940+0000 7ffbe6a00640 1 -- 172.21.15.7:0/2603047046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563608615970 con 0x563608952820 2024-09-10T21:45:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.941+0000 7ffbe6a00640 1 --2- 172.21.15.7:0/2603047046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608952820 0x56360894aa10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009950 tx=0x7ffbd8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=643d7560ecd5fe75 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.941+0000 7ffbcbe00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:45:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.941+0000 7ffbcbe00640 1 -- 172.21.15.7:0/2603047046 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd804f630 con 0x563608952820 2024-09-10T21:45:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.941+0000 7ffbcbe00640 1 -- 172.21.15.7:0/2603047046 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ffbd8046420 con 0x563608952820 2024-09-10T21:45:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.941+0000 7ffbf07c1740 1 -- 172.21.15.7:0/2603047046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbd006b3d0 con 0x563608952820 2024-09-10T21:45:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.941+0000 7ffbcbe00640 1 -- 172.21.15.7:0/2603047046 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ffbd804f7d0 con 0x563608952820 2024-09-10T21:45:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.941+0000 7ffbf07c1740 10 client.5901 fetch_fsmap learned FSMap version 36 2024-09-10T21:45:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.941+0000 7ffbf07c1740 1 -- 172.21.15.7:0/2603047046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x5636085f36a0 con 0x563608952820 2024-09-10T21:45:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.941+0000 7ffbcbe00640 1 -- 172.21.15.7:0/2603047046 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbd006b3d0 con 0x563608952820 2024-09-10T21:45:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.942+0000 7ffbcbe00640 1 -- 172.21.15.7:0/2603047046 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7ffbd804f970 con 0x563608952820 2024-09-10T21:45:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.942+0000 7ffbf07c1740 10 client.5901 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T21:45:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.942+0000 7ffbf07c1740 10 client.5901 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-10T21:45:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.942+0000 7ffbf07c1740 20 client.5901 populate_metadata read hostname 'smithi007' 2024-09-10T21:45:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.942+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/2603047046 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x5636089665a0 0x563608968f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.942+0000 7ffbf07c1740 4 client.5901 mds_command: new command op to 235016 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:45:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.942+0000 7ffbf07c1740 1 -- 172.21.15.7:0/2603047046 --> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5636088c67c0 con 0x5636089665a0 2024-09-10T21:45:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.942+0000 7ffbde000640 1 --2- 172.21.15.7:0/2603047046 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x5636089665a0 0x563608968f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:10.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.942+0000 7ffbcbe00640 10 client.5901 ms_handle_connect on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.942+0000 7ffbde000640 1 --2- 172.21.15.7:0/2603047046 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x5636089665a0 0x563608968f00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c45e762519c9837 server_cookie=9cc08679547977f4 in_seq=0 out_seq=0 2024-09-10T21:45:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.943+0000 7ffbcbe00640 1 -- 172.21.15.7:0/2603047046 <== mds.0 v2:172.21.15.7:6833/364473218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5636088c67c0 con 0x5636089665a0 2024-09-10T21:45:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.943+0000 7ffbcbe00640 10 client.5901 handle_command_reply: tid=0 2024-09-10T21:45:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.943+0000 7ffbf07c1740 1 client.5901 shutdown 2024-09-10T21:45:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.943+0000 7ffbf07c1740 1 -- 172.21.15.7:0/2603047046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608952820 msgr2=0x56360894aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.943+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/2603047046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608952820 0x56360894aa10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8009950 tx=0x7ffbd8005f90 comp rx=0 tx=0).stop 2024-09-10T21:45:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.943+0000 7ffbf07c1740 1 -- 172.21.15.7:0/2603047046 shutdown_connections 2024-09-10T21:45:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.943+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/2603047046 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x5636089665a0 0x563608968f00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.943+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/2603047046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56360894af50 0x563608995d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.943+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/2603047046 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563608997130 0x5636089957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.943+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/2603047046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608952820 0x56360894aa10 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.943+0000 7ffbcbe00640 0 client.5901 ms_handle_reset on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.943+0000 7ffbf07c1740 1 -- 172.21.15.7:0/2603047046 >> 172.21.15.7:0/2603047046 conn(0x5636089522a0 msgr2=0x56360894c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.944+0000 7ffbf07c1740 1 -- 172.21.15.7:0/2603047046 shutdown_connections 2024-09-10T21:45:10.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.944+0000 7ffbf07c1740 1 -- 172.21.15.7:0/2603047046 wait complete. 2024-09-10T21:45:10.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.944+0000 7ffbf07c1740 20 client.5901 trim_cache size 0 max 16384 2024-09-10T21:45:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.962+0000 7ffbf07c1740 1 Processor -- start 2024-09-10T21:45:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.962+0000 7ffbf07c1740 1 -- start start 2024-09-10T21:45:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.963+0000 7ffbf07c1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56360894af50 0x56360894c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.963+0000 7ffbf07c1740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563608952820 0x56360898f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.963+0000 7ffbf07c1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608997130 0x563608995770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.963+0000 7ffbf07c1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563608969960 con 0x563608997130 2024-09-10T21:45:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.963+0000 7ffbf07c1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe8201360 con 0x56360894af50 2024-09-10T21:45:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.963+0000 7ffbf07c1740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ffbe82003a0 con 0x563608952820 2024-09-10T21:45:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.963+0000 7ffbe7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608997130 0x563608995770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.963+0000 7ffbe7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608997130 0x563608995770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42044/0 (socket says 172.21.15.7:42044) 2024-09-10T21:45:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.963+0000 7ffbe7400640 1 -- 172.21.15.7:0/651084770 learned_addr learned my addr 172.21.15.7:0/651084770 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.963+0000 7ffbde000640 1 --2- 172.21.15.7:0/651084770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563608952820 0x56360898f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.963+0000 7ffbe6a00640 1 --2- 172.21.15.7:0/651084770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56360894af50 0x56360894c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.964+0000 7ffbe7400640 1 -- 172.21.15.7:0/651084770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563608952820 msgr2=0x56360898f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.964+0000 7ffbe7400640 1 --2- 172.21.15.7:0/651084770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563608952820 0x56360898f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.964+0000 7ffbe7400640 1 -- 172.21.15.7:0/651084770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56360894af50 msgr2=0x56360894c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.964+0000 7ffbe7400640 1 --2- 172.21.15.7:0/651084770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56360894af50 0x56360894c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.964+0000 7ffbe7400640 1 -- 172.21.15.7:0/651084770 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563608996530 con 0x563608997130 2024-09-10T21:45:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.964+0000 7ffbe7400640 1 --2- 172.21.15.7:0/651084770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608997130 0x563608995770 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0005550 tx=0x7ffbe0005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56bdf5aa47d6bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.964+0000 7ffbd6a00640 1 -- 172.21.15.7:0/651084770 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0019070 con 0x563608997130 2024-09-10T21:45:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.964+0000 7ffbd6a00640 1 -- 172.21.15.7:0/651084770 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbe0006e50 con 0x563608997130 2024-09-10T21:45:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.964+0000 7ffbd6a00640 1 -- 172.21.15.7:0/651084770 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0012cc0 con 0x563608997130 2024-09-10T21:45:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.965+0000 7ffbf07c1740 1 -- 172.21.15.7:0/651084770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608997130 msgr2=0x563608995770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.965+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/651084770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608997130 0x563608995770 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0005550 tx=0x7ffbe0005950 comp rx=0 tx=0).stop 2024-09-10T21:45:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.965+0000 7ffbf07c1740 1 -- 172.21.15.7:0/651084770 shutdown_connections 2024-09-10T21:45:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.965+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/651084770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608997130 0x563608995770 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.965+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/651084770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563608952820 0x56360898f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.965+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/651084770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56360894af50 0x56360894c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.965+0000 7ffbf07c1740 1 -- 172.21.15.7:0/651084770 >> 172.21.15.7:0/651084770 conn(0x5636089522a0 msgr2=0x563608946ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.965+0000 7ffbf07c1740 1 -- 172.21.15.7:0/651084770 shutdown_connections 2024-09-10T21:45:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.965+0000 7ffbf07c1740 1 -- 172.21.15.7:0/651084770 wait complete. 2024-09-10T21:45:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.966+0000 7ffbf07c1740 1 -- start start 2024-09-10T21:45:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.966+0000 7ffbf07c1740 1 Processor -- start 2024-09-10T21:45:10.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.967+0000 7ffbf07c1740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56360894af50 0x563608a15fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:10.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.967+0000 7ffbf07c1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563608952820 0x563608950c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:10.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.967+0000 7ffbf07c1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608997130 0x563608a19c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:10.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.967+0000 7ffbf07c1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81ffd50 con 0x563608997130 2024-09-10T21:45:10.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.967+0000 7ffbf07c1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ffbe81ff700 con 0x563608952820 2024-09-10T21:45:10.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.967+0000 7ffbf07c1740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x56360894cbb0 con 0x56360894af50 2024-09-10T21:45:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.967+0000 7ffbf07c1740 10 client.0 _ll_register_callbacks cb 0x7ffbd80096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:45:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.967+0000 7ffbe7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608997130 0x563608a19c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.967+0000 7ffbe7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608997130 0x563608a19c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42058/0 (socket says 172.21.15.7:42058) 2024-09-10T21:45:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.967+0000 7ffbe7400640 1 -- 172.21.15.7:0/2363242533 learned_addr learned my addr 172.21.15.7:0/2363242533 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.967+0000 7ffbe6a00640 1 --2- 172.21.15.7:0/2363242533 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56360894af50 0x563608a15fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.967+0000 7ffbde000640 1 --2- 172.21.15.7:0/2363242533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563608952820 0x563608950c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.967+0000 7ffbe7400640 1 -- 172.21.15.7:0/2363242533 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56360894af50 msgr2=0x563608a15fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.967+0000 7ffbe7400640 1 --2- 172.21.15.7:0/2363242533 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56360894af50 0x563608a15fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.968+0000 7ffbe7400640 1 -- 172.21.15.7:0/2363242533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563608952820 msgr2=0x563608950c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.968+0000 7ffbe7400640 1 --2- 172.21.15.7:0/2363242533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563608952820 0x563608950c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.968+0000 7ffbe7400640 1 -- 172.21.15.7:0/2363242533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563608951190 con 0x563608997130 2024-09-10T21:45:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.968+0000 7ffbe7400640 1 --2- 172.21.15.7:0/2363242533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608997130 0x563608a19c60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0007280 tx=0x7ffbe00072b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57eb1c0ab98a0af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.968+0000 7ffbcbe00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:45:10.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.968+0000 7ffbcbe00640 1 -- 172.21.15.7:0/2363242533 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbe0019050 con 0x563608997130 2024-09-10T21:45:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.968+0000 7ffbcbe00640 1 -- 172.21.15.7:0/2363242533 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ffbe0007860 con 0x563608997130 2024-09-10T21:45:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.968+0000 7ffbf07c1740 1 -- 172.21.15.7:0/2363242533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbe0006e50 con 0x563608997130 2024-09-10T21:45:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.968+0000 7ffbcbe00640 1 -- 172.21.15.7:0/2363242533 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ffbe00051a0 con 0x563608997130 2024-09-10T21:45:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.968+0000 7ffbf07c1740 10 client.5907 fetch_fsmap learned FSMap version 36 2024-09-10T21:45:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.968+0000 7ffbf07c1740 1 -- 172.21.15.7:0/2363242533 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x563608a1a5b0 con 0x563608997130 2024-09-10T21:45:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.969+0000 7ffbcbe00640 1 -- 172.21.15.7:0/2363242533 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbe0006e50 con 0x563608997130 2024-09-10T21:45:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.969+0000 7ffbcbe00640 1 -- 172.21.15.7:0/2363242533 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7ffbe0003b80 con 0x563608997130 2024-09-10T21:45:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.969+0000 7ffbf07c1740 10 client.5907 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T21:45:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.969+0000 7ffbf07c1740 10 client.5907 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-10T21:45:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.969+0000 7ffbf07c1740 20 client.5907 populate_metadata read hostname 'smithi007' 2024-09-10T21:45:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.969+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/2363242533 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x563608a1a1a0 0x563608995770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.969+0000 7ffbf07c1740 4 client.5907 mds_command: new command op to 235016 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:45:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.969+0000 7ffbf07c1740 1 -- 172.21.15.7:0/2363242533 --> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5636088c2270 con 0x563608a1a1a0 2024-09-10T21:45:10.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.969+0000 7ffbe6a00640 1 --2- 172.21.15.7:0/2363242533 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x563608a1a1a0 0x563608995770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.970+0000 7ffbe6a00640 1 --2- 172.21.15.7:0/2363242533 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x563608a1a1a0 0x563608995770 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7b3a27286105638 server_cookie=335a4fbf0eea822f in_seq=0 out_seq=0 2024-09-10T21:45:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.970+0000 7ffbcbe00640 10 client.5907 ms_handle_connect on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.970+0000 7ffbcbe00640 1 -- 172.21.15.7:0/2363242533 <== mds.0 v2:172.21.15.7:6833/364473218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x5636088c2270 con 0x563608a1a1a0 2024-09-10T21:45:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.970+0000 7ffbcbe00640 10 client.5907 handle_command_reply: tid=0 2024-09-10T21:45:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.970+0000 7ffbf07c1740 1 client.5907 shutdown 2024-09-10T21:45:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.970+0000 7ffbf07c1740 1 -- 172.21.15.7:0/2363242533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608997130 msgr2=0x563608a19c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.970+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/2363242533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608997130 0x563608a19c60 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0007280 tx=0x7ffbe00072b0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.970+0000 7ffbf07c1740 1 -- 172.21.15.7:0/2363242533 shutdown_connections 2024-09-10T21:45:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.970+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/2363242533 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x563608a1a1a0 0x563608995770 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.970+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/2363242533 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563608952820 0x563608950c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.970+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/2363242533 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563608997130 0x563608a19c60 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.970+0000 7ffbf07c1740 1 --2- 172.21.15.7:0/2363242533 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56360894af50 0x563608a15fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.970+0000 7ffbcbe00640 0 client.5907 ms_handle_reset on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:10.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.970+0000 7ffbf07c1740 1 -- 172.21.15.7:0/2363242533 >> 172.21.15.7:0/2363242533 conn(0x5636089522a0 msgr2=0x563608946ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:10.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.970+0000 7ffbf07c1740 1 -- 172.21.15.7:0/2363242533 shutdown_connections 2024-09-10T21:45:10.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.970+0000 7ffbf07c1740 1 -- 172.21.15.7:0/2363242533 wait complete. 2024-09-10T21:45:10.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.970+0000 7ffbf07c1740 20 client.5907 trim_cache size 0 max 16384 2024-09-10T21:45:10.976 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:45:10.976 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-09-10T21:45:10.976 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {}, 2024-09-10T21:45:10.976 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:45:10.976 INFO:teuthology.orchestra.run.smithi007.stdout: "cabd7ddd-fca9-48c5-a921-5e626bb8f21c": { 2024-09-10T21:45:10.976 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:44:40.510317+0000", 2024-09-10T21:45:10.977 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-09-10T21:45:10.977 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:45:10.977 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:45:10.977 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:45:10.977 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:45:10.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:10.977 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:10.977 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:45:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.971+0000 7ffbed800640 1 -- 172.21.15.7:0/632568188 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ffbc4077fa0 msgr2=0x7ffbc407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.971+0000 7ffbed800640 1 --2- 172.21.15.7:0/632568188 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ffbc4077fa0 0x7ffbc407a460 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7ffbd8034000 tx=0x7ffbd8004730 comp rx=0 tx=0).stop 2024-09-10T21:45:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.971+0000 7ffbed800640 1 -- 172.21.15.7:0/632568188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe8073e20 msgr2=0x7ffbe815e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.971+0000 7ffbed800640 1 --2- 172.21.15.7:0/632568188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe8073e20 0x7ffbe815e530 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000c420 tx=0x7ffbd000c8e0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.971+0000 7ffbed800640 1 -- 172.21.15.7:0/632568188 shutdown_connections 2024-09-10T21:45:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.971+0000 7ffbed800640 1 --2- 172.21.15.7:0/632568188 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ffbc4077fa0 0x7ffbc407a460 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.971+0000 7ffbed800640 1 --2- 172.21.15.7:0/632568188 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffbe8159680 0x7ffbe815ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.971+0000 7ffbed800640 1 --2- 172.21.15.7:0/632568188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffbe8073e20 0x7ffbe815e530 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.971+0000 7ffbed800640 1 --2- 172.21.15.7:0/632568188 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffbe806e580 0x7ffbe814c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:10.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.971+0000 7ffbed800640 1 -- 172.21.15.7:0/632568188 >> 172.21.15.7:0/632568188 conn(0x7ffbe80b16d0 msgr2=0x7ffbe815b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:10.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.971+0000 7ffbed800640 1 -- 172.21.15.7:0/632568188 shutdown_connections 2024-09-10T21:45:10.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:10.971+0000 7ffbed800640 1 -- 172.21.15.7:0/632568188 wait complete. 2024-09-10T21:45:10.988 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:45:10.988 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:45:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.149+0000 7f3c13600640 1 -- 172.21.15.7:0/1014442379 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c0c0ca360 msgr2=0x7f3c0c15a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.149+0000 7f3c13600640 1 --2- 172.21.15.7:0/1014442379 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c0c0ca360 0x7f3c0c15a8e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009c10 tx=0x7f3c0402a940 comp rx=0 tx=0).stop 2024-09-10T21:45:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.150+0000 7f3c13600640 1 -- 172.21.15.7:0/1014442379 shutdown_connections 2024-09-10T21:45:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.150+0000 7f3c13600640 1 --2- 172.21.15.7:0/1014442379 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c0c0ca360 0x7f3c0c15a8e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.150+0000 7f3c13600640 1 -- 172.21.15.7:0/1014442379 >> 172.21.15.7:0/1014442379 conn(0x7f3c0c0b16d0 msgr2=0x7f3c0c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.150+0000 7f3c13600640 1 -- 172.21.15.7:0/1014442379 shutdown_connections 2024-09-10T21:45:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.150+0000 7f3c13600640 1 -- 172.21.15.7:0/1014442379 wait complete. 2024-09-10T21:45:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.151+0000 7f3c13600640 1 Processor -- start 2024-09-10T21:45:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.151+0000 7f3c13600640 1 -- start start 2024-09-10T21:45:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.152+0000 7f3c13600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c0c070600 0x7f3c0c1f8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.152+0000 7f3c13600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c0ca360 0x7f3c0c0b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.152+0000 7f3c13600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c0c15da10 0x7f3c0c1f5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.152+0000 7f3c13600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c15b260 con 0x7f3c0c0ca360 2024-09-10T21:45:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.152+0000 7f3c13600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3c0c15b0e0 con 0x7f3c0c15da10 2024-09-10T21:45:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.152+0000 7f3c13600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f3c0c15b3e0 con 0x7f3c0c070600 2024-09-10T21:45:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.152+0000 7f3c0be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c0ca360 0x7f3c0c0b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.153+0000 7f3c0be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c0ca360 0x7f3c0c0b1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42060/0 (socket says 172.21.15.7:42060) 2024-09-10T21:45:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.153+0000 7f3c0be00640 1 -- 172.21.15.7:0/2932979580 learned_addr learned my addr 172.21.15.7:0/2932979580 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.153+0000 7f3c11600640 1 --2- 172.21.15.7:0/2932979580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c0c15da10 0x7f3c0c1f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.153+0000 7f3c10c00640 1 --2- 172.21.15.7:0/2932979580 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c0c070600 0x7f3c0c1f8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.153+0000 7f3c0be00640 1 -- 172.21.15.7:0/2932979580 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c0c070600 msgr2=0x7f3c0c1f8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.153+0000 7f3c0be00640 1 --2- 172.21.15.7:0/2932979580 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c0c070600 0x7f3c0c1f8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.153+0000 7f3c0be00640 1 -- 172.21.15.7:0/2932979580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c0c15da10 msgr2=0x7f3c0c1f5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.153+0000 7f3c0be00640 1 --2- 172.21.15.7:0/2932979580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c0c15da10 0x7f3c0c1f5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.153+0000 7f3c0be00640 1 -- 172.21.15.7:0/2932979580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c0c0c9eb0 con 0x7f3c0c0ca360 2024-09-10T21:45:11.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.153+0000 7f3c0be00640 1 --2- 172.21.15.7:0/2932979580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c0ca360 0x7f3c0c0b1070 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3bf800c400 tx=0x7f3bf800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:11.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.153+0000 7f3c08c00640 1 -- 172.21.15.7:0/2932979580 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8011850 con 0x7f3c0c0ca360 2024-09-10T21:45:11.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.154+0000 7f3c08c00640 1 -- 172.21.15.7:0/2932979580 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3bf8017070 con 0x7f3c0c0ca360 2024-09-10T21:45:11.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.154+0000 7f3c13600640 1 -- 172.21.15.7:0/2932979580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c0c1f4b80 con 0x7f3c0c0ca360 2024-09-10T21:45:11.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.154+0000 7f3c08c00640 1 -- 172.21.15.7:0/2932979580 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf8011da0 con 0x7f3c0c0ca360 2024-09-10T21:45:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.154+0000 7f3c13600640 1 -- 172.21.15.7:0/2932979580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c0c073340 con 0x7f3c0c0ca360 2024-09-10T21:45:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.155+0000 7f3c13600640 1 -- 172.21.15.7:0/2932979580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c0c0cad00 con 0x7f3c0c0ca360 2024-09-10T21:45:11.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.156+0000 7f3c08c00640 1 -- 172.21.15.7:0/2932979580 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3bf801c460 con 0x7f3c0c0ca360 2024-09-10T21:45:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.157+0000 7f3c08c00640 1 --2- 172.21.15.7:0/2932979580 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3bec077f70 0x7f3bec07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.157+0000 7f3c08c00640 1 -- 172.21.15.7:0/2932979580 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f3bf8099bd0 con 0x7f3c0c0ca360 2024-09-10T21:45:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.157+0000 7f3c10c00640 1 --2- 172.21.15.7:0/2932979580 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3bec077f70 0x7f3bec07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.158+0000 7f3c10c00640 1 --2- 172.21.15.7:0/2932979580 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3bec077f70 0x7f3bec07a430 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3c00001650 tx=0x7f3c00009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.162+0000 7f3c08c00640 1 -- 172.21.15.7:0/2932979580 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3c0c0cad00 con 0x7f3c0c0ca360 2024-09-10T21:45:11.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.400+0000 7f3c13600640 1 -- 172.21.15.7:0/2932979580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c0c0cd2a0 con 0x7f3c0c0ca360 2024-09-10T21:45:11.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.401+0000 7f3c08c00640 1 -- 172.21.15.7:0/2932979580 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5227 (secure 0 0 0) 0x7f3c0c0cd2a0 con 0x7f3c0c0ca360 2024-09-10T21:45:11.402 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:45:11.402 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":36,"btime":"2024-09-10T21:44:43:429122+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5473,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/2853074495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2853074495},{"type":"v1","addr":"172.21.15.133:6836","nonce":2853074495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3498706462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3498706462},{"type":"v1","addr":"172.21.15.133:6837","nonce":3498706462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/521190705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":521190705},{"type":"v1","addr":"172.21.15.7:6835","nonce":521190705}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:44:30.804578+0000","modified":"2024-09-10T21:44:43.409545+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235016},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235016":{"gid":235016,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":6,"addr":"172.21.15.7:6834/364473218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":364473218},{"type":"v1","addr":"172.21.15.7:6834","nonce":364473218}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235016,"qdb_cluster":[235016]},"id":4}]} 2024-09-10T21:45:11.402 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 36 2024-09-10T21:45:11.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.403+0000 7f3c13600640 1 -- 172.21.15.7:0/2932979580 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3bec077f70 msgr2=0x7f3bec07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:11.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.403+0000 7f3c13600640 1 --2- 172.21.15.7:0/2932979580 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3bec077f70 0x7f3bec07a430 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3c00001650 tx=0x7f3c00009040 comp rx=0 tx=0).stop 2024-09-10T21:45:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.403+0000 7f3c13600640 1 -- 172.21.15.7:0/2932979580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c0ca360 msgr2=0x7f3c0c0b1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.403+0000 7f3c13600640 1 --2- 172.21.15.7:0/2932979580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c0ca360 0x7f3c0c0b1070 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3bf800c400 tx=0x7f3bf800c8c0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.403+0000 7f3c13600640 1 -- 172.21.15.7:0/2932979580 shutdown_connections 2024-09-10T21:45:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.403+0000 7f3c13600640 1 --2- 172.21.15.7:0/2932979580 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3bec077f70 0x7f3bec07a430 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.403+0000 7f3c13600640 1 --2- 172.21.15.7:0/2932979580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c0c15da10 0x7f3c0c1f5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.403+0000 7f3c13600640 1 --2- 172.21.15.7:0/2932979580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c0c0ca360 0x7f3c0c0b1070 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.403+0000 7f3c13600640 1 --2- 172.21.15.7:0/2932979580 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c0c070600 0x7f3c0c1f8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.403+0000 7f3c13600640 1 -- 172.21.15.7:0/2932979580 >> 172.21.15.7:0/2932979580 conn(0x7f3c0c0b16d0 msgr2=0x7f3c0c0cf360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.403+0000 7f3c13600640 1 -- 172.21.15.7:0/2932979580 shutdown_connections 2024-09-10T21:45:11.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.403+0000 7f3c13600640 1 -- 172.21.15.7:0/2932979580 wait complete. 2024-09-10T21:45:11.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-09-10T21:45:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.583+0000 7f48a9000640 1 -- 172.21.15.7:0/1399036525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48a4159680 msgr2=0x7f48a415cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:11.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.583+0000 7f48a9000640 1 --2- 172.21.15.7:0/1399036525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48a4159680 0x7f48a415cc00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f4894005fd0 tx=0x7f489402a540 comp rx=0 tx=0).stop 2024-09-10T21:45:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.583+0000 7f48a9000640 1 -- 172.21.15.7:0/1399036525 shutdown_connections 2024-09-10T21:45:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.583+0000 7f48a9000640 1 --2- 172.21.15.7:0/1399036525 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48a4073e20 0x7f48a40742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.583+0000 7f48a9000640 1 --2- 172.21.15.7:0/1399036525 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48a4159680 0x7f48a415cc00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.583+0000 7f48a9000640 1 -- 172.21.15.7:0/1399036525 >> 172.21.15.7:0/1399036525 conn(0x7f48a40b16d0 msgr2=0x7f48a40b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.583+0000 7f48a9000640 1 -- 172.21.15.7:0/1399036525 shutdown_connections 2024-09-10T21:45:11.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.583+0000 7f48a9000640 1 -- 172.21.15.7:0/1399036525 wait complete. 2024-09-10T21:45:11.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.585+0000 7f48a9000640 1 Processor -- start 2024-09-10T21:45:11.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.585+0000 7f48a9000640 1 -- start start 2024-09-10T21:45:11.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.586+0000 7f48a9000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48a406e580 0x7f48a41f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.586+0000 7f48a9000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48a4073e20 0x7f48a40cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.586+0000 7f48a9000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48a4159680 0x7f48a40cc760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.586+0000 7f48a9000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48a40ca990 con 0x7f48a4073e20 2024-09-10T21:45:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.586+0000 7f48a9000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f48a40ca810 con 0x7f48a4159680 2024-09-10T21:45:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.586+0000 7f48a9000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f48a40cab10 con 0x7f48a406e580 2024-09-10T21:45:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.586+0000 7f4899400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48a4073e20 0x7f48a40cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.586+0000 7f4899400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48a4073e20 0x7f48a40cc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42080/0 (socket says 172.21.15.7:42080) 2024-09-10T21:45:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.586+0000 7f4899400640 1 -- 172.21.15.7:0/1506701859 learned_addr learned my addr 172.21.15.7:0/1506701859 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.586+0000 7f48a1e00640 1 --2- 172.21.15.7:0/1506701859 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48a406e580 0x7f48a41f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.586+0000 7f48a2800640 1 --2- 172.21.15.7:0/1506701859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48a4159680 0x7f48a40cc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.586+0000 7f4899400640 1 -- 172.21.15.7:0/1506701859 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48a406e580 msgr2=0x7f48a41f2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.586+0000 7f4899400640 1 --2- 172.21.15.7:0/1506701859 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48a406e580 0x7f48a41f2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.586+0000 7f4899400640 1 -- 172.21.15.7:0/1506701859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48a4159680 msgr2=0x7f48a40cc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.586+0000 7f4899400640 1 --2- 172.21.15.7:0/1506701859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48a4159680 0x7f48a40cc760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.586+0000 7f4899400640 1 -- 172.21.15.7:0/1506701859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a40b50d0 con 0x7f48a4073e20 2024-09-10T21:45:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.587+0000 7f4899400640 1 --2- 172.21.15.7:0/1506701859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48a4073e20 0x7f48a40cc220 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f488c00c420 tx=0x7f488c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.587+0000 7f489aa00640 1 -- 172.21.15.7:0/1506701859 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c017070 con 0x7f48a4073e20 2024-09-10T21:45:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.587+0000 7f489aa00640 1 -- 172.21.15.7:0/1506701859 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f488c0053c0 con 0x7f48a4073e20 2024-09-10T21:45:11.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.587+0000 7f489aa00640 1 -- 172.21.15.7:0/1506701859 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c003850 con 0x7f48a4073e20 2024-09-10T21:45:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.587+0000 7f48a9000640 1 -- 172.21.15.7:0/1506701859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a40b2b00 con 0x7f48a4073e20 2024-09-10T21:45:11.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.587+0000 7f48a9000640 1 -- 172.21.15.7:0/1506701859 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a40b3080 con 0x7f48a4073e20 2024-09-10T21:45:11.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.591+0000 7f489aa00640 1 -- 172.21.15.7:0/1506701859 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f488c0039f0 con 0x7f48a4073e20 2024-09-10T21:45:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.591+0000 7f489aa00640 1 --2- 172.21.15.7:0/1506701859 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4880077f70 0x7f488007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.592+0000 7f48abe27740 1 Processor -- start 2024-09-10T21:45:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.592+0000 7f489aa00640 1 -- 172.21.15.7:0/1506701859 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f488c0993c0 con 0x7f48a4073e20 2024-09-10T21:45:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.592+0000 7f48abe27740 1 -- start start 2024-09-10T21:45:11.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.592+0000 7f48a1e00640 1 --2- 172.21.15.7:0/1506701859 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4880077f70 0x7f488007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.593+0000 7f48abe27740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55c17a0 0x55d9a55be640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.593+0000 7f48abe27740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55d31c0 0x55d9a55c1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.593+0000 7f48a1e00640 1 --2- 172.21.15.7:0/1506701859 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4880077f70 0x7f488007a430 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4894033000 tx=0x7f4894004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.593+0000 7f48abe27740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55c20c0 0x55d9a5592ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.593+0000 7f48abe27740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d9a5509750 con 0x55d9a55d31c0 2024-09-10T21:45:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.593+0000 7f48abe27740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d9a54f8b80 con 0x55d9a55c17a0 2024-09-10T21:45:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.593+0000 7f48abe27740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55d9a54fc990 con 0x55d9a55c20c0 2024-09-10T21:45:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.593+0000 7f4899400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55d31c0 0x55d9a55c1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.593+0000 7f4899400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55d31c0 0x55d9a55c1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42090/0 (socket says 172.21.15.7:42090) 2024-09-10T21:45:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.593+0000 7f4899400640 1 -- 172.21.15.7:0/4175481618 learned_addr learned my addr 172.21.15.7:0/4175481618 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.593+0000 7f48a1e00640 1 --2- 172.21.15.7:0/4175481618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55c17a0 0x55d9a55be640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.593+0000 7f48a2800640 1 --2- 172.21.15.7:0/4175481618 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55c20c0 0x55d9a5592ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.594+0000 7f4899400640 1 -- 172.21.15.7:0/4175481618 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55c20c0 msgr2=0x55d9a5592ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.594+0000 7f4899400640 1 --2- 172.21.15.7:0/4175481618 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55c20c0 0x55d9a5592ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.594+0000 7f4899400640 1 -- 172.21.15.7:0/4175481618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55c17a0 msgr2=0x55d9a55be640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.594+0000 7f4899400640 1 --2- 172.21.15.7:0/4175481618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55c17a0 0x55d9a55be640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.594+0000 7f4899400640 1 -- 172.21.15.7:0/4175481618 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9a5486ad0 con 0x55d9a55d31c0 2024-09-10T21:45:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.594+0000 7f4899400640 1 --2- 172.21.15.7:0/4175481618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55d31c0 0x55d9a55c1b80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f488c05ead0 tx=0x7f488c061370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=611f890bb6402bb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.594+0000 7f4892a00640 1 -- 172.21.15.7:0/4175481618 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c066290 con 0x55d9a55d31c0 2024-09-10T21:45:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.594+0000 7f4892a00640 1 -- 172.21.15.7:0/4175481618 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f488c06acb0 con 0x55d9a55d31c0 2024-09-10T21:45:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.594+0000 7f4892a00640 1 -- 172.21.15.7:0/4175481618 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f488c060ac0 con 0x55d9a55d31c0 2024-09-10T21:45:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.595+0000 7f48abe27740 1 -- 172.21.15.7:0/4175481618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55d31c0 msgr2=0x55d9a55c1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.595+0000 7f48abe27740 1 --2- 172.21.15.7:0/4175481618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55d31c0 0x55d9a55c1b80 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f488c05ead0 tx=0x7f488c061370 comp rx=0 tx=0).stop 2024-09-10T21:45:11.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.595+0000 7f48abe27740 1 -- 172.21.15.7:0/4175481618 shutdown_connections 2024-09-10T21:45:11.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.595+0000 7f48abe27740 1 --2- 172.21.15.7:0/4175481618 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55c20c0 0x55d9a5592ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.595+0000 7f48abe27740 1 --2- 172.21.15.7:0/4175481618 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55d31c0 0x55d9a55c1b80 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.595+0000 7f48abe27740 1 --2- 172.21.15.7:0/4175481618 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55c17a0 0x55d9a55be640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.595+0000 7f48abe27740 1 -- 172.21.15.7:0/4175481618 >> 172.21.15.7:0/4175481618 conn(0x55d9a5593ce0 msgr2=0x55d9a5597700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:11.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.595+0000 7f48abe27740 1 -- 172.21.15.7:0/4175481618 shutdown_connections 2024-09-10T21:45:11.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.595+0000 7f48abe27740 1 -- 172.21.15.7:0/4175481618 wait complete. 2024-09-10T21:45:11.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.596+0000 7f48abe27740 1 -- start start 2024-09-10T21:45:11.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.596+0000 7f48abe27740 1 Processor -- start 2024-09-10T21:45:11.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.597+0000 7f48abe27740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55c20c0 0x55d9a55aad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.597+0000 7f48abe27740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55d31c0 0x55d9a55b8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.597+0000 7f48abe27740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55ec200 0x55d9a55ec680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.597+0000 7f48abe27740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d9a5269700 con 0x55d9a55c20c0 2024-09-10T21:45:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.597+0000 7f48abe27740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d9a549e430 con 0x55d9a55ec200 2024-09-10T21:45:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.597+0000 7f48abe27740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55d9a5598780 con 0x55d9a55d31c0 2024-09-10T21:45:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.597+0000 7f48abe27740 10 client.0 _ll_register_callbacks cb 0x55d9a54a4600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:45:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.597+0000 7f48a1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55c20c0 0x55d9a55aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.597+0000 7f48a1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55c20c0 0x55d9a55aad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42098/0 (socket says 172.21.15.7:42098) 2024-09-10T21:45:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.597+0000 7f48a1e00640 1 -- 172.21.15.7:0/2622216664 learned_addr learned my addr 172.21.15.7:0/2622216664 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.598+0000 7f48a2800640 1 --2- 172.21.15.7:0/2622216664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55ec200 0x55d9a55ec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.598+0000 7f4899400640 1 --2- 172.21.15.7:0/2622216664 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55d31c0 0x55d9a55b8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.598+0000 7f48a1e00640 1 -- 172.21.15.7:0/2622216664 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55d31c0 msgr2=0x55d9a55b8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.598+0000 7f48a1e00640 1 --2- 172.21.15.7:0/2622216664 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55d31c0 0x55d9a55b8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.598+0000 7f48a1e00640 1 -- 172.21.15.7:0/2622216664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55ec200 msgr2=0x55d9a55ec680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:11.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.598+0000 7f48a1e00640 1 --2- 172.21.15.7:0/2622216664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55ec200 0x55d9a55ec680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.598+0000 7f48a1e00640 1 -- 172.21.15.7:0/2622216664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9a52d37c0 con 0x55d9a55c20c0 2024-09-10T21:45:11.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.598+0000 7f48a1e00640 1 --2- 172.21.15.7:0/2622216664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55c20c0 0x55d9a55aad50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f489402ab30 tx=0x7f4894009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9af189e4652683 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:11.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.598+0000 7f4887400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:45:11.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.598+0000 7f4887400640 1 -- 172.21.15.7:0/2622216664 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489404d070 con 0x55d9a55c20c0 2024-09-10T21:45:11.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.598+0000 7f4887400640 1 -- 172.21.15.7:0/2622216664 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f489403cbc0 con 0x55d9a55c20c0 2024-09-10T21:45:11.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.598+0000 7f48abe27740 1 -- 172.21.15.7:0/2622216664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f488c06acb0 con 0x55d9a55c20c0 2024-09-10T21:45:11.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.598+0000 7f4887400640 1 -- 172.21.15.7:0/2622216664 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4894058050 con 0x55d9a55c20c0 2024-09-10T21:45:11.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.599+0000 7f48abe27740 10 client.5925 fetch_fsmap learned FSMap version 36 2024-09-10T21:45:11.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.599+0000 7f48abe27740 1 -- 172.21.15.7:0/2622216664 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55d9a5333f50 con 0x55d9a55c20c0 2024-09-10T21:45:11.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.599+0000 7f4887400640 1 -- 172.21.15.7:0/2622216664 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f488c06acb0 con 0x55d9a55c20c0 2024-09-10T21:45:11.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.599+0000 7f4887400640 1 -- 172.21.15.7:0/2622216664 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f489405b020 con 0x55d9a55c20c0 2024-09-10T21:45:11.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.599+0000 7f48abe27740 10 client.5925 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T21:45:11.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.599+0000 7f48abe27740 10 client.5925 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:45:11.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.599+0000 7f48abe27740 20 client.5925 populate_metadata read hostname 'smithi007' 2024-09-10T21:45:11.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.599+0000 7f48abe27740 1 --2- 172.21.15.7:0/2622216664 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55d9a55f7c50 0x55d9a55fa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.599+0000 7f48abe27740 4 client.5925 mds_command: new command op to 235016 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:45:11.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.599+0000 7f48abe27740 1 -- 172.21.15.7:0/2622216664 --> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9a550b6a0 con 0x55d9a55f7c50 2024-09-10T21:45:11.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.599+0000 7f4899400640 1 --2- 172.21.15.7:0/2622216664 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55d9a55f7c50 0x55d9a55fa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.600+0000 7f4899400640 1 --2- 172.21.15.7:0/2622216664 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55d9a55f7c50 0x55d9a55fa040 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=463599e3b93f34e0 server_cookie=81faac0bd881eeb1 in_seq=0 out_seq=0 2024-09-10T21:45:11.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.600+0000 7f4887400640 10 client.5925 ms_handle_connect on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:11.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.600+0000 7f4887400640 1 -- 172.21.15.7:0/2622216664 <== mds.0 v2:172.21.15.7:6833/364473218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d9a550b6a0 con 0x55d9a55f7c50 2024-09-10T21:45:11.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.600+0000 7f4887400640 10 client.5925 handle_command_reply: tid=0 2024-09-10T21:45:11.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.600+0000 7f48abe27740 1 client.5925 shutdown 2024-09-10T21:45:11.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.601+0000 7f48abe27740 1 -- 172.21.15.7:0/2622216664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55c20c0 msgr2=0x55d9a55aad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:11.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.601+0000 7f48abe27740 1 --2- 172.21.15.7:0/2622216664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55c20c0 0x55d9a55aad50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f489402ab30 tx=0x7f4894009260 comp rx=0 tx=0).stop 2024-09-10T21:45:11.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.601+0000 7f48abe27740 1 -- 172.21.15.7:0/2622216664 shutdown_connections 2024-09-10T21:45:11.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.601+0000 7f48abe27740 1 --2- 172.21.15.7:0/2622216664 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55d9a55f7c50 0x55d9a55fa040 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.601+0000 7f48abe27740 1 --2- 172.21.15.7:0/2622216664 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55ec200 0x55d9a55ec680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.601+0000 7f48abe27740 1 --2- 172.21.15.7:0/2622216664 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55d31c0 0x55d9a55b8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.601+0000 7f48abe27740 1 --2- 172.21.15.7:0/2622216664 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55c20c0 0x55d9a55aad50 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.601+0000 7f48abe27740 1 -- 172.21.15.7:0/2622216664 >> 172.21.15.7:0/2622216664 conn(0x55d9a55c17a0 msgr2=0x55d9a5590f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:11.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.601+0000 7f4887400640 0 client.5925 ms_handle_reset on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:11.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.601+0000 7f48abe27740 1 -- 172.21.15.7:0/2622216664 shutdown_connections 2024-09-10T21:45:11.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.601+0000 7f48abe27740 1 -- 172.21.15.7:0/2622216664 wait complete. 2024-09-10T21:45:11.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.601+0000 7f48abe27740 20 client.5925 trim_cache size 0 max 16384 2024-09-10T21:45:11.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.615+0000 7f48abe27740 1 Processor -- start 2024-09-10T21:45:11.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.615+0000 7f48abe27740 1 -- start start 2024-09-10T21:45:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.616+0000 7f48abe27740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55c20c0 0x55d9a55973d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.616+0000 7f48abe27740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55d31c0 0x55d9a5597910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.616+0000 7f48abe27740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55ec200 0x55d9a55f5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.616+0000 7f48abe27740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f48a40ceaa0 con 0x55d9a55ec200 2024-09-10T21:45:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.616+0000 7f48abe27740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f48a41f6a90 con 0x55d9a55d31c0 2024-09-10T21:45:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.616+0000 7f48abe27740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f48a414c3b0 con 0x55d9a55c20c0 2024-09-10T21:45:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.617+0000 7f48a2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55ec200 0x55d9a55f5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.617+0000 7f48a2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55ec200 0x55d9a55f5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42104/0 (socket says 172.21.15.7:42104) 2024-09-10T21:45:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.617+0000 7f48a2800640 1 -- 172.21.15.7:0/4247009552 learned_addr learned my addr 172.21.15.7:0/4247009552 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.617+0000 7f4899400640 1 --2- 172.21.15.7:0/4247009552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55d31c0 0x55d9a5597910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.617+0000 7f48a1e00640 1 --2- 172.21.15.7:0/4247009552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55c20c0 0x55d9a55973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.617+0000 7f48a2800640 1 -- 172.21.15.7:0/4247009552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55c20c0 msgr2=0x55d9a55973d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:11.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.617+0000 7f48a2800640 1 --2- 172.21.15.7:0/4247009552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55c20c0 0x55d9a55973d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.617+0000 7f48a2800640 1 -- 172.21.15.7:0/4247009552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55d31c0 msgr2=0x55d9a5597910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:11.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.617+0000 7f48a2800640 1 --2- 172.21.15.7:0/4247009552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55d31c0 0x55d9a5597910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.617+0000 7f48a2800640 1 -- 172.21.15.7:0/4247009552 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9a5238660 con 0x55d9a55ec200 2024-09-10T21:45:11.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.617+0000 7f48a2800640 1 --2- 172.21.15.7:0/4247009552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55ec200 0x55d9a55f5650 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f489c013590 tx=0x7f489c013a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22217f1e81bc00a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:11.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.617+0000 7f4892000640 1 -- 172.21.15.7:0/4247009552 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c021070 con 0x55d9a55ec200 2024-09-10T21:45:11.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.618+0000 7f4892000640 1 -- 172.21.15.7:0/4247009552 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f489c011560 con 0x55d9a55ec200 2024-09-10T21:45:11.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.618+0000 7f4892000640 1 -- 172.21.15.7:0/4247009552 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489c011990 con 0x55d9a55ec200 2024-09-10T21:45:11.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.618+0000 7f48abe27740 1 -- 172.21.15.7:0/4247009552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55ec200 msgr2=0x55d9a55f5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:11.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.618+0000 7f48abe27740 1 --2- 172.21.15.7:0/4247009552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55ec200 0x55d9a55f5650 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f489c013590 tx=0x7f489c013a50 comp rx=0 tx=0).stop 2024-09-10T21:45:11.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.618+0000 7f48abe27740 1 -- 172.21.15.7:0/4247009552 shutdown_connections 2024-09-10T21:45:11.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.618+0000 7f48abe27740 1 --2- 172.21.15.7:0/4247009552 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55d31c0 0x55d9a5597910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.618+0000 7f48abe27740 1 --2- 172.21.15.7:0/4247009552 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55ec200 0x55d9a55f5650 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.618+0000 7f48abe27740 1 --2- 172.21.15.7:0/4247009552 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55c20c0 0x55d9a55973d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.618+0000 7f48abe27740 1 -- 172.21.15.7:0/4247009552 >> 172.21.15.7:0/4247009552 conn(0x55d9a55c17a0 msgr2=0x55d9a55b89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:11.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.618+0000 7f48abe27740 1 -- 172.21.15.7:0/4247009552 shutdown_connections 2024-09-10T21:45:11.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.618+0000 7f48abe27740 1 -- 172.21.15.7:0/4247009552 wait complete. 2024-09-10T21:45:11.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.619+0000 7f48abe27740 1 -- start start 2024-09-10T21:45:11.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.619+0000 7f48abe27740 1 Processor -- start 2024-09-10T21:45:11.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.620+0000 7f48abe27740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55c20c0 0x55d9a56567d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.620+0000 7f48abe27740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55d31c0 0x55d9a565c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.620+0000 7f48abe27740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55ec200 0x55d9a5660360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.620+0000 7f48abe27740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d9a55b91b0 con 0x55d9a55c20c0 2024-09-10T21:45:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.620+0000 7f48abe27740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d9a55ab290 con 0x55d9a55ec200 2024-09-10T21:45:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.620+0000 7f48abe27740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55d9a55b8070 con 0x55d9a55d31c0 2024-09-10T21:45:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.620+0000 7f48abe27740 10 client.0 _ll_register_callbacks cb 0x7f4894003a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:45:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.620+0000 7f48a1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55c20c0 0x55d9a56567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.620+0000 7f48a1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55c20c0 0x55d9a56567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42118/0 (socket says 172.21.15.7:42118) 2024-09-10T21:45:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.620+0000 7f48a1e00640 1 -- 172.21.15.7:0/1272781731 learned_addr learned my addr 172.21.15.7:0/1272781731 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.620+0000 7f4899400640 1 --2- 172.21.15.7:0/1272781731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55d31c0 0x55d9a565c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.621+0000 7f48a2800640 1 --2- 172.21.15.7:0/1272781731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55ec200 0x55d9a5660360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.621+0000 7f48a1e00640 1 -- 172.21.15.7:0/1272781731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55d31c0 msgr2=0x55d9a565c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.621+0000 7f48a1e00640 1 --2- 172.21.15.7:0/1272781731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55d31c0 0x55d9a565c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.621+0000 7f48a1e00640 1 -- 172.21.15.7:0/1272781731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55ec200 msgr2=0x55d9a5660360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.621+0000 7f48a1e00640 1 --2- 172.21.15.7:0/1272781731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55ec200 0x55d9a5660360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.621+0000 7f48a1e00640 1 -- 172.21.15.7:0/1272781731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9a54c5630 con 0x55d9a55c20c0 2024-09-10T21:45:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.621+0000 7f48a1e00640 1 --2- 172.21.15.7:0/1272781731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55c20c0 0x55d9a56567d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4894009690 tx=0x7f489403c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19902c302a84ee9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.621+0000 7f4887400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:45:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.621+0000 7f4887400640 1 -- 172.21.15.7:0/1272781731 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894061060 con 0x55d9a55c20c0 2024-09-10T21:45:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.621+0000 7f48abe27740 1 -- 172.21.15.7:0/1272781731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f489c011560 con 0x55d9a55c20c0 2024-09-10T21:45:11.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.621+0000 7f4887400640 1 -- 172.21.15.7:0/1272781731 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f4894052b70 con 0x55d9a55c20c0 2024-09-10T21:45:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.622+0000 7f4887400640 1 -- 172.21.15.7:0/1272781731 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4894067050 con 0x55d9a55c20c0 2024-09-10T21:45:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.622+0000 7f48abe27740 10 client.5931 fetch_fsmap learned FSMap version 36 2024-09-10T21:45:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.622+0000 7f48abe27740 1 -- 172.21.15.7:0/1272781731 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55d9a55ebf60 con 0x55d9a55c20c0 2024-09-10T21:45:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.622+0000 7f4887400640 1 -- 172.21.15.7:0/1272781731 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f489c011560 con 0x55d9a55c20c0 2024-09-10T21:45:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.622+0000 7f4887400640 1 -- 172.21.15.7:0/1272781731 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f489406a020 con 0x55d9a55c20c0 2024-09-10T21:45:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.622+0000 7f48abe27740 10 client.5931 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T21:45:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.622+0000 7f48abe27740 10 client.5931 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:45:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.622+0000 7f48abe27740 20 client.5931 populate_metadata read hostname 'smithi007' 2024-09-10T21:45:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.622+0000 7f48abe27740 1 --2- 172.21.15.7:0/1272781731 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55d9a55976f0 0x55d9a55eaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.622+0000 7f48abe27740 4 client.5931 mds_command: new command op to 235016 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-10T21:45:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.622+0000 7f48abe27740 1 -- 172.21.15.7:0/1272781731 --> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x55d9a55d63f0 con 0x55d9a55976f0 2024-09-10T21:45:11.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.622+0000 7f4899400640 1 --2- 172.21.15.7:0/1272781731 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55d9a55976f0 0x55d9a55eaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.623+0000 7f4887400640 10 client.5931 ms_handle_connect on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:11.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.623+0000 7f4899400640 1 --2- 172.21.15.7:0/1272781731 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55d9a55976f0 0x55d9a55eaa10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ae267a2e07d2b1 server_cookie=a268c04f8cb2f943 in_seq=0 out_seq=0 2024-09-10T21:45:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.623+0000 7f4887400640 1 -- 172.21.15.7:0/1272781731 <== mds.0 v2:172.21.15.7:6833/364473218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+141 (crc 0 0 0) 0x55d9a55d63f0 con 0x55d9a55976f0 2024-09-10T21:45:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.623+0000 7f4887400640 10 client.5931 handle_command_reply: tid=0 2024-09-10T21:45:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.623+0000 7f48abe27740 1 client.5931 shutdown 2024-09-10T21:45:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.623+0000 7f48abe27740 1 -- 172.21.15.7:0/1272781731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55c20c0 msgr2=0x55d9a56567d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.623+0000 7f48abe27740 1 --2- 172.21.15.7:0/1272781731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55c20c0 0x55d9a56567d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4894009690 tx=0x7f489403c000 comp rx=0 tx=0).stop 2024-09-10T21:45:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.623+0000 7f48abe27740 1 -- 172.21.15.7:0/1272781731 shutdown_connections 2024-09-10T21:45:11.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.623+0000 7f48abe27740 1 --2- 172.21.15.7:0/1272781731 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55d9a55976f0 0x55d9a55eaa10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.623+0000 7f48abe27740 1 --2- 172.21.15.7:0/1272781731 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d9a55ec200 0x55d9a5660360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.623+0000 7f48abe27740 1 --2- 172.21.15.7:0/1272781731 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d9a55d31c0 0x55d9a565c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.623+0000 7f48abe27740 1 --2- 172.21.15.7:0/1272781731 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d9a55c20c0 0x55d9a56567d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.623+0000 7f48abe27740 1 -- 172.21.15.7:0/1272781731 >> 172.21.15.7:0/1272781731 conn(0x55d9a55c17a0 msgr2=0x55d9a55b88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.623+0000 7f4887400640 0 client.5931 ms_handle_reset on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.624+0000 7f48abe27740 1 -- 172.21.15.7:0/1272781731 shutdown_connections 2024-09-10T21:45:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.624+0000 7f48abe27740 1 -- 172.21.15.7:0/1272781731 wait complete. 2024-09-10T21:45:11.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.624+0000 7f48abe27740 20 client.5931 trim_cache size 0 max 16384 2024-09-10T21:45:11.629 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:45:11.630 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-09-10T21:45:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-09-10T21:45:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1205259754, 2024-09-10T21:45:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627777, 2024-09-10T21:45:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/dir_test/file" 2024-09-10T21:45:11.630 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:11.630 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-09-10T21:45:11.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.624+0000 7f48a9000640 1 -- 172.21.15.7:0/1506701859 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4880077f70 msgr2=0x7f488007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:11.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.624+0000 7f48a9000640 1 --2- 172.21.15.7:0/1506701859 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4880077f70 0x7f488007a430 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4894033000 tx=0x7f4894004730 comp rx=0 tx=0).stop 2024-09-10T21:45:11.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.624+0000 7f48a9000640 1 -- 172.21.15.7:0/1506701859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48a4073e20 msgr2=0x7f48a40cc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:11.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.624+0000 7f48a9000640 1 --2- 172.21.15.7:0/1506701859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48a4073e20 0x7f48a40cc220 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f488c00c420 tx=0x7f488c00c8e0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.624+0000 7f48a9000640 1 -- 172.21.15.7:0/1506701859 shutdown_connections 2024-09-10T21:45:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.624+0000 7f48a9000640 1 --2- 172.21.15.7:0/1506701859 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4880077f70 0x7f488007a430 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.624+0000 7f48a9000640 1 --2- 172.21.15.7:0/1506701859 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f48a4159680 0x7f48a40cc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.624+0000 7f48a9000640 1 --2- 172.21.15.7:0/1506701859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f48a4073e20 0x7f48a40cc220 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.624+0000 7f48a9000640 1 --2- 172.21.15.7:0/1506701859 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48a406e580 0x7f48a41f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.624+0000 7f48a9000640 1 -- 172.21.15.7:0/1506701859 >> 172.21.15.7:0/1506701859 conn(0x7f48a40b16d0 msgr2=0x7f48a415bd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.625+0000 7f48a9000640 1 -- 172.21.15.7:0/1506701859 shutdown_connections 2024-09-10T21:45:11.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.625+0000 7f48a9000640 1 -- 172.21.15.7:0/1506701859 wait complete. 2024-09-10T21:45:11.642 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:45:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.800+0000 7f6553e00640 1 -- 172.21.15.7:0/389712795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6534004210 msgr2=0x7f6534004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.800+0000 7f6553e00640 1 --2- 172.21.15.7:0/389712795 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6534004210 0x7f6534004800 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f6538006ef0 tx=0x7f6534005830 comp rx=0 tx=0).stop 2024-09-10T21:45:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.800+0000 7f6551e00640 1 -- 172.21.15.7:0/389712795 reap_dead start 2024-09-10T21:45:11.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.801+0000 7f6553e00640 1 -- 172.21.15.7:0/389712795 shutdown_connections 2024-09-10T21:45:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.801+0000 7f6553e00640 1 -- 172.21.15.7:0/389712795 >> 172.21.15.7:0/389712795 conn(0x7f654c0b16d0 msgr2=0x7f654c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.801+0000 7f6553e00640 1 -- 172.21.15.7:0/389712795 shutdown_connections 2024-09-10T21:45:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.801+0000 7f6553e00640 1 -- 172.21.15.7:0/389712795 wait complete. 2024-09-10T21:45:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.802+0000 7f6553e00640 1 Processor -- start 2024-09-10T21:45:11.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.803+0000 7f6553e00640 1 -- start start 2024-09-10T21:45:11.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.803+0000 7f6553e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6534004210 0x7f654c15c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.803+0000 7f6553e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f654c15a710 0x7f654c0b0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.803+0000 7f6553e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f654c0b47f0 0x7f654c0b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.803+0000 7f6553e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f654c0cd280 con 0x7f654c15a710 2024-09-10T21:45:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.803+0000 7f6553e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f654c0cd100 con 0x7f6534004210 2024-09-10T21:45:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.803+0000 7f6553e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f654c06fcf0 con 0x7f654c0b47f0 2024-09-10T21:45:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.804+0000 7f6550a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f654c15a710 0x7f654c0b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.804+0000 7f6550a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f654c15a710 0x7f654c0b0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42128/0 (socket says 172.21.15.7:42128) 2024-09-10T21:45:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.804+0000 7f6550a00640 1 -- 172.21.15.7:0/1589706294 learned_addr learned my addr 172.21.15.7:0/1589706294 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.804+0000 7f6551400640 1 --2- 172.21.15.7:0/1589706294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6534004210 0x7f654c15c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.804+0000 7f6551e00640 1 --2- 172.21.15.7:0/1589706294 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f654c0b47f0 0x7f654c0b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.804+0000 7f6550a00640 1 -- 172.21.15.7:0/1589706294 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f654c0b47f0 msgr2=0x7f654c0b4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:11.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.804+0000 7f6550a00640 1 --2- 172.21.15.7:0/1589706294 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f654c0b47f0 0x7f654c0b4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.804+0000 7f6550a00640 1 -- 172.21.15.7:0/1589706294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6534004210 msgr2=0x7f654c15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.804+0000 7f6550a00640 1 --2- 172.21.15.7:0/1589706294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6534004210 0x7f654c15c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.804+0000 7f6550a00640 1 -- 172.21.15.7:0/1589706294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654c0c6ea0 con 0x7f654c15a710 2024-09-10T21:45:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.805+0000 7f6550a00640 1 --2- 172.21.15.7:0/1589706294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f654c15a710 0x7f654c0b0540 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f653800c290 tx=0x7f65380066a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.805+0000 7f6549600640 1 -- 172.21.15.7:0/1589706294 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65380037f0 con 0x7f654c15a710 2024-09-10T21:45:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.805+0000 7f6549600640 1 -- 172.21.15.7:0/1589706294 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f653803f070 con 0x7f654c15a710 2024-09-10T21:45:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.805+0000 7f6549600640 1 -- 172.21.15.7:0/1589706294 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6538003d00 con 0x7f654c15a710 2024-09-10T21:45:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.805+0000 7f6553e00640 1 -- 172.21.15.7:0/1589706294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f654c0c7180 con 0x7f654c15a710 2024-09-10T21:45:11.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.805+0000 7f6553e00640 1 -- 172.21.15.7:0/1589706294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f654c0c7740 con 0x7f654c15a710 2024-09-10T21:45:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.806+0000 7f6553e00640 1 -- 172.21.15.7:0/1589706294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f654c0c6c30 con 0x7f654c15a710 2024-09-10T21:45:11.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.807+0000 7f6549600640 1 -- 172.21.15.7:0/1589706294 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f653802f460 con 0x7f654c15a710 2024-09-10T21:45:11.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.808+0000 7f6549600640 1 --2- 172.21.15.7:0/1589706294 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f652c077f70 0x7f652c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:11.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.808+0000 7f6549600640 1 -- 172.21.15.7:0/1589706294 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f65380ba900 con 0x7f654c15a710 2024-09-10T21:45:11.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.809+0000 7f6551400640 1 --2- 172.21.15.7:0/1589706294 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f652c077f70 0x7f652c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:11.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.809+0000 7f6551400640 1 --2- 172.21.15.7:0/1589706294 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f652c077f70 0x7f652c07a430 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6540006fd0 tx=0x7f6540006b10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:11.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:11.812+0000 7f6549600640 1 -- 172.21.15.7:0/1589706294 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6538082d10 con 0x7f654c15a710 2024-09-10T21:45:12.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.048+0000 7f6553e00640 1 -- 172.21.15.7:0/1589706294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f654c0b51b0 con 0x7f654c15a710 2024-09-10T21:45:12.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.049+0000 7f6549600640 1 -- 172.21.15.7:0/1589706294 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f65380ba020 con 0x7f654c15a710 2024-09-10T21:45:12.050 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:45:12.050 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:45:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.051+0000 7f6553e00640 1 -- 172.21.15.7:0/1589706294 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f652c077f70 msgr2=0x7f652c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.051+0000 7f6553e00640 1 --2- 172.21.15.7:0/1589706294 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f652c077f70 0x7f652c07a430 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6540006fd0 tx=0x7f6540006b10 comp rx=0 tx=0).stop 2024-09-10T21:45:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.051+0000 7f6553e00640 1 -- 172.21.15.7:0/1589706294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f654c15a710 msgr2=0x7f654c0b0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.051+0000 7f6553e00640 1 --2- 172.21.15.7:0/1589706294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f654c15a710 0x7f654c0b0540 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f653800c290 tx=0x7f65380066a0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.051+0000 7f6553e00640 1 -- 172.21.15.7:0/1589706294 shutdown_connections 2024-09-10T21:45:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.051+0000 7f6553e00640 1 --2- 172.21.15.7:0/1589706294 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f652c077f70 0x7f652c07a430 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.051+0000 7f6553e00640 1 --2- 172.21.15.7:0/1589706294 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f654c0b47f0 0x7f654c0b4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.051+0000 7f6553e00640 1 --2- 172.21.15.7:0/1589706294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f654c15a710 0x7f654c0b0540 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.051+0000 7f6553e00640 1 --2- 172.21.15.7:0/1589706294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6534004210 0x7f654c15c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.051+0000 7f6553e00640 1 -- 172.21.15.7:0/1589706294 >> 172.21.15.7:0/1589706294 conn(0x7f654c0b16d0 msgr2=0x7f654c0724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.051+0000 7f6553e00640 1 -- 172.21.15.7:0/1589706294 shutdown_connections 2024-09-10T21:45:12.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.051+0000 7f6553e00640 1 -- 172.21.15.7:0/1589706294 wait complete. 2024-09-10T21:45:12.068 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:45:12.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:45:12.068 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub start / repair,recursive,force 2024-09-10T21:45:12.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.239+0000 7fb90a200640 1 -- 172.21.15.7:0/2983676812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb90415bce0 msgr2=0x7fb90406cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.239+0000 7fb90a200640 1 --2- 172.21.15.7:0/2983676812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb90415bce0 0x7fb90406cb20 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc005fd0 tx=0x7fb8fc02aa10 comp rx=0 tx=0).stop 2024-09-10T21:45:12.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.240+0000 7fb90a200640 1 -- 172.21.15.7:0/2983676812 shutdown_connections 2024-09-10T21:45:12.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.240+0000 7fb90a200640 1 --2- 172.21.15.7:0/2983676812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb90415bce0 0x7fb90406cb20 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.240+0000 7fb90a200640 1 --2- 172.21.15.7:0/2983676812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb90415b900 0x7fb904070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.240+0000 7fb90a200640 1 -- 172.21.15.7:0/2983676812 >> 172.21.15.7:0/2983676812 conn(0x7fb9040b1c10 msgr2=0x7fb9040b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:12.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.240+0000 7fb90a200640 1 -- 172.21.15.7:0/2983676812 shutdown_connections 2024-09-10T21:45:12.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.240+0000 7fb90a200640 1 -- 172.21.15.7:0/2983676812 wait complete. 2024-09-10T21:45:12.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.241+0000 7fb90a200640 1 Processor -- start 2024-09-10T21:45:12.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.242+0000 7fb90a200640 1 -- start start 2024-09-10T21:45:12.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.242+0000 7fb90a200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9040ace80 0x7fb9041f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.242+0000 7fb90a200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb90415b900 0x7fb9041f4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.242+0000 7fb90a200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb90415bce0 0x7fb9040b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.243+0000 7fb90a200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb9040aed00 con 0x7fb90415b900 2024-09-10T21:45:12.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.243+0000 7fb90a200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb9040aeb80 con 0x7fb9040ace80 2024-09-10T21:45:12.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.243+0000 7fb90a200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fb9040aee80 con 0x7fb90415bce0 2024-09-10T21:45:12.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.243+0000 7fb902a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb90415b900 0x7fb9041f4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.243+0000 7fb903400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9040ace80 0x7fb9041f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.243+0000 7fb902a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb90415b900 0x7fb9041f4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42148/0 (socket says 172.21.15.7:42148) 2024-09-10T21:45:12.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.243+0000 7fb902a00640 1 -- 172.21.15.7:0/1696180771 learned_addr learned my addr 172.21.15.7:0/1696180771 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:12.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.243+0000 7fb903e00640 1 --2- 172.21.15.7:0/1696180771 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb90415bce0 0x7fb9040b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.243+0000 7fb902a00640 1 -- 172.21.15.7:0/1696180771 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb90415bce0 msgr2=0x7fb9040b0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.243+0000 7fb902a00640 1 --2- 172.21.15.7:0/1696180771 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb90415bce0 0x7fb9040b0c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.243+0000 7fb902a00640 1 -- 172.21.15.7:0/1696180771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9040ace80 msgr2=0x7fb9041f8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.243+0000 7fb902a00640 1 --2- 172.21.15.7:0/1696180771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9040ace80 0x7fb9041f8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.243+0000 7fb902a00640 1 -- 172.21.15.7:0/1696180771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9041f5550 con 0x7fb90415b900 2024-09-10T21:45:12.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.244+0000 7fb902a00640 1 --2- 172.21.15.7:0/1696180771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb90415b900 0x7fb9041f4d30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00c450 tx=0x7fb8ec00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:12.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.244+0000 7fb8fb400640 1 -- 172.21.15.7:0/1696180771 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec0052d0 con 0x7fb90415b900 2024-09-10T21:45:12.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.244+0000 7fb8fb400640 1 -- 172.21.15.7:0/1696180771 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8ec005470 con 0x7fb90415b900 2024-09-10T21:45:12.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.244+0000 7fb8fb400640 1 -- 172.21.15.7:0/1696180771 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec012870 con 0x7fb90415b900 2024-09-10T21:45:12.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.244+0000 7fb90a200640 1 -- 172.21.15.7:0/1696180771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9041f5890 con 0x7fb90415b900 2024-09-10T21:45:12.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.244+0000 7fb90a200640 1 -- 172.21.15.7:0/1696180771 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9041f5270 con 0x7fb90415b900 2024-09-10T21:45:12.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.246+0000 7fb8fb400640 1 -- 172.21.15.7:0/1696180771 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb8ec0035f0 con 0x7fb90415b900 2024-09-10T21:45:12.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.247+0000 7fb8fb400640 1 --2- 172.21.15.7:0/1696180771 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb8e0077f70 0x7fb8e007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.247+0000 7fb8fb400640 1 -- 172.21.15.7:0/1696180771 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb8ec099850 con 0x7fb90415b900 2024-09-10T21:45:12.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.247+0000 7fb90d26c740 1 Processor -- start 2024-09-10T21:45:12.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.247+0000 7fb903400640 1 --2- 172.21.15.7:0/1696180771 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb8e0077f70 0x7fb8e007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.248+0000 7fb90d26c740 1 -- start start 2024-09-10T21:45:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.248+0000 7fb903400640 1 --2- 172.21.15.7:0/1696180771 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb8e0077f70 0x7fb8e007a430 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40058c0 tx=0x7fb8f400a2d0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.248+0000 7fb90d26c740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51e9cb0 0x55e5c51ea090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.248+0000 7fb90d26c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51a4e20 0x55e5c51e8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.248+0000 7fb90d26c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 0x55e5c51e8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.248+0000 7fb90d26c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e5c51167d0 con 0x55e5c51a53a0 2024-09-10T21:45:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.248+0000 7fb90d26c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e5c5105c00 con 0x55e5c51a4e20 2024-09-10T21:45:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.248+0000 7fb90d26c740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55e5c5109a10 con 0x55e5c51e9cb0 2024-09-10T21:45:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.249+0000 7fb903e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 0x55e5c51e8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.249+0000 7fb903e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 0x55e5c51e8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42162/0 (socket says 172.21.15.7:42162) 2024-09-10T21:45:12.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.249+0000 7fb903e00640 1 -- 172.21.15.7:0/2893724217 learned_addr learned my addr 172.21.15.7:0/2893724217 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:12.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.249+0000 7fb902a00640 1 --2- 172.21.15.7:0/2893724217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51a4e20 0x55e5c51e8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.249+0000 7fb903400640 1 --2- 172.21.15.7:0/2893724217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51e9cb0 0x55e5c51ea090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.249+0000 7fb903e00640 1 -- 172.21.15.7:0/2893724217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51e9cb0 msgr2=0x55e5c51ea090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:12.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.249+0000 7fb903e00640 1 --2- 172.21.15.7:0/2893724217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51e9cb0 0x55e5c51ea090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.249+0000 7fb903e00640 1 -- 172.21.15.7:0/2893724217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51a4e20 msgr2=0x55e5c51e8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:12.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.249+0000 7fb903e00640 1 --2- 172.21.15.7:0/2893724217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51a4e20 0x55e5c51e8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.249+0000 7fb903e00640 1 -- 172.21.15.7:0/2893724217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5c5093b50 con 0x55e5c51a53a0 2024-09-10T21:45:12.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.249+0000 7fb903e00640 1 --2- 172.21.15.7:0/2893724217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 0x55e5c51e8cd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc02b000 tx=0x7fb8fc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98368f48eb1a2dc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:12.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.250+0000 7fb8f3400640 1 -- 172.21.15.7:0/2893724217 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc047070 con 0x55e5c51a53a0 2024-09-10T21:45:12.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.250+0000 7fb8f3400640 1 -- 172.21.15.7:0/2893724217 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8fc02f670 con 0x55e5c51a53a0 2024-09-10T21:45:12.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.250+0000 7fb8f3400640 1 -- 172.21.15.7:0/2893724217 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8fc033070 con 0x55e5c51a53a0 2024-09-10T21:45:12.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.250+0000 7fb90d26c740 1 -- 172.21.15.7:0/2893724217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 msgr2=0x55e5c51e8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.250+0000 7fb90d26c740 1 --2- 172.21.15.7:0/2893724217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 0x55e5c51e8cd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fb8fc02b000 tx=0x7fb8fc004ab0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.250+0000 7fb90d26c740 1 -- 172.21.15.7:0/2893724217 shutdown_connections 2024-09-10T21:45:12.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.250+0000 7fb90d26c740 1 --2- 172.21.15.7:0/2893724217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51a4e20 0x55e5c51e8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.250+0000 7fb90d26c740 1 --2- 172.21.15.7:0/2893724217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 0x55e5c51e8cd0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.250+0000 7fb90d26c740 1 --2- 172.21.15.7:0/2893724217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51e9cb0 0x55e5c51ea090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.250+0000 7fb90d26c740 1 -- 172.21.15.7:0/2893724217 >> 172.21.15.7:0/2893724217 conn(0x55e5c51a03f0 msgr2=0x55e5c51a3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:12.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.251+0000 7fb90d26c740 1 -- 172.21.15.7:0/2893724217 shutdown_connections 2024-09-10T21:45:12.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.251+0000 7fb90d26c740 1 -- 172.21.15.7:0/2893724217 wait complete. 2024-09-10T21:45:12.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.251+0000 7fb90d26c740 1 -- start start 2024-09-10T21:45:12.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.251+0000 7fb90d26c740 1 Processor -- start 2024-09-10T21:45:12.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.252+0000 7fb90d26c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51a53a0 0x55e5c51bc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.252+0000 7fb90d26c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51e9cb0 0x55e5c51e8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.252+0000 7fb90d26c740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51b8c30 0x55e5c51b9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.252+0000 7fb90d26c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e5c4e76880 con 0x55e5c51e9cb0 2024-09-10T21:45:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.252+0000 7fb90d26c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e5c50ab4b0 con 0x55e5c51a53a0 2024-09-10T21:45:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.252+0000 7fb90d26c740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55e5c519ee10 con 0x55e5c51b8c30 2024-09-10T21:45:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.252+0000 7fb90d26c740 10 client.0 _ll_register_callbacks cb 0x55e5c50b1680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:45:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.253+0000 7fb902a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51e9cb0 0x55e5c51e8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.253+0000 7fb902a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51e9cb0 0x55e5c51e8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42166/0 (socket says 172.21.15.7:42166) 2024-09-10T21:45:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.253+0000 7fb902a00640 1 -- 172.21.15.7:0/2138411424 learned_addr learned my addr 172.21.15.7:0/2138411424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.253+0000 7fb903400640 1 --2- 172.21.15.7:0/2138411424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51a53a0 0x55e5c51bc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.253+0000 7fb903e00640 1 --2- 172.21.15.7:0/2138411424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51b8c30 0x55e5c51b9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.253+0000 7fb902a00640 1 -- 172.21.15.7:0/2138411424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51b8c30 msgr2=0x55e5c51b9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.253+0000 7fb902a00640 1 --2- 172.21.15.7:0/2138411424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51b8c30 0x55e5c51b9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.253+0000 7fb902a00640 1 -- 172.21.15.7:0/2138411424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51a53a0 msgr2=0x55e5c51bc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.253+0000 7fb902a00640 1 --2- 172.21.15.7:0/2138411424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51a53a0 0x55e5c51bc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.253+0000 7fb902a00640 1 -- 172.21.15.7:0/2138411424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5c4ee0940 con 0x55e5c51e9cb0 2024-09-10T21:45:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.253+0000 7fb903400640 1 --2- 172.21.15.7:0/2138411424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51a53a0 0x55e5c51bc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:45:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.253+0000 7fb903e00640 1 --2- 172.21.15.7:0/2138411424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51b8c30 0x55e5c51b9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:45:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.253+0000 7fb902a00640 1 --2- 172.21.15.7:0/2138411424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51e9cb0 0x55e5c51e8980 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec061960 tx=0x7fb8ec0618a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03a6d49b6d3ebe2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.253+0000 7fb8e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:45:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.253+0000 7fb8e7e00640 1 -- 172.21.15.7:0/2138411424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8ec06b0b0 con 0x55e5c51e9cb0 2024-09-10T21:45:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.253+0000 7fb8e7e00640 1 -- 172.21.15.7:0/2138411424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb8ec060ec0 con 0x55e5c51e9cb0 2024-09-10T21:45:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.253+0000 7fb90d26c740 1 -- 172.21.15.7:0/2138411424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8fc02f670 con 0x55e5c51e9cb0 2024-09-10T21:45:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.254+0000 7fb8e7e00640 1 -- 172.21.15.7:0/2138411424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb8ec0610c0 con 0x55e5c51e9cb0 2024-09-10T21:45:12.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.254+0000 7fb90d26c740 10 client.5952 fetch_fsmap learned FSMap version 36 2024-09-10T21:45:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.254+0000 7fb90d26c740 1 -- 172.21.15.7:0/2138411424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55e5c4f410d0 con 0x55e5c51e9cb0 2024-09-10T21:45:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.254+0000 7fb8e7e00640 1 -- 172.21.15.7:0/2138411424 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8fc02f670 con 0x55e5c51e9cb0 2024-09-10T21:45:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.254+0000 7fb8e7e00640 1 -- 172.21.15.7:0/2138411424 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7fb8ec05d5b0 con 0x55e5c51e9cb0 2024-09-10T21:45:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.254+0000 7fb90d26c740 10 client.5952 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T21:45:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.254+0000 7fb90d26c740 10 client.5952 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-10T21:45:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.254+0000 7fb90d26c740 20 client.5952 populate_metadata read hostname 'smithi007' 2024-09-10T21:45:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.254+0000 7fb90d26c740 1 --2- 172.21.15.7:0/2138411424 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55e5c51e1d60 0x55e5c51e2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.254+0000 7fb90d26c740 4 client.5952 mds_command: new command op to 235016 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:45:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.254+0000 7fb90d26c740 1 -- 172.21.15.7:0/2138411424 --> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5c5118720 con 0x55e5c51e1d60 2024-09-10T21:45:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.254+0000 7fb903400640 1 --2- 172.21.15.7:0/2138411424 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55e5c51e1d60 0x55e5c51e2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.255+0000 7fb903400640 1 --2- 172.21.15.7:0/2138411424 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55e5c51e1d60 0x55e5c51e2140 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4ef4954dfb4ca6b server_cookie=a1560dc8abf04823 in_seq=0 out_seq=0 2024-09-10T21:45:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.255+0000 7fb8e7e00640 10 client.5952 ms_handle_connect on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:12.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.255+0000 7fb8e7e00640 1 -- 172.21.15.7:0/2138411424 <== mds.0 v2:172.21.15.7:6833/364473218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e5c5118720 con 0x55e5c51e1d60 2024-09-10T21:45:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.255+0000 7fb8e7e00640 10 client.5952 handle_command_reply: tid=0 2024-09-10T21:45:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.256+0000 7fb90d26c740 1 client.5952 shutdown 2024-09-10T21:45:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.256+0000 7fb90d26c740 1 -- 172.21.15.7:0/2138411424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51e9cb0 msgr2=0x55e5c51e8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.256+0000 7fb90d26c740 1 --2- 172.21.15.7:0/2138411424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51e9cb0 0x55e5c51e8980 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec061960 tx=0x7fb8ec0618a0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.256+0000 7fb90d26c740 1 -- 172.21.15.7:0/2138411424 shutdown_connections 2024-09-10T21:45:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.256+0000 7fb90d26c740 1 --2- 172.21.15.7:0/2138411424 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55e5c51e1d60 0x55e5c51e2140 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.256+0000 7fb90d26c740 1 --2- 172.21.15.7:0/2138411424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51b8c30 0x55e5c51b9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.256+0000 7fb90d26c740 1 --2- 172.21.15.7:0/2138411424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51e9cb0 0x55e5c51e8980 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.256+0000 7fb90d26c740 1 --2- 172.21.15.7:0/2138411424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51a53a0 0x55e5c51bc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.256+0000 7fb8e7e00640 0 client.5952 ms_handle_reset on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.256+0000 7fb90d26c740 1 -- 172.21.15.7:0/2138411424 >> 172.21.15.7:0/2138411424 conn(0x55e5c51a4e20 msgr2=0x55e5c51cd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.256+0000 7fb90d26c740 1 -- 172.21.15.7:0/2138411424 shutdown_connections 2024-09-10T21:45:12.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.256+0000 7fb90d26c740 1 -- 172.21.15.7:0/2138411424 wait complete. 2024-09-10T21:45:12.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.256+0000 7fb90d26c740 20 client.5952 trim_cache size 0 max 16384 2024-09-10T21:45:12.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.274+0000 7fb90d26c740 1 Processor -- start 2024-09-10T21:45:12.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.275+0000 7fb90d26c740 1 -- start start 2024-09-10T21:45:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.275+0000 7fb90d26c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 0x55e5c51e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.275+0000 7fb90d26c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51b8c30 0x55e5c51e7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.275+0000 7fb90d26c740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51e9cb0 0x55e5c5201190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.275+0000 7fb90d26c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb9040b0430 con 0x55e5c51a53a0 2024-09-10T21:45:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.275+0000 7fb90d26c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb9040d0a80 con 0x55e5c51b8c30 2024-09-10T21:45:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.275+0000 7fb90d26c740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fb9040cfde0 con 0x55e5c51e9cb0 2024-09-10T21:45:12.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.276+0000 7fb903400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 0x55e5c51e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.276+0000 7fb903400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 0x55e5c51e6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42182/0 (socket says 172.21.15.7:42182) 2024-09-10T21:45:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.276+0000 7fb903400640 1 -- 172.21.15.7:0/3710470686 learned_addr learned my addr 172.21.15.7:0/3710470686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.276+0000 7fb902a00640 1 --2- 172.21.15.7:0/3710470686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51b8c30 0x55e5c51e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.276+0000 7fb903e00640 1 --2- 172.21.15.7:0/3710470686 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51e9cb0 0x55e5c5201190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.276+0000 7fb903400640 1 -- 172.21.15.7:0/3710470686 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51e9cb0 msgr2=0x55e5c5201190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.276+0000 7fb903400640 1 --2- 172.21.15.7:0/3710470686 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51e9cb0 0x55e5c5201190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.276+0000 7fb903400640 1 -- 172.21.15.7:0/3710470686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51b8c30 msgr2=0x55e5c51e7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.276+0000 7fb903400640 1 --2- 172.21.15.7:0/3710470686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51b8c30 0x55e5c51e7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.276+0000 7fb903400640 1 -- 172.21.15.7:0/3710470686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5c51e8cf0 con 0x55e5c51a53a0 2024-09-10T21:45:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.276+0000 7fb903e00640 1 --2- 172.21.15.7:0/3710470686 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51e9cb0 0x55e5c5201190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:45:12.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.276+0000 7fb903400640 1 --2- 172.21.15.7:0/3710470686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 0x55e5c51e6c10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x55e5c51e27d0 tx=0x7fb8f4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f68ca38b3f6fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:12.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.277+0000 7fb8f2a00640 1 -- 172.21.15.7:0/3710470686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4034070 con 0x55e5c51a53a0 2024-09-10T21:45:12.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.277+0000 7fb8f2a00640 1 -- 172.21.15.7:0/3710470686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8f402f460 con 0x55e5c51a53a0 2024-09-10T21:45:12.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.277+0000 7fb8f2a00640 1 -- 172.21.15.7:0/3710470686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4016070 con 0x55e5c51a53a0 2024-09-10T21:45:12.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.277+0000 7fb90d26c740 1 -- 172.21.15.7:0/3710470686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 msgr2=0x55e5c51e6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.277+0000 7fb90d26c740 1 --2- 172.21.15.7:0/3710470686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 0x55e5c51e6c10 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x55e5c51e27d0 tx=0x7fb8f4012000 comp rx=0 tx=0).stop 2024-09-10T21:45:12.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.277+0000 7fb90d26c740 1 -- 172.21.15.7:0/3710470686 shutdown_connections 2024-09-10T21:45:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.277+0000 7fb90d26c740 1 --2- 172.21.15.7:0/3710470686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51b8c30 0x55e5c51e7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.277+0000 7fb90d26c740 1 --2- 172.21.15.7:0/3710470686 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51e9cb0 0x55e5c5201190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.277+0000 7fb90d26c740 1 --2- 172.21.15.7:0/3710470686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 0x55e5c51e6c10 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.277+0000 7fb90d26c740 1 -- 172.21.15.7:0/3710470686 >> 172.21.15.7:0/3710470686 conn(0x55e5c51a4e20 msgr2=0x55e5c519d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.277+0000 7fb90d26c740 1 -- 172.21.15.7:0/3710470686 shutdown_connections 2024-09-10T21:45:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.277+0000 7fb90d26c740 1 -- 172.21.15.7:0/3710470686 wait complete. 2024-09-10T21:45:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.278+0000 7fb90d26c740 1 -- start start 2024-09-10T21:45:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.278+0000 7fb90d26c740 1 Processor -- start 2024-09-10T21:45:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.279+0000 7fb90d26c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 0x55e5c51e6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.279+0000 7fb90d26c740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51b8c30 0x55e5c51e7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.279+0000 7fb90d26c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51e9cb0 0x55e5c520b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.279+0000 7fb90d26c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb9040d0430 con 0x55e5c51a53a0 2024-09-10T21:45:12.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.279+0000 7fb90d26c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e5c51ba1d0 con 0x55e5c51e9cb0 2024-09-10T21:45:12.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.279+0000 7fb90d26c740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55e5c51ec4d0 con 0x55e5c51b8c30 2024-09-10T21:45:12.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.279+0000 7fb90d26c740 10 client.0 _ll_register_callbacks cb 0x7fb8ec061440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:45:12.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.279+0000 7fb903400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 0x55e5c51e6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.279+0000 7fb903400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 0x55e5c51e6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42190/0 (socket says 172.21.15.7:42190) 2024-09-10T21:45:12.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.279+0000 7fb903400640 1 -- 172.21.15.7:0/4082406597 learned_addr learned my addr 172.21.15.7:0/4082406597 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:12.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.280+0000 7fb902a00640 1 --2- 172.21.15.7:0/4082406597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51b8c30 0x55e5c51e7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.280+0000 7fb903e00640 1 --2- 172.21.15.7:0/4082406597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51e9cb0 0x55e5c520b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.280+0000 7fb903400640 1 -- 172.21.15.7:0/4082406597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51b8c30 msgr2=0x55e5c51e7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:12.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.280+0000 7fb903400640 1 --2- 172.21.15.7:0/4082406597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51b8c30 0x55e5c51e7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.280+0000 7fb903400640 1 -- 172.21.15.7:0/4082406597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51e9cb0 msgr2=0x55e5c520b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:12.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.280+0000 7fb903400640 1 --2- 172.21.15.7:0/4082406597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51e9cb0 0x55e5c520b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.280+0000 7fb903400640 1 -- 172.21.15.7:0/4082406597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5c4e457e0 con 0x55e5c51a53a0 2024-09-10T21:45:12.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.280+0000 7fb902a00640 1 --2- 172.21.15.7:0/4082406597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51b8c30 0x55e5c51e7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:45:12.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.280+0000 7fb903e00640 1 --2- 172.21.15.7:0/4082406597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51e9cb0 0x55e5c520b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:45:12.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.280+0000 7fb903400640 1 --2- 172.21.15.7:0/4082406597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 0x55e5c51e6c10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4032040 tx=0x7fb8f4005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8f47fb7d43406b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:12.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.280+0000 7fb8e7e00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:45:12.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.280+0000 7fb8e7e00640 1 -- 172.21.15.7:0/4082406597 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8f4034040 con 0x55e5c51a53a0 2024-09-10T21:45:12.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.280+0000 7fb8e7e00640 1 -- 172.21.15.7:0/4082406597 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb8f4005020 con 0x55e5c51a53a0 2024-09-10T21:45:12.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.280+0000 7fb90d26c740 1 -- 172.21.15.7:0/4082406597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8f402f460 con 0x55e5c51a53a0 2024-09-10T21:45:12.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.280+0000 7fb8e7e00640 1 -- 172.21.15.7:0/4082406597 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb8f4044050 con 0x55e5c51a53a0 2024-09-10T21:45:12.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.280+0000 7fb90d26c740 10 client.5958 fetch_fsmap learned FSMap version 36 2024-09-10T21:45:12.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.280+0000 7fb90d26c740 1 -- 172.21.15.7:0/4082406597 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x55e5c50d26b0 con 0x55e5c51a53a0 2024-09-10T21:45:12.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.281+0000 7fb8e7e00640 1 -- 172.21.15.7:0/4082406597 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8f402f460 con 0x55e5c51a53a0 2024-09-10T21:45:12.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.281+0000 7fb8e7e00640 1 -- 172.21.15.7:0/4082406597 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7fb8f4047020 con 0x55e5c51a53a0 2024-09-10T21:45:12.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.281+0000 7fb90d26c740 10 client.5958 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T21:45:12.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.281+0000 7fb90d26c740 10 client.5958 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-10T21:45:12.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.281+0000 7fb90d26c740 20 client.5958 populate_metadata read hostname 'smithi007' 2024-09-10T21:45:12.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.281+0000 7fb90d26c740 1 --2- 172.21.15.7:0/4082406597 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55e5c51b80b0 0x55e5c520b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.281+0000 7fb90d26c740 4 client.5958 mds_command: new command op to 235016 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}] 2024-09-10T21:45:12.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.281+0000 7fb90d26c740 1 -- 172.21.15.7:0/4082406597 --> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive", "force"]}) -- 0x55e5c51141d0 con 0x55e5c51b80b0 2024-09-10T21:45:12.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.281+0000 7fb902a00640 1 --2- 172.21.15.7:0/4082406597 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55e5c51b80b0 0x55e5c520b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.282+0000 7fb902a00640 1 --2- 172.21.15.7:0/4082406597 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55e5c51b80b0 0x55e5c520b890 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13ad759ff3553616 server_cookie=1a4ea596d9787f49 in_seq=0 out_seq=0 2024-09-10T21:45:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.282+0000 7fb8e7e00640 10 client.5958 ms_handle_connect on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.282+0000 7fb8e7e00640 1 -- 172.21.15.7:0/4082406597 <== mds.0 v2:172.21.15.7:6833/364473218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e5c51141d0 con 0x55e5c51b80b0 2024-09-10T21:45:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.282+0000 7fb8e7e00640 10 client.5958 handle_command_reply: tid=0 2024-09-10T21:45:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.282+0000 7fb90d26c740 1 client.5958 shutdown 2024-09-10T21:45:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.283+0000 7fb90d26c740 1 -- 172.21.15.7:0/4082406597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 msgr2=0x55e5c51e6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.283+0000 7fb90d26c740 1 --2- 172.21.15.7:0/4082406597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 0x55e5c51e6c10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb8f4032040 tx=0x7fb8f4005dd0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.283+0000 7fb90d26c740 1 -- 172.21.15.7:0/4082406597 shutdown_connections 2024-09-10T21:45:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.283+0000 7fb90d26c740 1 --2- 172.21.15.7:0/4082406597 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55e5c51b80b0 0x55e5c520b890 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.283+0000 7fb90d26c740 1 --2- 172.21.15.7:0/4082406597 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e5c51e9cb0 0x55e5c520b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.283+0000 7fb90d26c740 1 --2- 172.21.15.7:0/4082406597 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e5c51b8c30 0x55e5c51e7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.283+0000 7fb90d26c740 1 --2- 172.21.15.7:0/4082406597 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e5c51a53a0 0x55e5c51e6c10 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.283+0000 7fb90d26c740 1 -- 172.21.15.7:0/4082406597 >> 172.21.15.7:0/4082406597 conn(0x55e5c51a4e20 msgr2=0x55e5c519e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:12.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.283+0000 7fb8e7e00640 0 client.5958 ms_handle_reset on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:12.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.283+0000 7fb90d26c740 1 -- 172.21.15.7:0/4082406597 shutdown_connections 2024-09-10T21:45:12.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.283+0000 7fb90d26c740 1 -- 172.21.15.7:0/4082406597 wait complete. 2024-09-10T21:45:12.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.283+0000 7fb90d26c740 20 client.5958 trim_cache size 0 max 16384 2024-09-10T21:45:12.289 INFO:tasks.ceph.mds.a.smithi007.stderr:2024-09-10T21:45:12.283+0000 7fe3f0e00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 pv6 ap=4 DIRTYPARENT s=0 n(v0 rc2024-09-10T21:44:39.176338+0000 1=1+0)->n(v0 rc2024-09-10T21:44:39.176338+0000 1=1+0) (ifile lock->xlock w=1) (iquiesce lock w=1) cr={5596=0-4194304@1} caps={5596=pAsLsXs/-@6} | ptrwaiter=1 request=1 lock=3 caps=1 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x5608cb1c6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-10T21:45:12.290 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:45:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-09-10T21:45:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "34168865-5a8d-4f70-a439-66b4e769eba6", 2024-09-10T21:45:12.290 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-09-10T21:45:12.290 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:45:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.284+0000 7fb90a200640 1 -- 172.21.15.7:0/1696180771 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb8e0077f70 msgr2=0x7fb8e007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.284+0000 7fb90a200640 1 --2- 172.21.15.7:0/1696180771 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb8e0077f70 0x7fb8e007a430 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb8f40058c0 tx=0x7fb8f400a2d0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.284+0000 7fb90a200640 1 -- 172.21.15.7:0/1696180771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb90415b900 msgr2=0x7fb9041f4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.284+0000 7fb90a200640 1 --2- 172.21.15.7:0/1696180771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb90415b900 0x7fb9041f4d30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec00c450 tx=0x7fb8ec00c910 comp rx=0 tx=0).stop 2024-09-10T21:45:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.284+0000 7fb90a200640 1 -- 172.21.15.7:0/1696180771 shutdown_connections 2024-09-10T21:45:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.284+0000 7fb90a200640 1 --2- 172.21.15.7:0/1696180771 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb8e0077f70 0x7fb8e007a430 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.284+0000 7fb90a200640 1 --2- 172.21.15.7:0/1696180771 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb90415bce0 0x7fb9040b0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.284+0000 7fb90a200640 1 --2- 172.21.15.7:0/1696180771 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb90415b900 0x7fb9041f4d30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.284+0000 7fb90a200640 1 --2- 172.21.15.7:0/1696180771 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9040ace80 0x7fb9041f8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.284+0000 7fb90a200640 1 -- 172.21.15.7:0/1696180771 >> 172.21.15.7:0/1696180771 conn(0x7fb9040b1c10 msgr2=0x7fb904158ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.284+0000 7fb90a200640 1 -- 172.21.15.7:0/1696180771 shutdown_connections 2024-09-10T21:45:12.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.284+0000 7fb90a200640 1 -- 172.21.15.7:0/1696180771 wait complete. 2024-09-10T21:45:12.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:45:12.301 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-09-10T21:45:12.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.464+0000 7f6ef2600640 1 -- 172.21.15.7:0/1274061550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eec155fb0 msgr2=0x7f6eec1600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.464+0000 7f6ef2600640 1 --2- 172.21.15.7:0/1274061550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eec155fb0 0x7f6eec1600d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0b5c90 tx=0x7f6edc02a560 comp rx=0 tx=0).stop 2024-09-10T21:45:12.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.465+0000 7f6ef2600640 1 -- 172.21.15.7:0/1274061550 shutdown_connections 2024-09-10T21:45:12.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.465+0000 7f6ef2600640 1 --2- 172.21.15.7:0/1274061550 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6eec074280 0x7f6eec06e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.465+0000 7f6ef2600640 1 --2- 172.21.15.7:0/1274061550 >> v2:172.21.15.133:3300/0 conn(0x7f6eec06fcf0 0x7f6eec073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.465+0000 7f6ef2600640 1 --2- 172.21.15.7:0/1274061550 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eec155fb0 0x7f6eec1600d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.465+0000 7f6ef2600640 1 -- 172.21.15.7:0/1274061550 >> 172.21.15.7:0/1274061550 conn(0x7f6eec0b16d0 msgr2=0x7f6eec0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.465+0000 7f6ef2600640 1 -- 172.21.15.7:0/1274061550 shutdown_connections 2024-09-10T21:45:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.465+0000 7f6ef2600640 1 -- 172.21.15.7:0/1274061550 wait complete. 2024-09-10T21:45:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.466+0000 7f6ef2600640 1 Processor -- start 2024-09-10T21:45:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.466+0000 7f6ef2600640 1 -- start start 2024-09-10T21:45:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.467+0000 7f6ef2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eec06fcf0 0x7f6eec1f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.467+0000 7f6ef2600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6eec074280 0x7f6eec1f1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.467+0000 7f6ef2600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6eec155fb0 0x7f6eec0c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.467+0000 7f6ef2600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec0cbc80 con 0x7f6eec06fcf0 2024-09-10T21:45:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.467+0000 7f6ef2600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec06eac0 con 0x7f6eec155fb0 2024-09-10T21:45:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.467+0000 7f6ef2600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f6eec1f47a0 con 0x7f6eec074280 2024-09-10T21:45:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.468+0000 7f6eeb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eec06fcf0 0x7f6eec1f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.468+0000 7f6eeb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eec06fcf0 0x7f6eec1f7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42202/0 (socket says 172.21.15.7:42202) 2024-09-10T21:45:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.468+0000 7f6eeb400640 1 -- 172.21.15.7:0/1015422168 learned_addr learned my addr 172.21.15.7:0/1015422168 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.468+0000 7f6eeaa00640 1 --2- 172.21.15.7:0/1015422168 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6eec074280 0x7f6eec1f1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.468+0000 7f6eebe00640 1 --2- 172.21.15.7:0/1015422168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6eec155fb0 0x7f6eec0c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.468+0000 7f6eeb400640 1 -- 172.21.15.7:0/1015422168 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6eec074280 msgr2=0x7f6eec1f1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.468+0000 7f6eeb400640 1 --2- 172.21.15.7:0/1015422168 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6eec074280 0x7f6eec1f1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.468+0000 7f6eeb400640 1 -- 172.21.15.7:0/1015422168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6eec155fb0 msgr2=0x7f6eec0c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.468+0000 7f6eeb400640 1 --2- 172.21.15.7:0/1015422168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6eec155fb0 0x7f6eec0c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.468+0000 7f6eeb400640 1 -- 172.21.15.7:0/1015422168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eec1f8240 con 0x7f6eec06fcf0 2024-09-10T21:45:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.468+0000 7f6eebe00640 1 --2- 172.21.15.7:0/1015422168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6eec155fb0 0x7f6eec0c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:45:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.468+0000 7f6eeb400640 1 --2- 172.21.15.7:0/1015422168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eec06fcf0 0x7f6eec1f7c60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6edc034040 tx=0x7f6edc003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.469+0000 7f6ee3400640 1 -- 172.21.15.7:0/1015422168 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc03e030 con 0x7f6eec06fcf0 2024-09-10T21:45:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.469+0000 7f6ee3400640 1 -- 172.21.15.7:0/1015422168 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6edc003e00 con 0x7f6eec06fcf0 2024-09-10T21:45:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.469+0000 7f6ee3400640 1 -- 172.21.15.7:0/1015422168 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc035040 con 0x7f6eec06fcf0 2024-09-10T21:45:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.469+0000 7f6ef2600640 1 -- 172.21.15.7:0/1015422168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eec1f8520 con 0x7f6eec06fcf0 2024-09-10T21:45:12.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.469+0000 7f6ef2600640 1 -- 172.21.15.7:0/1015422168 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eec1f2010 con 0x7f6eec06fcf0 2024-09-10T21:45:12.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.471+0000 7f6ee3400640 1 -- 172.21.15.7:0/1015422168 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6edc0049c0 con 0x7f6eec06fcf0 2024-09-10T21:45:12.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.473+0000 7f6ee3400640 1 --2- 172.21.15.7:0/1015422168 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6ecc077f70 0x7f6ecc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.473+0000 7f6ee3400640 1 -- 172.21.15.7:0/1015422168 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f6edc0ba6c0 con 0x7f6eec06fcf0 2024-09-10T21:45:12.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.473+0000 7f6eeaa00640 1 --2- 172.21.15.7:0/1015422168 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6ecc077f70 0x7f6ecc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.473+0000 7f6ef55ec740 1 Processor -- start 2024-09-10T21:45:12.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.473+0000 7f6ef55ec740 1 -- start start 2024-09-10T21:45:12.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.474+0000 7f6eeaa00640 1 --2- 172.21.15.7:0/1015422168 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6ecc077f70 0x7f6ecc07a430 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0c9dc0 tx=0x7f6ed800b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:12.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.474+0000 7f6ef55ec740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df39d6130 0x560df39d6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.474+0000 7f6ef55ec740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df39912a0 0x560df39d4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.474+0000 7f6ef55ec740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df3991820 0x560df39d5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.474+0000 7f6ef55ec740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560df38f01c0 con 0x560df3991820 2024-09-10T21:45:12.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.474+0000 7f6ef55ec740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560df3903870 con 0x560df39d6130 2024-09-10T21:45:12.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.474+0000 7f6ef55ec740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x560df38f2ca0 con 0x560df39912a0 2024-09-10T21:45:12.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.474+0000 7f6eebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df3991820 0x560df39d5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.474+0000 7f6eebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df3991820 0x560df39d5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42208/0 (socket says 172.21.15.7:42208) 2024-09-10T21:45:12.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.474+0000 7f6eebe00640 1 -- 172.21.15.7:0/2916590514 learned_addr learned my addr 172.21.15.7:0/2916590514 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:12.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.475+0000 7f6eeb400640 1 --2- 172.21.15.7:0/2916590514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df39d6130 0x560df39d6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.475+0000 7f6eeaa00640 1 --2- 172.21.15.7:0/2916590514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df39912a0 0x560df39d4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.475+0000 7f6eebe00640 1 -- 172.21.15.7:0/2916590514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df39912a0 msgr2=0x560df39d4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.475+0000 7f6eebe00640 1 --2- 172.21.15.7:0/2916590514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df39912a0 0x560df39d4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.475+0000 7f6eebe00640 1 -- 172.21.15.7:0/2916590514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df39d6130 msgr2=0x560df39d6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.475+0000 7f6eebe00640 1 --2- 172.21.15.7:0/2916590514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df39d6130 0x560df39d6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.475+0000 7f6eebe00640 1 -- 172.21.15.7:0/2916590514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df36cb7d0 con 0x560df3991820 2024-09-10T21:45:12.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.475+0000 7f6eeaa00640 1 --2- 172.21.15.7:0/2916590514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df39912a0 0x560df39d4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:45:12.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.475+0000 7f6eebe00640 1 --2- 172.21.15.7:0/2916590514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df3991820 0x560df39d5150 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4009ef0 tx=0x7f6ee4010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90dd313f1025d959 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:12.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.475+0000 7f6ed7e00640 1 -- 172.21.15.7:0/2916590514 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4016070 con 0x560df3991820 2024-09-10T21:45:12.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.475+0000 7f6ed7e00640 1 -- 172.21.15.7:0/2916590514 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ee401b750 con 0x560df3991820 2024-09-10T21:45:12.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.475+0000 7f6ed7e00640 1 -- 172.21.15.7:0/2916590514 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee401bb00 con 0x560df3991820 2024-09-10T21:45:12.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.476+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2916590514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df3991820 msgr2=0x560df39d5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.476+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/2916590514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df3991820 0x560df39d5150 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4009ef0 tx=0x7f6ee4010740 comp rx=0 tx=0).stop 2024-09-10T21:45:12.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.476+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2916590514 shutdown_connections 2024-09-10T21:45:12.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.476+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/2916590514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df3991820 0x560df39d5150 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.476+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/2916590514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df39912a0 0x560df39d4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.476+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/2916590514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df39d6130 0x560df39d6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.476+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2916590514 >> 172.21.15.7:0/2916590514 conn(0x560df398c870 msgr2=0x560df3990290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:12.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.476+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2916590514 shutdown_connections 2024-09-10T21:45:12.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.476+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2916590514 wait complete. 2024-09-10T21:45:12.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.477+0000 7f6ef55ec740 1 -- start start 2024-09-10T21:45:12.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.477+0000 7f6ef55ec740 1 Processor -- start 2024-09-10T21:45:12.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.478+0000 7f6ef55ec740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df3991820 0x560df39d4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.478+0000 7f6ef55ec740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df39d6130 0x560df39a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.478+0000 7f6ef55ec740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df398fef0 0x560df39902f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.478+0000 7f6ef55ec740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560df3899410 con 0x560df398fef0 2024-09-10T21:45:12.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.478+0000 7f6ef55ec740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560df38f6ab0 con 0x560df39d6130 2024-09-10T21:45:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.478+0000 7f6ef55ec740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x560df3663740 con 0x560df3991820 2024-09-10T21:45:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.478+0000 7f6ef55ec740 10 client.0 _ll_register_callbacks cb 0x560df3876360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:45:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.478+0000 7f6eebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df398fef0 0x560df39902f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.478+0000 7f6eebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df398fef0 0x560df39902f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42212/0 (socket says 172.21.15.7:42212) 2024-09-10T21:45:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.478+0000 7f6eebe00640 1 -- 172.21.15.7:0/2031259793 learned_addr learned my addr 172.21.15.7:0/2031259793 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.478+0000 7f6eeb400640 1 --2- 172.21.15.7:0/2031259793 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df3991820 0x560df39d4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.479+0000 7f6eeaa00640 1 --2- 172.21.15.7:0/2031259793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df39d6130 0x560df39a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.479+0000 7f6eebe00640 1 -- 172.21.15.7:0/2031259793 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df3991820 msgr2=0x560df39d4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.479+0000 7f6eebe00640 1 --2- 172.21.15.7:0/2031259793 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df3991820 0x560df39d4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.479+0000 7f6eebe00640 1 -- 172.21.15.7:0/2031259793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df39d6130 msgr2=0x560df39a52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.479+0000 7f6eebe00640 1 --2- 172.21.15.7:0/2031259793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df39d6130 0x560df39a52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.479+0000 7f6eebe00640 1 -- 172.21.15.7:0/2031259793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df3654970 con 0x560df398fef0 2024-09-10T21:45:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.479+0000 7f6eeaa00640 1 --2- 172.21.15.7:0/2031259793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df39d6130 0x560df39a52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:45:12.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.479+0000 7f6eebe00640 1 --2- 172.21.15.7:0/2031259793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df398fef0 0x560df39902f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x560df39d77b0 tx=0x7f6ee401b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=560841b1980230a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.479+0000 7f6ed4c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:45:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.479+0000 7f6ed4c00640 1 -- 172.21.15.7:0/2031259793 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4016040 con 0x560df398fef0 2024-09-10T21:45:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.479+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2031259793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ee401b750 con 0x560df398fef0 2024-09-10T21:45:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.479+0000 7f6ed4c00640 1 -- 172.21.15.7:0/2031259793 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f6ee4004b20 con 0x560df398fef0 2024-09-10T21:45:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.480+0000 7f6ed4c00640 1 -- 172.21.15.7:0/2031259793 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6ee402e050 con 0x560df398fef0 2024-09-10T21:45:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.480+0000 7f6ef55ec740 10 client.5970 fetch_fsmap learned FSMap version 36 2024-09-10T21:45:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.480+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2031259793 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x560df36326a0 con 0x560df398fef0 2024-09-10T21:45:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.480+0000 7f6ed4c00640 1 -- 172.21.15.7:0/2031259793 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ee401b750 con 0x560df398fef0 2024-09-10T21:45:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.480+0000 7f6ed4c00640 1 -- 172.21.15.7:0/2031259793 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f6ee4031020 con 0x560df398fef0 2024-09-10T21:45:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.480+0000 7f6ef55ec740 10 client.5970 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T21:45:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.480+0000 7f6ef55ec740 10 client.5970 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-10T21:45:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.480+0000 7f6ef55ec740 20 client.5970 populate_metadata read hostname 'smithi007' 2024-09-10T21:45:12.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.480+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/2031259793 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x560df39b9e70 0x560df39ba2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.480+0000 7f6ef55ec740 4 client.5970 mds_command: new command op to 235016 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:45:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.480+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2031259793 --> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560df39057c0 con 0x560df39b9e70 2024-09-10T21:45:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.480+0000 7f6eeb400640 1 --2- 172.21.15.7:0/2031259793 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x560df39b9e70 0x560df39ba2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.481+0000 7f6eeb400640 1 --2- 172.21.15.7:0/2031259793 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x560df39b9e70 0x560df39ba2a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d192e3d1bf0db82a server_cookie=1063905aafa65c1f in_seq=0 out_seq=0 2024-09-10T21:45:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.481+0000 7f6ed4c00640 10 client.5970 ms_handle_connect on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.481+0000 7f6ed4c00640 1 -- 172.21.15.7:0/2031259793 <== mds.0 v2:172.21.15.7:6833/364473218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560df39057c0 con 0x560df39b9e70 2024-09-10T21:45:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.481+0000 7f6ed4c00640 10 client.5970 handle_command_reply: tid=0 2024-09-10T21:45:12.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.481+0000 7f6ef55ec740 1 client.5970 shutdown 2024-09-10T21:45:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.482+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2031259793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df398fef0 msgr2=0x560df39902f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.482+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/2031259793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df398fef0 0x560df39902f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x560df39d77b0 tx=0x7f6ee401b5a0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.482+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2031259793 shutdown_connections 2024-09-10T21:45:12.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.482+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/2031259793 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x560df39b9e70 0x560df39ba2a0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.482+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/2031259793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df39d6130 0x560df39a52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.482+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/2031259793 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df398fef0 0x560df39902f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.482+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/2031259793 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df3991820 0x560df39d4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.482+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2031259793 >> 172.21.15.7:0/2031259793 conn(0x560df39912a0 msgr2=0x560df39ced00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.482+0000 7f6ed4c00640 0 client.5970 ms_handle_reset on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.482+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2031259793 shutdown_connections 2024-09-10T21:45:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.482+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2031259793 wait complete. 2024-09-10T21:45:12.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.482+0000 7f6ef55ec740 20 client.5970 trim_cache size 0 max 16384 2024-09-10T21:45:12.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.501+0000 7f6ef55ec740 1 Processor -- start 2024-09-10T21:45:12.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.502+0000 7f6ef55ec740 1 -- start start 2024-09-10T21:45:12.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.502+0000 7f6ef55ec740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df398fef0 0x560df39d2b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.502+0000 7f6ef55ec740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df3991820 0x560df39d4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.502+0000 7f6ef55ec740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df39d6130 0x560df39d4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.502+0000 7f6ef55ec740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec201140 con 0x560df39d6130 2024-09-10T21:45:12.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.502+0000 7f6ef55ec740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec0c5a20 con 0x560df398fef0 2024-09-10T21:45:12.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.502+0000 7f6ef55ec740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f6eec0b5910 con 0x560df3991820 2024-09-10T21:45:12.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.503+0000 7f6eebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df39d6130 0x560df39d4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.503+0000 7f6eebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df39d6130 0x560df39d4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42224/0 (socket says 172.21.15.7:42224) 2024-09-10T21:45:12.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.503+0000 7f6eebe00640 1 -- 172.21.15.7:0/85733894 learned_addr learned my addr 172.21.15.7:0/85733894 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:12.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.503+0000 7f6eebe00640 1 -- 172.21.15.7:0/85733894 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df3991820 msgr2=0x560df39d4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:12.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.503+0000 7f6eebe00640 1 --2- 172.21.15.7:0/85733894 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df3991820 0x560df39d4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.503+0000 7f6eebe00640 1 -- 172.21.15.7:0/85733894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df398fef0 msgr2=0x560df39d2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:12.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.503+0000 7f6eebe00640 1 --2- 172.21.15.7:0/85733894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df398fef0 0x560df39d2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.503+0000 7f6eebe00640 1 -- 172.21.15.7:0/85733894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560df39d72e0 con 0x560df39d6130 2024-09-10T21:45:12.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.503+0000 7f6eebe00640 1 --2- 172.21.15.7:0/85733894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df39d6130 0x560df39d4b40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4005fd0 tx=0x7f6ee40093c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d7199096426db92 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:12.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.503+0000 7f6ed7400640 1 -- 172.21.15.7:0/85733894 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4016040 con 0x560df39d6130 2024-09-10T21:45:12.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.504+0000 7f6ed7400640 1 -- 172.21.15.7:0/85733894 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ee401bc90 con 0x560df39d6130 2024-09-10T21:45:12.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.504+0000 7f6ed7400640 1 -- 172.21.15.7:0/85733894 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4005130 con 0x560df39d6130 2024-09-10T21:45:12.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.504+0000 7f6ef55ec740 1 -- 172.21.15.7:0/85733894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df39d6130 msgr2=0x560df39d4b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.504+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/85733894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df39d6130 0x560df39d4b40 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4005fd0 tx=0x7f6ee40093c0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.504+0000 7f6ef55ec740 1 -- 172.21.15.7:0/85733894 shutdown_connections 2024-09-10T21:45:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.504+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/85733894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df39d6130 0x560df39d4b40 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.504+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/85733894 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df3991820 0x560df39d4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.504+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/85733894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df398fef0 0x560df39d2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.504+0000 7f6ef55ec740 1 -- 172.21.15.7:0/85733894 >> 172.21.15.7:0/85733894 conn(0x560df39912a0 msgr2=0x560df398a6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.504+0000 7f6ef55ec740 1 -- 172.21.15.7:0/85733894 shutdown_connections 2024-09-10T21:45:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.504+0000 7f6ef55ec740 1 -- 172.21.15.7:0/85733894 wait complete. 2024-09-10T21:45:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.505+0000 7f6ef55ec740 1 -- start start 2024-09-10T21:45:12.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.505+0000 7f6ef55ec740 1 Processor -- start 2024-09-10T21:45:12.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.506+0000 7f6ef55ec740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df398fef0 0x560df3a4c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.506+0000 7f6ef55ec740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df3991820 0x560df39f3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.506+0000 7f6ef55ec740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df39d6130 0x560df39d1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.506+0000 7f6ef55ec740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6eec1f5370 con 0x560df398fef0 2024-09-10T21:45:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.506+0000 7f6ef55ec740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560df39d2500 con 0x560df3991820 2024-09-10T21:45:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.506+0000 7f6ef55ec740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x560df39f3fd0 con 0x560df39d6130 2024-09-10T21:45:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.506+0000 7f6ef55ec740 10 client.0 _ll_register_callbacks cb 0x7f6edc005b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:45:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.506+0000 7f6eeb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df398fef0 0x560df3a4c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.506+0000 7f6eeb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df398fef0 0x560df3a4c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:42230/0 (socket says 172.21.15.7:42230) 2024-09-10T21:45:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.506+0000 7f6eeb400640 1 -- 172.21.15.7:0/2335805483 learned_addr learned my addr 172.21.15.7:0/2335805483 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.507+0000 7f6eeaa00640 1 --2- 172.21.15.7:0/2335805483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df3991820 0x560df39f3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.507+0000 7f6eebe00640 1 --2- 172.21.15.7:0/2335805483 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df39d6130 0x560df39d1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.507+0000 7f6eeb400640 1 -- 172.21.15.7:0/2335805483 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df39d6130 msgr2=0x560df39d1740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.507+0000 7f6eeb400640 1 --2- 172.21.15.7:0/2335805483 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df39d6130 0x560df39d1740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.507+0000 7f6eeb400640 1 -- 172.21.15.7:0/2335805483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df3991820 msgr2=0x560df39f3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:12.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.507+0000 7f6eeb400640 1 --2- 172.21.15.7:0/2335805483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df3991820 0x560df39f3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.507+0000 7f6eeb400640 1 -- 172.21.15.7:0/2335805483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560df39f9800 con 0x560df398fef0 2024-09-10T21:45:12.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.507+0000 7f6eeb400640 1 --2- 172.21.15.7:0/2335805483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df398fef0 0x560df3a4c740 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6edc082860 tx=0x7f6edc0ba040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59b470e4c3ef33cf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:12.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.507+0000 7f6ed4c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:45:12.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.507+0000 7f6ed4c00640 1 -- 172.21.15.7:0/2335805483 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6edc048040 con 0x560df398fef0 2024-09-10T21:45:12.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.507+0000 7f6ed4c00640 1 -- 172.21.15.7:0/2335805483 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f6edc099c50 con 0x560df398fef0 2024-09-10T21:45:12.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.507+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2335805483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6ee401bc90 con 0x560df398fef0 2024-09-10T21:45:12.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.508+0000 7f6ed4c00640 1 -- 172.21.15.7:0/2335805483 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f6edc087390 con 0x560df398fef0 2024-09-10T21:45:12.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.508+0000 7f6ef55ec740 10 client.5976 fetch_fsmap learned FSMap version 36 2024-09-10T21:45:12.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.508+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2335805483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x560df39e5060 con 0x560df398fef0 2024-09-10T21:45:12.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.508+0000 7f6ed4c00640 1 -- 172.21.15.7:0/2335805483 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6ee401bc90 con 0x560df398fef0 2024-09-10T21:45:12.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.508+0000 7f6ed4c00640 1 -- 172.21.15.7:0/2335805483 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f6edc08bb20 con 0x560df398fef0 2024-09-10T21:45:12.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.508+0000 7f6ef55ec740 10 client.5976 fetch_fsmap finished waiting for FSMap version 36 2024-09-10T21:45:12.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.508+0000 7f6ef55ec740 10 client.5976 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-10T21:45:12.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.508+0000 7f6ef55ec740 20 client.5976 populate_metadata read hostname 'smithi007' 2024-09-10T21:45:12.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.508+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/2335805483 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x560df39d2a10 0x560df39d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:12.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.508+0000 7f6ef55ec740 4 client.5976 mds_command: new command op to 235016 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:45:12.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.508+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2335805483 --> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560df3901270 con 0x560df39d2a10 2024-09-10T21:45:12.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.508+0000 7f6eeaa00640 1 --2- 172.21.15.7:0/2335805483 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x560df39d2a10 0x560df39d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.509+0000 7f6eeaa00640 1 --2- 172.21.15.7:0/2335805483 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x560df39d2a10 0x560df39d1c80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6edc4358c2237b0a server_cookie=a6d1b1dc06a3b5e6 in_seq=0 out_seq=0 2024-09-10T21:45:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.509+0000 7f6ed4c00640 10 client.5976 ms_handle_connect on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.509+0000 7f6ed4c00640 1 -- 172.21.15.7:0/2335805483 <== mds.0 v2:172.21.15.7:6833/364473218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+861 (crc 0 0 0) 0x560df3901270 con 0x560df39d2a10 2024-09-10T21:45:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.509+0000 7f6ed4c00640 10 client.5976 handle_command_reply: tid=0 2024-09-10T21:45:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.509+0000 7f6ef55ec740 1 client.5976 shutdown 2024-09-10T21:45:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.509+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2335805483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df398fef0 msgr2=0x560df3a4c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.509+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/2335805483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df398fef0 0x560df3a4c740 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6edc082860 tx=0x7f6edc0ba040 comp rx=0 tx=0).stop 2024-09-10T21:45:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.509+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2335805483 shutdown_connections 2024-09-10T21:45:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.509+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/2335805483 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x560df39d2a10 0x560df39d1c80 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.509+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/2335805483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560df3991820 0x560df39f3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.509+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/2335805483 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560df39d6130 0x560df39d1740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.509+0000 7f6ef55ec740 1 --2- 172.21.15.7:0/2335805483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560df398fef0 0x560df3a4c740 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.509+0000 7f6ed4c00640 0 client.5976 ms_handle_reset on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.509+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2335805483 >> 172.21.15.7:0/2335805483 conn(0x560df39912a0 msgr2=0x560df398a590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.509+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2335805483 shutdown_connections 2024-09-10T21:45:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.510+0000 7f6ef55ec740 1 -- 172.21.15.7:0/2335805483 wait complete. 2024-09-10T21:45:12.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.510+0000 7f6ef55ec740 20 client.5976 trim_cache size 0 max 16384 2024-09-10T21:45:12.514 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:45:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:45:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-09-10T21:45:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "34168865-5a8d-4f70-a439-66b4e769eba6": { 2024-09-10T21:45:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-09-10T21:45:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "34168865-5a8d-4f70-a439-66b4e769eba6", 2024-09-10T21:45:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,repair,force" 2024-09-10T21:45:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:45:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "34168865-5a8d-4f70-a439-66b4e769eba6": { 2024-09-10T21:45:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:45:12.282824+0000", 2024-09-10T21:45:12.515 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-09-10T21:45:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:45:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:45:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:45:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:45:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "cabd7ddd-fca9-48c5-a921-5e626bb8f21c": { 2024-09-10T21:45:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:44:40.510317+0000", 2024-09-10T21:45:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-09-10T21:45:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:45:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:45:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:45:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:45:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:12.516 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:12.517 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:45:12.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.510+0000 7f6ef2600640 1 -- 172.21.15.7:0/1015422168 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6ecc077f70 msgr2=0x7f6ecc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.510+0000 7f6ef2600640 1 --2- 172.21.15.7:0/1015422168 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6ecc077f70 0x7f6ecc07a430 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f6eec0c9dc0 tx=0x7f6ed800b390 comp rx=0 tx=0).stop 2024-09-10T21:45:12.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.510+0000 7f6ef2600640 1 -- 172.21.15.7:0/1015422168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eec06fcf0 msgr2=0x7f6eec1f7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:12.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.510+0000 7f6ef2600640 1 --2- 172.21.15.7:0/1015422168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eec06fcf0 0x7f6eec1f7c60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6edc034040 tx=0x7f6edc003910 comp rx=0 tx=0).stop 2024-09-10T21:45:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.510+0000 7f6ef2600640 1 -- 172.21.15.7:0/1015422168 shutdown_connections 2024-09-10T21:45:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.510+0000 7f6ef2600640 1 --2- 172.21.15.7:0/1015422168 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6ecc077f70 0x7f6ecc07a430 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.510+0000 7f6ef2600640 1 --2- 172.21.15.7:0/1015422168 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6eec155fb0 0x7f6eec0c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.510+0000 7f6ef2600640 1 --2- 172.21.15.7:0/1015422168 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6eec074280 0x7f6eec1f1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.510+0000 7f6ef2600640 1 --2- 172.21.15.7:0/1015422168 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6eec06fcf0 0x7f6eec1f7c60 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.510+0000 7f6ef2600640 1 -- 172.21.15.7:0/1015422168 >> 172.21.15.7:0/1015422168 conn(0x7f6eec0b16d0 msgr2=0x7f6eec15e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.510+0000 7f6ef2600640 1 -- 172.21.15.7:0/1015422168 shutdown_connections 2024-09-10T21:45:12.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:12.511+0000 7f6ef2600640 1 -- 172.21.15.7:0/1015422168 wait complete. 2024-09-10T21:45:12.527 INFO:tasks.cephfs.filesystem:scrub status for tag:34168865-5a8d-4f70-a439-66b4e769eba6 - {'path': '/', 'tag': '34168865-5a8d-4f70-a439-66b4e769eba6', 'options': 'recursive,repair,force'} 2024-09-10T21:45:30.284 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:45:30.289 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:45:42.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:45:42.528 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.4:0 scrub status 2024-09-10T21:45:42.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.702+0000 7f0ead000640 1 -- 172.21.15.7:0/2374065085 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ea806fcf0 msgr2=0x7f0ea8160100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:42.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.702+0000 7f0ead000640 1 --2- 172.21.15.7:0/2374065085 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ea806fcf0 0x7f0ea8160100 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009be0 tx=0x7f0ea002a920 comp rx=0 tx=0).stop 2024-09-10T21:45:42.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.702+0000 7f0ead000640 1 -- 172.21.15.7:0/2374065085 shutdown_connections 2024-09-10T21:45:42.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.702+0000 7f0ead000640 1 --2- 172.21.15.7:0/2374065085 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ea806fcf0 0x7f0ea8160100 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.702+0000 7f0ead000640 1 -- 172.21.15.7:0/2374065085 >> 172.21.15.7:0/2374065085 conn(0x7f0ea80b16d0 msgr2=0x7f0ea80b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:42.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.702+0000 7f0ead000640 1 -- 172.21.15.7:0/2374065085 shutdown_connections 2024-09-10T21:45:42.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.702+0000 7f0ead000640 1 -- 172.21.15.7:0/2374065085 wait complete. 2024-09-10T21:45:42.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.703+0000 7f0ead000640 1 Processor -- start 2024-09-10T21:45:42.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.703+0000 7f0ead000640 1 -- start start 2024-09-10T21:45:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.704+0000 7f0ead000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ea806fcf0 0x7f0ea814dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.704+0000 7f0ead000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ea8074090 0x7f0ea80b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.704+0000 7f0ead000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea8156290 0x7f0ea81f87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.704+0000 7f0ead000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea80cad10 con 0x7f0ea8074090 2024-09-10T21:45:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.704+0000 7f0ead000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea80cab90 con 0x7f0ea8156290 2024-09-10T21:45:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.704+0000 7f0ead000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f0ea80cae90 con 0x7f0ea806fcf0 2024-09-10T21:45:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.705+0000 7f0e9d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ea8074090 0x7f0ea80b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.705+0000 7f0e9d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ea8074090 0x7f0ea80b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43688/0 (socket says 172.21.15.7:43688) 2024-09-10T21:45:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.705+0000 7f0e9d400640 1 -- 172.21.15.7:0/4144153842 learned_addr learned my addr 172.21.15.7:0/4144153842 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.705+0000 7f0ea6800640 1 --2- 172.21.15.7:0/4144153842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea8156290 0x7f0ea81f87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.705+0000 7f0ea5e00640 1 --2- 172.21.15.7:0/4144153842 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ea806fcf0 0x7f0ea814dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.705+0000 7f0e9d400640 1 -- 172.21.15.7:0/4144153842 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ea806fcf0 msgr2=0x7f0ea814dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.705+0000 7f0e9d400640 1 --2- 172.21.15.7:0/4144153842 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ea806fcf0 0x7f0ea814dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.705+0000 7f0e9d400640 1 -- 172.21.15.7:0/4144153842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea8156290 msgr2=0x7f0ea81f87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:42.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.705+0000 7f0e9d400640 1 --2- 172.21.15.7:0/4144153842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea8156290 0x7f0ea81f87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.705+0000 7f0e9d400640 1 -- 172.21.15.7:0/4144153842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ea81f8d30 con 0x7f0ea8074090 2024-09-10T21:45:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.705+0000 7f0e9d400640 1 --2- 172.21.15.7:0/4144153842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ea8074090 0x7f0ea80b41d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000c400 tx=0x7f0e9000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.706+0000 7f0e9ea00640 1 -- 172.21.15.7:0/4144153842 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90011790 con 0x7f0ea8074090 2024-09-10T21:45:42.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.706+0000 7f0e9ea00640 1 -- 172.21.15.7:0/4144153842 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e90017070 con 0x7f0ea8074090 2024-09-10T21:45:42.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.706+0000 7f0ead000640 1 -- 172.21.15.7:0/4144153842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ea80b4920 con 0x7f0ea8074090 2024-09-10T21:45:42.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.706+0000 7f0e9ea00640 1 -- 172.21.15.7:0/4144153842 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90011cc0 con 0x7f0ea8074090 2024-09-10T21:45:42.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.706+0000 7f0ead000640 1 -- 172.21.15.7:0/4144153842 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ea81f7ff0 con 0x7f0ea8074090 2024-09-10T21:45:42.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.708+0000 7f0e9ea00640 1 -- 172.21.15.7:0/4144153842 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0e9001c460 con 0x7f0ea8074090 2024-09-10T21:45:42.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.709+0000 7f0e9ea00640 1 --2- 172.21.15.7:0/4144153842 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0e84077f70 0x7f0e8407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.709+0000 7f0e9ea00640 1 -- 172.21.15.7:0/4144153842 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f0e90099990 con 0x7f0ea8074090 2024-09-10T21:45:42.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.709+0000 7f0eafe3b740 1 Processor -- start 2024-09-10T21:45:42.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.710+0000 7f0eafe3b740 1 -- start start 2024-09-10T21:45:42.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.710+0000 7f0ea5e00640 1 --2- 172.21.15.7:0/4144153842 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0e84077f70 0x7f0e8407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.710+0000 7f0eafe3b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef6130 0x55acd8ef6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.710+0000 7f0eafe3b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8eb12a0 0x55acd8ef4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.710+0000 7f0eafe3b740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8eb1820 0x55acd8ef5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.710+0000 7f0eafe3b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55acd8e101c0 con 0x55acd8eb12a0 2024-09-10T21:45:42.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.710+0000 7f0eafe3b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55acd8e23870 con 0x55acd8ef6130 2024-09-10T21:45:42.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.710+0000 7f0eafe3b740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55acd8e12ca0 con 0x55acd8eb1820 2024-09-10T21:45:42.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.710+0000 7f0ea5e00640 1 --2- 172.21.15.7:0/4144153842 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0e84077f70 0x7f0e8407a430 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0e98007fd0 tx=0x7f0e98009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:42.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.711+0000 7f0e9d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8eb12a0 0x55acd8ef4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.711+0000 7f0e9d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8eb12a0 0x55acd8ef4c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43696/0 (socket says 172.21.15.7:43696) 2024-09-10T21:45:42.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.711+0000 7f0e9d400640 1 -- 172.21.15.7:0/3939238258 learned_addr learned my addr 172.21.15.7:0/3939238258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:42.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.711+0000 7f0ea5e00640 1 --2- 172.21.15.7:0/3939238258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef6130 0x55acd8ef6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.711+0000 7f0ea6800640 1 --2- 172.21.15.7:0/3939238258 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8eb1820 0x55acd8ef5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.711+0000 7f0e9d400640 1 -- 172.21.15.7:0/3939238258 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8eb1820 msgr2=0x55acd8ef5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:42.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.711+0000 7f0e9d400640 1 --2- 172.21.15.7:0/3939238258 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8eb1820 0x55acd8ef5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.711+0000 7f0e9d400640 1 -- 172.21.15.7:0/3939238258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef6130 msgr2=0x55acd8ef6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:42.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.711+0000 7f0e9d400640 1 --2- 172.21.15.7:0/3939238258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef6130 0x55acd8ef6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.711+0000 7f0e9d400640 1 -- 172.21.15.7:0/3939238258 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acd8beb7d0 con 0x55acd8eb12a0 2024-09-10T21:45:42.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.711+0000 7f0e9d400640 1 --2- 172.21.15.7:0/3939238258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8eb12a0 0x55acd8ef4c10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0e90061e80 tx=0x7f0e90061790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65584ca97ac4a2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:42.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.712+0000 7f0e96a00640 1 -- 172.21.15.7:0/3939238258 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90064070 con 0x55acd8eb12a0 2024-09-10T21:45:42.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.712+0000 7f0e96a00640 1 -- 172.21.15.7:0/3939238258 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e90066920 con 0x55acd8eb12a0 2024-09-10T21:45:42.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.712+0000 7f0e96a00640 1 -- 172.21.15.7:0/3939238258 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90061370 con 0x55acd8eb12a0 2024-09-10T21:45:42.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.712+0000 7f0eafe3b740 1 -- 172.21.15.7:0/3939238258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8eb12a0 msgr2=0x55acd8ef4c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:42.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.712+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/3939238258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8eb12a0 0x55acd8ef4c10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0e90061e80 tx=0x7f0e90061790 comp rx=0 tx=0).stop 2024-09-10T21:45:42.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.712+0000 7f0eafe3b740 1 -- 172.21.15.7:0/3939238258 shutdown_connections 2024-09-10T21:45:42.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.712+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/3939238258 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8eb1820 0x55acd8ef5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.712+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/3939238258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8eb12a0 0x55acd8ef4c10 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.712+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/3939238258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef6130 0x55acd8ef6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.712+0000 7f0eafe3b740 1 -- 172.21.15.7:0/3939238258 >> 172.21.15.7:0/3939238258 conn(0x55acd8eac870 msgr2=0x55acd8eb0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:42.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.712+0000 7f0eafe3b740 1 -- 172.21.15.7:0/3939238258 shutdown_connections 2024-09-10T21:45:42.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.712+0000 7f0eafe3b740 1 -- 172.21.15.7:0/3939238258 wait complete. 2024-09-10T21:45:42.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.713+0000 7f0eafe3b740 1 -- start start 2024-09-10T21:45:42.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.713+0000 7f0eafe3b740 1 Processor -- start 2024-09-10T21:45:42.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.714+0000 7f0eafe3b740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8eb1820 0x55acd8ef81e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.714+0000 7f0eafe3b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef6130 0x55acd8ed9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.714+0000 7f0eafe3b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8ef8720 0x55acd8f054d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.714+0000 7f0eafe3b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55acd8db9410 con 0x55acd8ef8720 2024-09-10T21:45:42.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.714+0000 7f0eafe3b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55acd8e16ab0 con 0x55acd8ef6130 2024-09-10T21:45:42.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.714+0000 7f0eafe3b740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55acd8b83740 con 0x55acd8eb1820 2024-09-10T21:45:42.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.714+0000 7f0eafe3b740 10 client.0 _ll_register_callbacks cb 0x55acd8d96360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:45:42.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.714+0000 7f0ea6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8ef8720 0x55acd8f054d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.715+0000 7f0ea6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8ef8720 0x55acd8f054d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43702/0 (socket says 172.21.15.7:43702) 2024-09-10T21:45:42.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.715+0000 7f0ea6800640 1 -- 172.21.15.7:0/2301732169 learned_addr learned my addr 172.21.15.7:0/2301732169 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:42.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.715+0000 7f0ea5e00640 1 --2- 172.21.15.7:0/2301732169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8eb1820 0x55acd8ef81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.715+0000 7f0e9d400640 1 --2- 172.21.15.7:0/2301732169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef6130 0x55acd8ed9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.715+0000 7f0ea6800640 1 -- 172.21.15.7:0/2301732169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8eb1820 msgr2=0x55acd8ef81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:42.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.715+0000 7f0ea6800640 1 --2- 172.21.15.7:0/2301732169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8eb1820 0x55acd8ef81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.715+0000 7f0ea6800640 1 -- 172.21.15.7:0/2301732169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef6130 msgr2=0x55acd8ed9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:42.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.715+0000 7f0ea6800640 1 --2- 172.21.15.7:0/2301732169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef6130 0x55acd8ed9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.715+0000 7f0ea6800640 1 -- 172.21.15.7:0/2301732169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acd8b74970 con 0x55acd8ef8720 2024-09-10T21:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.715+0000 7f0e9d400640 1 --2- 172.21.15.7:0/2301732169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef6130 0x55acd8ed9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.715+0000 7f0ea5e00640 1 --2- 172.21.15.7:0/2301732169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8eb1820 0x55acd8ef81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.715+0000 7f0ea6800640 1 --2- 172.21.15.7:0/2301732169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8ef8720 0x55acd8f054d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0ea002af10 tx=0x7f0ea0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92f427e96b5c2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.715+0000 7f0e8b400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.716+0000 7f0e8b400640 1 -- 172.21.15.7:0/2301732169 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ea00047f0 con 0x55acd8ef8720 2024-09-10T21:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.716+0000 7f0eafe3b740 1 -- 172.21.15.7:0/2301732169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e90066920 con 0x55acd8ef8720 2024-09-10T21:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.716+0000 7f0e8b400640 1 -- 172.21.15.7:0/2301732169 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f0ea003b450 con 0x55acd8ef8720 2024-09-10T21:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.716+0000 7f0e8b400640 1 -- 172.21.15.7:0/2301732169 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0ea0051050 con 0x55acd8ef8720 2024-09-10T21:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.716+0000 7f0eafe3b740 10 client.5988 fetch_fsmap learned FSMap version 37 2024-09-10T21:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.716+0000 7f0eafe3b740 1 -- 172.21.15.7:0/2301732169 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x55acd8b526a0 con 0x55acd8ef8720 2024-09-10T21:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.716+0000 7f0e8b400640 1 -- 172.21.15.7:0/2301732169 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e90066920 con 0x55acd8ef8720 2024-09-10T21:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.716+0000 7f0e8b400640 1 -- 172.21.15.7:0/2301732169 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 3502+0+0 (secure 0 0 0) 0x7f0ea0054020 con 0x55acd8ef8720 2024-09-10T21:45:42.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.716+0000 7f0eafe3b740 10 client.5988 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T21:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.716+0000 7f0eafe3b740 10 client.5988 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-10T21:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.716+0000 7f0eafe3b740 20 client.5988 populate_metadata read hostname 'smithi007' 2024-09-10T21:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.716+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/2301732169 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55acd8ed8b00 0x55acd8ed8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.716+0000 7f0eafe3b740 4 client.5988 mds_command: new command op to 235016 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.716+0000 7f0eafe3b740 1 -- 172.21.15.7:0/2301732169 --> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acd8e257c0 con 0x55acd8ed8b00 2024-09-10T21:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.717+0000 7f0ea5e00640 1 --2- 172.21.15.7:0/2301732169 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55acd8ed8b00 0x55acd8ed8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.717+0000 7f0ea5e00640 1 --2- 172.21.15.7:0/2301732169 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55acd8ed8b00 0x55acd8ed8f10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=938d654a01b68849 server_cookie=706eed5643859a14 in_seq=0 out_seq=0 2024-09-10T21:45:42.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.717+0000 7f0e8b400640 10 client.5988 ms_handle_connect on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.718+0000 7f0e8b400640 1 -- 172.21.15.7:0/2301732169 <== mds.0 v2:172.21.15.7:6833/364473218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55acd8e257c0 con 0x55acd8ed8b00 2024-09-10T21:45:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.718+0000 7f0e8b400640 10 client.5988 handle_command_reply: tid=0 2024-09-10T21:45:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.718+0000 7f0eafe3b740 1 client.5988 shutdown 2024-09-10T21:45:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.718+0000 7f0eafe3b740 1 -- 172.21.15.7:0/2301732169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8ef8720 msgr2=0x55acd8f054d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:42.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.718+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/2301732169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8ef8720 0x55acd8f054d0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0ea002af10 tx=0x7f0ea0004270 comp rx=0 tx=0).stop 2024-09-10T21:45:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.718+0000 7f0eafe3b740 1 -- 172.21.15.7:0/2301732169 shutdown_connections 2024-09-10T21:45:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.718+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/2301732169 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55acd8ed8b00 0x55acd8ed8f10 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.718+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/2301732169 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef6130 0x55acd8ed9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.718+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/2301732169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8ef8720 0x55acd8f054d0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.718+0000 7f0e8b400640 0 client.5988 ms_handle_reset on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.718+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/2301732169 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8eb1820 0x55acd8ef81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.718+0000 7f0eafe3b740 1 -- 172.21.15.7:0/2301732169 >> 172.21.15.7:0/2301732169 conn(0x55acd8eb12a0 msgr2=0x55acd8ea9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.718+0000 7f0eafe3b740 1 -- 172.21.15.7:0/2301732169 shutdown_connections 2024-09-10T21:45:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.718+0000 7f0eafe3b740 1 -- 172.21.15.7:0/2301732169 wait complete. 2024-09-10T21:45:42.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.719+0000 7f0eafe3b740 20 client.5988 trim_cache size 0 max 16384 2024-09-10T21:45:42.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.737+0000 7f0eafe3b740 1 Processor -- start 2024-09-10T21:45:42.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.737+0000 7f0eafe3b740 1 -- start start 2024-09-10T21:45:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.737+0000 7f0eafe3b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8eb1820 0x55acd8eab5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.738+0000 7f0eafe3b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef6130 0x55acd8eadcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.738+0000 7f0eafe3b740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8ef8720 0x55acd8f18f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.738+0000 7f0eafe3b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea8200f50 con 0x55acd8eb1820 2024-09-10T21:45:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.738+0000 7f0eafe3b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea80b3810 con 0x55acd8ef6130 2024-09-10T21:45:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.738+0000 7f0eafe3b740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f0ea80b2b70 con 0x55acd8ef8720 2024-09-10T21:45:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.738+0000 7f0ea5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8eb1820 0x55acd8eab5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.738+0000 7f0ea5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8eb1820 0x55acd8eab5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43706/0 (socket says 172.21.15.7:43706) 2024-09-10T21:45:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.738+0000 7f0ea5e00640 1 -- 172.21.15.7:0/2355046118 learned_addr learned my addr 172.21.15.7:0/2355046118 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.738+0000 7f0e9d400640 1 --2- 172.21.15.7:0/2355046118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef6130 0x55acd8eadcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.738+0000 7f0ea6800640 1 --2- 172.21.15.7:0/2355046118 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8ef8720 0x55acd8f18f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.738+0000 7f0ea5e00640 1 -- 172.21.15.7:0/2355046118 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8ef8720 msgr2=0x55acd8f18f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.738+0000 7f0ea5e00640 1 --2- 172.21.15.7:0/2355046118 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8ef8720 0x55acd8f18f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.738+0000 7f0ea5e00640 1 -- 172.21.15.7:0/2355046118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef6130 msgr2=0x55acd8eadcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.738+0000 7f0ea5e00640 1 --2- 172.21.15.7:0/2355046118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef6130 0x55acd8eadcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.738+0000 7f0ea5e00640 1 -- 172.21.15.7:0/2355046118 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acd8e49d50 con 0x55acd8eb1820 2024-09-10T21:45:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.739+0000 7f0ea5e00640 1 --2- 172.21.15.7:0/2355046118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8eb1820 0x55acd8eab5a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80b3a90 tx=0x7f0e98012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28daf8885c7a1c84 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.739+0000 7f0e96000640 1 -- 172.21.15.7:0/2355046118 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e9802f070 con 0x55acd8eb1820 2024-09-10T21:45:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.739+0000 7f0e96000640 1 -- 172.21.15.7:0/2355046118 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e9801b040 con 0x55acd8eb1820 2024-09-10T21:45:42.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.739+0000 7f0e96000640 1 -- 172.21.15.7:0/2355046118 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e98016190 con 0x55acd8eb1820 2024-09-10T21:45:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.739+0000 7f0eafe3b740 1 -- 172.21.15.7:0/2355046118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8eb1820 msgr2=0x55acd8eab5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:42.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.739+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/2355046118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8eb1820 0x55acd8eab5a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0ea80b3a90 tx=0x7f0e98012000 comp rx=0 tx=0).stop 2024-09-10T21:45:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.739+0000 7f0eafe3b740 1 -- 172.21.15.7:0/2355046118 shutdown_connections 2024-09-10T21:45:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.739+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/2355046118 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef6130 0x55acd8eadcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.739+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/2355046118 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8ef8720 0x55acd8f18f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.739+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/2355046118 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8eb1820 0x55acd8eab5a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.739+0000 7f0eafe3b740 1 -- 172.21.15.7:0/2355046118 >> 172.21.15.7:0/2355046118 conn(0x55acd8eb12a0 msgr2=0x55acd8ed9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.740+0000 7f0eafe3b740 1 -- 172.21.15.7:0/2355046118 shutdown_connections 2024-09-10T21:45:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.740+0000 7f0eafe3b740 1 -- 172.21.15.7:0/2355046118 wait complete. 2024-09-10T21:45:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.740+0000 7f0eafe3b740 1 -- start start 2024-09-10T21:45:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.740+0000 7f0eafe3b740 1 Processor -- start 2024-09-10T21:45:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.741+0000 7f0eafe3b740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8eb1820 0x55acd8ed93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.741+0000 7f0eafe3b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8ef6130 0x55acd8eadee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.741+0000 7f0eafe3b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef8720 0x55acd8f6bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.741+0000 7f0eafe3b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55acd8eac460 con 0x55acd8ef6130 2024-09-10T21:45:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.741+0000 7f0eafe3b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0ea80b31c0 con 0x55acd8ef8720 2024-09-10T21:45:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.741+0000 7f0eafe3b740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55acd8dd9e80 con 0x55acd8eb1820 2024-09-10T21:45:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.741+0000 7f0eafe3b740 10 client.0 _ll_register_callbacks cb 0x7f0ea0009840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:45:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.742+0000 7f0e9d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8ef6130 0x55acd8eadee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.742+0000 7f0e9d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8ef6130 0x55acd8eadee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43718/0 (socket says 172.21.15.7:43718) 2024-09-10T21:45:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.742+0000 7f0e9d400640 1 -- 172.21.15.7:0/519519906 learned_addr learned my addr 172.21.15.7:0/519519906 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.742+0000 7f0ea5e00640 1 --2- 172.21.15.7:0/519519906 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8eb1820 0x55acd8ed93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.742+0000 7f0ea6800640 1 --2- 172.21.15.7:0/519519906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef8720 0x55acd8f6bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.742+0000 7f0e9d400640 1 -- 172.21.15.7:0/519519906 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8eb1820 msgr2=0x55acd8ed93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.742+0000 7f0e9d400640 1 --2- 172.21.15.7:0/519519906 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8eb1820 0x55acd8ed93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.742+0000 7f0e9d400640 1 -- 172.21.15.7:0/519519906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef8720 msgr2=0x55acd8f6bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:42.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.742+0000 7f0e9d400640 1 --2- 172.21.15.7:0/519519906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef8720 0x55acd8f6bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.742+0000 7f0e9d400640 1 -- 172.21.15.7:0/519519906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acd8ed5100 con 0x55acd8ef6130 2024-09-10T21:45:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.742+0000 7f0e9d400640 1 --2- 172.21.15.7:0/519519906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8ef6130 0x55acd8eadee0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0e90061e80 tx=0x7f0e90061380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a68af1c13e04b9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.742+0000 7f0e8b400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:45:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.743+0000 7f0e8b400640 1 -- 172.21.15.7:0/519519906 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e90080060 con 0x55acd8ef6130 2024-09-10T21:45:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.743+0000 7f0e8b400640 1 -- 172.21.15.7:0/519519906 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f0e9005d840 con 0x55acd8ef6130 2024-09-10T21:45:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.743+0000 7f0eafe3b740 1 -- 172.21.15.7:0/519519906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e9801b040 con 0x55acd8ef6130 2024-09-10T21:45:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.743+0000 7f0e8b400640 1 -- 172.21.15.7:0/519519906 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f0e90064040 con 0x55acd8ef6130 2024-09-10T21:45:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.743+0000 7f0eafe3b740 10 client.5994 fetch_fsmap learned FSMap version 37 2024-09-10T21:45:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.743+0000 7f0eafe3b740 1 -- 172.21.15.7:0/519519906 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x55acd8f184d0 con 0x55acd8ef6130 2024-09-10T21:45:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.743+0000 7f0e8b400640 1 -- 172.21.15.7:0/519519906 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e9801b040 con 0x55acd8ef6130 2024-09-10T21:45:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.743+0000 7f0e8b400640 1 -- 172.21.15.7:0/519519906 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 3502+0+0 (secure 0 0 0) 0x7f0e9005da40 con 0x55acd8ef6130 2024-09-10T21:45:42.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.743+0000 7f0eafe3b740 10 client.5994 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T21:45:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.743+0000 7f0eafe3b740 10 client.5994 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.a 2024-09-10T21:45:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.743+0000 7f0eafe3b740 20 client.5994 populate_metadata read hostname 'smithi007' 2024-09-10T21:45:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.743+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/519519906 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55acd8eae420 0x55acd8f6c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.743+0000 7f0eafe3b740 4 client.5994 mds_command: new command op to 235016 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:45:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.743+0000 7f0eafe3b740 1 -- 172.21.15.7:0/519519906 --> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55acd8e21270 con 0x55acd8eae420 2024-09-10T21:45:42.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.744+0000 7f0ea5e00640 1 --2- 172.21.15.7:0/519519906 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55acd8eae420 0x55acd8f6c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.744+0000 7f0ea5e00640 1 --2- 172.21.15.7:0/519519906 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55acd8eae420 0x55acd8f6c3e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7adeeb7a96f1c8d6 server_cookie=351284ddd158bbe9 in_seq=0 out_seq=0 2024-09-10T21:45:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.744+0000 7f0e8b400640 10 client.5994 ms_handle_connect on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.744+0000 7f0e8b400640 1 -- 172.21.15.7:0/519519906 <== mds.0 v2:172.21.15.7:6833/364473218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x55acd8e21270 con 0x55acd8eae420 2024-09-10T21:45:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.744+0000 7f0e8b400640 10 client.5994 handle_command_reply: tid=0 2024-09-10T21:45:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.744+0000 7f0eafe3b740 1 client.5994 shutdown 2024-09-10T21:45:42.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.744+0000 7f0eafe3b740 1 -- 172.21.15.7:0/519519906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8ef6130 msgr2=0x55acd8eadee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.744+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/519519906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8ef6130 0x55acd8eadee0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0e90061e80 tx=0x7f0e90061380 comp rx=0 tx=0).stop 2024-09-10T21:45:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.745+0000 7f0eafe3b740 1 -- 172.21.15.7:0/519519906 shutdown_connections 2024-09-10T21:45:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.745+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/519519906 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x55acd8eae420 0x55acd8f6c3e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.745+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/519519906 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55acd8ef8720 0x55acd8f6bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.745+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/519519906 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55acd8ef6130 0x55acd8eadee0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.745+0000 7f0eafe3b740 1 --2- 172.21.15.7:0/519519906 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55acd8eb1820 0x55acd8ed93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.745+0000 7f0eafe3b740 1 -- 172.21.15.7:0/519519906 >> 172.21.15.7:0/519519906 conn(0x55acd8eb12a0 msgr2=0x55acd8eab5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.745+0000 7f0e8b400640 0 client.5994 ms_handle_reset on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.745+0000 7f0eafe3b740 1 -- 172.21.15.7:0/519519906 shutdown_connections 2024-09-10T21:45:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.745+0000 7f0eafe3b740 1 -- 172.21.15.7:0/519519906 wait complete. 2024-09-10T21:45:42.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.745+0000 7f0eafe3b740 20 client.5994 trim_cache size 0 max 16384 2024-09-10T21:45:42.751 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:45:42.751 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-09-10T21:45:42.751 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {}, 2024-09-10T21:45:42.751 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:45:42.751 INFO:teuthology.orchestra.run.smithi007.stdout: "34168865-5a8d-4f70-a439-66b4e769eba6": { 2024-09-10T21:45:42.751 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:45:12.282824+0000", 2024-09-10T21:45:42.751 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-09-10T21:45:42.751 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:45:42.751 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:45:42.751 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:45:42.751 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:45:42.751 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:42.752 INFO:teuthology.orchestra.run.smithi007.stdout: "cabd7ddd-fca9-48c5-a921-5e626bb8f21c": { 2024-09-10T21:45:42.752 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:44:40.510317+0000", 2024-09-10T21:45:42.752 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-09-10T21:45:42.752 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:45:42.752 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:45:42.752 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:45:42.752 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:45:42.752 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:42.752 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:42.752 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:45:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.745+0000 7f0ead000640 1 -- 172.21.15.7:0/4144153842 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0e84077f70 msgr2=0x7f0e8407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.745+0000 7f0ead000640 1 --2- 172.21.15.7:0/4144153842 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0e84077f70 0x7f0e8407a430 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f0e98007fd0 tx=0x7f0e98009040 comp rx=0 tx=0).stop 2024-09-10T21:45:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.745+0000 7f0ead000640 1 -- 172.21.15.7:0/4144153842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ea8074090 msgr2=0x7f0ea80b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.745+0000 7f0ead000640 1 --2- 172.21.15.7:0/4144153842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ea8074090 0x7f0ea80b41d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f0e9000c400 tx=0x7f0e9000c8c0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.746+0000 7f0ead000640 1 -- 172.21.15.7:0/4144153842 shutdown_connections 2024-09-10T21:45:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.746+0000 7f0ead000640 1 --2- 172.21.15.7:0/4144153842 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0e84077f70 0x7f0e8407a430 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.746+0000 7f0ead000640 1 --2- 172.21.15.7:0/4144153842 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0ea8156290 0x7f0ea81f87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.746+0000 7f0ead000640 1 --2- 172.21.15.7:0/4144153842 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0ea8074090 0x7f0ea80b41d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.746+0000 7f0ead000640 1 --2- 172.21.15.7:0/4144153842 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0ea806fcf0 0x7f0ea814dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.746+0000 7f0ead000640 1 -- 172.21.15.7:0/4144153842 >> 172.21.15.7:0/4144153842 conn(0x7f0ea80b16d0 msgr2=0x7f0ea80b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.746+0000 7f0ead000640 1 -- 172.21.15.7:0/4144153842 shutdown_connections 2024-09-10T21:45:42.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.746+0000 7f0ead000640 1 -- 172.21.15.7:0/4144153842 wait complete. 2024-09-10T21:45:42.761 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:45:42.762 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:45:42.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.922+0000 7f97d3200640 1 -- 172.21.15.7:0/3303283596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97d40b2f80 msgr2=0x7f97d415f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:42.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.922+0000 7f97d3200640 1 --2- 172.21.15.7:0/3303283596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97d40b2f80 0x7f97d415f7d0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f97d40745c0 tx=0x7f97cc02aa30 comp rx=0 tx=0).stop 2024-09-10T21:45:42.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.922+0000 7f97d3200640 1 -- 172.21.15.7:0/3303283596 shutdown_connections 2024-09-10T21:45:42.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.922+0000 7f97d3200640 1 --2- 172.21.15.7:0/3303283596 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97d40b2f80 0x7f97d415f7d0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.922+0000 7f97d3200640 1 --2- 172.21.15.7:0/3303283596 >> v2:172.21.15.133:3300/0 conn(0x7f97d40b4ac0 0x7f97d40b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.922+0000 7f97d3200640 1 --2- 172.21.15.7:0/3303283596 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97d4145540 0x7f97d40b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.922+0000 7f97d3200640 1 -- 172.21.15.7:0/3303283596 >> 172.21.15.7:0/3303283596 conn(0x7f97d40b1d70 msgr2=0x7f97d40cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:42.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.922+0000 7f97d3200640 1 -- 172.21.15.7:0/3303283596 shutdown_connections 2024-09-10T21:45:42.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.922+0000 7f97d3200640 1 -- 172.21.15.7:0/3303283596 wait complete. 2024-09-10T21:45:42.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.923+0000 7f97d3200640 1 Processor -- start 2024-09-10T21:45:42.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.923+0000 7f97d3200640 1 -- start start 2024-09-10T21:45:42.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.924+0000 7f97d3200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d40b2f80 0x7f97d41f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.924+0000 7f97d3200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97d40b4ac0 0x7f97d40afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.924+0000 7f97d3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97d4145540 0x7f97d40b0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.924+0000 7f97d3200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f97d415fe90 con 0x7f97d4145540 2024-09-10T21:45:42.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.924+0000 7f97d3200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f97d415fd10 con 0x7f97d40b2f80 2024-09-10T21:45:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.924+0000 7f97d3200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f97d4160010 con 0x7f97d40b4ac0 2024-09-10T21:45:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.925+0000 7f97d2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97d4145540 0x7f97d40b0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.925+0000 7f97d2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97d4145540 0x7f97d40b0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43742/0 (socket says 172.21.15.7:43742) 2024-09-10T21:45:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.925+0000 7f97d1e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d40b2f80 0x7f97d41f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.925+0000 7f97d2800640 1 -- 172.21.15.7:0/2644742861 learned_addr learned my addr 172.21.15.7:0/2644742861 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.925+0000 7f97c9400640 1 --2- 172.21.15.7:0/2644742861 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97d40b4ac0 0x7f97d40afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.925+0000 7f97d2800640 1 -- 172.21.15.7:0/2644742861 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97d40b4ac0 msgr2=0x7f97d40afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.925+0000 7f97d2800640 1 --2- 172.21.15.7:0/2644742861 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97d40b4ac0 0x7f97d40afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.925+0000 7f97d2800640 1 -- 172.21.15.7:0/2644742861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d40b2f80 msgr2=0x7f97d41f4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.925+0000 7f97d2800640 1 --2- 172.21.15.7:0/2644742861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d40b2f80 0x7f97d41f4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.925+0000 7f97d2800640 1 -- 172.21.15.7:0/2644742861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97d41f8770 con 0x7f97d4145540 2024-09-10T21:45:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.925+0000 7f97d2800640 1 --2- 172.21.15.7:0/2644742861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97d4145540 0x7f97d40b0360 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009d60 tx=0x7f97cc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:42.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.926+0000 7f97caa00640 1 -- 172.21.15.7:0/2644742861 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc047070 con 0x7f97d4145540 2024-09-10T21:45:42.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.926+0000 7f97caa00640 1 -- 172.21.15.7:0/2644742861 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97cc004250 con 0x7f97d4145540 2024-09-10T21:45:42.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.926+0000 7f97caa00640 1 -- 172.21.15.7:0/2644742861 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97cc033050 con 0x7f97d4145540 2024-09-10T21:45:42.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.926+0000 7f97d3200640 1 -- 172.21.15.7:0/2644742861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97d41f79b0 con 0x7f97d4145540 2024-09-10T21:45:42.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.926+0000 7f97d3200640 1 -- 172.21.15.7:0/2644742861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97d41f7f70 con 0x7f97d4145540 2024-09-10T21:45:42.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.927+0000 7f97d3200640 1 -- 172.21.15.7:0/2644742861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97d41604c0 con 0x7f97d4145540 2024-09-10T21:45:42.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.928+0000 7f97caa00640 1 -- 172.21.15.7:0/2644742861 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f97cc004680 con 0x7f97d4145540 2024-09-10T21:45:42.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.929+0000 7f97caa00640 1 --2- 172.21.15.7:0/2644742861 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f97b0077f70 0x7f97b007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:42.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.929+0000 7f97caa00640 1 -- 172.21.15.7:0/2644742861 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f97cc0bb320 con 0x7f97d4145540 2024-09-10T21:45:42.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.930+0000 7f97d1e00640 1 --2- 172.21.15.7:0/2644742861 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f97b0077f70 0x7f97b007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.930+0000 7f97d1e00640 1 --2- 172.21.15.7:0/2644742861 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f97b0077f70 0x7f97b007a430 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f97c4004630 tx=0x7f97c400b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:42.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:42.934+0000 7f97caa00640 1 -- 172.21.15.7:0/2644742861 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f97cc083730 con 0x7f97d4145540 2024-09-10T21:45:43.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.167+0000 7f97d3200640 1 -- 172.21.15.7:0/2644742861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97d4155e10 con 0x7f97d4145540 2024-09-10T21:45:43.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.167+0000 7f97caa00640 1 -- 172.21.15.7:0/2644742861 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5228 (secure 0 0 0) 0x7f97cc088160 con 0x7f97d4145540 2024-09-10T21:45:43.169 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:45:43.170 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":37,"btime":"2024-09-10T21:45:19:514745+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5473,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/2853074495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2853074495},{"type":"v1","addr":"172.21.15.133:6836","nonce":2853074495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3498706462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3498706462},{"type":"v1","addr":"172.21.15.133:6837","nonce":3498706462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/521190705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":521190705},{"type":"v1","addr":"172.21.15.7:6835","nonce":521190705}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:44:30.804578+0000","modified":"2024-09-10T21:45:19.415172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235016},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235016":{"gid":235016,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":15,"addr":"172.21.15.7:6834/364473218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":364473218},{"type":"v1","addr":"172.21.15.7:6834","nonce":364473218}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235016,"qdb_cluster":[235016]},"id":4}]} 2024-09-10T21:45:43.170 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 37 2024-09-10T21:45:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.170+0000 7f97d3200640 1 -- 172.21.15.7:0/2644742861 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f97b0077f70 msgr2=0x7f97b007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.170+0000 7f97d3200640 1 --2- 172.21.15.7:0/2644742861 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f97b0077f70 0x7f97b007a430 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f97c4004630 tx=0x7f97c400b2e0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.170+0000 7f97d3200640 1 -- 172.21.15.7:0/2644742861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97d4145540 msgr2=0x7f97d40b0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:43.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.170+0000 7f97d3200640 1 --2- 172.21.15.7:0/2644742861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97d4145540 0x7f97d40b0360 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009d60 tx=0x7f97cc003f10 comp rx=0 tx=0).stop 2024-09-10T21:45:43.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.170+0000 7f97d3200640 1 -- 172.21.15.7:0/2644742861 shutdown_connections 2024-09-10T21:45:43.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.170+0000 7f97d3200640 1 --2- 172.21.15.7:0/2644742861 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f97b0077f70 0x7f97b007a430 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.170+0000 7f97d3200640 1 --2- 172.21.15.7:0/2644742861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97d4145540 0x7f97d40b0360 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.170+0000 7f97d3200640 1 --2- 172.21.15.7:0/2644742861 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97d40b4ac0 0x7f97d40afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.170+0000 7f97d3200640 1 --2- 172.21.15.7:0/2644742861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97d40b2f80 0x7f97d41f4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.170+0000 7f97d3200640 1 -- 172.21.15.7:0/2644742861 >> 172.21.15.7:0/2644742861 conn(0x7f97d40b1d70 msgr2=0x7f97d40b0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.170+0000 7f97d3200640 1 -- 172.21.15.7:0/2644742861 shutdown_connections 2024-09-10T21:45:43.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.170+0000 7f97d3200640 1 -- 172.21.15.7:0/2644742861 wait complete. 2024-09-10T21:45:43.187 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a damage ls --format=json-pretty 2024-09-10T21:45:43.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.349+0000 7f11ae200640 1 -- 172.21.15.7:0/2912690381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11a8159680 msgr2=0x7f11a815cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:43.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.349+0000 7f11ae200640 1 --2- 172.21.15.7:0/2912690381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11a8159680 0x7f11a815cc00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1198009830 tx=0x7f119802a480 comp rx=0 tx=0).stop 2024-09-10T21:45:43.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.349+0000 7f11ae200640 1 -- 172.21.15.7:0/2912690381 shutdown_connections 2024-09-10T21:45:43.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.349+0000 7f11ae200640 1 --2- 172.21.15.7:0/2912690381 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11a806e580 0x7f11a80cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.349+0000 7f11ae200640 1 --2- 172.21.15.7:0/2912690381 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11a8159680 0x7f11a815cc00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.349+0000 7f11ae200640 1 -- 172.21.15.7:0/2912690381 >> 172.21.15.7:0/2912690381 conn(0x7f11a80b16d0 msgr2=0x7f11a80b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:43.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.349+0000 7f11ae200640 1 -- 172.21.15.7:0/2912690381 shutdown_connections 2024-09-10T21:45:43.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.349+0000 7f11ae200640 1 -- 172.21.15.7:0/2912690381 wait complete. 2024-09-10T21:45:43.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.351+0000 7f11ae200640 1 Processor -- start 2024-09-10T21:45:43.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.351+0000 7f11ae200640 1 -- start start 2024-09-10T21:45:43.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.352+0000 7f11ae200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11a806e580 0x7f11a80b40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.352+0000 7f11ae200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11a8073e20 0x7f11a81520c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.352+0000 7f11ae200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11a8159680 0x7f11a81540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.352+0000 7f11ae200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f11a80ca990 con 0x7f11a806e580 2024-09-10T21:45:43.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.352+0000 7f11ae200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f11a80ca810 con 0x7f11a8159680 2024-09-10T21:45:43.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.352+0000 7f11ae200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f11a80cab10 con 0x7f11a8073e20 2024-09-10T21:45:43.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.352+0000 7f11a7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11a806e580 0x7f11a80b40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.352+0000 7f11a7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11a806e580 0x7f11a80b40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43754/0 (socket says 172.21.15.7:43754) 2024-09-10T21:45:43.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.352+0000 7f11a7400640 1 -- 172.21.15.7:0/3747437375 learned_addr learned my addr 172.21.15.7:0/3747437375 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:43.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.352+0000 7f119ea00640 1 --2- 172.21.15.7:0/3747437375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11a8073e20 0x7f11a81520c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.352+0000 7f11a7400640 1 -- 172.21.15.7:0/3747437375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11a8073e20 msgr2=0x7f11a81520c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:43.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.352+0000 7f11a7e00640 1 --2- 172.21.15.7:0/3747437375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11a8159680 0x7f11a81540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.352+0000 7f11a7400640 1 --2- 172.21.15.7:0/3747437375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11a8073e20 0x7f11a81520c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.352+0000 7f11a7400640 1 -- 172.21.15.7:0/3747437375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11a8159680 msgr2=0x7f11a81540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:43.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.352+0000 7f11a7400640 1 --2- 172.21.15.7:0/3747437375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11a8159680 0x7f11a81540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.352+0000 7f11a7400640 1 -- 172.21.15.7:0/3747437375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a80b4630 con 0x7f11a806e580 2024-09-10T21:45:43.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.353+0000 7f119ea00640 1 --2- 172.21.15.7:0/3747437375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11a8073e20 0x7f11a81520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:45:43.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.353+0000 7f11a7e00640 1 --2- 172.21.15.7:0/3747437375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11a8159680 0x7f11a81540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:45:43.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.353+0000 7f11a7400640 1 --2- 172.21.15.7:0/3747437375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11a806e580 0x7f11a80b40f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1198000c00 tx=0x7f1198033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:43.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.353+0000 7f119fe00640 1 -- 172.21.15.7:0/3747437375 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198031d60 con 0x7f11a806e580 2024-09-10T21:45:43.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.353+0000 7f119fe00640 1 -- 172.21.15.7:0/3747437375 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11980330d0 con 0x7f11a806e580 2024-09-10T21:45:43.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.353+0000 7f119fe00640 1 -- 172.21.15.7:0/3747437375 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198033520 con 0x7f11a806e580 2024-09-10T21:45:43.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.353+0000 7f11ae200640 1 -- 172.21.15.7:0/3747437375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a80b36e0 con 0x7f11a806e580 2024-09-10T21:45:43.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.353+0000 7f11ae200640 1 -- 172.21.15.7:0/3747437375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a80c3ce0 con 0x7f11a806e580 2024-09-10T21:45:43.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.356+0000 7f119fe00640 1 -- 172.21.15.7:0/3747437375 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f11980035f0 con 0x7f11a806e580 2024-09-10T21:45:43.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.357+0000 7f119fe00640 1 --2- 172.21.15.7:0/3747437375 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1184077f70 0x7f118407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.357+0000 7f11b11c6740 1 Processor -- start 2024-09-10T21:45:43.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.357+0000 7f119fe00640 1 -- 172.21.15.7:0/3747437375 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f11980bb150 con 0x7f11a806e580 2024-09-10T21:45:43.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.358+0000 7f11b11c6740 1 -- start start 2024-09-10T21:45:43.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.358+0000 7f119ea00640 1 --2- 172.21.15.7:0/3747437375 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1184077f70 0x7f118407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.358+0000 7f11b11c6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804b7a0 0x559248048640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.358+0000 7f11b11c6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924805d1c0 0x55924804bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.358+0000 7f11b11c6740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924804c0c0 0x55924801cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.358+0000 7f11b11c6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559247f93750 con 0x55924804b7a0 2024-09-10T21:45:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.358+0000 7f11b11c6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559247f82b80 con 0x55924805d1c0 2024-09-10T21:45:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.358+0000 7f11b11c6740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x559247f86990 con 0x55924804c0c0 2024-09-10T21:45:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.359+0000 7f119ea00640 1 --2- 172.21.15.7:0/3747437375 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1184077f70 0x7f118407a430 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f11a80b4080 tx=0x7f1190005db0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.359+0000 7f11a7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804b7a0 0x559248048640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.359+0000 7f11a7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804b7a0 0x559248048640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43758/0 (socket says 172.21.15.7:43758) 2024-09-10T21:45:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.359+0000 7f11a7400640 1 -- 172.21.15.7:0/3784960033 learned_addr learned my addr 172.21.15.7:0/3784960033 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.359+0000 7f119ea00640 1 --2- 172.21.15.7:0/3784960033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924805d1c0 0x55924804bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.359+0000 7f11a7e00640 1 --2- 172.21.15.7:0/3784960033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924804c0c0 0x55924801cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.359+0000 7f11a7400640 1 -- 172.21.15.7:0/3784960033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924804c0c0 msgr2=0x55924801cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.359+0000 7f11a7400640 1 --2- 172.21.15.7:0/3784960033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924804c0c0 0x55924801cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.359+0000 7f11a7400640 1 -- 172.21.15.7:0/3784960033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924805d1c0 msgr2=0x55924804bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.359+0000 7f11a7400640 1 --2- 172.21.15.7:0/3784960033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924805d1c0 0x55924804bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.359+0000 7f11a7400640 1 -- 172.21.15.7:0/3784960033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559247f10ad0 con 0x55924804b7a0 2024-09-10T21:45:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.359+0000 7f11a7400640 1 --2- 172.21.15.7:0/3784960033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804b7a0 0x559248048640 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f11980b9680 tx=0x7f1198082ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61db68a48ffa19ac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.360+0000 7f1197400640 1 -- 172.21.15.7:0/3784960033 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198048030 con 0x55924804b7a0 2024-09-10T21:45:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.360+0000 7f1197400640 1 -- 172.21.15.7:0/3784960033 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1198083c30 con 0x55924804b7a0 2024-09-10T21:45:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.360+0000 7f1197400640 1 -- 172.21.15.7:0/3784960033 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1198083530 con 0x55924804b7a0 2024-09-10T21:45:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.360+0000 7f11b11c6740 1 -- 172.21.15.7:0/3784960033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804b7a0 msgr2=0x559248048640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.360+0000 7f11b11c6740 1 --2- 172.21.15.7:0/3784960033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804b7a0 0x559248048640 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f11980b9680 tx=0x7f1198082ca0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.360+0000 7f11b11c6740 1 -- 172.21.15.7:0/3784960033 shutdown_connections 2024-09-10T21:45:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.360+0000 7f11b11c6740 1 --2- 172.21.15.7:0/3784960033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924805d1c0 0x55924804bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.360+0000 7f11b11c6740 1 --2- 172.21.15.7:0/3784960033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924804c0c0 0x55924801cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.360+0000 7f11b11c6740 1 --2- 172.21.15.7:0/3784960033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804b7a0 0x559248048640 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.360+0000 7f11b11c6740 1 -- 172.21.15.7:0/3784960033 >> 172.21.15.7:0/3784960033 conn(0x55924801dce0 msgr2=0x559248021700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.360+0000 7f11b11c6740 1 -- 172.21.15.7:0/3784960033 shutdown_connections 2024-09-10T21:45:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.360+0000 7f11b11c6740 1 -- 172.21.15.7:0/3784960033 wait complete. 2024-09-10T21:45:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.361+0000 7f11b11c6740 1 -- start start 2024-09-10T21:45:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.361+0000 7f11b11c6740 1 Processor -- start 2024-09-10T21:45:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.362+0000 7f11b11c6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924804c0c0 0x55924803f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.362+0000 7f11b11c6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924805d1c0 0x559248041e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.362+0000 7f11b11c6740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924803ffe0 0x559248042360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.362+0000 7f11b11c6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559247cf3700 con 0x55924805d1c0 2024-09-10T21:45:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.362+0000 7f11b11c6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559247f28430 con 0x55924804c0c0 2024-09-10T21:45:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.362+0000 7f11b11c6740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x559247f93750 con 0x55924803ffe0 2024-09-10T21:45:43.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.362+0000 7f11b11c6740 10 client.0 _ll_register_callbacks cb 0x559247f2e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:45:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.363+0000 7f119ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924805d1c0 0x559248041e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.363+0000 7f119ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924805d1c0 0x559248041e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43760/0 (socket says 172.21.15.7:43760) 2024-09-10T21:45:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.363+0000 7f119ea00640 1 -- 172.21.15.7:0/2020892367 learned_addr learned my addr 172.21.15.7:0/2020892367 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.363+0000 7f11a7400640 1 --2- 172.21.15.7:0/2020892367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924804c0c0 0x55924803f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.363+0000 7f11a7e00640 1 --2- 172.21.15.7:0/2020892367 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924803ffe0 0x559248042360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.363+0000 7f119ea00640 1 -- 172.21.15.7:0/2020892367 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924803ffe0 msgr2=0x559248042360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.363+0000 7f119ea00640 1 --2- 172.21.15.7:0/2020892367 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924803ffe0 0x559248042360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.363+0000 7f119ea00640 1 -- 172.21.15.7:0/2020892367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924804c0c0 msgr2=0x55924803f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:43.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.363+0000 7f119ea00640 1 --2- 172.21.15.7:0/2020892367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924804c0c0 0x55924803f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.363+0000 7f119ea00640 1 -- 172.21.15.7:0/2020892367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559247d5d7c0 con 0x55924805d1c0 2024-09-10T21:45:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.363+0000 7f11a7400640 1 --2- 172.21.15.7:0/2020892367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924804c0c0 0x55924803f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:45:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.363+0000 7f11a7e00640 1 --2- 172.21.15.7:0/2020892367 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924803ffe0 0x559248042360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:45:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.363+0000 7f119ea00640 1 --2- 172.21.15.7:0/2020892367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924805d1c0 0x559248041e20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f119000f9e0 tx=0x7f11900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6755be2ac0ac00 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.363+0000 7f118be00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:45:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.364+0000 7f118be00640 1 -- 172.21.15.7:0/2020892367 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190026470 con 0x55924805d1c0 2024-09-10T21:45:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.364+0000 7f11b11c6740 1 -- 172.21.15.7:0/2020892367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1198083c30 con 0x55924805d1c0 2024-09-10T21:45:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.364+0000 7f118be00640 1 -- 172.21.15.7:0/2020892367 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f1190026610 con 0x55924805d1c0 2024-09-10T21:45:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.364+0000 7f118be00640 1 -- 172.21.15.7:0/2020892367 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f119002e050 con 0x55924805d1c0 2024-09-10T21:45:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.364+0000 7f11b11c6740 10 client.6015 fetch_fsmap learned FSMap version 37 2024-09-10T21:45:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.364+0000 7f11b11c6740 1 -- 172.21.15.7:0/2020892367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x559247dbdf50 con 0x55924805d1c0 2024-09-10T21:45:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.364+0000 7f118be00640 1 -- 172.21.15.7:0/2020892367 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1198083c30 con 0x55924805d1c0 2024-09-10T21:45:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.364+0000 7f118be00640 1 -- 172.21.15.7:0/2020892367 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 3502+0+0 (secure 0 0 0) 0x7f1190026810 con 0x55924805d1c0 2024-09-10T21:45:43.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.364+0000 7f11b11c6740 10 client.6015 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T21:45:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.364+0000 7f11b11c6740 10 client.6015 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:45:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.364+0000 7f11b11c6740 20 client.6015 populate_metadata read hostname 'smithi007' 2024-09-10T21:45:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.364+0000 7f11b11c6740 1 --2- 172.21.15.7:0/2020892367 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x5592480795c0 0x5592480448c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.364+0000 7f11b11c6740 4 client.6015 mds_command: new command op to 235016 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:45:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.364+0000 7f11b11c6740 1 -- 172.21.15.7:0/2020892367 --> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559247f956a0 con 0x5592480795c0 2024-09-10T21:45:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.365+0000 7f11a7400640 1 --2- 172.21.15.7:0/2020892367 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x5592480795c0 0x5592480448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.365+0000 7f11a7400640 1 --2- 172.21.15.7:0/2020892367 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x5592480795c0 0x5592480448c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=569de3de55401943 server_cookie=f17b80cc009f68ce in_seq=0 out_seq=0 2024-09-10T21:45:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.365+0000 7f118be00640 10 client.6015 ms_handle_connect on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.366+0000 7f118be00640 1 -- 172.21.15.7:0/2020892367 <== mds.0 v2:172.21.15.7:6833/364473218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559247f956a0 con 0x5592480795c0 2024-09-10T21:45:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.366+0000 7f118be00640 10 client.6015 handle_command_reply: tid=0 2024-09-10T21:45:43.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.366+0000 7f11b11c6740 1 client.6015 shutdown 2024-09-10T21:45:43.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.366+0000 7f11b11c6740 1 -- 172.21.15.7:0/2020892367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924805d1c0 msgr2=0x559248041e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:43.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.366+0000 7f11b11c6740 1 --2- 172.21.15.7:0/2020892367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924805d1c0 0x559248041e20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f119000f9e0 tx=0x7f11900046c0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.366+0000 7f11b11c6740 1 -- 172.21.15.7:0/2020892367 shutdown_connections 2024-09-10T21:45:43.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.366+0000 7f11b11c6740 1 --2- 172.21.15.7:0/2020892367 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x5592480795c0 0x5592480448c0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.366+0000 7f11b11c6740 1 --2- 172.21.15.7:0/2020892367 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924803ffe0 0x559248042360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.366+0000 7f11b11c6740 1 --2- 172.21.15.7:0/2020892367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924805d1c0 0x559248041e20 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.366+0000 7f11b11c6740 1 --2- 172.21.15.7:0/2020892367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924804c0c0 0x55924803f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.366+0000 7f118be00640 0 client.6015 ms_handle_reset on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:43.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.366+0000 7f11b11c6740 1 -- 172.21.15.7:0/2020892367 >> 172.21.15.7:0/2020892367 conn(0x55924804b7a0 msgr2=0x5592480216d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.366+0000 7f11b11c6740 1 -- 172.21.15.7:0/2020892367 shutdown_connections 2024-09-10T21:45:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.366+0000 7f11b11c6740 1 -- 172.21.15.7:0/2020892367 wait complete. 2024-09-10T21:45:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.366+0000 7f11b11c6740 20 client.6015 trim_cache size 0 max 16384 2024-09-10T21:45:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.380+0000 7f11b11c6740 1 Processor -- start 2024-09-10T21:45:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.380+0000 7f11b11c6740 1 -- start start 2024-09-10T21:45:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.381+0000 7f11b11c6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924803ffe0 0x5592480435e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.381+0000 7f11b11c6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804c0c0 0x559248021a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.381+0000 7f11b11c6740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924805d1c0 0x5592480655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.381+0000 7f11b11c6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f11a82002a0 con 0x55924804c0c0 2024-09-10T21:45:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.381+0000 7f11b11c6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f11a81f8220 con 0x55924803ffe0 2024-09-10T21:45:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.381+0000 7f11b11c6740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f11a81f8870 con 0x55924805d1c0 2024-09-10T21:45:43.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.382+0000 7f119ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804c0c0 0x559248021a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.382+0000 7f119ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804c0c0 0x559248021a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43774/0 (socket says 172.21.15.7:43774) 2024-09-10T21:45:43.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.382+0000 7f119ea00640 1 -- 172.21.15.7:0/3387242474 learned_addr learned my addr 172.21.15.7:0/3387242474 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:43.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.382+0000 7f11a7400640 1 --2- 172.21.15.7:0/3387242474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924803ffe0 0x5592480435e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.382+0000 7f11a7e00640 1 --2- 172.21.15.7:0/3387242474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924805d1c0 0x5592480655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.382+0000 7f119ea00640 1 -- 172.21.15.7:0/3387242474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924805d1c0 msgr2=0x5592480655e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:43.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.382+0000 7f119ea00640 1 --2- 172.21.15.7:0/3387242474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924805d1c0 0x5592480655e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.382+0000 7f119ea00640 1 -- 172.21.15.7:0/3387242474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924803ffe0 msgr2=0x5592480435e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:43.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.382+0000 7f119ea00640 1 --2- 172.21.15.7:0/3387242474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924803ffe0 0x5592480435e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.382+0000 7f119ea00640 1 -- 172.21.15.7:0/3387242474 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559247cc2660 con 0x55924804c0c0 2024-09-10T21:45:43.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.382+0000 7f119ea00640 1 --2- 172.21.15.7:0/3387242474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804c0c0 0x559248021a90 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f119000db30 tx=0x7f119000db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ef9456c62ad448 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.383+0000 7f1196a00640 1 -- 172.21.15.7:0/3387242474 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190027e50 con 0x55924804c0c0 2024-09-10T21:45:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.383+0000 7f1196a00640 1 -- 172.21.15.7:0/3387242474 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f119002b040 con 0x55924804c0c0 2024-09-10T21:45:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.383+0000 7f1196a00640 1 -- 172.21.15.7:0/3387242474 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1190026690 con 0x55924804c0c0 2024-09-10T21:45:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.384+0000 7f11b11c6740 1 -- 172.21.15.7:0/3387242474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804c0c0 msgr2=0x559248021a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.384+0000 7f11b11c6740 1 --2- 172.21.15.7:0/3387242474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804c0c0 0x559248021a90 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f119000db30 tx=0x7f119000db60 comp rx=0 tx=0).stop 2024-09-10T21:45:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.384+0000 7f11b11c6740 1 -- 172.21.15.7:0/3387242474 shutdown_connections 2024-09-10T21:45:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.384+0000 7f11b11c6740 1 --2- 172.21.15.7:0/3387242474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924805d1c0 0x5592480655e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.384+0000 7f11b11c6740 1 --2- 172.21.15.7:0/3387242474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804c0c0 0x559248021a90 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.384+0000 7f11b11c6740 1 --2- 172.21.15.7:0/3387242474 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924803ffe0 0x5592480435e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.384+0000 7f11b11c6740 1 -- 172.21.15.7:0/3387242474 >> 172.21.15.7:0/3387242474 conn(0x55924804b7a0 msgr2=0x559248047000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.384+0000 7f11b11c6740 1 -- 172.21.15.7:0/3387242474 shutdown_connections 2024-09-10T21:45:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.384+0000 7f11b11c6740 1 -- 172.21.15.7:0/3387242474 wait complete. 2024-09-10T21:45:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.384+0000 7f11b11c6740 1 -- start start 2024-09-10T21:45:43.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.385+0000 7f11b11c6740 1 Processor -- start 2024-09-10T21:45:43.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.386+0000 7f11b11c6740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924803ffe0 0x55924801cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.386+0000 7f11b11c6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804c0c0 0x5592480e40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.386+0000 7f11b11c6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924805d1c0 0x559248047fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.386+0000 7f11b11c6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559248079440 con 0x55924804c0c0 2024-09-10T21:45:43.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.386+0000 7f11b11c6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55924803f750 con 0x55924805d1c0 2024-09-10T21:45:43.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.386+0000 7f11b11c6740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x559248079a90 con 0x55924803ffe0 2024-09-10T21:45:43.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.386+0000 7f11b11c6740 10 client.0 _ll_register_callbacks cb 0x7f119000df70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:45:43.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.386+0000 7f119ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804c0c0 0x5592480e40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.386+0000 7f119ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804c0c0 0x5592480e40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43780/0 (socket says 172.21.15.7:43780) 2024-09-10T21:45:43.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.386+0000 7f119ea00640 1 -- 172.21.15.7:0/1926400586 learned_addr learned my addr 172.21.15.7:0/1926400586 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:43.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.386+0000 7f11a7400640 1 --2- 172.21.15.7:0/1926400586 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924803ffe0 0x55924801cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.386+0000 7f11a7e00640 1 --2- 172.21.15.7:0/1926400586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924805d1c0 0x559248047fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.387+0000 7f119ea00640 1 -- 172.21.15.7:0/1926400586 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924803ffe0 msgr2=0x55924801cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:43.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.387+0000 7f119ea00640 1 --2- 172.21.15.7:0/1926400586 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924803ffe0 0x55924801cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.387+0000 7f119ea00640 1 -- 172.21.15.7:0/1926400586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924805d1c0 msgr2=0x559248047fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:45:43.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.387+0000 7f119ea00640 1 --2- 172.21.15.7:0/1926400586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924805d1c0 0x559248047fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.387+0000 7f119ea00640 1 -- 172.21.15.7:0/1926400586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559247f4f630 con 0x55924804c0c0 2024-09-10T21:45:43.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.387+0000 7f119ea00640 1 --2- 172.21.15.7:0/1926400586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804c0c0 0x5592480e40d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f119000f9e0 tx=0x7f11900266a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2baf7fbaef184d2a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:43.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.387+0000 7f118be00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:45:43.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.387+0000 7f118be00640 1 -- 172.21.15.7:0/1926400586 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f119001c790 con 0x55924804c0c0 2024-09-10T21:45:43.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.387+0000 7f118be00640 1 -- 172.21.15.7:0/1926400586 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f11900047e0 con 0x55924804c0c0 2024-09-10T21:45:43.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.387+0000 7f11b11c6740 1 -- 172.21.15.7:0/1926400586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f119002b040 con 0x55924804c0c0 2024-09-10T21:45:43.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.388+0000 7f118be00640 1 -- 172.21.15.7:0/1926400586 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f1190004a20 con 0x55924804c0c0 2024-09-10T21:45:43.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.388+0000 7f11b11c6740 10 client.6021 fetch_fsmap learned FSMap version 37 2024-09-10T21:45:43.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.388+0000 7f11b11c6740 1 -- 172.21.15.7:0/1926400586 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=2+}) -- 0x559248078c90 con 0x55924804c0c0 2024-09-10T21:45:43.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.388+0000 7f118be00640 1 -- 172.21.15.7:0/1926400586 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f119002b040 con 0x55924804c0c0 2024-09-10T21:45:43.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.388+0000 7f118be00640 1 -- 172.21.15.7:0/1926400586 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 37) ==== 3502+0+0 (secure 0 0 0) 0x7f1190004bc0 con 0x55924804c0c0 2024-09-10T21:45:43.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.388+0000 7f11b11c6740 10 client.6021 fetch_fsmap finished waiting for FSMap version 37 2024-09-10T21:45:43.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.388+0000 7f11b11c6740 10 client.6021 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:45:43.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.388+0000 7f11b11c6740 20 client.6021 populate_metadata read hostname 'smithi007' 2024-09-10T21:45:43.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.388+0000 7f11b11c6740 1 --2- 172.21.15.7:0/1926400586 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x559248065760 0x559248065b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.388+0000 7f11b11c6740 4 client.6021 mds_command: new command op to 235016 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-10T21:45:43.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.388+0000 7f11b11c6740 1 -- 172.21.15.7:0/1926400586 --> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x559247f91150 con 0x559248065760 2024-09-10T21:45:43.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.388+0000 7f11a7400640 1 --2- 172.21.15.7:0/1926400586 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x559248065760 0x559248065b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.389+0000 7f11a7400640 1 --2- 172.21.15.7:0/1926400586 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x559248065760 0x559248065b40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17a161af69a3651f server_cookie=b88e14c90db7a799 in_seq=0 out_seq=0 2024-09-10T21:45:43.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.389+0000 7f118be00640 10 client.6021 ms_handle_connect on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:43.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.389+0000 7f118be00640 1 -- 172.21.15.7:0/1926400586 <== mds.0 v2:172.21.15.7:6833/364473218 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559247f91150 con 0x559248065760 2024-09-10T21:45:43.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.389+0000 7f118be00640 10 client.6021 handle_command_reply: tid=0 2024-09-10T21:45:43.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.389+0000 7f11b11c6740 1 client.6021 shutdown 2024-09-10T21:45:43.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.389+0000 7f11b11c6740 1 -- 172.21.15.7:0/1926400586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804c0c0 msgr2=0x5592480e40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:43.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.389+0000 7f11b11c6740 1 --2- 172.21.15.7:0/1926400586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804c0c0 0x5592480e40d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f119000f9e0 tx=0x7f11900266a0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.390+0000 7f11b11c6740 1 -- 172.21.15.7:0/1926400586 shutdown_connections 2024-09-10T21:45:43.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.390+0000 7f11b11c6740 1 --2- 172.21.15.7:0/1926400586 >> [v2:172.21.15.7:6833/364473218,v1:172.21.15.7:6834/364473218] conn(0x559248065760 0x559248065b40 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.390+0000 7f11b11c6740 1 --2- 172.21.15.7:0/1926400586 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55924805d1c0 0x559248047fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.390+0000 7f11b11c6740 1 --2- 172.21.15.7:0/1926400586 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55924804c0c0 0x5592480e40d0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.390+0000 7f11b11c6740 1 --2- 172.21.15.7:0/1926400586 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55924803ffe0 0x55924801cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.390+0000 7f11b11c6740 1 -- 172.21.15.7:0/1926400586 >> 172.21.15.7:0/1926400586 conn(0x55924804b7a0 msgr2=0x55924801ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:43.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.390+0000 7f118be00640 0 client.6021 ms_handle_reset on v2:172.21.15.7:6833/364473218 2024-09-10T21:45:43.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.390+0000 7f11b11c6740 1 -- 172.21.15.7:0/1926400586 shutdown_connections 2024-09-10T21:45:43.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.390+0000 7f11b11c6740 1 -- 172.21.15.7:0/1926400586 wait complete. 2024-09-10T21:45:43.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.390+0000 7f11b11c6740 20 client.6021 trim_cache size 0 max 16384 2024-09-10T21:45:43.436 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:45:43.436 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-09-10T21:45:43.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.391+0000 7f11ae200640 1 -- 172.21.15.7:0/3747437375 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1184077f70 msgr2=0x7f118407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:43.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.391+0000 7f11ae200640 1 --2- 172.21.15.7:0/3747437375 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1184077f70 0x7f118407a430 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f11a80b4080 tx=0x7f1190005db0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.391+0000 7f11ae200640 1 -- 172.21.15.7:0/3747437375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11a806e580 msgr2=0x7f11a80b40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:43.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.391+0000 7f11ae200640 1 --2- 172.21.15.7:0/3747437375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11a806e580 0x7f11a80b40f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1198000c00 tx=0x7f1198033d60 comp rx=0 tx=0).stop 2024-09-10T21:45:43.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.391+0000 7f11ae200640 1 -- 172.21.15.7:0/3747437375 shutdown_connections 2024-09-10T21:45:43.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.391+0000 7f11ae200640 1 --2- 172.21.15.7:0/3747437375 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1184077f70 0x7f118407a430 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.391+0000 7f11ae200640 1 --2- 172.21.15.7:0/3747437375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f11a8159680 0x7f11a81540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.391+0000 7f11ae200640 1 --2- 172.21.15.7:0/3747437375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11a8073e20 0x7f11a81520c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.391+0000 7f11ae200640 1 --2- 172.21.15.7:0/3747437375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f11a806e580 0x7f11a80b40f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.391+0000 7f11ae200640 1 -- 172.21.15.7:0/3747437375 >> 172.21.15.7:0/3747437375 conn(0x7f11a80b16d0 msgr2=0x7f11a80cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:43.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.391+0000 7f11ae200640 1 -- 172.21.15.7:0/3747437375 shutdown_connections 2024-09-10T21:45:43.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.391+0000 7f11ae200640 1 -- 172.21.15.7:0/3747437375 wait complete. 2024-09-10T21:45:43.438 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:45:43.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.597+0000 7f585da00640 1 -- 172.21.15.7:0/50751535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585815be40 msgr2=0x7f58580cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:43.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.597+0000 7f585da00640 1 --2- 172.21.15.7:0/50751535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585815be40 0x7f58580cf3d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f5840005fd0 tx=0x7f584002a4a0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.597+0000 7f585da00640 1 -- 172.21.15.7:0/50751535 shutdown_connections 2024-09-10T21:45:43.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.597+0000 7f585da00640 1 --2- 172.21.15.7:0/50751535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585815be40 0x7f58580cf3d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.597+0000 7f585da00640 1 --2- 172.21.15.7:0/50751535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f585815ba60 0x7f58580cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.597+0000 7f585da00640 1 -- 172.21.15.7:0/50751535 >> 172.21.15.7:0/50751535 conn(0x7f58580b16d0 msgr2=0x7f58580b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:43.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.597+0000 7f585da00640 1 -- 172.21.15.7:0/50751535 shutdown_connections 2024-09-10T21:45:43.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.597+0000 7f585da00640 1 -- 172.21.15.7:0/50751535 wait complete. 2024-09-10T21:45:43.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.599+0000 7f585da00640 1 Processor -- start 2024-09-10T21:45:43.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.599+0000 7f585da00640 1 -- start start 2024-09-10T21:45:43.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.600+0000 7f585da00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58580cde10 0x7f58580c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.600+0000 7f585da00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f585815ba60 0x7f5858148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.600+0000 7f585da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585815be40 0x7f5858149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.600+0000 7f585da00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f585806fd60 con 0x7f585815be40 2024-09-10T21:45:43.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.600+0000 7f585da00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f58580ce5c0 con 0x7f585815ba60 2024-09-10T21:45:43.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.600+0000 7f585da00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f585806fee0 con 0x7f58580cde10 2024-09-10T21:45:43.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.600+0000 7f5857400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585815be40 0x7f5858149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.600+0000 7f5857400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585815be40 0x7f5858149150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43796/0 (socket says 172.21.15.7:43796) 2024-09-10T21:45:43.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.600+0000 7f5857400640 1 -- 172.21.15.7:0/429415521 learned_addr learned my addr 172.21.15.7:0/429415521 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:43.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.600+0000 7f584e000640 1 --2- 172.21.15.7:0/429415521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f585815ba60 0x7f5858148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.600+0000 7f5856a00640 1 --2- 172.21.15.7:0/429415521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58580cde10 0x7f58580c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.601+0000 7f5857400640 1 -- 172.21.15.7:0/429415521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58580cde10 msgr2=0x7f58580c36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:43.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.601+0000 7f5857400640 1 --2- 172.21.15.7:0/429415521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58580cde10 0x7f58580c36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.601+0000 7f5857400640 1 -- 172.21.15.7:0/429415521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f585815ba60 msgr2=0x7f5858148c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:43.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.601+0000 7f5857400640 1 --2- 172.21.15.7:0/429415521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f585815ba60 0x7f5858148c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.601+0000 7f5857400640 1 -- 172.21.15.7:0/429415521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f585815b460 con 0x7f585815be40 2024-09-10T21:45:43.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.601+0000 7f5857400640 1 --2- 172.21.15.7:0/429415521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585815be40 0x7f5858149150 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f585000c780 tx=0x7f585000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:43.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.601+0000 7f584f400640 1 -- 172.21.15.7:0/429415521 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5850004b10 con 0x7f585815be40 2024-09-10T21:45:43.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.601+0000 7f584f400640 1 -- 172.21.15.7:0/429415521 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5850004da0 con 0x7f585815be40 2024-09-10T21:45:43.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.601+0000 7f584f400640 1 -- 172.21.15.7:0/429415521 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585000aa50 con 0x7f585815be40 2024-09-10T21:45:43.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.601+0000 7f585da00640 1 -- 172.21.15.7:0/429415521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58580c6050 con 0x7f585815be40 2024-09-10T21:45:43.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.602+0000 7f585da00640 1 -- 172.21.15.7:0/429415521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58580c6610 con 0x7f585815be40 2024-09-10T21:45:43.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.603+0000 7f585da00640 1 -- 172.21.15.7:0/429415521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58580705e0 con 0x7f585815be40 2024-09-10T21:45:43.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.604+0000 7f584f400640 1 -- 172.21.15.7:0/429415521 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f585000abf0 con 0x7f585815be40 2024-09-10T21:45:43.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.605+0000 7f584f400640 1 --2- 172.21.15.7:0/429415521 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5834077fa0 0x7f583407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:43.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.605+0000 7f584f400640 1 -- 172.21.15.7:0/429415521 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f5850099dd0 con 0x7f585815be40 2024-09-10T21:45:43.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.605+0000 7f5856a00640 1 --2- 172.21.15.7:0/429415521 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5834077fa0 0x7f583407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:43.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.606+0000 7f5856a00640 1 --2- 172.21.15.7:0/429415521 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5834077fa0 0x7f583407a460 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f584800bbe0 tx=0x7f584800b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:43.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.609+0000 7f584f400640 1 -- 172.21.15.7:0/429415521 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f58580705e0 con 0x7f585815be40 2024-09-10T21:45:43.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.842+0000 7f585da00640 1 -- 172.21.15.7:0/429415521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f58580c9a60 con 0x7f585815be40 2024-09-10T21:45:43.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.843+0000 7f584f400640 1 -- 172.21.15.7:0/429415521 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f5850020070 con 0x7f585815be40 2024-09-10T21:45:43.844 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:45:43.844 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T21:45:43.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.845+0000 7f585da00640 1 -- 172.21.15.7:0/429415521 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5834077fa0 msgr2=0x7f583407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:43.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.845+0000 7f585da00640 1 --2- 172.21.15.7:0/429415521 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5834077fa0 0x7f583407a460 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f584800bbe0 tx=0x7f584800b390 comp rx=0 tx=0).stop 2024-09-10T21:45:43.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.845+0000 7f585da00640 1 -- 172.21.15.7:0/429415521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585815be40 msgr2=0x7f5858149150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:43.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.845+0000 7f585da00640 1 --2- 172.21.15.7:0/429415521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585815be40 0x7f5858149150 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f585000c780 tx=0x7f585000cc40 comp rx=0 tx=0).stop 2024-09-10T21:45:43.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.845+0000 7f585da00640 1 -- 172.21.15.7:0/429415521 shutdown_connections 2024-09-10T21:45:43.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.845+0000 7f585da00640 1 --2- 172.21.15.7:0/429415521 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5834077fa0 0x7f583407a460 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.845+0000 7f585da00640 1 --2- 172.21.15.7:0/429415521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f585815ba60 0x7f5858148c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.845+0000 7f585da00640 1 --2- 172.21.15.7:0/429415521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585815be40 0x7f5858149150 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.845+0000 7f585da00640 1 --2- 172.21.15.7:0/429415521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58580cde10 0x7f58580c36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:43.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.845+0000 7f585da00640 1 -- 172.21.15.7:0/429415521 >> 172.21.15.7:0/429415521 conn(0x7f58580b16d0 msgr2=0x7f58580c8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:43.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.845+0000 7f585da00640 1 -- 172.21.15.7:0/429415521 shutdown_connections 2024-09-10T21:45:43.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:43.845+0000 7f585da00640 1 -- 172.21.15.7:0/429415521 wait complete. 2024-09-10T21:45:43.863 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:45:43.864 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:45:43.864 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:45:43.960 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:45:44.050 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:45:44.050 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:45:44.050 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:45:44.050 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:45:44.086 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:45:44.086 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:45:44.161 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:45:44.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.362+0000 7fb9dac00640 1 -- 172.21.15.7:0/1000585219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cc940 msgr2=0x7fb9d406fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:44.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.362+0000 7fb9dac00640 1 --2- 172.21.15.7:0/1000585219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cc940 0x7fb9d406fdf0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4005fd0 tx=0x7fb9c402a670 comp rx=0 tx=0).stop 2024-09-10T21:45:44.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.362+0000 7fb9dac00640 1 -- 172.21.15.7:0/1000585219 shutdown_connections 2024-09-10T21:45:44.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.362+0000 7fb9dac00640 1 --2- 172.21.15.7:0/1000585219 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cc940 0x7fb9d406fdf0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:44.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.362+0000 7fb9dac00640 1 -- 172.21.15.7:0/1000585219 >> 172.21.15.7:0/1000585219 conn(0x7fb9d40b1660 msgr2=0x7fb9d40b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:44.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.362+0000 7fb9dac00640 1 -- 172.21.15.7:0/1000585219 shutdown_connections 2024-09-10T21:45:44.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.362+0000 7fb9dac00640 1 -- 172.21.15.7:0/1000585219 wait complete. 2024-09-10T21:45:44.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.364+0000 7fb9dac00640 1 Processor -- start 2024-09-10T21:45:44.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.364+0000 7fb9dac00640 1 -- start start 2024-09-10T21:45:44.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.365+0000 7fb9dac00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb9d4073cf0 0x7fb9d414df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:44.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.365+0000 7fb9dac00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cc940 0x7fb9d40cb570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.365+0000 7fb9dac00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9d415e7b0 0x7fb9d40acb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.365+0000 7fb9dac00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d40ccea0 con 0x7fb9d40cc940 2024-09-10T21:45:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.365+0000 7fb9dac00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb9d40ccd20 con 0x7fb9d415e7b0 2024-09-10T21:45:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.365+0000 7fb9dac00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fb9d40b4130 con 0x7fb9d4073cf0 2024-09-10T21:45:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.365+0000 7fb9d3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cc940 0x7fb9d40cb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.365+0000 7fb9d3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cc940 0x7fb9d40cb570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58038/0 (socket says 172.21.15.7:58038) 2024-09-10T21:45:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.365+0000 7fb9d3400640 1 -- 172.21.15.7:0/65450064 learned_addr learned my addr 172.21.15.7:0/65450064 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.365+0000 7fb9d3e00640 1 --2- 172.21.15.7:0/65450064 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb9d4073cf0 0x7fb9d414df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:44.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.365+0000 7fb9d8c00640 1 --2- 172.21.15.7:0/65450064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9d415e7b0 0x7fb9d40acb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.366+0000 7fb9d3400640 1 -- 172.21.15.7:0/65450064 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb9d4073cf0 msgr2=0x7fb9d414df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.366+0000 7fb9d3400640 1 --2- 172.21.15.7:0/65450064 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb9d4073cf0 0x7fb9d414df00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.366+0000 7fb9d3400640 1 -- 172.21.15.7:0/65450064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9d415e7b0 msgr2=0x7fb9d40acb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.366+0000 7fb9d3400640 1 --2- 172.21.15.7:0/65450064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9d415e7b0 0x7fb9d40acb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.366+0000 7fb9d3400640 1 -- 172.21.15.7:0/65450064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9d41508c0 con 0x7fb9d40cc940 2024-09-10T21:45:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.366+0000 7fb9d3400640 1 --2- 172.21.15.7:0/65450064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cc940 0x7fb9d40cb570 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000c340 tx=0x7fb9c000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.366+0000 7fb9cbe00640 1 -- 172.21.15.7:0/65450064 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0017070 con 0x7fb9d40cc940 2024-09-10T21:45:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.366+0000 7fb9cbe00640 1 -- 172.21.15.7:0/65450064 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9c00053c0 con 0x7fb9d40cc940 2024-09-10T21:45:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.366+0000 7fb9cbe00640 1 -- 172.21.15.7:0/65450064 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0003870 con 0x7fb9d40cc940 2024-09-10T21:45:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.366+0000 7fb9dac00640 1 -- 172.21.15.7:0/65450064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9d4150ba0 con 0x7fb9d40cc940 2024-09-10T21:45:44.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.367+0000 7fb9dac00640 1 -- 172.21.15.7:0/65450064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9d4151160 con 0x7fb9d40cc940 2024-09-10T21:45:44.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.368+0000 7fb9dac00640 1 -- 172.21.15.7:0/65450064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9d406fdf0 con 0x7fb9d40cc940 2024-09-10T21:45:44.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.369+0000 7fb9cbe00640 1 -- 172.21.15.7:0/65450064 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb9c0003a10 con 0x7fb9d40cc940 2024-09-10T21:45:44.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.370+0000 7fb9cbe00640 1 --2- 172.21.15.7:0/65450064 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb9b4077f70 0x7fb9b407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:44.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.370+0000 7fb9cbe00640 1 -- 172.21.15.7:0/65450064 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fb9c00993b0 con 0x7fb9d40cc940 2024-09-10T21:45:44.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.370+0000 7fb9d3e00640 1 --2- 172.21.15.7:0/65450064 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb9b4077f70 0x7fb9b407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:44.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.371+0000 7fb9d3e00640 1 --2- 172.21.15.7:0/65450064 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb9b4077f70 0x7fb9b407a430 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4033000 tx=0x7fb9c4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:44.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.374+0000 7fb9cbe00640 1 -- 172.21.15.7:0/65450064 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb9c00617c0 con 0x7fb9d40cc940 2024-09-10T21:45:44.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.581+0000 7fb9dac00640 1 -- 172.21.15.7:0/65450064 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb9d40af4c0 con 0x7fb9d40cc940 2024-09-10T21:45:44.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.582+0000 7fb9cbe00640 1 -- 172.21.15.7:0/65450064 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v58) ==== 80+0+0 (secure 0 0 0) 0x7fb9c00661f0 con 0x7fb9d40cc940 2024-09-10T21:45:44.583 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-10T21:45:44.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.584+0000 7fb9dac00640 1 -- 172.21.15.7:0/65450064 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb9b4077f70 msgr2=0x7fb9b407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:44.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.584+0000 7fb9dac00640 1 --2- 172.21.15.7:0/65450064 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb9b4077f70 0x7fb9b407a430 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb9c4033000 tx=0x7fb9c4004730 comp rx=0 tx=0).stop 2024-09-10T21:45:44.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.584+0000 7fb9dac00640 1 -- 172.21.15.7:0/65450064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cc940 msgr2=0x7fb9d40cb570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:44.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.584+0000 7fb9dac00640 1 --2- 172.21.15.7:0/65450064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cc940 0x7fb9d40cb570 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fb9c000c340 tx=0x7fb9c000c800 comp rx=0 tx=0).stop 2024-09-10T21:45:44.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.585+0000 7fb9dac00640 1 -- 172.21.15.7:0/65450064 shutdown_connections 2024-09-10T21:45:44.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.585+0000 7fb9dac00640 1 --2- 172.21.15.7:0/65450064 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb9b4077f70 0x7fb9b407a430 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:44.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.585+0000 7fb9dac00640 1 --2- 172.21.15.7:0/65450064 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb9d415e7b0 0x7fb9d40acb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:44.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.585+0000 7fb9dac00640 1 --2- 172.21.15.7:0/65450064 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb9d40cc940 0x7fb9d40cb570 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:44.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.585+0000 7fb9dac00640 1 --2- 172.21.15.7:0/65450064 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb9d4073cf0 0x7fb9d414df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:44.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.585+0000 7fb9dac00640 1 -- 172.21.15.7:0/65450064 >> 172.21.15.7:0/65450064 conn(0x7fb9d40b1660 msgr2=0x7fb9d415fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:44.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.585+0000 7fb9dac00640 1 -- 172.21.15.7:0/65450064 shutdown_connections 2024-09-10T21:45:44.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:44.585+0000 7fb9dac00640 1 -- 172.21.15.7:0/65450064 wait complete. 2024-09-10T21:45:44.601 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:45:44.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:45:44.601 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:45:44.702 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:45:44.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:45:44.702 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:45:44.747 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:45:44.747 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:45:44.747 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:45:44.784 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:45:44.785 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:45:44.785 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:45:44.785 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:45:44.824 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:45:44.824 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:45:44.824 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:45:44.824 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:45:44.861 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:45:44.862 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:45:45.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.023+0000 7ff5c2c00640 1 -- 172.21.15.7:0/1102033180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc15bc90 msgr2=0x7ff5bc0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.023+0000 7ff5c2c00640 1 --2- 172.21.15.7:0/1102033180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc15bc90 0x7ff5bc0ccef0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc15fa90 tx=0x7ff5ac02a540 comp rx=0 tx=0).stop 2024-09-10T21:45:45.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.023+0000 7ff5c2c00640 1 -- 172.21.15.7:0/1102033180 shutdown_connections 2024-09-10T21:45:45.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.023+0000 7ff5c2c00640 1 --2- 172.21.15.7:0/1102033180 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff5bc15fda0 0x7ff5bc1601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.023+0000 7ff5c2c00640 1 --2- 172.21.15.7:0/1102033180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc15bc90 0x7ff5bc0ccef0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.023+0000 7ff5c2c00640 1 -- 172.21.15.7:0/1102033180 >> 172.21.15.7:0/1102033180 conn(0x7ff5bc0b16d0 msgr2=0x7ff5bc0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:45.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.023+0000 7ff5c2c00640 1 -- 172.21.15.7:0/1102033180 shutdown_connections 2024-09-10T21:45:45.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.023+0000 7ff5c2c00640 1 -- 172.21.15.7:0/1102033180 wait complete. 2024-09-10T21:45:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.025+0000 7ff5c2c00640 1 Processor -- start 2024-09-10T21:45:45.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.025+0000 7ff5c2c00640 1 -- start start 2024-09-10T21:45:45.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.026+0000 7ff5c2c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5bc0cd750 0x7ff5bc1f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:45.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.026+0000 7ff5c2c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc15bc90 0x7ff5bc1565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:45.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.026+0000 7ff5c2c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff5bc15fda0 0x7ff5bc0c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:45.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.026+0000 7ff5c2c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc06fe70 con 0x7ff5bc15bc90 2024-09-10T21:45:45.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.026+0000 7ff5c2c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff5bc06fcf0 con 0x7ff5bc0cd750 2024-09-10T21:45:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.026+0000 7ff5c2c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ff5bc06fff0 con 0x7ff5bc15fda0 2024-09-10T21:45:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.026+0000 7ff5b3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc15bc90 0x7ff5bc1565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.026+0000 7ff5b3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc15bc90 0x7ff5bc1565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58054/0 (socket says 172.21.15.7:58054) 2024-09-10T21:45:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.026+0000 7ff5b3400640 1 -- 172.21.15.7:0/3290602950 learned_addr learned my addr 172.21.15.7:0/3290602950 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.026+0000 7ff5bbe00640 1 --2- 172.21.15.7:0/3290602950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5bc0cd750 0x7ff5bc1f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.026+0000 7ff5c0c00640 1 --2- 172.21.15.7:0/3290602950 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff5bc15fda0 0x7ff5bc0c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.027+0000 7ff5b3400640 1 -- 172.21.15.7:0/3290602950 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff5bc15fda0 msgr2=0x7ff5bc0c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.027+0000 7ff5b3400640 1 --2- 172.21.15.7:0/3290602950 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff5bc15fda0 0x7ff5bc0c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.027+0000 7ff5b3400640 1 -- 172.21.15.7:0/3290602950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5bc0cd750 msgr2=0x7ff5bc1f24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.027+0000 7ff5b3400640 1 --2- 172.21.15.7:0/3290602950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5bc0cd750 0x7ff5bc1f24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.027+0000 7ff5b3400640 1 -- 172.21.15.7:0/3290602950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5bc1f3760 con 0x7ff5bc15bc90 2024-09-10T21:45:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.027+0000 7ff5b3400640 1 --2- 172.21.15.7:0/3290602950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc15bc90 0x7ff5bc1565d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400c450 tx=0x7ff5a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.027+0000 7ff5b8c00640 1 -- 172.21.15.7:0/3290602950 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4017070 con 0x7ff5bc15bc90 2024-09-10T21:45:45.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.027+0000 7ff5b8c00640 1 -- 172.21.15.7:0/3290602950 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5a40053c0 con 0x7ff5bc15bc90 2024-09-10T21:45:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.027+0000 7ff5c2c00640 1 -- 172.21.15.7:0/3290602950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5bc0c7c20 con 0x7ff5bc15bc90 2024-09-10T21:45:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.027+0000 7ff5b8c00640 1 -- 172.21.15.7:0/3290602950 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4003850 con 0x7ff5bc15bc90 2024-09-10T21:45:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.027+0000 7ff5c2c00640 1 -- 172.21.15.7:0/3290602950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5bc1f2e30 con 0x7ff5bc15bc90 2024-09-10T21:45:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.029+0000 7ff5c2c00640 1 -- 172.21.15.7:0/3290602950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5bc0704a0 con 0x7ff5bc15bc90 2024-09-10T21:45:45.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.030+0000 7ff5b8c00640 1 -- 172.21.15.7:0/3290602950 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff5a40039f0 con 0x7ff5bc15bc90 2024-09-10T21:45:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.030+0000 7ff5b8c00640 1 --2- 172.21.15.7:0/3290602950 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff598077f70 0x7ff59807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:45.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.031+0000 7ff5b8c00640 1 -- 172.21.15.7:0/3290602950 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7ff5a40993d0 con 0x7ff5bc15bc90 2024-09-10T21:45:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.031+0000 7ff5bbe00640 1 --2- 172.21.15.7:0/3290602950 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff598077f70 0x7ff59807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:45.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.032+0000 7ff5bbe00640 1 --2- 172.21.15.7:0/3290602950 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff598077f70 0x7ff59807a430 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac033000 tx=0x7ff5ac004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:45.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.035+0000 7ff5b8c00640 1 -- 172.21.15.7:0/3290602950 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff5a4061880 con 0x7ff5bc15bc90 2024-09-10T21:45:45.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.269+0000 7ff5c2c00640 1 -- 172.21.15.7:0/3290602950 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5bc0b56e0 con 0x7ff5bc15bc90 2024-09-10T21:45:45.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.269+0000 7ff5b8c00640 1 -- 172.21.15.7:0/3290602950 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5228 (secure 0 0 0) 0x7ff5a40662b0 con 0x7ff5bc15bc90 2024-09-10T21:45:45.270 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:45:45.271 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":37,"btime":"2024-09-10T21:45:19:514745+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5473,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/2853074495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2853074495},{"type":"v1","addr":"172.21.15.133:6836","nonce":2853074495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3498706462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3498706462},{"type":"v1","addr":"172.21.15.133:6837","nonce":3498706462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/521190705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":521190705},{"type":"v1","addr":"172.21.15.7:6835","nonce":521190705}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:44:30.804578+0000","modified":"2024-09-10T21:45:19.415172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235016},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235016":{"gid":235016,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":15,"addr":"172.21.15.7:6834/364473218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":364473218},{"type":"v1","addr":"172.21.15.7:6834","nonce":364473218}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235016,"qdb_cluster":[235016]},"id":4}]} 2024-09-10T21:45:45.271 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 37 2024-09-10T21:45:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.272+0000 7ff5c2c00640 1 -- 172.21.15.7:0/3290602950 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff598077f70 msgr2=0x7ff59807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.272+0000 7ff5c2c00640 1 --2- 172.21.15.7:0/3290602950 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff598077f70 0x7ff59807a430 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac033000 tx=0x7ff5ac004730 comp rx=0 tx=0).stop 2024-09-10T21:45:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.272+0000 7ff5c2c00640 1 -- 172.21.15.7:0/3290602950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc15bc90 msgr2=0x7ff5bc1565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.272+0000 7ff5c2c00640 1 --2- 172.21.15.7:0/3290602950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc15bc90 0x7ff5bc1565d0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7ff5a400c450 tx=0x7ff5a400c910 comp rx=0 tx=0).stop 2024-09-10T21:45:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.272+0000 7ff5c2c00640 1 -- 172.21.15.7:0/3290602950 shutdown_connections 2024-09-10T21:45:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.272+0000 7ff5c2c00640 1 --2- 172.21.15.7:0/3290602950 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff598077f70 0x7ff59807a430 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.272+0000 7ff5c2c00640 1 --2- 172.21.15.7:0/3290602950 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff5bc15fda0 0x7ff5bc0c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.272+0000 7ff5c2c00640 1 --2- 172.21.15.7:0/3290602950 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5bc15bc90 0x7ff5bc1565d0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.272+0000 7ff5c2c00640 1 --2- 172.21.15.7:0/3290602950 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5bc0cd750 0x7ff5bc1f24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.272+0000 7ff5c2c00640 1 -- 172.21.15.7:0/3290602950 >> 172.21.15.7:0/3290602950 conn(0x7ff5bc0b16d0 msgr2=0x7ff5bc072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.272+0000 7ff5c2c00640 1 -- 172.21.15.7:0/3290602950 shutdown_connections 2024-09-10T21:45:45.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.272+0000 7ff5c2c00640 1 -- 172.21.15.7:0/3290602950 wait complete. 2024-09-10T21:45:45.289 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:45:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.449+0000 7fce60e00640 1 -- 172.21.15.7:0/2267196540 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce5c15fda0 msgr2=0x7fce5c1601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.449+0000 7fce60e00640 1 --2- 172.21.15.7:0/2267196540 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce5c15fda0 0x7fce5c1601d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fce5c15fa90 tx=0x7fce5402a800 comp rx=0 tx=0).stop 2024-09-10T21:45:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.449+0000 7fce60e00640 1 -- 172.21.15.7:0/2267196540 shutdown_connections 2024-09-10T21:45:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.449+0000 7fce60e00640 1 --2- 172.21.15.7:0/2267196540 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce5c15fda0 0x7fce5c1601d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.449+0000 7fce60e00640 1 --2- 172.21.15.7:0/2267196540 >> v2:172.21.15.133:3300/0 conn(0x7fce5c0cd750 0x7fce5c0ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.449+0000 7fce60e00640 1 -- 172.21.15.7:0/2267196540 >> 172.21.15.7:0/2267196540 conn(0x7fce5c0b16d0 msgr2=0x7fce5c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.449+0000 7fce60e00640 1 -- 172.21.15.7:0/2267196540 shutdown_connections 2024-09-10T21:45:45.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.449+0000 7fce60e00640 1 -- 172.21.15.7:0/2267196540 wait complete. 2024-09-10T21:45:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.451+0000 7fce60e00640 1 Processor -- start 2024-09-10T21:45:45.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.451+0000 7fce60e00640 1 -- start start 2024-09-10T21:45:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.452+0000 7fce60e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce5c0cd750 0x7fce5c15e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:45.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.452+0000 7fce60e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce5c15bc90 0x7fce5c14fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.452+0000 7fce60e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce5c15fda0 0x7fce5c0b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.452+0000 7fce60e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c06fe70 con 0x7fce5c15fda0 2024-09-10T21:45:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.452+0000 7fce60e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fce5c06fcf0 con 0x7fce5c15bc90 2024-09-10T21:45:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.452+0000 7fce60e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fce5c06fff0 con 0x7fce5c0cd750 2024-09-10T21:45:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.452+0000 7fce5a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce5c15fda0 0x7fce5c0b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.452+0000 7fce5a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce5c15fda0 0x7fce5c0b48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58068/0 (socket says 172.21.15.7:58068) 2024-09-10T21:45:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.452+0000 7fce5a800640 1 -- 172.21.15.7:0/1234704090 learned_addr learned my addr 172.21.15.7:0/1234704090 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.452+0000 7fce59e00640 1 --2- 172.21.15.7:0/1234704090 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce5c0cd750 0x7fce5c15e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.452+0000 7fce5a800640 1 -- 172.21.15.7:0/1234704090 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce5c0cd750 msgr2=0x7fce5c15e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.453+0000 7fce5a800640 1 --2- 172.21.15.7:0/1234704090 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce5c0cd750 0x7fce5c15e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.453+0000 7fce5a800640 1 -- 172.21.15.7:0/1234704090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce5c15bc90 msgr2=0x7fce5c14fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.452+0000 7fce59400640 1 --2- 172.21.15.7:0/1234704090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce5c15bc90 0x7fce5c14fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.453+0000 7fce5a800640 1 --2- 172.21.15.7:0/1234704090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce5c15bc90 0x7fce5c14fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.453+0000 7fce5a800640 1 -- 172.21.15.7:0/1234704090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce5c15ed70 con 0x7fce5c15fda0 2024-09-10T21:45:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.453+0000 7fce59e00640 1 --2- 172.21.15.7:0/1234704090 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce5c0cd750 0x7fce5c15e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:45:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.453+0000 7fce59400640 1 --2- 172.21.15.7:0/1234704090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce5c15bc90 0x7fce5c14fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:45:45.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.453+0000 7fce5a800640 1 --2- 172.21.15.7:0/1234704090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce5c15fda0 0x7fce5c0b48d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fce54009ca0 tx=0x7fce54033e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:45.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.453+0000 7fce52000640 1 -- 172.21.15.7:0/1234704090 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce54037050 con 0x7fce5c15fda0 2024-09-10T21:45:45.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.453+0000 7fce52000640 1 -- 172.21.15.7:0/1234704090 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce54043070 con 0x7fce5c15fda0 2024-09-10T21:45:45.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.453+0000 7fce52000640 1 -- 172.21.15.7:0/1234704090 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce540044b0 con 0x7fce5c15fda0 2024-09-10T21:45:45.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.453+0000 7fce60e00640 1 -- 172.21.15.7:0/1234704090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce5c0c7dd0 con 0x7fce5c15fda0 2024-09-10T21:45:45.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.453+0000 7fce60e00640 1 -- 172.21.15.7:0/1234704090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce5c0c8390 con 0x7fce5c15fda0 2024-09-10T21:45:45.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.454+0000 7fce60e00640 1 -- 172.21.15.7:0/1234704090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce5c06e730 con 0x7fce5c15fda0 2024-09-10T21:45:45.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.456+0000 7fce52000640 1 -- 172.21.15.7:0/1234704090 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fce54031070 con 0x7fce5c15fda0 2024-09-10T21:45:45.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.456+0000 7fce52000640 1 --2- 172.21.15.7:0/1234704090 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fce3c077f70 0x7fce3c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:45.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.457+0000 7fce52000640 1 -- 172.21.15.7:0/1234704090 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fce540ba470 con 0x7fce5c15fda0 2024-09-10T21:45:45.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.457+0000 7fce59e00640 1 --2- 172.21.15.7:0/1234704090 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fce3c077f70 0x7fce3c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:45.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.458+0000 7fce59e00640 1 --2- 172.21.15.7:0/1234704090 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fce3c077f70 0x7fce3c07a430 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fce4c001220 tx=0x7fce4c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:45.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.460+0000 7fce52000640 1 -- 172.21.15.7:0/1234704090 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fce5c06e730 con 0x7fce5c15fda0 2024-09-10T21:45:45.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.693+0000 7fce60e00640 1 -- 172.21.15.7:0/1234704090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fce5c0cc910 con 0x7fce5c15fda0 2024-09-10T21:45:45.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.694+0000 7fce52000640 1 -- 172.21.15.7:0/1234704090 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5228 (secure 0 0 0) 0x7fce5c0cc910 con 0x7fce5c15fda0 2024-09-10T21:45:45.695 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:45:45.695 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":37,"btime":"2024-09-10T21:45:19:514745+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5473,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/2853074495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2853074495},{"type":"v1","addr":"172.21.15.133:6836","nonce":2853074495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3498706462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3498706462},{"type":"v1","addr":"172.21.15.133:6837","nonce":3498706462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/521190705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":521190705},{"type":"v1","addr":"172.21.15.7:6835","nonce":521190705}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:44:30.804578+0000","modified":"2024-09-10T21:45:19.415172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235016},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235016":{"gid":235016,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":15,"addr":"172.21.15.7:6834/364473218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":364473218},{"type":"v1","addr":"172.21.15.7:6834","nonce":364473218}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235016,"qdb_cluster":[235016]},"id":4}]} 2024-09-10T21:45:45.695 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 37 2024-09-10T21:45:45.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.696+0000 7fce60e00640 1 -- 172.21.15.7:0/1234704090 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fce3c077f70 msgr2=0x7fce3c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.696+0000 7fce60e00640 1 --2- 172.21.15.7:0/1234704090 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fce3c077f70 0x7fce3c07a430 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fce4c001220 tx=0x7fce4c009040 comp rx=0 tx=0).stop 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.696+0000 7fce60e00640 1 -- 172.21.15.7:0/1234704090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce5c15fda0 msgr2=0x7fce5c0b48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.696+0000 7fce60e00640 1 --2- 172.21.15.7:0/1234704090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce5c15fda0 0x7fce5c0b48d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fce54009ca0 tx=0x7fce54033e50 comp rx=0 tx=0).stop 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.696+0000 7fce60e00640 1 -- 172.21.15.7:0/1234704090 shutdown_connections 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.696+0000 7fce60e00640 1 --2- 172.21.15.7:0/1234704090 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fce3c077f70 0x7fce3c07a430 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.696+0000 7fce60e00640 1 --2- 172.21.15.7:0/1234704090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce5c15bc90 0x7fce5c14fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.696+0000 7fce60e00640 1 --2- 172.21.15.7:0/1234704090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce5c15fda0 0x7fce5c0b48d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.696+0000 7fce60e00640 1 --2- 172.21.15.7:0/1234704090 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce5c0cd750 0x7fce5c15e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.696+0000 7fce60e00640 1 -- 172.21.15.7:0/1234704090 >> 172.21.15.7:0/1234704090 conn(0x7fce5c0b16d0 msgr2=0x7fce5c0c5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.696+0000 7fce60e00640 1 -- 172.21.15.7:0/1234704090 shutdown_connections 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.696+0000 7fce60e00640 1 -- 172.21.15.7:0/1234704090 wait complete. 2024-09-10T21:45:45.713 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:45:45.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.874+0000 7fc8e4c00640 1 -- 172.21.15.7:0/1606115946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8c4003cb0 msgr2=0x7fc8c40040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.874+0000 7fc8e4c00640 1 --2- 172.21.15.7:0/1606115946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8c4003cb0 0x7fc8c40040b0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0001830 tx=0x7fc8c4005830 comp rx=0 tx=0).stop 2024-09-10T21:45:45.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.874+0000 7fc8de800640 1 -- 172.21.15.7:0/1606115946 reap_dead start 2024-09-10T21:45:45.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.874+0000 7fc8e4c00640 1 -- 172.21.15.7:0/1606115946 shutdown_connections 2024-09-10T21:45:45.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.874+0000 7fc8e4c00640 1 -- 172.21.15.7:0/1606115946 >> 172.21.15.7:0/1606115946 conn(0x7fc8e00b16d0 msgr2=0x7fc8e00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:45.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.874+0000 7fc8e4c00640 1 -- 172.21.15.7:0/1606115946 shutdown_connections 2024-09-10T21:45:45.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.874+0000 7fc8e4c00640 1 -- 172.21.15.7:0/1606115946 wait complete. 2024-09-10T21:45:45.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.875+0000 7fc8e4c00640 1 Processor -- start 2024-09-10T21:45:45.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.876+0000 7fc8e4c00640 1 -- start start 2024-09-10T21:45:45.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.876+0000 7fc8e4c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8c4003cb0 0x7fc8e01476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:45.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.876+0000 7fc8e4c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8e014ad00 0x7fc8e01f6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:45.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.877+0000 7fc8e4c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8e01f6770 0x7fc8e015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:45.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.877+0000 7fc8e4c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e00cd2f0 con 0x7fc8e01f6770 2024-09-10T21:45:45.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.877+0000 7fc8e4c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e00cd170 con 0x7fc8c4003cb0 2024-09-10T21:45:45.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.877+0000 7fc8e4c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc8e0073cf0 con 0x7fc8e014ad00 2024-09-10T21:45:45.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.877+0000 7fc8de800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8e01f6770 0x7fc8e015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:45.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.877+0000 7fc8de800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8e01f6770 0x7fc8e015ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58088/0 (socket says 172.21.15.7:58088) 2024-09-10T21:45:45.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.877+0000 7fc8de800640 1 -- 172.21.15.7:0/1225511657 learned_addr learned my addr 172.21.15.7:0/1225511657 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:45.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.877+0000 7fc8dd400640 1 --2- 172.21.15.7:0/1225511657 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8e014ad00 0x7fc8e01f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:45.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.877+0000 7fc8dde00640 1 --2- 172.21.15.7:0/1225511657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8c4003cb0 0x7fc8e01476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:45.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.877+0000 7fc8de800640 1 -- 172.21.15.7:0/1225511657 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8e014ad00 msgr2=0x7fc8e01f6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.877+0000 7fc8de800640 1 --2- 172.21.15.7:0/1225511657 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8e014ad00 0x7fc8e01f6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.877+0000 7fc8de800640 1 -- 172.21.15.7:0/1225511657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8c4003cb0 msgr2=0x7fc8e01476b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.877+0000 7fc8de800640 1 --2- 172.21.15.7:0/1225511657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8c4003cb0 0x7fc8e01476b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.877+0000 7fc8de800640 1 -- 172.21.15.7:0/1225511657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e014b180 con 0x7fc8e01f6770 2024-09-10T21:45:45.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.878+0000 7fc8de800640 1 --2- 172.21.15.7:0/1225511657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8e01f6770 0x7fc8e015ed80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc8e006dac0 tx=0x7fc8d8006420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:45.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.878+0000 7fc8d6000640 1 -- 172.21.15.7:0/1225511657 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8015020 con 0x7fc8e01f6770 2024-09-10T21:45:45.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.878+0000 7fc8d6000640 1 -- 172.21.15.7:0/1225511657 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8d80119c0 con 0x7fc8e01f6770 2024-09-10T21:45:45.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.878+0000 7fc8d6000640 1 -- 172.21.15.7:0/1225511657 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8011e10 con 0x7fc8e01f6770 2024-09-10T21:45:45.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.878+0000 7fc8e4c00640 1 -- 172.21.15.7:0/1225511657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e015c340 con 0x7fc8e01f6770 2024-09-10T21:45:45.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.878+0000 7fc8e4c00640 1 -- 172.21.15.7:0/1225511657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e01f5a30 con 0x7fc8e01f6770 2024-09-10T21:45:45.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.879+0000 7fc8e4c00640 1 -- 172.21.15.7:0/1225511657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8e0074250 con 0x7fc8e01f6770 2024-09-10T21:45:45.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.880+0000 7fc8d6000640 1 -- 172.21.15.7:0/1225511657 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc8d80103c0 con 0x7fc8e01f6770 2024-09-10T21:45:45.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.882+0000 7fc8d6000640 1 --2- 172.21.15.7:0/1225511657 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc8c0077f70 0x7fc8c007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:45.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.882+0000 7fc8d6000640 1 -- 172.21.15.7:0/1225511657 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fc8d809a6e0 con 0x7fc8e01f6770 2024-09-10T21:45:45.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.882+0000 7fc8dde00640 1 --2- 172.21.15.7:0/1225511657 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc8c0077f70 0x7fc8c007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:45.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.883+0000 7fc8dde00640 1 --2- 172.21.15.7:0/1225511657 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc8c0077f70 0x7fc8c007a430 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0032000 tx=0x7fc8d0001360 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:45.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:45.886+0000 7fc8d6000640 1 -- 172.21.15.7:0/1225511657 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc8d801f030 con 0x7fc8e01f6770 2024-09-10T21:45:46.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.082+0000 7fc8e4c00640 1 -- 172.21.15.7:0/1225511657 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8e0157140 con 0x7fc8e01f6770 2024-09-10T21:45:46.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.083+0000 7fc8d6000640 1 -- 172.21.15.7:0/1225511657 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16689 (secure 0 0 0) 0x7fc8d8001410 con 0x7fc8e01f6770 2024-09-10T21:45:46.085 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:45:46.085 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":58,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:44:32.373354+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:44:28.642474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T21:44:29.765081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:45:46.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.085+0000 7fc8e4c00640 1 -- 172.21.15.7:0/1225511657 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc8c0077f70 msgr2=0x7fc8c007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.085+0000 7fc8e4c00640 1 --2- 172.21.15.7:0/1225511657 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc8c0077f70 0x7fc8c007a430 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0032000 tx=0x7fc8d0001360 comp rx=0 tx=0).stop 2024-09-10T21:45:46.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.086+0000 7fc8e4c00640 1 -- 172.21.15.7:0/1225511657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8e01f6770 msgr2=0x7fc8e015ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.086+0000 7fc8e4c00640 1 --2- 172.21.15.7:0/1225511657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8e01f6770 0x7fc8e015ed80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fc8e006dac0 tx=0x7fc8d8006420 comp rx=0 tx=0).stop 2024-09-10T21:45:46.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.086+0000 7fc8e4c00640 1 -- 172.21.15.7:0/1225511657 shutdown_connections 2024-09-10T21:45:46.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.086+0000 7fc8e4c00640 1 --2- 172.21.15.7:0/1225511657 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc8c0077f70 0x7fc8c007a430 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.086+0000 7fc8e4c00640 1 --2- 172.21.15.7:0/1225511657 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc8e01f6770 0x7fc8e015ed80 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.086+0000 7fc8e4c00640 1 --2- 172.21.15.7:0/1225511657 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc8e014ad00 0x7fc8e01f6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.086+0000 7fc8e4c00640 1 --2- 172.21.15.7:0/1225511657 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc8c4003cb0 0x7fc8e01476b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.086+0000 7fc8e4c00640 1 -- 172.21.15.7:0/1225511657 >> 172.21.15.7:0/1225511657 conn(0x7fc8e00b16d0 msgr2=0x7fc8e01590d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.086+0000 7fc8e4c00640 1 -- 172.21.15.7:0/1225511657 shutdown_connections 2024-09-10T21:45:46.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.086+0000 7fc8e4c00640 1 -- 172.21.15.7:0/1225511657 wait complete. 2024-09-10T21:45:46.103 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:45:46.103 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:45:46.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.263+0000 7f428da00640 1 -- 172.21.15.7:0/1115252365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f426c004020 msgr2=0x7f426c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.263+0000 7f428da00640 1 --2- 172.21.15.7:0/1115252365 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f426c004020 0x7f426c004800 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f42880cd6a0 tx=0x7f426c005830 comp rx=0 tx=0).stop 2024-09-10T21:45:46.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.263+0000 7f4287400640 1 -- 172.21.15.7:0/1115252365 reap_dead start 2024-09-10T21:45:46.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.263+0000 7f428da00640 1 -- 172.21.15.7:0/1115252365 shutdown_connections 2024-09-10T21:45:46.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.263+0000 7f428da00640 1 -- 172.21.15.7:0/1115252365 >> 172.21.15.7:0/1115252365 conn(0x7f42880b16d0 msgr2=0x7f42880b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:46.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.263+0000 7f428da00640 1 -- 172.21.15.7:0/1115252365 shutdown_connections 2024-09-10T21:45:46.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.263+0000 7f428da00640 1 -- 172.21.15.7:0/1115252365 wait complete. 2024-09-10T21:45:46.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.265+0000 7f428da00640 1 Processor -- start 2024-09-10T21:45:46.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.265+0000 7f428da00640 1 -- start start 2024-09-10T21:45:46.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.266+0000 7f428da00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f426c004020 0x7f42881f4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.266+0000 7f428da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42880c97d0 0x7f4288159420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.266+0000 7f428da00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4288159960 0x7f42880b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.266+0000 7f428da00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42880cdc90 con 0x7f42880c97d0 2024-09-10T21:45:46.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.266+0000 7f428da00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f428815e680 con 0x7f4288159960 2024-09-10T21:45:46.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.266+0000 7f428da00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f42880cde10 con 0x7f426c004020 2024-09-10T21:45:46.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.266+0000 7f427e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42880c97d0 0x7f4288159420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:46.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.266+0000 7f427e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42880c97d0 0x7f4288159420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58112/0 (socket says 172.21.15.7:58112) 2024-09-10T21:45:46.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.266+0000 7f427e000640 1 -- 172.21.15.7:0/13149668 learned_addr learned my addr 172.21.15.7:0/13149668 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:46.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.266+0000 7f4287400640 1 --2- 172.21.15.7:0/13149668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4288159960 0x7f42880b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:46.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.266+0000 7f4286a00640 1 --2- 172.21.15.7:0/13149668 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f426c004020 0x7f42881f4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:46.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.267+0000 7f427e000640 1 -- 172.21.15.7:0/13149668 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f426c004020 msgr2=0x7f42881f4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.267+0000 7f427e000640 1 --2- 172.21.15.7:0/13149668 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f426c004020 0x7f42881f4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.267+0000 7f427e000640 1 -- 172.21.15.7:0/13149668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4288159960 msgr2=0x7f42880b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.267+0000 7f427e000640 1 --2- 172.21.15.7:0/13149668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4288159960 0x7f42880b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.267+0000 7f427e000640 1 -- 172.21.15.7:0/13149668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4288073800 con 0x7f42880c97d0 2024-09-10T21:45:46.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.267+0000 7f427e000640 1 --2- 172.21.15.7:0/13149668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42880c97d0 0x7f4288159420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f427002b070 tx=0x7f4270006c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.267+0000 7f427f400640 1 -- 172.21.15.7:0/13149668 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270032030 con 0x7f42880c97d0 2024-09-10T21:45:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.267+0000 7f427f400640 1 -- 172.21.15.7:0/13149668 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4270003af0 con 0x7f42880c97d0 2024-09-10T21:45:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.267+0000 7f427f400640 1 -- 172.21.15.7:0/13149668 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270037450 con 0x7f42880c97d0 2024-09-10T21:45:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.267+0000 7f428da00640 1 -- 172.21.15.7:0/13149668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4288073ae0 con 0x7f42880c97d0 2024-09-10T21:45:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.267+0000 7f428da00640 1 -- 172.21.15.7:0/13149668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42881f4a30 con 0x7f42880c97d0 2024-09-10T21:45:46.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.268+0000 7f428da00640 1 -- 172.21.15.7:0/13149668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f428815e890 con 0x7f42880c97d0 2024-09-10T21:45:46.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.269+0000 7f427f400640 1 -- 172.21.15.7:0/13149668 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f42700375f0 con 0x7f42880c97d0 2024-09-10T21:45:46.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.270+0000 7f427f400640 1 --2- 172.21.15.7:0/13149668 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4264077f70 0x7f426407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.270+0000 7f427f400640 1 -- 172.21.15.7:0/13149668 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f42700ba540 con 0x7f42880c97d0 2024-09-10T21:45:46.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.271+0000 7f4286a00640 1 --2- 172.21.15.7:0/13149668 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4264077f70 0x7f426407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:46.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.272+0000 7f4286a00640 1 --2- 172.21.15.7:0/13149668 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4264077f70 0x7f426407a430 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4278009ab0 tx=0x7f4278005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:46.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.275+0000 7f427f400640 1 -- 172.21.15.7:0/13149668 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f42700830b0 con 0x7f42880c97d0 2024-09-10T21:45:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.507+0000 7f428da00640 1 -- 172.21.15.7:0/13149668 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f428815c980 con 0x7f42880c97d0 2024-09-10T21:45:46.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.507+0000 7f427f400640 1 -- 172.21.15.7:0/13149668 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5228 (secure 0 0 0) 0x7f42700829b0 con 0x7f42880c97d0 2024-09-10T21:45:46.509 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:45:46.509 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":37,"btime":"2024-09-10T21:45:19:514745+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5473,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/2853074495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2853074495},{"type":"v1","addr":"172.21.15.133:6836","nonce":2853074495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3498706462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3498706462},{"type":"v1","addr":"172.21.15.133:6837","nonce":3498706462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/521190705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":521190705},{"type":"v1","addr":"172.21.15.7:6835","nonce":521190705}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:44:30.804578+0000","modified":"2024-09-10T21:45:19.415172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235016},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235016":{"gid":235016,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":15,"addr":"172.21.15.7:6834/364473218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":364473218},{"type":"v1","addr":"172.21.15.7:6834","nonce":364473218}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235016,"qdb_cluster":[235016]},"id":4}]} 2024-09-10T21:45:46.509 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 37 2024-09-10T21:45:46.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.510+0000 7f428da00640 1 -- 172.21.15.7:0/13149668 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4264077f70 msgr2=0x7f426407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.510+0000 7f428da00640 1 --2- 172.21.15.7:0/13149668 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4264077f70 0x7f426407a430 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f4278009ab0 tx=0x7f4278005f90 comp rx=0 tx=0).stop 2024-09-10T21:45:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.510+0000 7f428da00640 1 -- 172.21.15.7:0/13149668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42880c97d0 msgr2=0x7f4288159420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.510+0000 7f428da00640 1 --2- 172.21.15.7:0/13149668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42880c97d0 0x7f4288159420 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f427002b070 tx=0x7f4270006c30 comp rx=0 tx=0).stop 2024-09-10T21:45:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.510+0000 7f428da00640 1 -- 172.21.15.7:0/13149668 shutdown_connections 2024-09-10T21:45:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.510+0000 7f428da00640 1 --2- 172.21.15.7:0/13149668 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4264077f70 0x7f426407a430 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.510+0000 7f428da00640 1 --2- 172.21.15.7:0/13149668 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4288159960 0x7f42880b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.510+0000 7f428da00640 1 --2- 172.21.15.7:0/13149668 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f42880c97d0 0x7f4288159420 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.510+0000 7f428da00640 1 --2- 172.21.15.7:0/13149668 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f426c004020 0x7f42881f4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.510+0000 7f428da00640 1 -- 172.21.15.7:0/13149668 >> 172.21.15.7:0/13149668 conn(0x7f42880b16d0 msgr2=0x7f428815db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.511+0000 7f428da00640 1 -- 172.21.15.7:0/13149668 shutdown_connections 2024-09-10T21:45:46.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.511+0000 7f428da00640 1 -- 172.21.15.7:0/13149668 wait complete. 2024-09-10T21:45:46.528 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:45:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.689+0000 7fd8c5400640 1 -- 172.21.15.7:0/2723330575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a4003fd0 msgr2=0x7fd8a4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.689+0000 7fd8c5400640 1 --2- 172.21.15.7:0/2723330575 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a4003fd0 0x7fd8a4004800 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00c260 tx=0x7fd8a4005830 comp rx=0 tx=0).stop 2024-09-10T21:45:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.689+0000 7fd8bf200640 1 -- 172.21.15.7:0/2723330575 reap_dead start 2024-09-10T21:45:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.689+0000 7fd8c5400640 1 -- 172.21.15.7:0/2723330575 shutdown_connections 2024-09-10T21:45:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.689+0000 7fd8c5400640 1 -- 172.21.15.7:0/2723330575 >> 172.21.15.7:0/2723330575 conn(0x7fd8c00b16d0 msgr2=0x7fd8c00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.689+0000 7fd8c5400640 1 -- 172.21.15.7:0/2723330575 shutdown_connections 2024-09-10T21:45:46.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.690+0000 7fd8c5400640 1 -- 172.21.15.7:0/2723330575 wait complete. 2024-09-10T21:45:46.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.691+0000 7fd8c5400640 1 Processor -- start 2024-09-10T21:45:46.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.691+0000 7fd8c5400640 1 -- start start 2024-09-10T21:45:46.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.692+0000 7fd8c5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a4003fd0 0x7fd8c00b5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.692+0000 7fd8c5400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8c00c9340 0x7fd8c00c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.692+0000 7fd8c5400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8c0158010 0x7fd8c01f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.692+0000 7fd8c5400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c00cdae0 con 0x7fd8a4003fd0 2024-09-10T21:45:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.692+0000 7fd8c5400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c00cd960 con 0x7fd8c0158010 2024-09-10T21:45:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.692+0000 7fd8c5400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fd8c00cdc60 con 0x7fd8c00c9340 2024-09-10T21:45:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.692+0000 7fd8be800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a4003fd0 0x7fd8c00b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.692+0000 7fd8be800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a4003fd0 0x7fd8c00b5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58128/0 (socket says 172.21.15.7:58128) 2024-09-10T21:45:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.692+0000 7fd8be800640 1 -- 172.21.15.7:0/2266130961 learned_addr learned my addr 172.21.15.7:0/2266130961 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.693+0000 7fd8bde00640 1 --2- 172.21.15.7:0/2266130961 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8c00c9340 0x7fd8c00c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:46.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.693+0000 7fd8bf200640 1 --2- 172.21.15.7:0/2266130961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8c0158010 0x7fd8c01f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.693+0000 7fd8be800640 1 -- 172.21.15.7:0/2266130961 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8c00c9340 msgr2=0x7fd8c00c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.693+0000 7fd8be800640 1 --2- 172.21.15.7:0/2266130961 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8c00c9340 0x7fd8c00c7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.693+0000 7fd8be800640 1 -- 172.21.15.7:0/2266130961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8c0158010 msgr2=0x7fd8c01f8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.693+0000 7fd8be800640 1 --2- 172.21.15.7:0/2266130961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8c0158010 0x7fd8c01f8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.693+0000 7fd8be800640 1 -- 172.21.15.7:0/2266130961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c00b5840 con 0x7fd8a4003fd0 2024-09-10T21:45:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.693+0000 7fd8bf200640 1 --2- 172.21.15.7:0/2266130961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8c0158010 0x7fd8c01f8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:45:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.693+0000 7fd8be800640 1 --2- 172.21.15.7:0/2266130961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a4003fd0 0x7fd8c00b5140 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0011b10 tx=0x7fd8b0009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.693+0000 7fd8b6a00640 1 -- 172.21.15.7:0/2266130961 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b000fce0 con 0x7fd8a4003fd0 2024-09-10T21:45:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.693+0000 7fd8b6a00640 1 -- 172.21.15.7:0/2266130961 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8b000fe80 con 0x7fd8a4003fd0 2024-09-10T21:45:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.693+0000 7fd8b6a00640 1 -- 172.21.15.7:0/2266130961 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b0004280 con 0x7fd8a4003fd0 2024-09-10T21:45:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.693+0000 7fd8c5400640 1 -- 172.21.15.7:0/2266130961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c00c8580 con 0x7fd8a4003fd0 2024-09-10T21:45:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.694+0000 7fd8c5400640 1 -- 172.21.15.7:0/2266130961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c00c8b40 con 0x7fd8a4003fd0 2024-09-10T21:45:46.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.695+0000 7fd8c5400640 1 -- 172.21.15.7:0/2266130961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8c015e3c0 con 0x7fd8a4003fd0 2024-09-10T21:45:46.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.696+0000 7fd8b6a00640 1 -- 172.21.15.7:0/2266130961 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd8b00035f0 con 0x7fd8a4003fd0 2024-09-10T21:45:46.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.697+0000 7fd8b6a00640 1 --2- 172.21.15.7:0/2266130961 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd8a0077fa0 0x7fd8a007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.697+0000 7fd8b6a00640 1 -- 172.21.15.7:0/2266130961 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7fd8b009a8c0 con 0x7fd8a4003fd0 2024-09-10T21:45:46.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.697+0000 7fd8bde00640 1 --2- 172.21.15.7:0/2266130961 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd8a0077fa0 0x7fd8a007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:46.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.698+0000 7fd8bde00640 1 --2- 172.21.15.7:0/2266130961 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd8a0077fa0 0x7fd8a007a460 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac033000 tx=0x7fd8c015ed80 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:46.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.701+0000 7fd8b6a00640 1 -- 172.21.15.7:0/2266130961 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd8b0063320 con 0x7fd8a4003fd0 2024-09-10T21:45:46.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.936+0000 7fd8c5400640 1 -- 172.21.15.7:0/2266130961 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd8c00b5b00 con 0x7fd8a4003fd0 2024-09-10T21:45:46.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.937+0000 7fd8b6a00640 1 -- 172.21.15.7:0/2266130961 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5228 (secure 0 0 0) 0x7fd8b0062c20 con 0x7fd8a4003fd0 2024-09-10T21:45:46.938 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:45:46.938 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":37,"btime":"2024-09-10T21:45:19:514745+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5473,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/2853074495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2853074495},{"type":"v1","addr":"172.21.15.133:6836","nonce":2853074495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3498706462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3498706462},{"type":"v1","addr":"172.21.15.133:6837","nonce":3498706462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/521190705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":521190705},{"type":"v1","addr":"172.21.15.7:6835","nonce":521190705}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:44:30.804578+0000","modified":"2024-09-10T21:45:19.415172+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235016},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235016":{"gid":235016,"name":"a","rank":0,"incarnation":31,"state":"up:active","state_seq":15,"addr":"172.21.15.7:6834/364473218","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":364473218},{"type":"v1","addr":"172.21.15.7:6834","nonce":364473218}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235016,"qdb_cluster":[235016]},"id":4}]} 2024-09-10T21:45:46.939 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 37 2024-09-10T21:45:46.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.940+0000 7fd8c5400640 1 -- 172.21.15.7:0/2266130961 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd8a0077fa0 msgr2=0x7fd8a007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.940+0000 7fd8c5400640 1 --2- 172.21.15.7:0/2266130961 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd8a0077fa0 0x7fd8a007a460 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac033000 tx=0x7fd8c015ed80 comp rx=0 tx=0).stop 2024-09-10T21:45:46.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.940+0000 7fd8c5400640 1 -- 172.21.15.7:0/2266130961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a4003fd0 msgr2=0x7fd8c00b5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.940+0000 7fd8c5400640 1 --2- 172.21.15.7:0/2266130961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a4003fd0 0x7fd8c00b5140 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fd8b0011b10 tx=0x7fd8b0009f10 comp rx=0 tx=0).stop 2024-09-10T21:45:46.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.940+0000 7fd8c5400640 1 -- 172.21.15.7:0/2266130961 shutdown_connections 2024-09-10T21:45:46.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.940+0000 7fd8c5400640 1 --2- 172.21.15.7:0/2266130961 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd8a0077fa0 0x7fd8a007a460 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.940+0000 7fd8c5400640 1 --2- 172.21.15.7:0/2266130961 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8c0158010 0x7fd8c01f8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.940+0000 7fd8c5400640 1 --2- 172.21.15.7:0/2266130961 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8c00c9340 0x7fd8c00c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.940+0000 7fd8c5400640 1 --2- 172.21.15.7:0/2266130961 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8a4003fd0 0x7fd8c00b5140 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.940+0000 7fd8c5400640 1 -- 172.21.15.7:0/2266130961 >> 172.21.15.7:0/2266130961 conn(0x7fd8c00b16d0 msgr2=0x7fd8c015cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:46.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.940+0000 7fd8c5400640 1 -- 172.21.15.7:0/2266130961 shutdown_connections 2024-09-10T21:45:46.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:46.940+0000 7fd8c5400640 1 -- 172.21.15.7:0/2266130961 wait complete. 2024-09-10T21:45:46.956 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:45:47.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.117+0000 7f2540400640 1 -- 172.21.15.7:0/1998809025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f253815aa30 msgr2=0x7f2538073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:47.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.117+0000 7f2540400640 1 --2- 172.21.15.7:0/1998809025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f253815aa30 0x7f2538073550 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f252c005fd0 tx=0x7f252c02a5d0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.117+0000 7f2540400640 1 -- 172.21.15.7:0/1998809025 shutdown_connections 2024-09-10T21:45:47.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.117+0000 7f2540400640 1 --2- 172.21.15.7:0/1998809025 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25380cbcd0 0x7f25380ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.117+0000 7f2540400640 1 --2- 172.21.15.7:0/1998809025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f253815aa30 0x7f2538073550 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.117+0000 7f2540400640 1 -- 172.21.15.7:0/1998809025 >> 172.21.15.7:0/1998809025 conn(0x7f25380b16d0 msgr2=0x7f25380b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:47.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.117+0000 7f2540400640 1 -- 172.21.15.7:0/1998809025 shutdown_connections 2024-09-10T21:45:47.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.117+0000 7f2540400640 1 -- 172.21.15.7:0/1998809025 wait complete. 2024-09-10T21:45:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.119+0000 7f2540400640 1 Processor -- start 2024-09-10T21:45:47.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.119+0000 7f2540400640 1 -- start start 2024-09-10T21:45:47.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.120+0000 7f2540400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25380cbcd0 0x7f253815b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.120+0000 7f2540400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25380ce860 0x7f2538145900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.120+0000 7f2540400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f253815aa30 0x7f25381f5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.120+0000 7f2540400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f253806ee30 con 0x7f25380cbcd0 2024-09-10T21:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.120+0000 7f2540400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f253806ecb0 con 0x7f253815aa30 2024-09-10T21:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.120+0000 7f2540400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f253815ed60 con 0x7f25380ce860 2024-09-10T21:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.120+0000 7f253da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25380cbcd0 0x7f253815b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.120+0000 7f253da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25380cbcd0 0x7f253815b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58150/0 (socket says 172.21.15.7:58150) 2024-09-10T21:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.120+0000 7f253da00640 1 -- 172.21.15.7:0/160308075 learned_addr learned my addr 172.21.15.7:0/160308075 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.120+0000 7f253d000640 1 --2- 172.21.15.7:0/160308075 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25380ce860 0x7f2538145900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.120+0000 7f253e400640 1 --2- 172.21.15.7:0/160308075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f253815aa30 0x7f25381f5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.120+0000 7f253da00640 1 -- 172.21.15.7:0/160308075 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25380ce860 msgr2=0x7f2538145900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.120+0000 7f253da00640 1 --2- 172.21.15.7:0/160308075 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25380ce860 0x7f2538145900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.120+0000 7f253da00640 1 -- 172.21.15.7:0/160308075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f253815aa30 msgr2=0x7f25381f5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.120+0000 7f253da00640 1 --2- 172.21.15.7:0/160308075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f253815aa30 0x7f25381f5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.120+0000 7f253da00640 1 -- 172.21.15.7:0/160308075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2538145e40 con 0x7f25380cbcd0 2024-09-10T21:45:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.121+0000 7f253da00640 1 --2- 172.21.15.7:0/160308075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25380cbcd0 0x7f253815b0b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f252c006010 tx=0x7f252c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:47.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.121+0000 7f2535600640 1 -- 172.21.15.7:0/160308075 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c03e030 con 0x7f25380cbcd0 2024-09-10T21:45:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.121+0000 7f2535600640 1 -- 172.21.15.7:0/160308075 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f252c003ca0 con 0x7f25380cbcd0 2024-09-10T21:45:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.121+0000 7f2535600640 1 -- 172.21.15.7:0/160308075 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252c035040 con 0x7f25380cbcd0 2024-09-10T21:45:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.121+0000 7f2540400640 1 -- 172.21.15.7:0/160308075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25380c6620 con 0x7f25380cbcd0 2024-09-10T21:45:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.121+0000 7f2540400640 1 -- 172.21.15.7:0/160308075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f253815da30 con 0x7f25380cbcd0 2024-09-10T21:45:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.122+0000 7f2540400640 1 -- 172.21.15.7:0/160308075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f253815f240 con 0x7f25380cbcd0 2024-09-10T21:45:47.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.123+0000 7f2535600640 1 -- 172.21.15.7:0/160308075 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f252c0049e0 con 0x7f25380cbcd0 2024-09-10T21:45:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.125+0000 7f2535600640 1 --2- 172.21.15.7:0/160308075 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2518077f70 0x7f251807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.125+0000 7f2535600640 1 -- 172.21.15.7:0/160308075 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f252c0ba550 con 0x7f25380cbcd0 2024-09-10T21:45:47.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.125+0000 7f253d000640 1 --2- 172.21.15.7:0/160308075 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2518077f70 0x7f251807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:47.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.126+0000 7f253d000640 1 --2- 172.21.15.7:0/160308075 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2518077f70 0x7f251807a430 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f25380cb500 tx=0x7f2524009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:47.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.129+0000 7f2535600640 1 -- 172.21.15.7:0/160308075 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f252c082fb0 con 0x7f25380cbcd0 2024-09-10T21:45:47.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.326+0000 7f2540400640 1 -- 172.21.15.7:0/160308075 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2538155460 con 0x7f25380cbcd0 2024-09-10T21:45:47.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.327+0000 7f2535600640 1 -- 172.21.15.7:0/160308075 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v58) ==== 74+0+16689 (secure 0 0 0) 0x7f252c0828b0 con 0x7f25380cbcd0 2024-09-10T21:45:47.328 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:45:47.328 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":58,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:44:32.373354+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:44:28.642474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-10T21:44:29.765081+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"58","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:45:47.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.329+0000 7f2540400640 1 -- 172.21.15.7:0/160308075 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2518077f70 msgr2=0x7f251807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:47.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.329+0000 7f2540400640 1 --2- 172.21.15.7:0/160308075 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2518077f70 0x7f251807a430 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f25380cb500 tx=0x7f2524009040 comp rx=0 tx=0).stop 2024-09-10T21:45:47.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.329+0000 7f2540400640 1 -- 172.21.15.7:0/160308075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25380cbcd0 msgr2=0x7f253815b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:47.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.329+0000 7f2540400640 1 --2- 172.21.15.7:0/160308075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25380cbcd0 0x7f253815b0b0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f252c006010 tx=0x7f252c0038a0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.330+0000 7f2540400640 1 -- 172.21.15.7:0/160308075 shutdown_connections 2024-09-10T21:45:47.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.330+0000 7f2540400640 1 --2- 172.21.15.7:0/160308075 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2518077f70 0x7f251807a430 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.330+0000 7f2540400640 1 --2- 172.21.15.7:0/160308075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f253815aa30 0x7f25381f5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.330+0000 7f2540400640 1 --2- 172.21.15.7:0/160308075 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25380ce860 0x7f2538145900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.330+0000 7f2540400640 1 --2- 172.21.15.7:0/160308075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25380cbcd0 0x7f253815b0b0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.330+0000 7f2540400640 1 -- 172.21.15.7:0/160308075 >> 172.21.15.7:0/160308075 conn(0x7f25380b16d0 msgr2=0x7f2538157040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:47.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.330+0000 7f2540400640 1 -- 172.21.15.7:0/160308075 shutdown_connections 2024-09-10T21:45:47.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.330+0000 7f2540400640 1 -- 172.21.15.7:0/160308075 wait complete. 2024-09-10T21:45:47.346 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:45:47.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.506+0000 7f8297600640 1 -- 172.21.15.7:0/1727460458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f829006ff50 msgr2=0x7f829015cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:47.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.506+0000 7f8297600640 1 --2- 172.21.15.7:0/1727460458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f829006ff50 0x7f829015cbe0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8280005fd0 tx=0x7f828002a4b0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.506+0000 7f8297600640 1 -- 172.21.15.7:0/1727460458 shutdown_connections 2024-09-10T21:45:47.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.506+0000 7f8297600640 1 --2- 172.21.15.7:0/1727460458 >> v2:172.21.15.133:3300/0 conn(0x7f82900743e0 0x7f829015f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.507+0000 7f8297600640 1 --2- 172.21.15.7:0/1727460458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f829006ff50 0x7f829015cbe0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.507+0000 7f8297600640 1 -- 172.21.15.7:0/1727460458 >> 172.21.15.7:0/1727460458 conn(0x7f82900b1660 msgr2=0x7f82900b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:47.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.507+0000 7f8297600640 1 -- 172.21.15.7:0/1727460458 shutdown_connections 2024-09-10T21:45:47.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.507+0000 7f8297600640 1 -- 172.21.15.7:0/1727460458 wait complete. 2024-09-10T21:45:47.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.508+0000 7f8297600640 1 Processor -- start 2024-09-10T21:45:47.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.508+0000 7f8297600640 1 -- start start 2024-09-10T21:45:47.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.509+0000 7f8297600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f829006e510 0x7f82900c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:47.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.509+0000 7f8297600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f829006ff50 0x7f82901f8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:47.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.509+0000 7f8297600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82900743e0 0x7f82900a9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:47.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.509+0000 7f8297600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f829006eb40 con 0x7f82900743e0 2024-09-10T21:45:47.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.509+0000 7f8297600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f829015fc50 con 0x7f829006e510 2024-09-10T21:45:47.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.509+0000 7f8297600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f82900c57d0 con 0x7f829006ff50 2024-09-10T21:45:47.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.510+0000 7f8295600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82900743e0 0x7f82900a9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:47.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.510+0000 7f8295600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82900743e0 0x7f82900a9750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58160/0 (socket says 172.21.15.7:58160) 2024-09-10T21:45:47.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.510+0000 7f8295600640 1 -- 172.21.15.7:0/1216852692 learned_addr learned my addr 172.21.15.7:0/1216852692 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:47.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.510+0000 7f8294c00640 1 --2- 172.21.15.7:0/1216852692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f829006e510 0x7f82900c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:47.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.510+0000 7f8287e00640 1 --2- 172.21.15.7:0/1216852692 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f829006ff50 0x7f82901f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:47.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.510+0000 7f8295600640 1 -- 172.21.15.7:0/1216852692 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f829006ff50 msgr2=0x7f82901f8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:47.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.510+0000 7f8295600640 1 --2- 172.21.15.7:0/1216852692 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f829006ff50 0x7f82901f8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.510+0000 7f8295600640 1 -- 172.21.15.7:0/1216852692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f829006e510 msgr2=0x7f82900c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:47.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.510+0000 7f8295600640 1 --2- 172.21.15.7:0/1216852692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f829006e510 0x7f82900c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.510+0000 7f8295600640 1 -- 172.21.15.7:0/1216852692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82901f46c0 con 0x7f82900743e0 2024-09-10T21:45:47.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.510+0000 7f8295600640 1 --2- 172.21.15.7:0/1216852692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82900743e0 0x7f82900a9750 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f828800e9d0 tx=0x7f828800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:47.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.511+0000 7f828d600640 1 -- 172.21.15.7:0/1216852692 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f828800cde0 con 0x7f82900743e0 2024-09-10T21:45:47.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.511+0000 7f828d600640 1 -- 172.21.15.7:0/1216852692 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8288004b20 con 0x7f82900743e0 2024-09-10T21:45:47.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.511+0000 7f828d600640 1 -- 172.21.15.7:0/1216852692 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8288005020 con 0x7f82900743e0 2024-09-10T21:45:47.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.511+0000 7f8297600640 1 -- 172.21.15.7:0/1216852692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82901f49a0 con 0x7f82900743e0 2024-09-10T21:45:47.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.511+0000 7f8297600640 1 -- 172.21.15.7:0/1216852692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82901f4f60 con 0x7f82900743e0 2024-09-10T21:45:47.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.512+0000 7f8297600640 1 -- 172.21.15.7:0/1216852692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82901600e0 con 0x7f82900743e0 2024-09-10T21:45:47.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.513+0000 7f828d600640 1 -- 172.21.15.7:0/1216852692 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f82880051c0 con 0x7f82900743e0 2024-09-10T21:45:47.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.514+0000 7f828d600640 1 --2- 172.21.15.7:0/1216852692 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f826c077fa0 0x7f826c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:47.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.514+0000 7f828d600640 1 -- 172.21.15.7:0/1216852692 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5709+0+0 (secure 0 0 0) 0x7f828809a750 con 0x7f82900743e0 2024-09-10T21:45:47.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.514+0000 7f8294c00640 1 --2- 172.21.15.7:0/1216852692 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f826c077fa0 0x7f826c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:47.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.515+0000 7f8294c00640 1 --2- 172.21.15.7:0/1216852692 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f826c077fa0 0x7f826c07a460 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8280033000 tx=0x7f8280004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:47.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.518+0000 7f828d600640 1 -- 172.21.15.7:0/1216852692 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8288062c90 con 0x7f82900743e0 2024-09-10T21:45:47.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.752+0000 7f8297600640 1 -- 172.21.15.7:0/1216852692 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f82900c6300 con 0x7f82900743e0 2024-09-10T21:45:47.753 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-10T21:45:47.752+0000 7feb3fa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:45:47.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.754+0000 7f828d600640 1 -- 172.21.15.7:0/1216852692 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7f82880676c0 con 0x7f82900743e0 2024-09-10T21:45:47.755 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:45:47.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.757+0000 7f8297600640 1 -- 172.21.15.7:0/1216852692 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f826c077fa0 msgr2=0x7f826c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:47.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.757+0000 7f8297600640 1 --2- 172.21.15.7:0/1216852692 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f826c077fa0 0x7f826c07a460 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8280033000 tx=0x7f8280004730 comp rx=0 tx=0).stop 2024-09-10T21:45:47.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.757+0000 7f8297600640 1 -- 172.21.15.7:0/1216852692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82900743e0 msgr2=0x7f82900a9750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:47.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.757+0000 7f8297600640 1 --2- 172.21.15.7:0/1216852692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82900743e0 0x7f82900a9750 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f828800e9d0 tx=0x7f828800ee90 comp rx=0 tx=0).stop 2024-09-10T21:45:47.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.757+0000 7f8297600640 1 -- 172.21.15.7:0/1216852692 shutdown_connections 2024-09-10T21:45:47.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.757+0000 7f8297600640 1 --2- 172.21.15.7:0/1216852692 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f826c077fa0 0x7f826c07a460 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.757+0000 7f8297600640 1 --2- 172.21.15.7:0/1216852692 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f82900743e0 0x7f82900a9750 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.757+0000 7f8297600640 1 --2- 172.21.15.7:0/1216852692 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f829006ff50 0x7f82901f8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.757+0000 7f8297600640 1 --2- 172.21.15.7:0/1216852692 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f829006e510 0x7f82900c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.757+0000 7f8297600640 1 -- 172.21.15.7:0/1216852692 >> 172.21.15.7:0/1216852692 conn(0x7f82900b1660 msgr2=0x7f82900b04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:47.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.757+0000 7f8297600640 1 -- 172.21.15.7:0/1216852692 shutdown_connections 2024-09-10T21:45:47.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.757+0000 7f8297600640 1 -- 172.21.15.7:0/1216852692 wait complete. 2024-09-10T21:45:47.769 INFO:tasks.ceph.mds.a.smithi007.stderr: -8883> 2024-09-10T21:44:37.323+0000 7fe3f5e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:45:47.773 INFO:tasks.ceph.mds.a.smithi007.stderr: -5784> 2024-09-10T21:44:40.512+0000 7fe3f0e00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 pv4 ap=4 s=0 n(v0 1=1+0)->n(v0 rc2024-09-10T21:44:39.176338+0000 1=1+0) (iauth excl) (ifile lock->excl w=1) (ixattr excl) (iquiesce lock w=1) cr={5596=0-4194304@1} caps={5596=pAsxLsXsxFcb/pAsxXsxFxwb@3},l=5596 | ptrwaiter=1 request=1 lock=3 caps=1 dirtyparent=0 dirty=0 waiter=1 authpin=1 scrubqueue=0 0x5608cb1c6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-10T21:45:47.776 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:45:47.777 INFO:tasks.ceph.mds.a.smithi007.stderr: -3574> 2024-09-10T21:45:12.283+0000 7fe3f0e00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v4 pv6 ap=4 DIRTYPARENT s=0 n(v0 rc2024-09-10T21:44:39.176338+0000 1=1+0)->n(v0 rc2024-09-10T21:44:39.176338+0000 1=1+0) (ifile lock->xlock w=1) (iquiesce lock w=1) cr={5596=0-4194304@1} caps={5596=pAsLsXs/-@6} | ptrwaiter=1 request=1 lock=3 caps=1 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x5608cb1c6b00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/file v4>,<0x1/dir_test v7>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-10T21:45:47.824 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-09-10T21:45:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.944+0000 7f90b3600640 1 -- 172.21.15.7:0/818242777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9094003a50 msgr2=0x7f9094003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.944+0000 7f90b3600640 1 --2- 172.21.15.7:0/818242777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9094003a50 0x7f9094003e70 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f90a0009b50 tx=0x7f9094005830 comp rx=0 tx=0).stop 2024-09-10T21:45:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.944+0000 7f90b1600640 1 -- 172.21.15.7:0/818242777 reap_dead start 2024-09-10T21:45:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.945+0000 7f90b3600640 1 -- 172.21.15.7:0/818242777 shutdown_connections 2024-09-10T21:45:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.945+0000 7f90b3600640 1 -- 172.21.15.7:0/818242777 >> 172.21.15.7:0/818242777 conn(0x7f90ac0d1db0 msgr2=0x7f90ac0d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.945+0000 7f90b3600640 1 -- 172.21.15.7:0/818242777 shutdown_connections 2024-09-10T21:45:47.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.945+0000 7f90b3600640 1 -- 172.21.15.7:0/818242777 wait complete. 2024-09-10T21:45:47.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.946+0000 7f90b3600640 1 Processor -- start 2024-09-10T21:45:47.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.946+0000 7f90b3600640 1 -- start start 2024-09-10T21:45:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.947+0000 7f90b3600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9094003a50 0x7f90ac1f3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.947+0000 7f90b3600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f90ac0c58e0 0x7f90ac1f7c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.947+0000 7f90b3600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90ac0c5d60 0x7f90ac1f8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.947+0000 7f90b3600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac0b19f0 con 0x7f90ac0c5d60 2024-09-10T21:45:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.947+0000 7f90b3600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac0b06b0 con 0x7f9094003a50 2024-09-10T21:45:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.947+0000 7f90b3600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f90ac0b1b70 con 0x7f90ac0c58e0 2024-09-10T21:45:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.948+0000 7f90b1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90ac0c5d60 0x7f90ac1f8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.948+0000 7f90b1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90ac0c5d60 0x7f90ac1f8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58196/0 (socket says 172.21.15.7:58196) 2024-09-10T21:45:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.948+0000 7f90b1600640 1 -- 172.21.15.7:0/807476012 learned_addr learned my addr 172.21.15.7:0/807476012 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.948+0000 7f90abe00640 1 --2- 172.21.15.7:0/807476012 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f90ac0c58e0 0x7f90ac1f7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:47.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.948+0000 7f90b0c00640 1 --2- 172.21.15.7:0/807476012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9094003a50 0x7f90ac1f3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:47.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.948+0000 7f90b1600640 1 -- 172.21.15.7:0/807476012 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f90ac0c58e0 msgr2=0x7f90ac1f7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:47.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.948+0000 7f90b1600640 1 --2- 172.21.15.7:0/807476012 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f90ac0c58e0 0x7f90ac1f7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.948+0000 7f90b1600640 1 -- 172.21.15.7:0/807476012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9094003a50 msgr2=0x7f90ac1f3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:47.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.948+0000 7f90b1600640 1 --2- 172.21.15.7:0/807476012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9094003a50 0x7f90ac1f3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:47.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.948+0000 7f90b1600640 1 -- 172.21.15.7:0/807476012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ac0ca890 con 0x7f90ac0c5d60 2024-09-10T21:45:47.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.948+0000 7f90b1600640 1 --2- 172.21.15.7:0/807476012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90ac0c5d60 0x7f90ac1f8150 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0702d0 tx=0x7f90a400ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:47.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.948+0000 7f90a8c00640 1 -- 172.21.15.7:0/807476012 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a400ca20 con 0x7f90ac0c5d60 2024-09-10T21:45:47.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.949+0000 7f90a8c00640 1 -- 172.21.15.7:0/807476012 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90a4018070 con 0x7f90ac0c5d60 2024-09-10T21:45:47.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.949+0000 7f90a8c00640 1 -- 172.21.15.7:0/807476012 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a4011460 con 0x7f90ac0c5d60 2024-09-10T21:45:47.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.949+0000 7f90b3600640 1 -- 172.21.15.7:0/807476012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ac0c82c0 con 0x7f90ac0c5d60 2024-09-10T21:45:47.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.949+0000 7f90b3600640 1 -- 172.21.15.7:0/807476012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ac0cab10 con 0x7f90ac0c5d60 2024-09-10T21:45:47.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.950+0000 7f90b3600640 1 -- 172.21.15.7:0/807476012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90ac0b14b0 con 0x7f90ac0c5d60 2024-09-10T21:45:47.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.951+0000 7f90a8c00640 1 -- 172.21.15.7:0/807476012 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f90a4011600 con 0x7f90ac0c5d60 2024-09-10T21:45:47.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.952+0000 7f90a8c00640 1 --2- 172.21.15.7:0/807476012 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f908c077fa0 0x7f908c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:47.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.952+0000 7f90a8c00640 1 -- 172.21.15.7:0/807476012 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5795+0+0 (secure 0 0 0) 0x7f90a409a170 con 0x7f90ac0c5d60 2024-09-10T21:45:47.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.952+0000 7f90b0c00640 1 --2- 172.21.15.7:0/807476012 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f908c077fa0 0x7f908c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:47.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.953+0000 7f90b0c00640 1 --2- 172.21.15.7:0/807476012 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f908c077fa0 0x7f908c07a460 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f90a0035000 tx=0x7f90a002d040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:47.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:47.956+0000 7f90a8c00640 1 -- 172.21.15.7:0/807476012 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f90ac0ca890 con 0x7f90ac0c5d60 2024-09-10T21:45:48.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.191+0000 7f90b3600640 1 -- 172.21.15.7:0/807476012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f90ac15b770 con 0x7f90ac0c5d60 2024-09-10T21:45:48.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.756+0000 7f90a8c00640 1 -- 172.21.15.7:0/807476012 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7f90a4062520 con 0x7f90ac0c5d60 2024-09-10T21:45:48.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.759+0000 7f90b3600640 1 -- 172.21.15.7:0/807476012 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f908c077fa0 msgr2=0x7f908c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:48.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.759+0000 7f90b3600640 1 --2- 172.21.15.7:0/807476012 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f908c077fa0 0x7f908c07a460 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f90a0035000 tx=0x7f90a002d040 comp rx=0 tx=0).stop 2024-09-10T21:45:48.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.759+0000 7f90b3600640 1 -- 172.21.15.7:0/807476012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90ac0c5d60 msgr2=0x7f90ac1f8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.759+0000 7f90b3600640 1 --2- 172.21.15.7:0/807476012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90ac0c5d60 0x7f90ac1f8150 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f90ac0702d0 tx=0x7f90a400ed10 comp rx=0 tx=0).stop 2024-09-10T21:45:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.759+0000 7f90b3600640 1 -- 172.21.15.7:0/807476012 shutdown_connections 2024-09-10T21:45:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.759+0000 7f90b3600640 1 --2- 172.21.15.7:0/807476012 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f908c077fa0 0x7f908c07a460 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.759+0000 7f90b3600640 1 --2- 172.21.15.7:0/807476012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90ac0c5d60 0x7f90ac1f8150 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.759+0000 7f90b3600640 1 --2- 172.21.15.7:0/807476012 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f90ac0c58e0 0x7f90ac1f7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.759+0000 7f90b3600640 1 --2- 172.21.15.7:0/807476012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9094003a50 0x7f90ac1f3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.759+0000 7f90b3600640 1 -- 172.21.15.7:0/807476012 >> 172.21.15.7:0/807476012 conn(0x7f90ac0d1db0 msgr2=0x7f90ac15a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.759+0000 7f90b3600640 1 -- 172.21.15.7:0/807476012 shutdown_connections 2024-09-10T21:45:48.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.759+0000 7f90b3600640 1 -- 172.21.15.7:0/807476012 wait complete. 2024-09-10T21:45:48.781 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:45:48.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.947+0000 7fe366400640 1 -- 172.21.15.7:0/2120799232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe360161150 msgr2=0x7fe360161600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.947+0000 7fe366400640 1 --2- 172.21.15.7:0/2120799232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe360161150 0x7fe360161600 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe358005fd0 tx=0x7fe35802a8b0 comp rx=0 tx=0).stop 2024-09-10T21:45:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.947+0000 7fe366400640 1 -- 172.21.15.7:0/2120799232 shutdown_connections 2024-09-10T21:45:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.947+0000 7fe366400640 1 --2- 172.21.15.7:0/2120799232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe360161150 0x7fe360161600 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.947+0000 7fe366400640 1 --2- 172.21.15.7:0/2120799232 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3601620d0 0x7fe3600d0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.947+0000 7fe366400640 1 -- 172.21.15.7:0/2120799232 >> 172.21.15.7:0/2120799232 conn(0x7fe3600d1dc0 msgr2=0x7fe3600d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.947+0000 7fe366400640 1 -- 172.21.15.7:0/2120799232 shutdown_connections 2024-09-10T21:45:48.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.947+0000 7fe366400640 1 -- 172.21.15.7:0/2120799232 wait complete. 2024-09-10T21:45:48.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.949+0000 7fe366400640 1 Processor -- start 2024-09-10T21:45:48.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.949+0000 7fe366400640 1 -- start start 2024-09-10T21:45:48.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.950+0000 7fe366400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3600b0ca0 0x7fe3601f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:48.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.950+0000 7fe366400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe360161150 0x7fe36015cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:48.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.950+0000 7fe366400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3601620d0 0x7fe3600b6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:48.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.950+0000 7fe366400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe360162d40 con 0x7fe3601620d0 2024-09-10T21:45:48.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.950+0000 7fe366400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe360162bc0 con 0x7fe360161150 2024-09-10T21:45:48.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.950+0000 7fe366400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe3600b16d0 con 0x7fe3600b0ca0 2024-09-10T21:45:48.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.950+0000 7fe35fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3601620d0 0x7fe3600b6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:48.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.950+0000 7fe35fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3601620d0 0x7fe3600b6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58204/0 (socket says 172.21.15.7:58204) 2024-09-10T21:45:48.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.950+0000 7fe35fe00640 1 -- 172.21.15.7:0/167612265 learned_addr learned my addr 172.21.15.7:0/167612265 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:48.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.950+0000 7fe35f400640 1 --2- 172.21.15.7:0/167612265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3600b0ca0 0x7fe3601f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:48.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.950+0000 7fe35ea00640 1 --2- 172.21.15.7:0/167612265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe360161150 0x7fe36015cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:48.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.950+0000 7fe35fe00640 1 -- 172.21.15.7:0/167612265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3600b0ca0 msgr2=0x7fe3601f5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:48.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.951+0000 7fe35fe00640 1 --2- 172.21.15.7:0/167612265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3600b0ca0 0x7fe3601f5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:48.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.951+0000 7fe35fe00640 1 -- 172.21.15.7:0/167612265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe360161150 msgr2=0x7fe36015cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:48.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.951+0000 7fe35fe00640 1 --2- 172.21.15.7:0/167612265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe360161150 0x7fe36015cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:48.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.951+0000 7fe35fe00640 1 -- 172.21.15.7:0/167612265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe360071bd0 con 0x7fe3601620d0 2024-09-10T21:45:48.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.951+0000 7fe35fe00640 1 --2- 172.21.15.7:0/167612265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3601620d0 0x7fe3600b6dc0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe358009b30 tx=0x7fe3580316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:48.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.951+0000 7fe357400640 1 -- 172.21.15.7:0/167612265 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe358040070 con 0x7fe3601620d0 2024-09-10T21:45:48.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.951+0000 7fe357400640 1 -- 172.21.15.7:0/167612265 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe358004040 con 0x7fe3601620d0 2024-09-10T21:45:48.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.951+0000 7fe357400640 1 -- 172.21.15.7:0/167612265 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe358004450 con 0x7fe3601620d0 2024-09-10T21:45:48.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.951+0000 7fe366400640 1 -- 172.21.15.7:0/167612265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3601f50c0 con 0x7fe3601620d0 2024-09-10T21:45:48.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.951+0000 7fe366400640 1 -- 172.21.15.7:0/167612265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3601f5360 con 0x7fe3601620d0 2024-09-10T21:45:48.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.952+0000 7fe366400640 1 -- 172.21.15.7:0/167612265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe360161600 con 0x7fe3601620d0 2024-09-10T21:45:48.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.954+0000 7fe357400640 1 -- 172.21.15.7:0/167612265 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe3580045f0 con 0x7fe3601620d0 2024-09-10T21:45:48.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.955+0000 7fe357400640 1 --2- 172.21.15.7:0/167612265 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe340077f70 0x7fe34007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.955+0000 7fe357400640 1 -- 172.21.15.7:0/167612265 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5795+0+0 (secure 0 0 0) 0x7fe3580bb650 con 0x7fe3601620d0 2024-09-10T21:45:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.955+0000 7fe35f400640 1 --2- 172.21.15.7:0/167612265 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe340077f70 0x7fe34007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:48.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.956+0000 7fe35f400640 1 --2- 172.21.15.7:0/167612265 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe340077f70 0x7fe34007a430 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe3600d0f90 tx=0x7fe350005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:48.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:48.959+0000 7fe357400640 1 -- 172.21.15.7:0/167612265 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe358083a10 con 0x7fe3601620d0 2024-09-10T21:45:49.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.154+0000 7fe366400640 1 -- 172.21.15.7:0/167612265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe36006fc20 con 0x7fe3601620d0 2024-09-10T21:45:49.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.762+0000 7fe357400640 1 -- 172.21.15.7:0/167612265 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v60) ==== 181+0+0 (secure 0 0 0) 0x7fe358088440 con 0x7fe3601620d0 2024-09-10T21:45:49.764 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:45:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.767+0000 7fe366400640 1 -- 172.21.15.7:0/167612265 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe340077f70 msgr2=0x7fe34007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.767+0000 7fe366400640 1 --2- 172.21.15.7:0/167612265 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe340077f70 0x7fe34007a430 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe3600d0f90 tx=0x7fe350005f90 comp rx=0 tx=0).stop 2024-09-10T21:45:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.767+0000 7fe366400640 1 -- 172.21.15.7:0/167612265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3601620d0 msgr2=0x7fe3600b6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.767+0000 7fe366400640 1 --2- 172.21.15.7:0/167612265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3601620d0 0x7fe3600b6dc0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe358009b30 tx=0x7fe3580316e0 comp rx=0 tx=0).stop 2024-09-10T21:45:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.767+0000 7fe366400640 1 -- 172.21.15.7:0/167612265 shutdown_connections 2024-09-10T21:45:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.767+0000 7fe366400640 1 --2- 172.21.15.7:0/167612265 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe340077f70 0x7fe34007a430 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:49.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.767+0000 7fe366400640 1 --2- 172.21.15.7:0/167612265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe360161150 0x7fe36015cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:49.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.767+0000 7fe366400640 1 --2- 172.21.15.7:0/167612265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe3601620d0 0x7fe3600b6dc0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:49.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.767+0000 7fe366400640 1 --2- 172.21.15.7:0/167612265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe3600b0ca0 0x7fe3601f5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:49.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.767+0000 7fe366400640 1 -- 172.21.15.7:0/167612265 >> 172.21.15.7:0/167612265 conn(0x7fe3600d1dc0 msgr2=0x7fe3600b37d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:49.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.767+0000 7fe366400640 1 -- 172.21.15.7:0/167612265 shutdown_connections 2024-09-10T21:45:49.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.767+0000 7fe366400640 1 -- 172.21.15.7:0/167612265 wait complete. 2024-09-10T21:45:49.789 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:45:49.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.963+0000 7f678f200640 1 -- 172.21.15.7:0/3143135288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900d10b0 msgr2=0x7f67900ce820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:49.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.963+0000 7f678f200640 1 --2- 172.21.15.7:0/3143135288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900d10b0 0x7f67900ce820 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f679006d200 tx=0x7f677c02a440 comp rx=0 tx=0).stop 2024-09-10T21:45:49.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.963+0000 7f678f200640 1 -- 172.21.15.7:0/3143135288 shutdown_connections 2024-09-10T21:45:49.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.963+0000 7f678f200640 1 --2- 172.21.15.7:0/3143135288 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67900ced60 0x7f67900cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:49.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.963+0000 7f678f200640 1 --2- 172.21.15.7:0/3143135288 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900d10b0 0x7f67900ce820 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:49.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.963+0000 7f678f200640 1 --2- 172.21.15.7:0/3143135288 >> v2:172.21.15.133:3300/0 conn(0x7f67900d0790 0x7f67900d0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:49.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.963+0000 7f678f200640 1 -- 172.21.15.7:0/3143135288 >> 172.21.15.7:0/3143135288 conn(0x7f67900d3d80 msgr2=0x7f67900d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:49.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.963+0000 7f678f200640 1 -- 172.21.15.7:0/3143135288 shutdown_connections 2024-09-10T21:45:49.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.963+0000 7f678f200640 1 -- 172.21.15.7:0/3143135288 wait complete. 2024-09-10T21:45:49.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.965+0000 7f678f200640 1 Processor -- start 2024-09-10T21:45:49.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.965+0000 7f678f200640 1 -- start start 2024-09-10T21:45:49.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.966+0000 7f678f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67900ced60 0x7f67900b9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:49.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.966+0000 7f678f200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67900d0790 0x7f6790155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:49.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.966+0000 7f678f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900d10b0 0x7f67901f84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:49.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.966+0000 7f678f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f67900b30e0 con 0x7f67900d10b0 2024-09-10T21:45:49.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.966+0000 7f678f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f67900b2f60 con 0x7f67900ced60 2024-09-10T21:45:49.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.966+0000 7f678f200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f67900d48a0 con 0x7f67900d0790 2024-09-10T21:45:49.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.966+0000 7f678e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900d10b0 0x7f67901f84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:49.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.966+0000 7f678e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900d10b0 0x7f67901f84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58216/0 (socket says 172.21.15.7:58216) 2024-09-10T21:45:49.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.966+0000 7f678e800640 1 -- 172.21.15.7:0/1937316726 learned_addr learned my addr 172.21.15.7:0/1937316726 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:49.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.966+0000 7f678d400640 1 --2- 172.21.15.7:0/1937316726 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67900d0790 0x7f6790155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:49.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.966+0000 7f678de00640 1 --2- 172.21.15.7:0/1937316726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67900ced60 0x7f67900b9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:49.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.966+0000 7f678e800640 1 -- 172.21.15.7:0/1937316726 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67900d0790 msgr2=0x7f6790155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:49.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.966+0000 7f678e800640 1 --2- 172.21.15.7:0/1937316726 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67900d0790 0x7f6790155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:49.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.966+0000 7f678e800640 1 -- 172.21.15.7:0/1937316726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67900ced60 msgr2=0x7f67900b9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:49.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.966+0000 7f678e800640 1 --2- 172.21.15.7:0/1937316726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67900ced60 0x7f67900b9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:49.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.966+0000 7f678e800640 1 -- 172.21.15.7:0/1937316726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67900cb0f0 con 0x7f67900d10b0 2024-09-10T21:45:49.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.966+0000 7f678de00640 1 --2- 172.21.15.7:0/1937316726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67900ced60 0x7f67900b9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:45:49.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.966+0000 7f678d400640 1 --2- 172.21.15.7:0/1937316726 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67900d0790 0x7f6790155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:45:49.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.967+0000 7f678e800640 1 --2- 172.21.15.7:0/1937316726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900d10b0 0x7f67901f84b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f678800cad0 tx=0x7f678800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:49.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.967+0000 7f6786000640 1 -- 172.21.15.7:0/1937316726 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678800ae20 con 0x7f67900d10b0 2024-09-10T21:45:49.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.967+0000 7f6786000640 1 -- 172.21.15.7:0/1937316726 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f678800f450 con 0x7f67900d10b0 2024-09-10T21:45:49.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.967+0000 7f6786000640 1 -- 172.21.15.7:0/1937316726 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f678800f8c0 con 0x7f67900d10b0 2024-09-10T21:45:49.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.967+0000 7f678f200640 1 -- 172.21.15.7:0/1937316726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67900cd3c0 con 0x7f67900d10b0 2024-09-10T21:45:49.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.967+0000 7f678f200640 1 -- 172.21.15.7:0/1937316726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67900cd980 con 0x7f67900d10b0 2024-09-10T21:45:49.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.968+0000 7f678f200640 1 -- 172.21.15.7:0/1937316726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67900cd150 con 0x7f67900d10b0 2024-09-10T21:45:49.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.969+0000 7f6786000640 1 -- 172.21.15.7:0/1937316726 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f678800fa80 con 0x7f67900d10b0 2024-09-10T21:45:49.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.970+0000 7f6786000640 1 --2- 172.21.15.7:0/1937316726 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6770077f70 0x7f677007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:49.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.970+0000 7f6786000640 1 -- 172.21.15.7:0/1937316726 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5357+0+0 (secure 0 0 0) 0x7f678809a7a0 con 0x7f67900d10b0 2024-09-10T21:45:49.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.971+0000 7f678de00640 1 --2- 172.21.15.7:0/1937316726 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6770077f70 0x7f677007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:49.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.971+0000 7f678de00640 1 --2- 172.21.15.7:0/1937316726 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6770077f70 0x7f677007a430 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6780006fd0 tx=0x7f6780008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:49.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:49.974+0000 7f6786000640 1 -- 172.21.15.7:0/1937316726 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6788014030 con 0x7f67900d10b0 2024-09-10T21:45:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.171+0000 7f678f200640 1 -- 172.21.15.7:0/1937316726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f67900d0b70 con 0x7f67900d10b0 2024-09-10T21:45:50.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.771+0000 7f6786000640 1 -- 172.21.15.7:0/1937316726 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v61) ==== 169+0+0 (secure 0 0 0) 0x7f678809a0d0 con 0x7f67900d10b0 2024-09-10T21:45:50.773 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-09-10T21:45:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.775+0000 7f678f200640 1 -- 172.21.15.7:0/1937316726 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6770077f70 msgr2=0x7f677007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.775+0000 7f678f200640 1 --2- 172.21.15.7:0/1937316726 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6770077f70 0x7f677007a430 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6780006fd0 tx=0x7f6780008040 comp rx=0 tx=0).stop 2024-09-10T21:45:50.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.775+0000 7f678f200640 1 -- 172.21.15.7:0/1937316726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900d10b0 msgr2=0x7f67901f84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.775+0000 7f678f200640 1 --2- 172.21.15.7:0/1937316726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900d10b0 0x7f67901f84b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f678800cad0 tx=0x7f678800cf90 comp rx=0 tx=0).stop 2024-09-10T21:45:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.776+0000 7f678f200640 1 -- 172.21.15.7:0/1937316726 shutdown_connections 2024-09-10T21:45:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.776+0000 7f678f200640 1 --2- 172.21.15.7:0/1937316726 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6770077f70 0x7f677007a430 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.776+0000 7f678f200640 1 --2- 172.21.15.7:0/1937316726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67900d10b0 0x7f67901f84b0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:50.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.776+0000 7f678f200640 1 --2- 172.21.15.7:0/1937316726 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67900d0790 0x7f6790155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.776+0000 7f678f200640 1 --2- 172.21.15.7:0/1937316726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67900ced60 0x7f67900b9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.776+0000 7f678f200640 1 -- 172.21.15.7:0/1937316726 >> 172.21.15.7:0/1937316726 conn(0x7f67900d3d80 msgr2=0x7f67900d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.776+0000 7f678f200640 1 -- 172.21.15.7:0/1937316726 shutdown_connections 2024-09-10T21:45:50.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.776+0000 7f678f200640 1 -- 172.21.15.7:0/1937316726 wait complete. 2024-09-10T21:45:50.800 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:45:50.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.963+0000 7f6a8d200640 1 -- 172.21.15.7:0/1538205272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a8806ff50 msgr2=0x7f6a8815cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:50.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.963+0000 7f6a8d200640 1 --2- 172.21.15.7:0/1538205272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a8806ff50 0x7f6a8815cbe0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009830 tx=0x7f6a7802a400 comp rx=0 tx=0).stop 2024-09-10T21:45:50.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.963+0000 7f6a8d200640 1 -- 172.21.15.7:0/1538205272 shutdown_connections 2024-09-10T21:45:50.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.963+0000 7f6a8d200640 1 --2- 172.21.15.7:0/1538205272 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a8806e510 0x7f6a88073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:50.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.963+0000 7f6a8d200640 1 --2- 172.21.15.7:0/1538205272 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a8806ff50 0x7f6a8815cbe0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:50.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.963+0000 7f6a8d200640 1 -- 172.21.15.7:0/1538205272 >> 172.21.15.7:0/1538205272 conn(0x7f6a880b1660 msgr2=0x7f6a880b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:50.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.963+0000 7f6a8d200640 1 -- 172.21.15.7:0/1538205272 shutdown_connections 2024-09-10T21:45:50.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.963+0000 7f6a8d200640 1 -- 172.21.15.7:0/1538205272 wait complete. 2024-09-10T21:45:50.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.965+0000 7f6a8d200640 1 Processor -- start 2024-09-10T21:45:50.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.965+0000 7f6a8d200640 1 -- start start 2024-09-10T21:45:50.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.966+0000 7f6a8d200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a8806e510 0x7f6a880ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:50.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.966+0000 7f6a8d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8806e960 0x7f6a881f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:50.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.966+0000 7f6a8d200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a8806ff50 0x7f6a881f14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:50.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.966+0000 7f6a8d200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6a880745a0 con 0x7f6a8806e960 2024-09-10T21:45:50.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.966+0000 7f6a8d200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6a88074420 con 0x7f6a8806ff50 2024-09-10T21:45:50.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.966+0000 7f6a8d200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f6a88074720 con 0x7f6a8806e510 2024-09-10T21:45:50.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.966+0000 7f6a7d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8806e960 0x7f6a881f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:50.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.966+0000 7f6a7d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8806e960 0x7f6a881f59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58228/0 (socket says 172.21.15.7:58228) 2024-09-10T21:45:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.966+0000 7f6a7d400640 1 -- 172.21.15.7:0/740378864 learned_addr learned my addr 172.21.15.7:0/740378864 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.966+0000 7f6a85e00640 1 --2- 172.21.15.7:0/740378864 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a8806e510 0x7f6a880ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.966+0000 7f6a86800640 1 --2- 172.21.15.7:0/740378864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a8806ff50 0x7f6a881f14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.966+0000 7f6a7d400640 1 -- 172.21.15.7:0/740378864 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a8806e510 msgr2=0x7f6a880ca2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.966+0000 7f6a7d400640 1 --2- 172.21.15.7:0/740378864 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a8806e510 0x7f6a880ca2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.966+0000 7f6a7d400640 1 -- 172.21.15.7:0/740378864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a8806ff50 msgr2=0x7f6a881f14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.966+0000 7f6a7d400640 1 --2- 172.21.15.7:0/740378864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a8806ff50 0x7f6a881f14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.966+0000 7f6a7d400640 1 -- 172.21.15.7:0/740378864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a880b3370 con 0x7f6a8806e960 2024-09-10T21:45:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.966+0000 7f6a86800640 1 --2- 172.21.15.7:0/740378864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a8806ff50 0x7f6a881f14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:45:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.967+0000 7f6a7d400640 1 --2- 172.21.15.7:0/740378864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8806e960 0x7f6a881f59e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000ea60 tx=0x7f6a7000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.967+0000 7f6a7ea00640 1 -- 172.21.15.7:0/740378864 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7000cde0 con 0x7f6a8806e960 2024-09-10T21:45:50.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.967+0000 7f6a7ea00640 1 -- 172.21.15.7:0/740378864 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a70004020 con 0x7f6a8806e960 2024-09-10T21:45:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.967+0000 7f6a7ea00640 1 -- 172.21.15.7:0/740378864 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a70004450 con 0x7f6a8806e960 2024-09-10T21:45:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.967+0000 7f6a8d200640 1 -- 172.21.15.7:0/740378864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a880b43b0 con 0x7f6a8806e960 2024-09-10T21:45:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.967+0000 7f6a8d200640 1 -- 172.21.15.7:0/740378864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a880b38d0 con 0x7f6a8806e960 2024-09-10T21:45:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.968+0000 7f6a8d200640 1 -- 172.21.15.7:0/740378864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a88073cd0 con 0x7f6a8806e960 2024-09-10T21:45:50.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.969+0000 7f6a7ea00640 1 -- 172.21.15.7:0/740378864 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6a700035f0 con 0x7f6a8806e960 2024-09-10T21:45:50.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.970+0000 7f6a7ea00640 1 --2- 172.21.15.7:0/740378864 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6a64077f70 0x7f6a6407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:50.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.970+0000 7f6a7ea00640 1 -- 172.21.15.7:0/740378864 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f6a7009a4d0 con 0x7f6a8806e960 2024-09-10T21:45:50.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.971+0000 7f6a85e00640 1 --2- 172.21.15.7:0/740378864 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6a64077f70 0x7f6a6407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:50.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.971+0000 7f6a85e00640 1 --2- 172.21.15.7:0/740378864 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6a64077f70 0x7f6a6407a430 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6a78034000 tx=0x7f6a78004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:50.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:50.974+0000 7f6a7ea00640 1 -- 172.21.15.7:0/740378864 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6a880b3370 con 0x7f6a8806e960 2024-09-10T21:45:51.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.178+0000 7f6a8d200640 1 -- 172.21.15.7:0/740378864 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f6a88072470 con 0x7f6a8806e960 2024-09-10T21:45:51.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.182+0000 7f6a7ea00640 1 -- 172.21.15.7:0/740378864 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7f6a70063230 con 0x7f6a8806e960 2024-09-10T21:45:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.185+0000 7f6a8d200640 1 -- 172.21.15.7:0/740378864 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6a64077f70 msgr2=0x7f6a6407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.185+0000 7f6a8d200640 1 --2- 172.21.15.7:0/740378864 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6a64077f70 0x7f6a6407a430 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6a78034000 tx=0x7f6a78004730 comp rx=0 tx=0).stop 2024-09-10T21:45:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.185+0000 7f6a8d200640 1 -- 172.21.15.7:0/740378864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8806e960 msgr2=0x7f6a881f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.185+0000 7f6a8d200640 1 --2- 172.21.15.7:0/740378864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8806e960 0x7f6a881f59e0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6a7000ea60 tx=0x7f6a7000ef20 comp rx=0 tx=0).stop 2024-09-10T21:45:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.185+0000 7f6a8d200640 1 -- 172.21.15.7:0/740378864 shutdown_connections 2024-09-10T21:45:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.185+0000 7f6a8d200640 1 --2- 172.21.15.7:0/740378864 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6a64077f70 0x7f6a6407a430 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.185+0000 7f6a8d200640 1 --2- 172.21.15.7:0/740378864 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6a8806ff50 0x7f6a881f14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.185+0000 7f6a8d200640 1 --2- 172.21.15.7:0/740378864 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6a8806e960 0x7f6a881f59e0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.185+0000 7f6a8d200640 1 --2- 172.21.15.7:0/740378864 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6a8806e510 0x7f6a880ca2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:51.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.185+0000 7f6a8d200640 1 -- 172.21.15.7:0/740378864 >> 172.21.15.7:0/740378864 conn(0x7f6a880b1660 msgr2=0x7f6a8815dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:51.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.185+0000 7f6a8d200640 1 -- 172.21.15.7:0/740378864 shutdown_connections 2024-09-10T21:45:51.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.185+0000 7f6a8d200640 1 -- 172.21.15.7:0/740378864 wait complete. 2024-09-10T21:45:51.202 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2024-09-10T21:45:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.364+0000 7f6d67c00640 1 -- 172.21.15.7:0/2297845150 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d6015bc90 msgr2=0x7f6d600ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.364+0000 7f6d67c00640 1 --2- 172.21.15.7:0/2297845150 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d6015bc90 0x7f6d600ccef0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6d50009830 tx=0x7f6d5002a3f0 comp rx=0 tx=0).stop 2024-09-10T21:45:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.364+0000 7f6d67c00640 1 -- 172.21.15.7:0/2297845150 shutdown_connections 2024-09-10T21:45:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.364+0000 7f6d67c00640 1 --2- 172.21.15.7:0/2297845150 >> v2:172.21.15.133:3300/0 conn(0x7f6d6015fda0 0x7f6d601601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.364+0000 7f6d67c00640 1 --2- 172.21.15.7:0/2297845150 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d6015bc90 0x7f6d600ccef0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.364+0000 7f6d67c00640 1 -- 172.21.15.7:0/2297845150 >> 172.21.15.7:0/2297845150 conn(0x7f6d600b16d0 msgr2=0x7f6d600b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.364+0000 7f6d67c00640 1 -- 172.21.15.7:0/2297845150 shutdown_connections 2024-09-10T21:45:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.364+0000 7f6d67c00640 1 -- 172.21.15.7:0/2297845150 wait complete. 2024-09-10T21:45:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.365+0000 7f6d67c00640 1 Processor -- start 2024-09-10T21:45:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.365+0000 7f6d67c00640 1 -- start start 2024-09-10T21:45:51.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.366+0000 7f6d67c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d600cd750 0x7f6d601f4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:51.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.366+0000 7f6d67c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d6015bc90 0x7f6d6015a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:51.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.366+0000 7f6d67c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d6015fda0 0x7f6d600c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:51.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.366+0000 7f6d67c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6006fe70 con 0x7f6d6015bc90 2024-09-10T21:45:51.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.366+0000 7f6d67c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6d6006fcf0 con 0x7f6d600cd750 2024-09-10T21:45:51.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.366+0000 7f6d67c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f6d6006fff0 con 0x7f6d6015fda0 2024-09-10T21:45:51.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.367+0000 7f6d5fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d6015bc90 0x7f6d6015a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.367+0000 7f6d5fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d6015bc90 0x7f6d6015a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58232/0 (socket says 172.21.15.7:58232) 2024-09-10T21:45:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.367+0000 7f6d5fe00640 1 -- 172.21.15.7:0/3405833097 learned_addr learned my addr 172.21.15.7:0/3405833097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.367+0000 7f6d65200640 1 --2- 172.21.15.7:0/3405833097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d600cd750 0x7f6d601f4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.367+0000 7f6d65c00640 1 --2- 172.21.15.7:0/3405833097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d6015fda0 0x7f6d600c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.367+0000 7f6d5fe00640 1 -- 172.21.15.7:0/3405833097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d6015fda0 msgr2=0x7f6d600c83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.367+0000 7f6d5fe00640 1 --2- 172.21.15.7:0/3405833097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d6015fda0 0x7f6d600c83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.367+0000 7f6d5fe00640 1 -- 172.21.15.7:0/3405833097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d600cd750 msgr2=0x7f6d601f4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.367+0000 7f6d5fe00640 1 --2- 172.21.15.7:0/3405833097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d600cd750 0x7f6d601f4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.367+0000 7f6d5fe00640 1 -- 172.21.15.7:0/3405833097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6015f4b0 con 0x7f6d6015bc90 2024-09-10T21:45:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.367+0000 7f6d65c00640 1 --2- 172.21.15.7:0/3405833097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d6015fda0 0x7f6d600c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:45:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.367+0000 7f6d5fe00640 1 --2- 172.21.15.7:0/3405833097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d6015bc90 0x7f6d6015a470 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00c4c0 tx=0x7f6d4c00c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.368+0000 7f6d5cc00640 1 -- 172.21.15.7:0/3405833097 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c0052d0 con 0x7f6d6015bc90 2024-09-10T21:45:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.368+0000 7f6d5cc00640 1 -- 172.21.15.7:0/3405833097 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d4c005490 con 0x7f6d6015bc90 2024-09-10T21:45:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.368+0000 7f6d5cc00640 1 -- 172.21.15.7:0/3405833097 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d4c012890 con 0x7f6d6015bc90 2024-09-10T21:45:51.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.368+0000 7f6d67c00640 1 -- 172.21.15.7:0/3405833097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d600c7350 con 0x7f6d6015bc90 2024-09-10T21:45:51.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.368+0000 7f6d67c00640 1 -- 172.21.15.7:0/3405833097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d600c7910 con 0x7f6d6015bc90 2024-09-10T21:45:51.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.369+0000 7f6d67c00640 1 -- 172.21.15.7:0/3405833097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d60070510 con 0x7f6d6015bc90 2024-09-10T21:45:51.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.370+0000 7f6d5cc00640 1 -- 172.21.15.7:0/3405833097 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6d4c012a30 con 0x7f6d6015bc90 2024-09-10T21:45:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.371+0000 7f6d5cc00640 1 --2- 172.21.15.7:0/3405833097 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6d40077f70 0x7f6d4007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.371+0000 7f6d5cc00640 1 -- 172.21.15.7:0/3405833097 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f6d4c099630 con 0x7f6d6015bc90 2024-09-10T21:45:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.372+0000 7f6d65200640 1 --2- 172.21.15.7:0/3405833097 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6d40077f70 0x7f6d4007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.372+0000 7f6d65200640 1 --2- 172.21.15.7:0/3405833097 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6d40077f70 0x7f6d4007a430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6d50034000 tx=0x7f6d50004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:51.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.376+0000 7f6d5cc00640 1 -- 172.21.15.7:0/3405833097 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6d4c062390 con 0x7f6d6015bc90 2024-09-10T21:45:51.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.650+0000 7f6d67c00640 1 -- 172.21.15.7:0/3405833097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) -- 0x7f6d60069810 con 0x7f6d6015bc90 2024-09-10T21:45:51.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.773+0000 7f6d5cc00640 1 -- 172.21.15.7:0/3405833097 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v193) ==== 169+0+0 (secure 0 0 0) 0x7f6d4c061c90 con 0x7f6d6015bc90 2024-09-10T21:45:51.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.776+0000 7f6d67c00640 1 -- 172.21.15.7:0/3405833097 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6d40077f70 msgr2=0x7f6d4007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:51.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.776+0000 7f6d67c00640 1 --2- 172.21.15.7:0/3405833097 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6d40077f70 0x7f6d4007a430 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6d50034000 tx=0x7f6d50004730 comp rx=0 tx=0).stop 2024-09-10T21:45:51.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.776+0000 7f6d67c00640 1 -- 172.21.15.7:0/3405833097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d6015bc90 msgr2=0x7f6d6015a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:51.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.776+0000 7f6d67c00640 1 --2- 172.21.15.7:0/3405833097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d6015bc90 0x7f6d6015a470 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f6d4c00c4c0 tx=0x7f6d4c00c980 comp rx=0 tx=0).stop 2024-09-10T21:45:51.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.776+0000 7f6d67c00640 1 -- 172.21.15.7:0/3405833097 shutdown_connections 2024-09-10T21:45:51.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.776+0000 7f6d67c00640 1 --2- 172.21.15.7:0/3405833097 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6d40077f70 0x7f6d4007a430 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:51.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.776+0000 7f6d67c00640 1 --2- 172.21.15.7:0/3405833097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6d6015fda0 0x7f6d600c83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:51.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.776+0000 7f6d67c00640 1 --2- 172.21.15.7:0/3405833097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6d6015bc90 0x7f6d6015a470 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:51.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.776+0000 7f6d67c00640 1 --2- 172.21.15.7:0/3405833097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6d600cd750 0x7f6d601f4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:51.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.776+0000 7f6d67c00640 1 -- 172.21.15.7:0/3405833097 >> 172.21.15.7:0/3405833097 conn(0x7f6d600b16d0 msgr2=0x7f6d600b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:51.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.776+0000 7f6d67c00640 1 -- 172.21.15.7:0/3405833097 shutdown_connections 2024-09-10T21:45:51.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.776+0000 7f6d67c00640 1 -- 172.21.15.7:0/3405833097 wait complete. 2024-09-10T21:45:51.793 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-09-10T21:45:51.793 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-09-10T21:45:51.793 INFO:tasks.cephfs_test_runner:test_health_status_after_dentry_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-10T21:45:51.794 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:45:51.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.954+0000 7fe1d2400640 1 -- 172.21.15.7:0/2244007149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1cc0cd750 msgr2=0x7fe1cc0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:51.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.954+0000 7fe1d2400640 1 --2- 172.21.15.7:0/2244007149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1cc0cd750 0x7fe1cc0ccef0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc0c3650 tx=0x7fe1b402a4b0 comp rx=0 tx=0).stop 2024-09-10T21:45:51.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.954+0000 7fe1d2400640 1 -- 172.21.15.7:0/2244007149 shutdown_connections 2024-09-10T21:45:51.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.954+0000 7fe1d2400640 1 --2- 172.21.15.7:0/2244007149 >> v2:172.21.15.133:3300/0 conn(0x7fe1cc15fda0 0x7fe1cc160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:51.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.954+0000 7fe1d2400640 1 --2- 172.21.15.7:0/2244007149 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1cc0cd750 0x7fe1cc0ccef0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:51.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.954+0000 7fe1d2400640 1 -- 172.21.15.7:0/2244007149 >> 172.21.15.7:0/2244007149 conn(0x7fe1cc0b16d0 msgr2=0x7fe1cc0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:51.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.954+0000 7fe1d2400640 1 -- 172.21.15.7:0/2244007149 shutdown_connections 2024-09-10T21:45:51.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.954+0000 7fe1d2400640 1 -- 172.21.15.7:0/2244007149 wait complete. 2024-09-10T21:45:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.956+0000 7fe1d2400640 1 Processor -- start 2024-09-10T21:45:51.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.956+0000 7fe1d2400640 1 -- start start 2024-09-10T21:45:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.957+0000 7fe1d2400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cd750 0x7fe1cc0c5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.957+0000 7fe1d2400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1cc15bc90 0x7fe1cc147b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.957+0000 7fe1d2400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1cc15fda0 0x7fe1cc1480b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.957+0000 7fe1d2400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc06fe70 con 0x7fe1cc15fda0 2024-09-10T21:45:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.957+0000 7fe1d2400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc06fcf0 con 0x7fe1cc0cd750 2024-09-10T21:45:51.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.957+0000 7fe1d2400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe1cc06fff0 con 0x7fe1cc15bc90 2024-09-10T21:45:51.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.957+0000 7fe1cbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1cc15fda0 0x7fe1cc1480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:51.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.957+0000 7fe1cbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1cc15fda0 0x7fe1cc1480b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58254/0 (socket says 172.21.15.7:58254) 2024-09-10T21:45:51.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.957+0000 7fe1cbe00640 1 -- 172.21.15.7:0/2731705969 learned_addr learned my addr 172.21.15.7:0/2731705969 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:51.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.957+0000 7fe1c2a00640 1 --2- 172.21.15.7:0/2731705969 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1cc15bc90 0x7fe1cc147b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:51.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.957+0000 7fe1cb400640 1 --2- 172.21.15.7:0/2731705969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cd750 0x7fe1cc0c5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.958+0000 7fe1cbe00640 1 -- 172.21.15.7:0/2731705969 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1cc15bc90 msgr2=0x7fe1cc147b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.958+0000 7fe1cbe00640 1 --2- 172.21.15.7:0/2731705969 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1cc15bc90 0x7fe1cc147b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.958+0000 7fe1cbe00640 1 -- 172.21.15.7:0/2731705969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cd750 msgr2=0x7fe1cc0c5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.958+0000 7fe1cbe00640 1 --2- 172.21.15.7:0/2731705969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cd750 0x7fe1cc0c5bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.958+0000 7fe1cbe00640 1 -- 172.21.15.7:0/2731705969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc155b50 con 0x7fe1cc15fda0 2024-09-10T21:45:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.958+0000 7fe1cbe00640 1 --2- 172.21.15.7:0/2731705969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1cc15fda0 0x7fe1cc1480b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400cab0 tx=0x7fe1c400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.958+0000 7fe1c3e00640 1 -- 172.21.15.7:0/2731705969 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c400adc0 con 0x7fe1cc15fda0 2024-09-10T21:45:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.958+0000 7fe1c3e00640 1 -- 172.21.15.7:0/2731705969 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1c400f450 con 0x7fe1cc15fda0 2024-09-10T21:45:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.958+0000 7fe1c3e00640 1 -- 172.21.15.7:0/2731705969 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1c400f8a0 con 0x7fe1cc15fda0 2024-09-10T21:45:51.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.958+0000 7fe1d2400640 1 -- 172.21.15.7:0/2731705969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1cc155e30 con 0x7fe1cc15fda0 2024-09-10T21:45:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.958+0000 7fe1d2400640 1 -- 172.21.15.7:0/2731705969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1cc156110 con 0x7fe1cc15fda0 2024-09-10T21:45:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.959+0000 7fe1d2400640 1 -- 172.21.15.7:0/2731705969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1cc06e730 con 0x7fe1cc15fda0 2024-09-10T21:45:51.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.961+0000 7fe1c3e00640 1 -- 172.21.15.7:0/2731705969 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe1c400fa40 con 0x7fe1cc15fda0 2024-09-10T21:45:51.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.962+0000 7fe1c3e00640 1 --2- 172.21.15.7:0/2731705969 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe1a8077f70 0x7fe1a807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:51.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.962+0000 7fe1c3e00640 1 -- 172.21.15.7:0/2731705969 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7fe1c409a3a0 con 0x7fe1cc15fda0 2024-09-10T21:45:51.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.962+0000 7fe1cb400640 1 --2- 172.21.15.7:0/2731705969 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe1a8077f70 0x7fe1a807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:51.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.963+0000 7fe1cb400640 1 --2- 172.21.15.7:0/2731705969 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe1a8077f70 0x7fe1a807a430 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc007fd0 tx=0x7fe1bc001410 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:51.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:51.966+0000 7fe1c3e00640 1 -- 172.21.15.7:0/2731705969 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe1cc06e730 con 0x7fe1cc15fda0 2024-09-10T21:45:52.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.160+0000 7fe1d2400640 1 -- 172.21.15.7:0/2731705969 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1cc145800 con 0x7fe1cc15fda0 2024-09-10T21:45:52.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.160+0000 7fe1c3e00640 1 -- 172.21.15.7:0/2731705969 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) ==== 74+0+12992 (secure 0 0 0) 0x7fe1cc145800 con 0x7fe1cc15fda0 2024-09-10T21:45:52.162 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:45:52.162 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":61,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:45:50.764225+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":56,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6834/364473218":"2024-09-11T21:45:47.753348+0000","172.21.15.7:6833/364473218":"2024-09-11T21:45:47.753348+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:45:52.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.163+0000 7fe1d2400640 1 -- 172.21.15.7:0/2731705969 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe1a8077f70 msgr2=0x7fe1a807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:52.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.163+0000 7fe1d2400640 1 --2- 172.21.15.7:0/2731705969 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe1a8077f70 0x7fe1a807a430 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc007fd0 tx=0x7fe1bc001410 comp rx=0 tx=0).stop 2024-09-10T21:45:52.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.163+0000 7fe1d2400640 1 -- 172.21.15.7:0/2731705969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1cc15fda0 msgr2=0x7fe1cc1480b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:52.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.163+0000 7fe1d2400640 1 --2- 172.21.15.7:0/2731705969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1cc15fda0 0x7fe1cc1480b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fe1c400cab0 tx=0x7fe1c400cf70 comp rx=0 tx=0).stop 2024-09-10T21:45:52.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.163+0000 7fe1d2400640 1 -- 172.21.15.7:0/2731705969 shutdown_connections 2024-09-10T21:45:52.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.163+0000 7fe1d2400640 1 --2- 172.21.15.7:0/2731705969 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe1a8077f70 0x7fe1a807a430 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:52.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.163+0000 7fe1d2400640 1 --2- 172.21.15.7:0/2731705969 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1cc15fda0 0x7fe1cc1480b0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:52.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.163+0000 7fe1d2400640 1 --2- 172.21.15.7:0/2731705969 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1cc15bc90 0x7fe1cc147b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:52.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.163+0000 7fe1d2400640 1 --2- 172.21.15.7:0/2731705969 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1cc0cd750 0x7fe1cc0c5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:52.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.163+0000 7fe1d2400640 1 -- 172.21.15.7:0/2731705969 >> 172.21.15.7:0/2731705969 conn(0x7fe1cc0b16d0 msgr2=0x7fe1cc0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:52.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.163+0000 7fe1d2400640 1 -- 172.21.15.7:0/2731705969 shutdown_connections 2024-09-10T21:45:52.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.163+0000 7fe1d2400640 1 -- 172.21.15.7:0/2731705969 wait complete. 2024-09-10T21:45:52.181 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:41:55.889618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:45:52.181 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:45:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.344+0000 7fd32be00640 1 -- 172.21.15.7:0/803299180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd32c0b0190 msgr2=0x7fd32c15f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.344+0000 7fd32be00640 1 --2- 172.21.15.7:0/803299180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd32c0b0190 0x7fd32c15f720 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0afb40 tx=0x7fd31402a440 comp rx=0 tx=0).stop 2024-09-10T21:45:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.344+0000 7fd32be00640 1 -- 172.21.15.7:0/803299180 shutdown_connections 2024-09-10T21:45:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.344+0000 7fd32be00640 1 --2- 172.21.15.7:0/803299180 >> v2:172.21.15.133:3300/0 conn(0x7fd32c06d230 0x7fd32c15fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.344+0000 7fd32be00640 1 --2- 172.21.15.7:0/803299180 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd32c0b0190 0x7fd32c15f720 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:52.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.344+0000 7fd32be00640 1 -- 172.21.15.7:0/803299180 >> 172.21.15.7:0/803299180 conn(0x7fd32c0d0b40 msgr2=0x7fd32c0d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:52.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.344+0000 7fd32be00640 1 -- 172.21.15.7:0/803299180 shutdown_connections 2024-09-10T21:45:52.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.344+0000 7fd32be00640 1 -- 172.21.15.7:0/803299180 wait complete. 2024-09-10T21:45:52.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.345+0000 7fd32be00640 1 Processor -- start 2024-09-10T21:45:52.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.346+0000 7fd32be00640 1 -- start start 2024-09-10T21:45:52.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.346+0000 7fd32be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd32c06d230 0x7fd32c15c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:52.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.346+0000 7fd32be00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd32c0b0190 0x7fd32c15ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:52.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.346+0000 7fd32be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd32c15bcd0 0x7fd32c1f2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:52.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.346+0000 7fd32be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c074770 con 0x7fd32c06d230 2024-09-10T21:45:52.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.346+0000 7fd32be00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd32c0745f0 con 0x7fd32c15bcd0 2024-09-10T21:45:52.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.346+0000 7fd32be00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fd32c0ce710 con 0x7fd32c0b0190 2024-09-10T21:45:52.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.347+0000 7fd32aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd32c06d230 0x7fd32c15c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:52.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.347+0000 7fd32aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd32c06d230 0x7fd32c15c4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58278/0 (socket says 172.21.15.7:58278) 2024-09-10T21:45:52.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.347+0000 7fd32aa00640 1 -- 172.21.15.7:0/834997819 learned_addr learned my addr 172.21.15.7:0/834997819 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:52.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.347+0000 7fd323e00640 1 --2- 172.21.15.7:0/834997819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd32c0b0190 0x7fd32c15ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:52.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.347+0000 7fd32b400640 1 --2- 172.21.15.7:0/834997819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd32c15bcd0 0x7fd32c1f2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:52.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.347+0000 7fd32aa00640 1 -- 172.21.15.7:0/834997819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd32c0b0190 msgr2=0x7fd32c15ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.347+0000 7fd32aa00640 1 --2- 172.21.15.7:0/834997819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd32c0b0190 0x7fd32c15ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.347+0000 7fd32aa00640 1 -- 172.21.15.7:0/834997819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd32c15bcd0 msgr2=0x7fd32c1f2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.347+0000 7fd32aa00640 1 --2- 172.21.15.7:0/834997819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd32c15bcd0 0x7fd32c1f2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.347+0000 7fd32aa00640 1 -- 172.21.15.7:0/834997819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd32c15cf40 con 0x7fd32c06d230 2024-09-10T21:45:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.347+0000 7fd32b400640 1 --2- 172.21.15.7:0/834997819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd32c15bcd0 0x7fd32c1f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:45:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.347+0000 7fd32aa00640 1 --2- 172.21.15.7:0/834997819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd32c06d230 0x7fd32c15c4c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00c430 tx=0x7fd31c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.348+0000 7fd322a00640 1 -- 172.21.15.7:0/834997819 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c011790 con 0x7fd32c06d230 2024-09-10T21:45:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.348+0000 7fd322a00640 1 -- 172.21.15.7:0/834997819 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd31c017070 con 0x7fd32c06d230 2024-09-10T21:45:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.348+0000 7fd322a00640 1 -- 172.21.15.7:0/834997819 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd31c011cc0 con 0x7fd32c06d230 2024-09-10T21:45:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.348+0000 7fd32be00640 1 -- 172.21.15.7:0/834997819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd32c15d210 con 0x7fd32c06d230 2024-09-10T21:45:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.348+0000 7fd32be00640 1 -- 172.21.15.7:0/834997819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd32c15d7d0 con 0x7fd32c06d230 2024-09-10T21:45:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.349+0000 7fd32be00640 1 -- 172.21.15.7:0/834997819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd32c0cb7d0 con 0x7fd32c06d230 2024-09-10T21:45:52.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.350+0000 7fd322a00640 1 -- 172.21.15.7:0/834997819 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd31c01c460 con 0x7fd32c06d230 2024-09-10T21:45:52.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.351+0000 7fd322a00640 1 --2- 172.21.15.7:0/834997819 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd308077fa0 0x7fd30807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:52.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.351+0000 7fd322a00640 1 -- 172.21.15.7:0/834997819 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7fd31c099850 con 0x7fd32c06d230 2024-09-10T21:45:52.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.351+0000 7fd323e00640 1 --2- 172.21.15.7:0/834997819 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd308077fa0 0x7fd30807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:52.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.352+0000 7fd323e00640 1 --2- 172.21.15.7:0/834997819 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd308077fa0 0x7fd30807a460 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd314033000 tx=0x7fd314004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:52.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.355+0000 7fd322a00640 1 -- 172.21.15.7:0/834997819 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd32c0cb7d0 con 0x7fd32c06d230 2024-09-10T21:45:52.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.549+0000 7fd32be00640 1 -- 172.21.15.7:0/834997819 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd32c160330 con 0x7fd32c06d230 2024-09-10T21:45:52.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.549+0000 7fd322a00640 1 -- 172.21.15.7:0/834997819 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v61) ==== 93+0+10 (secure 0 0 0) 0x7fd31c062640 con 0x7fd32c06d230 2024-09-10T21:45:52.550 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-10T21:45:52.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.551+0000 7fd32be00640 1 -- 172.21.15.7:0/834997819 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd308077fa0 msgr2=0x7fd30807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:52.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.551+0000 7fd32be00640 1 --2- 172.21.15.7:0/834997819 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd308077fa0 0x7fd30807a460 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd314033000 tx=0x7fd314004730 comp rx=0 tx=0).stop 2024-09-10T21:45:52.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.551+0000 7fd32be00640 1 -- 172.21.15.7:0/834997819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd32c06d230 msgr2=0x7fd32c15c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:52.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.551+0000 7fd32be00640 1 --2- 172.21.15.7:0/834997819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd32c06d230 0x7fd32c15c4c0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fd31c00c430 tx=0x7fd31c00c8f0 comp rx=0 tx=0).stop 2024-09-10T21:45:52.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.552+0000 7fd32be00640 1 -- 172.21.15.7:0/834997819 shutdown_connections 2024-09-10T21:45:52.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.552+0000 7fd32be00640 1 --2- 172.21.15.7:0/834997819 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd308077fa0 0x7fd30807a460 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:52.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.552+0000 7fd32be00640 1 --2- 172.21.15.7:0/834997819 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd32c15bcd0 0x7fd32c1f2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:52.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.552+0000 7fd32be00640 1 --2- 172.21.15.7:0/834997819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd32c0b0190 0x7fd32c15ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:52.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.552+0000 7fd32be00640 1 --2- 172.21.15.7:0/834997819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd32c06d230 0x7fd32c15c4c0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:52.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.552+0000 7fd32be00640 1 -- 172.21.15.7:0/834997819 >> 172.21.15.7:0/834997819 conn(0x7fd32c0d0b40 msgr2=0x7fd32c0b1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:52.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.552+0000 7fd32be00640 1 -- 172.21.15.7:0/834997819 shutdown_connections 2024-09-10T21:45:52.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.552+0000 7fd32be00640 1 -- 172.21.15.7:0/834997819 wait complete. 2024-09-10T21:45:52.568 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-09-10T21:45:52.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.727+0000 7f12d3c00640 1 -- 172.21.15.7:0/1119032315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12cc073c70 msgr2=0x7f12cc15fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:52.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.727+0000 7f12d3c00640 1 --2- 172.21.15.7:0/1119032315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12cc073c70 0x7f12cc15fda0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f12cc06db70 tx=0x7f12c402a8f0 comp rx=0 tx=0).stop 2024-09-10T21:45:52.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.728+0000 7f12d3c00640 1 -- 172.21.15.7:0/1119032315 shutdown_connections 2024-09-10T21:45:52.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.728+0000 7f12d3c00640 1 --2- 172.21.15.7:0/1119032315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12cc073c70 0x7f12cc15fda0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:52.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.728+0000 7f12d3c00640 1 --2- 172.21.15.7:0/1119032315 >> v2:172.21.15.133:3300/0 conn(0x7f12cc0cd750 0x7f12cc073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:52.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.728+0000 7f12d3c00640 1 --2- 172.21.15.7:0/1119032315 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12cc15bc90 0x7f12cc0ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:52.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.728+0000 7f12d3c00640 1 -- 172.21.15.7:0/1119032315 >> 172.21.15.7:0/1119032315 conn(0x7f12cc0b16d0 msgr2=0x7f12cc0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:52.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.728+0000 7f12d3c00640 1 -- 172.21.15.7:0/1119032315 shutdown_connections 2024-09-10T21:45:52.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.728+0000 7f12d3c00640 1 -- 172.21.15.7:0/1119032315 wait complete. 2024-09-10T21:45:52.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.729+0000 7f12d3c00640 1 Processor -- start 2024-09-10T21:45:52.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.729+0000 7f12d3c00640 1 -- start start 2024-09-10T21:45:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.730+0000 7f12d3c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12cc073c70 0x7f12cc14d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.730+0000 7f12d3c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12cc0cd750 0x7f12cc0a9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.730+0000 7f12d3c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12cc15bc90 0x7f12cc0aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.730+0000 7f12d3c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc160670 con 0x7f12cc0cd750 2024-09-10T21:45:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.730+0000 7f12d3c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f12cc1604f0 con 0x7f12cc15bc90 2024-09-10T21:45:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.730+0000 7f12d3c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f12cc06fcf0 con 0x7f12cc073c70 2024-09-10T21:45:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.731+0000 7f12c3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12cc0cd750 0x7f12cc0a9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.731+0000 7f12c3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12cc0cd750 0x7f12cc0a9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58290/0 (socket says 172.21.15.7:58290) 2024-09-10T21:45:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.731+0000 7f12c3e00640 1 -- 172.21.15.7:0/2933839967 learned_addr learned my addr 172.21.15.7:0/2933839967 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.731+0000 7f12c3e00640 1 -- 172.21.15.7:0/2933839967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12cc073c70 msgr2=0x7f12cc14d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.731+0000 7f12d1200640 1 --2- 172.21.15.7:0/2933839967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12cc073c70 0x7f12cc14d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:52.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.731+0000 7f12d1c00640 1 --2- 172.21.15.7:0/2933839967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12cc15bc90 0x7f12cc0aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.731+0000 7f12c3e00640 1 --2- 172.21.15.7:0/2933839967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12cc073c70 0x7f12cc14d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.731+0000 7f12c3e00640 1 -- 172.21.15.7:0/2933839967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12cc15bc90 msgr2=0x7f12cc0aa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.731+0000 7f12c3e00640 1 --2- 172.21.15.7:0/2933839967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12cc15bc90 0x7f12cc0aa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.731+0000 7f12c3e00640 1 -- 172.21.15.7:0/2933839967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12cc1f3030 con 0x7f12cc0cd750 2024-09-10T21:45:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.731+0000 7f12d1c00640 1 --2- 172.21.15.7:0/2933839967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12cc15bc90 0x7f12cc0aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:45:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.731+0000 7f12d1200640 1 --2- 172.21.15.7:0/2933839967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12cc073c70 0x7f12cc14d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:45:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.731+0000 7f12c3e00640 1 --2- 172.21.15.7:0/2933839967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12cc0cd750 0x7f12cc0a9e90 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f12b400e9f0 tx=0x7f12b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.732+0000 7f12c9600640 1 -- 172.21.15.7:0/2933839967 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b400cd80 con 0x7f12cc0cd750 2024-09-10T21:45:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.732+0000 7f12c9600640 1 -- 172.21.15.7:0/2933839967 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12b4010430 con 0x7f12cc0cd750 2024-09-10T21:45:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.732+0000 7f12c9600640 1 -- 172.21.15.7:0/2933839967 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4010840 con 0x7f12cc0cd750 2024-09-10T21:45:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.732+0000 7f12d3c00640 1 -- 172.21.15.7:0/2933839967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12cc1f3330 con 0x7f12cc0cd750 2024-09-10T21:45:52.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.732+0000 7f12d3c00640 1 -- 172.21.15.7:0/2933839967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12cc1f38f0 con 0x7f12cc0cd750 2024-09-10T21:45:52.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.733+0000 7f12d3c00640 1 -- 172.21.15.7:0/2933839967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12cc073730 con 0x7f12cc0cd750 2024-09-10T21:45:52.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.734+0000 7f12c9600640 1 -- 172.21.15.7:0/2933839967 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f12b40109e0 con 0x7f12cc0cd750 2024-09-10T21:45:52.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.735+0000 7f12c9600640 1 --2- 172.21.15.7:0/2933839967 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f12a8077f70 0x7f12a807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:52.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.736+0000 7f12c9600640 1 -- 172.21.15.7:0/2933839967 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f12b409a660 con 0x7f12cc0cd750 2024-09-10T21:45:52.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.736+0000 7f12d1200640 1 --2- 172.21.15.7:0/2933839967 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f12a8077f70 0x7f12a807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:52.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.737+0000 7f12d1200640 1 --2- 172.21.15.7:0/2933839967 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f12a8077f70 0x7f12a807a430 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009a20 tx=0x7f12bc005d30 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:52.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:52.740+0000 7f12c9600640 1 -- 172.21.15.7:0/2933839967 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f12b4063480 con 0x7f12cc0cd750 2024-09-10T21:45:53.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.010+0000 7f12d3c00640 1 -- 172.21.15.7:0/2933839967 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f12cc0b57c0 con 0x7f12cc0cd750 2024-09-10T21:45:53.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.781+0000 7f12c9600640 1 -- 172.21.15.7:0/2933839967 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v195) ==== 169+0+0 (secure 0 0 0) 0x7f12b4062d80 con 0x7f12cc0cd750 2024-09-10T21:45:53.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.784+0000 7f12d3c00640 1 -- 172.21.15.7:0/2933839967 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f12a8077f70 msgr2=0x7f12a807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:53.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.784+0000 7f12d3c00640 1 --2- 172.21.15.7:0/2933839967 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f12a8077f70 0x7f12a807a430 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009a20 tx=0x7f12bc005d30 comp rx=0 tx=0).stop 2024-09-10T21:45:53.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.784+0000 7f12d3c00640 1 -- 172.21.15.7:0/2933839967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12cc0cd750 msgr2=0x7f12cc0a9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:53.785 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.784+0000 7f12d3c00640 1 --2- 172.21.15.7:0/2933839967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12cc0cd750 0x7f12cc0a9e90 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f12b400e9f0 tx=0x7f12b400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:45:53.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.784+0000 7f12d3c00640 1 -- 172.21.15.7:0/2933839967 shutdown_connections 2024-09-10T21:45:53.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.784+0000 7f12d3c00640 1 --2- 172.21.15.7:0/2933839967 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f12a8077f70 0x7f12a807a430 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:53.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.784+0000 7f12d3c00640 1 --2- 172.21.15.7:0/2933839967 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12cc15bc90 0x7f12cc0aa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:53.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.784+0000 7f12d3c00640 1 --2- 172.21.15.7:0/2933839967 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12cc0cd750 0x7f12cc0a9e90 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:53.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.784+0000 7f12d3c00640 1 --2- 172.21.15.7:0/2933839967 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12cc073c70 0x7f12cc14d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:53.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.784+0000 7f12d3c00640 1 -- 172.21.15.7:0/2933839967 >> 172.21.15.7:0/2933839967 conn(0x7f12cc0b16d0 msgr2=0x7f12cc15b280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:53.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.784+0000 7f12d3c00640 1 -- 172.21.15.7:0/2933839967 shutdown_connections 2024-09-10T21:45:53.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.784+0000 7f12d3c00640 1 -- 172.21.15.7:0/2933839967 wait complete. 2024-09-10T21:45:53.806 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:45:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.970+0000 7fc6df000640 1 -- 172.21.15.7:0/1089625997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d80cbd50 msgr2=0x7fc6d80cc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.970+0000 7fc6df000640 1 --2- 172.21.15.7:0/1089625997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d80cbd50 0x7fc6d80cc190 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009830 tx=0x7fc6c002a510 comp rx=0 tx=0).stop 2024-09-10T21:45:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.970+0000 7fc6df000640 1 -- 172.21.15.7:0/1089625997 shutdown_connections 2024-09-10T21:45:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.970+0000 7fc6df000640 1 --2- 172.21.15.7:0/1089625997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d80cbd50 0x7fc6d80cc190 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.970+0000 7fc6df000640 1 --2- 172.21.15.7:0/1089625997 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6d81619e0 0x7fc6d815e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:53.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.971+0000 7fc6df000640 1 -- 172.21.15.7:0/1089625997 >> 172.21.15.7:0/1089625997 conn(0x7fc6d80b16d0 msgr2=0x7fc6d80b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:53.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.971+0000 7fc6df000640 1 -- 172.21.15.7:0/1089625997 shutdown_connections 2024-09-10T21:45:53.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.971+0000 7fc6df000640 1 -- 172.21.15.7:0/1089625997 wait complete. 2024-09-10T21:45:53.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.972+0000 7fc6df000640 1 Processor -- start 2024-09-10T21:45:53.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.972+0000 7fc6df000640 1 -- start start 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.973+0000 7fc6df000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d80ae020 0x7fc6d81f7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.973+0000 7fc6df000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6d80cbd50 0x7fc6d806f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.973+0000 7fc6df000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d81619e0 0x7fc6d81e6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.973+0000 7fc6df000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d80ae950 con 0x7fc6d80ae020 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.973+0000 7fc6df000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d80ae7d0 con 0x7fc6d81619e0 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.973+0000 7fc6df000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc6d8161dc0 con 0x7fc6d80cbd50 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.973+0000 7fc6d7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d80ae020 0x7fc6d81f7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.973+0000 7fc6d7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d80ae020 0x7fc6d81f7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60478/0 (socket says 172.21.15.7:60478) 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.973+0000 7fc6d7e00640 1 -- 172.21.15.7:0/4202184836 learned_addr learned my addr 172.21.15.7:0/4202184836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:53.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.974+0000 7fc6cf400640 1 --2- 172.21.15.7:0/4202184836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6d80cbd50 0x7fc6d806f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.974+0000 7fc6dd000640 1 --2- 172.21.15.7:0/4202184836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d81619e0 0x7fc6d81e6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.974+0000 7fc6d7e00640 1 -- 172.21.15.7:0/4202184836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6d80cbd50 msgr2=0x7fc6d806f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.974+0000 7fc6d7e00640 1 --2- 172.21.15.7:0/4202184836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6d80cbd50 0x7fc6d806f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.974+0000 7fc6d7e00640 1 -- 172.21.15.7:0/4202184836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d81619e0 msgr2=0x7fc6d81e6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.974+0000 7fc6d7e00640 1 --2- 172.21.15.7:0/4202184836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d81619e0 0x7fc6d81e6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.974+0000 7fc6d7e00640 1 -- 172.21.15.7:0/4202184836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d806e510 con 0x7fc6d80ae020 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.974+0000 7fc6cf400640 1 --2- 172.21.15.7:0/4202184836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6d80cbd50 0x7fc6d806f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:45:53.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.974+0000 7fc6dd000640 1 --2- 172.21.15.7:0/4202184836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d81619e0 0x7fc6d81e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:45:53.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.974+0000 7fc6d7e00640 1 --2- 172.21.15.7:0/4202184836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d80ae020 0x7fc6d81f7ec0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800d950 tx=0x7fc6c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:53.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.974+0000 7fc6d4c00640 1 -- 172.21.15.7:0/4202184836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c8017070 con 0x7fc6d80ae020 2024-09-10T21:45:53.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.974+0000 7fc6d4c00640 1 -- 172.21.15.7:0/4202184836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6c8009d20 con 0x7fc6d80ae020 2024-09-10T21:45:53.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.974+0000 7fc6d4c00640 1 -- 172.21.15.7:0/4202184836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c80106c0 con 0x7fc6d80ae020 2024-09-10T21:45:53.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.974+0000 7fc6df000640 1 -- 172.21.15.7:0/4202184836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d806e7f0 con 0x7fc6d80ae020 2024-09-10T21:45:53.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.975+0000 7fc6df000640 1 -- 172.21.15.7:0/4202184836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d806edb0 con 0x7fc6d80ae020 2024-09-10T21:45:53.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.976+0000 7fc6df000640 1 -- 172.21.15.7:0/4202184836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d80cc190 con 0x7fc6d80ae020 2024-09-10T21:45:53.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.977+0000 7fc6d4c00640 1 -- 172.21.15.7:0/4202184836 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc6c8010860 con 0x7fc6d80ae020 2024-09-10T21:45:53.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.978+0000 7fc6d4c00640 1 --2- 172.21.15.7:0/4202184836 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc6b4077f70 0x7fc6b407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:53.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.978+0000 7fc6d4c00640 1 -- 172.21.15.7:0/4202184836 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7fc6c8099f40 con 0x7fc6d80ae020 2024-09-10T21:45:53.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.978+0000 7fc6cf400640 1 --2- 172.21.15.7:0/4202184836 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc6b4077f70 0x7fc6b407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:53.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.979+0000 7fc6cf400640 1 --2- 172.21.15.7:0/4202184836 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc6b4077f70 0x7fc6b407a430 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0034000 tx=0x7fc6c0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:53.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:53.982+0000 7fc6d4c00640 1 -- 172.21.15.7:0/4202184836 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc6c8062ca0 con 0x7fc6d80ae020 2024-09-10T21:45:54.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.177+0000 7fc6df000640 1 -- 172.21.15.7:0/4202184836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fc6d80747d0 con 0x7fc6d80ae020 2024-09-10T21:45:54.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.178+0000 7fc6d4c00640 1 -- 172.21.15.7:0/4202184836 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+1998 (secure 0 0 0) 0x7fc6c80625a0 con 0x7fc6d80ae020 2024-09-10T21:45:54.179 INFO:teuthology.orchestra.run.smithi007.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:45:54.179 INFO:teuthology.orchestra.run.smithi007.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:45:54.179 INFO:teuthology.orchestra.run.smithi007.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:45:54.179 INFO:teuthology.orchestra.run.smithi007.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:45:54.179 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3770.177547 2024-09-10T21:45:54.179 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3542.062395 2024-09-10T21:45:54.179 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3788.037135 2024-09-10T21:45:54.180 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5808.986321 2024-09-10T21:45:54.180 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4630.109911 2024-09-10T21:45:54.180 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4510.255493 2024-09-10T21:45:54.180 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4792.698527 2024-09-10T21:45:54.180 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6842.370281 2024-09-10T21:45:54.180 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_mds 20/20 2024-09-10T21:45:54.180 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:45:54.180 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_ms 2/2 2024-09-10T21:45:54.180 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_debug_frag true 2024-09-10T21:45:54.180 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:45:54.180 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:45:54.180 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_verify_scatter true 2024-09-10T21:45:54.180 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:45:54.181 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:45:54.181 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:45:54.181 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced client_mount_timeout 600 2024-09-10T21:45:54.181 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced debug_client 20/20 2024-09-10T21:45:54.181 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced debug_ms 1/1 2024-09-10T21:45:54.181 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:45:54.181 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:45:54.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.180+0000 7fc6df000640 1 -- 172.21.15.7:0/4202184836 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc6b4077f70 msgr2=0x7fc6b407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:54.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.180+0000 7fc6df000640 1 --2- 172.21.15.7:0/4202184836 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc6b4077f70 0x7fc6b407a430 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0034000 tx=0x7fc6c0004730 comp rx=0 tx=0).stop 2024-09-10T21:45:54.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.180+0000 7fc6df000640 1 -- 172.21.15.7:0/4202184836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d80ae020 msgr2=0x7fc6d81f7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:54.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.180+0000 7fc6df000640 1 --2- 172.21.15.7:0/4202184836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d80ae020 0x7fc6d81f7ec0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc6c800d950 tx=0x7fc6c800de10 comp rx=0 tx=0).stop 2024-09-10T21:45:54.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.180+0000 7fc6df000640 1 -- 172.21.15.7:0/4202184836 shutdown_connections 2024-09-10T21:45:54.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.180+0000 7fc6df000640 1 --2- 172.21.15.7:0/4202184836 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc6b4077f70 0x7fc6b407a430 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:54.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.180+0000 7fc6df000640 1 --2- 172.21.15.7:0/4202184836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc6d81619e0 0x7fc6d81e6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:54.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.180+0000 7fc6df000640 1 --2- 172.21.15.7:0/4202184836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc6d80cbd50 0x7fc6d806f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:54.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.180+0000 7fc6df000640 1 --2- 172.21.15.7:0/4202184836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6d80ae020 0x7fc6d81f7ec0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:54.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.180+0000 7fc6df000640 1 -- 172.21.15.7:0/4202184836 >> 172.21.15.7:0/4202184836 conn(0x7fc6d80b16d0 msgr2=0x7fc6d80cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:54.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.180+0000 7fc6df000640 1 -- 172.21.15.7:0/4202184836 shutdown_connections 2024-09-10T21:45:54.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.181+0000 7fc6df000640 1 -- 172.21.15.7:0/4202184836 wait complete. 2024-09-10T21:45:54.197 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:45:54.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.359+0000 7f3cc9a00640 1 -- 172.21.15.7:0/3589765122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc4151170 msgr2=0x7f3cc406e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:54.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.359+0000 7f3cc9a00640 1 --2- 172.21.15.7:0/3589765122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc4151170 0x7f3cc406e4c0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40b4940 tx=0x7f3cb402a4d0 comp rx=0 tx=0).stop 2024-09-10T21:45:54.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.359+0000 7f3cc9a00640 1 -- 172.21.15.7:0/3589765122 shutdown_connections 2024-09-10T21:45:54.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.359+0000 7f3cc9a00640 1 --2- 172.21.15.7:0/3589765122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cc415ddb0 0x7f3cc415e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:54.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.359+0000 7f3cc9a00640 1 --2- 172.21.15.7:0/3589765122 >> v2:172.21.15.133:3300/0 conn(0x7f3cc40cf2e0 0x7f3cc40b3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:54.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.359+0000 7f3cc9a00640 1 --2- 172.21.15.7:0/3589765122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc4151170 0x7f3cc406e4c0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:54.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.359+0000 7f3cc9a00640 1 -- 172.21.15.7:0/3589765122 >> 172.21.15.7:0/3589765122 conn(0x7f3cc406cd00 msgr2=0x7f3cc406d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:54.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.359+0000 7f3cc9a00640 1 -- 172.21.15.7:0/3589765122 shutdown_connections 2024-09-10T21:45:54.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.359+0000 7f3cc9a00640 1 -- 172.21.15.7:0/3589765122 wait complete. 2024-09-10T21:45:54.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.360+0000 7f3cc9a00640 1 Processor -- start 2024-09-10T21:45:54.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.361+0000 7f3cc9a00640 1 -- start start 2024-09-10T21:45:54.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.361+0000 7f3cc9a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cc40cf2e0 0x7f3cc4139dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:54.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.361+0000 7f3cc9a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc4151170 0x7f3cc415a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:54.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.362+0000 7f3cc9a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cc415ddb0 0x7f3cc415bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:54.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.362+0000 7f3cc9a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc40cff40 con 0x7f3cc4151170 2024-09-10T21:45:54.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.362+0000 7f3cc9a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc40cfdc0 con 0x7f3cc415ddb0 2024-09-10T21:45:54.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.362+0000 7f3cc9a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f3cc40d00c0 con 0x7f3cc40cf2e0 2024-09-10T21:45:54.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.362+0000 7f3cc3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc4151170 0x7f3cc415a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:54.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.362+0000 7f3cc3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc4151170 0x7f3cc415a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60504/0 (socket says 172.21.15.7:60504) 2024-09-10T21:45:54.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.362+0000 7f3cc3400640 1 -- 172.21.15.7:0/3320093257 learned_addr learned my addr 172.21.15.7:0/3320093257 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:54.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.362+0000 7f3cc3e00640 1 --2- 172.21.15.7:0/3320093257 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cc40cf2e0 0x7f3cc4139dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:54.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.362+0000 7f3cc9000640 1 --2- 172.21.15.7:0/3320093257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cc415ddb0 0x7f3cc415bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:54.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.362+0000 7f3cc3400640 1 -- 172.21.15.7:0/3320093257 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cc40cf2e0 msgr2=0x7f3cc4139dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:54.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.362+0000 7f3cc3400640 1 --2- 172.21.15.7:0/3320093257 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cc40cf2e0 0x7f3cc4139dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:54.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.362+0000 7f3cc3400640 1 -- 172.21.15.7:0/3320093257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cc415ddb0 msgr2=0x7f3cc415bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:54.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.362+0000 7f3cc3400640 1 --2- 172.21.15.7:0/3320093257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cc415ddb0 0x7f3cc415bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:54.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.362+0000 7f3cc3400640 1 -- 172.21.15.7:0/3320093257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc415ac90 con 0x7f3cc4151170 2024-09-10T21:45:54.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.363+0000 7f3cc3400640 1 --2- 172.21.15.7:0/3320093257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc4151170 0x7f3cc415a750 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000d950 tx=0x7f3cb000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:54.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.363+0000 7f3cbbe00640 1 -- 172.21.15.7:0/3320093257 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0017070 con 0x7f3cc4151170 2024-09-10T21:45:54.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.363+0000 7f3cbbe00640 1 -- 172.21.15.7:0/3320093257 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3cb0009d20 con 0x7f3cc4151170 2024-09-10T21:45:54.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.363+0000 7f3cc9a00640 1 -- 172.21.15.7:0/3320093257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc41f8320 con 0x7f3cc4151170 2024-09-10T21:45:54.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.363+0000 7f3cbbe00640 1 -- 172.21.15.7:0/3320093257 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb00106c0 con 0x7f3cc4151170 2024-09-10T21:45:54.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.363+0000 7f3cc9a00640 1 -- 172.21.15.7:0/3320093257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc41f88e0 con 0x7f3cc4151170 2024-09-10T21:45:54.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.364+0000 7f3cc9a00640 1 -- 172.21.15.7:0/3320093257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cc415e1e0 con 0x7f3cc4151170 2024-09-10T21:45:54.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.365+0000 7f3cbbe00640 1 -- 172.21.15.7:0/3320093257 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3cb0010860 con 0x7f3cc4151170 2024-09-10T21:45:54.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.366+0000 7f3cbbe00640 1 --2- 172.21.15.7:0/3320093257 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3ca4077f70 0x7f3ca407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:54.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.366+0000 7f3cbbe00640 1 -- 172.21.15.7:0/3320093257 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f3cb0099e00 con 0x7f3cc4151170 2024-09-10T21:45:54.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.367+0000 7f3cc3e00640 1 --2- 172.21.15.7:0/3320093257 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3ca4077f70 0x7f3ca407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:54.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.367+0000 7f3cc3e00640 1 --2- 172.21.15.7:0/3320093257 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3ca4077f70 0x7f3ca407a430 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4033000 tx=0x7f3cb4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:54.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.370+0000 7f3cbbe00640 1 -- 172.21.15.7:0/3320093257 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3cb00625a0 con 0x7f3cc4151170 2024-09-10T21:45:54.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.564+0000 7f3cc9a00640 1 -- 172.21.15.7:0/3320093257 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f3cc40cd1c0 con 0x7f3cc4151170 2024-09-10T21:45:54.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.568+0000 7f3cbbe00640 1 -- 172.21.15.7:0/3320093257 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f3cb0066fd0 con 0x7f3cc4151170 2024-09-10T21:45:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.570+0000 7f3cc9a00640 1 -- 172.21.15.7:0/3320093257 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3ca4077f70 msgr2=0x7f3ca407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.570+0000 7f3cc9a00640 1 --2- 172.21.15.7:0/3320093257 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3ca4077f70 0x7f3ca407a430 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4033000 tx=0x7f3cb4004730 comp rx=0 tx=0).stop 2024-09-10T21:45:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.570+0000 7f3cc9a00640 1 -- 172.21.15.7:0/3320093257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc4151170 msgr2=0x7f3cc415a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:54.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.570+0000 7f3cc9a00640 1 --2- 172.21.15.7:0/3320093257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc4151170 0x7f3cc415a750 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000d950 tx=0x7f3cb000de10 comp rx=0 tx=0).stop 2024-09-10T21:45:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.571+0000 7f3cc9a00640 1 -- 172.21.15.7:0/3320093257 shutdown_connections 2024-09-10T21:45:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.571+0000 7f3cc9a00640 1 --2- 172.21.15.7:0/3320093257 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3ca4077f70 0x7f3ca407a430 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.571+0000 7f3cc9a00640 1 --2- 172.21.15.7:0/3320093257 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3cc415ddb0 0x7f3cc415bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.571+0000 7f3cc9a00640 1 --2- 172.21.15.7:0/3320093257 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3cc4151170 0x7f3cc415a750 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.571+0000 7f3cc9a00640 1 --2- 172.21.15.7:0/3320093257 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3cc40cf2e0 0x7f3cc4139dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.571+0000 7f3cc9a00640 1 -- 172.21.15.7:0/3320093257 >> 172.21.15.7:0/3320093257 conn(0x7f3cc406cd00 msgr2=0x7f3cc40cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.571+0000 7f3cc9a00640 1 -- 172.21.15.7:0/3320093257 shutdown_connections 2024-09-10T21:45:54.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:54.571+0000 7f3cc9a00640 1 -- 172.21.15.7:0/3320093257 wait complete. 2024-09-10T21:45:54.588 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:45:54.588 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:45:54.688 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:45:54.772 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:45:54.772 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:45:54.772 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:45:54.808 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:45:54.808 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:45:54.808 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:45:54.845 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:45:54.845 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:45:54.845 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:45:54.884 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:45:54.884 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:45:54.884 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:45:54.922 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:45:54.922 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:45:54.922 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:45:55.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.084+0000 7f8645c00640 1 -- 172.21.15.7:0/1815244943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86240039e0 msgr2=0x7f8624003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:55.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.084+0000 7f8645c00640 1 --2- 172.21.15.7:0/1815244943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86240039e0 0x7f8624003dc0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8630006f80 tx=0x7f8624005830 comp rx=0 tx=0).stop 2024-09-10T21:45:55.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.084+0000 7f863f400640 1 -- 172.21.15.7:0/1815244943 reap_dead start 2024-09-10T21:45:55.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.084+0000 7f8645c00640 1 -- 172.21.15.7:0/1815244943 shutdown_connections 2024-09-10T21:45:55.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.084+0000 7f8645c00640 1 -- 172.21.15.7:0/1815244943 >> 172.21.15.7:0/1815244943 conn(0x7f86400b16d0 msgr2=0x7f86400b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:55.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.084+0000 7f8645c00640 1 -- 172.21.15.7:0/1815244943 shutdown_connections 2024-09-10T21:45:55.086 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.084+0000 7f8645c00640 1 -- 172.21.15.7:0/1815244943 wait complete. 2024-09-10T21:45:55.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.086+0000 7f8645c00640 1 Processor -- start 2024-09-10T21:45:55.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.086+0000 7f8645c00640 1 -- start start 2024-09-10T21:45:55.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.087+0000 7f8645c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86240039e0 0x7f86401453c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:55.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.087+0000 7f8645c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f864015d7d0 0x7f8640148530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:55.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.087+0000 7f8645c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86401f5ca0 0x7f86401f6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:55.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.087+0000 7f8645c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8640070610 con 0x7f86240039e0 2024-09-10T21:45:55.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.087+0000 7f8645c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f86400ccef0 con 0x7f864015d7d0 2024-09-10T21:45:55.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.087+0000 7f8645c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f864015d340 con 0x7f86401f5ca0 2024-09-10T21:45:55.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.087+0000 7f863ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86240039e0 0x7f86401453c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:55.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.087+0000 7f863ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86240039e0 0x7f86401453c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60530/0 (socket says 172.21.15.7:60530) 2024-09-10T21:45:55.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.087+0000 7f863ea00640 1 -- 172.21.15.7:0/3257758514 learned_addr learned my addr 172.21.15.7:0/3257758514 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:55.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.087+0000 7f8636000640 1 --2- 172.21.15.7:0/3257758514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f864015d7d0 0x7f8640148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:55.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.087+0000 7f863f400640 1 --2- 172.21.15.7:0/3257758514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86401f5ca0 0x7f86401f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:55.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.088+0000 7f863ea00640 1 -- 172.21.15.7:0/3257758514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86401f5ca0 msgr2=0x7f86401f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:55.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.088+0000 7f863ea00640 1 --2- 172.21.15.7:0/3257758514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86401f5ca0 0x7f86401f6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:55.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.088+0000 7f863ea00640 1 -- 172.21.15.7:0/3257758514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f864015d7d0 msgr2=0x7f8640148530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:55.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.088+0000 7f863ea00640 1 --2- 172.21.15.7:0/3257758514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f864015d7d0 0x7f8640148530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:55.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.088+0000 7f863ea00640 1 -- 172.21.15.7:0/3257758514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f864015c730 con 0x7f86240039e0 2024-09-10T21:45:55.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.088+0000 7f863ea00640 1 --2- 172.21.15.7:0/3257758514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86240039e0 0x7f86401453c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f863000c320 tx=0x7f8630003e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:55.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.088+0000 7f8637400640 1 -- 172.21.15.7:0/3257758514 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8630032030 con 0x7f86240039e0 2024-09-10T21:45:55.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.088+0000 7f8637400640 1 -- 172.21.15.7:0/3257758514 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8630006530 con 0x7f86240039e0 2024-09-10T21:45:55.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.088+0000 7f8637400640 1 -- 172.21.15.7:0/3257758514 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f863002e460 con 0x7f86240039e0 2024-09-10T21:45:55.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.088+0000 7f8645c00640 1 -- 172.21.15.7:0/3257758514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f864015ca10 con 0x7f86240039e0 2024-09-10T21:45:55.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.088+0000 7f8645c00640 1 -- 172.21.15.7:0/3257758514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f864015cfd0 con 0x7f86240039e0 2024-09-10T21:45:55.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.090+0000 7f8645c00640 1 -- 172.21.15.7:0/3257758514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86401601f0 con 0x7f86240039e0 2024-09-10T21:45:55.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.091+0000 7f8637400640 1 -- 172.21.15.7:0/3257758514 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f863002e600 con 0x7f86240039e0 2024-09-10T21:45:55.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.092+0000 7f8637400640 1 --2- 172.21.15.7:0/3257758514 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f861c077f70 0x7f861c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:55.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.092+0000 7f8637400640 1 -- 172.21.15.7:0/3257758514 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f86300b97c0 con 0x7f86240039e0 2024-09-10T21:45:55.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.092+0000 7f8636000640 1 --2- 172.21.15.7:0/3257758514 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f861c077f70 0x7f861c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:55.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.093+0000 7f8636000640 1 --2- 172.21.15.7:0/3257758514 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f861c077f70 0x7f861c07a430 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f864015f310 tx=0x7f8628005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:55.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.096+0000 7f8637400640 1 -- 172.21.15.7:0/3257758514 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8630082520 con 0x7f86240039e0 2024-09-10T21:45:55.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.326+0000 7f8645c00640 1 -- 172.21.15.7:0/3257758514 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f864015f5c0 con 0x7f86240039e0 2024-09-10T21:45:55.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.327+0000 7f8637400640 1 -- 172.21.15.7:0/3257758514 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+3908 (secure 0 0 0) 0x7f8630081e20 con 0x7f86240039e0 2024-09-10T21:45:55.328 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:45:55.328 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":39,"btime":"2024-09-10T21:45:48:755623+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5473,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/2853074495","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2853074495},{"type":"v1","addr":"172.21.15.133:6836","nonce":2853074495}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":5476,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3498706462","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3498706462},{"type":"v1","addr":"172.21.15.133:6837","nonce":3498706462}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":5691,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/521190705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":521190705},{"type":"v1","addr":"172.21.15.7:6835","nonce":521190705}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":6084,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3681651186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3681651186},{"type":"v1","addr":"172.21.15.7:6834","nonce":3681651186}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39}],"filesystems":[]} 2024-09-10T21:45:55.329 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 39 2024-09-10T21:45:55.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.329+0000 7f8645c00640 1 -- 172.21.15.7:0/3257758514 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f861c077f70 msgr2=0x7f861c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:55.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.329+0000 7f8645c00640 1 --2- 172.21.15.7:0/3257758514 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f861c077f70 0x7f861c07a430 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f864015f310 tx=0x7f8628005f90 comp rx=0 tx=0).stop 2024-09-10T21:45:55.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.329+0000 7f8645c00640 1 -- 172.21.15.7:0/3257758514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86240039e0 msgr2=0x7f86401453c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:55.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.329+0000 7f8645c00640 1 --2- 172.21.15.7:0/3257758514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86240039e0 0x7f86401453c0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f863000c320 tx=0x7f8630003e40 comp rx=0 tx=0).stop 2024-09-10T21:45:55.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.329+0000 7f8645c00640 1 -- 172.21.15.7:0/3257758514 shutdown_connections 2024-09-10T21:45:55.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.329+0000 7f8645c00640 1 --2- 172.21.15.7:0/3257758514 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f861c077f70 0x7f861c07a430 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:55.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.329+0000 7f8645c00640 1 --2- 172.21.15.7:0/3257758514 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f864015d7d0 0x7f8640148530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:55.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.329+0000 7f8645c00640 1 --2- 172.21.15.7:0/3257758514 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f86401f5ca0 0x7f86401f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:55.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.329+0000 7f8645c00640 1 --2- 172.21.15.7:0/3257758514 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f86240039e0 0x7f86401453c0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:55.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.329+0000 7f8645c00640 1 -- 172.21.15.7:0/3257758514 >> 172.21.15.7:0/3257758514 conn(0x7f86400b16d0 msgr2=0x7f86400b49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:55.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.329+0000 7f8645c00640 1 -- 172.21.15.7:0/3257758514 shutdown_connections 2024-09-10T21:45:55.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.329+0000 7f8645c00640 1 -- 172.21.15.7:0/3257758514 wait complete. 2024-09-10T21:45:55.347 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:45:55.347 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:45:55.347 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:45:55.347 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:45:55.348 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:45:55.348 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:45:55.348 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:45:55.348 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:45:55.349 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:45:55.349 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:45:55.349 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:45:55.349 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:45:55.349 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:45:55.350 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:45:55.350 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:45:55.350 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:45:55.362 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:45:55.362 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:45:55.363 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:45:55.364 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:45:55.365 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:45:55.365 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:45:55.366 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:45:55.366 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:45:55.408 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:45:55.409 DEBUG:teuthology.parallel:result is None 2024-09-10T21:45:55.410 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:45:55.410 DEBUG:teuthology.parallel:result is None 2024-09-10T21:45:55.411 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:45:55.411 DEBUG:teuthology.parallel:result is None 2024-09-10T21:45:55.412 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:45:55.412 DEBUG:teuthology.parallel:result is None 2024-09-10T21:45:55.413 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:45:55.541 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2024-09-10T21:45:55.541 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2024-09-10T21:45:55.542 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-09-10T21:45:55.543 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-09-10T21:45:55.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.581+0000 7f35c0800640 1 -- 172.21.15.7:0/3045643167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35b81579b0 msgr2=0x7f35b80aec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:55.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.581+0000 7f35c0800640 1 --2- 172.21.15.7:0/3045643167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35b81579b0 0x7f35b80aec20 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009830 tx=0x7f35ac02a510 comp rx=0 tx=0).stop 2024-09-10T21:45:55.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.581+0000 7f35c0800640 1 -- 172.21.15.7:0/3045643167 shutdown_connections 2024-09-10T21:45:55.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.581+0000 7f35c0800640 1 --2- 172.21.15.7:0/3045643167 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35b80ccc30 0x7f35b81639b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:55.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.581+0000 7f35c0800640 1 --2- 172.21.15.7:0/3045643167 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35b81579b0 0x7f35b80aec20 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:55.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.581+0000 7f35c0800640 1 -- 172.21.15.7:0/3045643167 >> 172.21.15.7:0/3045643167 conn(0x7f35b80b1660 msgr2=0x7f35b80b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:55.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.581+0000 7f35c0800640 1 -- 172.21.15.7:0/3045643167 shutdown_connections 2024-09-10T21:45:55.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.581+0000 7f35c0800640 1 -- 172.21.15.7:0/3045643167 wait complete. 2024-09-10T21:45:55.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.583+0000 7f35c0800640 1 Processor -- start 2024-09-10T21:45:55.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.583+0000 7f35c0800640 1 -- start start 2024-09-10T21:45:55.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.584+0000 7f35c0800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35b80ccc30 0x7f35b814db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:55.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.584+0000 7f35c0800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35b80cf650 0x7f35b81661b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:55.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.584+0000 7f35c0800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35b81579b0 0x7f35b814d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:55.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.584+0000 7f35c0800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35b8070780 con 0x7f35b80cf650 2024-09-10T21:45:55.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.584+0000 7f35c0800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f35b8070600 con 0x7f35b81579b0 2024-09-10T21:45:55.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.584+0000 7f35c0800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f35b8163ef0 con 0x7f35b80ccc30 2024-09-10T21:45:55.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.584+0000 7f35bd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35b80cf650 0x7f35b81661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:55.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.584+0000 7f35bd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35b80cf650 0x7f35b81661b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60550/0 (socket says 172.21.15.7:60550) 2024-09-10T21:45:55.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.584+0000 7f35bd400640 1 -- 172.21.15.7:0/3654081191 learned_addr learned my addr 172.21.15.7:0/3654081191 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:55.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.584+0000 7f35bde00640 1 --2- 172.21.15.7:0/3654081191 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35b80ccc30 0x7f35b814db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:55.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.584+0000 7f35be800640 1 --2- 172.21.15.7:0/3654081191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35b81579b0 0x7f35b814d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:55.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.585+0000 7f35bd400640 1 -- 172.21.15.7:0/3654081191 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35b80ccc30 msgr2=0x7f35b814db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:55.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.585+0000 7f35bd400640 1 --2- 172.21.15.7:0/3654081191 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35b80ccc30 0x7f35b814db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:55.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.585+0000 7f35bd400640 1 -- 172.21.15.7:0/3654081191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35b81579b0 msgr2=0x7f35b814d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:55.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.585+0000 7f35bd400640 1 --2- 172.21.15.7:0/3654081191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35b81579b0 0x7f35b814d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:55.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.585+0000 7f35bd400640 1 -- 172.21.15.7:0/3654081191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35b815bdb0 con 0x7f35b80cf650 2024-09-10T21:45:55.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.585+0000 7f35bd400640 1 --2- 172.21.15.7:0/3654081191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35b80cf650 0x7f35b81661b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f35a800c530 tx=0x7f35a800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:55.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.585+0000 7f35b2000640 1 -- 172.21.15.7:0/3654081191 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a80052d0 con 0x7f35b80cf650 2024-09-10T21:45:55.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.585+0000 7f35b2000640 1 -- 172.21.15.7:0/3654081191 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f35a8005490 con 0x7f35b80cf650 2024-09-10T21:45:55.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.585+0000 7f35b2000640 1 -- 172.21.15.7:0/3654081191 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35a8012890 con 0x7f35b80cf650 2024-09-10T21:45:55.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.585+0000 7f35c0800640 1 -- 172.21.15.7:0/3654081191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35b815c090 con 0x7f35b80cf650 2024-09-10T21:45:55.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.586+0000 7f35c0800640 1 -- 172.21.15.7:0/3654081191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35b8165180 con 0x7f35b80cf650 2024-09-10T21:45:55.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.586+0000 7f35c0800640 1 -- 172.21.15.7:0/3654081191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35b80aec20 con 0x7f35b80cf650 2024-09-10T21:45:55.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.589+0000 7f35b2000640 1 -- 172.21.15.7:0/3654081191 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f35a8012a30 con 0x7f35b80cf650 2024-09-10T21:45:55.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.590+0000 7f35b2000640 1 --2- 172.21.15.7:0/3654081191 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f359c077f70 0x7f359c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.590+0000 7f35b2000640 1 -- 172.21.15.7:0/3654081191 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 4943+0+0 (secure 0 0 0) 0x7f35a8099ee0 con 0x7f35b80cf650 2024-09-10T21:45:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.590+0000 7f35bde00640 1 --2- 172.21.15.7:0/3654081191 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f359c077f70 0x7f359c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.591+0000 7f35bde00640 1 --2- 172.21.15.7:0/3654081191 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f359c077f70 0x7f359c07a430 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f35ac034000 tx=0x7f35ac004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.593+0000 7f35b2000640 1 -- 172.21.15.7:0/3654081191 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f35a8062700 con 0x7f35b80cf650 2024-09-10T21:45:55.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:55.804+0000 7f35c0800640 1 -- 172.21.15.7:0/3654081191 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f35b80ae4c0 con 0x7f35b80cf650 2024-09-10T21:45:56.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:56.815+0000 7f35b2000640 1 -- 172.21.15.7:0/3654081191 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v66) ==== 97+0+0 (secure 0 0 0) 0x7f35a8067130 con 0x7f35b80cf650 2024-09-10T21:45:56.817 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-09-10T21:45:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:56.819+0000 7f35c0800640 1 -- 172.21.15.7:0/3654081191 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f359c077f70 msgr2=0x7f359c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:56.819+0000 7f35c0800640 1 --2- 172.21.15.7:0/3654081191 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f359c077f70 0x7f359c07a430 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f35ac034000 tx=0x7f35ac004730 comp rx=0 tx=0).stop 2024-09-10T21:45:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:56.819+0000 7f35c0800640 1 -- 172.21.15.7:0/3654081191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35b80cf650 msgr2=0x7f35b81661b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:56.819+0000 7f35c0800640 1 --2- 172.21.15.7:0/3654081191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35b80cf650 0x7f35b81661b0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f35a800c530 tx=0x7f35a800c9f0 comp rx=0 tx=0).stop 2024-09-10T21:45:56.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:56.819+0000 7f35c0800640 1 -- 172.21.15.7:0/3654081191 shutdown_connections 2024-09-10T21:45:56.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:56.819+0000 7f35c0800640 1 --2- 172.21.15.7:0/3654081191 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f359c077f70 0x7f359c07a430 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:56.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:56.819+0000 7f35c0800640 1 --2- 172.21.15.7:0/3654081191 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35b81579b0 0x7f35b814d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:56.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:56.819+0000 7f35c0800640 1 --2- 172.21.15.7:0/3654081191 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35b80cf650 0x7f35b81661b0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:56.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:56.819+0000 7f35c0800640 1 --2- 172.21.15.7:0/3654081191 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35b80ccc30 0x7f35b814db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:56.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:56.819+0000 7f35c0800640 1 -- 172.21.15.7:0/3654081191 >> 172.21.15.7:0/3654081191 conn(0x7f35b80b1660 msgr2=0x7f35b80cc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:56.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:56.819+0000 7f35c0800640 1 -- 172.21.15.7:0/3654081191 shutdown_connections 2024-09-10T21:45:56.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:56.819+0000 7f35c0800640 1 -- 172.21.15.7:0/3654081191 wait complete. 2024-09-10T21:45:56.838 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:45:57.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.001+0000 7f793be00640 1 -- 172.21.15.7:0/1610507005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7920004020 msgr2=0x7f7920004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.001+0000 7f793be00640 1 --2- 172.21.15.7:0/1610507005 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7920004020 0x7f7920004800 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7928009b50 tx=0x7f7920005830 comp rx=0 tx=0).stop 2024-09-10T21:45:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.001+0000 7f793b400640 1 -- 172.21.15.7:0/1610507005 reap_dead start 2024-09-10T21:45:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.001+0000 7f793be00640 1 -- 172.21.15.7:0/1610507005 shutdown_connections 2024-09-10T21:45:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.001+0000 7f793be00640 1 -- 172.21.15.7:0/1610507005 >> 172.21.15.7:0/1610507005 conn(0x7f793c0d3d50 msgr2=0x7f793c0d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.001+0000 7f793be00640 1 -- 172.21.15.7:0/1610507005 shutdown_connections 2024-09-10T21:45:57.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.001+0000 7f793be00640 1 -- 172.21.15.7:0/1610507005 wait complete. 2024-09-10T21:45:57.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.002+0000 7f793be00640 1 Processor -- start 2024-09-10T21:45:57.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.003+0000 7f793be00640 1 -- start start 2024-09-10T21:45:57.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.003+0000 7f793be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7920004020 0x7f793c0b94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:57.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.003+0000 7f793be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f793c157940 0x7f793c0b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:57.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.004+0000 7f793be00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f793c157dc0 0x7f793c0b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:57.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.004+0000 7f793be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f793c0d49f0 con 0x7f7920004020 2024-09-10T21:45:57.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.004+0000 7f793be00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f793c0d4870 con 0x7f793c157940 2024-09-10T21:45:57.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.004+0000 7f793be00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f793c0d4b70 con 0x7f793c157dc0 2024-09-10T21:45:57.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.004+0000 7f793aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7920004020 0x7f793c0b94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:57.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.004+0000 7f793aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7920004020 0x7f793c0b94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60574/0 (socket says 172.21.15.7:60574) 2024-09-10T21:45:57.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.004+0000 7f793aa00640 1 -- 172.21.15.7:0/3911130651 learned_addr learned my addr 172.21.15.7:0/3911130651 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:57.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.004+0000 7f793b400640 1 --2- 172.21.15.7:0/3911130651 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f793c157dc0 0x7f793c0b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:57.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.004+0000 7f793a000640 1 --2- 172.21.15.7:0/3911130651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f793c157940 0x7f793c0b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:57.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.005+0000 7f793aa00640 1 -- 172.21.15.7:0/3911130651 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f793c157dc0 msgr2=0x7f793c0b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:57.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.005+0000 7f793aa00640 1 --2- 172.21.15.7:0/3911130651 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f793c157dc0 0x7f793c0b1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:57.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.005+0000 7f793aa00640 1 -- 172.21.15.7:0/3911130651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f793c157940 msgr2=0x7f793c0b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:57.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.005+0000 7f793aa00640 1 --2- 172.21.15.7:0/3911130651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f793c157940 0x7f793c0b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:57.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.005+0000 7f793aa00640 1 -- 172.21.15.7:0/3911130651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793c1f6f20 con 0x7f7920004020 2024-09-10T21:45:57.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.005+0000 7f793aa00640 1 --2- 172.21.15.7:0/3911130651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7920004020 0x7f793c0b94d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f792c001c20 tx=0x7f792c003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:57.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.005+0000 7f7932a00640 1 -- 172.21.15.7:0/3911130651 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c011700 con 0x7f7920004020 2024-09-10T21:45:57.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.005+0000 7f7932a00640 1 -- 172.21.15.7:0/3911130651 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f792c016020 con 0x7f7920004020 2024-09-10T21:45:57.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.005+0000 7f793be00640 1 -- 172.21.15.7:0/3911130651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f793c15aa60 con 0x7f7920004020 2024-09-10T21:45:57.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.005+0000 7f7932a00640 1 -- 172.21.15.7:0/3911130651 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c011b00 con 0x7f7920004020 2024-09-10T21:45:57.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.005+0000 7f793be00640 1 -- 172.21.15.7:0/3911130651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f793c1f8410 con 0x7f7920004020 2024-09-10T21:45:57.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.007+0000 7f793be00640 1 -- 172.21.15.7:0/3911130651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f793c0d1ee0 con 0x7f7920004020 2024-09-10T21:45:57.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.008+0000 7f7932a00640 1 -- 172.21.15.7:0/3911130651 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f792c011ca0 con 0x7f7920004020 2024-09-10T21:45:57.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.009+0000 7f7932a00640 1 --2- 172.21.15.7:0/3911130651 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f791c077f70 0x7f791c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:57.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.009+0000 7f7932a00640 1 -- 172.21.15.7:0/3911130651 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4857+0+0 (secure 0 0 0) 0x7f792c09a470 con 0x7f7920004020 2024-09-10T21:45:57.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.010+0000 7f793a000640 1 --2- 172.21.15.7:0/3911130651 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f791c077f70 0x7f791c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:57.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.011+0000 7f793a000640 1 --2- 172.21.15.7:0/3911130651 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f791c077f70 0x7f791c07a430 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7928035000 tx=0x7f79280046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:57.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.014+0000 7f7932a00640 1 -- 172.21.15.7:0/3911130651 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f792c020020 con 0x7f7920004020 2024-09-10T21:45:57.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.279+0000 7f793be00640 1 -- 172.21.15.7:0/3911130651 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f793c15d400 con 0x7f7920004020 2024-09-10T21:45:57.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.280+0000 7f7932a00640 1 -- 172.21.15.7:0/3911130651 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v34) ==== 80+0+5304 (secure 0 0 0) 0x7f792c0632b0 con 0x7f7920004020 2024-09-10T21:45:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:45:57.281 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:45:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-09-10T21:45:57.281 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-09-10T21:45:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmgRMuLBAA88QT12AXv2qf11IxU20Xbg==", 2024-09-10T21:45:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:45:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:45:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:45:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:45:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:45:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-09-10T21:45:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmRUM2BBAAwcQM6z+hjZI4ZWBlVrYMwQ==", 2024-09-10T21:45:57.282 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:45:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:45:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:45:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:45:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:45:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-09-10T21:45:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmnq31NRAAj1eJUAIS986idxppfeXEAA==", 2024-09-10T21:45:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:45:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:45:57.283 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:45:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:45:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:45:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-09-10T21:45:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmy43uDRAAzdKjLxZMGROuDABrsJ0zgQ==", 2024-09-10T21:45:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:45:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:45:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:45:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:45:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:45:57.284 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-09-10T21:45:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDwvOBmzfgtOxAAk6K5fU0Uar4VW9nUUIuNdw==", 2024-09-10T21:45:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:45:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:45:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:45:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:45:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.285 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-09-10T21:45:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD0vOBmHcRzChAA3z+CaGgu5NkyUmjpqzmKjw==", 2024-09-10T21:45:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:45:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:45:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:45:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:45:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-09-10T21:45:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD3vOBm9sArFBAAGQlyiw6qEcyumEBICWEBKA==", 2024-09-10T21:45:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:45:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:45:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:45:57.286 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:45:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-09-10T21:45:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD6vOBmkO3rHxAAW0zPydYZRGUKjGh3yIyPCA==", 2024-09-10T21:45:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:45:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:45:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:45:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:45:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-09-10T21:45:57.287 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAAveBmlPABMxAAeazVOzc1DhsVAwoHn87QIA==", 2024-09-10T21:45:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:45:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:45:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:45:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:45:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-09-10T21:45:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAEveBmL9qzARAApu8nWIY9HVsDl4yUnfhH9w==", 2024-09-10T21:45:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:45:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:45:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:45:57.288 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:45:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-09-10T21:45:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAHveBmJ4ofDRAAEiCFN8nGzVSR3u2gKoT+2Q==", 2024-09-10T21:45:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:45:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:45:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:45:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:45:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.289 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-09-10T21:45:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAKveBmoUGEGBAAhXmkUaniuAQrBiYPWr8Tog==", 2024-09-10T21:45:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:45:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:45:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:45:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:45:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-09-10T21:45:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBm0aqcFhAAtid2bnzjcpaYAbU7g0JcOA==", 2024-09-10T21:45:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:45:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:45:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-09-10T21:45:57.290 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:45:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-09-10T21:45:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmqf+MIxAAib5hVnbBsw/bJPqsQszJwg==", 2024-09-10T21:45:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:45:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:45:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:45:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:45:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:45:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:57.291 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-09-10T21:45:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmybQhHRAAZUwqnIsihEWnHZWQTYr+uQ==", 2024-09-10T21:45:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:45:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:45:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:45:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:45:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:45:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-09-10T21:45:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmDbrrKRAAUxb3EBfKIJ4FAGHW3QIsLg==", 2024-09-10T21:45:57.292 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:45:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:45:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:45:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:45:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:45:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-09-10T21:45:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDtvOBmF+eEOBAA3icvNqI0nWDShE49DZ3bvg==", 2024-09-10T21:45:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:45:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:45:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:45:57.293 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:45:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:45:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-09-10T21:45:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmsu6OJRAAhdgDvpeHY/zDKUeUOv6xxA==", 2024-09-10T21:45:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:45:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:45:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-10T21:45:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:45:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:45:57.294 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-09-10T21:45:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmKBHLHhAA0W5My19tIhN1v2qev4cBTA==", 2024-09-10T21:45:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:45:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:45:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-10T21:45:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:45:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.295 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T21:45:57.295 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:45:57.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.282+0000 7f793be00640 1 -- 172.21.15.7:0/3911130651 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f791c077f70 msgr2=0x7f791c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:57.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.282+0000 7f793be00640 1 --2- 172.21.15.7:0/3911130651 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f791c077f70 0x7f791c07a430 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f7928035000 tx=0x7f79280046c0 comp rx=0 tx=0).stop 2024-09-10T21:45:57.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.282+0000 7f793be00640 1 -- 172.21.15.7:0/3911130651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7920004020 msgr2=0x7f793c0b94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:57.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.282+0000 7f793be00640 1 --2- 172.21.15.7:0/3911130651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7920004020 0x7f793c0b94d0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f792c001c20 tx=0x7f792c003a80 comp rx=0 tx=0).stop 2024-09-10T21:45:57.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.282+0000 7f793be00640 1 -- 172.21.15.7:0/3911130651 shutdown_connections 2024-09-10T21:45:57.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.282+0000 7f793be00640 1 --2- 172.21.15.7:0/3911130651 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f791c077f70 0x7f791c07a430 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:57.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.282+0000 7f793be00640 1 --2- 172.21.15.7:0/3911130651 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f793c157940 0x7f793c0b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:57.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.282+0000 7f793be00640 1 --2- 172.21.15.7:0/3911130651 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f793c157dc0 0x7f793c0b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:57.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.282+0000 7f793be00640 1 --2- 172.21.15.7:0/3911130651 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7920004020 0x7f793c0b94d0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:57.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.282+0000 7f793be00640 1 -- 172.21.15.7:0/3911130651 >> 172.21.15.7:0/3911130651 conn(0x7f793c0d3d50 msgr2=0x7f793c0d2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:57.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.283+0000 7f793be00640 1 -- 172.21.15.7:0/3911130651 shutdown_connections 2024-09-10T21:45:57.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.283+0000 7f793be00640 1 -- 172.21.15.7:0/3911130651 wait complete. 2024-09-10T21:45:57.299 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:45:57.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.461+0000 7fb3ff400640 1 -- 172.21.15.7:0/751704542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3dc004800 msgr2=0x7fb3dc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:57.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.461+0000 7fb3ff400640 1 --2- 172.21.15.7:0/751704542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3dc004800 0x7fb3dc004c80 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb3e0009b20 tx=0x7fb3dc005830 comp rx=0 tx=0).stop 2024-09-10T21:45:57.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.461+0000 7fb3fd400640 1 -- 172.21.15.7:0/751704542 reap_dead start 2024-09-10T21:45:57.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.461+0000 7fb3ff400640 1 -- 172.21.15.7:0/751704542 shutdown_connections 2024-09-10T21:45:57.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.461+0000 7fb3ff400640 1 -- 172.21.15.7:0/751704542 >> 172.21.15.7:0/751704542 conn(0x7fb3f80b16d0 msgr2=0x7fb3f80b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:57.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.461+0000 7fb3ff400640 1 -- 172.21.15.7:0/751704542 shutdown_connections 2024-09-10T21:45:57.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.461+0000 7fb3ff400640 1 -- 172.21.15.7:0/751704542 wait complete. 2024-09-10T21:45:57.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.463+0000 7fb3ff400640 1 Processor -- start 2024-09-10T21:45:57.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.463+0000 7fb3ff400640 1 -- start start 2024-09-10T21:45:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.464+0000 7fb3ff400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3dc004800 0x7fb3f81f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.464+0000 7fb3ff400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3f81e6670 0x7fb3f81e7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.464+0000 7fb3ff400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3f80cbd10 0x7fb3f815cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.464+0000 7fb3ff400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f80cd070 con 0x7fb3f80cbd10 2024-09-10T21:45:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.464+0000 7fb3ff400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f80ccef0 con 0x7fb3f81e6670 2024-09-10T21:45:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.464+0000 7fb3ff400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f80cd1f0 con 0x7fb3dc004800 2024-09-10T21:45:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.464+0000 7fb3fd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3f80cbd10 0x7fb3f815cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.464+0000 7fb3fca00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3dc004800 0x7fb3f81f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.464+0000 7fb3fd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3f80cbd10 0x7fb3f815cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60590/0 (socket says 172.21.15.7:60590) 2024-09-10T21:45:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.464+0000 7fb3fd400640 1 -- 172.21.15.7:0/257156781 learned_addr learned my addr 172.21.15.7:0/257156781 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.464+0000 7fb3efe00640 1 --2- 172.21.15.7:0/257156781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3f81e6670 0x7fb3f81e7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.465+0000 7fb3fd400640 1 -- 172.21.15.7:0/257156781 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3dc004800 msgr2=0x7fb3f81f8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:57.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.465+0000 7fb3fd400640 1 --2- 172.21.15.7:0/257156781 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3dc004800 0x7fb3f81f8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:57.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.465+0000 7fb3fd400640 1 -- 172.21.15.7:0/257156781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3f81e6670 msgr2=0x7fb3f81e7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:57.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.465+0000 7fb3fd400640 1 --2- 172.21.15.7:0/257156781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3f81e6670 0x7fb3f81e7bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:57.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.465+0000 7fb3fd400640 1 -- 172.21.15.7:0/257156781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f80cc1c0 con 0x7fb3f80cbd10 2024-09-10T21:45:57.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.465+0000 7fb3fd400640 1 --2- 172.21.15.7:0/257156781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3f80cbd10 0x7fb3f815cc20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0005fd0 tx=0x7fb3f0011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:57.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.465+0000 7fb3f5600640 1 -- 172.21.15.7:0/257156781 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0004b20 con 0x7fb3f80cbd10 2024-09-10T21:45:57.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.465+0000 7fb3f5600640 1 -- 172.21.15.7:0/257156781 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3f0004cc0 con 0x7fb3f80cbd10 2024-09-10T21:45:57.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.465+0000 7fb3f5600640 1 -- 172.21.15.7:0/257156781 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0003bd0 con 0x7fb3f80cbd10 2024-09-10T21:45:57.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.465+0000 7fb3ff400640 1 -- 172.21.15.7:0/257156781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f81e6b50 con 0x7fb3f80cbd10 2024-09-10T21:45:57.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.465+0000 7fb3ff400640 1 -- 172.21.15.7:0/257156781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f81e7110 con 0x7fb3f80cbd10 2024-09-10T21:45:57.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.466+0000 7fb3ff400640 1 -- 172.21.15.7:0/257156781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3f81601f0 con 0x7fb3f80cbd10 2024-09-10T21:45:57.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.468+0000 7fb3f5600640 1 -- 172.21.15.7:0/257156781 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb3f0018070 con 0x7fb3f80cbd10 2024-09-10T21:45:57.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.469+0000 7fb3f5600640 1 --2- 172.21.15.7:0/257156781 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb3d4077fa0 0x7fb3d407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:57.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.469+0000 7fb3f5600640 1 -- 172.21.15.7:0/257156781 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4857+0+0 (secure 0 0 0) 0x7fb3f0099ea0 con 0x7fb3f80cbd10 2024-09-10T21:45:57.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.469+0000 7fb3fca00640 1 --2- 172.21.15.7:0/257156781 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb3d4077fa0 0x7fb3d407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:57.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.470+0000 7fb3fca00640 1 --2- 172.21.15.7:0/257156781 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb3d4077fa0 0x7fb3d407a460 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8001160 tx=0x7fb3e8006040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:57.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.473+0000 7fb3f5600640 1 -- 172.21.15.7:0/257156781 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb3f0062600 con 0x7fb3f80cbd10 2024-09-10T21:45:57.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.668+0000 7fb3ff400640 1 -- 172.21.15.7:0/257156781 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb3f8073780 con 0x7fb3f80cbd10 2024-09-10T21:45:57.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.669+0000 7fb3f5600640 1 -- 172.21.15.7:0/257156781 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v66) ==== 84+0+65 (secure 0 0 0) 0x7fb3f0067030 con 0x7fb3f80cbd10 2024-09-10T21:45:57.670 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:45:57.670 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-09-10T21:45:57.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:45:57.670 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-09-10T21:45:57.670 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-09-10T21:45:57.670 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:45:57.670 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-09-10T21:45:57.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.671+0000 7fb3ff400640 1 -- 172.21.15.7:0/257156781 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb3d4077fa0 msgr2=0x7fb3d407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:57.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.671+0000 7fb3ff400640 1 --2- 172.21.15.7:0/257156781 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb3d4077fa0 0x7fb3d407a460 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8001160 tx=0x7fb3e8006040 comp rx=0 tx=0).stop 2024-09-10T21:45:57.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.671+0000 7fb3ff400640 1 -- 172.21.15.7:0/257156781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3f80cbd10 msgr2=0x7fb3f815cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:57.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.671+0000 7fb3ff400640 1 --2- 172.21.15.7:0/257156781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3f80cbd10 0x7fb3f815cc20 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0005fd0 tx=0x7fb3f0011f40 comp rx=0 tx=0).stop 2024-09-10T21:45:57.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.671+0000 7fb3ff400640 1 -- 172.21.15.7:0/257156781 shutdown_connections 2024-09-10T21:45:57.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.671+0000 7fb3ff400640 1 --2- 172.21.15.7:0/257156781 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb3d4077fa0 0x7fb3d407a460 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.671+0000 7fb3ff400640 1 --2- 172.21.15.7:0/257156781 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb3f81e6670 0x7fb3f81e7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.671+0000 7fb3ff400640 1 --2- 172.21.15.7:0/257156781 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb3f80cbd10 0x7fb3f815cc20 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.671+0000 7fb3ff400640 1 --2- 172.21.15.7:0/257156781 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb3dc004800 0x7fb3f81f8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.671+0000 7fb3ff400640 1 -- 172.21.15.7:0/257156781 >> 172.21.15.7:0/257156781 conn(0x7fb3f80b16d0 msgr2=0x7fb3f80b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.671+0000 7fb3ff400640 1 -- 172.21.15.7:0/257156781 shutdown_connections 2024-09-10T21:45:57.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.671+0000 7fb3ff400640 1 -- 172.21.15.7:0/257156781 wait complete. 2024-09-10T21:45:57.687 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:45:57.687 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:45:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.853+0000 7fb1f0600640 1 -- 172.21.15.7:0/1067057190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1d0004d40 msgr2=0x7fb1d00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.853+0000 7fb1f0600640 1 --2- 172.21.15.7:0/1067057190 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1d0004d40 0x7fb1d00051c0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0005fd0 tx=0x7fb1d0005830 comp rx=0 tx=0).stop 2024-09-10T21:45:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.853+0000 7fb1ee600640 1 -- 172.21.15.7:0/1067057190 reap_dead start 2024-09-10T21:45:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.853+0000 7fb1f0600640 1 -- 172.21.15.7:0/1067057190 shutdown_connections 2024-09-10T21:45:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.854+0000 7fb1f0600640 1 -- 172.21.15.7:0/1067057190 >> 172.21.15.7:0/1067057190 conn(0x7fb1e80b1c10 msgr2=0x7fb1e80b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.854+0000 7fb1f0600640 1 -- 172.21.15.7:0/1067057190 shutdown_connections 2024-09-10T21:45:57.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.854+0000 7fb1f0600640 1 -- 172.21.15.7:0/1067057190 wait complete. 2024-09-10T21:45:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.855+0000 7fb1f0600640 1 Processor -- start 2024-09-10T21:45:57.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.855+0000 7fb1f0600640 1 -- start start 2024-09-10T21:45:57.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.856+0000 7fb1f0600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1d0004d40 0x7fb1e814a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.856+0000 7fb1f0600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e809ce20 0x7fb1e8156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.856+0000 7fb1f0600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e815ed20 0x7fb1e8156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.856+0000 7fb1f0600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e80705c0 con 0x7fb1e815ed20 2024-09-10T21:45:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.856+0000 7fb1f0600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb1e8070440 con 0x7fb1d0004d40 2024-09-10T21:45:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.856+0000 7fb1f0600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fb1e80aeb80 con 0x7fb1e809ce20 2024-09-10T21:45:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.856+0000 7fb1ee600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e815ed20 0x7fb1e8156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.857+0000 7fb1ee600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e815ed20 0x7fb1e8156d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60608/0 (socket says 172.21.15.7:60608) 2024-09-10T21:45:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.857+0000 7fb1ee600640 1 -- 172.21.15.7:0/3256437674 learned_addr learned my addr 172.21.15.7:0/3256437674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.857+0000 7fb1edc00640 1 --2- 172.21.15.7:0/3256437674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1d0004d40 0x7fb1e814a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.857+0000 7fb1ed200640 1 --2- 172.21.15.7:0/3256437674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e809ce20 0x7fb1e8156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.857+0000 7fb1ee600640 1 -- 172.21.15.7:0/3256437674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e809ce20 msgr2=0x7fb1e8156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.857+0000 7fb1ee600640 1 --2- 172.21.15.7:0/3256437674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e809ce20 0x7fb1e8156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:57.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.857+0000 7fb1ee600640 1 -- 172.21.15.7:0/3256437674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1d0004d40 msgr2=0x7fb1e814a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.857+0000 7fb1ee600640 1 --2- 172.21.15.7:0/3256437674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1d0004d40 0x7fb1e814a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.857+0000 7fb1ee600640 1 -- 172.21.15.7:0/3256437674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1e80c5900 con 0x7fb1e815ed20 2024-09-10T21:45:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.857+0000 7fb1ee600640 1 --2- 172.21.15.7:0/3256437674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e815ed20 0x7fb1e8156d40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0036040 tx=0x7fb1e0033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.857+0000 7fb1e5600640 1 -- 172.21.15.7:0/3256437674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0042070 con 0x7fb1e815ed20 2024-09-10T21:45:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.857+0000 7fb1e5600640 1 -- 172.21.15.7:0/3256437674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1e0003970 con 0x7fb1e815ed20 2024-09-10T21:45:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.858+0000 7fb1e5600640 1 -- 172.21.15.7:0/3256437674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0038040 con 0x7fb1e815ed20 2024-09-10T21:45:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.858+0000 7fb1f0600640 1 -- 172.21.15.7:0/3256437674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1e81f4ab0 con 0x7fb1e815ed20 2024-09-10T21:45:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.858+0000 7fb1f0600640 1 -- 172.21.15.7:0/3256437674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1e81f47b0 con 0x7fb1e815ed20 2024-09-10T21:45:57.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.859+0000 7fb1f0600640 1 -- 172.21.15.7:0/3256437674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1e81f50c0 con 0x7fb1e815ed20 2024-09-10T21:45:57.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.860+0000 7fb1e5600640 1 -- 172.21.15.7:0/3256437674 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb1e003b070 con 0x7fb1e815ed20 2024-09-10T21:45:57.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.861+0000 7fb1e5600640 1 --2- 172.21.15.7:0/3256437674 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb1c8077fa0 0x7fb1c807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:57.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.861+0000 7fb1e5600640 1 -- 172.21.15.7:0/3256437674 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 4857+0+0 (secure 0 0 0) 0x7fb1e00b97b0 con 0x7fb1e815ed20 2024-09-10T21:45:57.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.861+0000 7fb1edc00640 1 --2- 172.21.15.7:0/3256437674 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb1c8077fa0 0x7fb1c807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:57.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.862+0000 7fb1edc00640 1 --2- 172.21.15.7:0/3256437674 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb1c8077fa0 0x7fb1c807a460 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb1e814af90 tx=0x7fb1dc001090 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:57.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:57.865+0000 7fb1e5600640 1 -- 172.21.15.7:0/3256437674 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb1e0081e80 con 0x7fb1e815ed20 2024-09-10T21:45:58.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:58.059+0000 7fb1f0600640 1 -- 172.21.15.7:0/3256437674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fb1e806d050 con 0x7fb1e815ed20 2024-09-10T21:45:58.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:58.805+0000 7fb1e5600640 1 -- 172.21.15.7:0/3256437674 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v67) ==== 138+0+0 (secure 0 0 0) 0x7fb1e806d050 con 0x7fb1e815ed20 2024-09-10T21:45:58.806 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-09-10T21:45:58.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:58.808+0000 7fb1f0600640 1 -- 172.21.15.7:0/3256437674 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb1c8077fa0 msgr2=0x7fb1c807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:58.808+0000 7fb1f0600640 1 --2- 172.21.15.7:0/3256437674 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb1c8077fa0 0x7fb1c807a460 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb1e814af90 tx=0x7fb1dc001090 comp rx=0 tx=0).stop 2024-09-10T21:45:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:58.808+0000 7fb1f0600640 1 -- 172.21.15.7:0/3256437674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e815ed20 msgr2=0x7fb1e8156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:58.808+0000 7fb1f0600640 1 --2- 172.21.15.7:0/3256437674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e815ed20 0x7fb1e8156d40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0036040 tx=0x7fb1e0033ba0 comp rx=0 tx=0).stop 2024-09-10T21:45:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:58.808+0000 7fb1f0600640 1 -- 172.21.15.7:0/3256437674 shutdown_connections 2024-09-10T21:45:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:58.808+0000 7fb1f0600640 1 --2- 172.21.15.7:0/3256437674 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb1c8077fa0 0x7fb1c807a460 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:58.808+0000 7fb1f0600640 1 --2- 172.21.15.7:0/3256437674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb1e815ed20 0x7fb1e8156d40 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:58.808+0000 7fb1f0600640 1 --2- 172.21.15.7:0/3256437674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb1e809ce20 0x7fb1e8156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:58.808+0000 7fb1f0600640 1 --2- 172.21.15.7:0/3256437674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb1d0004d40 0x7fb1e814a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:58.808+0000 7fb1f0600640 1 -- 172.21.15.7:0/3256437674 >> 172.21.15.7:0/3256437674 conn(0x7fb1e80b1c10 msgr2=0x7fb1e8159dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:58.808+0000 7fb1f0600640 1 -- 172.21.15.7:0/3256437674 shutdown_connections 2024-09-10T21:45:58.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:58.809+0000 7fb1f0600640 1 -- 172.21.15.7:0/3256437674 wait complete. 2024-09-10T21:45:58.833 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:45:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.004+0000 7f7709c00640 1 -- 172.21.15.7:0/382205764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77040b4ac0 msgr2=0x7f77040b2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.004+0000 7f7709c00640 1 --2- 172.21.15.7:0/382205764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77040b4ac0 0x7f77040b2a40 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f77040706a0 tx=0x7f76ec02a540 comp rx=0 tx=0).stop 2024-09-10T21:45:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.004+0000 7f7709c00640 1 -- 172.21.15.7:0/382205764 shutdown_connections 2024-09-10T21:45:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.004+0000 7f7709c00640 1 --2- 172.21.15.7:0/382205764 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77040b4ac0 0x7f77040b2a40 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.004+0000 7f7709c00640 1 --2- 172.21.15.7:0/382205764 >> v2:172.21.15.133:3300/0 conn(0x7f7704145540 0x7f77040b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.004+0000 7f7709c00640 1 -- 172.21.15.7:0/382205764 >> 172.21.15.7:0/382205764 conn(0x7f77040b1d70 msgr2=0x7f77040cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.004+0000 7f7709c00640 1 -- 172.21.15.7:0/382205764 shutdown_connections 2024-09-10T21:45:59.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.004+0000 7f7709c00640 1 -- 172.21.15.7:0/382205764 wait complete. 2024-09-10T21:45:59.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.006+0000 7f7709c00640 1 Processor -- start 2024-09-10T21:45:59.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.006+0000 7f7709c00640 1 -- start start 2024-09-10T21:45:59.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.007+0000 7f7709c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77040b2f80 0x7f77040ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:59.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.007+0000 7f7709c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77040b4ac0 0x7f77041f5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:59.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.007+0000 7f7709c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7704145540 0x7f77041f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:59.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.007+0000 7f7709c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f77040cf700 con 0x7f77040b2f80 2024-09-10T21:45:59.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.007+0000 7f7709c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f77040cf880 con 0x7f77040b4ac0 2024-09-10T21:45:59.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.007+0000 7f7709c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f770415fee0 con 0x7f7704145540 2024-09-10T21:45:59.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.007+0000 7f7703e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77040b2f80 0x7f77040ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:59.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.007+0000 7f7703e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77040b2f80 0x7f77040ae5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60634/0 (socket says 172.21.15.7:60634) 2024-09-10T21:45:59.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.007+0000 7f7703e00640 1 -- 172.21.15.7:0/1495536629 learned_addr learned my addr 172.21.15.7:0/1495536629 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:45:59.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.008+0000 7f76fb400640 1 --2- 172.21.15.7:0/1495536629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77040b4ac0 0x7f77041f5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:59.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.008+0000 7f7709200640 1 --2- 172.21.15.7:0/1495536629 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7704145540 0x7f77041f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:59.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.008+0000 7f7703e00640 1 -- 172.21.15.7:0/1495536629 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7704145540 msgr2=0x7f77041f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:59.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.008+0000 7f7703e00640 1 --2- 172.21.15.7:0/1495536629 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7704145540 0x7f77041f62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:59.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.008+0000 7f7703e00640 1 -- 172.21.15.7:0/1495536629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77040b4ac0 msgr2=0x7f77041f5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:59.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.008+0000 7f7703e00640 1 --2- 172.21.15.7:0/1495536629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77040b4ac0 0x7f77041f5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:59.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.008+0000 7f7703e00640 1 -- 172.21.15.7:0/1495536629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7704157760 con 0x7f77040b2f80 2024-09-10T21:45:59.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.008+0000 7f7703e00640 1 --2- 172.21.15.7:0/1495536629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77040b2f80 0x7f77040ae5a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f76f400e790 tx=0x7f76f400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:59.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.009+0000 7f7700c00640 1 -- 172.21.15.7:0/1495536629 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f400eea0 con 0x7f77040b2f80 2024-09-10T21:45:59.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.009+0000 7f7700c00640 1 -- 172.21.15.7:0/1495536629 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76f400cdb0 con 0x7f77040b2f80 2024-09-10T21:45:59.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.009+0000 7f7700c00640 1 -- 172.21.15.7:0/1495536629 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76f4010690 con 0x7f77040b2f80 2024-09-10T21:45:59.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.009+0000 7f7709c00640 1 -- 172.21.15.7:0/1495536629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7704157a40 con 0x7f77040b2f80 2024-09-10T21:45:59.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.009+0000 7f7709c00640 1 -- 172.21.15.7:0/1495536629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7704158000 con 0x7f77040b2f80 2024-09-10T21:45:59.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.010+0000 7f7709c00640 1 -- 172.21.15.7:0/1495536629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7704160360 con 0x7f77040b2f80 2024-09-10T21:45:59.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.011+0000 7f7700c00640 1 -- 172.21.15.7:0/1495536629 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f76f4010830 con 0x7f77040b2f80 2024-09-10T21:45:59.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.012+0000 7f7700c00640 1 --2- 172.21.15.7:0/1495536629 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f76e0077f70 0x7f76e007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:59.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.012+0000 7f7700c00640 1 -- 172.21.15.7:0/1495536629 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 5227+0+0 (secure 0 0 0) 0x7f76f409a5a0 con 0x7f77040b2f80 2024-09-10T21:45:59.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.013+0000 7f76fb400640 1 --2- 172.21.15.7:0/1495536629 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f76e0077f70 0x7f76e007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:59.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.013+0000 7f76fb400640 1 --2- 172.21.15.7:0/1495536629 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f76e0077f70 0x7f76e007a430 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f76ec034000 tx=0x7f76ec004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:59.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.017+0000 7f7700c00640 1 -- 172.21.15.7:0/1495536629 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f76f4062b90 con 0x7f77040b2f80 2024-09-10T21:45:59.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.215+0000 7f7709c00640 1 -- 172.21.15.7:0/1495536629 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f7704141b00 con 0x7f77040b2f80 2024-09-10T21:45:59.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.808+0000 7f7700c00640 1 -- 172.21.15.7:0/1495536629 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v68) ==== 170+0+0 (secure 0 0 0) 0x7f76f40675c0 con 0x7f77040b2f80 2024-09-10T21:45:59.809 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-09-10T21:45:59.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.811+0000 7f7709c00640 1 -- 172.21.15.7:0/1495536629 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f76e0077f70 msgr2=0x7f76e007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:59.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.811+0000 7f7709c00640 1 --2- 172.21.15.7:0/1495536629 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f76e0077f70 0x7f76e007a430 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f76ec034000 tx=0x7f76ec004730 comp rx=0 tx=0).stop 2024-09-10T21:45:59.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.811+0000 7f7709c00640 1 -- 172.21.15.7:0/1495536629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77040b2f80 msgr2=0x7f77040ae5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:59.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.811+0000 7f7709c00640 1 --2- 172.21.15.7:0/1495536629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77040b2f80 0x7f77040ae5a0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f76f400e790 tx=0x7f76f400ec50 comp rx=0 tx=0).stop 2024-09-10T21:45:59.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.812+0000 7f7709c00640 1 -- 172.21.15.7:0/1495536629 shutdown_connections 2024-09-10T21:45:59.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.812+0000 7f7709c00640 1 --2- 172.21.15.7:0/1495536629 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f76e0077f70 0x7f76e007a430 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:59.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.812+0000 7f7709c00640 1 --2- 172.21.15.7:0/1495536629 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77040b4ac0 0x7f77041f5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:59.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.812+0000 7f7709c00640 1 --2- 172.21.15.7:0/1495536629 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7704145540 0x7f77041f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:59.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.812+0000 7f7709c00640 1 --2- 172.21.15.7:0/1495536629 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77040b2f80 0x7f77040ae5a0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:59.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.812+0000 7f7709c00640 1 -- 172.21.15.7:0/1495536629 >> 172.21.15.7:0/1495536629 conn(0x7f77040b1d70 msgr2=0x7f77040b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:59.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.812+0000 7f7709c00640 1 -- 172.21.15.7:0/1495536629 shutdown_connections 2024-09-10T21:45:59.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:45:59.812+0000 7f7709c00640 1 -- 172.21.15.7:0/1495536629 wait complete. 2024-09-10T21:45:59.847 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:46:00.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.012+0000 7f7b4cc00640 1 -- 172.21.15.7:0/1605859705 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b48155fb0 msgr2=0x7f7b481600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:00.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.012+0000 7f7b4cc00640 1 --2- 172.21.15.7:0/1605859705 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b48155fb0 0x7f7b481600d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7b38005fd0 tx=0x7f7b3802a5d0 comp rx=0 tx=0).stop 2024-09-10T21:46:00.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.012+0000 7f7b4cc00640 1 -- 172.21.15.7:0/1605859705 shutdown_connections 2024-09-10T21:46:00.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.012+0000 7f7b4cc00640 1 --2- 172.21.15.7:0/1605859705 >> v2:172.21.15.133:3300/0 conn(0x7f7b48074450 0x7f7b480ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:00.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.012+0000 7f7b4cc00640 1 --2- 172.21.15.7:0/1605859705 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b48155fb0 0x7f7b481600d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:00.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.012+0000 7f7b4cc00640 1 -- 172.21.15.7:0/1605859705 >> 172.21.15.7:0/1605859705 conn(0x7f7b480b16d0 msgr2=0x7f7b480b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:00.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.012+0000 7f7b4cc00640 1 -- 172.21.15.7:0/1605859705 shutdown_connections 2024-09-10T21:46:00.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.012+0000 7f7b4cc00640 1 -- 172.21.15.7:0/1605859705 wait complete. 2024-09-10T21:46:00.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.014+0000 7f7b4cc00640 1 Processor -- start 2024-09-10T21:46:00.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.014+0000 7f7b4cc00640 1 -- start start 2024-09-10T21:46:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.015+0000 7f7b4cc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b4806fcf0 0x7f7b4814cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.015+0000 7f7b4cc00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b48074450 0x7f7b4814c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:00.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.015+0000 7f7b4cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b48155fb0 0x7f7b480b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.015+0000 7f7b4cc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b480ca930 con 0x7f7b48155fb0 2024-09-10T21:46:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.015+0000 7f7b4cc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7b480ca7b0 con 0x7f7b4806fcf0 2024-09-10T21:46:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.015+0000 7f7b4cc00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f7b480caab0 con 0x7f7b48074450 2024-09-10T21:46:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.015+0000 7f7b46800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b48155fb0 0x7f7b480b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.015+0000 7f7b46800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b48155fb0 0x7f7b480b33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60642/0 (socket says 172.21.15.7:60642) 2024-09-10T21:46:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.015+0000 7f7b46800640 1 -- 172.21.15.7:0/564603695 learned_addr learned my addr 172.21.15.7:0/564603695 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:00.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.015+0000 7f7b45e00640 1 --2- 172.21.15.7:0/564603695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b4806fcf0 0x7f7b4814cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:00.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.015+0000 7f7b3d400640 1 --2- 172.21.15.7:0/564603695 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b48074450 0x7f7b4814c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:00.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.016+0000 7f7b46800640 1 -- 172.21.15.7:0/564603695 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b48074450 msgr2=0x7f7b4814c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:00.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.016+0000 7f7b46800640 1 --2- 172.21.15.7:0/564603695 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b48074450 0x7f7b4814c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:00.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.016+0000 7f7b46800640 1 -- 172.21.15.7:0/564603695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b4806fcf0 msgr2=0x7f7b4814cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:00.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.016+0000 7f7b46800640 1 --2- 172.21.15.7:0/564603695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b4806fcf0 0x7f7b4814cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:00.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.016+0000 7f7b46800640 1 -- 172.21.15.7:0/564603695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b480c6bf0 con 0x7f7b48155fb0 2024-09-10T21:46:00.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.016+0000 7f7b46800640 1 --2- 172.21.15.7:0/564603695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b48155fb0 0x7f7b480b33e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7b4000eb40 tx=0x7f7b4000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:00.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.016+0000 7f7b3ea00640 1 -- 172.21.15.7:0/564603695 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b40004b20 con 0x7f7b48155fb0 2024-09-10T21:46:00.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.016+0000 7f7b3ea00640 1 -- 172.21.15.7:0/564603695 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b40004ce0 con 0x7f7b48155fb0 2024-09-10T21:46:00.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.016+0000 7f7b3ea00640 1 -- 172.21.15.7:0/564603695 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b40005280 con 0x7f7b48155fb0 2024-09-10T21:46:00.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.016+0000 7f7b4cc00640 1 -- 172.21.15.7:0/564603695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b480c6ed0 con 0x7f7b48155fb0 2024-09-10T21:46:00.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.017+0000 7f7b4cc00640 1 -- 172.21.15.7:0/564603695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b480c9920 con 0x7f7b48155fb0 2024-09-10T21:46:00.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.018+0000 7f7b4cc00640 1 -- 172.21.15.7:0/564603695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b480cafd0 con 0x7f7b48155fb0 2024-09-10T21:46:00.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.019+0000 7f7b3ea00640 1 -- 172.21.15.7:0/564603695 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7b40005420 con 0x7f7b48155fb0 2024-09-10T21:46:00.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.020+0000 7f7b3ea00640 1 --2- 172.21.15.7:0/564603695 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b24077f70 0x7f7b2407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:00.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.020+0000 7f7b3ea00640 1 -- 172.21.15.7:0/564603695 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 5609+0+0 (secure 0 0 0) 0x7f7b4009a8e0 con 0x7f7b48155fb0 2024-09-10T21:46:00.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.020+0000 7f7b45e00640 1 --2- 172.21.15.7:0/564603695 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b24077f70 0x7f7b2407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:00.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.021+0000 7f7b45e00640 1 --2- 172.21.15.7:0/564603695 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b24077f70 0x7f7b2407a430 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7b38033000 tx=0x7f7b38004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.025+0000 7f7b3ea00640 1 -- 172.21.15.7:0/564603695 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7b40062de0 con 0x7f7b48155fb0 2024-09-10T21:46:00.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.259+0000 7f7b4cc00640 1 -- 172.21.15.7:0/564603695 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7b48070550 con 0x7f7b48155fb0 2024-09-10T21:46:00.380 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:46:00.385 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:46:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.832+0000 7f7b3ea00640 1 -- 172.21.15.7:0/564603695 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v42) ==== 174+0+0 (secure 0 0 0) 0x7f7b40067810 con 0x7f7b48155fb0 2024-09-10T21:46:00.834 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 10 and data pool 11 2024-09-10T21:46:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.841+0000 7f7b4cc00640 1 -- 172.21.15.7:0/564603695 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b24077f70 msgr2=0x7f7b2407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.841+0000 7f7b4cc00640 1 --2- 172.21.15.7:0/564603695 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b24077f70 0x7f7b2407a430 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7b38033000 tx=0x7f7b38004730 comp rx=0 tx=0).stop 2024-09-10T21:46:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.841+0000 7f7b4cc00640 1 -- 172.21.15.7:0/564603695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b48155fb0 msgr2=0x7f7b480b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:00.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.841+0000 7f7b4cc00640 1 --2- 172.21.15.7:0/564603695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b48155fb0 0x7f7b480b33e0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f7b4000eb40 tx=0x7f7b4000c2d0 comp rx=0 tx=0).stop 2024-09-10T21:46:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.841+0000 7f7b4cc00640 1 -- 172.21.15.7:0/564603695 shutdown_connections 2024-09-10T21:46:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.841+0000 7f7b4cc00640 1 --2- 172.21.15.7:0/564603695 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b24077f70 0x7f7b2407a430 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.841+0000 7f7b4cc00640 1 --2- 172.21.15.7:0/564603695 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b48155fb0 0x7f7b480b33e0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.841+0000 7f7b4cc00640 1 --2- 172.21.15.7:0/564603695 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b48074450 0x7f7b4814c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.841+0000 7f7b4cc00640 1 --2- 172.21.15.7:0/564603695 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b4806fcf0 0x7f7b4814cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.841+0000 7f7b4cc00640 1 -- 172.21.15.7:0/564603695 >> 172.21.15.7:0/564603695 conn(0x7f7b480b16d0 msgr2=0x7f7b4815e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.841+0000 7f7b4cc00640 1 -- 172.21.15.7:0/564603695 shutdown_connections 2024-09-10T21:46:00.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:00.842+0000 7f7b4cc00640 1 -- 172.21.15.7:0/564603695 wait complete. 2024-09-10T21:46:00.872 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:46:01.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.032+0000 7f2c1e800640 1 -- 172.21.15.7:0/721159278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c180b4b00 msgr2=0x7f2c180b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:01.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.032+0000 7f2c1e800640 1 --2- 172.21.15.7:0/721159278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c180b4b00 0x7f2c180b4f90 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f2c1815cc60 tx=0x7f2c0402a510 comp rx=0 tx=0).stop 2024-09-10T21:46:01.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.032+0000 7f2c1e800640 1 -- 172.21.15.7:0/721159278 shutdown_connections 2024-09-10T21:46:01.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.032+0000 7f2c1e800640 1 --2- 172.21.15.7:0/721159278 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c1815c150 0x7f2c1815c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.032+0000 7f2c1e800640 1 --2- 172.21.15.7:0/721159278 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c180b4b00 0x7f2c180b4f90 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.032+0000 7f2c1e800640 1 -- 172.21.15.7:0/721159278 >> 172.21.15.7:0/721159278 conn(0x7f2c1806d400 msgr2=0x7f2c180b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:01.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.032+0000 7f2c1e800640 1 -- 172.21.15.7:0/721159278 shutdown_connections 2024-09-10T21:46:01.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.032+0000 7f2c1e800640 1 -- 172.21.15.7:0/721159278 wait complete. 2024-09-10T21:46:01.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.033+0000 7f2c1e800640 1 Processor -- start 2024-09-10T21:46:01.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.034+0000 7f2c1e800640 1 -- start start 2024-09-10T21:46:01.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.035+0000 7f2c1e800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c180b4b00 0x7f2c18155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:01.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.035+0000 7f2c1e800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c180d0430 0x7f2c181f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:01.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.035+0000 7f2c1e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c1815c150 0x7f2c181f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:01.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.035+0000 7f2c1e800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2c180b30e0 con 0x7f2c1815c150 2024-09-10T21:46:01.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.035+0000 7f2c1e800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2c180b2f60 con 0x7f2c180b4b00 2024-09-10T21:46:01.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.035+0000 7f2c1e800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f2c180b3260 con 0x7f2c180d0430 2024-09-10T21:46:01.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.035+0000 7f2c17e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c1815c150 0x7f2c181f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:01.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.035+0000 7f2c17e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c1815c150 0x7f2c181f4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60656/0 (socket says 172.21.15.7:60656) 2024-09-10T21:46:01.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.035+0000 7f2c17e00640 1 -- 172.21.15.7:0/1684181982 learned_addr learned my addr 172.21.15.7:0/1684181982 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:01.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.035+0000 7f2c16a00640 1 --2- 172.21.15.7:0/1684181982 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c180d0430 0x7f2c181f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.035+0000 7f2c17400640 1 --2- 172.21.15.7:0/1684181982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c180b4b00 0x7f2c18155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.035+0000 7f2c17e00640 1 -- 172.21.15.7:0/1684181982 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c180d0430 msgr2=0x7f2c181f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.035+0000 7f2c17e00640 1 --2- 172.21.15.7:0/1684181982 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c180d0430 0x7f2c181f7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.035+0000 7f2c17e00640 1 -- 172.21.15.7:0/1684181982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c180b4b00 msgr2=0x7f2c18155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.035+0000 7f2c17e00640 1 --2- 172.21.15.7:0/1684181982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c180b4b00 0x7f2c18155f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.035+0000 7f2c17e00640 1 -- 172.21.15.7:0/1684181982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c181f80b0 con 0x7f2c1815c150 2024-09-10T21:46:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.035+0000 7f2c17400640 1 --2- 172.21.15.7:0/1684181982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c180b4b00 0x7f2c18155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.036+0000 7f2c17e00640 1 --2- 172.21.15.7:0/1684181982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c1815c150 0x7f2c181f4dd0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000dac0 tx=0x7f2c1000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.036+0000 7f2c0f400640 1 -- 172.21.15.7:0/1684181982 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10017070 con 0x7f2c1815c150 2024-09-10T21:46:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.036+0000 7f2c1e800640 1 -- 172.21.15.7:0/1684181982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c181f8350 con 0x7f2c1815c150 2024-09-10T21:46:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.036+0000 7f2c0f400640 1 -- 172.21.15.7:0/1684181982 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c10003970 con 0x7f2c1815c150 2024-09-10T21:46:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.036+0000 7f2c0f400640 1 -- 172.21.15.7:0/1684181982 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c10003dc0 con 0x7f2c1815c150 2024-09-10T21:46:01.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.036+0000 7f2c1e800640 1 -- 172.21.15.7:0/1684181982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c181f8910 con 0x7f2c1815c150 2024-09-10T21:46:01.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.037+0000 7f2c1e800640 1 -- 172.21.15.7:0/1684181982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c180b4f90 con 0x7f2c1815c150 2024-09-10T21:46:01.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.038+0000 7f2c0f400640 1 -- 172.21.15.7:0/1684181982 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2c10003f60 con 0x7f2c1815c150 2024-09-10T21:46:01.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.039+0000 7f2c0f400640 1 --2- 172.21.15.7:0/1684181982 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2bf8077f70 0x7f2bf807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:01.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.039+0000 7f2c0f400640 1 -- 172.21.15.7:0/1684181982 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f2c1009a200 con 0x7f2c1815c150 2024-09-10T21:46:01.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.040+0000 7f2c17400640 1 --2- 172.21.15.7:0/1684181982 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2bf8077f70 0x7f2bf807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:01.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.040+0000 7f2c17400640 1 --2- 172.21.15.7:0/1684181982 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2bf8077f70 0x7f2bf807a430 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2c08007a50 tx=0x7f2c08009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:01.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.043+0000 7f2c0f400640 1 -- 172.21.15.7:0/1684181982 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2c10062590 con 0x7f2c1815c150 2024-09-10T21:46:01.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.237+0000 7f2c1e800640 1 -- 172.21.15.7:0/1684181982 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2c1815f260 con 0x7f2c1815c150 2024-09-10T21:46:01.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.238+0000 7f2c0f400640 1 -- 172.21.15.7:0/1684181982 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16705 (secure 0 0 0) 0x7f2c10066fc0 con 0x7f2c1815c150 2024-09-10T21:46:01.240 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:46:01.240 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":69,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:46:00.261012+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:45:58.069464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T21:45:59.225636+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:46:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.240+0000 7f2c1e800640 1 -- 172.21.15.7:0/1684181982 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2bf8077f70 msgr2=0x7f2bf807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.240+0000 7f2c1e800640 1 --2- 172.21.15.7:0/1684181982 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2bf8077f70 0x7f2bf807a430 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f2c08007a50 tx=0x7f2c08009040 comp rx=0 tx=0).stop 2024-09-10T21:46:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.240+0000 7f2c1e800640 1 -- 172.21.15.7:0/1684181982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c1815c150 msgr2=0x7f2c181f4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.240+0000 7f2c1e800640 1 --2- 172.21.15.7:0/1684181982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c1815c150 0x7f2c181f4dd0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f2c1000dac0 tx=0x7f2c1000df80 comp rx=0 tx=0).stop 2024-09-10T21:46:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.241+0000 7f2c1e800640 1 -- 172.21.15.7:0/1684181982 shutdown_connections 2024-09-10T21:46:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.241+0000 7f2c1e800640 1 --2- 172.21.15.7:0/1684181982 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2bf8077f70 0x7f2bf807a430 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.241+0000 7f2c1e800640 1 --2- 172.21.15.7:0/1684181982 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2c1815c150 0x7f2c181f4dd0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.241+0000 7f2c1e800640 1 --2- 172.21.15.7:0/1684181982 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2c180d0430 0x7f2c181f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.241+0000 7f2c1e800640 1 --2- 172.21.15.7:0/1684181982 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2c180b4b00 0x7f2c18155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.241+0000 7f2c1e800640 1 -- 172.21.15.7:0/1684181982 >> 172.21.15.7:0/1684181982 conn(0x7f2c1806d400 msgr2=0x7f2c18070880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.241+0000 7f2c1e800640 1 -- 172.21.15.7:0/1684181982 shutdown_connections 2024-09-10T21:46:01.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.241+0000 7f2c1e800640 1 -- 172.21.15.7:0/1684181982 wait complete. 2024-09-10T21:46:01.258 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:46:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.417+0000 7f64f2600640 1 -- 172.21.15.7:0/689550743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64ec0cd750 msgr2=0x7f64ec0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.417+0000 7f64f2600640 1 --2- 172.21.15.7:0/689550743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64ec0cd750 0x7f64ec0ccef0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f64ec15fa90 tx=0x7f64d402a470 comp rx=0 tx=0).stop 2024-09-10T21:46:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.417+0000 7f64f2600640 1 -- 172.21.15.7:0/689550743 shutdown_connections 2024-09-10T21:46:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.417+0000 7f64f2600640 1 --2- 172.21.15.7:0/689550743 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64ec15fda0 0x7f64ec1601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.417+0000 7f64f2600640 1 --2- 172.21.15.7:0/689550743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64ec0cd750 0x7f64ec0ccef0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.417+0000 7f64f2600640 1 -- 172.21.15.7:0/689550743 >> 172.21.15.7:0/689550743 conn(0x7f64ec0b16d0 msgr2=0x7f64ec0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.417+0000 7f64f2600640 1 -- 172.21.15.7:0/689550743 shutdown_connections 2024-09-10T21:46:01.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.417+0000 7f64f2600640 1 -- 172.21.15.7:0/689550743 wait complete. 2024-09-10T21:46:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.419+0000 7f64f2600640 1 Processor -- start 2024-09-10T21:46:01.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.419+0000 7f64f2600640 1 -- start start 2024-09-10T21:46:01.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.420+0000 7f64f2600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64ec0cd750 0x7f64ec15e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:01.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.420+0000 7f64f2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64ec15bc90 0x7f64ec0c4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:01.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.420+0000 7f64f2600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64ec15fda0 0x7f64ec0c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:01.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.420+0000 7f64f2600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec06fe70 con 0x7f64ec15bc90 2024-09-10T21:46:01.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.420+0000 7f64f2600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec06fcf0 con 0x7f64ec15fda0 2024-09-10T21:46:01.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.420+0000 7f64f2600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f64ec06fff0 con 0x7f64ec0cd750 2024-09-10T21:46:01.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.420+0000 7f64e2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64ec15bc90 0x7f64ec0c4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:01.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.420+0000 7f64e2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64ec15bc90 0x7f64ec0c4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60660/0 (socket says 172.21.15.7:60660) 2024-09-10T21:46:01.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.420+0000 7f64e2a00640 1 -- 172.21.15.7:0/1307202397 learned_addr learned my addr 172.21.15.7:0/1307202397 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:01.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.420+0000 7f64eb400640 1 --2- 172.21.15.7:0/1307202397 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64ec0cd750 0x7f64ec15e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:01.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.420+0000 7f64ebe00640 1 --2- 172.21.15.7:0/1307202397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64ec15fda0 0x7f64ec0c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:01.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.421+0000 7f64e2a00640 1 -- 172.21.15.7:0/1307202397 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64ec0cd750 msgr2=0x7f64ec15e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:01.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.421+0000 7f64e2a00640 1 --2- 172.21.15.7:0/1307202397 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64ec0cd750 0x7f64ec15e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.421+0000 7f64e2a00640 1 -- 172.21.15.7:0/1307202397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64ec15fda0 msgr2=0x7f64ec0c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:01.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.421+0000 7f64e2a00640 1 --2- 172.21.15.7:0/1307202397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64ec15fda0 0x7f64ec0c4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.421+0000 7f64e2a00640 1 -- 172.21.15.7:0/1307202397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ec1f88a0 con 0x7f64ec15bc90 2024-09-10T21:46:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.421+0000 7f64e2a00640 1 --2- 172.21.15.7:0/1307202397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64ec15bc90 0x7f64ec0c4700 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009780 tx=0x7f64d4032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.421+0000 7f64e3e00640 1 -- 172.21.15.7:0/1307202397 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4040070 con 0x7f64ec15bc90 2024-09-10T21:46:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.421+0000 7f64e3e00640 1 -- 172.21.15.7:0/1307202397 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64d4005310 con 0x7f64ec15bc90 2024-09-10T21:46:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.421+0000 7f64f2600640 1 -- 172.21.15.7:0/1307202397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ec15dc90 con 0x7f64ec15bc90 2024-09-10T21:46:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.421+0000 7f64e3e00640 1 -- 172.21.15.7:0/1307202397 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64d4003800 con 0x7f64ec15bc90 2024-09-10T21:46:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.422+0000 7f64f2600640 1 -- 172.21.15.7:0/1307202397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ec1f85c0 con 0x7f64ec15bc90 2024-09-10T21:46:01.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.423+0000 7f64f2600640 1 -- 172.21.15.7:0/1307202397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64ec0704c0 con 0x7f64ec15bc90 2024-09-10T21:46:01.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.424+0000 7f64e3e00640 1 -- 172.21.15.7:0/1307202397 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f64d404c030 con 0x7f64ec15bc90 2024-09-10T21:46:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.425+0000 7f64e3e00640 1 --2- 172.21.15.7:0/1307202397 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f64c8077f70 0x7f64c807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:01.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.426+0000 7f64e3e00640 1 -- 172.21.15.7:0/1307202397 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f64d40baf70 con 0x7f64ec15bc90 2024-09-10T21:46:01.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.426+0000 7f64eb400640 1 --2- 172.21.15.7:0/1307202397 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f64c8077f70 0x7f64c807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:01.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.426+0000 7f64eb400640 1 --2- 172.21.15.7:0/1307202397 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f64c8077f70 0x7f64c807a430 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0017c0 tx=0x7f64dc007e90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:01.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.430+0000 7f64e3e00640 1 -- 172.21.15.7:0/1307202397 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f64d4083410 con 0x7f64ec15bc90 2024-09-10T21:46:01.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.624+0000 7f64f2600640 1 -- 172.21.15.7:0/1307202397 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f64ec069810 con 0x7f64ec15bc90 2024-09-10T21:46:01.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.625+0000 7f64e3e00640 1 -- 172.21.15.7:0/1307202397 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) ==== 74+0+16705 (secure 0 0 0) 0x7f64d4087e40 con 0x7f64ec15bc90 2024-09-10T21:46:01.627 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:46:01.627 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":69,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:46:00.261012+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:45:58.069464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T21:45:59.225636+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:46:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.628+0000 7f64f2600640 1 -- 172.21.15.7:0/1307202397 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f64c8077f70 msgr2=0x7f64c807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.628+0000 7f64f2600640 1 --2- 172.21.15.7:0/1307202397 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f64c8077f70 0x7f64c807a430 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f64dc0017c0 tx=0x7f64dc007e90 comp rx=0 tx=0).stop 2024-09-10T21:46:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.628+0000 7f64f2600640 1 -- 172.21.15.7:0/1307202397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64ec15bc90 msgr2=0x7f64ec0c4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:01.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.628+0000 7f64f2600640 1 --2- 172.21.15.7:0/1307202397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64ec15bc90 0x7f64ec0c4700 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009780 tx=0x7f64d4032d50 comp rx=0 tx=0).stop 2024-09-10T21:46:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.628+0000 7f64f2600640 1 -- 172.21.15.7:0/1307202397 shutdown_connections 2024-09-10T21:46:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.628+0000 7f64f2600640 1 --2- 172.21.15.7:0/1307202397 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f64c8077f70 0x7f64c807a430 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.628+0000 7f64f2600640 1 --2- 172.21.15.7:0/1307202397 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f64ec15fda0 0x7f64ec0c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.628+0000 7f64f2600640 1 --2- 172.21.15.7:0/1307202397 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f64ec15bc90 0x7f64ec0c4700 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.628+0000 7f64f2600640 1 --2- 172.21.15.7:0/1307202397 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f64ec0cd750 0x7f64ec15e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.628+0000 7f64f2600640 1 -- 172.21.15.7:0/1307202397 >> 172.21.15.7:0/1307202397 conn(0x7f64ec0b16d0 msgr2=0x7f64ec0b2b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.628+0000 7f64f2600640 1 -- 172.21.15.7:0/1307202397 shutdown_connections 2024-09-10T21:46:01.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.628+0000 7f64f2600640 1 -- 172.21.15.7:0/1307202397 wait complete. 2024-09-10T21:46:01.646 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:46:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.809+0000 7f905c000640 1 -- 172.21.15.7:0/81196819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90540affb0 msgr2=0x7f90540cb830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.809+0000 7f905c000640 1 --2- 172.21.15.7:0/81196819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90540affb0 0x7f90540cb830 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f9054073d20 tx=0x7f904c02aa30 comp rx=0 tx=0).stop 2024-09-10T21:46:01.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.809+0000 7f905c000640 1 -- 172.21.15.7:0/81196819 shutdown_connections 2024-09-10T21:46:01.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.809+0000 7f905c000640 1 --2- 172.21.15.7:0/81196819 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90540affb0 0x7f90540cb830 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.809+0000 7f905c000640 1 --2- 172.21.15.7:0/81196819 >> v2:172.21.15.133:3300/0 conn(0x7f90540cc380 0x7f90540afa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.809+0000 7f905c000640 1 --2- 172.21.15.7:0/81196819 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f905415aba0 0x7f90540cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.809+0000 7f905c000640 1 -- 172.21.15.7:0/81196819 >> 172.21.15.7:0/81196819 conn(0x7f90540b1f00 msgr2=0x7f90540b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:01.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.809+0000 7f905c000640 1 -- 172.21.15.7:0/81196819 shutdown_connections 2024-09-10T21:46:01.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.809+0000 7f905c000640 1 -- 172.21.15.7:0/81196819 wait complete. 2024-09-10T21:46:01.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.810+0000 7f905c000640 1 Processor -- start 2024-09-10T21:46:01.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.811+0000 7f905c000640 1 -- start start 2024-09-10T21:46:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.812+0000 7f905c000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90540affb0 0x7f90540af6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.812+0000 7f905c000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90540cc380 0x7f90540c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.812+0000 7f905c000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f905415aba0 0x7f9054158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.812+0000 7f905c000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f90540cc100 con 0x7f90540cc380 2024-09-10T21:46:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.812+0000 7f905c000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f90540cbf80 con 0x7f90540affb0 2024-09-10T21:46:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.812+0000 7f905c000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f90540741b0 con 0x7f905415aba0 2024-09-10T21:46:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.812+0000 7f9058c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90540cc380 0x7f90540c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:01.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.812+0000 7f9058c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90540cc380 0x7f90540c5ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60688/0 (socket says 172.21.15.7:60688) 2024-09-10T21:46:01.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.812+0000 7f9058c00640 1 -- 172.21.15.7:0/103239983 learned_addr learned my addr 172.21.15.7:0/103239983 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.812+0000 7f9059600640 1 --2- 172.21.15.7:0/103239983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90540affb0 0x7f90540af6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.812+0000 7f905a000640 1 --2- 172.21.15.7:0/103239983 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f905415aba0 0x7f9054158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.812+0000 7f9058c00640 1 -- 172.21.15.7:0/103239983 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f905415aba0 msgr2=0x7f9054158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.812+0000 7f9058c00640 1 --2- 172.21.15.7:0/103239983 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f905415aba0 0x7f9054158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.812+0000 7f9058c00640 1 -- 172.21.15.7:0/103239983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90540affb0 msgr2=0x7f90540af6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.812+0000 7f9058c00640 1 --2- 172.21.15.7:0/103239983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90540affb0 0x7f90540af6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.812+0000 7f9058c00640 1 -- 172.21.15.7:0/103239983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f905414ddc0 con 0x7f90540cc380 2024-09-10T21:46:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.812+0000 7f905a000640 1 --2- 172.21.15.7:0/103239983 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f905415aba0 0x7f9054158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.812+0000 7f9059600640 1 --2- 172.21.15.7:0/103239983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90540affb0 0x7f90540af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.813+0000 7f9058c00640 1 --2- 172.21.15.7:0/103239983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90540cc380 0x7f90540c5ca0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f904000d720 tx=0x7f904000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.813+0000 7f9051600640 1 -- 172.21.15.7:0/103239983 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040017070 con 0x7f90540cc380 2024-09-10T21:46:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.813+0000 7f9051600640 1 -- 172.21.15.7:0/103239983 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9040009d20 con 0x7f90540cc380 2024-09-10T21:46:01.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.813+0000 7f9051600640 1 -- 172.21.15.7:0/103239983 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9040004280 con 0x7f90540cc380 2024-09-10T21:46:01.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.813+0000 7f905c000640 1 -- 172.21.15.7:0/103239983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f905414e060 con 0x7f90540cc380 2024-09-10T21:46:01.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.813+0000 7f905c000640 1 -- 172.21.15.7:0/103239983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f905414e300 con 0x7f90540cc380 2024-09-10T21:46:01.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.814+0000 7f905c000640 1 -- 172.21.15.7:0/103239983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90540cb830 con 0x7f90540cc380 2024-09-10T21:46:01.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.815+0000 7f9051600640 1 -- 172.21.15.7:0/103239983 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9040005020 con 0x7f90540cc380 2024-09-10T21:46:01.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.816+0000 7f9051600640 1 --2- 172.21.15.7:0/103239983 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9034077f70 0x7f903407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:01.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.816+0000 7f9051600640 1 -- 172.21.15.7:0/103239983 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 5709+0+0 (secure 0 0 0) 0x7f904009a170 con 0x7f90540cc380 2024-09-10T21:46:01.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.817+0000 7f9059600640 1 --2- 172.21.15.7:0/103239983 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9034077f70 0x7f903407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:01.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.817+0000 7f9059600640 1 --2- 172.21.15.7:0/103239983 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9034077f70 0x7f903407a430 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9048005fd0 tx=0x7f9048005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:01.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:01.820+0000 7f9051600640 1 -- 172.21.15.7:0/103239983 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9040020020 con 0x7f90540cc380 2024-09-10T21:46:02.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:02.056+0000 7f905c000640 1 -- 172.21.15.7:0/103239983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f90540cd620 con 0x7f90540cc380 2024-09-10T21:46:02.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:02.853+0000 7f9051600640 1 -- 172.21.15.7:0/103239983 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) ==== 118+0+0 (secure 0 0 0) 0x7f9040062c60 con 0x7f90540cc380 2024-09-10T21:46:02.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:02.856+0000 7f905c000640 1 -- 172.21.15.7:0/103239983 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9034077f70 msgr2=0x7f903407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:02.856+0000 7f905c000640 1 --2- 172.21.15.7:0/103239983 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9034077f70 0x7f903407a430 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f9048005fd0 tx=0x7f9048005850 comp rx=0 tx=0).stop 2024-09-10T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:02.856+0000 7f905c000640 1 -- 172.21.15.7:0/103239983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90540cc380 msgr2=0x7f90540c5ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:02.856+0000 7f905c000640 1 --2- 172.21.15.7:0/103239983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90540cc380 0x7f90540c5ca0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f904000d720 tx=0x7f904000dbe0 comp rx=0 tx=0).stop 2024-09-10T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:02.856+0000 7f905c000640 1 -- 172.21.15.7:0/103239983 shutdown_connections 2024-09-10T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:02.856+0000 7f905c000640 1 --2- 172.21.15.7:0/103239983 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9034077f70 0x7f903407a430 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:02.856+0000 7f905c000640 1 --2- 172.21.15.7:0/103239983 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f905415aba0 0x7f9054158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:02.856+0000 7f905c000640 1 --2- 172.21.15.7:0/103239983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f90540cc380 0x7f90540c5ca0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:02.856+0000 7f905c000640 1 --2- 172.21.15.7:0/103239983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f90540affb0 0x7f90540af6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:02.856+0000 7f905c000640 1 -- 172.21.15.7:0/103239983 >> 172.21.15.7:0/103239983 conn(0x7f90540b1f00 msgr2=0x7f90540b0e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:02.857+0000 7f905c000640 1 -- 172.21.15.7:0/103239983 shutdown_connections 2024-09-10T21:46:02.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:02.857+0000 7f905c000640 1 -- 172.21.15.7:0/103239983 wait complete. 2024-09-10T21:46:02.877 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:46:02.877 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:46:03.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.044+0000 7f3ad7c00640 1 -- 172.21.15.7:0/3463275176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad0159680 msgr2=0x7f3ad015cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.044+0000 7f3ad7c00640 1 --2- 172.21.15.7:0/3463275176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad0159680 0x7f3ad015cc00 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00c3620 tx=0x7f3ac002a4a0 comp rx=0 tx=0).stop 2024-09-10T21:46:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.045+0000 7f3ad7c00640 1 -- 172.21.15.7:0/3463275176 shutdown_connections 2024-09-10T21:46:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.045+0000 7f3ad7c00640 1 --2- 172.21.15.7:0/3463275176 >> v2:172.21.15.133:3300/0 conn(0x7f3ad0073e20 0x7f3ad00742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.045+0000 7f3ad7c00640 1 --2- 172.21.15.7:0/3463275176 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad0159680 0x7f3ad015cc00 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.045+0000 7f3ad7c00640 1 -- 172.21.15.7:0/3463275176 >> 172.21.15.7:0/3463275176 conn(0x7f3ad00b16d0 msgr2=0x7f3ad00b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.045+0000 7f3ad7c00640 1 -- 172.21.15.7:0/3463275176 shutdown_connections 2024-09-10T21:46:03.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.045+0000 7f3ad7c00640 1 -- 172.21.15.7:0/3463275176 wait complete. 2024-09-10T21:46:03.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.046+0000 7f3ad7c00640 1 Processor -- start 2024-09-10T21:46:03.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.046+0000 7f3ad7c00640 1 -- start start 2024-09-10T21:46:03.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.047+0000 7f3ad7c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ad006e580 0x7f3ad014bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:03.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.047+0000 7f3ad7c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad0073e20 0x7f3ad00cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:03.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.047+0000 7f3ad7c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ad0159680 0x7f3ad01f51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:03.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.047+0000 7f3ad7c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad00ca990 con 0x7f3ad0073e20 2024-09-10T21:46:03.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.047+0000 7f3ad7c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3ad00ca810 con 0x7f3ad006e580 2024-09-10T21:46:03.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.047+0000 7f3ad7c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f3ad00cab10 con 0x7f3ad0159680 2024-09-10T21:46:03.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.047+0000 7f3ac7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad0073e20 0x7f3ad00cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:03.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.048+0000 7f3ac7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad0073e20 0x7f3ad00cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60710/0 (socket says 172.21.15.7:60710) 2024-09-10T21:46:03.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.048+0000 7f3ac7e00640 1 -- 172.21.15.7:0/3555131666 learned_addr learned my addr 172.21.15.7:0/3555131666 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.048+0000 7f3ad5200640 1 --2- 172.21.15.7:0/3555131666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ad006e580 0x7f3ad014bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.048+0000 7f3ad5c00640 1 --2- 172.21.15.7:0/3555131666 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ad0159680 0x7f3ad01f51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.048+0000 7f3ac7e00640 1 -- 172.21.15.7:0/3555131666 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ad0159680 msgr2=0x7f3ad01f51d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.048+0000 7f3ac7e00640 1 --2- 172.21.15.7:0/3555131666 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ad0159680 0x7f3ad01f51d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.048+0000 7f3ac7e00640 1 -- 172.21.15.7:0/3555131666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ad006e580 msgr2=0x7f3ad014bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.048+0000 7f3ac7e00640 1 --2- 172.21.15.7:0/3555131666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ad006e580 0x7f3ad014bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.048+0000 7f3ac7e00640 1 -- 172.21.15.7:0/3555131666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ad01f4a40 con 0x7f3ad0073e20 2024-09-10T21:46:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.048+0000 7f3ac7e00640 1 --2- 172.21.15.7:0/3555131666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad0073e20 0x7f3ad00cd6f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800c730 tx=0x7f3ab800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.049+0000 7f3acd600640 1 -- 172.21.15.7:0/3555131666 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab8017070 con 0x7f3ad0073e20 2024-09-10T21:46:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.049+0000 7f3acd600640 1 -- 172.21.15.7:0/3555131666 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ab8005160 con 0x7f3ad0073e20 2024-09-10T21:46:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.049+0000 7f3acd600640 1 -- 172.21.15.7:0/3555131666 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ab800a640 con 0x7f3ad0073e20 2024-09-10T21:46:03.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.049+0000 7f3ad7c00640 1 -- 172.21.15.7:0/3555131666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ad014c220 con 0x7f3ad0073e20 2024-09-10T21:46:03.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.049+0000 7f3ad7c00640 1 -- 172.21.15.7:0/3555131666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ad00ce1e0 con 0x7f3ad0073e20 2024-09-10T21:46:03.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.050+0000 7f3ad7c00640 1 -- 172.21.15.7:0/3555131666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ad00cb7f0 con 0x7f3ad0073e20 2024-09-10T21:46:03.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.051+0000 7f3acd600640 1 -- 172.21.15.7:0/3555131666 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3ab800a7e0 con 0x7f3ad0073e20 2024-09-10T21:46:03.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.052+0000 7f3acd600640 1 --2- 172.21.15.7:0/3555131666 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3aac077f70 0x7f3aac07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:03.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.052+0000 7f3acd600640 1 -- 172.21.15.7:0/3555131666 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f3ab8099390 con 0x7f3ad0073e20 2024-09-10T21:46:03.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.052+0000 7f3ad5200640 1 --2- 172.21.15.7:0/3555131666 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3aac077f70 0x7f3aac07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:03.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.053+0000 7f3ad5200640 1 --2- 172.21.15.7:0/3555131666 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3aac077f70 0x7f3aac07a430 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0034000 tx=0x7f3ac0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:03.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.056+0000 7f3acd600640 1 -- 172.21.15.7:0/3555131666 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3ad0072830 con 0x7f3ad0073e20 2024-09-10T21:46:03.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.288+0000 7f3ad7c00640 1 -- 172.21.15.7:0/3555131666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3ad015b5c0 con 0x7f3ad0073e20 2024-09-10T21:46:03.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.854+0000 7f3acd600640 1 -- 172.21.15.7:0/3555131666 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7f3ab8061830 con 0x7f3ad0073e20 2024-09-10T21:46:03.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.857+0000 7f3ad7c00640 1 -- 172.21.15.7:0/3555131666 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3aac077f70 msgr2=0x7f3aac07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:03.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.857+0000 7f3ad7c00640 1 --2- 172.21.15.7:0/3555131666 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3aac077f70 0x7f3aac07a430 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0034000 tx=0x7f3ac0004730 comp rx=0 tx=0).stop 2024-09-10T21:46:03.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.857+0000 7f3ad7c00640 1 -- 172.21.15.7:0/3555131666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad0073e20 msgr2=0x7f3ad00cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:03.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.857+0000 7f3ad7c00640 1 --2- 172.21.15.7:0/3555131666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad0073e20 0x7f3ad00cd6f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f3ab800c730 tx=0x7f3ab800cbf0 comp rx=0 tx=0).stop 2024-09-10T21:46:03.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.857+0000 7f3ad7c00640 1 -- 172.21.15.7:0/3555131666 shutdown_connections 2024-09-10T21:46:03.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.857+0000 7f3ad7c00640 1 --2- 172.21.15.7:0/3555131666 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3aac077f70 0x7f3aac07a430 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:03.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.857+0000 7f3ad7c00640 1 --2- 172.21.15.7:0/3555131666 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3ad0159680 0x7f3ad01f51d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:03.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.857+0000 7f3ad7c00640 1 --2- 172.21.15.7:0/3555131666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3ad0073e20 0x7f3ad00cd6f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:03.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.857+0000 7f3ad7c00640 1 --2- 172.21.15.7:0/3555131666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3ad006e580 0x7f3ad014bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.857+0000 7f3ad7c00640 1 -- 172.21.15.7:0/3555131666 >> 172.21.15.7:0/3555131666 conn(0x7f3ad00b16d0 msgr2=0x7f3ad0159f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.857+0000 7f3ad7c00640 1 -- 172.21.15.7:0/3555131666 shutdown_connections 2024-09-10T21:46:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:03.857+0000 7f3ad7c00640 1 -- 172.21.15.7:0/3555131666 wait complete. 2024-09-10T21:46:03.876 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:46:03.876 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:46:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.047+0000 7f3d99c00640 1 -- 172.21.15.7:0/3332506217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d9415bc90 msgr2=0x7f3d940ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.047+0000 7f3d99c00640 1 --2- 172.21.15.7:0/3332506217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d9415bc90 0x7f3d940ccef0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3d84009830 tx=0x7f3d8402a530 comp rx=0 tx=0).stop 2024-09-10T21:46:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.047+0000 7f3d99c00640 1 -- 172.21.15.7:0/3332506217 shutdown_connections 2024-09-10T21:46:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.047+0000 7f3d99c00640 1 --2- 172.21.15.7:0/3332506217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d9415bc90 0x7f3d940ccef0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.047+0000 7f3d99c00640 1 -- 172.21.15.7:0/3332506217 >> 172.21.15.7:0/3332506217 conn(0x7f3d940b16d0 msgr2=0x7f3d940b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.047+0000 7f3d99c00640 1 -- 172.21.15.7:0/3332506217 shutdown_connections 2024-09-10T21:46:04.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.047+0000 7f3d99c00640 1 -- 172.21.15.7:0/3332506217 wait complete. 2024-09-10T21:46:04.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.049+0000 7f3d99c00640 1 Processor -- start 2024-09-10T21:46:04.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.049+0000 7f3d99c00640 1 -- start start 2024-09-10T21:46:04.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.050+0000 7f3d99c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d940cd750 0x7f3d940c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:04.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.050+0000 7f3d99c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d9415bc90 0x7f3d9414d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:04.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.050+0000 7f3d99c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d9415fda0 0x7f3d94150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:04.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.050+0000 7f3d99c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3d94070080 con 0x7f3d940cd750 2024-09-10T21:46:04.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.050+0000 7f3d99c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3d9406ff00 con 0x7f3d9415fda0 2024-09-10T21:46:04.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.050+0000 7f3d99c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f3d94070200 con 0x7f3d9415bc90 2024-09-10T21:46:04.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.050+0000 7f3d92a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d940cd750 0x7f3d940c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:04.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.050+0000 7f3d92a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d940cd750 0x7f3d940c7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44730/0 (socket says 172.21.15.7:44730) 2024-09-10T21:46:04.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.050+0000 7f3d92a00640 1 -- 172.21.15.7:0/3224613713 learned_addr learned my addr 172.21.15.7:0/3224613713 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:04.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.051+0000 7f3d8a000640 1 --2- 172.21.15.7:0/3224613713 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d9415bc90 0x7f3d9414d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:04.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.051+0000 7f3d93400640 1 --2- 172.21.15.7:0/3224613713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d9415fda0 0x7f3d94150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:04.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.051+0000 7f3d92a00640 1 -- 172.21.15.7:0/3224613713 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d9415bc90 msgr2=0x7f3d9414d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:04.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.051+0000 7f3d92a00640 1 --2- 172.21.15.7:0/3224613713 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d9415bc90 0x7f3d9414d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.051+0000 7f3d92a00640 1 -- 172.21.15.7:0/3224613713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d9415fda0 msgr2=0x7f3d94150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:04.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.051+0000 7f3d92a00640 1 --2- 172.21.15.7:0/3224613713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d9415fda0 0x7f3d94150040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.051+0000 7f3d92a00640 1 -- 172.21.15.7:0/3224613713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d94150a00 con 0x7f3d940cd750 2024-09-10T21:46:04.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.051+0000 7f3d92a00640 1 --2- 172.21.15.7:0/3224613713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d940cd750 0x7f3d940c7e50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3d84009950 tx=0x7f3d84032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:04.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.051+0000 7f3d8b400640 1 -- 172.21.15.7:0/3224613713 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d84040070 con 0x7f3d940cd750 2024-09-10T21:46:04.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.051+0000 7f3d8b400640 1 -- 172.21.15.7:0/3224613713 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d84005310 con 0x7f3d940cd750 2024-09-10T21:46:04.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.051+0000 7f3d99c00640 1 -- 172.21.15.7:0/3224613713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d94150ce0 con 0x7f3d940cd750 2024-09-10T21:46:04.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.051+0000 7f3d8b400640 1 -- 172.21.15.7:0/3224613713 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d840047a0 con 0x7f3d940cd750 2024-09-10T21:46:04.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.052+0000 7f3d99c00640 1 -- 172.21.15.7:0/3224613713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d940afef0 con 0x7f3d940cd750 2024-09-10T21:46:04.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.053+0000 7f3d99c00640 1 -- 172.21.15.7:0/3224613713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d940b5740 con 0x7f3d940cd750 2024-09-10T21:46:04.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.054+0000 7f3d8b400640 1 -- 172.21.15.7:0/3224613713 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3d8404c030 con 0x7f3d940cd750 2024-09-10T21:46:04.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.054+0000 7f3d8b400640 1 --2- 172.21.15.7:0/3224613713 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3d70077f70 0x7f3d7007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:04.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.055+0000 7f3d8b400640 1 -- 172.21.15.7:0/3224613713 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f3d840baee0 con 0x7f3d940cd750 2024-09-10T21:46:04.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.055+0000 7f3d8a000640 1 --2- 172.21.15.7:0/3224613713 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3d70077f70 0x7f3d7007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:04.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.056+0000 7f3d8a000640 1 --2- 172.21.15.7:0/3224613713 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3d70077f70 0x7f3d7007a430 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3d9415f600 tx=0x7f3d7c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:04.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.058+0000 7f3d8b400640 1 -- 172.21.15.7:0/3224613713 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3d940b5740 con 0x7f3d940cd750 2024-09-10T21:46:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.291+0000 7f3d99c00640 1 -- 172.21.15.7:0/3224613713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d940706b0 con 0x7f3d940cd750 2024-09-10T21:46:04.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.291+0000 7f3d8b400640 1 -- 172.21.15.7:0/3224613713 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5235 (secure 0 0 0) 0x7f3d84083940 con 0x7f3d940cd750 2024-09-10T21:46:04.293 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:46:04.293 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":46,"btime":"2024-09-10T21:46:03:853860+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5884,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3488410478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3488410478},{"type":"v1","addr":"172.21.15.133:6837","nonce":3488410478}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5890,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3660400727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3660400727},{"type":"v1","addr":"172.21.15.7:6835","nonce":3660400727}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2388873704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2388873704},{"type":"v1","addr":"172.21.15.7:6834","nonce":2388873704}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:46:00.260975+0000","modified":"2024-09-10T21:46:03.290119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235361":{"gid":235361,"name":"b","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6836/2752881644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2752881644},{"type":"v1","addr":"172.21.15.133:6836","nonce":2752881644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235361,"qdb_cluster":[235361]},"id":5}]} 2024-09-10T21:46:04.293 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 46 2024-09-10T21:46:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.294+0000 7f3d99c00640 1 -- 172.21.15.7:0/3224613713 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3d70077f70 msgr2=0x7f3d7007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.294+0000 7f3d99c00640 1 --2- 172.21.15.7:0/3224613713 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3d70077f70 0x7f3d7007a430 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3d9415f600 tx=0x7f3d7c009040 comp rx=0 tx=0).stop 2024-09-10T21:46:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.294+0000 7f3d99c00640 1 -- 172.21.15.7:0/3224613713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d940cd750 msgr2=0x7f3d940c7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.294+0000 7f3d99c00640 1 --2- 172.21.15.7:0/3224613713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d940cd750 0x7f3d940c7e50 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f3d84009950 tx=0x7f3d84032b90 comp rx=0 tx=0).stop 2024-09-10T21:46:04.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.294+0000 7f3d99c00640 1 -- 172.21.15.7:0/3224613713 shutdown_connections 2024-09-10T21:46:04.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.294+0000 7f3d99c00640 1 --2- 172.21.15.7:0/3224613713 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3d70077f70 0x7f3d7007a430 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.294+0000 7f3d99c00640 1 --2- 172.21.15.7:0/3224613713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3d9415fda0 0x7f3d94150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.294+0000 7f3d99c00640 1 --2- 172.21.15.7:0/3224613713 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3d9415bc90 0x7f3d9414d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.294+0000 7f3d99c00640 1 --2- 172.21.15.7:0/3224613713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3d940cd750 0x7f3d940c7e50 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.294+0000 7f3d99c00640 1 -- 172.21.15.7:0/3224613713 >> 172.21.15.7:0/3224613713 conn(0x7f3d940b16d0 msgr2=0x7f3d940b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:04.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.294+0000 7f3d99c00640 1 -- 172.21.15.7:0/3224613713 shutdown_connections 2024-09-10T21:46:04.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.294+0000 7f3d99c00640 1 -- 172.21.15.7:0/3224613713 wait complete. 2024-09-10T21:46:04.313 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:46:04.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.473+0000 7f39e8e00640 1 -- 172.21.15.7:0/730586307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e40cd750 msgr2=0x7f39e40ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:04.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.473+0000 7f39e8e00640 1 --2- 172.21.15.7:0/730586307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e40cd750 0x7f39e40ccef0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f39e40c3650 tx=0x7f39d002a4d0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.473+0000 7f39e8e00640 1 -- 172.21.15.7:0/730586307 shutdown_connections 2024-09-10T21:46:04.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.473+0000 7f39e8e00640 1 --2- 172.21.15.7:0/730586307 >> v2:172.21.15.133:3300/0 conn(0x7f39e415fda0 0x7f39e4160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.473+0000 7f39e8e00640 1 --2- 172.21.15.7:0/730586307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e40cd750 0x7f39e40ccef0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.473+0000 7f39e8e00640 1 -- 172.21.15.7:0/730586307 >> 172.21.15.7:0/730586307 conn(0x7f39e40b16d0 msgr2=0x7f39e40b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:04.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.473+0000 7f39e8e00640 1 -- 172.21.15.7:0/730586307 shutdown_connections 2024-09-10T21:46:04.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.473+0000 7f39e8e00640 1 -- 172.21.15.7:0/730586307 wait complete. 2024-09-10T21:46:04.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.474+0000 7f39e8e00640 1 Processor -- start 2024-09-10T21:46:04.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.475+0000 7f39e8e00640 1 -- start start 2024-09-10T21:46:04.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.475+0000 7f39e8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e40cd750 0x7f39e415e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:04.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.475+0000 7f39e8e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39e415bc90 0x7f39e4155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:04.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.475+0000 7f39e8e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39e415fda0 0x7f39e40c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:04.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.475+0000 7f39e8e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f39e406fe70 con 0x7f39e40cd750 2024-09-10T21:46:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.475+0000 7f39e8e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f39e406fcf0 con 0x7f39e415fda0 2024-09-10T21:46:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.475+0000 7f39e8e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f39e406fff0 con 0x7f39e415bc90 2024-09-10T21:46:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.476+0000 7f39e1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e40cd750 0x7f39e415e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.476+0000 7f39e1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e40cd750 0x7f39e415e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44758/0 (socket says 172.21.15.7:44758) 2024-09-10T21:46:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.476+0000 7f39e1e00640 1 -- 172.21.15.7:0/721471265 learned_addr learned my addr 172.21.15.7:0/721471265 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.476+0000 7f39e2800640 1 --2- 172.21.15.7:0/721471265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39e415fda0 0x7f39e40c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.476+0000 7f39e1400640 1 --2- 172.21.15.7:0/721471265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39e415bc90 0x7f39e4155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.476+0000 7f39e1e00640 1 -- 172.21.15.7:0/721471265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39e415bc90 msgr2=0x7f39e4155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.476+0000 7f39e1e00640 1 --2- 172.21.15.7:0/721471265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39e415bc90 0x7f39e4155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.476+0000 7f39e1e00640 1 -- 172.21.15.7:0/721471265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39e415fda0 msgr2=0x7f39e40c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.476+0000 7f39e1e00640 1 --2- 172.21.15.7:0/721471265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39e415fda0 0x7f39e40c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.476+0000 7f39e1e00640 1 -- 172.21.15.7:0/721471265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e41f5d00 con 0x7f39e40cd750 2024-09-10T21:46:04.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.477+0000 7f39e1e00640 1 --2- 172.21.15.7:0/721471265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e40cd750 0x7f39e415e070 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f39d400c430 tx=0x7f39d400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:04.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.477+0000 7f39da000640 1 -- 172.21.15.7:0/721471265 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4011790 con 0x7f39e40cd750 2024-09-10T21:46:04.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.477+0000 7f39da000640 1 -- 172.21.15.7:0/721471265 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f39d4017070 con 0x7f39e40cd750 2024-09-10T21:46:04.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.477+0000 7f39e8e00640 1 -- 172.21.15.7:0/721471265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e41f5fe0 con 0x7f39e40cd750 2024-09-10T21:46:04.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.477+0000 7f39da000640 1 -- 172.21.15.7:0/721471265 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d4011d00 con 0x7f39e40cd750 2024-09-10T21:46:04.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.477+0000 7f39e8e00640 1 -- 172.21.15.7:0/721471265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e41f65a0 con 0x7f39e40cd750 2024-09-10T21:46:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.478+0000 7f39e8e00640 1 -- 172.21.15.7:0/721471265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39e406a380 con 0x7f39e40cd750 2024-09-10T21:46:04.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.479+0000 7f39da000640 1 -- 172.21.15.7:0/721471265 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f39d401c460 con 0x7f39e40cd750 2024-09-10T21:46:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.480+0000 7f39da000640 1 --2- 172.21.15.7:0/721471265 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f39c4077f70 0x7f39c407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.480+0000 7f39da000640 1 -- 172.21.15.7:0/721471265 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f39d4099b30 con 0x7f39e40cd750 2024-09-10T21:46:04.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.481+0000 7f39e1400640 1 --2- 172.21.15.7:0/721471265 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f39c4077f70 0x7f39c407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:04.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.481+0000 7f39e1400640 1 --2- 172.21.15.7:0/721471265 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f39c4077f70 0x7f39c407a430 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f39d0033000 tx=0x7f39d0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:04.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.484+0000 7f39da000640 1 -- 172.21.15.7:0/721471265 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f39d4061f40 con 0x7f39e40cd750 2024-09-10T21:46:04.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.678+0000 7f39e8e00640 1 -- 172.21.15.7:0/721471265 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39e4070510 con 0x7f39e40cd750 2024-09-10T21:46:04.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.679+0000 7f39da000640 1 -- 172.21.15.7:0/721471265 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+16692 (secure 0 0 0) 0x7f39d4066970 con 0x7f39e40cd750 2024-09-10T21:46:04.681 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:46:04.681 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":70,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:46:01.821559+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:45:58.069464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T21:45:59.225636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:46:04.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.682+0000 7f39e8e00640 1 -- 172.21.15.7:0/721471265 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f39c4077f70 msgr2=0x7f39c407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:04.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.682+0000 7f39e8e00640 1 --2- 172.21.15.7:0/721471265 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f39c4077f70 0x7f39c407a430 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f39d0033000 tx=0x7f39d0004730 comp rx=0 tx=0).stop 2024-09-10T21:46:04.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.682+0000 7f39e8e00640 1 -- 172.21.15.7:0/721471265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e40cd750 msgr2=0x7f39e415e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:04.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.682+0000 7f39e8e00640 1 --2- 172.21.15.7:0/721471265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e40cd750 0x7f39e415e070 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f39d400c430 tx=0x7f39d400c8f0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.682+0000 7f39e8e00640 1 -- 172.21.15.7:0/721471265 shutdown_connections 2024-09-10T21:46:04.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.682+0000 7f39e8e00640 1 --2- 172.21.15.7:0/721471265 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f39c4077f70 0x7f39c407a430 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.682+0000 7f39e8e00640 1 --2- 172.21.15.7:0/721471265 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f39e415fda0 0x7f39e40c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.682+0000 7f39e8e00640 1 --2- 172.21.15.7:0/721471265 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f39e415bc90 0x7f39e4155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.682+0000 7f39e8e00640 1 --2- 172.21.15.7:0/721471265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f39e40cd750 0x7f39e415e070 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.682+0000 7f39e8e00640 1 -- 172.21.15.7:0/721471265 >> 172.21.15.7:0/721471265 conn(0x7f39e40b16d0 msgr2=0x7f39e415b310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:04.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.682+0000 7f39e8e00640 1 -- 172.21.15.7:0/721471265 shutdown_connections 2024-09-10T21:46:04.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.682+0000 7f39e8e00640 1 -- 172.21.15.7:0/721471265 wait complete. 2024-09-10T21:46:04.700 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:46:04.700 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:46:04.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.861+0000 7f9a99600640 1 -- 172.21.15.7:0/4239818820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a940d2be0 msgr2=0x7f9a940b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:04.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.861+0000 7f9a99600640 1 --2- 172.21.15.7:0/4239818820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a940d2be0 0x7f9a940b4450 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f9a8c005fd0 tx=0x7f9a8c02a8b0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.861+0000 7f9a99600640 1 -- 172.21.15.7:0/4239818820 shutdown_connections 2024-09-10T21:46:04.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.861+0000 7f9a99600640 1 --2- 172.21.15.7:0/4239818820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a940d2be0 0x7f9a940b4450 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.861+0000 7f9a99600640 1 --2- 172.21.15.7:0/4239818820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9a9415e090 0x7f9a940d1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.861+0000 7f9a99600640 1 -- 172.21.15.7:0/4239818820 >> 172.21.15.7:0/4239818820 conn(0x7f9a940d3f10 msgr2=0x7f9a940d4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:04.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.861+0000 7f9a99600640 1 -- 172.21.15.7:0/4239818820 shutdown_connections 2024-09-10T21:46:04.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.861+0000 7f9a99600640 1 -- 172.21.15.7:0/4239818820 wait complete. 2024-09-10T21:46:04.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.863+0000 7f9a99600640 1 Processor -- start 2024-09-10T21:46:04.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.863+0000 7f9a99600640 1 -- start start 2024-09-10T21:46:04.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.864+0000 7f9a99600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a940d1ae0 0x7f9a941564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:04.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.864+0000 7f9a99600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9a940d2be0 0x7f9a940a1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.864+0000 7f9a99600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a9415e090 0x7f9a940cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.864+0000 7f9a99600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a940b4d20 con 0x7f9a940d1ae0 2024-09-10T21:46:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.864+0000 7f9a99600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9a940b4ba0 con 0x7f9a9415e090 2024-09-10T21:46:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.864+0000 7f9a99600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f9a940b4ea0 con 0x7f9a940d2be0 2024-09-10T21:46:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.864+0000 7f9a92a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a940d1ae0 0x7f9a941564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.864+0000 7f9a92a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a940d1ae0 0x7f9a941564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44774/0 (socket says 172.21.15.7:44774) 2024-09-10T21:46:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.864+0000 7f9a92a00640 1 -- 172.21.15.7:0/3151221207 learned_addr learned my addr 172.21.15.7:0/3151221207 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.864+0000 7f9a8be00640 1 --2- 172.21.15.7:0/3151221207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9a940d2be0 0x7f9a940a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.864+0000 7f9a93400640 1 --2- 172.21.15.7:0/3151221207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a9415e090 0x7f9a940cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.864+0000 7f9a92a00640 1 -- 172.21.15.7:0/3151221207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9a940d2be0 msgr2=0x7f9a940a1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.864+0000 7f9a92a00640 1 --2- 172.21.15.7:0/3151221207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9a940d2be0 0x7f9a940a1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.864+0000 7f9a92a00640 1 -- 172.21.15.7:0/3151221207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a9415e090 msgr2=0x7f9a940cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:04.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.864+0000 7f9a92a00640 1 --2- 172.21.15.7:0/3151221207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a9415e090 0x7f9a940cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:04.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.864+0000 7f9a92a00640 1 -- 172.21.15.7:0/3151221207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a940cd5e0 con 0x7f9a940d1ae0 2024-09-10T21:46:04.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.865+0000 7f9a92a00640 1 --2- 172.21.15.7:0/3151221207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a940d1ae0 0x7f9a941564f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400ea20 tx=0x7f9a8400eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:04.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.865+0000 7f9a8aa00640 1 -- 172.21.15.7:0/3151221207 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a8400ce40 con 0x7f9a940d1ae0 2024-09-10T21:46:04.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.865+0000 7f9a8aa00640 1 -- 172.21.15.7:0/3151221207 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a84004020 con 0x7f9a940d1ae0 2024-09-10T21:46:04.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.865+0000 7f9a8aa00640 1 -- 172.21.15.7:0/3151221207 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a84004470 con 0x7f9a940d1ae0 2024-09-10T21:46:04.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.865+0000 7f9a99600640 1 -- 172.21.15.7:0/3151221207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a941f49e0 con 0x7f9a940d1ae0 2024-09-10T21:46:04.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.865+0000 7f9a99600640 1 -- 172.21.15.7:0/3151221207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a941f4fa0 con 0x7f9a940d1ae0 2024-09-10T21:46:04.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.866+0000 7f9a99600640 1 -- 172.21.15.7:0/3151221207 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a940b4450 con 0x7f9a940d1ae0 2024-09-10T21:46:04.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.867+0000 7f9a8aa00640 1 -- 172.21.15.7:0/3151221207 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9a840035f0 con 0x7f9a940d1ae0 2024-09-10T21:46:04.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.868+0000 7f9a8aa00640 1 --2- 172.21.15.7:0/3151221207 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9a70077f70 0x7f9a7007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:04.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.868+0000 7f9a8aa00640 1 -- 172.21.15.7:0/3151221207 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f9a8409a7b0 con 0x7f9a940d1ae0 2024-09-10T21:46:04.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.869+0000 7f9a8be00640 1 --2- 172.21.15.7:0/3151221207 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9a70077f70 0x7f9a7007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:04.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.870+0000 7f9a8be00640 1 --2- 172.21.15.7:0/3151221207 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9a70077f70 0x7f9a7007a430 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c007fd0 tx=0x7f9a7c001430 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:04.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:04.873+0000 7f9a8aa00640 1 -- 172.21.15.7:0/3151221207 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9a84062bc0 con 0x7f9a940d1ae0 2024-09-10T21:46:05.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.066+0000 7f9a99600640 1 -- 172.21.15.7:0/3151221207 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9a9406e590 con 0x7f9a70077f70 2024-09-10T21:46:05.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.071+0000 7f9a8aa00640 1 -- 172.21.15.7:0/3151221207 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252053 (secure 0 0 0) 0x7f9a9406e590 con 0x7f9a70077f70 2024-09-10T21:46:05.075 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:46:05.077 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:46:05.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.076+0000 7f9a99600640 1 -- 172.21.15.7:0/3151221207 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9a70077f70 msgr2=0x7f9a7007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:05.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.076+0000 7f9a99600640 1 --2- 172.21.15.7:0/3151221207 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9a70077f70 0x7f9a7007a430 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c007fd0 tx=0x7f9a7c001430 comp rx=0 tx=0).stop 2024-09-10T21:46:05.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.076+0000 7f9a99600640 1 -- 172.21.15.7:0/3151221207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a940d1ae0 msgr2=0x7f9a941564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.076+0000 7f9a99600640 1 --2- 172.21.15.7:0/3151221207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a940d1ae0 0x7f9a941564f0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400ea20 tx=0x7f9a8400eee0 comp rx=0 tx=0).stop 2024-09-10T21:46:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.076+0000 7f9a99600640 1 -- 172.21.15.7:0/3151221207 shutdown_connections 2024-09-10T21:46:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.076+0000 7f9a99600640 1 --2- 172.21.15.7:0/3151221207 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9a70077f70 0x7f9a7007a430 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.076+0000 7f9a99600640 1 --2- 172.21.15.7:0/3151221207 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a9415e090 0x7f9a940cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.076+0000 7f9a99600640 1 --2- 172.21.15.7:0/3151221207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9a940d2be0 0x7f9a940a1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.076+0000 7f9a99600640 1 --2- 172.21.15.7:0/3151221207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a940d1ae0 0x7f9a941564f0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.076+0000 7f9a99600640 1 -- 172.21.15.7:0/3151221207 >> 172.21.15.7:0/3151221207 conn(0x7f9a940d3f10 msgr2=0x7f9a9406ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.076+0000 7f9a99600640 1 -- 172.21.15.7:0/3151221207 shutdown_connections 2024-09-10T21:46:05.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.076+0000 7f9a99600640 1 -- 172.21.15.7:0/3151221207 wait complete. 2024-09-10T21:46:05.094 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-10T21:46:03.916000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257320,"kb_used_data":7768,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717400,"statfs":{"total":773094113280,"available":772830617600,"internally_reserved":0,"allocated":7954432,"data_stored":3984884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.105338"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.857623+0000","last_change":"2024-09-10T21:46:00.845405+0000","last_active":"2024-09-10T21:46:00.857623+0000","last_peered":"2024-09-10T21:46:00.857623+0000","last_clean":"2024-09-10T21:46:00.857623+0000","last_became_active":"2024-09-10T21:46:00.845093+0000","last_became_peered":"2024-09-10T21:46:00.845093+0000","last_unstale":"2024-09-10T21:46:00.857623+0000","last_undegraded":"2024-09-10T21:46:00.857623+0000","last_fullsized":"2024-09-10T21:46:00.857623+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:01:48.977667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.829269+0000","last_change":"2024-09-10T21:46:00.839425+0000","last_active":"2024-09-10T21:46:01.829269+0000","last_peered":"2024-09-10T21:46:01.829269+0000","last_clean":"2024-09-10T21:46:01.829269+0000","last_became_active":"2024-09-10T21:46:00.839055+0000","last_became_peered":"2024-09-10T21:46:00.839055+0000","last_unstale":"2024-09-10T21:46:01.829269+0000","last_undegraded":"2024-09-10T21:46:01.829269+0000","last_fullsized":"2024-09-10T21:46:01.829269+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:37.238972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.845162+0000","last_change":"2024-09-10T21:46:00.841041+0000","last_active":"2024-09-10T21:46:00.845162+0000","last_peered":"2024-09-10T21:46:00.845162+0000","last_clean":"2024-09-10T21:46:00.845162+0000","last_became_active":"2024-09-10T21:46:00.840517+0000","last_became_peered":"2024-09-10T21:46:00.840517+0000","last_unstale":"2024-09-10T21:46:00.845162+0000","last_undegraded":"2024-09-10T21:46:00.845162+0000","last_fullsized":"2024-09-10T21:46:00.845162+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:19.662566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.710960+0000","last_change":"2024-09-10T21:46:00.841498+0000","last_active":"2024-09-10T21:46:02.710960+0000","last_peered":"2024-09-10T21:46:02.710960+0000","last_clean":"2024-09-10T21:46:02.710960+0000","last_became_active":"2024-09-10T21:46:00.841104+0000","last_became_peered":"2024-09-10T21:46:00.841104+0000","last_unstale":"2024-09-10T21:46:02.710960+0000","last_undegraded":"2024-09-10T21:46:02.710960+0000","last_fullsized":"2024-09-10T21:46:02.710960+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:25.233074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709731+0000","last_change":"2024-09-10T21:46:00.840826+0000","last_active":"2024-09-10T21:46:02.709731+0000","last_peered":"2024-09-10T21:46:02.709731+0000","last_clean":"2024-09-10T21:46:02.709731+0000","last_became_active":"2024-09-10T21:46:00.839302+0000","last_became_peered":"2024-09-10T21:46:00.839302+0000","last_unstale":"2024-09-10T21:46:02.709731+0000","last_undegraded":"2024-09-10T21:46:02.709731+0000","last_fullsized":"2024-09-10T21:46:02.709731+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:10.316342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828592+0000","last_change":"2024-09-10T21:46:00.854710+0000","last_active":"2024-09-10T21:46:01.828592+0000","last_peered":"2024-09-10T21:46:01.828592+0000","last_clean":"2024-09-10T21:46:01.828592+0000","last_became_active":"2024-09-10T21:46:00.854455+0000","last_became_peered":"2024-09-10T21:46:00.854455+0000","last_unstale":"2024-09-10T21:46:01.828592+0000","last_undegraded":"2024-09-10T21:46:01.828592+0000","last_fullsized":"2024-09-10T21:46:01.828592+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:19:51.897259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830917+0000","last_change":"2024-09-10T21:46:00.848967+0000","last_active":"2024-09-10T21:46:01.830917+0000","last_peered":"2024-09-10T21:46:01.830917+0000","last_clean":"2024-09-10T21:46:01.830917+0000","last_became_active":"2024-09-10T21:46:00.848450+0000","last_became_peered":"2024-09-10T21:46:00.848450+0000","last_unstale":"2024-09-10T21:46:01.830917+0000","last_undegraded":"2024-09-10T21:46:01.830917+0000","last_fullsized":"2024-09-10T21:46:01.830917+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:56.519958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828082+0000","last_change":"2024-09-10T21:46:00.844463+0000","last_active":"2024-09-10T21:46:01.828082+0000","last_peered":"2024-09-10T21:46:01.828082+0000","last_clean":"2024-09-10T21:46:01.828082+0000","last_became_active":"2024-09-10T21:46:00.841533+0000","last_became_peered":"2024-09-10T21:46:00.841533+0000","last_unstale":"2024-09-10T21:46:01.828082+0000","last_undegraded":"2024-09-10T21:46:01.828082+0000","last_fullsized":"2024-09-10T21:46:01.828082+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:29.186264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709860+0000","last_change":"2024-09-10T21:46:00.844670+0000","last_active":"2024-09-10T21:46:02.709860+0000","last_peered":"2024-09-10T21:46:02.709860+0000","last_clean":"2024-09-10T21:46:02.709860+0000","last_became_active":"2024-09-10T21:46:00.844340+0000","last_became_peered":"2024-09-10T21:46:00.844340+0000","last_unstale":"2024-09-10T21:46:02.709860+0000","last_undegraded":"2024-09-10T21:46:02.709860+0000","last_fullsized":"2024-09-10T21:46:02.709860+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:10.664153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830583+0000","last_change":"2024-09-10T21:46:00.845549+0000","last_active":"2024-09-10T21:46:01.830583+0000","last_peered":"2024-09-10T21:46:01.830583+0000","last_clean":"2024-09-10T21:46:01.830583+0000","last_became_active":"2024-09-10T21:46:00.845289+0000","last_became_peered":"2024-09-10T21:46:00.845289+0000","last_unstale":"2024-09-10T21:46:01.830583+0000","last_undegraded":"2024-09-10T21:46:01.830583+0000","last_fullsized":"2024-09-10T21:46:01.830583+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:34.031166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.857698+0000","last_change":"2024-09-10T21:46:00.846160+0000","last_active":"2024-09-10T21:46:00.857698+0000","last_peered":"2024-09-10T21:46:00.857698+0000","last_clean":"2024-09-10T21:46:00.857698+0000","last_became_active":"2024-09-10T21:46:00.845817+0000","last_became_peered":"2024-09-10T21:46:00.845817+0000","last_unstale":"2024-09-10T21:46:00.857698+0000","last_undegraded":"2024-09-10T21:46:00.857698+0000","last_fullsized":"2024-09-10T21:46:00.857698+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:04.317280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.710991+0000","last_change":"2024-09-10T21:46:00.843787+0000","last_active":"2024-09-10T21:46:02.710991+0000","last_peered":"2024-09-10T21:46:02.710991+0000","last_clean":"2024-09-10T21:46:02.710991+0000","last_became_active":"2024-09-10T21:46:00.843497+0000","last_became_peered":"2024-09-10T21:46:00.843497+0000","last_unstale":"2024-09-10T21:46:02.710991+0000","last_undegraded":"2024-09-10T21:46:02.710991+0000","last_fullsized":"2024-09-10T21:46:02.710991+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:19.504216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.850517+0000","last_change":"2024-09-10T21:46:00.839289+0000","last_active":"2024-09-10T21:46:01.850517+0000","last_peered":"2024-09-10T21:46:01.850517+0000","last_clean":"2024-09-10T21:46:01.850517+0000","last_became_active":"2024-09-10T21:46:00.839002+0000","last_became_peered":"2024-09-10T21:46:00.839002+0000","last_unstale":"2024-09-10T21:46:01.850517+0000","last_undegraded":"2024-09-10T21:46:01.850517+0000","last_fullsized":"2024-09-10T21:46:01.850517+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:37.624857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709810+0000","last_change":"2024-09-10T21:46:00.854653+0000","last_active":"2024-09-10T21:46:02.709810+0000","last_peered":"2024-09-10T21:46:02.709810+0000","last_clean":"2024-09-10T21:46:02.709810+0000","last_became_active":"2024-09-10T21:46:00.854198+0000","last_became_peered":"2024-09-10T21:46:00.854198+0000","last_unstale":"2024-09-10T21:46:02.709810+0000","last_undegraded":"2024-09-10T21:46:02.709810+0000","last_fullsized":"2024-09-10T21:46:02.709810+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:24.506727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.850511+0000","last_change":"2024-09-10T21:46:00.847153+0000","last_active":"2024-09-10T21:46:01.850511+0000","last_peered":"2024-09-10T21:46:01.850511+0000","last_clean":"2024-09-10T21:46:01.850511+0000","last_became_active":"2024-09-10T21:46:00.846597+0000","last_became_peered":"2024-09-10T21:46:00.846597+0000","last_unstale":"2024-09-10T21:46:01.850511+0000","last_undegraded":"2024-09-10T21:46:01.850511+0000","last_fullsized":"2024-09-10T21:46:01.850511+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:03.399674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.829825+0000","last_change":"2024-09-10T21:46:00.847779+0000","last_active":"2024-09-10T21:46:01.829825+0000","last_peered":"2024-09-10T21:46:01.829825+0000","last_clean":"2024-09-10T21:46:01.829825+0000","last_became_active":"2024-09-10T21:46:00.847407+0000","last_became_peered":"2024-09-10T21:46:00.847407+0000","last_unstale":"2024-09-10T21:46:01.829825+0000","last_undegraded":"2024-09-10T21:46:01.829825+0000","last_fullsized":"2024-09-10T21:46:01.829825+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:20.998836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.710109+0000","last_change":"2024-09-10T21:46:00.845442+0000","last_active":"2024-09-10T21:46:02.710109+0000","last_peered":"2024-09-10T21:46:02.710109+0000","last_clean":"2024-09-10T21:46:02.710109+0000","last_became_active":"2024-09-10T21:46:00.845134+0000","last_became_peered":"2024-09-10T21:46:00.845134+0000","last_unstale":"2024-09-10T21:46:02.710109+0000","last_undegraded":"2024-09-10T21:46:02.710109+0000","last_fullsized":"2024-09-10T21:46:02.710109+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:30.650327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.711363+0000","last_change":"2024-09-10T21:46:00.835386+0000","last_active":"2024-09-10T21:46:02.711363+0000","last_peered":"2024-09-10T21:46:02.711363+0000","last_clean":"2024-09-10T21:46:02.711363+0000","last_became_active":"2024-09-10T21:46:00.835115+0000","last_became_peered":"2024-09-10T21:46:00.835115+0000","last_unstale":"2024-09-10T21:46:02.711363+0000","last_undegraded":"2024-09-10T21:46:02.711363+0000","last_fullsized":"2024-09-10T21:46:02.711363+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.555199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828651+0000","last_change":"2024-09-10T21:46:00.830508+0000","last_active":"2024-09-10T21:46:01.828651+0000","last_peered":"2024-09-10T21:46:01.828651+0000","last_clean":"2024-09-10T21:46:01.828651+0000","last_became_active":"2024-09-10T21:46:00.830232+0000","last_became_peered":"2024-09-10T21:46:00.830232+0000","last_unstale":"2024-09-10T21:46:01.828651+0000","last_undegraded":"2024-09-10T21:46:01.828651+0000","last_fullsized":"2024-09-10T21:46:01.828651+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:39.679370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.850612+0000","last_change":"2024-09-10T21:46:00.853233+0000","last_active":"2024-09-10T21:46:01.850612+0000","last_peered":"2024-09-10T21:46:01.850612+0000","last_clean":"2024-09-10T21:46:01.850612+0000","last_became_active":"2024-09-10T21:46:00.852834+0000","last_became_peered":"2024-09-10T21:46:00.852834+0000","last_unstale":"2024-09-10T21:46:01.850612+0000","last_undegraded":"2024-09-10T21:46:01.850612+0000","last_fullsized":"2024-09-10T21:46:01.850612+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:34.784319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830406+0000","last_change":"2024-09-10T21:46:00.843228+0000","last_active":"2024-09-10T21:46:01.830406+0000","last_peered":"2024-09-10T21:46:01.830406+0000","last_clean":"2024-09-10T21:46:01.830406+0000","last_became_active":"2024-09-10T21:46:00.842891+0000","last_became_peered":"2024-09-10T21:46:00.842891+0000","last_unstale":"2024-09-10T21:46:01.830406+0000","last_undegraded":"2024-09-10T21:46:01.830406+0000","last_fullsized":"2024-09-10T21:46:01.830406+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:03.394188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830180+0000","last_change":"2024-09-10T21:46:00.842985+0000","last_active":"2024-09-10T21:46:01.830180+0000","last_peered":"2024-09-10T21:46:01.830180+0000","last_clean":"2024-09-10T21:46:01.830180+0000","last_became_active":"2024-09-10T21:46:00.842604+0000","last_became_peered":"2024-09-10T21:46:00.842604+0000","last_unstale":"2024-09-10T21:46:01.830180+0000","last_undegraded":"2024-09-10T21:46:01.830180+0000","last_fullsized":"2024-09-10T21:46:01.830180+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:00.984325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.859026+0000","last_change":"2024-09-10T21:46:00.854428+0000","last_active":"2024-09-10T21:46:00.859026+0000","last_peered":"2024-09-10T21:46:00.859026+0000","last_clean":"2024-09-10T21:46:00.859026+0000","last_became_active":"2024-09-10T21:46:00.854124+0000","last_became_peered":"2024-09-10T21:46:00.854124+0000","last_unstale":"2024-09-10T21:46:00.859026+0000","last_undegraded":"2024-09-10T21:46:00.859026+0000","last_fullsized":"2024-09-10T21:46:00.859026+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:01.690462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.858049+0000","last_change":"2024-09-10T21:46:00.847631+0000","last_active":"2024-09-10T21:46:00.858049+0000","last_peered":"2024-09-10T21:46:00.858049+0000","last_clean":"2024-09-10T21:46:00.858049+0000","last_became_active":"2024-09-10T21:46:00.847364+0000","last_became_peered":"2024-09-10T21:46:00.847364+0000","last_unstale":"2024-09-10T21:46:00.858049+0000","last_undegraded":"2024-09-10T21:46:00.858049+0000","last_fullsized":"2024-09-10T21:46:00.858049+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:19.153947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830929+0000","last_change":"2024-09-10T21:46:00.838371+0000","last_active":"2024-09-10T21:46:01.830929+0000","last_peered":"2024-09-10T21:46:01.830929+0000","last_clean":"2024-09-10T21:46:01.830929+0000","last_became_active":"2024-09-10T21:46:00.838080+0000","last_became_peered":"2024-09-10T21:46:00.838080+0000","last_unstale":"2024-09-10T21:46:01.830929+0000","last_undegraded":"2024-09-10T21:46:01.830929+0000","last_fullsized":"2024-09-10T21:46:01.830929+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:35.423503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.831256+0000","last_change":"2024-09-10T21:46:00.864076+0000","last_active":"2024-09-10T21:46:01.831256+0000","last_peered":"2024-09-10T21:46:01.831256+0000","last_clean":"2024-09-10T21:46:01.831256+0000","last_became_active":"2024-09-10T21:46:00.863795+0000","last_became_peered":"2024-09-10T21:46:00.863795+0000","last_unstale":"2024-09-10T21:46:01.831256+0000","last_undegraded":"2024-09-10T21:46:01.831256+0000","last_fullsized":"2024-09-10T21:46:01.831256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:32.715496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.852256+0000","last_change":"2024-09-10T21:46:00.850433+0000","last_active":"2024-09-10T21:46:00.852256+0000","last_peered":"2024-09-10T21:46:00.852256+0000","last_clean":"2024-09-10T21:46:00.852256+0000","last_became_active":"2024-09-10T21:46:00.850127+0000","last_became_peered":"2024-09-10T21:46:00.850127+0000","last_unstale":"2024-09-10T21:46:00.852256+0000","last_undegraded":"2024-09-10T21:46:00.852256+0000","last_fullsized":"2024-09-10T21:46:00.852256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:14.666971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830481+0000","last_change":"2024-09-10T21:46:00.840169+0000","last_active":"2024-09-10T21:46:01.830481+0000","last_peered":"2024-09-10T21:46:01.830481+0000","last_clean":"2024-09-10T21:46:01.830481+0000","last_became_active":"2024-09-10T21:46:00.838227+0000","last_became_peered":"2024-09-10T21:46:00.838227+0000","last_unstale":"2024-09-10T21:46:01.830481+0000","last_undegraded":"2024-09-10T21:46:01.830481+0000","last_fullsized":"2024-09-10T21:46:01.830481+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:52.428566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.852058+0000","last_change":"2024-09-10T21:46:00.849867+0000","last_active":"2024-09-10T21:46:00.852058+0000","last_peered":"2024-09-10T21:46:00.852058+0000","last_clean":"2024-09-10T21:46:00.852058+0000","last_became_active":"2024-09-10T21:46:00.849205+0000","last_became_peered":"2024-09-10T21:46:00.849205+0000","last_unstale":"2024-09-10T21:46:00.852058+0000","last_undegraded":"2024-09-10T21:46:00.852058+0000","last_fullsized":"2024-09-10T21:46:00.852058+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:14.339273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.831292+0000","last_change":"2024-09-10T21:46:00.864905+0000","last_active":"2024-09-10T21:46:01.831292+0000","last_peered":"2024-09-10T21:46:01.831292+0000","last_clean":"2024-09-10T21:46:01.831292+0000","last_became_active":"2024-09-10T21:46:00.864529+0000","last_became_peered":"2024-09-10T21:46:00.864529+0000","last_unstale":"2024-09-10T21:46:01.831292+0000","last_undegraded":"2024-09-10T21:46:01.831292+0000","last_fullsized":"2024-09-10T21:46:01.831292+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:05.727840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.831565+0000","last_change":"2024-09-10T21:46:00.846590+0000","last_active":"2024-09-10T21:46:01.831565+0000","last_peered":"2024-09-10T21:46:01.831565+0000","last_clean":"2024-09-10T21:46:01.831565+0000","last_became_active":"2024-09-10T21:46:00.846328+0000","last_became_peered":"2024-09-10T21:46:00.846328+0000","last_unstale":"2024-09-10T21:46:01.831565+0000","last_undegraded":"2024-09-10T21:46:01.831565+0000","last_fullsized":"2024-09-10T21:46:01.831565+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:57.554644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828289+0000","last_change":"2024-09-10T21:46:00.852671+0000","last_active":"2024-09-10T21:46:01.828289+0000","last_peered":"2024-09-10T21:46:01.828289+0000","last_clean":"2024-09-10T21:46:01.828289+0000","last_became_active":"2024-09-10T21:46:00.849824+0000","last_became_peered":"2024-09-10T21:46:00.849824+0000","last_unstale":"2024-09-10T21:46:01.828289+0000","last_undegraded":"2024-09-10T21:46:01.828289+0000","last_fullsized":"2024-09-10T21:46:01.828289+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:37.967117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.857787+0000","last_change":"2024-09-10T21:46:00.847058+0000","last_active":"2024-09-10T21:46:00.857787+0000","last_peered":"2024-09-10T21:46:00.857787+0000","last_clean":"2024-09-10T21:46:00.857787+0000","last_became_active":"2024-09-10T21:46:00.846826+0000","last_became_peered":"2024-09-10T21:46:00.846826+0000","last_unstale":"2024-09-10T21:46:00.857787+0000","last_undegraded":"2024-09-10T21:46:00.857787+0000","last_fullsized":"2024-09-10T21:46:00.857787+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:43.687249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"69'4","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.870886+0000","last_change":"2024-09-10T21:45:59.839520+0000","last_active":"2024-09-10T21:46:00.870886+0000","last_peered":"2024-09-10T21:46:00.870886+0000","last_clean":"2024-09-10T21:46:00.870886+0000","last_became_active":"2024-09-10T21:45:59.839036+0000","last_became_peered":"2024-09-10T21:45:59.839036+0000","last_unstale":"2024-09-10T21:46:00.870886+0000","last_undegraded":"2024-09-10T21:46:00.870886+0000","last_fullsized":"2024-09-10T21:46:00.870886+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:02.998584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.710338+0000","last_change":"2024-09-10T21:46:00.845798+0000","last_active":"2024-09-10T21:46:02.710338+0000","last_peered":"2024-09-10T21:46:02.710338+0000","last_clean":"2024-09-10T21:46:02.710338+0000","last_became_active":"2024-09-10T21:46:00.843962+0000","last_became_peered":"2024-09-10T21:46:00.843962+0000","last_unstale":"2024-09-10T21:46:02.710338+0000","last_undegraded":"2024-09-10T21:46:02.710338+0000","last_fullsized":"2024-09-10T21:46:02.710338+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:19.486516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"69'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.850739+0000","last_change":"2024-09-10T21:45:59.834997+0000","last_active":"2024-09-10T21:46:01.850739+0000","last_peered":"2024-09-10T21:46:01.850739+0000","last_clean":"2024-09-10T21:46:01.850739+0000","last_became_active":"2024-09-10T21:45:59.834651+0000","last_became_peered":"2024-09-10T21:45:59.834651+0000","last_unstale":"2024-09-10T21:46:01.850739+0000","last_undegraded":"2024-09-10T21:46:01.850739+0000","last_fullsized":"2024-09-10T21:46:01.850739+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:40.832467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828553+0000","last_change":"2024-09-10T21:46:00.849908+0000","last_active":"2024-09-10T21:46:01.828553+0000","last_peered":"2024-09-10T21:46:01.828553+0000","last_clean":"2024-09-10T21:46:01.828553+0000","last_became_active":"2024-09-10T21:46:00.849592+0000","last_became_peered":"2024-09-10T21:46:00.849592+0000","last_unstale":"2024-09-10T21:46:01.828553+0000","last_undegraded":"2024-09-10T21:46:01.828553+0000","last_fullsized":"2024-09-10T21:46:01.828553+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:03.394188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"69'2","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830754+0000","last_change":"2024-09-10T21:45:59.838236+0000","last_active":"2024-09-10T21:46:01.830754+0000","last_peered":"2024-09-10T21:46:01.830754+0000","last_clean":"2024-09-10T21:46:01.830754+0000","last_became_active":"2024-09-10T21:45:59.836821+0000","last_became_peered":"2024-09-10T21:45:59.836821+0000","last_unstale":"2024-09-10T21:46:01.830754+0000","last_undegraded":"2024-09-10T21:46:01.830754+0000","last_fullsized":"2024-09-10T21:46:01.830754+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:35.084439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.855541+0000","last_change":"2024-09-10T21:46:00.841820+0000","last_active":"2024-09-10T21:46:00.855541+0000","last_peered":"2024-09-10T21:46:00.855541+0000","last_clean":"2024-09-10T21:46:00.855541+0000","last_became_active":"2024-09-10T21:46:00.841524+0000","last_became_peered":"2024-09-10T21:46:00.841524+0000","last_unstale":"2024-09-10T21:46:00.855541+0000","last_undegraded":"2024-09-10T21:46:00.855541+0000","last_fullsized":"2024-09-10T21:46:00.855541+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:09.586859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"69'2","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828883+0000","last_change":"2024-09-10T21:45:59.837757+0000","last_active":"2024-09-10T21:46:01.828883+0000","last_peered":"2024-09-10T21:46:01.828883+0000","last_clean":"2024-09-10T21:46:01.828883+0000","last_became_active":"2024-09-10T21:45:59.837466+0000","last_became_peered":"2024-09-10T21:45:59.837466+0000","last_unstale":"2024-09-10T21:46:01.828883+0000","last_undegraded":"2024-09-10T21:46:01.828883+0000","last_fullsized":"2024-09-10T21:46:01.828883+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:31.504429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.831512+0000","last_change":"2024-09-10T21:46:00.846072+0000","last_active":"2024-09-10T21:46:01.831512+0000","last_peered":"2024-09-10T21:46:01.831512+0000","last_clean":"2024-09-10T21:46:01.831512+0000","last_became_active":"2024-09-10T21:46:00.845756+0000","last_became_peered":"2024-09-10T21:46:00.845756+0000","last_unstale":"2024-09-10T21:46:01.831512+0000","last_undegraded":"2024-09-10T21:46:01.831512+0000","last_fullsized":"2024-09-10T21:46:01.831512+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:37.624857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"69'1","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.858794+0000","last_change":"2024-09-10T21:45:59.830252+0000","last_active":"2024-09-10T21:46:00.858794+0000","last_peered":"2024-09-10T21:46:00.858794+0000","last_clean":"2024-09-10T21:46:00.858794+0000","last_became_active":"2024-09-10T21:45:59.829958+0000","last_became_peered":"2024-09-10T21:45:59.829958+0000","last_unstale":"2024-09-10T21:46:00.858794+0000","last_undegraded":"2024-09-10T21:46:00.858794+0000","last_fullsized":"2024-09-10T21:46:00.858794+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:00.503511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830221+0000","last_change":"2024-09-10T21:46:00.845982+0000","last_active":"2024-09-10T21:46:01.830221+0000","last_peered":"2024-09-10T21:46:01.830221+0000","last_clean":"2024-09-10T21:46:01.830221+0000","last_became_active":"2024-09-10T21:46:00.844701+0000","last_became_peered":"2024-09-10T21:46:00.844701+0000","last_unstale":"2024-09-10T21:46:01.830221+0000","last_undegraded":"2024-09-10T21:46:01.830221+0000","last_fullsized":"2024-09-10T21:46:01.830221+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:33.689497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"69'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828460+0000","last_change":"2024-09-10T21:45:59.833692+0000","last_active":"2024-09-10T21:46:01.828460+0000","last_peered":"2024-09-10T21:46:01.828460+0000","last_clean":"2024-09-10T21:46:01.828460+0000","last_became_active":"2024-09-10T21:45:59.833393+0000","last_became_peered":"2024-09-10T21:45:59.833393+0000","last_unstale":"2024-09-10T21:46:01.828460+0000","last_undegraded":"2024-09-10T21:46:01.828460+0000","last_fullsized":"2024-09-10T21:46:01.828460+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:18:28.299604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709131+0000","last_change":"2024-09-10T21:46:00.838228+0000","last_active":"2024-09-10T21:46:02.709131+0000","last_peered":"2024-09-10T21:46:02.709131+0000","last_clean":"2024-09-10T21:46:02.709131+0000","last_became_active":"2024-09-10T21:46:00.837919+0000","last_became_peered":"2024-09-10T21:46:00.837919+0000","last_unstale":"2024-09-10T21:46:02.709131+0000","last_undegraded":"2024-09-10T21:46:02.709131+0000","last_fullsized":"2024-09-10T21:46:02.709131+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:05.473475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.710686+0000","last_change":"2024-09-10T21:45:59.830360+0000","last_active":"2024-09-10T21:46:02.710686+0000","last_peered":"2024-09-10T21:46:02.710686+0000","last_clean":"2024-09-10T21:46:02.710686+0000","last_became_active":"2024-09-10T21:45:59.830049+0000","last_became_peered":"2024-09-10T21:45:59.830049+0000","last_unstale":"2024-09-10T21:46:02.710686+0000","last_undegraded":"2024-09-10T21:46:02.710686+0000","last_fullsized":"2024-09-10T21:46:02.710686+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:32.210607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.850518+0000","last_change":"2024-09-10T21:46:00.846853+0000","last_active":"2024-09-10T21:46:01.850518+0000","last_peered":"2024-09-10T21:46:01.850518+0000","last_clean":"2024-09-10T21:46:01.850518+0000","last_became_active":"2024-09-10T21:46:00.846224+0000","last_became_peered":"2024-09-10T21:46:00.846224+0000","last_unstale":"2024-09-10T21:46:01.850518+0000","last_undegraded":"2024-09-10T21:46:01.850518+0000","last_fullsized":"2024-09-10T21:46:01.850518+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:43.160809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709066+0000","last_change":"2024-09-10T21:45:59.827108+0000","last_active":"2024-09-10T21:46:02.709066+0000","last_peered":"2024-09-10T21:46:02.709066+0000","last_clean":"2024-09-10T21:46:02.709066+0000","last_became_active":"2024-09-10T21:45:59.826809+0000","last_became_peered":"2024-09-10T21:45:59.826809+0000","last_unstale":"2024-09-10T21:46:02.709066+0000","last_undegraded":"2024-09-10T21:46:02.709066+0000","last_fullsized":"2024-09-10T21:46:02.709066+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:34.953708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.831420+0000","last_change":"2024-09-10T21:46:00.858254+0000","last_active":"2024-09-10T21:46:01.831420+0000","last_peered":"2024-09-10T21:46:01.831420+0000","last_clean":"2024-09-10T21:46:01.831420+0000","last_became_active":"2024-09-10T21:46:00.857949+0000","last_became_peered":"2024-09-10T21:46:00.857949+0000","last_unstale":"2024-09-10T21:46:01.831420+0000","last_undegraded":"2024-09-10T21:46:01.831420+0000","last_fullsized":"2024-09-10T21:46:01.831420+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:05.211407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830914+0000","last_change":"2024-09-10T21:45:59.828239+0000","last_active":"2024-09-10T21:46:01.830914+0000","last_peered":"2024-09-10T21:46:01.830914+0000","last_clean":"2024-09-10T21:46:01.830914+0000","last_became_active":"2024-09-10T21:45:59.827825+0000","last_became_peered":"2024-09-10T21:45:59.827825+0000","last_unstale":"2024-09-10T21:46:01.830914+0000","last_undegraded":"2024-09-10T21:46:01.830914+0000","last_fullsized":"2024-09-10T21:46:01.830914+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:50.008776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.865057+0000","last_change":"2024-09-10T21:46:00.861904+0000","last_active":"2024-09-10T21:46:00.865057+0000","last_peered":"2024-09-10T21:46:00.865057+0000","last_clean":"2024-09-10T21:46:00.865057+0000","last_became_active":"2024-09-10T21:46:00.861641+0000","last_became_peered":"2024-09-10T21:46:00.861641+0000","last_unstale":"2024-09-10T21:46:00.865057+0000","last_undegraded":"2024-09-10T21:46:00.865057+0000","last_fullsized":"2024-09-10T21:46:00.865057+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:08.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.829259+0000","last_change":"2024-09-10T21:45:59.835170+0000","last_active":"2024-09-10T21:46:01.829259+0000","last_peered":"2024-09-10T21:46:01.829259+0000","last_clean":"2024-09-10T21:46:01.829259+0000","last_became_active":"2024-09-10T21:45:59.833573+0000","last_became_peered":"2024-09-10T21:45:59.833573+0000","last_unstale":"2024-09-10T21:46:01.829259+0000","last_undegraded":"2024-09-10T21:46:01.829259+0000","last_fullsized":"2024-09-10T21:46:01.829259+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:14.083982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828773+0000","last_change":"2024-09-10T21:46:00.841108+0000","last_active":"2024-09-10T21:46:01.828773+0000","last_peered":"2024-09-10T21:46:01.828773+0000","last_clean":"2024-09-10T21:46:01.828773+0000","last_became_active":"2024-09-10T21:46:00.840794+0000","last_became_peered":"2024-09-10T21:46:00.840794+0000","last_unstale":"2024-09-10T21:46:01.828773+0000","last_undegraded":"2024-09-10T21:46:01.828773+0000","last_fullsized":"2024-09-10T21:46:01.828773+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:02.804991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"69'3","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.863092+0000","last_change":"2024-09-10T21:45:59.835930+0000","last_active":"2024-09-10T21:46:00.863092+0000","last_peered":"2024-09-10T21:46:00.863092+0000","last_clean":"2024-09-10T21:46:00.863092+0000","last_became_active":"2024-09-10T21:45:59.834886+0000","last_became_peered":"2024-09-10T21:45:59.834886+0000","last_unstale":"2024-09-10T21:46:00.863092+0000","last_undegraded":"2024-09-10T21:46:00.863092+0000","last_fullsized":"2024-09-10T21:46:00.863092+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:30.172190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709672+0000","last_change":"2024-09-10T21:46:00.835699+0000","last_active":"2024-09-10T21:46:02.709672+0000","last_peered":"2024-09-10T21:46:02.709672+0000","last_clean":"2024-09-10T21:46:02.709672+0000","last_became_active":"2024-09-10T21:46:00.835365+0000","last_became_peered":"2024-09-10T21:46:00.835365+0000","last_unstale":"2024-09-10T21:46:02.709672+0000","last_undegraded":"2024-09-10T21:46:02.709672+0000","last_fullsized":"2024-09-10T21:46:02.709672+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:01.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"69'6","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.711320+0000","last_change":"2024-09-10T21:45:59.832994+0000","last_active":"2024-09-10T21:46:02.711320+0000","last_peered":"2024-09-10T21:46:02.711320+0000","last_clean":"2024-09-10T21:46:02.711320+0000","last_became_active":"2024-09-10T21:45:59.832695+0000","last_became_peered":"2024-09-10T21:45:59.832695+0000","last_unstale":"2024-09-10T21:46:02.711320+0000","last_undegraded":"2024-09-10T21:46:02.711320+0000","last_fullsized":"2024-09-10T21:46:02.711320+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:36.881454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.840833+0000","last_change":"2024-09-10T21:46:00.836737+0000","last_active":"2024-09-10T21:46:00.840833+0000","last_peered":"2024-09-10T21:46:00.840833+0000","last_clean":"2024-09-10T21:46:00.840833+0000","last_became_active":"2024-09-10T21:46:00.836137+0000","last_became_peered":"2024-09-10T21:46:00.836137+0000","last_unstale":"2024-09-10T21:46:00.840833+0000","last_undegraded":"2024-09-10T21:46:00.840833+0000","last_fullsized":"2024-09-10T21:46:00.840833+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:47.277936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"69'2","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.711080+0000","last_change":"2024-09-10T21:45:59.834396+0000","last_active":"2024-09-10T21:46:02.711080+0000","last_peered":"2024-09-10T21:46:02.711080+0000","last_clean":"2024-09-10T21:46:02.711080+0000","last_became_active":"2024-09-10T21:45:59.834132+0000","last_became_peered":"2024-09-10T21:45:59.834132+0000","last_unstale":"2024-09-10T21:46:02.711080+0000","last_undegraded":"2024-09-10T21:46:02.711080+0000","last_fullsized":"2024-09-10T21:46:02.711080+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:23.405630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.831242+0000","last_change":"2024-09-10T21:46:00.854001+0000","last_active":"2024-09-10T21:46:01.831242+0000","last_peered":"2024-09-10T21:46:01.831242+0000","last_clean":"2024-09-10T21:46:01.831242+0000","last_became_active":"2024-09-10T21:46:00.853583+0000","last_became_peered":"2024-09-10T21:46:00.853583+0000","last_unstale":"2024-09-10T21:46:01.831242+0000","last_undegraded":"2024-09-10T21:46:01.831242+0000","last_fullsized":"2024-09-10T21:46:01.831242+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:30.666063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828564+0000","last_change":"2024-09-10T21:45:59.838484+0000","last_active":"2024-09-10T21:46:01.828564+0000","last_peered":"2024-09-10T21:46:01.828564+0000","last_clean":"2024-09-10T21:46:01.828564+0000","last_became_active":"2024-09-10T21:45:59.838214+0000","last_became_peered":"2024-09-10T21:45:59.838214+0000","last_unstale":"2024-09-10T21:46:01.828564+0000","last_undegraded":"2024-09-10T21:46:01.828564+0000","last_fullsized":"2024-09-10T21:46:01.828564+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:50.361968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.850424+0000","last_change":"2024-09-10T21:46:00.840558+0000","last_active":"2024-09-10T21:46:01.850424+0000","last_peered":"2024-09-10T21:46:01.850424+0000","last_clean":"2024-09-10T21:46:01.850424+0000","last_became_active":"2024-09-10T21:46:00.840246+0000","last_became_peered":"2024-09-10T21:46:00.840246+0000","last_unstale":"2024-09-10T21:46:01.850424+0000","last_undegraded":"2024-09-10T21:46:01.850424+0000","last_fullsized":"2024-09-10T21:46:01.850424+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:12.588863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830943+0000","last_change":"2024-09-10T21:45:59.827139+0000","last_active":"2024-09-10T21:46:01.830943+0000","last_peered":"2024-09-10T21:46:01.830943+0000","last_clean":"2024-09-10T21:46:01.830943+0000","last_became_active":"2024-09-10T21:45:59.826823+0000","last_became_peered":"2024-09-10T21:45:59.826823+0000","last_unstale":"2024-09-10T21:46:01.830943+0000","last_undegraded":"2024-09-10T21:46:01.830943+0000","last_fullsized":"2024-09-10T21:46:01.830943+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:23.851492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709190+0000","last_change":"2024-09-10T21:46:00.837215+0000","last_active":"2024-09-10T21:46:02.709190+0000","last_peered":"2024-09-10T21:46:02.709190+0000","last_clean":"2024-09-10T21:46:02.709190+0000","last_became_active":"2024-09-10T21:46:00.836728+0000","last_became_peered":"2024-09-10T21:46:00.836728+0000","last_unstale":"2024-09-10T21:46:02.709190+0000","last_undegraded":"2024-09-10T21:46:02.709190+0000","last_fullsized":"2024-09-10T21:46:02.709190+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:40:23.589785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"69'8","reported_seq":23,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.829971+0000","last_change":"2024-09-10T21:45:59.831460+0000","last_active":"2024-09-10T21:46:01.829971+0000","last_peered":"2024-09-10T21:46:01.829971+0000","last_clean":"2024-09-10T21:46:01.829971+0000","last_became_active":"2024-09-10T21:45:59.831161+0000","last_became_peered":"2024-09-10T21:45:59.831161+0000","last_unstale":"2024-09-10T21:46:01.829971+0000","last_undegraded":"2024-09-10T21:46:01.829971+0000","last_fullsized":"2024-09-10T21:46:01.829971+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:02.348733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830792+0000","last_change":"2024-09-10T21:46:00.838850+0000","last_active":"2024-09-10T21:46:01.830792+0000","last_peered":"2024-09-10T21:46:01.830792+0000","last_clean":"2024-09-10T21:46:01.830792+0000","last_became_active":"2024-09-10T21:46:00.838462+0000","last_became_peered":"2024-09-10T21:46:00.838462+0000","last_unstale":"2024-09-10T21:46:01.830792+0000","last_undegraded":"2024-09-10T21:46:01.830792+0000","last_fullsized":"2024-09-10T21:46:01.830792+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:14.199837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"69'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.829086+0000","last_change":"2024-09-10T21:45:59.837920+0000","last_active":"2024-09-10T21:46:01.829086+0000","last_peered":"2024-09-10T21:46:01.829086+0000","last_clean":"2024-09-10T21:46:01.829086+0000","last_became_active":"2024-09-10T21:45:59.837372+0000","last_became_peered":"2024-09-10T21:45:59.837372+0000","last_unstale":"2024-09-10T21:46:01.829086+0000","last_undegraded":"2024-09-10T21:46:01.829086+0000","last_fullsized":"2024-09-10T21:46:01.829086+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:17.497947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.857032+0000","last_change":"2024-09-10T21:46:00.843932+0000","last_active":"2024-09-10T21:46:00.857032+0000","last_peered":"2024-09-10T21:46:00.857032+0000","last_clean":"2024-09-10T21:46:00.857032+0000","last_became_active":"2024-09-10T21:46:00.843650+0000","last_became_peered":"2024-09-10T21:46:00.843650+0000","last_unstale":"2024-09-10T21:46:00.857032+0000","last_undegraded":"2024-09-10T21:46:00.857032+0000","last_fullsized":"2024-09-10T21:46:00.857032+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:51.932442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"69'2","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830711+0000","last_change":"2024-09-10T21:45:59.834683+0000","last_active":"2024-09-10T21:46:01.830711+0000","last_peered":"2024-09-10T21:46:01.830711+0000","last_clean":"2024-09-10T21:46:01.830711+0000","last_became_active":"2024-09-10T21:45:59.834398+0000","last_became_peered":"2024-09-10T21:45:59.834398+0000","last_unstale":"2024-09-10T21:46:01.830711+0000","last_undegraded":"2024-09-10T21:46:01.830711+0000","last_fullsized":"2024-09-10T21:46:01.830711+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:21.733557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.711204+0000","last_change":"2024-09-10T21:46:00.843778+0000","last_active":"2024-09-10T21:46:02.711204+0000","last_peered":"2024-09-10T21:46:02.711204+0000","last_clean":"2024-09-10T21:46:02.711204+0000","last_became_active":"2024-09-10T21:46:00.843455+0000","last_became_peered":"2024-09-10T21:46:00.843455+0000","last_unstale":"2024-09-10T21:46:02.711204+0000","last_undegraded":"2024-09-10T21:46:02.711204+0000","last_fullsized":"2024-09-10T21:46:02.711204+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:32.511194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"69'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828797+0000","last_change":"2024-09-10T21:45:59.835900+0000","last_active":"2024-09-10T21:46:01.828797+0000","last_peered":"2024-09-10T21:46:01.828797+0000","last_clean":"2024-09-10T21:46:01.828797+0000","last_became_active":"2024-09-10T21:45:59.835055+0000","last_became_peered":"2024-09-10T21:45:59.835055+0000","last_unstale":"2024-09-10T21:46:01.828797+0000","last_undegraded":"2024-09-10T21:46:01.828797+0000","last_fullsized":"2024-09-10T21:46:01.828797+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:18.497451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.710563+0000","last_change":"2024-09-10T21:46:00.838328+0000","last_active":"2024-09-10T21:46:02.710563+0000","last_peered":"2024-09-10T21:46:02.710563+0000","last_clean":"2024-09-10T21:46:02.710563+0000","last_became_active":"2024-09-10T21:46:00.838034+0000","last_became_peered":"2024-09-10T21:46:00.838034+0000","last_unstale":"2024-09-10T21:46:02.710563+0000","last_undegraded":"2024-09-10T21:46:02.710563+0000","last_fullsized":"2024-09-10T21:46:02.710563+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:51.514167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.827043+0000","last_change":"2024-09-10T21:45:59.836594+0000","last_active":"2024-09-10T21:46:00.827043+0000","last_peered":"2024-09-10T21:46:00.827043+0000","last_clean":"2024-09-10T21:46:00.827043+0000","last_became_active":"2024-09-10T21:45:59.836325+0000","last_became_peered":"2024-09-10T21:45:59.836325+0000","last_unstale":"2024-09-10T21:46:00.827043+0000","last_undegraded":"2024-09-10T21:46:00.827043+0000","last_fullsized":"2024-09-10T21:46:00.827043+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:19:39.295788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.844196+0000","last_change":"2024-09-10T21:46:00.837935+0000","last_active":"2024-09-10T21:46:00.844196+0000","last_peered":"2024-09-10T21:46:00.844196+0000","last_clean":"2024-09-10T21:46:00.844196+0000","last_became_active":"2024-09-10T21:46:00.837645+0000","last_became_peered":"2024-09-10T21:46:00.837645+0000","last_unstale":"2024-09-10T21:46:00.844196+0000","last_undegraded":"2024-09-10T21:46:00.844196+0000","last_fullsized":"2024-09-10T21:46:00.844196+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:13.337873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828666+0000","last_change":"2024-09-10T21:45:59.833884+0000","last_active":"2024-09-10T21:46:01.828666+0000","last_peered":"2024-09-10T21:46:01.828666+0000","last_clean":"2024-09-10T21:46:01.828666+0000","last_became_active":"2024-09-10T21:45:59.833348+0000","last_became_peered":"2024-09-10T21:45:59.833348+0000","last_unstale":"2024-09-10T21:46:01.828666+0000","last_undegraded":"2024-09-10T21:46:01.828666+0000","last_fullsized":"2024-09-10T21:46:01.828666+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:24.226309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.870492+0000","last_change":"2024-09-10T21:46:00.869022+0000","last_active":"2024-09-10T21:46:00.870492+0000","last_peered":"2024-09-10T21:46:00.870492+0000","last_clean":"2024-09-10T21:46:00.870492+0000","last_became_active":"2024-09-10T21:46:00.868103+0000","last_became_peered":"2024-09-10T21:46:00.868103+0000","last_unstale":"2024-09-10T21:46:00.870492+0000","last_undegraded":"2024-09-10T21:46:00.870492+0000","last_fullsized":"2024-09-10T21:46:00.870492+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:40.002164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.831360+0000","last_change":"2024-09-10T21:45:59.830947+0000","last_active":"2024-09-10T21:46:01.831360+0000","last_peered":"2024-09-10T21:46:01.831360+0000","last_clean":"2024-09-10T21:46:01.831360+0000","last_became_active":"2024-09-10T21:45:59.829692+0000","last_became_peered":"2024-09-10T21:45:59.829692+0000","last_unstale":"2024-09-10T21:46:01.831360+0000","last_undegraded":"2024-09-10T21:46:01.831360+0000","last_fullsized":"2024-09-10T21:46:01.831360+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:36.232207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.863457+0000","last_change":"2024-09-10T21:46:00.862041+0000","last_active":"2024-09-10T21:46:00.863457+0000","last_peered":"2024-09-10T21:46:00.863457+0000","last_clean":"2024-09-10T21:46:00.863457+0000","last_became_active":"2024-09-10T21:46:00.861716+0000","last_became_peered":"2024-09-10T21:46:00.861716+0000","last_unstale":"2024-09-10T21:46:00.863457+0000","last_undegraded":"2024-09-10T21:46:00.863457+0000","last_fullsized":"2024-09-10T21:46:00.863457+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:40.331928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.850307+0000","last_change":"2024-09-10T21:45:59.835947+0000","last_active":"2024-09-10T21:46:01.850307+0000","last_peered":"2024-09-10T21:46:01.850307+0000","last_clean":"2024-09-10T21:46:01.850307+0000","last_became_active":"2024-09-10T21:45:59.835118+0000","last_became_peered":"2024-09-10T21:45:59.835118+0000","last_unstale":"2024-09-10T21:46:01.850307+0000","last_undegraded":"2024-09-10T21:46:01.850307+0000","last_fullsized":"2024-09-10T21:46:01.850307+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:38.399726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.850725+0000","last_change":"2024-09-10T21:46:00.846564+0000","last_active":"2024-09-10T21:46:01.850725+0000","last_peered":"2024-09-10T21:46:01.850725+0000","last_clean":"2024-09-10T21:46:01.850725+0000","last_became_active":"2024-09-10T21:46:00.846321+0000","last_became_peered":"2024-09-10T21:46:00.846321+0000","last_unstale":"2024-09-10T21:46:01.850725+0000","last_undegraded":"2024-09-10T21:46:01.850725+0000","last_fullsized":"2024-09-10T21:46:01.850725+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:35.510698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"69'2","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.829187+0000","last_change":"2024-09-10T21:45:59.834909+0000","last_active":"2024-09-10T21:46:01.829187+0000","last_peered":"2024-09-10T21:46:01.829187+0000","last_clean":"2024-09-10T21:46:01.829187+0000","last_became_active":"2024-09-10T21:45:59.834603+0000","last_became_peered":"2024-09-10T21:45:59.834603+0000","last_unstale":"2024-09-10T21:46:01.829187+0000","last_undegraded":"2024-09-10T21:46:01.829187+0000","last_fullsized":"2024-09-10T21:46:01.829187+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:04.899566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.831273+0000","last_change":"2024-09-10T21:46:00.855694+0000","last_active":"2024-09-10T21:46:01.831273+0000","last_peered":"2024-09-10T21:46:01.831273+0000","last_clean":"2024-09-10T21:46:01.831273+0000","last_became_active":"2024-09-10T21:46:00.853229+0000","last_became_peered":"2024-09-10T21:46:00.853229+0000","last_unstale":"2024-09-10T21:46:01.831273+0000","last_undegraded":"2024-09-10T21:46:01.831273+0000","last_fullsized":"2024-09-10T21:46:01.831273+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:01:31.844366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"70'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.849822+0000","last_change":"2024-09-10T21:45:59.828584+0000","last_active":"2024-09-10T21:46:01.849822+0000","last_peered":"2024-09-10T21:46:01.849822+0000","last_clean":"2024-09-10T21:46:01.849822+0000","last_became_active":"2024-09-10T21:45:59.827977+0000","last_became_peered":"2024-09-10T21:45:59.827977+0000","last_unstale":"2024-09-10T21:46:01.849822+0000","last_undegraded":"2024-09-10T21:46:01.849822+0000","last_fullsized":"2024-09-10T21:46:01.849822+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:01.798226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.864657+0000","last_change":"2024-09-10T21:46:00.862536+0000","last_active":"2024-09-10T21:46:00.864657+0000","last_peered":"2024-09-10T21:46:00.864657+0000","last_clean":"2024-09-10T21:46:00.864657+0000","last_became_active":"2024-09-10T21:46:00.862244+0000","last_became_peered":"2024-09-10T21:46:00.862244+0000","last_unstale":"2024-09-10T21:46:00.864657+0000","last_undegraded":"2024-09-10T21:46:00.864657+0000","last_fullsized":"2024-09-10T21:46:00.864657+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:33.601733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709884+0000","last_change":"2024-09-10T21:45:59.830095+0000","last_active":"2024-09-10T21:46:02.709884+0000","last_peered":"2024-09-10T21:46:02.709884+0000","last_clean":"2024-09-10T21:46:02.709884+0000","last_became_active":"2024-09-10T21:45:59.829796+0000","last_became_peered":"2024-09-10T21:45:59.829796+0000","last_unstale":"2024-09-10T21:46:02.709884+0000","last_undegraded":"2024-09-10T21:46:02.709884+0000","last_fullsized":"2024-09-10T21:46:02.709884+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:02.704482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.858295+0000","last_change":"2024-09-10T21:46:00.855637+0000","last_active":"2024-09-10T21:46:00.858295+0000","last_peered":"2024-09-10T21:46:00.858295+0000","last_clean":"2024-09-10T21:46:00.858295+0000","last_became_active":"2024-09-10T21:46:00.855388+0000","last_became_peered":"2024-09-10T21:46:00.855388+0000","last_unstale":"2024-09-10T21:46:00.858295+0000","last_undegraded":"2024-09-10T21:46:00.858295+0000","last_fullsized":"2024-09-10T21:46:00.858295+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:48.018328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.850547+0000","last_change":"2024-09-10T21:45:59.826722+0000","last_active":"2024-09-10T21:46:01.850547+0000","last_peered":"2024-09-10T21:46:01.850547+0000","last_clean":"2024-09-10T21:46:01.850547+0000","last_became_active":"2024-09-10T21:45:59.826436+0000","last_became_peered":"2024-09-10T21:45:59.826436+0000","last_unstale":"2024-09-10T21:46:01.850547+0000","last_undegraded":"2024-09-10T21:46:01.850547+0000","last_fullsized":"2024-09-10T21:46:01.850547+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:11.996778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.710082+0000","last_change":"2024-09-10T21:46:00.839723+0000","last_active":"2024-09-10T21:46:02.710082+0000","last_peered":"2024-09-10T21:46:02.710082+0000","last_clean":"2024-09-10T21:46:02.710082+0000","last_became_active":"2024-09-10T21:46:00.839475+0000","last_became_peered":"2024-09-10T21:46:00.839475+0000","last_unstale":"2024-09-10T21:46:02.710082+0000","last_undegraded":"2024-09-10T21:46:02.710082+0000","last_fullsized":"2024-09-10T21:46:02.710082+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:31.317327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709825+0000","last_change":"2024-09-10T21:45:59.821286+0000","last_active":"2024-09-10T21:46:02.709825+0000","last_peered":"2024-09-10T21:46:02.709825+0000","last_clean":"2024-09-10T21:46:02.709825+0000","last_became_active":"2024-09-10T21:45:59.820999+0000","last_became_peered":"2024-09-10T21:45:59.820999+0000","last_unstale":"2024-09-10T21:46:02.709825+0000","last_undegraded":"2024-09-10T21:46:02.709825+0000","last_fullsized":"2024-09-10T21:46:02.709825+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:35:51.376484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828127+0000","last_change":"2024-09-10T21:46:00.849338+0000","last_active":"2024-09-10T21:46:01.828127+0000","last_peered":"2024-09-10T21:46:01.828127+0000","last_clean":"2024-09-10T21:46:01.828127+0000","last_became_active":"2024-09-10T21:46:00.848819+0000","last_became_peered":"2024-09-10T21:46:00.848819+0000","last_unstale":"2024-09-10T21:46:01.828127+0000","last_undegraded":"2024-09-10T21:46:01.828127+0000","last_fullsized":"2024-09-10T21:46:01.828127+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:05.714091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.710702+0000","last_change":"2024-09-10T21:45:59.832047+0000","last_active":"2024-09-10T21:46:02.710702+0000","last_peered":"2024-09-10T21:46:02.710702+0000","last_clean":"2024-09-10T21:46:02.710702+0000","last_became_active":"2024-09-10T21:45:59.831762+0000","last_became_peered":"2024-09-10T21:45:59.831762+0000","last_unstale":"2024-09-10T21:46:02.710702+0000","last_undegraded":"2024-09-10T21:46:02.710702+0000","last_fullsized":"2024-09-10T21:46:02.710702+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:36.934813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709755+0000","last_change":"2024-09-10T21:46:00.835798+0000","last_active":"2024-09-10T21:46:02.709755+0000","last_peered":"2024-09-10T21:46:02.709755+0000","last_clean":"2024-09-10T21:46:02.709755+0000","last_became_active":"2024-09-10T21:46:00.835507+0000","last_became_peered":"2024-09-10T21:46:00.835507+0000","last_unstale":"2024-09-10T21:46:02.709755+0000","last_undegraded":"2024-09-10T21:46:02.709755+0000","last_fullsized":"2024-09-10T21:46:02.709755+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:15.090747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.829884+0000","last_change":"2024-09-10T21:45:59.830690+0000","last_active":"2024-09-10T21:46:01.829884+0000","last_peered":"2024-09-10T21:46:01.829884+0000","last_clean":"2024-09-10T21:46:01.829884+0000","last_became_active":"2024-09-10T21:45:59.830407+0000","last_became_peered":"2024-09-10T21:45:59.830407+0000","last_unstale":"2024-09-10T21:46:01.829884+0000","last_undegraded":"2024-09-10T21:46:01.829884+0000","last_fullsized":"2024-09-10T21:46:01.829884+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:31.228948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.827918+0000","last_change":"2024-09-10T21:46:00.847377+0000","last_active":"2024-09-10T21:46:01.827918+0000","last_peered":"2024-09-10T21:46:01.827918+0000","last_clean":"2024-09-10T21:46:01.827918+0000","last_became_active":"2024-09-10T21:46:00.847047+0000","last_became_peered":"2024-09-10T21:46:00.847047+0000","last_unstale":"2024-09-10T21:46:01.827918+0000","last_undegraded":"2024-09-10T21:46:01.827918+0000","last_fullsized":"2024-09-10T21:46:01.827918+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:59.158149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828257+0000","last_change":"2024-09-10T21:45:59.836973+0000","last_active":"2024-09-10T21:46:01.828257+0000","last_peered":"2024-09-10T21:46:01.828257+0000","last_clean":"2024-09-10T21:46:01.828257+0000","last_became_active":"2024-09-10T21:45:59.836688+0000","last_became_peered":"2024-09-10T21:45:59.836688+0000","last_unstale":"2024-09-10T21:46:01.828257+0000","last_undegraded":"2024-09-10T21:46:01.828257+0000","last_fullsized":"2024-09-10T21:46:01.828257+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:10:58.722699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.863459+0000","last_change":"2024-09-10T21:46:00.862053+0000","last_active":"2024-09-10T21:46:00.863459+0000","last_peered":"2024-09-10T21:46:00.863459+0000","last_clean":"2024-09-10T21:46:00.863459+0000","last_became_active":"2024-09-10T21:46:00.861698+0000","last_became_peered":"2024-09-10T21:46:00.861698+0000","last_unstale":"2024-09-10T21:46:00.863459+0000","last_undegraded":"2024-09-10T21:46:00.863459+0000","last_fullsized":"2024-09-10T21:46:00.863459+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:58.548218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"69'2","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828079+0000","last_change":"2024-09-10T21:45:59.836987+0000","last_active":"2024-09-10T21:46:01.828079+0000","last_peered":"2024-09-10T21:46:01.828079+0000","last_clean":"2024-09-10T21:46:01.828079+0000","last_became_active":"2024-09-10T21:45:59.836711+0000","last_became_peered":"2024-09-10T21:45:59.836711+0000","last_unstale":"2024-09-10T21:46:01.828079+0000","last_undegraded":"2024-09-10T21:46:01.828079+0000","last_fullsized":"2024-09-10T21:46:01.828079+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:51.311564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":198,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.826392+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:46:00.826392+0000","last_peered":"2024-09-10T21:46:00.826392+0000","last_clean":"2024-09-10T21:46:00.826392+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:46:00.826392+0000","last_undegraded":"2024-09-10T21:46:00.826392+0000","last_fullsized":"2024-09-10T21:46:00.826392+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509493,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":851968,"data_stored":353158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509493,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":847872,"data_stored":353156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509492,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":847872,"data_stored":353136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509492,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":843776,"data_stored":348378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509494,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":851968,"data_stored":348994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509492,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1437696,"data_stored":943434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542195,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1433600,"data_stored":937750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542196,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":839680,"data_stored":346878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:46:05.099 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:46:05.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.293+0000 7fe93fc00640 1 -- 172.21.15.7:0/3147173447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9380cca30 msgr2=0x7fe938074710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:05.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.293+0000 7fe93fc00640 1 --2- 172.21.15.7:0/3147173447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9380cca30 0x7fe938074710 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe938072680 tx=0x7fe92402a4d0 comp rx=0 tx=0).stop 2024-09-10T21:46:05.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.293+0000 7fe93fc00640 1 -- 172.21.15.7:0/3147173447 shutdown_connections 2024-09-10T21:46:05.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.293+0000 7fe93fc00640 1 --2- 172.21.15.7:0/3147173447 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9380cca30 0x7fe938074710 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:05.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.293+0000 7fe93fc00640 1 --2- 172.21.15.7:0/3147173447 >> v2:172.21.15.133:3300/0 conn(0x7fe93815e4c0 0x7fe9380741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:05.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.293+0000 7fe93fc00640 1 -- 172.21.15.7:0/3147173447 >> 172.21.15.7:0/3147173447 conn(0x7fe9380d0430 msgr2=0x7fe9380d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:05.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.293+0000 7fe93fc00640 1 -- 172.21.15.7:0/3147173447 shutdown_connections 2024-09-10T21:46:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.293+0000 7fe93fc00640 1 -- 172.21.15.7:0/3147173447 wait complete. 2024-09-10T21:46:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.294+0000 7fe93fc00640 1 Processor -- start 2024-09-10T21:46:05.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.294+0000 7fe93fc00640 1 -- start start 2024-09-10T21:46:05.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.295+0000 7fe93fc00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9380af570 0x7fe938099930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:05.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.295+0000 7fe93fc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9380cca30 0x7fe9380c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:05.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.295+0000 7fe93fc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe93815e4c0 0x7fe9380c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:05.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.295+0000 7fe93fc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe9380b0c50 con 0x7fe9380cca30 2024-09-10T21:46:05.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.295+0000 7fe93fc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe9380b0ad0 con 0x7fe93815e4c0 2024-09-10T21:46:05.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.295+0000 7fe93fc00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe93815f1d0 con 0x7fe9380af570 2024-09-10T21:46:05.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.296+0000 7fe937e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9380cca30 0x7fe9380c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:05.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.296+0000 7fe937e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9380cca30 0x7fe9380c5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44792/0 (socket says 172.21.15.7:44792) 2024-09-10T21:46:05.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.296+0000 7fe937e00640 1 -- 172.21.15.7:0/718125346 learned_addr learned my addr 172.21.15.7:0/718125346 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:05.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.296+0000 7fe93d200640 1 --2- 172.21.15.7:0/718125346 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9380af570 0x7fe938099930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:05.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.296+0000 7fe93dc00640 1 --2- 172.21.15.7:0/718125346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe93815e4c0 0x7fe9380c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:05.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.296+0000 7fe937e00640 1 -- 172.21.15.7:0/718125346 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9380af570 msgr2=0x7fe938099930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:05.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.296+0000 7fe937e00640 1 --2- 172.21.15.7:0/718125346 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9380af570 0x7fe938099930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:05.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.296+0000 7fe937e00640 1 -- 172.21.15.7:0/718125346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe93815e4c0 msgr2=0x7fe9380c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:05.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.296+0000 7fe937e00640 1 --2- 172.21.15.7:0/718125346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe93815e4c0 0x7fe9380c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:05.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.296+0000 7fe937e00640 1 -- 172.21.15.7:0/718125346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9380c5f90 con 0x7fe9380cca30 2024-09-10T21:46:05.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.296+0000 7fe937e00640 1 --2- 172.21.15.7:0/718125346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9380cca30 0x7fe9380c5680 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe9240099c0 tx=0x7fe9240039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:05.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.297+0000 7fe934c00640 1 -- 172.21.15.7:0/718125346 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92403f070 con 0x7fe9380cca30 2024-09-10T21:46:05.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.297+0000 7fe934c00640 1 -- 172.21.15.7:0/718125346 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe924004890 con 0x7fe9380cca30 2024-09-10T21:46:05.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.297+0000 7fe934c00640 1 -- 172.21.15.7:0/718125346 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924035030 con 0x7fe9380cca30 2024-09-10T21:46:05.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.297+0000 7fe93fc00640 1 -- 172.21.15.7:0/718125346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9380c40e0 con 0x7fe9380cca30 2024-09-10T21:46:05.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.297+0000 7fe93fc00640 1 -- 172.21.15.7:0/718125346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9380c46a0 con 0x7fe9380cca30 2024-09-10T21:46:05.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.298+0000 7fe93fc00640 1 -- 172.21.15.7:0/718125346 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe938050ee0 con 0x7fe9380cca30 2024-09-10T21:46:05.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.299+0000 7fe934c00640 1 -- 172.21.15.7:0/718125346 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe924004ce0 con 0x7fe9380cca30 2024-09-10T21:46:05.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.300+0000 7fe934c00640 1 --2- 172.21.15.7:0/718125346 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe918077f70 0x7fe91807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:05.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.300+0000 7fe934c00640 1 -- 172.21.15.7:0/718125346 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fe9240bae00 con 0x7fe9380cca30 2024-09-10T21:46:05.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.301+0000 7fe93d200640 1 --2- 172.21.15.7:0/718125346 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe918077f70 0x7fe91807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:05.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.301+0000 7fe93d200640 1 --2- 172.21.15.7:0/718125346 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe918077f70 0x7fe91807a430 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe938070550 tx=0x7fe92c00b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:05.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.304+0000 7fe934c00640 1 -- 172.21.15.7:0/718125346 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe938050ee0 con 0x7fe9380cca30 2024-09-10T21:46:05.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.493+0000 7fe93fc00640 1 -- 172.21.15.7:0/718125346 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe9380c6270 con 0x7fe918077f70 2024-09-10T21:46:05.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.498+0000 7fe934c00640 1 -- 172.21.15.7:0/718125346 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252053 (secure 0 0 0) 0x7fe9380c6270 con 0x7fe918077f70 2024-09-10T21:46:05.504 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:46:05.505 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:46:05.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.503+0000 7fe93fc00640 1 -- 172.21.15.7:0/718125346 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe918077f70 msgr2=0x7fe91807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:05.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.503+0000 7fe93fc00640 1 --2- 172.21.15.7:0/718125346 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe918077f70 0x7fe91807a430 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe938070550 tx=0x7fe92c00b2e0 comp rx=0 tx=0).stop 2024-09-10T21:46:05.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.503+0000 7fe93fc00640 1 -- 172.21.15.7:0/718125346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9380cca30 msgr2=0x7fe9380c5680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:05.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.503+0000 7fe93fc00640 1 --2- 172.21.15.7:0/718125346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9380cca30 0x7fe9380c5680 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe9240099c0 tx=0x7fe9240039e0 comp rx=0 tx=0).stop 2024-09-10T21:46:05.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.503+0000 7fe93fc00640 1 -- 172.21.15.7:0/718125346 shutdown_connections 2024-09-10T21:46:05.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.503+0000 7fe93fc00640 1 --2- 172.21.15.7:0/718125346 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe918077f70 0x7fe91807a430 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:05.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.503+0000 7fe93fc00640 1 --2- 172.21.15.7:0/718125346 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe93815e4c0 0x7fe9380c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:05.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.503+0000 7fe93fc00640 1 --2- 172.21.15.7:0/718125346 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9380cca30 0x7fe9380c5680 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:05.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.503+0000 7fe93fc00640 1 --2- 172.21.15.7:0/718125346 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe9380af570 0x7fe938099930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:05.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.503+0000 7fe93fc00640 1 -- 172.21.15.7:0/718125346 >> 172.21.15.7:0/718125346 conn(0x7fe9380d0430 msgr2=0x7fe93806f0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:05.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.503+0000 7fe93fc00640 1 -- 172.21.15.7:0/718125346 shutdown_connections 2024-09-10T21:46:05.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.503+0000 7fe93fc00640 1 -- 172.21.15.7:0/718125346 wait complete. 2024-09-10T21:46:05.520 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":187,"stamp":"2024-09-10T21:46:03.916000+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":74,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257320,"kb_used_data":7768,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717400,"statfs":{"total":773094113280,"available":772830617600,"internally_reserved":0,"allocated":7954432,"data_stored":3984884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.105338"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.857623+0000","last_change":"2024-09-10T21:46:00.845405+0000","last_active":"2024-09-10T21:46:00.857623+0000","last_peered":"2024-09-10T21:46:00.857623+0000","last_clean":"2024-09-10T21:46:00.857623+0000","last_became_active":"2024-09-10T21:46:00.845093+0000","last_became_peered":"2024-09-10T21:46:00.845093+0000","last_unstale":"2024-09-10T21:46:00.857623+0000","last_undegraded":"2024-09-10T21:46:00.857623+0000","last_fullsized":"2024-09-10T21:46:00.857623+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:01:48.977667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.829269+0000","last_change":"2024-09-10T21:46:00.839425+0000","last_active":"2024-09-10T21:46:01.829269+0000","last_peered":"2024-09-10T21:46:01.829269+0000","last_clean":"2024-09-10T21:46:01.829269+0000","last_became_active":"2024-09-10T21:46:00.839055+0000","last_became_peered":"2024-09-10T21:46:00.839055+0000","last_unstale":"2024-09-10T21:46:01.829269+0000","last_undegraded":"2024-09-10T21:46:01.829269+0000","last_fullsized":"2024-09-10T21:46:01.829269+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:37.238972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.845162+0000","last_change":"2024-09-10T21:46:00.841041+0000","last_active":"2024-09-10T21:46:00.845162+0000","last_peered":"2024-09-10T21:46:00.845162+0000","last_clean":"2024-09-10T21:46:00.845162+0000","last_became_active":"2024-09-10T21:46:00.840517+0000","last_became_peered":"2024-09-10T21:46:00.840517+0000","last_unstale":"2024-09-10T21:46:00.845162+0000","last_undegraded":"2024-09-10T21:46:00.845162+0000","last_fullsized":"2024-09-10T21:46:00.845162+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:19.662566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.710960+0000","last_change":"2024-09-10T21:46:00.841498+0000","last_active":"2024-09-10T21:46:02.710960+0000","last_peered":"2024-09-10T21:46:02.710960+0000","last_clean":"2024-09-10T21:46:02.710960+0000","last_became_active":"2024-09-10T21:46:00.841104+0000","last_became_peered":"2024-09-10T21:46:00.841104+0000","last_unstale":"2024-09-10T21:46:02.710960+0000","last_undegraded":"2024-09-10T21:46:02.710960+0000","last_fullsized":"2024-09-10T21:46:02.710960+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:25.233074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709731+0000","last_change":"2024-09-10T21:46:00.840826+0000","last_active":"2024-09-10T21:46:02.709731+0000","last_peered":"2024-09-10T21:46:02.709731+0000","last_clean":"2024-09-10T21:46:02.709731+0000","last_became_active":"2024-09-10T21:46:00.839302+0000","last_became_peered":"2024-09-10T21:46:00.839302+0000","last_unstale":"2024-09-10T21:46:02.709731+0000","last_undegraded":"2024-09-10T21:46:02.709731+0000","last_fullsized":"2024-09-10T21:46:02.709731+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:10.316342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828592+0000","last_change":"2024-09-10T21:46:00.854710+0000","last_active":"2024-09-10T21:46:01.828592+0000","last_peered":"2024-09-10T21:46:01.828592+0000","last_clean":"2024-09-10T21:46:01.828592+0000","last_became_active":"2024-09-10T21:46:00.854455+0000","last_became_peered":"2024-09-10T21:46:00.854455+0000","last_unstale":"2024-09-10T21:46:01.828592+0000","last_undegraded":"2024-09-10T21:46:01.828592+0000","last_fullsized":"2024-09-10T21:46:01.828592+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:19:51.897259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830917+0000","last_change":"2024-09-10T21:46:00.848967+0000","last_active":"2024-09-10T21:46:01.830917+0000","last_peered":"2024-09-10T21:46:01.830917+0000","last_clean":"2024-09-10T21:46:01.830917+0000","last_became_active":"2024-09-10T21:46:00.848450+0000","last_became_peered":"2024-09-10T21:46:00.848450+0000","last_unstale":"2024-09-10T21:46:01.830917+0000","last_undegraded":"2024-09-10T21:46:01.830917+0000","last_fullsized":"2024-09-10T21:46:01.830917+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:56.519958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828082+0000","last_change":"2024-09-10T21:46:00.844463+0000","last_active":"2024-09-10T21:46:01.828082+0000","last_peered":"2024-09-10T21:46:01.828082+0000","last_clean":"2024-09-10T21:46:01.828082+0000","last_became_active":"2024-09-10T21:46:00.841533+0000","last_became_peered":"2024-09-10T21:46:00.841533+0000","last_unstale":"2024-09-10T21:46:01.828082+0000","last_undegraded":"2024-09-10T21:46:01.828082+0000","last_fullsized":"2024-09-10T21:46:01.828082+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:15:29.186264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709860+0000","last_change":"2024-09-10T21:46:00.844670+0000","last_active":"2024-09-10T21:46:02.709860+0000","last_peered":"2024-09-10T21:46:02.709860+0000","last_clean":"2024-09-10T21:46:02.709860+0000","last_became_active":"2024-09-10T21:46:00.844340+0000","last_became_peered":"2024-09-10T21:46:00.844340+0000","last_unstale":"2024-09-10T21:46:02.709860+0000","last_undegraded":"2024-09-10T21:46:02.709860+0000","last_fullsized":"2024-09-10T21:46:02.709860+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:10.664153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830583+0000","last_change":"2024-09-10T21:46:00.845549+0000","last_active":"2024-09-10T21:46:01.830583+0000","last_peered":"2024-09-10T21:46:01.830583+0000","last_clean":"2024-09-10T21:46:01.830583+0000","last_became_active":"2024-09-10T21:46:00.845289+0000","last_became_peered":"2024-09-10T21:46:00.845289+0000","last_unstale":"2024-09-10T21:46:01.830583+0000","last_undegraded":"2024-09-10T21:46:01.830583+0000","last_fullsized":"2024-09-10T21:46:01.830583+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:11:34.031166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.857698+0000","last_change":"2024-09-10T21:46:00.846160+0000","last_active":"2024-09-10T21:46:00.857698+0000","last_peered":"2024-09-10T21:46:00.857698+0000","last_clean":"2024-09-10T21:46:00.857698+0000","last_became_active":"2024-09-10T21:46:00.845817+0000","last_became_peered":"2024-09-10T21:46:00.845817+0000","last_unstale":"2024-09-10T21:46:00.857698+0000","last_undegraded":"2024-09-10T21:46:00.857698+0000","last_fullsized":"2024-09-10T21:46:00.857698+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:04.317280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.710991+0000","last_change":"2024-09-10T21:46:00.843787+0000","last_active":"2024-09-10T21:46:02.710991+0000","last_peered":"2024-09-10T21:46:02.710991+0000","last_clean":"2024-09-10T21:46:02.710991+0000","last_became_active":"2024-09-10T21:46:00.843497+0000","last_became_peered":"2024-09-10T21:46:00.843497+0000","last_unstale":"2024-09-10T21:46:02.710991+0000","last_undegraded":"2024-09-10T21:46:02.710991+0000","last_fullsized":"2024-09-10T21:46:02.710991+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:19.504216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.850517+0000","last_change":"2024-09-10T21:46:00.839289+0000","last_active":"2024-09-10T21:46:01.850517+0000","last_peered":"2024-09-10T21:46:01.850517+0000","last_clean":"2024-09-10T21:46:01.850517+0000","last_became_active":"2024-09-10T21:46:00.839002+0000","last_became_peered":"2024-09-10T21:46:00.839002+0000","last_unstale":"2024-09-10T21:46:01.850517+0000","last_undegraded":"2024-09-10T21:46:01.850517+0000","last_fullsized":"2024-09-10T21:46:01.850517+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:37.624857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709810+0000","last_change":"2024-09-10T21:46:00.854653+0000","last_active":"2024-09-10T21:46:02.709810+0000","last_peered":"2024-09-10T21:46:02.709810+0000","last_clean":"2024-09-10T21:46:02.709810+0000","last_became_active":"2024-09-10T21:46:00.854198+0000","last_became_peered":"2024-09-10T21:46:00.854198+0000","last_unstale":"2024-09-10T21:46:02.709810+0000","last_undegraded":"2024-09-10T21:46:02.709810+0000","last_fullsized":"2024-09-10T21:46:02.709810+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:24.506727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.850511+0000","last_change":"2024-09-10T21:46:00.847153+0000","last_active":"2024-09-10T21:46:01.850511+0000","last_peered":"2024-09-10T21:46:01.850511+0000","last_clean":"2024-09-10T21:46:01.850511+0000","last_became_active":"2024-09-10T21:46:00.846597+0000","last_became_peered":"2024-09-10T21:46:00.846597+0000","last_unstale":"2024-09-10T21:46:01.850511+0000","last_undegraded":"2024-09-10T21:46:01.850511+0000","last_fullsized":"2024-09-10T21:46:01.850511+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:26:03.399674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.829825+0000","last_change":"2024-09-10T21:46:00.847779+0000","last_active":"2024-09-10T21:46:01.829825+0000","last_peered":"2024-09-10T21:46:01.829825+0000","last_clean":"2024-09-10T21:46:01.829825+0000","last_became_active":"2024-09-10T21:46:00.847407+0000","last_became_peered":"2024-09-10T21:46:00.847407+0000","last_unstale":"2024-09-10T21:46:01.829825+0000","last_undegraded":"2024-09-10T21:46:01.829825+0000","last_fullsized":"2024-09-10T21:46:01.829825+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:20.998836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.710109+0000","last_change":"2024-09-10T21:46:00.845442+0000","last_active":"2024-09-10T21:46:02.710109+0000","last_peered":"2024-09-10T21:46:02.710109+0000","last_clean":"2024-09-10T21:46:02.710109+0000","last_became_active":"2024-09-10T21:46:00.845134+0000","last_became_peered":"2024-09-10T21:46:00.845134+0000","last_unstale":"2024-09-10T21:46:02.710109+0000","last_undegraded":"2024-09-10T21:46:02.710109+0000","last_fullsized":"2024-09-10T21:46:02.710109+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:30.650327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.711363+0000","last_change":"2024-09-10T21:46:00.835386+0000","last_active":"2024-09-10T21:46:02.711363+0000","last_peered":"2024-09-10T21:46:02.711363+0000","last_clean":"2024-09-10T21:46:02.711363+0000","last_became_active":"2024-09-10T21:46:00.835115+0000","last_became_peered":"2024-09-10T21:46:00.835115+0000","last_unstale":"2024-09-10T21:46:02.711363+0000","last_undegraded":"2024-09-10T21:46:02.711363+0000","last_fullsized":"2024-09-10T21:46:02.711363+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:12.555199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828651+0000","last_change":"2024-09-10T21:46:00.830508+0000","last_active":"2024-09-10T21:46:01.828651+0000","last_peered":"2024-09-10T21:46:01.828651+0000","last_clean":"2024-09-10T21:46:01.828651+0000","last_became_active":"2024-09-10T21:46:00.830232+0000","last_became_peered":"2024-09-10T21:46:00.830232+0000","last_unstale":"2024-09-10T21:46:01.828651+0000","last_undegraded":"2024-09-10T21:46:01.828651+0000","last_fullsized":"2024-09-10T21:46:01.828651+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:45:39.679370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.850612+0000","last_change":"2024-09-10T21:46:00.853233+0000","last_active":"2024-09-10T21:46:01.850612+0000","last_peered":"2024-09-10T21:46:01.850612+0000","last_clean":"2024-09-10T21:46:01.850612+0000","last_became_active":"2024-09-10T21:46:00.852834+0000","last_became_peered":"2024-09-10T21:46:00.852834+0000","last_unstale":"2024-09-10T21:46:01.850612+0000","last_undegraded":"2024-09-10T21:46:01.850612+0000","last_fullsized":"2024-09-10T21:46:01.850612+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:06:34.784319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830406+0000","last_change":"2024-09-10T21:46:00.843228+0000","last_active":"2024-09-10T21:46:01.830406+0000","last_peered":"2024-09-10T21:46:01.830406+0000","last_clean":"2024-09-10T21:46:01.830406+0000","last_became_active":"2024-09-10T21:46:00.842891+0000","last_became_peered":"2024-09-10T21:46:00.842891+0000","last_unstale":"2024-09-10T21:46:01.830406+0000","last_undegraded":"2024-09-10T21:46:01.830406+0000","last_fullsized":"2024-09-10T21:46:01.830406+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:03.394188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830180+0000","last_change":"2024-09-10T21:46:00.842985+0000","last_active":"2024-09-10T21:46:01.830180+0000","last_peered":"2024-09-10T21:46:01.830180+0000","last_clean":"2024-09-10T21:46:01.830180+0000","last_became_active":"2024-09-10T21:46:00.842604+0000","last_became_peered":"2024-09-10T21:46:00.842604+0000","last_unstale":"2024-09-10T21:46:01.830180+0000","last_undegraded":"2024-09-10T21:46:01.830180+0000","last_fullsized":"2024-09-10T21:46:01.830180+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:03:00.984325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.859026+0000","last_change":"2024-09-10T21:46:00.854428+0000","last_active":"2024-09-10T21:46:00.859026+0000","last_peered":"2024-09-10T21:46:00.859026+0000","last_clean":"2024-09-10T21:46:00.859026+0000","last_became_active":"2024-09-10T21:46:00.854124+0000","last_became_peered":"2024-09-10T21:46:00.854124+0000","last_unstale":"2024-09-10T21:46:00.859026+0000","last_undegraded":"2024-09-10T21:46:00.859026+0000","last_fullsized":"2024-09-10T21:46:00.859026+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:14:01.690462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.858049+0000","last_change":"2024-09-10T21:46:00.847631+0000","last_active":"2024-09-10T21:46:00.858049+0000","last_peered":"2024-09-10T21:46:00.858049+0000","last_clean":"2024-09-10T21:46:00.858049+0000","last_became_active":"2024-09-10T21:46:00.847364+0000","last_became_peered":"2024-09-10T21:46:00.847364+0000","last_unstale":"2024-09-10T21:46:00.858049+0000","last_undegraded":"2024-09-10T21:46:00.858049+0000","last_fullsized":"2024-09-10T21:46:00.858049+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:19.153947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830929+0000","last_change":"2024-09-10T21:46:00.838371+0000","last_active":"2024-09-10T21:46:01.830929+0000","last_peered":"2024-09-10T21:46:01.830929+0000","last_clean":"2024-09-10T21:46:01.830929+0000","last_became_active":"2024-09-10T21:46:00.838080+0000","last_became_peered":"2024-09-10T21:46:00.838080+0000","last_unstale":"2024-09-10T21:46:01.830929+0000","last_undegraded":"2024-09-10T21:46:01.830929+0000","last_fullsized":"2024-09-10T21:46:01.830929+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:34:35.423503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.831256+0000","last_change":"2024-09-10T21:46:00.864076+0000","last_active":"2024-09-10T21:46:01.831256+0000","last_peered":"2024-09-10T21:46:01.831256+0000","last_clean":"2024-09-10T21:46:01.831256+0000","last_became_active":"2024-09-10T21:46:00.863795+0000","last_became_peered":"2024-09-10T21:46:00.863795+0000","last_unstale":"2024-09-10T21:46:01.831256+0000","last_undegraded":"2024-09-10T21:46:01.831256+0000","last_fullsized":"2024-09-10T21:46:01.831256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:32.715496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.852256+0000","last_change":"2024-09-10T21:46:00.850433+0000","last_active":"2024-09-10T21:46:00.852256+0000","last_peered":"2024-09-10T21:46:00.852256+0000","last_clean":"2024-09-10T21:46:00.852256+0000","last_became_active":"2024-09-10T21:46:00.850127+0000","last_became_peered":"2024-09-10T21:46:00.850127+0000","last_unstale":"2024-09-10T21:46:00.852256+0000","last_undegraded":"2024-09-10T21:46:00.852256+0000","last_fullsized":"2024-09-10T21:46:00.852256+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:26:14.666971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830481+0000","last_change":"2024-09-10T21:46:00.840169+0000","last_active":"2024-09-10T21:46:01.830481+0000","last_peered":"2024-09-10T21:46:01.830481+0000","last_clean":"2024-09-10T21:46:01.830481+0000","last_became_active":"2024-09-10T21:46:00.838227+0000","last_became_peered":"2024-09-10T21:46:00.838227+0000","last_unstale":"2024-09-10T21:46:01.830481+0000","last_undegraded":"2024-09-10T21:46:01.830481+0000","last_fullsized":"2024-09-10T21:46:01.830481+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:28:52.428566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.852058+0000","last_change":"2024-09-10T21:46:00.849867+0000","last_active":"2024-09-10T21:46:00.852058+0000","last_peered":"2024-09-10T21:46:00.852058+0000","last_clean":"2024-09-10T21:46:00.852058+0000","last_became_active":"2024-09-10T21:46:00.849205+0000","last_became_peered":"2024-09-10T21:46:00.849205+0000","last_unstale":"2024-09-10T21:46:00.852058+0000","last_undegraded":"2024-09-10T21:46:00.852058+0000","last_fullsized":"2024-09-10T21:46:00.852058+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:14.339273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.831292+0000","last_change":"2024-09-10T21:46:00.864905+0000","last_active":"2024-09-10T21:46:01.831292+0000","last_peered":"2024-09-10T21:46:01.831292+0000","last_clean":"2024-09-10T21:46:01.831292+0000","last_became_active":"2024-09-10T21:46:00.864529+0000","last_became_peered":"2024-09-10T21:46:00.864529+0000","last_unstale":"2024-09-10T21:46:01.831292+0000","last_undegraded":"2024-09-10T21:46:01.831292+0000","last_fullsized":"2024-09-10T21:46:01.831292+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:05.727840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.831565+0000","last_change":"2024-09-10T21:46:00.846590+0000","last_active":"2024-09-10T21:46:01.831565+0000","last_peered":"2024-09-10T21:46:01.831565+0000","last_clean":"2024-09-10T21:46:01.831565+0000","last_became_active":"2024-09-10T21:46:00.846328+0000","last_became_peered":"2024-09-10T21:46:00.846328+0000","last_unstale":"2024-09-10T21:46:01.831565+0000","last_undegraded":"2024-09-10T21:46:01.831565+0000","last_fullsized":"2024-09-10T21:46:01.831565+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:57.554644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828289+0000","last_change":"2024-09-10T21:46:00.852671+0000","last_active":"2024-09-10T21:46:01.828289+0000","last_peered":"2024-09-10T21:46:01.828289+0000","last_clean":"2024-09-10T21:46:01.828289+0000","last_became_active":"2024-09-10T21:46:00.849824+0000","last_became_peered":"2024-09-10T21:46:00.849824+0000","last_unstale":"2024-09-10T21:46:01.828289+0000","last_undegraded":"2024-09-10T21:46:01.828289+0000","last_fullsized":"2024-09-10T21:46:01.828289+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:24:37.967117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.857787+0000","last_change":"2024-09-10T21:46:00.847058+0000","last_active":"2024-09-10T21:46:00.857787+0000","last_peered":"2024-09-10T21:46:00.857787+0000","last_clean":"2024-09-10T21:46:00.857787+0000","last_became_active":"2024-09-10T21:46:00.846826+0000","last_became_peered":"2024-09-10T21:46:00.846826+0000","last_unstale":"2024-09-10T21:46:00.857787+0000","last_undegraded":"2024-09-10T21:46:00.857787+0000","last_fullsized":"2024-09-10T21:46:00.857787+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:43.687249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"69'4","reported_seq":17,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.870886+0000","last_change":"2024-09-10T21:45:59.839520+0000","last_active":"2024-09-10T21:46:00.870886+0000","last_peered":"2024-09-10T21:46:00.870886+0000","last_clean":"2024-09-10T21:46:00.870886+0000","last_became_active":"2024-09-10T21:45:59.839036+0000","last_became_peered":"2024-09-10T21:45:59.839036+0000","last_unstale":"2024-09-10T21:46:00.870886+0000","last_undegraded":"2024-09-10T21:46:00.870886+0000","last_fullsized":"2024-09-10T21:46:00.870886+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:41:02.998584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.710338+0000","last_change":"2024-09-10T21:46:00.845798+0000","last_active":"2024-09-10T21:46:02.710338+0000","last_peered":"2024-09-10T21:46:02.710338+0000","last_clean":"2024-09-10T21:46:02.710338+0000","last_became_active":"2024-09-10T21:46:00.843962+0000","last_became_peered":"2024-09-10T21:46:00.843962+0000","last_unstale":"2024-09-10T21:46:02.710338+0000","last_undegraded":"2024-09-10T21:46:02.710338+0000","last_fullsized":"2024-09-10T21:46:02.710338+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:19.486516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"69'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.850739+0000","last_change":"2024-09-10T21:45:59.834997+0000","last_active":"2024-09-10T21:46:01.850739+0000","last_peered":"2024-09-10T21:46:01.850739+0000","last_clean":"2024-09-10T21:46:01.850739+0000","last_became_active":"2024-09-10T21:45:59.834651+0000","last_became_peered":"2024-09-10T21:45:59.834651+0000","last_unstale":"2024-09-10T21:46:01.850739+0000","last_undegraded":"2024-09-10T21:46:01.850739+0000","last_fullsized":"2024-09-10T21:46:01.850739+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:40.832467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828553+0000","last_change":"2024-09-10T21:46:00.849908+0000","last_active":"2024-09-10T21:46:01.828553+0000","last_peered":"2024-09-10T21:46:01.828553+0000","last_clean":"2024-09-10T21:46:01.828553+0000","last_became_active":"2024-09-10T21:46:00.849592+0000","last_became_peered":"2024-09-10T21:46:00.849592+0000","last_unstale":"2024-09-10T21:46:01.828553+0000","last_undegraded":"2024-09-10T21:46:01.828553+0000","last_fullsized":"2024-09-10T21:46:01.828553+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:52:03.394188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"69'2","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830754+0000","last_change":"2024-09-10T21:45:59.838236+0000","last_active":"2024-09-10T21:46:01.830754+0000","last_peered":"2024-09-10T21:46:01.830754+0000","last_clean":"2024-09-10T21:46:01.830754+0000","last_became_active":"2024-09-10T21:45:59.836821+0000","last_became_peered":"2024-09-10T21:45:59.836821+0000","last_unstale":"2024-09-10T21:46:01.830754+0000","last_undegraded":"2024-09-10T21:46:01.830754+0000","last_fullsized":"2024-09-10T21:46:01.830754+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:39:35.084439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.855541+0000","last_change":"2024-09-10T21:46:00.841820+0000","last_active":"2024-09-10T21:46:00.855541+0000","last_peered":"2024-09-10T21:46:00.855541+0000","last_clean":"2024-09-10T21:46:00.855541+0000","last_became_active":"2024-09-10T21:46:00.841524+0000","last_became_peered":"2024-09-10T21:46:00.841524+0000","last_unstale":"2024-09-10T21:46:00.855541+0000","last_undegraded":"2024-09-10T21:46:00.855541+0000","last_fullsized":"2024-09-10T21:46:00.855541+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:09.586859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"69'2","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828883+0000","last_change":"2024-09-10T21:45:59.837757+0000","last_active":"2024-09-10T21:46:01.828883+0000","last_peered":"2024-09-10T21:46:01.828883+0000","last_clean":"2024-09-10T21:46:01.828883+0000","last_became_active":"2024-09-10T21:45:59.837466+0000","last_became_peered":"2024-09-10T21:45:59.837466+0000","last_unstale":"2024-09-10T21:46:01.828883+0000","last_undegraded":"2024-09-10T21:46:01.828883+0000","last_fullsized":"2024-09-10T21:46:01.828883+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:31.504429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.831512+0000","last_change":"2024-09-10T21:46:00.846072+0000","last_active":"2024-09-10T21:46:01.831512+0000","last_peered":"2024-09-10T21:46:01.831512+0000","last_clean":"2024-09-10T21:46:01.831512+0000","last_became_active":"2024-09-10T21:46:00.845756+0000","last_became_peered":"2024-09-10T21:46:00.845756+0000","last_unstale":"2024-09-10T21:46:01.831512+0000","last_undegraded":"2024-09-10T21:46:01.831512+0000","last_fullsized":"2024-09-10T21:46:01.831512+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:37.624857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"69'1","reported_seq":14,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.858794+0000","last_change":"2024-09-10T21:45:59.830252+0000","last_active":"2024-09-10T21:46:00.858794+0000","last_peered":"2024-09-10T21:46:00.858794+0000","last_clean":"2024-09-10T21:46:00.858794+0000","last_became_active":"2024-09-10T21:45:59.829958+0000","last_became_peered":"2024-09-10T21:45:59.829958+0000","last_unstale":"2024-09-10T21:46:00.858794+0000","last_undegraded":"2024-09-10T21:46:00.858794+0000","last_fullsized":"2024-09-10T21:46:00.858794+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:00.503511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830221+0000","last_change":"2024-09-10T21:46:00.845982+0000","last_active":"2024-09-10T21:46:01.830221+0000","last_peered":"2024-09-10T21:46:01.830221+0000","last_clean":"2024-09-10T21:46:01.830221+0000","last_became_active":"2024-09-10T21:46:00.844701+0000","last_became_peered":"2024-09-10T21:46:00.844701+0000","last_unstale":"2024-09-10T21:46:01.830221+0000","last_undegraded":"2024-09-10T21:46:01.830221+0000","last_fullsized":"2024-09-10T21:46:01.830221+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:33.689497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"69'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828460+0000","last_change":"2024-09-10T21:45:59.833692+0000","last_active":"2024-09-10T21:46:01.828460+0000","last_peered":"2024-09-10T21:46:01.828460+0000","last_clean":"2024-09-10T21:46:01.828460+0000","last_became_active":"2024-09-10T21:45:59.833393+0000","last_became_peered":"2024-09-10T21:45:59.833393+0000","last_unstale":"2024-09-10T21:46:01.828460+0000","last_undegraded":"2024-09-10T21:46:01.828460+0000","last_fullsized":"2024-09-10T21:46:01.828460+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:18:28.299604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709131+0000","last_change":"2024-09-10T21:46:00.838228+0000","last_active":"2024-09-10T21:46:02.709131+0000","last_peered":"2024-09-10T21:46:02.709131+0000","last_clean":"2024-09-10T21:46:02.709131+0000","last_became_active":"2024-09-10T21:46:00.837919+0000","last_became_peered":"2024-09-10T21:46:00.837919+0000","last_unstale":"2024-09-10T21:46:02.709131+0000","last_undegraded":"2024-09-10T21:46:02.709131+0000","last_fullsized":"2024-09-10T21:46:02.709131+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:43:05.473475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.710686+0000","last_change":"2024-09-10T21:45:59.830360+0000","last_active":"2024-09-10T21:46:02.710686+0000","last_peered":"2024-09-10T21:46:02.710686+0000","last_clean":"2024-09-10T21:46:02.710686+0000","last_became_active":"2024-09-10T21:45:59.830049+0000","last_became_peered":"2024-09-10T21:45:59.830049+0000","last_unstale":"2024-09-10T21:46:02.710686+0000","last_undegraded":"2024-09-10T21:46:02.710686+0000","last_fullsized":"2024-09-10T21:46:02.710686+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:32.210607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.850518+0000","last_change":"2024-09-10T21:46:00.846853+0000","last_active":"2024-09-10T21:46:01.850518+0000","last_peered":"2024-09-10T21:46:01.850518+0000","last_clean":"2024-09-10T21:46:01.850518+0000","last_became_active":"2024-09-10T21:46:00.846224+0000","last_became_peered":"2024-09-10T21:46:00.846224+0000","last_unstale":"2024-09-10T21:46:01.850518+0000","last_undegraded":"2024-09-10T21:46:01.850518+0000","last_fullsized":"2024-09-10T21:46:01.850518+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:05:43.160809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709066+0000","last_change":"2024-09-10T21:45:59.827108+0000","last_active":"2024-09-10T21:46:02.709066+0000","last_peered":"2024-09-10T21:46:02.709066+0000","last_clean":"2024-09-10T21:46:02.709066+0000","last_became_active":"2024-09-10T21:45:59.826809+0000","last_became_peered":"2024-09-10T21:45:59.826809+0000","last_unstale":"2024-09-10T21:46:02.709066+0000","last_undegraded":"2024-09-10T21:46:02.709066+0000","last_fullsized":"2024-09-10T21:46:02.709066+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:34.953708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.831420+0000","last_change":"2024-09-10T21:46:00.858254+0000","last_active":"2024-09-10T21:46:01.831420+0000","last_peered":"2024-09-10T21:46:01.831420+0000","last_clean":"2024-09-10T21:46:01.831420+0000","last_became_active":"2024-09-10T21:46:00.857949+0000","last_became_peered":"2024-09-10T21:46:00.857949+0000","last_unstale":"2024-09-10T21:46:01.831420+0000","last_undegraded":"2024-09-10T21:46:01.831420+0000","last_fullsized":"2024-09-10T21:46:01.831420+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:51:05.211407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830914+0000","last_change":"2024-09-10T21:45:59.828239+0000","last_active":"2024-09-10T21:46:01.830914+0000","last_peered":"2024-09-10T21:46:01.830914+0000","last_clean":"2024-09-10T21:46:01.830914+0000","last_became_active":"2024-09-10T21:45:59.827825+0000","last_became_peered":"2024-09-10T21:45:59.827825+0000","last_unstale":"2024-09-10T21:46:01.830914+0000","last_undegraded":"2024-09-10T21:46:01.830914+0000","last_fullsized":"2024-09-10T21:46:01.830914+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:59:50.008776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.865057+0000","last_change":"2024-09-10T21:46:00.861904+0000","last_active":"2024-09-10T21:46:00.865057+0000","last_peered":"2024-09-10T21:46:00.865057+0000","last_clean":"2024-09-10T21:46:00.865057+0000","last_became_active":"2024-09-10T21:46:00.861641+0000","last_became_peered":"2024-09-10T21:46:00.861641+0000","last_unstale":"2024-09-10T21:46:00.865057+0000","last_undegraded":"2024-09-10T21:46:00.865057+0000","last_fullsized":"2024-09-10T21:46:00.865057+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:08:08.004631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.829259+0000","last_change":"2024-09-10T21:45:59.835170+0000","last_active":"2024-09-10T21:46:01.829259+0000","last_peered":"2024-09-10T21:46:01.829259+0000","last_clean":"2024-09-10T21:46:01.829259+0000","last_became_active":"2024-09-10T21:45:59.833573+0000","last_became_peered":"2024-09-10T21:45:59.833573+0000","last_unstale":"2024-09-10T21:46:01.829259+0000","last_undegraded":"2024-09-10T21:46:01.829259+0000","last_fullsized":"2024-09-10T21:46:01.829259+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:14.083982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828773+0000","last_change":"2024-09-10T21:46:00.841108+0000","last_active":"2024-09-10T21:46:01.828773+0000","last_peered":"2024-09-10T21:46:01.828773+0000","last_clean":"2024-09-10T21:46:01.828773+0000","last_became_active":"2024-09-10T21:46:00.840794+0000","last_became_peered":"2024-09-10T21:46:00.840794+0000","last_unstale":"2024-09-10T21:46:01.828773+0000","last_undegraded":"2024-09-10T21:46:01.828773+0000","last_fullsized":"2024-09-10T21:46:01.828773+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:02.804991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"69'3","reported_seq":16,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.863092+0000","last_change":"2024-09-10T21:45:59.835930+0000","last_active":"2024-09-10T21:46:00.863092+0000","last_peered":"2024-09-10T21:46:00.863092+0000","last_clean":"2024-09-10T21:46:00.863092+0000","last_became_active":"2024-09-10T21:45:59.834886+0000","last_became_peered":"2024-09-10T21:45:59.834886+0000","last_unstale":"2024-09-10T21:46:00.863092+0000","last_undegraded":"2024-09-10T21:46:00.863092+0000","last_fullsized":"2024-09-10T21:46:00.863092+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:30.172190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709672+0000","last_change":"2024-09-10T21:46:00.835699+0000","last_active":"2024-09-10T21:46:02.709672+0000","last_peered":"2024-09-10T21:46:02.709672+0000","last_clean":"2024-09-10T21:46:02.709672+0000","last_became_active":"2024-09-10T21:46:00.835365+0000","last_became_peered":"2024-09-10T21:46:00.835365+0000","last_unstale":"2024-09-10T21:46:02.709672+0000","last_undegraded":"2024-09-10T21:46:02.709672+0000","last_fullsized":"2024-09-10T21:46:02.709672+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:01.217551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"69'6","reported_seq":21,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.711320+0000","last_change":"2024-09-10T21:45:59.832994+0000","last_active":"2024-09-10T21:46:02.711320+0000","last_peered":"2024-09-10T21:46:02.711320+0000","last_clean":"2024-09-10T21:46:02.711320+0000","last_became_active":"2024-09-10T21:45:59.832695+0000","last_became_peered":"2024-09-10T21:45:59.832695+0000","last_unstale":"2024-09-10T21:46:02.711320+0000","last_undegraded":"2024-09-10T21:46:02.711320+0000","last_fullsized":"2024-09-10T21:46:02.711320+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:12:36.881454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.840833+0000","last_change":"2024-09-10T21:46:00.836737+0000","last_active":"2024-09-10T21:46:00.840833+0000","last_peered":"2024-09-10T21:46:00.840833+0000","last_clean":"2024-09-10T21:46:00.840833+0000","last_became_active":"2024-09-10T21:46:00.836137+0000","last_became_peered":"2024-09-10T21:46:00.836137+0000","last_unstale":"2024-09-10T21:46:00.840833+0000","last_undegraded":"2024-09-10T21:46:00.840833+0000","last_fullsized":"2024-09-10T21:46:00.840833+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:47.277936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"69'2","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.711080+0000","last_change":"2024-09-10T21:45:59.834396+0000","last_active":"2024-09-10T21:46:02.711080+0000","last_peered":"2024-09-10T21:46:02.711080+0000","last_clean":"2024-09-10T21:46:02.711080+0000","last_became_active":"2024-09-10T21:45:59.834132+0000","last_became_peered":"2024-09-10T21:45:59.834132+0000","last_unstale":"2024-09-10T21:46:02.711080+0000","last_undegraded":"2024-09-10T21:46:02.711080+0000","last_fullsized":"2024-09-10T21:46:02.711080+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:23.405630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.831242+0000","last_change":"2024-09-10T21:46:00.854001+0000","last_active":"2024-09-10T21:46:01.831242+0000","last_peered":"2024-09-10T21:46:01.831242+0000","last_clean":"2024-09-10T21:46:01.831242+0000","last_became_active":"2024-09-10T21:46:00.853583+0000","last_became_peered":"2024-09-10T21:46:00.853583+0000","last_unstale":"2024-09-10T21:46:01.831242+0000","last_undegraded":"2024-09-10T21:46:01.831242+0000","last_fullsized":"2024-09-10T21:46:01.831242+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:30.666063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828564+0000","last_change":"2024-09-10T21:45:59.838484+0000","last_active":"2024-09-10T21:46:01.828564+0000","last_peered":"2024-09-10T21:46:01.828564+0000","last_clean":"2024-09-10T21:46:01.828564+0000","last_became_active":"2024-09-10T21:45:59.838214+0000","last_became_peered":"2024-09-10T21:45:59.838214+0000","last_unstale":"2024-09-10T21:46:01.828564+0000","last_undegraded":"2024-09-10T21:46:01.828564+0000","last_fullsized":"2024-09-10T21:46:01.828564+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:50.361968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.850424+0000","last_change":"2024-09-10T21:46:00.840558+0000","last_active":"2024-09-10T21:46:01.850424+0000","last_peered":"2024-09-10T21:46:01.850424+0000","last_clean":"2024-09-10T21:46:01.850424+0000","last_became_active":"2024-09-10T21:46:00.840246+0000","last_became_peered":"2024-09-10T21:46:00.840246+0000","last_unstale":"2024-09-10T21:46:01.850424+0000","last_undegraded":"2024-09-10T21:46:01.850424+0000","last_fullsized":"2024-09-10T21:46:01.850424+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:08:12.588863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830943+0000","last_change":"2024-09-10T21:45:59.827139+0000","last_active":"2024-09-10T21:46:01.830943+0000","last_peered":"2024-09-10T21:46:01.830943+0000","last_clean":"2024-09-10T21:46:01.830943+0000","last_became_active":"2024-09-10T21:45:59.826823+0000","last_became_peered":"2024-09-10T21:45:59.826823+0000","last_unstale":"2024-09-10T21:46:01.830943+0000","last_undegraded":"2024-09-10T21:46:01.830943+0000","last_fullsized":"2024-09-10T21:46:01.830943+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:23:23.851492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709190+0000","last_change":"2024-09-10T21:46:00.837215+0000","last_active":"2024-09-10T21:46:02.709190+0000","last_peered":"2024-09-10T21:46:02.709190+0000","last_clean":"2024-09-10T21:46:02.709190+0000","last_became_active":"2024-09-10T21:46:00.836728+0000","last_became_peered":"2024-09-10T21:46:00.836728+0000","last_unstale":"2024-09-10T21:46:02.709190+0000","last_undegraded":"2024-09-10T21:46:02.709190+0000","last_fullsized":"2024-09-10T21:46:02.709190+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:40:23.589785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"69'8","reported_seq":23,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.829971+0000","last_change":"2024-09-10T21:45:59.831460+0000","last_active":"2024-09-10T21:46:01.829971+0000","last_peered":"2024-09-10T21:46:01.829971+0000","last_clean":"2024-09-10T21:46:01.829971+0000","last_became_active":"2024-09-10T21:45:59.831161+0000","last_became_peered":"2024-09-10T21:45:59.831161+0000","last_unstale":"2024-09-10T21:46:01.829971+0000","last_undegraded":"2024-09-10T21:46:01.829971+0000","last_fullsized":"2024-09-10T21:46:01.829971+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:02.348733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830792+0000","last_change":"2024-09-10T21:46:00.838850+0000","last_active":"2024-09-10T21:46:01.830792+0000","last_peered":"2024-09-10T21:46:01.830792+0000","last_clean":"2024-09-10T21:46:01.830792+0000","last_became_active":"2024-09-10T21:46:00.838462+0000","last_became_peered":"2024-09-10T21:46:00.838462+0000","last_unstale":"2024-09-10T21:46:01.830792+0000","last_undegraded":"2024-09-10T21:46:01.830792+0000","last_fullsized":"2024-09-10T21:46:01.830792+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:29:14.199837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"69'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.829086+0000","last_change":"2024-09-10T21:45:59.837920+0000","last_active":"2024-09-10T21:46:01.829086+0000","last_peered":"2024-09-10T21:46:01.829086+0000","last_clean":"2024-09-10T21:46:01.829086+0000","last_became_active":"2024-09-10T21:45:59.837372+0000","last_became_peered":"2024-09-10T21:45:59.837372+0000","last_unstale":"2024-09-10T21:46:01.829086+0000","last_undegraded":"2024-09-10T21:46:01.829086+0000","last_fullsized":"2024-09-10T21:46:01.829086+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:17.497947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.857032+0000","last_change":"2024-09-10T21:46:00.843932+0000","last_active":"2024-09-10T21:46:00.857032+0000","last_peered":"2024-09-10T21:46:00.857032+0000","last_clean":"2024-09-10T21:46:00.857032+0000","last_became_active":"2024-09-10T21:46:00.843650+0000","last_became_peered":"2024-09-10T21:46:00.843650+0000","last_unstale":"2024-09-10T21:46:00.857032+0000","last_undegraded":"2024-09-10T21:46:00.857032+0000","last_fullsized":"2024-09-10T21:46:00.857032+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:51.932442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"69'2","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.830711+0000","last_change":"2024-09-10T21:45:59.834683+0000","last_active":"2024-09-10T21:46:01.830711+0000","last_peered":"2024-09-10T21:46:01.830711+0000","last_clean":"2024-09-10T21:46:01.830711+0000","last_became_active":"2024-09-10T21:45:59.834398+0000","last_became_peered":"2024-09-10T21:45:59.834398+0000","last_unstale":"2024-09-10T21:46:01.830711+0000","last_undegraded":"2024-09-10T21:46:01.830711+0000","last_fullsized":"2024-09-10T21:46:01.830711+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:21.733557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.711204+0000","last_change":"2024-09-10T21:46:00.843778+0000","last_active":"2024-09-10T21:46:02.711204+0000","last_peered":"2024-09-10T21:46:02.711204+0000","last_clean":"2024-09-10T21:46:02.711204+0000","last_became_active":"2024-09-10T21:46:00.843455+0000","last_became_peered":"2024-09-10T21:46:00.843455+0000","last_unstale":"2024-09-10T21:46:02.711204+0000","last_undegraded":"2024-09-10T21:46:02.711204+0000","last_fullsized":"2024-09-10T21:46:02.711204+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:32.511194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"69'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828797+0000","last_change":"2024-09-10T21:45:59.835900+0000","last_active":"2024-09-10T21:46:01.828797+0000","last_peered":"2024-09-10T21:46:01.828797+0000","last_clean":"2024-09-10T21:46:01.828797+0000","last_became_active":"2024-09-10T21:45:59.835055+0000","last_became_peered":"2024-09-10T21:45:59.835055+0000","last_unstale":"2024-09-10T21:46:01.828797+0000","last_undegraded":"2024-09-10T21:46:01.828797+0000","last_fullsized":"2024-09-10T21:46:01.828797+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:18.497451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.710563+0000","last_change":"2024-09-10T21:46:00.838328+0000","last_active":"2024-09-10T21:46:02.710563+0000","last_peered":"2024-09-10T21:46:02.710563+0000","last_clean":"2024-09-10T21:46:02.710563+0000","last_became_active":"2024-09-10T21:46:00.838034+0000","last_became_peered":"2024-09-10T21:46:00.838034+0000","last_unstale":"2024-09-10T21:46:02.710563+0000","last_undegraded":"2024-09-10T21:46:02.710563+0000","last_fullsized":"2024-09-10T21:46:02.710563+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:51.514167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.827043+0000","last_change":"2024-09-10T21:45:59.836594+0000","last_active":"2024-09-10T21:46:00.827043+0000","last_peered":"2024-09-10T21:46:00.827043+0000","last_clean":"2024-09-10T21:46:00.827043+0000","last_became_active":"2024-09-10T21:45:59.836325+0000","last_became_peered":"2024-09-10T21:45:59.836325+0000","last_unstale":"2024-09-10T21:46:00.827043+0000","last_undegraded":"2024-09-10T21:46:00.827043+0000","last_fullsized":"2024-09-10T21:46:00.827043+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:19:39.295788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.844196+0000","last_change":"2024-09-10T21:46:00.837935+0000","last_active":"2024-09-10T21:46:00.844196+0000","last_peered":"2024-09-10T21:46:00.844196+0000","last_clean":"2024-09-10T21:46:00.844196+0000","last_became_active":"2024-09-10T21:46:00.837645+0000","last_became_peered":"2024-09-10T21:46:00.837645+0000","last_unstale":"2024-09-10T21:46:00.844196+0000","last_undegraded":"2024-09-10T21:46:00.844196+0000","last_fullsized":"2024-09-10T21:46:00.844196+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:35:13.337873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828666+0000","last_change":"2024-09-10T21:45:59.833884+0000","last_active":"2024-09-10T21:46:01.828666+0000","last_peered":"2024-09-10T21:46:01.828666+0000","last_clean":"2024-09-10T21:46:01.828666+0000","last_became_active":"2024-09-10T21:45:59.833348+0000","last_became_peered":"2024-09-10T21:45:59.833348+0000","last_unstale":"2024-09-10T21:46:01.828666+0000","last_undegraded":"2024-09-10T21:46:01.828666+0000","last_fullsized":"2024-09-10T21:46:01.828666+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:13:24.226309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.870492+0000","last_change":"2024-09-10T21:46:00.869022+0000","last_active":"2024-09-10T21:46:00.870492+0000","last_peered":"2024-09-10T21:46:00.870492+0000","last_clean":"2024-09-10T21:46:00.870492+0000","last_became_active":"2024-09-10T21:46:00.868103+0000","last_became_peered":"2024-09-10T21:46:00.868103+0000","last_unstale":"2024-09-10T21:46:00.870492+0000","last_undegraded":"2024-09-10T21:46:00.870492+0000","last_fullsized":"2024-09-10T21:46:00.870492+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:40.002164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.831360+0000","last_change":"2024-09-10T21:45:59.830947+0000","last_active":"2024-09-10T21:46:01.831360+0000","last_peered":"2024-09-10T21:46:01.831360+0000","last_clean":"2024-09-10T21:46:01.831360+0000","last_became_active":"2024-09-10T21:45:59.829692+0000","last_became_peered":"2024-09-10T21:45:59.829692+0000","last_unstale":"2024-09-10T21:46:01.831360+0000","last_undegraded":"2024-09-10T21:46:01.831360+0000","last_fullsized":"2024-09-10T21:46:01.831360+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:36.232207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.863457+0000","last_change":"2024-09-10T21:46:00.862041+0000","last_active":"2024-09-10T21:46:00.863457+0000","last_peered":"2024-09-10T21:46:00.863457+0000","last_clean":"2024-09-10T21:46:00.863457+0000","last_became_active":"2024-09-10T21:46:00.861716+0000","last_became_peered":"2024-09-10T21:46:00.861716+0000","last_unstale":"2024-09-10T21:46:00.863457+0000","last_undegraded":"2024-09-10T21:46:00.863457+0000","last_fullsized":"2024-09-10T21:46:00.863457+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:40.331928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.850307+0000","last_change":"2024-09-10T21:45:59.835947+0000","last_active":"2024-09-10T21:46:01.850307+0000","last_peered":"2024-09-10T21:46:01.850307+0000","last_clean":"2024-09-10T21:46:01.850307+0000","last_became_active":"2024-09-10T21:45:59.835118+0000","last_became_peered":"2024-09-10T21:45:59.835118+0000","last_unstale":"2024-09-10T21:46:01.850307+0000","last_undegraded":"2024-09-10T21:46:01.850307+0000","last_fullsized":"2024-09-10T21:46:01.850307+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:44:38.399726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.850725+0000","last_change":"2024-09-10T21:46:00.846564+0000","last_active":"2024-09-10T21:46:01.850725+0000","last_peered":"2024-09-10T21:46:01.850725+0000","last_clean":"2024-09-10T21:46:01.850725+0000","last_became_active":"2024-09-10T21:46:00.846321+0000","last_became_peered":"2024-09-10T21:46:00.846321+0000","last_unstale":"2024-09-10T21:46:01.850725+0000","last_undegraded":"2024-09-10T21:46:01.850725+0000","last_fullsized":"2024-09-10T21:46:01.850725+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:35.510698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"69'2","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.829187+0000","last_change":"2024-09-10T21:45:59.834909+0000","last_active":"2024-09-10T21:46:01.829187+0000","last_peered":"2024-09-10T21:46:01.829187+0000","last_clean":"2024-09-10T21:46:01.829187+0000","last_became_active":"2024-09-10T21:45:59.834603+0000","last_became_peered":"2024-09-10T21:45:59.834603+0000","last_unstale":"2024-09-10T21:46:01.829187+0000","last_undegraded":"2024-09-10T21:46:01.829187+0000","last_fullsized":"2024-09-10T21:46:01.829187+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:04.899566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.831273+0000","last_change":"2024-09-10T21:46:00.855694+0000","last_active":"2024-09-10T21:46:01.831273+0000","last_peered":"2024-09-10T21:46:01.831273+0000","last_clean":"2024-09-10T21:46:01.831273+0000","last_became_active":"2024-09-10T21:46:00.853229+0000","last_became_peered":"2024-09-10T21:46:00.853229+0000","last_unstale":"2024-09-10T21:46:01.831273+0000","last_undegraded":"2024-09-10T21:46:01.831273+0000","last_fullsized":"2024-09-10T21:46:01.831273+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:01:31.844366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"70'1","reported_seq":16,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.849822+0000","last_change":"2024-09-10T21:45:59.828584+0000","last_active":"2024-09-10T21:46:01.849822+0000","last_peered":"2024-09-10T21:46:01.849822+0000","last_clean":"2024-09-10T21:46:01.849822+0000","last_became_active":"2024-09-10T21:45:59.827977+0000","last_became_peered":"2024-09-10T21:45:59.827977+0000","last_unstale":"2024-09-10T21:46:01.849822+0000","last_undegraded":"2024-09-10T21:46:01.849822+0000","last_fullsized":"2024-09-10T21:46:01.849822+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:01.798226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.864657+0000","last_change":"2024-09-10T21:46:00.862536+0000","last_active":"2024-09-10T21:46:00.864657+0000","last_peered":"2024-09-10T21:46:00.864657+0000","last_clean":"2024-09-10T21:46:00.864657+0000","last_became_active":"2024-09-10T21:46:00.862244+0000","last_became_peered":"2024-09-10T21:46:00.862244+0000","last_unstale":"2024-09-10T21:46:00.864657+0000","last_undegraded":"2024-09-10T21:46:00.864657+0000","last_fullsized":"2024-09-10T21:46:00.864657+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:05:33.601733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709884+0000","last_change":"2024-09-10T21:45:59.830095+0000","last_active":"2024-09-10T21:46:02.709884+0000","last_peered":"2024-09-10T21:46:02.709884+0000","last_clean":"2024-09-10T21:46:02.709884+0000","last_became_active":"2024-09-10T21:45:59.829796+0000","last_became_peered":"2024-09-10T21:45:59.829796+0000","last_unstale":"2024-09-10T21:46:02.709884+0000","last_undegraded":"2024-09-10T21:46:02.709884+0000","last_fullsized":"2024-09-10T21:46:02.709884+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:02.704482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.858295+0000","last_change":"2024-09-10T21:46:00.855637+0000","last_active":"2024-09-10T21:46:00.858295+0000","last_peered":"2024-09-10T21:46:00.858295+0000","last_clean":"2024-09-10T21:46:00.858295+0000","last_became_active":"2024-09-10T21:46:00.855388+0000","last_became_peered":"2024-09-10T21:46:00.855388+0000","last_unstale":"2024-09-10T21:46:00.858295+0000","last_undegraded":"2024-09-10T21:46:00.858295+0000","last_fullsized":"2024-09-10T21:46:00.858295+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:48.018328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.850547+0000","last_change":"2024-09-10T21:45:59.826722+0000","last_active":"2024-09-10T21:46:01.850547+0000","last_peered":"2024-09-10T21:46:01.850547+0000","last_clean":"2024-09-10T21:46:01.850547+0000","last_became_active":"2024-09-10T21:45:59.826436+0000","last_became_peered":"2024-09-10T21:45:59.826436+0000","last_unstale":"2024-09-10T21:46:01.850547+0000","last_undegraded":"2024-09-10T21:46:01.850547+0000","last_fullsized":"2024-09-10T21:46:01.850547+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:13:11.996778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.710082+0000","last_change":"2024-09-10T21:46:00.839723+0000","last_active":"2024-09-10T21:46:02.710082+0000","last_peered":"2024-09-10T21:46:02.710082+0000","last_clean":"2024-09-10T21:46:02.710082+0000","last_became_active":"2024-09-10T21:46:00.839475+0000","last_became_peered":"2024-09-10T21:46:00.839475+0000","last_unstale":"2024-09-10T21:46:02.710082+0000","last_undegraded":"2024-09-10T21:46:02.710082+0000","last_fullsized":"2024-09-10T21:46:02.710082+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:31.317327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709825+0000","last_change":"2024-09-10T21:45:59.821286+0000","last_active":"2024-09-10T21:46:02.709825+0000","last_peered":"2024-09-10T21:46:02.709825+0000","last_clean":"2024-09-10T21:46:02.709825+0000","last_became_active":"2024-09-10T21:45:59.820999+0000","last_became_peered":"2024-09-10T21:45:59.820999+0000","last_unstale":"2024-09-10T21:46:02.709825+0000","last_undegraded":"2024-09-10T21:46:02.709825+0000","last_fullsized":"2024-09-10T21:46:02.709825+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:35:51.376484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828127+0000","last_change":"2024-09-10T21:46:00.849338+0000","last_active":"2024-09-10T21:46:01.828127+0000","last_peered":"2024-09-10T21:46:01.828127+0000","last_clean":"2024-09-10T21:46:01.828127+0000","last_became_active":"2024-09-10T21:46:00.848819+0000","last_became_peered":"2024-09-10T21:46:00.848819+0000","last_unstale":"2024-09-10T21:46:01.828127+0000","last_undegraded":"2024-09-10T21:46:01.828127+0000","last_fullsized":"2024-09-10T21:46:01.828127+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:17:05.714091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.710702+0000","last_change":"2024-09-10T21:45:59.832047+0000","last_active":"2024-09-10T21:46:02.710702+0000","last_peered":"2024-09-10T21:46:02.710702+0000","last_clean":"2024-09-10T21:46:02.710702+0000","last_became_active":"2024-09-10T21:45:59.831762+0000","last_became_peered":"2024-09-10T21:45:59.831762+0000","last_unstale":"2024-09-10T21:46:02.710702+0000","last_undegraded":"2024-09-10T21:46:02.710702+0000","last_fullsized":"2024-09-10T21:46:02.710702+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:02:36.934813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:02.709755+0000","last_change":"2024-09-10T21:46:00.835798+0000","last_active":"2024-09-10T21:46:02.709755+0000","last_peered":"2024-09-10T21:46:02.709755+0000","last_clean":"2024-09-10T21:46:02.709755+0000","last_became_active":"2024-09-10T21:46:00.835507+0000","last_became_peered":"2024-09-10T21:46:00.835507+0000","last_unstale":"2024-09-10T21:46:02.709755+0000","last_undegraded":"2024-09-10T21:46:02.709755+0000","last_fullsized":"2024-09-10T21:46:02.709755+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:43:15.090747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.829884+0000","last_change":"2024-09-10T21:45:59.830690+0000","last_active":"2024-09-10T21:46:01.829884+0000","last_peered":"2024-09-10T21:46:01.829884+0000","last_clean":"2024-09-10T21:46:01.829884+0000","last_became_active":"2024-09-10T21:45:59.830407+0000","last_became_peered":"2024-09-10T21:45:59.830407+0000","last_unstale":"2024-09-10T21:46:01.829884+0000","last_undegraded":"2024-09-10T21:46:01.829884+0000","last_fullsized":"2024-09-10T21:46:01.829884+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:31.228948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":13,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.827918+0000","last_change":"2024-09-10T21:46:00.847377+0000","last_active":"2024-09-10T21:46:01.827918+0000","last_peered":"2024-09-10T21:46:01.827918+0000","last_clean":"2024-09-10T21:46:01.827918+0000","last_became_active":"2024-09-10T21:46:00.847047+0000","last_became_peered":"2024-09-10T21:46:00.847047+0000","last_unstale":"2024-09-10T21:46:01.827918+0000","last_undegraded":"2024-09-10T21:46:01.827918+0000","last_fullsized":"2024-09-10T21:46:01.827918+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:48:59.158149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":15,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828257+0000","last_change":"2024-09-10T21:45:59.836973+0000","last_active":"2024-09-10T21:46:01.828257+0000","last_peered":"2024-09-10T21:46:01.828257+0000","last_clean":"2024-09-10T21:46:01.828257+0000","last_became_active":"2024-09-10T21:45:59.836688+0000","last_became_peered":"2024-09-10T21:45:59.836688+0000","last_unstale":"2024-09-10T21:46:01.828257+0000","last_undegraded":"2024-09-10T21:46:01.828257+0000","last_fullsized":"2024-09-10T21:46:01.828257+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:10:58.722699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.863459+0000","last_change":"2024-09-10T21:46:00.862053+0000","last_active":"2024-09-10T21:46:00.863459+0000","last_peered":"2024-09-10T21:46:00.863459+0000","last_clean":"2024-09-10T21:46:00.863459+0000","last_became_active":"2024-09-10T21:46:00.861698+0000","last_became_peered":"2024-09-10T21:46:00.861698+0000","last_unstale":"2024-09-10T21:46:00.863459+0000","last_undegraded":"2024-09-10T21:46:00.863459+0000","last_fullsized":"2024-09-10T21:46:00.863459+0000","mapping_epoch":68,"log_start":"0'0","ondisk_log_start":"0'0","created":68,"last_epoch_clean":69,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:59.806002+0000","last_clean_scrub_stamp":"2024-09-10T21:45:59.806002+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:58.548218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"69'2","reported_seq":17,"reported_epoch":70,"state":"active+clean","last_fresh":"2024-09-10T21:46:01.828079+0000","last_change":"2024-09-10T21:45:59.836987+0000","last_active":"2024-09-10T21:46:01.828079+0000","last_peered":"2024-09-10T21:46:01.828079+0000","last_clean":"2024-09-10T21:46:01.828079+0000","last_became_active":"2024-09-10T21:45:59.836711+0000","last_became_peered":"2024-09-10T21:45:59.836711+0000","last_unstale":"2024-09-10T21:46:01.828079+0000","last_undegraded":"2024-09-10T21:46:01.828079+0000","last_fullsized":"2024-09-10T21:46:01.828079+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:45:58.799237+0000","last_clean_scrub_stamp":"2024-09-10T21:45:58.799237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:45:51.311564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":198,"reported_epoch":69,"state":"active+clean","last_fresh":"2024-09-10T21:46:00.826392+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:46:00.826392+0000","last_peered":"2024-09-10T21:46:00.826392+0000","last_clean":"2024-09-10T21:46:00.826392+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:46:00.826392+0000","last_undegraded":"2024-09-10T21:46:00.826392+0000","last_fullsized":"2024-09-10T21:46:00.826392+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509493,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32028,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339812,"statfs":{"total":96636764160,"available":96603967488,"internally_reserved":0,"allocated":851968,"data_stored":353158,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509493,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":847872,"data_stored":353156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509492,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":847872,"data_stored":353136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509492,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":843776,"data_stored":348378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509494,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":851968,"data_stored":348994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509492,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32600,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339240,"statfs":{"total":96636764160,"available":96603381760,"internally_reserved":0,"allocated":1437696,"data_stored":943434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542195,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32588,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339252,"statfs":{"total":96636764160,"available":96603394048,"internally_reserved":0,"allocated":1433600,"data_stored":937750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542196,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":839680,"data_stored":346878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:46:05.524 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:46:05.525 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:46:05.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.727+0000 7faf98600640 1 -- 172.21.15.7:0/3439271100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf78004800 msgr2=0x7faf78004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.727+0000 7faf98600640 1 --2- 172.21.15.7:0/3439271100 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf78004800 0x7faf78004c80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7faf7c001690 tx=0x7faf78005830 comp rx=0 tx=0).stop 2024-09-10T21:46:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.727+0000 7faf96600640 1 -- 172.21.15.7:0/3439271100 reap_dead start 2024-09-10T21:46:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.728+0000 7faf98600640 1 -- 172.21.15.7:0/3439271100 shutdown_connections 2024-09-10T21:46:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.728+0000 7faf98600640 1 -- 172.21.15.7:0/3439271100 >> 172.21.15.7:0/3439271100 conn(0x7faf900b16d0 msgr2=0x7faf900b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.728+0000 7faf98600640 1 -- 172.21.15.7:0/3439271100 shutdown_connections 2024-09-10T21:46:05.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.728+0000 7faf98600640 1 -- 172.21.15.7:0/3439271100 wait complete. 2024-09-10T21:46:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.729+0000 7faf98600640 1 Processor -- start 2024-09-10T21:46:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.729+0000 7faf98600640 1 -- start start 2024-09-10T21:46:05.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.730+0000 7faf98600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf78004800 0x7faf901efc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.730+0000 7faf98600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faf90150920 0x7faf90152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.730+0000 7faf98600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf901f67f0 0x7faf901f5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.730+0000 7faf98600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faf90160490 con 0x7faf901f67f0 2024-09-10T21:46:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.730+0000 7faf98600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faf90160310 con 0x7faf78004800 2024-09-10T21:46:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.730+0000 7faf98600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7faf900ca980 con 0x7faf90150920 2024-09-10T21:46:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.731+0000 7faf96600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf901f67f0 0x7faf901f5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:05.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.731+0000 7faf96600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf901f67f0 0x7faf901f5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44810/0 (socket says 172.21.15.7:44810) 2024-09-10T21:46:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.731+0000 7faf96600640 1 -- 172.21.15.7:0/735146125 learned_addr learned my addr 172.21.15.7:0/735146125 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.731+0000 7faf95c00640 1 --2- 172.21.15.7:0/735146125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf78004800 0x7faf901efc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.731+0000 7faf95200640 1 --2- 172.21.15.7:0/735146125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faf90150920 0x7faf90152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.731+0000 7faf96600640 1 -- 172.21.15.7:0/735146125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faf90150920 msgr2=0x7faf90152080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.731+0000 7faf96600640 1 --2- 172.21.15.7:0/735146125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faf90150920 0x7faf90152080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:05.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.731+0000 7faf96600640 1 -- 172.21.15.7:0/735146125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf78004800 msgr2=0x7faf901efc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.731+0000 7faf96600640 1 --2- 172.21.15.7:0/735146125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf78004800 0x7faf901efc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.731+0000 7faf96600640 1 -- 172.21.15.7:0/735146125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9014f880 con 0x7faf901f67f0 2024-09-10T21:46:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.731+0000 7faf96600640 1 --2- 172.21.15.7:0/735146125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf901f67f0 0x7faf901f5c10 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7faf900701a0 tx=0x7faf8800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.732+0000 7faf8d600640 1 -- 172.21.15.7:0/735146125 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88017070 con 0x7faf901f67f0 2024-09-10T21:46:05.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.732+0000 7faf8d600640 1 -- 172.21.15.7:0/735146125 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf88015070 con 0x7faf901f67f0 2024-09-10T21:46:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.732+0000 7faf8d600640 1 -- 172.21.15.7:0/735146125 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88005280 con 0x7faf901f67f0 2024-09-10T21:46:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.732+0000 7faf98600640 1 -- 172.21.15.7:0/735146125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9014fb60 con 0x7faf901f67f0 2024-09-10T21:46:05.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.732+0000 7faf98600640 1 -- 172.21.15.7:0/735146125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf90150120 con 0x7faf901f67f0 2024-09-10T21:46:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.733+0000 7faf98600640 1 -- 172.21.15.7:0/735146125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf900caf10 con 0x7faf901f67f0 2024-09-10T21:46:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.734+0000 7faf8d600640 1 -- 172.21.15.7:0/735146125 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7faf88023030 con 0x7faf901f67f0 2024-09-10T21:46:05.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.735+0000 7faf8d600640 1 --2- 172.21.15.7:0/735146125 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7faf70077f70 0x7faf7007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.735+0000 7faf8d600640 1 -- 172.21.15.7:0/735146125 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7faf8809a380 con 0x7faf901f67f0 2024-09-10T21:46:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.735+0000 7faf95c00640 1 --2- 172.21.15.7:0/735146125 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7faf70077f70 0x7faf7007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.736+0000 7faf95c00640 1 --2- 172.21.15.7:0/735146125 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7faf70077f70 0x7faf7007a430 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7faf84001eb0 tx=0x7faf84006040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:05.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:05.739+0000 7faf8d600640 1 -- 172.21.15.7:0/735146125 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7faf88010e60 con 0x7faf901f67f0 2024-09-10T21:46:05.801 INFO:tasks.ceph.mds.b.smithi133.stderr:2024-09-10T21:46:05.801+0000 7ffbc4a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:46:06.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.004+0000 7faf98600640 1 -- 172.21.15.7:0/735146125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7faf900d0780 con 0x7faf901f67f0 2024-09-10T21:46:06.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.007+0000 7faf8d600640 1 -- 172.21.15.7:0/735146125 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v35) ==== 190+0+147 (secure 0 0 0) 0x7faf88062820 con 0x7faf901f67f0 2024-09-10T21:46:06.009 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-09-10T21:46:06.009 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDvvOBm0aqcFhAAtid2bnzjcpaYAbU7g0JcOA== 2024-09-10T21:46:06.009 INFO:teuthology.orchestra.run.smithi007.stdout: caps mds = "allow" 2024-09-10T21:46:06.009 INFO:teuthology.orchestra.run.smithi007.stdout: caps mon = "allow r" 2024-09-10T21:46:06.009 INFO:teuthology.orchestra.run.smithi007.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:46:06.010 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-09-10T21:46:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.010+0000 7faf98600640 1 -- 172.21.15.7:0/735146125 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7faf70077f70 msgr2=0x7faf7007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:06.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.010+0000 7faf98600640 1 --2- 172.21.15.7:0/735146125 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7faf70077f70 0x7faf7007a430 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7faf84001eb0 tx=0x7faf84006040 comp rx=0 tx=0).stop 2024-09-10T21:46:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.010+0000 7faf98600640 1 -- 172.21.15.7:0/735146125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf901f67f0 msgr2=0x7faf901f5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.010+0000 7faf98600640 1 --2- 172.21.15.7:0/735146125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf901f67f0 0x7faf901f5c10 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7faf900701a0 tx=0x7faf8800e720 comp rx=0 tx=0).stop 2024-09-10T21:46:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.010+0000 7faf98600640 1 -- 172.21.15.7:0/735146125 shutdown_connections 2024-09-10T21:46:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.010+0000 7faf98600640 1 --2- 172.21.15.7:0/735146125 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7faf70077f70 0x7faf7007a430 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.010+0000 7faf98600640 1 --2- 172.21.15.7:0/735146125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faf901f67f0 0x7faf901f5c10 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.010+0000 7faf98600640 1 --2- 172.21.15.7:0/735146125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faf90150920 0x7faf90152080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.010+0000 7faf98600640 1 --2- 172.21.15.7:0/735146125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faf78004800 0x7faf901efc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.010+0000 7faf98600640 1 -- 172.21.15.7:0/735146125 >> 172.21.15.7:0/735146125 conn(0x7faf900b16d0 msgr2=0x7faf9015d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.010+0000 7faf98600640 1 -- 172.21.15.7:0/735146125 shutdown_connections 2024-09-10T21:46:06.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.010+0000 7faf98600640 1 -- 172.21.15.7:0/735146125 wait complete. 2024-09-10T21:46:06.029 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:46:06.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.187+0000 7fcc43c00640 1 -- 172.21.15.7:0/3979437426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc3c162940 msgr2=0x7fcc3c161150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:06.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.187+0000 7fcc43c00640 1 --2- 172.21.15.7:0/3979437426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc3c162940 0x7fcc3c161150 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fcc28005fd0 tx=0x7fcc2802a5d0 comp rx=0 tx=0).stop 2024-09-10T21:46:06.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.187+0000 7fcc43c00640 1 -- 172.21.15.7:0/3979437426 shutdown_connections 2024-09-10T21:46:06.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.187+0000 7fcc43c00640 1 --2- 172.21.15.7:0/3979437426 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc3c162360 0x7fcc3c161690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:06.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.187+0000 7fcc43c00640 1 --2- 172.21.15.7:0/3979437426 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc3c162940 0x7fcc3c161150 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:06.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.187+0000 7fcc43c00640 1 -- 172.21.15.7:0/3979437426 >> 172.21.15.7:0/3979437426 conn(0x7fcc3c09d260 msgr2=0x7fcc3c0d3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:06.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.187+0000 7fcc43c00640 1 -- 172.21.15.7:0/3979437426 shutdown_connections 2024-09-10T21:46:06.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.187+0000 7fcc43c00640 1 -- 172.21.15.7:0/3979437426 wait complete. 2024-09-10T21:46:06.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.188+0000 7fcc43c00640 1 Processor -- start 2024-09-10T21:46:06.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.189+0000 7fcc43c00640 1 -- start start 2024-09-10T21:46:06.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.189+0000 7fcc43c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc3c161f80 0x7fcc3c0b81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:06.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.189+0000 7fcc43c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc3c162360 0x7fcc3c0d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:06.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.189+0000 7fcc43c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc3c162940 0x7fcc3c0d1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:06.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.190+0000 7fcc43c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c0b1180 con 0x7fcc3c162360 2024-09-10T21:46:06.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.190+0000 7fcc43c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcc3c0b1000 con 0x7fcc3c161f80 2024-09-10T21:46:06.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.190+0000 7fcc43c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fcc3c0b1300 con 0x7fcc3c162940 2024-09-10T21:46:06.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.190+0000 7fcc3be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc3c162360 0x7fcc3c0d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:06.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.190+0000 7fcc3be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc3c162360 0x7fcc3c0d0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44834/0 (socket says 172.21.15.7:44834) 2024-09-10T21:46:06.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.190+0000 7fcc3be00640 1 -- 172.21.15.7:0/201055727 learned_addr learned my addr 172.21.15.7:0/201055727 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:06.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.190+0000 7fcc41c00640 1 --2- 172.21.15.7:0/201055727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc3c162940 0x7fcc3c0d1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:06.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.190+0000 7fcc41200640 1 --2- 172.21.15.7:0/201055727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc3c161f80 0x7fcc3c0b81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:06.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.190+0000 7fcc3be00640 1 -- 172.21.15.7:0/201055727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc3c162940 msgr2=0x7fcc3c0d1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:06.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.190+0000 7fcc3be00640 1 --2- 172.21.15.7:0/201055727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc3c162940 0x7fcc3c0d1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:06.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.190+0000 7fcc3be00640 1 -- 172.21.15.7:0/201055727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc3c161f80 msgr2=0x7fcc3c0b81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:06.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.190+0000 7fcc3be00640 1 --2- 172.21.15.7:0/201055727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc3c161f80 0x7fcc3c0b81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:06.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.190+0000 7fcc3be00640 1 -- 172.21.15.7:0/201055727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc3c156f70 con 0x7fcc3c162360 2024-09-10T21:46:06.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.190+0000 7fcc41c00640 1 --2- 172.21.15.7:0/201055727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc3c162940 0x7fcc3c0d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:46:06.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.190+0000 7fcc41200640 1 --2- 172.21.15.7:0/201055727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc3c161f80 0x7fcc3c0b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:46:06.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.190+0000 7fcc3be00640 1 --2- 172.21.15.7:0/201055727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc3c162360 0x7fcc3c0d0ec0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fcc28034040 tx=0x7fcc280038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:06.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.191+0000 7fcc38c00640 1 -- 172.21.15.7:0/201055727 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc2803e030 con 0x7fcc3c162360 2024-09-10T21:46:06.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.191+0000 7fcc38c00640 1 -- 172.21.15.7:0/201055727 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc28003ca0 con 0x7fcc3c162360 2024-09-10T21:46:06.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.191+0000 7fcc38c00640 1 -- 172.21.15.7:0/201055727 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc28035040 con 0x7fcc3c162360 2024-09-10T21:46:06.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.191+0000 7fcc43c00640 1 -- 172.21.15.7:0/201055727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc3c0b8820 con 0x7fcc3c162360 2024-09-10T21:46:06.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.191+0000 7fcc43c00640 1 -- 172.21.15.7:0/201055727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc3c0b8de0 con 0x7fcc3c162360 2024-09-10T21:46:06.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.192+0000 7fcc43c00640 1 -- 172.21.15.7:0/201055727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc3c0b3e70 con 0x7fcc3c162360 2024-09-10T21:46:06.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.193+0000 7fcc38c00640 1 -- 172.21.15.7:0/201055727 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fcc280049e0 con 0x7fcc3c162360 2024-09-10T21:46:06.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.194+0000 7fcc38c00640 1 --2- 172.21.15.7:0/201055727 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcc1c077fa0 0x7fcc1c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:06.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.194+0000 7fcc38c00640 1 -- 172.21.15.7:0/201055727 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fcc280ba540 con 0x7fcc3c162360 2024-09-10T21:46:06.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.195+0000 7fcc41200640 1 --2- 172.21.15.7:0/201055727 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcc1c077fa0 0x7fcc1c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:06.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.195+0000 7fcc41200640 1 --2- 172.21.15.7:0/201055727 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcc1c077fa0 0x7fcc1c07a460 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0b3be0 tx=0x7fcc30009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:06.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.198+0000 7fcc38c00640 1 -- 172.21.15.7:0/201055727 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fcc3c156f70 con 0x7fcc3c162360 2024-09-10T21:46:06.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.431+0000 7fcc43c00640 1 -- 172.21.15.7:0/201055727 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc3c08d6e0 con 0x7fcc3c162360 2024-09-10T21:46:06.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.432+0000 7fcc38c00640 1 -- 172.21.15.7:0/201055727 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5235 (secure 0 0 0) 0x7fcc28082fa0 con 0x7fcc3c162360 2024-09-10T21:46:06.434 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:46:06.436 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":46,"btime":"2024-09-10T21:46:03:853860+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5884,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3488410478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3488410478},{"type":"v1","addr":"172.21.15.133:6837","nonce":3488410478}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5890,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3660400727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3660400727},{"type":"v1","addr":"172.21.15.7:6835","nonce":3660400727}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2388873704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2388873704},{"type":"v1","addr":"172.21.15.7:6834","nonce":2388873704}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:46:00.260975+0000","modified":"2024-09-10T21:46:03.290119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235361":{"gid":235361,"name":"b","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6836/2752881644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2752881644},{"type":"v1","addr":"172.21.15.133:6836","nonce":2752881644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235361,"qdb_cluster":[235361]},"id":5}]} 2024-09-10T21:46:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 46 2024-09-10T21:46:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.434+0000 7fcc43c00640 1 -- 172.21.15.7:0/201055727 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcc1c077fa0 msgr2=0x7fcc1c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.434+0000 7fcc43c00640 1 --2- 172.21.15.7:0/201055727 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcc1c077fa0 0x7fcc1c07a460 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c0b3be0 tx=0x7fcc30009040 comp rx=0 tx=0).stop 2024-09-10T21:46:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.434+0000 7fcc43c00640 1 -- 172.21.15.7:0/201055727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc3c162360 msgr2=0x7fcc3c0d0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:06.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.434+0000 7fcc43c00640 1 --2- 172.21.15.7:0/201055727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc3c162360 0x7fcc3c0d0ec0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fcc28034040 tx=0x7fcc280038a0 comp rx=0 tx=0).stop 2024-09-10T21:46:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.434+0000 7fcc43c00640 1 -- 172.21.15.7:0/201055727 shutdown_connections 2024-09-10T21:46:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.434+0000 7fcc43c00640 1 --2- 172.21.15.7:0/201055727 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcc1c077fa0 0x7fcc1c07a460 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.434+0000 7fcc43c00640 1 --2- 172.21.15.7:0/201055727 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc3c162940 0x7fcc3c0d1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.434+0000 7fcc43c00640 1 --2- 172.21.15.7:0/201055727 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc3c162360 0x7fcc3c0d0ec0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.434+0000 7fcc43c00640 1 --2- 172.21.15.7:0/201055727 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc3c161f80 0x7fcc3c0b81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.434+0000 7fcc43c00640 1 -- 172.21.15.7:0/201055727 >> 172.21.15.7:0/201055727 conn(0x7fcc3c09d260 msgr2=0x7fcc3c15fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.434+0000 7fcc43c00640 1 -- 172.21.15.7:0/201055727 shutdown_connections 2024-09-10T21:46:06.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:06.434+0000 7fcc43c00640 1 -- 172.21.15.7:0/201055727 wait complete. 2024-09-10T21:46:06.856 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:46:00.260975+0000', 'modified': '2024-09-10T21:46:03.290119+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 235361}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_235361': {'gid': 235361, 'name': 'b', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6836/2752881644', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 2752881644}, {'type': 'v1', 'addr': '172.21.15.133:6836', 'nonce': 2752881644}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 235361, 'qdb_cluster': [235361]} 2024-09-10T21:46:06.856 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:46:06.856 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T21:46:07.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.019+0000 7f00e3000640 1 -- 172.21.15.7:0/1516593682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00dc15dfd0 msgr2=0x7f00dc0d4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:07.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.019+0000 7f00e3000640 1 --2- 172.21.15.7:0/1516593682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00dc15dfd0 0x7f00dc0d4870 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f00dc074d80 tx=0x7f00d002a4a0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.019+0000 7f00e3000640 1 -- 172.21.15.7:0/1516593682 shutdown_connections 2024-09-10T21:46:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.019+0000 7f00e3000640 1 --2- 172.21.15.7:0/1516593682 >> v2:172.21.15.133:3300/0 conn(0x7f00dc0b2b50 0x7f00dc0d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.019+0000 7f00e3000640 1 --2- 172.21.15.7:0/1516593682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00dc15dfd0 0x7f00dc0d4870 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.019+0000 7f00e3000640 1 -- 172.21.15.7:0/1516593682 >> 172.21.15.7:0/1516593682 conn(0x7f00dc0d3d50 msgr2=0x7f00dc0d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.019+0000 7f00e3000640 1 -- 172.21.15.7:0/1516593682 shutdown_connections 2024-09-10T21:46:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.019+0000 7f00e3000640 1 -- 172.21.15.7:0/1516593682 wait complete. 2024-09-10T21:46:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.020+0000 7f00e3000640 1 Processor -- start 2024-09-10T21:46:07.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.021+0000 7f00e3000640 1 -- start start 2024-09-10T21:46:07.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.021+0000 7f00e3000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00dc0b2b50 0x7f00dc160d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.021+0000 7f00e3000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00dc0d0ca0 0x7f00dc1f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.021+0000 7f00e3000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00dc15dfd0 0x7f00dc0b8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.021+0000 7f00e3000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc0b08c0 con 0x7f00dc0d0ca0 2024-09-10T21:46:07.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.021+0000 7f00e3000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc0d2ef0 con 0x7f00dc15dfd0 2024-09-10T21:46:07.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.021+0000 7f00e3000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f00dc0b0a40 con 0x7f00dc0b2b50 2024-09-10T21:46:07.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.022+0000 7f00e1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00dc0d0ca0 0x7f00dc1f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.022+0000 7f00e1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00dc0d0ca0 0x7f00dc1f5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44860/0 (socket says 172.21.15.7:44860) 2024-09-10T21:46:07.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.022+0000 7f00e1200640 1 -- 172.21.15.7:0/1940665034 learned_addr learned my addr 172.21.15.7:0/1940665034 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:07.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.022+0000 7f00e1c00640 1 --2- 172.21.15.7:0/1940665034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00dc0b2b50 0x7f00dc160d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.022+0000 7f00e2600640 1 --2- 172.21.15.7:0/1940665034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00dc15dfd0 0x7f00dc0b8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.022+0000 7f00e1200640 1 -- 172.21.15.7:0/1940665034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00dc0b2b50 msgr2=0x7f00dc160d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:07.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.022+0000 7f00e1200640 1 --2- 172.21.15.7:0/1940665034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00dc0b2b50 0x7f00dc160d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.022+0000 7f00e1200640 1 -- 172.21.15.7:0/1940665034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00dc15dfd0 msgr2=0x7f00dc0b8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:07.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.022+0000 7f00e1200640 1 --2- 172.21.15.7:0/1940665034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00dc15dfd0 0x7f00dc0b8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.022+0000 7f00e1200640 1 -- 172.21.15.7:0/1940665034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00dc161280 con 0x7f00dc0d0ca0 2024-09-10T21:46:07.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.022+0000 7f00e1200640 1 --2- 172.21.15.7:0/1940665034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00dc0d0ca0 0x7f00dc1f5790 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f00c800d950 tx=0x7f00c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:07.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.023+0000 7f00d9600640 1 -- 172.21.15.7:0/1940665034 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8017070 con 0x7f00dc0d0ca0 2024-09-10T21:46:07.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.023+0000 7f00d9600640 1 -- 172.21.15.7:0/1940665034 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00c8009d20 con 0x7f00dc0d0ca0 2024-09-10T21:46:07.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.023+0000 7f00d9600640 1 -- 172.21.15.7:0/1940665034 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8004280 con 0x7f00dc0d0ca0 2024-09-10T21:46:07.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.023+0000 7f00e3000640 1 -- 172.21.15.7:0/1940665034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00dc0b6dd0 con 0x7f00dc0d0ca0 2024-09-10T21:46:07.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.023+0000 7f00e3000640 1 -- 172.21.15.7:0/1940665034 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00dc0b7390 con 0x7f00dc0d0ca0 2024-09-10T21:46:07.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.025+0000 7f00d9600640 1 -- 172.21.15.7:0/1940665034 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f00c80035f0 con 0x7f00dc0d0ca0 2024-09-10T21:46:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.026+0000 7f00d9600640 1 --2- 172.21.15.7:0/1940665034 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f00bc077f70 0x7f00bc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.026+0000 7f00d9600640 1 -- 172.21.15.7:0/1940665034 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f00c8099ee0 con 0x7f00dc0d0ca0 2024-09-10T21:46:07.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.026+0000 7f00e1c00640 1 --2- 172.21.15.7:0/1940665034 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f00bc077f70 0x7f00bc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.027+0000 7f00e7427740 1 Processor -- start 2024-09-10T21:46:07.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.027+0000 7f00e7427740 1 -- start start 2024-09-10T21:46:07.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.027+0000 7f00e1c00640 1 --2- 172.21.15.7:0/1940665034 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f00bc077f70 0x7f00bc07a430 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f00d0034000 tx=0x7f00d0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:07.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.028+0000 7f00e7427740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b341acaa0 0x556b341ace80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.028+0000 7f00e7427740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b34167c10 0x556b341ab580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.028+0000 7f00e7427740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b34168190 0x556b341abac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.028+0000 7f00e7427740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556b340ea430 con 0x556b34168190 2024-09-10T21:46:07.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.028+0000 7f00e7427740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556b340da5d0 con 0x556b34167c10 2024-09-10T21:46:07.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.028+0000 7f00e7427740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x556b340c9a00 con 0x556b341acaa0 2024-09-10T21:46:07.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.028+0000 7f00e2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b34168190 0x556b341abac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.028+0000 7f00e2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b34168190 0x556b341abac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44866/0 (socket says 172.21.15.7:44866) 2024-09-10T21:46:07.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.028+0000 7f00e2600640 1 -- 172.21.15.7:0/3345359156 learned_addr learned my addr 172.21.15.7:0/3345359156 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:07.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.028+0000 7f00e1200640 1 --2- 172.21.15.7:0/3345359156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b34167c10 0x556b341ab580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.028+0000 7f00e1c00640 1 --2- 172.21.15.7:0/3345359156 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b341acaa0 0x556b341ace80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.028+0000 7f00e2600640 1 -- 172.21.15.7:0/3345359156 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b341acaa0 msgr2=0x556b341ace80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:07.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.028+0000 7f00e2600640 1 --2- 172.21.15.7:0/3345359156 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b341acaa0 0x556b341ace80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.028+0000 7f00e2600640 1 -- 172.21.15.7:0/3345359156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b34167c10 msgr2=0x556b341ab580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:07.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.028+0000 7f00e2600640 1 --2- 172.21.15.7:0/3345359156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b34167c10 0x556b341ab580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.028+0000 7f00e2600640 1 -- 172.21.15.7:0/3345359156 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b34092520 con 0x556b34168190 2024-09-10T21:46:07.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.029+0000 7f00e2600640 1 --2- 172.21.15.7:0/3345359156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b34168190 0x556b341abac0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f00d4002080 tx=0x7f00d400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df6376707870f57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:07.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.029+0000 7f00ce000640 1 -- 172.21.15.7:0/3345359156 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4004a40 con 0x556b34168190 2024-09-10T21:46:07.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.029+0000 7f00ce000640 1 -- 172.21.15.7:0/3345359156 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00d4004be0 con 0x556b34168190 2024-09-10T21:46:07.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.029+0000 7f00ce000640 1 -- 172.21.15.7:0/3345359156 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4004dd0 con 0x556b34168190 2024-09-10T21:46:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.030+0000 7f00e7427740 1 -- 172.21.15.7:0/3345359156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b34168190 msgr2=0x556b341abac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.030+0000 7f00e7427740 1 --2- 172.21.15.7:0/3345359156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b34168190 0x556b341abac0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f00d4002080 tx=0x7f00d400e530 comp rx=0 tx=0).stop 2024-09-10T21:46:07.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.030+0000 7f00e7427740 1 -- 172.21.15.7:0/3345359156 shutdown_connections 2024-09-10T21:46:07.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.030+0000 7f00e7427740 1 --2- 172.21.15.7:0/3345359156 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b34167c10 0x556b341ab580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.030+0000 7f00e7427740 1 --2- 172.21.15.7:0/3345359156 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b34168190 0x556b341abac0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.030+0000 7f00e7427740 1 --2- 172.21.15.7:0/3345359156 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b341acaa0 0x556b341ace80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.030+0000 7f00e7427740 1 -- 172.21.15.7:0/3345359156 >> 172.21.15.7:0/3345359156 conn(0x556b341631e0 msgr2=0x556b34166c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:07.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.030+0000 7f00e7427740 1 -- 172.21.15.7:0/3345359156 shutdown_connections 2024-09-10T21:46:07.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.030+0000 7f00e7427740 1 -- 172.21.15.7:0/3345359156 wait complete. 2024-09-10T21:46:07.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.031+0000 7f00e7427740 1 -- start start 2024-09-10T21:46:07.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.031+0000 7f00e7427740 1 Processor -- start 2024-09-10T21:46:07.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.032+0000 7f00e7427740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b34168190 0x556b3417baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.032+0000 7f00e7427740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b341acaa0 0x556b3417e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.032+0000 7f00e7427740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b3417e860 0x556b3417c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.032+0000 7f00e7427740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556b34070330 con 0x556b34168190 2024-09-10T21:46:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.032+0000 7f00e7427740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556b340cd810 con 0x556b3417e860 2024-09-10T21:46:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.032+0000 7f00e7427740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x556b33e3a620 con 0x556b341acaa0 2024-09-10T21:46:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.032+0000 7f00e7427740 10 client.0 _ll_register_callbacks cb 0x556b34075420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:46:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.032+0000 7f00e1c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b34168190 0x556b3417baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.032+0000 7f00e1c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b34168190 0x556b3417baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44882/0 (socket says 172.21.15.7:44882) 2024-09-10T21:46:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.032+0000 7f00e1c00640 1 -- 172.21.15.7:0/3936737897 learned_addr learned my addr 172.21.15.7:0/3936737897 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.032+0000 7f00e1200640 1 --2- 172.21.15.7:0/3936737897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b341acaa0 0x556b3417e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.032+0000 7f00e2600640 1 --2- 172.21.15.7:0/3936737897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b3417e860 0x556b3417c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.032+0000 7f00e1c00640 1 -- 172.21.15.7:0/3936737897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b341acaa0 msgr2=0x556b3417e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.032+0000 7f00e1c00640 1 --2- 172.21.15.7:0/3936737897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b341acaa0 0x556b3417e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.032+0000 7f00e1c00640 1 -- 172.21.15.7:0/3936737897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b3417e860 msgr2=0x556b3417c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:07.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.032+0000 7f00e1c00640 1 --2- 172.21.15.7:0/3936737897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b3417e860 0x556b3417c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.032+0000 7f00e1c00640 1 -- 172.21.15.7:0/3936737897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b33e2b850 con 0x556b34168190 2024-09-10T21:46:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.032+0000 7f00e2600640 1 --2- 172.21.15.7:0/3936737897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b3417e860 0x556b3417c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.033+0000 7f00e1c00640 1 --2- 172.21.15.7:0/3936737897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b34168190 0x556b3417baa0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f00d002aa90 tx=0x7f00d0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61d9e870cb4616 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.033+0000 7f00c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:46:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.033+0000 7f00c2a00640 1 -- 172.21.15.7:0/3936737897 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d0049630 con 0x556b34168190 2024-09-10T21:46:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.033+0000 7f00c2a00640 1 -- 172.21.15.7:0/3936737897 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f00d00497d0 con 0x556b34168190 2024-09-10T21:46:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.033+0000 7f00e7427740 1 -- 172.21.15.7:0/3936737897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00d4004be0 con 0x556b34168190 2024-09-10T21:46:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.033+0000 7f00c2a00640 1 -- 172.21.15.7:0/3936737897 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f00d00499d0 con 0x556b34168190 2024-09-10T21:46:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.033+0000 7f00e7427740 10 client.6297 fetch_fsmap learned FSMap version 46 2024-09-10T21:46:07.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.033+0000 7f00e7427740 1 -- 172.21.15.7:0/3936737897 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x556b33ea26b0 con 0x556b34168190 2024-09-10T21:46:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.033+0000 7f00c2a00640 1 -- 172.21.15.7:0/3936737897 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00d4004be0 con 0x556b34168190 2024-09-10T21:46:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.034+0000 7f00c2a00640 1 -- 172.21.15.7:0/3936737897 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7f00d005b080 con 0x556b34168190 2024-09-10T21:46:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.034+0000 7f00e7427740 10 client.6297 fetch_fsmap finished waiting for FSMap version 46 2024-09-10T21:46:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.034+0000 7f00e7427740 10 client.6297 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:46:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.034+0000 7f00e7427740 20 client.6297 populate_metadata read hostname 'smithi007' 2024-09-10T21:46:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.034+0000 7f00e7427740 1 --2- 172.21.15.7:0/3936737897 >> [v2:172.21.15.133:6834/2752881644,v1:172.21.15.133:6836/2752881644] conn(0x556b34190990 0x556b34190db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.034+0000 7f00e7427740 4 client.6297 mds_command: new command op to 235361 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:46:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.034+0000 7f00e7427740 1 -- 172.21.15.7:0/3936737897 --> [v2:172.21.15.133:6834/2752881644,v1:172.21.15.133:6836/2752881644] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b340dc520 con 0x556b34190990 2024-09-10T21:46:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.034+0000 7f00e1200640 1 --2- 172.21.15.7:0/3936737897 >> [v2:172.21.15.133:6834/2752881644,v1:172.21.15.133:6836/2752881644] conn(0x556b34190990 0x556b34190db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.035+0000 7f00e1200640 1 --2- 172.21.15.7:0/3936737897 >> [v2:172.21.15.133:6834/2752881644,v1:172.21.15.133:6836/2752881644] conn(0x556b34190990 0x556b34190db0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89c4fb731f9ee11a server_cookie=9ae441d593adb2a3 in_seq=0 out_seq=0 2024-09-10T21:46:07.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.035+0000 7f00c2a00640 10 client.6297 ms_handle_connect on v2:172.21.15.133:6834/2752881644 2024-09-10T21:46:07.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.036+0000 7f00c2a00640 1 -- 172.21.15.7:0/3936737897 <== mds.0 v2:172.21.15.133:6834/2752881644 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556b340dc520 con 0x556b34190990 2024-09-10T21:46:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.036+0000 7f00c2a00640 10 client.6297 handle_command_reply: tid=0 2024-09-10T21:46:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.036+0000 7f00e7427740 1 client.6297 shutdown 2024-09-10T21:46:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.036+0000 7f00e7427740 1 -- 172.21.15.7:0/3936737897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b34168190 msgr2=0x556b3417baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.036+0000 7f00e7427740 1 --2- 172.21.15.7:0/3936737897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b34168190 0x556b3417baa0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f00d002aa90 tx=0x7f00d0005f90 comp rx=0 tx=0).stop 2024-09-10T21:46:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.036+0000 7f00e7427740 1 -- 172.21.15.7:0/3936737897 shutdown_connections 2024-09-10T21:46:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.036+0000 7f00e7427740 1 --2- 172.21.15.7:0/3936737897 >> [v2:172.21.15.133:6834/2752881644,v1:172.21.15.133:6836/2752881644] conn(0x556b34190990 0x556b34190db0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.036+0000 7f00e7427740 1 --2- 172.21.15.7:0/3936737897 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b3417e860 0x556b3417c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.036+0000 7f00e7427740 1 --2- 172.21.15.7:0/3936737897 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b341acaa0 0x556b3417e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.036+0000 7f00e7427740 1 --2- 172.21.15.7:0/3936737897 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b34168190 0x556b3417baa0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.036+0000 7f00c2a00640 0 client.6297 ms_handle_reset on v2:172.21.15.133:6834/2752881644 2024-09-10T21:46:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.036+0000 7f00e7427740 1 -- 172.21.15.7:0/3936737897 >> 172.21.15.7:0/3936737897 conn(0x556b34167c10 msgr2=0x556b341a5610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.036+0000 7f00e7427740 1 -- 172.21.15.7:0/3936737897 shutdown_connections 2024-09-10T21:46:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.036+0000 7f00e7427740 1 -- 172.21.15.7:0/3936737897 wait complete. 2024-09-10T21:46:07.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.037+0000 7f00e7427740 20 client.6297 trim_cache size 0 max 16384 2024-09-10T21:46:07.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.056+0000 7f00e7427740 1 Processor -- start 2024-09-10T21:46:07.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.056+0000 7f00e7427740 1 -- start start 2024-09-10T21:46:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.056+0000 7f00e7427740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b34168190 0x556b341a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.056+0000 7f00e7427740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b3417e860 0x556b341a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.056+0000 7f00e7427740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b341acaa0 0x556b341cf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.056+0000 7f00e7427740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc1f9140 con 0x556b341acaa0 2024-09-10T21:46:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.056+0000 7f00e7427740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc1f84a0 con 0x556b3417e860 2024-09-10T21:46:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.056+0000 7f00e7427740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f00dc1f8af0 con 0x556b34168190 2024-09-10T21:46:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.057+0000 7f00e2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b341acaa0 0x556b341cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.057+0000 7f00e2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b341acaa0 0x556b341cf180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44898/0 (socket says 172.21.15.7:44898) 2024-09-10T21:46:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.057+0000 7f00e2600640 1 -- 172.21.15.7:0/3851010624 learned_addr learned my addr 172.21.15.7:0/3851010624 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.057+0000 7f00e1c00640 1 --2- 172.21.15.7:0/3851010624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b34168190 0x556b341a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.057+0000 7f00e1200640 1 --2- 172.21.15.7:0/3851010624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b3417e860 0x556b341a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.057+0000 7f00e2600640 1 -- 172.21.15.7:0/3851010624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b34168190 msgr2=0x556b341a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.057+0000 7f00e2600640 1 --2- 172.21.15.7:0/3851010624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b34168190 0x556b341a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.057+0000 7f00e2600640 1 -- 172.21.15.7:0/3851010624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b3417e860 msgr2=0x556b341a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.057+0000 7f00e2600640 1 --2- 172.21.15.7:0/3851010624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b3417e860 0x556b341a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.057+0000 7f00e2600640 1 -- 172.21.15.7:0/3851010624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b3417f280 con 0x556b341acaa0 2024-09-10T21:46:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.057+0000 7f00e1200640 1 --2- 172.21.15.7:0/3851010624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b3417e860 0x556b341a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.057+0000 7f00e2600640 1 --2- 172.21.15.7:0/3851010624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b341acaa0 0x556b341cf180 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x556b341ae120 tx=0x7f00d4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d12bcf19e1ae3d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.058+0000 7f00cd600640 1 -- 172.21.15.7:0/3851010624 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4014480 con 0x556b341acaa0 2024-09-10T21:46:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.058+0000 7f00cd600640 1 -- 172.21.15.7:0/3851010624 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00d4014750 con 0x556b341acaa0 2024-09-10T21:46:07.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.058+0000 7f00cd600640 1 -- 172.21.15.7:0/3851010624 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4014ba0 con 0x556b341acaa0 2024-09-10T21:46:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.058+0000 7f00e7427740 1 -- 172.21.15.7:0/3851010624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b341acaa0 msgr2=0x556b341cf180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.058+0000 7f00e7427740 1 --2- 172.21.15.7:0/3851010624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b341acaa0 0x556b341cf180 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x556b341ae120 tx=0x7f00d4011000 comp rx=0 tx=0).stop 2024-09-10T21:46:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.058+0000 7f00e7427740 1 -- 172.21.15.7:0/3851010624 shutdown_connections 2024-09-10T21:46:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.058+0000 7f00e7427740 1 --2- 172.21.15.7:0/3851010624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b3417e860 0x556b341a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.058+0000 7f00e7427740 1 --2- 172.21.15.7:0/3851010624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b341acaa0 0x556b341cf180 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.058+0000 7f00e7427740 1 --2- 172.21.15.7:0/3851010624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b34168190 0x556b341a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.058+0000 7f00e7427740 1 -- 172.21.15.7:0/3851010624 >> 172.21.15.7:0/3851010624 conn(0x556b34167c10 msgr2=0x556b34161e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.058+0000 7f00e7427740 1 -- 172.21.15.7:0/3851010624 shutdown_connections 2024-09-10T21:46:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.058+0000 7f00e7427740 1 -- 172.21.15.7:0/3851010624 wait complete. 2024-09-10T21:46:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.059+0000 7f00e7427740 1 -- start start 2024-09-10T21:46:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.059+0000 7f00e7427740 1 Processor -- start 2024-09-10T21:46:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.060+0000 7f00e7427740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b34168190 0x556b3422fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.060+0000 7f00e7427740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b3417e860 0x556b341cf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.060+0000 7f00e7427740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b341acaa0 0x556b341cf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.060+0000 7f00e7427740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556b3417ce30 con 0x556b3417e860 2024-09-10T21:46:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.060+0000 7f00e7427740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556b341ab2c0 con 0x556b34168190 2024-09-10T21:46:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.060+0000 7f00e7427740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x556b34166bc0 con 0x556b341acaa0 2024-09-10T21:46:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.060+0000 7f00e7427740 10 client.0 _ll_register_callbacks cb 0x7f00d0003750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:46:07.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.060+0000 7f00e1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b3417e860 0x556b341cf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.060+0000 7f00e1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b3417e860 0x556b341cf460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44910/0 (socket says 172.21.15.7:44910) 2024-09-10T21:46:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.060+0000 7f00e1200640 1 -- 172.21.15.7:0/2270319708 learned_addr learned my addr 172.21.15.7:0/2270319708 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.061+0000 7f00e2600640 1 --2- 172.21.15.7:0/2270319708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b341acaa0 0x556b341cf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.061+0000 7f00e1c00640 1 --2- 172.21.15.7:0/2270319708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b34168190 0x556b3422fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.061+0000 7f00e1200640 1 -- 172.21.15.7:0/2270319708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b341acaa0 msgr2=0x556b341cf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.061+0000 7f00e1200640 1 --2- 172.21.15.7:0/2270319708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b341acaa0 0x556b341cf9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.061+0000 7f00e1200640 1 -- 172.21.15.7:0/2270319708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b34168190 msgr2=0x556b3422fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.061+0000 7f00e1200640 1 --2- 172.21.15.7:0/2270319708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b34168190 0x556b3422fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.061+0000 7f00e1200640 1 -- 172.21.15.7:0/2270319708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b341bf940 con 0x556b3417e860 2024-09-10T21:46:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.061+0000 7f00e1200640 1 --2- 172.21.15.7:0/2270319708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b3417e860 0x556b341cf460 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f00c8098400 tx=0x7f00c806b110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f8d5460d13e748 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.061+0000 7f00c2a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:46:07.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.061+0000 7f00c2a00640 1 -- 172.21.15.7:0/2270319708 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c8020040 con 0x556b3417e860 2024-09-10T21:46:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.061+0000 7f00c2a00640 1 -- 172.21.15.7:0/2270319708 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f00c806b340 con 0x556b3417e860 2024-09-10T21:46:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.061+0000 7f00e7427740 1 -- 172.21.15.7:0/2270319708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00d4014750 con 0x556b3417e860 2024-09-10T21:46:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.061+0000 7f00c2a00640 1 -- 172.21.15.7:0/2270319708 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f00c80622d0 con 0x556b3417e860 2024-09-10T21:46:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.061+0000 7f00e7427740 10 client.6303 fetch_fsmap learned FSMap version 46 2024-09-10T21:46:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.061+0000 7f00e7427740 1 -- 172.21.15.7:0/2270319708 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x556b341c1600 con 0x556b3417e860 2024-09-10T21:46:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.062+0000 7f00c2a00640 1 -- 172.21.15.7:0/2270319708 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00d4014750 con 0x556b3417e860 2024-09-10T21:46:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.062+0000 7f00c2a00640 1 -- 172.21.15.7:0/2270319708 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7f00c8061530 con 0x556b3417e860 2024-09-10T21:46:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.062+0000 7f00e7427740 10 client.6303 fetch_fsmap finished waiting for FSMap version 46 2024-09-10T21:46:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.062+0000 7f00e7427740 10 client.6303 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:46:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.062+0000 7f00e7427740 20 client.6303 populate_metadata read hostname 'smithi007' 2024-09-10T21:46:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.062+0000 7f00e7427740 1 --2- 172.21.15.7:0/2270319708 >> [v2:172.21.15.133:6834/2752881644,v1:172.21.15.133:6836/2752881644] conn(0x556b34231210 0x556b3417cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.062+0000 7f00e7427740 4 client.6303 mds_command: new command op to 235361 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:46:07.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.062+0000 7f00e7427740 1 -- 172.21.15.7:0/2270319708 --> [v2:172.21.15.133:6834/2752881644,v1:172.21.15.133:6836/2752881644] -- command(tid 0: {"prefix": "status"}) -- 0x556b340d7fd0 con 0x556b34231210 2024-09-10T21:46:07.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.062+0000 7f00e1c00640 1 --2- 172.21.15.7:0/2270319708 >> [v2:172.21.15.133:6834/2752881644,v1:172.21.15.133:6836/2752881644] conn(0x556b34231210 0x556b3417cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.063+0000 7f00e1c00640 1 --2- 172.21.15.7:0/2270319708 >> [v2:172.21.15.133:6834/2752881644,v1:172.21.15.133:6836/2752881644] conn(0x556b34231210 0x556b3417cfb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=344216b06cb7996c server_cookie=a409fb9986a4fc53 in_seq=0 out_seq=0 2024-09-10T21:46:07.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.063+0000 7f00c2a00640 10 client.6303 ms_handle_connect on v2:172.21.15.133:6834/2752881644 2024-09-10T21:46:07.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.063+0000 7f00c2a00640 1 -- 172.21.15.7:0/2270319708 <== mds.0 v2:172.21.15.133:6834/2752881644 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x556b340d7fd0 con 0x556b34231210 2024-09-10T21:46:07.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.063+0000 7f00c2a00640 10 client.6303 handle_command_reply: tid=0 2024-09-10T21:46:07.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.063+0000 7f00e7427740 1 client.6303 shutdown 2024-09-10T21:46:07.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.063+0000 7f00e7427740 1 -- 172.21.15.7:0/2270319708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b3417e860 msgr2=0x556b341cf460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:07.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.063+0000 7f00e7427740 1 --2- 172.21.15.7:0/2270319708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b3417e860 0x556b341cf460 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f00c8098400 tx=0x7f00c806b110 comp rx=0 tx=0).stop 2024-09-10T21:46:07.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.063+0000 7f00e7427740 1 -- 172.21.15.7:0/2270319708 shutdown_connections 2024-09-10T21:46:07.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.063+0000 7f00e7427740 1 --2- 172.21.15.7:0/2270319708 >> [v2:172.21.15.133:6834/2752881644,v1:172.21.15.133:6836/2752881644] conn(0x556b34231210 0x556b3417cfb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.063+0000 7f00e7427740 1 --2- 172.21.15.7:0/2270319708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556b341acaa0 0x556b341cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.063+0000 7f00c2a00640 0 client.6303 ms_handle_reset on v2:172.21.15.133:6834/2752881644 2024-09-10T21:46:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.063+0000 7f00e7427740 1 --2- 172.21.15.7:0/2270319708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556b3417e860 0x556b341cf460 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.063+0000 7f00e7427740 1 --2- 172.21.15.7:0/2270319708 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556b34168190 0x556b3422fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.063+0000 7f00e7427740 1 -- 172.21.15.7:0/2270319708 >> 172.21.15.7:0/2270319708 conn(0x556b34167c10 msgr2=0x556b3415c770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.063+0000 7f00e7427740 1 -- 172.21.15.7:0/2270319708 shutdown_connections 2024-09-10T21:46:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.063+0000 7f00e7427740 1 -- 172.21.15.7:0/2270319708 wait complete. 2024-09-10T21:46:07.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.064+0000 7f00e7427740 20 client.6303 trim_cache size 0 max 16384 2024-09-10T21:46:07.070 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:46:07.070 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2029451a-3c26-48c7-b874-0597197278ac", 2024-09-10T21:46:07.070 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-10T21:46:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 235361, 2024-09-10T21:46:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T21:46:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T21:46:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T21:46:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.2259506699999996, 2024-09-10T21:46:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 46, 2024-09-10T21:46:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 70, 2024-09-10T21:46:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 70, 2024-09-10T21:46:07.071 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 11.521051569000001 2024-09-10T21:46:07.071 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:46:07.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.064+0000 7f00e3000640 1 -- 172.21.15.7:0/1940665034 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f00bc077f70 msgr2=0x7f00bc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:07.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.064+0000 7f00e3000640 1 --2- 172.21.15.7:0/1940665034 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f00bc077f70 0x7f00bc07a430 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f00d0034000 tx=0x7f00d0004730 comp rx=0 tx=0).stop 2024-09-10T21:46:07.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.064+0000 7f00e3000640 1 -- 172.21.15.7:0/1940665034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00dc0d0ca0 msgr2=0x7f00dc1f5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:07.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.064+0000 7f00e3000640 1 --2- 172.21.15.7:0/1940665034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00dc0d0ca0 0x7f00dc1f5790 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f00c800d950 tx=0x7f00c800de10 comp rx=0 tx=0).stop 2024-09-10T21:46:07.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.064+0000 7f00e3000640 1 -- 172.21.15.7:0/1940665034 shutdown_connections 2024-09-10T21:46:07.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.064+0000 7f00e3000640 1 --2- 172.21.15.7:0/1940665034 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f00bc077f70 0x7f00bc07a430 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.064+0000 7f00e3000640 1 --2- 172.21.15.7:0/1940665034 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f00dc15dfd0 0x7f00dc0b8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.064+0000 7f00e3000640 1 --2- 172.21.15.7:0/1940665034 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f00dc0d0ca0 0x7f00dc1f5790 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.064+0000 7f00e3000640 1 --2- 172.21.15.7:0/1940665034 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f00dc0b2b50 0x7f00dc160d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.064+0000 7f00e3000640 1 -- 172.21.15.7:0/1940665034 >> 172.21.15.7:0/1940665034 conn(0x7f00dc0d3d50 msgr2=0x7f00dc0d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.064+0000 7f00e3000640 1 -- 172.21.15.7:0/1940665034 shutdown_connections 2024-09-10T21:46:07.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.064+0000 7f00e3000640 1 -- 172.21.15.7:0/1940665034 wait complete. 2024-09-10T21:46:07.082 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:46:07.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.240+0000 7f1416200640 1 -- 172.21.15.7:0/4067957474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14100ce6a0 msgr2=0x7f14100ceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:07.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.240+0000 7f1416200640 1 --2- 172.21.15.7:0/4067957474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14100ce6a0 0x7f14100ceb50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f1408005fd0 tx=0x7f140802aa00 comp rx=0 tx=0).stop 2024-09-10T21:46:07.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.240+0000 7f1416200640 1 -- 172.21.15.7:0/4067957474 shutdown_connections 2024-09-10T21:46:07.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.240+0000 7f1416200640 1 --2- 172.21.15.7:0/4067957474 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14100ce6a0 0x7f14100ceb50 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.240+0000 7f1416200640 1 --2- 172.21.15.7:0/4067957474 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f14100cdc00 0x7f14100ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.240+0000 7f1416200640 1 -- 172.21.15.7:0/4067957474 >> 172.21.15.7:0/4067957474 conn(0x7f14100d1d20 msgr2=0x7f14100d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:07.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.240+0000 7f1416200640 1 -- 172.21.15.7:0/4067957474 shutdown_connections 2024-09-10T21:46:07.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.240+0000 7f1416200640 1 -- 172.21.15.7:0/4067957474 wait complete. 2024-09-10T21:46:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.242+0000 7f1416200640 1 Processor -- start 2024-09-10T21:46:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.242+0000 7f1416200640 1 -- start start 2024-09-10T21:46:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.243+0000 7f1416200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14100cdc00 0x7f14100c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.243+0000 7f1416200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14100ce6a0 0x7f14100b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.243+0000 7f1416200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1410161640 0x7f14100b5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.243+0000 7f1416200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f141006daf0 con 0x7f14100ce6a0 2024-09-10T21:46:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.243+0000 7f1416200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f141006d970 con 0x7f14100cdc00 2024-09-10T21:46:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.243+0000 7f1416200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f141006dc70 con 0x7f1410161640 2024-09-10T21:46:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.243+0000 7f140ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14100ce6a0 0x7f14100b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.243+0000 7f140ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14100ce6a0 0x7f14100b4b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44920/0 (socket says 172.21.15.7:44920) 2024-09-10T21:46:07.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.243+0000 7f140ea00640 1 -- 172.21.15.7:0/3758525111 learned_addr learned my addr 172.21.15.7:0/3758525111 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:07.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.243+0000 7f140f400640 1 --2- 172.21.15.7:0/3758525111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14100cdc00 0x7f14100c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.243+0000 7f140fe00640 1 --2- 172.21.15.7:0/3758525111 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1410161640 0x7f14100b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.243+0000 7f140ea00640 1 -- 172.21.15.7:0/3758525111 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1410161640 msgr2=0x7f14100b5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:07.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.243+0000 7f140ea00640 1 --2- 172.21.15.7:0/3758525111 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1410161640 0x7f14100b5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.243+0000 7f140ea00640 1 -- 172.21.15.7:0/3758525111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14100cdc00 msgr2=0x7f14100c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:07.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.243+0000 7f140ea00640 1 --2- 172.21.15.7:0/3758525111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14100cdc00 0x7f14100c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.244+0000 7f140ea00640 1 -- 172.21.15.7:0/3758525111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14101f5a80 con 0x7f14100ce6a0 2024-09-10T21:46:07.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.244+0000 7f140fe00640 1 --2- 172.21.15.7:0/3758525111 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1410161640 0x7f14100b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:07.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.244+0000 7f140ea00640 1 --2- 172.21.15.7:0/3758525111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14100ce6a0 0x7f14100b4b50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00d980 tx=0x7f13fc00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:07.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.244+0000 7f1407400640 1 -- 172.21.15.7:0/3758525111 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc017070 con 0x7f14100ce6a0 2024-09-10T21:46:07.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.244+0000 7f1407400640 1 -- 172.21.15.7:0/3758525111 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13fc009d20 con 0x7f14100ce6a0 2024-09-10T21:46:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.244+0000 7f1407400640 1 -- 172.21.15.7:0/3758525111 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13fc0106c0 con 0x7f14100ce6a0 2024-09-10T21:46:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.244+0000 7f1416200640 1 -- 172.21.15.7:0/3758525111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1410147f40 con 0x7f14100ce6a0 2024-09-10T21:46:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.244+0000 7f1416200640 1 -- 172.21.15.7:0/3758525111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1410148500 con 0x7f14100ce6a0 2024-09-10T21:46:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.245+0000 7f1416200640 1 -- 172.21.15.7:0/3758525111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f141006e200 con 0x7f14100ce6a0 2024-09-10T21:46:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.246+0000 7f1407400640 1 -- 172.21.15.7:0/3758525111 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f13fc010860 con 0x7f14100ce6a0 2024-09-10T21:46:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.248+0000 7f1407400640 1 --2- 172.21.15.7:0/3758525111 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f13f0077fa0 0x7f13f007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:07.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.248+0000 7f1407400640 1 -- 172.21.15.7:0/3758525111 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f13fc09a200 con 0x7f14100ce6a0 2024-09-10T21:46:07.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.248+0000 7f140f400640 1 --2- 172.21.15.7:0/3758525111 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f13f0077fa0 0x7f13f007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:07.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.249+0000 7f140f400640 1 --2- 172.21.15.7:0/3758525111 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f13f0077fa0 0x7f13f007a460 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1400006fd0 tx=0x7f1400009060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:07.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.252+0000 7f1407400640 1 -- 172.21.15.7:0/3758525111 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f13fc062610 con 0x7f14100ce6a0 2024-09-10T21:46:07.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.484+0000 7f1416200640 1 -- 172.21.15.7:0/3758525111 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f14100709e0 con 0x7f14100ce6a0 2024-09-10T21:46:07.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.484+0000 7f1407400640 1 -- 172.21.15.7:0/3758525111 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7f13fc067040 con 0x7f14100ce6a0 2024-09-10T21:46:07.487 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:46:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.486+0000 7f1416200640 1 -- 172.21.15.7:0/3758525111 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f13f0077fa0 msgr2=0x7f13f007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.486+0000 7f1416200640 1 --2- 172.21.15.7:0/3758525111 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f13f0077fa0 0x7f13f007a460 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1400006fd0 tx=0x7f1400009060 comp rx=0 tx=0).stop 2024-09-10T21:46:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.486+0000 7f1416200640 1 -- 172.21.15.7:0/3758525111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14100ce6a0 msgr2=0x7f14100b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.486+0000 7f1416200640 1 --2- 172.21.15.7:0/3758525111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14100ce6a0 0x7f14100b4b50 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f13fc00d980 tx=0x7f13fc00de40 comp rx=0 tx=0).stop 2024-09-10T21:46:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.487+0000 7f1416200640 1 -- 172.21.15.7:0/3758525111 shutdown_connections 2024-09-10T21:46:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.487+0000 7f1416200640 1 --2- 172.21.15.7:0/3758525111 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f13f0077fa0 0x7f13f007a460 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.487+0000 7f1416200640 1 --2- 172.21.15.7:0/3758525111 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1410161640 0x7f14100b5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.487+0000 7f1416200640 1 --2- 172.21.15.7:0/3758525111 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f14100ce6a0 0x7f14100b4b50 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.487+0000 7f1416200640 1 --2- 172.21.15.7:0/3758525111 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f14100cdc00 0x7f14100c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:07.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.487+0000 7f1416200640 1 -- 172.21.15.7:0/3758525111 >> 172.21.15.7:0/3758525111 conn(0x7f14100d1d20 msgr2=0x7f14100cd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:07.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.487+0000 7f1416200640 1 -- 172.21.15.7:0/3758525111 shutdown_connections 2024-09-10T21:46:07.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07.487+0000 7f1416200640 1 -- 172.21.15.7:0/3758525111 wait complete. 2024-09-10T21:46:07.505 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:46:07.505 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:46:07.505 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-10T21:46:07.505 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:46:07.505 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:46:07.505 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:46:07.505 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:46:07.505 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:46:07.506 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:46:07.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:07.506 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-10T21:46:07.541 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:46:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:46:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:46:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:46:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T21:46:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:46:07.542 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:46:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T21:46:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-10T21:46:07.542 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:46:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T21:46:07.542 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-10T21:46:07.542 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:46:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-10T21:46:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-10T21:46:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T21:46:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42377sec preferred_lft 42377sec 2024-09-10T21:46:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-10T21:46:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:46:07.543 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:46:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-10T21:46:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-10T21:46:07.543 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:46:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff 2024-09-10T21:46:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:46:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:46:07.543 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::3c3d:dbff:fe02:4c15/64 scope link 2024-09-10T21:46:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:46:07.544 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:46:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether f6:f9:69:5a:86:f5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:46:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f4f9:69ff:fe5a:86f5/64 scope link 2024-09-10T21:46:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:46:07.544 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:46:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:46:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::433:5cff:fe44:4a5b/64 scope link 2024-09-10T21:46:07.544 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:46:07.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:07.544 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:46:07.636 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:46:07.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:07.639 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-10T21:46:07.675 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-10T21:46:07.675 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:46:07.675 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T21:46:07.675 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T21:46:07.675 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:46:07.676 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:46:07.676 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:46:07.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:46:07.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:46:07.676 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:46:07.676 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:46:07.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:46:07.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:46:07.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:07.875 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:46:07.910 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:46:07.910 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:46:07.911 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:46:07.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:07.911 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-10T21:46:08.007 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:46:08.007 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:46:08.042 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:46:08.042 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:46:08.042 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:46:08.116 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:46:08.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:08.116 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:46:08.245 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:46:08.245 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:46:08.273 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:46:08.273 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:46:08.273 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.133:3300,172.21.15.133:3301" 2024-09-10T21:46:08.273 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@2029451a-3c26-48c7-b874-0597197278ac.cephfs=/ 2024-09-10T21:46:08.273 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.133:3300/172.21.15.133:3301" will pass to kernel 2024-09-10T21:46:08.274 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T21:46:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:46:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:46:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:46:08.274 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:46:08.277 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:46:08.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:08.277 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:46:08.334 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:46:08.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:08 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:46:08.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:46:08.434 INFO:teuthology.orchestra.run.smithi007.stdout:client6312 2024-09-10T21:46:08.434 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:46:08.434 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:46:08.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:08.470 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir/) 2024-09-10T21:46:08.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:08.552 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_undamaged) 2024-09-10T21:46:08.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:08.631 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch subdir/file_to_be_damaged) 2024-09-10T21:46:08.708 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:46:08.708 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:46:08.709 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:46:08.709 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:46:08.709 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/subdir").st_ino) 2024-09-10T21:46:08.709 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:46:08.850 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627776 2024-09-10T21:46:08.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:46:08.862 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:46:08.862 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:46:08.937 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:46:08.938 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:46:09.013 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:46:09.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.213+0000 7fda7e600640 1 -- 172.21.15.7:0/827996091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7806ff50 msgr2=0x7fda7815cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:09.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.213+0000 7fda7e600640 1 --2- 172.21.15.7:0/827996091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7806ff50 0x7fda7815cbe0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fda78073840 tx=0x7fda6802a440 comp rx=0 tx=0).stop 2024-09-10T21:46:09.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.213+0000 7fda7e600640 1 -- 172.21.15.7:0/827996091 shutdown_connections 2024-09-10T21:46:09.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.213+0000 7fda7e600640 1 --2- 172.21.15.7:0/827996091 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda7806e960 0x7fda78074210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:09.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.213+0000 7fda7e600640 1 --2- 172.21.15.7:0/827996091 >> v2:172.21.15.133:3300/0 conn(0x7fda7806e510 0x7fda78073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:09.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.213+0000 7fda7e600640 1 --2- 172.21.15.7:0/827996091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7806ff50 0x7fda7815cbe0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:09.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.213+0000 7fda7e600640 1 -- 172.21.15.7:0/827996091 >> 172.21.15.7:0/827996091 conn(0x7fda780b1660 msgr2=0x7fda780b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:09.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.213+0000 7fda7e600640 1 -- 172.21.15.7:0/827996091 shutdown_connections 2024-09-10T21:46:09.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.213+0000 7fda7e600640 1 -- 172.21.15.7:0/827996091 wait complete. 2024-09-10T21:46:09.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.215+0000 7fda7e600640 1 Processor -- start 2024-09-10T21:46:09.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.215+0000 7fda7e600640 1 -- start start 2024-09-10T21:46:09.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.216+0000 7fda7e600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda7806e510 0x7fda781f5160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:09.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.216+0000 7fda7e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7806e960 0x7fda781574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:09.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.216+0000 7fda7e600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda7806ff50 0x7fda781579e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.216+0000 7fda7e600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fda7815f890 con 0x7fda7806e960 2024-09-10T21:46:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.216+0000 7fda7e600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fda7815f710 con 0x7fda7806ff50 2024-09-10T21:46:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.216+0000 7fda7e600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fda7815fa10 con 0x7fda7806e510 2024-09-10T21:46:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.216+0000 7fda6fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7806e960 0x7fda781574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.216+0000 7fda6fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7806e960 0x7fda781574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44936/0 (socket says 172.21.15.7:44936) 2024-09-10T21:46:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.216+0000 7fda6fe00640 1 -- 172.21.15.7:0/1154935053 learned_addr learned my addr 172.21.15.7:0/1154935053 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.216+0000 7fda77400640 1 --2- 172.21.15.7:0/1154935053 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda7806e510 0x7fda781f5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.216+0000 7fda77e00640 1 --2- 172.21.15.7:0/1154935053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda7806ff50 0x7fda781579e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.217+0000 7fda6fe00640 1 -- 172.21.15.7:0/1154935053 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda7806e510 msgr2=0x7fda781f5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.217+0000 7fda6fe00640 1 --2- 172.21.15.7:0/1154935053 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda7806e510 0x7fda781f5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.217+0000 7fda6fe00640 1 -- 172.21.15.7:0/1154935053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda7806ff50 msgr2=0x7fda781579e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.217+0000 7fda6fe00640 1 --2- 172.21.15.7:0/1154935053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda7806ff50 0x7fda781579e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:09.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.217+0000 7fda6fe00640 1 -- 172.21.15.7:0/1154935053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda781f48c0 con 0x7fda7806e960 2024-09-10T21:46:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.217+0000 7fda6fe00640 1 --2- 172.21.15.7:0/1154935053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7806e960 0x7fda781574a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fda6000e9f0 tx=0x7fda6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:09.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.217+0000 7fda6f400640 1 -- 172.21.15.7:0/1154935053 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda6000cd80 con 0x7fda7806e960 2024-09-10T21:46:09.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.217+0000 7fda6f400640 1 -- 172.21.15.7:0/1154935053 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda60010430 con 0x7fda7806e960 2024-09-10T21:46:09.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.217+0000 7fda6f400640 1 -- 172.21.15.7:0/1154935053 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda60010840 con 0x7fda7806e960 2024-09-10T21:46:09.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.217+0000 7fda7e600640 1 -- 172.21.15.7:0/1154935053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7815a1e0 con 0x7fda7806e960 2024-09-10T21:46:09.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.217+0000 7fda7e600640 1 -- 172.21.15.7:0/1154935053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda7815a7a0 con 0x7fda7806e960 2024-09-10T21:46:09.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.218+0000 7fda7e600640 1 -- 172.21.15.7:0/1154935053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda7815cbe0 con 0x7fda7806e960 2024-09-10T21:46:09.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.220+0000 7fda6f400640 1 -- 172.21.15.7:0/1154935053 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fda600109e0 con 0x7fda7806e960 2024-09-10T21:46:09.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.221+0000 7fda6f400640 1 --2- 172.21.15.7:0/1154935053 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fda54077fa0 0x7fda5407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:09.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.221+0000 7fda6f400640 1 -- 172.21.15.7:0/1154935053 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fda6009a980 con 0x7fda7806e960 2024-09-10T21:46:09.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.221+0000 7fda77400640 1 --2- 172.21.15.7:0/1154935053 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fda54077fa0 0x7fda5407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:09.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.222+0000 7fda77400640 1 --2- 172.21.15.7:0/1154935053 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fda54077fa0 0x7fda5407a460 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fda68033000 tx=0x7fda68004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:09.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.225+0000 7fda6f400640 1 -- 172.21.15.7:0/1154935053 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fda781f48c0 con 0x7fda7806e960 2024-09-10T21:46:09.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.432+0000 7fda7e600640 1 -- 172.21.15.7:0/1154935053 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fda780caff0 con 0x7fda7806e960 2024-09-10T21:46:09.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.432+0000 7fda6f400640 1 -- 172.21.15.7:0/1154935053 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v70) ==== 80+0+0 (secure 0 0 0) 0x7fda600634a0 con 0x7fda7806e960 2024-09-10T21:46:09.434 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-10T21:46:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.435+0000 7fda7e600640 1 -- 172.21.15.7:0/1154935053 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fda54077fa0 msgr2=0x7fda5407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.435+0000 7fda7e600640 1 --2- 172.21.15.7:0/1154935053 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fda54077fa0 0x7fda5407a460 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fda68033000 tx=0x7fda68004730 comp rx=0 tx=0).stop 2024-09-10T21:46:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.435+0000 7fda7e600640 1 -- 172.21.15.7:0/1154935053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7806e960 msgr2=0x7fda781574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.435+0000 7fda7e600640 1 --2- 172.21.15.7:0/1154935053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7806e960 0x7fda781574a0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fda6000e9f0 tx=0x7fda6000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:46:09.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.435+0000 7fda7e600640 1 -- 172.21.15.7:0/1154935053 shutdown_connections 2024-09-10T21:46:09.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.435+0000 7fda7e600640 1 --2- 172.21.15.7:0/1154935053 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fda54077fa0 0x7fda5407a460 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:09.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.435+0000 7fda7e600640 1 --2- 172.21.15.7:0/1154935053 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda7806ff50 0x7fda781579e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:09.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.435+0000 7fda7e600640 1 --2- 172.21.15.7:0/1154935053 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7806e960 0x7fda781574a0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:09.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.435+0000 7fda7e600640 1 --2- 172.21.15.7:0/1154935053 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda7806e510 0x7fda781f5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:09.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.435+0000 7fda7e600640 1 -- 172.21.15.7:0/1154935053 >> 172.21.15.7:0/1154935053 conn(0x7fda780b1660 msgr2=0x7fda780af620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:09.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.435+0000 7fda7e600640 1 -- 172.21.15.7:0/1154935053 shutdown_connections 2024-09-10T21:46:09.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.435+0000 7fda7e600640 1 -- 172.21.15.7:0/1154935053 wait complete. 2024-09-10T21:46:09.453 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:46:09.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:09.453 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:46:09.553 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:46:09.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:09.553 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:46:09.593 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:46:09.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.792+0000 7fa3af200640 1 -- 172.21.15.7:0/945613353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a80b21f0 msgr2=0x7fa3a80b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:09.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.792+0000 7fa3af200640 1 --2- 172.21.15.7:0/945613353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a80b21f0 0x7fa3a80b2680 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4005fd0 tx=0x7fa3a402a820 comp rx=0 tx=0).stop 2024-09-10T21:46:09.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.792+0000 7fa3af200640 1 -- 172.21.15.7:0/945613353 shutdown_connections 2024-09-10T21:46:09.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.792+0000 7fa3af200640 1 --2- 172.21.15.7:0/945613353 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a80b21f0 0x7fa3a80b2680 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.792+0000 7fa3af200640 1 --2- 172.21.15.7:0/945613353 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3a80d4900 0x7fa3a80b6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.792+0000 7fa3af200640 1 -- 172.21.15.7:0/945613353 >> 172.21.15.7:0/945613353 conn(0x7fa3a80d3de0 msgr2=0x7fa3a80d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.792+0000 7fa3af200640 1 -- 172.21.15.7:0/945613353 shutdown_connections 2024-09-10T21:46:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.792+0000 7fa3af200640 1 -- 172.21.15.7:0/945613353 wait complete. 2024-09-10T21:46:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.794+0000 7fa3af200640 1 Processor -- start 2024-09-10T21:46:09.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.795+0000 7fa3af200640 1 -- start start 2024-09-10T21:46:09.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.795+0000 7fa3af200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a80b21f0 0x7fa3a80c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.795+0000 7fa3af200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3a80d4900 0x7fa3a815fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.795+0000 7fa3af200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3a815dff0 0x7fa3a806e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.795+0000 7fa3af200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a80cfb60 con 0x7fa3a80b21f0 2024-09-10T21:46:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.795+0000 7fa3af200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a80cf9e0 con 0x7fa3a815dff0 2024-09-10T21:46:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.795+0000 7fa3af200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa3a80cfce0 con 0x7fa3a80d4900 2024-09-10T21:46:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.795+0000 7fa3ade00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a80b21f0 0x7fa3a80c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.795+0000 7fa3ade00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a80b21f0 0x7fa3a80c6df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44956/0 (socket says 172.21.15.7:44956) 2024-09-10T21:46:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.795+0000 7fa3ade00640 1 -- 172.21.15.7:0/1619804073 learned_addr learned my addr 172.21.15.7:0/1619804073 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.795+0000 7fa3ae800640 1 --2- 172.21.15.7:0/1619804073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3a815dff0 0x7fa3a806e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.795+0000 7fa3ad400640 1 --2- 172.21.15.7:0/1619804073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3a80d4900 0x7fa3a815fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.795+0000 7fa3ade00640 1 -- 172.21.15.7:0/1619804073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3a80d4900 msgr2=0x7fa3a815fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.795+0000 7fa3ade00640 1 --2- 172.21.15.7:0/1619804073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3a80d4900 0x7fa3a815fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:09.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.795+0000 7fa3ade00640 1 -- 172.21.15.7:0/1619804073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3a815dff0 msgr2=0x7fa3a806e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:09.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.795+0000 7fa3ade00640 1 --2- 172.21.15.7:0/1619804073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3a815dff0 0x7fa3a806e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:09.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.795+0000 7fa3ade00640 1 -- 172.21.15.7:0/1619804073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a80b4d20 con 0x7fa3a80b21f0 2024-09-10T21:46:09.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.795+0000 7fa3ade00640 1 --2- 172.21.15.7:0/1619804073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a80b21f0 0x7fa3a80c6df0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00c440 tx=0x7fa39c00c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:09.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.796+0000 7fa3a2000640 1 -- 172.21.15.7:0/1619804073 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c017070 con 0x7fa3a80b21f0 2024-09-10T21:46:09.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.796+0000 7fa3a2000640 1 -- 172.21.15.7:0/1619804073 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa39c0035f0 con 0x7fa3a80b21f0 2024-09-10T21:46:09.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.796+0000 7fa3a2000640 1 -- 172.21.15.7:0/1619804073 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c003a40 con 0x7fa3a80b21f0 2024-09-10T21:46:09.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.796+0000 7fa3af200640 1 -- 172.21.15.7:0/1619804073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a80b4fc0 con 0x7fa3a80b21f0 2024-09-10T21:46:09.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.796+0000 7fa3af200640 1 -- 172.21.15.7:0/1619804073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a80b5580 con 0x7fa3a80b21f0 2024-09-10T21:46:09.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.797+0000 7fa3af200640 1 -- 172.21.15.7:0/1619804073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3a80b6c30 con 0x7fa3a80b21f0 2024-09-10T21:46:09.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.798+0000 7fa3a2000640 1 -- 172.21.15.7:0/1619804073 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa39c003be0 con 0x7fa3a80b21f0 2024-09-10T21:46:09.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.799+0000 7fa3a2000640 1 --2- 172.21.15.7:0/1619804073 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa38c077f70 0x7fa38c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:09.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.799+0000 7fa3a2000640 1 -- 172.21.15.7:0/1619804073 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7fa39c099430 con 0x7fa3a80b21f0 2024-09-10T21:46:09.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.800+0000 7fa3ad400640 1 --2- 172.21.15.7:0/1619804073 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa38c077f70 0x7fa38c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:09.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.800+0000 7fa3ad400640 1 --2- 172.21.15.7:0/1619804073 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa38c077f70 0x7fa38c07a430 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa398005fd0 tx=0x7fa398005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:09.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:09.804+0000 7fa3a2000640 1 -- 172.21.15.7:0/1619804073 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fa39c061e90 con 0x7fa3a80b21f0 2024-09-10T21:46:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.035+0000 7fa3af200640 1 -- 172.21.15.7:0/1619804073 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3a80b6700 con 0x7fa3a80b21f0 2024-09-10T21:46:10.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.035+0000 7fa3a2000640 1 -- 172.21.15.7:0/1619804073 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5235 (secure 0 0 0) 0x7fa39c061790 con 0x7fa3a80b21f0 2024-09-10T21:46:10.038 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:46:10.038 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":46,"btime":"2024-09-10T21:46:03:853860+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5884,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3488410478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3488410478},{"type":"v1","addr":"172.21.15.133:6837","nonce":3488410478}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5890,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3660400727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3660400727},{"type":"v1","addr":"172.21.15.7:6835","nonce":3660400727}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2388873704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2388873704},{"type":"v1","addr":"172.21.15.7:6834","nonce":2388873704}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:46:00.260975+0000","modified":"2024-09-10T21:46:03.290119+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235361},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235361":{"gid":235361,"name":"b","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6836/2752881644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2752881644},{"type":"v1","addr":"172.21.15.133:6836","nonce":2752881644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235361,"qdb_cluster":[235361]},"id":5}]} 2024-09-10T21:46:10.038 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 46 2024-09-10T21:46:10.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.038+0000 7fa3af200640 1 -- 172.21.15.7:0/1619804073 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa38c077f70 msgr2=0x7fa38c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.038+0000 7fa3af200640 1 --2- 172.21.15.7:0/1619804073 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa38c077f70 0x7fa38c07a430 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa398005fd0 tx=0x7fa398005850 comp rx=0 tx=0).stop 2024-09-10T21:46:10.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.038+0000 7fa3af200640 1 -- 172.21.15.7:0/1619804073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a80b21f0 msgr2=0x7fa3a80c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.038+0000 7fa3af200640 1 --2- 172.21.15.7:0/1619804073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a80b21f0 0x7fa3a80c6df0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00c440 tx=0x7fa39c00c900 comp rx=0 tx=0).stop 2024-09-10T21:46:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.038+0000 7fa3af200640 1 -- 172.21.15.7:0/1619804073 shutdown_connections 2024-09-10T21:46:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.038+0000 7fa3af200640 1 --2- 172.21.15.7:0/1619804073 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa38c077f70 0x7fa38c07a430 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.038+0000 7fa3af200640 1 --2- 172.21.15.7:0/1619804073 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3a815dff0 0x7fa3a806e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.038+0000 7fa3af200640 1 --2- 172.21.15.7:0/1619804073 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3a80d4900 0x7fa3a815fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.038+0000 7fa3af200640 1 --2- 172.21.15.7:0/1619804073 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3a80b21f0 0x7fa3a80c6df0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.038+0000 7fa3af200640 1 -- 172.21.15.7:0/1619804073 >> 172.21.15.7:0/1619804073 conn(0x7fa3a80d3de0 msgr2=0x7fa3a80b7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.038+0000 7fa3af200640 1 -- 172.21.15.7:0/1619804073 shutdown_connections 2024-09-10T21:46:10.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.038+0000 7fa3af200640 1 -- 172.21.15.7:0/1619804073 wait complete. 2024-09-10T21:46:10.056 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json flush journal 2024-09-10T21:46:10.247 INFO:teuthology.orchestra.run.smithi133.stdout:{"message":"","return_code":0} 2024-09-10T21:46:10.248 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-10T21:46:10.248 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:46:10.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.410+0000 7f444b200640 1 -- 172.21.15.7:0/3537069113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4430004800 msgr2=0x7f4430004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.410+0000 7f444b200640 1 --2- 172.21.15.7:0/3537069113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4430004800 0x7f4430004c80 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f4438001830 tx=0x7f4430005830 comp rx=0 tx=0).stop 2024-09-10T21:46:10.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.410+0000 7f444a800640 1 -- 172.21.15.7:0/3537069113 reap_dead start 2024-09-10T21:46:10.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.410+0000 7f444b200640 1 -- 172.21.15.7:0/3537069113 shutdown_connections 2024-09-10T21:46:10.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.410+0000 7f444b200640 1 -- 172.21.15.7:0/3537069113 >> 172.21.15.7:0/3537069113 conn(0x7f444c074080 msgr2=0x7f444c074460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:10.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.410+0000 7f444b200640 1 -- 172.21.15.7:0/3537069113 shutdown_connections 2024-09-10T21:46:10.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.410+0000 7f444b200640 1 -- 172.21.15.7:0/3537069113 wait complete. 2024-09-10T21:46:10.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.412+0000 7f444b200640 1 Processor -- start 2024-09-10T21:46:10.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.412+0000 7f444b200640 1 -- start start 2024-09-10T21:46:10.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.412+0000 7f444b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4430004800 0x7f444c0c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.412+0000 7f444b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444c1f45a0 0x7f444c0b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.412+0000 7f444b200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f444c0adc50 0x7f444c0ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.412+0000 7f444b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f444c0cc7b0 con 0x7f444c1f45a0 2024-09-10T21:46:10.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.412+0000 7f444b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f444c0cd160 con 0x7f4430004800 2024-09-10T21:46:10.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.412+0000 7f444b200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f444c0b0c70 con 0x7f444c0adc50 2024-09-10T21:46:10.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.414+0000 7f4449400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444c1f45a0 0x7f444c0b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.414+0000 7f4449400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444c1f45a0 0x7f444c0b41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44982/0 (socket says 172.21.15.7:44982) 2024-09-10T21:46:10.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.414+0000 7f4449400640 1 -- 172.21.15.7:0/1427604694 learned_addr learned my addr 172.21.15.7:0/1427604694 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:10.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.414+0000 7f4449e00640 1 --2- 172.21.15.7:0/1427604694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4430004800 0x7f444c0c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.414+0000 7f444a800640 1 --2- 172.21.15.7:0/1427604694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f444c0adc50 0x7f444c0ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.414+0000 7f4449400640 1 -- 172.21.15.7:0/1427604694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f444c0adc50 msgr2=0x7f444c0ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.414+0000 7f4449400640 1 --2- 172.21.15.7:0/1427604694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f444c0adc50 0x7f444c0ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.414+0000 7f4449400640 1 -- 172.21.15.7:0/1427604694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4430004800 msgr2=0x7f444c0c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.414+0000 7f4449400640 1 --2- 172.21.15.7:0/1427604694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4430004800 0x7f444c0c62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.414+0000 7f4449400640 1 -- 172.21.15.7:0/1427604694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f444c0c6820 con 0x7f444c1f45a0 2024-09-10T21:46:10.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.414+0000 7f444a800640 1 --2- 172.21.15.7:0/1427604694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f444c0adc50 0x7f444c0ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:10.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.414+0000 7f4449400640 1 --2- 172.21.15.7:0/1427604694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444c1f45a0 0x7f444c0b41a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4438001950 tx=0x7f44380060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:10.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.415+0000 7f4442000640 1 -- 172.21.15.7:0/1427604694 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4438043070 con 0x7f444c1f45a0 2024-09-10T21:46:10.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.415+0000 7f4442000640 1 -- 172.21.15.7:0/1427604694 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4438003af0 con 0x7f444c1f45a0 2024-09-10T21:46:10.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.415+0000 7f4442000640 1 -- 172.21.15.7:0/1427604694 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44380064c0 con 0x7f444c1f45a0 2024-09-10T21:46:10.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.415+0000 7f444b200640 1 -- 172.21.15.7:0/1427604694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f444c0b4b10 con 0x7f444c1f45a0 2024-09-10T21:46:10.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.415+0000 7f444b200640 1 -- 172.21.15.7:0/1427604694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f444c0b50d0 con 0x7f444c1f45a0 2024-09-10T21:46:10.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.416+0000 7f444b200640 1 -- 172.21.15.7:0/1427604694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f444c15f720 con 0x7f444c1f45a0 2024-09-10T21:46:10.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.417+0000 7f4442000640 1 -- 172.21.15.7:0/1427604694 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4438033480 con 0x7f444c1f45a0 2024-09-10T21:46:10.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.417+0000 7f4442000640 1 --2- 172.21.15.7:0/1427604694 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f442c077fa0 0x7f442c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.417+0000 7f4442000640 1 -- 172.21.15.7:0/1427604694 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5709+0+0 (secure 0 0 0) 0x7f44380b9aa0 con 0x7f444c1f45a0 2024-09-10T21:46:10.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.418+0000 7f4449e00640 1 --2- 172.21.15.7:0/1427604694 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f442c077fa0 0x7f442c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.420+0000 7f4449e00640 1 --2- 172.21.15.7:0/1427604694 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f442c077fa0 0x7f442c07a460 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f444c076280 tx=0x7f443c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:10.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.423+0000 7f4442000640 1 -- 172.21.15.7:0/1427604694 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4438081f40 con 0x7f444c1f45a0 2024-09-10T21:46:10.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.657+0000 7f444b200640 1 -- 172.21.15.7:0/1427604694 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f444c0b6240 con 0x7f444c1f45a0 2024-09-10T21:46:10.659 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-10T21:46:10.658+0000 7feb3fa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:46:10.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.660+0000 7f4442000640 1 -- 172.21.15.7:0/1427604694 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) ==== 161+0+0 (secure 0 0 0) 0x7f4438086970 con 0x7f444c1f45a0 2024-09-10T21:46:10.661 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:46:10.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.663+0000 7f444b200640 1 -- 172.21.15.7:0/1427604694 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f442c077fa0 msgr2=0x7f442c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.663+0000 7f444b200640 1 --2- 172.21.15.7:0/1427604694 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f442c077fa0 0x7f442c07a460 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f444c076280 tx=0x7f443c005f90 comp rx=0 tx=0).stop 2024-09-10T21:46:10.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.663+0000 7f444b200640 1 -- 172.21.15.7:0/1427604694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444c1f45a0 msgr2=0x7f444c0b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.663+0000 7f444b200640 1 --2- 172.21.15.7:0/1427604694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444c1f45a0 0x7f444c0b41a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f4438001950 tx=0x7f44380060e0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.663+0000 7f444b200640 1 -- 172.21.15.7:0/1427604694 shutdown_connections 2024-09-10T21:46:10.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.663+0000 7f444b200640 1 --2- 172.21.15.7:0/1427604694 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f442c077fa0 0x7f442c07a460 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.663+0000 7f444b200640 1 --2- 172.21.15.7:0/1427604694 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f444c0adc50 0x7f444c0ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.663+0000 7f444b200640 1 --2- 172.21.15.7:0/1427604694 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f444c1f45a0 0x7f444c0b41a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.663+0000 7f444b200640 1 --2- 172.21.15.7:0/1427604694 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4430004800 0x7f444c0c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.663+0000 7f444b200640 1 -- 172.21.15.7:0/1427604694 >> 172.21.15.7:0/1427604694 conn(0x7f444c074080 msgr2=0x7f444c0d0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:10.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.663+0000 7f444b200640 1 -- 172.21.15.7:0/1427604694 shutdown_connections 2024-09-10T21:46:10.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.663+0000 7f444b200640 1 -- 172.21.15.7:0/1427604694 wait complete. 2024-09-10T21:46:10.674 INFO:tasks.ceph.mds.b.smithi133.stderr: -5137> 2024-09-10T21:46:05.801+0000 7ffbc4a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:46:10.680 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T21:46:10.722 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2024-09-10T21:46:10.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.758+0000 7f98432eaf40 1 Processor -- start 2024-09-10T21:46:10.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.758+0000 7f98432eaf40 1 -- start start 2024-09-10T21:46:10.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.759+0000 7f98432eaf40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563faf7d40e0 0x563faf8afe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.759+0000 7f98432eaf40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563faf809210 0x563faf8a3050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.759+0000 7f98432eaf40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563faf8aab90 0x563faf8a3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.759+0000 7f98432eaf40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563faf821d90 con 0x563faf7d40e0 2024-09-10T21:46:10.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.759+0000 7f98432eaf40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563faf725c60 con 0x563faf809210 2024-09-10T21:46:10.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.759+0000 7f98432eaf40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x563faf821f10 con 0x563faf8aab90 2024-09-10T21:46:10.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.759+0000 7f9840400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563faf7d40e0 0x563faf8afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.759+0000 7f9840400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563faf7d40e0 0x563faf8afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44998/0 (socket says 172.21.15.7:44998) 2024-09-10T21:46:10.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.759+0000 7f9840400640 1 -- 172.21.15.7:0/604951884 learned_addr learned my addr 172.21.15.7:0/604951884 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:10.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.759+0000 7f9837e00640 1 --2- 172.21.15.7:0/604951884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563faf809210 0x563faf8a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.759+0000 7f9840e00640 1 --2- 172.21.15.7:0/604951884 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563faf8aab90 0x563faf8a3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.759+0000 7f9840400640 1 -- 172.21.15.7:0/604951884 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563faf8aab90 msgr2=0x563faf8a3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:10.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.759+0000 7f9840400640 1 --2- 172.21.15.7:0/604951884 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563faf8aab90 0x563faf8a3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.759+0000 7f9840400640 1 -- 172.21.15.7:0/604951884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563faf809210 msgr2=0x563faf8a3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:10.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.759+0000 7f9840400640 1 --2- 172.21.15.7:0/604951884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563faf809210 0x563faf8a3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.759+0000 7f9840400640 1 -- 172.21.15.7:0/604951884 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563faf6fdf10 con 0x563faf7d40e0 2024-09-10T21:46:10.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.759+0000 7f9837e00640 1 --2- 172.21.15.7:0/604951884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563faf809210 0x563faf8a3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:10.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.760+0000 7f9840400640 1 --2- 172.21.15.7:0/604951884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563faf7d40e0 0x563faf8afe50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f983000b600 tx=0x7f983000bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a6ec808346443da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:10.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.760+0000 7f983f000640 1 -- 172.21.15.7:0/604951884 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983000a910 con 0x563faf7d40e0 2024-09-10T21:46:10.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.760+0000 7f983f000640 1 -- 172.21.15.7:0/604951884 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f983000aab0 con 0x563faf7d40e0 2024-09-10T21:46:10.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.760+0000 7f983f000640 1 -- 172.21.15.7:0/604951884 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830008450 con 0x563faf7d40e0 2024-09-10T21:46:10.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.760+0000 7f98432eaf40 1 -- 172.21.15.7:0/604951884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563faf7d40e0 msgr2=0x563faf8afe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.760+0000 7f98432eaf40 1 --2- 172.21.15.7:0/604951884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563faf7d40e0 0x563faf8afe50 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f983000b600 tx=0x7f983000bac0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.760+0000 7f98432eaf40 1 -- 172.21.15.7:0/604951884 shutdown_connections 2024-09-10T21:46:10.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.760+0000 7f98432eaf40 1 --2- 172.21.15.7:0/604951884 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563faf809210 0x563faf8a3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.760+0000 7f98432eaf40 1 --2- 172.21.15.7:0/604951884 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563faf8aab90 0x563faf8a3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.760+0000 7f98432eaf40 1 --2- 172.21.15.7:0/604951884 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563faf7d40e0 0x563faf8afe50 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.760+0000 7f98432eaf40 1 -- 172.21.15.7:0/604951884 >> 172.21.15.7:0/604951884 conn(0x563faf823df0 msgr2=0x563faf807510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:10.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.760+0000 7f98432eaf40 1 -- 172.21.15.7:0/604951884 shutdown_connections 2024-09-10T21:46:10.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.760+0000 7f98432eaf40 1 -- 172.21.15.7:0/604951884 wait complete. 2024-09-10T21:46:10.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.761+0000 7f98432eaf40 1 Processor -- start 2024-09-10T21:46:10.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.761+0000 7f98432eaf40 1 -- start start 2024-09-10T21:46:10.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.762+0000 7f98432eaf40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563faf7d40e0 0x563faf8084f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.762+0000 7f98432eaf40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563faf809210 0x563faf803890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.762+0000 7f98432eaf40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563faf8aab90 0x563faf8a48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.762+0000 7f98432eaf40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563faf8ad2e0 con 0x563faf8aab90 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.762+0000 7f98432eaf40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563faf8a7980 con 0x563faf7d40e0 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.762+0000 7f98432eaf40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x563faf8aa090 con 0x563faf809210 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.762+0000 7f9840e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563faf8aab90 0x563faf8a48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.762+0000 7f9840e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563faf8aab90 0x563faf8a48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45000/0 (socket says 172.21.15.7:45000) 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.762+0000 7f9840e00640 1 -- 172.21.15.7:0/3674562973 learned_addr learned my addr 172.21.15.7:0/3674562973 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.762+0000 7f9837e00640 1 --2- 172.21.15.7:0/3674562973 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563faf809210 0x563faf803890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.762+0000 7f9840400640 1 --2- 172.21.15.7:0/3674562973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563faf7d40e0 0x563faf8084f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.763+0000 7f9840e00640 1 -- 172.21.15.7:0/3674562973 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563faf809210 msgr2=0x563faf803890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.763+0000 7f9840e00640 1 --2- 172.21.15.7:0/3674562973 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563faf809210 0x563faf803890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.763+0000 7f9840e00640 1 -- 172.21.15.7:0/3674562973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563faf7d40e0 msgr2=0x563faf8084f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.763+0000 7f9840e00640 1 --2- 172.21.15.7:0/3674562973 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563faf7d40e0 0x563faf8084f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.763+0000 7f9840e00640 1 -- 172.21.15.7:0/3674562973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563faf8a6690 con 0x563faf8aab90 2024-09-10T21:46:10.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.763+0000 7f9840e00640 1 --2- 172.21.15.7:0/3674562973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563faf8aab90 0x563faf8a48b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9838025e30 tx=0x7f9838004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:10.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.763+0000 7f983dc00640 1 -- 172.21.15.7:0/3674562973 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838064030 con 0x563faf8aab90 2024-09-10T21:46:10.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.763+0000 7f983dc00640 1 -- 172.21.15.7:0/3674562973 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f983804d880 con 0x563faf8aab90 2024-09-10T21:46:10.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.763+0000 7f983dc00640 1 -- 172.21.15.7:0/3674562973 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838004020 con 0x563faf8aab90 2024-09-10T21:46:10.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.763+0000 7f98432eaf40 1 -- 172.21.15.7:0/3674562973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563faf8a43d0 con 0x563faf8aab90 2024-09-10T21:46:10.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.764+0000 7f98432eaf40 1 -- 172.21.15.7:0/3674562973 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563faf8a6bf0 con 0x563faf8aab90 2024-09-10T21:46:10.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.765+0000 7f983dc00640 1 -- 172.21.15.7:0/3674562973 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9838060440 con 0x563faf8aab90 2024-09-10T21:46:10.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.766+0000 7f983dc00640 1 --2- 172.21.15.7:0/3674562973 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f981c076980 0x7f981c078e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.766+0000 7f983dc00640 1 -- 172.21.15.7:0/3674562973 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7f98380dcdd0 con 0x563faf8aab90 2024-09-10T21:46:10.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.767+0000 7f98432eaf40 1 --2- 172.21.15.7:0/3674562973 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x563faf808a30 0x563faf89eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.767+0000 7f9840400640 1 --2- 172.21.15.7:0/3674562973 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f981c076980 0x7f981c078e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.767+0000 7f98432eaf40 1 -- 172.21.15.7:0/3674562973 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e71) -- 0x563faf8a3320 con 0x563faf808a30 2024-09-10T21:46:10.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.767+0000 7f9837e00640 1 --2- 172.21.15.7:0/3674562973 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x563faf808a30 0x563faf89eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.768+0000 7f9840400640 1 --2- 172.21.15.7:0/3674562973 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f981c076980 0x7f981c078e40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9830002040 tx=0x7f9830019040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:10.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.768+0000 7f9837e00640 1 --2- 172.21.15.7:0/3674562973 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x563faf808a30 0x563faf89eaa0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:10.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.769+0000 7f9837e00640 1 -- 172.21.15.7:0/3674562973 <== osd.4 v2:172.21.15.133:6826/1857710445 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v71'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f982800c940 con 0x563faf808a30 2024-09-10T21:46:10.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.769+0000 7f98432eaf40 1 -- 172.21.15.7:0/3674562973 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x563faf808a30 msgr2=0x563faf89eaa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.769+0000 7f98432eaf40 1 --2- 172.21.15.7:0/3674562973 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x563faf808a30 0x563faf89eaa0 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.770+0000 7f98432eaf40 1 -- 172.21.15.7:0/3674562973 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f981c076980 msgr2=0x7f981c078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.770+0000 7f98432eaf40 1 --2- 172.21.15.7:0/3674562973 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f981c076980 0x7f981c078e40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9830002040 tx=0x7f9830019040 comp rx=0 tx=0).stop 2024-09-10T21:46:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.770+0000 7f98432eaf40 1 -- 172.21.15.7:0/3674562973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563faf8aab90 msgr2=0x563faf8a48b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.770+0000 7f98432eaf40 1 --2- 172.21.15.7:0/3674562973 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563faf8aab90 0x563faf8a48b0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f9838025e30 tx=0x7f9838004ab0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.770+0000 7f9840e00640 1 -- 172.21.15.7:0/3674562973 reap_dead start 2024-09-10T21:46:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.770+0000 7f98432eaf40 1 -- 172.21.15.7:0/3674562973 shutdown_connections 2024-09-10T21:46:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.770+0000 7f98432eaf40 1 -- 172.21.15.7:0/3674562973 >> 172.21.15.7:0/3674562973 conn(0x563faf823df0 msgr2=0x563faf806a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.770+0000 7f98432eaf40 1 -- 172.21.15.7:0/3674562973 shutdown_connections 2024-09-10T21:46:10.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.770+0000 7f98432eaf40 1 -- 172.21.15.7:0/3674562973 wait complete. 2024-09-10T21:46:10.775 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:46:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.938+0000 7f1c09000640 1 -- 172.21.15.7:0/3292117155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c04073e20 msgr2=0x7f1c04074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.938+0000 7f1c09000640 1 --2- 172.21.15.7:0/3292117155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c04073e20 0x7f1c04074280 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc005fd0 tx=0x7f1bfc02a970 comp rx=0 tx=0).stop 2024-09-10T21:46:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.938+0000 7f1c09000640 1 -- 172.21.15.7:0/3292117155 shutdown_connections 2024-09-10T21:46:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.938+0000 7f1c09000640 1 --2- 172.21.15.7:0/3292117155 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c04073e20 0x7f1c04074280 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.938+0000 7f1c09000640 1 --2- 172.21.15.7:0/3292117155 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c04159680 0x7f1c0415cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.938+0000 7f1c09000640 1 -- 172.21.15.7:0/3292117155 >> 172.21.15.7:0/3292117155 conn(0x7f1c040b16d0 msgr2=0x7f1c040b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.939+0000 7f1c09000640 1 -- 172.21.15.7:0/3292117155 shutdown_connections 2024-09-10T21:46:10.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.939+0000 7f1c09000640 1 -- 172.21.15.7:0/3292117155 wait complete. 2024-09-10T21:46:10.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.940+0000 7f1c09000640 1 Processor -- start 2024-09-10T21:46:10.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.940+0000 7f1c09000640 1 -- start start 2024-09-10T21:46:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.941+0000 7f1c09000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c0406e580 0x7f1c040b1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.941+0000 7f1c09000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c04073e20 0x7f1c040c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.941+0000 7f1c09000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c04159680 0x7f1c041f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.941+0000 7f1c09000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c040ca990 con 0x7f1c04073e20 2024-09-10T21:46:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.941+0000 7f1c09000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1c040ca810 con 0x7f1c04159680 2024-09-10T21:46:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.941+0000 7f1c09000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f1c040cab10 con 0x7f1c0406e580 2024-09-10T21:46:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.942+0000 7f1bfbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c04073e20 0x7f1c040c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.942+0000 7f1c01e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c0406e580 0x7f1c040b1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.942+0000 7f1bfbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c04073e20 0x7f1c040c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45014/0 (socket says 172.21.15.7:45014) 2024-09-10T21:46:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.942+0000 7f1bfbe00640 1 -- 172.21.15.7:0/99042990 learned_addr learned my addr 172.21.15.7:0/99042990 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.942+0000 7f1c02800640 1 --2- 172.21.15.7:0/99042990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c04159680 0x7f1c041f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.942+0000 7f1bfbe00640 1 -- 172.21.15.7:0/99042990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c0406e580 msgr2=0x7f1c040b1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.942+0000 7f1bfbe00640 1 --2- 172.21.15.7:0/99042990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c0406e580 0x7f1c040b1180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.942+0000 7f1bfbe00640 1 -- 172.21.15.7:0/99042990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c04159680 msgr2=0x7f1c041f7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.942+0000 7f1bfbe00640 1 --2- 172.21.15.7:0/99042990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c04159680 0x7f1c041f7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.942+0000 7f1bfbe00640 1 -- 172.21.15.7:0/99042990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c040c63b0 con 0x7f1c04073e20 2024-09-10T21:46:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.942+0000 7f1bfbe00640 1 --2- 172.21.15.7:0/99042990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c04073e20 0x7f1c040c5a80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00c430 tx=0x7f1bec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.942+0000 7f1bfa000640 1 -- 172.21.15.7:0/99042990 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec011790 con 0x7f1c04073e20 2024-09-10T21:46:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.942+0000 7f1bfa000640 1 -- 172.21.15.7:0/99042990 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1bec017070 con 0x7f1c04073e20 2024-09-10T21:46:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.943+0000 7f1bfa000640 1 -- 172.21.15.7:0/99042990 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bec011d00 con 0x7f1c04073e20 2024-09-10T21:46:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.943+0000 7f1c09000640 1 -- 172.21.15.7:0/99042990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c040c6690 con 0x7f1c04073e20 2024-09-10T21:46:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.943+0000 7f1c09000640 1 -- 172.21.15.7:0/99042990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c04159ec0 con 0x7f1c04073e20 2024-09-10T21:46:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.944+0000 7f1c09000640 1 -- 172.21.15.7:0/99042990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c040cafc0 con 0x7f1c04073e20 2024-09-10T21:46:10.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.945+0000 7f1bfa000640 1 -- 172.21.15.7:0/99042990 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1bec01c460 con 0x7f1c04073e20 2024-09-10T21:46:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.946+0000 7f1bfa000640 1 --2- 172.21.15.7:0/99042990 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1be0077f70 0x7f1be007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.946+0000 7f1bfa000640 1 -- 172.21.15.7:0/99042990 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7f1bec099b00 con 0x7f1c04073e20 2024-09-10T21:46:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.946+0000 7f1c01e00640 1 --2- 172.21.15.7:0/99042990 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1be0077f70 0x7f1be007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.947+0000 7f1c01e00640 1 --2- 172.21.15.7:0/99042990 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1be0077f70 0x7f1be007a430 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1c0415b810 tx=0x7f1bf400a250 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:10.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:10.950+0000 7f1bfa000640 1 -- 172.21.15.7:0/99042990 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1bec061f40 con 0x7f1c04073e20 2024-09-10T21:46:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.183+0000 7f1c09000640 1 -- 172.21.15.7:0/99042990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f1c0415cb50 con 0x7f1c04073e20 2024-09-10T21:46:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.662+0000 7f1bfa000640 1 -- 172.21.15.7:0/99042990 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v48) ==== 162+0+0 (secure 0 0 0) 0x7f1bec099e20 con 0x7f1c04073e20 2024-09-10T21:46:11.663 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:46:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.665+0000 7f1c09000640 1 -- 172.21.15.7:0/99042990 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1be0077f70 msgr2=0x7f1be007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.665+0000 7f1c09000640 1 --2- 172.21.15.7:0/99042990 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1be0077f70 0x7f1be007a430 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f1c0415b810 tx=0x7f1bf400a250 comp rx=0 tx=0).stop 2024-09-10T21:46:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.665+0000 7f1c09000640 1 -- 172.21.15.7:0/99042990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c04073e20 msgr2=0x7f1c040c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.665+0000 7f1c09000640 1 --2- 172.21.15.7:0/99042990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c04073e20 0x7f1c040c5a80 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f1bec00c430 tx=0x7f1bec00c8f0 comp rx=0 tx=0).stop 2024-09-10T21:46:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.665+0000 7f1c09000640 1 -- 172.21.15.7:0/99042990 shutdown_connections 2024-09-10T21:46:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.665+0000 7f1c09000640 1 --2- 172.21.15.7:0/99042990 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1be0077f70 0x7f1be007a430 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.665+0000 7f1c09000640 1 --2- 172.21.15.7:0/99042990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c04159680 0x7f1c041f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:11.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.665+0000 7f1c09000640 1 --2- 172.21.15.7:0/99042990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c04073e20 0x7f1c040c5a80 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.665+0000 7f1c09000640 1 --2- 172.21.15.7:0/99042990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c0406e580 0x7f1c040b1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.665+0000 7f1c09000640 1 -- 172.21.15.7:0/99042990 >> 172.21.15.7:0/99042990 conn(0x7f1c040b16d0 msgr2=0x7f1c040b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.665+0000 7f1c09000640 1 -- 172.21.15.7:0/99042990 shutdown_connections 2024-09-10T21:46:11.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.665+0000 7f1c09000640 1 -- 172.21.15.7:0/99042990 wait complete. 2024-09-10T21:46:11.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:46:11.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.847+0000 7f7f70c00640 1 -- 172.21.15.7:0/3533868027 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f6c15bc90 msgr2=0x7f7f6c0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:11.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.847+0000 7f7f70c00640 1 --2- 172.21.15.7:0/3533868027 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f6c15bc90 0x7f7f6c0ccef0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009830 tx=0x7f7f5c02a3f0 comp rx=0 tx=0).stop 2024-09-10T21:46:11.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.847+0000 7f7f70c00640 1 -- 172.21.15.7:0/3533868027 shutdown_connections 2024-09-10T21:46:11.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.847+0000 7f7f70c00640 1 --2- 172.21.15.7:0/3533868027 >> v2:172.21.15.133:3300/0 conn(0x7f7f6c15fda0 0x7f7f6c1601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:11.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.847+0000 7f7f70c00640 1 --2- 172.21.15.7:0/3533868027 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f6c15bc90 0x7f7f6c0ccef0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:11.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.847+0000 7f7f70c00640 1 -- 172.21.15.7:0/3533868027 >> 172.21.15.7:0/3533868027 conn(0x7f7f6c0b16d0 msgr2=0x7f7f6c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:11.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.847+0000 7f7f70c00640 1 -- 172.21.15.7:0/3533868027 shutdown_connections 2024-09-10T21:46:11.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.847+0000 7f7f70c00640 1 -- 172.21.15.7:0/3533868027 wait complete. 2024-09-10T21:46:11.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.849+0000 7f7f70c00640 1 Processor -- start 2024-09-10T21:46:11.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.849+0000 7f7f70c00640 1 -- start start 2024-09-10T21:46:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.850+0000 7f7f70c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c0cd750 0x7f7f6c0aea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.850+0000 7f7f70c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c15bc90 0x7f7f6c15a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.850+0000 7f7f70c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f6c15fda0 0x7f7f6c15e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.850+0000 7f7f70c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c06fe70 con 0x7f7f6c0cd750 2024-09-10T21:46:11.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.850+0000 7f7f70c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7f6c06fcf0 con 0x7f7f6c15bc90 2024-09-10T21:46:11.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.850+0000 7f7f70c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f7f6c06fff0 con 0x7f7f6c15fda0 2024-09-10T21:46:11.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.850+0000 7f7f69e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c0cd750 0x7f7f6c0aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:11.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.850+0000 7f7f69e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c0cd750 0x7f7f6c0aea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45028/0 (socket says 172.21.15.7:45028) 2024-09-10T21:46:11.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.850+0000 7f7f69e00640 1 -- 172.21.15.7:0/1770391623 learned_addr learned my addr 172.21.15.7:0/1770391623 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:11.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.851+0000 7f7f69400640 1 --2- 172.21.15.7:0/1770391623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c15bc90 0x7f7f6c15a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:11.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.851+0000 7f7f6a800640 1 --2- 172.21.15.7:0/1770391623 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f6c15fda0 0x7f7f6c15e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:11.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.851+0000 7f7f69e00640 1 -- 172.21.15.7:0/1770391623 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f6c15fda0 msgr2=0x7f7f6c15e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:11.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.851+0000 7f7f69e00640 1 --2- 172.21.15.7:0/1770391623 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f6c15fda0 0x7f7f6c15e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:11.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.851+0000 7f7f69e00640 1 -- 172.21.15.7:0/1770391623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c15bc90 msgr2=0x7f7f6c15a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:11.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.851+0000 7f7f69e00640 1 --2- 172.21.15.7:0/1770391623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c15bc90 0x7f7f6c15a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.851+0000 7f7f69e00640 1 -- 172.21.15.7:0/1770391623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f6c0aef70 con 0x7f7f6c0cd750 2024-09-10T21:46:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.851+0000 7f7f69400640 1 --2- 172.21.15.7:0/1770391623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c15bc90 0x7f7f6c15a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.851+0000 7f7f69e00640 1 --2- 172.21.15.7:0/1770391623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c0cd750 0x7f7f6c0aea30 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c000c00 tx=0x7f7f5c032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.852+0000 7f7f62000640 1 -- 172.21.15.7:0/1770391623 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c040070 con 0x7f7f6c0cd750 2024-09-10T21:46:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.852+0000 7f7f70c00640 1 -- 172.21.15.7:0/1770391623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f6c15dad0 con 0x7f7f6c0cd750 2024-09-10T21:46:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.852+0000 7f7f62000640 1 -- 172.21.15.7:0/1770391623 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f5c0038e0 con 0x7f7f6c0cd750 2024-09-10T21:46:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.852+0000 7f7f62000640 1 -- 172.21.15.7:0/1770391623 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c003d30 con 0x7f7f6c0cd750 2024-09-10T21:46:11.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.852+0000 7f7f70c00640 1 -- 172.21.15.7:0/1770391623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f6c0c3da0 con 0x7f7f6c0cd750 2024-09-10T21:46:11.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.853+0000 7f7f70c00640 1 -- 172.21.15.7:0/1770391623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f6c0b1b50 con 0x7f7f6c0cd750 2024-09-10T21:46:11.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.854+0000 7f7f62000640 1 -- 172.21.15.7:0/1770391623 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7f5c045670 con 0x7f7f6c0cd750 2024-09-10T21:46:11.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.855+0000 7f7f62000640 1 --2- 172.21.15.7:0/1770391623 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f4c077fa0 0x7f7f4c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:11.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.855+0000 7f7f62000640 1 -- 172.21.15.7:0/1770391623 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7f7f5c0bb8c0 con 0x7f7f6c0cd750 2024-09-10T21:46:11.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.855+0000 7f7f69400640 1 --2- 172.21.15.7:0/1770391623 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f4c077fa0 0x7f7f4c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:11.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.856+0000 7f7f69400640 1 --2- 172.21.15.7:0/1770391623 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f4c077fa0 0x7f7f4c07a460 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7f58007fd0 tx=0x7f7f58009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:11.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:11.859+0000 7f7f62000640 1 -- 172.21.15.7:0/1770391623 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7f5c084240 con 0x7f7f6c0cd750 2024-09-10T21:46:12.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:12.091+0000 7f7f70c00640 1 -- 172.21.15.7:0/1770391623 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f6c0c3870 con 0x7f7f6c0cd750 2024-09-10T21:46:12.092 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:12.091+0000 7f7f62000640 1 -- 172.21.15.7:0/1770391623 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+5225 (secure 0 0 0) 0x7f7f5c083b40 con 0x7f7f6c0cd750 2024-09-10T21:46:12.092 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:46:12.092 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":49,"btime":"2024-09-10T21:46:11:665681+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5884,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3488410478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3488410478},{"type":"v1","addr":"172.21.15.133:6837","nonce":3488410478}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5890,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3660400727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3660400727},{"type":"v1","addr":"172.21.15.7:6835","nonce":3660400727}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2388873704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2388873704},{"type":"v1","addr":"172.21.15.7:6834","nonce":2388873704}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":49,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:46:00.260975+0000","modified":"2024-09-10T21:46:11.665664+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235490":{"gid":235490,"name":"b","rank":0,"incarnation":49,"state":"up:replay","state_seq":1,"addr":"172.21.15.133:6836/2622530239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2622530239},{"type":"v1","addr":"172.21.15.133:6836","nonce":2622530239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-09-10T21:46:12.093 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 49 2024-09-10T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:12.094+0000 7f7f70c00640 1 -- 172.21.15.7:0/1770391623 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f4c077fa0 msgr2=0x7f7f4c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:12.094+0000 7f7f70c00640 1 --2- 172.21.15.7:0/1770391623 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f4c077fa0 0x7f7f4c07a460 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7f58007fd0 tx=0x7f7f58009040 comp rx=0 tx=0).stop 2024-09-10T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:12.094+0000 7f7f70c00640 1 -- 172.21.15.7:0/1770391623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c0cd750 msgr2=0x7f7f6c0aea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:12.094+0000 7f7f70c00640 1 --2- 172.21.15.7:0/1770391623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c0cd750 0x7f7f6c0aea30 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c000c00 tx=0x7f7f5c032c10 comp rx=0 tx=0).stop 2024-09-10T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:12.094+0000 7f7f70c00640 1 -- 172.21.15.7:0/1770391623 shutdown_connections 2024-09-10T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:12.094+0000 7f7f70c00640 1 --2- 172.21.15.7:0/1770391623 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7f4c077fa0 0x7f7f4c07a460 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:12.094+0000 7f7f70c00640 1 --2- 172.21.15.7:0/1770391623 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7f6c15bc90 0x7f7f6c15a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:12.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:12.094+0000 7f7f70c00640 1 --2- 172.21.15.7:0/1770391623 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7f6c15fda0 0x7f7f6c15e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:12.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:12.094+0000 7f7f70c00640 1 --2- 172.21.15.7:0/1770391623 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7f6c0cd750 0x7f7f6c0aea30 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:12.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:12.094+0000 7f7f70c00640 1 -- 172.21.15.7:0/1770391623 >> 172.21.15.7:0/1770391623 conn(0x7f7f6c0b16d0 msgr2=0x7f7f6c0b2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:12.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:12.094+0000 7f7f70c00640 1 -- 172.21.15.7:0/1770391623 shutdown_connections 2024-09-10T21:46:12.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:12.094+0000 7f7f70c00640 1 -- 172.21.15.7:0/1770391623 wait complete. 2024-09-10T21:46:12.513 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 49, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:46:00.260975+0000', 'modified': '2024-09-10T21:46:11.665664+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 235490}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_235490': {'gid': 235490, 'name': 'b', 'rank': 0, 'incarnation': 49, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.133:6836/2622530239', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 2622530239}, {'type': 'v1', 'addr': '172.21.15.133:6836', 'nonce': 2622530239}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:46:12.514 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_235490:up:replay 2024-09-10T21:46:13.515 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:46:13.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.685+0000 7f77e0a00640 1 -- 172.21.15.7:0/3141630499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77dc073c70 msgr2=0x7f77dc15fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:13.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.685+0000 7f77e0a00640 1 --2- 172.21.15.7:0/3141630499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77dc073c70 0x7f77dc15fda0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f77dc06db70 tx=0x7f77d402a8f0 comp rx=0 tx=0).stop 2024-09-10T21:46:13.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.685+0000 7f77e0a00640 1 -- 172.21.15.7:0/3141630499 shutdown_connections 2024-09-10T21:46:13.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.685+0000 7f77e0a00640 1 --2- 172.21.15.7:0/3141630499 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77dc073c70 0x7f77dc15fda0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:13.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.685+0000 7f77e0a00640 1 --2- 172.21.15.7:0/3141630499 >> v2:172.21.15.133:3300/0 conn(0x7f77dc0cd750 0x7f77dc073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:13.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.685+0000 7f77e0a00640 1 --2- 172.21.15.7:0/3141630499 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77dc15bc90 0x7f77dc0ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:13.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.685+0000 7f77e0a00640 1 -- 172.21.15.7:0/3141630499 >> 172.21.15.7:0/3141630499 conn(0x7f77dc0b16d0 msgr2=0x7f77dc0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:13.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.685+0000 7f77e0a00640 1 -- 172.21.15.7:0/3141630499 shutdown_connections 2024-09-10T21:46:13.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.685+0000 7f77e0a00640 1 -- 172.21.15.7:0/3141630499 wait complete. 2024-09-10T21:46:13.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.687+0000 7f77e0a00640 1 Processor -- start 2024-09-10T21:46:13.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.687+0000 7f77e0a00640 1 -- start start 2024-09-10T21:46:13.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.688+0000 7f77e0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77dc073c70 0x7f77dc15f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:13.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.688+0000 7f77e0a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77dc0cd750 0x7f77dc15cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:13.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.688+0000 7f77e0a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77dc15bc90 0x7f77dc0b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:13.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.688+0000 7f77e0a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc160670 con 0x7f77dc073c70 2024-09-10T21:46:13.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.688+0000 7f77e0a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f77dc1604f0 con 0x7f77dc15bc90 2024-09-10T21:46:13.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.688+0000 7f77e0a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f77dc06fcf0 con 0x7f77dc0cd750 2024-09-10T21:46:13.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.688+0000 7f77d9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77dc073c70 0x7f77dc15f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:13.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.688+0000 7f77d9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77dc073c70 0x7f77dc15f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45046/0 (socket says 172.21.15.7:45046) 2024-09-10T21:46:13.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.688+0000 7f77d9e00640 1 -- 172.21.15.7:0/2893463957 learned_addr learned my addr 172.21.15.7:0/2893463957 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:13.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.688+0000 7f77da800640 1 --2- 172.21.15.7:0/2893463957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77dc15bc90 0x7f77dc0b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:13.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.689+0000 7f77d1400640 1 --2- 172.21.15.7:0/2893463957 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77dc0cd750 0x7f77dc15cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:13.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.689+0000 7f77d9e00640 1 -- 172.21.15.7:0/2893463957 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77dc0cd750 msgr2=0x7f77dc15cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:13.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.689+0000 7f77d9e00640 1 --2- 172.21.15.7:0/2893463957 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77dc0cd750 0x7f77dc15cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:13.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.689+0000 7f77d9e00640 1 -- 172.21.15.7:0/2893463957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77dc15bc90 msgr2=0x7f77dc0b31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:13.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.689+0000 7f77d9e00640 1 --2- 172.21.15.7:0/2893463957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77dc15bc90 0x7f77dc0b31e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:13.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.689+0000 7f77d9e00640 1 -- 172.21.15.7:0/2893463957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77dc0cbb20 con 0x7f77dc073c70 2024-09-10T21:46:13.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.689+0000 7f77d1400640 1 --2- 172.21.15.7:0/2893463957 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77dc0cd750 0x7f77dc15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:46:13.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.689+0000 7f77d9e00640 1 --2- 172.21.15.7:0/2893463957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77dc073c70 0x7f77dc15f250 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00c970 tx=0x7f77cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:13.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.689+0000 7f77d2a00640 1 -- 172.21.15.7:0/2893463957 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc007bb0 con 0x7f77dc073c70 2024-09-10T21:46:13.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.689+0000 7f77d2a00640 1 -- 172.21.15.7:0/2893463957 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77cc007e40 con 0x7f77dc073c70 2024-09-10T21:46:13.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.689+0000 7f77d2a00640 1 -- 172.21.15.7:0/2893463957 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc00f690 con 0x7f77dc073c70 2024-09-10T21:46:13.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.689+0000 7f77e0a00640 1 -- 172.21.15.7:0/2893463957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77dc0cbe00 con 0x7f77dc073c70 2024-09-10T21:46:13.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.689+0000 7f77e0a00640 1 -- 172.21.15.7:0/2893463957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77dc0b3720 con 0x7f77dc073c70 2024-09-10T21:46:13.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.691+0000 7f77e0a00640 1 -- 172.21.15.7:0/2893463957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77dc073730 con 0x7f77dc073c70 2024-09-10T21:46:13.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.692+0000 7f77d2a00640 1 -- 172.21.15.7:0/2893463957 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f77cc020030 con 0x7f77dc073c70 2024-09-10T21:46:13.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.692+0000 7f77d2a00640 1 --2- 172.21.15.7:0/2893463957 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f77b8077f70 0x7f77b807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:13.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.693+0000 7f77d2a00640 1 -- 172.21.15.7:0/2893463957 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7f77cc09a2d0 con 0x7f77dc073c70 2024-09-10T21:46:13.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.693+0000 7f77d1400640 1 --2- 172.21.15.7:0/2893463957 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f77b8077f70 0x7f77b807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:13.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.694+0000 7f77d1400640 1 --2- 172.21.15.7:0/2893463957 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f77b8077f70 0x7f77b807a430 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f77dc15c7a0 tx=0x7f77c4005d90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:13.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.697+0000 7f77d2a00640 1 -- 172.21.15.7:0/2893463957 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f77cc014030 con 0x7f77dc073c70 2024-09-10T21:46:13.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.931+0000 7f77e0a00640 1 -- 172.21.15.7:0/2893463957 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f77dc0b1ea0 con 0x7f77dc073c70 2024-09-10T21:46:13.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.932+0000 7f77d2a00640 1 -- 172.21.15.7:0/2893463957 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 51 v51) ==== 94+0+5225 (secure 0 0 0) 0x7f77cc062d60 con 0x7f77dc073c70 2024-09-10T21:46:13.933 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:46:13.933 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":51,"btime":"2024-09-10T21:46:13:670284+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5884,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3488410478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3488410478},{"type":"v1","addr":"172.21.15.133:6837","nonce":3488410478}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5890,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3660400727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3660400727},{"type":"v1","addr":"172.21.15.7:6835","nonce":3660400727}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2388873704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2388873704},{"type":"v1","addr":"172.21.15.7:6834","nonce":2388873704}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":51,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:46:00.260975+0000","modified":"2024-09-10T21:46:12.673847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235490":{"gid":235490,"name":"b","rank":0,"incarnation":49,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.133:6836/2622530239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2622530239},{"type":"v1","addr":"172.21.15.133:6836","nonce":2622530239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":5}]} 2024-09-10T21:46:13.934 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 51 2024-09-10T21:46:13.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.935+0000 7f77e0a00640 1 -- 172.21.15.7:0/2893463957 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f77b8077f70 msgr2=0x7f77b807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:13.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.935+0000 7f77e0a00640 1 --2- 172.21.15.7:0/2893463957 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f77b8077f70 0x7f77b807a430 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f77dc15c7a0 tx=0x7f77c4005d90 comp rx=0 tx=0).stop 2024-09-10T21:46:13.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.935+0000 7f77e0a00640 1 -- 172.21.15.7:0/2893463957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77dc073c70 msgr2=0x7f77dc15f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:13.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.935+0000 7f77e0a00640 1 --2- 172.21.15.7:0/2893463957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77dc073c70 0x7f77dc15f250 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00c970 tx=0x7f77cc00ce30 comp rx=0 tx=0).stop 2024-09-10T21:46:13.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.935+0000 7f77e0a00640 1 -- 172.21.15.7:0/2893463957 shutdown_connections 2024-09-10T21:46:13.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.935+0000 7f77e0a00640 1 --2- 172.21.15.7:0/2893463957 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f77b8077f70 0x7f77b807a430 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:13.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.935+0000 7f77e0a00640 1 --2- 172.21.15.7:0/2893463957 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f77dc15bc90 0x7f77dc0b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:13.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.935+0000 7f77e0a00640 1 --2- 172.21.15.7:0/2893463957 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f77dc0cd750 0x7f77dc15cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:13.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.935+0000 7f77e0a00640 1 --2- 172.21.15.7:0/2893463957 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f77dc073c70 0x7f77dc15f250 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:13.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.935+0000 7f77e0a00640 1 -- 172.21.15.7:0/2893463957 >> 172.21.15.7:0/2893463957 conn(0x7f77dc0b16d0 msgr2=0x7f77dc15b210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:13.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.935+0000 7f77e0a00640 1 -- 172.21.15.7:0/2893463957 shutdown_connections 2024-09-10T21:46:13.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:13.935+0000 7f77e0a00640 1 -- 172.21.15.7:0/2893463957 wait complete. 2024-09-10T21:46:14.355 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 51, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:46:00.260975+0000', 'modified': '2024-09-10T21:46:12.673847+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 235490}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_235490': {'gid': 235490, 'name': 'b', 'rank': 0, 'incarnation': 49, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.133:6836/2622530239', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 2622530239}, {'type': 'v1', 'addr': '172.21.15.133:6836', 'nonce': 2622530239}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:46:14.355 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_235490:up:rejoin 2024-09-10T21:46:15.356 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:46:15.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.527+0000 7fdc0be00640 1 -- 172.21.15.7:0/2708151031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc0415bc90 msgr2=0x7fdc040ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:15.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.527+0000 7fdc0be00640 1 --2- 172.21.15.7:0/2708151031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc0415bc90 0x7fdc040ccef0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8009830 tx=0x7fdbf802a520 comp rx=0 tx=0).stop 2024-09-10T21:46:15.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.528+0000 7fdc0be00640 1 -- 172.21.15.7:0/2708151031 shutdown_connections 2024-09-10T21:46:15.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.528+0000 7fdc0be00640 1 --2- 172.21.15.7:0/2708151031 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc0415bc90 0x7fdc040ccef0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:15.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.528+0000 7fdc0be00640 1 -- 172.21.15.7:0/2708151031 >> 172.21.15.7:0/2708151031 conn(0x7fdc040b16d0 msgr2=0x7fdc040b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:15.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.528+0000 7fdc0be00640 1 -- 172.21.15.7:0/2708151031 shutdown_connections 2024-09-10T21:46:15.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.528+0000 7fdc0be00640 1 -- 172.21.15.7:0/2708151031 wait complete. 2024-09-10T21:46:15.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.529+0000 7fdc0be00640 1 Processor -- start 2024-09-10T21:46:15.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.529+0000 7fdc0be00640 1 -- start start 2024-09-10T21:46:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.530+0000 7fdc0be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc040cd750 0x7fdc04156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.530+0000 7fdc0be00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc0415bc90 0x7fdc0415ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.530+0000 7fdc0be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc0415fda0 0x7fdc0415efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.530+0000 7fdc0be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdc04070080 con 0x7fdc0415fda0 2024-09-10T21:46:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.530+0000 7fdc0be00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdc0406ff00 con 0x7fdc040cd750 2024-09-10T21:46:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.530+0000 7fdc0be00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fdc04070200 con 0x7fdc0415bc90 2024-09-10T21:46:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.531+0000 7fdc09e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc0415fda0 0x7fdc0415efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.531+0000 7fdc09e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc0415fda0 0x7fdc0415efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49370/0 (socket says 172.21.15.7:49370) 2024-09-10T21:46:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.531+0000 7fdc09e00640 1 -- 172.21.15.7:0/2046799898 learned_addr learned my addr 172.21.15.7:0/2046799898 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.531+0000 7fdc09400640 1 --2- 172.21.15.7:0/2046799898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc040cd750 0x7fdc04156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.531+0000 7fdc08a00640 1 --2- 172.21.15.7:0/2046799898 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc0415bc90 0x7fdc0415ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:15.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.531+0000 7fdc09e00640 1 -- 172.21.15.7:0/2046799898 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc0415bc90 msgr2=0x7fdc0415ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.531+0000 7fdc09e00640 1 --2- 172.21.15.7:0/2046799898 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc0415bc90 0x7fdc0415ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.531+0000 7fdc09e00640 1 -- 172.21.15.7:0/2046799898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc040cd750 msgr2=0x7fdc04156d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.531+0000 7fdc09e00640 1 --2- 172.21.15.7:0/2046799898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc040cd750 0x7fdc04156d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.531+0000 7fdc09e00640 1 -- 172.21.15.7:0/2046799898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc041f30e0 con 0x7fdc0415fda0 2024-09-10T21:46:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.531+0000 7fdc08a00640 1 --2- 172.21.15.7:0/2046799898 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc0415bc90 0x7fdc0415ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.531+0000 7fdc09e00640 1 --2- 172.21.15.7:0/2046799898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc0415fda0 0x7fdc0415efc0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00c700 tx=0x7fdbfc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.532+0000 7fdc01600640 1 -- 172.21.15.7:0/2046799898 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc003b70 con 0x7fdc0415fda0 2024-09-10T21:46:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.532+0000 7fdc0be00640 1 -- 172.21.15.7:0/2046799898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc040c7dd0 con 0x7fdc0415fda0 2024-09-10T21:46:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.532+0000 7fdc01600640 1 -- 172.21.15.7:0/2046799898 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbfc003d10 con 0x7fdc0415fda0 2024-09-10T21:46:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.532+0000 7fdc01600640 1 -- 172.21.15.7:0/2046799898 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc005330 con 0x7fdc0415fda0 2024-09-10T21:46:15.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.532+0000 7fdc0be00640 1 -- 172.21.15.7:0/2046799898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc040c8390 con 0x7fdc0415fda0 2024-09-10T21:46:15.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.533+0000 7fdc0be00640 1 -- 172.21.15.7:0/2046799898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc040706b0 con 0x7fdc0415fda0 2024-09-10T21:46:15.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.534+0000 7fdc01600640 1 -- 172.21.15.7:0/2046799898 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fdbfc0054d0 con 0x7fdc0415fda0 2024-09-10T21:46:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.535+0000 7fdc01600640 1 --2- 172.21.15.7:0/2046799898 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdbe4077f70 0x7fdbe407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.535+0000 7fdc01600640 1 -- 172.21.15.7:0/2046799898 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7fdbfc099ac0 con 0x7fdc0415fda0 2024-09-10T21:46:15.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.536+0000 7fdc09400640 1 --2- 172.21.15.7:0/2046799898 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdbe4077f70 0x7fdbe407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:15.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.536+0000 7fdc09400640 1 --2- 172.21.15.7:0/2046799898 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdbe4077f70 0x7fdbe407a430 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8034000 tx=0x7fdbf8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:15.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.539+0000 7fdc01600640 1 -- 172.21.15.7:0/2046799898 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fdbfc0057f0 con 0x7fdc0415fda0 2024-09-10T21:46:15.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.770+0000 7fdc0be00640 1 -- 172.21.15.7:0/2046799898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc040c8560 con 0x7fdc0415fda0 2024-09-10T21:46:15.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.771+0000 7fdc01600640 1 -- 172.21.15.7:0/2046799898 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) ==== 94+0+5236 (secure 0 0 0) 0x7fdbfc0624d0 con 0x7fdc0415fda0 2024-09-10T21:46:15.772 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:46:15.772 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":52,"btime":"2024-09-10T21:46:14:672054+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5884,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3488410478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3488410478},{"type":"v1","addr":"172.21.15.133:6837","nonce":3488410478}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5890,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3660400727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3660400727},{"type":"v1","addr":"172.21.15.7:6835","nonce":3660400727}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2388873704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2388873704},{"type":"v1","addr":"172.21.15.7:6834","nonce":2388873704}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:46:00.260975+0000","modified":"2024-09-10T21:46:14.672047+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235490":{"gid":235490,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6836/2622530239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2622530239},{"type":"v1","addr":"172.21.15.133:6836","nonce":2622530239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235490,"qdb_cluster":[235490]},"id":5}]} 2024-09-10T21:46:15.772 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 52 2024-09-10T21:46:15.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.773+0000 7fdc0be00640 1 -- 172.21.15.7:0/2046799898 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdbe4077f70 msgr2=0x7fdbe407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:15.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.773+0000 7fdc0be00640 1 --2- 172.21.15.7:0/2046799898 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdbe4077f70 0x7fdbe407a430 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fdbf8034000 tx=0x7fdbf8004730 comp rx=0 tx=0).stop 2024-09-10T21:46:15.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.773+0000 7fdc0be00640 1 -- 172.21.15.7:0/2046799898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc0415fda0 msgr2=0x7fdc0415efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:15.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.773+0000 7fdc0be00640 1 --2- 172.21.15.7:0/2046799898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc0415fda0 0x7fdc0415efc0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc00c700 tx=0x7fdbfc00cbc0 comp rx=0 tx=0).stop 2024-09-10T21:46:15.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.773+0000 7fdc0be00640 1 -- 172.21.15.7:0/2046799898 shutdown_connections 2024-09-10T21:46:15.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.773+0000 7fdc0be00640 1 --2- 172.21.15.7:0/2046799898 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdbe4077f70 0x7fdbe407a430 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:15.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.773+0000 7fdc0be00640 1 --2- 172.21.15.7:0/2046799898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdc0415fda0 0x7fdc0415efc0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:15.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.773+0000 7fdc0be00640 1 --2- 172.21.15.7:0/2046799898 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdc0415bc90 0x7fdc0415ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:15.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.773+0000 7fdc0be00640 1 --2- 172.21.15.7:0/2046799898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdc040cd750 0x7fdc04156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:15.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.773+0000 7fdc0be00640 1 -- 172.21.15.7:0/2046799898 >> 172.21.15.7:0/2046799898 conn(0x7fdc040b16d0 msgr2=0x7fdc040b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:15.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.773+0000 7fdc0be00640 1 -- 172.21.15.7:0/2046799898 shutdown_connections 2024-09-10T21:46:15.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:15.773+0000 7fdc0be00640 1 -- 172.21.15.7:0/2046799898 wait complete. 2024-09-10T21:46:16.194 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:46:00.260975+0000', 'modified': '2024-09-10T21:46:14.672047+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 235490}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_235490': {'gid': 235490, 'name': 'b', 'rank': 0, 'incarnation': 49, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.133:6836/2622530239', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 2622530239}, {'type': 'v1', 'addr': '172.21.15.133:6836', 'nonce': 2622530239}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 235490, 'qdb_cluster': [235490]} 2024-09-10T21:46:16.194 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:46:16.194 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T21:46:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.357+0000 7f22a3200640 1 -- 172.21.15.7:0/3713784870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a40d2a20 msgr2=0x7f22a40d2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.357+0000 7f22a3200640 1 --2- 172.21.15.7:0/3713784870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a40d2a20 0x7f22a40d2240 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f22a40d1e10 tx=0x7f228c02a6c0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.357+0000 7f22a3200640 1 -- 172.21.15.7:0/3713784870 shutdown_connections 2024-09-10T21:46:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.357+0000 7f22a3200640 1 --2- 172.21.15.7:0/3713784870 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a40d2a20 0x7f22a40d2240 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.357+0000 7f22a3200640 1 --2- 172.21.15.7:0/3713784870 >> v2:172.21.15.133:3300/0 conn(0x7f22a40d1e60 0x7f22a40d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.357+0000 7f22a3200640 1 -- 172.21.15.7:0/3713784870 >> 172.21.15.7:0/3713784870 conn(0x7f22a40d3de0 msgr2=0x7f22a40d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:16.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.357+0000 7f22a3200640 1 -- 172.21.15.7:0/3713784870 shutdown_connections 2024-09-10T21:46:16.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.357+0000 7f22a3200640 1 -- 172.21.15.7:0/3713784870 wait complete. 2024-09-10T21:46:16.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.358+0000 7f22a3200640 1 Processor -- start 2024-09-10T21:46:16.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.358+0000 7f22a3200640 1 -- start start 2024-09-10T21:46:16.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.359+0000 7f22a3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a40d07f0 0x7f22a40cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.359+0000 7f22a3200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a40d1e60 0x7f22a4155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.359+0000 7f22a3200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a40d2a20 0x7f22a409fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.359+0000 7f22a3200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22a40d1530 con 0x7f22a40d07f0 2024-09-10T21:46:16.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.359+0000 7f22a3200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f22a40d13b0 con 0x7f22a40d2a20 2024-09-10T21:46:16.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.359+0000 7f22a3200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f22a40c92e0 con 0x7f22a40d1e60 2024-09-10T21:46:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.360+0000 7f22a1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a40d07f0 0x7f22a40cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.360+0000 7f22a1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a40d07f0 0x7f22a40cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49376/0 (socket says 172.21.15.7:49376) 2024-09-10T21:46:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.360+0000 7f22a1e00640 1 -- 172.21.15.7:0/2560070505 learned_addr learned my addr 172.21.15.7:0/2560070505 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.360+0000 7f2299400640 1 --2- 172.21.15.7:0/2560070505 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a40d1e60 0x7f22a4155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.360+0000 7f22a2800640 1 --2- 172.21.15.7:0/2560070505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a40d2a20 0x7f22a409fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.360+0000 7f22a1e00640 1 -- 172.21.15.7:0/2560070505 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a40d1e60 msgr2=0x7f22a4155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:16.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.360+0000 7f22a1e00640 1 --2- 172.21.15.7:0/2560070505 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a40d1e60 0x7f22a4155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.360+0000 7f22a1e00640 1 -- 172.21.15.7:0/2560070505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a40d2a20 msgr2=0x7f22a409fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.360+0000 7f22a1e00640 1 --2- 172.21.15.7:0/2560070505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a40d2a20 0x7f22a409fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.360+0000 7f22a1e00640 1 -- 172.21.15.7:0/2560070505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a409e5c0 con 0x7f22a40d07f0 2024-09-10T21:46:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.360+0000 7f22a1e00640 1 --2- 172.21.15.7:0/2560070505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a40d07f0 0x7f22a40cd8a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f229400eab0 tx=0x7f229400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.361+0000 7f229aa00640 1 -- 172.21.15.7:0/2560070505 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229400ce40 con 0x7f22a40d07f0 2024-09-10T21:46:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.361+0000 7f229aa00640 1 -- 172.21.15.7:0/2560070505 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2294011560 con 0x7f22a40d07f0 2024-09-10T21:46:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.361+0000 7f229aa00640 1 -- 172.21.15.7:0/2560070505 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294011930 con 0x7f22a40d07f0 2024-09-10T21:46:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.361+0000 7f22a3200640 1 -- 172.21.15.7:0/2560070505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a415d870 con 0x7f22a40d07f0 2024-09-10T21:46:16.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.361+0000 7f22a3200640 1 -- 172.21.15.7:0/2560070505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a409eb10 con 0x7f22a40d07f0 2024-09-10T21:46:16.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.363+0000 7f229aa00640 1 -- 172.21.15.7:0/2560070505 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2294022030 con 0x7f22a40d07f0 2024-09-10T21:46:16.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.364+0000 7f229aa00640 1 --2- 172.21.15.7:0/2560070505 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2280077fa0 0x7f228007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.364+0000 7f229aa00640 1 -- 172.21.15.7:0/2560070505 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7f229409a680 con 0x7f22a40d07f0 2024-09-10T21:46:16.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.364+0000 7f22ac072740 1 Processor -- start 2024-09-10T21:46:16.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.365+0000 7f22ac072740 1 -- start start 2024-09-10T21:46:16.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.365+0000 7f2299400640 1 --2- 172.21.15.7:0/2560070505 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2280077fa0 0x7f228007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.366+0000 7f2299400640 1 --2- 172.21.15.7:0/2560070505 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2280077fa0 0x7f228007a460 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f228c033000 tx=0x7f228c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.366+0000 7f22ac072740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f8c8aa0 0x55613f8c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.366+0000 7f22ac072740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55613f883c10 0x55613f8c7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.366+0000 7f22ac072740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f884190 0x55613f8c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.366+0000 7f22ac072740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55613f806430 con 0x55613f8c8aa0 2024-09-10T21:46:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.366+0000 7f22ac072740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55613f7f65d0 con 0x55613f884190 2024-09-10T21:46:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.366+0000 7f22ac072740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55613f7e5a00 con 0x55613f883c10 2024-09-10T21:46:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.366+0000 7f22a1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f8c8aa0 0x55613f8c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.366+0000 7f22a1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f8c8aa0 0x55613f8c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49392/0 (socket says 172.21.15.7:49392) 2024-09-10T21:46:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.366+0000 7f22a1e00640 1 -- 172.21.15.7:0/4648203 learned_addr learned my addr 172.21.15.7:0/4648203 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.366+0000 7f22a2800640 1 --2- 172.21.15.7:0/4648203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f884190 0x55613f8c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.366+0000 7f2299400640 1 --2- 172.21.15.7:0/4648203 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55613f883c10 0x55613f8c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.366+0000 7f22a1e00640 1 -- 172.21.15.7:0/4648203 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55613f883c10 msgr2=0x55613f8c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.366+0000 7f22a1e00640 1 --2- 172.21.15.7:0/4648203 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55613f883c10 0x55613f8c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.366+0000 7f22a1e00640 1 -- 172.21.15.7:0/4648203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f884190 msgr2=0x55613f8c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.366+0000 7f22a1e00640 1 --2- 172.21.15.7:0/4648203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f884190 0x55613f8c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.366+0000 7f22a1e00640 1 -- 172.21.15.7:0/4648203 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55613f7ae520 con 0x55613f8c8aa0 2024-09-10T21:46:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.367+0000 7f22a1e00640 1 --2- 172.21.15.7:0/4648203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f8c8aa0 0x55613f8c8e80 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2294062a50 tx=0x7f2294062710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2f40d9bfdab6b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.367+0000 7f2292a00640 1 -- 172.21.15.7:0/4648203 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229406bf10 con 0x55613f8c8aa0 2024-09-10T21:46:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.367+0000 7f2292a00640 1 -- 172.21.15.7:0/4648203 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2294067070 con 0x55613f8c8aa0 2024-09-10T21:46:16.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.367+0000 7f2292a00640 1 -- 172.21.15.7:0/4648203 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294084b50 con 0x55613f8c8aa0 2024-09-10T21:46:16.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.367+0000 7f22ac072740 1 -- 172.21.15.7:0/4648203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f8c8aa0 msgr2=0x55613f8c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:16.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.367+0000 7f22ac072740 1 --2- 172.21.15.7:0/4648203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f8c8aa0 0x55613f8c8e80 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2294062a50 tx=0x7f2294062710 comp rx=0 tx=0).stop 2024-09-10T21:46:16.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.368+0000 7f22ac072740 1 -- 172.21.15.7:0/4648203 shutdown_connections 2024-09-10T21:46:16.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.368+0000 7f22ac072740 1 --2- 172.21.15.7:0/4648203 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f884190 0x55613f8c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.368+0000 7f22ac072740 1 --2- 172.21.15.7:0/4648203 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55613f883c10 0x55613f8c7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.368+0000 7f22ac072740 1 --2- 172.21.15.7:0/4648203 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f8c8aa0 0x55613f8c8e80 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.368+0000 7f22ac072740 1 -- 172.21.15.7:0/4648203 >> 172.21.15.7:0/4648203 conn(0x55613f87f1e0 msgr2=0x55613f882c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:16.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.368+0000 7f22ac072740 1 -- 172.21.15.7:0/4648203 shutdown_connections 2024-09-10T21:46:16.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.368+0000 7f22ac072740 1 -- 172.21.15.7:0/4648203 wait complete. 2024-09-10T21:46:16.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.368+0000 7f22ac072740 1 -- start start 2024-09-10T21:46:16.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.369+0000 7f22ac072740 1 Processor -- start 2024-09-10T21:46:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.370+0000 7f22ac072740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f884190 0x55613f8c7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.370+0000 7f22ac072740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55613f8c8aa0 0x55613f89ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.370+0000 7f22ac072740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f89e490 0x55613f8983f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.370+0000 7f22ac072740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55613f78c330 con 0x55613f884190 2024-09-10T21:46:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.370+0000 7f22ac072740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55613f7e9810 con 0x55613f89e490 2024-09-10T21:46:16.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.370+0000 7f22ac072740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55613f556620 con 0x55613f8c8aa0 2024-09-10T21:46:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.370+0000 7f22ac072740 10 client.0 _ll_register_callbacks cb 0x55613f791420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:46:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.370+0000 7f22a1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f884190 0x55613f8c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.370+0000 7f22a1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f884190 0x55613f8c7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49408/0 (socket says 172.21.15.7:49408) 2024-09-10T21:46:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.370+0000 7f22a1e00640 1 -- 172.21.15.7:0/2415908416 learned_addr learned my addr 172.21.15.7:0/2415908416 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.370+0000 7f22a2800640 1 --2- 172.21.15.7:0/2415908416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f89e490 0x55613f8983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.370+0000 7f22a1e00640 1 -- 172.21.15.7:0/2415908416 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55613f8c8aa0 msgr2=0x55613f89ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.370+0000 7f2299400640 1 --2- 172.21.15.7:0/2415908416 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55613f8c8aa0 0x55613f89ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.370+0000 7f22a1e00640 1 --2- 172.21.15.7:0/2415908416 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55613f8c8aa0 0x55613f89ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.370+0000 7f22a1e00640 1 -- 172.21.15.7:0/2415908416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f89e490 msgr2=0x55613f8983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:16.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.370+0000 7f22a1e00640 1 --2- 172.21.15.7:0/2415908416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f89e490 0x55613f8983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.370+0000 7f22a1e00640 1 -- 172.21.15.7:0/2415908416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55613f547850 con 0x55613f884190 2024-09-10T21:46:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.370+0000 7f22a2800640 1 --2- 172.21.15.7:0/2415908416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f89e490 0x55613f8983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:46:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.371+0000 7f22a1e00640 1 --2- 172.21.15.7:0/2415908416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f884190 0x55613f8c7850 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2294062a50 tx=0x7f229400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=931ae3f8487071c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.371+0000 7f2287400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:46:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.371+0000 7f2287400640 1 -- 172.21.15.7:0/2415908416 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294061d20 con 0x55613f884190 2024-09-10T21:46:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.371+0000 7f22ac072740 1 -- 172.21.15.7:0/2415908416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2294067070 con 0x55613f884190 2024-09-10T21:46:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.371+0000 7f2287400640 1 -- 172.21.15.7:0/2415908416 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7f2294061ec0 con 0x55613f884190 2024-09-10T21:46:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.371+0000 7f2287400640 1 -- 172.21.15.7:0/2415908416 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f229405e410 con 0x55613f884190 2024-09-10T21:46:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.371+0000 7f22ac072740 10 client.6378 fetch_fsmap learned FSMap version 52 2024-09-10T21:46:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.371+0000 7f22ac072740 1 -- 172.21.15.7:0/2415908416 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55613f5be6b0 con 0x55613f884190 2024-09-10T21:46:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.371+0000 7f2287400640 1 -- 172.21.15.7:0/2415908416 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2294067070 con 0x55613f884190 2024-09-10T21:46:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.371+0000 7f2287400640 1 -- 172.21.15.7:0/2415908416 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7f2294091ce0 con 0x55613f884190 2024-09-10T21:46:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.371+0000 7f22ac072740 10 client.6378 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T21:46:16.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.371+0000 7f22ac072740 10 client.6378 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:46:16.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.371+0000 7f22ac072740 20 client.6378 populate_metadata read hostname 'smithi007' 2024-09-10T21:46:16.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.371+0000 7f22ac072740 1 --2- 172.21.15.7:0/2415908416 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55613f8c5650 0x55613f8c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.372+0000 7f22ac072740 4 client.6378 mds_command: new command op to 235490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:46:16.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.372+0000 7f22ac072740 1 -- 172.21.15.7:0/2415908416 --> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55613f7f8520 con 0x55613f8c5650 2024-09-10T21:46:16.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.372+0000 7f2299400640 1 --2- 172.21.15.7:0/2415908416 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55613f8c5650 0x55613f8c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.373+0000 7f2299400640 1 --2- 172.21.15.7:0/2415908416 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55613f8c5650 0x55613f8c5a30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b72ec7fce3df0f39 server_cookie=8786619770cb12da in_seq=0 out_seq=0 2024-09-10T21:46:16.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.373+0000 7f2287400640 10 client.6378 ms_handle_connect on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:16.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.374+0000 7f2287400640 1 -- 172.21.15.7:0/2415908416 <== mds.0 v2:172.21.15.133:6834/2622530239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55613f7f8520 con 0x55613f8c5650 2024-09-10T21:46:16.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.374+0000 7f2287400640 10 client.6378 handle_command_reply: tid=0 2024-09-10T21:46:16.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.374+0000 7f22ac072740 1 client.6378 shutdown 2024-09-10T21:46:16.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.374+0000 7f22ac072740 1 -- 172.21.15.7:0/2415908416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f884190 msgr2=0x55613f8c7850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:16.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.374+0000 7f22ac072740 1 --2- 172.21.15.7:0/2415908416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f884190 0x55613f8c7850 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2294062a50 tx=0x7f229400c830 comp rx=0 tx=0).stop 2024-09-10T21:46:16.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.374+0000 7f22ac072740 1 -- 172.21.15.7:0/2415908416 shutdown_connections 2024-09-10T21:46:16.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.374+0000 7f22ac072740 1 --2- 172.21.15.7:0/2415908416 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f89e490 0x55613f8983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.374+0000 7f22ac072740 1 --2- 172.21.15.7:0/2415908416 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55613f8c5650 0x55613f8c5a30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.374+0000 7f22ac072740 1 --2- 172.21.15.7:0/2415908416 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55613f8c8aa0 0x55613f89ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.374+0000 7f22ac072740 1 --2- 172.21.15.7:0/2415908416 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f884190 0x55613f8c7850 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.374+0000 7f2287400640 0 client.6378 ms_handle_reset on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:16.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.374+0000 7f22ac072740 1 -- 172.21.15.7:0/2415908416 >> 172.21.15.7:0/2415908416 conn(0x55613f883c10 msgr2=0x55613f87ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:16.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.374+0000 7f22ac072740 1 -- 172.21.15.7:0/2415908416 shutdown_connections 2024-09-10T21:46:16.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.374+0000 7f22ac072740 1 -- 172.21.15.7:0/2415908416 wait complete. 2024-09-10T21:46:16.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.374+0000 7f22ac072740 20 client.6378 trim_cache size 0 max 16384 2024-09-10T21:46:16.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.394+0000 7f22ac072740 1 Processor -- start 2024-09-10T21:46:16.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.394+0000 7f22ac072740 1 -- start start 2024-09-10T21:46:16.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.394+0000 7f22ac072740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f884190 0x55613f87c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.394+0000 7f22ac072740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55613f89e490 0x55613f8c0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.394+0000 7f22ac072740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f8c8aa0 0x55613f87d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.394+0000 7f22ac072740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22a42001e0 con 0x55613f884190 2024-09-10T21:46:16.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.394+0000 7f22ac072740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f22a41ff570 con 0x55613f8c8aa0 2024-09-10T21:46:16.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.394+0000 7f22ac072740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f22a41ffb90 con 0x55613f89e490 2024-09-10T21:46:16.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.395+0000 7f22a1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f884190 0x55613f87c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.395+0000 7f22a1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f884190 0x55613f87c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49410/0 (socket says 172.21.15.7:49410) 2024-09-10T21:46:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.395+0000 7f22a1e00640 1 -- 172.21.15.7:0/1666737470 learned_addr learned my addr 172.21.15.7:0/1666737470 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.395+0000 7f22a1e00640 1 -- 172.21.15.7:0/1666737470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55613f89e490 msgr2=0x55613f8c0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.395+0000 7f22a1e00640 1 --2- 172.21.15.7:0/1666737470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55613f89e490 0x55613f8c0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.395+0000 7f22a1e00640 1 -- 172.21.15.7:0/1666737470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f8c8aa0 msgr2=0x55613f87d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.395+0000 7f22a1e00640 1 --2- 172.21.15.7:0/1666737470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f8c8aa0 0x55613f87d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.395+0000 7f22a1e00640 1 -- 172.21.15.7:0/1666737470 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55613f8ca230 con 0x55613f884190 2024-09-10T21:46:16.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.395+0000 7f22a1e00640 1 --2- 172.21.15.7:0/1666737470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f884190 0x55613f87c0f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f22940627c0 tx=0x7f229406bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fb5ac6bc1f8bd7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.396+0000 7f2292000640 1 -- 172.21.15.7:0/1666737470 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229405f110 con 0x55613f884190 2024-09-10T21:46:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.396+0000 7f2292000640 1 -- 172.21.15.7:0/1666737470 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2294067040 con 0x55613f884190 2024-09-10T21:46:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.396+0000 7f2292000640 1 -- 172.21.15.7:0/1666737470 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2294061f40 con 0x55613f884190 2024-09-10T21:46:16.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.396+0000 7f22ac072740 1 -- 172.21.15.7:0/1666737470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f884190 msgr2=0x55613f87c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:16.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.396+0000 7f22ac072740 1 --2- 172.21.15.7:0/1666737470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f884190 0x55613f87c0f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f22940627c0 tx=0x7f229406bf90 comp rx=0 tx=0).stop 2024-09-10T21:46:16.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.396+0000 7f22ac072740 1 -- 172.21.15.7:0/1666737470 shutdown_connections 2024-09-10T21:46:16.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.396+0000 7f22ac072740 1 --2- 172.21.15.7:0/1666737470 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f8c8aa0 0x55613f87d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.396+0000 7f22ac072740 1 --2- 172.21.15.7:0/1666737470 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55613f89e490 0x55613f8c0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.396+0000 7f22ac072740 1 --2- 172.21.15.7:0/1666737470 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f884190 0x55613f87c0f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.396+0000 7f22ac072740 1 -- 172.21.15.7:0/1666737470 >> 172.21.15.7:0/1666737470 conn(0x55613f883c10 msgr2=0x55613f882860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:16.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.396+0000 7f22ac072740 1 -- 172.21.15.7:0/1666737470 shutdown_connections 2024-09-10T21:46:16.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.396+0000 7f22ac072740 1 -- 172.21.15.7:0/1666737470 wait complete. 2024-09-10T21:46:16.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.397+0000 7f22ac072740 1 -- start start 2024-09-10T21:46:16.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.397+0000 7f22ac072740 1 Processor -- start 2024-09-10T21:46:16.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.398+0000 7f22ac072740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55613f884190 0x55613f8eca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.398+0000 7f22ac072740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f89e490 0x55613f94bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.398+0000 7f22ac072740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f8c8aa0 0x55613f87d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.398+0000 7f22ac072740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55613f898db0 con 0x55613f8c8aa0 2024-09-10T21:46:16.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.398+0000 7f22ac072740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55613f78c330 con 0x55613f89e490 2024-09-10T21:46:16.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.398+0000 7f22ac072740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55613f898060 con 0x55613f884190 2024-09-10T21:46:16.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.398+0000 7f22ac072740 10 client.0 _ll_register_callbacks cb 0x7f2294067630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:46:16.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.398+0000 7f22a2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f8c8aa0 0x55613f87d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.398+0000 7f2299400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f89e490 0x55613f94bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.398+0000 7f22a2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f8c8aa0 0x55613f87d070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49420/0 (socket says 172.21.15.7:49420) 2024-09-10T21:46:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.398+0000 7f22a2800640 1 -- 172.21.15.7:0/1026134916 learned_addr learned my addr 172.21.15.7:0/1026134916 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.398+0000 7f22a1e00640 1 --2- 172.21.15.7:0/1026134916 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55613f884190 0x55613f8eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.399+0000 7f22a2800640 1 -- 172.21.15.7:0/1026134916 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55613f884190 msgr2=0x55613f8eca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.399+0000 7f22a2800640 1 --2- 172.21.15.7:0/1026134916 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55613f884190 0x55613f8eca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.399+0000 7f22a2800640 1 -- 172.21.15.7:0/1026134916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f89e490 msgr2=0x55613f94bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.399+0000 7f22a2800640 1 --2- 172.21.15.7:0/1026134916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f89e490 0x55613f94bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.399+0000 7f22a2800640 1 -- 172.21.15.7:0/1026134916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55613f87e330 con 0x55613f8c8aa0 2024-09-10T21:46:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.399+0000 7f2299400640 1 --2- 172.21.15.7:0/1026134916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f89e490 0x55613f94bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:46:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.399+0000 7f22a2800640 1 --2- 172.21.15.7:0/1026134916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f8c8aa0 0x55613f87d070 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f229c000f60 tx=0x7f229c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e828cb4bfbec346 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.399+0000 7f2287400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:46:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.399+0000 7f2287400640 1 -- 172.21.15.7:0/1026134916 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f229c001bf0 con 0x55613f8c8aa0 2024-09-10T21:46:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.399+0000 7f2287400640 1 -- 172.21.15.7:0/1026134916 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7f229c010040 con 0x55613f8c8aa0 2024-09-10T21:46:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.399+0000 7f22ac072740 1 -- 172.21.15.7:0/1026134916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2294067040 con 0x55613f8c8aa0 2024-09-10T21:46:16.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.400+0000 7f2287400640 1 -- 172.21.15.7:0/1026134916 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f229c016450 con 0x55613f8c8aa0 2024-09-10T21:46:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.400+0000 7f22ac072740 10 client.6384 fetch_fsmap learned FSMap version 52 2024-09-10T21:46:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.400+0000 7f22ac072740 1 -- 172.21.15.7:0/1026134916 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55613f8ecf50 con 0x55613f8c8aa0 2024-09-10T21:46:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.400+0000 7f2287400640 1 -- 172.21.15.7:0/1026134916 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2294067040 con 0x55613f8c8aa0 2024-09-10T21:46:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.400+0000 7f2287400640 1 -- 172.21.15.7:0/1026134916 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7f229c0165f0 con 0x55613f8c8aa0 2024-09-10T21:46:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.400+0000 7f22ac072740 10 client.6384 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T21:46:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.400+0000 7f22ac072740 10 client.6384 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:46:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.400+0000 7f22ac072740 20 client.6384 populate_metadata read hostname 'smithi007' 2024-09-10T21:46:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.400+0000 7f22ac072740 1 --2- 172.21.15.7:0/1026134916 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55613f87d740 0x55613f8c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.400+0000 7f22ac072740 4 client.6384 mds_command: new command op to 235490 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:46:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.400+0000 7f22ac072740 1 -- 172.21.15.7:0/1026134916 --> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] -- command(tid 0: {"prefix": "status"}) -- 0x55613f7f3fd0 con 0x55613f87d740 2024-09-10T21:46:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.400+0000 7f22a1e00640 1 --2- 172.21.15.7:0/1026134916 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55613f87d740 0x55613f8c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.401+0000 7f22a1e00640 1 --2- 172.21.15.7:0/1026134916 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55613f87d740 0x55613f8c3dc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26284ad1a5ef7f7f server_cookie=b6b122b473f92e43 in_seq=0 out_seq=0 2024-09-10T21:46:16.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.401+0000 7f2287400640 10 client.6384 ms_handle_connect on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:16.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.401+0000 7f2287400640 1 -- 172.21.15.7:0/1026134916 <== mds.0 v2:172.21.15.133:6834/2622530239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55613f7f3fd0 con 0x55613f87d740 2024-09-10T21:46:16.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.401+0000 7f2287400640 10 client.6384 handle_command_reply: tid=0 2024-09-10T21:46:16.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.401+0000 7f22ac072740 1 client.6384 shutdown 2024-09-10T21:46:16.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.401+0000 7f22ac072740 1 -- 172.21.15.7:0/1026134916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f8c8aa0 msgr2=0x55613f87d070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:16.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.401+0000 7f22ac072740 1 --2- 172.21.15.7:0/1026134916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f8c8aa0 0x55613f87d070 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f229c000f60 tx=0x7f229c003f60 comp rx=0 tx=0).stop 2024-09-10T21:46:16.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.402+0000 7f22ac072740 1 -- 172.21.15.7:0/1026134916 shutdown_connections 2024-09-10T21:46:16.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.402+0000 7f22ac072740 1 --2- 172.21.15.7:0/1026134916 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55613f89e490 0x55613f94bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.402+0000 7f22ac072740 1 --2- 172.21.15.7:0/1026134916 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55613f87d740 0x55613f8c3dc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.402+0000 7f22ac072740 1 --2- 172.21.15.7:0/1026134916 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55613f8c8aa0 0x55613f87d070 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.402+0000 7f22ac072740 1 --2- 172.21.15.7:0/1026134916 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55613f884190 0x55613f8eca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.402+0000 7f2287400640 0 client.6384 ms_handle_reset on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:16.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.402+0000 7f22ac072740 1 -- 172.21.15.7:0/1026134916 >> 172.21.15.7:0/1026134916 conn(0x55613f883c10 msgr2=0x55613f89d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:16.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.402+0000 7f22ac072740 1 -- 172.21.15.7:0/1026134916 shutdown_connections 2024-09-10T21:46:16.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.402+0000 7f22ac072740 1 -- 172.21.15.7:0/1026134916 wait complete. 2024-09-10T21:46:16.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.402+0000 7f22ac072740 20 client.6384 trim_cache size 0 max 16384 2024-09-10T21:46:16.407 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:46:16.407 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2029451a-3c26-48c7-b874-0597197278ac", 2024-09-10T21:46:16.407 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-10T21:46:16.407 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 235490, 2024-09-10T21:46:16.407 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T21:46:16.407 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T21:46:16.407 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T21:46:16.407 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 4.7296572560000003, 2024-09-10T21:46:16.407 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 52, 2024-09-10T21:46:16.408 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 71, 2024-09-10T21:46:16.408 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 71, 2024-09-10T21:46:16.408 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 5.6765470999999996 2024-09-10T21:46:16.408 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:46:16.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.402+0000 7f22a3200640 1 -- 172.21.15.7:0/2560070505 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2280077fa0 msgr2=0x7f228007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:16.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.402+0000 7f22a3200640 1 --2- 172.21.15.7:0/2560070505 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2280077fa0 0x7f228007a460 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f228c033000 tx=0x7f228c004730 comp rx=0 tx=0).stop 2024-09-10T21:46:16.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.402+0000 7f22a3200640 1 -- 172.21.15.7:0/2560070505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a40d07f0 msgr2=0x7f22a40cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:16.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.402+0000 7f22a3200640 1 --2- 172.21.15.7:0/2560070505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a40d07f0 0x7f22a40cd8a0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f229400eab0 tx=0x7f229400ef70 comp rx=0 tx=0).stop 2024-09-10T21:46:16.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.403+0000 7f22a3200640 1 -- 172.21.15.7:0/2560070505 shutdown_connections 2024-09-10T21:46:16.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.403+0000 7f22a3200640 1 --2- 172.21.15.7:0/2560070505 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2280077fa0 0x7f228007a460 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.403+0000 7f22a3200640 1 --2- 172.21.15.7:0/2560070505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a40d2a20 0x7f22a409fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.403+0000 7f22a3200640 1 --2- 172.21.15.7:0/2560070505 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a40d1e60 0x7f22a4155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.403+0000 7f22a3200640 1 --2- 172.21.15.7:0/2560070505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a40d07f0 0x7f22a40cd8a0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.403+0000 7f22a3200640 1 -- 172.21.15.7:0/2560070505 >> 172.21.15.7:0/2560070505 conn(0x7f22a40d3de0 msgr2=0x7f22a40b7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:16.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.403+0000 7f22a3200640 1 -- 172.21.15.7:0/2560070505 shutdown_connections 2024-09-10T21:46:16.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.403+0000 7f22a3200640 1 -- 172.21.15.7:0/2560070505 wait complete. 2024-09-10T21:46:16.419 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:46:16.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.578+0000 7fd35fe00640 1 -- 172.21.15.7:0/2833285222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3580cd750 msgr2=0x7fd3580ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:16.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.578+0000 7fd35fe00640 1 --2- 172.21.15.7:0/2833285222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3580cd750 0x7fd3580ccf60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fd344009830 tx=0x7fd34402a510 comp rx=0 tx=0).stop 2024-09-10T21:46:16.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.578+0000 7fd35fe00640 1 -- 172.21.15.7:0/2833285222 shutdown_connections 2024-09-10T21:46:16.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.578+0000 7fd35fe00640 1 --2- 172.21.15.7:0/2833285222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd35806fcf0 0x7fd358070120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.578+0000 7fd35fe00640 1 --2- 172.21.15.7:0/2833285222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd3580cd750 0x7fd3580ccf60 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.578+0000 7fd35fe00640 1 -- 172.21.15.7:0/2833285222 >> 172.21.15.7:0/2833285222 conn(0x7fd3580b16d0 msgr2=0x7fd3580b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:16.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.578+0000 7fd35fe00640 1 -- 172.21.15.7:0/2833285222 shutdown_connections 2024-09-10T21:46:16.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.578+0000 7fd35fe00640 1 -- 172.21.15.7:0/2833285222 wait complete. 2024-09-10T21:46:16.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.580+0000 7fd35fe00640 1 Processor -- start 2024-09-10T21:46:16.581 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.580+0000 7fd35fe00640 1 -- start start 2024-09-10T21:46:16.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.581+0000 7fd35fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd35806fcf0 0x7fd358158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.582 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.581+0000 7fd35fe00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd3580cd750 0x7fd3580b4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.581+0000 7fd35fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd35815bc00 0x7fd358151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.581+0000 7fd35fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd358073e70 con 0x7fd35815bc00 2024-09-10T21:46:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.581+0000 7fd35fe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd358073cf0 con 0x7fd35806fcf0 2024-09-10T21:46:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.581+0000 7fd35fe00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fd358073ff0 con 0x7fd3580cd750 2024-09-10T21:46:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.581+0000 7fd35de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd35815bc00 0x7fd358151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.581+0000 7fd35de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd35815bc00 0x7fd358151af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49428/0 (socket says 172.21.15.7:49428) 2024-09-10T21:46:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.581+0000 7fd35de00640 1 -- 172.21.15.7:0/990552033 learned_addr learned my addr 172.21.15.7:0/990552033 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.581+0000 7fd35ca00640 1 --2- 172.21.15.7:0/990552033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd3580cd750 0x7fd3580b4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.581+0000 7fd35d400640 1 --2- 172.21.15.7:0/990552033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd35806fcf0 0x7fd358158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.581+0000 7fd35de00640 1 -- 172.21.15.7:0/990552033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd3580cd750 msgr2=0x7fd3580b4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.581+0000 7fd35de00640 1 --2- 172.21.15.7:0/990552033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd3580cd750 0x7fd3580b4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.581+0000 7fd35de00640 1 -- 172.21.15.7:0/990552033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd35806fcf0 msgr2=0x7fd358158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.581+0000 7fd35de00640 1 --2- 172.21.15.7:0/990552033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd35806fcf0 0x7fd358158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.581+0000 7fd35de00640 1 -- 172.21.15.7:0/990552033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3580c9960 con 0x7fd35815bc00 2024-09-10T21:46:16.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.581+0000 7fd35d400640 1 --2- 172.21.15.7:0/990552033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd35806fcf0 0x7fd358158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:46:16.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.581+0000 7fd35ca00640 1 --2- 172.21.15.7:0/990552033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd3580cd750 0x7fd3580b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:46:16.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.582+0000 7fd35de00640 1 --2- 172.21.15.7:0/990552033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd35815bc00 0x7fd358151af0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd35000ea60 tx=0x7fd35000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:16.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.582+0000 7fd355600640 1 -- 172.21.15.7:0/990552033 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35000cde0 con 0x7fd35815bc00 2024-09-10T21:46:16.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.582+0000 7fd355600640 1 -- 172.21.15.7:0/990552033 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd350004b20 con 0x7fd35815bc00 2024-09-10T21:46:16.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.582+0000 7fd355600640 1 -- 172.21.15.7:0/990552033 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd350005020 con 0x7fd35815bc00 2024-09-10T21:46:16.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.582+0000 7fd35fe00640 1 -- 172.21.15.7:0/990552033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3580c9c00 con 0x7fd35815bc00 2024-09-10T21:46:16.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.582+0000 7fd35fe00640 1 -- 172.21.15.7:0/990552033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3580c4f30 con 0x7fd35815bc00 2024-09-10T21:46:16.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.583+0000 7fd35fe00640 1 -- 172.21.15.7:0/990552033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3580745b0 con 0x7fd35815bc00 2024-09-10T21:46:16.585 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.585+0000 7fd355600640 1 -- 172.21.15.7:0/990552033 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd3500051c0 con 0x7fd35815bc00 2024-09-10T21:46:16.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.585+0000 7fd355600640 1 --2- 172.21.15.7:0/990552033 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd338077f70 0x7fd33807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:16.586 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.586+0000 7fd355600640 1 -- 172.21.15.7:0/990552033 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7fd35009aa80 con 0x7fd35815bc00 2024-09-10T21:46:16.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.586+0000 7fd35d400640 1 --2- 172.21.15.7:0/990552033 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd338077f70 0x7fd33807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:16.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.587+0000 7fd35d400640 1 --2- 172.21.15.7:0/990552033 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd338077f70 0x7fd33807a430 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd34c000f60 tx=0x7fd34c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:16.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.589+0000 7fd355600640 1 -- 172.21.15.7:0/990552033 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd35009ce20 con 0x7fd35815bc00 2024-09-10T21:46:16.675 INFO:tasks.ceph.mds.b.smithi133.stderr:2024-09-10T21:46:16.674+0000 7f567f600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:46:16.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.827+0000 7fd35fe00640 1 -- 172.21.15.7:0/990552033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd35806e730 con 0x7fd35815bc00 2024-09-10T21:46:16.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.827+0000 7fd355600640 1 -- 172.21.15.7:0/990552033 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) ==== 53+0+73 (secure 0 0 0) 0x7fd350063510 con 0x7fd35815bc00 2024-09-10T21:46:16.829 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:46:16.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.830+0000 7fd35fe00640 1 -- 172.21.15.7:0/990552033 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd338077f70 msgr2=0x7fd33807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:16.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.830+0000 7fd35fe00640 1 --2- 172.21.15.7:0/990552033 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd338077f70 0x7fd33807a430 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd34c000f60 tx=0x7fd34c009040 comp rx=0 tx=0).stop 2024-09-10T21:46:16.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.830+0000 7fd35fe00640 1 -- 172.21.15.7:0/990552033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd35815bc00 msgr2=0x7fd358151af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:16.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.830+0000 7fd35fe00640 1 --2- 172.21.15.7:0/990552033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd35815bc00 0x7fd358151af0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fd35000ea60 tx=0x7fd35000ef20 comp rx=0 tx=0).stop 2024-09-10T21:46:16.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.830+0000 7fd35fe00640 1 -- 172.21.15.7:0/990552033 shutdown_connections 2024-09-10T21:46:16.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.830+0000 7fd35fe00640 1 --2- 172.21.15.7:0/990552033 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd338077f70 0x7fd33807a430 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.830+0000 7fd35fe00640 1 --2- 172.21.15.7:0/990552033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd35815bc00 0x7fd358151af0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.830+0000 7fd35fe00640 1 --2- 172.21.15.7:0/990552033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd3580cd750 0x7fd3580b4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.830+0000 7fd35fe00640 1 --2- 172.21.15.7:0/990552033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd35806fcf0 0x7fd358158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:16.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.830+0000 7fd35fe00640 1 -- 172.21.15.7:0/990552033 >> 172.21.15.7:0/990552033 conn(0x7fd3580b16d0 msgr2=0x7fd3580b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:16.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.830+0000 7fd35fe00640 1 -- 172.21.15.7:0/990552033 shutdown_connections 2024-09-10T21:46:16.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:16.830+0000 7fd35fe00640 1 -- 172.21.15.7:0/990552033 wait complete. 2024-09-10T21:46:16.849 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:46:16.849 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:46:16.849 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-10T21:46:16.849 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:46:16.849 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:46:16.849 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:46:16.850 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:46:16.850 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:46:16.850 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:46:16.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:16.850 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-10T21:46:16.886 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:46:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:46:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:46:16.886 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-10T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-10T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-10T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-10T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T21:46:16.887 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42368sec preferred_lft 42368sec 2024-09-10T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-10T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-10T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-10T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff 2024-09-10T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::3c3d:dbff:fe02:4c15/64 scope link 2024-09-10T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:46:16.888 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether f6:f9:69:5a:86:f5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:46:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f4f9:69ff:fe5a:86f5/64 scope link 2024-09-10T21:46:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:46:16.889 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:46:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:46:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::433:5cff:fe44:4a5b/64 scope link 2024-09-10T21:46:16.889 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:46:16.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:16.892 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:46:16.981 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:46:16.984 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:16.985 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-10T21:46:17.021 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-10T21:46:17.021 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:46:17.021 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T21:46:17.021 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T21:46:17.021 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:46:17.022 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:46:17.022 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:46:17.022 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:46:17.022 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:46:17.022 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:46:17.022 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:46:17.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:17 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:46:17.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:46:17.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:17.220 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:46:17.256 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:46:17.257 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:46:17.257 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:46:17.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:17.257 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-10T21:46:17.353 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:46:17.353 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:46:17.388 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:46:17.389 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:46:17.389 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:46:17.464 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:46:17.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:17.464 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:46:17.592 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:46:17.592 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:46:17.620 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:46:17.620 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:46:17.620 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.133:3300,172.21.15.133:3301" 2024-09-10T21:46:17.621 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@2029451a-3c26-48c7-b874-0597197278ac.cephfs=/ 2024-09-10T21:46:17.621 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.133:3300/172.21.15.133:3301" will pass to kernel 2024-09-10T21:46:17.622 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T21:46:17.622 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:46:17.622 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:46:17.622 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:46:17.622 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:46:17.625 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:46:17.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:17.625 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:46:17.681 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:46:17.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:17 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:46:17.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:17 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:46:17.772 INFO:teuthology.orchestra.run.smithi007.stdout:client6396 2024-09-10T21:46:17.772 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:46:17.773 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:46:17.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:17.808 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir/) 2024-09-10T21:46:17.886 INFO:teuthology.orchestra.run.smithi007.stdout:file_undamaged 2024-09-10T21:46:17.888 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:46:17.888 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:46:17.888 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:46:17.888 DEBUG:teuthology.orchestra.run.smithi007:> import json 2024-09-10T21:46:17.888 DEBUG:teuthology.orchestra.run.smithi007:> import sys 2024-09-10T21:46:17.888 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:46:17.888 DEBUG:teuthology.orchestra.run.smithi007:> try: 2024-09-10T21:46:17.888 DEBUG:teuthology.orchestra.run.smithi007:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/file_to_be_damaged'"'"') 2024-09-10T21:46:17.888 DEBUG:teuthology.orchestra.run.smithi007:> except OSError as e: 2024-09-10T21:46:17.888 DEBUG:teuthology.orchestra.run.smithi007:> sys.exit(e.errno) 2024-09-10T21:46:17.888 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:46:17.888 DEBUG:teuthology.orchestra.run.smithi007:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-09-10T21:46:17.889 DEBUG:teuthology.orchestra.run.smithi007:> print(json.dumps( 2024-09-10T21:46:17.889 DEBUG:teuthology.orchestra.run.smithi007:> dict([(a, getattr(s, a)) for a in attrs]), 2024-09-10T21:46:17.889 DEBUG:teuthology.orchestra.run.smithi007:> indent=2)) 2024-09-10T21:46:17.889 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:46:18.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-09-10T21:46:18.042 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-10T21:46:18.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:18.043 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-09-10T21:46:18.121 INFO:teuthology.orchestra.run.smithi007.stdout:2 2024-09-10T21:46:18.122 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:46:18.122 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:46:18.198 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:46:18.198 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:46:18.273 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:46:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.476+0000 7f7ff2c00640 1 -- 172.21.15.7:0/352981636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fd0003fd0 msgr2=0x7f7fd0004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.476+0000 7f7ff2c00640 1 --2- 172.21.15.7:0/352981636 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fd0003fd0 0x7f7fd0004800 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f7fec162b20 tx=0x7f7fd0005830 comp rx=0 tx=0).stop 2024-09-10T21:46:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.476+0000 7f7ff0c00640 1 -- 172.21.15.7:0/352981636 reap_dead start 2024-09-10T21:46:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.476+0000 7f7ff2c00640 1 -- 172.21.15.7:0/352981636 shutdown_connections 2024-09-10T21:46:18.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.476+0000 7f7ff2c00640 1 -- 172.21.15.7:0/352981636 >> 172.21.15.7:0/352981636 conn(0x7f7fec0b1660 msgr2=0x7f7fec0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.476+0000 7f7ff2c00640 1 -- 172.21.15.7:0/352981636 shutdown_connections 2024-09-10T21:46:18.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.476+0000 7f7ff2c00640 1 -- 172.21.15.7:0/352981636 wait complete. 2024-09-10T21:46:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.478+0000 7f7ff2c00640 1 Processor -- start 2024-09-10T21:46:18.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.478+0000 7f7ff2c00640 1 -- start start 2024-09-10T21:46:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.479+0000 7f7ff2c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fd0003fd0 0x7f7fec0c6be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:18.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.479+0000 7f7ff2c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fec0b2960 0x7f7fec161e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.479+0000 7f7ff2c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fec158260 0x7f7fec14ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.479+0000 7f7ff2c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec0adc80 con 0x7f7fec0b2960 2024-09-10T21:46:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.479+0000 7f7ff2c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7fec0adb00 con 0x7f7fec158260 2024-09-10T21:46:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.479+0000 7f7ff2c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f7fec15f230 con 0x7f7fd0003fd0 2024-09-10T21:46:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.479+0000 7f7fe3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fec0b2960 0x7f7fec161e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.479+0000 7f7fe3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fec0b2960 0x7f7fec161e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49454/0 (socket says 172.21.15.7:49454) 2024-09-10T21:46:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.479+0000 7f7fe3e00640 1 -- 172.21.15.7:0/2716734341 learned_addr learned my addr 172.21.15.7:0/2716734341 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.479+0000 7f7febe00640 1 --2- 172.21.15.7:0/2716734341 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fd0003fd0 0x7f7fec0c6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.479+0000 7f7ff0c00640 1 --2- 172.21.15.7:0/2716734341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fec158260 0x7f7fec14ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.479+0000 7f7fe3e00640 1 -- 172.21.15.7:0/2716734341 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fd0003fd0 msgr2=0x7f7fec0c6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.479+0000 7f7fe3e00640 1 --2- 172.21.15.7:0/2716734341 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fd0003fd0 0x7f7fec0c6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.479+0000 7f7fe3e00640 1 -- 172.21.15.7:0/2716734341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fec158260 msgr2=0x7f7fec14ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:18.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.479+0000 7f7fe3e00640 1 --2- 172.21.15.7:0/2716734341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fec158260 0x7f7fec14ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.479+0000 7f7fe3e00640 1 -- 172.21.15.7:0/2716734341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fec072d40 con 0x7f7fec0b2960 2024-09-10T21:46:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.480+0000 7f7fe3e00640 1 --2- 172.21.15.7:0/2716734341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fec0b2960 0x7f7fec161e60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4036040 tx=0x7f7fd4033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.480+0000 7f7fe8c00640 1 -- 172.21.15.7:0/2716734341 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4038040 con 0x7f7fec0b2960 2024-09-10T21:46:18.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.480+0000 7f7fe8c00640 1 -- 172.21.15.7:0/2716734341 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fd4033860 con 0x7f7fec0b2960 2024-09-10T21:46:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.480+0000 7f7ff2c00640 1 -- 172.21.15.7:0/2716734341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fec073020 con 0x7f7fec0b2960 2024-09-10T21:46:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.480+0000 7f7fe8c00640 1 -- 172.21.15.7:0/2716734341 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4033cb0 con 0x7f7fec0b2960 2024-09-10T21:46:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.480+0000 7f7ff2c00640 1 -- 172.21.15.7:0/2716734341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fec0c7120 con 0x7f7fec0b2960 2024-09-10T21:46:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.481+0000 7f7ff2c00640 1 -- 172.21.15.7:0/2716734341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fec0cc950 con 0x7f7fec0b2960 2024-09-10T21:46:18.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.482+0000 7f7fe8c00640 1 -- 172.21.15.7:0/2716734341 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7fd403b070 con 0x7f7fec0b2960 2024-09-10T21:46:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.483+0000 7f7fe8c00640 1 --2- 172.21.15.7:0/2716734341 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7fc8077f70 0x7f7fc807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:18.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.483+0000 7f7fe8c00640 1 -- 172.21.15.7:0/2716734341 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7f7fd40b9aa0 con 0x7f7fec0b2960 2024-09-10T21:46:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.484+0000 7f7febe00640 1 --2- 172.21.15.7:0/2716734341 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7fc8077f70 0x7f7fc807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:18.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.484+0000 7f7febe00640 1 --2- 172.21.15.7:0/2716734341 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7fc8077f70 0x7f7fc807a430 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc006c50 tx=0x7f7fdc0067a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:18.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.488+0000 7f7fe8c00640 1 -- 172.21.15.7:0/2716734341 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7fd404a020 con 0x7f7fec0b2960 2024-09-10T21:46:18.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.694+0000 7f7ff2c00640 1 -- 172.21.15.7:0/2716734341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7fec0747d0 con 0x7f7fec0b2960 2024-09-10T21:46:18.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.694+0000 7f7fe8c00640 1 -- 172.21.15.7:0/2716734341 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v71) ==== 80+0+124 (secure 0 0 0) 0x7f7fd4082530 con 0x7f7fec0b2960 2024-09-10T21:46:18.696 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.133:6836/2752881644 2024-09-11T21:46:10.658995+0000 2024-09-10T21:46:18.696 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.133:6834/2752881644 2024-09-11T21:46:10.658995+0000 2024-09-10T21:46:18.696 INFO:teuthology.orchestra.run.smithi007.stderr:listed 2 entries 2024-09-10T21:46:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.696+0000 7f7ff2c00640 1 -- 172.21.15.7:0/2716734341 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7fc8077f70 msgr2=0x7f7fc807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.696+0000 7f7ff2c00640 1 --2- 172.21.15.7:0/2716734341 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7fc8077f70 0x7f7fc807a430 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7fdc006c50 tx=0x7f7fdc0067a0 comp rx=0 tx=0).stop 2024-09-10T21:46:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.696+0000 7f7ff2c00640 1 -- 172.21.15.7:0/2716734341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fec0b2960 msgr2=0x7f7fec161e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.696+0000 7f7ff2c00640 1 --2- 172.21.15.7:0/2716734341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fec0b2960 0x7f7fec161e60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4036040 tx=0x7f7fd4033670 comp rx=0 tx=0).stop 2024-09-10T21:46:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.697+0000 7f7ff2c00640 1 -- 172.21.15.7:0/2716734341 shutdown_connections 2024-09-10T21:46:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.697+0000 7f7ff2c00640 1 --2- 172.21.15.7:0/2716734341 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7fc8077f70 0x7f7fc807a430 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.697+0000 7f7ff2c00640 1 --2- 172.21.15.7:0/2716734341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7fec158260 0x7f7fec14ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.697+0000 7f7ff2c00640 1 --2- 172.21.15.7:0/2716734341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7fec0b2960 0x7f7fec161e60 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:18.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.697+0000 7f7ff2c00640 1 --2- 172.21.15.7:0/2716734341 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7fd0003fd0 0x7f7fec0c6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:18.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.697+0000 7f7ff2c00640 1 -- 172.21.15.7:0/2716734341 >> 172.21.15.7:0/2716734341 conn(0x7f7fec0b1660 msgr2=0x7f7fec0cb370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:18.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.697+0000 7f7ff2c00640 1 -- 172.21.15.7:0/2716734341 shutdown_connections 2024-09-10T21:46:18.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:18.697+0000 7f7ff2c00640 1 -- 172.21.15.7:0/2716734341 wait complete. 2024-09-10T21:46:18.714 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:46:18.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:18.714 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:46:18.776 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:46:18.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:18.776 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:46:18.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:18.813 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir recursive 2024-09-10T21:46:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.016+0000 7f1cb9600640 1 -- 172.21.15.7:0/3630150081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb415edd0 msgr2=0x7f1cb406cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.016+0000 7f1cb9600640 1 --2- 172.21.15.7:0/3630150081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb415edd0 0x7f1cb406cb20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1cb406d890 tx=0x7f1c9c02a4d0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.016+0000 7f1cb9600640 1 -- 172.21.15.7:0/3630150081 shutdown_connections 2024-09-10T21:46:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.016+0000 7f1cb9600640 1 --2- 172.21.15.7:0/3630150081 >> v2:172.21.15.133:3300/0 conn(0x7f1cb406d060 0x7f1cb4073810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.016+0000 7f1cb9600640 1 --2- 172.21.15.7:0/3630150081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb415edd0 0x7f1cb406cb20 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.016+0000 7f1cb9600640 1 --2- 172.21.15.7:0/3630150081 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1cb4158300 0x7f1cb40b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.016+0000 7f1cb9600640 1 -- 172.21.15.7:0/3630150081 >> 172.21.15.7:0/3630150081 conn(0x7f1cb40b1c10 msgr2=0x7f1cb40b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.016+0000 7f1cb9600640 1 -- 172.21.15.7:0/3630150081 shutdown_connections 2024-09-10T21:46:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.016+0000 7f1cb9600640 1 -- 172.21.15.7:0/3630150081 wait complete. 2024-09-10T21:46:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.018+0000 7f1cb9600640 1 Processor -- start 2024-09-10T21:46:19.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.018+0000 7f1cb9600640 1 -- start start 2024-09-10T21:46:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.019+0000 7f1cb9600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb406d060 0x7f1cb41f4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.019+0000 7f1cb9600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb4158300 0x7f1cb409c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.019+0000 7f1cb9600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1cb415edd0 0x7f1cb4157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.019+0000 7f1cb9600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb40acf20 con 0x7f1cb4158300 2024-09-10T21:46:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.019+0000 7f1cb9600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb40acda0 con 0x7f1cb406d060 2024-09-10T21:46:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.019+0000 7f1cb9600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb4073d50 con 0x7f1cb415edd0 2024-09-10T21:46:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.019+0000 7f1caa000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb4158300 0x7f1cb409c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.019+0000 7f1caa000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb4158300 0x7f1cb409c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49476/0 (socket says 172.21.15.7:49476) 2024-09-10T21:46:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.019+0000 7f1caa000640 1 -- 172.21.15.7:0/3179662676 learned_addr learned my addr 172.21.15.7:0/3179662676 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.020+0000 7f1cb2a00640 1 --2- 172.21.15.7:0/3179662676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb406d060 0x7f1cb41f4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.020+0000 7f1cb3400640 1 --2- 172.21.15.7:0/3179662676 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1cb415edd0 0x7f1cb4157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.020+0000 7f1caa000640 1 -- 172.21.15.7:0/3179662676 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1cb415edd0 msgr2=0x7f1cb4157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.020+0000 7f1caa000640 1 --2- 172.21.15.7:0/3179662676 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1cb415edd0 0x7f1cb4157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.020+0000 7f1caa000640 1 -- 172.21.15.7:0/3179662676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb406d060 msgr2=0x7f1cb41f4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.020+0000 7f1caa000640 1 --2- 172.21.15.7:0/3179662676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb406d060 0x7f1cb41f4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.020+0000 7f1caa000640 1 -- 172.21.15.7:0/3179662676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cb409c920 con 0x7f1cb4158300 2024-09-10T21:46:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.020+0000 7f1caa000640 1 --2- 172.21.15.7:0/3179662676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb4158300 0x7f1cb409c3e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0099c0 tx=0x7f1c9c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.020+0000 7f1cab400640 1 -- 172.21.15.7:0/3179662676 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c03e030 con 0x7f1cb4158300 2024-09-10T21:46:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.020+0000 7f1cab400640 1 -- 172.21.15.7:0/3179662676 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c9c0047a0 con 0x7f1cb4158300 2024-09-10T21:46:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.020+0000 7f1cb9600640 1 -- 172.21.15.7:0/3179662676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cb409c0a0 con 0x7f1cb4158300 2024-09-10T21:46:19.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.020+0000 7f1cab400640 1 -- 172.21.15.7:0/3179662676 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c035040 con 0x7f1cb4158300 2024-09-10T21:46:19.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.021+0000 7f1cb9600640 1 -- 172.21.15.7:0/3179662676 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cb41594a0 con 0x7f1cb4158300 2024-09-10T21:46:19.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.023+0000 7f1cab400640 1 -- 172.21.15.7:0/3179662676 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1c9c004bf0 con 0x7f1cb4158300 2024-09-10T21:46:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.024+0000 7f1cab400640 1 --2- 172.21.15.7:0/3179662676 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1c90077f70 0x7f1c9007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.024+0000 7f1cab400640 1 -- 172.21.15.7:0/3179662676 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7f1c9c0ba890 con 0x7f1cb4158300 2024-09-10T21:46:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.024+0000 7f1cb2a00640 1 --2- 172.21.15.7:0/3179662676 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1c90077f70 0x7f1c9007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.024+0000 7f1cbc49f740 1 Processor -- start 2024-09-10T21:46:19.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.024+0000 7f1cbc49f740 1 -- start start 2024-09-10T21:46:19.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.025+0000 7f1cb2a00640 1 --2- 172.21.15.7:0/3179662676 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1c90077f70 0x7f1c9007a430 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400cac0 tx=0x7f1ca400c4a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:19.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.025+0000 7f1cbc49f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb678b50 0x559cfb678f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.025+0000 7f1cbc49f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb633b70 0x559cfb6775e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.025+0000 7f1cbc49f740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb659fc0 0x559cfb677b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.025+0000 7f1cbc49f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559cfb5a5970 con 0x559cfb633b70 2024-09-10T21:46:19.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.025+0000 7f1cbc49f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559cfb594da0 con 0x559cfb678b50 2024-09-10T21:46:19.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.025+0000 7f1cbc49f740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x559cfb598bb0 con 0x559cfb659fc0 2024-09-10T21:46:19.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.026+0000 7f1caa000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb633b70 0x559cfb6775e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.026+0000 7f1cb2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb678b50 0x559cfb678f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.026+0000 7f1cb2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb678b50 0x559cfb678f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:39956/0 (socket says 172.21.15.7:39956) 2024-09-10T21:46:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.026+0000 7f1caa000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb633b70 0x559cfb6775e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49480/0 (socket says 172.21.15.7:49480) 2024-09-10T21:46:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.026+0000 7f1cb2a00640 1 -- 172.21.15.7:0/3037605039 learned_addr learned my addr 172.21.15.7:0/3037605039 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.026+0000 7f1cb3400640 1 --2- 172.21.15.7:0/3037605039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb659fc0 0x559cfb677b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.026+0000 7f1caa000640 1 -- 172.21.15.7:0/3037605039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb659fc0 msgr2=0x559cfb677b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:19.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.026+0000 7f1caa000640 1 --2- 172.21.15.7:0/3037605039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb659fc0 0x559cfb677b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.026+0000 7f1caa000640 1 -- 172.21.15.7:0/3037605039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb678b50 msgr2=0x559cfb678f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.026+0000 7f1caa000640 1 --2- 172.21.15.7:0/3037605039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb678b50 0x559cfb678f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.026+0000 7f1caa000640 1 -- 172.21.15.7:0/3037605039 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cfb36fa20 con 0x559cfb633b70 2024-09-10T21:46:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.026+0000 7f1caa000640 1 --2- 172.21.15.7:0/3037605039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb633b70 0x559cfb6775e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c082b20 tx=0x7f1c9c082a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d92213b7367c0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.026+0000 7f1ca3000640 1 -- 172.21.15.7:0/3037605039 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c048040 con 0x559cfb633b70 2024-09-10T21:46:19.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.027+0000 7f1ca3000640 1 -- 172.21.15.7:0/3037605039 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c9c087070 con 0x559cfb633b70 2024-09-10T21:46:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.027+0000 7f1ca3000640 1 -- 172.21.15.7:0/3037605039 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c083290 con 0x559cfb633b70 2024-09-10T21:46:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.027+0000 7f1cbc49f740 1 -- 172.21.15.7:0/3037605039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb633b70 msgr2=0x559cfb6775e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.027+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/3037605039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb633b70 0x559cfb6775e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c082b20 tx=0x7f1c9c082a20 comp rx=0 tx=0).stop 2024-09-10T21:46:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.027+0000 7f1cbc49f740 1 -- 172.21.15.7:0/3037605039 shutdown_connections 2024-09-10T21:46:19.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.027+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/3037605039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb659fc0 0x559cfb677b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.027+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/3037605039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb633b70 0x559cfb6775e0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.027+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/3037605039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb678b50 0x559cfb678f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.027+0000 7f1cbc49f740 1 -- 172.21.15.7:0/3037605039 >> 172.21.15.7:0/3037605039 conn(0x559cfb62f0a0 msgr2=0x559cfb632ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:19.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.027+0000 7f1cbc49f740 1 -- 172.21.15.7:0/3037605039 shutdown_connections 2024-09-10T21:46:19.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.027+0000 7f1cbc49f740 1 -- 172.21.15.7:0/3037605039 wait complete. 2024-09-10T21:46:19.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.028+0000 7f1cbc49f740 1 -- start start 2024-09-10T21:46:19.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.028+0000 7f1cbc49f740 1 Processor -- start 2024-09-10T21:46:19.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.029+0000 7f1cbc49f740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb659fc0 0x559cfb670ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.029+0000 7f1cbc49f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb678b50 0x559cfb656ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.029+0000 7f1cbc49f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb6734c0 0x559cfb673900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.029+0000 7f1cbc49f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559cfb305960 con 0x559cfb6734c0 2024-09-10T21:46:19.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.029+0000 7f1cbc49f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559cfb53a780 con 0x559cfb678b50 2024-09-10T21:46:19.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.029+0000 7f1cbc49f740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x559cfb594da0 con 0x559cfb659fc0 2024-09-10T21:46:19.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.029+0000 7f1cbc49f740 10 client.0 _ll_register_callbacks cb 0x559cfb5914b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:46:19.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.029+0000 7f1cb3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb6734c0 0x559cfb673900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.030+0000 7f1cb3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb6734c0 0x559cfb673900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49496/0 (socket says 172.21.15.7:49496) 2024-09-10T21:46:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.030+0000 7f1cb3400640 1 -- 172.21.15.7:0/1896642380 learned_addr learned my addr 172.21.15.7:0/1896642380 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.030+0000 7f1caa000640 1 --2- 172.21.15.7:0/1896642380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb678b50 0x559cfb656ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.030+0000 7f1cb2a00640 1 --2- 172.21.15.7:0/1896642380 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb659fc0 0x559cfb670ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.030+0000 7f1cb3400640 1 -- 172.21.15.7:0/1896642380 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb659fc0 msgr2=0x559cfb670ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:19.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.030+0000 7f1cb3400640 1 --2- 172.21.15.7:0/1896642380 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb659fc0 0x559cfb670ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.030+0000 7f1cb3400640 1 -- 172.21.15.7:0/1896642380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb678b50 msgr2=0x559cfb656ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:19.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.030+0000 7f1cb3400640 1 --2- 172.21.15.7:0/1896642380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb678b50 0x559cfb656ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.030+0000 7f1cb3400640 1 -- 172.21.15.7:0/1896642380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cfb2d48c0 con 0x559cfb6734c0 2024-09-10T21:46:19.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.030+0000 7f1cb3400640 1 --2- 172.21.15.7:0/1896642380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb6734c0 0x559cfb673900 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0129e0 tx=0x7f1cac012ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a6c49ab0d02365f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:19.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.030+0000 7f1c97400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:46:19.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.030+0000 7f1c97400640 1 -- 172.21.15.7:0/1896642380 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac01f060 con 0x559cfb6734c0 2024-09-10T21:46:19.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.031+0000 7f1c97400640 1 -- 172.21.15.7:0/1896642380 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7f1cac00cd90 con 0x559cfb6734c0 2024-09-10T21:46:19.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.031+0000 7f1cbc49f740 1 -- 172.21.15.7:0/1896642380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c9c087070 con 0x559cfb6734c0 2024-09-10T21:46:19.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.031+0000 7f1c97400640 1 -- 172.21.15.7:0/1896642380 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f1cac025050 con 0x559cfb6734c0 2024-09-10T21:46:19.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.031+0000 7f1cbc49f740 10 client.6414 fetch_fsmap learned FSMap version 52 2024-09-10T21:46:19.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.031+0000 7f1cbc49f740 1 -- 172.21.15.7:0/1896642380 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x559cfb3d01b0 con 0x559cfb6734c0 2024-09-10T21:46:19.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.031+0000 7f1c97400640 1 -- 172.21.15.7:0/1896642380 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c9c087070 con 0x559cfb6734c0 2024-09-10T21:46:19.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.031+0000 7f1c97400640 1 -- 172.21.15.7:0/1896642380 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7f1cac019030 con 0x559cfb6734c0 2024-09-10T21:46:19.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.031+0000 7f1cbc49f740 10 client.6414 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T21:46:19.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.031+0000 7f1cbc49f740 10 client.6414 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-10T21:46:19.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.031+0000 7f1cbc49f740 20 client.6414 populate_metadata read hostname 'smithi007' 2024-09-10T21:46:19.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.031+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/1896642380 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x559cfb66ec00 0x559cfb66f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.031+0000 7f1cbc49f740 4 client.6414 mds_command: new command op to 235490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:46:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.031+0000 7f1cbc49f740 1 -- 172.21.15.7:0/1896642380 --> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559cfb5a78c0 con 0x559cfb66ec00 2024-09-10T21:46:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.032+0000 7f1cb2a00640 1 --2- 172.21.15.7:0/1896642380 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x559cfb66ec00 0x559cfb66f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.032+0000 7f1cb2a00640 1 --2- 172.21.15.7:0/1896642380 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x559cfb66ec00 0x559cfb66f010 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a84a09e8889f35b server_cookie=f847c545feb5a90b in_seq=0 out_seq=0 2024-09-10T21:46:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.032+0000 7f1c97400640 10 client.6414 ms_handle_connect on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:19.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.033+0000 7f1c97400640 1 -- 172.21.15.7:0/1896642380 <== mds.0 v2:172.21.15.133:6834/2622530239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559cfb5a78c0 con 0x559cfb66ec00 2024-09-10T21:46:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.033+0000 7f1c97400640 10 client.6414 handle_command_reply: tid=0 2024-09-10T21:46:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.033+0000 7f1cbc49f740 1 client.6414 shutdown 2024-09-10T21:46:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.033+0000 7f1cbc49f740 1 -- 172.21.15.7:0/1896642380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb6734c0 msgr2=0x559cfb673900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.033+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/1896642380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb6734c0 0x559cfb673900 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0129e0 tx=0x7f1cac012ea0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.033+0000 7f1cbc49f740 1 -- 172.21.15.7:0/1896642380 shutdown_connections 2024-09-10T21:46:19.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.033+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/1896642380 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb678b50 0x559cfb656ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.033+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/1896642380 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x559cfb66ec00 0x559cfb66f010 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.033+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/1896642380 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb6734c0 0x559cfb673900 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.033+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/1896642380 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb659fc0 0x559cfb670ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.033+0000 7f1c97400640 0 client.6414 ms_handle_reset on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:19.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.034+0000 7f1cbc49f740 1 -- 172.21.15.7:0/1896642380 >> 172.21.15.7:0/1896642380 conn(0x559cfb633b70 msgr2=0x559cfb630520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:19.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.034+0000 7f1cbc49f740 1 -- 172.21.15.7:0/1896642380 shutdown_connections 2024-09-10T21:46:19.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.034+0000 7f1cbc49f740 1 -- 172.21.15.7:0/1896642380 wait complete. 2024-09-10T21:46:19.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.034+0000 7f1cbc49f740 20 client.6414 trim_cache size 0 max 16384 2024-09-10T21:46:19.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.053+0000 7f1cbc49f740 1 Processor -- start 2024-09-10T21:46:19.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.053+0000 7f1cbc49f740 1 -- start start 2024-09-10T21:46:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.053+0000 7f1cbc49f740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb659fc0 0x559cfb66faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.053+0000 7f1cbc49f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb6734c0 0x559cfb671f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.053+0000 7f1cbc49f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb678b50 0x559cfb62e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.053+0000 7f1cbc49f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb4201080 con 0x559cfb6734c0 2024-09-10T21:46:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.053+0000 7f1cbc49f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb42000c0 con 0x559cfb678b50 2024-09-10T21:46:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.053+0000 7f1cbc49f740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f1cb409b900 con 0x559cfb659fc0 2024-09-10T21:46:19.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.054+0000 7f1caa000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb6734c0 0x559cfb671f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.054+0000 7f1caa000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb6734c0 0x559cfb671f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49500/0 (socket says 172.21.15.7:49500) 2024-09-10T21:46:19.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.054+0000 7f1caa000640 1 -- 172.21.15.7:0/680743942 learned_addr learned my addr 172.21.15.7:0/680743942 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:19.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.054+0000 7f1caa000640 1 -- 172.21.15.7:0/680743942 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb659fc0 msgr2=0x559cfb66faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:19.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.054+0000 7f1cb2a00640 1 --2- 172.21.15.7:0/680743942 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb659fc0 0x559cfb66faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.054+0000 7f1caa000640 1 --2- 172.21.15.7:0/680743942 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb659fc0 0x559cfb66faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.054+0000 7f1cb3400640 1 --2- 172.21.15.7:0/680743942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb678b50 0x559cfb62e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.054+0000 7f1caa000640 1 -- 172.21.15.7:0/680743942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb678b50 msgr2=0x559cfb62e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:19.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.054+0000 7f1caa000640 1 --2- 172.21.15.7:0/680743942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb678b50 0x559cfb62e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.054+0000 7f1caa000640 1 -- 172.21.15.7:0/680743942 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cfb66f380 con 0x559cfb6734c0 2024-09-10T21:46:19.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.054+0000 7f1cb2a00640 1 --2- 172.21.15.7:0/680743942 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb659fc0 0x559cfb66faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:46:19.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.054+0000 7f1cb3400640 1 --2- 172.21.15.7:0/680743942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb678b50 0x559cfb62e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:46:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.054+0000 7f1caa000640 1 --2- 172.21.15.7:0/680743942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb6734c0 0x559cfb671f00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c082b20 tx=0x7f1c9c0a6040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0db55637c1baded server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.055+0000 7f1ca2600640 1 -- 172.21.15.7:0/680743942 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c0b1e60 con 0x559cfb6734c0 2024-09-10T21:46:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.055+0000 7f1ca2600640 1 -- 172.21.15.7:0/680743942 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c9c06e030 con 0x559cfb6734c0 2024-09-10T21:46:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.055+0000 7f1ca2600640 1 -- 172.21.15.7:0/680743942 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c048030 con 0x559cfb6734c0 2024-09-10T21:46:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.055+0000 7f1cbc49f740 1 -- 172.21.15.7:0/680743942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb6734c0 msgr2=0x559cfb671f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.055+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/680743942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb6734c0 0x559cfb671f00 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c082b20 tx=0x7f1c9c0a6040 comp rx=0 tx=0).stop 2024-09-10T21:46:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.055+0000 7f1cbc49f740 1 -- 172.21.15.7:0/680743942 shutdown_connections 2024-09-10T21:46:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.055+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/680743942 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb678b50 0x559cfb62e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.055+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/680743942 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb6734c0 0x559cfb671f00 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.055+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/680743942 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb659fc0 0x559cfb66faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.055+0000 7f1cbc49f740 1 -- 172.21.15.7:0/680743942 >> 172.21.15.7:0/680743942 conn(0x559cfb633b70 msgr2=0x559cfb628660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:19.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.055+0000 7f1cbc49f740 1 -- 172.21.15.7:0/680743942 shutdown_connections 2024-09-10T21:46:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.055+0000 7f1cbc49f740 1 -- 172.21.15.7:0/680743942 wait complete. 2024-09-10T21:46:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.056+0000 7f1cbc49f740 1 -- start start 2024-09-10T21:46:19.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.056+0000 7f1cbc49f740 1 Processor -- start 2024-09-10T21:46:19.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.057+0000 7f1cbc49f740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb659fc0 0x559cfb6f8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.057+0000 7f1cbc49f740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb6734c0 0x559cfb6890f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.057+0000 7f1cbc49f740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb678b50 0x559cfb69b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.057+0000 7f1cbc49f740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559cfb5a5970 con 0x559cfb678b50 2024-09-10T21:46:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.057+0000 7f1cbc49f740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1cb41f86e0 con 0x559cfb659fc0 2024-09-10T21:46:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.057+0000 7f1cbc49f740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x559cfb670030 con 0x559cfb6734c0 2024-09-10T21:46:19.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.057+0000 7f1cbc49f740 10 client.0 _ll_register_callbacks cb 0x7f1cac009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:46:19.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.057+0000 7f1cb3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb678b50 0x559cfb69b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.057+0000 7f1cb3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb678b50 0x559cfb69b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49516/0 (socket says 172.21.15.7:49516) 2024-09-10T21:46:19.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.057+0000 7f1cb3400640 1 -- 172.21.15.7:0/1871793339 learned_addr learned my addr 172.21.15.7:0/1871793339 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:19.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.057+0000 7f1cb2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb659fc0 0x559cfb6f8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.057+0000 7f1caa000640 1 --2- 172.21.15.7:0/1871793339 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb6734c0 0x559cfb6890f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.058+0000 7f1cb3400640 1 -- 172.21.15.7:0/1871793339 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb6734c0 msgr2=0x559cfb6890f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:19.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.058+0000 7f1cb3400640 1 --2- 172.21.15.7:0/1871793339 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb6734c0 0x559cfb6890f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.058+0000 7f1cb3400640 1 -- 172.21.15.7:0/1871793339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb659fc0 msgr2=0x559cfb6f8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.058+0000 7f1cb3400640 1 --2- 172.21.15.7:0/1871793339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb659fc0 0x559cfb6f8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.058+0000 7f1cb3400640 1 -- 172.21.15.7:0/1871793339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cfb36dd20 con 0x559cfb678b50 2024-09-10T21:46:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.058+0000 7f1cb3400640 1 --2- 172.21.15.7:0/1871793339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb678b50 0x559cfb69b090 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00cef0 tx=0x7f1cac00cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6f5d0e72cf8a8e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.058+0000 7f1c97400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:46:19.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.058+0000 7f1c97400640 1 -- 172.21.15.7:0/1871793339 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cac004e20 con 0x559cfb678b50 2024-09-10T21:46:19.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.058+0000 7f1c97400640 1 -- 172.21.15.7:0/1871793339 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7f1cac01f880 con 0x559cfb678b50 2024-09-10T21:46:19.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.058+0000 7f1cbc49f740 1 -- 172.21.15.7:0/1871793339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c9c06e030 con 0x559cfb678b50 2024-09-10T21:46:19.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.059+0000 7f1c97400640 1 -- 172.21.15.7:0/1871793339 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7f1cac01fa80 con 0x559cfb678b50 2024-09-10T21:46:19.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.059+0000 7f1cbc49f740 10 client.6420 fetch_fsmap learned FSMap version 52 2024-09-10T21:46:19.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.059+0000 7f1cbc49f740 1 -- 172.21.15.7:0/1871793339 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x559cfb672080 con 0x559cfb678b50 2024-09-10T21:46:19.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.059+0000 7f1c97400640 1 -- 172.21.15.7:0/1871793339 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c9c06e030 con 0x559cfb678b50 2024-09-10T21:46:19.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.059+0000 7f1c97400640 1 -- 172.21.15.7:0/1871793339 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7f1cac019070 con 0x559cfb678b50 2024-09-10T21:46:19.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.059+0000 7f1cbc49f740 10 client.6420 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T21:46:19.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.059+0000 7f1cbc49f740 10 client.6420 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-10T21:46:19.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.059+0000 7f1cbc49f740 20 client.6420 populate_metadata read hostname 'smithi007' 2024-09-10T21:46:19.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.059+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/1871793339 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x559cfb657b50 0x559cfb69b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.059+0000 7f1cbc49f740 4 client.6420 mds_command: new command op to 235490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}] 2024-09-10T21:46:19.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.059+0000 7f1cbc49f740 1 -- 172.21.15.7:0/1871793339 --> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["recursive"]}) -- 0x559cfb5a3370 con 0x559cfb657b50 2024-09-10T21:46:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.059+0000 7f1cb2a00640 1 --2- 172.21.15.7:0/1871793339 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x559cfb657b50 0x559cfb69b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.060+0000 7f1cb2a00640 1 --2- 172.21.15.7:0/1871793339 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x559cfb657b50 0x559cfb69b5d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff0cabf935ae3a60 server_cookie=1f790dac0aff254 in_seq=0 out_seq=0 2024-09-10T21:46:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.060+0000 7f1c97400640 10 client.6420 ms_handle_connect on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.061+0000 7f1c97400640 1 -- 172.21.15.7:0/1871793339 <== mds.0 v2:172.21.15.133:6834/2622530239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559cfb5a3370 con 0x559cfb657b50 2024-09-10T21:46:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.061+0000 7f1c97400640 10 client.6420 handle_command_reply: tid=0 2024-09-10T21:46:19.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.061+0000 7f1cbc49f740 1 client.6420 shutdown 2024-09-10T21:46:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.061+0000 7f1cbc49f740 1 -- 172.21.15.7:0/1871793339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb678b50 msgr2=0x559cfb69b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.061+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/1871793339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb678b50 0x559cfb69b090 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f1cac00cef0 tx=0x7f1cac00cf20 comp rx=0 tx=0).stop 2024-09-10T21:46:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.061+0000 7f1cbc49f740 1 -- 172.21.15.7:0/1871793339 shutdown_connections 2024-09-10T21:46:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.061+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/1871793339 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x559cfb657b50 0x559cfb69b5d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.061+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/1871793339 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559cfb678b50 0x559cfb69b090 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.061+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/1871793339 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559cfb6734c0 0x559cfb6890f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.061+0000 7f1cbc49f740 1 --2- 172.21.15.7:0/1871793339 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559cfb659fc0 0x559cfb6f8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.061+0000 7f1c97400640 0 client.6420 ms_handle_reset on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.061+0000 7f1cbc49f740 1 -- 172.21.15.7:0/1871793339 >> 172.21.15.7:0/1871793339 conn(0x559cfb633b70 msgr2=0x559cfb62d5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.061+0000 7f1cbc49f740 1 -- 172.21.15.7:0/1871793339 shutdown_connections 2024-09-10T21:46:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.061+0000 7f1cbc49f740 1 -- 172.21.15.7:0/1871793339 wait complete. 2024-09-10T21:46:19.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.062+0000 7f1cbc49f740 20 client.6420 trim_cache size 0 max 16384 2024-09-10T21:46:19.073 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:46:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-09-10T21:46:19.073 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "d158ff07-06ba-4bd2-9cbd-cc6aad170cea", 2024-09-10T21:46:19.074 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-09-10T21:46:19.074 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:46:19.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.062+0000 7f1cb9600640 1 -- 172.21.15.7:0/3179662676 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1c90077f70 msgr2=0x7f1c9007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:19.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.062+0000 7f1cb9600640 1 --2- 172.21.15.7:0/3179662676 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1c90077f70 0x7f1c9007a430 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1ca400cac0 tx=0x7f1ca400c4a0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.062+0000 7f1cb9600640 1 -- 172.21.15.7:0/3179662676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb4158300 msgr2=0x7f1cb409c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:19.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.062+0000 7f1cb9600640 1 --2- 172.21.15.7:0/3179662676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb4158300 0x7f1cb409c3e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c0099c0 tx=0x7f1c9c0039e0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.062+0000 7f1cb9600640 1 -- 172.21.15.7:0/3179662676 shutdown_connections 2024-09-10T21:46:19.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.062+0000 7f1cb9600640 1 --2- 172.21.15.7:0/3179662676 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1c90077f70 0x7f1c9007a430 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.062+0000 7f1cb9600640 1 --2- 172.21.15.7:0/3179662676 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1cb415edd0 0x7f1cb4157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.062+0000 7f1cb9600640 1 --2- 172.21.15.7:0/3179662676 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1cb4158300 0x7f1cb409c3e0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.062+0000 7f1cb9600640 1 --2- 172.21.15.7:0/3179662676 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1cb406d060 0x7f1cb41f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.062+0000 7f1cb9600640 1 -- 172.21.15.7:0/3179662676 >> 172.21.15.7:0/3179662676 conn(0x7f1cb40b1c10 msgr2=0x7f1cb40afeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:19.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.062+0000 7f1cb9600640 1 -- 172.21.15.7:0/3179662676 shutdown_connections 2024-09-10T21:46:19.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.062+0000 7f1cb9600640 1 -- 172.21.15.7:0/3179662676 wait complete. 2024-09-10T21:46:19.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:19.079 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-09-10T21:46:19.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.240+0000 7fda08c00640 1 -- 172.21.15.7:0/989097765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0406fcf0 msgr2=0x7fda04073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:19.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.240+0000 7fda08c00640 1 --2- 172.21.15.7:0/989097765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0406fcf0 0x7fda04073d40 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fda040708e0 tx=0x7fd9ec02a530 comp rx=0 tx=0).stop 2024-09-10T21:46:19.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.240+0000 7fda08c00640 1 -- 172.21.15.7:0/989097765 shutdown_connections 2024-09-10T21:46:19.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.240+0000 7fda08c00640 1 --2- 172.21.15.7:0/989097765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda0406fcf0 0x7fda04073d40 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.240+0000 7fda08c00640 1 --2- 172.21.15.7:0/989097765 >> v2:172.21.15.133:3300/0 conn(0x7fda04155fb0 0x7fda041600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.240+0000 7fda08c00640 1 -- 172.21.15.7:0/989097765 >> 172.21.15.7:0/989097765 conn(0x7fda040b16d0 msgr2=0x7fda040b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:19.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.240+0000 7fda08c00640 1 -- 172.21.15.7:0/989097765 shutdown_connections 2024-09-10T21:46:19.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.240+0000 7fda08c00640 1 -- 172.21.15.7:0/989097765 wait complete. 2024-09-10T21:46:19.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.242+0000 7fda08c00640 1 Processor -- start 2024-09-10T21:46:19.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.242+0000 7fda08c00640 1 -- start start 2024-09-10T21:46:19.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.243+0000 7fda08c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda0406fcf0 0x7fda040b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.243+0000 7fda08c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04074280 0x7fda04152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.243+0000 7fda08c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda04155fb0 0x7fda040b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.243+0000 7fda08c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fda04156510 con 0x7fda04074280 2024-09-10T21:46:19.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.243+0000 7fda08c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fda04156390 con 0x7fda04155fb0 2024-09-10T21:46:19.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.243+0000 7fda08c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fda0406ec90 con 0x7fda0406fcf0 2024-09-10T21:46:19.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.243+0000 7fd9f9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04074280 0x7fda04152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.243+0000 7fd9f9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04074280 0x7fda04152290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49534/0 (socket says 172.21.15.7:49534) 2024-09-10T21:46:19.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.243+0000 7fd9f9400640 1 -- 172.21.15.7:0/2444687424 learned_addr learned my addr 172.21.15.7:0/2444687424 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:19.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.244+0000 7fda01e00640 1 --2- 172.21.15.7:0/2444687424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda0406fcf0 0x7fda040b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.244+0000 7fda02800640 1 --2- 172.21.15.7:0/2444687424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda04155fb0 0x7fda040b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.244+0000 7fd9f9400640 1 -- 172.21.15.7:0/2444687424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda0406fcf0 msgr2=0x7fda040b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:19.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.244+0000 7fd9f9400640 1 --2- 172.21.15.7:0/2444687424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda0406fcf0 0x7fda040b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.244+0000 7fd9f9400640 1 -- 172.21.15.7:0/2444687424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda04155fb0 msgr2=0x7fda040b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.244+0000 7fd9f9400640 1 --2- 172.21.15.7:0/2444687424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda04155fb0 0x7fda040b4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.244+0000 7fd9f9400640 1 -- 172.21.15.7:0/2444687424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda040b59a0 con 0x7fda04074280 2024-09-10T21:46:19.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.244+0000 7fd9f9400640 1 --2- 172.21.15.7:0/2444687424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04074280 0x7fda04152290 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec02ab20 tx=0x7fd9ec032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:19.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.244+0000 7fd9faa00640 1 -- 172.21.15.7:0/2444687424 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec040070 con 0x7fda04074280 2024-09-10T21:46:19.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.244+0000 7fd9faa00640 1 -- 172.21.15.7:0/2444687424 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9ec005310 con 0x7fda04074280 2024-09-10T21:46:19.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.244+0000 7fd9faa00640 1 -- 172.21.15.7:0/2444687424 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec0047a0 con 0x7fda04074280 2024-09-10T21:46:19.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.244+0000 7fda08c00640 1 -- 172.21.15.7:0/2444687424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda040c6d20 con 0x7fda04074280 2024-09-10T21:46:19.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.245+0000 7fda08c00640 1 -- 172.21.15.7:0/2444687424 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda040b56c0 con 0x7fda04074280 2024-09-10T21:46:19.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.247+0000 7fd9faa00640 1 -- 172.21.15.7:0/2444687424 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd9ec04c030 con 0x7fda04074280 2024-09-10T21:46:19.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.248+0000 7fd9faa00640 1 --2- 172.21.15.7:0/2444687424 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd9e0077f70 0x7fd9e007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.248+0000 7fd9faa00640 1 -- 172.21.15.7:0/2444687424 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7fd9ec0bb1c0 con 0x7fda04074280 2024-09-10T21:46:19.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.248+0000 7fda01e00640 1 --2- 172.21.15.7:0/2444687424 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd9e0077f70 0x7fd9e007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.248+0000 7fda0bc40740 1 Processor -- start 2024-09-10T21:46:19.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.248+0000 7fda0bc40740 1 -- start start 2024-09-10T21:46:19.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.249+0000 7fda01e00640 1 --2- 172.21.15.7:0/2444687424 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd9e0077f70 0x7fd9e007a430 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4005fd0 tx=0x7fd9f4005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:19.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.249+0000 7fda0bc40740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028c8130 0x55e1028c8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.249+0000 7fda0bc40740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028832a0 0x55e1028c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.249+0000 7fda0bc40740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e102883820 0x55e1028c7150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.249+0000 7fda0bc40740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e1027e21c0 con 0x55e1028c8130 2024-09-10T21:46:19.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.249+0000 7fda0bc40740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e1027f5870 con 0x55e1028832a0 2024-09-10T21:46:19.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.249+0000 7fda0bc40740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55e1027e4ca0 con 0x55e102883820 2024-09-10T21:46:19.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.249+0000 7fda01e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028c8130 0x55e1028c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.250+0000 7fda01e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028c8130 0x55e1028c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49550/0 (socket says 172.21.15.7:49550) 2024-09-10T21:46:19.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.250+0000 7fda01e00640 1 -- 172.21.15.7:0/3454104940 learned_addr learned my addr 172.21.15.7:0/3454104940 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:19.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.250+0000 7fda02800640 1 --2- 172.21.15.7:0/3454104940 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e102883820 0x55e1028c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.250+0000 7fd9f9400640 1 --2- 172.21.15.7:0/3454104940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028832a0 0x55e1028c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.250+0000 7fda01e00640 1 -- 172.21.15.7:0/3454104940 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e102883820 msgr2=0x55e1028c7150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:19.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.250+0000 7fda01e00640 1 --2- 172.21.15.7:0/3454104940 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e102883820 0x55e1028c7150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.250+0000 7fda01e00640 1 -- 172.21.15.7:0/3454104940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028832a0 msgr2=0x55e1028c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:19.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.250+0000 7fda01e00640 1 --2- 172.21.15.7:0/3454104940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028832a0 0x55e1028c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.250+0000 7fda01e00640 1 -- 172.21.15.7:0/3454104940 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1025bd7d0 con 0x55e1028c8130 2024-09-10T21:46:19.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.250+0000 7fd9f9400640 1 --2- 172.21.15.7:0/3454104940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028832a0 0x55e1028c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:46:19.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.250+0000 7fda01e00640 1 --2- 172.21.15.7:0/3454104940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028c8130 0x55e1028c8510 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400fdd0 tx=0x7fd9f4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e9697a66729573 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:19.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.250+0000 7fd9f2600640 1 -- 172.21.15.7:0/3454104940 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4022070 con 0x55e1028c8130 2024-09-10T21:46:19.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.251+0000 7fd9f2600640 1 -- 172.21.15.7:0/3454104940 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9f401e450 con 0x55e1028c8130 2024-09-10T21:46:19.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.251+0000 7fd9f2600640 1 -- 172.21.15.7:0/3454104940 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f401e860 con 0x55e1028c8130 2024-09-10T21:46:19.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.251+0000 7fda0bc40740 1 -- 172.21.15.7:0/3454104940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028c8130 msgr2=0x55e1028c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:19.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.251+0000 7fda0bc40740 1 --2- 172.21.15.7:0/3454104940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028c8130 0x55e1028c8510 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400fdd0 tx=0x7fd9f4012000 comp rx=0 tx=0).stop 2024-09-10T21:46:19.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.251+0000 7fda0bc40740 1 -- 172.21.15.7:0/3454104940 shutdown_connections 2024-09-10T21:46:19.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.251+0000 7fda0bc40740 1 --2- 172.21.15.7:0/3454104940 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028832a0 0x55e1028c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.251+0000 7fda0bc40740 1 --2- 172.21.15.7:0/3454104940 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e102883820 0x55e1028c7150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.251+0000 7fda0bc40740 1 --2- 172.21.15.7:0/3454104940 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028c8130 0x55e1028c8510 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.251+0000 7fda0bc40740 1 -- 172.21.15.7:0/3454104940 >> 172.21.15.7:0/3454104940 conn(0x55e10287e870 msgr2=0x55e102882290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:19.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.251+0000 7fda0bc40740 1 -- 172.21.15.7:0/3454104940 shutdown_connections 2024-09-10T21:46:19.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.251+0000 7fda0bc40740 1 -- 172.21.15.7:0/3454104940 wait complete. 2024-09-10T21:46:19.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.252+0000 7fda0bc40740 1 -- start start 2024-09-10T21:46:19.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.252+0000 7fda0bc40740 1 Processor -- start 2024-09-10T21:46:19.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.253+0000 7fda0bc40740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e102883820 0x55e1028a6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.253+0000 7fda0bc40740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028c8130 0x55e1028c2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.253+0000 7fda0bc40740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028ac890 0x55e1028c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.253+0000 7fda0bc40740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e10278b410 con 0x55e1028ac890 2024-09-10T21:46:19.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.253+0000 7fda0bc40740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e1027e8ab0 con 0x55e1028c8130 2024-09-10T21:46:19.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.253+0000 7fda0bc40740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55e102555740 con 0x55e102883820 2024-09-10T21:46:19.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.253+0000 7fda0bc40740 10 client.0 _ll_register_callbacks cb 0x55e102768360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:46:19.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.253+0000 7fda02800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028ac890 0x55e1028c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.254+0000 7fda01e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e102883820 0x55e1028a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.254+0000 7fda02800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028ac890 0x55e1028c3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49566/0 (socket says 172.21.15.7:49566) 2024-09-10T21:46:19.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.254+0000 7fda02800640 1 -- 172.21.15.7:0/84424747 learned_addr learned my addr 172.21.15.7:0/84424747 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:19.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.254+0000 7fd9f9400640 1 --2- 172.21.15.7:0/84424747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028c8130 0x55e1028c2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.254+0000 7fda02800640 1 -- 172.21.15.7:0/84424747 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e102883820 msgr2=0x55e1028a6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:19.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.254+0000 7fda02800640 1 --2- 172.21.15.7:0/84424747 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e102883820 0x55e1028a6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.254+0000 7fda02800640 1 -- 172.21.15.7:0/84424747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028c8130 msgr2=0x55e1028c2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:19.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.254+0000 7fda02800640 1 --2- 172.21.15.7:0/84424747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028c8130 0x55e1028c2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.254+0000 7fda02800640 1 -- 172.21.15.7:0/84424747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e102546970 con 0x55e1028ac890 2024-09-10T21:46:19.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.254+0000 7fda02800640 1 --2- 172.21.15.7:0/84424747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028ac890 0x55e1028c3280 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc010780 tx=0x7fd9fc010c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32e22fc8f3d66298 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:19.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.254+0000 7fd9e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:46:19.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.254+0000 7fd9e6a00640 1 -- 172.21.15.7:0/84424747 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc01a020 con 0x55e1028ac890 2024-09-10T21:46:19.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.254+0000 7fd9e6a00640 1 -- 172.21.15.7:0/84424747 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7fd9fc00e8e0 con 0x55e1028ac890 2024-09-10T21:46:19.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.254+0000 7fda0bc40740 1 -- 172.21.15.7:0/84424747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9f401e450 con 0x55e1028ac890 2024-09-10T21:46:19.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.255+0000 7fd9e6a00640 1 -- 172.21.15.7:0/84424747 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fd9fc024050 con 0x55e1028ac890 2024-09-10T21:46:19.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.255+0000 7fda0bc40740 10 client.6435 fetch_fsmap learned FSMap version 52 2024-09-10T21:46:19.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.255+0000 7fda0bc40740 1 -- 172.21.15.7:0/84424747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55e1025246a0 con 0x55e1028ac890 2024-09-10T21:46:19.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.255+0000 7fd9e6a00640 1 -- 172.21.15.7:0/84424747 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9f401e450 con 0x55e1028ac890 2024-09-10T21:46:19.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.255+0000 7fd9e6a00640 1 -- 172.21.15.7:0/84424747 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7fd9fc027020 con 0x55e1028ac890 2024-09-10T21:46:19.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.255+0000 7fda0bc40740 10 client.6435 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T21:46:19.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.255+0000 7fda0bc40740 10 client.6435 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-10T21:46:19.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.255+0000 7fda0bc40740 20 client.6435 populate_metadata read hostname 'smithi007' 2024-09-10T21:46:19.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.255+0000 7fda0bc40740 1 --2- 172.21.15.7:0/84424747 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55e1028da1b0 0x55e1028da590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.255+0000 7fda0bc40740 4 client.6435 mds_command: new command op to 235490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:46:19.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.255+0000 7fda0bc40740 1 -- 172.21.15.7:0/84424747 --> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1027f77c0 con 0x55e1028da1b0 2024-09-10T21:46:19.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.256+0000 7fda01e00640 1 --2- 172.21.15.7:0/84424747 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55e1028da1b0 0x55e1028da590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.256+0000 7fda01e00640 1 --2- 172.21.15.7:0/84424747 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55e1028da1b0 0x55e1028da590 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d90fa46e3fad8295 server_cookie=4a9a305877509634 in_seq=0 out_seq=0 2024-09-10T21:46:19.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.256+0000 7fd9e6a00640 10 client.6435 ms_handle_connect on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:19.281 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.257+0000 7fd9e6a00640 1 -- 172.21.15.7:0/84424747 <== mds.0 v2:172.21.15.133:6834/2622530239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e1027f77c0 con 0x55e1028da1b0 2024-09-10T21:46:19.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.257+0000 7fd9e6a00640 10 client.6435 handle_command_reply: tid=0 2024-09-10T21:46:19.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.257+0000 7fda0bc40740 1 client.6435 shutdown 2024-09-10T21:46:19.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.257+0000 7fda0bc40740 1 -- 172.21.15.7:0/84424747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028ac890 msgr2=0x55e1028c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:19.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.257+0000 7fda0bc40740 1 --2- 172.21.15.7:0/84424747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028ac890 0x55e1028c3280 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc010780 tx=0x7fd9fc010c40 comp rx=0 tx=0).stop 2024-09-10T21:46:19.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.257+0000 7fda0bc40740 1 -- 172.21.15.7:0/84424747 shutdown_connections 2024-09-10T21:46:19.282 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.257+0000 7fda0bc40740 1 --2- 172.21.15.7:0/84424747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028c8130 0x55e1028c2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.257+0000 7fda0bc40740 1 --2- 172.21.15.7:0/84424747 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55e1028da1b0 0x55e1028da590 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.257+0000 7fda0bc40740 1 --2- 172.21.15.7:0/84424747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028ac890 0x55e1028c3280 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.257+0000 7fda0bc40740 1 --2- 172.21.15.7:0/84424747 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e102883820 0x55e1028a6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.257+0000 7fd9e6a00640 0 client.6435 ms_handle_reset on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:19.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.257+0000 7fda0bc40740 1 -- 172.21.15.7:0/84424747 >> 172.21.15.7:0/84424747 conn(0x55e1028832a0 msgr2=0x55e1028a89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:19.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.258+0000 7fda0bc40740 1 -- 172.21.15.7:0/84424747 shutdown_connections 2024-09-10T21:46:19.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.258+0000 7fda0bc40740 1 -- 172.21.15.7:0/84424747 wait complete. 2024-09-10T21:46:19.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.258+0000 7fda0bc40740 20 client.6435 trim_cache size 0 max 16384 2024-09-10T21:46:19.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.276+0000 7fda0bc40740 1 Processor -- start 2024-09-10T21:46:19.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.276+0000 7fda0bc40740 1 -- start start 2024-09-10T21:46:19.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.277+0000 7fda0bc40740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e102883820 0x55e1028bd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.277+0000 7fda0bc40740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e1028ac890 0x55e1028eab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.277+0000 7fda0bc40740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028c8130 0x55e1028eb070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.277+0000 7fda0bc40740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fda041fef80 con 0x55e102883820 2024-09-10T21:46:19.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.277+0000 7fda0bc40740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fda041fdfb0 con 0x55e1028c8130 2024-09-10T21:46:19.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.277+0000 7fda0bc40740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fda041f6120 con 0x55e1028ac890 2024-09-10T21:46:19.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.277+0000 7fda01e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e102883820 0x55e1028bd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.278+0000 7fda01e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e102883820 0x55e1028bd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49572/0 (socket says 172.21.15.7:49572) 2024-09-10T21:46:19.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.278+0000 7fda01e00640 1 -- 172.21.15.7:0/3011982924 learned_addr learned my addr 172.21.15.7:0/3011982924 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:19.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.278+0000 7fd9f9400640 1 --2- 172.21.15.7:0/3011982924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e1028ac890 0x55e1028eab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.278+0000 7fda01e00640 1 -- 172.21.15.7:0/3011982924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e1028ac890 msgr2=0x55e1028eab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:19.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.278+0000 7fda02800640 1 --2- 172.21.15.7:0/3011982924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028c8130 0x55e1028eb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.278+0000 7fda01e00640 1 --2- 172.21.15.7:0/3011982924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e1028ac890 0x55e1028eab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.278+0000 7fda01e00640 1 -- 172.21.15.7:0/3011982924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028c8130 msgr2=0x55e1028eb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:19.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.278+0000 7fda01e00640 1 --2- 172.21.15.7:0/3011982924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028c8130 0x55e1028eb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.278+0000 7fda01e00640 1 -- 172.21.15.7:0/3011982924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1027abec0 con 0x55e102883820 2024-09-10T21:46:19.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.278+0000 7fd9f9400640 1 --2- 172.21.15.7:0/3011982924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e1028ac890 0x55e1028eab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:46:19.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.278+0000 7fda02800640 1 --2- 172.21.15.7:0/3011982924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028c8130 0x55e1028eb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:46:19.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.278+0000 7fda01e00640 1 --2- 172.21.15.7:0/3011982924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e102883820 0x55e1028bd560 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40058c0 tx=0x7fd9f4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=127ee0f245c97239 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:19.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.278+0000 7fd9f1c00640 1 -- 172.21.15.7:0/3011982924 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f403f070 con 0x55e102883820 2024-09-10T21:46:19.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.279+0000 7fd9f1c00640 1 -- 172.21.15.7:0/3011982924 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9f401f8d0 con 0x55e102883820 2024-09-10T21:46:19.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.279+0000 7fd9f1c00640 1 -- 172.21.15.7:0/3011982924 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4015040 con 0x55e102883820 2024-09-10T21:46:19.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.279+0000 7fda0bc40740 1 -- 172.21.15.7:0/3011982924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e102883820 msgr2=0x55e1028bd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:19.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.279+0000 7fda0bc40740 1 --2- 172.21.15.7:0/3011982924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e102883820 0x55e1028bd560 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd9f40058c0 tx=0x7fd9f4004730 comp rx=0 tx=0).stop 2024-09-10T21:46:19.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.279+0000 7fda0bc40740 1 -- 172.21.15.7:0/3011982924 shutdown_connections 2024-09-10T21:46:19.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.279+0000 7fda0bc40740 1 --2- 172.21.15.7:0/3011982924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028c8130 0x55e1028eb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.279+0000 7fda0bc40740 1 --2- 172.21.15.7:0/3011982924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e1028ac890 0x55e1028eab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.279+0000 7fda0bc40740 1 --2- 172.21.15.7:0/3011982924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e102883820 0x55e1028bd560 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.279+0000 7fda0bc40740 1 -- 172.21.15.7:0/3011982924 >> 172.21.15.7:0/3011982924 conn(0x55e1028832a0 msgr2=0x55e1028abff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:19.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.279+0000 7fda0bc40740 1 -- 172.21.15.7:0/3011982924 shutdown_connections 2024-09-10T21:46:19.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.279+0000 7fda0bc40740 1 -- 172.21.15.7:0/3011982924 wait complete. 2024-09-10T21:46:19.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.280+0000 7fda0bc40740 1 -- start start 2024-09-10T21:46:19.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.280+0000 7fda0bc40740 1 Processor -- start 2024-09-10T21:46:19.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.281+0000 7fda0bc40740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e102883820 0x55e1028802a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.281+0000 7fda0bc40740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028ac890 0x55e102940cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.281+0000 7fda0bc40740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028c8130 0x55e102941210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.281+0000 7fda0bc40740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e1027f77c0 con 0x55e1028c8130 2024-09-10T21:46:19.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.281+0000 7fda0bc40740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fda041fdba0 con 0x55e1028ac890 2024-09-10T21:46:19.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.281+0000 7fda0bc40740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55e1028826d0 con 0x55e102883820 2024-09-10T21:46:19.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.281+0000 7fda0bc40740 10 client.0 _ll_register_callbacks cb 0x7fd9fc001b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:46:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.281+0000 7fda02800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028c8130 0x55e102941210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.281+0000 7fda02800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028c8130 0x55e102941210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49576/0 (socket says 172.21.15.7:49576) 2024-09-10T21:46:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.281+0000 7fda02800640 1 -- 172.21.15.7:0/3629495835 learned_addr learned my addr 172.21.15.7:0/3629495835 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.282+0000 7fda01e00640 1 --2- 172.21.15.7:0/3629495835 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e102883820 0x55e1028802a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.282+0000 7fd9f9400640 1 --2- 172.21.15.7:0/3629495835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028ac890 0x55e102940cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.282+0000 7fda02800640 1 -- 172.21.15.7:0/3629495835 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e102883820 msgr2=0x55e1028802a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.282+0000 7fda02800640 1 --2- 172.21.15.7:0/3629495835 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e102883820 0x55e1028802a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.282+0000 7fda02800640 1 -- 172.21.15.7:0/3629495835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028ac890 msgr2=0x55e102940cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.282+0000 7fda02800640 1 --2- 172.21.15.7:0/3629495835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028ac890 0x55e102940cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.282+0000 7fda02800640 1 -- 172.21.15.7:0/3629495835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1028bdb00 con 0x55e1028c8130 2024-09-10T21:46:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.282+0000 7fd9f9400640 1 --2- 172.21.15.7:0/3629495835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028ac890 0x55e102940cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:19.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.282+0000 7fda02800640 1 --2- 172.21.15.7:0/3629495835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028c8130 0x55e102941210 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc009010 tx=0x7fd9fc010480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5689272d3453b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.282+0000 7fd9e6a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:46:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.282+0000 7fd9e6a00640 1 -- 172.21.15.7:0/3629495835 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9fc02c020 con 0x55e1028c8130 2024-09-10T21:46:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.282+0000 7fd9e6a00640 1 -- 172.21.15.7:0/3629495835 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(71..71 src has 1..71) ==== 5795+0+0 (secure 0 0 0) 0x7fd9fc003b70 con 0x55e1028c8130 2024-09-10T21:46:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.282+0000 7fda0bc40740 1 -- 172.21.15.7:0/3629495835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9f401f8d0 con 0x55e1028c8130 2024-09-10T21:46:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.283+0000 7fd9e6a00640 1 -- 172.21.15.7:0/3629495835 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) ==== 24+0+0 (secure 0 0 0) 0x7fd9fc036050 con 0x55e1028c8130 2024-09-10T21:46:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.283+0000 7fda0bc40740 10 client.6441 fetch_fsmap learned FSMap version 52 2024-09-10T21:46:19.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.283+0000 7fda0bc40740 1 -- 172.21.15.7:0/3629495835 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) -- 0x55e10293e630 con 0x55e1028c8130 2024-09-10T21:46:19.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.283+0000 7fd9e6a00640 1 -- 172.21.15.7:0/3629495835 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9f401f8d0 con 0x55e1028c8130 2024-09-10T21:46:19.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.283+0000 7fd9e6a00640 1 -- 172.21.15.7:0/3629495835 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 52) ==== 3502+0+0 (secure 0 0 0) 0x7fd9fc039020 con 0x55e1028c8130 2024-09-10T21:46:19.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.283+0000 7fda0bc40740 10 client.6441 fetch_fsmap finished waiting for FSMap version 52 2024-09-10T21:46:19.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.283+0000 7fda0bc40740 10 client.6441 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-10T21:46:19.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.283+0000 7fda0bc40740 20 client.6441 populate_metadata read hostname 'smithi007' 2024-09-10T21:46:19.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.283+0000 7fda0bc40740 1 --2- 172.21.15.7:0/3629495835 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55e1028807e0 0x55e1028a7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.283+0000 7fda0bc40740 4 client.6441 mds_command: new command op to 235490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:46:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.283+0000 7fda0bc40740 1 -- 172.21.15.7:0/3629495835 --> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e1027f3270 con 0x55e1028807e0 2024-09-10T21:46:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.283+0000 7fda01e00640 1 --2- 172.21.15.7:0/3629495835 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55e1028807e0 0x55e1028a7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.284+0000 7fda01e00640 1 --2- 172.21.15.7:0/3629495835 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55e1028807e0 0x55e1028a7320 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77b913a9d0917031 server_cookie=e3ffc28dc882f1f in_seq=0 out_seq=0 2024-09-10T21:46:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.284+0000 7fd9e6a00640 10 client.6441 ms_handle_connect on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:19.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.284+0000 7fd9e6a00640 1 -- 172.21.15.7:0/3629495835 <== mds.0 v2:172.21.15.133:6834/2622530239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+576 (crc 0 0 0) 0x55e1027f3270 con 0x55e1028807e0 2024-09-10T21:46:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.284+0000 7fd9e6a00640 10 client.6441 handle_command_reply: tid=0 2024-09-10T21:46:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.284+0000 7fda0bc40740 1 client.6441 shutdown 2024-09-10T21:46:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.284+0000 7fda0bc40740 1 -- 172.21.15.7:0/3629495835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028c8130 msgr2=0x55e102941210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.284+0000 7fda0bc40740 1 --2- 172.21.15.7:0/3629495835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028c8130 0x55e102941210 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fd9fc009010 tx=0x7fd9fc010480 comp rx=0 tx=0).stop 2024-09-10T21:46:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.285+0000 7fda0bc40740 1 -- 172.21.15.7:0/3629495835 shutdown_connections 2024-09-10T21:46:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.285+0000 7fda0bc40740 1 --2- 172.21.15.7:0/3629495835 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e1028ac890 0x55e102940cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.285+0000 7fda0bc40740 1 --2- 172.21.15.7:0/3629495835 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55e1028807e0 0x55e1028a7320 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.285+0000 7fda0bc40740 1 --2- 172.21.15.7:0/3629495835 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e1028c8130 0x55e102941210 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.285+0000 7fda0bc40740 1 --2- 172.21.15.7:0/3629495835 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e102883820 0x55e1028802a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.285+0000 7fd9e6a00640 0 client.6441 ms_handle_reset on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.285+0000 7fda0bc40740 1 -- 172.21.15.7:0/3629495835 >> 172.21.15.7:0/3629495835 conn(0x55e1028832a0 msgr2=0x55e1028c1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:19.307 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.285+0000 7fda0bc40740 1 -- 172.21.15.7:0/3629495835 shutdown_connections 2024-09-10T21:46:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.285+0000 7fda0bc40740 1 -- 172.21.15.7:0/3629495835 wait complete. 2024-09-10T21:46:19.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.285+0000 7fda0bc40740 20 client.6441 trim_cache size 0 max 16384 2024-09-10T21:46:19.308 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:46:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:46:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-09-10T21:46:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "d158ff07-06ba-4bd2-9cbd-cc6aad170cea": { 2024-09-10T21:46:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/subdir", 2024-09-10T21:46:19.308 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "d158ff07-06ba-4bd2-9cbd-cc6aad170cea", 2024-09-10T21:46:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive" 2024-09-10T21:46:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:46:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:46:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:46:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "d158ff07-06ba-4bd2-9cbd-cc6aad170cea": { 2024-09-10T21:46:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:46:19.061276+0000", 2024-09-10T21:46:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/subdir", 2024-09-10T21:46:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:46:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:46:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:46:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:46:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:46:19.309 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:46:19.309 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:46:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.285+0000 7fda08c00640 1 -- 172.21.15.7:0/2444687424 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd9e0077f70 msgr2=0x7fd9e007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.285+0000 7fda08c00640 1 --2- 172.21.15.7:0/2444687424 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd9e0077f70 0x7fd9e007a430 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd9f4005fd0 tx=0x7fd9f4005850 comp rx=0 tx=0).stop 2024-09-10T21:46:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.285+0000 7fda08c00640 1 -- 172.21.15.7:0/2444687424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04074280 msgr2=0x7fda04152290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.285+0000 7fda08c00640 1 --2- 172.21.15.7:0/2444687424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04074280 0x7fda04152290 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec02ab20 tx=0x7fd9ec032c10 comp rx=0 tx=0).stop 2024-09-10T21:46:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.285+0000 7fda08c00640 1 -- 172.21.15.7:0/2444687424 shutdown_connections 2024-09-10T21:46:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.285+0000 7fda08c00640 1 --2- 172.21.15.7:0/2444687424 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd9e0077f70 0x7fd9e007a430 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.285+0000 7fda08c00640 1 --2- 172.21.15.7:0/2444687424 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda04155fb0 0x7fda040b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.285+0000 7fda08c00640 1 --2- 172.21.15.7:0/2444687424 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda04074280 0x7fda04152290 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.285+0000 7fda08c00640 1 --2- 172.21.15.7:0/2444687424 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda0406fcf0 0x7fda040b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.286+0000 7fda08c00640 1 -- 172.21.15.7:0/2444687424 >> 172.21.15.7:0/2444687424 conn(0x7fda040b16d0 msgr2=0x7fda0415f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.286+0000 7fda08c00640 1 -- 172.21.15.7:0/2444687424 shutdown_connections 2024-09-10T21:46:19.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:19.286+0000 7fda08c00640 1 -- 172.21.15.7:0/2444687424 wait complete. 2024-09-10T21:46:19.312 INFO:tasks.cephfs.filesystem:scrub status for tag:d158ff07-06ba-4bd2-9cbd-cc6aad170cea - {'path': '/subdir', 'tag': 'd158ff07-06ba-4bd2-9cbd-cc6aad170cea', 'options': 'recursive'} 2024-09-10T21:46:21.984 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-10T21:46:21.982+0000 7feb42c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-10T21:46:30.494 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:46:30.499 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:46:49.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:49.314 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-09-10T21:46:49.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.486+0000 7ff514a00640 1 -- 172.21.15.7:0/2793815829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff51006fcf0 msgr2=0x7ff5101600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:49.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.486+0000 7ff514a00640 1 --2- 172.21.15.7:0/2793815829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff51006fcf0 0x7ff5101600d0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff5100cd510 tx=0x7ff4fc02a4b0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.487+0000 7ff514a00640 1 -- 172.21.15.7:0/2793815829 shutdown_connections 2024-09-10T21:46:49.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.487+0000 7ff514a00640 1 --2- 172.21.15.7:0/2793815829 >> v2:172.21.15.133:3300/0 conn(0x7ff510074450 0x7ff5100ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.487+0000 7ff514a00640 1 --2- 172.21.15.7:0/2793815829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff51006fcf0 0x7ff5101600d0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.487+0000 7ff514a00640 1 -- 172.21.15.7:0/2793815829 >> 172.21.15.7:0/2793815829 conn(0x7ff5100b16d0 msgr2=0x7ff5100b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:49.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.487+0000 7ff514a00640 1 -- 172.21.15.7:0/2793815829 shutdown_connections 2024-09-10T21:46:49.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.487+0000 7ff514a00640 1 -- 172.21.15.7:0/2793815829 wait complete. 2024-09-10T21:46:49.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.488+0000 7ff514a00640 1 Processor -- start 2024-09-10T21:46:49.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.488+0000 7ff514a00640 1 -- start start 2024-09-10T21:46:49.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.489+0000 7ff514a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff51006fcf0 0x7ff51014aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.489+0000 7ff514a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff510074450 0x7ff5100b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.489+0000 7ff514a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff510155fb0 0x7ff5100b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.489+0000 7ff514a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff5100ca930 con 0x7ff510155fb0 2024-09-10T21:46:49.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.489+0000 7ff514a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff5100ca7b0 con 0x7ff510074450 2024-09-10T21:46:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.489+0000 7ff514a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ff5100caab0 con 0x7ff51006fcf0 2024-09-10T21:46:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.490+0000 7ff50e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff510155fb0 0x7ff5100b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.490+0000 7ff50e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff510155fb0 0x7ff5100b4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44978/0 (socket says 172.21.15.7:44978) 2024-09-10T21:46:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.490+0000 7ff50e800640 1 -- 172.21.15.7:0/980149448 learned_addr learned my addr 172.21.15.7:0/980149448 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.490+0000 7ff50d400640 1 --2- 172.21.15.7:0/980149448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff510074450 0x7ff5100b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.490+0000 7ff50de00640 1 --2- 172.21.15.7:0/980149448 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff51006fcf0 0x7ff51014aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.490+0000 7ff50e800640 1 -- 172.21.15.7:0/980149448 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff51006fcf0 msgr2=0x7ff51014aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.490+0000 7ff50e800640 1 --2- 172.21.15.7:0/980149448 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff51006fcf0 0x7ff51014aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.490+0000 7ff50e800640 1 -- 172.21.15.7:0/980149448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff510074450 msgr2=0x7ff5100b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.490+0000 7ff50e800640 1 --2- 172.21.15.7:0/980149448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff510074450 0x7ff5100b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.490+0000 7ff50e800640 1 -- 172.21.15.7:0/980149448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5101f8ba0 con 0x7ff510155fb0 2024-09-10T21:46:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.490+0000 7ff50de00640 1 --2- 172.21.15.7:0/980149448 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff51006fcf0 0x7ff51014aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:49.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.490+0000 7ff50e800640 1 --2- 172.21.15.7:0/980149448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff510155fb0 0x7ff5100b4f20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff50800eaa0 tx=0x7ff50800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.491+0000 7ff506000640 1 -- 172.21.15.7:0/980149448 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50800cdb0 con 0x7ff510155fb0 2024-09-10T21:46:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.491+0000 7ff506000640 1 -- 172.21.15.7:0/980149448 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff508004c10 con 0x7ff510155fb0 2024-09-10T21:46:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.491+0000 7ff506000640 1 -- 172.21.15.7:0/980149448 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff508005020 con 0x7ff510155fb0 2024-09-10T21:46:49.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.491+0000 7ff514a00640 1 -- 172.21.15.7:0/980149448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5101f4800 con 0x7ff510155fb0 2024-09-10T21:46:49.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.491+0000 7ff514a00640 1 -- 172.21.15.7:0/980149448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5101f4dc0 con 0x7ff510155fb0 2024-09-10T21:46:49.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.493+0000 7ff506000640 1 -- 172.21.15.7:0/980149448 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff50800f040 con 0x7ff510155fb0 2024-09-10T21:46:49.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.494+0000 7ff506000640 1 --2- 172.21.15.7:0/980149448 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff4f0077f70 0x7ff4f007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.494+0000 7ff506000640 1 -- 172.21.15.7:0/980149448 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7ff50809a430 con 0x7ff510155fb0 2024-09-10T21:46:49.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.495+0000 7ff5177dd740 1 Processor -- start 2024-09-10T21:46:49.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.495+0000 7ff50de00640 1 --2- 172.21.15.7:0/980149448 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff4f0077f70 0x7ff4f007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.495+0000 7ff5177dd740 1 -- start start 2024-09-10T21:46:49.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.495+0000 7ff50de00640 1 --2- 172.21.15.7:0/980149448 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff4f0077f70 0x7ff4f007a430 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff5000016a0 tx=0x7ff500009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.495+0000 7ff5177dd740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c677130 0x55d39c677510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.495+0000 7ff5177dd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c6322a0 0x55d39c675c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.495+0000 7ff5177dd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c632820 0x55d39c676150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.495+0000 7ff5177dd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d39c5911c0 con 0x55d39c6322a0 2024-09-10T21:46:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.495+0000 7ff5177dd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d39c5a4870 con 0x55d39c632820 2024-09-10T21:46:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.495+0000 7ff5177dd740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55d39c593ca0 con 0x55d39c677130 2024-09-10T21:46:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.496+0000 7ff50d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c6322a0 0x55d39c675c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.496+0000 7ff50d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c6322a0 0x55d39c675c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44984/0 (socket says 172.21.15.7:44984) 2024-09-10T21:46:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.496+0000 7ff50d400640 1 -- 172.21.15.7:0/4117015085 learned_addr learned my addr 172.21.15.7:0/4117015085 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.496+0000 7ff50de00640 1 --2- 172.21.15.7:0/4117015085 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c677130 0x55d39c677510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.496+0000 7ff50e800640 1 --2- 172.21.15.7:0/4117015085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c632820 0x55d39c676150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.496+0000 7ff50d400640 1 -- 172.21.15.7:0/4117015085 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c677130 msgr2=0x55d39c677510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.496+0000 7ff50d400640 1 --2- 172.21.15.7:0/4117015085 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c677130 0x55d39c677510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.496+0000 7ff50d400640 1 -- 172.21.15.7:0/4117015085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c632820 msgr2=0x55d39c676150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.496+0000 7ff50d400640 1 --2- 172.21.15.7:0/4117015085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c632820 0x55d39c676150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.496+0000 7ff50d400640 1 -- 172.21.15.7:0/4117015085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d39c36c7d0 con 0x55d39c6322a0 2024-09-10T21:46:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.497+0000 7ff50d400640 1 --2- 172.21.15.7:0/4117015085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c6322a0 0x55d39c675c10 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc035040 tx=0x7ff4fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9941cf21fa06ae93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.497+0000 7ff4fa600640 1 -- 172.21.15.7:0/4117015085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc005220 con 0x55d39c6322a0 2024-09-10T21:46:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.497+0000 7ff4fa600640 1 -- 172.21.15.7:0/4117015085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4fc0054b0 con 0x55d39c6322a0 2024-09-10T21:46:49.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.497+0000 7ff4fa600640 1 -- 172.21.15.7:0/4117015085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc0049c0 con 0x55d39c6322a0 2024-09-10T21:46:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.497+0000 7ff5177dd740 1 -- 172.21.15.7:0/4117015085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c6322a0 msgr2=0x55d39c675c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:49.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.497+0000 7ff5177dd740 1 --2- 172.21.15.7:0/4117015085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c6322a0 0x55d39c675c10 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc035040 tx=0x7ff4fc0046c0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.498+0000 7ff5177dd740 1 -- 172.21.15.7:0/4117015085 shutdown_connections 2024-09-10T21:46:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.498+0000 7ff5177dd740 1 --2- 172.21.15.7:0/4117015085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c632820 0x55d39c676150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.498+0000 7ff5177dd740 1 --2- 172.21.15.7:0/4117015085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c6322a0 0x55d39c675c10 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.498+0000 7ff5177dd740 1 --2- 172.21.15.7:0/4117015085 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c677130 0x55d39c677510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.498+0000 7ff5177dd740 1 -- 172.21.15.7:0/4117015085 >> 172.21.15.7:0/4117015085 conn(0x55d39c62d870 msgr2=0x55d39c631290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.498+0000 7ff5177dd740 1 -- 172.21.15.7:0/4117015085 shutdown_connections 2024-09-10T21:46:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.498+0000 7ff5177dd740 1 -- 172.21.15.7:0/4117015085 wait complete. 2024-09-10T21:46:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.498+0000 7ff5177dd740 1 -- start start 2024-09-10T21:46:49.502 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.499+0000 7ff5177dd740 1 Processor -- start 2024-09-10T21:46:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.499+0000 7ff5177dd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c632820 0x55d39c675850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.499+0000 7ff5177dd740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c677130 0x55d39c630ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.500+0000 7ff5177dd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c675d90 0x55d39c631430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.500+0000 7ff5177dd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d39c53a410 con 0x55d39c632820 2024-09-10T21:46:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.500+0000 7ff5177dd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d39c597ab0 con 0x55d39c675d90 2024-09-10T21:46:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.500+0000 7ff5177dd740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55d39c304740 con 0x55d39c677130 2024-09-10T21:46:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.500+0000 7ff5177dd740 10 client.0 _ll_register_callbacks cb 0x55d39c517360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:46:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.500+0000 7ff50de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c632820 0x55d39c675850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.500+0000 7ff50de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c632820 0x55d39c675850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44994/0 (socket says 172.21.15.7:44994) 2024-09-10T21:46:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.500+0000 7ff50de00640 1 -- 172.21.15.7:0/1585690182 learned_addr learned my addr 172.21.15.7:0/1585690182 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.500+0000 7ff50d400640 1 --2- 172.21.15.7:0/1585690182 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c677130 0x55d39c630ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.500+0000 7ff50e800640 1 --2- 172.21.15.7:0/1585690182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c675d90 0x55d39c631430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.500+0000 7ff50de00640 1 -- 172.21.15.7:0/1585690182 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c677130 msgr2=0x55d39c630ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:49.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.500+0000 7ff50de00640 1 --2- 172.21.15.7:0/1585690182 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c677130 0x55d39c630ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.500+0000 7ff50de00640 1 -- 172.21.15.7:0/1585690182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c675d90 msgr2=0x55d39c631430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:49.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.500+0000 7ff50de00640 1 --2- 172.21.15.7:0/1585690182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c675d90 0x55d39c631430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.500+0000 7ff50de00640 1 -- 172.21.15.7:0/1585690182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d39c2f5970 con 0x55d39c632820 2024-09-10T21:46:49.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.501+0000 7ff50de00640 1 --2- 172.21.15.7:0/1585690182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c632820 0x55d39c675850 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff50000ee00 tx=0x7ff500000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ec81ce04cb527 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:49.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.501+0000 7ff4eea00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:46:49.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.501+0000 7ff4eea00640 1 -- 172.21.15.7:0/1585690182 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500025030 con 0x55d39c632820 2024-09-10T21:46:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.501+0000 7ff4eea00640 1 -- 172.21.15.7:0/1585690182 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7ff50001e040 con 0x55d39c632820 2024-09-10T21:46:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.501+0000 7ff5177dd740 1 -- 172.21.15.7:0/1585690182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4fc0054b0 con 0x55d39c632820 2024-09-10T21:46:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.501+0000 7ff4eea00640 1 -- 172.21.15.7:0/1585690182 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7ff50002e050 con 0x55d39c632820 2024-09-10T21:46:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.501+0000 7ff5177dd740 10 client.6456 fetch_fsmap learned FSMap version 53 2024-09-10T21:46:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.501+0000 7ff5177dd740 1 -- 172.21.15.7:0/1585690182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55d39c2d36a0 con 0x55d39c632820 2024-09-10T21:46:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.502+0000 7ff4eea00640 1 -- 172.21.15.7:0/1585690182 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4fc0054b0 con 0x55d39c632820 2024-09-10T21:46:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.502+0000 7ff4eea00640 1 -- 172.21.15.7:0/1585690182 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7ff500031020 con 0x55d39c632820 2024-09-10T21:46:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.502+0000 7ff5177dd740 10 client.6456 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T21:46:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.502+0000 7ff5177dd740 10 client.6456 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-10T21:46:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.502+0000 7ff5177dd740 20 client.6456 populate_metadata read hostname 'smithi007' 2024-09-10T21:46:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.502+0000 7ff5177dd740 1 --2- 172.21.15.7:0/1585690182 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55d39c6463a0 0x55d39c649020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.502+0000 7ff5177dd740 4 client.6456 mds_command: new command op to 235490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:46:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.502+0000 7ff5177dd740 1 -- 172.21.15.7:0/1585690182 --> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d39c5a67c0 con 0x55d39c6463a0 2024-09-10T21:46:49.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.502+0000 7ff50d400640 1 --2- 172.21.15.7:0/1585690182 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55d39c6463a0 0x55d39c649020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.503+0000 7ff50d400640 1 --2- 172.21.15.7:0/1585690182 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55d39c6463a0 0x55d39c649020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c3f5ea497c02953 server_cookie=6e179101e205a9c0 in_seq=0 out_seq=0 2024-09-10T21:46:49.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.503+0000 7ff4eea00640 10 client.6456 ms_handle_connect on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:49.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.504+0000 7ff4eea00640 1 -- 172.21.15.7:0/1585690182 <== mds.0 v2:172.21.15.133:6834/2622530239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d39c5a67c0 con 0x55d39c6463a0 2024-09-10T21:46:49.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.504+0000 7ff4eea00640 10 client.6456 handle_command_reply: tid=0 2024-09-10T21:46:49.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.504+0000 7ff5177dd740 1 client.6456 shutdown 2024-09-10T21:46:49.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.504+0000 7ff5177dd740 1 -- 172.21.15.7:0/1585690182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c632820 msgr2=0x55d39c675850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:49.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.504+0000 7ff5177dd740 1 --2- 172.21.15.7:0/1585690182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c632820 0x55d39c675850 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff50000ee00 tx=0x7ff500000ed0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.504+0000 7ff5177dd740 1 -- 172.21.15.7:0/1585690182 shutdown_connections 2024-09-10T21:46:49.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.504+0000 7ff5177dd740 1 --2- 172.21.15.7:0/1585690182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c675d90 0x55d39c631430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.504+0000 7ff5177dd740 1 --2- 172.21.15.7:0/1585690182 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55d39c6463a0 0x55d39c649020 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.504+0000 7ff5177dd740 1 --2- 172.21.15.7:0/1585690182 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c677130 0x55d39c630ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.504+0000 7ff5177dd740 1 --2- 172.21.15.7:0/1585690182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c632820 0x55d39c675850 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.504+0000 7ff4eea00640 0 client.6456 ms_handle_reset on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:49.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.504+0000 7ff5177dd740 1 -- 172.21.15.7:0/1585690182 >> 172.21.15.7:0/1585690182 conn(0x55d39c6322a0 msgr2=0x55d39c62c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:49.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.504+0000 7ff5177dd740 1 -- 172.21.15.7:0/1585690182 shutdown_connections 2024-09-10T21:46:49.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.504+0000 7ff5177dd740 1 -- 172.21.15.7:0/1585690182 wait complete. 2024-09-10T21:46:49.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.505+0000 7ff5177dd740 20 client.6456 trim_cache size 0 max 16384 2024-09-10T21:46:49.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.523+0000 7ff5177dd740 1 Processor -- start 2024-09-10T21:46:49.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.523+0000 7ff5177dd740 1 -- start start 2024-09-10T21:46:49.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.524+0000 7ff5177dd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c632820 0x55d39c630c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.524+0000 7ff5177dd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c675d90 0x55d39c649210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.524+0000 7ff5177dd740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c677130 0x55d39c657270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.524+0000 7ff5177dd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff510201320 con 0x55d39c675d90 2024-09-10T21:46:49.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.524+0000 7ff5177dd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff510200390 con 0x55d39c632820 2024-09-10T21:46:49.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.524+0000 7ff5177dd740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ff5100c5250 con 0x55d39c677130 2024-09-10T21:46:49.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.524+0000 7ff50d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c675d90 0x55d39c649210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.524+0000 7ff50d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c675d90 0x55d39c649210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45010/0 (socket says 172.21.15.7:45010) 2024-09-10T21:46:49.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.524+0000 7ff50d400640 1 -- 172.21.15.7:0/2921437268 learned_addr learned my addr 172.21.15.7:0/2921437268 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:49.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.524+0000 7ff50de00640 1 --2- 172.21.15.7:0/2921437268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c632820 0x55d39c630c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.524+0000 7ff50e800640 1 --2- 172.21.15.7:0/2921437268 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c677130 0x55d39c657270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.524+0000 7ff50d400640 1 -- 172.21.15.7:0/2921437268 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c677130 msgr2=0x55d39c657270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:49.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.524+0000 7ff50d400640 1 --2- 172.21.15.7:0/2921437268 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c677130 0x55d39c657270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.525+0000 7ff50d400640 1 -- 172.21.15.7:0/2921437268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c632820 msgr2=0x55d39c630c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:49.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.525+0000 7ff50d400640 1 --2- 172.21.15.7:0/2921437268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c632820 0x55d39c630c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.525+0000 7ff50d400640 1 -- 172.21.15.7:0/2921437268 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d39c66cbe0 con 0x55d39c675d90 2024-09-10T21:46:49.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.525+0000 7ff50e800640 1 --2- 172.21.15.7:0/2921437268 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c677130 0x55d39c657270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:46:49.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.525+0000 7ff50de00640 1 --2- 172.21.15.7:0/2921437268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c632820 0x55d39c630c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:49.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.525+0000 7ff50d400640 1 --2- 172.21.15.7:0/2921437268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c675d90 0x55d39c649210 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0098f0 tx=0x7ff4fc02a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202b403fee61b526 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:49.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.525+0000 7ff4f9c00640 1 -- 172.21.15.7:0/2921437268 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc047110 con 0x55d39c675d90 2024-09-10T21:46:49.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.525+0000 7ff4f9c00640 1 -- 172.21.15.7:0/2921437268 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4fc0472b0 con 0x55d39c675d90 2024-09-10T21:46:49.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.525+0000 7ff4f9c00640 1 -- 172.21.15.7:0/2921437268 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc047780 con 0x55d39c675d90 2024-09-10T21:46:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.526+0000 7ff5177dd740 1 -- 172.21.15.7:0/2921437268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c675d90 msgr2=0x55d39c649210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.526+0000 7ff5177dd740 1 --2- 172.21.15.7:0/2921437268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c675d90 0x55d39c649210 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc0098f0 tx=0x7ff4fc02a4b0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.526+0000 7ff5177dd740 1 -- 172.21.15.7:0/2921437268 shutdown_connections 2024-09-10T21:46:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.526+0000 7ff5177dd740 1 --2- 172.21.15.7:0/2921437268 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c677130 0x55d39c657270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.526+0000 7ff5177dd740 1 --2- 172.21.15.7:0/2921437268 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c675d90 0x55d39c649210 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.526+0000 7ff5177dd740 1 --2- 172.21.15.7:0/2921437268 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c632820 0x55d39c630c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.526+0000 7ff5177dd740 1 -- 172.21.15.7:0/2921437268 >> 172.21.15.7:0/2921437268 conn(0x55d39c6322a0 msgr2=0x55d39c62a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.526+0000 7ff5177dd740 1 -- 172.21.15.7:0/2921437268 shutdown_connections 2024-09-10T21:46:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.526+0000 7ff5177dd740 1 -- 172.21.15.7:0/2921437268 wait complete. 2024-09-10T21:46:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.526+0000 7ff5177dd740 1 -- start start 2024-09-10T21:46:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.527+0000 7ff5177dd740 1 Processor -- start 2024-09-10T21:46:49.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.527+0000 7ff5177dd740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c632820 0x55d39c62a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.527+0000 7ff5177dd740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c675d90 0x55d39c62b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.527+0000 7ff5177dd740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c677130 0x55d39c6f6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.528+0000 7ff5177dd740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff5100c58a0 con 0x55d39c677130 2024-09-10T21:46:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.528+0000 7ff5177dd740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d39c6f8920 con 0x55d39c675d90 2024-09-10T21:46:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.528+0000 7ff5177dd740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55d39c657ed0 con 0x55d39c632820 2024-09-10T21:46:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.528+0000 7ff5177dd740 10 client.0 _ll_register_callbacks cb 0x7ff50000ea50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:46:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.528+0000 7ff50e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c677130 0x55d39c6f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.528+0000 7ff50e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c677130 0x55d39c6f6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45018/0 (socket says 172.21.15.7:45018) 2024-09-10T21:46:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.528+0000 7ff50e800640 1 -- 172.21.15.7:0/3395693754 learned_addr learned my addr 172.21.15.7:0/3395693754 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.528+0000 7ff50e800640 1 -- 172.21.15.7:0/3395693754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c632820 msgr2=0x55d39c62a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.528+0000 7ff50e800640 1 --2- 172.21.15.7:0/3395693754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c632820 0x55d39c62a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.528+0000 7ff50e800640 1 -- 172.21.15.7:0/3395693754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c675d90 msgr2=0x55d39c62b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:49.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.528+0000 7ff50e800640 1 --2- 172.21.15.7:0/3395693754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c675d90 0x55d39c62b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.528+0000 7ff50e800640 1 -- 172.21.15.7:0/3395693754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d39c67a360 con 0x55d39c677130 2024-09-10T21:46:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.528+0000 7ff50e800640 1 --2- 172.21.15.7:0/3395693754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c677130 0x55d39c6f6720 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff508098860 tx=0x7ff508009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a1543ec0be814af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.528+0000 7ff4eea00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:46:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.529+0000 7ff4eea00640 1 -- 172.21.15.7:0/3395693754 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff50801f040 con 0x55d39c677130 2024-09-10T21:46:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.529+0000 7ff4eea00640 1 -- 172.21.15.7:0/3395693754 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7ff5080881e0 con 0x55d39c677130 2024-09-10T21:46:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.529+0000 7ff5177dd740 1 -- 172.21.15.7:0/3395693754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4fc0472b0 con 0x55d39c677130 2024-09-10T21:46:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.529+0000 7ff4eea00640 1 -- 172.21.15.7:0/3395693754 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7ff50806bc90 con 0x55d39c677130 2024-09-10T21:46:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.529+0000 7ff5177dd740 10 client.6462 fetch_fsmap learned FSMap version 53 2024-09-10T21:46:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.529+0000 7ff5177dd740 1 -- 172.21.15.7:0/3395693754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55d39c62aef0 con 0x55d39c677130 2024-09-10T21:46:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.529+0000 7ff4eea00640 1 -- 172.21.15.7:0/3395693754 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4fc0472b0 con 0x55d39c677130 2024-09-10T21:46:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.529+0000 7ff4eea00640 1 -- 172.21.15.7:0/3395693754 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7ff50805e190 con 0x55d39c677130 2024-09-10T21:46:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.529+0000 7ff5177dd740 10 client.6462 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T21:46:49.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.529+0000 7ff5177dd740 10 client.6462 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-10T21:46:49.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.529+0000 7ff5177dd740 20 client.6462 populate_metadata read hostname 'smithi007' 2024-09-10T21:46:49.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.529+0000 7ff5177dd740 1 --2- 172.21.15.7:0/3395693754 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55d39c6f6cb0 0x55d39c646e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.529+0000 7ff5177dd740 4 client.6462 mds_command: new command op to 235490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:46:49.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.529+0000 7ff5177dd740 1 -- 172.21.15.7:0/3395693754 --> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d39c5a2270 con 0x55d39c6f6cb0 2024-09-10T21:46:49.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.530+0000 7ff50de00640 1 --2- 172.21.15.7:0/3395693754 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55d39c6f6cb0 0x55d39c646e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.530+0000 7ff50de00640 1 --2- 172.21.15.7:0/3395693754 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55d39c6f6cb0 0x55d39c646e70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a305433ed0f11ff9 server_cookie=14d019ef816d64bb in_seq=0 out_seq=0 2024-09-10T21:46:49.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.530+0000 7ff4eea00640 10 client.6462 ms_handle_connect on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:49.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.530+0000 7ff4eea00640 1 -- 172.21.15.7:0/3395693754 <== mds.0 v2:172.21.15.133:6834/2622530239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+380 (crc 0 0 0) 0x55d39c5a2270 con 0x55d39c6f6cb0 2024-09-10T21:46:49.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.530+0000 7ff4eea00640 10 client.6462 handle_command_reply: tid=0 2024-09-10T21:46:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.530+0000 7ff5177dd740 1 client.6462 shutdown 2024-09-10T21:46:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.531+0000 7ff5177dd740 1 -- 172.21.15.7:0/3395693754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c677130 msgr2=0x55d39c6f6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.531+0000 7ff5177dd740 1 --2- 172.21.15.7:0/3395693754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c677130 0x55d39c6f6720 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff508098860 tx=0x7ff508009730 comp rx=0 tx=0).stop 2024-09-10T21:46:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.531+0000 7ff5177dd740 1 -- 172.21.15.7:0/3395693754 shutdown_connections 2024-09-10T21:46:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.531+0000 7ff5177dd740 1 --2- 172.21.15.7:0/3395693754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d39c675d90 0x55d39c62b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.531+0000 7ff5177dd740 1 --2- 172.21.15.7:0/3395693754 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55d39c6f6cb0 0x55d39c646e70 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.531+0000 7ff5177dd740 1 --2- 172.21.15.7:0/3395693754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d39c677130 0x55d39c6f6720 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.531+0000 7ff4eea00640 0 client.6462 ms_handle_reset on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.531+0000 7ff5177dd740 1 --2- 172.21.15.7:0/3395693754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d39c632820 0x55d39c62a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.531+0000 7ff5177dd740 1 -- 172.21.15.7:0/3395693754 >> 172.21.15.7:0/3395693754 conn(0x55d39c6322a0 msgr2=0x55d39c62c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.531+0000 7ff5177dd740 1 -- 172.21.15.7:0/3395693754 shutdown_connections 2024-09-10T21:46:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.531+0000 7ff5177dd740 1 -- 172.21.15.7:0/3395693754 wait complete. 2024-09-10T21:46:49.536 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.531+0000 7ff5177dd740 20 client.6462 trim_cache size 0 max 16384 2024-09-10T21:46:49.537 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:46:49.537 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-09-10T21:46:49.537 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {}, 2024-09-10T21:46:49.537 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:46:49.537 INFO:teuthology.orchestra.run.smithi007.stdout: "d158ff07-06ba-4bd2-9cbd-cc6aad170cea": { 2024-09-10T21:46:49.537 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:46:19.061276+0000", 2024-09-10T21:46:49.537 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/subdir", 2024-09-10T21:46:49.537 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:46:49.537 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:46:49.537 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:46:49.538 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:46:49.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:46:49.538 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:46:49.538 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:46:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.532+0000 7ff514a00640 1 -- 172.21.15.7:0/980149448 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff4f0077f70 msgr2=0x7ff4f007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.532+0000 7ff514a00640 1 --2- 172.21.15.7:0/980149448 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff4f0077f70 0x7ff4f007a430 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff5000016a0 tx=0x7ff500009040 comp rx=0 tx=0).stop 2024-09-10T21:46:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.532+0000 7ff514a00640 1 -- 172.21.15.7:0/980149448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff510155fb0 msgr2=0x7ff5100b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.532+0000 7ff514a00640 1 --2- 172.21.15.7:0/980149448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff510155fb0 0x7ff5100b4f20 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff50800eaa0 tx=0x7ff50800ef60 comp rx=0 tx=0).stop 2024-09-10T21:46:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.532+0000 7ff514a00640 1 -- 172.21.15.7:0/980149448 shutdown_connections 2024-09-10T21:46:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.532+0000 7ff514a00640 1 --2- 172.21.15.7:0/980149448 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff4f0077f70 0x7ff4f007a430 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.532+0000 7ff514a00640 1 --2- 172.21.15.7:0/980149448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff510074450 0x7ff5100b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.532+0000 7ff514a00640 1 --2- 172.21.15.7:0/980149448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff510155fb0 0x7ff5100b4f20 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.532+0000 7ff514a00640 1 --2- 172.21.15.7:0/980149448 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff51006fcf0 0x7ff51014aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.532+0000 7ff514a00640 1 -- 172.21.15.7:0/980149448 >> 172.21.15.7:0/980149448 conn(0x7ff5100b16d0 msgr2=0x7ff51015da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:49.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.532+0000 7ff514a00640 1 -- 172.21.15.7:0/980149448 shutdown_connections 2024-09-10T21:46:49.540 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.532+0000 7ff514a00640 1 -- 172.21.15.7:0/980149448 wait complete. 2024-09-10T21:46:49.548 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:46:49.548 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:46:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.707+0000 7f557b200640 1 -- 172.21.15.7:0/3653339341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f557406f590 msgr2=0x7f5574072ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.707+0000 7f557b200640 1 --2- 172.21.15.7:0/3653339341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f557406f590 0x7f5574072ac0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5564009830 tx=0x7f556402a530 comp rx=0 tx=0).stop 2024-09-10T21:46:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.707+0000 7f557b200640 1 -- 172.21.15.7:0/3653339341 shutdown_connections 2024-09-10T21:46:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.707+0000 7f557b200640 1 --2- 172.21.15.7:0/3653339341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f557406f590 0x7f5574072ac0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.707+0000 7f557b200640 1 -- 172.21.15.7:0/3653339341 >> 172.21.15.7:0/3653339341 conn(0x7f55740b1d70 msgr2=0x7f55740cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.707+0000 7f557b200640 1 -- 172.21.15.7:0/3653339341 shutdown_connections 2024-09-10T21:46:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.707+0000 7f557b200640 1 -- 172.21.15.7:0/3653339341 wait complete. 2024-09-10T21:46:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.709+0000 7f557b200640 1 Processor -- start 2024-09-10T21:46:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.709+0000 7f557b200640 1 -- start start 2024-09-10T21:46:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.710+0000 7f557b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f557406f590 0x7f55741f1d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.710+0000 7f557b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5574146c50 0x7f5574131f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.710+0000 7f557b200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f55741470b0 0x7f55740c7610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.710+0000 7f557b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f55740ae170 con 0x7f5574146c50 2024-09-10T21:46:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.710+0000 7f557b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f55740adff0 con 0x7f557406f590 2024-09-10T21:46:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.710+0000 7f557b200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f55740ae2f0 con 0x7f55741470b0 2024-09-10T21:46:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.710+0000 7f5579400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5574146c50 0x7f5574131f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.710+0000 7f5579400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5574146c50 0x7f5574131f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45026/0 (socket says 172.21.15.7:45026) 2024-09-10T21:46:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.710+0000 7f5579400640 1 -- 172.21.15.7:0/3241603050 learned_addr learned my addr 172.21.15.7:0/3241603050 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.711+0000 7f557a800640 1 --2- 172.21.15.7:0/3241603050 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f55741470b0 0x7f55740c7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.711+0000 7f5579e00640 1 --2- 172.21.15.7:0/3241603050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f557406f590 0x7f55741f1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.711+0000 7f5579400640 1 -- 172.21.15.7:0/3241603050 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f55741470b0 msgr2=0x7f55740c7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.711+0000 7f5579400640 1 --2- 172.21.15.7:0/3241603050 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f55741470b0 0x7f55740c7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.711+0000 7f5579400640 1 -- 172.21.15.7:0/3241603050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f557406f590 msgr2=0x7f55741f1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.711+0000 7f5579400640 1 --2- 172.21.15.7:0/3241603050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f557406f590 0x7f55741f1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.711+0000 7f5579400640 1 -- 172.21.15.7:0/3241603050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55741f5750 con 0x7f5574146c50 2024-09-10T21:46:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.711+0000 7f5579400640 1 --2- 172.21.15.7:0/3241603050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5574146c50 0x7f5574131f90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5564000c00 tx=0x7f5564032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:49.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.711+0000 7f556e000640 1 -- 172.21.15.7:0/3241603050 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564040070 con 0x7f5574146c50 2024-09-10T21:46:49.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.711+0000 7f556e000640 1 -- 172.21.15.7:0/3241603050 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55640038e0 con 0x7f5574146c50 2024-09-10T21:46:49.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.711+0000 7f557b200640 1 -- 172.21.15.7:0/3241603050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5574144290 con 0x7f5574146c50 2024-09-10T21:46:49.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.711+0000 7f556e000640 1 -- 172.21.15.7:0/3241603050 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5564003d30 con 0x7f5574146c50 2024-09-10T21:46:49.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.712+0000 7f557b200640 1 -- 172.21.15.7:0/3241603050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5574144850 con 0x7f5574146c50 2024-09-10T21:46:49.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.713+0000 7f557b200640 1 -- 172.21.15.7:0/3241603050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55740ae7a0 con 0x7f5574146c50 2024-09-10T21:46:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.714+0000 7f556e000640 1 -- 172.21.15.7:0/3241603050 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f556404d030 con 0x7f5574146c50 2024-09-10T21:46:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.715+0000 7f556e000640 1 --2- 172.21.15.7:0/3241603050 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5558077f70 0x7f555807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.715+0000 7f556e000640 1 -- 172.21.15.7:0/3241603050 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f55640bb530 con 0x7f5574146c50 2024-09-10T21:46:49.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.715+0000 7f5579e00640 1 --2- 172.21.15.7:0/3241603050 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5558077f70 0x7f555807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:49.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.716+0000 7f5579e00640 1 --2- 172.21.15.7:0/3241603050 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5558077f70 0x7f555807a430 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5568000f60 tx=0x7f5568009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:49.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.719+0000 7f556e000640 1 -- 172.21.15.7:0/3241603050 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5564083f90 con 0x7f5574146c50 2024-09-10T21:46:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.947+0000 7f557b200640 1 -- 172.21.15.7:0/3241603050 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5574161570 con 0x7f5574146c50 2024-09-10T21:46:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.947+0000 7f556e000640 1 -- 172.21.15.7:0/3241603050 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) ==== 94+0+5236 (secure 0 0 0) 0x7f5564083890 con 0x7f5574146c50 2024-09-10T21:46:49.948 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:46:49.948 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":53,"btime":"2024-09-10T21:46:21:983782+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5884,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3488410478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3488410478},{"type":"v1","addr":"172.21.15.133:6837","nonce":3488410478}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5890,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3660400727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3660400727},{"type":"v1","addr":"172.21.15.7:6835","nonce":3660400727}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2388873704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2388873704},{"type":"v1","addr":"172.21.15.7:6834","nonce":2388873704}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":53,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:46:00.260975+0000","modified":"2024-09-10T21:46:21.673749+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235490":{"gid":235490,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":6,"addr":"172.21.15.133:6836/2622530239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2622530239},{"type":"v1","addr":"172.21.15.133:6836","nonce":2622530239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235490,"qdb_cluster":[235490]},"id":5}]} 2024-09-10T21:46:49.949 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 53 2024-09-10T21:46:49.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.950+0000 7f557b200640 1 -- 172.21.15.7:0/3241603050 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5558077f70 msgr2=0x7f555807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:49.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.950+0000 7f557b200640 1 --2- 172.21.15.7:0/3241603050 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5558077f70 0x7f555807a430 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5568000f60 tx=0x7f5568009040 comp rx=0 tx=0).stop 2024-09-10T21:46:49.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.950+0000 7f557b200640 1 -- 172.21.15.7:0/3241603050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5574146c50 msgr2=0x7f5574131f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:49.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.950+0000 7f557b200640 1 --2- 172.21.15.7:0/3241603050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5574146c50 0x7f5574131f90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5564000c00 tx=0x7f5564032c10 comp rx=0 tx=0).stop 2024-09-10T21:46:49.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.950+0000 7f557b200640 1 -- 172.21.15.7:0/3241603050 shutdown_connections 2024-09-10T21:46:49.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.950+0000 7f557b200640 1 --2- 172.21.15.7:0/3241603050 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5558077f70 0x7f555807a430 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.950+0000 7f557b200640 1 --2- 172.21.15.7:0/3241603050 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f55741470b0 0x7f55740c7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.950+0000 7f557b200640 1 --2- 172.21.15.7:0/3241603050 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5574146c50 0x7f5574131f90 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.950+0000 7f557b200640 1 --2- 172.21.15.7:0/3241603050 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f557406f590 0x7f55741f1d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.950+0000 7f557b200640 1 -- 172.21.15.7:0/3241603050 >> 172.21.15.7:0/3241603050 conn(0x7f55740b1d70 msgr2=0x7f55740b3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.950+0000 7f557b200640 1 -- 172.21.15.7:0/3241603050 shutdown_connections 2024-09-10T21:46:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:49.950+0000 7f557b200640 1 -- 172.21.15.7:0/3241603050 wait complete. 2024-09-10T21:46:49.968 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-09-10T21:46:50.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.129+0000 7f8c6bc00640 1 -- 172.21.15.7:0/3650098103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c640ca960 msgr2=0x7f8c6415e350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.129+0000 7f8c6bc00640 1 --2- 172.21.15.7:0/3650098103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c640ca960 0x7f8c6415e350 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c005fd0 tx=0x7f8c5c02aa10 comp rx=0 tx=0).stop 2024-09-10T21:46:50.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.129+0000 7f8c6bc00640 1 -- 172.21.15.7:0/3650098103 shutdown_connections 2024-09-10T21:46:50.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.130+0000 7f8c6bc00640 1 --2- 172.21.15.7:0/3650098103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c640ca960 0x7f8c6415e350 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.130+0000 7f8c6bc00640 1 --2- 172.21.15.7:0/3650098103 >> v2:172.21.15.133:3300/0 conn(0x7f8c640cec80 0x7f8c640cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.130+0000 7f8c6bc00640 1 -- 172.21.15.7:0/3650098103 >> 172.21.15.7:0/3650098103 conn(0x7f8c640b16d0 msgr2=0x7f8c640b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:50.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.130+0000 7f8c6bc00640 1 -- 172.21.15.7:0/3650098103 shutdown_connections 2024-09-10T21:46:50.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.130+0000 7f8c6bc00640 1 -- 172.21.15.7:0/3650098103 wait complete. 2024-09-10T21:46:50.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.131+0000 7f8c6bc00640 1 Processor -- start 2024-09-10T21:46:50.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.131+0000 7f8c6bc00640 1 -- start start 2024-09-10T21:46:50.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.132+0000 7f8c6bc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c640ca960 0x7f8c6414ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.132+0000 7f8c6bc00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c640cec80 0x7f8c6414b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.132+0000 7f8c6bc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c640cf060 0x7f8c641492d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.132+0000 7f8c6bc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8c640cc720 con 0x7f8c640cf060 2024-09-10T21:46:50.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.132+0000 7f8c6bc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8c640cc5a0 con 0x7f8c640ca960 2024-09-10T21:46:50.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.132+0000 7f8c6bc00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f8c640cc8a0 con 0x7f8c640cec80 2024-09-10T21:46:50.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.133+0000 7f8c69c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c640cf060 0x7f8c641492d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.133+0000 7f8c69c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c640cf060 0x7f8c641492d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45038/0 (socket says 172.21.15.7:45038) 2024-09-10T21:46:50.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.133+0000 7f8c5be00640 1 --2- 172.21.15.7:0/1775356404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c640cec80 0x7f8c6414b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.133+0000 7f8c69200640 1 --2- 172.21.15.7:0/1775356404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c640ca960 0x7f8c6414ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.133+0000 7f8c69c00640 1 -- 172.21.15.7:0/1775356404 learned_addr learned my addr 172.21.15.7:0/1775356404 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:50.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.133+0000 7f8c69c00640 1 -- 172.21.15.7:0/1775356404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c640cec80 msgr2=0x7f8c6414b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.133+0000 7f8c69c00640 1 --2- 172.21.15.7:0/1775356404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c640cec80 0x7f8c6414b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.133+0000 7f8c69c00640 1 -- 172.21.15.7:0/1775356404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c640ca960 msgr2=0x7f8c6414ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.133+0000 7f8c69c00640 1 --2- 172.21.15.7:0/1775356404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c640ca960 0x7f8c6414ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.133+0000 7f8c69c00640 1 -- 172.21.15.7:0/1775356404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c641f7f00 con 0x7f8c640cf060 2024-09-10T21:46:50.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.133+0000 7f8c69c00640 1 --2- 172.21.15.7:0/1775356404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c640cf060 0x7f8c641492d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c006010 tx=0x7f8c5c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:50.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.134+0000 7f8c61600640 1 -- 172.21.15.7:0/1775356404 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c046070 con 0x7f8c640cf060 2024-09-10T21:46:50.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.134+0000 7f8c61600640 1 -- 172.21.15.7:0/1775356404 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c5c004540 con 0x7f8c640cf060 2024-09-10T21:46:50.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.134+0000 7f8c61600640 1 -- 172.21.15.7:0/1775356404 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c033070 con 0x7f8c640cf060 2024-09-10T21:46:50.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.134+0000 7f8c6bc00640 1 -- 172.21.15.7:0/1775356404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c64072840 con 0x7f8c640cf060 2024-09-10T21:46:50.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.134+0000 7f8c6bc00640 1 -- 172.21.15.7:0/1775356404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c641f84a0 con 0x7f8c640cf060 2024-09-10T21:46:50.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.136+0000 7f8c61600640 1 -- 172.21.15.7:0/1775356404 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8c5c040030 con 0x7f8c640cf060 2024-09-10T21:46:50.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.137+0000 7f8c61600640 1 --2- 172.21.15.7:0/1775356404 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8c40077f70 0x7f8c4007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.137+0000 7f8c61600640 1 -- 172.21.15.7:0/1775356404 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f8c5c0bad90 con 0x7f8c640cf060 2024-09-10T21:46:50.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.137+0000 7f8c6ec79740 1 Processor -- start 2024-09-10T21:46:50.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.137+0000 7f8c6ec79740 1 -- start start 2024-09-10T21:46:50.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.138+0000 7f8c69200640 1 --2- 172.21.15.7:0/1775356404 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8c40077f70 0x7f8c4007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.138+0000 7f8c69200640 1 --2- 172.21.15.7:0/1775356404 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8c40077f70 0x7f8c4007a430 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8c54009bd0 tx=0x7f8c54005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:50.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.138+0000 7f8c6ec79740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6bd47a0 0x5578e6bd1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.138+0000 7f8c6ec79740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 0x5578e6bd4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.138+0000 7f8c6ec79740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6bd50c0 0x5578e6ba5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.138+0000 7f8c6ec79740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5578e6b1c750 con 0x5578e6be61c0 2024-09-10T21:46:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.138+0000 7f8c6ec79740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5578e6b0bb80 con 0x5578e6bd47a0 2024-09-10T21:46:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.138+0000 7f8c6ec79740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5578e6b0f990 con 0x5578e6bd50c0 2024-09-10T21:46:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.139+0000 7f8c5be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 0x5578e6bd4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.139+0000 7f8c69200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6bd47a0 0x5578e6bd1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.139+0000 7f8c5be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 0x5578e6bd4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45042/0 (socket says 172.21.15.7:45042) 2024-09-10T21:46:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.139+0000 7f8c5be00640 1 -- 172.21.15.7:0/590752995 learned_addr learned my addr 172.21.15.7:0/590752995 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.139+0000 7f8c69c00640 1 --2- 172.21.15.7:0/590752995 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6bd50c0 0x5578e6ba5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.139+0000 7f8c5be00640 1 -- 172.21.15.7:0/590752995 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6bd50c0 msgr2=0x5578e6ba5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.139+0000 7f8c5be00640 1 --2- 172.21.15.7:0/590752995 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6bd50c0 0x5578e6ba5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.139+0000 7f8c5be00640 1 -- 172.21.15.7:0/590752995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6bd47a0 msgr2=0x5578e6bd1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:50.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.139+0000 7f8c5be00640 1 --2- 172.21.15.7:0/590752995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6bd47a0 0x5578e6bd1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.139+0000 7f8c5be00640 1 -- 172.21.15.7:0/590752995 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578e6a99ad0 con 0x5578e6be61c0 2024-09-10T21:46:50.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.139+0000 7f8c5be00640 1 --2- 172.21.15.7:0/590752995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 0x5578e6bd4b80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c001840 tx=0x7f8c4c007e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d2bea1dc7b6b9f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:50.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.140+0000 7f8c59600640 1 -- 172.21.15.7:0/590752995 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0052e0 con 0x5578e6be61c0 2024-09-10T21:46:50.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.140+0000 7f8c59600640 1 -- 172.21.15.7:0/590752995 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c4c018070 con 0x5578e6be61c0 2024-09-10T21:46:50.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.140+0000 7f8c59600640 1 -- 172.21.15.7:0/590752995 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c00fa10 con 0x5578e6be61c0 2024-09-10T21:46:50.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.140+0000 7f8c6ec79740 1 -- 172.21.15.7:0/590752995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 msgr2=0x5578e6bd4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.140+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/590752995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 0x5578e6bd4b80 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c001840 tx=0x7f8c4c007e30 comp rx=0 tx=0).stop 2024-09-10T21:46:50.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.140+0000 7f8c6ec79740 1 -- 172.21.15.7:0/590752995 shutdown_connections 2024-09-10T21:46:50.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.140+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/590752995 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6bd50c0 0x5578e6ba5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.140+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/590752995 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 0x5578e6bd4b80 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.140+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/590752995 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6bd47a0 0x5578e6bd1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.140+0000 7f8c6ec79740 1 -- 172.21.15.7:0/590752995 >> 172.21.15.7:0/590752995 conn(0x5578e6ba6ce0 msgr2=0x5578e6baa700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:50.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.140+0000 7f8c6ec79740 1 -- 172.21.15.7:0/590752995 shutdown_connections 2024-09-10T21:46:50.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.141+0000 7f8c6ec79740 1 -- 172.21.15.7:0/590752995 wait complete. 2024-09-10T21:46:50.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.141+0000 7f8c6ec79740 1 -- start start 2024-09-10T21:46:50.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.141+0000 7f8c6ec79740 1 Processor -- start 2024-09-10T21:46:50.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.142+0000 7f8c6ec79740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6bd50c0 0x5578e6beff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.142+0000 7f8c6ec79740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6be61c0 0x5578e6bff160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.142+0000 7f8c6ec79740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6be9f90 0x5578e6c00350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.142+0000 7f8c6ec79740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5578e687c700 con 0x5578e6bd50c0 2024-09-10T21:46:50.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.142+0000 7f8c6ec79740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5578e6ab1430 con 0x5578e6be9f90 2024-09-10T21:46:50.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.142+0000 7f8c6ec79740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5578e6bab780 con 0x5578e6be61c0 2024-09-10T21:46:50.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.142+0000 7f8c6ec79740 10 client.0 _ll_register_callbacks cb 0x5578e6ab7600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:46:50.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.143+0000 7f8c69200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6bd50c0 0x5578e6beff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.143+0000 7f8c69200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6bd50c0 0x5578e6beff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45056/0 (socket says 172.21.15.7:45056) 2024-09-10T21:46:50.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.143+0000 7f8c69200640 1 -- 172.21.15.7:0/476019821 learned_addr learned my addr 172.21.15.7:0/476019821 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:50.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.143+0000 7f8c5be00640 1 --2- 172.21.15.7:0/476019821 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6be61c0 0x5578e6bff160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.143+0000 7f8c69c00640 1 --2- 172.21.15.7:0/476019821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6be9f90 0x5578e6c00350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.143+0000 7f8c69200640 1 -- 172.21.15.7:0/476019821 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6be61c0 msgr2=0x5578e6bff160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:50.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.143+0000 7f8c69200640 1 --2- 172.21.15.7:0/476019821 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6be61c0 0x5578e6bff160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.143+0000 7f8c69200640 1 -- 172.21.15.7:0/476019821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6be9f90 msgr2=0x5578e6c00350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:50.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.143+0000 7f8c69200640 1 --2- 172.21.15.7:0/476019821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6be9f90 0x5578e6c00350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.143+0000 7f8c69200640 1 -- 172.21.15.7:0/476019821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578e68e67c0 con 0x5578e6bd50c0 2024-09-10T21:46:50.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.143+0000 7f8c69200640 1 --2- 172.21.15.7:0/476019821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6bd50c0 0x5578e6beff90 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8c540039f0 tx=0x7f8c54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72dd0f8fd2deedb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:50.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.143+0000 7f8c52000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:46:50.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.144+0000 7f8c52000640 1 -- 172.21.15.7:0/476019821 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c5401d6e0 con 0x5578e6bd50c0 2024-09-10T21:46:50.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.144+0000 7f8c52000640 1 -- 172.21.15.7:0/476019821 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f8c54028b60 con 0x5578e6bd50c0 2024-09-10T21:46:50.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.144+0000 7f8c6ec79740 1 -- 172.21.15.7:0/476019821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c4c018070 con 0x5578e6bd50c0 2024-09-10T21:46:50.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.144+0000 7f8c52000640 1 -- 172.21.15.7:0/476019821 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f8c54028d60 con 0x5578e6bd50c0 2024-09-10T21:46:50.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.144+0000 7f8c6ec79740 10 client.6477 fetch_fsmap learned FSMap version 53 2024-09-10T21:46:50.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.144+0000 7f8c6ec79740 1 -- 172.21.15.7:0/476019821 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x5578e6946f50 con 0x5578e6bd50c0 2024-09-10T21:46:50.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.144+0000 7f8c52000640 1 -- 172.21.15.7:0/476019821 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c4c018070 con 0x5578e6bd50c0 2024-09-10T21:46:50.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.144+0000 7f8c52000640 1 -- 172.21.15.7:0/476019821 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7f8c5401d880 con 0x5578e6bd50c0 2024-09-10T21:46:50.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.144+0000 7f8c6ec79740 10 client.6477 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T21:46:50.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.144+0000 7f8c6ec79740 10 client.6477 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:46:50.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.144+0000 7f8c6ec79740 20 client.6477 populate_metadata read hostname 'smithi007' 2024-09-10T21:46:50.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.144+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/476019821 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x5578e6bbd730 0x5578e6bbdb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.145+0000 7f8c6ec79740 4 client.6477 mds_command: new command op to 235490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:46:50.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.145+0000 7f8c6ec79740 1 -- 172.21.15.7:0/476019821 --> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578e6b1e6a0 con 0x5578e6bbd730 2024-09-10T21:46:50.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.145+0000 7f8c5be00640 1 --2- 172.21.15.7:0/476019821 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x5578e6bbd730 0x5578e6bbdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.145+0000 7f8c5be00640 1 --2- 172.21.15.7:0/476019821 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x5578e6bbd730 0x5578e6bbdb10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2582a7d033a49be6 server_cookie=d01d2d7d96b9c4bd in_seq=0 out_seq=0 2024-09-10T21:46:50.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.145+0000 7f8c52000640 10 client.6477 ms_handle_connect on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:50.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.146+0000 7f8c52000640 1 -- 172.21.15.7:0/476019821 <== mds.0 v2:172.21.15.133:6834/2622530239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5578e6b1e6a0 con 0x5578e6bbd730 2024-09-10T21:46:50.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.146+0000 7f8c52000640 10 client.6477 handle_command_reply: tid=0 2024-09-10T21:46:50.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.146+0000 7f8c6ec79740 1 client.6477 shutdown 2024-09-10T21:46:50.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.147+0000 7f8c6ec79740 1 -- 172.21.15.7:0/476019821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6bd50c0 msgr2=0x5578e6beff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.147+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/476019821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6bd50c0 0x5578e6beff90 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8c540039f0 tx=0x7f8c54004730 comp rx=0 tx=0).stop 2024-09-10T21:46:50.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.147+0000 7f8c6ec79740 1 -- 172.21.15.7:0/476019821 shutdown_connections 2024-09-10T21:46:50.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.147+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/476019821 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6be9f90 0x5578e6c00350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.147+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/476019821 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x5578e6bbd730 0x5578e6bbdb10 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.147+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/476019821 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6be61c0 0x5578e6bff160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.147+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/476019821 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6bd50c0 0x5578e6beff90 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.147+0000 7f8c52000640 0 client.6477 ms_handle_reset on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:50.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.147+0000 7f8c6ec79740 1 -- 172.21.15.7:0/476019821 >> 172.21.15.7:0/476019821 conn(0x5578e6bd47a0 msgr2=0x5578e6bd0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:50.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.147+0000 7f8c6ec79740 1 -- 172.21.15.7:0/476019821 shutdown_connections 2024-09-10T21:46:50.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.147+0000 7f8c6ec79740 1 -- 172.21.15.7:0/476019821 wait complete. 2024-09-10T21:46:50.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.147+0000 7f8c6ec79740 20 client.6477 trim_cache size 0 max 16384 2024-09-10T21:46:50.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.161+0000 7f8c6ec79740 1 Processor -- start 2024-09-10T21:46:50.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.161+0000 7f8c6ec79740 1 -- start start 2024-09-10T21:46:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.162+0000 7f8c6ec79740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6bd50c0 0x5578e6bd3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.162+0000 7f8c6ec79740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 0x5578e6ba5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.162+0000 7f8c6ec79740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6be9f90 0x5578e6ba6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.162+0000 7f8c6ec79740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8c64200590 con 0x5578e6be61c0 2024-09-10T21:46:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.162+0000 7f8c6ec79740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8c641f5230 con 0x5578e6be9f90 2024-09-10T21:46:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.162+0000 7f8c6ec79740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f8c641f5880 con 0x5578e6bd50c0 2024-09-10T21:46:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.162+0000 7f8c5be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 0x5578e6ba5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.162+0000 7f8c5be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 0x5578e6ba5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45068/0 (socket says 172.21.15.7:45068) 2024-09-10T21:46:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.162+0000 7f8c5be00640 1 -- 172.21.15.7:0/3568846871 learned_addr learned my addr 172.21.15.7:0/3568846871 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.162+0000 7f8c69200640 1 --2- 172.21.15.7:0/3568846871 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6bd50c0 0x5578e6bd3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.162+0000 7f8c69c00640 1 --2- 172.21.15.7:0/3568846871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6be9f90 0x5578e6ba6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.162+0000 7f8c5be00640 1 -- 172.21.15.7:0/3568846871 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6bd50c0 msgr2=0x5578e6bd3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.162+0000 7f8c5be00640 1 --2- 172.21.15.7:0/3568846871 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6bd50c0 0x5578e6bd3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.162+0000 7f8c5be00640 1 -- 172.21.15.7:0/3568846871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6be9f90 msgr2=0x5578e6ba6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.162+0000 7f8c5be00640 1 --2- 172.21.15.7:0/3568846871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6be9f90 0x5578e6ba6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.162+0000 7f8c5be00640 1 -- 172.21.15.7:0/3568846871 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578e684b660 con 0x5578e6be61c0 2024-09-10T21:46:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.162+0000 7f8c69200640 1 --2- 172.21.15.7:0/3568846871 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6bd50c0 0x5578e6bd3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:46:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.162+0000 7f8c69c00640 1 --2- 172.21.15.7:0/3568846871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6be9f90 0x5578e6ba6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:46:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.163+0000 7f8c5be00640 1 --2- 172.21.15.7:0/3568846871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 0x5578e6ba5fe0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c014000 tx=0x7f8c4c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e417ac338bf9dd7d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.163+0000 7f8c58c00640 1 -- 172.21.15.7:0/3568846871 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c02f070 con 0x5578e6be61c0 2024-09-10T21:46:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.163+0000 7f8c58c00640 1 -- 172.21.15.7:0/3568846871 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c4c018070 con 0x5578e6be61c0 2024-09-10T21:46:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.163+0000 7f8c58c00640 1 -- 172.21.15.7:0/3568846871 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c0035f0 con 0x5578e6be61c0 2024-09-10T21:46:50.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.163+0000 7f8c6ec79740 1 -- 172.21.15.7:0/3568846871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 msgr2=0x5578e6ba5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.163+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/3568846871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 0x5578e6ba5fe0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c014000 tx=0x7f8c4c000bc0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.163+0000 7f8c6ec79740 1 -- 172.21.15.7:0/3568846871 shutdown_connections 2024-09-10T21:46:50.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.163+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/3568846871 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6be9f90 0x5578e6ba6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.163+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/3568846871 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 0x5578e6ba5fe0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.163+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/3568846871 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6bd50c0 0x5578e6bd3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.163+0000 7f8c6ec79740 1 -- 172.21.15.7:0/3568846871 >> 172.21.15.7:0/3568846871 conn(0x5578e6bd47a0 msgr2=0x5578e6bd0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:50.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.164+0000 7f8c6ec79740 1 -- 172.21.15.7:0/3568846871 shutdown_connections 2024-09-10T21:46:50.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.164+0000 7f8c6ec79740 1 -- 172.21.15.7:0/3568846871 wait complete. 2024-09-10T21:46:50.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.164+0000 7f8c6ec79740 1 -- start start 2024-09-10T21:46:50.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.164+0000 7f8c6ec79740 1 Processor -- start 2024-09-10T21:46:50.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.165+0000 7f8c6ec79740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6bd50c0 0x5578e6c72680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.165+0000 7f8c6ec79740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 0x5578e6ba5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.165+0000 7f8c6ec79740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6be9f90 0x5578e6ba6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.165+0000 7f8c6ec79740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5578e6bc8ec0 con 0x5578e6be61c0 2024-09-10T21:46:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.165+0000 7f8c6ec79740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5578e6c00c30 con 0x5578e6be9f90 2024-09-10T21:46:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.165+0000 7f8c6ec79740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5578e6bea3d0 con 0x5578e6bd50c0 2024-09-10T21:46:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.165+0000 7f8c6ec79740 10 client.0 _ll_register_callbacks cb 0x7f8c54003610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:46:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.165+0000 7f8c5be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 0x5578e6ba5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.166+0000 7f8c5be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 0x5578e6ba5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45082/0 (socket says 172.21.15.7:45082) 2024-09-10T21:46:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.166+0000 7f8c5be00640 1 -- 172.21.15.7:0/1057874828 learned_addr learned my addr 172.21.15.7:0/1057874828 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.166+0000 7f8c69c00640 1 --2- 172.21.15.7:0/1057874828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6be9f90 0x5578e6ba6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.166+0000 7f8c69200640 1 --2- 172.21.15.7:0/1057874828 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6bd50c0 0x5578e6c72680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.166+0000 7f8c5be00640 1 -- 172.21.15.7:0/1057874828 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6bd50c0 msgr2=0x5578e6c72680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.166+0000 7f8c5be00640 1 --2- 172.21.15.7:0/1057874828 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6bd50c0 0x5578e6c72680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.166+0000 7f8c5be00640 1 -- 172.21.15.7:0/1057874828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6be9f90 msgr2=0x5578e6ba6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.166+0000 7f8c5be00640 1 --2- 172.21.15.7:0/1057874828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6be9f90 0x5578e6ba6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.166+0000 7f8c5be00640 1 -- 172.21.15.7:0/1057874828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578e6ad8630 con 0x5578e6be61c0 2024-09-10T21:46:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.166+0000 7f8c69c00640 1 --2- 172.21.15.7:0/1057874828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6be9f90 0x5578e6ba6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.166+0000 7f8c69200640 1 --2- 172.21.15.7:0/1057874828 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6bd50c0 0x5578e6c72680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.166+0000 7f8c5be00640 1 --2- 172.21.15.7:0/1057874828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 0x5578e6ba5fe0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c02d040 tx=0x7f8c4c00f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745fed6a3a39dfe7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.166+0000 7f8c52000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:46:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.166+0000 7f8c52000640 1 -- 172.21.15.7:0/1057874828 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c4c02f070 con 0x5578e6be61c0 2024-09-10T21:46:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.166+0000 7f8c52000640 1 -- 172.21.15.7:0/1057874828 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f8c4c0035f0 con 0x5578e6be61c0 2024-09-10T21:46:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.166+0000 7f8c6ec79740 1 -- 172.21.15.7:0/1057874828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c4c018070 con 0x5578e6be61c0 2024-09-10T21:46:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.167+0000 7f8c52000640 1 -- 172.21.15.7:0/1057874828 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f8c4c00fe50 con 0x5578e6be61c0 2024-09-10T21:46:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.167+0000 7f8c6ec79740 10 client.6483 fetch_fsmap learned FSMap version 53 2024-09-10T21:46:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.167+0000 7f8c6ec79740 1 -- 172.21.15.7:0/1057874828 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x5578e6ba4810 con 0x5578e6be61c0 2024-09-10T21:46:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.167+0000 7f8c52000640 1 -- 172.21.15.7:0/1057874828 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c4c018070 con 0x5578e6be61c0 2024-09-10T21:46:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.167+0000 7f8c52000640 1 -- 172.21.15.7:0/1057874828 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7f8c4c0037f0 con 0x5578e6be61c0 2024-09-10T21:46:50.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.167+0000 7f8c6ec79740 10 client.6483 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T21:46:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.167+0000 7f8c6ec79740 10 client.6483 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:46:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.167+0000 7f8c6ec79740 20 client.6483 populate_metadata read hostname 'smithi007' 2024-09-10T21:46:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.167+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/1057874828 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x5578e6c731f0 0x5578e6c735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.167+0000 7f8c6ec79740 4 client.6483 mds_command: new command op to 235490 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-10T21:46:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.167+0000 7f8c6ec79740 1 -- 172.21.15.7:0/1057874828 --> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5578e6b1a150 con 0x5578e6c731f0 2024-09-10T21:46:50.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.168+0000 7f8c69200640 1 --2- 172.21.15.7:0/1057874828 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x5578e6c731f0 0x5578e6c735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.168+0000 7f8c69200640 1 --2- 172.21.15.7:0/1057874828 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x5578e6c731f0 0x5578e6c735d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bca02f727e981d45 server_cookie=58d9467ddde23c28 in_seq=0 out_seq=0 2024-09-10T21:46:50.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.168+0000 7f8c52000640 10 client.6483 ms_handle_connect on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:50.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.168+0000 7f8c52000640 1 -- 172.21.15.7:0/1057874828 <== mds.0 v2:172.21.15.133:6834/2622530239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+237 (crc 0 0 0) 0x5578e6b1a150 con 0x5578e6c731f0 2024-09-10T21:46:50.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.168+0000 7f8c52000640 10 client.6483 handle_command_reply: tid=0 2024-09-10T21:46:50.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.169+0000 7f8c6ec79740 1 client.6483 shutdown 2024-09-10T21:46:50.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.169+0000 7f8c6ec79740 1 -- 172.21.15.7:0/1057874828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 msgr2=0x5578e6ba5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.169+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/1057874828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 0x5578e6ba5fe0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f8c4c02d040 tx=0x7f8c4c00f7b0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.169+0000 7f8c6ec79740 1 -- 172.21.15.7:0/1057874828 shutdown_connections 2024-09-10T21:46:50.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.169+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/1057874828 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578e6be9f90 0x5578e6ba6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.169+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/1057874828 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x5578e6c731f0 0x5578e6c735d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.169+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/1057874828 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578e6be61c0 0x5578e6ba5fe0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.169+0000 7f8c6ec79740 1 --2- 172.21.15.7:0/1057874828 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578e6bd50c0 0x5578e6c72680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.169+0000 7f8c52000640 0 client.6483 ms_handle_reset on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:50.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.169+0000 7f8c6ec79740 1 -- 172.21.15.7:0/1057874828 >> 172.21.15.7:0/1057874828 conn(0x5578e6bd47a0 msgr2=0x5578e6be5a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:50.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.169+0000 7f8c6ec79740 1 -- 172.21.15.7:0/1057874828 shutdown_connections 2024-09-10T21:46:50.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.169+0000 7f8c6ec79740 1 -- 172.21.15.7:0/1057874828 wait complete. 2024-09-10T21:46:50.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.169+0000 7f8c6ec79740 20 client.6483 trim_cache size 0 max 16384 2024-09-10T21:46:50.175 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:46:50.175 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-09-10T21:46:50.175 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:46:50.175 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "dentry", 2024-09-10T21:46:50.175 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 2381848182, 2024-09-10T21:46:50.175 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627776, 2024-09-10T21:46:50.175 INFO:teuthology.orchestra.run.smithi007.stdout: "frag": "*", 2024-09-10T21:46:50.175 INFO:teuthology.orchestra.run.smithi007.stdout: "dname": "file_to_be_damaged", 2024-09-10T21:46:50.175 INFO:teuthology.orchestra.run.smithi007.stdout: "snap_id": "head", 2024-09-10T21:46:50.175 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/subdir/file_to_be_damaged" 2024-09-10T21:46:50.175 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:46:50.175 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-09-10T21:46:50.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.170+0000 7f8c6bc00640 1 -- 172.21.15.7:0/1775356404 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8c40077f70 msgr2=0x7f8c4007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.170+0000 7f8c6bc00640 1 --2- 172.21.15.7:0/1775356404 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8c40077f70 0x7f8c4007a430 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8c54009bd0 tx=0x7f8c54005f90 comp rx=0 tx=0).stop 2024-09-10T21:46:50.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.170+0000 7f8c6bc00640 1 -- 172.21.15.7:0/1775356404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c640cf060 msgr2=0x7f8c641492d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.170+0000 7f8c6bc00640 1 --2- 172.21.15.7:0/1775356404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c640cf060 0x7f8c641492d0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c006010 tx=0x7f8c5c003f10 comp rx=0 tx=0).stop 2024-09-10T21:46:50.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.170+0000 7f8c6bc00640 1 -- 172.21.15.7:0/1775356404 shutdown_connections 2024-09-10T21:46:50.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.170+0000 7f8c6bc00640 1 --2- 172.21.15.7:0/1775356404 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8c40077f70 0x7f8c4007a430 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.170+0000 7f8c6bc00640 1 --2- 172.21.15.7:0/1775356404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c640cf060 0x7f8c641492d0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.170+0000 7f8c6bc00640 1 --2- 172.21.15.7:0/1775356404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c640cec80 0x7f8c6414b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.170+0000 7f8c6bc00640 1 --2- 172.21.15.7:0/1775356404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c640ca960 0x7f8c6414ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.170+0000 7f8c6bc00640 1 -- 172.21.15.7:0/1775356404 >> 172.21.15.7:0/1775356404 conn(0x7f8c640b16d0 msgr2=0x7f8c640b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.170+0000 7f8c6bc00640 1 -- 172.21.15.7:0/1775356404 shutdown_connections 2024-09-10T21:46:50.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.170+0000 7f8c6bc00640 1 -- 172.21.15.7:0/1775356404 wait complete. 2024-09-10T21:46:50.186 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:46:50.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.345+0000 7f0241200640 1 -- 172.21.15.7:0/4128339174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f023c15bc90 msgr2=0x7f023c0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.345+0000 7f0241200640 1 --2- 172.21.15.7:0/4128339174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f023c15bc90 0x7f023c0ccef0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f023c15fa90 tx=0x7f022c02a470 comp rx=0 tx=0).stop 2024-09-10T21:46:50.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.345+0000 7f0241200640 1 -- 172.21.15.7:0/4128339174 shutdown_connections 2024-09-10T21:46:50.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.346+0000 7f0241200640 1 --2- 172.21.15.7:0/4128339174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f023c15fda0 0x7f023c1601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.346+0000 7f0241200640 1 --2- 172.21.15.7:0/4128339174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f023c15bc90 0x7f023c0ccef0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.346+0000 7f0241200640 1 -- 172.21.15.7:0/4128339174 >> 172.21.15.7:0/4128339174 conn(0x7f023c0b16d0 msgr2=0x7f023c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:50.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.346+0000 7f0241200640 1 -- 172.21.15.7:0/4128339174 shutdown_connections 2024-09-10T21:46:50.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.346+0000 7f0241200640 1 -- 172.21.15.7:0/4128339174 wait complete. 2024-09-10T21:46:50.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.347+0000 7f0241200640 1 Processor -- start 2024-09-10T21:46:50.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.347+0000 7f0241200640 1 -- start start 2024-09-10T21:46:50.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.348+0000 7f0241200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f023c0cd750 0x7f023c15ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.348+0000 7f0241200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f023c15bc90 0x7f023c1f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.348+0000 7f0241200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f023c15fda0 0x7f023c0c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.348+0000 7f0241200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f023c06fe70 con 0x7f023c0cd750 2024-09-10T21:46:50.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.348+0000 7f0241200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f023c06fcf0 con 0x7f023c15fda0 2024-09-10T21:46:50.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.348+0000 7f0241200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f023c06fff0 con 0x7f023c15bc90 2024-09-10T21:46:50.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.348+0000 7f0239e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f023c0cd750 0x7f023c15ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.349+0000 7f0239e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f023c0cd750 0x7f023c15ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45096/0 (socket says 172.21.15.7:45096) 2024-09-10T21:46:50.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.349+0000 7f0239e00640 1 -- 172.21.15.7:0/787197729 learned_addr learned my addr 172.21.15.7:0/787197729 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:50.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.349+0000 7f0231400640 1 --2- 172.21.15.7:0/787197729 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f023c15bc90 0x7f023c1f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.349+0000 7f023a800640 1 --2- 172.21.15.7:0/787197729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f023c15fda0 0x7f023c0c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.349+0000 7f0239e00640 1 -- 172.21.15.7:0/787197729 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f023c15bc90 msgr2=0x7f023c1f4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.349+0000 7f0239e00640 1 --2- 172.21.15.7:0/787197729 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f023c15bc90 0x7f023c1f4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.349+0000 7f0239e00640 1 -- 172.21.15.7:0/787197729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f023c15fda0 msgr2=0x7f023c0c5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.349+0000 7f0239e00640 1 --2- 172.21.15.7:0/787197729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f023c15fda0 0x7f023c0c5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.349+0000 7f0239e00640 1 -- 172.21.15.7:0/787197729 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f023c0ca570 con 0x7f023c0cd750 2024-09-10T21:46:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.349+0000 7f0231400640 1 --2- 172.21.15.7:0/787197729 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f023c15bc90 0x7f023c1f4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.349+0000 7f023a800640 1 --2- 172.21.15.7:0/787197729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f023c15fda0 0x7f023c0c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.349+0000 7f0239e00640 1 --2- 172.21.15.7:0/787197729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f023c0cd750 0x7f023c15ecb0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f022c02aa60 tx=0x7f022c032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.350+0000 7f0232a00640 1 -- 172.21.15.7:0/787197729 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c040070 con 0x7f023c0cd750 2024-09-10T21:46:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.350+0000 7f0232a00640 1 -- 172.21.15.7:0/787197729 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f022c005310 con 0x7f023c0cd750 2024-09-10T21:46:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.350+0000 7f0241200640 1 -- 172.21.15.7:0/787197729 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f023c0ca850 con 0x7f023c0cd750 2024-09-10T21:46:50.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.350+0000 7f0232a00640 1 -- 172.21.15.7:0/787197729 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f022c005c90 con 0x7f023c0cd750 2024-09-10T21:46:50.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.350+0000 7f0241200640 1 -- 172.21.15.7:0/787197729 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f023c0cae10 con 0x7f023c0cd750 2024-09-10T21:46:50.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.351+0000 7f0241200640 1 -- 172.21.15.7:0/787197729 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f023c0cb240 con 0x7f023c0cd750 2024-09-10T21:46:50.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.352+0000 7f0232a00640 1 -- 172.21.15.7:0/787197729 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f022c0049b0 con 0x7f023c0cd750 2024-09-10T21:46:50.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.353+0000 7f0232a00640 1 --2- 172.21.15.7:0/787197729 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0218077fa0 0x7f021807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.353+0000 7f0232a00640 1 -- 172.21.15.7:0/787197729 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f022c0bb450 con 0x7f023c0cd750 2024-09-10T21:46:50.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.353+0000 7f0231400640 1 --2- 172.21.15.7:0/787197729 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0218077fa0 0x7f021807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.354+0000 7f0231400640 1 --2- 172.21.15.7:0/787197729 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0218077fa0 0x7f021807a460 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f023c159e80 tx=0x7f0224009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:50.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.357+0000 7f0232a00640 1 -- 172.21.15.7:0/787197729 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f022c083da0 con 0x7f023c0cd750 2024-09-10T21:46:50.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.595+0000 7f0241200640 1 -- 172.21.15.7:0/787197729 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f023c0d09c0 con 0x7f023c0cd750 2024-09-10T21:46:50.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.596+0000 7f0232a00640 1 -- 172.21.15.7:0/787197729 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7f022c0836a0 con 0x7f023c0cd750 2024-09-10T21:46:50.597 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:46:50.597 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:46:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.599+0000 7f0241200640 1 -- 172.21.15.7:0/787197729 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0218077fa0 msgr2=0x7f021807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.599+0000 7f0241200640 1 --2- 172.21.15.7:0/787197729 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0218077fa0 0x7f021807a460 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f023c159e80 tx=0x7f0224009040 comp rx=0 tx=0).stop 2024-09-10T21:46:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.599+0000 7f0241200640 1 -- 172.21.15.7:0/787197729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f023c0cd750 msgr2=0x7f023c15ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.599+0000 7f0241200640 1 --2- 172.21.15.7:0/787197729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f023c0cd750 0x7f023c15ecb0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f022c02aa60 tx=0x7f022c032c10 comp rx=0 tx=0).stop 2024-09-10T21:46:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.599+0000 7f0241200640 1 -- 172.21.15.7:0/787197729 shutdown_connections 2024-09-10T21:46:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.599+0000 7f0241200640 1 --2- 172.21.15.7:0/787197729 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0218077fa0 0x7f021807a460 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.599+0000 7f0241200640 1 --2- 172.21.15.7:0/787197729 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f023c15fda0 0x7f023c0c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.599+0000 7f0241200640 1 --2- 172.21.15.7:0/787197729 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f023c15bc90 0x7f023c1f4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.599+0000 7f0241200640 1 --2- 172.21.15.7:0/787197729 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f023c0cd750 0x7f023c15ecb0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.599+0000 7f0241200640 1 -- 172.21.15.7:0/787197729 >> 172.21.15.7:0/787197729 conn(0x7f023c0b16d0 msgr2=0x7f023c072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.599+0000 7f0241200640 1 -- 172.21.15.7:0/787197729 shutdown_connections 2024-09-10T21:46:50.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.599+0000 7f0241200640 1 -- 172.21.15.7:0/787197729 wait complete. 2024-09-10T21:46:50.616 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:46:50.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:50.616 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start /subdir repair,recursive 2024-09-10T21:46:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.781+0000 7f0648c00640 1 -- 172.21.15.7:0/3742352022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0628003a50 msgr2=0x7f0628003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.781+0000 7f0648c00640 1 --2- 172.21.15.7:0/3742352022 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0628003a50 0x7f0628003e70 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f063400bb20 tx=0x7f0628005830 comp rx=0 tx=0).stop 2024-09-10T21:46:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.781+0000 7f0642800640 1 -- 172.21.15.7:0/3742352022 reap_dead start 2024-09-10T21:46:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.781+0000 7f0648c00640 1 -- 172.21.15.7:0/3742352022 shutdown_connections 2024-09-10T21:46:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.781+0000 7f0648c00640 1 -- 172.21.15.7:0/3742352022 >> 172.21.15.7:0/3742352022 conn(0x7f06440b1c10 msgr2=0x7f06440b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.781+0000 7f0648c00640 1 -- 172.21.15.7:0/3742352022 shutdown_connections 2024-09-10T21:46:50.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.781+0000 7f0648c00640 1 -- 172.21.15.7:0/3742352022 wait complete. 2024-09-10T21:46:50.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.782+0000 7f0648c00640 1 Processor -- start 2024-09-10T21:46:50.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.783+0000 7f0648c00640 1 -- start start 2024-09-10T21:46:50.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.783+0000 7f0648c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0628003a50 0x7f0644149440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.783+0000 7f0648c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0644159b30 0x7f0644073850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.783+0000 7f0648c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0644159fb0 0x7f0644073d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.783+0000 7f0648c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f06440703b0 con 0x7f0644159fb0 2024-09-10T21:46:50.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.783+0000 7f0648c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0644070230 con 0x7f0644159b30 2024-09-10T21:46:50.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.783+0000 7f0648c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f0644070530 con 0x7f0628003a50 2024-09-10T21:46:50.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.784+0000 7f0642800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0644159fb0 0x7f0644073d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.784+0000 7f0642800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0644159fb0 0x7f0644073d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45112/0 (socket says 172.21.15.7:45112) 2024-09-10T21:46:50.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.784+0000 7f0642800640 1 -- 172.21.15.7:0/256089704 learned_addr learned my addr 172.21.15.7:0/256089704 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:50.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.784+0000 7f0641e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0628003a50 0x7f0644149440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.784+0000 7f0641400640 1 --2- 172.21.15.7:0/256089704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0644159b30 0x7f0644073850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.784+0000 7f0642800640 1 -- 172.21.15.7:0/256089704 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0628003a50 msgr2=0x7f0644149440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.784+0000 7f0642800640 1 --2- 172.21.15.7:0/256089704 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0628003a50 0x7f0644149440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.784+0000 7f0642800640 1 -- 172.21.15.7:0/256089704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0644159b30 msgr2=0x7f0644073850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.784+0000 7f0642800640 1 --2- 172.21.15.7:0/256089704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0644159b30 0x7f0644073850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.784+0000 7f0642800640 1 -- 172.21.15.7:0/256089704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f064415a4a0 con 0x7f0644159fb0 2024-09-10T21:46:50.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.784+0000 7f0642800640 1 --2- 172.21.15.7:0/256089704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0644159fb0 0x7f0644073d90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f06440ad2d0 tx=0x7f063c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:50.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.785+0000 7f063a000640 1 -- 172.21.15.7:0/256089704 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c015070 con 0x7f0644159fb0 2024-09-10T21:46:50.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.785+0000 7f063a000640 1 -- 172.21.15.7:0/256089704 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f063c00cb40 con 0x7f0644159fb0 2024-09-10T21:46:50.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.785+0000 7f0648c00640 1 -- 172.21.15.7:0/256089704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06441f7f10 con 0x7f0644159fb0 2024-09-10T21:46:50.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.785+0000 7f063a000640 1 -- 172.21.15.7:0/256089704 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c011460 con 0x7f0644159fb0 2024-09-10T21:46:50.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.785+0000 7f0648c00640 1 -- 172.21.15.7:0/256089704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06441f84d0 con 0x7f0644159fb0 2024-09-10T21:46:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.787+0000 7f063a000640 1 -- 172.21.15.7:0/256089704 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f063c011600 con 0x7f0644159fb0 2024-09-10T21:46:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.789+0000 7f063a000640 1 --2- 172.21.15.7:0/256089704 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0624077fa0 0x7f062407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.789+0000 7f064ba04740 1 Processor -- start 2024-09-10T21:46:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.789+0000 7f063a000640 1 -- 172.21.15.7:0/256089704 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f063c09a480 con 0x7f0644159fb0 2024-09-10T21:46:50.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.789+0000 7f064ba04740 1 -- start start 2024-09-10T21:46:50.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.790+0000 7f0641e00640 1 --2- 172.21.15.7:0/256089704 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0624077fa0 0x7f062407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.790+0000 7f064ba04740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759bed00 0x55b4759bf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.790+0000 7f064ba04740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b4759d0720 0x55b4759bae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.790+0000 7f064ba04740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b475996290 0x55b4759bb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.790+0000 7f064ba04740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b4758f40c0 con 0x55b4759d0720 2024-09-10T21:46:50.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.790+0000 7f064ba04740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b475907f90 con 0x55b4759bed00 2024-09-10T21:46:50.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.790+0000 7f064ba04740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55b4758f73c0 con 0x55b475996290 2024-09-10T21:46:50.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.790+0000 7f0641e00640 1 --2- 172.21.15.7:0/256089704 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0624077fa0 0x7f062407a460 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0634036000 tx=0x7f063403c040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:50.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.791+0000 7f0641400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b4759d0720 0x55b4759bae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.791+0000 7f0641400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b4759d0720 0x55b4759bae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45126/0 (socket says 172.21.15.7:45126) 2024-09-10T21:46:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.791+0000 7f0641400640 1 -- 172.21.15.7:0/3744235060 learned_addr learned my addr 172.21.15.7:0/3744235060 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.791+0000 7f0641e00640 1 --2- 172.21.15.7:0/3744235060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759bed00 0x55b4759bf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.791+0000 7f0642800640 1 --2- 172.21.15.7:0/3744235060 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b475996290 0x55b4759bb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.791+0000 7f0641400640 1 -- 172.21.15.7:0/3744235060 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b475996290 msgr2=0x55b4759bb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.791+0000 7f0641400640 1 --2- 172.21.15.7:0/3744235060 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b475996290 0x55b4759bb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.791+0000 7f0641400640 1 -- 172.21.15.7:0/3744235060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759bed00 msgr2=0x55b4759bf0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.791+0000 7f0641400640 1 --2- 172.21.15.7:0/3744235060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759bed00 0x55b4759bf0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.791+0000 7f0641400640 1 -- 172.21.15.7:0/3744235060 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4756cda50 con 0x55b4759d0720 2024-09-10T21:46:50.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.791+0000 7f0641400640 1 --2- 172.21.15.7:0/3744235060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b4759d0720 0x55b4759bae90 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0630009c40 tx=0x7f0630012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75fa0243896ef345 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:50.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.792+0000 7f062e600640 1 -- 172.21.15.7:0/3744235060 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630009df0 con 0x55b4759d0720 2024-09-10T21:46:50.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.792+0000 7f062e600640 1 -- 172.21.15.7:0/3744235060 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06300120d0 con 0x55b4759d0720 2024-09-10T21:46:50.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.792+0000 7f062e600640 1 -- 172.21.15.7:0/3744235060 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0630010630 con 0x55b4759d0720 2024-09-10T21:46:50.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.792+0000 7f064ba04740 1 -- 172.21.15.7:0/3744235060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b4759d0720 msgr2=0x55b4759bae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.792+0000 7f064ba04740 1 --2- 172.21.15.7:0/3744235060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b4759d0720 0x55b4759bae90 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f0630009c40 tx=0x7f0630012820 comp rx=0 tx=0).stop 2024-09-10T21:46:50.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.792+0000 7f064ba04740 1 -- 172.21.15.7:0/3744235060 shutdown_connections 2024-09-10T21:46:50.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.792+0000 7f064ba04740 1 --2- 172.21.15.7:0/3744235060 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b475996290 0x55b4759bb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.792+0000 7f064ba04740 1 --2- 172.21.15.7:0/3744235060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b4759d0720 0x55b4759bae90 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.792+0000 7f064ba04740 1 --2- 172.21.15.7:0/3744235060 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759bed00 0x55b4759bf0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.792+0000 7f064ba04740 1 -- 172.21.15.7:0/3744235060 >> 172.21.15.7:0/3744235060 conn(0x55b4759912e0 msgr2=0x55b475994d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:50.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.792+0000 7f064ba04740 1 -- 172.21.15.7:0/3744235060 shutdown_connections 2024-09-10T21:46:50.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.792+0000 7f064ba04740 1 -- 172.21.15.7:0/3744235060 wait complete. 2024-09-10T21:46:50.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.793+0000 7f064ba04740 1 -- start start 2024-09-10T21:46:50.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.793+0000 7f064ba04740 1 Processor -- start 2024-09-10T21:46:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.794+0000 7f064ba04740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759bed00 0x55b4759ed020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.794+0000 7f064ba04740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b4759d0720 0x55b4759dcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.794+0000 7f064ba04740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b4759ed6f0 0x55b4759edb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.794+0000 7f064ba04740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b47589caf0 con 0x55b4759ed6f0 2024-09-10T21:46:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.794+0000 7f064ba04740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55b4758fb1d0 con 0x55b4759bed00 2024-09-10T21:46:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.794+0000 7f064ba04740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55b475667a30 con 0x55b4759d0720 2024-09-10T21:46:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.794+0000 7f064ba04740 10 client.0 _ll_register_callbacks cb 0x55b4758a2b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:46:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.794+0000 7f0642800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b4759ed6f0 0x55b4759edb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.794+0000 7f0642800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b4759ed6f0 0x55b4759edb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45134/0 (socket says 172.21.15.7:45134) 2024-09-10T21:46:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.795+0000 7f0642800640 1 -- 172.21.15.7:0/1231755877 learned_addr learned my addr 172.21.15.7:0/1231755877 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.795+0000 7f0641e00640 1 --2- 172.21.15.7:0/1231755877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759bed00 0x55b4759ed020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.795+0000 7f0641400640 1 --2- 172.21.15.7:0/1231755877 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b4759d0720 0x55b4759dcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.795+0000 7f0642800640 1 -- 172.21.15.7:0/1231755877 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b4759d0720 msgr2=0x55b4759dcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:50.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.795+0000 7f0642800640 1 --2- 172.21.15.7:0/1231755877 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b4759d0720 0x55b4759dcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.795+0000 7f0642800640 1 -- 172.21.15.7:0/1231755877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759bed00 msgr2=0x55b4759ed020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:50.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.795+0000 7f0642800640 1 --2- 172.21.15.7:0/1231755877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759bed00 0x55b4759ed020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.795+0000 7f0642800640 1 -- 172.21.15.7:0/1231755877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b475636990 con 0x55b4759ed6f0 2024-09-10T21:46:50.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.795+0000 7f0641400640 1 --2- 172.21.15.7:0/1231755877 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b4759d0720 0x55b4759dcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:50.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.795+0000 7f0642800640 1 --2- 172.21.15.7:0/1231755877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b4759ed6f0 0x55b4759edb70 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f063c002020 tx=0x7f063c062310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc500dc806363fd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:50.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.795+0000 7f0622a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:46:50.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.795+0000 7f0622a00640 1 -- 172.21.15.7:0/1231755877 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063c01f040 con 0x55b4759ed6f0 2024-09-10T21:46:50.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.795+0000 7f0622a00640 1 -- 172.21.15.7:0/1231755877 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f063c061af0 con 0x55b4759ed6f0 2024-09-10T21:46:50.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.795+0000 7f064ba04740 1 -- 172.21.15.7:0/1231755877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06300120d0 con 0x55b4759ed6f0 2024-09-10T21:46:50.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.796+0000 7f0622a00640 1 -- 172.21.15.7:0/1231755877 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f063c0672c0 con 0x55b4759ed6f0 2024-09-10T21:46:50.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.796+0000 7f064ba04740 10 client.6501 fetch_fsmap learned FSMap version 53 2024-09-10T21:46:50.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.796+0000 7f064ba04740 1 -- 172.21.15.7:0/1231755877 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55b47572e1e0 con 0x55b4759ed6f0 2024-09-10T21:46:50.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.796+0000 7f0622a00640 1 -- 172.21.15.7:0/1231755877 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06300120d0 con 0x55b4759ed6f0 2024-09-10T21:46:50.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.796+0000 7f0622a00640 1 -- 172.21.15.7:0/1231755877 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7f063c098ce0 con 0x55b4759ed6f0 2024-09-10T21:46:50.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.796+0000 7f064ba04740 10 client.6501 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T21:46:50.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.796+0000 7f064ba04740 10 client.6501 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-10T21:46:50.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.796+0000 7f064ba04740 20 client.6501 populate_metadata read hostname 'smithi007' 2024-09-10T21:46:50.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.796+0000 7f064ba04740 1 --2- 172.21.15.7:0/1231755877 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55b4759ea580 0x55b4759ea990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.796+0000 7f064ba04740 4 client.6501 mds_command: new command op to 235490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:46:50.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.796+0000 7f064ba04740 1 -- 172.21.15.7:0/1231755877 --> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b475909ee0 con 0x55b4759ea580 2024-09-10T21:46:50.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.796+0000 7f0641e00640 1 --2- 172.21.15.7:0/1231755877 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55b4759ea580 0x55b4759ea990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.797+0000 7f0641e00640 1 --2- 172.21.15.7:0/1231755877 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55b4759ea580 0x55b4759ea990 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9613fa761af32d98 server_cookie=742d35fa37b28953 in_seq=0 out_seq=0 2024-09-10T21:46:50.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.797+0000 7f0622a00640 10 client.6501 ms_handle_connect on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.798+0000 7f0622a00640 1 -- 172.21.15.7:0/1231755877 <== mds.0 v2:172.21.15.133:6834/2622530239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b475909ee0 con 0x55b4759ea580 2024-09-10T21:46:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.798+0000 7f0622a00640 10 client.6501 handle_command_reply: tid=0 2024-09-10T21:46:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.798+0000 7f064ba04740 1 client.6501 shutdown 2024-09-10T21:46:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.798+0000 7f064ba04740 1 -- 172.21.15.7:0/1231755877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b4759ed6f0 msgr2=0x55b4759edb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.798+0000 7f064ba04740 1 --2- 172.21.15.7:0/1231755877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b4759ed6f0 0x55b4759edb70 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f063c002020 tx=0x7f063c062310 comp rx=0 tx=0).stop 2024-09-10T21:46:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.798+0000 7f064ba04740 1 -- 172.21.15.7:0/1231755877 shutdown_connections 2024-09-10T21:46:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.798+0000 7f064ba04740 1 --2- 172.21.15.7:0/1231755877 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55b4759ea580 0x55b4759ea990 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.798+0000 7f064ba04740 1 --2- 172.21.15.7:0/1231755877 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b4759ed6f0 0x55b4759edb70 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.798+0000 7f064ba04740 1 --2- 172.21.15.7:0/1231755877 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b4759d0720 0x55b4759dcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.798+0000 7f064ba04740 1 --2- 172.21.15.7:0/1231755877 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759bed00 0x55b4759ed020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.798+0000 7f0622a00640 0 client.6501 ms_handle_reset on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.798+0000 7f064ba04740 1 -- 172.21.15.7:0/1231755877 >> 172.21.15.7:0/1231755877 conn(0x55b475996290 msgr2=0x55b475990490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.799+0000 7f064ba04740 1 -- 172.21.15.7:0/1231755877 shutdown_connections 2024-09-10T21:46:50.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.799+0000 7f064ba04740 1 -- 172.21.15.7:0/1231755877 wait complete. 2024-09-10T21:46:50.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.799+0000 7f064ba04740 20 client.6501 trim_cache size 0 max 16384 2024-09-10T21:46:50.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.817+0000 7f064ba04740 1 Processor -- start 2024-09-10T21:46:50.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.818+0000 7f064ba04740 1 -- start start 2024-09-10T21:46:50.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.818+0000 7f064ba04740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b475996290 0x55b4759ee160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.818+0000 7f064ba04740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b4759bed00 0x55b4759903a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.818+0000 7f064ba04740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759d0720 0x55b475a55e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.818+0000 7f064ba04740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0644201940 con 0x55b475996290 2024-09-10T21:46:50.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.818+0000 7f064ba04740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f06442009e0 con 0x55b4759d0720 2024-09-10T21:46:50.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.818+0000 7f064ba04740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f0644158bd0 con 0x55b4759bed00 2024-09-10T21:46:50.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.819+0000 7f0641e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b475996290 0x55b4759ee160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.819+0000 7f0641e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b475996290 0x55b4759ee160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45146/0 (socket says 172.21.15.7:45146) 2024-09-10T21:46:50.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.819+0000 7f0641e00640 1 -- 172.21.15.7:0/3758434605 learned_addr learned my addr 172.21.15.7:0/3758434605 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:50.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.819+0000 7f0641400640 1 --2- 172.21.15.7:0/3758434605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b4759bed00 0x55b4759903a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.819+0000 7f0642800640 1 --2- 172.21.15.7:0/3758434605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759d0720 0x55b475a55e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.819+0000 7f0641e00640 1 -- 172.21.15.7:0/3758434605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b4759bed00 msgr2=0x55b4759903a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:50.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.819+0000 7f0641e00640 1 --2- 172.21.15.7:0/3758434605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b4759bed00 0x55b4759903a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.819+0000 7f0641e00640 1 -- 172.21.15.7:0/3758434605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759d0720 msgr2=0x55b475a55e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:50.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.819+0000 7f0641e00640 1 --2- 172.21.15.7:0/3758434605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759d0720 0x55b475a55e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.819+0000 7f0641e00640 1 -- 172.21.15.7:0/3758434605 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b4756cbd50 con 0x55b475996290 2024-09-10T21:46:50.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.819+0000 7f0641e00640 1 --2- 172.21.15.7:0/3758434605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b475996290 0x55b4759ee160 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0634034040 tx=0x7f0628005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c8e2ac201265f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:50.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.819+0000 7f062dc00640 1 -- 172.21.15.7:0/3758434605 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063405e070 con 0x55b475996290 2024-09-10T21:46:50.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.819+0000 7f062dc00640 1 -- 172.21.15.7:0/3758434605 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0634044a50 con 0x55b475996290 2024-09-10T21:46:50.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.819+0000 7f062dc00640 1 -- 172.21.15.7:0/3758434605 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0634035040 con 0x55b475996290 2024-09-10T21:46:50.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.820+0000 7f064ba04740 1 -- 172.21.15.7:0/3758434605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b475996290 msgr2=0x55b4759ee160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.820+0000 7f064ba04740 1 --2- 172.21.15.7:0/3758434605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b475996290 0x55b4759ee160 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f0634034040 tx=0x7f0628005830 comp rx=0 tx=0).stop 2024-09-10T21:46:50.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.820+0000 7f064ba04740 1 -- 172.21.15.7:0/3758434605 shutdown_connections 2024-09-10T21:46:50.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.820+0000 7f064ba04740 1 --2- 172.21.15.7:0/3758434605 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759d0720 0x55b475a55e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.820+0000 7f064ba04740 1 --2- 172.21.15.7:0/3758434605 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b4759bed00 0x55b4759903a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.820+0000 7f064ba04740 1 --2- 172.21.15.7:0/3758434605 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b475996290 0x55b4759ee160 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.820+0000 7f064ba04740 1 -- 172.21.15.7:0/3758434605 >> 172.21.15.7:0/3758434605 conn(0x55b475995e40 msgr2=0x55b4759dc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:50.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.820+0000 7f064ba04740 1 -- 172.21.15.7:0/3758434605 shutdown_connections 2024-09-10T21:46:50.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.820+0000 7f064ba04740 1 -- 172.21.15.7:0/3758434605 wait complete. 2024-09-10T21:46:50.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.821+0000 7f064ba04740 1 -- start start 2024-09-10T21:46:50.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.821+0000 7f064ba04740 1 Processor -- start 2024-09-10T21:46:50.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.822+0000 7f064ba04740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b475996290 0x55b475a5bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.822+0000 7f064ba04740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759bed00 0x55b475a445f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.822+0000 7f064ba04740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b4759d0720 0x55b4759efb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.822+0000 7f064ba04740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55b4759d6190 con 0x55b475996290 2024-09-10T21:46:50.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.822+0000 7f064ba04740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0644200390 con 0x55b4759bed00 2024-09-10T21:46:50.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.822+0000 7f064ba04740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55b475989ed0 con 0x55b4759d0720 2024-09-10T21:46:50.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.822+0000 7f064ba04740 10 client.0 _ll_register_callbacks cb 0x7f063c062660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:46:50.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.822+0000 7f0641e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b475996290 0x55b475a5bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.822+0000 7f0641e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b475996290 0x55b475a5bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45152/0 (socket says 172.21.15.7:45152) 2024-09-10T21:46:50.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.822+0000 7f0641e00640 1 -- 172.21.15.7:0/1936698976 learned_addr learned my addr 172.21.15.7:0/1936698976 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:50.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.822+0000 7f0642800640 1 --2- 172.21.15.7:0/1936698976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b4759d0720 0x55b4759efb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.822+0000 7f0641400640 1 --2- 172.21.15.7:0/1936698976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759bed00 0x55b475a445f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.822+0000 7f0641e00640 1 -- 172.21.15.7:0/1936698976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b4759d0720 msgr2=0x55b4759efb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:50.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.822+0000 7f0641e00640 1 --2- 172.21.15.7:0/1936698976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b4759d0720 0x55b4759efb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.822+0000 7f0641e00640 1 -- 172.21.15.7:0/1936698976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759bed00 msgr2=0x55b475a445f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:50.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.822+0000 7f0641e00640 1 --2- 172.21.15.7:0/1936698976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759bed00 0x55b475a445f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.822+0000 7f0641e00640 1 -- 172.21.15.7:0/1936698976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b475a44b30 con 0x55b475996290 2024-09-10T21:46:50.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.823+0000 7f0642800640 1 --2- 172.21.15.7:0/1936698976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b4759d0720 0x55b4759efb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:50.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.823+0000 7f0641e00640 1 --2- 172.21.15.7:0/1936698976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b475996290 0x55b475a5bfb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f0634005920 tx=0x7f063403bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5b5240437d3645 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:50.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.823+0000 7f0622a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:46:50.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.823+0000 7f0622a00640 1 -- 172.21.15.7:0/1936698976 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f063405e070 con 0x55b475996290 2024-09-10T21:46:50.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.823+0000 7f0622a00640 1 -- 172.21.15.7:0/1936698976 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f0634044e00 con 0x55b475996290 2024-09-10T21:46:50.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.823+0000 7f064ba04740 1 -- 172.21.15.7:0/1936698976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0634044a50 con 0x55b475996290 2024-09-10T21:46:50.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.823+0000 7f0622a00640 1 -- 172.21.15.7:0/1936698976 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7f063406c050 con 0x55b475996290 2024-09-10T21:46:50.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.823+0000 7f064ba04740 10 client.6507 fetch_fsmap learned FSMap version 53 2024-09-10T21:46:50.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.823+0000 7f064ba04740 1 -- 172.21.15.7:0/1936698976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x55b475a5ba90 con 0x55b475996290 2024-09-10T21:46:50.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.824+0000 7f0622a00640 1 -- 172.21.15.7:0/1936698976 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0634044a50 con 0x55b475996290 2024-09-10T21:46:50.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.824+0000 7f0622a00640 1 -- 172.21.15.7:0/1936698976 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7f063406f020 con 0x55b475996290 2024-09-10T21:46:50.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.824+0000 7f064ba04740 10 client.6507 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T21:46:50.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.824+0000 7f064ba04740 10 client.6507 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-10T21:46:50.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.824+0000 7f064ba04740 20 client.6507 populate_metadata read hostname 'smithi007' 2024-09-10T21:46:50.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.824+0000 7f064ba04740 1 --2- 172.21.15.7:0/1936698976 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55b4759ed6f0 0x55b4759f00c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:50.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.824+0000 7f064ba04740 4 client.6507 mds_command: new command op to 235490 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}] 2024-09-10T21:46:50.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.824+0000 7f064ba04740 1 -- 172.21.15.7:0/1936698976 --> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] -- command(tid 0: {"prefix": "scrub start", "path": "/subdir", "scrubops": ["repair", "recursive"]}) -- 0x55b475905990 con 0x55b4759ed6f0 2024-09-10T21:46:50.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.824+0000 7f0641400640 1 --2- 172.21.15.7:0/1936698976 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55b4759ed6f0 0x55b4759f00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:50.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.825+0000 7f0641400640 1 --2- 172.21.15.7:0/1936698976 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55b4759ed6f0 0x55b4759f00c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33a1c1c8b2ec975d server_cookie=aa3c5d731947980d in_seq=0 out_seq=0 2024-09-10T21:46:50.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.825+0000 7f0622a00640 10 client.6507 ms_handle_connect on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:50.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.826+0000 7f0622a00640 1 -- 172.21.15.7:0/1936698976 <== mds.0 v2:172.21.15.133:6834/2622530239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b475905990 con 0x55b4759ed6f0 2024-09-10T21:46:50.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.826+0000 7f0622a00640 10 client.6507 handle_command_reply: tid=0 2024-09-10T21:46:50.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.826+0000 7f064ba04740 1 client.6507 shutdown 2024-09-10T21:46:50.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.826+0000 7f064ba04740 1 -- 172.21.15.7:0/1936698976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b475996290 msgr2=0x55b475a5bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.826+0000 7f064ba04740 1 --2- 172.21.15.7:0/1936698976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b475996290 0x55b475a5bfb0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f0634005920 tx=0x7f063403bf30 comp rx=0 tx=0).stop 2024-09-10T21:46:50.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.826+0000 7f064ba04740 1 -- 172.21.15.7:0/1936698976 shutdown_connections 2024-09-10T21:46:50.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.826+0000 7f064ba04740 1 --2- 172.21.15.7:0/1936698976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55b4759bed00 0x55b475a445f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.826+0000 7f064ba04740 1 --2- 172.21.15.7:0/1936698976 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x55b4759ed6f0 0x55b4759f00c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.826+0000 7f064ba04740 1 --2- 172.21.15.7:0/1936698976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55b4759d0720 0x55b4759efb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.826+0000 7f064ba04740 1 --2- 172.21.15.7:0/1936698976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55b475996290 0x55b475a5bfb0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.826+0000 7f0622a00640 0 client.6507 ms_handle_reset on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:50.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.826+0000 7f064ba04740 1 -- 172.21.15.7:0/1936698976 >> 172.21.15.7:0/1936698976 conn(0x55b475995e40 msgr2=0x55b4759db120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:50.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.826+0000 7f064ba04740 1 -- 172.21.15.7:0/1936698976 shutdown_connections 2024-09-10T21:46:50.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.826+0000 7f064ba04740 1 -- 172.21.15.7:0/1936698976 wait complete. 2024-09-10T21:46:50.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.826+0000 7f064ba04740 20 client.6507 trim_cache size 0 max 16384 2024-09-10T21:46:50.832 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:46:50.832 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-09-10T21:46:50.832 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "a08b336a-e18a-4f20-8d7c-ebce794cf011", 2024-09-10T21:46:50.832 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-09-10T21:46:50.832 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:46:50.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.827+0000 7f0648c00640 1 -- 172.21.15.7:0/256089704 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0624077fa0 msgr2=0x7f062407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.827+0000 7f0648c00640 1 --2- 172.21.15.7:0/256089704 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0624077fa0 0x7f062407a460 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0634036000 tx=0x7f063403c040 comp rx=0 tx=0).stop 2024-09-10T21:46:50.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.827+0000 7f0648c00640 1 -- 172.21.15.7:0/256089704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0644159fb0 msgr2=0x7f0644073d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:50.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.827+0000 7f0648c00640 1 --2- 172.21.15.7:0/256089704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0644159fb0 0x7f0644073d90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f06440ad2d0 tx=0x7f063c00ec50 comp rx=0 tx=0).stop 2024-09-10T21:46:50.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.827+0000 7f0648c00640 1 -- 172.21.15.7:0/256089704 shutdown_connections 2024-09-10T21:46:50.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.827+0000 7f0648c00640 1 --2- 172.21.15.7:0/256089704 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0624077fa0 0x7f062407a460 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.827+0000 7f0648c00640 1 --2- 172.21.15.7:0/256089704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0644159b30 0x7f0644073850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.827+0000 7f0648c00640 1 --2- 172.21.15.7:0/256089704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0644159fb0 0x7f0644073d90 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.827+0000 7f0648c00640 1 --2- 172.21.15.7:0/256089704 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0628003a50 0x7f0644149440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:50.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.827+0000 7f0648c00640 1 -- 172.21.15.7:0/256089704 >> 172.21.15.7:0/256089704 conn(0x7f06440b1c10 msgr2=0x7f06440d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:50.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.827+0000 7f0648c00640 1 -- 172.21.15.7:0/256089704 shutdown_connections 2024-09-10T21:46:50.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:50.827+0000 7f0648c00640 1 -- 172.21.15.7:0/256089704 wait complete. 2024-09-10T21:46:50.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:46:50.843 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-09-10T21:46:51.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.006+0000 7fa726e00640 1 -- 172.21.15.7:0/1346624471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa72006fcf0 msgr2=0x7fa720073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:51.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.006+0000 7fa726e00640 1 --2- 172.21.15.7:0/1346624471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa72006fcf0 0x7fa720073d40 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fa708005fd0 tx=0x7fa70802a5d0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.006+0000 7fa726e00640 1 -- 172.21.15.7:0/1346624471 shutdown_connections 2024-09-10T21:46:51.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.006+0000 7fa726e00640 1 --2- 172.21.15.7:0/1346624471 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa72006fcf0 0x7fa720073d40 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.006+0000 7fa726e00640 1 --2- 172.21.15.7:0/1346624471 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa720155fb0 0x7fa7201600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.006+0000 7fa726e00640 1 -- 172.21.15.7:0/1346624471 >> 172.21.15.7:0/1346624471 conn(0x7fa7200b16d0 msgr2=0x7fa7200b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:51.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.006+0000 7fa726e00640 1 -- 172.21.15.7:0/1346624471 shutdown_connections 2024-09-10T21:46:51.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.006+0000 7fa726e00640 1 -- 172.21.15.7:0/1346624471 wait complete. 2024-09-10T21:46:51.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.008+0000 7fa726e00640 1 Processor -- start 2024-09-10T21:46:51.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.008+0000 7fa726e00640 1 -- start start 2024-09-10T21:46:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.009+0000 7fa726e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa72006fcf0 0x7fa7200cf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.009+0000 7fa726e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa720074280 0x7fa72015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.009+0000 7fa726e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa720155fb0 0x7fa72015afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.009+0000 7fa726e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa720156510 con 0x7fa72006fcf0 2024-09-10T21:46:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.009+0000 7fa726e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa720156390 con 0x7fa720074280 2024-09-10T21:46:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.009+0000 7fa726e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa72006ec90 con 0x7fa720155fb0 2024-09-10T21:46:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.009+0000 7fa71fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa72006fcf0 0x7fa7200cf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.009+0000 7fa71fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa72006fcf0 0x7fa7200cf0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45154/0 (socket says 172.21.15.7:45154) 2024-09-10T21:46:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.009+0000 7fa71fe00640 1 -- 172.21.15.7:0/4191010507 learned_addr learned my addr 172.21.15.7:0/4191010507 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:51.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.009+0000 7fa717400640 1 --2- 172.21.15.7:0/4191010507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa720074280 0x7fa72015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.009+0000 7fa724e00640 1 --2- 172.21.15.7:0/4191010507 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa720155fb0 0x7fa72015afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.009+0000 7fa71fe00640 1 -- 172.21.15.7:0/4191010507 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa720155fb0 msgr2=0x7fa72015afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:51.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.009+0000 7fa71fe00640 1 --2- 172.21.15.7:0/4191010507 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa720155fb0 0x7fa72015afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.009+0000 7fa71fe00640 1 -- 172.21.15.7:0/4191010507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa720074280 msgr2=0x7fa72015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:51.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.009+0000 7fa71fe00640 1 --2- 172.21.15.7:0/4191010507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa720074280 0x7fa72015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.010+0000 7fa71fe00640 1 -- 172.21.15.7:0/4191010507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7200c7ce0 con 0x7fa72006fcf0 2024-09-10T21:46:51.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.010+0000 7fa71fe00640 1 --2- 172.21.15.7:0/4191010507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa72006fcf0 0x7fa7200cf0b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa71000e790 tx=0x7fa71000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:51.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.010+0000 7fa71cc00640 1 -- 172.21.15.7:0/4191010507 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71000eea0 con 0x7fa72006fcf0 2024-09-10T21:46:51.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.010+0000 7fa71cc00640 1 -- 172.21.15.7:0/4191010507 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa71000cdb0 con 0x7fa72006fcf0 2024-09-10T21:46:51.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.010+0000 7fa71cc00640 1 -- 172.21.15.7:0/4191010507 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa710010690 con 0x7fa72006fcf0 2024-09-10T21:46:51.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.010+0000 7fa726e00640 1 -- 172.21.15.7:0/4191010507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7200c7fc0 con 0x7fa72006fcf0 2024-09-10T21:46:51.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.010+0000 7fa726e00640 1 -- 172.21.15.7:0/4191010507 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7200c8580 con 0x7fa72006fcf0 2024-09-10T21:46:51.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.014+0000 7fa71cc00640 1 -- 172.21.15.7:0/4191010507 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa710010830 con 0x7fa72006fcf0 2024-09-10T21:46:51.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.015+0000 7fa729c6e740 1 Processor -- start 2024-09-10T21:46:51.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.015+0000 7fa71cc00640 1 --2- 172.21.15.7:0/4191010507 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa6fc077f70 0x7fa6fc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.015+0000 7fa729c6e740 1 -- start start 2024-09-10T21:46:51.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.015+0000 7fa71cc00640 1 -- 172.21.15.7:0/4191010507 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7fa71009a810 con 0x7fa72006fcf0 2024-09-10T21:46:51.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.015+0000 7fa724e00640 1 --2- 172.21.15.7:0/4191010507 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa6fc077f70 0x7fa6fc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.016+0000 7fa729c6e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f92130 0x559f84f92510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.016+0000 7fa729c6e740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f4d2a0 0x559f84f90c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.016+0000 7fa729c6e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f4d820 0x559f84f91150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.016+0000 7fa729c6e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559f84eac1c0 con 0x559f84f4d820 2024-09-10T21:46:51.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.016+0000 7fa729c6e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559f84ebf870 con 0x559f84f92130 2024-09-10T21:46:51.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.016+0000 7fa729c6e740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x559f84eaeca0 con 0x559f84f4d2a0 2024-09-10T21:46:51.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.016+0000 7fa71fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f92130 0x559f84f92510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.016+0000 7fa724e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f4d820 0x559f84f91150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.016+0000 7fa717400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f4d2a0 0x559f84f90c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.016+0000 7fa71fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f92130 0x559f84f92510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:37758/0 (socket says 172.21.15.7:37758) 2024-09-10T21:46:51.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.016+0000 7fa71fe00640 1 -- 172.21.15.7:0/3147007778 learned_addr learned my addr 172.21.15.7:0/3147007778 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:51.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.016+0000 7fa724e00640 1 --2- 172.21.15.7:0/4191010507 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa6fc077f70 0x7fa6fc07a430 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa7200cb450 tx=0x7fa718009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:51.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.016+0000 7fa724e00640 1 -- 172.21.15.7:0/3147007778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f4d2a0 msgr2=0x559f84f90c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:51.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.016+0000 7fa724e00640 1 --2- 172.21.15.7:0/3147007778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f4d2a0 0x559f84f90c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.016+0000 7fa724e00640 1 -- 172.21.15.7:0/3147007778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f92130 msgr2=0x559f84f92510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:51.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.016+0000 7fa724e00640 1 --2- 172.21.15.7:0/3147007778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f92130 0x559f84f92510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.016+0000 7fa724e00640 1 -- 172.21.15.7:0/3147007778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f84c877d0 con 0x559f84f4d820 2024-09-10T21:46:51.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.016+0000 7fa71fe00640 1 --2- 172.21.15.7:0/3147007778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f92130 0x559f84f92510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:46:51.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.017+0000 7fa724e00640 1 --2- 172.21.15.7:0/3147007778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f4d820 0x559f84f91150 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa71800fa30 tx=0x7fa718012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c70cb075081e245 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:51.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.017+0000 7fa714c00640 1 -- 172.21.15.7:0/3147007778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718026070 con 0x559f84f4d820 2024-09-10T21:46:51.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.017+0000 7fa714c00640 1 -- 172.21.15.7:0/3147007778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa718019070 con 0x559f84f4d820 2024-09-10T21:46:51.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.017+0000 7fa714c00640 1 -- 172.21.15.7:0/3147007778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71801b030 con 0x559f84f4d820 2024-09-10T21:46:51.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.018+0000 7fa729c6e740 1 -- 172.21.15.7:0/3147007778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f4d820 msgr2=0x559f84f91150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:51.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.018+0000 7fa729c6e740 1 --2- 172.21.15.7:0/3147007778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f4d820 0x559f84f91150 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fa71800fa30 tx=0x7fa718012000 comp rx=0 tx=0).stop 2024-09-10T21:46:51.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.018+0000 7fa729c6e740 1 -- 172.21.15.7:0/3147007778 shutdown_connections 2024-09-10T21:46:51.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.018+0000 7fa729c6e740 1 --2- 172.21.15.7:0/3147007778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f4d820 0x559f84f91150 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.018+0000 7fa729c6e740 1 --2- 172.21.15.7:0/3147007778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f4d2a0 0x559f84f90c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.018+0000 7fa729c6e740 1 --2- 172.21.15.7:0/3147007778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f92130 0x559f84f92510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.018+0000 7fa729c6e740 1 -- 172.21.15.7:0/3147007778 >> 172.21.15.7:0/3147007778 conn(0x559f84f48870 msgr2=0x559f84f4c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:51.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.018+0000 7fa729c6e740 1 -- 172.21.15.7:0/3147007778 shutdown_connections 2024-09-10T21:46:51.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.018+0000 7fa729c6e740 1 -- 172.21.15.7:0/3147007778 wait complete. 2024-09-10T21:46:51.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.019+0000 7fa729c6e740 1 -- start start 2024-09-10T21:46:51.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.019+0000 7fa729c6e740 1 Processor -- start 2024-09-10T21:46:51.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.020+0000 7fa729c6e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f4d820 0x559f84fa1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.020+0000 7fa729c6e740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f92130 0x559f84f708e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.020+0000 7fa729c6e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f73740 0x559f84f75c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.020+0000 7fa729c6e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559f84e55410 con 0x559f84f73740 2024-09-10T21:46:51.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.020+0000 7fa729c6e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x559f84eb2ab0 con 0x559f84f4d820 2024-09-10T21:46:51.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.020+0000 7fa729c6e740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x559f84c1f740 con 0x559f84f92130 2024-09-10T21:46:51.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.020+0000 7fa729c6e740 10 client.0 _ll_register_callbacks cb 0x559f84e32360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:46:51.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.020+0000 7fa724e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f73740 0x559f84f75c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.020+0000 7fa724e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f73740 0x559f84f75c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45172/0 (socket says 172.21.15.7:45172) 2024-09-10T21:46:51.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.020+0000 7fa724e00640 1 -- 172.21.15.7:0/2149895323 learned_addr learned my addr 172.21.15.7:0/2149895323 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:51.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.020+0000 7fa717400640 1 --2- 172.21.15.7:0/2149895323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f92130 0x559f84f708e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.020+0000 7fa71fe00640 1 --2- 172.21.15.7:0/2149895323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f4d820 0x559f84fa1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.020+0000 7fa724e00640 1 -- 172.21.15.7:0/2149895323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f92130 msgr2=0x559f84f708e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:51.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.020+0000 7fa724e00640 1 --2- 172.21.15.7:0/2149895323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f92130 0x559f84f708e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.020+0000 7fa724e00640 1 -- 172.21.15.7:0/2149895323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f4d820 msgr2=0x559f84fa1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:51.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.020+0000 7fa724e00640 1 --2- 172.21.15.7:0/2149895323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f4d820 0x559f84fa1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.020+0000 7fa724e00640 1 -- 172.21.15.7:0/2149895323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f84c10970 con 0x559f84f73740 2024-09-10T21:46:51.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.021+0000 7fa717400640 1 --2- 172.21.15.7:0/2149895323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f92130 0x559f84f708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:51.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.021+0000 7fa71fe00640 1 --2- 172.21.15.7:0/2149895323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f4d820 0x559f84fa1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:46:51.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.021+0000 7fa724e00640 1 --2- 172.21.15.7:0/2149895323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f73740 0x559f84f75c70 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa718009010 tx=0x7fa718007300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a083056b71ada81 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:51.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.021+0000 7fa70d600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:46:51.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.021+0000 7fa70d600640 1 -- 172.21.15.7:0/2149895323 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71801b070 con 0x559f84f73740 2024-09-10T21:46:51.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.021+0000 7fa70d600640 1 -- 172.21.15.7:0/2149895323 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7fa718007900 con 0x559f84f73740 2024-09-10T21:46:51.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.021+0000 7fa729c6e740 1 -- 172.21.15.7:0/2149895323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa718019070 con 0x559f84f73740 2024-09-10T21:46:51.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.021+0000 7fa70d600640 1 -- 172.21.15.7:0/2149895323 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fa718037050 con 0x559f84f73740 2024-09-10T21:46:51.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.021+0000 7fa729c6e740 10 client.6516 fetch_fsmap learned FSMap version 53 2024-09-10T21:46:51.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.021+0000 7fa729c6e740 1 -- 172.21.15.7:0/2149895323 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x559f84bee6a0 con 0x559f84f73740 2024-09-10T21:46:51.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.022+0000 7fa70d600640 1 -- 172.21.15.7:0/2149895323 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa718019070 con 0x559f84f73740 2024-09-10T21:46:51.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.022+0000 7fa70d600640 1 -- 172.21.15.7:0/2149895323 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7fa71803a020 con 0x559f84f73740 2024-09-10T21:46:51.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.022+0000 7fa729c6e740 10 client.6516 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T21:46:51.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.022+0000 7fa729c6e740 10 client.6516 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-10T21:46:51.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.022+0000 7fa729c6e740 20 client.6516 populate_metadata read hostname 'smithi007' 2024-09-10T21:46:51.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.022+0000 7fa729c6e740 1 --2- 172.21.15.7:0/2149895323 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x559f84fa2210 0x559f84fa2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.022+0000 7fa729c6e740 4 client.6516 mds_command: new command op to 235490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:46:51.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.022+0000 7fa729c6e740 1 -- 172.21.15.7:0/2149895323 --> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f84ec17c0 con 0x559f84fa2210 2024-09-10T21:46:51.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.022+0000 7fa71fe00640 1 --2- 172.21.15.7:0/2149895323 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x559f84fa2210 0x559f84fa2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.023+0000 7fa71fe00640 1 --2- 172.21.15.7:0/2149895323 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x559f84fa2210 0x559f84fa2620 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da6296d693b2ce00 server_cookie=4ca9bc2604592a38 in_seq=0 out_seq=0 2024-09-10T21:46:51.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.023+0000 7fa70d600640 10 client.6516 ms_handle_connect on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:51.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.024+0000 7fa70d600640 1 -- 172.21.15.7:0/2149895323 <== mds.0 v2:172.21.15.133:6834/2622530239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559f84ec17c0 con 0x559f84fa2210 2024-09-10T21:46:51.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.024+0000 7fa70d600640 10 client.6516 handle_command_reply: tid=0 2024-09-10T21:46:51.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.024+0000 7fa729c6e740 1 client.6516 shutdown 2024-09-10T21:46:51.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.024+0000 7fa729c6e740 1 -- 172.21.15.7:0/2149895323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f73740 msgr2=0x559f84f75c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:51.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.024+0000 7fa729c6e740 1 --2- 172.21.15.7:0/2149895323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f73740 0x559f84f75c70 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fa718009010 tx=0x7fa718007300 comp rx=0 tx=0).stop 2024-09-10T21:46:51.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.024+0000 7fa729c6e740 1 -- 172.21.15.7:0/2149895323 shutdown_connections 2024-09-10T21:46:51.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.024+0000 7fa729c6e740 1 --2- 172.21.15.7:0/2149895323 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x559f84fa2210 0x559f84fa2620 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.024+0000 7fa729c6e740 1 --2- 172.21.15.7:0/2149895323 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f73740 0x559f84f75c70 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.024+0000 7fa729c6e740 1 --2- 172.21.15.7:0/2149895323 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f92130 0x559f84f708e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.024+0000 7fa70d600640 0 client.6516 ms_handle_reset on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:51.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.024+0000 7fa729c6e740 1 --2- 172.21.15.7:0/2149895323 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f4d820 0x559f84fa1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.024+0000 7fa729c6e740 1 -- 172.21.15.7:0/2149895323 >> 172.21.15.7:0/2149895323 conn(0x559f84f4d2a0 msgr2=0x559f84f45fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:51.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.024+0000 7fa729c6e740 1 -- 172.21.15.7:0/2149895323 shutdown_connections 2024-09-10T21:46:51.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.024+0000 7fa729c6e740 1 -- 172.21.15.7:0/2149895323 wait complete. 2024-09-10T21:46:51.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.025+0000 7fa729c6e740 20 client.6516 trim_cache size 0 max 16384 2024-09-10T21:46:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.043+0000 7fa729c6e740 1 Processor -- start 2024-09-10T21:46:51.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.043+0000 7fa729c6e740 1 -- start start 2024-09-10T21:46:51.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.044+0000 7fa729c6e740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f4d820 0x559f84f92510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.044+0000 7fa729c6e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f73740 0x559f84f8e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.044+0000 7fa729c6e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f92130 0x559f84fa25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.044+0000 7fa729c6e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa7202013a0 con 0x559f84f92130 2024-09-10T21:46:51.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.044+0000 7fa729c6e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa7202003e0 con 0x559f84f73740 2024-09-10T21:46:51.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.044+0000 7fa729c6e740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa7200ce150 con 0x559f84f4d820 2024-09-10T21:46:51.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.044+0000 7fa724e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f92130 0x559f84fa25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.044+0000 7fa724e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f92130 0x559f84fa25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45176/0 (socket says 172.21.15.7:45176) 2024-09-10T21:46:51.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.044+0000 7fa724e00640 1 -- 172.21.15.7:0/1476133223 learned_addr learned my addr 172.21.15.7:0/1476133223 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:51.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.044+0000 7fa724e00640 1 -- 172.21.15.7:0/1476133223 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f4d820 msgr2=0x559f84f92510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:51.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.044+0000 7fa724e00640 1 --2- 172.21.15.7:0/1476133223 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f4d820 0x559f84f92510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.044+0000 7fa724e00640 1 -- 172.21.15.7:0/1476133223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f73740 msgr2=0x559f84f8e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:51.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.044+0000 7fa724e00640 1 --2- 172.21.15.7:0/1476133223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f73740 0x559f84f8e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.044+0000 7fa724e00640 1 -- 172.21.15.7:0/1476133223 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f84f73e70 con 0x559f84f92130 2024-09-10T21:46:51.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.045+0000 7fa724e00640 1 --2- 172.21.15.7:0/1476133223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f92130 0x559f84fa25f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa718001330 tx=0x7fa718001540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895076cb5e0ed1b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:51.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.045+0000 7fa70fe00640 1 -- 172.21.15.7:0/1476133223 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa718040070 con 0x559f84f92130 2024-09-10T21:46:51.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.045+0000 7fa70fe00640 1 -- 172.21.15.7:0/1476133223 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa718019070 con 0x559f84f92130 2024-09-10T21:46:51.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.045+0000 7fa70fe00640 1 -- 172.21.15.7:0/1476133223 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71801b040 con 0x559f84f92130 2024-09-10T21:46:51.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.045+0000 7fa729c6e740 1 -- 172.21.15.7:0/1476133223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f92130 msgr2=0x559f84fa25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:51.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.045+0000 7fa729c6e740 1 --2- 172.21.15.7:0/1476133223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f92130 0x559f84fa25f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fa718001330 tx=0x7fa718001540 comp rx=0 tx=0).stop 2024-09-10T21:46:51.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.045+0000 7fa729c6e740 1 -- 172.21.15.7:0/1476133223 shutdown_connections 2024-09-10T21:46:51.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.045+0000 7fa729c6e740 1 --2- 172.21.15.7:0/1476133223 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f73740 0x559f84f8e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.045+0000 7fa729c6e740 1 --2- 172.21.15.7:0/1476133223 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f92130 0x559f84fa25f0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.045+0000 7fa729c6e740 1 --2- 172.21.15.7:0/1476133223 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f4d820 0x559f84f92510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.045+0000 7fa729c6e740 1 -- 172.21.15.7:0/1476133223 >> 172.21.15.7:0/1476133223 conn(0x559f84f4d2a0 msgr2=0x559f84f94510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:51.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.046+0000 7fa729c6e740 1 -- 172.21.15.7:0/1476133223 shutdown_connections 2024-09-10T21:46:51.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.046+0000 7fa729c6e740 1 -- 172.21.15.7:0/1476133223 wait complete. 2024-09-10T21:46:51.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.046+0000 7fa729c6e740 1 -- start start 2024-09-10T21:46:51.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.046+0000 7fa729c6e740 1 Processor -- start 2024-09-10T21:46:51.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.047+0000 7fa729c6e740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f4d820 0x559f850119a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.047+0000 7fa729c6e740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f73740 0x559f84faff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.047+0000 7fa729c6e740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f92130 0x559f850129a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.047+0000 7fa729c6e740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x559f84f74f60 con 0x559f84f92130 2024-09-10T21:46:51.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.047+0000 7fa729c6e740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa7200ce7a0 con 0x559f84f73740 2024-09-10T21:46:51.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.047+0000 7fa729c6e740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x559f84f75690 con 0x559f84f4d820 2024-09-10T21:46:51.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.047+0000 7fa729c6e740 10 client.0 _ll_register_callbacks cb 0x7fa718024070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:46:51.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.048+0000 7fa724e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f92130 0x559f850129a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.048+0000 7fa724e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f92130 0x559f850129a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45178/0 (socket says 172.21.15.7:45178) 2024-09-10T21:46:51.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.048+0000 7fa724e00640 1 -- 172.21.15.7:0/646160376 learned_addr learned my addr 172.21.15.7:0/646160376 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:46:51.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.048+0000 7fa717400640 1 --2- 172.21.15.7:0/646160376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f73740 0x559f84faff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.048+0000 7fa71fe00640 1 --2- 172.21.15.7:0/646160376 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f4d820 0x559f850119a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.048+0000 7fa724e00640 1 -- 172.21.15.7:0/646160376 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f4d820 msgr2=0x559f850119a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:51.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.048+0000 7fa724e00640 1 --2- 172.21.15.7:0/646160376 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f4d820 0x559f850119a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.048+0000 7fa724e00640 1 -- 172.21.15.7:0/646160376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f73740 msgr2=0x559f84faff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:46:51.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.048+0000 7fa724e00640 1 --2- 172.21.15.7:0/646160376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f73740 0x559f84faff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.048+0000 7fa724e00640 1 -- 172.21.15.7:0/646160376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f84f917b0 con 0x559f84f92130 2024-09-10T21:46:51.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.048+0000 7fa724e00640 1 --2- 172.21.15.7:0/646160376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f92130 0x559f850129a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa7180019d0 tx=0x7fa71802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f40c2e04c7687e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:51.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.048+0000 7fa70d600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:46:51.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.049+0000 7fa70d600640 1 -- 172.21.15.7:0/646160376 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa71804a060 con 0x559f84f92130 2024-09-10T21:46:51.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.049+0000 7fa70d600640 1 -- 172.21.15.7:0/646160376 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7fa718020610 con 0x559f84f92130 2024-09-10T21:46:51.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.049+0000 7fa729c6e740 1 -- 172.21.15.7:0/646160376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa718019070 con 0x559f84f92130 2024-09-10T21:46:51.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.049+0000 7fa70d600640 1 -- 172.21.15.7:0/646160376 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) ==== 24+0+0 (secure 0 0 0) 0x7fa718050050 con 0x559f84f92130 2024-09-10T21:46:51.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.049+0000 7fa729c6e740 10 client.6522 fetch_fsmap learned FSMap version 53 2024-09-10T21:46:51.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.049+0000 7fa729c6e740 1 -- 172.21.15.7:0/646160376 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=2+}) -- 0x559f850133b0 con 0x559f84f92130 2024-09-10T21:46:51.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.049+0000 7fa70d600640 1 -- 172.21.15.7:0/646160376 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa718019070 con 0x559f84f92130 2024-09-10T21:46:51.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.049+0000 7fa70d600640 1 -- 172.21.15.7:0/646160376 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 53) ==== 3502+0+0 (secure 0 0 0) 0x7fa718053020 con 0x559f84f92130 2024-09-10T21:46:51.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.049+0000 7fa729c6e740 10 client.6522 fetch_fsmap finished waiting for FSMap version 53 2024-09-10T21:46:51.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.049+0000 7fa729c6e740 10 client.6522 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-10T21:46:51.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.049+0000 7fa729c6e740 20 client.6522 populate_metadata read hostname 'smithi007' 2024-09-10T21:46:51.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.049+0000 7fa729c6e740 1 --2- 172.21.15.7:0/646160376 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x559f85012ee0 0x559f84fa0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.049+0000 7fa729c6e740 4 client.6522 mds_command: new command op to 235490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:46:51.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.049+0000 7fa729c6e740 1 -- 172.21.15.7:0/646160376 --> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559f84ebd270 con 0x559f85012ee0 2024-09-10T21:46:51.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.050+0000 7fa71fe00640 1 --2- 172.21.15.7:0/646160376 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x559f85012ee0 0x559f84fa0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.050+0000 7fa71fe00640 1 --2- 172.21.15.7:0/646160376 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x559f85012ee0 0x559f84fa0fb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c73095ff70d242c server_cookie=1032172750543a2f in_seq=0 out_seq=0 2024-09-10T21:46:51.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.050+0000 7fa70d600640 10 client.6522 ms_handle_connect on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:51.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.050+0000 7fa70d600640 1 -- 172.21.15.7:0/646160376 <== mds.0 v2:172.21.15.133:6834/2622530239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+873 (crc 0 0 0) 0x559f84ebd270 con 0x559f85012ee0 2024-09-10T21:46:51.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.051+0000 7fa70d600640 10 client.6522 handle_command_reply: tid=0 2024-09-10T21:46:51.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.051+0000 7fa729c6e740 1 client.6522 shutdown 2024-09-10T21:46:51.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.051+0000 7fa729c6e740 1 -- 172.21.15.7:0/646160376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f92130 msgr2=0x559f850129a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:51.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.051+0000 7fa729c6e740 1 --2- 172.21.15.7:0/646160376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f92130 0x559f850129a0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fa7180019d0 tx=0x7fa71802bc30 comp rx=0 tx=0).stop 2024-09-10T21:46:51.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.051+0000 7fa729c6e740 1 -- 172.21.15.7:0/646160376 shutdown_connections 2024-09-10T21:46:51.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.051+0000 7fa729c6e740 1 --2- 172.21.15.7:0/646160376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x559f84f73740 0x559f84faff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.051+0000 7fa729c6e740 1 --2- 172.21.15.7:0/646160376 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x559f85012ee0 0x559f84fa0fb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.051+0000 7fa729c6e740 1 --2- 172.21.15.7:0/646160376 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x559f84f92130 0x559f850129a0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.051+0000 7fa729c6e740 1 --2- 172.21.15.7:0/646160376 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x559f84f4d820 0x559f850119a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.051+0000 7fa70d600640 0 client.6522 ms_handle_reset on v2:172.21.15.133:6834/2622530239 2024-09-10T21:46:51.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.051+0000 7fa729c6e740 1 -- 172.21.15.7:0/646160376 >> 172.21.15.7:0/646160376 conn(0x559f84f4d2a0 msgr2=0x559f84fa25f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:51.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.051+0000 7fa729c6e740 1 -- 172.21.15.7:0/646160376 shutdown_connections 2024-09-10T21:46:51.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.051+0000 7fa729c6e740 1 -- 172.21.15.7:0/646160376 wait complete. 2024-09-10T21:46:51.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.051+0000 7fa729c6e740 20 client.6522 trim_cache size 0 max 16384 2024-09-10T21:46:51.057 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:46:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:46:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-09-10T21:46:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "a08b336a-e18a-4f20-8d7c-ebce794cf011": { 2024-09-10T21:46:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/subdir", 2024-09-10T21:46:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "a08b336a-e18a-4f20-8d7c-ebce794cf011", 2024-09-10T21:46:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,repair" 2024-09-10T21:46:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:46:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:46:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:46:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "a08b336a-e18a-4f20-8d7c-ebce794cf011": { 2024-09-10T21:46:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:46:50.825662+0000", 2024-09-10T21:46:51.057 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/subdir", 2024-09-10T21:46:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:46:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:46:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:46:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:46:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:46:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "d158ff07-06ba-4bd2-9cbd-cc6aad170cea": { 2024-09-10T21:46:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:46:19.061276+0000", 2024-09-10T21:46:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/subdir", 2024-09-10T21:46:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:46:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:46:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:46:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:46:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:46:51.058 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:46:51.059 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:46:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.052+0000 7fa726e00640 1 -- 172.21.15.7:0/4191010507 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa6fc077f70 msgr2=0x7fa6fc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.052+0000 7fa726e00640 1 --2- 172.21.15.7:0/4191010507 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa6fc077f70 0x7fa6fc07a430 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa7200cb450 tx=0x7fa718009040 comp rx=0 tx=0).stop 2024-09-10T21:46:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.052+0000 7fa726e00640 1 -- 172.21.15.7:0/4191010507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa72006fcf0 msgr2=0x7fa7200cf0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.052+0000 7fa726e00640 1 --2- 172.21.15.7:0/4191010507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa72006fcf0 0x7fa7200cf0b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa71000e790 tx=0x7fa71000ec50 comp rx=0 tx=0).stop 2024-09-10T21:46:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.052+0000 7fa726e00640 1 -- 172.21.15.7:0/4191010507 shutdown_connections 2024-09-10T21:46:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.052+0000 7fa726e00640 1 --2- 172.21.15.7:0/4191010507 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa6fc077f70 0x7fa6fc07a430 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.052+0000 7fa726e00640 1 --2- 172.21.15.7:0/4191010507 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa720074280 0x7fa72015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.052+0000 7fa726e00640 1 --2- 172.21.15.7:0/4191010507 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa720155fb0 0x7fa72015afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.052+0000 7fa726e00640 1 --2- 172.21.15.7:0/4191010507 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa72006fcf0 0x7fa7200cf0b0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.052+0000 7fa726e00640 1 -- 172.21.15.7:0/4191010507 >> 172.21.15.7:0/4191010507 conn(0x7fa7200b16d0 msgr2=0x7fa72015e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:51.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.052+0000 7fa726e00640 1 -- 172.21.15.7:0/4191010507 shutdown_connections 2024-09-10T21:46:51.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:46:51.052+0000 7fa726e00640 1 -- 172.21.15.7:0/4191010507 wait complete. 2024-09-10T21:46:51.070 INFO:tasks.cephfs.filesystem:scrub status for tag:a08b336a-e18a-4f20-8d7c-ebce794cf011 - {'path': '/subdir', 'tag': 'a08b336a-e18a-4f20-8d7c-ebce794cf011', 'options': 'recursive,repair'} 2024-09-10T21:47:00.643 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:47:00.648 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:47:21.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:21.071 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-09-10T21:47:21.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.247+0000 7f68f9200640 1 -- 172.21.15.7:0/3968431012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f40cec80 msgr2=0x7f68f40cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.247+0000 7f68f9200640 1 --2- 172.21.15.7:0/3968431012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f40cec80 0x7f68f40cd6f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f68e400a940 tx=0x7f68e402b7b0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.247+0000 7f68f9200640 1 -- 172.21.15.7:0/3968431012 shutdown_connections 2024-09-10T21:47:21.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.247+0000 7f68f9200640 1 --2- 172.21.15.7:0/3968431012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f40cec80 0x7f68f40cd6f0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.247+0000 7f68f9200640 1 -- 172.21.15.7:0/3968431012 >> 172.21.15.7:0/3968431012 conn(0x7f68f40b16d0 msgr2=0x7f68f40b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.247+0000 7f68f9200640 1 -- 172.21.15.7:0/3968431012 shutdown_connections 2024-09-10T21:47:21.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.247+0000 7f68f9200640 1 -- 172.21.15.7:0/3968431012 wait complete. 2024-09-10T21:47:21.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.249+0000 7f68f9200640 1 Processor -- start 2024-09-10T21:47:21.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.249+0000 7f68f9200640 1 -- start start 2024-09-10T21:47:21.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.250+0000 7f68f9200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f40ca960 0x7f68f41577e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.250+0000 7f68f9200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68f40cec80 0x7f68f4154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.250+0000 7f68f9200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68f40cf060 0x7f68f4155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.250+0000 7f68f9200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68f40cc930 con 0x7f68f40ca960 2024-09-10T21:47:21.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.250+0000 7f68f9200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f68f40cc7b0 con 0x7f68f40cec80 2024-09-10T21:47:21.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.250+0000 7f68f9200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f68f40ccab0 con 0x7f68f40cf060 2024-09-10T21:47:21.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.250+0000 7f68f1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f40ca960 0x7f68f41577e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.250+0000 7f68f1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f40ca960 0x7f68f41577e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60834/0 (socket says 172.21.15.7:60834) 2024-09-10T21:47:21.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.250+0000 7f68f1e00640 1 -- 172.21.15.7:0/2403814231 learned_addr learned my addr 172.21.15.7:0/2403814231 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:21.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.251+0000 7f68f1400640 1 --2- 172.21.15.7:0/2403814231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68f40cec80 0x7f68f4154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.251+0000 7f68f1e00640 1 -- 172.21.15.7:0/2403814231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68f40cf060 msgr2=0x7f68f4155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.251+0000 7f68f2800640 1 --2- 172.21.15.7:0/2403814231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68f40cf060 0x7f68f4155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.251+0000 7f68f1e00640 1 --2- 172.21.15.7:0/2403814231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68f40cf060 0x7f68f4155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.251+0000 7f68f1e00640 1 -- 172.21.15.7:0/2403814231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68f40cec80 msgr2=0x7f68f4154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.251+0000 7f68f1e00640 1 --2- 172.21.15.7:0/2403814231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68f40cec80 0x7f68f4154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.251+0000 7f68f1e00640 1 -- 172.21.15.7:0/2403814231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68f4157d20 con 0x7f68f40ca960 2024-09-10T21:47:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.251+0000 7f68f2800640 1 --2- 172.21.15.7:0/2403814231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68f40cf060 0x7f68f4155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:47:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.251+0000 7f68f1e00640 1 --2- 172.21.15.7:0/2403814231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f40ca960 0x7f68f41577e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f68e4000c00 tx=0x7f68e4004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.251+0000 7f68ea000640 1 -- 172.21.15.7:0/2403814231 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e4003a00 con 0x7f68f40ca960 2024-09-10T21:47:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.251+0000 7f68ea000640 1 -- 172.21.15.7:0/2403814231 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68e4003c90 con 0x7f68f40ca960 2024-09-10T21:47:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.251+0000 7f68ea000640 1 -- 172.21.15.7:0/2403814231 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e403c670 con 0x7f68f40ca960 2024-09-10T21:47:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.251+0000 7f68f9200640 1 -- 172.21.15.7:0/2403814231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68f41f5380 con 0x7f68f40ca960 2024-09-10T21:47:21.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.252+0000 7f68f9200640 1 -- 172.21.15.7:0/2403814231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68f41f5940 con 0x7f68f40ca960 2024-09-10T21:47:21.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.254+0000 7f68ea000640 1 -- 172.21.15.7:0/2403814231 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f68e404c030 con 0x7f68f40ca960 2024-09-10T21:47:21.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.255+0000 7f68ea000640 1 --2- 172.21.15.7:0/2403814231 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f68d4077f70 0x7f68d407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.255+0000 7f68ea000640 1 -- 172.21.15.7:0/2403814231 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f68e40bb3a0 con 0x7f68f40ca960 2024-09-10T21:47:21.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.255+0000 7f68f1400640 1 --2- 172.21.15.7:0/2403814231 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f68d4077f70 0x7f68d407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.255+0000 7f68fc1ff740 1 Processor -- start 2024-09-10T21:47:21.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.256+0000 7f68fc1ff740 1 -- start start 2024-09-10T21:47:21.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.256+0000 7f68f1400640 1 --2- 172.21.15.7:0/2403814231 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f68d4077f70 0x7f68d407a430 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f68f4143d30 tx=0x7f68e0009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.257+0000 7f68fc1ff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1ee130 0x5620dd1ee510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.257+0000 7f68fc1ff740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1a92a0 0x5620dd1ecc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.257+0000 7f68fc1ff740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd1a9820 0x5620dd1ed150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.257+0000 7f68fc1ff740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5620dd1081c0 con 0x5620dd1ee130 2024-09-10T21:47:21.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.257+0000 7f68fc1ff740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5620dd11b870 con 0x5620dd1a9820 2024-09-10T21:47:21.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.257+0000 7f68fc1ff740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5620dd10aca0 con 0x5620dd1a92a0 2024-09-10T21:47:21.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.257+0000 7f68f1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1ee130 0x5620dd1ee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.257+0000 7f68f1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1ee130 0x5620dd1ee510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60844/0 (socket says 172.21.15.7:60844) 2024-09-10T21:47:21.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.257+0000 7f68f1e00640 1 -- 172.21.15.7:0/806770786 learned_addr learned my addr 172.21.15.7:0/806770786 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.257+0000 7f68f2800640 1 --2- 172.21.15.7:0/806770786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd1a9820 0x5620dd1ed150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.257+0000 7f68f1400640 1 --2- 172.21.15.7:0/806770786 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1a92a0 0x5620dd1ecc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.257+0000 7f68f1e00640 1 -- 172.21.15.7:0/806770786 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1a92a0 msgr2=0x5620dd1ecc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.257+0000 7f68f1e00640 1 --2- 172.21.15.7:0/806770786 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1a92a0 0x5620dd1ecc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.257+0000 7f68f1e00640 1 -- 172.21.15.7:0/806770786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd1a9820 msgr2=0x5620dd1ed150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.257+0000 7f68f1e00640 1 --2- 172.21.15.7:0/806770786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd1a9820 0x5620dd1ed150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.257+0000 7f68f1e00640 1 -- 172.21.15.7:0/806770786 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620dcee37d0 con 0x5620dd1ee130 2024-09-10T21:47:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.257+0000 7f68f1400640 1 --2- 172.21.15.7:0/806770786 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1a92a0 0x5620dd1ecc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:47:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.258+0000 7f68f1e00640 1 --2- 172.21.15.7:0/806770786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1ee130 0x5620dd1ee510 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f68e4083370 tx=0x7f68e4082de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5a1e8a03af2b54 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.258+0000 7f68dea00640 1 -- 172.21.15.7:0/806770786 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e4083180 con 0x5620dd1ee130 2024-09-10T21:47:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.258+0000 7f68dea00640 1 -- 172.21.15.7:0/806770786 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68e4083d70 con 0x5620dd1ee130 2024-09-10T21:47:21.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.258+0000 7f68dea00640 1 -- 172.21.15.7:0/806770786 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e4083670 con 0x5620dd1ee130 2024-09-10T21:47:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.258+0000 7f68fc1ff740 1 -- 172.21.15.7:0/806770786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1ee130 msgr2=0x5620dd1ee510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.258+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/806770786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1ee130 0x5620dd1ee510 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f68e4083370 tx=0x7f68e4082de0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.258+0000 7f68fc1ff740 1 -- 172.21.15.7:0/806770786 shutdown_connections 2024-09-10T21:47:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.258+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/806770786 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd1a9820 0x5620dd1ed150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.258+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/806770786 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1a92a0 0x5620dd1ecc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.258+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/806770786 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1ee130 0x5620dd1ee510 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.258+0000 7f68fc1ff740 1 -- 172.21.15.7:0/806770786 >> 172.21.15.7:0/806770786 conn(0x5620dd1a4870 msgr2=0x5620dd1a8290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.259+0000 7f68fc1ff740 1 -- 172.21.15.7:0/806770786 shutdown_connections 2024-09-10T21:47:21.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.259+0000 7f68fc1ff740 1 -- 172.21.15.7:0/806770786 wait complete. 2024-09-10T21:47:21.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.259+0000 7f68fc1ff740 1 -- start start 2024-09-10T21:47:21.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.259+0000 7f68fc1ff740 1 Processor -- start 2024-09-10T21:47:21.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.260+0000 7f68fc1ff740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1a9820 0x5620dd1ffbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.260+0000 7f68fc1ff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1ee130 0x5620dd1e6070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.260+0000 7f68fc1ff740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd200120 0x5620dd1d2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.260+0000 7f68fc1ff740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5620dd0b1410 con 0x5620dd1ee130 2024-09-10T21:47:21.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.260+0000 7f68fc1ff740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5620dd10eab0 con 0x5620dd200120 2024-09-10T21:47:21.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.260+0000 7f68fc1ff740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5620dce7b740 con 0x5620dd1a9820 2024-09-10T21:47:21.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.260+0000 7f68fc1ff740 10 client.0 _ll_register_callbacks cb 0x5620dd08e360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:47:21.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.261+0000 7f68f1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1ee130 0x5620dd1e6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.261+0000 7f68f2800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd200120 0x5620dd1d2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.261+0000 7f68f1e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1a9820 0x5620dd1ffbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.261+0000 7f68f2800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd200120 0x5620dd1d2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:59944/0 (socket says 172.21.15.7:59944) 2024-09-10T21:47:21.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.261+0000 7f68f2800640 1 -- 172.21.15.7:0/2441725377 learned_addr learned my addr 172.21.15.7:0/2441725377 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:21.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.261+0000 7f68f1400640 1 -- 172.21.15.7:0/2441725377 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1a9820 msgr2=0x5620dd1ffbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:21.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.261+0000 7f68f1400640 1 --2- 172.21.15.7:0/2441725377 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1a9820 0x5620dd1ffbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.261+0000 7f68f1400640 1 -- 172.21.15.7:0/2441725377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd200120 msgr2=0x5620dd1d2230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:21.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.261+0000 7f68f1400640 1 --2- 172.21.15.7:0/2441725377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd200120 0x5620dd1d2230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.261+0000 7f68f1400640 1 -- 172.21.15.7:0/2441725377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620dce6c970 con 0x5620dd1ee130 2024-09-10T21:47:21.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.261+0000 7f68f1400640 1 --2- 172.21.15.7:0/2441725377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1ee130 0x5620dd1e6070 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f68f40b1d60 tx=0x7f68e00036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13728e243dda19a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.261+0000 7f68d3400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:47:21.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.262+0000 7f68d3400640 1 -- 172.21.15.7:0/2441725377 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e0015e10 con 0x5620dd1ee130 2024-09-10T21:47:21.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.262+0000 7f68d3400640 1 -- 172.21.15.7:0/2441725377 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f68e0010040 con 0x5620dd1ee130 2024-09-10T21:47:21.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.262+0000 7f68fc1ff740 1 -- 172.21.15.7:0/2441725377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68e4083d70 con 0x5620dd1ee130 2024-09-10T21:47:21.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.262+0000 7f68d3400640 1 -- 172.21.15.7:0/2441725377 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f68e0003800 con 0x5620dd1ee130 2024-09-10T21:47:21.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.262+0000 7f68fc1ff740 10 client.6537 fetch_fsmap learned FSMap version 54 2024-09-10T21:47:21.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.262+0000 7f68fc1ff740 1 -- 172.21.15.7:0/2441725377 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x5620dce4a6a0 con 0x5620dd1ee130 2024-09-10T21:47:21.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.262+0000 7f68d3400640 1 -- 172.21.15.7:0/2441725377 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68e4083d70 con 0x5620dd1ee130 2024-09-10T21:47:21.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.262+0000 7f68d3400640 1 -- 172.21.15.7:0/2441725377 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7f68e002f080 con 0x5620dd1ee130 2024-09-10T21:47:21.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.262+0000 7f68fc1ff740 10 client.6537 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T21:47:21.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.262+0000 7f68fc1ff740 10 client.6537 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-10T21:47:21.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.262+0000 7f68fc1ff740 20 client.6537 populate_metadata read hostname 'smithi007' 2024-09-10T21:47:21.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.262+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/2441725377 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x5620dd1d0750 0x5620dd1d0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.262+0000 7f68fc1ff740 4 client.6537 mds_command: new command op to 235490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:47:21.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.262+0000 7f68fc1ff740 1 -- 172.21.15.7:0/2441725377 --> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620dd11d7c0 con 0x5620dd1d0750 2024-09-10T21:47:21.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.263+0000 7f68f1e00640 1 --2- 172.21.15.7:0/2441725377 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x5620dd1d0750 0x5620dd1d0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.263+0000 7f68f1e00640 1 --2- 172.21.15.7:0/2441725377 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x5620dd1d0750 0x5620dd1d0b80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1e02244fc55ea2e server_cookie=2554d418b30e822c in_seq=0 out_seq=0 2024-09-10T21:47:21.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.263+0000 7f68d3400640 10 client.6537 ms_handle_connect on v2:172.21.15.133:6834/2622530239 2024-09-10T21:47:21.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.264+0000 7f68d3400640 1 -- 172.21.15.7:0/2441725377 <== mds.0 v2:172.21.15.133:6834/2622530239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5620dd11d7c0 con 0x5620dd1d0750 2024-09-10T21:47:21.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.264+0000 7f68d3400640 10 client.6537 handle_command_reply: tid=0 2024-09-10T21:47:21.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.264+0000 7f68fc1ff740 1 client.6537 shutdown 2024-09-10T21:47:21.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.264+0000 7f68fc1ff740 1 -- 172.21.15.7:0/2441725377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1ee130 msgr2=0x5620dd1e6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.264+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/2441725377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1ee130 0x5620dd1e6070 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f68f40b1d60 tx=0x7f68e00036f0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.264+0000 7f68fc1ff740 1 -- 172.21.15.7:0/2441725377 shutdown_connections 2024-09-10T21:47:21.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.264+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/2441725377 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd200120 0x5620dd1d2230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.264+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/2441725377 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x5620dd1d0750 0x5620dd1d0b80 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.264+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/2441725377 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1ee130 0x5620dd1e6070 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.264+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/2441725377 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1a9820 0x5620dd1ffbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.264+0000 7f68d3400640 0 client.6537 ms_handle_reset on v2:172.21.15.133:6834/2622530239 2024-09-10T21:47:21.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.264+0000 7f68fc1ff740 1 -- 172.21.15.7:0/2441725377 >> 172.21.15.7:0/2441725377 conn(0x5620dd1a92a0 msgr2=0x5620dd1a3440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.264+0000 7f68fc1ff740 1 -- 172.21.15.7:0/2441725377 shutdown_connections 2024-09-10T21:47:21.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.265+0000 7f68fc1ff740 1 -- 172.21.15.7:0/2441725377 wait complete. 2024-09-10T21:47:21.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.265+0000 7f68fc1ff740 20 client.6537 trim_cache size 0 max 16384 2024-09-10T21:47:21.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.283+0000 7f68fc1ff740 1 Processor -- start 2024-09-10T21:47:21.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.283+0000 7f68fc1ff740 1 -- start start 2024-09-10T21:47:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.284+0000 7f68fc1ff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1a9820 0x5620dd1d11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.284+0000 7f68fc1ff740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd1ee130 0x5620dd1a35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.284+0000 7f68fc1ff740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1a3ae0 0x5620dd1e32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.284+0000 7f68fc1ff740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68f42013b0 con 0x5620dd1a9820 2024-09-10T21:47:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.284+0000 7f68fc1ff740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f68f4141730 con 0x5620dd1ee130 2024-09-10T21:47:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.284+0000 7f68fc1ff740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f68f41f9370 con 0x5620dd1a3ae0 2024-09-10T21:47:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.284+0000 7f68f1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1a9820 0x5620dd1d11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.284+0000 7f68f1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1a9820 0x5620dd1d11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60860/0 (socket says 172.21.15.7:60860) 2024-09-10T21:47:21.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.284+0000 7f68f1e00640 1 -- 172.21.15.7:0/3578227547 learned_addr learned my addr 172.21.15.7:0/3578227547 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:21.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.284+0000 7f68f1400640 1 --2- 172.21.15.7:0/3578227547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd1ee130 0x5620dd1a35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.284+0000 7f68f2800640 1 --2- 172.21.15.7:0/3578227547 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1a3ae0 0x5620dd1e32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.284+0000 7f68f1e00640 1 -- 172.21.15.7:0/3578227547 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1a3ae0 msgr2=0x5620dd1e32b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:21.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.284+0000 7f68f1e00640 1 --2- 172.21.15.7:0/3578227547 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1a3ae0 0x5620dd1e32b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.284+0000 7f68f1e00640 1 -- 172.21.15.7:0/3578227547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd1ee130 msgr2=0x5620dd1a35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:21.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.284+0000 7f68f1e00640 1 --2- 172.21.15.7:0/3578227547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd1ee130 0x5620dd1a35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.284+0000 7f68f1e00640 1 -- 172.21.15.7:0/3578227547 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620dd1d25a0 con 0x5620dd1a9820 2024-09-10T21:47:21.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.285+0000 7f68f1e00640 1 --2- 172.21.15.7:0/3578227547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1a9820 0x5620dd1d11b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f68e4083370 tx=0x7f68e4082de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc7d415134bb47c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.285+0000 7f68de000640 1 -- 172.21.15.7:0/3578227547 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e4088150 con 0x5620dd1a9820 2024-09-10T21:47:21.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.285+0000 7f68de000640 1 -- 172.21.15.7:0/3578227547 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68e408cb70 con 0x5620dd1a9820 2024-09-10T21:47:21.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.285+0000 7f68de000640 1 -- 172.21.15.7:0/3578227547 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e4083670 con 0x5620dd1a9820 2024-09-10T21:47:21.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.285+0000 7f68fc1ff740 1 -- 172.21.15.7:0/3578227547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1a9820 msgr2=0x5620dd1d11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.285+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/3578227547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1a9820 0x5620dd1d11b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f68e4083370 tx=0x7f68e4082de0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.286+0000 7f68fc1ff740 1 -- 172.21.15.7:0/3578227547 shutdown_connections 2024-09-10T21:47:21.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.286+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/3578227547 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd1ee130 0x5620dd1a35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.286+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/3578227547 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1a3ae0 0x5620dd1e32b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.286+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/3578227547 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1a9820 0x5620dd1d11b0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.286+0000 7f68fc1ff740 1 -- 172.21.15.7:0/3578227547 >> 172.21.15.7:0/3578227547 conn(0x5620dd1a92a0 msgr2=0x5620dd1ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.286+0000 7f68fc1ff740 1 -- 172.21.15.7:0/3578227547 shutdown_connections 2024-09-10T21:47:21.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.286+0000 7f68fc1ff740 1 -- 172.21.15.7:0/3578227547 wait complete. 2024-09-10T21:47:21.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.286+0000 7f68fc1ff740 1 -- start start 2024-09-10T21:47:21.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.286+0000 7f68fc1ff740 1 Processor -- start 2024-09-10T21:47:21.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.287+0000 7f68fc1ff740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd1a3ae0 0x5620dd1cd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.287+0000 7f68fc1ff740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1a9820 0x5620dd1cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.287+0000 7f68fc1ff740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1ee130 0x5620dd1f0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.287+0000 7f68fc1ff740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68f41410e0 con 0x5620dd1a9820 2024-09-10T21:47:21.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.287+0000 7f68fc1ff740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5620dd1ce4a0 con 0x5620dd1a3ae0 2024-09-10T21:47:21.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.287+0000 7f68fc1ff740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5620dd265ac0 con 0x5620dd1ee130 2024-09-10T21:47:21.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.288+0000 7f68fc1ff740 10 client.0 _ll_register_callbacks cb 0x7f68e0007d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:47:21.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.288+0000 7f68f1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1a9820 0x5620dd1cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.288+0000 7f68f1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1a9820 0x5620dd1cdb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60872/0 (socket says 172.21.15.7:60872) 2024-09-10T21:47:21.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.288+0000 7f68f1400640 1 -- 172.21.15.7:0/3158550174 learned_addr learned my addr 172.21.15.7:0/3158550174 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:21.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.288+0000 7f68f1e00640 1 --2- 172.21.15.7:0/3158550174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd1a3ae0 0x5620dd1cd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.288+0000 7f68f2800640 1 --2- 172.21.15.7:0/3158550174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1ee130 0x5620dd1f0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.288+0000 7f68f1400640 1 -- 172.21.15.7:0/3158550174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1ee130 msgr2=0x5620dd1f0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:21.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.288+0000 7f68f1400640 1 --2- 172.21.15.7:0/3158550174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1ee130 0x5620dd1f0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.288+0000 7f68f1400640 1 -- 172.21.15.7:0/3158550174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd1a3ae0 msgr2=0x5620dd1cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:21.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.288+0000 7f68f1400640 1 --2- 172.21.15.7:0/3158550174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd1a3ae0 0x5620dd1cd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.288+0000 7f68f1400640 1 -- 172.21.15.7:0/3158550174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620dd265310 con 0x5620dd1a9820 2024-09-10T21:47:21.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.288+0000 7f68f1e00640 1 --2- 172.21.15.7:0/3158550174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd1a3ae0 0x5620dd1cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:47:21.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.289+0000 7f68f1400640 1 --2- 172.21.15.7:0/3158550174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1a9820 0x5620dd1cdb30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009010 tx=0x7f68e0004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68865d027870ddaf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.289+0000 7f68d3400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:47:21.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.289+0000 7f68d3400640 1 -- 172.21.15.7:0/3158550174 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e0004620 con 0x5620dd1a9820 2024-09-10T21:47:21.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.289+0000 7f68d3400640 1 -- 172.21.15.7:0/3158550174 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f68e0025040 con 0x5620dd1a9820 2024-09-10T21:47:21.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.289+0000 7f68fc1ff740 1 -- 172.21.15.7:0/3158550174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68e408cb70 con 0x5620dd1a9820 2024-09-10T21:47:21.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.289+0000 7f68d3400640 1 -- 172.21.15.7:0/3158550174 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f68e003f050 con 0x5620dd1a9820 2024-09-10T21:47:21.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.289+0000 7f68fc1ff740 10 client.6543 fetch_fsmap learned FSMap version 54 2024-09-10T21:47:21.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.289+0000 7f68fc1ff740 1 -- 172.21.15.7:0/3158550174 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x5620dd2657c0 con 0x5620dd1a9820 2024-09-10T21:47:21.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.289+0000 7f68d3400640 1 -- 172.21.15.7:0/3158550174 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68e408cb70 con 0x5620dd1a9820 2024-09-10T21:47:21.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.289+0000 7f68d3400640 1 -- 172.21.15.7:0/3158550174 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7f68e0014070 con 0x5620dd1a9820 2024-09-10T21:47:21.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.289+0000 7f68fc1ff740 10 client.6543 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T21:47:21.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.289+0000 7f68fc1ff740 10 client.6543 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.b 2024-09-10T21:47:21.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.289+0000 7f68fc1ff740 20 client.6543 populate_metadata read hostname 'smithi007' 2024-09-10T21:47:21.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.289+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/3158550174 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x5620dd1cffb0 0x5620dd1f08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.290+0000 7f68fc1ff740 4 client.6543 mds_command: new command op to 235490 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:47:21.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.290+0000 7f68fc1ff740 1 -- 172.21.15.7:0/3158550174 --> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5620dd119270 con 0x5620dd1cffb0 2024-09-10T21:47:21.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.290+0000 7f68f1e00640 1 --2- 172.21.15.7:0/3158550174 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x5620dd1cffb0 0x5620dd1f08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.291+0000 7f68f1e00640 1 --2- 172.21.15.7:0/3158550174 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x5620dd1cffb0 0x5620dd1f08a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71bb0a0d96378d4e server_cookie=2dd89c6018a30613 in_seq=0 out_seq=0 2024-09-10T21:47:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.291+0000 7f68d3400640 10 client.6543 ms_handle_connect on v2:172.21.15.133:6834/2622530239 2024-09-10T21:47:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.291+0000 7f68d3400640 1 -- 172.21.15.7:0/3158550174 <== mds.0 v2:172.21.15.133:6834/2622530239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+670 (crc 0 0 0) 0x5620dd119270 con 0x5620dd1cffb0 2024-09-10T21:47:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.291+0000 7f68d3400640 10 client.6543 handle_command_reply: tid=0 2024-09-10T21:47:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.291+0000 7f68fc1ff740 1 client.6543 shutdown 2024-09-10T21:47:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.291+0000 7f68fc1ff740 1 -- 172.21.15.7:0/3158550174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1a9820 msgr2=0x5620dd1cdb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.291+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/3158550174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1a9820 0x5620dd1cdb30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009010 tx=0x7f68e0004220 comp rx=0 tx=0).stop 2024-09-10T21:47:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.291+0000 7f68fc1ff740 1 -- 172.21.15.7:0/3158550174 shutdown_connections 2024-09-10T21:47:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.291+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/3158550174 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x5620dd1cffb0 0x5620dd1f08a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.291+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/3158550174 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5620dd1ee130 0x5620dd1f0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.291+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/3158550174 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5620dd1a9820 0x5620dd1cdb30 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.291+0000 7f68d3400640 0 client.6543 ms_handle_reset on v2:172.21.15.133:6834/2622530239 2024-09-10T21:47:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.291+0000 7f68fc1ff740 1 --2- 172.21.15.7:0/3158550174 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5620dd1a3ae0 0x5620dd1cd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.291+0000 7f68fc1ff740 1 -- 172.21.15.7:0/3158550174 >> 172.21.15.7:0/3158550174 conn(0x5620dd1a92a0 msgr2=0x5620dd1e56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.291+0000 7f68fc1ff740 1 -- 172.21.15.7:0/3158550174 shutdown_connections 2024-09-10T21:47:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.291+0000 7f68fc1ff740 1 -- 172.21.15.7:0/3158550174 wait complete. 2024-09-10T21:47:21.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.292+0000 7f68fc1ff740 20 client.6543 trim_cache size 0 max 16384 2024-09-10T21:47:21.298 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:47:21.298 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-09-10T21:47:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {}, 2024-09-10T21:47:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:47:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "a08b336a-e18a-4f20-8d7c-ebce794cf011": { 2024-09-10T21:47:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:46:50.825662+0000", 2024-09-10T21:47:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/subdir", 2024-09-10T21:47:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:47:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:47:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:47:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:47:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:21.299 INFO:teuthology.orchestra.run.smithi007.stdout: "d158ff07-06ba-4bd2-9cbd-cc6aad170cea": { 2024-09-10T21:47:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:46:19.061276+0000", 2024-09-10T21:47:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/subdir", 2024-09-10T21:47:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:47:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:47:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:47:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:47:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:21.300 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:21.300 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:47:21.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.292+0000 7f68f9200640 1 -- 172.21.15.7:0/2403814231 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f68d4077f70 msgr2=0x7f68d407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.292+0000 7f68f9200640 1 --2- 172.21.15.7:0/2403814231 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f68d4077f70 0x7f68d407a430 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f68f4143d30 tx=0x7f68e0009040 comp rx=0 tx=0).stop 2024-09-10T21:47:21.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.292+0000 7f68f9200640 1 -- 172.21.15.7:0/2403814231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f40ca960 msgr2=0x7f68f41577e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.292+0000 7f68f9200640 1 --2- 172.21.15.7:0/2403814231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f40ca960 0x7f68f41577e0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f68e4000c00 tx=0x7f68e4004430 comp rx=0 tx=0).stop 2024-09-10T21:47:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.292+0000 7f68f9200640 1 -- 172.21.15.7:0/2403814231 shutdown_connections 2024-09-10T21:47:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.292+0000 7f68f9200640 1 --2- 172.21.15.7:0/2403814231 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f68d4077f70 0x7f68d407a430 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.292+0000 7f68f9200640 1 --2- 172.21.15.7:0/2403814231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68f40cec80 0x7f68f4154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.292+0000 7f68f9200640 1 --2- 172.21.15.7:0/2403814231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68f40cf060 0x7f68f4155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.292+0000 7f68f9200640 1 --2- 172.21.15.7:0/2403814231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f68f40ca960 0x7f68f41577e0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.292+0000 7f68f9200640 1 -- 172.21.15.7:0/2403814231 >> 172.21.15.7:0/2403814231 conn(0x7f68f40b16d0 msgr2=0x7f68f40cd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.292+0000 7f68f9200640 1 -- 172.21.15.7:0/2403814231 shutdown_connections 2024-09-10T21:47:21.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.292+0000 7f68f9200640 1 -- 172.21.15.7:0/2403814231 wait complete. 2024-09-10T21:47:21.308 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:47:21.308 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:47:21.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.467+0000 7f0e91c00640 1 -- 172.21.15.7:0/3590641225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e8c1619e0 msgr2=0x7f0e8c15e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.467+0000 7f0e91c00640 1 --2- 172.21.15.7:0/3590641225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e8c1619e0 0x7f0e8c15e3c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c009860 tx=0x7f0e7c02a5a0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.467+0000 7f0e91c00640 1 -- 172.21.15.7:0/3590641225 shutdown_connections 2024-09-10T21:47:21.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.467+0000 7f0e91c00640 1 --2- 172.21.15.7:0/3590641225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e8c1619e0 0x7f0e8c15e3c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.467+0000 7f0e91c00640 1 -- 172.21.15.7:0/3590641225 >> 172.21.15.7:0/3590641225 conn(0x7f0e8c0b16d0 msgr2=0x7f0e8c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.468+0000 7f0e91c00640 1 -- 172.21.15.7:0/3590641225 shutdown_connections 2024-09-10T21:47:21.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.468+0000 7f0e91c00640 1 -- 172.21.15.7:0/3590641225 wait complete. 2024-09-10T21:47:21.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.469+0000 7f0e91c00640 1 Processor -- start 2024-09-10T21:47:21.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.469+0000 7f0e91c00640 1 -- start start 2024-09-10T21:47:21.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.470+0000 7f0e91c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8c0cbd50 0x7f0e8c148930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.470+0000 7f0e91c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e8c0cc190 0x7f0e8c1f8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.470+0000 7f0e91c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8c1619e0 0x7f0e8c1f87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.470+0000 7f0e91c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c161f40 con 0x7f0e8c0cbd50 2024-09-10T21:47:21.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.470+0000 7f0e91c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0e8c161dc0 con 0x7f0e8c1619e0 2024-09-10T21:47:21.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.470+0000 7f0e91c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f0e8c1620c0 con 0x7f0e8c0cc190 2024-09-10T21:47:21.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.470+0000 7f0e8aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8c0cbd50 0x7f0e8c148930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.471+0000 7f0e8aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8c0cbd50 0x7f0e8c148930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60882/0 (socket says 172.21.15.7:60882) 2024-09-10T21:47:21.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.471+0000 7f0e8aa00640 1 -- 172.21.15.7:0/4273467895 learned_addr learned my addr 172.21.15.7:0/4273467895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:21.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.471+0000 7f0e82000640 1 --2- 172.21.15.7:0/4273467895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e8c0cc190 0x7f0e8c1f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.471+0000 7f0e8b400640 1 --2- 172.21.15.7:0/4273467895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8c1619e0 0x7f0e8c1f87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.471+0000 7f0e8aa00640 1 -- 172.21.15.7:0/4273467895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e8c0cc190 msgr2=0x7f0e8c1f8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.471+0000 7f0e8aa00640 1 --2- 172.21.15.7:0/4273467895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e8c0cc190 0x7f0e8c1f8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.471+0000 7f0e8aa00640 1 -- 172.21.15.7:0/4273467895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8c1619e0 msgr2=0x7f0e8c1f87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.471+0000 7f0e8aa00640 1 --2- 172.21.15.7:0/4273467895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8c1619e0 0x7f0e8c1f87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.471+0000 7f0e8aa00640 1 -- 172.21.15.7:0/4273467895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e8c0c5f50 con 0x7f0e8c0cbd50 2024-09-10T21:47:21.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.471+0000 7f0e8b400640 1 --2- 172.21.15.7:0/4273467895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8c1619e0 0x7f0e8c1f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:21.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.471+0000 7f0e8aa00640 1 --2- 172.21.15.7:0/4273467895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8c0cbd50 0x7f0e8c148930 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c02ab90 tx=0x7f0e7c032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.472+0000 7f0e83400640 1 -- 172.21.15.7:0/4273467895 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c040070 con 0x7f0e8c0cbd50 2024-09-10T21:47:21.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.472+0000 7f0e91c00640 1 -- 172.21.15.7:0/4273467895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e8c0c6230 con 0x7f0e8c0cbd50 2024-09-10T21:47:21.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.472+0000 7f0e83400640 1 -- 172.21.15.7:0/4273467895 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e7c005310 con 0x7f0e8c0cbd50 2024-09-10T21:47:21.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.472+0000 7f0e83400640 1 -- 172.21.15.7:0/4273467895 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7c0047a0 con 0x7f0e8c0cbd50 2024-09-10T21:47:21.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.472+0000 7f0e91c00640 1 -- 172.21.15.7:0/4273467895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e8c0c67f0 con 0x7f0e8c0cbd50 2024-09-10T21:47:21.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.473+0000 7f0e91c00640 1 -- 172.21.15.7:0/4273467895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e8c1625e0 con 0x7f0e8c0cbd50 2024-09-10T21:47:21.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.474+0000 7f0e83400640 1 -- 172.21.15.7:0/4273467895 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0e7c004940 con 0x7f0e8c0cbd50 2024-09-10T21:47:21.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.475+0000 7f0e83400640 1 --2- 172.21.15.7:0/4273467895 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0e68077f70 0x7f0e6807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.475+0000 7f0e83400640 1 -- 172.21.15.7:0/4273467895 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f0e7c0bb170 con 0x7f0e8c0cbd50 2024-09-10T21:47:21.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.475+0000 7f0e82000640 1 --2- 172.21.15.7:0/4273467895 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0e68077f70 0x7f0e6807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.476+0000 7f0e82000640 1 --2- 172.21.15.7:0/4273467895 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0e68077f70 0x7f0e6807a430 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c0c6d00 tx=0x7f0e74007cc0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.479+0000 7f0e83400640 1 -- 172.21.15.7:0/4273467895 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0e7c083b60 con 0x7f0e8c0cbd50 2024-09-10T21:47:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.712+0000 7f0e91c00640 1 -- 172.21.15.7:0/4273467895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e8c0b3a20 con 0x7f0e8c0cbd50 2024-09-10T21:47:21.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.712+0000 7f0e83400640 1 -- 172.21.15.7:0/4273467895 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5237 (secure 0 0 0) 0x7f0e7c083460 con 0x7f0e8c0cbd50 2024-09-10T21:47:21.714 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:21.714 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":54,"btime":"2024-09-10T21:46:58:078261+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5884,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3488410478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3488410478},{"type":"v1","addr":"172.21.15.133:6837","nonce":3488410478}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5890,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3660400727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3660400727},{"type":"v1","addr":"172.21.15.7:6835","nonce":3660400727}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2388873704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2388873704},{"type":"v1","addr":"172.21.15.7:6834","nonce":2388873704}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:46:00.260975+0000","modified":"2024-09-10T21:46:57.678038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235490":{"gid":235490,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":15,"addr":"172.21.15.133:6836/2622530239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2622530239},{"type":"v1","addr":"172.21.15.133:6836","nonce":2622530239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235490,"qdb_cluster":[235490]},"id":5}]} 2024-09-10T21:47:21.714 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 54 2024-09-10T21:47:21.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.714+0000 7f0e91c00640 1 -- 172.21.15.7:0/4273467895 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0e68077f70 msgr2=0x7f0e6807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.714+0000 7f0e91c00640 1 --2- 172.21.15.7:0/4273467895 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0e68077f70 0x7f0e6807a430 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c0c6d00 tx=0x7f0e74007cc0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.715+0000 7f0e91c00640 1 -- 172.21.15.7:0/4273467895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8c0cbd50 msgr2=0x7f0e8c148930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.715+0000 7f0e91c00640 1 --2- 172.21.15.7:0/4273467895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8c0cbd50 0x7f0e8c148930 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f0e7c02ab90 tx=0x7f0e7c032b90 comp rx=0 tx=0).stop 2024-09-10T21:47:21.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.715+0000 7f0e91c00640 1 -- 172.21.15.7:0/4273467895 shutdown_connections 2024-09-10T21:47:21.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.715+0000 7f0e91c00640 1 --2- 172.21.15.7:0/4273467895 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0e68077f70 0x7f0e6807a430 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.715+0000 7f0e91c00640 1 --2- 172.21.15.7:0/4273467895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0e8c1619e0 0x7f0e8c1f87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.715+0000 7f0e91c00640 1 --2- 172.21.15.7:0/4273467895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0e8c0cc190 0x7f0e8c1f8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.715+0000 7f0e91c00640 1 --2- 172.21.15.7:0/4273467895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0e8c0cbd50 0x7f0e8c148930 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.715+0000 7f0e91c00640 1 -- 172.21.15.7:0/4273467895 >> 172.21.15.7:0/4273467895 conn(0x7f0e8c0b16d0 msgr2=0x7f0e8c160850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.715+0000 7f0e91c00640 1 -- 172.21.15.7:0/4273467895 shutdown_connections 2024-09-10T21:47:21.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.715+0000 7f0e91c00640 1 -- 172.21.15.7:0/4273467895 wait complete. 2024-09-10T21:47:21.732 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b damage ls --format=json-pretty 2024-09-10T21:47:21.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.895+0000 7f1c0fe00640 1 -- 172.21.15.7:0/3087510662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c08073e20 msgr2=0x7f1c080742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.895+0000 7f1c0fe00640 1 --2- 172.21.15.7:0/3087510662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c08073e20 0x7f1c080742d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1c00005fd0 tx=0x7f1c0002a820 comp rx=0 tx=0).stop 2024-09-10T21:47:21.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.895+0000 7f1c0fe00640 1 -- 172.21.15.7:0/3087510662 shutdown_connections 2024-09-10T21:47:21.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.895+0000 7f1c0fe00640 1 --2- 172.21.15.7:0/3087510662 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c08073e20 0x7f1c080742d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.895+0000 7f1c0fe00640 1 --2- 172.21.15.7:0/3087510662 >> v2:172.21.15.133:3300/0 conn(0x7f1c0806e580 0x7f1c0815cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.895+0000 7f1c0fe00640 1 -- 172.21.15.7:0/3087510662 >> 172.21.15.7:0/3087510662 conn(0x7f1c080b16d0 msgr2=0x7f1c080b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.895+0000 7f1c0fe00640 1 -- 172.21.15.7:0/3087510662 shutdown_connections 2024-09-10T21:47:21.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.895+0000 7f1c0fe00640 1 -- 172.21.15.7:0/3087510662 wait complete. 2024-09-10T21:47:21.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.897+0000 7f1c0fe00640 1 Processor -- start 2024-09-10T21:47:21.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.897+0000 7f1c0fe00640 1 -- start start 2024-09-10T21:47:21.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.898+0000 7f1c0fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c0806e580 0x7f1c081f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.898+0000 7f1c0fe00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c08073e20 0x7f1c08154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.898+0000 7f1c0fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c08159680 0x7f1c080c4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.898+0000 7f1c0fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c080ca990 con 0x7f1c0806e580 2024-09-10T21:47:21.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.898+0000 7f1c0fe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1c080ca810 con 0x7f1c08159680 2024-09-10T21:47:21.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.898+0000 7f1c0fe00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f1c080cab10 con 0x7f1c08073e20 2024-09-10T21:47:21.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.898+0000 7f1c0d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c0806e580 0x7f1c081f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.898+0000 7f1c0d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c0806e580 0x7f1c081f50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60890/0 (socket says 172.21.15.7:60890) 2024-09-10T21:47:21.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.898+0000 7f1c0d400640 1 -- 172.21.15.7:0/589042675 learned_addr learned my addr 172.21.15.7:0/589042675 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:21.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.898+0000 7f1c0de00640 1 --2- 172.21.15.7:0/589042675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c08159680 0x7f1c080c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.898+0000 7f1c0ca00640 1 --2- 172.21.15.7:0/589042675 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c08073e20 0x7f1c08154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.899+0000 7f1c0d400640 1 -- 172.21.15.7:0/589042675 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c08073e20 msgr2=0x7f1c08154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.899+0000 7f1c0d400640 1 --2- 172.21.15.7:0/589042675 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c08073e20 0x7f1c08154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.899+0000 7f1c0d400640 1 -- 172.21.15.7:0/589042675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c08159680 msgr2=0x7f1c080c4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.899+0000 7f1c0d400640 1 --2- 172.21.15.7:0/589042675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c08159680 0x7f1c080c4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.899+0000 7f1c0d400640 1 -- 172.21.15.7:0/589042675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c080c36b0 con 0x7f1c0806e580 2024-09-10T21:47:21.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.899+0000 7f1c0d400640 1 --2- 172.21.15.7:0/589042675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c0806e580 0x7f1c081f50a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00c3a0 tx=0x7f1bfc00c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.899+0000 7f1c05600640 1 -- 172.21.15.7:0/589042675 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc017070 con 0x7f1c0806e580 2024-09-10T21:47:21.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.899+0000 7f1c0fe00640 1 -- 172.21.15.7:0/589042675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c081f55e0 con 0x7f1c0806e580 2024-09-10T21:47:21.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.899+0000 7f1c05600640 1 -- 172.21.15.7:0/589042675 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1bfc0053c0 con 0x7f1c0806e580 2024-09-10T21:47:21.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.899+0000 7f1c05600640 1 -- 172.21.15.7:0/589042675 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc003850 con 0x7f1c0806e580 2024-09-10T21:47:21.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.899+0000 7f1c0fe00640 1 -- 172.21.15.7:0/589042675 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c080b3c80 con 0x7f1c0806e580 2024-09-10T21:47:21.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.902+0000 7f1c05600640 1 -- 172.21.15.7:0/589042675 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1bfc0039f0 con 0x7f1c0806e580 2024-09-10T21:47:21.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.903+0000 7f1c05600640 1 --2- 172.21.15.7:0/589042675 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1be8077f70 0x7f1be807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.903+0000 7f1c05600640 1 -- 172.21.15.7:0/589042675 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f1bfc099420 con 0x7f1c0806e580 2024-09-10T21:47:21.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.903+0000 7f1c12dd6740 1 Processor -- start 2024-09-10T21:47:21.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.903+0000 7f1c12dd6740 1 -- start start 2024-09-10T21:47:21.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.903+0000 7f1c0ca00640 1 --2- 172.21.15.7:0/589042675 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1be8077f70 0x7f1be807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.904+0000 7f1c12dd6740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad95587a0 0x556ad9555640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.904+0000 7f1c12dd6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad956a1c0 0x556ad9558b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.904+0000 7f1c12dd6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad95590c0 0x556ad9529ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.904+0000 7f1c12dd6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556ad94a0750 con 0x556ad95590c0 2024-09-10T21:47:21.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.904+0000 7f1c12dd6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556ad948fb80 con 0x556ad956a1c0 2024-09-10T21:47:21.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.904+0000 7f1c12dd6740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x556ad9493990 con 0x556ad95587a0 2024-09-10T21:47:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.904+0000 7f1c0ca00640 1 --2- 172.21.15.7:0/589042675 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1be8077f70 0x7f1be807a430 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1c080b5750 tx=0x7f1bf4009260 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.904+0000 7f1c0de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad95590c0 0x556ad9529ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.904+0000 7f1c0d400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad95587a0 0x556ad9555640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.904+0000 7f1c0de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad95590c0 0x556ad9529ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60892/0 (socket says 172.21.15.7:60892) 2024-09-10T21:47:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.904+0000 7f1c0de00640 1 -- 172.21.15.7:0/2855374218 learned_addr learned my addr 172.21.15.7:0/2855374218 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.904+0000 7f1c0ca00640 1 --2- 172.21.15.7:0/2855374218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad956a1c0 0x556ad9558b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.904+0000 7f1c0d400640 1 -- 172.21.15.7:0/2855374218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad956a1c0 msgr2=0x556ad9558b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.904+0000 7f1c0d400640 1 --2- 172.21.15.7:0/2855374218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad956a1c0 0x556ad9558b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.905+0000 7f1c0d400640 1 -- 172.21.15.7:0/2855374218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad95590c0 msgr2=0x556ad9529ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.905+0000 7f1c0d400640 1 --2- 172.21.15.7:0/2855374218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad95590c0 0x556ad9529ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.905+0000 7f1c0d400640 1 -- 172.21.15.7:0/2855374218 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad941dad0 con 0x556ad95587a0 2024-09-10T21:47:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.905+0000 7f1c0d400640 1 --2- 172.21.15.7:0/2855374218 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad95587a0 0x556ad9555640 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00cfd0 tx=0x7f1bfc000ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e42c9c5a48eafc33 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.905+0000 7f1bf9c00640 1 -- 172.21.15.7:0/2855374218 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc0661e0 con 0x556ad95587a0 2024-09-10T21:47:21.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.905+0000 7f1bf9c00640 1 -- 172.21.15.7:0/2855374218 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1bfc05da20 con 0x556ad95587a0 2024-09-10T21:47:21.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.905+0000 7f1bf9c00640 1 -- 172.21.15.7:0/2855374218 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc060c70 con 0x556ad95587a0 2024-09-10T21:47:21.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.906+0000 7f1c12dd6740 1 -- 172.21.15.7:0/2855374218 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad95587a0 msgr2=0x556ad9555640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.906+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/2855374218 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad95587a0 0x556ad9555640 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00cfd0 tx=0x7f1bfc000ed0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.906+0000 7f1c12dd6740 1 -- 172.21.15.7:0/2855374218 shutdown_connections 2024-09-10T21:47:21.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.906+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/2855374218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad956a1c0 0x556ad9558b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.906+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/2855374218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad95590c0 0x556ad9529ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.906+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/2855374218 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad95587a0 0x556ad9555640 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.906+0000 7f1c12dd6740 1 -- 172.21.15.7:0/2855374218 >> 172.21.15.7:0/2855374218 conn(0x556ad952ace0 msgr2=0x556ad952e700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.906+0000 7f1c12dd6740 1 -- 172.21.15.7:0/2855374218 shutdown_connections 2024-09-10T21:47:21.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.906+0000 7f1c12dd6740 1 -- 172.21.15.7:0/2855374218 wait complete. 2024-09-10T21:47:21.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.907+0000 7f1c12dd6740 1 -- start start 2024-09-10T21:47:21.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.907+0000 7f1c12dd6740 1 Processor -- start 2024-09-10T21:47:21.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.908+0000 7f1c12dd6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad95590c0 0x556ad9541ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.908+0000 7f1c12dd6740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad956a1c0 0x556ad9577180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.908+0000 7f1c12dd6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad9575420 0x556ad9557570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.908+0000 7f1c12dd6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556ad9200700 con 0x556ad9575420 2024-09-10T21:47:21.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.908+0000 7f1c12dd6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556ad9435430 con 0x556ad95590c0 2024-09-10T21:47:21.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.908+0000 7f1c12dd6740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x556ad952f780 con 0x556ad956a1c0 2024-09-10T21:47:21.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.908+0000 7f1c12dd6740 10 client.0 _ll_register_callbacks cb 0x556ad943b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:47:21.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.908+0000 7f1c0de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad9575420 0x556ad9557570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.908+0000 7f1c0de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad9575420 0x556ad9557570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60898/0 (socket says 172.21.15.7:60898) 2024-09-10T21:47:21.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.908+0000 7f1c0de00640 1 -- 172.21.15.7:0/2799226301 learned_addr learned my addr 172.21.15.7:0/2799226301 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:21.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.908+0000 7f1c0d400640 1 --2- 172.21.15.7:0/2799226301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad95590c0 0x556ad9541ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.908+0000 7f1c0ca00640 1 --2- 172.21.15.7:0/2799226301 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad956a1c0 0x556ad9577180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.908+0000 7f1c0de00640 1 -- 172.21.15.7:0/2799226301 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad956a1c0 msgr2=0x556ad9577180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:21.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.908+0000 7f1c0de00640 1 --2- 172.21.15.7:0/2799226301 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad956a1c0 0x556ad9577180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.908+0000 7f1c0de00640 1 -- 172.21.15.7:0/2799226301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad95590c0 msgr2=0x556ad9541ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:21.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.908+0000 7f1c0de00640 1 --2- 172.21.15.7:0/2799226301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad95590c0 0x556ad9541ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.908+0000 7f1c0de00640 1 -- 172.21.15.7:0/2799226301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad926a7c0 con 0x556ad9575420 2024-09-10T21:47:21.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.908+0000 7f1c0ca00640 1 --2- 172.21.15.7:0/2799226301 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad956a1c0 0x556ad9577180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:47:21.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.909+0000 7f1c0de00640 1 --2- 172.21.15.7:0/2799226301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad9575420 0x556ad9557570 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1c00003f40 tx=0x7f1c00003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f313df2f702bdc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.909+0000 7f1bee000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:47:21.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.909+0000 7f1bee000640 1 -- 172.21.15.7:0/2799226301 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00042070 con 0x556ad9575420 2024-09-10T21:47:21.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.909+0000 7f1bee000640 1 -- 172.21.15.7:0/2799226301 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f1c00004250 con 0x556ad9575420 2024-09-10T21:47:21.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.909+0000 7f1c12dd6740 1 -- 172.21.15.7:0/2799226301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1bfc05da20 con 0x556ad9575420 2024-09-10T21:47:21.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.909+0000 7f1bee000640 1 -- 172.21.15.7:0/2799226301 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f1c00037040 con 0x556ad9575420 2024-09-10T21:47:21.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.909+0000 7f1c12dd6740 10 client.6558 fetch_fsmap learned FSMap version 54 2024-09-10T21:47:21.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.909+0000 7f1c12dd6740 1 -- 172.21.15.7:0/2799226301 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x556ad92caf50 con 0x556ad9575420 2024-09-10T21:47:21.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.909+0000 7f1bee000640 1 -- 172.21.15.7:0/2799226301 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1bfc05da20 con 0x556ad9575420 2024-09-10T21:47:21.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.909+0000 7f1bee000640 1 -- 172.21.15.7:0/2799226301 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7f1c00052080 con 0x556ad9575420 2024-09-10T21:47:21.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.910+0000 7f1c12dd6740 10 client.6558 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T21:47:21.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.910+0000 7f1c12dd6740 10 client.6558 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:47:21.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.910+0000 7f1c12dd6740 20 client.6558 populate_metadata read hostname 'smithi007' 2024-09-10T21:47:21.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.910+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/2799226301 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x556ad9573630 0x556ad9573a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.910+0000 7f1c12dd6740 4 client.6558 mds_command: new command op to 235490 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:47:21.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.910+0000 7f1c12dd6740 1 -- 172.21.15.7:0/2799226301 --> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ad94a26a0 con 0x556ad9573630 2024-09-10T21:47:21.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.910+0000 7f1c0d400640 1 --2- 172.21.15.7:0/2799226301 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x556ad9573630 0x556ad9573a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.911+0000 7f1c0d400640 1 --2- 172.21.15.7:0/2799226301 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x556ad9573630 0x556ad9573a10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=884780417b4eeec4 server_cookie=31a8f918d2508277 in_seq=0 out_seq=0 2024-09-10T21:47:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.911+0000 7f1bee000640 10 client.6558 ms_handle_connect on v2:172.21.15.133:6834/2622530239 2024-09-10T21:47:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.911+0000 7f1bee000640 1 -- 172.21.15.7:0/2799226301 <== mds.0 v2:172.21.15.133:6834/2622530239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556ad94a26a0 con 0x556ad9573630 2024-09-10T21:47:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.911+0000 7f1bee000640 10 client.6558 handle_command_reply: tid=0 2024-09-10T21:47:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.912+0000 7f1c12dd6740 1 client.6558 shutdown 2024-09-10T21:47:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.912+0000 7f1c12dd6740 1 -- 172.21.15.7:0/2799226301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad9575420 msgr2=0x556ad9557570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.912+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/2799226301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad9575420 0x556ad9557570 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f1c00003f40 tx=0x7f1c00003f70 comp rx=0 tx=0).stop 2024-09-10T21:47:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.912+0000 7f1c12dd6740 1 -- 172.21.15.7:0/2799226301 shutdown_connections 2024-09-10T21:47:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.912+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/2799226301 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x556ad9573630 0x556ad9573a10 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.912+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/2799226301 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad9575420 0x556ad9557570 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.912+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/2799226301 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad956a1c0 0x556ad9577180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.912+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/2799226301 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad95590c0 0x556ad9541ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.912+0000 7f1bee000640 0 client.6558 ms_handle_reset on v2:172.21.15.133:6834/2622530239 2024-09-10T21:47:21.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.912+0000 7f1c12dd6740 1 -- 172.21.15.7:0/2799226301 >> 172.21.15.7:0/2799226301 conn(0x556ad95587a0 msgr2=0x556ad952c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.912+0000 7f1c12dd6740 1 -- 172.21.15.7:0/2799226301 shutdown_connections 2024-09-10T21:47:21.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.912+0000 7f1c12dd6740 1 -- 172.21.15.7:0/2799226301 wait complete. 2024-09-10T21:47:21.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.912+0000 7f1c12dd6740 20 client.6558 trim_cache size 0 max 16384 2024-09-10T21:47:21.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.926+0000 7f1c12dd6740 1 Processor -- start 2024-09-10T21:47:21.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.926+0000 7f1c12dd6740 1 -- start start 2024-09-10T21:47:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.927+0000 7f1c12dd6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad95590c0 0x556ad9573a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.927+0000 7f1c12dd6740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad956a1c0 0x556ad9573f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.927+0000 7f1c12dd6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad9575420 0x556ad952e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.927+0000 7f1c12dd6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1c082003a0 con 0x556ad9575420 2024-09-10T21:47:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.927+0000 7f1c12dd6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1c080b4d80 con 0x556ad95590c0 2024-09-10T21:47:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.927+0000 7f1c12dd6740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f1c080b53d0 con 0x556ad956a1c0 2024-09-10T21:47:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.927+0000 7f1c0de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad9575420 0x556ad952e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.927+0000 7f1c0de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad9575420 0x556ad952e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60906/0 (socket says 172.21.15.7:60906) 2024-09-10T21:47:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.927+0000 7f1c0de00640 1 -- 172.21.15.7:0/28169319 learned_addr learned my addr 172.21.15.7:0/28169319 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:21.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.928+0000 7f1c0ca00640 1 --2- 172.21.15.7:0/28169319 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad956a1c0 0x556ad9573f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.928+0000 7f1c0d400640 1 --2- 172.21.15.7:0/28169319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad95590c0 0x556ad9573a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.928+0000 7f1c0de00640 1 -- 172.21.15.7:0/28169319 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad956a1c0 msgr2=0x556ad9573f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:21.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.928+0000 7f1c0de00640 1 --2- 172.21.15.7:0/28169319 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad956a1c0 0x556ad9573f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.928+0000 7f1c0de00640 1 -- 172.21.15.7:0/28169319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad95590c0 msgr2=0x556ad9573a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:21.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.928+0000 7f1c0de00640 1 --2- 172.21.15.7:0/28169319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad95590c0 0x556ad9573a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.928+0000 7f1c0de00640 1 -- 172.21.15.7:0/28169319 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ad91cf660 con 0x556ad9575420 2024-09-10T21:47:21.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.928+0000 7f1c0de00640 1 --2- 172.21.15.7:0/28169319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad9575420 0x556ad952e3d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009880 tx=0x7f1c000098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17dc28800e5d8f20 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.928+0000 7f1bf9200640 1 -- 172.21.15.7:0/28169319 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00055030 con 0x556ad9575420 2024-09-10T21:47:21.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.928+0000 7f1bf9200640 1 -- 172.21.15.7:0/28169319 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c00004630 con 0x556ad9575420 2024-09-10T21:47:21.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.929+0000 7f1bf9200640 1 -- 172.21.15.7:0/28169319 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c00042040 con 0x556ad9575420 2024-09-10T21:47:21.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.929+0000 7f1c12dd6740 1 -- 172.21.15.7:0/28169319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad9575420 msgr2=0x556ad952e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.929+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/28169319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad9575420 0x556ad952e3d0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f1c00009880 tx=0x7f1c000098b0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.929+0000 7f1c12dd6740 1 -- 172.21.15.7:0/28169319 shutdown_connections 2024-09-10T21:47:21.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.929+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/28169319 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad9575420 0x556ad952e3d0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.929+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/28169319 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad956a1c0 0x556ad9573f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.929+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/28169319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad95590c0 0x556ad9573a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.929+0000 7f1c12dd6740 1 -- 172.21.15.7:0/28169319 >> 172.21.15.7:0/28169319 conn(0x556ad95587a0 msgr2=0x556ad9541ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.929+0000 7f1c12dd6740 1 -- 172.21.15.7:0/28169319 shutdown_connections 2024-09-10T21:47:21.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.929+0000 7f1c12dd6740 1 -- 172.21.15.7:0/28169319 wait complete. 2024-09-10T21:47:21.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.930+0000 7f1c12dd6740 1 -- start start 2024-09-10T21:47:21.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.930+0000 7f1c12dd6740 1 Processor -- start 2024-09-10T21:47:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.931+0000 7f1c12dd6740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad95590c0 0x556ad95f3190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.931+0000 7f1c12dd6740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad956a1c0 0x556ad9573a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.931+0000 7f1c12dd6740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad9575420 0x556ad9573f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.931+0000 7f1c12dd6740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x556ad95758d0 con 0x556ad95590c0 2024-09-10T21:47:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.931+0000 7f1c12dd6740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x556ad9583d30 con 0x556ad9575420 2024-09-10T21:47:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.931+0000 7f1c12dd6740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x556ad9586ce0 con 0x556ad956a1c0 2024-09-10T21:47:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.931+0000 7f1c12dd6740 10 client.0 _ll_register_callbacks cb 0x7f1c00003de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:47:21.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.931+0000 7f1c0d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad95590c0 0x556ad95f3190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.931+0000 7f1c0d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad95590c0 0x556ad95f3190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60920/0 (socket says 172.21.15.7:60920) 2024-09-10T21:47:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.931+0000 7f1c0d400640 1 -- 172.21.15.7:0/3968118200 learned_addr learned my addr 172.21.15.7:0/3968118200 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.931+0000 7f1c0de00640 1 --2- 172.21.15.7:0/3968118200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad9575420 0x556ad9573f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.931+0000 7f1c0ca00640 1 --2- 172.21.15.7:0/3968118200 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad956a1c0 0x556ad9573a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.932+0000 7f1c0d400640 1 -- 172.21.15.7:0/3968118200 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad956a1c0 msgr2=0x556ad9573a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.932+0000 7f1c0d400640 1 --2- 172.21.15.7:0/3968118200 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad956a1c0 0x556ad9573a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.932+0000 7f1c0d400640 1 -- 172.21.15.7:0/3968118200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad9575420 msgr2=0x556ad9573f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.932+0000 7f1c0d400640 1 --2- 172.21.15.7:0/3968118200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad9575420 0x556ad9573f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.932+0000 7f1c0d400640 1 -- 172.21.15.7:0/3968118200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ad945c630 con 0x556ad95590c0 2024-09-10T21:47:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.932+0000 7f1c0de00640 1 --2- 172.21.15.7:0/3968118200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad9575420 0x556ad9573f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.932+0000 7f1c0d400640 1 --2- 172.21.15.7:0/3968118200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad95590c0 0x556ad95f3190 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc05ea20 tx=0x7f1bfc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dce8ecf30cb91d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.932+0000 7f1bee000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:47:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.932+0000 7f1bee000640 1 -- 172.21.15.7:0/3968118200 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bfc060c10 con 0x556ad95590c0 2024-09-10T21:47:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.932+0000 7f1c12dd6740 1 -- 172.21.15.7:0/3968118200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c00004630 con 0x556ad95590c0 2024-09-10T21:47:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.932+0000 7f1bee000640 1 -- 172.21.15.7:0/3968118200 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f1bfc0683d0 con 0x556ad95590c0 2024-09-10T21:47:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.932+0000 7f1bee000640 1 -- 172.21.15.7:0/3968118200 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) ==== 24+0+0 (secure 0 0 0) 0x7f1bfc05da20 con 0x556ad95590c0 2024-09-10T21:47:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.932+0000 7f1c12dd6740 10 client.6564 fetch_fsmap learned FSMap version 54 2024-09-10T21:47:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.932+0000 7f1c12dd6740 1 -- 172.21.15.7:0/3968118200 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) -- 0x556ad95818a0 con 0x556ad95590c0 2024-09-10T21:47:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.933+0000 7f1bee000640 1 -- 172.21.15.7:0/3968118200 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c00004630 con 0x556ad95590c0 2024-09-10T21:47:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.933+0000 7f1bee000640 1 -- 172.21.15.7:0/3968118200 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 54) ==== 3502+0+0 (secure 0 0 0) 0x7f1bfc05d100 con 0x556ad95590c0 2024-09-10T21:47:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.933+0000 7f1c12dd6740 10 client.6564 fetch_fsmap finished waiting for FSMap version 54 2024-09-10T21:47:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.933+0000 7f1c12dd6740 10 client.6564 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:47:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.933+0000 7f1c12dd6740 20 client.6564 populate_metadata read hostname 'smithi007' 2024-09-10T21:47:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.933+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/3968118200 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x556ad956e1c0 0x556ad956e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.933+0000 7f1c12dd6740 4 client.6564 mds_command: new command op to 235490 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-10T21:47:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.933+0000 7f1c12dd6740 1 -- 172.21.15.7:0/3968118200 --> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x556ad949e150 con 0x556ad956e1c0 2024-09-10T21:47:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.933+0000 7f1c0ca00640 1 --2- 172.21.15.7:0/3968118200 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x556ad956e1c0 0x556ad956e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.934+0000 7f1c0ca00640 1 --2- 172.21.15.7:0/3968118200 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x556ad956e1c0 0x556ad956e5a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c5c3b9265258e97 server_cookie=d6b49c7a7c4b57a2 in_seq=0 out_seq=0 2024-09-10T21:47:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.934+0000 7f1bee000640 10 client.6564 ms_handle_connect on v2:172.21.15.133:6834/2622530239 2024-09-10T21:47:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.934+0000 7f1bee000640 1 -- 172.21.15.7:0/3968118200 <== mds.0 v2:172.21.15.133:6834/2622530239 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ad949e150 con 0x556ad956e1c0 2024-09-10T21:47:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.934+0000 7f1bee000640 10 client.6564 handle_command_reply: tid=0 2024-09-10T21:47:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.934+0000 7f1c12dd6740 1 client.6564 shutdown 2024-09-10T21:47:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.934+0000 7f1c12dd6740 1 -- 172.21.15.7:0/3968118200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad95590c0 msgr2=0x556ad95f3190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.934+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/3968118200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad95590c0 0x556ad95f3190 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc05ea20 tx=0x7f1bfc000bc0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.934+0000 7f1c12dd6740 1 -- 172.21.15.7:0/3968118200 shutdown_connections 2024-09-10T21:47:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.934+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/3968118200 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x556ad9575420 0x556ad9573f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.934+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/3968118200 >> [v2:172.21.15.133:6834/2622530239,v1:172.21.15.133:6836/2622530239] conn(0x556ad956e1c0 0x556ad956e5a0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.934+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/3968118200 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x556ad956a1c0 0x556ad9573a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.934+0000 7f1c12dd6740 1 --2- 172.21.15.7:0/3968118200 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x556ad95590c0 0x556ad95f3190 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.934+0000 7f1bee000640 0 client.6564 ms_handle_reset on v2:172.21.15.133:6834/2622530239 2024-09-10T21:47:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.934+0000 7f1c12dd6740 1 -- 172.21.15.7:0/3968118200 >> 172.21.15.7:0/3968118200 conn(0x556ad95587a0 msgr2=0x556ad9541ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.935+0000 7f1c12dd6740 1 -- 172.21.15.7:0/3968118200 shutdown_connections 2024-09-10T21:47:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.935+0000 7f1c12dd6740 1 -- 172.21.15.7:0/3968118200 wait complete. 2024-09-10T21:47:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.935+0000 7f1c12dd6740 20 client.6564 trim_cache size 0 max 16384 2024-09-10T21:47:21.940 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:21.941 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-09-10T21:47:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.935+0000 7f1c0fe00640 1 -- 172.21.15.7:0/589042675 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1be8077f70 msgr2=0x7f1be807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.935+0000 7f1c0fe00640 1 --2- 172.21.15.7:0/589042675 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1be8077f70 0x7f1be807a430 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1c080b5750 tx=0x7f1bf4009260 comp rx=0 tx=0).stop 2024-09-10T21:47:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.935+0000 7f1c0fe00640 1 -- 172.21.15.7:0/589042675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c0806e580 msgr2=0x7f1c081f50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.935+0000 7f1c0fe00640 1 --2- 172.21.15.7:0/589042675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c0806e580 0x7f1c081f50a0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00c3a0 tx=0x7f1bfc00c7a0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.935+0000 7f1c0fe00640 1 -- 172.21.15.7:0/589042675 shutdown_connections 2024-09-10T21:47:21.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.935+0000 7f1c0fe00640 1 --2- 172.21.15.7:0/589042675 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1be8077f70 0x7f1be807a430 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.935+0000 7f1c0fe00640 1 --2- 172.21.15.7:0/589042675 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1c08159680 0x7f1c080c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.935+0000 7f1c0fe00640 1 --2- 172.21.15.7:0/589042675 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1c08073e20 0x7f1c08154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.935+0000 7f1c0fe00640 1 --2- 172.21.15.7:0/589042675 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1c0806e580 0x7f1c081f50a0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.935+0000 7f1c0fe00640 1 -- 172.21.15.7:0/589042675 >> 172.21.15.7:0/589042675 conn(0x7f1c080b16d0 msgr2=0x7f1c0815bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.936+0000 7f1c0fe00640 1 -- 172.21.15.7:0/589042675 shutdown_connections 2024-09-10T21:47:21.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:21.936+0000 7f1c0fe00640 1 -- 172.21.15.7:0/589042675 wait complete. 2024-09-10T21:47:21.952 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:47:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.110+0000 7ff2e1200640 1 -- 172.21.15.7:0/361782939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2dc145540 msgr2=0x7ff2dc0b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.110+0000 7ff2e1200640 1 --2- 172.21.15.7:0/361782939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2dc145540 0x7ff2dc0b4580 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc009860 tx=0x7ff2cc02a540 comp rx=0 tx=0).stop 2024-09-10T21:47:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.110+0000 7ff2e1200640 1 -- 172.21.15.7:0/361782939 shutdown_connections 2024-09-10T21:47:22.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.110+0000 7ff2e1200640 1 --2- 172.21.15.7:0/361782939 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2dc0b3150 0x7ff2dc0ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.110+0000 7ff2e1200640 1 --2- 172.21.15.7:0/361782939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2dc145540 0x7ff2dc0b4580 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.110+0000 7ff2e1200640 1 -- 172.21.15.7:0/361782939 >> 172.21.15.7:0/361782939 conn(0x7ff2dc0b1d70 msgr2=0x7ff2dc0cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.110+0000 7ff2e1200640 1 -- 172.21.15.7:0/361782939 shutdown_connections 2024-09-10T21:47:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.110+0000 7ff2e1200640 1 -- 172.21.15.7:0/361782939 wait complete. 2024-09-10T21:47:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.111+0000 7ff2e1200640 1 Processor -- start 2024-09-10T21:47:22.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.112+0000 7ff2e1200640 1 -- start start 2024-09-10T21:47:22.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.112+0000 7ff2e1200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2dc0b3150 0x7ff2dc141640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.112+0000 7ff2e1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2dc0b4ac0 0x7ff2dc06e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.112+0000 7ff2e1200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2dc145540 0x7ff2dc140050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.112+0000 7ff2e1200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc0cedb0 con 0x7ff2dc0b4ac0 2024-09-10T21:47:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.112+0000 7ff2e1200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc0cec30 con 0x7ff2dc145540 2024-09-10T21:47:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.112+0000 7ff2e1200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ff2dc0cef30 con 0x7ff2dc0b3150 2024-09-10T21:47:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.113+0000 7ff2daa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2dc0b4ac0 0x7ff2dc06e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.113+0000 7ff2daa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2dc0b4ac0 0x7ff2dc06e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60938/0 (socket says 172.21.15.7:60938) 2024-09-10T21:47:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.113+0000 7ff2daa00640 1 -- 172.21.15.7:0/3027381666 learned_addr learned my addr 172.21.15.7:0/3027381666 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.113+0000 7ff2daa00640 1 -- 172.21.15.7:0/3027381666 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2dc0b3150 msgr2=0x7ff2dc141640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.113+0000 7ff2daa00640 1 --2- 172.21.15.7:0/3027381666 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2dc0b3150 0x7ff2dc141640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.113+0000 7ff2dbe00640 1 --2- 172.21.15.7:0/3027381666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2dc145540 0x7ff2dc140050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.113+0000 7ff2daa00640 1 -- 172.21.15.7:0/3027381666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2dc145540 msgr2=0x7ff2dc140050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.113+0000 7ff2daa00640 1 --2- 172.21.15.7:0/3027381666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2dc145540 0x7ff2dc140050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.113+0000 7ff2daa00640 1 -- 172.21.15.7:0/3027381666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2dc0a94d0 con 0x7ff2dc0b4ac0 2024-09-10T21:47:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.113+0000 7ff2dbe00640 1 --2- 172.21.15.7:0/3027381666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2dc145540 0x7ff2dc140050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:47:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.113+0000 7ff2daa00640 1 --2- 172.21.15.7:0/3027381666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2dc0b4ac0 0x7ff2dc06e650 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800c480 tx=0x7ff2c800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.114+0000 7ff2d3400640 1 -- 172.21.15.7:0/3027381666 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8016020 con 0x7ff2dc0b4ac0 2024-09-10T21:47:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.114+0000 7ff2d3400640 1 -- 172.21.15.7:0/3027381666 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2c8012920 con 0x7ff2dc0b4ac0 2024-09-10T21:47:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.114+0000 7ff2d3400640 1 -- 172.21.15.7:0/3027381666 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8012d70 con 0x7ff2dc0b4ac0 2024-09-10T21:47:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.114+0000 7ff2e1200640 1 -- 172.21.15.7:0/3027381666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2dc141f20 con 0x7ff2dc0b4ac0 2024-09-10T21:47:22.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.114+0000 7ff2e1200640 1 -- 172.21.15.7:0/3027381666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2dc0a9a70 con 0x7ff2dc0b4ac0 2024-09-10T21:47:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.115+0000 7ff2e1200640 1 -- 172.21.15.7:0/3027381666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2dc0ce6f0 con 0x7ff2dc0b4ac0 2024-09-10T21:47:22.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.116+0000 7ff2d3400640 1 -- 172.21.15.7:0/3027381666 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff2c801b400 con 0x7ff2dc0b4ac0 2024-09-10T21:47:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.117+0000 7ff2d3400640 1 --2- 172.21.15.7:0/3027381666 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff2bc077f70 0x7ff2bc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.117+0000 7ff2d3400640 1 -- 172.21.15.7:0/3027381666 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7ff2c809a110 con 0x7ff2dc0b4ac0 2024-09-10T21:47:22.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.117+0000 7ff2db400640 1 --2- 172.21.15.7:0/3027381666 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff2bc077f70 0x7ff2bc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.118+0000 7ff2db400640 1 --2- 172.21.15.7:0/3027381666 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff2bc077f70 0x7ff2bc07a430 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc034000 tx=0x7ff2cc0046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:22.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.121+0000 7ff2d3400640 1 -- 172.21.15.7:0/3027381666 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff2c8020020 con 0x7ff2dc0b4ac0 2024-09-10T21:47:22.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.353+0000 7ff2e1200640 1 -- 172.21.15.7:0/3027381666 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff2dc0a9f50 con 0x7ff2dc0b4ac0 2024-09-10T21:47:22.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.353+0000 7ff2d3400640 1 -- 172.21.15.7:0/3027381666 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff2dc0a9f50 con 0x7ff2dc0b4ac0 2024-09-10T21:47:22.354 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:22.354 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T21:47:22.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.356+0000 7ff2e1200640 1 -- 172.21.15.7:0/3027381666 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff2bc077f70 msgr2=0x7ff2bc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.356+0000 7ff2e1200640 1 --2- 172.21.15.7:0/3027381666 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff2bc077f70 0x7ff2bc07a430 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc034000 tx=0x7ff2cc0046c0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.356+0000 7ff2e1200640 1 -- 172.21.15.7:0/3027381666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2dc0b4ac0 msgr2=0x7ff2dc06e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.356+0000 7ff2e1200640 1 --2- 172.21.15.7:0/3027381666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2dc0b4ac0 0x7ff2dc06e650 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff2c800c480 tx=0x7ff2c800c940 comp rx=0 tx=0).stop 2024-09-10T21:47:22.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.356+0000 7ff2e1200640 1 -- 172.21.15.7:0/3027381666 shutdown_connections 2024-09-10T21:47:22.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.356+0000 7ff2e1200640 1 --2- 172.21.15.7:0/3027381666 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff2bc077f70 0x7ff2bc07a430 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.356+0000 7ff2e1200640 1 --2- 172.21.15.7:0/3027381666 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff2dc145540 0x7ff2dc140050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.356+0000 7ff2e1200640 1 --2- 172.21.15.7:0/3027381666 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff2dc0b4ac0 0x7ff2dc06e650 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.356+0000 7ff2e1200640 1 --2- 172.21.15.7:0/3027381666 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff2dc0b3150 0x7ff2dc141640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.356+0000 7ff2e1200640 1 -- 172.21.15.7:0/3027381666 >> 172.21.15.7:0/3027381666 conn(0x7ff2dc0b1d70 msgr2=0x7ff2dc144800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:22.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.356+0000 7ff2e1200640 1 -- 172.21.15.7:0/3027381666 shutdown_connections 2024-09-10T21:47:22.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.356+0000 7ff2e1200640 1 -- 172.21.15.7:0/3027381666 wait complete. 2024-09-10T21:47:22.372 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:47:22.372 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:47:22.533 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.532+0000 7f5693a00640 1 -- 172.21.15.7:0/2810659869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5670004d40 msgr2=0x7f56700051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.532+0000 7f5693a00640 1 --2- 172.21.15.7:0/2810659869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5670004d40 0x7f56700051c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f5684009eb0 tx=0x7f5670005830 comp rx=0 tx=0).stop 2024-09-10T21:47:22.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.532+0000 7f5691a00640 1 -- 172.21.15.7:0/2810659869 reap_dead start 2024-09-10T21:47:22.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.532+0000 7f5693a00640 1 -- 172.21.15.7:0/2810659869 shutdown_connections 2024-09-10T21:47:22.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.532+0000 7f5693a00640 1 -- 172.21.15.7:0/2810659869 >> 172.21.15.7:0/2810659869 conn(0x7f568c0b16d0 msgr2=0x7f568c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:22.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.532+0000 7f5693a00640 1 -- 172.21.15.7:0/2810659869 shutdown_connections 2024-09-10T21:47:22.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.532+0000 7f5693a00640 1 -- 172.21.15.7:0/2810659869 wait complete. 2024-09-10T21:47:22.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.534+0000 7f5693a00640 1 Processor -- start 2024-09-10T21:47:22.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.534+0000 7f5693a00640 1 -- start start 2024-09-10T21:47:22.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.535+0000 7f5693a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5670004d40 0x7f568c1f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.535+0000 7f5693a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f568c0cb910 0x7f568c1f80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.535+0000 7f5693a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f568c15c380 0x7f568c1f8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.535+0000 7f5693a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f568c0cd070 con 0x7f568c15c380 2024-09-10T21:47:22.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.535+0000 7f5693a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f568c0ccef0 con 0x7f5670004d40 2024-09-10T21:47:22.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.535+0000 7f5693a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f568c0cd1f0 con 0x7f568c0cb910 2024-09-10T21:47:22.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.535+0000 7f5691a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f568c15c380 0x7f568c1f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.535+0000 7f5691a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f568c15c380 0x7f568c1f8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:60958/0 (socket says 172.21.15.7:60958) 2024-09-10T21:47:22.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.535+0000 7f5691a00640 1 -- 172.21.15.7:0/2110995291 learned_addr learned my addr 172.21.15.7:0/2110995291 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:22.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.536+0000 7f5683e00640 1 --2- 172.21.15.7:0/2110995291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f568c0cb910 0x7f568c1f80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.536+0000 7f5691000640 1 --2- 172.21.15.7:0/2110995291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5670004d40 0x7f568c1f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.536+0000 7f5691a00640 1 -- 172.21.15.7:0/2110995291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f568c0cb910 msgr2=0x7f568c1f80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.536+0000 7f5691a00640 1 --2- 172.21.15.7:0/2110995291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f568c0cb910 0x7f568c1f80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.536+0000 7f5691a00640 1 -- 172.21.15.7:0/2110995291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5670004d40 msgr2=0x7f568c1f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.536+0000 7f5691a00640 1 --2- 172.21.15.7:0/2110995291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5670004d40 0x7f568c1f7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.536+0000 7f5691a00640 1 -- 172.21.15.7:0/2110995291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568c0b1480 con 0x7f568c15c380 2024-09-10T21:47:22.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.536+0000 7f5691a00640 1 --2- 172.21.15.7:0/2110995291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f568c15c380 0x7f568c1f8610 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5684009fd0 tx=0x7f5684033c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:22.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.536+0000 7f5689600640 1 -- 172.21.15.7:0/2110995291 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684042030 con 0x7f568c15c380 2024-09-10T21:47:22.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.536+0000 7f5689600640 1 -- 172.21.15.7:0/2110995291 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f568402bc00 con 0x7f568c15c380 2024-09-10T21:47:22.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.536+0000 7f5689600640 1 -- 172.21.15.7:0/2110995291 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5684009ab0 con 0x7f568c15c380 2024-09-10T21:47:22.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.536+0000 7f5693a00640 1 -- 172.21.15.7:0/2110995291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568c15c900 con 0x7f568c15c380 2024-09-10T21:47:22.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.537+0000 7f5693a00640 1 -- 172.21.15.7:0/2110995291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568c0cb110 con 0x7f568c15c380 2024-09-10T21:47:22.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.538+0000 7f5693a00640 1 -- 172.21.15.7:0/2110995291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f568c1601f0 con 0x7f568c15c380 2024-09-10T21:47:22.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.539+0000 7f5689600640 1 -- 172.21.15.7:0/2110995291 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f568403e6b0 con 0x7f568c15c380 2024-09-10T21:47:22.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.540+0000 7f5689600640 1 --2- 172.21.15.7:0/2110995291 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5668077fa0 0x7f566807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.540+0000 7f5689600640 1 -- 172.21.15.7:0/2110995291 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f56840bb260 con 0x7f568c15c380 2024-09-10T21:47:22.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.541+0000 7f5691000640 1 --2- 172.21.15.7:0/2110995291 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5668077fa0 0x7f566807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.541+0000 7f5691000640 1 --2- 172.21.15.7:0/2110995291 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5668077fa0 0x7f566807a460 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f567c001160 tx=0x7f567c006720 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:22.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.544+0000 7f5689600640 1 -- 172.21.15.7:0/2110995291 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f568404d020 con 0x7f568c15c380 2024-09-10T21:47:22.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.775+0000 7f5693a00640 1 -- 172.21.15.7:0/2110995291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f568c0b2ff0 con 0x7f568c15c380 2024-09-10T21:47:22.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.776+0000 7f5689600640 1 -- 172.21.15.7:0/2110995291 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) ==== 53+0+73 (secure 0 0 0) 0x7f5684083cc0 con 0x7f568c15c380 2024-09-10T21:47:22.777 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:47:22.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.778+0000 7f5693a00640 1 -- 172.21.15.7:0/2110995291 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5668077fa0 msgr2=0x7f566807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.778+0000 7f5693a00640 1 --2- 172.21.15.7:0/2110995291 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5668077fa0 0x7f566807a460 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f567c001160 tx=0x7f567c006720 comp rx=0 tx=0).stop 2024-09-10T21:47:22.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.778+0000 7f5693a00640 1 -- 172.21.15.7:0/2110995291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f568c15c380 msgr2=0x7f568c1f8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.778+0000 7f5693a00640 1 --2- 172.21.15.7:0/2110995291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f568c15c380 0x7f568c1f8610 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f5684009fd0 tx=0x7f5684033c00 comp rx=0 tx=0).stop 2024-09-10T21:47:22.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.779+0000 7f5693a00640 1 -- 172.21.15.7:0/2110995291 shutdown_connections 2024-09-10T21:47:22.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.779+0000 7f5693a00640 1 --2- 172.21.15.7:0/2110995291 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5668077fa0 0x7f566807a460 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.779+0000 7f5693a00640 1 --2- 172.21.15.7:0/2110995291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f568c15c380 0x7f568c1f8610 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.779+0000 7f5693a00640 1 --2- 172.21.15.7:0/2110995291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f568c0cb910 0x7f568c1f80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.779+0000 7f5693a00640 1 --2- 172.21.15.7:0/2110995291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5670004d40 0x7f568c1f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.779+0000 7f5693a00640 1 -- 172.21.15.7:0/2110995291 >> 172.21.15.7:0/2110995291 conn(0x7f568c0b16d0 msgr2=0x7f568c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:22.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.779+0000 7f5693a00640 1 -- 172.21.15.7:0/2110995291 shutdown_connections 2024-09-10T21:47:22.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:22.779+0000 7f5693a00640 1 -- 172.21.15.7:0/2110995291 wait complete. 2024-09-10T21:47:22.795 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:47:22.795 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:47:22.795 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-10T21:47:22.795 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:47:22.795 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:47:22.796 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:47:22.796 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:47:22.796 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:47:22.796 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:47:22.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:22.796 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-10T21:47:22.833 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:47:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:47:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:47:22.833 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T21:47:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:22.834 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:47:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T21:47:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-10T21:47:22.834 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:47:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T21:47:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-10T21:47:22.834 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:47:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-10T21:47:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-10T21:47:22.834 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T21:47:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42302sec preferred_lft 42302sec 2024-09-10T21:47:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-10T21:47:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:22.835 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:47:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-10T21:47:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-10T21:47:22.835 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:47:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff 2024-09-10T21:47:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:47:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::3c3d:dbff:fe02:4c15/64 scope link 2024-09-10T21:47:22.835 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:22.835 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:47:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether f6:f9:69:5a:86:f5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:47:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f4f9:69ff:fe5a:86f5/64 scope link 2024-09-10T21:47:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:22.836 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:47:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:47:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::433:5cff:fe44:4a5b/64 scope link 2024-09-10T21:47:22.836 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:22.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:22.836 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:47:22.926 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:47:22.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:22.929 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-10T21:47:22.965 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-10T21:47:22.965 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:47:22.966 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T21:47:22.966 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T21:47:22.966 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:47:22.966 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:47:22.966 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:47:22.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:47:22.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:47:22.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:47:22.966 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:47:23.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:47:23.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:47:23.163 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:23.164 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:47:23.199 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:47:23.199 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:47:23.200 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:47:23.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:23.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-10T21:47:23.296 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:47:23.296 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:47:23.331 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:47:23.331 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:47:23.331 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:47:23.406 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:47:23.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:23.406 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:47:23.535 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:47:23.535 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:47:23.564 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:47:23.564 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:47:23.564 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.133:3300,172.21.15.133:3301" 2024-09-10T21:47:23.564 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@2029451a-3c26-48c7-b874-0597197278ac.cephfs=/ 2024-09-10T21:47:23.564 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.133:3300/172.21.15.133:3301" will pass to kernel 2024-09-10T21:47:23.566 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T21:47:23.567 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:47:23.567 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:47:23.567 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:47:23.567 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:47:23.569 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:47:23.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:23.570 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:47:23.625 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:47:23.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:47:23.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:47:23.717 INFO:teuthology.orchestra.run.smithi007.stdout:client6585 2024-09-10T21:47:23.717 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:47:23.717 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:47:23.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:23.754 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.files ./subdir) 2024-09-10T21:47:23.831 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:47:23.832 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 file_to_be_damaged_head deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef 2024-09-10T21:47:23.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.949+0000 7f94dc4d2f40 1 Processor -- start 2024-09-10T21:47:23.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.949+0000 7f94dc4d2f40 1 -- start start 2024-09-10T21:47:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.950+0000 7f94dc4d2f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d033470e0 0x558d03377aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.950+0000 7f94dc4d2f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d0337c210 0x558d0341ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.950+0000 7f94dc4d2f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558d0341db90 0x558d03422290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.950+0000 7f94dc4d2f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558d03394d90 con 0x558d0337c210 2024-09-10T21:47:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.950+0000 7f94dc4d2f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558d03298c60 con 0x558d033470e0 2024-09-10T21:47:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.950+0000 7f94dc4d2f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x558d03394f10 con 0x558d0341db90 2024-09-10T21:47:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.950+0000 7f94d8c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d0337c210 0x558d0341ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.950+0000 7f94d8c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d0337c210 0x558d0341ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36894/0 (socket says 172.21.15.7:36894) 2024-09-10T21:47:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.950+0000 7f94d8c00640 1 -- 172.21.15.7:0/2494501189 learned_addr learned my addr 172.21.15.7:0/2494501189 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.951+0000 7f94d9600640 1 --2- 172.21.15.7:0/2494501189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d033470e0 0x558d03377aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.951+0000 7f94da000640 1 --2- 172.21.15.7:0/2494501189 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558d0341db90 0x558d03422290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.951+0000 7f94d8c00640 1 -- 172.21.15.7:0/2494501189 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558d0341db90 msgr2=0x558d03422290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.951+0000 7f94d8c00640 1 --2- 172.21.15.7:0/2494501189 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558d0341db90 0x558d03422290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.951+0000 7f94d8c00640 1 -- 172.21.15.7:0/2494501189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d033470e0 msgr2=0x558d03377aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.951+0000 7f94d8c00640 1 --2- 172.21.15.7:0/2494501189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d033470e0 0x558d03377aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.951+0000 7f94d8c00640 1 -- 172.21.15.7:0/2494501189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d03270f10 con 0x558d0337c210 2024-09-10T21:47:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.951+0000 7f94d9600640 1 --2- 172.21.15.7:0/2494501189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d033470e0 0x558d03377aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.951+0000 7f94d8c00640 1 --2- 172.21.15.7:0/2494501189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d0337c210 0x558d0341ba10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f94c800a5e0 tx=0x7f94c8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf740326d529438f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.951+0000 7f94d3400640 1 -- 172.21.15.7:0/2494501189 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c8008a40 con 0x558d0337c210 2024-09-10T21:47:23.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.951+0000 7f94d3400640 1 -- 172.21.15.7:0/2494501189 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94c8008c00 con 0x558d0337c210 2024-09-10T21:47:23.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.951+0000 7f94d3400640 1 -- 172.21.15.7:0/2494501189 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94c8013440 con 0x558d0337c210 2024-09-10T21:47:23.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.952+0000 7f94dc4d2f40 1 -- 172.21.15.7:0/2494501189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d0337c210 msgr2=0x558d0341ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.952+0000 7f94dc4d2f40 1 --2- 172.21.15.7:0/2494501189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d0337c210 0x558d0341ba10 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f94c800a5e0 tx=0x7f94c8005f90 comp rx=0 tx=0).stop 2024-09-10T21:47:23.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.952+0000 7f94dc4d2f40 1 -- 172.21.15.7:0/2494501189 shutdown_connections 2024-09-10T21:47:23.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.952+0000 7f94dc4d2f40 1 --2- 172.21.15.7:0/2494501189 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558d0341db90 0x558d03422290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.952+0000 7f94dc4d2f40 1 --2- 172.21.15.7:0/2494501189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d0337c210 0x558d0341ba10 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.952+0000 7f94dc4d2f40 1 --2- 172.21.15.7:0/2494501189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d033470e0 0x558d03377aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.952+0000 7f94dc4d2f40 1 -- 172.21.15.7:0/2494501189 >> 172.21.15.7:0/2494501189 conn(0x558d03396df0 msgr2=0x558d03423530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:23.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.952+0000 7f94dc4d2f40 1 -- 172.21.15.7:0/2494501189 shutdown_connections 2024-09-10T21:47:23.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.952+0000 7f94dc4d2f40 1 -- 172.21.15.7:0/2494501189 wait complete. 2024-09-10T21:47:23.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.953+0000 7f94dc4d2f40 1 Processor -- start 2024-09-10T21:47:23.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.953+0000 7f94dc4d2f40 1 -- start start 2024-09-10T21:47:23.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.954+0000 7f94dc4d2f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558d033470e0 0x558d0341c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.954+0000 7f94dc4d2f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d0337c210 0x558d0341a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.954+0000 7f94dc4d2f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d0341db90 0x558d0341aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.954+0000 7f94dc4d2f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558d0341c4a0 con 0x558d0341db90 2024-09-10T21:47:23.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.954+0000 7f94dc4d2f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558d0341bf50 con 0x558d0337c210 2024-09-10T21:47:23.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.954+0000 7f94dc4d2f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x558d034232c0 con 0x558d033470e0 2024-09-10T21:47:23.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.954+0000 7f94da000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d0341db90 0x558d0341aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.954+0000 7f94da000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d0341db90 0x558d0341aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36908/0 (socket says 172.21.15.7:36908) 2024-09-10T21:47:23.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.954+0000 7f94da000640 1 -- 172.21.15.7:0/684145202 learned_addr learned my addr 172.21.15.7:0/684145202 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:23.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.954+0000 7f94d8c00640 1 --2- 172.21.15.7:0/684145202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d0337c210 0x558d0341a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.954+0000 7f94d9600640 1 --2- 172.21.15.7:0/684145202 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558d033470e0 0x558d0341c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.954+0000 7f94da000640 1 -- 172.21.15.7:0/684145202 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558d033470e0 msgr2=0x558d0341c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.954+0000 7f94da000640 1 --2- 172.21.15.7:0/684145202 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558d033470e0 0x558d0341c840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.954+0000 7f94da000640 1 -- 172.21.15.7:0/684145202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d0337c210 msgr2=0x558d0341a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.954+0000 7f94da000640 1 --2- 172.21.15.7:0/684145202 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558d0337c210 0x558d0341a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.954+0000 7f94da000640 1 -- 172.21.15.7:0/684145202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d03414da0 con 0x558d0341db90 2024-09-10T21:47:23.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.955+0000 7f94da000640 1 --2- 172.21.15.7:0/684145202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d0341db90 0x558d0341aa40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f94d400b770 tx=0x7f94d400bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:23.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.955+0000 7f94d2000640 1 -- 172.21.15.7:0/684145202 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d4009a90 con 0x558d0341db90 2024-09-10T21:47:23.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.955+0000 7f94d2000640 1 -- 172.21.15.7:0/684145202 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94d4009c30 con 0x558d0341db90 2024-09-10T21:47:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.955+0000 7f94dc4d2f40 1 -- 172.21.15.7:0/684145202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558d03414590 con 0x558d0341db90 2024-09-10T21:47:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.955+0000 7f94d2000640 1 -- 172.21.15.7:0/684145202 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d400f690 con 0x558d0341db90 2024-09-10T21:47:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.955+0000 7f94dc4d2f40 1 -- 172.21.15.7:0/684145202 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d03415040 con 0x558d0341db90 2024-09-10T21:47:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.957+0000 7f94d2000640 1 -- 172.21.15.7:0/684145202 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f94d400fa40 con 0x558d0341db90 2024-09-10T21:47:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.958+0000 7f94d2000640 1 --2- 172.21.15.7:0/684145202 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f94bc076980 0x7f94bc078e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.958+0000 7f94d2000640 1 -- 172.21.15.7:0/684145202 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f94d4099b00 con 0x558d0341db90 2024-09-10T21:47:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.958+0000 7f94dc4d2f40 1 --2- 172.21.15.7:0/684145202 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x558d0341cd80 0x558d0341af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.958+0000 7f94dc4d2f40 1 -- 172.21.15.7:0/684145202 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:1 10.10 10:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=115b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e73) -- 0x558d0341ba10 con 0x558d0341cd80 2024-09-10T21:47:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.958+0000 7f94d9600640 1 --2- 172.21.15.7:0/684145202 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f94bc076980 0x7f94bc078e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.959+0000 7f94d8c00640 1 --2- 172.21.15.7:0/684145202 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x558d0341cd80 0x558d0341af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.959+0000 7f94d8c00640 1 --2- 172.21.15.7:0/684145202 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x558d0341cd80 0x558d0341af80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.960+0000 7f94d9600640 1 --2- 172.21.15.7:0/684145202 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f94bc076980 0x7f94bc078e40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f94cc055000 tx=0x7f94cc003020 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:23.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.961+0000 7f94d8c00640 1 -- 172.21.15.7:0/684145202 <== osd.4 v2:172.21.15.133:6826/1857710445 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v73'12 uv12 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f94c8008940 con 0x558d0341cd80 2024-09-10T21:47:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.962+0000 7f94dc4d2f40 1 -- 172.21.15.7:0/684145202 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x558d0341cd80 msgr2=0x558d0341af80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.962+0000 7f94dc4d2f40 1 --2- 172.21.15.7:0/684145202 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x558d0341cd80 0x558d0341af80 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.962+0000 7f94dc4d2f40 1 -- 172.21.15.7:0/684145202 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f94bc076980 msgr2=0x7f94bc078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.962+0000 7f94dc4d2f40 1 --2- 172.21.15.7:0/684145202 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f94bc076980 0x7f94bc078e40 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f94cc055000 tx=0x7f94cc003020 comp rx=0 tx=0).stop 2024-09-10T21:47:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.962+0000 7f94dc4d2f40 1 -- 172.21.15.7:0/684145202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d0341db90 msgr2=0x558d0341aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.962+0000 7f94dc4d2f40 1 --2- 172.21.15.7:0/684145202 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558d0341db90 0x558d0341aa40 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f94d400b770 tx=0x7f94d400bc30 comp rx=0 tx=0).stop 2024-09-10T21:47:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.962+0000 7f94da000640 1 -- 172.21.15.7:0/684145202 reap_dead start 2024-09-10T21:47:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.962+0000 7f94dc4d2f40 1 -- 172.21.15.7:0/684145202 shutdown_connections 2024-09-10T21:47:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.962+0000 7f94dc4d2f40 1 -- 172.21.15.7:0/684145202 >> 172.21.15.7:0/684145202 conn(0x558d03396df0 msgr2=0x558d034234a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.962+0000 7f94dc4d2f40 1 -- 172.21.15.7:0/684145202 shutdown_connections 2024-09-10T21:47:23.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:23.962+0000 7f94dc4d2f40 1 -- 172.21.15.7:0/684145202 wait complete. 2024-09-10T21:47:23.967 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:47:23.968 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:47:24.064 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:47:24.155 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:47:24.156 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:47:24.156 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:47:24.156 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:47:24.192 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:47:24.192 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:47:24.267 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:47:24.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.468+0000 7efd42200640 1 -- 172.21.15.7:0/1525547289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd3c0cc3c0 msgr2=0x7efd3c0702b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.468+0000 7efd42200640 1 --2- 172.21.15.7:0/1525547289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd3c0cc3c0 0x7efd3c0702b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7efd28005fd0 tx=0x7efd2802a5d0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.468+0000 7efd42200640 1 -- 172.21.15.7:0/1525547289 shutdown_connections 2024-09-10T21:47:24.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.468+0000 7efd42200640 1 --2- 172.21.15.7:0/1525547289 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd3c0cddd0 0x7efd3c0ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.468+0000 7efd42200640 1 --2- 172.21.15.7:0/1525547289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd3c0cc3c0 0x7efd3c0702b0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.468+0000 7efd42200640 1 -- 172.21.15.7:0/1525547289 >> 172.21.15.7:0/1525547289 conn(0x7efd3c0b16d0 msgr2=0x7efd3c0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:24.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.468+0000 7efd42200640 1 -- 172.21.15.7:0/1525547289 shutdown_connections 2024-09-10T21:47:24.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.468+0000 7efd42200640 1 -- 172.21.15.7:0/1525547289 wait complete. 2024-09-10T21:47:24.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.470+0000 7efd42200640 1 Processor -- start 2024-09-10T21:47:24.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.470+0000 7efd42200640 1 -- start start 2024-09-10T21:47:24.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.471+0000 7efd42200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd3c0cc3c0 0x7efd3c14f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.471+0000 7efd42200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd3c0cddd0 0x7efd3c1f4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.471+0000 7efd42200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd3c15f4d0 0x7efd3c0caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.471+0000 7efd42200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c0ce920 con 0x7efd3c0cddd0 2024-09-10T21:47:24.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.471+0000 7efd42200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7efd3c0ce7a0 con 0x7efd3c15f4d0 2024-09-10T21:47:24.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.471+0000 7efd42200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7efd3c0ceaa0 con 0x7efd3c0cc3c0 2024-09-10T21:47:24.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.471+0000 7efd3be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd3c0cddd0 0x7efd3c1f4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.471+0000 7efd3be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd3c0cddd0 0x7efd3c1f4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36924/0 (socket says 172.21.15.7:36924) 2024-09-10T21:47:24.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.471+0000 7efd3be00640 1 -- 172.21.15.7:0/3233145531 learned_addr learned my addr 172.21.15.7:0/3233145531 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:24.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.471+0000 7efd41800640 1 --2- 172.21.15.7:0/3233145531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd3c15f4d0 0x7efd3c0caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.471+0000 7efd40e00640 1 --2- 172.21.15.7:0/3233145531 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd3c0cc3c0 0x7efd3c14f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.471+0000 7efd3be00640 1 -- 172.21.15.7:0/3233145531 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd3c0cc3c0 msgr2=0x7efd3c14f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.471+0000 7efd3be00640 1 --2- 172.21.15.7:0/3233145531 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd3c0cc3c0 0x7efd3c14f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.471+0000 7efd3be00640 1 -- 172.21.15.7:0/3233145531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd3c15f4d0 msgr2=0x7efd3c0caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.471+0000 7efd3be00640 1 --2- 172.21.15.7:0/3233145531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd3c15f4d0 0x7efd3c0caf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.471+0000 7efd3be00640 1 -- 172.21.15.7:0/3233145531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3c0c8d50 con 0x7efd3c0cddd0 2024-09-10T21:47:24.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.471+0000 7efd41800640 1 --2- 172.21.15.7:0/3233145531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd3c15f4d0 0x7efd3c0caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:24.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.472+0000 7efd3be00640 1 --2- 172.21.15.7:0/3233145531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd3c0cddd0 0x7efd3c1f4930 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7efd28006010 tx=0x7efd280038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:24.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.472+0000 7efd38c00640 1 -- 172.21.15.7:0/3233145531 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd2803f070 con 0x7efd3c0cddd0 2024-09-10T21:47:24.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.472+0000 7efd38c00640 1 -- 172.21.15.7:0/3233145531 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd28005600 con 0x7efd3c0cddd0 2024-09-10T21:47:24.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.472+0000 7efd38c00640 1 -- 172.21.15.7:0/3233145531 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd28035030 con 0x7efd3c0cddd0 2024-09-10T21:47:24.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.472+0000 7efd42200640 1 -- 172.21.15.7:0/3233145531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3c0c9030 con 0x7efd3c0cddd0 2024-09-10T21:47:24.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.472+0000 7efd42200640 1 -- 172.21.15.7:0/3233145531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd3c0c95f0 con 0x7efd3c0cddd0 2024-09-10T21:47:24.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.473+0000 7efd42200640 1 -- 172.21.15.7:0/3233145531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd3c0702b0 con 0x7efd3c0cddd0 2024-09-10T21:47:24.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.474+0000 7efd38c00640 1 -- 172.21.15.7:0/3233145531 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7efd280049e0 con 0x7efd3c0cddd0 2024-09-10T21:47:24.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.475+0000 7efd38c00640 1 --2- 172.21.15.7:0/3233145531 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efd1c077f70 0x7efd1c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.475+0000 7efd38c00640 1 -- 172.21.15.7:0/3233145531 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7efd280bacd0 con 0x7efd3c0cddd0 2024-09-10T21:47:24.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.476+0000 7efd40e00640 1 --2- 172.21.15.7:0/3233145531 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efd1c077f70 0x7efd1c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.477+0000 7efd40e00640 1 --2- 172.21.15.7:0/3233145531 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efd1c077f70 0x7efd1c07a430 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efd30001720 tx=0x7efd30009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:24.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.480+0000 7efd38c00640 1 -- 172.21.15.7:0/3233145531 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7efd28083050 con 0x7efd3c0cddd0 2024-09-10T21:47:24.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.686+0000 7efd42200640 1 -- 172.21.15.7:0/3233145531 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efd3c14dd00 con 0x7efd3c0cddd0 2024-09-10T21:47:24.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.686+0000 7efd38c00640 1 -- 172.21.15.7:0/3233145531 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v73) ==== 80+0+124 (secure 0 0 0) 0x7efd28087a80 con 0x7efd3c0cddd0 2024-09-10T21:47:24.687 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.133:6836/2752881644 2024-09-11T21:46:10.658995+0000 2024-09-10T21:47:24.687 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.133:6834/2752881644 2024-09-11T21:46:10.658995+0000 2024-09-10T21:47:24.688 INFO:teuthology.orchestra.run.smithi007.stderr:listed 2 entries 2024-09-10T21:47:24.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.689+0000 7efd42200640 1 -- 172.21.15.7:0/3233145531 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efd1c077f70 msgr2=0x7efd1c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.689+0000 7efd42200640 1 --2- 172.21.15.7:0/3233145531 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efd1c077f70 0x7efd1c07a430 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7efd30001720 tx=0x7efd30009040 comp rx=0 tx=0).stop 2024-09-10T21:47:24.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.689+0000 7efd42200640 1 -- 172.21.15.7:0/3233145531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd3c0cddd0 msgr2=0x7efd3c1f4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.689+0000 7efd42200640 1 --2- 172.21.15.7:0/3233145531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd3c0cddd0 0x7efd3c1f4930 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7efd28006010 tx=0x7efd280038a0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.689+0000 7efd42200640 1 -- 172.21.15.7:0/3233145531 shutdown_connections 2024-09-10T21:47:24.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.689+0000 7efd42200640 1 --2- 172.21.15.7:0/3233145531 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7efd1c077f70 0x7efd1c07a430 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.689+0000 7efd42200640 1 --2- 172.21.15.7:0/3233145531 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7efd3c15f4d0 0x7efd3c0caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.689+0000 7efd42200640 1 --2- 172.21.15.7:0/3233145531 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7efd3c0cddd0 0x7efd3c1f4930 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.689+0000 7efd42200640 1 --2- 172.21.15.7:0/3233145531 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7efd3c0cc3c0 0x7efd3c14f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.689+0000 7efd42200640 1 -- 172.21.15.7:0/3233145531 >> 172.21.15.7:0/3233145531 conn(0x7efd3c0b16d0 msgr2=0x7efd3c159b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:24.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.689+0000 7efd42200640 1 -- 172.21.15.7:0/3233145531 shutdown_connections 2024-09-10T21:47:24.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:24.689+0000 7efd42200640 1 -- 172.21.15.7:0/3233145531 wait complete. 2024-09-10T21:47:24.706 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:47:24.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:24.706 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:47:24.767 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:47:24.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:24.767 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:47:24.803 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:47:24.803 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:47:24.803 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:47:24.843 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:47:24.843 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:47:24.843 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:47:24.844 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:47:24.880 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:47:24.880 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:47:24.880 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:47:24.880 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:47:24.921 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:47:24.922 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:47:25.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.082+0000 7f1ed8c00640 1 -- 172.21.15.7:0/4258477165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb8003a00 msgr2=0x7f1eb8003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.082+0000 7f1ed8c00640 1 --2- 172.21.15.7:0/4258477165 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1eb8003a00 0x7f1eb8003e20 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009b50 tx=0x7f1eb8005830 comp rx=0 tx=0).stop 2024-09-10T21:47:25.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.082+0000 7f1ed3e00640 1 -- 172.21.15.7:0/4258477165 reap_dead start 2024-09-10T21:47:25.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.082+0000 7f1ed8c00640 1 -- 172.21.15.7:0/4258477165 shutdown_connections 2024-09-10T21:47:25.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.082+0000 7f1ed8c00640 1 -- 172.21.15.7:0/4258477165 >> 172.21.15.7:0/4258477165 conn(0x7f1ed40b1d70 msgr2=0x7f1ed40cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:25.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.082+0000 7f1ed8c00640 1 -- 172.21.15.7:0/4258477165 shutdown_connections 2024-09-10T21:47:25.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.082+0000 7f1ed8c00640 1 -- 172.21.15.7:0/4258477165 wait complete. 2024-09-10T21:47:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.083+0000 7f1ed8c00640 1 Processor -- start 2024-09-10T21:47:25.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.084+0000 7f1ed8c00640 1 -- start start 2024-09-10T21:47:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.084+0000 7f1ed8c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1eb8003a00 0x7f1ed41560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.084+0000 7f1ed8c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed413a7f0 0x7f1ed413e350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.084+0000 7f1ed8c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed413e890 0x7f1ed41f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.084+0000 7f1ed8c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed40cf880 con 0x7f1ed413e890 2024-09-10T21:47:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.084+0000 7f1ed8c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed40cf700 con 0x7f1ed413a7f0 2024-09-10T21:47:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.084+0000 7f1ed8c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed40b2a40 con 0x7f1eb8003a00 2024-09-10T21:47:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.085+0000 7f1ed3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed413e890 0x7f1ed41f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.085+0000 7f1ed3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed413e890 0x7f1ed41f5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36948/0 (socket says 172.21.15.7:36948) 2024-09-10T21:47:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.085+0000 7f1ed3e00640 1 -- 172.21.15.7:0/4121915998 learned_addr learned my addr 172.21.15.7:0/4121915998 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:25.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.085+0000 7f1ed3400640 1 --2- 172.21.15.7:0/4121915998 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1eb8003a00 0x7f1ed41560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:25.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.085+0000 7f1ed2a00640 1 --2- 172.21.15.7:0/4121915998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed413a7f0 0x7f1ed413e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:25.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.085+0000 7f1ed3e00640 1 -- 172.21.15.7:0/4121915998 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1eb8003a00 msgr2=0x7f1ed41560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.085+0000 7f1ed3e00640 1 --2- 172.21.15.7:0/4121915998 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1eb8003a00 0x7f1ed41560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.085+0000 7f1ed3e00640 1 -- 172.21.15.7:0/4121915998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed413a7f0 msgr2=0x7f1ed413e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.085+0000 7f1ed3e00640 1 --2- 172.21.15.7:0/4121915998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed413a7f0 0x7f1ed413e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.085+0000 7f1ed3e00640 1 -- 172.21.15.7:0/4121915998 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed41f5760 con 0x7f1ed413e890 2024-09-10T21:47:25.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.085+0000 7f1ed3e00640 1 --2- 172.21.15.7:0/4121915998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed413e890 0x7f1ed41f5c60 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc001be0 tx=0x7f1ecc006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:25.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.086+0000 7f1ecb400640 1 -- 172.21.15.7:0/4121915998 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc00f8b0 con 0x7f1ed413e890 2024-09-10T21:47:25.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.086+0000 7f1ecb400640 1 -- 172.21.15.7:0/4121915998 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ecc016020 con 0x7f1ed413e890 2024-09-10T21:47:25.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.086+0000 7f1ecb400640 1 -- 172.21.15.7:0/4121915998 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc00fcf0 con 0x7f1ed413e890 2024-09-10T21:47:25.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.086+0000 7f1ed8c00640 1 -- 172.21.15.7:0/4121915998 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed41e7e50 con 0x7f1ed413e890 2024-09-10T21:47:25.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.086+0000 7f1ed8c00640 1 -- 172.21.15.7:0/4121915998 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed41e8410 con 0x7f1ed413e890 2024-09-10T21:47:25.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.087+0000 7f1ed8c00640 1 -- 172.21.15.7:0/4121915998 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ed40b4580 con 0x7f1ed413e890 2024-09-10T21:47:25.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.088+0000 7f1ecb400640 1 -- 172.21.15.7:0/4121915998 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1ecc011430 con 0x7f1ed413e890 2024-09-10T21:47:25.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.089+0000 7f1ecb400640 1 --2- 172.21.15.7:0/4121915998 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1eb4077f70 0x7f1eb407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:25.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.089+0000 7f1ecb400640 1 -- 172.21.15.7:0/4121915998 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f1ecc09a8f0 con 0x7f1ed413e890 2024-09-10T21:47:25.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.090+0000 7f1ed3400640 1 --2- 172.21.15.7:0/4121915998 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1eb4077f70 0x7f1eb407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:25.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.090+0000 7f1ed3400640 1 --2- 172.21.15.7:0/4121915998 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1eb4077f70 0x7f1eb407a430 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4035000 tx=0x7f1ec402d040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:25.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.093+0000 7f1ecb400640 1 -- 172.21.15.7:0/4121915998 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1ecc020020 con 0x7f1ed413e890 2024-09-10T21:47:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.327+0000 7f1ed8c00640 1 -- 172.21.15.7:0/4121915998 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ed41e85e0 con 0x7f1ed413e890 2024-09-10T21:47:25.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.328+0000 7f1ecb400640 1 -- 172.21.15.7:0/4121915998 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5237 (secure 0 0 0) 0x7f1ecc063350 con 0x7f1ed413e890 2024-09-10T21:47:25.329 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:25.329 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":54,"btime":"2024-09-10T21:46:58:078261+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5884,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3488410478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3488410478},{"type":"v1","addr":"172.21.15.133:6837","nonce":3488410478}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5890,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3660400727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3660400727},{"type":"v1","addr":"172.21.15.7:6835","nonce":3660400727}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2388873704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2388873704},{"type":"v1","addr":"172.21.15.7:6834","nonce":2388873704}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:46:00.260975+0000","modified":"2024-09-10T21:46:57.678038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235490":{"gid":235490,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":15,"addr":"172.21.15.133:6836/2622530239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2622530239},{"type":"v1","addr":"172.21.15.133:6836","nonce":2622530239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235490,"qdb_cluster":[235490]},"id":5}]} 2024-09-10T21:47:25.329 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 54 2024-09-10T21:47:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.330+0000 7f1ed8c00640 1 -- 172.21.15.7:0/4121915998 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1eb4077f70 msgr2=0x7f1eb407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.330+0000 7f1ed8c00640 1 --2- 172.21.15.7:0/4121915998 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1eb4077f70 0x7f1eb407a430 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4035000 tx=0x7f1ec402d040 comp rx=0 tx=0).stop 2024-09-10T21:47:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.330+0000 7f1ed8c00640 1 -- 172.21.15.7:0/4121915998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed413e890 msgr2=0x7f1ed41f5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.330+0000 7f1ed8c00640 1 --2- 172.21.15.7:0/4121915998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed413e890 0x7f1ed41f5c60 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc001be0 tx=0x7f1ecc006660 comp rx=0 tx=0).stop 2024-09-10T21:47:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.330+0000 7f1ed8c00640 1 -- 172.21.15.7:0/4121915998 shutdown_connections 2024-09-10T21:47:25.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.330+0000 7f1ed8c00640 1 --2- 172.21.15.7:0/4121915998 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1eb4077f70 0x7f1eb407a430 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.330+0000 7f1ed8c00640 1 --2- 172.21.15.7:0/4121915998 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed413a7f0 0x7f1ed413e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.330+0000 7f1ed8c00640 1 --2- 172.21.15.7:0/4121915998 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed413e890 0x7f1ed41f5c60 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.330+0000 7f1ed8c00640 1 --2- 172.21.15.7:0/4121915998 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1eb8003a00 0x7f1ed41560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.330+0000 7f1ed8c00640 1 -- 172.21.15.7:0/4121915998 >> 172.21.15.7:0/4121915998 conn(0x7f1ed40b1d70 msgr2=0x7f1ed415ebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:25.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.330+0000 7f1ed8c00640 1 -- 172.21.15.7:0/4121915998 shutdown_connections 2024-09-10T21:47:25.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.330+0000 7f1ed8c00640 1 -- 172.21.15.7:0/4121915998 wait complete. 2024-09-10T21:47:25.346 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:47:25.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.505+0000 7f5604a00640 1 -- 172.21.15.7:0/3315384783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56000d2690 msgr2=0x7f56000b2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.505+0000 7f5604a00640 1 --2- 172.21.15.7:0/3315384783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56000d2690 0x7f56000b2290 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009830 tx=0x7f55ec02a480 comp rx=0 tx=0).stop 2024-09-10T21:47:25.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.505+0000 7f5604a00640 1 -- 172.21.15.7:0/3315384783 shutdown_connections 2024-09-10T21:47:25.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.505+0000 7f5604a00640 1 --2- 172.21.15.7:0/3315384783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56000d2690 0x7f56000b2290 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.505+0000 7f5604a00640 1 -- 172.21.15.7:0/3315384783 >> 172.21.15.7:0/3315384783 conn(0x7f56000d1ba0 msgr2=0x7f56000d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:25.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.506+0000 7f5604a00640 1 -- 172.21.15.7:0/3315384783 shutdown_connections 2024-09-10T21:47:25.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.506+0000 7f5604a00640 1 -- 172.21.15.7:0/3315384783 wait complete. 2024-09-10T21:47:25.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.507+0000 7f5604a00640 1 Processor -- start 2024-09-10T21:47:25.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.507+0000 7f5604a00640 1 -- start start 2024-09-10T21:47:25.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.508+0000 7f5604a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56000d2690 0x7f56000c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:25.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.508+0000 7f5604a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f560015f0a0 0x7f56000c7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:25.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.508+0000 7f5604a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f560015f480 0x7f56000c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:25.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.508+0000 7f5604a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f56000b0ef0 con 0x7f56000d2690 2024-09-10T21:47:25.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.508+0000 7f5604a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f56000b0d70 con 0x7f560015f480 2024-09-10T21:47:25.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.508+0000 7f5604a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f56000b1070 con 0x7f560015f0a0 2024-09-10T21:47:25.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.508+0000 7f55fe800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56000d2690 0x7f56000c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:25.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.508+0000 7f55fe800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56000d2690 0x7f56000c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36956/0 (socket says 172.21.15.7:36956) 2024-09-10T21:47:25.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.508+0000 7f55fe800640 1 -- 172.21.15.7:0/2344969375 learned_addr learned my addr 172.21.15.7:0/2344969375 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:25.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.509+0000 7f55fde00640 1 --2- 172.21.15.7:0/2344969375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f560015f0a0 0x7f56000c7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:25.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.509+0000 7f55ff200640 1 --2- 172.21.15.7:0/2344969375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f560015f480 0x7f56000c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:25.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.509+0000 7f55fe800640 1 -- 172.21.15.7:0/2344969375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f560015f0a0 msgr2=0x7f56000c7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.509+0000 7f55fe800640 1 --2- 172.21.15.7:0/2344969375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f560015f0a0 0x7f56000c7850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.509+0000 7f55fe800640 1 -- 172.21.15.7:0/2344969375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f560015f480 msgr2=0x7f56000c43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.509+0000 7f55fe800640 1 --2- 172.21.15.7:0/2344969375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f560015f480 0x7f56000c43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.509+0000 7f55fe800640 1 -- 172.21.15.7:0/2344969375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56000c82a0 con 0x7f56000d2690 2024-09-10T21:47:25.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.509+0000 7f55fe800640 1 --2- 172.21.15.7:0/2344969375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56000d2690 0x7f56000c5860 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f55f000c430 tx=0x7f55f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:25.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.509+0000 7f55f6a00640 1 -- 172.21.15.7:0/2344969375 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0011790 con 0x7f56000d2690 2024-09-10T21:47:25.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.510+0000 7f55f6a00640 1 -- 172.21.15.7:0/2344969375 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55f0017070 con 0x7f56000d2690 2024-09-10T21:47:25.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.510+0000 7f5604a00640 1 -- 172.21.15.7:0/2344969375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56000c8580 con 0x7f56000d2690 2024-09-10T21:47:25.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.510+0000 7f55f6a00640 1 -- 172.21.15.7:0/2344969375 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55f0011d00 con 0x7f56000d2690 2024-09-10T21:47:25.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.510+0000 7f5604a00640 1 -- 172.21.15.7:0/2344969375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56000b5eb0 con 0x7f56000d2690 2024-09-10T21:47:25.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.511+0000 7f5604a00640 1 -- 172.21.15.7:0/2344969375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56000ce930 con 0x7f56000d2690 2024-09-10T21:47:25.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.512+0000 7f55f6a00640 1 -- 172.21.15.7:0/2344969375 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f55f001c460 con 0x7f56000d2690 2024-09-10T21:47:25.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.513+0000 7f55f6a00640 1 --2- 172.21.15.7:0/2344969375 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f55e0077f70 0x7f55e007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:25.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.513+0000 7f55f6a00640 1 -- 172.21.15.7:0/2344969375 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f55f0099c90 con 0x7f56000d2690 2024-09-10T21:47:25.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.513+0000 7f55fde00640 1 --2- 172.21.15.7:0/2344969375 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f55e0077f70 0x7f55e007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:25.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.514+0000 7f55fde00640 1 --2- 172.21.15.7:0/2344969375 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f55e0077f70 0x7f55e007a430 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f55ec034000 tx=0x7f55ec004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:25.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.517+0000 7f55f6a00640 1 -- 172.21.15.7:0/2344969375 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f55f00626f0 con 0x7f56000d2690 2024-09-10T21:47:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.750+0000 7f5604a00640 1 -- 172.21.15.7:0/2344969375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56000af320 con 0x7f56000d2690 2024-09-10T21:47:25.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.750+0000 7f55f6a00640 1 -- 172.21.15.7:0/2344969375 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5237 (secure 0 0 0) 0x7f55f0061ff0 con 0x7f56000d2690 2024-09-10T21:47:25.752 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:25.752 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":54,"btime":"2024-09-10T21:46:58:078261+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5884,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3488410478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3488410478},{"type":"v1","addr":"172.21.15.133:6837","nonce":3488410478}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5890,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3660400727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3660400727},{"type":"v1","addr":"172.21.15.7:6835","nonce":3660400727}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2388873704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2388873704},{"type":"v1","addr":"172.21.15.7:6834","nonce":2388873704}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:46:00.260975+0000","modified":"2024-09-10T21:46:57.678038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235490":{"gid":235490,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":15,"addr":"172.21.15.133:6836/2622530239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2622530239},{"type":"v1","addr":"172.21.15.133:6836","nonce":2622530239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235490,"qdb_cluster":[235490]},"id":5}]} 2024-09-10T21:47:25.752 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 54 2024-09-10T21:47:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.753+0000 7f5604a00640 1 -- 172.21.15.7:0/2344969375 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f55e0077f70 msgr2=0x7f55e007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.753+0000 7f5604a00640 1 --2- 172.21.15.7:0/2344969375 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f55e0077f70 0x7f55e007a430 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f55ec034000 tx=0x7f55ec004730 comp rx=0 tx=0).stop 2024-09-10T21:47:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.753+0000 7f5604a00640 1 -- 172.21.15.7:0/2344969375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56000d2690 msgr2=0x7f56000c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.753+0000 7f5604a00640 1 --2- 172.21.15.7:0/2344969375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56000d2690 0x7f56000c5860 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f55f000c430 tx=0x7f55f000c8f0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.753+0000 7f5604a00640 1 -- 172.21.15.7:0/2344969375 shutdown_connections 2024-09-10T21:47:25.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.753+0000 7f5604a00640 1 --2- 172.21.15.7:0/2344969375 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f55e0077f70 0x7f55e007a430 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.753+0000 7f5604a00640 1 --2- 172.21.15.7:0/2344969375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f560015f480 0x7f56000c43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.753+0000 7f5604a00640 1 --2- 172.21.15.7:0/2344969375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f560015f0a0 0x7f56000c7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.753+0000 7f5604a00640 1 --2- 172.21.15.7:0/2344969375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f56000d2690 0x7f56000c5860 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.753+0000 7f5604a00640 1 -- 172.21.15.7:0/2344969375 >> 172.21.15.7:0/2344969375 conn(0x7f56000d1ba0 msgr2=0x7f56000d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.753+0000 7f5604a00640 1 -- 172.21.15.7:0/2344969375 shutdown_connections 2024-09-10T21:47:25.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.753+0000 7f5604a00640 1 -- 172.21.15.7:0/2344969375 wait complete. 2024-09-10T21:47:25.770 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.931+0000 7f4f54800640 1 -- 172.21.15.7:0/946844895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f4c15bc90 msgr2=0x7f4f4c0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.931+0000 7f4f54800640 1 --2- 172.21.15.7:0/946844895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f4c15bc90 0x7f4f4c0ccef0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4f40005fd0 tx=0x7f4f4002a4b0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.931+0000 7f4f54800640 1 -- 172.21.15.7:0/946844895 shutdown_connections 2024-09-10T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.931+0000 7f4f54800640 1 --2- 172.21.15.7:0/946844895 >> v2:172.21.15.133:3300/0 conn(0x7f4f4c15fda0 0x7f4f4c1601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.931+0000 7f4f54800640 1 --2- 172.21.15.7:0/946844895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f4c15bc90 0x7f4f4c0ccef0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.931+0000 7f4f54800640 1 -- 172.21.15.7:0/946844895 >> 172.21.15.7:0/946844895 conn(0x7f4f4c0b16d0 msgr2=0x7f4f4c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.931+0000 7f4f54800640 1 -- 172.21.15.7:0/946844895 shutdown_connections 2024-09-10T21:47:25.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.931+0000 7f4f54800640 1 -- 172.21.15.7:0/946844895 wait complete. 2024-09-10T21:47:25.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.932+0000 7f4f54800640 1 Processor -- start 2024-09-10T21:47:25.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.933+0000 7f4f54800640 1 -- start start 2024-09-10T21:47:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.933+0000 7f4f54800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f4c0cd750 0x7f4f4c1f6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.933+0000 7f4f54800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f4c15bc90 0x7f4f4c0c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:25.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.933+0000 7f4f54800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f4c15fda0 0x7f4f4c15e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.933+0000 7f4f54800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c06fe70 con 0x7f4f4c15bc90 2024-09-10T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.933+0000 7f4f54800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c06fcf0 con 0x7f4f4c15fda0 2024-09-10T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.933+0000 7f4f54800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4c06fff0 con 0x7f4f4c0cd750 2024-09-10T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.934+0000 7f4f51400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f4c15bc90 0x7f4f4c0c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.934+0000 7f4f51e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f4c0cd750 0x7f4f4c1f6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.934+0000 7f4f51400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f4c15bc90 0x7f4f4c0c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36972/0 (socket says 172.21.15.7:36972) 2024-09-10T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.934+0000 7f4f51400640 1 -- 172.21.15.7:0/809006258 learned_addr learned my addr 172.21.15.7:0/809006258 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.934+0000 7f4f51e00640 1 --2- 172.21.15.7:0/809006258 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f4c0cd750 0x7f4f4c1f6460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:57790/0 (socket says 172.21.15.7:57790) 2024-09-10T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.934+0000 7f4f52800640 1 --2- 172.21.15.7:0/809006258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f4c15fda0 0x7f4f4c15e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.934+0000 7f4f51e00640 1 -- 172.21.15.7:0/809006258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f4c15fda0 msgr2=0x7f4f4c15e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.934+0000 7f4f51e00640 1 --2- 172.21.15.7:0/809006258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f4c15fda0 0x7f4f4c15e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.934+0000 7f4f51e00640 1 -- 172.21.15.7:0/809006258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f4c15bc90 msgr2=0x7f4f4c0c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.934+0000 7f4f51e00640 1 --2- 172.21.15.7:0/809006258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f4c15bc90 0x7f4f4c0c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.934+0000 7f4f51e00640 1 -- 172.21.15.7:0/809006258 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c1f69a0 con 0x7f4f4c0cd750 2024-09-10T21:47:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.934+0000 7f4f51400640 1 --2- 172.21.15.7:0/809006258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f4c15bc90 0x7f4f4c0c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:47:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.935+0000 7f4f51e00640 1 --2- 172.21.15.7:0/809006258 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f4c0cd750 0x7f4f4c1f6460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4f40031000 tx=0x7f4f400038a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:25.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.935+0000 7f4f46000640 1 -- 172.21.15.7:0/809006258 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4003f070 con 0x7f4f4c0cd750 2024-09-10T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.935+0000 7f4f46000640 1 -- 172.21.15.7:0/809006258 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f400047a0 con 0x7f4f4c0cd750 2024-09-10T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.935+0000 7f4f46000640 1 -- 172.21.15.7:0/809006258 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40035030 con 0x7f4f4c0cd750 2024-09-10T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.935+0000 7f4f54800640 1 -- 172.21.15.7:0/809006258 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4c15d450 con 0x7f4f4c0cd750 2024-09-10T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.935+0000 7f4f54800640 1 -- 172.21.15.7:0/809006258 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f4c15da10 con 0x7f4f4c0cd750 2024-09-10T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.936+0000 7f4f54800640 1 -- 172.21.15.7:0/809006258 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f4c070510 con 0x7f4f4c0cd750 2024-09-10T21:47:25.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.938+0000 7f4f46000640 1 -- 172.21.15.7:0/809006258 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4f40004bb0 con 0x7f4f4c0cd750 2024-09-10T21:47:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.938+0000 7f4f46000640 1 --2- 172.21.15.7:0/809006258 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4f30077f70 0x7f4f3007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.939+0000 7f4f46000640 1 -- 172.21.15.7:0/809006258 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f4f400bb600 con 0x7f4f4c0cd750 2024-09-10T21:47:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.939+0000 7f4f51400640 1 --2- 172.21.15.7:0/809006258 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4f30077f70 0x7f4f3007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:25.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.940+0000 7f4f51400640 1 --2- 172.21.15.7:0/809006258 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4f30077f70 0x7f4f3007a430 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c15b030 tx=0x7f4f3c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:25.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:25.943+0000 7f4f46000640 1 -- 172.21.15.7:0/809006258 <== mon.2 v2:172.21.15.133:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4f40083980 con 0x7f4f4c0cd750 2024-09-10T21:47:26.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.139+0000 7f4f54800640 1 -- 172.21.15.7:0/809006258 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4f4c073250 con 0x7f4f4c0cd750 2024-09-10T21:47:26.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.145+0000 7f4f46000640 1 -- 172.21.15.7:0/809006258 <== mon.2 v2:172.21.15.133:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+16920 (secure 0 0 0) 0x7f4f400b9e20 con 0x7f4f4c0cd750 2024-09-10T21:47:26.146 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:26.146 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":73,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:46:49.043545+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:45:58.069464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T21:45:59.225636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.a","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6836/2752881644":"2024-09-11T21:46:10.658995+0000","172.21.15.133:6834/2752881644":"2024-09-11T21:46:10.658995+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:47:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.147+0000 7f4f54800640 1 -- 172.21.15.7:0/809006258 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4f30077f70 msgr2=0x7f4f3007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.147+0000 7f4f54800640 1 --2- 172.21.15.7:0/809006258 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4f30077f70 0x7f4f3007a430 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c15b030 tx=0x7f4f3c009040 comp rx=0 tx=0).stop 2024-09-10T21:47:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.147+0000 7f4f54800640 1 -- 172.21.15.7:0/809006258 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f4c0cd750 msgr2=0x7f4f4c1f6460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.147+0000 7f4f54800640 1 --2- 172.21.15.7:0/809006258 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f4c0cd750 0x7f4f4c1f6460 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4f40031000 tx=0x7f4f400038a0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.147+0000 7f4f54800640 1 -- 172.21.15.7:0/809006258 shutdown_connections 2024-09-10T21:47:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.147+0000 7f4f54800640 1 --2- 172.21.15.7:0/809006258 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4f30077f70 0x7f4f3007a430 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.147+0000 7f4f54800640 1 --2- 172.21.15.7:0/809006258 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f4c15fda0 0x7f4f4c15e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.147+0000 7f4f54800640 1 --2- 172.21.15.7:0/809006258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f4c15bc90 0x7f4f4c0c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.147+0000 7f4f54800640 1 --2- 172.21.15.7:0/809006258 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f4c0cd750 0x7f4f4c1f6460 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.147+0000 7f4f54800640 1 -- 172.21.15.7:0/809006258 >> 172.21.15.7:0/809006258 conn(0x7f4f4c0b16d0 msgr2=0x7f4f4c0b4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.147+0000 7f4f54800640 1 -- 172.21.15.7:0/809006258 shutdown_connections 2024-09-10T21:47:26.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.148+0000 7f4f54800640 1 -- 172.21.15.7:0/809006258 wait complete. 2024-09-10T21:47:26.165 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:47:26.165 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:47:26.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.326+0000 7f2715600640 1 -- 172.21.15.7:0/1236835004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f271015bc90 msgr2=0x7f27100ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:26.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.326+0000 7f2715600640 1 --2- 172.21.15.7:0/1236835004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f271015bc90 0x7f27100ccef0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2700005fd0 tx=0x7f270002a4b0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.326+0000 7f2715600640 1 -- 172.21.15.7:0/1236835004 shutdown_connections 2024-09-10T21:47:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.326+0000 7f2715600640 1 --2- 172.21.15.7:0/1236835004 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27100cd750 0x7f2710073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.326+0000 7f2715600640 1 --2- 172.21.15.7:0/1236835004 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f271015bc90 0x7f27100ccef0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.326+0000 7f2715600640 1 -- 172.21.15.7:0/1236835004 >> 172.21.15.7:0/1236835004 conn(0x7f27100b16d0 msgr2=0x7f27100b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.326+0000 7f2715600640 1 -- 172.21.15.7:0/1236835004 shutdown_connections 2024-09-10T21:47:26.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.326+0000 7f2715600640 1 -- 172.21.15.7:0/1236835004 wait complete. 2024-09-10T21:47:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.328+0000 7f2715600640 1 Processor -- start 2024-09-10T21:47:26.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.328+0000 7f2715600640 1 -- start start 2024-09-10T21:47:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.329+0000 7f2715600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2710073c70 0x7f2710150450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.329+0000 7f2715600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27100cd750 0x7f27100c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.329+0000 7f2715600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f271015bc90 0x7f27100b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.329+0000 7f2715600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f271006fcf0 con 0x7f271015bc90 2024-09-10T21:47:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.329+0000 7f2715600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f27101606c0 con 0x7f2710073c70 2024-09-10T21:47:26.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.329+0000 7f2715600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f271006fe70 con 0x7f27100cd750 2024-09-10T21:47:26.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.329+0000 7f270f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f271015bc90 0x7f27100b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:26.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.329+0000 7f270f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f271015bc90 0x7f27100b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36984/0 (socket says 172.21.15.7:36984) 2024-09-10T21:47:26.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.329+0000 7f270f400640 1 -- 172.21.15.7:0/4174514712 learned_addr learned my addr 172.21.15.7:0/4174514712 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:26.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.329+0000 7f2706000640 1 --2- 172.21.15.7:0/4174514712 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27100cd750 0x7f27100c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:26.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.329+0000 7f270ea00640 1 --2- 172.21.15.7:0/4174514712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2710073c70 0x7f2710150450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:26.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.330+0000 7f270f400640 1 -- 172.21.15.7:0/4174514712 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27100cd750 msgr2=0x7f27100c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.330+0000 7f270f400640 1 --2- 172.21.15.7:0/4174514712 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27100cd750 0x7f27100c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.330+0000 7f270f400640 1 -- 172.21.15.7:0/4174514712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2710073c70 msgr2=0x7f2710150450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.330+0000 7f270f400640 1 --2- 172.21.15.7:0/4174514712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2710073c70 0x7f2710150450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.330+0000 7f270f400640 1 -- 172.21.15.7:0/4174514712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27100c8680 con 0x7f271015bc90 2024-09-10T21:47:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.330+0000 7f270f400640 1 --2- 172.21.15.7:0/4174514712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f271015bc90 0x7f27100b4960 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f270800c730 tx=0x7f270800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.330+0000 7f2707400640 1 -- 172.21.15.7:0/4174514712 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2708003b70 con 0x7f271015bc90 2024-09-10T21:47:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.330+0000 7f2707400640 1 -- 172.21.15.7:0/4174514712 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2708003d10 con 0x7f271015bc90 2024-09-10T21:47:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.330+0000 7f2707400640 1 -- 172.21.15.7:0/4174514712 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f270800aab0 con 0x7f271015bc90 2024-09-10T21:47:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.330+0000 7f2715600640 1 -- 172.21.15.7:0/4174514712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27100c8960 con 0x7f271015bc90 2024-09-10T21:47:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.331+0000 7f2715600640 1 -- 172.21.15.7:0/4174514712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27101f5360 con 0x7f271015bc90 2024-09-10T21:47:26.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.331+0000 7f2715600640 1 -- 172.21.15.7:0/4174514712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27100aec60 con 0x7f271015bc90 2024-09-10T21:47:26.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.333+0000 7f2707400640 1 -- 172.21.15.7:0/4174514712 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f270800ac50 con 0x7f271015bc90 2024-09-10T21:47:26.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.334+0000 7f2707400640 1 --2- 172.21.15.7:0/4174514712 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f26ec077f70 0x7f26ec07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:26.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.334+0000 7f2707400640 1 -- 172.21.15.7:0/4174514712 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f2708099a20 con 0x7f271015bc90 2024-09-10T21:47:26.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.334+0000 7f270ea00640 1 --2- 172.21.15.7:0/4174514712 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f26ec077f70 0x7f26ec07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:26.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.335+0000 7f270ea00640 1 --2- 172.21.15.7:0/4174514712 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f26ec077f70 0x7f26ec07a430 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2700033000 tx=0x7f2700004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:26.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.338+0000 7f2707400640 1 -- 172.21.15.7:0/4174514712 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2708061e30 con 0x7f271015bc90 2024-09-10T21:47:26.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.569+0000 7f2715600640 1 -- 172.21.15.7:0/4174514712 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2710073730 con 0x7f271015bc90 2024-09-10T21:47:26.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.570+0000 7f2707400640 1 -- 172.21.15.7:0/4174514712 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5237 (secure 0 0 0) 0x7f2708066860 con 0x7f271015bc90 2024-09-10T21:47:26.571 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:26.571 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":54,"btime":"2024-09-10T21:46:58:078261+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5884,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3488410478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3488410478},{"type":"v1","addr":"172.21.15.133:6837","nonce":3488410478}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5890,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3660400727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3660400727},{"type":"v1","addr":"172.21.15.7:6835","nonce":3660400727}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2388873704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2388873704},{"type":"v1","addr":"172.21.15.7:6834","nonce":2388873704}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:46:00.260975+0000","modified":"2024-09-10T21:46:57.678038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235490":{"gid":235490,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":15,"addr":"172.21.15.133:6836/2622530239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2622530239},{"type":"v1","addr":"172.21.15.133:6836","nonce":2622530239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235490,"qdb_cluster":[235490]},"id":5}]} 2024-09-10T21:47:26.571 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 54 2024-09-10T21:47:26.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.572+0000 7f2715600640 1 -- 172.21.15.7:0/4174514712 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f26ec077f70 msgr2=0x7f26ec07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:26.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.572+0000 7f2715600640 1 --2- 172.21.15.7:0/4174514712 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f26ec077f70 0x7f26ec07a430 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2700033000 tx=0x7f2700004730 comp rx=0 tx=0).stop 2024-09-10T21:47:26.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.572+0000 7f2715600640 1 -- 172.21.15.7:0/4174514712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f271015bc90 msgr2=0x7f27100b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:26.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.572+0000 7f2715600640 1 --2- 172.21.15.7:0/4174514712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f271015bc90 0x7f27100b4960 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f270800c730 tx=0x7f270800cbf0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.572+0000 7f2715600640 1 -- 172.21.15.7:0/4174514712 shutdown_connections 2024-09-10T21:47:26.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.572+0000 7f2715600640 1 --2- 172.21.15.7:0/4174514712 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f26ec077f70 0x7f26ec07a430 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.572+0000 7f2715600640 1 --2- 172.21.15.7:0/4174514712 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f271015bc90 0x7f27100b4960 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.572+0000 7f2715600640 1 --2- 172.21.15.7:0/4174514712 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f27100cd750 0x7f27100c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.572+0000 7f2715600640 1 --2- 172.21.15.7:0/4174514712 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2710073c70 0x7f2710150450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.572+0000 7f2715600640 1 -- 172.21.15.7:0/4174514712 >> 172.21.15.7:0/4174514712 conn(0x7f27100b16d0 msgr2=0x7f27100740a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:26.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.573+0000 7f2715600640 1 -- 172.21.15.7:0/4174514712 shutdown_connections 2024-09-10T21:47:26.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.573+0000 7f2715600640 1 -- 172.21.15.7:0/4174514712 wait complete. 2024-09-10T21:47:26.590 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:47:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.749+0000 7f1a1fa00640 1 -- 172.21.15.7:0/1041014645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a180cd750 msgr2=0x7f1a180ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.749+0000 7f1a1fa00640 1 --2- 172.21.15.7:0/1041014645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a180cd750 0x7f1a180ccef0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f1a180c3650 tx=0x7f1a0402a4d0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.749+0000 7f1a1fa00640 1 -- 172.21.15.7:0/1041014645 shutdown_connections 2024-09-10T21:47:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.749+0000 7f1a1fa00640 1 --2- 172.21.15.7:0/1041014645 >> v2:172.21.15.133:3300/0 conn(0x7f1a1815fda0 0x7f1a18160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.749+0000 7f1a1fa00640 1 --2- 172.21.15.7:0/1041014645 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a180cd750 0x7f1a180ccef0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.749+0000 7f1a1fa00640 1 -- 172.21.15.7:0/1041014645 >> 172.21.15.7:0/1041014645 conn(0x7f1a180b16d0 msgr2=0x7f1a180b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:26.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.749+0000 7f1a1fa00640 1 -- 172.21.15.7:0/1041014645 shutdown_connections 2024-09-10T21:47:26.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.749+0000 7f1a1fa00640 1 -- 172.21.15.7:0/1041014645 wait complete. 2024-09-10T21:47:26.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.750+0000 7f1a1fa00640 1 Processor -- start 2024-09-10T21:47:26.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.750+0000 7f1a1fa00640 1 -- start start 2024-09-10T21:47:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.751+0000 7f1a1fa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a180cd750 0x7f1a1815c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.751+0000 7f1a1fa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a1815bc90 0x7f1a1814dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.751+0000 7f1a1fa00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a1815fda0 0x7f1a180c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.751+0000 7f1a1fa00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1806fe70 con 0x7f1a180cd750 2024-09-10T21:47:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.751+0000 7f1a1fa00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1806fcf0 con 0x7f1a1815bc90 2024-09-10T21:47:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.751+0000 7f1a1fa00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f1a1806fff0 con 0x7f1a1815fda0 2024-09-10T21:47:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.752+0000 7f1a1d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a180cd750 0x7f1a1815c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.752+0000 7f1a1d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a180cd750 0x7f1a1815c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37002/0 (socket says 172.21.15.7:37002) 2024-09-10T21:47:26.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.752+0000 7f1a1d000640 1 -- 172.21.15.7:0/1527593602 learned_addr learned my addr 172.21.15.7:0/1527593602 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:26.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.752+0000 7f1a17e00640 1 --2- 172.21.15.7:0/1527593602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a1815bc90 0x7f1a1814dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:26.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.752+0000 7f1a1da00640 1 --2- 172.21.15.7:0/1527593602 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a1815fda0 0x7f1a180c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:26.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.752+0000 7f1a1d000640 1 -- 172.21.15.7:0/1527593602 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a1815fda0 msgr2=0x7f1a180c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:26.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.752+0000 7f1a1d000640 1 --2- 172.21.15.7:0/1527593602 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a1815fda0 0x7f1a180c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.752+0000 7f1a1d000640 1 -- 172.21.15.7:0/1527593602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a1815bc90 msgr2=0x7f1a1814dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:26.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.752+0000 7f1a1d000640 1 --2- 172.21.15.7:0/1527593602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a1815bc90 0x7f1a1814dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.752+0000 7f1a1d000640 1 -- 172.21.15.7:0/1527593602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a181f2010 con 0x7f1a180cd750 2024-09-10T21:47:26.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.752+0000 7f1a1d000640 1 --2- 172.21.15.7:0/1527593602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a180cd750 0x7f1a1815c260 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00c430 tx=0x7f1a0c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:26.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.753+0000 7f1a14c00640 1 -- 172.21.15.7:0/1527593602 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c011790 con 0x7f1a180cd750 2024-09-10T21:47:26.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.753+0000 7f1a14c00640 1 -- 172.21.15.7:0/1527593602 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a0c017070 con 0x7f1a180cd750 2024-09-10T21:47:26.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.753+0000 7f1a14c00640 1 -- 172.21.15.7:0/1527593602 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a0c011d00 con 0x7f1a180cd750 2024-09-10T21:47:26.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.753+0000 7f1a1fa00640 1 -- 172.21.15.7:0/1527593602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1815c8b0 con 0x7f1a180cd750 2024-09-10T21:47:26.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.753+0000 7f1a1fa00640 1 -- 172.21.15.7:0/1527593602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a1815ce70 con 0x7f1a180cd750 2024-09-10T21:47:26.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.754+0000 7f1a1fa00640 1 -- 172.21.15.7:0/1527593602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a18070510 con 0x7f1a180cd750 2024-09-10T21:47:26.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.755+0000 7f1a14c00640 1 -- 172.21.15.7:0/1527593602 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1a0c01c460 con 0x7f1a180cd750 2024-09-10T21:47:26.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.756+0000 7f1a14c00640 1 --2- 172.21.15.7:0/1527593602 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f19f8077f70 0x7f19f807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:26.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.756+0000 7f1a14c00640 1 -- 172.21.15.7:0/1527593602 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f1a0c099b90 con 0x7f1a180cd750 2024-09-10T21:47:26.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.757+0000 7f1a17e00640 1 --2- 172.21.15.7:0/1527593602 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f19f8077f70 0x7f19f807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:26.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.757+0000 7f1a17e00640 1 --2- 172.21.15.7:0/1527593602 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f19f8077f70 0x7f19f807a430 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1a04033000 tx=0x7f1a04004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:26.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.760+0000 7f1a14c00640 1 -- 172.21.15.7:0/1527593602 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1a0c061fa0 con 0x7f1a180cd750 2024-09-10T21:47:26.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.991+0000 7f1a1fa00640 1 -- 172.21.15.7:0/1527593602 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1a18069810 con 0x7f1a180cd750 2024-09-10T21:47:26.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.992+0000 7f1a14c00640 1 -- 172.21.15.7:0/1527593602 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) ==== 94+0+5237 (secure 0 0 0) 0x7f1a0c0669d0 con 0x7f1a180cd750 2024-09-10T21:47:26.993 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:26.993 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":54,"btime":"2024-09-10T21:46:58:078261+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5884,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3488410478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3488410478},{"type":"v1","addr":"172.21.15.133:6837","nonce":3488410478}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5890,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3660400727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3660400727},{"type":"v1","addr":"172.21.15.7:6835","nonce":3660400727}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2388873704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2388873704},{"type":"v1","addr":"172.21.15.7:6834","nonce":2388873704}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:46:00.260975+0000","modified":"2024-09-10T21:46:57.678038+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235490},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235490":{"gid":235490,"name":"b","rank":0,"incarnation":49,"state":"up:active","state_seq":15,"addr":"172.21.15.133:6836/2622530239","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2622530239},{"type":"v1","addr":"172.21.15.133:6836","nonce":2622530239}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235490,"qdb_cluster":[235490]},"id":5}]} 2024-09-10T21:47:26.993 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 54 2024-09-10T21:47:26.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.994+0000 7f1a1fa00640 1 -- 172.21.15.7:0/1527593602 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f19f8077f70 msgr2=0x7f19f807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.994+0000 7f1a1fa00640 1 --2- 172.21.15.7:0/1527593602 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f19f8077f70 0x7f19f807a430 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1a04033000 tx=0x7f1a04004730 comp rx=0 tx=0).stop 2024-09-10T21:47:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.994+0000 7f1a1fa00640 1 -- 172.21.15.7:0/1527593602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a180cd750 msgr2=0x7f1a1815c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.994+0000 7f1a1fa00640 1 --2- 172.21.15.7:0/1527593602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a180cd750 0x7f1a1815c260 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c00c430 tx=0x7f1a0c00c8f0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.994+0000 7f1a1fa00640 1 -- 172.21.15.7:0/1527593602 shutdown_connections 2024-09-10T21:47:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.994+0000 7f1a1fa00640 1 --2- 172.21.15.7:0/1527593602 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f19f8077f70 0x7f19f807a430 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.994+0000 7f1a1fa00640 1 --2- 172.21.15.7:0/1527593602 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1a1815bc90 0x7f1a1814dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.994+0000 7f1a1fa00640 1 --2- 172.21.15.7:0/1527593602 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1a1815fda0 0x7f1a180c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.994+0000 7f1a1fa00640 1 --2- 172.21.15.7:0/1527593602 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1a180cd750 0x7f1a1815c260 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.994+0000 7f1a1fa00640 1 -- 172.21.15.7:0/1527593602 >> 172.21.15.7:0/1527593602 conn(0x7f1a180b16d0 msgr2=0x7f1a180b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.994+0000 7f1a1fa00640 1 -- 172.21.15.7:0/1527593602 shutdown_connections 2024-09-10T21:47:26.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:26.994+0000 7f1a1fa00640 1 -- 172.21.15.7:0/1527593602 wait complete. 2024-09-10T21:47:27.011 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:47:27.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.172+0000 7fe061200640 1 -- 172.21.15.7:0/3236775040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe040003fd0 msgr2=0x7fe040004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:27.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.172+0000 7fe061200640 1 --2- 172.21.15.7:0/3236775040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe040003fd0 0x7fe040004800 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe044009b80 tx=0x7fe040005830 comp rx=0 tx=0).stop 2024-09-10T21:47:27.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.172+0000 7fe05a800640 1 -- 172.21.15.7:0/3236775040 reap_dead start 2024-09-10T21:47:27.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.172+0000 7fe061200640 1 -- 172.21.15.7:0/3236775040 shutdown_connections 2024-09-10T21:47:27.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.172+0000 7fe061200640 1 -- 172.21.15.7:0/3236775040 >> 172.21.15.7:0/3236775040 conn(0x7fe05c0b16d0 msgr2=0x7fe05c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:27.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.172+0000 7fe061200640 1 -- 172.21.15.7:0/3236775040 shutdown_connections 2024-09-10T21:47:27.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.172+0000 7fe061200640 1 -- 172.21.15.7:0/3236775040 wait complete. 2024-09-10T21:47:27.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.174+0000 7fe061200640 1 Processor -- start 2024-09-10T21:47:27.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.174+0000 7fe061200640 1 -- start start 2024-09-10T21:47:27.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.175+0000 7fe061200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe040003fd0 0x7fe05c14c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:27.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.175+0000 7fe061200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe05c158fc0 0x7fe05c1f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:27.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.175+0000 7fe061200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05c0c3dc0 0x7fe05c0c4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:27.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.175+0000 7fe061200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c0cb0c0 con 0x7fe040003fd0 2024-09-10T21:47:27.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.175+0000 7fe061200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe05c06ee50 con 0x7fe05c158fc0 2024-09-10T21:47:27.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.175+0000 7fe061200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe05c0cb240 con 0x7fe05c0c3dc0 2024-09-10T21:47:27.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.175+0000 7fe059e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe040003fd0 0x7fe05c14c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:27.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.176+0000 7fe059e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe040003fd0 0x7fe05c14c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37020/0 (socket says 172.21.15.7:37020) 2024-09-10T21:47:27.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.176+0000 7fe059e00640 1 -- 172.21.15.7:0/893328166 learned_addr learned my addr 172.21.15.7:0/893328166 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:27.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.176+0000 7fe051400640 1 --2- 172.21.15.7:0/893328166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe05c158fc0 0x7fe05c1f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:27.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.176+0000 7fe05a800640 1 --2- 172.21.15.7:0/893328166 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05c0c3dc0 0x7fe05c0c4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:27.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.176+0000 7fe059e00640 1 -- 172.21.15.7:0/893328166 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05c0c3dc0 msgr2=0x7fe05c0c4270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:27.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.176+0000 7fe059e00640 1 --2- 172.21.15.7:0/893328166 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05c0c3dc0 0x7fe05c0c4270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.176+0000 7fe059e00640 1 -- 172.21.15.7:0/893328166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe05c158fc0 msgr2=0x7fe05c1f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:27.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.176+0000 7fe059e00640 1 --2- 172.21.15.7:0/893328166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe05c158fc0 0x7fe05c1f5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.176+0000 7fe059e00640 1 -- 172.21.15.7:0/893328166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05c0c6a20 con 0x7fe040003fd0 2024-09-10T21:47:27.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.176+0000 7fe059e00640 1 --2- 172.21.15.7:0/893328166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe040003fd0 0x7fe05c14c070 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fe04c001160 tx=0x7fe04c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:27.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.176+0000 7fe052a00640 1 -- 172.21.15.7:0/893328166 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c015070 con 0x7fe040003fd0 2024-09-10T21:47:27.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.176+0000 7fe052a00640 1 -- 172.21.15.7:0/893328166 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe04c011900 con 0x7fe040003fd0 2024-09-10T21:47:27.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.176+0000 7fe052a00640 1 -- 172.21.15.7:0/893328166 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe04c011d30 con 0x7fe040003fd0 2024-09-10T21:47:27.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.176+0000 7fe061200640 1 -- 172.21.15.7:0/893328166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe05c0c5c20 con 0x7fe040003fd0 2024-09-10T21:47:27.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.177+0000 7fe061200640 1 -- 172.21.15.7:0/893328166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe05c0c5ea0 con 0x7fe040003fd0 2024-09-10T21:47:27.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.178+0000 7fe061200640 1 -- 172.21.15.7:0/893328166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe05c06e510 con 0x7fe040003fd0 2024-09-10T21:47:27.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.179+0000 7fe052a00640 1 -- 172.21.15.7:0/893328166 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe04c0103c0 con 0x7fe040003fd0 2024-09-10T21:47:27.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.180+0000 7fe052a00640 1 --2- 172.21.15.7:0/893328166 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe038077f70 0x7fe03807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:27.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.180+0000 7fe052a00640 1 -- 172.21.15.7:0/893328166 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7fe04c09aeb0 con 0x7fe040003fd0 2024-09-10T21:47:27.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.181+0000 7fe051400640 1 --2- 172.21.15.7:0/893328166 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe038077f70 0x7fe03807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:27.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.182+0000 7fe051400640 1 --2- 172.21.15.7:0/893328166 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe038077f70 0x7fe03807a430 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe044035000 tx=0x7fe0440046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:27.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.185+0000 7fe052a00640 1 -- 172.21.15.7:0/893328166 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe05c0c5c20 con 0x7fe040003fd0 2024-09-10T21:47:27.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.379+0000 7fe061200640 1 -- 172.21.15.7:0/893328166 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe05c0c7f40 con 0x7fe040003fd0 2024-09-10T21:47:27.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.380+0000 7fe052a00640 1 -- 172.21.15.7:0/893328166 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+16920 (secure 0 0 0) 0x7fe04c0632c0 con 0x7fe040003fd0 2024-09-10T21:47:27.382 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:27.382 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":73,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:46:49.043545+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:45:58.069464+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-10T21:45:59.225636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.a","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6836/2752881644":"2024-09-11T21:46:10.658995+0000","172.21.15.133:6834/2752881644":"2024-09-11T21:46:10.658995+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:47:27.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.383+0000 7fe061200640 1 -- 172.21.15.7:0/893328166 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe038077f70 msgr2=0x7fe03807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:27.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.383+0000 7fe061200640 1 --2- 172.21.15.7:0/893328166 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe038077f70 0x7fe03807a430 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe044035000 tx=0x7fe0440046c0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.383+0000 7fe061200640 1 -- 172.21.15.7:0/893328166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe040003fd0 msgr2=0x7fe05c14c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:27.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.383+0000 7fe061200640 1 --2- 172.21.15.7:0/893328166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe040003fd0 0x7fe05c14c070 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fe04c001160 tx=0x7fe04c0039c0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.383+0000 7fe061200640 1 -- 172.21.15.7:0/893328166 shutdown_connections 2024-09-10T21:47:27.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.383+0000 7fe061200640 1 --2- 172.21.15.7:0/893328166 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe038077f70 0x7fe03807a430 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.383+0000 7fe061200640 1 --2- 172.21.15.7:0/893328166 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe05c158fc0 0x7fe05c1f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.383+0000 7fe061200640 1 --2- 172.21.15.7:0/893328166 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05c0c3dc0 0x7fe05c0c4270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.383+0000 7fe061200640 1 --2- 172.21.15.7:0/893328166 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe040003fd0 0x7fe05c14c070 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.383+0000 7fe061200640 1 -- 172.21.15.7:0/893328166 >> 172.21.15.7:0/893328166 conn(0x7fe05c0b16d0 msgr2=0x7fe05c0cc690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:27.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.383+0000 7fe061200640 1 -- 172.21.15.7:0/893328166 shutdown_connections 2024-09-10T21:47:27.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.383+0000 7fe061200640 1 -- 172.21.15.7:0/893328166 wait complete. 2024-09-10T21:47:27.401 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:47:27.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.561+0000 7f91dde00640 1 -- 172.21.15.7:0/320454518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91d815dff0 msgr2=0x7f91d80d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:27.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.561+0000 7f91dde00640 1 --2- 172.21.15.7:0/320454518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91d815dff0 0x7f91d80d4900 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f91d80d3db0 tx=0x7f91cc02a4d0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.561+0000 7f91dde00640 1 -- 172.21.15.7:0/320454518 shutdown_connections 2024-09-10T21:47:27.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.561+0000 7f91dde00640 1 --2- 172.21.15.7:0/320454518 >> v2:172.21.15.133:3300/0 conn(0x7f91d80b21f0 0x7f91d80b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.561+0000 7f91dde00640 1 --2- 172.21.15.7:0/320454518 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91d80b0970 0x7f91d80b6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.561+0000 7f91dde00640 1 --2- 172.21.15.7:0/320454518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91d815dff0 0x7f91d80d4900 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.561+0000 7f91dde00640 1 -- 172.21.15.7:0/320454518 >> 172.21.15.7:0/320454518 conn(0x7f91d80d3de0 msgr2=0x7f91d80d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:27.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.561+0000 7f91dde00640 1 -- 172.21.15.7:0/320454518 shutdown_connections 2024-09-10T21:47:27.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.561+0000 7f91dde00640 1 -- 172.21.15.7:0/320454518 wait complete. 2024-09-10T21:47:27.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.562+0000 7f91dde00640 1 Processor -- start 2024-09-10T21:47:27.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.563+0000 7f91dde00640 1 -- start start 2024-09-10T21:47:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.563+0000 7f91dde00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91d80b0970 0x7f91d81f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.563+0000 7f91dde00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91d80b21f0 0x7f91d80cc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.563+0000 7f91dde00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d815dff0 0x7f91d815b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.563+0000 7f91dde00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f91d80ce940 con 0x7f91d80b21f0 2024-09-10T21:47:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.563+0000 7f91dde00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f91d80ce7c0 con 0x7f91d815dff0 2024-09-10T21:47:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.563+0000 7f91dde00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f91d80ceac0 con 0x7f91d80b0970 2024-09-10T21:47:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.564+0000 7f91d7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91d80b21f0 0x7f91d80cc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.564+0000 7f91d7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91d80b21f0 0x7f91d80cc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37052/0 (socket says 172.21.15.7:37052) 2024-09-10T21:47:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.564+0000 7f91d7e00640 1 -- 172.21.15.7:0/1693531684 learned_addr learned my addr 172.21.15.7:0/1693531684 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.564+0000 7f91dd400640 1 --2- 172.21.15.7:0/1693531684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d815dff0 0x7f91d815b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:27.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.564+0000 7f91dca00640 1 --2- 172.21.15.7:0/1693531684 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91d80b0970 0x7f91d81f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:27.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.564+0000 7f91d7e00640 1 -- 172.21.15.7:0/1693531684 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91d80b0970 msgr2=0x7f91d81f83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:27.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.564+0000 7f91d7e00640 1 --2- 172.21.15.7:0/1693531684 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91d80b0970 0x7f91d81f83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.564+0000 7f91d7e00640 1 -- 172.21.15.7:0/1693531684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d815dff0 msgr2=0x7f91d815b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:27.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.564+0000 7f91d7e00640 1 --2- 172.21.15.7:0/1693531684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d815dff0 0x7f91d815b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.564+0000 7f91d7e00640 1 -- 172.21.15.7:0/1693531684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91d81f48c0 con 0x7f91d80b21f0 2024-09-10T21:47:27.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.565+0000 7f91d7e00640 1 --2- 172.21.15.7:0/1693531684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91d80b21f0 0x7f91d80cc7e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f91c400d770 tx=0x7f91c400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:27.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.565+0000 7f91d4c00640 1 -- 172.21.15.7:0/1693531684 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c4017070 con 0x7f91d80b21f0 2024-09-10T21:47:27.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.565+0000 7f91d4c00640 1 -- 172.21.15.7:0/1693531684 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91c4009d20 con 0x7f91d80b21f0 2024-09-10T21:47:27.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.565+0000 7f91dde00640 1 -- 172.21.15.7:0/1693531684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91d81f4ba0 con 0x7f91d80b21f0 2024-09-10T21:47:27.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.565+0000 7f91d4c00640 1 -- 172.21.15.7:0/1693531684 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c40106c0 con 0x7f91d80b21f0 2024-09-10T21:47:27.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.565+0000 7f91dde00640 1 -- 172.21.15.7:0/1693531684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91d81f5160 con 0x7f91d80b21f0 2024-09-10T21:47:27.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.566+0000 7f91dde00640 1 -- 172.21.15.7:0/1693531684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91d80b6c30 con 0x7f91d80b21f0 2024-09-10T21:47:27.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.567+0000 7f91d4c00640 1 -- 172.21.15.7:0/1693531684 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f91c4010860 con 0x7f91d80b21f0 2024-09-10T21:47:27.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.569+0000 7f91d4c00640 1 --2- 172.21.15.7:0/1693531684 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f91b8077f70 0x7f91b807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:27.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.569+0000 7f91d4c00640 1 -- 172.21.15.7:0/1693531684 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5853+0+0 (secure 0 0 0) 0x7f91c409a280 con 0x7f91d80b21f0 2024-09-10T21:47:27.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.569+0000 7f91dca00640 1 --2- 172.21.15.7:0/1693531684 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f91b8077f70 0x7f91b807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:27.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.570+0000 7f91dca00640 1 --2- 172.21.15.7:0/1693531684 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f91b8077f70 0x7f91b807a430 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f91cc033000 tx=0x7f91cc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:27.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.573+0000 7f91d4c00640 1 -- 172.21.15.7:0/1693531684 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f91c4062600 con 0x7f91d80b21f0 2024-09-10T21:47:27.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.802+0000 7f91dde00640 1 -- 172.21.15.7:0/1693531684 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f91d8160f10 con 0x7f91d80b21f0 2024-09-10T21:47:27.803 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-10T21:47:27.803+0000 7feb3fa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:47:27.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.805+0000 7f91d4c00640 1 -- 172.21.15.7:0/1693531684 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) ==== 161+0+0 (secure 0 0 0) 0x7f91c4067030 con 0x7f91d80b21f0 2024-09-10T21:47:27.806 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:47:27.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.807+0000 7f91dde00640 1 -- 172.21.15.7:0/1693531684 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f91b8077f70 msgr2=0x7f91b807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:27.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.807+0000 7f91dde00640 1 --2- 172.21.15.7:0/1693531684 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f91b8077f70 0x7f91b807a430 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f91cc033000 tx=0x7f91cc004730 comp rx=0 tx=0).stop 2024-09-10T21:47:27.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.807+0000 7f91dde00640 1 -- 172.21.15.7:0/1693531684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91d80b21f0 msgr2=0x7f91d80cc7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:27.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.807+0000 7f91dde00640 1 --2- 172.21.15.7:0/1693531684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91d80b21f0 0x7f91d80cc7e0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f91c400d770 tx=0x7f91c400dc30 comp rx=0 tx=0).stop 2024-09-10T21:47:27.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.807+0000 7f91dde00640 1 -- 172.21.15.7:0/1693531684 shutdown_connections 2024-09-10T21:47:27.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.807+0000 7f91dde00640 1 --2- 172.21.15.7:0/1693531684 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f91b8077f70 0x7f91b807a430 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.807+0000 7f91dde00640 1 --2- 172.21.15.7:0/1693531684 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f91d815dff0 0x7f91d815b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.807+0000 7f91dde00640 1 --2- 172.21.15.7:0/1693531684 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f91d80b21f0 0x7f91d80cc7e0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.807+0000 7f91dde00640 1 --2- 172.21.15.7:0/1693531684 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f91d80b0970 0x7f91d81f83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.807+0000 7f91dde00640 1 -- 172.21.15.7:0/1693531684 >> 172.21.15.7:0/1693531684 conn(0x7f91d80d3de0 msgr2=0x7f91d80b7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:27.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.807+0000 7f91dde00640 1 -- 172.21.15.7:0/1693531684 shutdown_connections 2024-09-10T21:47:27.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.807+0000 7f91dde00640 1 -- 172.21.15.7:0/1693531684 wait complete. 2024-09-10T21:47:27.821 INFO:tasks.ceph.mds.b.smithi133.stderr: -4400> 2024-09-10T21:46:16.674+0000 7f567f600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:47:27.825 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:47:27.871 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2024-09-10T21:47:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.988+0000 7fcc26800640 1 -- 172.21.15.7:0/1369173027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc200ccc40 msgr2=0x7fcc2015e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.988+0000 7fcc26800640 1 --2- 172.21.15.7:0/1369173027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc200ccc40 0x7fcc2015e8d0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fcc18005fd0 tx=0x7fcc1802a820 comp rx=0 tx=0).stop 2024-09-10T21:47:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.989+0000 7fcc26800640 1 -- 172.21.15.7:0/1369173027 shutdown_connections 2024-09-10T21:47:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.989+0000 7fcc26800640 1 --2- 172.21.15.7:0/1369173027 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc200ccc40 0x7fcc2015e8d0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.989+0000 7fcc26800640 1 --2- 172.21.15.7:0/1369173027 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc2015ba00 0x7fcc200ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.989+0000 7fcc26800640 1 -- 172.21.15.7:0/1369173027 >> 172.21.15.7:0/1369173027 conn(0x7fcc200b16d0 msgr2=0x7fcc200b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:27.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.989+0000 7fcc26800640 1 -- 172.21.15.7:0/1369173027 shutdown_connections 2024-09-10T21:47:27.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.989+0000 7fcc26800640 1 -- 172.21.15.7:0/1369173027 wait complete. 2024-09-10T21:47:27.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.990+0000 7fcc26800640 1 Processor -- start 2024-09-10T21:47:27.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.990+0000 7fcc26800640 1 -- start start 2024-09-10T21:47:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.991+0000 7fcc26800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc200ccc40 0x7fcc201f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.991+0000 7fcc26800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc2015ba00 0x7fcc20158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.991+0000 7fcc26800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc200c9b80 0x7fcc2015efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.991+0000 7fcc26800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc200af280 con 0x7fcc200c9b80 2024-09-10T21:47:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.991+0000 7fcc26800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcc200af100 con 0x7fcc2015ba00 2024-09-10T21:47:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.991+0000 7fcc26800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fcc200af400 con 0x7fcc200ccc40 2024-09-10T21:47:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.992+0000 7fcc1fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc200c9b80 0x7fcc2015efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:27.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.992+0000 7fcc1fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc200c9b80 0x7fcc2015efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37060/0 (socket says 172.21.15.7:37060) 2024-09-10T21:47:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.992+0000 7fcc1fe00640 1 -- 172.21.15.7:0/3497412105 learned_addr learned my addr 172.21.15.7:0/3497412105 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.992+0000 7fcc1f400640 1 --2- 172.21.15.7:0/3497412105 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc200ccc40 0x7fcc201f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.992+0000 7fcc16a00640 1 --2- 172.21.15.7:0/3497412105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc2015ba00 0x7fcc20158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.992+0000 7fcc1fe00640 1 -- 172.21.15.7:0/3497412105 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc200ccc40 msgr2=0x7fcc201f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.992+0000 7fcc1fe00640 1 --2- 172.21.15.7:0/3497412105 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc200ccc40 0x7fcc201f5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.992+0000 7fcc1fe00640 1 -- 172.21.15.7:0/3497412105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc2015ba00 msgr2=0x7fcc20158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.992+0000 7fcc1fe00640 1 --2- 172.21.15.7:0/3497412105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc2015ba00 0x7fcc20158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.992+0000 7fcc1fe00640 1 -- 172.21.15.7:0/3497412105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc200b29d0 con 0x7fcc200c9b80 2024-09-10T21:47:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.992+0000 7fcc16a00640 1 --2- 172.21.15.7:0/3497412105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc2015ba00 0x7fcc20158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:27.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.992+0000 7fcc1fe00640 1 --2- 172.21.15.7:0/3497412105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc200c9b80 0x7fcc2015efd0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fcc18006010 tx=0x7fcc18032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:27.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.993+0000 7fcc17e00640 1 -- 172.21.15.7:0/3497412105 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18037040 con 0x7fcc200c9b80 2024-09-10T21:47:27.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.993+0000 7fcc17e00640 1 -- 172.21.15.7:0/3497412105 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc18042070 con 0x7fcc200c9b80 2024-09-10T21:47:27.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.993+0000 7fcc17e00640 1 -- 172.21.15.7:0/3497412105 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc18003e60 con 0x7fcc200c9b80 2024-09-10T21:47:27.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.993+0000 7fcc26800640 1 -- 172.21.15.7:0/3497412105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc200b2cb0 con 0x7fcc200c9b80 2024-09-10T21:47:27.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.993+0000 7fcc26800640 1 -- 172.21.15.7:0/3497412105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc200b3270 con 0x7fcc200c9b80 2024-09-10T21:47:27.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.994+0000 7fcc26800640 1 -- 172.21.15.7:0/3497412105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc200708b0 con 0x7fcc200c9b80 2024-09-10T21:47:27.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.995+0000 7fcc17e00640 1 -- 172.21.15.7:0/3497412105 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fcc1803a030 con 0x7fcc200c9b80 2024-09-10T21:47:27.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.996+0000 7fcc17e00640 1 --2- 172.21.15.7:0/3497412105 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcbfc077f70 0x7fcbfc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:27.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.996+0000 7fcc17e00640 1 -- 172.21.15.7:0/3497412105 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5939+0+0 (secure 0 0 0) 0x7fcc180ba7d0 con 0x7fcc200c9b80 2024-09-10T21:47:27.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.996+0000 7fcc1f400640 1 --2- 172.21.15.7:0/3497412105 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcbfc077f70 0x7fcbfc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:27.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:27.997+0000 7fcc1f400640 1 --2- 172.21.15.7:0/3497412105 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcbfc077f70 0x7fcbfc07a430 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009bd0 tx=0x7fcc10005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:28.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.000+0000 7fcc17e00640 1 -- 172.21.15.7:0/3497412105 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fcc200708b0 con 0x7fcc200c9b80 2024-09-10T21:47:28.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.234+0000 7fcc26800640 1 -- 172.21.15.7:0/3497412105 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fcc2006fb50 con 0x7fcc200c9b80 2024-09-10T21:47:28.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.812+0000 7fcc17e00640 1 -- 172.21.15.7:0/3497412105 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) ==== 104+0+0 (secure 0 0 0) 0x7fcc180831e0 con 0x7fcc200c9b80 2024-09-10T21:47:28.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.814+0000 7fcc26800640 1 -- 172.21.15.7:0/3497412105 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcbfc077f70 msgr2=0x7fcbfc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:28.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.814+0000 7fcc26800640 1 --2- 172.21.15.7:0/3497412105 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcbfc077f70 0x7fcbfc07a430 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009bd0 tx=0x7fcc10005f90 comp rx=0 tx=0).stop 2024-09-10T21:47:28.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.814+0000 7fcc26800640 1 -- 172.21.15.7:0/3497412105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc200c9b80 msgr2=0x7fcc2015efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:28.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.814+0000 7fcc26800640 1 --2- 172.21.15.7:0/3497412105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc200c9b80 0x7fcc2015efd0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fcc18006010 tx=0x7fcc18032d10 comp rx=0 tx=0).stop 2024-09-10T21:47:28.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.814+0000 7fcc26800640 1 -- 172.21.15.7:0/3497412105 shutdown_connections 2024-09-10T21:47:28.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.814+0000 7fcc26800640 1 --2- 172.21.15.7:0/3497412105 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcbfc077f70 0x7fcbfc07a430 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:28.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.814+0000 7fcc26800640 1 --2- 172.21.15.7:0/3497412105 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc2015ba00 0x7fcc20158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:28.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.814+0000 7fcc26800640 1 --2- 172.21.15.7:0/3497412105 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc200c9b80 0x7fcc2015efd0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:28.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.814+0000 7fcc26800640 1 --2- 172.21.15.7:0/3497412105 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc200ccc40 0x7fcc201f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:28.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.814+0000 7fcc26800640 1 -- 172.21.15.7:0/3497412105 >> 172.21.15.7:0/3497412105 conn(0x7fcc200b16d0 msgr2=0x7fcc200af950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:28.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.814+0000 7fcc26800640 1 -- 172.21.15.7:0/3497412105 shutdown_connections 2024-09-10T21:47:28.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.814+0000 7fcc26800640 1 -- 172.21.15.7:0/3497412105 wait complete. 2024-09-10T21:47:28.833 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:47:29.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.999+0000 7fe288a00640 1 -- 172.21.15.7:0/2973645826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe28406d060 msgr2=0x7fe284073810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:29.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.999+0000 7fe288a00640 1 --2- 172.21.15.7:0/2973645826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe28406d060 0x7fe284073810 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fe284070a30 tx=0x7fe27c02a980 comp rx=0 tx=0).stop 2024-09-10T21:47:29.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.999+0000 7fe288a00640 1 -- 172.21.15.7:0/2973645826 shutdown_connections 2024-09-10T21:47:29.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.999+0000 7fe288a00640 1 --2- 172.21.15.7:0/2973645826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe28406d060 0x7fe284073810 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:29.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.999+0000 7fe288a00640 1 --2- 172.21.15.7:0/2973645826 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe28415edd0 0x7fe28406cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:29.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.999+0000 7fe288a00640 1 --2- 172.21.15.7:0/2973645826 >> v2:172.21.15.133:3300/0 conn(0x7fe284158300 0x7fe2840b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:29.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:28.999+0000 7fe288a00640 1 -- 172.21.15.7:0/2973645826 >> 172.21.15.7:0/2973645826 conn(0x7fe2840b1c10 msgr2=0x7fe2840b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:29.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.000+0000 7fe288a00640 1 -- 172.21.15.7:0/2973645826 shutdown_connections 2024-09-10T21:47:29.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.000+0000 7fe288a00640 1 -- 172.21.15.7:0/2973645826 wait complete. 2024-09-10T21:47:29.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.001+0000 7fe288a00640 1 Processor -- start 2024-09-10T21:47:29.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.001+0000 7fe288a00640 1 -- start start 2024-09-10T21:47:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.002+0000 7fe288a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe28406d060 0x7fe2841f7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.002+0000 7fe288a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe284158300 0x7fe284156f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.002+0000 7fe288a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe28415edd0 0x7fe2840cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.002+0000 7fe288a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe2840acf20 con 0x7fe284158300 2024-09-10T21:47:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.002+0000 7fe288a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe2840acda0 con 0x7fe28406d060 2024-09-10T21:47:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.002+0000 7fe288a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe284073d50 con 0x7fe28415edd0 2024-09-10T21:47:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.003+0000 7fe279400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe284158300 0x7fe284156f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.003+0000 7fe279400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe284158300 0x7fe284156f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37072/0 (socket says 172.21.15.7:37072) 2024-09-10T21:47:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.003+0000 7fe279400640 1 -- 172.21.15.7:0/294177713 learned_addr learned my addr 172.21.15.7:0/294177713 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.003+0000 7fe281e00640 1 --2- 172.21.15.7:0/294177713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe28406d060 0x7fe2841f7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.003+0000 7fe282800640 1 --2- 172.21.15.7:0/294177713 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe28415edd0 0x7fe2840cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.003+0000 7fe279400640 1 -- 172.21.15.7:0/294177713 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe28415edd0 msgr2=0x7fe2840cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.003+0000 7fe279400640 1 --2- 172.21.15.7:0/294177713 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe28415edd0 0x7fe2840cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.003+0000 7fe279400640 1 -- 172.21.15.7:0/294177713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe28406d060 msgr2=0x7fe2841f7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.003+0000 7fe279400640 1 --2- 172.21.15.7:0/294177713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe28406d060 0x7fe2841f7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.003+0000 7fe279400640 1 -- 172.21.15.7:0/294177713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe28409bd60 con 0x7fe284158300 2024-09-10T21:47:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.003+0000 7fe279400640 1 --2- 172.21.15.7:0/294177713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe284158300 0x7fe284156f40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe26c00d740 tx=0x7fe26c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.004+0000 7fe27aa00640 1 -- 172.21.15.7:0/294177713 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c017070 con 0x7fe284158300 2024-09-10T21:47:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.004+0000 7fe27aa00640 1 -- 172.21.15.7:0/294177713 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe26c009cc0 con 0x7fe284158300 2024-09-10T21:47:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.004+0000 7fe27aa00640 1 -- 172.21.15.7:0/294177713 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26c0106c0 con 0x7fe284158300 2024-09-10T21:47:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.004+0000 7fe288a00640 1 -- 172.21.15.7:0/294177713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2841f8400 con 0x7fe284158300 2024-09-10T21:47:29.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.004+0000 7fe288a00640 1 -- 172.21.15.7:0/294177713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2841f89c0 con 0x7fe284158300 2024-09-10T21:47:29.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.005+0000 7fe288a00640 1 -- 172.21.15.7:0/294177713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2841f8b90 con 0x7fe284158300 2024-09-10T21:47:29.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.006+0000 7fe27aa00640 1 -- 172.21.15.7:0/294177713 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe26c010860 con 0x7fe284158300 2024-09-10T21:47:29.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.007+0000 7fe27aa00640 1 --2- 172.21.15.7:0/294177713 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe260077f70 0x7fe26007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:29.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.007+0000 7fe27aa00640 1 -- 172.21.15.7:0/294177713 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 5939+0+0 (secure 0 0 0) 0x7fe26c09a150 con 0x7fe284158300 2024-09-10T21:47:29.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.007+0000 7fe281e00640 1 --2- 172.21.15.7:0/294177713 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe260077f70 0x7fe26007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:29.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.008+0000 7fe281e00640 1 --2- 172.21.15.7:0/294177713 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe260077f70 0x7fe26007a430 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe27400bbe0 tx=0x7fe27400b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:29.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.011+0000 7fe27aa00640 1 -- 172.21.15.7:0/294177713 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe26c062500 con 0x7fe284158300 2024-09-10T21:47:29.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:29.206+0000 7fe288a00640 1 -- 172.21.15.7:0/294177713 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe2840b08c0 con 0x7fe284158300 2024-09-10T21:47:30.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.152+0000 7fe27aa00640 1 -- 172.21.15.7:0/294177713 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v75) ==== 181+0+0 (secure 0 0 0) 0x7fe26c066f30 con 0x7fe284158300 2024-09-10T21:47:30.153 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:47:30.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.155+0000 7fe288a00640 1 -- 172.21.15.7:0/294177713 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe260077f70 msgr2=0x7fe26007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:30.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.155+0000 7fe288a00640 1 --2- 172.21.15.7:0/294177713 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe260077f70 0x7fe26007a430 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fe27400bbe0 tx=0x7fe27400b390 comp rx=0 tx=0).stop 2024-09-10T21:47:30.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.155+0000 7fe288a00640 1 -- 172.21.15.7:0/294177713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe284158300 msgr2=0x7fe284156f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:30.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.155+0000 7fe288a00640 1 --2- 172.21.15.7:0/294177713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe284158300 0x7fe284156f40 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fe26c00d740 tx=0x7fe26c00dc00 comp rx=0 tx=0).stop 2024-09-10T21:47:30.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.155+0000 7fe288a00640 1 -- 172.21.15.7:0/294177713 shutdown_connections 2024-09-10T21:47:30.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.155+0000 7fe288a00640 1 --2- 172.21.15.7:0/294177713 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe260077f70 0x7fe26007a430 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:30.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.155+0000 7fe288a00640 1 --2- 172.21.15.7:0/294177713 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe28415edd0 0x7fe2840cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:30.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.155+0000 7fe288a00640 1 --2- 172.21.15.7:0/294177713 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe284158300 0x7fe284156f40 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:30.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.155+0000 7fe288a00640 1 --2- 172.21.15.7:0/294177713 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe28406d060 0x7fe2841f7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:30.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.155+0000 7fe288a00640 1 -- 172.21.15.7:0/294177713 >> 172.21.15.7:0/294177713 conn(0x7fe2840b1c10 msgr2=0x7fe2840b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:30.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.155+0000 7fe288a00640 1 -- 172.21.15.7:0/294177713 shutdown_connections 2024-09-10T21:47:30.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.155+0000 7fe288a00640 1 -- 172.21.15.7:0/294177713 wait complete. 2024-09-10T21:47:30.175 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:47:30.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.342+0000 7f5084a00640 1 -- 172.21.15.7:0/3869873207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f508015bce0 msgr2=0x7f508006cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:30.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.342+0000 7f5084a00640 1 --2- 172.21.15.7:0/3869873207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f508015bce0 0x7f508006cb20 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f5078005fd0 tx=0x7f507802a820 comp rx=0 tx=0).stop 2024-09-10T21:47:30.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.342+0000 7f5084a00640 1 -- 172.21.15.7:0/3869873207 shutdown_connections 2024-09-10T21:47:30.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.342+0000 7f5084a00640 1 --2- 172.21.15.7:0/3869873207 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f508015bce0 0x7f508006cb20 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:30.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.342+0000 7f5084a00640 1 --2- 172.21.15.7:0/3869873207 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f50800ace80 0x7f5080070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:30.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.342+0000 7f5084a00640 1 -- 172.21.15.7:0/3869873207 >> 172.21.15.7:0/3869873207 conn(0x7f50800b1c10 msgr2=0x7f50800b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:30.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.342+0000 7f5084a00640 1 -- 172.21.15.7:0/3869873207 shutdown_connections 2024-09-10T21:47:30.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.342+0000 7f5084a00640 1 -- 172.21.15.7:0/3869873207 wait complete. 2024-09-10T21:47:30.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.343+0000 7f5084a00640 1 Processor -- start 2024-09-10T21:47:30.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.344+0000 7f5084a00640 1 -- start start 2024-09-10T21:47:30.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.344+0000 7f5084a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50800ace80 0x7f50801523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:30.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.344+0000 7f5084a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f508015b900 0x7f5080150420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:30.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.344+0000 7f5084a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f508015bce0 0x7f50801536f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:30.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.344+0000 7f5084a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f50800aed00 con 0x7f508015b900 2024-09-10T21:47:30.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.344+0000 7f5084a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f50800aeb80 con 0x7f50800ace80 2024-09-10T21:47:30.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.344+0000 7f5084a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f50800aee80 con 0x7f508015bce0 2024-09-10T21:47:30.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.345+0000 7f5075400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f508015b900 0x7f5080150420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:30.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.345+0000 7f5075400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f508015b900 0x7f5080150420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37088/0 (socket says 172.21.15.7:37088) 2024-09-10T21:47:30.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.345+0000 7f5075400640 1 -- 172.21.15.7:0/873467985 learned_addr learned my addr 172.21.15.7:0/873467985 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:30.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.345+0000 7f507e800640 1 --2- 172.21.15.7:0/873467985 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f508015bce0 0x7f50801536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:30.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.345+0000 7f507de00640 1 --2- 172.21.15.7:0/873467985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50800ace80 0x7f50801523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:30.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.345+0000 7f5075400640 1 -- 172.21.15.7:0/873467985 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f508015bce0 msgr2=0x7f50801536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:30.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.345+0000 7f5075400640 1 --2- 172.21.15.7:0/873467985 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f508015bce0 0x7f50801536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:30.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.345+0000 7f5075400640 1 -- 172.21.15.7:0/873467985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50800ace80 msgr2=0x7f50801523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:30.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.345+0000 7f5075400640 1 --2- 172.21.15.7:0/873467985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50800ace80 0x7f50801523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:30.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.345+0000 7f5075400640 1 -- 172.21.15.7:0/873467985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5080152930 con 0x7f508015b900 2024-09-10T21:47:30.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.345+0000 7f507de00640 1 --2- 172.21.15.7:0/873467985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50800ace80 0x7f50801523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:30.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.346+0000 7f5075400640 1 --2- 172.21.15.7:0/873467985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f508015b900 0x7f5080150420 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f506800d950 tx=0x7f506800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:30.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.346+0000 7f5076a00640 1 -- 172.21.15.7:0/873467985 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5068017070 con 0x7f508015b900 2024-09-10T21:47:30.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.346+0000 7f5076a00640 1 -- 172.21.15.7:0/873467985 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5068009d20 con 0x7f508015b900 2024-09-10T21:47:30.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.346+0000 7f5076a00640 1 -- 172.21.15.7:0/873467985 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50680106c0 con 0x7f508015b900 2024-09-10T21:47:30.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.346+0000 7f5084a00640 1 -- 172.21.15.7:0/873467985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5080150960 con 0x7f508015b900 2024-09-10T21:47:30.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.346+0000 7f5084a00640 1 -- 172.21.15.7:0/873467985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50800b1280 con 0x7f508015b900 2024-09-10T21:47:30.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.347+0000 7f5084a00640 1 -- 172.21.15.7:0/873467985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50800af330 con 0x7f508015b900 2024-09-10T21:47:30.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.348+0000 7f5076a00640 1 -- 172.21.15.7:0/873467985 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5068010860 con 0x7f508015b900 2024-09-10T21:47:30.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.349+0000 7f5076a00640 1 --2- 172.21.15.7:0/873467985 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f505c077fa0 0x7f505c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:30.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.349+0000 7f5076a00640 1 -- 172.21.15.7:0/873467985 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 5501+0+0 (secure 0 0 0) 0x7f506809a0d0 con 0x7f508015b900 2024-09-10T21:47:30.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.350+0000 7f507de00640 1 --2- 172.21.15.7:0/873467985 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f505c077fa0 0x7f505c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:30.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.350+0000 7f507de00640 1 --2- 172.21.15.7:0/873467985 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f505c077fa0 0x7f505c07a460 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5070007a50 tx=0x7f5070009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:30.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.353+0000 7f5076a00640 1 -- 172.21.15.7:0/873467985 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f50680625b0 con 0x7f508015b900 2024-09-10T21:47:30.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:30.548+0000 7f5084a00640 1 -- 172.21.15.7:0/873467985 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f5080159370 con 0x7f508015b900 2024-09-10T21:47:30.745 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:47:30.749 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:47:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.158+0000 7f5076a00640 1 -- 172.21.15.7:0/873467985 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v76) ==== 169+0+0 (secure 0 0 0) 0x7f5068066fe0 con 0x7f508015b900 2024-09-10T21:47:31.159 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-09-10T21:47:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.161+0000 7f5084a00640 1 -- 172.21.15.7:0/873467985 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f505c077fa0 msgr2=0x7f505c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.161+0000 7f5084a00640 1 --2- 172.21.15.7:0/873467985 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f505c077fa0 0x7f505c07a460 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f5070007a50 tx=0x7f5070009040 comp rx=0 tx=0).stop 2024-09-10T21:47:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.161+0000 7f5084a00640 1 -- 172.21.15.7:0/873467985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f508015b900 msgr2=0x7f5080150420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.161+0000 7f5084a00640 1 --2- 172.21.15.7:0/873467985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f508015b900 0x7f5080150420 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f506800d950 tx=0x7f506800de10 comp rx=0 tx=0).stop 2024-09-10T21:47:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.161+0000 7f5084a00640 1 -- 172.21.15.7:0/873467985 shutdown_connections 2024-09-10T21:47:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.161+0000 7f5084a00640 1 --2- 172.21.15.7:0/873467985 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f505c077fa0 0x7f505c07a460 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:31.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.161+0000 7f5084a00640 1 --2- 172.21.15.7:0/873467985 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f508015bce0 0x7f50801536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.161+0000 7f5084a00640 1 --2- 172.21.15.7:0/873467985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f508015b900 0x7f5080150420 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.161+0000 7f5084a00640 1 --2- 172.21.15.7:0/873467985 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f50800ace80 0x7f50801523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.161+0000 7f5084a00640 1 -- 172.21.15.7:0/873467985 >> 172.21.15.7:0/873467985 conn(0x7f50800b1c10 msgr2=0x7f5080074090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.161+0000 7f5084a00640 1 -- 172.21.15.7:0/873467985 shutdown_connections 2024-09-10T21:47:31.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.161+0000 7f5084a00640 1 -- 172.21.15.7:0/873467985 wait complete. 2024-09-10T21:47:31.179 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:47:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.344+0000 7f06cae00640 1 -- 172.21.15.7:0/4188208146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c415bc20 msgr2=0x7f06c40ad8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.344+0000 7f06cae00640 1 --2- 172.21.15.7:0/4188208146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c415bc20 0x7f06c40ad8f0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009830 tx=0x7f06b402a480 comp rx=0 tx=0).stop 2024-09-10T21:47:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.345+0000 7f06cae00640 1 -- 172.21.15.7:0/4188208146 shutdown_connections 2024-09-10T21:47:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.345+0000 7f06cae00640 1 --2- 172.21.15.7:0/4188208146 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06c40cc520 0x7f06c40cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.345+0000 7f06cae00640 1 --2- 172.21.15.7:0/4188208146 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c415bc20 0x7f06c40ad8f0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.345+0000 7f06cae00640 1 -- 172.21.15.7:0/4188208146 >> 172.21.15.7:0/4188208146 conn(0x7f06c40b1660 msgr2=0x7f06c40b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.345+0000 7f06cae00640 1 -- 172.21.15.7:0/4188208146 shutdown_connections 2024-09-10T21:47:31.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.345+0000 7f06cae00640 1 -- 172.21.15.7:0/4188208146 wait complete. 2024-09-10T21:47:31.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.346+0000 7f06cae00640 1 Processor -- start 2024-09-10T21:47:31.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.346+0000 7f06cae00640 1 -- start start 2024-09-10T21:47:31.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.347+0000 7f06cae00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c40cc520 0x7f06c41f5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:31.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.347+0000 7f06cae00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06c415bc20 0x7f06c414bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:31.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.347+0000 7f06cae00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06c415e470 0x7f06c40c7e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:31.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.347+0000 7f06cae00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f06c406f150 con 0x7f06c40cc520 2024-09-10T21:47:31.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.347+0000 7f06cae00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f06c406efd0 con 0x7f06c415e470 2024-09-10T21:47:31.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.347+0000 7f06cae00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f06c406f2d0 con 0x7f06c415bc20 2024-09-10T21:47:31.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.348+0000 7f06c3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c40cc520 0x7f06c41f5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:31.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.348+0000 7f06c3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c40cc520 0x7f06c41f5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37106/0 (socket says 172.21.15.7:37106) 2024-09-10T21:47:31.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.348+0000 7f06c3e00640 1 -- 172.21.15.7:0/620527233 learned_addr learned my addr 172.21.15.7:0/620527233 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:31.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.348+0000 7f06c3400640 1 --2- 172.21.15.7:0/620527233 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06c415bc20 0x7f06c414bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:31.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.348+0000 7f06c8e00640 1 --2- 172.21.15.7:0/620527233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06c415e470 0x7f06c40c7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:31.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.348+0000 7f06c3e00640 1 -- 172.21.15.7:0/620527233 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06c415bc20 msgr2=0x7f06c414bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:31.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.348+0000 7f06c3e00640 1 --2- 172.21.15.7:0/620527233 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06c415bc20 0x7f06c414bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:31.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.348+0000 7f06c3e00640 1 -- 172.21.15.7:0/620527233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06c415e470 msgr2=0x7f06c40c7e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:31.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.348+0000 7f06c3e00640 1 --2- 172.21.15.7:0/620527233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06c415e470 0x7f06c40c7e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:31.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.348+0000 7f06c3e00640 1 -- 172.21.15.7:0/620527233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c41f5b80 con 0x7f06c40cc520 2024-09-10T21:47:31.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.348+0000 7f06c8e00640 1 --2- 172.21.15.7:0/620527233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06c415e470 0x7f06c40c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:31.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.348+0000 7f06c3e00640 1 --2- 172.21.15.7:0/620527233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c40cc520 0x7f06c41f5640 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009950 tx=0x7f06b4032c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:31.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.349+0000 7f06bbe00640 1 -- 172.21.15.7:0/620527233 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4040070 con 0x7f06c40cc520 2024-09-10T21:47:31.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.349+0000 7f06bbe00640 1 -- 172.21.15.7:0/620527233 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06b4032e70 con 0x7f06c40cc520 2024-09-10T21:47:31.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.349+0000 7f06bbe00640 1 -- 172.21.15.7:0/620527233 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4003d90 con 0x7f06c40cc520 2024-09-10T21:47:31.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.349+0000 7f06cae00640 1 -- 172.21.15.7:0/620527233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c41619f0 con 0x7f06c40cc520 2024-09-10T21:47:31.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.349+0000 7f06cae00640 1 -- 172.21.15.7:0/620527233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c4161fb0 con 0x7f06c40cc520 2024-09-10T21:47:31.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.350+0000 7f06cae00640 1 -- 172.21.15.7:0/620527233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c406f7f0 con 0x7f06c40cc520 2024-09-10T21:47:31.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.351+0000 7f06bbe00640 1 -- 172.21.15.7:0/620527233 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f06b4045460 con 0x7f06c40cc520 2024-09-10T21:47:31.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.353+0000 7f06bbe00640 1 --2- 172.21.15.7:0/620527233 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f06a4077f70 0x7f06a407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:31.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.353+0000 7f06bbe00640 1 -- 172.21.15.7:0/620527233 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5029+0+0 (secure 0 0 0) 0x7f06b40bb3a0 con 0x7f06c40cc520 2024-09-10T21:47:31.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.353+0000 7f06c3400640 1 --2- 172.21.15.7:0/620527233 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f06a4077f70 0x7f06a407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:31.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.354+0000 7f06c3400640 1 --2- 172.21.15.7:0/620527233 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f06a4077f70 0x7f06a407a430 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f06c4074770 tx=0x7f06b0007e90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:31.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.357+0000 7f06bbe00640 1 -- 172.21.15.7:0/620527233 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f06c41f5b80 con 0x7f06c40cc520 2024-09-10T21:47:31.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.558+0000 7f06cae00640 1 -- 172.21.15.7:0/620527233 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f06c40c9d30 con 0x7f06c40cc520 2024-09-10T21:47:31.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.561+0000 7f06bbe00640 1 -- 172.21.15.7:0/620527233 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v35) ==== 71+0+0 (secure 0 0 0) 0x7f06b40840a0 con 0x7f06c40cc520 2024-09-10T21:47:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.564+0000 7f06cae00640 1 -- 172.21.15.7:0/620527233 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f06a4077f70 msgr2=0x7f06a407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.564+0000 7f06cae00640 1 --2- 172.21.15.7:0/620527233 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f06a4077f70 0x7f06a407a430 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f06c4074770 tx=0x7f06b0007e90 comp rx=0 tx=0).stop 2024-09-10T21:47:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.564+0000 7f06cae00640 1 -- 172.21.15.7:0/620527233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c40cc520 msgr2=0x7f06c41f5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:31.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.564+0000 7f06cae00640 1 --2- 172.21.15.7:0/620527233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c40cc520 0x7f06c41f5640 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009950 tx=0x7f06b4032c80 comp rx=0 tx=0).stop 2024-09-10T21:47:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.564+0000 7f06cae00640 1 -- 172.21.15.7:0/620527233 shutdown_connections 2024-09-10T21:47:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.564+0000 7f06cae00640 1 --2- 172.21.15.7:0/620527233 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f06a4077f70 0x7f06a407a430 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.564+0000 7f06cae00640 1 --2- 172.21.15.7:0/620527233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f06c415e470 0x7f06c40c7e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.564+0000 7f06cae00640 1 --2- 172.21.15.7:0/620527233 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f06c415bc20 0x7f06c414bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.564+0000 7f06cae00640 1 --2- 172.21.15.7:0/620527233 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f06c40cc520 0x7f06c41f5640 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.564+0000 7f06cae00640 1 -- 172.21.15.7:0/620527233 >> 172.21.15.7:0/620527233 conn(0x7f06c40b1660 msgr2=0x7f06c415f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.564+0000 7f06cae00640 1 -- 172.21.15.7:0/620527233 shutdown_connections 2024-09-10T21:47:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.564+0000 7f06cae00640 1 -- 172.21.15.7:0/620527233 wait complete. 2024-09-10T21:47:31.581 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair' 2024-09-10T21:47:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.743+0000 7f2f78a00640 1 -- 172.21.15.7:0/2887918215 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f7415af40 msgr2=0x7f2f74164960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.743+0000 7f2f78a00640 1 --2- 172.21.15.7:0/2887918215 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f7415af40 0x7f2f74164960 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c009be0 tx=0x7f2f6c02a8b0 comp rx=0 tx=0).stop 2024-09-10T21:47:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.743+0000 7f2f78a00640 1 -- 172.21.15.7:0/2887918215 shutdown_connections 2024-09-10T21:47:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.743+0000 7f2f78a00640 1 --2- 172.21.15.7:0/2887918215 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f7415af40 0x7f2f74164960 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.743+0000 7f2f78a00640 1 -- 172.21.15.7:0/2887918215 >> 172.21.15.7:0/2887918215 conn(0x7f2f740b1d70 msgr2=0x7f2f740cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.743+0000 7f2f78a00640 1 -- 172.21.15.7:0/2887918215 shutdown_connections 2024-09-10T21:47:31.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.743+0000 7f2f78a00640 1 -- 172.21.15.7:0/2887918215 wait complete. 2024-09-10T21:47:31.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.744+0000 7f2f78a00640 1 Processor -- start 2024-09-10T21:47:31.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.745+0000 7f2f78a00640 1 -- start start 2024-09-10T21:47:31.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.745+0000 7f2f78a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f740704a0 0x7f2f741f7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:31.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.745+0000 7f2f78a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f7415a400 0x7f2f740cda20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:31.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.746+0000 7f2f78a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f7415af40 0x7f2f741f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:31.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.746+0000 7f2f78a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2f74164ea0 con 0x7f2f7415af40 2024-09-10T21:47:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.746+0000 7f2f78a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2f740b3250 con 0x7f2f7415a400 2024-09-10T21:47:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.746+0000 7f2f78a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f2f74165020 con 0x7f2f740704a0 2024-09-10T21:47:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.746+0000 7f2f73e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f7415af40 0x7f2f741f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.746+0000 7f2f73e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f7415af40 0x7f2f741f5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37108/0 (socket says 172.21.15.7:37108) 2024-09-10T21:47:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.746+0000 7f2f73e00640 1 -- 172.21.15.7:0/3735560122 learned_addr learned my addr 172.21.15.7:0/3735560122 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.746+0000 7f2f73400640 1 --2- 172.21.15.7:0/3735560122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f740704a0 0x7f2f741f7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.746+0000 7f2f6aa00640 1 --2- 172.21.15.7:0/3735560122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f7415a400 0x7f2f740cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.746+0000 7f2f73e00640 1 -- 172.21.15.7:0/3735560122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f740704a0 msgr2=0x7f2f741f7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.746+0000 7f2f73e00640 1 --2- 172.21.15.7:0/3735560122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f740704a0 0x7f2f741f7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.746+0000 7f2f73e00640 1 -- 172.21.15.7:0/3735560122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f7415a400 msgr2=0x7f2f740cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.746+0000 7f2f73e00640 1 --2- 172.21.15.7:0/3735560122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f7415a400 0x7f2f740cda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:31.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.746+0000 7f2f73e00640 1 -- 172.21.15.7:0/3735560122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f741f8390 con 0x7f2f7415af40 2024-09-10T21:47:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.746+0000 7f2f73400640 1 --2- 172.21.15.7:0/3735560122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f740704a0 0x7f2f741f7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:47:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.746+0000 7f2f6aa00640 1 --2- 172.21.15.7:0/3735560122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f7415a400 0x7f2f740cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:47:31.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.747+0000 7f2f73e00640 1 --2- 172.21.15.7:0/3735560122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f7415af40 0x7f2f741f5180 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c035040 tx=0x7f2f6c032a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:31.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.747+0000 7f2f6be00640 1 -- 172.21.15.7:0/3735560122 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c041070 con 0x7f2f7415af40 2024-09-10T21:47:31.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.747+0000 7f2f6be00640 1 -- 172.21.15.7:0/3735560122 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f6c03c750 con 0x7f2f7415af40 2024-09-10T21:47:31.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.747+0000 7f2f6be00640 1 -- 172.21.15.7:0/3735560122 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c037040 con 0x7f2f7415af40 2024-09-10T21:47:31.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.747+0000 7f2f78a00640 1 -- 172.21.15.7:0/3735560122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f741f8670 con 0x7f2f7415af40 2024-09-10T21:47:31.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.747+0000 7f2f78a00640 1 -- 172.21.15.7:0/3735560122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f740726b0 con 0x7f2f7415af40 2024-09-10T21:47:31.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.748+0000 7f2f78a00640 1 -- 172.21.15.7:0/3735560122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f741f8b90 con 0x7f2f7415af40 2024-09-10T21:47:31.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.749+0000 7f2f6be00640 1 -- 172.21.15.7:0/3735560122 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2f6c03cb80 con 0x7f2f7415af40 2024-09-10T21:47:31.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.750+0000 7f2f6be00640 1 --2- 172.21.15.7:0/3735560122 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f50077f70 0x7f2f5007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:31.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.750+0000 7f2f6be00640 1 -- 172.21.15.7:0/3735560122 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5029+0+0 (secure 0 0 0) 0x7f2f6c0ba6d0 con 0x7f2f7415af40 2024-09-10T21:47:31.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.751+0000 7f2f73400640 1 --2- 172.21.15.7:0/3735560122 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f50077f70 0x7f2f5007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:31.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.751+0000 7f2f73400640 1 --2- 172.21.15.7:0/3735560122 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f50077f70 0x7f2f5007a430 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2f64000f60 tx=0x7f2f64009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:31.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:31.754+0000 7f2f6be00640 1 -- 172.21.15.7:0/3735560122 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2f741f8b90 con 0x7f2f7415af40 2024-09-10T21:47:32.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.021+0000 7f2f78a00640 1 -- 172.21.15.7:0/3735560122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]} v 0) -- 0x7f2f7406e250 con 0x7f2f7415af40 2024-09-10T21:47:32.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.160+0000 7f2f6be00640 1 -- 172.21.15.7:0/3735560122 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dentry_repair"]}]=0 v265) ==== 166+0+0 (secure 0 0 0) 0x7f2f6c082d80 con 0x7f2f7415af40 2024-09-10T21:47:32.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.162+0000 7f2f78a00640 1 -- 172.21.15.7:0/3735560122 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f50077f70 msgr2=0x7f2f5007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:32.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.162+0000 7f2f78a00640 1 --2- 172.21.15.7:0/3735560122 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f50077f70 0x7f2f5007a430 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f2f64000f60 tx=0x7f2f64009040 comp rx=0 tx=0).stop 2024-09-10T21:47:32.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.162+0000 7f2f78a00640 1 -- 172.21.15.7:0/3735560122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f7415af40 msgr2=0x7f2f741f5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:32.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.162+0000 7f2f78a00640 1 --2- 172.21.15.7:0/3735560122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f7415af40 0x7f2f741f5180 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c035040 tx=0x7f2f6c032a30 comp rx=0 tx=0).stop 2024-09-10T21:47:32.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.163+0000 7f2f78a00640 1 -- 172.21.15.7:0/3735560122 shutdown_connections 2024-09-10T21:47:32.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.163+0000 7f2f78a00640 1 --2- 172.21.15.7:0/3735560122 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f50077f70 0x7f2f5007a430 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.163+0000 7f2f78a00640 1 --2- 172.21.15.7:0/3735560122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f7415a400 0x7f2f740cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.163+0000 7f2f78a00640 1 --2- 172.21.15.7:0/3735560122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f7415af40 0x7f2f741f5180 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.163+0000 7f2f78a00640 1 --2- 172.21.15.7:0/3735560122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f740704a0 0x7f2f741f7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.163+0000 7f2f78a00640 1 -- 172.21.15.7:0/3735560122 >> 172.21.15.7:0/3735560122 conn(0x7f2f740b1d70 msgr2=0x7f2f740b01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:32.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.163+0000 7f2f78a00640 1 -- 172.21.15.7:0/3735560122 shutdown_connections 2024-09-10T21:47:32.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.163+0000 7f2f78a00640 1 -- 172.21.15.7:0/3735560122 wait complete. 2024-09-10T21:47:32.179 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-09-10T21:47:32.180 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2024-09-10T21:47:32.180 INFO:tasks.cephfs_test_runner:test_health_status_after_dirfrag_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-10T21:47:32.180 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:47:32.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.339+0000 7f947da00640 1 -- 172.21.15.7:0/1753379265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94780c82d0 msgr2=0x7f94780c8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:32.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.339+0000 7f947da00640 1 --2- 172.21.15.7:0/1753379265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94780c82d0 0x7f94780c8780 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f94780d1e10 tx=0x7f947002aac0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.340+0000 7f947da00640 1 -- 172.21.15.7:0/1753379265 shutdown_connections 2024-09-10T21:47:32.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.340+0000 7f947da00640 1 --2- 172.21.15.7:0/1753379265 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94780c82d0 0x7f94780c8780 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.340+0000 7f947da00640 1 --2- 172.21.15.7:0/1753379265 >> v2:172.21.15.133:3300/0 conn(0x7f94780d1e60 0x7f94780d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.340+0000 7f947da00640 1 -- 172.21.15.7:0/1753379265 >> 172.21.15.7:0/1753379265 conn(0x7f94780d3de0 msgr2=0x7f94780d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:32.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.340+0000 7f947da00640 1 -- 172.21.15.7:0/1753379265 shutdown_connections 2024-09-10T21:47:32.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.340+0000 7f947da00640 1 -- 172.21.15.7:0/1753379265 wait complete. 2024-09-10T21:47:32.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.341+0000 7f947da00640 1 Processor -- start 2024-09-10T21:47:32.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.341+0000 7f947da00640 1 -- start start 2024-09-10T21:47:32.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.342+0000 7f947da00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f94780c82d0 0x7f947815ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:32.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.342+0000 7f947da00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94780d1e60 0x7f9478158c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:32.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.342+0000 7f947da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94780d2a20 0x7f947814bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:32.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.342+0000 7f947da00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f94780c8e40 con 0x7f94780d2a20 2024-09-10T21:47:32.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.342+0000 7f947da00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f94780c8cc0 con 0x7f94780d1e60 2024-09-10T21:47:32.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.342+0000 7f947da00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f94780c8fc0 con 0x7f94780c82d0 2024-09-10T21:47:32.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.343+0000 7f947d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94780d2a20 0x7f947814bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:32.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.343+0000 7f947d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94780d2a20 0x7f947814bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37130/0 (socket says 172.21.15.7:37130) 2024-09-10T21:47:32.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.343+0000 7f947d000640 1 -- 172.21.15.7:0/319453235 learned_addr learned my addr 172.21.15.7:0/319453235 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:32.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.343+0000 7f9477e00640 1 --2- 172.21.15.7:0/319453235 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f94780c82d0 0x7f947815ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:32.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.343+0000 7f9477400640 1 --2- 172.21.15.7:0/319453235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94780d1e60 0x7f9478158c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:32.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.343+0000 7f947d000640 1 -- 172.21.15.7:0/319453235 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f94780c82d0 msgr2=0x7f947815ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:32.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.343+0000 7f947d000640 1 --2- 172.21.15.7:0/319453235 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f94780c82d0 0x7f947815ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.343+0000 7f947d000640 1 -- 172.21.15.7:0/319453235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94780d1e60 msgr2=0x7f9478158c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:32.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.343+0000 7f947d000640 1 --2- 172.21.15.7:0/319453235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94780d1e60 0x7f9478158c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.343+0000 7f947d000640 1 -- 172.21.15.7:0/319453235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947815bf70 con 0x7f94780d2a20 2024-09-10T21:47:32.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.343+0000 7f947d000640 1 --2- 172.21.15.7:0/319453235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94780d2a20 0x7f947814bef0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f947002b0b0 tx=0x7f9470003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:32.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.344+0000 7f946fe00640 1 -- 172.21.15.7:0/319453235 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9470044070 con 0x7f94780d2a20 2024-09-10T21:47:32.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.344+0000 7f946fe00640 1 -- 172.21.15.7:0/319453235 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9470004250 con 0x7f94780d2a20 2024-09-10T21:47:32.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.344+0000 7f946fe00640 1 -- 172.21.15.7:0/319453235 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94700046a0 con 0x7f94780d2a20 2024-09-10T21:47:32.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.344+0000 7f947da00640 1 -- 172.21.15.7:0/319453235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947815f340 con 0x7f94780d2a20 2024-09-10T21:47:32.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.344+0000 7f947da00640 1 -- 172.21.15.7:0/319453235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f947815b9b0 con 0x7f94780d2a20 2024-09-10T21:47:32.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.345+0000 7f947da00640 1 -- 172.21.15.7:0/319453235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94780751b0 con 0x7f94780d2a20 2024-09-10T21:47:32.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.346+0000 7f946fe00640 1 -- 172.21.15.7:0/319453235 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f947003f410 con 0x7f94780d2a20 2024-09-10T21:47:32.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.347+0000 7f946fe00640 1 --2- 172.21.15.7:0/319453235 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9458077f70 0x7f945807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:32.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.347+0000 7f946fe00640 1 -- 172.21.15.7:0/319453235 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5029+0+0 (secure 0 0 0) 0x7f94700ba1c0 con 0x7f94780d2a20 2024-09-10T21:47:32.348 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.347+0000 7f9477e00640 1 --2- 172.21.15.7:0/319453235 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9458077f70 0x7f945807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:32.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.348+0000 7f9477e00640 1 --2- 172.21.15.7:0/319453235 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9458077f70 0x7f945807a430 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9468005fd0 tx=0x7f946800b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:32.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.351+0000 7f946fe00640 1 -- 172.21.15.7:0/319453235 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9470032030 con 0x7f94780d2a20 2024-09-10T21:47:32.547 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.546+0000 7f947da00640 1 -- 172.21.15.7:0/319453235 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94780698f0 con 0x7f94780d2a20 2024-09-10T21:47:32.548 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.547+0000 7f946fe00640 1 -- 172.21.15.7:0/319453235 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+13131 (secure 0 0 0) 0x7f9470082900 con 0x7f94780d2a20 2024-09-10T21:47:32.549 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:32.549 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":76,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:47:31.153273+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6836/2622530239":"2024-09-11T21:47:27.803823+0000","172.21.15.133:6834/2622530239":"2024-09-11T21:47:27.803823+0000","172.21.15.133:6836/2752881644":"2024-09-11T21:46:10.658995+0000","172.21.15.133:6834/2752881644":"2024-09-11T21:46:10.658995+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:47:32.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.549+0000 7f947da00640 1 -- 172.21.15.7:0/319453235 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9458077f70 msgr2=0x7f945807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:32.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.549+0000 7f947da00640 1 --2- 172.21.15.7:0/319453235 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9458077f70 0x7f945807a430 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f9468005fd0 tx=0x7f946800b390 comp rx=0 tx=0).stop 2024-09-10T21:47:32.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.549+0000 7f947da00640 1 -- 172.21.15.7:0/319453235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94780d2a20 msgr2=0x7f947814bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:32.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.549+0000 7f947da00640 1 --2- 172.21.15.7:0/319453235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94780d2a20 0x7f947814bef0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f947002b0b0 tx=0x7f9470003ce0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.549+0000 7f947da00640 1 -- 172.21.15.7:0/319453235 shutdown_connections 2024-09-10T21:47:32.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.549+0000 7f947da00640 1 --2- 172.21.15.7:0/319453235 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9458077f70 0x7f945807a430 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.549+0000 7f947da00640 1 --2- 172.21.15.7:0/319453235 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f94780d1e60 0x7f9478158c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.549+0000 7f947da00640 1 --2- 172.21.15.7:0/319453235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94780d2a20 0x7f947814bef0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.549+0000 7f947da00640 1 --2- 172.21.15.7:0/319453235 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f94780c82d0 0x7f947815ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.549+0000 7f947da00640 1 -- 172.21.15.7:0/319453235 >> 172.21.15.7:0/319453235 conn(0x7f94780d3de0 msgr2=0x7f94780d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:32.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.549+0000 7f947da00640 1 -- 172.21.15.7:0/319453235 shutdown_connections 2024-09-10T21:47:32.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.549+0000 7f947da00640 1 -- 172.21.15.7:0/319453235 wait complete. 2024-09-10T21:47:32.566 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:41:55.889618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:47:32.566 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:47:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.729+0000 7f7022c00640 1 -- 172.21.15.7:0/3964922615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7000003a00 msgr2=0x7f7000003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:32.730 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.729+0000 7f7022c00640 1 --2- 172.21.15.7:0/3964922615 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7000003a00 0x7f7000003e20 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f700c009b50 tx=0x7f7000005830 comp rx=0 tx=0).stop 2024-09-10T21:47:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.729+0000 7f7020c00640 1 -- 172.21.15.7:0/3964922615 reap_dead start 2024-09-10T21:47:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.729+0000 7f7022c00640 1 -- 172.21.15.7:0/3964922615 shutdown_connections 2024-09-10T21:47:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.729+0000 7f7022c00640 1 -- 172.21.15.7:0/3964922615 >> 172.21.15.7:0/3964922615 conn(0x7f701c0b16d0 msgr2=0x7f701c0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.729+0000 7f7022c00640 1 -- 172.21.15.7:0/3964922615 shutdown_connections 2024-09-10T21:47:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.729+0000 7f7022c00640 1 -- 172.21.15.7:0/3964922615 wait complete. 2024-09-10T21:47:32.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.731+0000 7f7022c00640 1 Processor -- start 2024-09-10T21:47:32.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.731+0000 7f7022c00640 1 -- start start 2024-09-10T21:47:32.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.732+0000 7f7022c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7000003a00 0x7f701c157ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:32.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.732+0000 7f7022c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f701c159a40 0x7f701c157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:32.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.732+0000 7f7022c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f701c157650 0x7f701c151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.732+0000 7f7022c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f701c160310 con 0x7f701c159a40 2024-09-10T21:47:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.732+0000 7f7022c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f701c0747d0 con 0x7f701c157650 2024-09-10T21:47:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.732+0000 7f7022c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f701c160490 con 0x7f7000003a00 2024-09-10T21:47:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.732+0000 7f7013400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f701c159a40 0x7f701c157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.732+0000 7f7013400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f701c159a40 0x7f701c157110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37156/0 (socket says 172.21.15.7:37156) 2024-09-10T21:47:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.732+0000 7f7013400640 1 -- 172.21.15.7:0/1543252012 learned_addr learned my addr 172.21.15.7:0/1543252012 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.732+0000 7f7020c00640 1 --2- 172.21.15.7:0/1543252012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f701c157650 0x7f701c151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.732+0000 7f701be00640 1 --2- 172.21.15.7:0/1543252012 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7000003a00 0x7f701c157ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.732+0000 7f7013400640 1 -- 172.21.15.7:0/1543252012 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7000003a00 msgr2=0x7f701c157ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.732+0000 7f7013400640 1 --2- 172.21.15.7:0/1543252012 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7000003a00 0x7f701c157ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.732+0000 7f7013400640 1 -- 172.21.15.7:0/1543252012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f701c157650 msgr2=0x7f701c151bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.732+0000 7f7013400640 1 --2- 172.21.15.7:0/1543252012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f701c157650 0x7f701c151bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.732+0000 7f7013400640 1 -- 172.21.15.7:0/1543252012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c0c8e40 con 0x7f701c159a40 2024-09-10T21:47:32.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.732+0000 7f7020c00640 1 --2- 172.21.15.7:0/1543252012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f701c157650 0x7f701c151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:32.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.732+0000 7f701be00640 1 --2- 172.21.15.7:0/1543252012 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7000003a00 0x7f701c157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:32.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.733+0000 7f7013400640 1 --2- 172.21.15.7:0/1543252012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f701c159a40 0x7f701c157110 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f700400e620 tx=0x7f700400eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:32.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.733+0000 7f7018c00640 1 -- 172.21.15.7:0/1543252012 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004015020 con 0x7f701c159a40 2024-09-10T21:47:32.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.733+0000 7f7018c00640 1 -- 172.21.15.7:0/1543252012 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f70040035f0 con 0x7f701c159a40 2024-09-10T21:47:32.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.733+0000 7f7022c00640 1 -- 172.21.15.7:0/1543252012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701c0c6c90 con 0x7f701c159a40 2024-09-10T21:47:32.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.733+0000 7f7018c00640 1 -- 172.21.15.7:0/1543252012 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7004003a40 con 0x7f701c159a40 2024-09-10T21:47:32.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.733+0000 7f7022c00640 1 -- 172.21.15.7:0/1543252012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f701c0c9120 con 0x7f701c159a40 2024-09-10T21:47:32.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.734+0000 7f7022c00640 1 -- 172.21.15.7:0/1543252012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f701c0ca270 con 0x7f701c159a40 2024-09-10T21:47:32.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.735+0000 7f7018c00640 1 -- 172.21.15.7:0/1543252012 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7004003be0 con 0x7f701c159a40 2024-09-10T21:47:32.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.736+0000 7f7018c00640 1 --2- 172.21.15.7:0/1543252012 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6ff8077f70 0x7f6ff807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:32.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.737+0000 7f7018c00640 1 -- 172.21.15.7:0/1543252012 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5029+0+0 (secure 0 0 0) 0x7f700409a200 con 0x7f701c159a40 2024-09-10T21:47:32.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.737+0000 7f701be00640 1 --2- 172.21.15.7:0/1543252012 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6ff8077f70 0x7f6ff807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:32.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.738+0000 7f701be00640 1 --2- 172.21.15.7:0/1543252012 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6ff8077f70 0x7f6ff807a430 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f700c035000 tx=0x7f700c02d040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:32.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.740+0000 7f7018c00640 1 -- 172.21.15.7:0/1543252012 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f700401f030 con 0x7f701c159a40 2024-09-10T21:47:32.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.937+0000 7f7022c00640 1 -- 172.21.15.7:0/1543252012 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f701c0c7020 con 0x7f701c159a40 2024-09-10T21:47:32.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.937+0000 7f7018c00640 1 -- 172.21.15.7:0/1543252012 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7f7004062f90 con 0x7f701c159a40 2024-09-10T21:47:32.938 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-10T21:47:32.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.939+0000 7f7022c00640 1 -- 172.21.15.7:0/1543252012 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6ff8077f70 msgr2=0x7f6ff807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:32.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.939+0000 7f7022c00640 1 --2- 172.21.15.7:0/1543252012 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6ff8077f70 0x7f6ff807a430 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f700c035000 tx=0x7f700c02d040 comp rx=0 tx=0).stop 2024-09-10T21:47:32.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.940+0000 7f7022c00640 1 -- 172.21.15.7:0/1543252012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f701c159a40 msgr2=0x7f701c157110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:32.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.940+0000 7f7022c00640 1 --2- 172.21.15.7:0/1543252012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f701c159a40 0x7f701c157110 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f700400e620 tx=0x7f700400eae0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.940+0000 7f7022c00640 1 -- 172.21.15.7:0/1543252012 shutdown_connections 2024-09-10T21:47:32.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.940+0000 7f7022c00640 1 --2- 172.21.15.7:0/1543252012 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6ff8077f70 0x7f6ff807a430 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.940+0000 7f7022c00640 1 --2- 172.21.15.7:0/1543252012 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f701c157650 0x7f701c151bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.940+0000 7f7022c00640 1 --2- 172.21.15.7:0/1543252012 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f701c159a40 0x7f701c157110 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.940+0000 7f7022c00640 1 --2- 172.21.15.7:0/1543252012 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7000003a00 0x7f701c157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:32.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.940+0000 7f7022c00640 1 -- 172.21.15.7:0/1543252012 >> 172.21.15.7:0/1543252012 conn(0x7f701c0b16d0 msgr2=0x7f701c0d0340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:32.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.940+0000 7f7022c00640 1 -- 172.21.15.7:0/1543252012 shutdown_connections 2024-09-10T21:47:32.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:32.940+0000 7f7022c00640 1 -- 172.21.15.7:0/1543252012 wait complete. 2024-09-10T21:47:32.956 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-09-10T21:47:33.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.116+0000 7f71b0c00640 1 -- 172.21.15.7:0/2677508769 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71ac0cc1c0 msgr2=0x7f71ac0cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:33.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.116+0000 7f71b0c00640 1 --2- 172.21.15.7:0/2677508769 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71ac0cc1c0 0x7f71ac0cd750 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f71a4005fd0 tx=0x7f71a402a840 comp rx=0 tx=0).stop 2024-09-10T21:47:33.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.116+0000 7f71b0c00640 1 -- 172.21.15.7:0/2677508769 shutdown_connections 2024-09-10T21:47:33.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.116+0000 7f71b0c00640 1 --2- 172.21.15.7:0/2677508769 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71ac0cc1c0 0x7f71ac0cd750 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:33.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.116+0000 7f71b0c00640 1 --2- 172.21.15.7:0/2677508769 >> v2:172.21.15.133:3300/0 conn(0x7f71ac1619e0 0x7f71ac15e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:33.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.116+0000 7f71b0c00640 1 -- 172.21.15.7:0/2677508769 >> 172.21.15.7:0/2677508769 conn(0x7f71ac0b16d0 msgr2=0x7f71ac0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:33.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.116+0000 7f71b0c00640 1 -- 172.21.15.7:0/2677508769 shutdown_connections 2024-09-10T21:47:33.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.116+0000 7f71b0c00640 1 -- 172.21.15.7:0/2677508769 wait complete. 2024-09-10T21:47:33.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.118+0000 7f71b0c00640 1 Processor -- start 2024-09-10T21:47:33.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.118+0000 7f71b0c00640 1 -- start start 2024-09-10T21:47:33.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.119+0000 7f71b0c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71ac0cbd50 0x7f71ac0b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:33.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.119+0000 7f71b0c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71ac0cc1c0 0x7f71ac1577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:33.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.119+0000 7f71b0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac1619e0 0x7f71ac073610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:33.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.119+0000 7f71b0c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac0ce050 con 0x7f71ac1619e0 2024-09-10T21:47:33.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.119+0000 7f71b0c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f71ac0cded0 con 0x7f71ac0cbd50 2024-09-10T21:47:33.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.119+0000 7f71b0c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f71ac161dc0 con 0x7f71ac0cc1c0 2024-09-10T21:47:33.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.119+0000 7f71aa800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac1619e0 0x7f71ac073610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:33.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.119+0000 7f71aa800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac1619e0 0x7f71ac073610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37164/0 (socket says 172.21.15.7:37164) 2024-09-10T21:47:33.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.119+0000 7f71aa800640 1 -- 172.21.15.7:0/4274752263 learned_addr learned my addr 172.21.15.7:0/4274752263 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:33.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.119+0000 7f71a9e00640 1 --2- 172.21.15.7:0/4274752263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71ac0cbd50 0x7f71ac0b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:33.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.119+0000 7f71a1400640 1 --2- 172.21.15.7:0/4274752263 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71ac0cc1c0 0x7f71ac1577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:33.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.119+0000 7f71aa800640 1 -- 172.21.15.7:0/4274752263 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71ac0cc1c0 msgr2=0x7f71ac1577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:33.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.119+0000 7f71aa800640 1 --2- 172.21.15.7:0/4274752263 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71ac0cc1c0 0x7f71ac1577b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:33.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.119+0000 7f71aa800640 1 -- 172.21.15.7:0/4274752263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71ac0cbd50 msgr2=0x7f71ac0b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:33.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.119+0000 7f71aa800640 1 --2- 172.21.15.7:0/4274752263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71ac0cbd50 0x7f71ac0b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:33.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.119+0000 7f71aa800640 1 -- 172.21.15.7:0/4274752263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71ac0b4e70 con 0x7f71ac1619e0 2024-09-10T21:47:33.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.120+0000 7f71aa800640 1 --2- 172.21.15.7:0/4274752263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac1619e0 0x7f71ac073610 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f71a4005fd0 tx=0x7f71a4032c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.120+0000 7f71a2a00640 1 -- 172.21.15.7:0/4274752263 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4040070 con 0x7f71ac1619e0 2024-09-10T21:47:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.120+0000 7f71a2a00640 1 -- 172.21.15.7:0/4274752263 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71a4004560 con 0x7f71ac1619e0 2024-09-10T21:47:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.120+0000 7f71a2a00640 1 -- 172.21.15.7:0/4274752263 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a4003e60 con 0x7f71ac1619e0 2024-09-10T21:47:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.120+0000 7f71b0c00640 1 -- 172.21.15.7:0/4274752263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71ac1f56e0 con 0x7f71ac1619e0 2024-09-10T21:47:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.120+0000 7f71b0c00640 1 -- 172.21.15.7:0/4274752263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71ac0b2ec0 con 0x7f71ac1619e0 2024-09-10T21:47:33.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.121+0000 7f71b0c00640 1 -- 172.21.15.7:0/4274752263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71ac0b29d0 con 0x7f71ac1619e0 2024-09-10T21:47:33.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.123+0000 7f71a2a00640 1 -- 172.21.15.7:0/4274752263 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f71a403c420 con 0x7f71ac1619e0 2024-09-10T21:47:33.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.124+0000 7f71a2a00640 1 --2- 172.21.15.7:0/4274752263 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7188077f70 0x7f718807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:33.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.124+0000 7f71a2a00640 1 -- 172.21.15.7:0/4274752263 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5029+0+0 (secure 0 0 0) 0x7f71a40bade0 con 0x7f71ac1619e0 2024-09-10T21:47:33.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.124+0000 7f71a9e00640 1 --2- 172.21.15.7:0/4274752263 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7188077f70 0x7f718807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:33.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.125+0000 7f71a9e00640 1 --2- 172.21.15.7:0/4274752263 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7188077f70 0x7f718807a430 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f719c009b60 tx=0x7f719c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:33.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.131+0000 7f71a2a00640 1 -- 172.21.15.7:0/4274752263 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f71a4083ba0 con 0x7f71ac1619e0 2024-09-10T21:47:33.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:33.401+0000 7f71b0c00640 1 -- 172.21.15.7:0/4274752263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f71ac0b5b00 con 0x7f71ac1619e0 2024-09-10T21:47:34.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.168+0000 7f71a2a00640 1 -- 172.21.15.7:0/4274752263 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v267) ==== 170+0+0 (secure 0 0 0) 0x7f71a408c030 con 0x7f71ac1619e0 2024-09-10T21:47:34.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.171+0000 7f71b0c00640 1 -- 172.21.15.7:0/4274752263 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7188077f70 msgr2=0x7f718807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:34.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.171+0000 7f71b0c00640 1 --2- 172.21.15.7:0/4274752263 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7188077f70 0x7f718807a430 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f719c009b60 tx=0x7f719c005f90 comp rx=0 tx=0).stop 2024-09-10T21:47:34.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.171+0000 7f71b0c00640 1 -- 172.21.15.7:0/4274752263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac1619e0 msgr2=0x7f71ac073610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:34.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.171+0000 7f71b0c00640 1 --2- 172.21.15.7:0/4274752263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac1619e0 0x7f71ac073610 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f71a4005fd0 tx=0x7f71a4032c40 comp rx=0 tx=0).stop 2024-09-10T21:47:34.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.171+0000 7f71b0c00640 1 -- 172.21.15.7:0/4274752263 shutdown_connections 2024-09-10T21:47:34.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.171+0000 7f71b0c00640 1 --2- 172.21.15.7:0/4274752263 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7188077f70 0x7f718807a430 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:34.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.171+0000 7f71b0c00640 1 --2- 172.21.15.7:0/4274752263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71ac1619e0 0x7f71ac073610 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:34.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.171+0000 7f71b0c00640 1 --2- 172.21.15.7:0/4274752263 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71ac0cc1c0 0x7f71ac1577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:34.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.171+0000 7f71b0c00640 1 --2- 172.21.15.7:0/4274752263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71ac0cbd50 0x7f71ac0b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:34.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.171+0000 7f71b0c00640 1 -- 172.21.15.7:0/4274752263 >> 172.21.15.7:0/4274752263 conn(0x7f71ac0b16d0 msgr2=0x7f71ac0b01f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:34.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.171+0000 7f71b0c00640 1 -- 172.21.15.7:0/4274752263 shutdown_connections 2024-09-10T21:47:34.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.171+0000 7f71b0c00640 1 -- 172.21.15.7:0/4274752263 wait complete. 2024-09-10T21:47:34.191 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:47:34.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.356+0000 7fce96c00640 1 -- 172.21.15.7:0/170436671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce78003aa0 msgr2=0x7fce78003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:34.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.356+0000 7fce96c00640 1 --2- 172.21.15.7:0/170436671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce78003aa0 0x7fce78003ec0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fce84001690 tx=0x7fce78005830 comp rx=0 tx=0).stop 2024-09-10T21:47:34.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.356+0000 7fce96c00640 1 -- 172.21.15.7:0/170436671 shutdown_connections 2024-09-10T21:47:34.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.356+0000 7fce96c00640 1 --2- 172.21.15.7:0/170436671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce78003aa0 0x7fce78003ec0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:34.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.356+0000 7fce96c00640 1 -- 172.21.15.7:0/170436671 >> 172.21.15.7:0/170436671 conn(0x7fce900d3de0 msgr2=0x7fce900d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:34.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.356+0000 7fce96c00640 1 -- 172.21.15.7:0/170436671 shutdown_connections 2024-09-10T21:47:34.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.356+0000 7fce96c00640 1 -- 172.21.15.7:0/170436671 wait complete. 2024-09-10T21:47:34.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.357+0000 7fce96c00640 1 Processor -- start 2024-09-10T21:47:34.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.358+0000 7fce96c00640 1 -- start start 2024-09-10T21:47:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.358+0000 7fce96c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce78003aa0 0x7fce900cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.358+0000 7fce96c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9014fc80 0x7fce90152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.358+0000 7fce96c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce9014ef10 0x7fce9014f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.359+0000 7fce96c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fce900d23c0 con 0x7fce9014fc80 2024-09-10T21:47:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.359+0000 7fce96c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fce900d2240 con 0x7fce9014ef10 2024-09-10T21:47:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.359+0000 7fce96c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fce900d2540 con 0x7fce78003aa0 2024-09-10T21:47:34.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.359+0000 7fce94e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9014fc80 0x7fce90152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.359+0000 7fce94e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9014fc80 0x7fce90152680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53988/0 (socket says 172.21.15.7:53988) 2024-09-10T21:47:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.359+0000 7fce94e00640 1 -- 172.21.15.7:0/2466923259 learned_addr learned my addr 172.21.15.7:0/2466923259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.359+0000 7fce95800640 1 --2- 172.21.15.7:0/2466923259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce78003aa0 0x7fce900cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.359+0000 7fce96200640 1 --2- 172.21.15.7:0/2466923259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce9014ef10 0x7fce9014f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.359+0000 7fce94e00640 1 -- 172.21.15.7:0/2466923259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce78003aa0 msgr2=0x7fce900cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.359+0000 7fce94e00640 1 --2- 172.21.15.7:0/2466923259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce78003aa0 0x7fce900cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.359+0000 7fce94e00640 1 -- 172.21.15.7:0/2466923259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce9014ef10 msgr2=0x7fce9014f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.359+0000 7fce94e00640 1 --2- 172.21.15.7:0/2466923259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce9014ef10 0x7fce9014f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.359+0000 7fce94e00640 1 -- 172.21.15.7:0/2466923259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce9015feb0 con 0x7fce9014fc80 2024-09-10T21:47:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.360+0000 7fce94e00640 1 --2- 172.21.15.7:0/2466923259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9014fc80 0x7fce90152680 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00e590 tx=0x7fce7c00ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.360+0000 7fce8d600640 1 -- 172.21.15.7:0/2466923259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c015070 con 0x7fce9014fc80 2024-09-10T21:47:34.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.360+0000 7fce8d600640 1 -- 172.21.15.7:0/2466923259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce7c010a60 con 0x7fce9014fc80 2024-09-10T21:47:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.360+0000 7fce96c00640 1 -- 172.21.15.7:0/2466923259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce90160150 con 0x7fce9014fc80 2024-09-10T21:47:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.360+0000 7fce8d600640 1 -- 172.21.15.7:0/2466923259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7c011460 con 0x7fce9014fc80 2024-09-10T21:47:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.360+0000 7fce96c00640 1 -- 172.21.15.7:0/2466923259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce90160710 con 0x7fce9014fc80 2024-09-10T21:47:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.361+0000 7fce96c00640 1 -- 172.21.15.7:0/2466923259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce900751b0 con 0x7fce9014fc80 2024-09-10T21:47:34.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.362+0000 7fce8d600640 1 -- 172.21.15.7:0/2466923259 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fce7c011600 con 0x7fce9014fc80 2024-09-10T21:47:34.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.363+0000 7fce8d600640 1 --2- 172.21.15.7:0/2466923259 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fce70077fa0 0x7fce7007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:34.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.363+0000 7fce8d600640 1 -- 172.21.15.7:0/2466923259 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5029+0+0 (secure 0 0 0) 0x7fce7c09a0c0 con 0x7fce9014fc80 2024-09-10T21:47:34.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.364+0000 7fce95800640 1 --2- 172.21.15.7:0/2466923259 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fce70077fa0 0x7fce7007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:34.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.364+0000 7fce95800640 1 --2- 172.21.15.7:0/2466923259 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fce70077fa0 0x7fce7007a460 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fce84032000 tx=0x7fce8402c040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:34.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.367+0000 7fce8d600640 1 -- 172.21.15.7:0/2466923259 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fce7c062770 con 0x7fce9014fc80 2024-09-10T21:47:34.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.563+0000 7fce96c00640 1 -- 172.21.15.7:0/2466923259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fce900d1120 con 0x7fce9014fc80 2024-09-10T21:47:34.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.563+0000 7fce8d600640 1 -- 172.21.15.7:0/2466923259 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v35) ==== 59+0+1998 (secure 0 0 0) 0x7fce7c0671a0 con 0x7fce9014fc80 2024-09-10T21:47:34.564 INFO:teuthology.orchestra.run.smithi007.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:47:34.564 INFO:teuthology.orchestra.run.smithi007.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:47:34.564 INFO:teuthology.orchestra.run.smithi007.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:47:34.565 INFO:teuthology.orchestra.run.smithi007.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:47:34.565 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3770.177547 2024-09-10T21:47:34.565 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3542.062395 2024-09-10T21:47:34.565 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3788.037135 2024-09-10T21:47:34.565 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5808.986321 2024-09-10T21:47:34.565 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4630.109911 2024-09-10T21:47:34.565 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4510.255493 2024-09-10T21:47:34.565 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4792.698527 2024-09-10T21:47:34.565 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6842.370281 2024-09-10T21:47:34.565 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_mds 20/20 2024-09-10T21:47:34.565 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:47:34.565 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_ms 2/2 2024-09-10T21:47:34.565 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_debug_frag true 2024-09-10T21:47:34.565 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:47:34.566 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:47:34.566 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_verify_scatter true 2024-09-10T21:47:34.566 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:47:34.566 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:47:34.566 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:47:34.566 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced client_mount_timeout 600 2024-09-10T21:47:34.566 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced debug_client 20/20 2024-09-10T21:47:34.566 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced debug_ms 1/1 2024-09-10T21:47:34.566 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:47:34.566 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:47:34.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.565+0000 7fce96c00640 1 -- 172.21.15.7:0/2466923259 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fce70077fa0 msgr2=0x7fce7007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:34.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.565+0000 7fce96c00640 1 --2- 172.21.15.7:0/2466923259 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fce70077fa0 0x7fce7007a460 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fce84032000 tx=0x7fce8402c040 comp rx=0 tx=0).stop 2024-09-10T21:47:34.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.565+0000 7fce96c00640 1 -- 172.21.15.7:0/2466923259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9014fc80 msgr2=0x7fce90152680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:34.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.565+0000 7fce96c00640 1 --2- 172.21.15.7:0/2466923259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9014fc80 0x7fce90152680 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00e590 tx=0x7fce7c00ea50 comp rx=0 tx=0).stop 2024-09-10T21:47:34.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.565+0000 7fce96c00640 1 -- 172.21.15.7:0/2466923259 shutdown_connections 2024-09-10T21:47:34.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.565+0000 7fce96c00640 1 --2- 172.21.15.7:0/2466923259 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fce70077fa0 0x7fce7007a460 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:34.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.565+0000 7fce96c00640 1 --2- 172.21.15.7:0/2466923259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fce9014ef10 0x7fce9014f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:34.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.565+0000 7fce96c00640 1 --2- 172.21.15.7:0/2466923259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fce9014fc80 0x7fce90152680 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:34.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.565+0000 7fce96c00640 1 --2- 172.21.15.7:0/2466923259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fce78003aa0 0x7fce900cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:34.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.566+0000 7fce96c00640 1 -- 172.21.15.7:0/2466923259 >> 172.21.15.7:0/2466923259 conn(0x7fce900d3de0 msgr2=0x7fce900b9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:34.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.566+0000 7fce96c00640 1 -- 172.21.15.7:0/2466923259 shutdown_connections 2024-09-10T21:47:34.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.566+0000 7fce96c00640 1 -- 172.21.15.7:0/2466923259 wait complete. 2024-09-10T21:47:34.583 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:47:34.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.746+0000 7fae3e600640 1 -- 172.21.15.7:0/2655504907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1c004d40 msgr2=0x7fae1c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:34.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.746+0000 7fae3e600640 1 --2- 172.21.15.7:0/2655504907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1c004d40 0x7fae1c0051c0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fae380c3620 tx=0x7fae1c005830 comp rx=0 tx=0).stop 2024-09-10T21:47:34.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.746+0000 7fae37e00640 1 -- 172.21.15.7:0/2655504907 reap_dead start 2024-09-10T21:47:34.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.746+0000 7fae3e600640 1 -- 172.21.15.7:0/2655504907 shutdown_connections 2024-09-10T21:47:34.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.746+0000 7fae3e600640 1 -- 172.21.15.7:0/2655504907 >> 172.21.15.7:0/2655504907 conn(0x7fae380b16d0 msgr2=0x7fae380b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:34.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.746+0000 7fae3e600640 1 -- 172.21.15.7:0/2655504907 shutdown_connections 2024-09-10T21:47:34.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.746+0000 7fae3e600640 1 -- 172.21.15.7:0/2655504907 wait complete. 2024-09-10T21:47:34.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.748+0000 7fae3e600640 1 Processor -- start 2024-09-10T21:47:34.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.748+0000 7fae3e600640 1 -- start start 2024-09-10T21:47:34.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.749+0000 7fae3e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1c004d40 0x7fae380b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:34.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.749+0000 7fae3e600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae3814cd80 0x7fae3814d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:34.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.749+0000 7fae3e600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae380c8330 0x7fae380c87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:34.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.749+0000 7fae3e600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fae380cbc90 con 0x7fae1c004d40 2024-09-10T21:47:34.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.749+0000 7fae3e600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fae380cbb10 con 0x7fae380c8330 2024-09-10T21:47:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.749+0000 7fae3e600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fae380cbe10 con 0x7fae3814cd80 2024-09-10T21:47:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.749+0000 7fae37400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1c004d40 0x7fae380b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.749+0000 7fae37400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1c004d40 0x7fae380b0e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54014/0 (socket says 172.21.15.7:54014) 2024-09-10T21:47:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.749+0000 7fae37400640 1 -- 172.21.15.7:0/3387463869 learned_addr learned my addr 172.21.15.7:0/3387463869 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.749+0000 7fae37400640 1 -- 172.21.15.7:0/3387463869 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae3814cd80 msgr2=0x7fae3814d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.749+0000 7fae37400640 1 --2- 172.21.15.7:0/3387463869 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae3814cd80 0x7fae3814d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.749+0000 7fae37400640 1 -- 172.21.15.7:0/3387463869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae380c8330 msgr2=0x7fae380c87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.749+0000 7fae37400640 1 --2- 172.21.15.7:0/3387463869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae380c8330 0x7fae380c87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.749+0000 7fae37400640 1 -- 172.21.15.7:0/3387463869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae380c8d00 con 0x7fae1c004d40 2024-09-10T21:47:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.750+0000 7fae37400640 1 --2- 172.21.15.7:0/3387463869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1c004d40 0x7fae380b0e70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fae28011ae0 tx=0x7fae28009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.750+0000 7fae2fe00640 1 -- 172.21.15.7:0/3387463869 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2800fc30 con 0x7fae1c004d40 2024-09-10T21:47:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.750+0000 7fae2fe00640 1 -- 172.21.15.7:0/3387463869 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae2800fdd0 con 0x7fae1c004d40 2024-09-10T21:47:34.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.750+0000 7fae2fe00640 1 -- 172.21.15.7:0/3387463869 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28004280 con 0x7fae1c004d40 2024-09-10T21:47:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.750+0000 7fae3e600640 1 -- 172.21.15.7:0/3387463869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae380c9630 con 0x7fae1c004d40 2024-09-10T21:47:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.750+0000 7fae3e600640 1 -- 172.21.15.7:0/3387463869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae3814e040 con 0x7fae1c004d40 2024-09-10T21:47:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.751+0000 7fae3e600640 1 -- 172.21.15.7:0/3387463869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae3814e210 con 0x7fae1c004d40 2024-09-10T21:47:34.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.752+0000 7fae2fe00640 1 -- 172.21.15.7:0/3387463869 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fae280035f0 con 0x7fae1c004d40 2024-09-10T21:47:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.753+0000 7fae2fe00640 1 --2- 172.21.15.7:0/3387463869 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fae14077f70 0x7fae1407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.753+0000 7fae2fe00640 1 -- 172.21.15.7:0/3387463869 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5029+0+0 (secure 0 0 0) 0x7fae2809a600 con 0x7fae1c004d40 2024-09-10T21:47:34.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.754+0000 7fae2ea00640 1 --2- 172.21.15.7:0/3387463869 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fae14077f70 0x7fae1407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:34.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.754+0000 7fae2ea00640 1 --2- 172.21.15.7:0/3387463869 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fae14077f70 0x7fae1407a430 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fae38153310 tx=0x7fae20009550 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:34.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.757+0000 7fae2fe00640 1 -- 172.21.15.7:0/3387463869 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fae28063390 con 0x7fae1c004d40 2024-09-10T21:47:34.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.951+0000 7fae3e600640 1 -- 172.21.15.7:0/3387463869 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fae3815cc00 con 0x7fae1c004d40 2024-09-10T21:47:34.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.955+0000 7fae2fe00640 1 -- 172.21.15.7:0/3387463869 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7fae28062c90 con 0x7fae1c004d40 2024-09-10T21:47:34.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.957+0000 7fae3e600640 1 -- 172.21.15.7:0/3387463869 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fae14077f70 msgr2=0x7fae1407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:34.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.957+0000 7fae3e600640 1 --2- 172.21.15.7:0/3387463869 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fae14077f70 0x7fae1407a430 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fae38153310 tx=0x7fae20009550 comp rx=0 tx=0).stop 2024-09-10T21:47:34.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.957+0000 7fae3e600640 1 -- 172.21.15.7:0/3387463869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1c004d40 msgr2=0x7fae380b0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:34.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.957+0000 7fae3e600640 1 --2- 172.21.15.7:0/3387463869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1c004d40 0x7fae380b0e70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fae28011ae0 tx=0x7fae28009f10 comp rx=0 tx=0).stop 2024-09-10T21:47:34.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.957+0000 7fae3e600640 1 -- 172.21.15.7:0/3387463869 shutdown_connections 2024-09-10T21:47:34.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.957+0000 7fae3e600640 1 --2- 172.21.15.7:0/3387463869 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fae14077f70 0x7fae1407a430 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:34.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.957+0000 7fae3e600640 1 --2- 172.21.15.7:0/3387463869 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fae380c8330 0x7fae380c87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:34.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.957+0000 7fae3e600640 1 --2- 172.21.15.7:0/3387463869 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fae3814cd80 0x7fae3814d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:34.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.957+0000 7fae3e600640 1 --2- 172.21.15.7:0/3387463869 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fae1c004d40 0x7fae380b0e70 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:34.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.957+0000 7fae3e600640 1 -- 172.21.15.7:0/3387463869 >> 172.21.15.7:0/3387463869 conn(0x7fae380b16d0 msgr2=0x7fae380cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:34.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.957+0000 7fae3e600640 1 -- 172.21.15.7:0/3387463869 shutdown_connections 2024-09-10T21:47:34.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:34.957+0000 7fae3e600640 1 -- 172.21.15.7:0/3387463869 wait complete. 2024-09-10T21:47:34.974 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:47:34.975 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:47:35.066 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:47:35.166 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:47:35.166 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:47:35.166 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:47:35.202 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:47:35.202 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:47:35.202 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:47:35.238 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:47:35.239 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:47:35.239 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:47:35.277 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:47:35.277 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:47:35.277 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:47:35.314 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:47:35.314 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:47:35.314 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:47:35.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.475+0000 7fc797e00640 1 -- 172.21.15.7:0/1943920997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7980b2b50 msgr2=0x7fc7980d0760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:35.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.475+0000 7fc797e00640 1 --2- 172.21.15.7:0/1943920997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7980b2b50 0x7fc7980d0760 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fc79808c3b0 tx=0x7fc78002a4d0 comp rx=0 tx=0).stop 2024-09-10T21:47:35.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.475+0000 7fc797e00640 1 -- 172.21.15.7:0/1943920997 shutdown_connections 2024-09-10T21:47:35.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.475+0000 7fc797e00640 1 --2- 172.21.15.7:0/1943920997 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc7980d0ca0 0x7fc798161b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:35.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.475+0000 7fc797e00640 1 --2- 172.21.15.7:0/1943920997 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7980b2b50 0x7fc7980d0760 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:35.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.475+0000 7fc797e00640 1 --2- 172.21.15.7:0/1943920997 >> v2:172.21.15.133:3300/0 conn(0x7fc79815dfd0 0x7fc7980d4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:35.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.475+0000 7fc797e00640 1 -- 172.21.15.7:0/1943920997 >> 172.21.15.7:0/1943920997 conn(0x7fc7980d3d50 msgr2=0x7fc7980d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:35.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.475+0000 7fc797e00640 1 -- 172.21.15.7:0/1943920997 shutdown_connections 2024-09-10T21:47:35.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.475+0000 7fc797e00640 1 -- 172.21.15.7:0/1943920997 wait complete. 2024-09-10T21:47:35.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.477+0000 7fc797e00640 1 Processor -- start 2024-09-10T21:47:35.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.477+0000 7fc797e00640 1 -- start start 2024-09-10T21:47:35.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.478+0000 7fc797e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7980b2b50 0x7fc79815edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:35.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.478+0000 7fc797e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc7980d0ca0 0x7fc7981f4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:35.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.478+0000 7fc797e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc79815dfd0 0x7fc7981f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:35.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.478+0000 7fc797e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc7980b0a40 con 0x7fc7980b2b50 2024-09-10T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.478+0000 7fc797e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc7980b08c0 con 0x7fc79815dfd0 2024-09-10T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.478+0000 7fc797e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc798162070 con 0x7fc7980d0ca0 2024-09-10T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.478+0000 7fc796a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7980b2b50 0x7fc79815edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.478+0000 7fc796a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7980b2b50 0x7fc79815edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54040/0 (socket says 172.21.15.7:54040) 2024-09-10T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.478+0000 7fc796a00640 1 -- 172.21.15.7:0/3658929892 learned_addr learned my addr 172.21.15.7:0/3658929892 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.478+0000 7fc797400640 1 --2- 172.21.15.7:0/3658929892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc79815dfd0 0x7fc7981f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.478+0000 7fc78e000640 1 --2- 172.21.15.7:0/3658929892 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc7980d0ca0 0x7fc7981f4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.478+0000 7fc796a00640 1 -- 172.21.15.7:0/3658929892 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc7980d0ca0 msgr2=0x7fc7981f4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.478+0000 7fc796a00640 1 --2- 172.21.15.7:0/3658929892 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc7980d0ca0 0x7fc7981f4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.478+0000 7fc796a00640 1 -- 172.21.15.7:0/3658929892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc79815dfd0 msgr2=0x7fc7981f5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.478+0000 7fc796a00640 1 --2- 172.21.15.7:0/3658929892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc79815dfd0 0x7fc7981f5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.478+0000 7fc796a00640 1 -- 172.21.15.7:0/3658929892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7981f58b0 con 0x7fc7980b2b50 2024-09-10T21:47:35.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.479+0000 7fc796a00640 1 --2- 172.21.15.7:0/3658929892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7980b2b50 0x7fc79815edb0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc78800e7f0 tx=0x7fc78800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.479+0000 7fc78f400640 1 -- 172.21.15.7:0/3658929892 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc78800ccc0 con 0x7fc7980b2b50 2024-09-10T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.479+0000 7fc78f400640 1 -- 172.21.15.7:0/3658929892 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc78800ce80 con 0x7fc7980b2b50 2024-09-10T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.479+0000 7fc78f400640 1 -- 172.21.15.7:0/3658929892 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc788005280 con 0x7fc7980b2b50 2024-09-10T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.479+0000 7fc797e00640 1 -- 172.21.15.7:0/3658929892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7981f5b90 con 0x7fc7980b2b50 2024-09-10T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.479+0000 7fc797e00640 1 -- 172.21.15.7:0/3658929892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7980b7f50 con 0x7fc7980b2b50 2024-09-10T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.480+0000 7fc797e00640 1 -- 172.21.15.7:0/3658929892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc798161b30 con 0x7fc7980b2b50 2024-09-10T21:47:35.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.481+0000 7fc78f400640 1 -- 172.21.15.7:0/3658929892 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc788005420 con 0x7fc7980b2b50 2024-09-10T21:47:35.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.483+0000 7fc78f400640 1 --2- 172.21.15.7:0/3658929892 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc774077f70 0x7fc77407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:35.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.483+0000 7fc78f400640 1 -- 172.21.15.7:0/3658929892 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5029+0+0 (secure 0 0 0) 0x7fc78809a530 con 0x7fc7980b2b50 2024-09-10T21:47:35.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.483+0000 7fc78e000640 1 --2- 172.21.15.7:0/3658929892 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc774077f70 0x7fc77407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:35.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.484+0000 7fc78e000640 1 --2- 172.21.15.7:0/3658929892 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc774077f70 0x7fc77407a430 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc780033000 tx=0x7fc780004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:35.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.487+0000 7fc78f400640 1 -- 172.21.15.7:0/3658929892 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc788063230 con 0x7fc7980b2b50 2024-09-10T21:47:35.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.719+0000 7fc797e00640 1 -- 172.21.15.7:0/3658929892 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7980caa50 con 0x7fc7980b2b50 2024-09-10T21:47:35.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.720+0000 7fc78f400640 1 -- 172.21.15.7:0/3658929892 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) ==== 94+0+3911 (secure 0 0 0) 0x7fc788062b30 con 0x7fc7980b2b50 2024-09-10T21:47:35.721 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:35.721 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":56,"btime":"2024-09-10T21:47:28:806530+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5884,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3488410478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3488410478},{"type":"v1","addr":"172.21.15.133:6837","nonce":3488410478}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5890,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3660400727","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3660400727},{"type":"v1","addr":"172.21.15.7:6835","nonce":3660400727}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/2388873704","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":2388873704},{"type":"v1","addr":"172.21.15.7:6834","nonce":2388873704}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6301,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/2287865297","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2287865297},{"type":"v1","addr":"172.21.15.133:6836","nonce":2287865297}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":56}],"filesystems":[]} 2024-09-10T21:47:35.721 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 56 2024-09-10T21:47:35.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.722+0000 7fc797e00640 1 -- 172.21.15.7:0/3658929892 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc774077f70 msgr2=0x7fc77407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:35.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.722+0000 7fc797e00640 1 --2- 172.21.15.7:0/3658929892 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc774077f70 0x7fc77407a430 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fc780033000 tx=0x7fc780004730 comp rx=0 tx=0).stop 2024-09-10T21:47:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.722+0000 7fc797e00640 1 -- 172.21.15.7:0/3658929892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7980b2b50 msgr2=0x7fc79815edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.722+0000 7fc797e00640 1 --2- 172.21.15.7:0/3658929892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7980b2b50 0x7fc79815edb0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fc78800e7f0 tx=0x7fc78800ecb0 comp rx=0 tx=0).stop 2024-09-10T21:47:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.722+0000 7fc797e00640 1 -- 172.21.15.7:0/3658929892 shutdown_connections 2024-09-10T21:47:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.722+0000 7fc797e00640 1 --2- 172.21.15.7:0/3658929892 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc774077f70 0x7fc77407a430 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.722+0000 7fc797e00640 1 --2- 172.21.15.7:0/3658929892 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc79815dfd0 0x7fc7981f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.722+0000 7fc797e00640 1 --2- 172.21.15.7:0/3658929892 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc7980d0ca0 0x7fc7981f4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.722+0000 7fc797e00640 1 --2- 172.21.15.7:0/3658929892 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc7980b2b50 0x7fc79815edb0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.722+0000 7fc797e00640 1 -- 172.21.15.7:0/3658929892 >> 172.21.15.7:0/3658929892 conn(0x7fc7980d3d50 msgr2=0x7fc7980b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.722+0000 7fc797e00640 1 -- 172.21.15.7:0/3658929892 shutdown_connections 2024-09-10T21:47:35.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.722+0000 7fc797e00640 1 -- 172.21.15.7:0/3658929892 wait complete. 2024-09-10T21:47:35.739 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:47:35.739 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:47:35.739 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:47:35.740 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:47:35.740 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:47:35.740 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:47:35.740 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:47:35.740 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:47:35.740 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:47:35.740 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:47:35.741 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:47:35.741 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:47:35.741 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:47:35.741 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:47:35.741 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:47:35.741 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:47:35.754 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:47:35.754 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:47:35.755 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:47:35.755 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:47:35.759 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:47:35.759 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:47:35.760 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:47:35.760 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:47:35.798 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:47:35.798 DEBUG:teuthology.parallel:result is None 2024-09-10T21:47:35.800 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:47:35.800 DEBUG:teuthology.parallel:result is None 2024-09-10T21:47:35.804 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:47:35.804 DEBUG:teuthology.parallel:result is None 2024-09-10T21:47:35.807 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:47:35.807 DEBUG:teuthology.parallel:result is None 2024-09-10T21:47:35.807 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:47:35.931 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-09-10T21:47:35.932 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2024-09-10T21:47:35.932 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2024-09-10T21:47:35.933 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-09-10T21:47:35.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.976+0000 7fc161200640 1 -- 172.21.15.7:0/3280898887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c0688e0 msgr2=0x7fc15c0b1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:35.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.976+0000 7fc161200640 1 --2- 172.21.15.7:0/3280898887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c0688e0 0x7fc15c0b1710 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fc154005fd0 tx=0x7fc15402a820 comp rx=0 tx=0).stop 2024-09-10T21:47:35.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.976+0000 7fc161200640 1 -- 172.21.15.7:0/3280898887 shutdown_connections 2024-09-10T21:47:35.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.976+0000 7fc161200640 1 --2- 172.21.15.7:0/3280898887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c0688e0 0x7fc15c0b1710 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:35.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.976+0000 7fc161200640 1 --2- 172.21.15.7:0/3280898887 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc15c15bbf0 0x7fc15c15bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:35.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.976+0000 7fc161200640 1 -- 172.21.15.7:0/3280898887 >> 172.21.15.7:0/3280898887 conn(0x7fc15c0b1f00 msgr2=0x7fc15c0b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:35.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.976+0000 7fc161200640 1 -- 172.21.15.7:0/3280898887 shutdown_connections 2024-09-10T21:47:35.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.976+0000 7fc161200640 1 -- 172.21.15.7:0/3280898887 wait complete. 2024-09-10T21:47:35.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.978+0000 7fc161200640 1 Processor -- start 2024-09-10T21:47:35.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.978+0000 7fc161200640 1 -- start start 2024-09-10T21:47:35.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.979+0000 7fc161200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc15c0688e0 0x7fc15c0b34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:35.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.979+0000 7fc161200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc15c15bbf0 0x7fc15c0c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:35.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.979+0000 7fc161200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c15c510 0x7fc15c0aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:35.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.979+0000 7fc161200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c0cc560 con 0x7fc15c15c510 2024-09-10T21:47:35.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.979+0000 7fc161200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c0cc3e0 con 0x7fc15c0688e0 2024-09-10T21:47:35.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.979+0000 7fc161200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc15c0cc6e0 con 0x7fc15c15bbf0 2024-09-10T21:47:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.979+0000 7fc15a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c15c510 0x7fc15c0aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.979+0000 7fc15a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c15c510 0x7fc15c0aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54096/0 (socket says 172.21.15.7:54096) 2024-09-10T21:47:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.979+0000 7fc15a800640 1 -- 172.21.15.7:0/2340220889 learned_addr learned my addr 172.21.15.7:0/2340220889 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.979+0000 7fc159e00640 1 --2- 172.21.15.7:0/2340220889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc15c0688e0 0x7fc15c0b34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.979+0000 7fc153e00640 1 --2- 172.21.15.7:0/2340220889 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc15c15bbf0 0x7fc15c0c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.979+0000 7fc15a800640 1 -- 172.21.15.7:0/2340220889 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc15c15bbf0 msgr2=0x7fc15c0c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.979+0000 7fc15a800640 1 --2- 172.21.15.7:0/2340220889 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc15c15bbf0 0x7fc15c0c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.979+0000 7fc15a800640 1 -- 172.21.15.7:0/2340220889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc15c0688e0 msgr2=0x7fc15c0b34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.979+0000 7fc15a800640 1 --2- 172.21.15.7:0/2340220889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc15c0688e0 0x7fc15c0b34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:35.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.979+0000 7fc15a800640 1 -- 172.21.15.7:0/2340220889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15c0c62a0 con 0x7fc15c15c510 2024-09-10T21:47:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.980+0000 7fc15a800640 1 --2- 172.21.15.7:0/2340220889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c15c510 0x7fc15c0aee80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc154006010 tx=0x7fc154032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.980+0000 7fc152000640 1 -- 172.21.15.7:0/2340220889 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154037040 con 0x7fc15c15c510 2024-09-10T21:47:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.980+0000 7fc152000640 1 -- 172.21.15.7:0/2340220889 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc154042070 con 0x7fc15c15c510 2024-09-10T21:47:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.980+0000 7fc152000640 1 -- 172.21.15.7:0/2340220889 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1540042a0 con 0x7fc15c15c510 2024-09-10T21:47:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.980+0000 7fc161200640 1 -- 172.21.15.7:0/2340220889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc15c0ca2a0 con 0x7fc15c15c510 2024-09-10T21:47:35.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.980+0000 7fc161200640 1 -- 172.21.15.7:0/2340220889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc15c0c76b0 con 0x7fc15c15c510 2024-09-10T21:47:35.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.981+0000 7fc161200640 1 -- 172.21.15.7:0/2340220889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc15c0cb600 con 0x7fc15c15c510 2024-09-10T21:47:35.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.982+0000 7fc152000640 1 -- 172.21.15.7:0/2340220889 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc15403a030 con 0x7fc15c15c510 2024-09-10T21:47:35.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.983+0000 7fc152000640 1 --2- 172.21.15.7:0/2340220889 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc138077f70 0x7fc13807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:35.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.983+0000 7fc152000640 1 -- 172.21.15.7:0/2340220889 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5029+0+0 (secure 0 0 0) 0x7fc1540ba440 con 0x7fc15c15c510 2024-09-10T21:47:35.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.984+0000 7fc159e00640 1 --2- 172.21.15.7:0/2340220889 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc138077f70 0x7fc13807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:35.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.984+0000 7fc159e00640 1 --2- 172.21.15.7:0/2340220889 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc138077f70 0x7fc13807a430 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00bbe0 tx=0x7fc14c00b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:35.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:35.987+0000 7fc152000640 1 -- 172.21.15.7:0/2340220889 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc15c0cb600 con 0x7fc15c15c510 2024-09-10T21:47:36.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:36.197+0000 7fc161200640 1 -- 172.21.15.7:0/2340220889 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fc15c15bfd0 con 0x7fc15c15c510 2024-09-10T21:47:37.197 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.196+0000 7fc152000640 1 -- 172.21.15.7:0/2340220889 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v81) ==== 97+0+0 (secure 0 0 0) 0x7fc154082af0 con 0x7fc15c15c510 2024-09-10T21:47:37.197 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-09-10T21:47:37.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.199+0000 7fc161200640 1 -- 172.21.15.7:0/2340220889 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc138077f70 msgr2=0x7fc13807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:37.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.199+0000 7fc161200640 1 --2- 172.21.15.7:0/2340220889 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc138077f70 0x7fc13807a430 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc14c00bbe0 tx=0x7fc14c00b390 comp rx=0 tx=0).stop 2024-09-10T21:47:37.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.199+0000 7fc161200640 1 -- 172.21.15.7:0/2340220889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c15c510 msgr2=0x7fc15c0aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:37.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.199+0000 7fc161200640 1 --2- 172.21.15.7:0/2340220889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c15c510 0x7fc15c0aee80 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fc154006010 tx=0x7fc154032c90 comp rx=0 tx=0).stop 2024-09-10T21:47:37.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.199+0000 7fc161200640 1 -- 172.21.15.7:0/2340220889 shutdown_connections 2024-09-10T21:47:37.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.199+0000 7fc161200640 1 --2- 172.21.15.7:0/2340220889 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc138077f70 0x7fc13807a430 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:37.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.199+0000 7fc161200640 1 --2- 172.21.15.7:0/2340220889 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc15c15c510 0x7fc15c0aee80 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:37.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.199+0000 7fc161200640 1 --2- 172.21.15.7:0/2340220889 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc15c15bbf0 0x7fc15c0c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:37.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.199+0000 7fc161200640 1 --2- 172.21.15.7:0/2340220889 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc15c0688e0 0x7fc15c0b34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:37.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.199+0000 7fc161200640 1 -- 172.21.15.7:0/2340220889 >> 172.21.15.7:0/2340220889 conn(0x7fc15c0b1f00 msgr2=0x7fc15c0b22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:37.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.199+0000 7fc161200640 1 -- 172.21.15.7:0/2340220889 shutdown_connections 2024-09-10T21:47:37.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.199+0000 7fc161200640 1 -- 172.21.15.7:0/2340220889 wait complete. 2024-09-10T21:47:37.219 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:47:37.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.383+0000 7f3824400640 1 -- 172.21.15.7:0/4047840774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c0cbd50 msgr2=0x7f381c0cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:37.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.383+0000 7f3824400640 1 --2- 172.21.15.7:0/4047840774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c0cbd50 0x7f381c0cc1c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f381c15bf30 tx=0x7f380802a540 comp rx=0 tx=0).stop 2024-09-10T21:47:37.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.383+0000 7f3824400640 1 -- 172.21.15.7:0/4047840774 shutdown_connections 2024-09-10T21:47:37.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.383+0000 7f3824400640 1 --2- 172.21.15.7:0/4047840774 >> v2:172.21.15.133:3300/0 conn(0x7f381c0ae020 0x7f381c0ae470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:37.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.383+0000 7f3824400640 1 --2- 172.21.15.7:0/4047840774 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c0cbd50 0x7f381c0cc1c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:37.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.383+0000 7f3824400640 1 -- 172.21.15.7:0/4047840774 >> 172.21.15.7:0/4047840774 conn(0x7f381c0b16d0 msgr2=0x7f381c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:37.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.383+0000 7f3824400640 1 -- 172.21.15.7:0/4047840774 shutdown_connections 2024-09-10T21:47:37.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.383+0000 7f3824400640 1 -- 172.21.15.7:0/4047840774 wait complete. 2024-09-10T21:47:37.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.384+0000 7f3824400640 1 Processor -- start 2024-09-10T21:47:37.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.385+0000 7f3824400640 1 -- start start 2024-09-10T21:47:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.385+0000 7f3824400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c0ae020 0x7f381c157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.385+0000 7f3824400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c0cbd50 0x7f381c0b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.385+0000 7f3824400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381c1619e0 0x7f381c0b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:37.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.385+0000 7f3824400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f381c0ce030 con 0x7f381c0ae020 2024-09-10T21:47:37.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.385+0000 7f3824400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f381c0cdeb0 con 0x7f381c1619e0 2024-09-10T21:47:37.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.385+0000 7f3824400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f381c161dc0 con 0x7f381c0cbd50 2024-09-10T21:47:37.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.386+0000 7f3821a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c0ae020 0x7f381c157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:37.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.386+0000 7f3821a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c0ae020 0x7f381c157550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54118/0 (socket says 172.21.15.7:54118) 2024-09-10T21:47:37.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.386+0000 7f3821a00640 1 -- 172.21.15.7:0/3696360608 learned_addr learned my addr 172.21.15.7:0/3696360608 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:37.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.386+0000 7f3821000640 1 --2- 172.21.15.7:0/3696360608 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c0cbd50 0x7f381c0b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:37.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.386+0000 7f3822400640 1 --2- 172.21.15.7:0/3696360608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381c1619e0 0x7f381c0b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:37.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.386+0000 7f3821a00640 1 -- 172.21.15.7:0/3696360608 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c0cbd50 msgr2=0x7f381c0b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:37.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.386+0000 7f3821a00640 1 --2- 172.21.15.7:0/3696360608 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c0cbd50 0x7f381c0b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:37.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.386+0000 7f3821a00640 1 -- 172.21.15.7:0/3696360608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381c1619e0 msgr2=0x7f381c0b4fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:37.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.386+0000 7f3821a00640 1 --2- 172.21.15.7:0/3696360608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381c1619e0 0x7f381c0b4fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:37.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.386+0000 7f3821a00640 1 -- 172.21.15.7:0/3696360608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381c157a90 con 0x7f381c0ae020 2024-09-10T21:47:37.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.386+0000 7f3822400640 1 --2- 172.21.15.7:0/3696360608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381c1619e0 0x7f381c0b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:37.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.387+0000 7f3821a00640 1 --2- 172.21.15.7:0/3696360608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c0ae020 0x7f381c157550 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f381000c450 tx=0x7f381000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:37.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.387+0000 7f3819600640 1 -- 172.21.15.7:0/3696360608 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810016020 con 0x7f381c0ae020 2024-09-10T21:47:37.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.387+0000 7f3819600640 1 -- 172.21.15.7:0/3696360608 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3810012920 con 0x7f381c0ae020 2024-09-10T21:47:37.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.387+0000 7f3824400640 1 -- 172.21.15.7:0/3696360608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381c0c70e0 con 0x7f381c0ae020 2024-09-10T21:47:37.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.387+0000 7f3819600640 1 -- 172.21.15.7:0/3696360608 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3810012d70 con 0x7f381c0ae020 2024-09-10T21:47:37.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.387+0000 7f3824400640 1 -- 172.21.15.7:0/3696360608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f381c0c7980 con 0x7f381c0ae020 2024-09-10T21:47:37.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.388+0000 7f3824400640 1 -- 172.21.15.7:0/3696360608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f381c0ae470 con 0x7f381c0ae020 2024-09-10T21:47:37.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.389+0000 7f3819600640 1 -- 172.21.15.7:0/3696360608 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f381001b400 con 0x7f381c0ae020 2024-09-10T21:47:37.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.390+0000 7f3819600640 1 --2- 172.21.15.7:0/3696360608 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f37fc077f70 0x7f37fc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:37.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.390+0000 7f3819600640 1 -- 172.21.15.7:0/3696360608 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4857+0+0 (secure 0 0 0) 0x7f3810099ac0 con 0x7f381c0ae020 2024-09-10T21:47:37.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.390+0000 7f3821000640 1 --2- 172.21.15.7:0/3696360608 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f37fc077f70 0x7f37fc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:37.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.391+0000 7f3821000640 1 --2- 172.21.15.7:0/3696360608 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f37fc077f70 0x7f37fc07a430 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3808034000 tx=0x7f3808004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:37.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.394+0000 7f3819600640 1 -- 172.21.15.7:0/3696360608 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3810020030 con 0x7f381c0ae020 2024-09-10T21:47:37.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.660+0000 7f3824400640 1 -- 172.21.15.7:0/3696360608 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f381c0cb550 con 0x7f381c0ae020 2024-09-10T21:47:37.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.660+0000 7f3819600640 1 -- 172.21.15.7:0/3696360608 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v35) ==== 80+0+5304 (secure 0 0 0) 0x7f381c157a90 con 0x7f381c0ae020 2024-09-10T21:47:37.662 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:37.662 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:47:37.662 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-09-10T21:47:37.662 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:37.662 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-09-10T21:47:37.662 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmgRMuLBAA88QT12AXv2qf11IxU20Xbg==", 2024-09-10T21:47:37.662 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:47:37.662 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:47:37.662 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:47:37.662 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:47:37.662 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:47:37.662 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.662 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:37.663 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:37.663 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-09-10T21:47:37.663 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmRUM2BBAAwcQM6z+hjZI4ZWBlVrYMwQ==", 2024-09-10T21:47:37.663 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:47:37.663 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:47:37.663 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:47:37.663 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:47:37.663 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:47:37.663 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.663 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:37.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:37.664 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-09-10T21:47:37.664 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmnq31NRAAj1eJUAIS986idxppfeXEAA==", 2024-09-10T21:47:37.664 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:47:37.664 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:47:37.664 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:47:37.664 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:47:37.664 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:47:37.664 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.664 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:37.664 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:37.664 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-09-10T21:47:37.665 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmy43uDRAAzdKjLxZMGROuDABrsJ0zgQ==", 2024-09-10T21:47:37.665 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:47:37.665 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:47:37.665 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:47:37.665 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:47:37.665 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:47:37.665 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.665 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:37.665 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:37.665 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-09-10T21:47:37.665 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDwvOBmzfgtOxAAk6K5fU0Uar4VW9nUUIuNdw==", 2024-09-10T21:47:37.665 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:47:37.665 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:47:37.665 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:47:37.666 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:47:37.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.666 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:37.666 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:37.666 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-09-10T21:47:37.666 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD0vOBmHcRzChAA3z+CaGgu5NkyUmjpqzmKjw==", 2024-09-10T21:47:37.666 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:47:37.666 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:47:37.666 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:47:37.666 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:47:37.666 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:37.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:37.667 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-09-10T21:47:37.667 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD3vOBm9sArFBAAGQlyiw6qEcyumEBICWEBKA==", 2024-09-10T21:47:37.667 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:47:37.667 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:47:37.667 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:47:37.667 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:47:37.667 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.667 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:37.667 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:37.667 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-09-10T21:47:37.667 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD6vOBmkO3rHxAAW0zPydYZRGUKjGh3yIyPCA==", 2024-09-10T21:47:37.667 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:47:37.668 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:47:37.668 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:47:37.668 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:47:37.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.668 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:37.668 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:37.668 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-09-10T21:47:37.668 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAAveBmlPABMxAAeazVOzc1DhsVAwoHn87QIA==", 2024-09-10T21:47:37.668 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:47:37.668 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:47:37.668 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:47:37.668 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:47:37.668 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:37.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:37.669 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-09-10T21:47:37.669 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAEveBmL9qzARAApu8nWIY9HVsDl4yUnfhH9w==", 2024-09-10T21:47:37.669 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:47:37.669 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:47:37.669 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:47:37.669 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:47:37.669 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.669 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:37.669 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:37.669 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-09-10T21:47:37.669 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAHveBmJ4ofDRAAEiCFN8nGzVSR3u2gKoT+2Q==", 2024-09-10T21:47:37.670 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:47:37.670 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:47:37.670 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:47:37.670 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:47:37.670 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.670 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:37.670 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:37.670 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-09-10T21:47:37.670 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAKveBmoUGEGBAAhXmkUaniuAQrBiYPWr8Tog==", 2024-09-10T21:47:37.670 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:47:37.670 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:47:37.670 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:47:37.670 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:47:37.670 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:37.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:37.671 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-09-10T21:47:37.671 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBm0aqcFhAAtid2bnzjcpaYAbU7g0JcOA==", 2024-09-10T21:47:37.671 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:47:37.671 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:47:37.671 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-09-10T21:47:37.671 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:47:37.671 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.671 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:37.671 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:37.671 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-09-10T21:47:37.671 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmqf+MIxAAib5hVnbBsw/bJPqsQszJwg==", 2024-09-10T21:47:37.672 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:47:37.672 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:47:37.672 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:47:37.672 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:47:37.672 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:47:37.672 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.672 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:37.672 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:37.672 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-09-10T21:47:37.672 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmybQhHRAAZUwqnIsihEWnHZWQTYr+uQ==", 2024-09-10T21:47:37.672 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:47:37.672 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:47:37.672 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:47:37.673 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:47:37.673 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:47:37.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:37.673 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:37.673 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-09-10T21:47:37.673 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmDbrrKRAAUxb3EBfKIJ4FAGHW3QIsLg==", 2024-09-10T21:47:37.673 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:47:37.673 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:47:37.673 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:47:37.673 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:47:37.673 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:47:37.673 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.673 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:37.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:37.674 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-09-10T21:47:37.674 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDtvOBmF+eEOBAA3icvNqI0nWDShE49DZ3bvg==", 2024-09-10T21:47:37.674 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:47:37.674 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:47:37.674 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:47:37.674 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:47:37.674 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:47:37.674 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.674 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:37.674 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:37.674 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-09-10T21:47:37.674 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmsu6OJRAAhdgDvpeHY/zDKUeUOv6xxA==", 2024-09-10T21:47:37.675 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:47:37.675 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:47:37.675 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-10T21:47:37.675 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:47:37.675 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.675 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:37.675 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:37.675 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-09-10T21:47:37.675 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmKBHLHhAA0W5My19tIhN1v2qev4cBTA==", 2024-09-10T21:47:37.675 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:47:37.675 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:47:37.675 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-10T21:47:37.675 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:47:37.675 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.676 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:37.676 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T21:47:37.676 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:47:37.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.662+0000 7f3824400640 1 -- 172.21.15.7:0/3696360608 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f37fc077f70 msgr2=0x7f37fc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:37.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.662+0000 7f3824400640 1 --2- 172.21.15.7:0/3696360608 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f37fc077f70 0x7f37fc07a430 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f3808034000 tx=0x7f3808004730 comp rx=0 tx=0).stop 2024-09-10T21:47:37.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.662+0000 7f3824400640 1 -- 172.21.15.7:0/3696360608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c0ae020 msgr2=0x7f381c157550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:37.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.662+0000 7f3824400640 1 --2- 172.21.15.7:0/3696360608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c0ae020 0x7f381c157550 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f381000c450 tx=0x7f381000c910 comp rx=0 tx=0).stop 2024-09-10T21:47:37.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.663+0000 7f3824400640 1 -- 172.21.15.7:0/3696360608 shutdown_connections 2024-09-10T21:47:37.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.663+0000 7f3824400640 1 --2- 172.21.15.7:0/3696360608 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f37fc077f70 0x7f37fc07a430 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:37.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.663+0000 7f3824400640 1 --2- 172.21.15.7:0/3696360608 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f381c1619e0 0x7f381c0b4fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:37.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.663+0000 7f3824400640 1 --2- 172.21.15.7:0/3696360608 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f381c0cbd50 0x7f381c0b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:37.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.663+0000 7f3824400640 1 --2- 172.21.15.7:0/3696360608 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f381c0ae020 0x7f381c157550 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:37.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.663+0000 7f3824400640 1 -- 172.21.15.7:0/3696360608 >> 172.21.15.7:0/3696360608 conn(0x7f381c0b16d0 msgr2=0x7f381c15d510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:37.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.663+0000 7f3824400640 1 -- 172.21.15.7:0/3696360608 shutdown_connections 2024-09-10T21:47:37.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.663+0000 7f3824400640 1 -- 172.21.15.7:0/3696360608 wait complete. 2024-09-10T21:47:37.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:47:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.843+0000 7f4a88600640 1 -- 172.21.15.7:0/655252887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a68004800 msgr2=0x7f4a68004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.843+0000 7f4a88600640 1 --2- 172.21.15.7:0/655252887 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a68004800 0x7f4a68004c80 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c006f90 tx=0x7f4a68005830 comp rx=0 tx=0).stop 2024-09-10T21:47:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.843+0000 7f4a86600640 1 -- 172.21.15.7:0/655252887 reap_dead start 2024-09-10T21:47:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.844+0000 7f4a88600640 1 -- 172.21.15.7:0/655252887 shutdown_connections 2024-09-10T21:47:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.844+0000 7f4a88600640 1 -- 172.21.15.7:0/655252887 >> 172.21.15.7:0/655252887 conn(0x7f4a800b16d0 msgr2=0x7f4a800b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:37.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.844+0000 7f4a88600640 1 -- 172.21.15.7:0/655252887 shutdown_connections 2024-09-10T21:47:37.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.844+0000 7f4a88600640 1 -- 172.21.15.7:0/655252887 wait complete. 2024-09-10T21:47:37.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.845+0000 7f4a88600640 1 Processor -- start 2024-09-10T21:47:37.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.846+0000 7f4a88600640 1 -- start start 2024-09-10T21:47:37.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.846+0000 7f4a88600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a68004800 0x7f4a801f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:37.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.846+0000 7f4a88600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a800c7ca0 0x7f4a800c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:37.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.846+0000 7f4a88600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a8014ff40 0x7f4a801503f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:37.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.847+0000 7f4a88600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4a80070610 con 0x7f4a68004800 2024-09-10T21:47:37.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.847+0000 7f4a88600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4a800ccef0 con 0x7f4a800c7ca0 2024-09-10T21:47:37.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.847+0000 7f4a88600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4a800ae0c0 con 0x7f4a8014ff40 2024-09-10T21:47:37.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.847+0000 7f4a85c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a68004800 0x7f4a801f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:37.849 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.847+0000 7f4a85c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a68004800 0x7f4a801f8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54136/0 (socket says 172.21.15.7:54136) 2024-09-10T21:47:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.847+0000 7f4a85c00640 1 -- 172.21.15.7:0/622018762 learned_addr learned my addr 172.21.15.7:0/622018762 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.847+0000 7f4a85200640 1 --2- 172.21.15.7:0/622018762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a800c7ca0 0x7f4a800c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.847+0000 7f4a86600640 1 --2- 172.21.15.7:0/622018762 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a8014ff40 0x7f4a801503f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.847+0000 7f4a85c00640 1 -- 172.21.15.7:0/622018762 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a8014ff40 msgr2=0x7f4a801503f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.847+0000 7f4a85c00640 1 --2- 172.21.15.7:0/622018762 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a8014ff40 0x7f4a801503f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.847+0000 7f4a85c00640 1 -- 172.21.15.7:0/622018762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a800c7ca0 msgr2=0x7f4a800c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.847+0000 7f4a85c00640 1 --2- 172.21.15.7:0/622018762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a800c7ca0 0x7f4a800c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.847+0000 7f4a85c00640 1 -- 172.21.15.7:0/622018762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a8015b280 con 0x7f4a68004800 2024-09-10T21:47:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.847+0000 7f4a86600640 1 --2- 172.21.15.7:0/622018762 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a8014ff40 0x7f4a801503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.848+0000 7f4a85c00640 1 --2- 172.21.15.7:0/622018762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a68004800 0x7f4a801f8ba0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4a74011980 tx=0x7f4a74011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.848+0000 7f4a7d600640 1 -- 172.21.15.7:0/622018762 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a7400fdb0 con 0x7f4a68004800 2024-09-10T21:47:37.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.848+0000 7f4a7d600640 1 -- 172.21.15.7:0/622018762 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a74005020 con 0x7f4a68004800 2024-09-10T21:47:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.848+0000 7f4a7d600640 1 -- 172.21.15.7:0/622018762 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a74005470 con 0x7f4a68004800 2024-09-10T21:47:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.848+0000 7f4a88600640 1 -- 172.21.15.7:0/622018762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a800c9c20 con 0x7f4a68004800 2024-09-10T21:47:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.848+0000 7f4a88600640 1 -- 172.21.15.7:0/622018762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a800adde0 con 0x7f4a68004800 2024-09-10T21:47:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.849+0000 7f4a88600640 1 -- 172.21.15.7:0/622018762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a800c6640 con 0x7f4a68004800 2024-09-10T21:47:37.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.850+0000 7f4a7d600640 1 -- 172.21.15.7:0/622018762 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4a740047a0 con 0x7f4a68004800 2024-09-10T21:47:37.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.851+0000 7f4a7d600640 1 --2- 172.21.15.7:0/622018762 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4a60077fa0 0x7f4a6007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:37.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.851+0000 7f4a7d600640 1 -- 172.21.15.7:0/622018762 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4857+0+0 (secure 0 0 0) 0x7f4a7409a6f0 con 0x7f4a68004800 2024-09-10T21:47:37.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.851+0000 7f4a85200640 1 --2- 172.21.15.7:0/622018762 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4a60077fa0 0x7f4a6007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:37.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.852+0000 7f4a85200640 1 --2- 172.21.15.7:0/622018762 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4a60077fa0 0x7f4a6007a460 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c033000 tx=0x7f4a6c03b040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:37.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:37.855+0000 7f4a7d600640 1 -- 172.21.15.7:0/622018762 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4a800c6640 con 0x7f4a68004800 2024-09-10T21:47:38.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.055+0000 7f4a88600640 1 -- 172.21.15.7:0/622018762 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f4a801601f0 con 0x7f4a68004800 2024-09-10T21:47:38.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.055+0000 7f4a7d600640 1 -- 172.21.15.7:0/622018762 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f4a8015b280 con 0x7f4a68004800 2024-09-10T21:47:38.056 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:38.056 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-09-10T21:47:38.056 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:47:38.057 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-09-10T21:47:38.057 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-09-10T21:47:38.057 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:38.057 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-09-10T21:47:38.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.057+0000 7f4a88600640 1 -- 172.21.15.7:0/622018762 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4a60077fa0 msgr2=0x7f4a6007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:38.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.057+0000 7f4a88600640 1 --2- 172.21.15.7:0/622018762 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4a60077fa0 0x7f4a6007a460 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c033000 tx=0x7f4a6c03b040 comp rx=0 tx=0).stop 2024-09-10T21:47:38.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.057+0000 7f4a88600640 1 -- 172.21.15.7:0/622018762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a68004800 msgr2=0x7f4a801f8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:38.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.057+0000 7f4a88600640 1 --2- 172.21.15.7:0/622018762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a68004800 0x7f4a801f8ba0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f4a74011980 tx=0x7f4a74011e40 comp rx=0 tx=0).stop 2024-09-10T21:47:38.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.057+0000 7f4a88600640 1 -- 172.21.15.7:0/622018762 shutdown_connections 2024-09-10T21:47:38.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.057+0000 7f4a88600640 1 --2- 172.21.15.7:0/622018762 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4a60077fa0 0x7f4a6007a460 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:38.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.057+0000 7f4a88600640 1 --2- 172.21.15.7:0/622018762 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a800c7ca0 0x7f4a800c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:38.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.057+0000 7f4a88600640 1 --2- 172.21.15.7:0/622018762 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a8014ff40 0x7f4a801503f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:38.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.057+0000 7f4a88600640 1 --2- 172.21.15.7:0/622018762 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a68004800 0x7f4a801f8ba0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:38.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.057+0000 7f4a88600640 1 -- 172.21.15.7:0/622018762 >> 172.21.15.7:0/622018762 conn(0x7f4a800b16d0 msgr2=0x7f4a800c3890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:38.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.058+0000 7f4a88600640 1 -- 172.21.15.7:0/622018762 shutdown_connections 2024-09-10T21:47:38.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.058+0000 7f4a88600640 1 -- 172.21.15.7:0/622018762 wait complete. 2024-09-10T21:47:38.075 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:47:38.075 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:47:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.240+0000 7fa119000640 1 -- 172.21.15.7:0/790985865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa11406f170 msgr2=0x7fa11406e480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.240+0000 7fa119000640 1 --2- 172.21.15.7:0/790985865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa11406f170 0x7fa11406e480 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fa1140ae8c0 tx=0x7fa10402a530 comp rx=0 tx=0).stop 2024-09-10T21:47:38.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.240+0000 7fa119000640 1 -- 172.21.15.7:0/790985865 shutdown_connections 2024-09-10T21:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.240+0000 7fa119000640 1 --2- 172.21.15.7:0/790985865 >> v2:172.21.15.133:3300/0 conn(0x7fa1140700c0 0x7fa1140cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.240+0000 7fa119000640 1 --2- 172.21.15.7:0/790985865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa11406f170 0x7fa11406e480 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.240+0000 7fa119000640 1 -- 172.21.15.7:0/790985865 >> 172.21.15.7:0/790985865 conn(0x7fa1140b1580 msgr2=0x7fa1140b1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.240+0000 7fa119000640 1 -- 172.21.15.7:0/790985865 shutdown_connections 2024-09-10T21:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.240+0000 7fa119000640 1 -- 172.21.15.7:0/790985865 wait complete. 2024-09-10T21:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.242+0000 7fa119000640 1 Processor -- start 2024-09-10T21:47:38.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.242+0000 7fa119000640 1 -- start start 2024-09-10T21:47:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.243+0000 7fa119000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11406f170 0x7fa1141575d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.243+0000 7fa119000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1140700c0 0x7fa1140b2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.243+0000 7fa119000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1140cb760 0x7fa1140b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:38.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.243+0000 7fa119000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa11415c610 con 0x7fa1140700c0 2024-09-10T21:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.243+0000 7fa119000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa11415c490 con 0x7fa11406f170 2024-09-10T21:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.243+0000 7fa119000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa11415c790 con 0x7fa1140cb760 2024-09-10T21:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.243+0000 7fa112a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1140700c0 0x7fa1140b2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.243+0000 7fa112a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1140700c0 0x7fa1140b2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54154/0 (socket says 172.21.15.7:54154) 2024-09-10T21:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.243+0000 7fa112a00640 1 -- 172.21.15.7:0/1330230263 learned_addr learned my addr 172.21.15.7:0/1330230263 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.243+0000 7fa113400640 1 --2- 172.21.15.7:0/1330230263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11406f170 0x7fa1141575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.243+0000 7fa113e00640 1 --2- 172.21.15.7:0/1330230263 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1140cb760 0x7fa1140b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.243+0000 7fa112a00640 1 -- 172.21.15.7:0/1330230263 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1140cb760 msgr2=0x7fa1140b4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.243+0000 7fa112a00640 1 --2- 172.21.15.7:0/1330230263 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1140cb760 0x7fa1140b4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.243+0000 7fa112a00640 1 -- 172.21.15.7:0/1330230263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11406f170 msgr2=0x7fa1141575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.243+0000 7fa112a00640 1 --2- 172.21.15.7:0/1330230263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11406f170 0x7fa1141575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:38.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.243+0000 7fa112a00640 1 -- 172.21.15.7:0/1330230263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1140b36e0 con 0x7fa1140700c0 2024-09-10T21:47:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.244+0000 7fa112a00640 1 --2- 172.21.15.7:0/1330230263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1140700c0 0x7fa1140b2880 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa10000c460 tx=0x7fa10000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.244+0000 7fa10b400640 1 -- 172.21.15.7:0/1330230263 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100016020 con 0x7fa1140700c0 2024-09-10T21:47:38.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.244+0000 7fa10b400640 1 -- 172.21.15.7:0/1330230263 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa100012940 con 0x7fa1140700c0 2024-09-10T21:47:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.244+0000 7fa119000640 1 -- 172.21.15.7:0/1330230263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1141f4520 con 0x7fa1140700c0 2024-09-10T21:47:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.244+0000 7fa10b400640 1 -- 172.21.15.7:0/1330230263 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa100012d90 con 0x7fa1140700c0 2024-09-10T21:47:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.244+0000 7fa119000640 1 -- 172.21.15.7:0/1330230263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1140b39c0 con 0x7fa1140700c0 2024-09-10T21:47:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.245+0000 7fa119000640 1 -- 172.21.15.7:0/1330230263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa11415cc40 con 0x7fa1140700c0 2024-09-10T21:47:38.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.246+0000 7fa10b400640 1 -- 172.21.15.7:0/1330230263 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa10001a3b0 con 0x7fa1140700c0 2024-09-10T21:47:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.247+0000 7fa10b400640 1 --2- 172.21.15.7:0/1330230263 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa0f4077f70 0x7fa0f407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:38.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.247+0000 7fa10b400640 1 -- 172.21.15.7:0/1330230263 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 4857+0+0 (secure 0 0 0) 0x7fa100099900 con 0x7fa1140700c0 2024-09-10T21:47:38.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.248+0000 7fa113400640 1 --2- 172.21.15.7:0/1330230263 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa0f4077f70 0x7fa0f407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:38.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.249+0000 7fa113400640 1 --2- 172.21.15.7:0/1330230263 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa0f4077f70 0x7fa0f407a430 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa104034000 tx=0x7fa104004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:38.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.251+0000 7fa10b400640 1 -- 172.21.15.7:0/1330230263 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fa100020030 con 0x7fa1140700c0 2024-09-10T21:47:38.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:38.446+0000 7fa119000640 1 -- 172.21.15.7:0/1330230263 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fa11415b5e0 con 0x7fa1140700c0 2024-09-10T21:47:39.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.208+0000 7fa10b400640 1 -- 172.21.15.7:0/1330230263 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7fa100062740 con 0x7fa1140700c0 2024-09-10T21:47:39.210 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-09-10T21:47:39.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.212+0000 7fa119000640 1 -- 172.21.15.7:0/1330230263 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa0f4077f70 msgr2=0x7fa0f407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:39.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.212+0000 7fa119000640 1 --2- 172.21.15.7:0/1330230263 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa0f4077f70 0x7fa0f407a430 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa104034000 tx=0x7fa104004730 comp rx=0 tx=0).stop 2024-09-10T21:47:39.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.212+0000 7fa119000640 1 -- 172.21.15.7:0/1330230263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1140700c0 msgr2=0x7fa1140b2880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.212+0000 7fa119000640 1 --2- 172.21.15.7:0/1330230263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1140700c0 0x7fa1140b2880 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fa10000c460 tx=0x7fa10000c920 comp rx=0 tx=0).stop 2024-09-10T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.213+0000 7fa119000640 1 -- 172.21.15.7:0/1330230263 shutdown_connections 2024-09-10T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.213+0000 7fa119000640 1 --2- 172.21.15.7:0/1330230263 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa0f4077f70 0x7fa0f407a430 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.213+0000 7fa119000640 1 --2- 172.21.15.7:0/1330230263 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa1140cb760 0x7fa1140b4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.213+0000 7fa119000640 1 --2- 172.21.15.7:0/1330230263 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa1140700c0 0x7fa1140b2880 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.213+0000 7fa119000640 1 --2- 172.21.15.7:0/1330230263 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa11406f170 0x7fa1141575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:39.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.213+0000 7fa119000640 1 -- 172.21.15.7:0/1330230263 >> 172.21.15.7:0/1330230263 conn(0x7fa1140b1580 msgr2=0x7fa11406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.213+0000 7fa119000640 1 -- 172.21.15.7:0/1330230263 shutdown_connections 2024-09-10T21:47:39.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.214+0000 7fa119000640 1 -- 172.21.15.7:0/1330230263 wait complete. 2024-09-10T21:47:39.238 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:47:39.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.409+0000 7f1e9fc00640 1 -- 172.21.15.7:0/1548353331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e980ae000 msgr2=0x7f1e980cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:39.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.409+0000 7f1e9fc00640 1 --2- 172.21.15.7:0/1548353331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e980ae000 0x7f1e980cdc30 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f1e980cd460 tx=0x7f1e9002a980 comp rx=0 tx=0).stop 2024-09-10T21:47:39.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.409+0000 7f1e9fc00640 1 -- 172.21.15.7:0/1548353331 shutdown_connections 2024-09-10T21:47:39.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.409+0000 7f1e9fc00640 1 --2- 172.21.15.7:0/1548353331 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e980ae000 0x7f1e980cdc30 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:39.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.409+0000 7f1e9fc00640 1 --2- 172.21.15.7:0/1548353331 >> v2:172.21.15.133:3300/0 conn(0x7f1e98158220 0x7f1e980cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:39.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.409+0000 7f1e9fc00640 1 -- 172.21.15.7:0/1548353331 >> 172.21.15.7:0/1548353331 conn(0x7f1e980b16d0 msgr2=0x7f1e980b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:39.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.410+0000 7f1e9fc00640 1 -- 172.21.15.7:0/1548353331 shutdown_connections 2024-09-10T21:47:39.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.410+0000 7f1e9fc00640 1 -- 172.21.15.7:0/1548353331 wait complete. 2024-09-10T21:47:39.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.411+0000 7f1e9fc00640 1 Processor -- start 2024-09-10T21:47:39.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.411+0000 7f1e9fc00640 1 -- start start 2024-09-10T21:47:39.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.412+0000 7f1e9fc00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e980ae000 0x7f1e981f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:39.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.412+0000 7f1e9fc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98158220 0x7f1e980c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:39.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.412+0000 7f1e9fc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e98158600 0x7f1e980b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:39.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.412+0000 7f1e9fc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9806f9e0 con 0x7f1e98158220 2024-09-10T21:47:39.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.412+0000 7f1e9fc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9806f860 con 0x7f1e98158600 2024-09-10T21:47:39.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.412+0000 7f1e9fc00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f1e9806fb60 con 0x7f1e980ae000 2024-09-10T21:47:39.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.413+0000 7f1e8fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98158220 0x7f1e980c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:39.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.413+0000 7f1e8fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98158220 0x7f1e980c5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54174/0 (socket says 172.21.15.7:54174) 2024-09-10T21:47:39.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.413+0000 7f1e8fe00640 1 -- 172.21.15.7:0/4216655918 learned_addr learned my addr 172.21.15.7:0/4216655918 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:39.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.413+0000 7f1e9d200640 1 --2- 172.21.15.7:0/4216655918 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e980ae000 0x7f1e981f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:39.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.413+0000 7f1e9dc00640 1 --2- 172.21.15.7:0/4216655918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e98158600 0x7f1e980b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:39.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.413+0000 7f1e8fe00640 1 -- 172.21.15.7:0/4216655918 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e980ae000 msgr2=0x7f1e981f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:39.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.413+0000 7f1e8fe00640 1 --2- 172.21.15.7:0/4216655918 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e980ae000 0x7f1e981f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:39.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.413+0000 7f1e8fe00640 1 -- 172.21.15.7:0/4216655918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e98158600 msgr2=0x7f1e980b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:39.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.413+0000 7f1e8fe00640 1 --2- 172.21.15.7:0/4216655918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e98158600 0x7f1e980b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:39.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.413+0000 7f1e8fe00640 1 -- 172.21.15.7:0/4216655918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e980b01c0 con 0x7f1e98158220 2024-09-10T21:47:39.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.413+0000 7f1e8fe00640 1 --2- 172.21.15.7:0/4216655918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98158220 0x7f1e980c5cc0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000b410 tx=0x7f1e8000b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:39.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.414+0000 7f1e95600640 1 -- 172.21.15.7:0/4216655918 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80015020 con 0x7f1e98158220 2024-09-10T21:47:39.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.414+0000 7f1e95600640 1 -- 172.21.15.7:0/4216655918 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e80011940 con 0x7f1e98158220 2024-09-10T21:47:39.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.414+0000 7f1e9fc00640 1 -- 172.21.15.7:0/4216655918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e980c54c0 con 0x7f1e98158220 2024-09-10T21:47:39.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.414+0000 7f1e95600640 1 -- 172.21.15.7:0/4216655918 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e80011d90 con 0x7f1e98158220 2024-09-10T21:47:39.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.414+0000 7f1e9fc00640 1 -- 172.21.15.7:0/4216655918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e980c32c0 con 0x7f1e98158220 2024-09-10T21:47:39.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.415+0000 7f1e9fc00640 1 -- 172.21.15.7:0/4216655918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e98070090 con 0x7f1e98158220 2024-09-10T21:47:39.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.416+0000 7f1e95600640 1 -- 172.21.15.7:0/4216655918 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1e8001a400 con 0x7f1e98158220 2024-09-10T21:47:39.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.417+0000 7f1e95600640 1 --2- 172.21.15.7:0/4216655918 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1e74077f70 0x7f1e7407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:39.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.417+0000 7f1e95600640 1 -- 172.21.15.7:0/4216655918 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 5227+0+0 (secure 0 0 0) 0x7f1e80099d00 con 0x7f1e98158220 2024-09-10T21:47:39.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.418+0000 7f1e9d200640 1 --2- 172.21.15.7:0/4216655918 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1e74077f70 0x7f1e7407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:39.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.418+0000 7f1e9d200640 1 --2- 172.21.15.7:0/4216655918 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1e74077f70 0x7f1e7407a430 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009990 tx=0x7f1e88009110 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:39.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.422+0000 7f1e95600640 1 -- 172.21.15.7:0/4216655918 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1e8001f020 con 0x7f1e98158220 2024-09-10T21:47:39.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:39.619+0000 7f1e9fc00640 1 -- 172.21.15.7:0/4216655918 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1e980ce320 con 0x7f1e98158220 2024-09-10T21:47:40.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.215+0000 7f1e95600640 1 -- 172.21.15.7:0/4216655918 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) ==== 170+0+0 (secure 0 0 0) 0x7f1e800629d0 con 0x7f1e98158220 2024-09-10T21:47:40.217 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-09-10T21:47:40.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.219+0000 7f1e9fc00640 1 -- 172.21.15.7:0/4216655918 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1e74077f70 msgr2=0x7f1e7407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:40.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.219+0000 7f1e9fc00640 1 --2- 172.21.15.7:0/4216655918 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1e74077f70 0x7f1e7407a430 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009990 tx=0x7f1e88009110 comp rx=0 tx=0).stop 2024-09-10T21:47:40.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.219+0000 7f1e9fc00640 1 -- 172.21.15.7:0/4216655918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98158220 msgr2=0x7f1e980c5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:40.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.219+0000 7f1e9fc00640 1 --2- 172.21.15.7:0/4216655918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98158220 0x7f1e980c5cc0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000b410 tx=0x7f1e8000b810 comp rx=0 tx=0).stop 2024-09-10T21:47:40.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.219+0000 7f1e9fc00640 1 -- 172.21.15.7:0/4216655918 shutdown_connections 2024-09-10T21:47:40.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.219+0000 7f1e9fc00640 1 --2- 172.21.15.7:0/4216655918 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1e74077f70 0x7f1e7407a430 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:40.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.219+0000 7f1e9fc00640 1 --2- 172.21.15.7:0/4216655918 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e98158600 0x7f1e980b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:40.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.219+0000 7f1e9fc00640 1 --2- 172.21.15.7:0/4216655918 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e98158220 0x7f1e980c5cc0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:40.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.219+0000 7f1e9fc00640 1 --2- 172.21.15.7:0/4216655918 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e980ae000 0x7f1e981f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:40.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.219+0000 7f1e9fc00640 1 -- 172.21.15.7:0/4216655918 >> 172.21.15.7:0/4216655918 conn(0x7f1e980b16d0 msgr2=0x7f1e981546e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.219+0000 7f1e9fc00640 1 -- 172.21.15.7:0/4216655918 shutdown_connections 2024-09-10T21:47:40.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.220+0000 7f1e9fc00640 1 -- 172.21.15.7:0/4216655918 wait complete. 2024-09-10T21:47:40.253 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:47:40.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.418+0000 7f356a800640 1 -- 172.21.15.7:0/2134414772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35640d22b0 msgr2=0x7f35640d26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:40.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.418+0000 7f356a800640 1 --2- 172.21.15.7:0/2134414772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35640d22b0 0x7f35640d26f0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f3564069890 tx=0x7f355002a540 comp rx=0 tx=0).stop 2024-09-10T21:47:40.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.418+0000 7f356a800640 1 -- 172.21.15.7:0/2134414772 shutdown_connections 2024-09-10T21:47:40.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.418+0000 7f356a800640 1 --2- 172.21.15.7:0/2134414772 >> v2:172.21.15.133:3300/0 conn(0x7f35640b0a10 0x7f35640b0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:40.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.418+0000 7f356a800640 1 --2- 172.21.15.7:0/2134414772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35640d22b0 0x7f35640d26f0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:40.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.418+0000 7f356a800640 1 -- 172.21.15.7:0/2134414772 >> 172.21.15.7:0/2134414772 conn(0x7f35640d3e80 msgr2=0x7f35640d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:40.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.418+0000 7f356a800640 1 -- 172.21.15.7:0/2134414772 shutdown_connections 2024-09-10T21:47:40.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.418+0000 7f356a800640 1 -- 172.21.15.7:0/2134414772 wait complete. 2024-09-10T21:47:40.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.420+0000 7f356a800640 1 Processor -- start 2024-09-10T21:47:40.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.420+0000 7f356a800640 1 -- start start 2024-09-10T21:47:40.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.421+0000 7f356a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35640b0a10 0x7f356415cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:40.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.421+0000 7f356a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35640d22b0 0x7f35641f5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:40.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.421+0000 7f356a800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f356415df70 0x7f35640a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:40.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.421+0000 7f356a800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35640b2960 con 0x7f35640d22b0 2024-09-10T21:47:40.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.421+0000 7f356a800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f35640b27e0 con 0x7f35640b0a10 2024-09-10T21:47:40.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.421+0000 7f356a800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f35640b2ae0 con 0x7f356415df70 2024-09-10T21:47:40.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.421+0000 7f3568a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35640d22b0 0x7f35641f5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:40.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.421+0000 7f3568a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35640d22b0 0x7f35641f5150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54186/0 (socket says 172.21.15.7:54186) 2024-09-10T21:47:40.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.421+0000 7f3568a00640 1 -- 172.21.15.7:0/441653922 learned_addr learned my addr 172.21.15.7:0/441653922 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:40.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.421+0000 7f3569400640 1 --2- 172.21.15.7:0/441653922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35640b0a10 0x7f356415cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:40.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.421+0000 7f3569e00640 1 --2- 172.21.15.7:0/441653922 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f356415df70 0x7f35640a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:40.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.421+0000 7f3568a00640 1 -- 172.21.15.7:0/441653922 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f356415df70 msgr2=0x7f35640a14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:40.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.421+0000 7f3568a00640 1 --2- 172.21.15.7:0/441653922 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f356415df70 0x7f35640a14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:40.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.421+0000 7f3568a00640 1 -- 172.21.15.7:0/441653922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35640b0a10 msgr2=0x7f356415cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:40.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.421+0000 7f3568a00640 1 --2- 172.21.15.7:0/441653922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35640b0a10 0x7f356415cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:40.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.421+0000 7f3568a00640 1 -- 172.21.15.7:0/441653922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35640cb900 con 0x7f35640d22b0 2024-09-10T21:47:40.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.421+0000 7f3569e00640 1 --2- 172.21.15.7:0/441653922 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f356415df70 0x7f35640a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:40.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.422+0000 7f3568a00640 1 --2- 172.21.15.7:0/441653922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35640d22b0 0x7f35641f5150 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3550009780 tx=0x7f3550032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:40.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.422+0000 7f3561600640 1 -- 172.21.15.7:0/441653922 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3550040070 con 0x7f35640d22b0 2024-09-10T21:47:40.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.422+0000 7f3561600640 1 -- 172.21.15.7:0/441653922 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3550005310 con 0x7f35640d22b0 2024-09-10T21:47:40.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.422+0000 7f356a800640 1 -- 172.21.15.7:0/441653922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35640cbc40 con 0x7f35640d22b0 2024-09-10T21:47:40.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.422+0000 7f3561600640 1 -- 172.21.15.7:0/441653922 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35500047a0 con 0x7f35640d22b0 2024-09-10T21:47:40.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.422+0000 7f356a800640 1 -- 172.21.15.7:0/441653922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35641f48e0 con 0x7f35640d22b0 2024-09-10T21:47:40.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.423+0000 7f356a800640 1 -- 172.21.15.7:0/441653922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35640b0e30 con 0x7f35640d22b0 2024-09-10T21:47:40.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.424+0000 7f3561600640 1 -- 172.21.15.7:0/441653922 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f355004c030 con 0x7f35640d22b0 2024-09-10T21:47:40.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.425+0000 7f3561600640 1 --2- 172.21.15.7:0/441653922 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3544077f70 0x7f354407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:40.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.425+0000 7f3561600640 1 -- 172.21.15.7:0/441653922 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5609+0+0 (secure 0 0 0) 0x7f35500bafd0 con 0x7f35640d22b0 2024-09-10T21:47:40.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.426+0000 7f3569400640 1 --2- 172.21.15.7:0/441653922 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3544077f70 0x7f354407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:40.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.427+0000 7f3569400640 1 --2- 172.21.15.7:0/441653922 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3544077f70 0x7f354407a430 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3558007fd0 tx=0x7f3558001370 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:40.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.430+0000 7f3561600640 1 -- 172.21.15.7:0/441653922 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f35640cbc40 con 0x7f35640d22b0 2024-09-10T21:47:40.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:40.662+0000 7f356a800640 1 -- 172.21.15.7:0/441653922 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f35640b86a0 con 0x7f35640d22b0 2024-09-10T21:47:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.218+0000 7f3561600640 1 -- 172.21.15.7:0/441653922 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v59) ==== 174+0+0 (secure 0 0 0) 0x7f3550083b20 con 0x7f35640d22b0 2024-09-10T21:47:41.219 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 12 and data pool 13 2024-09-10T21:47:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.222+0000 7f356a800640 1 -- 172.21.15.7:0/441653922 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3544077f70 msgr2=0x7f354407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.222+0000 7f356a800640 1 --2- 172.21.15.7:0/441653922 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3544077f70 0x7f354407a430 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3558007fd0 tx=0x7f3558001370 comp rx=0 tx=0).stop 2024-09-10T21:47:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.222+0000 7f356a800640 1 -- 172.21.15.7:0/441653922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35640d22b0 msgr2=0x7f35641f5150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:41.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.222+0000 7f356a800640 1 --2- 172.21.15.7:0/441653922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35640d22b0 0x7f35641f5150 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f3550009780 tx=0x7f3550032c10 comp rx=0 tx=0).stop 2024-09-10T21:47:41.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.223+0000 7f356a800640 1 -- 172.21.15.7:0/441653922 shutdown_connections 2024-09-10T21:47:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.223+0000 7f356a800640 1 --2- 172.21.15.7:0/441653922 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3544077f70 0x7f354407a430 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.223+0000 7f356a800640 1 --2- 172.21.15.7:0/441653922 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f356415df70 0x7f35640a14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.223+0000 7f356a800640 1 --2- 172.21.15.7:0/441653922 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35640d22b0 0x7f35641f5150 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.223+0000 7f356a800640 1 --2- 172.21.15.7:0/441653922 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35640b0a10 0x7f356415cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.223+0000 7f356a800640 1 -- 172.21.15.7:0/441653922 >> 172.21.15.7:0/441653922 conn(0x7f35640d3e80 msgr2=0x7f35640b7d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.224+0000 7f356a800640 1 -- 172.21.15.7:0/441653922 shutdown_connections 2024-09-10T21:47:41.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.224+0000 7f356a800640 1 -- 172.21.15.7:0/441653922 wait complete. 2024-09-10T21:47:41.274 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:47:41.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.436+0000 7f5b67200640 1 -- 172.21.15.7:0/2796671375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b6015bc90 msgr2=0x7f5b600ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:41.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.436+0000 7f5b67200640 1 --2- 172.21.15.7:0/2796671375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b6015bc90 0x7f5b600ccef0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f5b600c3650 tx=0x7f5b5002a4b0 comp rx=0 tx=0).stop 2024-09-10T21:47:41.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.436+0000 7f5b67200640 1 -- 172.21.15.7:0/2796671375 shutdown_connections 2024-09-10T21:47:41.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.436+0000 7f5b67200640 1 --2- 172.21.15.7:0/2796671375 >> v2:172.21.15.133:3300/0 conn(0x7f5b6015fda0 0x7f5b60160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:41.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.436+0000 7f5b67200640 1 --2- 172.21.15.7:0/2796671375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b6015bc90 0x7f5b600ccef0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:41.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.436+0000 7f5b67200640 1 -- 172.21.15.7:0/2796671375 >> 172.21.15.7:0/2796671375 conn(0x7f5b600b16d0 msgr2=0x7f5b600b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:41.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.436+0000 7f5b67200640 1 -- 172.21.15.7:0/2796671375 shutdown_connections 2024-09-10T21:47:41.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.437+0000 7f5b67200640 1 -- 172.21.15.7:0/2796671375 wait complete. 2024-09-10T21:47:41.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.438+0000 7f5b67200640 1 Processor -- start 2024-09-10T21:47:41.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.438+0000 7f5b67200640 1 -- start start 2024-09-10T21:47:41.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.439+0000 7f5b67200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b600cd750 0x7f5b6014d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:41.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.439+0000 7f5b67200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6015bc90 0x7f5b60150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:41.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.439+0000 7f5b67200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b6015fda0 0x7f5b600ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:41.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.439+0000 7f5b67200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6006fe70 con 0x7f5b6015fda0 2024-09-10T21:47:41.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.439+0000 7f5b67200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5b6006fcf0 con 0x7f5b6015bc90 2024-09-10T21:47:41.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.439+0000 7f5b67200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f5b6006fff0 con 0x7f5b600cd750 2024-09-10T21:47:41.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.439+0000 7f5b65200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b6015fda0 0x7f5b600ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:41.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.439+0000 7f5b65200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b6015fda0 0x7f5b600ad8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54190/0 (socket says 172.21.15.7:54190) 2024-09-10T21:47:41.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.439+0000 7f5b65200640 1 -- 172.21.15.7:0/3869823704 learned_addr learned my addr 172.21.15.7:0/3869823704 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:41.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.439+0000 7f5b5f400640 1 --2- 172.21.15.7:0/3869823704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6015bc90 0x7f5b60150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:41.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.439+0000 7f5b5fe00640 1 --2- 172.21.15.7:0/3869823704 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b600cd750 0x7f5b6014d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:41.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.440+0000 7f5b65200640 1 -- 172.21.15.7:0/3869823704 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b600cd750 msgr2=0x7f5b6014d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:41.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.440+0000 7f5b65200640 1 --2- 172.21.15.7:0/3869823704 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b600cd750 0x7f5b6014d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:41.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.440+0000 7f5b65200640 1 -- 172.21.15.7:0/3869823704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6015bc90 msgr2=0x7f5b60150930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:41.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.440+0000 7f5b65200640 1 --2- 172.21.15.7:0/3869823704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6015bc90 0x7f5b60150930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:41.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.440+0000 7f5b65200640 1 -- 172.21.15.7:0/3869823704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b600af8a0 con 0x7f5b6015fda0 2024-09-10T21:47:41.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.440+0000 7f5b65200640 1 --2- 172.21.15.7:0/3869823704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b6015fda0 0x7f5b600ad8f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800e8b0 tx=0x7f5b5800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:41.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.441+0000 7f5b57e00640 1 -- 172.21.15.7:0/3869823704 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58004d20 con 0x7f5b6015fda0 2024-09-10T21:47:41.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.441+0000 7f5b57e00640 1 -- 172.21.15.7:0/3869823704 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b5800ca60 con 0x7f5b6015fda0 2024-09-10T21:47:41.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.441+0000 7f5b67200640 1 -- 172.21.15.7:0/3869823704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b601f8750 con 0x7f5b6015fda0 2024-09-10T21:47:41.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.441+0000 7f5b57e00640 1 -- 172.21.15.7:0/3869823704 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b58005020 con 0x7f5b6015fda0 2024-09-10T21:47:41.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.441+0000 7f5b67200640 1 -- 172.21.15.7:0/3869823704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b601f8d10 con 0x7f5b6015fda0 2024-09-10T21:47:41.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.442+0000 7f5b67200640 1 -- 172.21.15.7:0/3869823704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b600704a0 con 0x7f5b6015fda0 2024-09-10T21:47:41.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.443+0000 7f5b57e00640 1 -- 172.21.15.7:0/3869823704 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5b5800f040 con 0x7f5b6015fda0 2024-09-10T21:47:41.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.444+0000 7f5b57e00640 1 --2- 172.21.15.7:0/3869823704 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5b40077f70 0x7f5b4007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:41.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.444+0000 7f5b57e00640 1 -- 172.21.15.7:0/3869823704 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f5b5809a230 con 0x7f5b6015fda0 2024-09-10T21:47:41.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.444+0000 7f5b5fe00640 1 --2- 172.21.15.7:0/3869823704 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5b40077f70 0x7f5b4007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:41.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.445+0000 7f5b5fe00640 1 --2- 172.21.15.7:0/3869823704 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5b40077f70 0x7f5b4007a430 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5b50034000 tx=0x7f5b50004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:41.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.448+0000 7f5b57e00640 1 -- 172.21.15.7:0/3869823704 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5b58016030 con 0x7f5b6015fda0 2024-09-10T21:47:41.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.642+0000 7f5b67200640 1 -- 172.21.15.7:0/3869823704 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b600afa70 con 0x7f5b6015fda0 2024-09-10T21:47:41.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.643+0000 7f5b57e00640 1 -- 172.21.15.7:0/3869823704 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16697 (secure 0 0 0) 0x7f5b580626d0 con 0x7f5b6015fda0 2024-09-10T21:47:41.644 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:41.645 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:47:40.664125+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:47:38.456723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T21:47:39.629851+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:47:41.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.645+0000 7f5b67200640 1 -- 172.21.15.7:0/3869823704 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5b40077f70 msgr2=0x7f5b4007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:41.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.645+0000 7f5b67200640 1 --2- 172.21.15.7:0/3869823704 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5b40077f70 0x7f5b4007a430 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5b50034000 tx=0x7f5b50004730 comp rx=0 tx=0).stop 2024-09-10T21:47:41.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.645+0000 7f5b67200640 1 -- 172.21.15.7:0/3869823704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b6015fda0 msgr2=0x7f5b600ad8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:41.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.645+0000 7f5b67200640 1 --2- 172.21.15.7:0/3869823704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b6015fda0 0x7f5b600ad8f0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800e8b0 tx=0x7f5b5800ed70 comp rx=0 tx=0).stop 2024-09-10T21:47:41.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.646+0000 7f5b67200640 1 -- 172.21.15.7:0/3869823704 shutdown_connections 2024-09-10T21:47:41.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.646+0000 7f5b67200640 1 --2- 172.21.15.7:0/3869823704 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5b40077f70 0x7f5b4007a430 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:41.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.646+0000 7f5b67200640 1 --2- 172.21.15.7:0/3869823704 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b6015bc90 0x7f5b60150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:41.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.646+0000 7f5b67200640 1 --2- 172.21.15.7:0/3869823704 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b6015fda0 0x7f5b600ad8f0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:41.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.646+0000 7f5b67200640 1 --2- 172.21.15.7:0/3869823704 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b600cd750 0x7f5b6014d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:41.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.646+0000 7f5b67200640 1 -- 172.21.15.7:0/3869823704 >> 172.21.15.7:0/3869823704 conn(0x7f5b600b16d0 msgr2=0x7f5b600c56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:41.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.646+0000 7f5b67200640 1 -- 172.21.15.7:0/3869823704 shutdown_connections 2024-09-10T21:47:41.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.646+0000 7f5b67200640 1 -- 172.21.15.7:0/3869823704 wait complete. 2024-09-10T21:47:41.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:47:41.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.825+0000 7f3826c00640 1 -- 172.21.15.7:0/1967173558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3820145540 msgr2=0x7f38200b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.825+0000 7f3826c00640 1 --2- 172.21.15.7:0/1967173558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3820145540 0x7f38200b4580 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f38200af640 tx=0x7f381402a560 comp rx=0 tx=0).stop 2024-09-10T21:47:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.825+0000 7f3826c00640 1 -- 172.21.15.7:0/1967173558 shutdown_connections 2024-09-10T21:47:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.825+0000 7f3826c00640 1 --2- 172.21.15.7:0/1967173558 >> v2:172.21.15.133:3300/0 conn(0x7f38200b2f80 0x7f382015f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.825+0000 7f3826c00640 1 --2- 172.21.15.7:0/1967173558 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38200b4ac0 0x7f38200b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.825+0000 7f3826c00640 1 --2- 172.21.15.7:0/1967173558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3820145540 0x7f38200b4580 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.825+0000 7f3826c00640 1 -- 172.21.15.7:0/1967173558 >> 172.21.15.7:0/1967173558 conn(0x7f38200b1d70 msgr2=0x7f38200cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.825+0000 7f3826c00640 1 -- 172.21.15.7:0/1967173558 shutdown_connections 2024-09-10T21:47:41.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.825+0000 7f3826c00640 1 -- 172.21.15.7:0/1967173558 wait complete. 2024-09-10T21:47:41.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.827+0000 7f3826c00640 1 Processor -- start 2024-09-10T21:47:41.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.827+0000 7f3826c00640 1 -- start start 2024-09-10T21:47:41.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.828+0000 7f3826c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38200b2f80 0x7f38200afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:41.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.828+0000 7f3826c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38200b4ac0 0x7f3820141060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:41.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.828+0000 7f3826c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3820145540 0x7f38201415a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:41.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.828+0000 7f3826c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f382015fd10 con 0x7f38200b4ac0 2024-09-10T21:47:41.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.828+0000 7f3826c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3820145a30 con 0x7f3820145540 2024-09-10T21:47:41.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.828+0000 7f3826c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f382015fe90 con 0x7f38200b2f80 2024-09-10T21:47:41.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.828+0000 7f3824e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38200b4ac0 0x7f3820141060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:41.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.828+0000 7f3825800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38200b2f80 0x7f38200afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:41.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.828+0000 7f3825800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38200b2f80 0x7f38200afea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:60430/0 (socket says 172.21.15.7:60430) 2024-09-10T21:47:41.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.828+0000 7f3824e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38200b4ac0 0x7f3820141060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54214/0 (socket says 172.21.15.7:54214) 2024-09-10T21:47:41.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.828+0000 7f3825800640 1 -- 172.21.15.7:0/1738010521 learned_addr learned my addr 172.21.15.7:0/1738010521 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:41.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.828+0000 7f3826200640 1 --2- 172.21.15.7:0/1738010521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3820145540 0x7f38201415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:41.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.829+0000 7f3824e00640 1 -- 172.21.15.7:0/1738010521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38200b2f80 msgr2=0x7f38200afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:41.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.829+0000 7f3824e00640 1 --2- 172.21.15.7:0/1738010521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38200b2f80 0x7f38200afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:41.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.829+0000 7f3824e00640 1 -- 172.21.15.7:0/1738010521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3820145540 msgr2=0x7f38201415a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:41.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.829+0000 7f3824e00640 1 --2- 172.21.15.7:0/1738010521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3820145540 0x7f38201415a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:41.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.829+0000 7f3824e00640 1 -- 172.21.15.7:0/1738010521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38201f2580 con 0x7f38200b4ac0 2024-09-10T21:47:41.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.829+0000 7f3824e00640 1 --2- 172.21.15.7:0/1738010521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38200b4ac0 0x7f3820141060 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f380c00b3f0 tx=0x7f380c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:41.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.829+0000 7f381d600640 1 -- 172.21.15.7:0/1738010521 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c015020 con 0x7f38200b4ac0 2024-09-10T21:47:41.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.829+0000 7f381d600640 1 -- 172.21.15.7:0/1738010521 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f380c011920 con 0x7f38200b4ac0 2024-09-10T21:47:41.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.829+0000 7f3826c00640 1 -- 172.21.15.7:0/1738010521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38201f2840 con 0x7f38200b4ac0 2024-09-10T21:47:41.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.829+0000 7f381d600640 1 -- 172.21.15.7:0/1738010521 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f380c011d70 con 0x7f38200b4ac0 2024-09-10T21:47:41.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.829+0000 7f3826c00640 1 -- 172.21.15.7:0/1738010521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38201f2e00 con 0x7f38200b4ac0 2024-09-10T21:47:41.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.830+0000 7f3826c00640 1 -- 172.21.15.7:0/1738010521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38201f34c0 con 0x7f38200b4ac0 2024-09-10T21:47:41.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.832+0000 7f381d600640 1 -- 172.21.15.7:0/1738010521 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f380c01a400 con 0x7f38200b4ac0 2024-09-10T21:47:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.832+0000 7f381d600640 1 --2- 172.21.15.7:0/1738010521 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3800077fa0 0x7f380007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:41.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.833+0000 7f381d600640 1 -- 172.21.15.7:0/1738010521 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5709+0+0 (secure 0 0 0) 0x7f380c099e50 con 0x7f38200b4ac0 2024-09-10T21:47:41.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.833+0000 7f3825800640 1 --2- 172.21.15.7:0/1738010521 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3800077fa0 0x7f380007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:41.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.834+0000 7f3825800640 1 --2- 172.21.15.7:0/1738010521 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3800077fa0 0x7f380007a460 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3814009230 tx=0x7f3814004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:41.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:41.837+0000 7f381d600640 1 -- 172.21.15.7:0/1738010521 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f38201f34c0 con 0x7f38200b4ac0 2024-09-10T21:47:42.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.032+0000 7f3826c00640 1 -- 172.21.15.7:0/1738010521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38201603d0 con 0x7f38200b4ac0 2024-09-10T21:47:42.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.033+0000 7f381d600640 1 -- 172.21.15.7:0/1738010521 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+16697 (secure 0 0 0) 0x7f38201603d0 con 0x7f38200b4ac0 2024-09-10T21:47:42.034 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:42.035 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":84,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:47:40.664125+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:47:38.456723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T21:47:39.629851+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:47:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.036+0000 7f3826c00640 1 -- 172.21.15.7:0/1738010521 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3800077fa0 msgr2=0x7f380007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.036+0000 7f3826c00640 1 --2- 172.21.15.7:0/1738010521 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3800077fa0 0x7f380007a460 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3814009230 tx=0x7f3814004730 comp rx=0 tx=0).stop 2024-09-10T21:47:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.036+0000 7f3826c00640 1 -- 172.21.15.7:0/1738010521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38200b4ac0 msgr2=0x7f3820141060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.036+0000 7f3826c00640 1 --2- 172.21.15.7:0/1738010521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38200b4ac0 0x7f3820141060 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f380c00b3f0 tx=0x7f380c00b8b0 comp rx=0 tx=0).stop 2024-09-10T21:47:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.036+0000 7f3826c00640 1 -- 172.21.15.7:0/1738010521 shutdown_connections 2024-09-10T21:47:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.036+0000 7f3826c00640 1 --2- 172.21.15.7:0/1738010521 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3800077fa0 0x7f380007a460 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.036+0000 7f3826c00640 1 --2- 172.21.15.7:0/1738010521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3820145540 0x7f38201415a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.036+0000 7f3826c00640 1 --2- 172.21.15.7:0/1738010521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38200b4ac0 0x7f3820141060 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.036+0000 7f3826c00640 1 --2- 172.21.15.7:0/1738010521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38200b2f80 0x7f38200afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.036+0000 7f3826c00640 1 -- 172.21.15.7:0/1738010521 >> 172.21.15.7:0/1738010521 conn(0x7f38200b1d70 msgr2=0x7f38200b4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.036+0000 7f3826c00640 1 -- 172.21.15.7:0/1738010521 shutdown_connections 2024-09-10T21:47:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.036+0000 7f3826c00640 1 -- 172.21.15.7:0/1738010521 wait complete. 2024-09-10T21:47:42.053 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:47:42.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.215+0000 7f97e1400640 1 -- 172.21.15.7:0/2266436324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97dc156290 msgr2=0x7f97dc0ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:42.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.215+0000 7f97e1400640 1 --2- 172.21.15.7:0/2266436324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97dc156290 0x7f97dc0ca270 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f97dc0708e0 tx=0x7f97d402a8b0 comp rx=0 tx=0).stop 2024-09-10T21:47:42.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.215+0000 7f97e1400640 1 -- 172.21.15.7:0/2266436324 shutdown_connections 2024-09-10T21:47:42.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.215+0000 7f97e1400640 1 --2- 172.21.15.7:0/2266436324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97dc156290 0x7f97dc0ca270 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:42.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.215+0000 7f97e1400640 1 --2- 172.21.15.7:0/2266436324 >> v2:172.21.15.133:3300/0 conn(0x7f97dc074090 0x7f97dc160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:42.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.215+0000 7f97e1400640 1 -- 172.21.15.7:0/2266436324 >> 172.21.15.7:0/2266436324 conn(0x7f97dc0b16d0 msgr2=0x7f97dc0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:42.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.215+0000 7f97e1400640 1 -- 172.21.15.7:0/2266436324 shutdown_connections 2024-09-10T21:47:42.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.215+0000 7f97e1400640 1 -- 172.21.15.7:0/2266436324 wait complete. 2024-09-10T21:47:42.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.217+0000 7f97e1400640 1 Processor -- start 2024-09-10T21:47:42.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.217+0000 7f97e1400640 1 -- start start 2024-09-10T21:47:42.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.218+0000 7f97e1400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97dc06fcf0 0x7f97dc1456f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:42.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.218+0000 7f97e1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97dc074090 0x7f97dc0c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:42.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.218+0000 7f97e1400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97dc156290 0x7f97dc1f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:42.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.218+0000 7f97e1400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc0cab40 con 0x7f97dc074090 2024-09-10T21:47:42.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.218+0000 7f97e1400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f97dc0ca9c0 con 0x7f97dc156290 2024-09-10T21:47:42.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.218+0000 7f97e1400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f97dc0cacc0 con 0x7f97dc06fcf0 2024-09-10T21:47:42.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.218+0000 7f97d9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97dc074090 0x7f97dc0c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:42.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.218+0000 7f97d9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97dc074090 0x7f97dc0c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54228/0 (socket says 172.21.15.7:54228) 2024-09-10T21:47:42.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.218+0000 7f97d9e00640 1 -- 172.21.15.7:0/685768999 learned_addr learned my addr 172.21.15.7:0/685768999 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:42.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.218+0000 7f97da800640 1 --2- 172.21.15.7:0/685768999 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97dc06fcf0 0x7f97dc1456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:42.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.218+0000 7f97db200640 1 --2- 172.21.15.7:0/685768999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97dc156290 0x7f97dc1f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:42.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.219+0000 7f97d9e00640 1 -- 172.21.15.7:0/685768999 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97dc06fcf0 msgr2=0x7f97dc1456f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:42.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.219+0000 7f97d9e00640 1 --2- 172.21.15.7:0/685768999 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97dc06fcf0 0x7f97dc1456f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:42.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.219+0000 7f97d9e00640 1 -- 172.21.15.7:0/685768999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97dc156290 msgr2=0x7f97dc1f5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:42.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.219+0000 7f97d9e00640 1 --2- 172.21.15.7:0/685768999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97dc156290 0x7f97dc1f5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:42.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.219+0000 7f97d9e00640 1 -- 172.21.15.7:0/685768999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97dc1f64d0 con 0x7f97dc074090 2024-09-10T21:47:42.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.219+0000 7f97db200640 1 --2- 172.21.15.7:0/685768999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97dc156290 0x7f97dc1f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:42.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.219+0000 7f97d9e00640 1 --2- 172.21.15.7:0/685768999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97dc074090 0x7f97dc0c5fd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f97c800c500 tx=0x7f97c800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:42.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.219+0000 7f97d2a00640 1 -- 172.21.15.7:0/685768999 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c8018070 con 0x7f97dc074090 2024-09-10T21:47:42.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.219+0000 7f97e1400640 1 -- 172.21.15.7:0/685768999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97dc1f67b0 con 0x7f97dc074090 2024-09-10T21:47:42.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.219+0000 7f97e1400640 1 -- 172.21.15.7:0/685768999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97dc1451d0 con 0x7f97dc074090 2024-09-10T21:47:42.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.220+0000 7f97e1400640 1 -- 172.21.15.7:0/685768999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97dc06acb0 con 0x7f97dc074090 2024-09-10T21:47:42.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.222+0000 7f97d2a00640 1 -- 172.21.15.7:0/685768999 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97c80035f0 con 0x7f97dc074090 2024-09-10T21:47:42.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.222+0000 7f97d2a00640 1 -- 172.21.15.7:0/685768999 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97c8022060 con 0x7f97dc074090 2024-09-10T21:47:42.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.224+0000 7f97d2a00640 1 -- 172.21.15.7:0/685768999 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f97c8025080 con 0x7f97dc074090 2024-09-10T21:47:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.226+0000 7f97d2a00640 1 --2- 172.21.15.7:0/685768999 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f97bc077f70 0x7f97bc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:42.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.226+0000 7f97d2a00640 1 -- 172.21.15.7:0/685768999 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f97c8099a20 con 0x7f97dc074090 2024-09-10T21:47:42.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.226+0000 7f97da800640 1 --2- 172.21.15.7:0/685768999 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f97bc077f70 0x7f97bc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:42.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.227+0000 7f97da800640 1 --2- 172.21.15.7:0/685768999 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f97bc077f70 0x7f97bc07a430 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009bd0 tx=0x7f97cc005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:42.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.228+0000 7f97d2a00640 1 -- 172.21.15.7:0/685768999 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f97dc06acb0 con 0x7f97dc074090 2024-09-10T21:47:42.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:42.465+0000 7f97e1400640 1 -- 172.21.15.7:0/685768999 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f97dc0cb1e0 con 0x7f97dc074090 2024-09-10T21:47:43.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.248+0000 7f97d2a00640 1 -- 172.21.15.7:0/685768999 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v62) ==== 118+0+0 (secure 0 0 0) 0x7f97c8061ec0 con 0x7f97dc074090 2024-09-10T21:47:43.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.251+0000 7f97e1400640 1 -- 172.21.15.7:0/685768999 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f97bc077f70 msgr2=0x7f97bc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:43.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.251+0000 7f97e1400640 1 --2- 172.21.15.7:0/685768999 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f97bc077f70 0x7f97bc07a430 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f97cc009bd0 tx=0x7f97cc005f90 comp rx=0 tx=0).stop 2024-09-10T21:47:43.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.251+0000 7f97e1400640 1 -- 172.21.15.7:0/685768999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97dc074090 msgr2=0x7f97dc0c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:43.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.251+0000 7f97e1400640 1 --2- 172.21.15.7:0/685768999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97dc074090 0x7f97dc0c5fd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f97c800c500 tx=0x7f97c800c9c0 comp rx=0 tx=0).stop 2024-09-10T21:47:43.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.251+0000 7f97e1400640 1 -- 172.21.15.7:0/685768999 shutdown_connections 2024-09-10T21:47:43.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.251+0000 7f97e1400640 1 --2- 172.21.15.7:0/685768999 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f97bc077f70 0x7f97bc07a430 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:43.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.251+0000 7f97e1400640 1 --2- 172.21.15.7:0/685768999 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97dc156290 0x7f97dc1f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:43.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.251+0000 7f97e1400640 1 --2- 172.21.15.7:0/685768999 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97dc074090 0x7f97dc0c5fd0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:43.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.251+0000 7f97e1400640 1 --2- 172.21.15.7:0/685768999 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97dc06fcf0 0x7f97dc1456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:43.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.251+0000 7f97e1400640 1 -- 172.21.15.7:0/685768999 >> 172.21.15.7:0/685768999 conn(0x7f97dc0b16d0 msgr2=0x7f97dc0cfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:43.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.251+0000 7f97e1400640 1 -- 172.21.15.7:0/685768999 shutdown_connections 2024-09-10T21:47:43.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.251+0000 7f97e1400640 1 -- 172.21.15.7:0/685768999 wait complete. 2024-09-10T21:47:43.270 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:47:43.271 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:47:43.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.437+0000 7f41c4200640 1 -- 172.21.15.7:0/3145609311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc06fcf0 msgr2=0x7f41bc1600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:43.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.437+0000 7f41c4200640 1 --2- 172.21.15.7:0/3145609311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc06fcf0 0x7f41bc1600d0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f41bc0cd510 tx=0x7f41a802a4d0 comp rx=0 tx=0).stop 2024-09-10T21:47:43.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.437+0000 7f41c4200640 1 -- 172.21.15.7:0/3145609311 shutdown_connections 2024-09-10T21:47:43.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.437+0000 7f41c4200640 1 --2- 172.21.15.7:0/3145609311 >> v2:172.21.15.133:3300/0 conn(0x7f41bc074450 0x7f41bc0ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:43.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.437+0000 7f41c4200640 1 --2- 172.21.15.7:0/3145609311 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc06fcf0 0x7f41bc1600d0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:43.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.437+0000 7f41c4200640 1 -- 172.21.15.7:0/3145609311 >> 172.21.15.7:0/3145609311 conn(0x7f41bc0b16d0 msgr2=0x7f41bc0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:43.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.437+0000 7f41c4200640 1 -- 172.21.15.7:0/3145609311 shutdown_connections 2024-09-10T21:47:43.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.438+0000 7f41c4200640 1 -- 172.21.15.7:0/3145609311 wait complete. 2024-09-10T21:47:43.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.439+0000 7f41c4200640 1 Processor -- start 2024-09-10T21:47:43.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.439+0000 7f41c4200640 1 -- start start 2024-09-10T21:47:43.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.440+0000 7f41c4200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc06fcf0 0x7f41bc150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:43.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.440+0000 7f41c4200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41bc074450 0x7f41bc151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:43.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.440+0000 7f41c4200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41bc155fb0 0x7f41bc1f4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:43.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.440+0000 7f41c4200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc0ca930 con 0x7f41bc06fcf0 2024-09-10T21:47:43.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.440+0000 7f41c4200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f41bc0ca7b0 con 0x7f41bc074450 2024-09-10T21:47:43.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.440+0000 7f41c4200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f41bc0caab0 con 0x7f41bc155fb0 2024-09-10T21:47:43.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.440+0000 7f41c1800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc06fcf0 0x7f41bc150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:43.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.441+0000 7f41c1800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc06fcf0 0x7f41bc150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:54246/0 (socket says 172.21.15.7:54246) 2024-09-10T21:47:43.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.441+0000 7f41c1800640 1 -- 172.21.15.7:0/3444073383 learned_addr learned my addr 172.21.15.7:0/3444073383 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:43.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.441+0000 7f41c0e00640 1 --2- 172.21.15.7:0/3444073383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41bc074450 0x7f41bc151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:43.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.441+0000 7f41c2200640 1 --2- 172.21.15.7:0/3444073383 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41bc155fb0 0x7f41bc1f4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:43.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.441+0000 7f41c1800640 1 -- 172.21.15.7:0/3444073383 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41bc155fb0 msgr2=0x7f41bc1f4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:43.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.441+0000 7f41c1800640 1 --2- 172.21.15.7:0/3444073383 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41bc155fb0 0x7f41bc1f4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:43.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.441+0000 7f41c1800640 1 -- 172.21.15.7:0/3444073383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41bc074450 msgr2=0x7f41bc151120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:43.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.441+0000 7f41c1800640 1 --2- 172.21.15.7:0/3444073383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41bc074450 0x7f41bc151120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:43.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.441+0000 7f41c1800640 1 -- 172.21.15.7:0/3444073383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41bc1f5880 con 0x7f41bc06fcf0 2024-09-10T21:47:43.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.441+0000 7f41c0e00640 1 --2- 172.21.15.7:0/3444073383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41bc074450 0x7f41bc151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:43.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.441+0000 7f41c2200640 1 --2- 172.21.15.7:0/3444073383 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41bc155fb0 0x7f41bc1f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:43.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.441+0000 7f41c1800640 1 --2- 172.21.15.7:0/3444073383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc06fcf0 0x7f41bc150be0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f41b000b420 tx=0x7f41b000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:43.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.441+0000 7f41b9600640 1 -- 172.21.15.7:0/3444073383 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b0011830 con 0x7f41bc06fcf0 2024-09-10T21:47:43.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.442+0000 7f41b9600640 1 -- 172.21.15.7:0/3444073383 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41b0016070 con 0x7f41bc06fcf0 2024-09-10T21:47:43.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.442+0000 7f41b9600640 1 -- 172.21.15.7:0/3444073383 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41b0011d60 con 0x7f41bc06fcf0 2024-09-10T21:47:43.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.442+0000 7f41c4200640 1 -- 172.21.15.7:0/3444073383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41bc0cb4d0 con 0x7f41bc06fcf0 2024-09-10T21:47:43.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.442+0000 7f41c4200640 1 -- 172.21.15.7:0/3444073383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41bc0cb7b0 con 0x7f41bc06fcf0 2024-09-10T21:47:43.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.443+0000 7f41c4200640 1 -- 172.21.15.7:0/3444073383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41bc0caf60 con 0x7f41bc06fcf0 2024-09-10T21:47:43.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.444+0000 7f41b9600640 1 -- 172.21.15.7:0/3444073383 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f41b001b460 con 0x7f41bc06fcf0 2024-09-10T21:47:43.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.445+0000 7f41b9600640 1 --2- 172.21.15.7:0/3444073383 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f419c077f70 0x7f419c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:43.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.445+0000 7f41b9600640 1 -- 172.21.15.7:0/3444073383 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f41b0099860 con 0x7f41bc06fcf0 2024-09-10T21:47:43.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.445+0000 7f41c0e00640 1 --2- 172.21.15.7:0/3444073383 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f419c077f70 0x7f419c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:43.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.446+0000 7f41c0e00640 1 --2- 172.21.15.7:0/3444073383 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f419c077f70 0x7f419c07a430 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f41a8033000 tx=0x7f41a8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:43.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.449+0000 7f41b9600640 1 -- 172.21.15.7:0/3444073383 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f41b0020020 con 0x7f41bc06fcf0 2024-09-10T21:47:43.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:43.679+0000 7f41c4200640 1 -- 172.21.15.7:0/3444073383 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f41bc0704c0 con 0x7f41bc06fcf0 2024-09-10T21:47:44.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.253+0000 7f41b9600640 1 -- 172.21.15.7:0/3444073383 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v63) ==== 122+0+0 (secure 0 0 0) 0x7f41b0062350 con 0x7f41bc06fcf0 2024-09-10T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.256+0000 7f41c4200640 1 -- 172.21.15.7:0/3444073383 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f419c077f70 msgr2=0x7f419c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.256+0000 7f41c4200640 1 --2- 172.21.15.7:0/3444073383 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f419c077f70 0x7f419c07a430 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f41a8033000 tx=0x7f41a8004730 comp rx=0 tx=0).stop 2024-09-10T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.256+0000 7f41c4200640 1 -- 172.21.15.7:0/3444073383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc06fcf0 msgr2=0x7f41bc150be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.256+0000 7f41c4200640 1 --2- 172.21.15.7:0/3444073383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc06fcf0 0x7f41bc150be0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f41b000b420 tx=0x7f41b000b8e0 comp rx=0 tx=0).stop 2024-09-10T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.257+0000 7f41c4200640 1 -- 172.21.15.7:0/3444073383 shutdown_connections 2024-09-10T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.257+0000 7f41c4200640 1 --2- 172.21.15.7:0/3444073383 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f419c077f70 0x7f419c07a430 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.257+0000 7f41c4200640 1 --2- 172.21.15.7:0/3444073383 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f41bc074450 0x7f41bc151120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.257+0000 7f41c4200640 1 --2- 172.21.15.7:0/3444073383 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f41bc155fb0 0x7f41bc1f4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.257+0000 7f41c4200640 1 --2- 172.21.15.7:0/3444073383 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f41bc06fcf0 0x7f41bc150be0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.257+0000 7f41c4200640 1 -- 172.21.15.7:0/3444073383 >> 172.21.15.7:0/3444073383 conn(0x7f41bc0b16d0 msgr2=0x7f41bc06e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:44.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.257+0000 7f41c4200640 1 -- 172.21.15.7:0/3444073383 shutdown_connections 2024-09-10T21:47:44.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.257+0000 7f41c4200640 1 -- 172.21.15.7:0/3444073383 wait complete. 2024-09-10T21:47:44.277 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:47:44.277 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:47:44.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.442+0000 7f3030a00640 1 -- 172.21.15.7:0/134712399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3010004020 msgr2=0x7f3010004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.442+0000 7f3030a00640 1 --2- 172.21.15.7:0/134712399 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3010004020 0x7f3010004800 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f3014006f60 tx=0x7f3010005830 comp rx=0 tx=0).stop 2024-09-10T21:47:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.442+0000 7f302a800640 1 -- 172.21.15.7:0/134712399 reap_dead start 2024-09-10T21:47:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.442+0000 7f3030a00640 1 -- 172.21.15.7:0/134712399 shutdown_connections 2024-09-10T21:47:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.442+0000 7f3030a00640 1 -- 172.21.15.7:0/134712399 >> 172.21.15.7:0/134712399 conn(0x7f302c0b16d0 msgr2=0x7f302c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.442+0000 7f3030a00640 1 -- 172.21.15.7:0/134712399 shutdown_connections 2024-09-10T21:47:44.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.442+0000 7f3030a00640 1 -- 172.21.15.7:0/134712399 wait complete. 2024-09-10T21:47:44.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.444+0000 7f3030a00640 1 Processor -- start 2024-09-10T21:47:44.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.444+0000 7f3030a00640 1 -- start start 2024-09-10T21:47:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.445+0000 7f3030a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3010004020 0x7f302c152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.445+0000 7f3030a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f302c152360 0x7f302c159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.445+0000 7f3030a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f302c152860 0x7f302c159c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.445+0000 7f3030a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f302c0cc710 con 0x7f302c152860 2024-09-10T21:47:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.445+0000 7f3030a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f302c0cc590 con 0x7f302c152360 2024-09-10T21:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.445+0000 7f3030a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f302c0cc890 con 0x7f3010004020 2024-09-10T21:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.445+0000 7f302a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f302c152860 0x7f302c159c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.445+0000 7f302a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f302c152860 0x7f302c159c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57694/0 (socket says 172.21.15.7:57694) 2024-09-10T21:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.445+0000 7f302a800640 1 -- 172.21.15.7:0/1872679320 learned_addr learned my addr 172.21.15.7:0/1872679320 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.445+0000 7f3021400640 1 --2- 172.21.15.7:0/1872679320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f302c152360 0x7f302c159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.445+0000 7f3029e00640 1 --2- 172.21.15.7:0/1872679320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3010004020 0x7f302c152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:44.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.446+0000 7f302a800640 1 -- 172.21.15.7:0/1872679320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3010004020 msgr2=0x7f302c152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.446+0000 7f302a800640 1 --2- 172.21.15.7:0/1872679320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3010004020 0x7f302c152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.446+0000 7f302a800640 1 -- 172.21.15.7:0/1872679320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f302c152360 msgr2=0x7f302c159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.446+0000 7f302a800640 1 --2- 172.21.15.7:0/1872679320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f302c152360 0x7f302c159740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.446+0000 7f302a800640 1 -- 172.21.15.7:0/1872679320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f302c0b58d0 con 0x7f302c152860 2024-09-10T21:47:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.446+0000 7f3021400640 1 --2- 172.21.15.7:0/1872679320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f302c152360 0x7f302c159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.446+0000 7f302a800640 1 --2- 172.21.15.7:0/1872679320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f302c152860 0x7f302c159c80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f302c15de40 tx=0x7f3024009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.446+0000 7f3022a00640 1 -- 172.21.15.7:0/1872679320 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302400fd40 con 0x7f302c152860 2024-09-10T21:47:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.446+0000 7f3022a00640 1 -- 172.21.15.7:0/1872679320 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3024004b20 con 0x7f302c152860 2024-09-10T21:47:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.446+0000 7f3030a00640 1 -- 172.21.15.7:0/1872679320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f302c0b1360 con 0x7f302c152860 2024-09-10T21:47:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.446+0000 7f3022a00640 1 -- 172.21.15.7:0/1872679320 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3024005020 con 0x7f302c152860 2024-09-10T21:47:44.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.446+0000 7f3030a00640 1 -- 172.21.15.7:0/1872679320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f302c0af900 con 0x7f302c152860 2024-09-10T21:47:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.448+0000 7f3030a00640 1 -- 172.21.15.7:0/1872679320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f302c06fcd0 con 0x7f302c152860 2024-09-10T21:47:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.449+0000 7f3022a00640 1 -- 172.21.15.7:0/1872679320 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f30240051c0 con 0x7f302c152860 2024-09-10T21:47:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.450+0000 7f3022a00640 1 --2- 172.21.15.7:0/1872679320 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3008077fa0 0x7f300807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.450+0000 7f3022a00640 1 -- 172.21.15.7:0/1872679320 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f302409a640 con 0x7f302c152860 2024-09-10T21:47:44.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.450+0000 7f3029e00640 1 --2- 172.21.15.7:0/1872679320 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3008077fa0 0x7f300807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:44.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.451+0000 7f3029e00640 1 --2- 172.21.15.7:0/1872679320 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3008077fa0 0x7f300807a460 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f302c0c6bc0 tx=0x7f301c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:44.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.454+0000 7f3022a00640 1 -- 172.21.15.7:0/1872679320 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f302c06fcd0 con 0x7f302c152860 2024-09-10T21:47:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.686+0000 7f3030a00640 1 -- 172.21.15.7:0/1872679320 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f302c1573b0 con 0x7f302c152860 2024-09-10T21:47:44.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.686+0000 7f3022a00640 1 -- 172.21.15.7:0/1872679320 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+5233 (secure 0 0 0) 0x7f302c1573b0 con 0x7f302c152860 2024-09-10T21:47:44.688 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:44.688 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":63,"btime":"2024-09-10T21:47:44:249448+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/978763848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":978763848},{"type":"v1","addr":"172.21.15.7:6835","nonce":978763848}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":235775,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/760980985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":760980985},{"type":"v1","addr":"172.21.15.133:6836","nonce":760980985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":235778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3312604545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3312604545},{"type":"v1","addr":"172.21.15.133:6837","nonce":3312604545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:47:40.664091+0000","modified":"2024-09-10T21:47:43.681126+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235781},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235781":{"gid":235781,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6833/1499133391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1499133391},{"type":"v1","addr":"172.21.15.7:6833","nonce":1499133391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235781,"qdb_cluster":[235781]},"id":6}]} 2024-09-10T21:47:44.688 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 63 2024-09-10T21:47:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.689+0000 7f3030a00640 1 -- 172.21.15.7:0/1872679320 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3008077fa0 msgr2=0x7f300807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.689+0000 7f3030a00640 1 --2- 172.21.15.7:0/1872679320 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3008077fa0 0x7f300807a460 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f302c0c6bc0 tx=0x7f301c005f90 comp rx=0 tx=0).stop 2024-09-10T21:47:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.689+0000 7f3030a00640 1 -- 172.21.15.7:0/1872679320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f302c152860 msgr2=0x7f302c159c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.689+0000 7f3030a00640 1 --2- 172.21.15.7:0/1872679320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f302c152860 0x7f302c159c80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f302c15de40 tx=0x7f3024009f70 comp rx=0 tx=0).stop 2024-09-10T21:47:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.689+0000 7f3030a00640 1 -- 172.21.15.7:0/1872679320 shutdown_connections 2024-09-10T21:47:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.689+0000 7f3030a00640 1 --2- 172.21.15.7:0/1872679320 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3008077fa0 0x7f300807a460 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.689+0000 7f3030a00640 1 --2- 172.21.15.7:0/1872679320 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f302c152360 0x7f302c159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.689+0000 7f3030a00640 1 --2- 172.21.15.7:0/1872679320 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f302c152860 0x7f302c159c80 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.689+0000 7f3030a00640 1 --2- 172.21.15.7:0/1872679320 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3010004020 0x7f302c152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.689+0000 7f3030a00640 1 -- 172.21.15.7:0/1872679320 >> 172.21.15.7:0/1872679320 conn(0x7f302c0b16d0 msgr2=0x7f302c155b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.689+0000 7f3030a00640 1 -- 172.21.15.7:0/1872679320 shutdown_connections 2024-09-10T21:47:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.689+0000 7f3030a00640 1 -- 172.21.15.7:0/1872679320 wait complete. 2024-09-10T21:47:44.706 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:47:44.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.867+0000 7fd24d400640 1 -- 172.21.15.7:0/3569513765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2480cbd50 msgr2=0x7fd2480cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:44.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.867+0000 7fd24d400640 1 --2- 172.21.15.7:0/3569513765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2480cbd50 0x7fd2480cc1c0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fd234005fd0 tx=0x7fd23402a520 comp rx=0 tx=0).stop 2024-09-10T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.867+0000 7fd24d400640 1 -- 172.21.15.7:0/3569513765 shutdown_connections 2024-09-10T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.867+0000 7fd24d400640 1 --2- 172.21.15.7:0/3569513765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2480ae020 0x7fd2480ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.867+0000 7fd24d400640 1 --2- 172.21.15.7:0/3569513765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2480cbd50 0x7fd2480cc1c0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.867+0000 7fd24d400640 1 -- 172.21.15.7:0/3569513765 >> 172.21.15.7:0/3569513765 conn(0x7fd2480b16d0 msgr2=0x7fd2480b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.867+0000 7fd24d400640 1 -- 172.21.15.7:0/3569513765 shutdown_connections 2024-09-10T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.867+0000 7fd24d400640 1 -- 172.21.15.7:0/3569513765 wait complete. 2024-09-10T21:47:44.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.869+0000 7fd24d400640 1 Processor -- start 2024-09-10T21:47:44.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.869+0000 7fd24d400640 1 -- start start 2024-09-10T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.870+0000 7fd24d400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2480ae020 0x7fd2480b51b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.870+0000 7fd24d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2480cbd50 0x7fd2481f8410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.870+0000 7fd24d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2481619e0 0x7fd2481f8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.870+0000 7fd24d400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd2480ce0d0 con 0x7fd2481619e0 2024-09-10T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.870+0000 7fd24d400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd2480cdf50 con 0x7fd2480cbd50 2024-09-10T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.870+0000 7fd24d400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fd248161dc0 con 0x7fd2480ae020 2024-09-10T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.870+0000 7fd247200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2481619e0 0x7fd2481f8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.870+0000 7fd247200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2481619e0 0x7fd2481f8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57712/0 (socket says 172.21.15.7:57712) 2024-09-10T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.870+0000 7fd247200640 1 -- 172.21.15.7:0/3039048544 learned_addr learned my addr 172.21.15.7:0/3039048544 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.870+0000 7fd245e00640 1 --2- 172.21.15.7:0/3039048544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2480cbd50 0x7fd2481f8410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.870+0000 7fd246800640 1 --2- 172.21.15.7:0/3039048544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2480ae020 0x7fd2480b51b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.870+0000 7fd247200640 1 -- 172.21.15.7:0/3039048544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2480ae020 msgr2=0x7fd2480b51b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.870+0000 7fd247200640 1 --2- 172.21.15.7:0/3039048544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2480ae020 0x7fd2480b51b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.870+0000 7fd247200640 1 -- 172.21.15.7:0/3039048544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2480cbd50 msgr2=0x7fd2481f8410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.870+0000 7fd247200640 1 --2- 172.21.15.7:0/3039048544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2480cbd50 0x7fd2481f8410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.870+0000 7fd247200640 1 -- 172.21.15.7:0/3039048544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2480b56f0 con 0x7fd2481619e0 2024-09-10T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.871+0000 7fd247200640 1 --2- 172.21.15.7:0/3039048544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2481619e0 0x7fd2481f8950 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd24000e8b0 tx=0x7fd24000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.871+0000 7fd23ea00640 1 -- 172.21.15.7:0/3039048544 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240004d20 con 0x7fd2481619e0 2024-09-10T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.871+0000 7fd23ea00640 1 -- 172.21.15.7:0/3039048544 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd24000ca60 con 0x7fd2481619e0 2024-09-10T21:47:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.871+0000 7fd23ea00640 1 -- 172.21.15.7:0/3039048544 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd240005020 con 0x7fd2481619e0 2024-09-10T21:47:44.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.871+0000 7fd24d400640 1 -- 172.21.15.7:0/3039048544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2480c5880 con 0x7fd2481619e0 2024-09-10T21:47:44.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.871+0000 7fd24d400640 1 -- 172.21.15.7:0/3039048544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2480c5e40 con 0x7fd2481619e0 2024-09-10T21:47:44.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.872+0000 7fd24d400640 1 -- 172.21.15.7:0/3039048544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2480ae4b0 con 0x7fd2481619e0 2024-09-10T21:47:44.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.873+0000 7fd23ea00640 1 -- 172.21.15.7:0/3039048544 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd24000f040 con 0x7fd2481619e0 2024-09-10T21:47:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.874+0000 7fd23ea00640 1 --2- 172.21.15.7:0/3039048544 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd228077f70 0x7fd22807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.874+0000 7fd23ea00640 1 -- 172.21.15.7:0/3039048544 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fd24009a230 con 0x7fd2481619e0 2024-09-10T21:47:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.875+0000 7fd246800640 1 --2- 172.21.15.7:0/3039048544 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd228077f70 0x7fd22807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.875+0000 7fd246800640 1 --2- 172.21.15.7:0/3039048544 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd228077f70 0x7fd22807a430 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd238004f60 tx=0x7fd238007e30 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:44.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:44.879+0000 7fd23ea00640 1 -- 172.21.15.7:0/3039048544 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd2480b56f0 con 0x7fd2481619e0 2024-09-10T21:47:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.077+0000 7fd24d400640 1 -- 172.21.15.7:0/3039048544 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd2480b5b00 con 0x7fd2481619e0 2024-09-10T21:47:45.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.078+0000 7fd23ea00640 1 -- 172.21.15.7:0/3039048544 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16684 (secure 0 0 0) 0x7fd240016030 con 0x7fd2481619e0 2024-09-10T21:47:45.079 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:45.079 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":85,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:47:42.218840+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:47:38.456723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T21:47:39.629851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:47:45.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.080+0000 7fd24d400640 1 -- 172.21.15.7:0/3039048544 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd228077f70 msgr2=0x7fd22807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.080+0000 7fd24d400640 1 --2- 172.21.15.7:0/3039048544 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd228077f70 0x7fd22807a430 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd238004f60 tx=0x7fd238007e30 comp rx=0 tx=0).stop 2024-09-10T21:47:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.080+0000 7fd24d400640 1 -- 172.21.15.7:0/3039048544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2481619e0 msgr2=0x7fd2481f8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.080+0000 7fd24d400640 1 --2- 172.21.15.7:0/3039048544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2481619e0 0x7fd2481f8950 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fd24000e8b0 tx=0x7fd24000ed70 comp rx=0 tx=0).stop 2024-09-10T21:47:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.080+0000 7fd24d400640 1 -- 172.21.15.7:0/3039048544 shutdown_connections 2024-09-10T21:47:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.080+0000 7fd24d400640 1 --2- 172.21.15.7:0/3039048544 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd228077f70 0x7fd22807a430 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.080+0000 7fd24d400640 1 --2- 172.21.15.7:0/3039048544 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd2480cbd50 0x7fd2481f8410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.080+0000 7fd24d400640 1 --2- 172.21.15.7:0/3039048544 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2481619e0 0x7fd2481f8950 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.080+0000 7fd24d400640 1 --2- 172.21.15.7:0/3039048544 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2480ae020 0x7fd2480b51b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.080+0000 7fd24d400640 1 -- 172.21.15.7:0/3039048544 >> 172.21.15.7:0/3039048544 conn(0x7fd2480b16d0 msgr2=0x7fd2480b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.080+0000 7fd24d400640 1 -- 172.21.15.7:0/3039048544 shutdown_connections 2024-09-10T21:47:45.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.081+0000 7fd24d400640 1 -- 172.21.15.7:0/3039048544 wait complete. 2024-09-10T21:47:45.097 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:47:45.097 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:47:45.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.258+0000 7feb83a00640 1 -- 172.21.15.7:0/2132616465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb7c0cd750 msgr2=0x7feb7c073730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:45.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.258+0000 7feb83a00640 1 --2- 172.21.15.7:0/2132616465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb7c0cd750 0x7feb7c073730 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7feb64005fd0 tx=0x7feb6402a4b0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.258+0000 7feb83a00640 1 -- 172.21.15.7:0/2132616465 shutdown_connections 2024-09-10T21:47:45.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.258+0000 7feb83a00640 1 --2- 172.21.15.7:0/2132616465 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb7c0cd750 0x7feb7c073730 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.258+0000 7feb83a00640 1 --2- 172.21.15.7:0/2132616465 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb7c15bc90 0x7feb7c0ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.258+0000 7feb83a00640 1 -- 172.21.15.7:0/2132616465 >> 172.21.15.7:0/2132616465 conn(0x7feb7c0b16d0 msgr2=0x7feb7c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:45.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.258+0000 7feb83a00640 1 -- 172.21.15.7:0/2132616465 shutdown_connections 2024-09-10T21:47:45.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.258+0000 7feb83a00640 1 -- 172.21.15.7:0/2132616465 wait complete. 2024-09-10T21:47:45.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.260+0000 7feb83a00640 1 Processor -- start 2024-09-10T21:47:45.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.260+0000 7feb83a00640 1 -- start start 2024-09-10T21:47:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.261+0000 7feb83a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb7c073c70 0x7feb7c1564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.261+0000 7feb83a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb7c0cd750 0x7feb7c1f44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.261+0000 7feb83a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb7c15bc90 0x7feb7c15cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.261+0000 7feb83a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c06fcf0 con 0x7feb7c15bc90 2024-09-10T21:47:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.261+0000 7feb83a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feb7c1606c0 con 0x7feb7c0cd750 2024-09-10T21:47:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.261+0000 7feb83a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7feb7c06fe70 con 0x7feb7c073c70 2024-09-10T21:47:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.261+0000 7feb81a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb7c15bc90 0x7feb7c15cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.261+0000 7feb81000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb7c073c70 0x7feb7c1564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:45.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.261+0000 7feb81a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb7c15bc90 0x7feb7c15cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57716/0 (socket says 172.21.15.7:57716) 2024-09-10T21:47:45.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.261+0000 7feb81a00640 1 -- 172.21.15.7:0/1810541878 learned_addr learned my addr 172.21.15.7:0/1810541878 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:45.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.261+0000 7feb73e00640 1 --2- 172.21.15.7:0/1810541878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb7c0cd750 0x7feb7c1f44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.262+0000 7feb81a00640 1 -- 172.21.15.7:0/1810541878 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb7c073c70 msgr2=0x7feb7c1564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.262+0000 7feb81a00640 1 --2- 172.21.15.7:0/1810541878 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb7c073c70 0x7feb7c1564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.262+0000 7feb81a00640 1 -- 172.21.15.7:0/1810541878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb7c0cd750 msgr2=0x7feb7c1f44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.262+0000 7feb81a00640 1 --2- 172.21.15.7:0/1810541878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb7c0cd750 0x7feb7c1f44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.262+0000 7feb81a00640 1 -- 172.21.15.7:0/1810541878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb7c1569e0 con 0x7feb7c15bc90 2024-09-10T21:47:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.262+0000 7feb81a00640 1 --2- 172.21.15.7:0/1810541878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb7c15bc90 0x7feb7c15cb00 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7feb7400c730 tx=0x7feb7400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.262+0000 7feb79600640 1 -- 172.21.15.7:0/1810541878 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb74003b70 con 0x7feb7c15bc90 2024-09-10T21:47:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.262+0000 7feb79600640 1 -- 172.21.15.7:0/1810541878 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb74003d10 con 0x7feb7c15bc90 2024-09-10T21:47:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.262+0000 7feb79600640 1 -- 172.21.15.7:0/1810541878 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb7400aab0 con 0x7feb7c15bc90 2024-09-10T21:47:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.262+0000 7feb83a00640 1 -- 172.21.15.7:0/1810541878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb7c1f4d30 con 0x7feb7c15bc90 2024-09-10T21:47:45.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.262+0000 7feb83a00640 1 -- 172.21.15.7:0/1810541878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb7c15c300 con 0x7feb7c15bc90 2024-09-10T21:47:45.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.263+0000 7feb83a00640 1 -- 172.21.15.7:0/1810541878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb7c1f5120 con 0x7feb7c15bc90 2024-09-10T21:47:45.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.264+0000 7feb79600640 1 -- 172.21.15.7:0/1810541878 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7feb7400ac50 con 0x7feb7c15bc90 2024-09-10T21:47:45.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.265+0000 7feb79600640 1 --2- 172.21.15.7:0/1810541878 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7feb58077f70 0x7feb5807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:45.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.265+0000 7feb79600640 1 -- 172.21.15.7:0/1810541878 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7feb74099ae0 con 0x7feb7c15bc90 2024-09-10T21:47:45.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.266+0000 7feb81000640 1 --2- 172.21.15.7:0/1810541878 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7feb58077f70 0x7feb5807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:45.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.266+0000 7feb81000640 1 --2- 172.21.15.7:0/1810541878 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7feb58077f70 0x7feb5807a430 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009b30 tx=0x7feb6c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:45.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.269+0000 7feb79600640 1 -- 172.21.15.7:0/1810541878 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7feb74061e60 con 0x7feb7c15bc90 2024-09-10T21:47:45.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.461+0000 7feb83a00640 1 -- 172.21.15.7:0/1810541878 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feb7c073730 con 0x7feb58077f70 2024-09-10T21:47:45.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.466+0000 7feb79600640 1 -- 172.21.15.7:0/1810541878 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252070 (secure 0 0 0) 0x7feb7c073730 con 0x7feb58077f70 2024-09-10T21:47:45.470 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:45.472 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:47:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.470+0000 7feb83a00640 1 -- 172.21.15.7:0/1810541878 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7feb58077f70 msgr2=0x7feb5807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.470+0000 7feb83a00640 1 --2- 172.21.15.7:0/1810541878 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7feb58077f70 0x7feb5807a430 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7feb6c009b30 tx=0x7feb6c005f90 comp rx=0 tx=0).stop 2024-09-10T21:47:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.470+0000 7feb83a00640 1 -- 172.21.15.7:0/1810541878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb7c15bc90 msgr2=0x7feb7c15cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.470+0000 7feb83a00640 1 --2- 172.21.15.7:0/1810541878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb7c15bc90 0x7feb7c15cb00 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7feb7400c730 tx=0x7feb7400cbf0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.470+0000 7feb83a00640 1 -- 172.21.15.7:0/1810541878 shutdown_connections 2024-09-10T21:47:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.470+0000 7feb83a00640 1 --2- 172.21.15.7:0/1810541878 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7feb58077f70 0x7feb5807a430 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.470+0000 7feb83a00640 1 --2- 172.21.15.7:0/1810541878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feb7c0cd750 0x7feb7c1f44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.470+0000 7feb83a00640 1 --2- 172.21.15.7:0/1810541878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7feb7c15bc90 0x7feb7c15cb00 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.470+0000 7feb83a00640 1 --2- 172.21.15.7:0/1810541878 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feb7c073c70 0x7feb7c1564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.470+0000 7feb83a00640 1 -- 172.21.15.7:0/1810541878 >> 172.21.15.7:0/1810541878 conn(0x7feb7c0b16d0 msgr2=0x7feb7c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.470+0000 7feb83a00640 1 -- 172.21.15.7:0/1810541878 shutdown_connections 2024-09-10T21:47:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.470+0000 7feb83a00640 1 -- 172.21.15.7:0/1810541878 wait complete. 2024-09-10T21:47:45.486 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-09-10T21:47:43.943155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262940,"kb_used_data":9248,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754711780,"statfs":{"total":773094113280,"available":772824862720,"internally_reserved":0,"allocated":9469952,"data_stored":4677281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.732214"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235125+0000","last_change":"2024-09-10T21:47:41.242942+0000","last_active":"2024-09-10T21:47:42.235125+0000","last_peered":"2024-09-10T21:47:42.235125+0000","last_clean":"2024-09-10T21:47:42.235125+0000","last_became_active":"2024-09-10T21:47:41.242607+0000","last_became_peered":"2024-09-10T21:47:41.242607+0000","last_unstale":"2024-09-10T21:47:42.235125+0000","last_undegraded":"2024-09-10T21:47:42.235125+0000","last_fullsized":"2024-09-10T21:47:42.235125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:40.161098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235567+0000","last_change":"2024-09-10T21:47:41.248615+0000","last_active":"2024-09-10T21:47:42.235567+0000","last_peered":"2024-09-10T21:47:42.235567+0000","last_clean":"2024-09-10T21:47:42.235567+0000","last_became_active":"2024-09-10T21:47:41.247819+0000","last_became_peered":"2024-09-10T21:47:41.247819+0000","last_unstale":"2024-09-10T21:47:42.235567+0000","last_undegraded":"2024-09-10T21:47:42.235567+0000","last_fullsized":"2024-09-10T21:47:42.235567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:49.531431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.241295+0000","last_change":"2024-09-10T21:47:41.238692+0000","last_active":"2024-09-10T21:47:41.241295+0000","last_peered":"2024-09-10T21:47:41.241295+0000","last_clean":"2024-09-10T21:47:41.241295+0000","last_became_active":"2024-09-10T21:47:41.237972+0000","last_became_peered":"2024-09-10T21:47:41.237972+0000","last_unstale":"2024-09-10T21:47:41.241295+0000","last_undegraded":"2024-09-10T21:47:41.241295+0000","last_fullsized":"2024-09-10T21:47:41.241295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:32.440482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.223249+0000","last_change":"2024-09-10T21:47:41.255766+0000","last_active":"2024-09-10T21:47:42.223249+0000","last_peered":"2024-09-10T21:47:42.223249+0000","last_clean":"2024-09-10T21:47:42.223249+0000","last_became_active":"2024-09-10T21:47:41.254110+0000","last_became_peered":"2024-09-10T21:47:41.254110+0000","last_unstale":"2024-09-10T21:47:42.223249+0000","last_undegraded":"2024-09-10T21:47:42.223249+0000","last_fullsized":"2024-09-10T21:47:42.223249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:13.319515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229056+0000","last_change":"2024-09-10T21:47:41.249886+0000","last_active":"2024-09-10T21:47:42.229056+0000","last_peered":"2024-09-10T21:47:42.229056+0000","last_clean":"2024-09-10T21:47:42.229056+0000","last_became_active":"2024-09-10T21:47:41.247649+0000","last_became_peered":"2024-09-10T21:47:41.247649+0000","last_unstale":"2024-09-10T21:47:42.229056+0000","last_undegraded":"2024-09-10T21:47:42.229056+0000","last_fullsized":"2024-09-10T21:47:42.229056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:44.426814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234218+0000","last_change":"2024-09-10T21:47:41.243328+0000","last_active":"2024-09-10T21:47:42.234218+0000","last_peered":"2024-09-10T21:47:42.234218+0000","last_clean":"2024-09-10T21:47:42.234218+0000","last_became_active":"2024-09-10T21:47:41.243075+0000","last_became_peered":"2024-09-10T21:47:41.243075+0000","last_unstale":"2024-09-10T21:47:42.234218+0000","last_undegraded":"2024-09-10T21:47:42.234218+0000","last_fullsized":"2024-09-10T21:47:42.234218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:11.635440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.222627+0000","last_change":"2024-09-10T21:47:41.243710+0000","last_active":"2024-09-10T21:47:42.222627+0000","last_peered":"2024-09-10T21:47:42.222627+0000","last_clean":"2024-09-10T21:47:42.222627+0000","last_became_active":"2024-09-10T21:47:41.240792+0000","last_became_peered":"2024-09-10T21:47:41.240792+0000","last_unstale":"2024-09-10T21:47:42.222627+0000","last_undegraded":"2024-09-10T21:47:42.222627+0000","last_fullsized":"2024-09-10T21:47:42.222627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:14.435322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234034+0000","last_change":"2024-09-10T21:47:41.261163+0000","last_active":"2024-09-10T21:47:42.234034+0000","last_peered":"2024-09-10T21:47:42.234034+0000","last_clean":"2024-09-10T21:47:42.234034+0000","last_became_active":"2024-09-10T21:47:41.260878+0000","last_became_peered":"2024-09-10T21:47:41.260878+0000","last_unstale":"2024-09-10T21:47:42.234034+0000","last_undegraded":"2024-09-10T21:47:42.234034+0000","last_fullsized":"2024-09-10T21:47:42.234034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:01.015998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235157+0000","last_change":"2024-09-10T21:47:41.253326+0000","last_active":"2024-09-10T21:47:42.235157+0000","last_peered":"2024-09-10T21:47:42.235157+0000","last_clean":"2024-09-10T21:47:42.235157+0000","last_became_active":"2024-09-10T21:47:41.252434+0000","last_became_peered":"2024-09-10T21:47:41.252434+0000","last_unstale":"2024-09-10T21:47:42.235157+0000","last_undegraded":"2024-09-10T21:47:42.235157+0000","last_fullsized":"2024-09-10T21:47:42.235157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:59.893460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235246+0000","last_change":"2024-09-10T21:47:41.235235+0000","last_active":"2024-09-10T21:47:42.235246+0000","last_peered":"2024-09-10T21:47:42.235246+0000","last_clean":"2024-09-10T21:47:42.235246+0000","last_became_active":"2024-09-10T21:47:41.234969+0000","last_became_peered":"2024-09-10T21:47:41.234969+0000","last_unstale":"2024-09-10T21:47:42.235246+0000","last_undegraded":"2024-09-10T21:47:42.235246+0000","last_fullsized":"2024-09-10T21:47:42.235246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:25.433073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235086+0000","last_change":"2024-09-10T21:47:41.243912+0000","last_active":"2024-09-10T21:47:42.235086+0000","last_peered":"2024-09-10T21:47:42.235086+0000","last_clean":"2024-09-10T21:47:42.235086+0000","last_became_active":"2024-09-10T21:47:41.243584+0000","last_became_peered":"2024-09-10T21:47:41.243584+0000","last_unstale":"2024-09-10T21:47:42.235086+0000","last_undegraded":"2024-09-10T21:47:42.235086+0000","last_fullsized":"2024-09-10T21:47:42.235086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:53.542101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234547+0000","last_change":"2024-09-10T21:47:41.235211+0000","last_active":"2024-09-10T21:47:42.234547+0000","last_peered":"2024-09-10T21:47:42.234547+0000","last_clean":"2024-09-10T21:47:42.234547+0000","last_became_active":"2024-09-10T21:47:41.234939+0000","last_became_peered":"2024-09-10T21:47:41.234939+0000","last_unstale":"2024-09-10T21:47:42.234547+0000","last_undegraded":"2024-09-10T21:47:42.234547+0000","last_fullsized":"2024-09-10T21:47:42.234547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:29.163811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.249504+0000","last_change":"2024-09-10T21:47:41.245469+0000","last_active":"2024-09-10T21:47:41.249504+0000","last_peered":"2024-09-10T21:47:41.249504+0000","last_clean":"2024-09-10T21:47:41.249504+0000","last_became_active":"2024-09-10T21:47:41.245114+0000","last_became_peered":"2024-09-10T21:47:41.245114+0000","last_unstale":"2024-09-10T21:47:41.249504+0000","last_undegraded":"2024-09-10T21:47:41.249504+0000","last_fullsized":"2024-09-10T21:47:41.249504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:13.145479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.222837+0000","last_change":"2024-09-10T21:47:41.256279+0000","last_active":"2024-09-10T21:47:42.222837+0000","last_peered":"2024-09-10T21:47:42.222837+0000","last_clean":"2024-09-10T21:47:42.222837+0000","last_became_active":"2024-09-10T21:47:41.255992+0000","last_became_peered":"2024-09-10T21:47:41.255992+0000","last_unstale":"2024-09-10T21:47:42.222837+0000","last_undegraded":"2024-09-10T21:47:42.222837+0000","last_fullsized":"2024-09-10T21:47:42.222837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:24.910353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.241215+0000","last_change":"2024-09-10T21:47:41.238041+0000","last_active":"2024-09-10T21:47:41.241215+0000","last_peered":"2024-09-10T21:47:41.241215+0000","last_clean":"2024-09-10T21:47:41.241215+0000","last_became_active":"2024-09-10T21:47:41.237691+0000","last_became_peered":"2024-09-10T21:47:41.237691+0000","last_unstale":"2024-09-10T21:47:41.241215+0000","last_undegraded":"2024-09-10T21:47:41.241215+0000","last_fullsized":"2024-09-10T21:47:41.241215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:38.220940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.228680+0000","last_change":"2024-09-10T21:47:41.233514+0000","last_active":"2024-09-10T21:47:42.228680+0000","last_peered":"2024-09-10T21:47:42.228680+0000","last_clean":"2024-09-10T21:47:42.228680+0000","last_became_active":"2024-09-10T21:47:41.233255+0000","last_became_peered":"2024-09-10T21:47:41.233255+0000","last_unstale":"2024-09-10T21:47:42.228680+0000","last_undegraded":"2024-09-10T21:47:42.228680+0000","last_fullsized":"2024-09-10T21:47:42.228680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:46.210802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229519+0000","last_change":"2024-09-10T21:47:41.251338+0000","last_active":"2024-09-10T21:47:42.229519+0000","last_peered":"2024-09-10T21:47:42.229519+0000","last_clean":"2024-09-10T21:47:42.229519+0000","last_became_active":"2024-09-10T21:47:41.250168+0000","last_became_peered":"2024-09-10T21:47:41.250168+0000","last_unstale":"2024-09-10T21:47:42.229519+0000","last_undegraded":"2024-09-10T21:47:42.229519+0000","last_fullsized":"2024-09-10T21:47:42.229519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:11.635440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.222913+0000","last_change":"2024-09-10T21:47:41.236969+0000","last_active":"2024-09-10T21:47:42.222913+0000","last_peered":"2024-09-10T21:47:42.222913+0000","last_clean":"2024-09-10T21:47:42.222913+0000","last_became_active":"2024-09-10T21:47:41.234889+0000","last_became_peered":"2024-09-10T21:47:41.234889+0000","last_unstale":"2024-09-10T21:47:42.222913+0000","last_undegraded":"2024-09-10T21:47:42.222913+0000","last_fullsized":"2024-09-10T21:47:42.222913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:22.243388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235536+0000","last_change":"2024-09-10T21:47:41.249077+0000","last_active":"2024-09-10T21:47:42.235536+0000","last_peered":"2024-09-10T21:47:42.235536+0000","last_clean":"2024-09-10T21:47:42.235536+0000","last_became_active":"2024-09-10T21:47:41.247147+0000","last_became_peered":"2024-09-10T21:47:41.247147+0000","last_unstale":"2024-09-10T21:47:42.235536+0000","last_undegraded":"2024-09-10T21:47:42.235536+0000","last_fullsized":"2024-09-10T21:47:42.235536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:03.990747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.223309+0000","last_change":"2024-09-10T21:47:41.253091+0000","last_active":"2024-09-10T21:47:42.223309+0000","last_peered":"2024-09-10T21:47:42.223309+0000","last_clean":"2024-09-10T21:47:42.223309+0000","last_became_active":"2024-09-10T21:47:41.252586+0000","last_became_peered":"2024-09-10T21:47:41.252586+0000","last_unstale":"2024-09-10T21:47:42.223309+0000","last_undegraded":"2024-09-10T21:47:42.223309+0000","last_fullsized":"2024-09-10T21:47:42.223309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:01.443545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.222996+0000","last_change":"2024-09-10T21:47:41.245414+0000","last_active":"2024-09-10T21:47:42.222996+0000","last_peered":"2024-09-10T21:47:42.222996+0000","last_clean":"2024-09-10T21:47:42.222996+0000","last_became_active":"2024-09-10T21:47:41.245153+0000","last_became_peered":"2024-09-10T21:47:41.245153+0000","last_unstale":"2024-09-10T21:47:42.222996+0000","last_undegraded":"2024-09-10T21:47:42.222996+0000","last_fullsized":"2024-09-10T21:47:42.222996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:53.407700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234889+0000","last_change":"2024-09-10T21:47:41.251176+0000","last_active":"2024-09-10T21:47:42.234889+0000","last_peered":"2024-09-10T21:47:42.234889+0000","last_clean":"2024-09-10T21:47:42.234889+0000","last_became_active":"2024-09-10T21:47:41.249983+0000","last_became_peered":"2024-09-10T21:47:41.249983+0000","last_unstale":"2024-09-10T21:47:42.234889+0000","last_undegraded":"2024-09-10T21:47:42.234889+0000","last_fullsized":"2024-09-10T21:47:42.234889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:06.277899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.233991+0000","last_change":"2024-09-10T21:47:41.243620+0000","last_active":"2024-09-10T21:47:42.233991+0000","last_peered":"2024-09-10T21:47:42.233991+0000","last_clean":"2024-09-10T21:47:42.233991+0000","last_became_active":"2024-09-10T21:47:41.243367+0000","last_became_peered":"2024-09-10T21:47:41.243367+0000","last_unstale":"2024-09-10T21:47:42.233991+0000","last_undegraded":"2024-09-10T21:47:42.233991+0000","last_fullsized":"2024-09-10T21:47:42.233991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:46.523445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.222784+0000","last_change":"2024-09-10T21:47:41.235364+0000","last_active":"2024-09-10T21:47:42.222784+0000","last_peered":"2024-09-10T21:47:42.222784+0000","last_clean":"2024-09-10T21:47:42.222784+0000","last_became_active":"2024-09-10T21:47:41.235083+0000","last_became_peered":"2024-09-10T21:47:41.235083+0000","last_unstale":"2024-09-10T21:47:42.222784+0000","last_undegraded":"2024-09-10T21:47:42.222784+0000","last_fullsized":"2024-09-10T21:47:42.222784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:15.827659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235526+0000","last_change":"2024-09-10T21:47:41.250729+0000","last_active":"2024-09-10T21:47:42.235526+0000","last_peered":"2024-09-10T21:47:42.235526+0000","last_clean":"2024-09-10T21:47:42.235526+0000","last_became_active":"2024-09-10T21:47:41.250469+0000","last_became_peered":"2024-09-10T21:47:41.250469+0000","last_unstale":"2024-09-10T21:47:42.235526+0000","last_undegraded":"2024-09-10T21:47:42.235526+0000","last_fullsized":"2024-09-10T21:47:42.235526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:13.625135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.258071+0000","last_change":"2024-09-10T21:47:41.254500+0000","last_active":"2024-09-10T21:47:41.258071+0000","last_peered":"2024-09-10T21:47:41.258071+0000","last_clean":"2024-09-10T21:47:41.258071+0000","last_became_active":"2024-09-10T21:47:41.254077+0000","last_became_peered":"2024-09-10T21:47:41.254077+0000","last_unstale":"2024-09-10T21:47:41.258071+0000","last_undegraded":"2024-09-10T21:47:41.258071+0000","last_fullsized":"2024-09-10T21:47:41.258071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:07:29.557158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.228998+0000","last_change":"2024-09-10T21:47:41.243643+0000","last_active":"2024-09-10T21:47:42.228998+0000","last_peered":"2024-09-10T21:47:42.228998+0000","last_clean":"2024-09-10T21:47:42.228998+0000","last_became_active":"2024-09-10T21:47:41.240706+0000","last_became_peered":"2024-09-10T21:47:41.240706+0000","last_unstale":"2024-09-10T21:47:42.228998+0000","last_undegraded":"2024-09-10T21:47:42.228998+0000","last_fullsized":"2024-09-10T21:47:42.228998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:33.342304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234601+0000","last_change":"2024-09-10T21:47:41.242344+0000","last_active":"2024-09-10T21:47:42.234601+0000","last_peered":"2024-09-10T21:47:42.234601+0000","last_clean":"2024-09-10T21:47:42.234601+0000","last_became_active":"2024-09-10T21:47:41.242006+0000","last_became_peered":"2024-09-10T21:47:41.242006+0000","last_unstale":"2024-09-10T21:47:42.234601+0000","last_undegraded":"2024-09-10T21:47:42.234601+0000","last_fullsized":"2024-09-10T21:47:42.234601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:01.015998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235153+0000","last_change":"2024-09-10T21:47:41.239244+0000","last_active":"2024-09-10T21:47:42.235153+0000","last_peered":"2024-09-10T21:47:42.235153+0000","last_clean":"2024-09-10T21:47:42.235153+0000","last_became_active":"2024-09-10T21:47:41.238892+0000","last_became_peered":"2024-09-10T21:47:41.238892+0000","last_unstale":"2024-09-10T21:47:42.235153+0000","last_undegraded":"2024-09-10T21:47:42.235153+0000","last_fullsized":"2024-09-10T21:47:42.235153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:09:11.571250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235175+0000","last_change":"2024-09-10T21:47:41.245615+0000","last_active":"2024-09-10T21:47:42.235175+0000","last_peered":"2024-09-10T21:47:42.235175+0000","last_clean":"2024-09-10T21:47:42.235175+0000","last_became_active":"2024-09-10T21:47:41.245331+0000","last_became_peered":"2024-09-10T21:47:41.245331+0000","last_unstale":"2024-09-10T21:47:42.235175+0000","last_undegraded":"2024-09-10T21:47:42.235175+0000","last_fullsized":"2024-09-10T21:47:42.235175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:55.772901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.228872+0000","last_change":"2024-09-10T21:47:41.237200+0000","last_active":"2024-09-10T21:47:42.228872+0000","last_peered":"2024-09-10T21:47:42.228872+0000","last_clean":"2024-09-10T21:47:42.228872+0000","last_became_active":"2024-09-10T21:47:41.236822+0000","last_became_peered":"2024-09-10T21:47:41.236822+0000","last_unstale":"2024-09-10T21:47:42.228872+0000","last_undegraded":"2024-09-10T21:47:42.228872+0000","last_fullsized":"2024-09-10T21:47:42.228872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:14.273935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.233955+0000","last_change":"2024-09-10T21:47:41.251791+0000","last_active":"2024-09-10T21:47:42.233955+0000","last_peered":"2024-09-10T21:47:42.233955+0000","last_clean":"2024-09-10T21:47:42.233955+0000","last_became_active":"2024-09-10T21:47:41.251272+0000","last_became_peered":"2024-09-10T21:47:41.251272+0000","last_unstale":"2024-09-10T21:47:42.233955+0000","last_undegraded":"2024-09-10T21:47:42.233955+0000","last_fullsized":"2024-09-10T21:47:42.233955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:46.063011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229257+0000","last_change":"2024-09-10T21:47:41.234915+0000","last_active":"2024-09-10T21:47:42.229257+0000","last_peered":"2024-09-10T21:47:42.229257+0000","last_clean":"2024-09-10T21:47:42.229257+0000","last_became_active":"2024-09-10T21:47:41.234582+0000","last_became_peered":"2024-09-10T21:47:41.234582+0000","last_unstale":"2024-09-10T21:47:42.229257+0000","last_undegraded":"2024-09-10T21:47:42.229257+0000","last_fullsized":"2024-09-10T21:47:42.229257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:57.517851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"84'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235667+0000","last_change":"2024-09-10T21:47:40.238034+0000","last_active":"2024-09-10T21:47:42.235667+0000","last_peered":"2024-09-10T21:47:42.235667+0000","last_clean":"2024-09-10T21:47:42.235667+0000","last_became_active":"2024-09-10T21:47:40.237748+0000","last_became_peered":"2024-09-10T21:47:40.237748+0000","last_unstale":"2024-09-10T21:47:42.235667+0000","last_undegraded":"2024-09-10T21:47:42.235667+0000","last_fullsized":"2024-09-10T21:47:42.235667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:23.214036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235550+0000","last_change":"2024-09-10T21:47:41.241430+0000","last_active":"2024-09-10T21:47:42.235550+0000","last_peered":"2024-09-10T21:47:42.235550+0000","last_clean":"2024-09-10T21:47:42.235550+0000","last_became_active":"2024-09-10T21:47:41.241077+0000","last_became_peered":"2024-09-10T21:47:41.241077+0000","last_unstale":"2024-09-10T21:47:42.235550+0000","last_undegraded":"2024-09-10T21:47:42.235550+0000","last_fullsized":"2024-09-10T21:47:42.235550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:59.893460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229399+0000","last_change":"2024-09-10T21:47:40.227811+0000","last_active":"2024-09-10T21:47:42.229399+0000","last_peered":"2024-09-10T21:47:42.229399+0000","last_clean":"2024-09-10T21:47:42.229399+0000","last_became_active":"2024-09-10T21:47:40.227399+0000","last_became_peered":"2024-09-10T21:47:40.227399+0000","last_unstale":"2024-09-10T21:47:42.229399+0000","last_undegraded":"2024-09-10T21:47:42.229399+0000","last_fullsized":"2024-09-10T21:47:42.229399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:42.598409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234854+0000","last_change":"2024-09-10T21:47:41.255071+0000","last_active":"2024-09-10T21:47:42.234854+0000","last_peered":"2024-09-10T21:47:42.234854+0000","last_clean":"2024-09-10T21:47:42.234854+0000","last_became_active":"2024-09-10T21:47:41.254678+0000","last_became_peered":"2024-09-10T21:47:41.254678+0000","last_unstale":"2024-09-10T21:47:42.234854+0000","last_undegraded":"2024-09-10T21:47:42.234854+0000","last_fullsized":"2024-09-10T21:47:42.234854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:11.635440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235478+0000","last_change":"2024-09-10T21:47:40.245284+0000","last_active":"2024-09-10T21:47:42.235478+0000","last_peered":"2024-09-10T21:47:42.235478+0000","last_clean":"2024-09-10T21:47:42.235478+0000","last_became_active":"2024-09-10T21:47:40.244945+0000","last_became_peered":"2024-09-10T21:47:40.244945+0000","last_unstale":"2024-09-10T21:47:42.235478+0000","last_undegraded":"2024-09-10T21:47:42.235478+0000","last_fullsized":"2024-09-10T21:47:42.235478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:35.716582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.249422+0000","last_change":"2024-09-10T21:47:41.241823+0000","last_active":"2024-09-10T21:47:41.249422+0000","last_peered":"2024-09-10T21:47:41.249422+0000","last_clean":"2024-09-10T21:47:41.249422+0000","last_became_active":"2024-09-10T21:47:41.241510+0000","last_became_peered":"2024-09-10T21:47:41.241510+0000","last_unstale":"2024-09-10T21:47:41.249422+0000","last_undegraded":"2024-09-10T21:47:41.249422+0000","last_fullsized":"2024-09-10T21:47:41.249422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:45:01.147786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235355+0000","last_change":"2024-09-10T21:47:40.245896+0000","last_active":"2024-09-10T21:47:42.235355+0000","last_peered":"2024-09-10T21:47:42.235355+0000","last_clean":"2024-09-10T21:47:42.235355+0000","last_became_active":"2024-09-10T21:47:40.245556+0000","last_became_peered":"2024-09-10T21:47:40.245556+0000","last_unstale":"2024-09-10T21:47:42.235355+0000","last_undegraded":"2024-09-10T21:47:42.235355+0000","last_fullsized":"2024-09-10T21:47:42.235355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:00.004793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235129+0000","last_change":"2024-09-10T21:47:41.236627+0000","last_active":"2024-09-10T21:47:42.235129+0000","last_peered":"2024-09-10T21:47:42.235129+0000","last_clean":"2024-09-10T21:47:42.235129+0000","last_became_active":"2024-09-10T21:47:41.236212+0000","last_became_peered":"2024-09-10T21:47:41.236212+0000","last_unstale":"2024-09-10T21:47:42.235129+0000","last_undegraded":"2024-09-10T21:47:42.235129+0000","last_fullsized":"2024-09-10T21:47:42.235129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:24.225240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229511+0000","last_change":"2024-09-10T21:47:40.241481+0000","last_active":"2024-09-10T21:47:42.229511+0000","last_peered":"2024-09-10T21:47:42.229511+0000","last_clean":"2024-09-10T21:47:42.229511+0000","last_became_active":"2024-09-10T21:47:40.241205+0000","last_became_peered":"2024-09-10T21:47:40.241205+0000","last_unstale":"2024-09-10T21:47:42.229511+0000","last_undegraded":"2024-09-10T21:47:42.229511+0000","last_fullsized":"2024-09-10T21:47:42.229511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:10:39.700005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229450+0000","last_change":"2024-09-10T21:47:41.250933+0000","last_active":"2024-09-10T21:47:42.229450+0000","last_peered":"2024-09-10T21:47:42.229450+0000","last_clean":"2024-09-10T21:47:42.229450+0000","last_became_active":"2024-09-10T21:47:41.250330+0000","last_became_peered":"2024-09-10T21:47:41.250330+0000","last_unstale":"2024-09-10T21:47:42.229450+0000","last_undegraded":"2024-09-10T21:47:42.229450+0000","last_fullsized":"2024-09-10T21:47:42.229450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:29.163811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.228836+0000","last_change":"2024-09-10T21:47:40.235952+0000","last_active":"2024-09-10T21:47:42.228836+0000","last_peered":"2024-09-10T21:47:42.228836+0000","last_clean":"2024-09-10T21:47:42.228836+0000","last_became_active":"2024-09-10T21:47:40.235614+0000","last_became_peered":"2024-09-10T21:47:40.235614+0000","last_unstale":"2024-09-10T21:47:42.228836+0000","last_undegraded":"2024-09-10T21:47:42.228836+0000","last_fullsized":"2024-09-10T21:47:42.228836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:26.500061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.249607+0000","last_change":"2024-09-10T21:47:41.243593+0000","last_active":"2024-09-10T21:47:41.249607+0000","last_peered":"2024-09-10T21:47:41.249607+0000","last_clean":"2024-09-10T21:47:41.249607+0000","last_became_active":"2024-09-10T21:47:41.243348+0000","last_became_peered":"2024-09-10T21:47:41.243348+0000","last_unstale":"2024-09-10T21:47:41.249607+0000","last_undegraded":"2024-09-10T21:47:41.249607+0000","last_fullsized":"2024-09-10T21:47:41.249607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:38.319192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234283+0000","last_change":"2024-09-10T21:47:40.239186+0000","last_active":"2024-09-10T21:47:42.234283+0000","last_peered":"2024-09-10T21:47:42.234283+0000","last_clean":"2024-09-10T21:47:42.234283+0000","last_became_active":"2024-09-10T21:47:40.238895+0000","last_became_peered":"2024-09-10T21:47:40.238895+0000","last_unstale":"2024-09-10T21:47:42.234283+0000","last_undegraded":"2024-09-10T21:47:42.234283+0000","last_fullsized":"2024-09-10T21:47:42.234283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:24.421869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.243659+0000","last_change":"2024-09-10T21:47:41.240091+0000","last_active":"2024-09-10T21:47:41.243659+0000","last_peered":"2024-09-10T21:47:41.243659+0000","last_clean":"2024-09-10T21:47:41.243659+0000","last_became_active":"2024-09-10T21:47:41.239768+0000","last_became_peered":"2024-09-10T21:47:41.239768+0000","last_unstale":"2024-09-10T21:47:41.243659+0000","last_undegraded":"2024-09-10T21:47:41.243659+0000","last_fullsized":"2024-09-10T21:47:41.243659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:13.425629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234462+0000","last_change":"2024-09-10T21:47:40.240093+0000","last_active":"2024-09-10T21:47:42.234462+0000","last_peered":"2024-09-10T21:47:42.234462+0000","last_clean":"2024-09-10T21:47:42.234462+0000","last_became_active":"2024-09-10T21:47:40.238125+0000","last_became_peered":"2024-09-10T21:47:40.238125+0000","last_unstale":"2024-09-10T21:47:42.234462+0000","last_undegraded":"2024-09-10T21:47:42.234462+0000","last_fullsized":"2024-09-10T21:47:42.234462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:12.956184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.258714+0000","last_change":"2024-09-10T21:47:41.249707+0000","last_active":"2024-09-10T21:47:41.258714+0000","last_peered":"2024-09-10T21:47:41.258714+0000","last_clean":"2024-09-10T21:47:41.258714+0000","last_became_active":"2024-09-10T21:47:41.249384+0000","last_became_peered":"2024-09-10T21:47:41.249384+0000","last_unstale":"2024-09-10T21:47:41.258714+0000","last_undegraded":"2024-09-10T21:47:41.258714+0000","last_fullsized":"2024-09-10T21:47:41.258714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:40.670356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235664+0000","last_change":"2024-09-10T21:47:40.245932+0000","last_active":"2024-09-10T21:47:42.235664+0000","last_peered":"2024-09-10T21:47:42.235664+0000","last_clean":"2024-09-10T21:47:42.235664+0000","last_became_active":"2024-09-10T21:47:40.245641+0000","last_became_peered":"2024-09-10T21:47:40.245641+0000","last_unstale":"2024-09-10T21:47:42.235664+0000","last_undegraded":"2024-09-10T21:47:42.235664+0000","last_fullsized":"2024-09-10T21:47:42.235664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:09:10.560046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235355+0000","last_change":"2024-09-10T21:47:41.249417+0000","last_active":"2024-09-10T21:47:42.235355+0000","last_peered":"2024-09-10T21:47:42.235355+0000","last_clean":"2024-09-10T21:47:42.235355+0000","last_became_active":"2024-09-10T21:47:41.249161+0000","last_became_peered":"2024-09-10T21:47:41.249161+0000","last_unstale":"2024-09-10T21:47:42.235355+0000","last_undegraded":"2024-09-10T21:47:42.235355+0000","last_fullsized":"2024-09-10T21:47:42.235355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:32.957441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235348+0000","last_change":"2024-09-10T21:47:40.235977+0000","last_active":"2024-09-10T21:47:42.235348+0000","last_peered":"2024-09-10T21:47:42.235348+0000","last_clean":"2024-09-10T21:47:42.235348+0000","last_became_active":"2024-09-10T21:47:40.233997+0000","last_became_peered":"2024-09-10T21:47:40.233997+0000","last_unstale":"2024-09-10T21:47:42.235348+0000","last_undegraded":"2024-09-10T21:47:42.235348+0000","last_fullsized":"2024-09-10T21:47:42.235348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:29.623676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235693+0000","last_change":"2024-09-10T21:47:41.235249+0000","last_active":"2024-09-10T21:47:42.235693+0000","last_peered":"2024-09-10T21:47:42.235693+0000","last_clean":"2024-09-10T21:47:42.235693+0000","last_became_active":"2024-09-10T21:47:41.234842+0000","last_became_peered":"2024-09-10T21:47:41.234842+0000","last_unstale":"2024-09-10T21:47:42.235693+0000","last_undegraded":"2024-09-10T21:47:42.235693+0000","last_fullsized":"2024-09-10T21:47:42.235693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:37.640217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"84'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234867+0000","last_change":"2024-09-10T21:47:40.242146+0000","last_active":"2024-09-10T21:47:42.234867+0000","last_peered":"2024-09-10T21:47:42.234867+0000","last_clean":"2024-09-10T21:47:42.234867+0000","last_became_active":"2024-09-10T21:47:40.241782+0000","last_became_peered":"2024-09-10T21:47:40.241782+0000","last_unstale":"2024-09-10T21:47:42.234867+0000","last_undegraded":"2024-09-10T21:47:42.234867+0000","last_fullsized":"2024-09-10T21:47:42.234867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:12.613930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229174+0000","last_change":"2024-09-10T21:47:41.242912+0000","last_active":"2024-09-10T21:47:42.229174+0000","last_peered":"2024-09-10T21:47:42.229174+0000","last_clean":"2024-09-10T21:47:42.229174+0000","last_became_active":"2024-09-10T21:47:41.241840+0000","last_became_peered":"2024-09-10T21:47:41.241840+0000","last_unstale":"2024-09-10T21:47:42.229174+0000","last_undegraded":"2024-09-10T21:47:42.229174+0000","last_fullsized":"2024-09-10T21:47:42.229174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:20.509204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"84'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234599+0000","last_change":"2024-09-10T21:47:40.235137+0000","last_active":"2024-09-10T21:47:42.234599+0000","last_peered":"2024-09-10T21:47:42.234599+0000","last_clean":"2024-09-10T21:47:42.234599+0000","last_became_active":"2024-09-10T21:47:40.234813+0000","last_became_peered":"2024-09-10T21:47:40.234813+0000","last_unstale":"2024-09-10T21:47:42.234599+0000","last_undegraded":"2024-09-10T21:47:42.234599+0000","last_fullsized":"2024-09-10T21:47:42.234599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:13.262730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234559+0000","last_change":"2024-09-10T21:47:41.247024+0000","last_active":"2024-09-10T21:47:42.234559+0000","last_peered":"2024-09-10T21:47:42.234559+0000","last_clean":"2024-09-10T21:47:42.234559+0000","last_became_active":"2024-09-10T21:47:41.244824+0000","last_became_peered":"2024-09-10T21:47:41.244824+0000","last_unstale":"2024-09-10T21:47:42.234559+0000","last_undegraded":"2024-09-10T21:47:42.234559+0000","last_fullsized":"2024-09-10T21:47:42.234559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:58.764672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234987+0000","last_change":"2024-09-10T21:47:40.238459+0000","last_active":"2024-09-10T21:47:42.234987+0000","last_peered":"2024-09-10T21:47:42.234987+0000","last_clean":"2024-09-10T21:47:42.234987+0000","last_became_active":"2024-09-10T21:47:40.237990+0000","last_became_peered":"2024-09-10T21:47:40.237990+0000","last_unstale":"2024-09-10T21:47:42.234987+0000","last_undegraded":"2024-09-10T21:47:42.234987+0000","last_fullsized":"2024-09-10T21:47:42.234987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:30.359167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.249459+0000","last_change":"2024-09-10T21:47:41.243816+0000","last_active":"2024-09-10T21:47:41.249459+0000","last_peered":"2024-09-10T21:47:41.249459+0000","last_clean":"2024-09-10T21:47:41.249459+0000","last_became_active":"2024-09-10T21:47:41.242574+0000","last_became_peered":"2024-09-10T21:47:41.242574+0000","last_unstale":"2024-09-10T21:47:41.249459+0000","last_undegraded":"2024-09-10T21:47:41.249459+0000","last_fullsized":"2024-09-10T21:47:41.249459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:35.540097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.222903+0000","last_change":"2024-09-10T21:47:40.246734+0000","last_active":"2024-09-10T21:47:41.222903+0000","last_peered":"2024-09-10T21:47:41.222903+0000","last_clean":"2024-09-10T21:47:41.222903+0000","last_became_active":"2024-09-10T21:47:40.246495+0000","last_became_peered":"2024-09-10T21:47:40.246495+0000","last_unstale":"2024-09-10T21:47:41.222903+0000","last_undegraded":"2024-09-10T21:47:41.222903+0000","last_fullsized":"2024-09-10T21:47:41.222903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:03.725986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234366+0000","last_change":"2024-09-10T21:47:41.238652+0000","last_active":"2024-09-10T21:47:42.234366+0000","last_peered":"2024-09-10T21:47:42.234366+0000","last_clean":"2024-09-10T21:47:42.234366+0000","last_became_active":"2024-09-10T21:47:41.238347+0000","last_became_peered":"2024-09-10T21:47:41.238347+0000","last_unstale":"2024-09-10T21:47:42.234366+0000","last_undegraded":"2024-09-10T21:47:42.234366+0000","last_fullsized":"2024-09-10T21:47:42.234366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:14.087344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234335+0000","last_change":"2024-09-10T21:47:40.234139+0000","last_active":"2024-09-10T21:47:42.234335+0000","last_peered":"2024-09-10T21:47:42.234335+0000","last_clean":"2024-09-10T21:47:42.234335+0000","last_became_active":"2024-09-10T21:47:40.233823+0000","last_became_peered":"2024-09-10T21:47:40.233823+0000","last_unstale":"2024-09-10T21:47:42.234335+0000","last_undegraded":"2024-09-10T21:47:42.234335+0000","last_fullsized":"2024-09-10T21:47:42.234335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:45.199598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234703+0000","last_change":"2024-09-10T21:47:41.244209+0000","last_active":"2024-09-10T21:47:42.234703+0000","last_peered":"2024-09-10T21:47:42.234703+0000","last_clean":"2024-09-10T21:47:42.234703+0000","last_became_active":"2024-09-10T21:47:41.243402+0000","last_became_peered":"2024-09-10T21:47:41.243402+0000","last_unstale":"2024-09-10T21:47:42.234703+0000","last_undegraded":"2024-09-10T21:47:42.234703+0000","last_fullsized":"2024-09-10T21:47:42.234703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:29.272552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"84'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234180+0000","last_change":"2024-09-10T21:47:40.235945+0000","last_active":"2024-09-10T21:47:42.234180+0000","last_peered":"2024-09-10T21:47:42.234180+0000","last_clean":"2024-09-10T21:47:42.234180+0000","last_became_active":"2024-09-10T21:47:40.235672+0000","last_became_peered":"2024-09-10T21:47:40.235672+0000","last_unstale":"2024-09-10T21:47:42.234180+0000","last_undegraded":"2024-09-10T21:47:42.234180+0000","last_fullsized":"2024-09-10T21:47:42.234180+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:11.334128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.247535+0000","last_change":"2024-09-10T21:47:41.240308+0000","last_active":"2024-09-10T21:47:41.247535+0000","last_peered":"2024-09-10T21:47:41.247535+0000","last_clean":"2024-09-10T21:47:41.247535+0000","last_became_active":"2024-09-10T21:47:41.240058+0000","last_became_peered":"2024-09-10T21:47:41.240058+0000","last_unstale":"2024-09-10T21:47:41.247535+0000","last_undegraded":"2024-09-10T21:47:41.247535+0000","last_fullsized":"2024-09-10T21:47:41.247535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:42.696781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235794+0000","last_change":"2024-09-10T21:47:40.233571+0000","last_active":"2024-09-10T21:47:42.235794+0000","last_peered":"2024-09-10T21:47:42.235794+0000","last_clean":"2024-09-10T21:47:42.235794+0000","last_became_active":"2024-09-10T21:47:40.233087+0000","last_became_peered":"2024-09-10T21:47:40.233087+0000","last_unstale":"2024-09-10T21:47:42.235794+0000","last_undegraded":"2024-09-10T21:47:42.235794+0000","last_fullsized":"2024-09-10T21:47:42.235794+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:12.949983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235547+0000","last_change":"2024-09-10T21:47:41.241298+0000","last_active":"2024-09-10T21:47:42.235547+0000","last_peered":"2024-09-10T21:47:42.235547+0000","last_clean":"2024-09-10T21:47:42.235547+0000","last_became_active":"2024-09-10T21:47:41.240689+0000","last_became_peered":"2024-09-10T21:47:41.240689+0000","last_unstale":"2024-09-10T21:47:42.235547+0000","last_undegraded":"2024-09-10T21:47:42.235547+0000","last_fullsized":"2024-09-10T21:47:42.235547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:46.703097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235360+0000","last_change":"2024-09-10T21:47:40.241879+0000","last_active":"2024-09-10T21:47:42.235360+0000","last_peered":"2024-09-10T21:47:42.235360+0000","last_clean":"2024-09-10T21:47:42.235360+0000","last_became_active":"2024-09-10T21:47:40.241542+0000","last_became_peered":"2024-09-10T21:47:40.241542+0000","last_unstale":"2024-09-10T21:47:42.235360+0000","last_undegraded":"2024-09-10T21:47:42.235360+0000","last_fullsized":"2024-09-10T21:47:42.235360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:28.152606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229256+0000","last_change":"2024-09-10T21:47:41.239089+0000","last_active":"2024-09-10T21:47:42.229256+0000","last_peered":"2024-09-10T21:47:42.229256+0000","last_clean":"2024-09-10T21:47:42.229256+0000","last_became_active":"2024-09-10T21:47:41.238786+0000","last_became_peered":"2024-09-10T21:47:41.238786+0000","last_unstale":"2024-09-10T21:47:42.229256+0000","last_undegraded":"2024-09-10T21:47:42.229256+0000","last_fullsized":"2024-09-10T21:47:42.229256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:46.063011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235454+0000","last_change":"2024-09-10T21:47:40.240002+0000","last_active":"2024-09-10T21:47:42.235454+0000","last_peered":"2024-09-10T21:47:42.235454+0000","last_clean":"2024-09-10T21:47:42.235454+0000","last_became_active":"2024-09-10T21:47:40.239707+0000","last_became_peered":"2024-09-10T21:47:40.239707+0000","last_unstale":"2024-09-10T21:47:42.235454+0000","last_undegraded":"2024-09-10T21:47:42.235454+0000","last_fullsized":"2024-09-10T21:47:42.235454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:10.624235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.222845+0000","last_change":"2024-09-10T21:47:41.254933+0000","last_active":"2024-09-10T21:47:42.222845+0000","last_peered":"2024-09-10T21:47:42.222845+0000","last_clean":"2024-09-10T21:47:42.222845+0000","last_became_active":"2024-09-10T21:47:41.254661+0000","last_became_peered":"2024-09-10T21:47:41.254661+0000","last_unstale":"2024-09-10T21:47:42.222845+0000","last_undegraded":"2024-09-10T21:47:42.222845+0000","last_fullsized":"2024-09-10T21:47:42.222845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:26.316387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"84'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.267032+0000","last_change":"2024-09-10T21:47:40.243074+0000","last_active":"2024-09-10T21:47:41.267032+0000","last_peered":"2024-09-10T21:47:41.267032+0000","last_clean":"2024-09-10T21:47:41.267032+0000","last_became_active":"2024-09-10T21:47:40.242818+0000","last_became_peered":"2024-09-10T21:47:40.242818+0000","last_unstale":"2024-09-10T21:47:41.267032+0000","last_undegraded":"2024-09-10T21:47:41.267032+0000","last_fullsized":"2024-09-10T21:47:41.267032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:01.117060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234612+0000","last_change":"2024-09-10T21:47:41.245761+0000","last_active":"2024-09-10T21:47:42.234612+0000","last_peered":"2024-09-10T21:47:42.234612+0000","last_clean":"2024-09-10T21:47:42.234612+0000","last_became_active":"2024-09-10T21:47:41.245464+0000","last_became_peered":"2024-09-10T21:47:41.245464+0000","last_unstale":"2024-09-10T21:47:42.234612+0000","last_undegraded":"2024-09-10T21:47:42.234612+0000","last_fullsized":"2024-09-10T21:47:42.234612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:25.433073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.222966+0000","last_change":"2024-09-10T21:47:40.242433+0000","last_active":"2024-09-10T21:47:42.222966+0000","last_peered":"2024-09-10T21:47:42.222966+0000","last_clean":"2024-09-10T21:47:42.222966+0000","last_became_active":"2024-09-10T21:47:40.242190+0000","last_became_peered":"2024-09-10T21:47:40.242190+0000","last_unstale":"2024-09-10T21:47:42.222966+0000","last_undegraded":"2024-09-10T21:47:42.222966+0000","last_fullsized":"2024-09-10T21:47:42.222966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:33.355716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.222682+0000","last_change":"2024-09-10T21:47:41.243121+0000","last_active":"2024-09-10T21:47:42.222682+0000","last_peered":"2024-09-10T21:47:42.222682+0000","last_clean":"2024-09-10T21:47:42.222682+0000","last_became_active":"2024-09-10T21:47:41.242794+0000","last_became_peered":"2024-09-10T21:47:41.242794+0000","last_unstale":"2024-09-10T21:47:42.222682+0000","last_undegraded":"2024-09-10T21:47:42.222682+0000","last_fullsized":"2024-09-10T21:47:42.222682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:54.603994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.222578+0000","last_change":"2024-09-10T21:47:40.241083+0000","last_active":"2024-09-10T21:47:42.222578+0000","last_peered":"2024-09-10T21:47:42.222578+0000","last_clean":"2024-09-10T21:47:42.222578+0000","last_became_active":"2024-09-10T21:47:40.240753+0000","last_became_peered":"2024-09-10T21:47:40.240753+0000","last_unstale":"2024-09-10T21:47:42.222578+0000","last_undegraded":"2024-09-10T21:47:42.222578+0000","last_fullsized":"2024-09-10T21:47:42.222578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:42:08.697455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234226+0000","last_change":"2024-09-10T21:47:41.242449+0000","last_active":"2024-09-10T21:47:42.234226+0000","last_peered":"2024-09-10T21:47:42.234226+0000","last_clean":"2024-09-10T21:47:42.234226+0000","last_became_active":"2024-09-10T21:47:41.242174+0000","last_became_peered":"2024-09-10T21:47:41.242174+0000","last_unstale":"2024-09-10T21:47:42.234226+0000","last_undegraded":"2024-09-10T21:47:42.234226+0000","last_fullsized":"2024-09-10T21:47:42.234226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:03.481250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.222006+0000","last_change":"2024-09-10T21:47:40.243627+0000","last_active":"2024-09-10T21:47:41.222006+0000","last_peered":"2024-09-10T21:47:41.222006+0000","last_clean":"2024-09-10T21:47:41.222006+0000","last_became_active":"2024-09-10T21:47:40.241668+0000","last_became_peered":"2024-09-10T21:47:40.241668+0000","last_unstale":"2024-09-10T21:47:41.222006+0000","last_undegraded":"2024-09-10T21:47:41.222006+0000","last_fullsized":"2024-09-10T21:47:41.222006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:07.637571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229357+0000","last_change":"2024-09-10T21:47:41.241102+0000","last_active":"2024-09-10T21:47:42.229357+0000","last_peered":"2024-09-10T21:47:42.229357+0000","last_clean":"2024-09-10T21:47:42.229357+0000","last_became_active":"2024-09-10T21:47:41.240802+0000","last_became_peered":"2024-09-10T21:47:41.240802+0000","last_unstale":"2024-09-10T21:47:42.229357+0000","last_undegraded":"2024-09-10T21:47:42.229357+0000","last_fullsized":"2024-09-10T21:47:42.229357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:12.345333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235590+0000","last_change":"2024-09-10T21:47:40.241148+0000","last_active":"2024-09-10T21:47:42.235590+0000","last_peered":"2024-09-10T21:47:42.235590+0000","last_clean":"2024-09-10T21:47:42.235590+0000","last_became_active":"2024-09-10T21:47:40.240394+0000","last_became_peered":"2024-09-10T21:47:40.240394+0000","last_unstale":"2024-09-10T21:47:42.235590+0000","last_undegraded":"2024-09-10T21:47:42.235590+0000","last_fullsized":"2024-09-10T21:47:42.235590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:42.958092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.249612+0000","last_change":"2024-09-10T21:47:41.244199+0000","last_active":"2024-09-10T21:47:41.249612+0000","last_peered":"2024-09-10T21:47:41.249612+0000","last_clean":"2024-09-10T21:47:41.249612+0000","last_became_active":"2024-09-10T21:47:41.242308+0000","last_became_peered":"2024-09-10T21:47:41.242308+0000","last_unstale":"2024-09-10T21:47:41.249612+0000","last_undegraded":"2024-09-10T21:47:41.249612+0000","last_fullsized":"2024-09-10T21:47:41.249612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:27.320599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235668+0000","last_change":"2024-09-10T21:47:40.240006+0000","last_active":"2024-09-10T21:47:42.235668+0000","last_peered":"2024-09-10T21:47:42.235668+0000","last_clean":"2024-09-10T21:47:42.235668+0000","last_became_active":"2024-09-10T21:47:40.238454+0000","last_became_peered":"2024-09-10T21:47:40.238454+0000","last_unstale":"2024-09-10T21:47:42.235668+0000","last_undegraded":"2024-09-10T21:47:42.235668+0000","last_fullsized":"2024-09-10T21:47:42.235668+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:31.052079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234946+0000","last_change":"2024-09-10T21:47:41.249745+0000","last_active":"2024-09-10T21:47:42.234946+0000","last_peered":"2024-09-10T21:47:42.234946+0000","last_clean":"2024-09-10T21:47:42.234946+0000","last_became_active":"2024-09-10T21:47:41.248663+0000","last_became_peered":"2024-09-10T21:47:41.248663+0000","last_unstale":"2024-09-10T21:47:42.234946+0000","last_undegraded":"2024-09-10T21:47:42.234946+0000","last_fullsized":"2024-09-10T21:47:42.234946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:36.727787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234969+0000","last_change":"2024-09-10T21:47:40.237681+0000","last_active":"2024-09-10T21:47:42.234969+0000","last_peered":"2024-09-10T21:47:42.234969+0000","last_clean":"2024-09-10T21:47:42.234969+0000","last_became_active":"2024-09-10T21:47:40.237382+0000","last_became_peered":"2024-09-10T21:47:40.237382+0000","last_unstale":"2024-09-10T21:47:42.234969+0000","last_undegraded":"2024-09-10T21:47:42.234969+0000","last_fullsized":"2024-09-10T21:47:42.234969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:48.520227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229478+0000","last_change":"2024-09-10T21:47:41.243208+0000","last_active":"2024-09-10T21:47:42.229478+0000","last_peered":"2024-09-10T21:47:42.229478+0000","last_clean":"2024-09-10T21:47:42.229478+0000","last_became_active":"2024-09-10T21:47:41.241208+0000","last_became_peered":"2024-09-10T21:47:41.241208+0000","last_unstale":"2024-09-10T21:47:42.229478+0000","last_undegraded":"2024-09-10T21:47:42.229478+0000","last_fullsized":"2024-09-10T21:47:42.229478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:01.015998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234980+0000","last_change":"2024-09-10T21:47:40.241599+0000","last_active":"2024-09-10T21:47:42.234980+0000","last_peered":"2024-09-10T21:47:42.234980+0000","last_clean":"2024-09-10T21:47:42.234980+0000","last_became_active":"2024-09-10T21:47:40.241261+0000","last_became_peered":"2024-09-10T21:47:40.241261+0000","last_unstale":"2024-09-10T21:47:42.234980+0000","last_undegraded":"2024-09-10T21:47:42.234980+0000","last_fullsized":"2024-09-10T21:47:42.234980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:54.761697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234297+0000","last_change":"2024-09-10T21:47:41.240907+0000","last_active":"2024-09-10T21:47:42.234297+0000","last_peered":"2024-09-10T21:47:42.234297+0000","last_clean":"2024-09-10T21:47:42.234297+0000","last_became_active":"2024-09-10T21:47:41.240623+0000","last_became_peered":"2024-09-10T21:47:41.240623+0000","last_unstale":"2024-09-10T21:47:42.234297+0000","last_undegraded":"2024-09-10T21:47:42.234297+0000","last_fullsized":"2024-09-10T21:47:42.234297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:40.161098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235132+0000","last_change":"2024-09-10T21:47:40.239305+0000","last_active":"2024-09-10T21:47:42.235132+0000","last_peered":"2024-09-10T21:47:42.235132+0000","last_clean":"2024-09-10T21:47:42.235132+0000","last_became_active":"2024-09-10T21:47:40.239033+0000","last_became_peered":"2024-09-10T21:47:40.239033+0000","last_unstale":"2024-09-10T21:47:42.235132+0000","last_undegraded":"2024-09-10T21:47:42.235132+0000","last_fullsized":"2024-09-10T21:47:42.235132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:39.149894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235298+0000","last_change":"2024-09-10T21:47:41.250246+0000","last_active":"2024-09-10T21:47:42.235298+0000","last_peered":"2024-09-10T21:47:42.235298+0000","last_clean":"2024-09-10T21:47:42.235298+0000","last_became_active":"2024-09-10T21:47:41.245991+0000","last_became_peered":"2024-09-10T21:47:41.245991+0000","last_unstale":"2024-09-10T21:47:42.235298+0000","last_undegraded":"2024-09-10T21:47:42.235298+0000","last_fullsized":"2024-09-10T21:47:42.235298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:54.204331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.228917+0000","last_change":"2024-09-10T21:47:40.230275+0000","last_active":"2024-09-10T21:47:42.228917+0000","last_peered":"2024-09-10T21:47:42.228917+0000","last_clean":"2024-09-10T21:47:42.228917+0000","last_became_active":"2024-09-10T21:47:40.230015+0000","last_became_peered":"2024-09-10T21:47:40.230015+0000","last_unstale":"2024-09-10T21:47:42.228917+0000","last_undegraded":"2024-09-10T21:47:42.228917+0000","last_fullsized":"2024-09-10T21:47:42.228917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:21.852224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.258051+0000","last_change":"2024-09-10T21:47:41.254403+0000","last_active":"2024-09-10T21:47:41.258051+0000","last_peered":"2024-09-10T21:47:41.258051+0000","last_clean":"2024-09-10T21:47:41.258051+0000","last_became_active":"2024-09-10T21:47:41.254093+0000","last_became_peered":"2024-09-10T21:47:41.254093+0000","last_unstale":"2024-09-10T21:47:41.258051+0000","last_undegraded":"2024-09-10T21:47:41.258051+0000","last_fullsized":"2024-09-10T21:47:41.258051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:39.704570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234209+0000","last_change":"2024-09-10T21:47:40.248048+0000","last_active":"2024-09-10T21:47:42.234209+0000","last_peered":"2024-09-10T21:47:42.234209+0000","last_clean":"2024-09-10T21:47:42.234209+0000","last_became_active":"2024-09-10T21:47:40.247833+0000","last_became_peered":"2024-09-10T21:47:40.247833+0000","last_unstale":"2024-09-10T21:47:42.234209+0000","last_undegraded":"2024-09-10T21:47:42.234209+0000","last_fullsized":"2024-09-10T21:47:42.234209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:54.026456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234296+0000","last_change":"2024-09-10T21:47:41.248910+0000","last_active":"2024-09-10T21:47:42.234296+0000","last_peered":"2024-09-10T21:47:42.234296+0000","last_clean":"2024-09-10T21:47:42.234296+0000","last_became_active":"2024-09-10T21:47:41.248542+0000","last_became_peered":"2024-09-10T21:47:41.248542+0000","last_unstale":"2024-09-10T21:47:42.234296+0000","last_undegraded":"2024-09-10T21:47:42.234296+0000","last_fullsized":"2024-09-10T21:47:42.234296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:12.345333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.228607+0000","last_change":"2024-09-10T21:47:40.237251+0000","last_active":"2024-09-10T21:47:42.228607+0000","last_peered":"2024-09-10T21:47:42.228607+0000","last_clean":"2024-09-10T21:47:42.228607+0000","last_became_active":"2024-09-10T21:47:40.236969+0000","last_became_peered":"2024-09-10T21:47:40.236969+0000","last_unstale":"2024-09-10T21:47:42.228607+0000","last_undegraded":"2024-09-10T21:47:42.228607+0000","last_fullsized":"2024-09-10T21:47:42.228607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:09:10.560046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235018+0000","last_change":"2024-09-10T21:47:41.244219+0000","last_active":"2024-09-10T21:47:42.235018+0000","last_peered":"2024-09-10T21:47:42.235018+0000","last_clean":"2024-09-10T21:47:42.235018+0000","last_became_active":"2024-09-10T21:47:41.243918+0000","last_became_peered":"2024-09-10T21:47:41.243918+0000","last_unstale":"2024-09-10T21:47:42.235018+0000","last_undegraded":"2024-09-10T21:47:42.235018+0000","last_fullsized":"2024-09-10T21:47:42.235018+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:31.044003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.264630+0000","last_change":"2024-09-10T21:47:40.242532+0000","last_active":"2024-09-10T21:47:41.264630+0000","last_peered":"2024-09-10T21:47:41.264630+0000","last_clean":"2024-09-10T21:47:41.264630+0000","last_became_active":"2024-09-10T21:47:40.241942+0000","last_became_peered":"2024-09-10T21:47:40.241942+0000","last_unstale":"2024-09-10T21:47:41.264630+0000","last_undegraded":"2024-09-10T21:47:41.264630+0000","last_fullsized":"2024-09-10T21:47:41.264630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:27.266025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":225,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.231048+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:47:41.231048+0000","last_peered":"2024-09-10T21:47:41.231048+0000","last_clean":"2024-09-10T21:47:41.231048+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:47:41.231048+0000","last_undegraded":"2024-09-10T21:47:41.231048+0000","last_fullsized":"2024-09-10T21:47:41.231048+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509513,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1048576,"data_stored":441358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509513,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1052672,"data_stored":442298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509512,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1036288,"data_stored":440664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509512,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1028096,"data_stored":434946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509514,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1048576,"data_stored":440858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509512,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1622016,"data_stored":1024766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542215,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32776,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339064,"statfs":{"total":96636764160,"available":96603201536,"internally_reserved":0,"allocated":1622016,"data_stored":1024802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542216,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36340,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335500,"statfs":{"total":96636764160,"available":96599552000,"internally_reserved":0,"allocated":1011712,"data_stored":427589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:47:45.492 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:47:45.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.686+0000 7f96a0400640 1 -- 172.21.15.7:0/327231145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96980cc190 msgr2=0x7f96980cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:45.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.686+0000 7f96a0400640 1 --2- 172.21.15.7:0/327231145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96980cc190 0x7f96980cd750 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f9690005fd0 tx=0x7f969002a8b0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.686+0000 7f96a0400640 1 -- 172.21.15.7:0/327231145 shutdown_connections 2024-09-10T21:47:45.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.686+0000 7f96a0400640 1 --2- 172.21.15.7:0/327231145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96980cc190 0x7f96980cd750 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.686+0000 7f96a0400640 1 --2- 172.21.15.7:0/327231145 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96981619e0 0x7f969815e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.686+0000 7f96a0400640 1 -- 172.21.15.7:0/327231145 >> 172.21.15.7:0/327231145 conn(0x7f96980b16d0 msgr2=0x7f96980b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:45.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.687+0000 7f96a0400640 1 -- 172.21.15.7:0/327231145 shutdown_connections 2024-09-10T21:47:45.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.687+0000 7f96a0400640 1 -- 172.21.15.7:0/327231145 wait complete. 2024-09-10T21:47:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.688+0000 7f96a0400640 1 Processor -- start 2024-09-10T21:47:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.688+0000 7f96a0400640 1 -- start start 2024-09-10T21:47:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.689+0000 7f96a0400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96980cbd50 0x7f9698157020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.689+0000 7f96a0400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96980cc190 0x7f96981f49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.689+0000 7f96a0400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96981619e0 0x7f96980c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:45.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.689+0000 7f96a0400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9698161dc0 con 0x7f96980cc190 2024-09-10T21:47:45.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.689+0000 7f96a0400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f96980cdfb0 con 0x7f96981619e0 2024-09-10T21:47:45.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.689+0000 7f96a0400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f9698161f40 con 0x7f96980cbd50 2024-09-10T21:47:45.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.689+0000 7f969d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96980cc190 0x7f96981f49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:45.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.689+0000 7f969d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96980cc190 0x7f96981f49c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57732/0 (socket says 172.21.15.7:57732) 2024-09-10T21:47:45.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.689+0000 7f969d000640 1 -- 172.21.15.7:0/2135391855 learned_addr learned my addr 172.21.15.7:0/2135391855 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:45.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.690+0000 7f969da00640 1 --2- 172.21.15.7:0/2135391855 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96980cbd50 0x7f9698157020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:45.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.690+0000 7f969e400640 1 --2- 172.21.15.7:0/2135391855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96981619e0 0x7f96980c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:45.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.690+0000 7f969d000640 1 -- 172.21.15.7:0/2135391855 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96980cbd50 msgr2=0x7f9698157020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:45.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.690+0000 7f969d000640 1 --2- 172.21.15.7:0/2135391855 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96980cbd50 0x7f9698157020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.690+0000 7f969d000640 1 -- 172.21.15.7:0/2135391855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96981619e0 msgr2=0x7f96980c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:45.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.690+0000 7f969d000640 1 --2- 172.21.15.7:0/2135391855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96981619e0 0x7f96980c59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.690+0000 7f969d000640 1 -- 172.21.15.7:0/2135391855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96981f5440 con 0x7f96980cc190 2024-09-10T21:47:45.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.690+0000 7f969d000640 1 --2- 172.21.15.7:0/2135391855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96980cc190 0x7f96981f49c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f968400c390 tx=0x7f968400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:45.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.690+0000 7f9695600640 1 -- 172.21.15.7:0/2135391855 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684017070 con 0x7f96980cc190 2024-09-10T21:47:45.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.690+0000 7f9695600640 1 -- 172.21.15.7:0/2135391855 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96840035f0 con 0x7f96980cc190 2024-09-10T21:47:45.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.690+0000 7f9695600640 1 -- 172.21.15.7:0/2135391855 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9684003a60 con 0x7f96980cc190 2024-09-10T21:47:45.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.690+0000 7f96a0400640 1 -- 172.21.15.7:0/2135391855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96981f5720 con 0x7f96980cc190 2024-09-10T21:47:45.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.690+0000 7f96a0400640 1 -- 172.21.15.7:0/2135391855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9698072f10 con 0x7f96980cc190 2024-09-10T21:47:45.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.691+0000 7f96a0400640 1 -- 172.21.15.7:0/2135391855 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96980cda60 con 0x7f96980cc190 2024-09-10T21:47:45.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.693+0000 7f9695600640 1 -- 172.21.15.7:0/2135391855 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9684003c00 con 0x7f96980cc190 2024-09-10T21:47:45.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.693+0000 7f9695600640 1 --2- 172.21.15.7:0/2135391855 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9678077f70 0x7f967807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:45.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.694+0000 7f9695600640 1 -- 172.21.15.7:0/2135391855 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f96840993c0 con 0x7f96980cc190 2024-09-10T21:47:45.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.694+0000 7f969da00640 1 --2- 172.21.15.7:0/2135391855 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9678077f70 0x7f967807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:45.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.695+0000 7f969da00640 1 --2- 172.21.15.7:0/2135391855 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9678077f70 0x7f967807a430 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f968c005fd0 tx=0x7f968c005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:45.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.698+0000 7f9695600640 1 -- 172.21.15.7:0/2135391855 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f96980cda60 con 0x7f96980cc190 2024-09-10T21:47:45.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.887+0000 7f96a0400640 1 -- 172.21.15.7:0/2135391855 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f96980cd750 con 0x7f9678077f70 2024-09-10T21:47:45.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.892+0000 7f9695600640 1 -- 172.21.15.7:0/2135391855 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252070 (secure 0 0 0) 0x7f96980cd750 con 0x7f9678077f70 2024-09-10T21:47:45.896 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:45.897 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:47:45.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.896+0000 7f96a0400640 1 -- 172.21.15.7:0/2135391855 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9678077f70 msgr2=0x7f967807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.896+0000 7f96a0400640 1 --2- 172.21.15.7:0/2135391855 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9678077f70 0x7f967807a430 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f968c005fd0 tx=0x7f968c005850 comp rx=0 tx=0).stop 2024-09-10T21:47:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.896+0000 7f96a0400640 1 -- 172.21.15.7:0/2135391855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96980cc190 msgr2=0x7f96981f49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.896+0000 7f96a0400640 1 --2- 172.21.15.7:0/2135391855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96980cc190 0x7f96981f49c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f968400c390 tx=0x7f968400c850 comp rx=0 tx=0).stop 2024-09-10T21:47:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.896+0000 7f96a0400640 1 -- 172.21.15.7:0/2135391855 shutdown_connections 2024-09-10T21:47:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.896+0000 7f96a0400640 1 --2- 172.21.15.7:0/2135391855 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9678077f70 0x7f967807a430 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.896+0000 7f96a0400640 1 --2- 172.21.15.7:0/2135391855 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96981619e0 0x7f96980c59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.896+0000 7f96a0400640 1 --2- 172.21.15.7:0/2135391855 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96980cc190 0x7f96981f49c0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.896+0000 7f96a0400640 1 --2- 172.21.15.7:0/2135391855 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96980cbd50 0x7f9698157020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.896+0000 7f96a0400640 1 -- 172.21.15.7:0/2135391855 >> 172.21.15.7:0/2135391855 conn(0x7f96980b16d0 msgr2=0x7f9698157f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.896+0000 7f96a0400640 1 -- 172.21.15.7:0/2135391855 shutdown_connections 2024-09-10T21:47:45.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:45.896+0000 7f96a0400640 1 -- 172.21.15.7:0/2135391855 wait complete. 2024-09-10T21:47:45.914 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-09-10T21:47:43.943155+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262940,"kb_used_data":9248,"kb_used_omap":12,"kb_used_meta":253491,"kb_avail":754711780,"statfs":{"total":773094113280,"available":772824862720,"internally_reserved":0,"allocated":9469952,"data_stored":4677281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":259575345},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.732214"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235125+0000","last_change":"2024-09-10T21:47:41.242942+0000","last_active":"2024-09-10T21:47:42.235125+0000","last_peered":"2024-09-10T21:47:42.235125+0000","last_clean":"2024-09-10T21:47:42.235125+0000","last_became_active":"2024-09-10T21:47:41.242607+0000","last_became_peered":"2024-09-10T21:47:41.242607+0000","last_unstale":"2024-09-10T21:47:42.235125+0000","last_undegraded":"2024-09-10T21:47:42.235125+0000","last_fullsized":"2024-09-10T21:47:42.235125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:40.161098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235567+0000","last_change":"2024-09-10T21:47:41.248615+0000","last_active":"2024-09-10T21:47:42.235567+0000","last_peered":"2024-09-10T21:47:42.235567+0000","last_clean":"2024-09-10T21:47:42.235567+0000","last_became_active":"2024-09-10T21:47:41.247819+0000","last_became_peered":"2024-09-10T21:47:41.247819+0000","last_unstale":"2024-09-10T21:47:42.235567+0000","last_undegraded":"2024-09-10T21:47:42.235567+0000","last_fullsized":"2024-09-10T21:47:42.235567+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:49.531431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.241295+0000","last_change":"2024-09-10T21:47:41.238692+0000","last_active":"2024-09-10T21:47:41.241295+0000","last_peered":"2024-09-10T21:47:41.241295+0000","last_clean":"2024-09-10T21:47:41.241295+0000","last_became_active":"2024-09-10T21:47:41.237972+0000","last_became_peered":"2024-09-10T21:47:41.237972+0000","last_unstale":"2024-09-10T21:47:41.241295+0000","last_undegraded":"2024-09-10T21:47:41.241295+0000","last_fullsized":"2024-09-10T21:47:41.241295+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:45:32.440482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.223249+0000","last_change":"2024-09-10T21:47:41.255766+0000","last_active":"2024-09-10T21:47:42.223249+0000","last_peered":"2024-09-10T21:47:42.223249+0000","last_clean":"2024-09-10T21:47:42.223249+0000","last_became_active":"2024-09-10T21:47:41.254110+0000","last_became_peered":"2024-09-10T21:47:41.254110+0000","last_unstale":"2024-09-10T21:47:42.223249+0000","last_undegraded":"2024-09-10T21:47:42.223249+0000","last_fullsized":"2024-09-10T21:47:42.223249+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:10:13.319515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229056+0000","last_change":"2024-09-10T21:47:41.249886+0000","last_active":"2024-09-10T21:47:42.229056+0000","last_peered":"2024-09-10T21:47:42.229056+0000","last_clean":"2024-09-10T21:47:42.229056+0000","last_became_active":"2024-09-10T21:47:41.247649+0000","last_became_peered":"2024-09-10T21:47:41.247649+0000","last_unstale":"2024-09-10T21:47:42.229056+0000","last_undegraded":"2024-09-10T21:47:42.229056+0000","last_fullsized":"2024-09-10T21:47:42.229056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:44.426814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234218+0000","last_change":"2024-09-10T21:47:41.243328+0000","last_active":"2024-09-10T21:47:42.234218+0000","last_peered":"2024-09-10T21:47:42.234218+0000","last_clean":"2024-09-10T21:47:42.234218+0000","last_became_active":"2024-09-10T21:47:41.243075+0000","last_became_peered":"2024-09-10T21:47:41.243075+0000","last_unstale":"2024-09-10T21:47:42.234218+0000","last_undegraded":"2024-09-10T21:47:42.234218+0000","last_fullsized":"2024-09-10T21:47:42.234218+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:11.635440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.222627+0000","last_change":"2024-09-10T21:47:41.243710+0000","last_active":"2024-09-10T21:47:42.222627+0000","last_peered":"2024-09-10T21:47:42.222627+0000","last_clean":"2024-09-10T21:47:42.222627+0000","last_became_active":"2024-09-10T21:47:41.240792+0000","last_became_peered":"2024-09-10T21:47:41.240792+0000","last_unstale":"2024-09-10T21:47:42.222627+0000","last_undegraded":"2024-09-10T21:47:42.222627+0000","last_fullsized":"2024-09-10T21:47:42.222627+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:14.435322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234034+0000","last_change":"2024-09-10T21:47:41.261163+0000","last_active":"2024-09-10T21:47:42.234034+0000","last_peered":"2024-09-10T21:47:42.234034+0000","last_clean":"2024-09-10T21:47:42.234034+0000","last_became_active":"2024-09-10T21:47:41.260878+0000","last_became_peered":"2024-09-10T21:47:41.260878+0000","last_unstale":"2024-09-10T21:47:42.234034+0000","last_undegraded":"2024-09-10T21:47:42.234034+0000","last_fullsized":"2024-09-10T21:47:42.234034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:01.015998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235157+0000","last_change":"2024-09-10T21:47:41.253326+0000","last_active":"2024-09-10T21:47:42.235157+0000","last_peered":"2024-09-10T21:47:42.235157+0000","last_clean":"2024-09-10T21:47:42.235157+0000","last_became_active":"2024-09-10T21:47:41.252434+0000","last_became_peered":"2024-09-10T21:47:41.252434+0000","last_unstale":"2024-09-10T21:47:42.235157+0000","last_undegraded":"2024-09-10T21:47:42.235157+0000","last_fullsized":"2024-09-10T21:47:42.235157+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:59.893460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235246+0000","last_change":"2024-09-10T21:47:41.235235+0000","last_active":"2024-09-10T21:47:42.235246+0000","last_peered":"2024-09-10T21:47:42.235246+0000","last_clean":"2024-09-10T21:47:42.235246+0000","last_became_active":"2024-09-10T21:47:41.234969+0000","last_became_peered":"2024-09-10T21:47:41.234969+0000","last_unstale":"2024-09-10T21:47:42.235246+0000","last_undegraded":"2024-09-10T21:47:42.235246+0000","last_fullsized":"2024-09-10T21:47:42.235246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:25.433073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235086+0000","last_change":"2024-09-10T21:47:41.243912+0000","last_active":"2024-09-10T21:47:42.235086+0000","last_peered":"2024-09-10T21:47:42.235086+0000","last_clean":"2024-09-10T21:47:42.235086+0000","last_became_active":"2024-09-10T21:47:41.243584+0000","last_became_peered":"2024-09-10T21:47:41.243584+0000","last_unstale":"2024-09-10T21:47:42.235086+0000","last_undegraded":"2024-09-10T21:47:42.235086+0000","last_fullsized":"2024-09-10T21:47:42.235086+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:53.542101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234547+0000","last_change":"2024-09-10T21:47:41.235211+0000","last_active":"2024-09-10T21:47:42.234547+0000","last_peered":"2024-09-10T21:47:42.234547+0000","last_clean":"2024-09-10T21:47:42.234547+0000","last_became_active":"2024-09-10T21:47:41.234939+0000","last_became_peered":"2024-09-10T21:47:41.234939+0000","last_unstale":"2024-09-10T21:47:42.234547+0000","last_undegraded":"2024-09-10T21:47:42.234547+0000","last_fullsized":"2024-09-10T21:47:42.234547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:29.163811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.249504+0000","last_change":"2024-09-10T21:47:41.245469+0000","last_active":"2024-09-10T21:47:41.249504+0000","last_peered":"2024-09-10T21:47:41.249504+0000","last_clean":"2024-09-10T21:47:41.249504+0000","last_became_active":"2024-09-10T21:47:41.245114+0000","last_became_peered":"2024-09-10T21:47:41.245114+0000","last_unstale":"2024-09-10T21:47:41.249504+0000","last_undegraded":"2024-09-10T21:47:41.249504+0000","last_fullsized":"2024-09-10T21:47:41.249504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:13.145479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.222837+0000","last_change":"2024-09-10T21:47:41.256279+0000","last_active":"2024-09-10T21:47:42.222837+0000","last_peered":"2024-09-10T21:47:42.222837+0000","last_clean":"2024-09-10T21:47:42.222837+0000","last_became_active":"2024-09-10T21:47:41.255992+0000","last_became_peered":"2024-09-10T21:47:41.255992+0000","last_unstale":"2024-09-10T21:47:42.222837+0000","last_undegraded":"2024-09-10T21:47:42.222837+0000","last_fullsized":"2024-09-10T21:47:42.222837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:24.910353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.241215+0000","last_change":"2024-09-10T21:47:41.238041+0000","last_active":"2024-09-10T21:47:41.241215+0000","last_peered":"2024-09-10T21:47:41.241215+0000","last_clean":"2024-09-10T21:47:41.241215+0000","last_became_active":"2024-09-10T21:47:41.237691+0000","last_became_peered":"2024-09-10T21:47:41.237691+0000","last_unstale":"2024-09-10T21:47:41.241215+0000","last_undegraded":"2024-09-10T21:47:41.241215+0000","last_fullsized":"2024-09-10T21:47:41.241215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:38.220940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.228680+0000","last_change":"2024-09-10T21:47:41.233514+0000","last_active":"2024-09-10T21:47:42.228680+0000","last_peered":"2024-09-10T21:47:42.228680+0000","last_clean":"2024-09-10T21:47:42.228680+0000","last_became_active":"2024-09-10T21:47:41.233255+0000","last_became_peered":"2024-09-10T21:47:41.233255+0000","last_unstale":"2024-09-10T21:47:42.228680+0000","last_undegraded":"2024-09-10T21:47:42.228680+0000","last_fullsized":"2024-09-10T21:47:42.228680+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:46.210802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229519+0000","last_change":"2024-09-10T21:47:41.251338+0000","last_active":"2024-09-10T21:47:42.229519+0000","last_peered":"2024-09-10T21:47:42.229519+0000","last_clean":"2024-09-10T21:47:42.229519+0000","last_became_active":"2024-09-10T21:47:41.250168+0000","last_became_peered":"2024-09-10T21:47:41.250168+0000","last_unstale":"2024-09-10T21:47:42.229519+0000","last_undegraded":"2024-09-10T21:47:42.229519+0000","last_fullsized":"2024-09-10T21:47:42.229519+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:11.635440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.222913+0000","last_change":"2024-09-10T21:47:41.236969+0000","last_active":"2024-09-10T21:47:42.222913+0000","last_peered":"2024-09-10T21:47:42.222913+0000","last_clean":"2024-09-10T21:47:42.222913+0000","last_became_active":"2024-09-10T21:47:41.234889+0000","last_became_peered":"2024-09-10T21:47:41.234889+0000","last_unstale":"2024-09-10T21:47:42.222913+0000","last_undegraded":"2024-09-10T21:47:42.222913+0000","last_fullsized":"2024-09-10T21:47:42.222913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:22.243388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235536+0000","last_change":"2024-09-10T21:47:41.249077+0000","last_active":"2024-09-10T21:47:42.235536+0000","last_peered":"2024-09-10T21:47:42.235536+0000","last_clean":"2024-09-10T21:47:42.235536+0000","last_became_active":"2024-09-10T21:47:41.247147+0000","last_became_peered":"2024-09-10T21:47:41.247147+0000","last_unstale":"2024-09-10T21:47:42.235536+0000","last_undegraded":"2024-09-10T21:47:42.235536+0000","last_fullsized":"2024-09-10T21:47:42.235536+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:02:03.990747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.223309+0000","last_change":"2024-09-10T21:47:41.253091+0000","last_active":"2024-09-10T21:47:42.223309+0000","last_peered":"2024-09-10T21:47:42.223309+0000","last_clean":"2024-09-10T21:47:42.223309+0000","last_became_active":"2024-09-10T21:47:41.252586+0000","last_became_peered":"2024-09-10T21:47:41.252586+0000","last_unstale":"2024-09-10T21:47:42.223309+0000","last_undegraded":"2024-09-10T21:47:42.223309+0000","last_fullsized":"2024-09-10T21:47:42.223309+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:31:01.443545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.222996+0000","last_change":"2024-09-10T21:47:41.245414+0000","last_active":"2024-09-10T21:47:42.222996+0000","last_peered":"2024-09-10T21:47:42.222996+0000","last_clean":"2024-09-10T21:47:42.222996+0000","last_became_active":"2024-09-10T21:47:41.245153+0000","last_became_peered":"2024-09-10T21:47:41.245153+0000","last_unstale":"2024-09-10T21:47:42.222996+0000","last_undegraded":"2024-09-10T21:47:42.222996+0000","last_fullsized":"2024-09-10T21:47:42.222996+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:53.407700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234889+0000","last_change":"2024-09-10T21:47:41.251176+0000","last_active":"2024-09-10T21:47:42.234889+0000","last_peered":"2024-09-10T21:47:42.234889+0000","last_clean":"2024-09-10T21:47:42.234889+0000","last_became_active":"2024-09-10T21:47:41.249983+0000","last_became_peered":"2024-09-10T21:47:41.249983+0000","last_unstale":"2024-09-10T21:47:42.234889+0000","last_undegraded":"2024-09-10T21:47:42.234889+0000","last_fullsized":"2024-09-10T21:47:42.234889+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:06.277899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.233991+0000","last_change":"2024-09-10T21:47:41.243620+0000","last_active":"2024-09-10T21:47:42.233991+0000","last_peered":"2024-09-10T21:47:42.233991+0000","last_clean":"2024-09-10T21:47:42.233991+0000","last_became_active":"2024-09-10T21:47:41.243367+0000","last_became_peered":"2024-09-10T21:47:41.243367+0000","last_unstale":"2024-09-10T21:47:42.233991+0000","last_undegraded":"2024-09-10T21:47:42.233991+0000","last_fullsized":"2024-09-10T21:47:42.233991+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:46.523445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.222784+0000","last_change":"2024-09-10T21:47:41.235364+0000","last_active":"2024-09-10T21:47:42.222784+0000","last_peered":"2024-09-10T21:47:42.222784+0000","last_clean":"2024-09-10T21:47:42.222784+0000","last_became_active":"2024-09-10T21:47:41.235083+0000","last_became_peered":"2024-09-10T21:47:41.235083+0000","last_unstale":"2024-09-10T21:47:42.222784+0000","last_undegraded":"2024-09-10T21:47:42.222784+0000","last_fullsized":"2024-09-10T21:47:42.222784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:36:15.827659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235526+0000","last_change":"2024-09-10T21:47:41.250729+0000","last_active":"2024-09-10T21:47:42.235526+0000","last_peered":"2024-09-10T21:47:42.235526+0000","last_clean":"2024-09-10T21:47:42.235526+0000","last_became_active":"2024-09-10T21:47:41.250469+0000","last_became_peered":"2024-09-10T21:47:41.250469+0000","last_unstale":"2024-09-10T21:47:42.235526+0000","last_undegraded":"2024-09-10T21:47:42.235526+0000","last_fullsized":"2024-09-10T21:47:42.235526+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:13.625135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.258071+0000","last_change":"2024-09-10T21:47:41.254500+0000","last_active":"2024-09-10T21:47:41.258071+0000","last_peered":"2024-09-10T21:47:41.258071+0000","last_clean":"2024-09-10T21:47:41.258071+0000","last_became_active":"2024-09-10T21:47:41.254077+0000","last_became_peered":"2024-09-10T21:47:41.254077+0000","last_unstale":"2024-09-10T21:47:41.258071+0000","last_undegraded":"2024-09-10T21:47:41.258071+0000","last_fullsized":"2024-09-10T21:47:41.258071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:07:29.557158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.228998+0000","last_change":"2024-09-10T21:47:41.243643+0000","last_active":"2024-09-10T21:47:42.228998+0000","last_peered":"2024-09-10T21:47:42.228998+0000","last_clean":"2024-09-10T21:47:42.228998+0000","last_became_active":"2024-09-10T21:47:41.240706+0000","last_became_peered":"2024-09-10T21:47:41.240706+0000","last_unstale":"2024-09-10T21:47:42.228998+0000","last_undegraded":"2024-09-10T21:47:42.228998+0000","last_fullsized":"2024-09-10T21:47:42.228998+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:33.342304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234601+0000","last_change":"2024-09-10T21:47:41.242344+0000","last_active":"2024-09-10T21:47:42.234601+0000","last_peered":"2024-09-10T21:47:42.234601+0000","last_clean":"2024-09-10T21:47:42.234601+0000","last_became_active":"2024-09-10T21:47:41.242006+0000","last_became_peered":"2024-09-10T21:47:41.242006+0000","last_unstale":"2024-09-10T21:47:42.234601+0000","last_undegraded":"2024-09-10T21:47:42.234601+0000","last_fullsized":"2024-09-10T21:47:42.234601+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:01.015998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235153+0000","last_change":"2024-09-10T21:47:41.239244+0000","last_active":"2024-09-10T21:47:42.235153+0000","last_peered":"2024-09-10T21:47:42.235153+0000","last_clean":"2024-09-10T21:47:42.235153+0000","last_became_active":"2024-09-10T21:47:41.238892+0000","last_became_peered":"2024-09-10T21:47:41.238892+0000","last_unstale":"2024-09-10T21:47:42.235153+0000","last_undegraded":"2024-09-10T21:47:42.235153+0000","last_fullsized":"2024-09-10T21:47:42.235153+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:09:11.571250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235175+0000","last_change":"2024-09-10T21:47:41.245615+0000","last_active":"2024-09-10T21:47:42.235175+0000","last_peered":"2024-09-10T21:47:42.235175+0000","last_clean":"2024-09-10T21:47:42.235175+0000","last_became_active":"2024-09-10T21:47:41.245331+0000","last_became_peered":"2024-09-10T21:47:41.245331+0000","last_unstale":"2024-09-10T21:47:42.235175+0000","last_undegraded":"2024-09-10T21:47:42.235175+0000","last_fullsized":"2024-09-10T21:47:42.235175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:55.772901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.228872+0000","last_change":"2024-09-10T21:47:41.237200+0000","last_active":"2024-09-10T21:47:42.228872+0000","last_peered":"2024-09-10T21:47:42.228872+0000","last_clean":"2024-09-10T21:47:42.228872+0000","last_became_active":"2024-09-10T21:47:41.236822+0000","last_became_peered":"2024-09-10T21:47:41.236822+0000","last_unstale":"2024-09-10T21:47:42.228872+0000","last_undegraded":"2024-09-10T21:47:42.228872+0000","last_fullsized":"2024-09-10T21:47:42.228872+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:14.273935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.233955+0000","last_change":"2024-09-10T21:47:41.251791+0000","last_active":"2024-09-10T21:47:42.233955+0000","last_peered":"2024-09-10T21:47:42.233955+0000","last_clean":"2024-09-10T21:47:42.233955+0000","last_became_active":"2024-09-10T21:47:41.251272+0000","last_became_peered":"2024-09-10T21:47:41.251272+0000","last_unstale":"2024-09-10T21:47:42.233955+0000","last_undegraded":"2024-09-10T21:47:42.233955+0000","last_fullsized":"2024-09-10T21:47:42.233955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:46.063011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229257+0000","last_change":"2024-09-10T21:47:41.234915+0000","last_active":"2024-09-10T21:47:42.229257+0000","last_peered":"2024-09-10T21:47:42.229257+0000","last_clean":"2024-09-10T21:47:42.229257+0000","last_became_active":"2024-09-10T21:47:41.234582+0000","last_became_peered":"2024-09-10T21:47:41.234582+0000","last_unstale":"2024-09-10T21:47:42.229257+0000","last_undegraded":"2024-09-10T21:47:42.229257+0000","last_fullsized":"2024-09-10T21:47:42.229257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:25:57.517851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"84'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235667+0000","last_change":"2024-09-10T21:47:40.238034+0000","last_active":"2024-09-10T21:47:42.235667+0000","last_peered":"2024-09-10T21:47:42.235667+0000","last_clean":"2024-09-10T21:47:42.235667+0000","last_became_active":"2024-09-10T21:47:40.237748+0000","last_became_peered":"2024-09-10T21:47:40.237748+0000","last_unstale":"2024-09-10T21:47:42.235667+0000","last_undegraded":"2024-09-10T21:47:42.235667+0000","last_fullsized":"2024-09-10T21:47:42.235667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:23.214036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235550+0000","last_change":"2024-09-10T21:47:41.241430+0000","last_active":"2024-09-10T21:47:42.235550+0000","last_peered":"2024-09-10T21:47:42.235550+0000","last_clean":"2024-09-10T21:47:42.235550+0000","last_became_active":"2024-09-10T21:47:41.241077+0000","last_became_peered":"2024-09-10T21:47:41.241077+0000","last_unstale":"2024-09-10T21:47:42.235550+0000","last_undegraded":"2024-09-10T21:47:42.235550+0000","last_fullsized":"2024-09-10T21:47:42.235550+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:24:59.893460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229399+0000","last_change":"2024-09-10T21:47:40.227811+0000","last_active":"2024-09-10T21:47:42.229399+0000","last_peered":"2024-09-10T21:47:42.229399+0000","last_clean":"2024-09-10T21:47:42.229399+0000","last_became_active":"2024-09-10T21:47:40.227399+0000","last_became_peered":"2024-09-10T21:47:40.227399+0000","last_unstale":"2024-09-10T21:47:42.229399+0000","last_undegraded":"2024-09-10T21:47:42.229399+0000","last_fullsized":"2024-09-10T21:47:42.229399+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:42.598409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234854+0000","last_change":"2024-09-10T21:47:41.255071+0000","last_active":"2024-09-10T21:47:42.234854+0000","last_peered":"2024-09-10T21:47:42.234854+0000","last_clean":"2024-09-10T21:47:42.234854+0000","last_became_active":"2024-09-10T21:47:41.254678+0000","last_became_peered":"2024-09-10T21:47:41.254678+0000","last_unstale":"2024-09-10T21:47:42.234854+0000","last_undegraded":"2024-09-10T21:47:42.234854+0000","last_fullsized":"2024-09-10T21:47:42.234854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:11.635440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235478+0000","last_change":"2024-09-10T21:47:40.245284+0000","last_active":"2024-09-10T21:47:42.235478+0000","last_peered":"2024-09-10T21:47:42.235478+0000","last_clean":"2024-09-10T21:47:42.235478+0000","last_became_active":"2024-09-10T21:47:40.244945+0000","last_became_peered":"2024-09-10T21:47:40.244945+0000","last_unstale":"2024-09-10T21:47:42.235478+0000","last_undegraded":"2024-09-10T21:47:42.235478+0000","last_fullsized":"2024-09-10T21:47:42.235478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:35.716582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.249422+0000","last_change":"2024-09-10T21:47:41.241823+0000","last_active":"2024-09-10T21:47:41.249422+0000","last_peered":"2024-09-10T21:47:41.249422+0000","last_clean":"2024-09-10T21:47:41.249422+0000","last_became_active":"2024-09-10T21:47:41.241510+0000","last_became_peered":"2024-09-10T21:47:41.241510+0000","last_unstale":"2024-09-10T21:47:41.249422+0000","last_undegraded":"2024-09-10T21:47:41.249422+0000","last_fullsized":"2024-09-10T21:47:41.249422+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:45:01.147786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235355+0000","last_change":"2024-09-10T21:47:40.245896+0000","last_active":"2024-09-10T21:47:42.235355+0000","last_peered":"2024-09-10T21:47:42.235355+0000","last_clean":"2024-09-10T21:47:42.235355+0000","last_became_active":"2024-09-10T21:47:40.245556+0000","last_became_peered":"2024-09-10T21:47:40.245556+0000","last_unstale":"2024-09-10T21:47:42.235355+0000","last_undegraded":"2024-09-10T21:47:42.235355+0000","last_fullsized":"2024-09-10T21:47:42.235355+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:00.004793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235129+0000","last_change":"2024-09-10T21:47:41.236627+0000","last_active":"2024-09-10T21:47:42.235129+0000","last_peered":"2024-09-10T21:47:42.235129+0000","last_clean":"2024-09-10T21:47:42.235129+0000","last_became_active":"2024-09-10T21:47:41.236212+0000","last_became_peered":"2024-09-10T21:47:41.236212+0000","last_unstale":"2024-09-10T21:47:42.235129+0000","last_undegraded":"2024-09-10T21:47:42.235129+0000","last_fullsized":"2024-09-10T21:47:42.235129+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:24.225240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229511+0000","last_change":"2024-09-10T21:47:40.241481+0000","last_active":"2024-09-10T21:47:42.229511+0000","last_peered":"2024-09-10T21:47:42.229511+0000","last_clean":"2024-09-10T21:47:42.229511+0000","last_became_active":"2024-09-10T21:47:40.241205+0000","last_became_peered":"2024-09-10T21:47:40.241205+0000","last_unstale":"2024-09-10T21:47:42.229511+0000","last_undegraded":"2024-09-10T21:47:42.229511+0000","last_fullsized":"2024-09-10T21:47:42.229511+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:10:39.700005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229450+0000","last_change":"2024-09-10T21:47:41.250933+0000","last_active":"2024-09-10T21:47:42.229450+0000","last_peered":"2024-09-10T21:47:42.229450+0000","last_clean":"2024-09-10T21:47:42.229450+0000","last_became_active":"2024-09-10T21:47:41.250330+0000","last_became_peered":"2024-09-10T21:47:41.250330+0000","last_unstale":"2024-09-10T21:47:42.229450+0000","last_undegraded":"2024-09-10T21:47:42.229450+0000","last_fullsized":"2024-09-10T21:47:42.229450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:29.163811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.228836+0000","last_change":"2024-09-10T21:47:40.235952+0000","last_active":"2024-09-10T21:47:42.228836+0000","last_peered":"2024-09-10T21:47:42.228836+0000","last_clean":"2024-09-10T21:47:42.228836+0000","last_became_active":"2024-09-10T21:47:40.235614+0000","last_became_peered":"2024-09-10T21:47:40.235614+0000","last_unstale":"2024-09-10T21:47:42.228836+0000","last_undegraded":"2024-09-10T21:47:42.228836+0000","last_fullsized":"2024-09-10T21:47:42.228836+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:26.500061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.249607+0000","last_change":"2024-09-10T21:47:41.243593+0000","last_active":"2024-09-10T21:47:41.249607+0000","last_peered":"2024-09-10T21:47:41.249607+0000","last_clean":"2024-09-10T21:47:41.249607+0000","last_became_active":"2024-09-10T21:47:41.243348+0000","last_became_peered":"2024-09-10T21:47:41.243348+0000","last_unstale":"2024-09-10T21:47:41.249607+0000","last_undegraded":"2024-09-10T21:47:41.249607+0000","last_fullsized":"2024-09-10T21:47:41.249607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:38.319192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234283+0000","last_change":"2024-09-10T21:47:40.239186+0000","last_active":"2024-09-10T21:47:42.234283+0000","last_peered":"2024-09-10T21:47:42.234283+0000","last_clean":"2024-09-10T21:47:42.234283+0000","last_became_active":"2024-09-10T21:47:40.238895+0000","last_became_peered":"2024-09-10T21:47:40.238895+0000","last_unstale":"2024-09-10T21:47:42.234283+0000","last_undegraded":"2024-09-10T21:47:42.234283+0000","last_fullsized":"2024-09-10T21:47:42.234283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:24.421869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.243659+0000","last_change":"2024-09-10T21:47:41.240091+0000","last_active":"2024-09-10T21:47:41.243659+0000","last_peered":"2024-09-10T21:47:41.243659+0000","last_clean":"2024-09-10T21:47:41.243659+0000","last_became_active":"2024-09-10T21:47:41.239768+0000","last_became_peered":"2024-09-10T21:47:41.239768+0000","last_unstale":"2024-09-10T21:47:41.243659+0000","last_undegraded":"2024-09-10T21:47:41.243659+0000","last_fullsized":"2024-09-10T21:47:41.243659+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:13.425629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234462+0000","last_change":"2024-09-10T21:47:40.240093+0000","last_active":"2024-09-10T21:47:42.234462+0000","last_peered":"2024-09-10T21:47:42.234462+0000","last_clean":"2024-09-10T21:47:42.234462+0000","last_became_active":"2024-09-10T21:47:40.238125+0000","last_became_peered":"2024-09-10T21:47:40.238125+0000","last_unstale":"2024-09-10T21:47:42.234462+0000","last_undegraded":"2024-09-10T21:47:42.234462+0000","last_fullsized":"2024-09-10T21:47:42.234462+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:12.956184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.258714+0000","last_change":"2024-09-10T21:47:41.249707+0000","last_active":"2024-09-10T21:47:41.258714+0000","last_peered":"2024-09-10T21:47:41.258714+0000","last_clean":"2024-09-10T21:47:41.258714+0000","last_became_active":"2024-09-10T21:47:41.249384+0000","last_became_peered":"2024-09-10T21:47:41.249384+0000","last_unstale":"2024-09-10T21:47:41.258714+0000","last_undegraded":"2024-09-10T21:47:41.258714+0000","last_fullsized":"2024-09-10T21:47:41.258714+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:19:40.670356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235664+0000","last_change":"2024-09-10T21:47:40.245932+0000","last_active":"2024-09-10T21:47:42.235664+0000","last_peered":"2024-09-10T21:47:42.235664+0000","last_clean":"2024-09-10T21:47:42.235664+0000","last_became_active":"2024-09-10T21:47:40.245641+0000","last_became_peered":"2024-09-10T21:47:40.245641+0000","last_unstale":"2024-09-10T21:47:42.235664+0000","last_undegraded":"2024-09-10T21:47:42.235664+0000","last_fullsized":"2024-09-10T21:47:42.235664+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:09:10.560046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235355+0000","last_change":"2024-09-10T21:47:41.249417+0000","last_active":"2024-09-10T21:47:42.235355+0000","last_peered":"2024-09-10T21:47:42.235355+0000","last_clean":"2024-09-10T21:47:42.235355+0000","last_became_active":"2024-09-10T21:47:41.249161+0000","last_became_peered":"2024-09-10T21:47:41.249161+0000","last_unstale":"2024-09-10T21:47:42.235355+0000","last_undegraded":"2024-09-10T21:47:42.235355+0000","last_fullsized":"2024-09-10T21:47:42.235355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:51:32.957441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235348+0000","last_change":"2024-09-10T21:47:40.235977+0000","last_active":"2024-09-10T21:47:42.235348+0000","last_peered":"2024-09-10T21:47:42.235348+0000","last_clean":"2024-09-10T21:47:42.235348+0000","last_became_active":"2024-09-10T21:47:40.233997+0000","last_became_peered":"2024-09-10T21:47:40.233997+0000","last_unstale":"2024-09-10T21:47:42.235348+0000","last_undegraded":"2024-09-10T21:47:42.235348+0000","last_fullsized":"2024-09-10T21:47:42.235348+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:24:29.623676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235693+0000","last_change":"2024-09-10T21:47:41.235249+0000","last_active":"2024-09-10T21:47:42.235693+0000","last_peered":"2024-09-10T21:47:42.235693+0000","last_clean":"2024-09-10T21:47:42.235693+0000","last_became_active":"2024-09-10T21:47:41.234842+0000","last_became_peered":"2024-09-10T21:47:41.234842+0000","last_unstale":"2024-09-10T21:47:42.235693+0000","last_undegraded":"2024-09-10T21:47:42.235693+0000","last_fullsized":"2024-09-10T21:47:42.235693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:37.640217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"84'3","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234867+0000","last_change":"2024-09-10T21:47:40.242146+0000","last_active":"2024-09-10T21:47:42.234867+0000","last_peered":"2024-09-10T21:47:42.234867+0000","last_clean":"2024-09-10T21:47:42.234867+0000","last_became_active":"2024-09-10T21:47:40.241782+0000","last_became_peered":"2024-09-10T21:47:40.241782+0000","last_unstale":"2024-09-10T21:47:42.234867+0000","last_undegraded":"2024-09-10T21:47:42.234867+0000","last_fullsized":"2024-09-10T21:47:42.234867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:26:12.613930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229174+0000","last_change":"2024-09-10T21:47:41.242912+0000","last_active":"2024-09-10T21:47:42.229174+0000","last_peered":"2024-09-10T21:47:42.229174+0000","last_clean":"2024-09-10T21:47:42.229174+0000","last_became_active":"2024-09-10T21:47:41.241840+0000","last_became_peered":"2024-09-10T21:47:41.241840+0000","last_unstale":"2024-09-10T21:47:42.229174+0000","last_undegraded":"2024-09-10T21:47:42.229174+0000","last_fullsized":"2024-09-10T21:47:42.229174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:07:20.509204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"84'6","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234599+0000","last_change":"2024-09-10T21:47:40.235137+0000","last_active":"2024-09-10T21:47:42.234599+0000","last_peered":"2024-09-10T21:47:42.234599+0000","last_clean":"2024-09-10T21:47:42.234599+0000","last_became_active":"2024-09-10T21:47:40.234813+0000","last_became_peered":"2024-09-10T21:47:40.234813+0000","last_unstale":"2024-09-10T21:47:42.234599+0000","last_undegraded":"2024-09-10T21:47:42.234599+0000","last_fullsized":"2024-09-10T21:47:42.234599+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:59:13.262730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234559+0000","last_change":"2024-09-10T21:47:41.247024+0000","last_active":"2024-09-10T21:47:42.234559+0000","last_peered":"2024-09-10T21:47:42.234559+0000","last_clean":"2024-09-10T21:47:42.234559+0000","last_became_active":"2024-09-10T21:47:41.244824+0000","last_became_peered":"2024-09-10T21:47:41.244824+0000","last_unstale":"2024-09-10T21:47:42.234559+0000","last_undegraded":"2024-09-10T21:47:42.234559+0000","last_fullsized":"2024-09-10T21:47:42.234559+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:58.764672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234987+0000","last_change":"2024-09-10T21:47:40.238459+0000","last_active":"2024-09-10T21:47:42.234987+0000","last_peered":"2024-09-10T21:47:42.234987+0000","last_clean":"2024-09-10T21:47:42.234987+0000","last_became_active":"2024-09-10T21:47:40.237990+0000","last_became_peered":"2024-09-10T21:47:40.237990+0000","last_unstale":"2024-09-10T21:47:42.234987+0000","last_undegraded":"2024-09-10T21:47:42.234987+0000","last_fullsized":"2024-09-10T21:47:42.234987+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:30.359167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.249459+0000","last_change":"2024-09-10T21:47:41.243816+0000","last_active":"2024-09-10T21:47:41.249459+0000","last_peered":"2024-09-10T21:47:41.249459+0000","last_clean":"2024-09-10T21:47:41.249459+0000","last_became_active":"2024-09-10T21:47:41.242574+0000","last_became_peered":"2024-09-10T21:47:41.242574+0000","last_unstale":"2024-09-10T21:47:41.249459+0000","last_undegraded":"2024-09-10T21:47:41.249459+0000","last_fullsized":"2024-09-10T21:47:41.249459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:06:35.540097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.222903+0000","last_change":"2024-09-10T21:47:40.246734+0000","last_active":"2024-09-10T21:47:41.222903+0000","last_peered":"2024-09-10T21:47:41.222903+0000","last_clean":"2024-09-10T21:47:41.222903+0000","last_became_active":"2024-09-10T21:47:40.246495+0000","last_became_peered":"2024-09-10T21:47:40.246495+0000","last_unstale":"2024-09-10T21:47:41.222903+0000","last_undegraded":"2024-09-10T21:47:41.222903+0000","last_fullsized":"2024-09-10T21:47:41.222903+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:48:03.725986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234366+0000","last_change":"2024-09-10T21:47:41.238652+0000","last_active":"2024-09-10T21:47:42.234366+0000","last_peered":"2024-09-10T21:47:42.234366+0000","last_clean":"2024-09-10T21:47:42.234366+0000","last_became_active":"2024-09-10T21:47:41.238347+0000","last_became_peered":"2024-09-10T21:47:41.238347+0000","last_unstale":"2024-09-10T21:47:42.234366+0000","last_undegraded":"2024-09-10T21:47:42.234366+0000","last_fullsized":"2024-09-10T21:47:42.234366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:14:14.087344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234335+0000","last_change":"2024-09-10T21:47:40.234139+0000","last_active":"2024-09-10T21:47:42.234335+0000","last_peered":"2024-09-10T21:47:42.234335+0000","last_clean":"2024-09-10T21:47:42.234335+0000","last_became_active":"2024-09-10T21:47:40.233823+0000","last_became_peered":"2024-09-10T21:47:40.233823+0000","last_unstale":"2024-09-10T21:47:42.234335+0000","last_undegraded":"2024-09-10T21:47:42.234335+0000","last_fullsized":"2024-09-10T21:47:42.234335+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:03:45.199598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234703+0000","last_change":"2024-09-10T21:47:41.244209+0000","last_active":"2024-09-10T21:47:42.234703+0000","last_peered":"2024-09-10T21:47:42.234703+0000","last_clean":"2024-09-10T21:47:42.234703+0000","last_became_active":"2024-09-10T21:47:41.243402+0000","last_became_peered":"2024-09-10T21:47:41.243402+0000","last_unstale":"2024-09-10T21:47:42.234703+0000","last_undegraded":"2024-09-10T21:47:42.234703+0000","last_fullsized":"2024-09-10T21:47:42.234703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:29.272552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"84'8","reported_seq":23,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234180+0000","last_change":"2024-09-10T21:47:40.235945+0000","last_active":"2024-09-10T21:47:42.234180+0000","last_peered":"2024-09-10T21:47:42.234180+0000","last_clean":"2024-09-10T21:47:42.234180+0000","last_became_active":"2024-09-10T21:47:40.235672+0000","last_became_peered":"2024-09-10T21:47:40.235672+0000","last_unstale":"2024-09-10T21:47:42.234180+0000","last_undegraded":"2024-09-10T21:47:42.234180+0000","last_fullsized":"2024-09-10T21:47:42.234180+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:11.334128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.247535+0000","last_change":"2024-09-10T21:47:41.240308+0000","last_active":"2024-09-10T21:47:41.247535+0000","last_peered":"2024-09-10T21:47:41.247535+0000","last_clean":"2024-09-10T21:47:41.247535+0000","last_became_active":"2024-09-10T21:47:41.240058+0000","last_became_peered":"2024-09-10T21:47:41.240058+0000","last_unstale":"2024-09-10T21:47:41.247535+0000","last_undegraded":"2024-09-10T21:47:41.247535+0000","last_fullsized":"2024-09-10T21:47:41.247535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:42.696781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235794+0000","last_change":"2024-09-10T21:47:40.233571+0000","last_active":"2024-09-10T21:47:42.235794+0000","last_peered":"2024-09-10T21:47:42.235794+0000","last_clean":"2024-09-10T21:47:42.235794+0000","last_became_active":"2024-09-10T21:47:40.233087+0000","last_became_peered":"2024-09-10T21:47:40.233087+0000","last_unstale":"2024-09-10T21:47:42.235794+0000","last_undegraded":"2024-09-10T21:47:42.235794+0000","last_fullsized":"2024-09-10T21:47:42.235794+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:12.949983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235547+0000","last_change":"2024-09-10T21:47:41.241298+0000","last_active":"2024-09-10T21:47:42.235547+0000","last_peered":"2024-09-10T21:47:42.235547+0000","last_clean":"2024-09-10T21:47:42.235547+0000","last_became_active":"2024-09-10T21:47:41.240689+0000","last_became_peered":"2024-09-10T21:47:41.240689+0000","last_unstale":"2024-09-10T21:47:42.235547+0000","last_undegraded":"2024-09-10T21:47:42.235547+0000","last_fullsized":"2024-09-10T21:47:42.235547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:46.703097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235360+0000","last_change":"2024-09-10T21:47:40.241879+0000","last_active":"2024-09-10T21:47:42.235360+0000","last_peered":"2024-09-10T21:47:42.235360+0000","last_clean":"2024-09-10T21:47:42.235360+0000","last_became_active":"2024-09-10T21:47:40.241542+0000","last_became_peered":"2024-09-10T21:47:40.241542+0000","last_unstale":"2024-09-10T21:47:42.235360+0000","last_undegraded":"2024-09-10T21:47:42.235360+0000","last_fullsized":"2024-09-10T21:47:42.235360+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:28.152606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229256+0000","last_change":"2024-09-10T21:47:41.239089+0000","last_active":"2024-09-10T21:47:42.229256+0000","last_peered":"2024-09-10T21:47:42.229256+0000","last_clean":"2024-09-10T21:47:42.229256+0000","last_became_active":"2024-09-10T21:47:41.238786+0000","last_became_peered":"2024-09-10T21:47:41.238786+0000","last_unstale":"2024-09-10T21:47:42.229256+0000","last_undegraded":"2024-09-10T21:47:42.229256+0000","last_fullsized":"2024-09-10T21:47:42.229256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:46.063011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235454+0000","last_change":"2024-09-10T21:47:40.240002+0000","last_active":"2024-09-10T21:47:42.235454+0000","last_peered":"2024-09-10T21:47:42.235454+0000","last_clean":"2024-09-10T21:47:42.235454+0000","last_became_active":"2024-09-10T21:47:40.239707+0000","last_became_peered":"2024-09-10T21:47:40.239707+0000","last_unstale":"2024-09-10T21:47:42.235454+0000","last_undegraded":"2024-09-10T21:47:42.235454+0000","last_fullsized":"2024-09-10T21:47:42.235454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:46:10.624235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.222845+0000","last_change":"2024-09-10T21:47:41.254933+0000","last_active":"2024-09-10T21:47:42.222845+0000","last_peered":"2024-09-10T21:47:42.222845+0000","last_clean":"2024-09-10T21:47:42.222845+0000","last_became_active":"2024-09-10T21:47:41.254661+0000","last_became_peered":"2024-09-10T21:47:41.254661+0000","last_unstale":"2024-09-10T21:47:42.222845+0000","last_undegraded":"2024-09-10T21:47:42.222845+0000","last_fullsized":"2024-09-10T21:47:42.222845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:12:26.316387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"84'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.267032+0000","last_change":"2024-09-10T21:47:40.243074+0000","last_active":"2024-09-10T21:47:41.267032+0000","last_peered":"2024-09-10T21:47:41.267032+0000","last_clean":"2024-09-10T21:47:41.267032+0000","last_became_active":"2024-09-10T21:47:40.242818+0000","last_became_peered":"2024-09-10T21:47:40.242818+0000","last_unstale":"2024-09-10T21:47:41.267032+0000","last_undegraded":"2024-09-10T21:47:41.267032+0000","last_fullsized":"2024-09-10T21:47:41.267032+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:01.117060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234612+0000","last_change":"2024-09-10T21:47:41.245761+0000","last_active":"2024-09-10T21:47:42.234612+0000","last_peered":"2024-09-10T21:47:42.234612+0000","last_clean":"2024-09-10T21:47:42.234612+0000","last_became_active":"2024-09-10T21:47:41.245464+0000","last_became_peered":"2024-09-10T21:47:41.245464+0000","last_unstale":"2024-09-10T21:47:42.234612+0000","last_undegraded":"2024-09-10T21:47:42.234612+0000","last_fullsized":"2024-09-10T21:47:42.234612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:25.433073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.222966+0000","last_change":"2024-09-10T21:47:40.242433+0000","last_active":"2024-09-10T21:47:42.222966+0000","last_peered":"2024-09-10T21:47:42.222966+0000","last_clean":"2024-09-10T21:47:42.222966+0000","last_became_active":"2024-09-10T21:47:40.242190+0000","last_became_peered":"2024-09-10T21:47:40.242190+0000","last_unstale":"2024-09-10T21:47:42.222966+0000","last_undegraded":"2024-09-10T21:47:42.222966+0000","last_fullsized":"2024-09-10T21:47:42.222966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:08:33.355716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.222682+0000","last_change":"2024-09-10T21:47:41.243121+0000","last_active":"2024-09-10T21:47:42.222682+0000","last_peered":"2024-09-10T21:47:42.222682+0000","last_clean":"2024-09-10T21:47:42.222682+0000","last_became_active":"2024-09-10T21:47:41.242794+0000","last_became_peered":"2024-09-10T21:47:41.242794+0000","last_unstale":"2024-09-10T21:47:42.222682+0000","last_undegraded":"2024-09-10T21:47:42.222682+0000","last_fullsized":"2024-09-10T21:47:42.222682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:54.603994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.222578+0000","last_change":"2024-09-10T21:47:40.241083+0000","last_active":"2024-09-10T21:47:42.222578+0000","last_peered":"2024-09-10T21:47:42.222578+0000","last_clean":"2024-09-10T21:47:42.222578+0000","last_became_active":"2024-09-10T21:47:40.240753+0000","last_became_peered":"2024-09-10T21:47:40.240753+0000","last_unstale":"2024-09-10T21:47:42.222578+0000","last_undegraded":"2024-09-10T21:47:42.222578+0000","last_fullsized":"2024-09-10T21:47:42.222578+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:42:08.697455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234226+0000","last_change":"2024-09-10T21:47:41.242449+0000","last_active":"2024-09-10T21:47:42.234226+0000","last_peered":"2024-09-10T21:47:42.234226+0000","last_clean":"2024-09-10T21:47:42.234226+0000","last_became_active":"2024-09-10T21:47:41.242174+0000","last_became_peered":"2024-09-10T21:47:41.242174+0000","last_unstale":"2024-09-10T21:47:42.234226+0000","last_undegraded":"2024-09-10T21:47:42.234226+0000","last_fullsized":"2024-09-10T21:47:42.234226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:03.481250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.222006+0000","last_change":"2024-09-10T21:47:40.243627+0000","last_active":"2024-09-10T21:47:41.222006+0000","last_peered":"2024-09-10T21:47:41.222006+0000","last_clean":"2024-09-10T21:47:41.222006+0000","last_became_active":"2024-09-10T21:47:40.241668+0000","last_became_peered":"2024-09-10T21:47:40.241668+0000","last_unstale":"2024-09-10T21:47:41.222006+0000","last_undegraded":"2024-09-10T21:47:41.222006+0000","last_fullsized":"2024-09-10T21:47:41.222006+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:07.637571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229357+0000","last_change":"2024-09-10T21:47:41.241102+0000","last_active":"2024-09-10T21:47:42.229357+0000","last_peered":"2024-09-10T21:47:42.229357+0000","last_clean":"2024-09-10T21:47:42.229357+0000","last_became_active":"2024-09-10T21:47:41.240802+0000","last_became_peered":"2024-09-10T21:47:41.240802+0000","last_unstale":"2024-09-10T21:47:42.229357+0000","last_undegraded":"2024-09-10T21:47:42.229357+0000","last_fullsized":"2024-09-10T21:47:42.229357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:12.345333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"84'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235590+0000","last_change":"2024-09-10T21:47:40.241148+0000","last_active":"2024-09-10T21:47:42.235590+0000","last_peered":"2024-09-10T21:47:42.235590+0000","last_clean":"2024-09-10T21:47:42.235590+0000","last_became_active":"2024-09-10T21:47:40.240394+0000","last_became_peered":"2024-09-10T21:47:40.240394+0000","last_unstale":"2024-09-10T21:47:42.235590+0000","last_undegraded":"2024-09-10T21:47:42.235590+0000","last_fullsized":"2024-09-10T21:47:42.235590+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:42.958092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.249612+0000","last_change":"2024-09-10T21:47:41.244199+0000","last_active":"2024-09-10T21:47:41.249612+0000","last_peered":"2024-09-10T21:47:41.249612+0000","last_clean":"2024-09-10T21:47:41.249612+0000","last_became_active":"2024-09-10T21:47:41.242308+0000","last_became_peered":"2024-09-10T21:47:41.242308+0000","last_unstale":"2024-09-10T21:47:41.249612+0000","last_undegraded":"2024-09-10T21:47:41.249612+0000","last_fullsized":"2024-09-10T21:47:41.249612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:27.320599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235668+0000","last_change":"2024-09-10T21:47:40.240006+0000","last_active":"2024-09-10T21:47:42.235668+0000","last_peered":"2024-09-10T21:47:42.235668+0000","last_clean":"2024-09-10T21:47:42.235668+0000","last_became_active":"2024-09-10T21:47:40.238454+0000","last_became_peered":"2024-09-10T21:47:40.238454+0000","last_unstale":"2024-09-10T21:47:42.235668+0000","last_undegraded":"2024-09-10T21:47:42.235668+0000","last_fullsized":"2024-09-10T21:47:42.235668+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:31.052079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234946+0000","last_change":"2024-09-10T21:47:41.249745+0000","last_active":"2024-09-10T21:47:42.234946+0000","last_peered":"2024-09-10T21:47:42.234946+0000","last_clean":"2024-09-10T21:47:42.234946+0000","last_became_active":"2024-09-10T21:47:41.248663+0000","last_became_peered":"2024-09-10T21:47:41.248663+0000","last_unstale":"2024-09-10T21:47:42.234946+0000","last_undegraded":"2024-09-10T21:47:42.234946+0000","last_fullsized":"2024-09-10T21:47:42.234946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:12:36.727787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234969+0000","last_change":"2024-09-10T21:47:40.237681+0000","last_active":"2024-09-10T21:47:42.234969+0000","last_peered":"2024-09-10T21:47:42.234969+0000","last_clean":"2024-09-10T21:47:42.234969+0000","last_became_active":"2024-09-10T21:47:40.237382+0000","last_became_peered":"2024-09-10T21:47:40.237382+0000","last_unstale":"2024-09-10T21:47:42.234969+0000","last_undegraded":"2024-09-10T21:47:42.234969+0000","last_fullsized":"2024-09-10T21:47:42.234969+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:48.520227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.229478+0000","last_change":"2024-09-10T21:47:41.243208+0000","last_active":"2024-09-10T21:47:42.229478+0000","last_peered":"2024-09-10T21:47:42.229478+0000","last_clean":"2024-09-10T21:47:42.229478+0000","last_became_active":"2024-09-10T21:47:41.241208+0000","last_became_peered":"2024-09-10T21:47:41.241208+0000","last_unstale":"2024-09-10T21:47:42.229478+0000","last_undegraded":"2024-09-10T21:47:42.229478+0000","last_fullsized":"2024-09-10T21:47:42.229478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:11:01.015998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234980+0000","last_change":"2024-09-10T21:47:40.241599+0000","last_active":"2024-09-10T21:47:42.234980+0000","last_peered":"2024-09-10T21:47:42.234980+0000","last_clean":"2024-09-10T21:47:42.234980+0000","last_became_active":"2024-09-10T21:47:40.241261+0000","last_became_peered":"2024-09-10T21:47:40.241261+0000","last_unstale":"2024-09-10T21:47:42.234980+0000","last_undegraded":"2024-09-10T21:47:42.234980+0000","last_fullsized":"2024-09-10T21:47:42.234980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:54.761697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234297+0000","last_change":"2024-09-10T21:47:41.240907+0000","last_active":"2024-09-10T21:47:42.234297+0000","last_peered":"2024-09-10T21:47:42.234297+0000","last_clean":"2024-09-10T21:47:42.234297+0000","last_became_active":"2024-09-10T21:47:41.240623+0000","last_became_peered":"2024-09-10T21:47:41.240623+0000","last_unstale":"2024-09-10T21:47:42.234297+0000","last_undegraded":"2024-09-10T21:47:42.234297+0000","last_fullsized":"2024-09-10T21:47:42.234297+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:40.161098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235132+0000","last_change":"2024-09-10T21:47:40.239305+0000","last_active":"2024-09-10T21:47:42.235132+0000","last_peered":"2024-09-10T21:47:42.235132+0000","last_clean":"2024-09-10T21:47:42.235132+0000","last_became_active":"2024-09-10T21:47:40.239033+0000","last_became_peered":"2024-09-10T21:47:40.239033+0000","last_unstale":"2024-09-10T21:47:42.235132+0000","last_undegraded":"2024-09-10T21:47:42.235132+0000","last_fullsized":"2024-09-10T21:47:42.235132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:03:39.149894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235298+0000","last_change":"2024-09-10T21:47:41.250246+0000","last_active":"2024-09-10T21:47:42.235298+0000","last_peered":"2024-09-10T21:47:42.235298+0000","last_clean":"2024-09-10T21:47:42.235298+0000","last_became_active":"2024-09-10T21:47:41.245991+0000","last_became_peered":"2024-09-10T21:47:41.245991+0000","last_unstale":"2024-09-10T21:47:42.235298+0000","last_undegraded":"2024-09-10T21:47:42.235298+0000","last_fullsized":"2024-09-10T21:47:42.235298+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:54.204331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.228917+0000","last_change":"2024-09-10T21:47:40.230275+0000","last_active":"2024-09-10T21:47:42.228917+0000","last_peered":"2024-09-10T21:47:42.228917+0000","last_clean":"2024-09-10T21:47:42.228917+0000","last_became_active":"2024-09-10T21:47:40.230015+0000","last_became_peered":"2024-09-10T21:47:40.230015+0000","last_unstale":"2024-09-10T21:47:42.228917+0000","last_undegraded":"2024-09-10T21:47:42.228917+0000","last_fullsized":"2024-09-10T21:47:42.228917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:21.852224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.258051+0000","last_change":"2024-09-10T21:47:41.254403+0000","last_active":"2024-09-10T21:47:41.258051+0000","last_peered":"2024-09-10T21:47:41.258051+0000","last_clean":"2024-09-10T21:47:41.258051+0000","last_became_active":"2024-09-10T21:47:41.254093+0000","last_became_peered":"2024-09-10T21:47:41.254093+0000","last_unstale":"2024-09-10T21:47:41.258051+0000","last_undegraded":"2024-09-10T21:47:41.258051+0000","last_fullsized":"2024-09-10T21:47:41.258051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:33:39.704570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234209+0000","last_change":"2024-09-10T21:47:40.248048+0000","last_active":"2024-09-10T21:47:42.234209+0000","last_peered":"2024-09-10T21:47:42.234209+0000","last_clean":"2024-09-10T21:47:42.234209+0000","last_became_active":"2024-09-10T21:47:40.247833+0000","last_became_peered":"2024-09-10T21:47:40.247833+0000","last_unstale":"2024-09-10T21:47:42.234209+0000","last_undegraded":"2024-09-10T21:47:42.234209+0000","last_fullsized":"2024-09-10T21:47:42.234209+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:11:54.026456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.234296+0000","last_change":"2024-09-10T21:47:41.248910+0000","last_active":"2024-09-10T21:47:42.234296+0000","last_peered":"2024-09-10T21:47:42.234296+0000","last_clean":"2024-09-10T21:47:42.234296+0000","last_became_active":"2024-09-10T21:47:41.248542+0000","last_became_peered":"2024-09-10T21:47:41.248542+0000","last_unstale":"2024-09-10T21:47:42.234296+0000","last_undegraded":"2024-09-10T21:47:42.234296+0000","last_fullsized":"2024-09-10T21:47:42.234296+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:12.345333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.228607+0000","last_change":"2024-09-10T21:47:40.237251+0000","last_active":"2024-09-10T21:47:42.228607+0000","last_peered":"2024-09-10T21:47:42.228607+0000","last_clean":"2024-09-10T21:47:42.228607+0000","last_became_active":"2024-09-10T21:47:40.236969+0000","last_became_peered":"2024-09-10T21:47:40.236969+0000","last_unstale":"2024-09-10T21:47:42.228607+0000","last_undegraded":"2024-09-10T21:47:42.228607+0000","last_fullsized":"2024-09-10T21:47:42.228607+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:09:10.560046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.235018+0000","last_change":"2024-09-10T21:47:41.244219+0000","last_active":"2024-09-10T21:47:42.235018+0000","last_peered":"2024-09-10T21:47:42.235018+0000","last_clean":"2024-09-10T21:47:42.235018+0000","last_became_active":"2024-09-10T21:47:41.243918+0000","last_became_peered":"2024-09-10T21:47:41.243918+0000","last_unstale":"2024-09-10T21:47:42.235018+0000","last_undegraded":"2024-09-10T21:47:42.235018+0000","last_fullsized":"2024-09-10T21:47:42.235018+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:40.210159+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.210159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:40:31.044003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.264630+0000","last_change":"2024-09-10T21:47:40.242532+0000","last_active":"2024-09-10T21:47:41.264630+0000","last_peered":"2024-09-10T21:47:41.264630+0000","last_clean":"2024-09-10T21:47:41.264630+0000","last_became_active":"2024-09-10T21:47:40.241942+0000","last_became_peered":"2024-09-10T21:47:40.241942+0000","last_unstale":"2024-09-10T21:47:41.264630+0000","last_undegraded":"2024-09-10T21:47:41.264630+0000","last_fullsized":"2024-09-10T21:47:41.264630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:47:39.198955+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.198955+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:27.266025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":225,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.231048+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:47:41.231048+0000","last_peered":"2024-09-10T21:47:41.231048+0000","last_clean":"2024-09-10T21:47:41.231048+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:47:41.231048+0000","last_undegraded":"2024-09-10T21:47:41.231048+0000","last_fullsized":"2024-09-10T21:47:41.231048+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509513,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1048576,"data_stored":441358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509513,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1052672,"data_stored":442298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509512,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32204,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339636,"statfs":{"total":96636764160,"available":96603787264,"internally_reserved":0,"allocated":1036288,"data_stored":440664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509512,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32200,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339640,"statfs":{"total":96636764160,"available":96603791360,"internally_reserved":0,"allocated":1028096,"data_stored":434946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509514,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1048576,"data_stored":440858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509512,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32780,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339060,"statfs":{"total":96636764160,"available":96603197440,"internally_reserved":0,"allocated":1622016,"data_stored":1024766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542215,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32776,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339064,"statfs":{"total":96636764160,"available":96603201536,"internally_reserved":0,"allocated":1622016,"data_stored":1024802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542216,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36340,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335500,"statfs":{"total":96636764160,"available":96599552000,"internally_reserved":0,"allocated":1011712,"data_stored":427589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:47:45.961 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:47:45.961 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:47:46.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.130+0000 7fbf46800640 1 -- 172.21.15.7:0/2054281734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf40073e20 msgr2=0x7fbf40074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:46.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.130+0000 7fbf46800640 1 --2- 172.21.15.7:0/2054281734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf40073e20 0x7fbf40074280 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbf38005fd0 tx=0x7fbf3802aa10 comp rx=0 tx=0).stop 2024-09-10T21:47:46.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.130+0000 7fbf46800640 1 -- 172.21.15.7:0/2054281734 shutdown_connections 2024-09-10T21:47:46.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.130+0000 7fbf46800640 1 --2- 172.21.15.7:0/2054281734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf40073e20 0x7fbf40074280 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.130+0000 7fbf46800640 1 --2- 172.21.15.7:0/2054281734 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf40159680 0x7fbf4015cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.130+0000 7fbf46800640 1 -- 172.21.15.7:0/2054281734 >> 172.21.15.7:0/2054281734 conn(0x7fbf400b16d0 msgr2=0x7fbf400b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.130+0000 7fbf46800640 1 -- 172.21.15.7:0/2054281734 shutdown_connections 2024-09-10T21:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.130+0000 7fbf46800640 1 -- 172.21.15.7:0/2054281734 wait complete. 2024-09-10T21:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.131+0000 7fbf46800640 1 Processor -- start 2024-09-10T21:47:46.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.132+0000 7fbf46800640 1 -- start start 2024-09-10T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.132+0000 7fbf46800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf4006e580 0x7fbf40152400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.132+0000 7fbf46800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf40073e20 0x7fbf400b5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.132+0000 7fbf46800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf40159680 0x7fbf4014ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.132+0000 7fbf46800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbf400ca990 con 0x7fbf40073e20 2024-09-10T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.132+0000 7fbf46800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbf400ca810 con 0x7fbf4006e580 2024-09-10T21:47:46.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.132+0000 7fbf46800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fbf400cab10 con 0x7fbf40159680 2024-09-10T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.133+0000 7fbf36a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf40073e20 0x7fbf400b5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.133+0000 7fbf36a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf40073e20 0x7fbf400b5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57746/0 (socket says 172.21.15.7:57746) 2024-09-10T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.133+0000 7fbf36a00640 1 -- 172.21.15.7:0/3931439847 learned_addr learned my addr 172.21.15.7:0/3931439847 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.133+0000 7fbf3f400640 1 --2- 172.21.15.7:0/3931439847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf4006e580 0x7fbf40152400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.133+0000 7fbf3fe00640 1 --2- 172.21.15.7:0/3931439847 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf40159680 0x7fbf4014ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.133+0000 7fbf36a00640 1 -- 172.21.15.7:0/3931439847 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf40159680 msgr2=0x7fbf4014ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.133+0000 7fbf36a00640 1 --2- 172.21.15.7:0/3931439847 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf40159680 0x7fbf4014ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.133+0000 7fbf36a00640 1 -- 172.21.15.7:0/3931439847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf4006e580 msgr2=0x7fbf40152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.133+0000 7fbf36a00640 1 --2- 172.21.15.7:0/3931439847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf4006e580 0x7fbf40152400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.133+0000 7fbf36a00640 1 -- 172.21.15.7:0/3931439847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf40152a10 con 0x7fbf40073e20 2024-09-10T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.133+0000 7fbf3f400640 1 --2- 172.21.15.7:0/3931439847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf4006e580 0x7fbf40152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:46.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.134+0000 7fbf36a00640 1 --2- 172.21.15.7:0/3931439847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf40073e20 0x7fbf400b5450 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800c450 tx=0x7fbf2800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:46.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.134+0000 7fbf37e00640 1 -- 172.21.15.7:0/3931439847 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf280052d0 con 0x7fbf40073e20 2024-09-10T21:47:46.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.134+0000 7fbf37e00640 1 -- 172.21.15.7:0/3931439847 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf28005470 con 0x7fbf40073e20 2024-09-10T21:47:46.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.134+0000 7fbf37e00640 1 -- 172.21.15.7:0/3931439847 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf28012870 con 0x7fbf40073e20 2024-09-10T21:47:46.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.134+0000 7fbf46800640 1 -- 172.21.15.7:0/3931439847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf40152cf0 con 0x7fbf40073e20 2024-09-10T21:47:46.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.134+0000 7fbf46800640 1 -- 172.21.15.7:0/3931439847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf400b2e50 con 0x7fbf40073e20 2024-09-10T21:47:46.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.135+0000 7fbf46800640 1 -- 172.21.15.7:0/3931439847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf400cafc0 con 0x7fbf40073e20 2024-09-10T21:47:46.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.136+0000 7fbf37e00640 1 -- 172.21.15.7:0/3931439847 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fbf280035f0 con 0x7fbf40073e20 2024-09-10T21:47:46.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.137+0000 7fbf37e00640 1 --2- 172.21.15.7:0/3931439847 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf1c077f70 0x7fbf1c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:46.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.137+0000 7fbf37e00640 1 -- 172.21.15.7:0/3931439847 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fbf28099950 con 0x7fbf40073e20 2024-09-10T21:47:46.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.138+0000 7fbf3f400640 1 --2- 172.21.15.7:0/3931439847 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf1c077f70 0x7fbf1c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:46.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.138+0000 7fbf3f400640 1 --2- 172.21.15.7:0/3931439847 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf1c077f70 0x7fbf1c07a430 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009b80 tx=0x7fbf30005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:46.142 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.141+0000 7fbf37e00640 1 -- 172.21.15.7:0/3931439847 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fbf40072830 con 0x7fbf40073e20 2024-09-10T21:47:46.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.413+0000 7fbf46800640 1 -- 172.21.15.7:0/3931439847 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fbf400b3020 con 0x7fbf40073e20 2024-09-10T21:47:46.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.416+0000 7fbf37e00640 1 -- 172.21.15.7:0/3931439847 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v36) ==== 190+0+147 (secure 0 0 0) 0x7fbf280623b0 con 0x7fbf40073e20 2024-09-10T21:47:46.417 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-09-10T21:47:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDvvOBm0aqcFhAAtid2bnzjcpaYAbU7g0JcOA== 2024-09-10T21:47:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: caps mds = "allow" 2024-09-10T21:47:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: caps mon = "allow r" 2024-09-10T21:47:46.417 INFO:teuthology.orchestra.run.smithi007.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:47:46.417 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-09-10T21:47:46.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.418+0000 7fbf46800640 1 -- 172.21.15.7:0/3931439847 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf1c077f70 msgr2=0x7fbf1c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:46.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.418+0000 7fbf46800640 1 --2- 172.21.15.7:0/3931439847 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf1c077f70 0x7fbf1c07a430 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009b80 tx=0x7fbf30005f90 comp rx=0 tx=0).stop 2024-09-10T21:47:46.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.418+0000 7fbf46800640 1 -- 172.21.15.7:0/3931439847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf40073e20 msgr2=0x7fbf400b5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:46.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.418+0000 7fbf46800640 1 --2- 172.21.15.7:0/3931439847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf40073e20 0x7fbf400b5450 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800c450 tx=0x7fbf2800c910 comp rx=0 tx=0).stop 2024-09-10T21:47:46.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.418+0000 7fbf46800640 1 -- 172.21.15.7:0/3931439847 shutdown_connections 2024-09-10T21:47:46.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.418+0000 7fbf46800640 1 --2- 172.21.15.7:0/3931439847 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbf1c077f70 0x7fbf1c07a430 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.418+0000 7fbf46800640 1 --2- 172.21.15.7:0/3931439847 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf40159680 0x7fbf4014ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.418+0000 7fbf46800640 1 --2- 172.21.15.7:0/3931439847 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbf40073e20 0x7fbf400b5450 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.418+0000 7fbf46800640 1 --2- 172.21.15.7:0/3931439847 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf4006e580 0x7fbf40152400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.418+0000 7fbf46800640 1 -- 172.21.15.7:0/3931439847 >> 172.21.15.7:0/3931439847 conn(0x7fbf400b16d0 msgr2=0x7fbf4015e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.418+0000 7fbf46800640 1 -- 172.21.15.7:0/3931439847 shutdown_connections 2024-09-10T21:47:46.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.418+0000 7fbf46800640 1 -- 172.21.15.7:0/3931439847 wait complete. 2024-09-10T21:47:46.436 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:47:46.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.595+0000 7feaaf800640 1 -- 172.21.15.7:0/2718587697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90004040 msgr2=0x7fea90004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:46.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.595+0000 7feaaf800640 1 --2- 172.21.15.7:0/2718587697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90004040 0x7fea90004800 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fea9400c290 tx=0x7fea90005830 comp rx=0 tx=0).stop 2024-09-10T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.595+0000 7feaad800640 1 -- 172.21.15.7:0/2718587697 reap_dead start 2024-09-10T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.595+0000 7feaaf800640 1 -- 172.21.15.7:0/2718587697 shutdown_connections 2024-09-10T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.595+0000 7feaaf800640 1 -- 172.21.15.7:0/2718587697 >> 172.21.15.7:0/2718587697 conn(0x7feaa80b16d0 msgr2=0x7feaa80b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.595+0000 7feaaf800640 1 -- 172.21.15.7:0/2718587697 shutdown_connections 2024-09-10T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.595+0000 7feaaf800640 1 -- 172.21.15.7:0/2718587697 wait complete. 2024-09-10T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.597+0000 7feaaf800640 1 Processor -- start 2024-09-10T21:47:46.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.597+0000 7feaaf800640 1 -- start start 2024-09-10T21:47:46.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.598+0000 7feaaf800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90004040 0x7feaa8149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:46.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.598+0000 7feaaf800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feaa815ec80 0x7feaa81598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.598+0000 7feaaf800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feaa815f100 0x7feaa815c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.598+0000 7feaaf800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7feaa81601d0 con 0x7fea90004040 2024-09-10T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.598+0000 7feaaf800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7feaa8073ef0 con 0x7feaa815f100 2024-09-10T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.598+0000 7feaaf800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7feaa8160350 con 0x7feaa815ec80 2024-09-10T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.598+0000 7feaace00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90004040 0x7feaa8149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.598+0000 7feaace00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90004040 0x7feaa8149d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57762/0 (socket says 172.21.15.7:57762) 2024-09-10T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.598+0000 7feaace00640 1 -- 172.21.15.7:0/567001624 learned_addr learned my addr 172.21.15.7:0/567001624 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.598+0000 7feaa7e00640 1 --2- 172.21.15.7:0/567001624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feaa815ec80 0x7feaa81598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.598+0000 7feaad800640 1 --2- 172.21.15.7:0/567001624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feaa815f100 0x7feaa815c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.598+0000 7feaace00640 1 -- 172.21.15.7:0/567001624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feaa815ec80 msgr2=0x7feaa81598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.598+0000 7feaace00640 1 --2- 172.21.15.7:0/567001624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feaa815ec80 0x7feaa81598e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:46.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.598+0000 7feaace00640 1 -- 172.21.15.7:0/567001624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feaa815f100 msgr2=0x7feaa815c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:46.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.598+0000 7feaace00640 1 --2- 172.21.15.7:0/567001624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feaa815f100 0x7feaa815c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:46.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.598+0000 7feaace00640 1 -- 172.21.15.7:0/567001624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa814a2d0 con 0x7fea90004040 2024-09-10T21:47:46.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.599+0000 7feaace00640 1 --2- 172.21.15.7:0/567001624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90004040 0x7feaa8149d90 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fea9c011b10 tx=0x7fea9c009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:46.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.599+0000 7feaa4c00640 1 -- 172.21.15.7:0/567001624 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c00fc20 con 0x7fea90004040 2024-09-10T21:47:46.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.599+0000 7feaa4c00640 1 -- 172.21.15.7:0/567001624 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea9c00fdc0 con 0x7fea90004040 2024-09-10T21:47:46.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.599+0000 7feaa4c00640 1 -- 172.21.15.7:0/567001624 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c004280 con 0x7fea90004040 2024-09-10T21:47:46.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.599+0000 7feaaf800640 1 -- 172.21.15.7:0/567001624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa8158840 con 0x7fea90004040 2024-09-10T21:47:46.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.599+0000 7feaaf800640 1 -- 172.21.15.7:0/567001624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa8158e00 con 0x7fea90004040 2024-09-10T21:47:46.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.600+0000 7feaaf800640 1 -- 172.21.15.7:0/567001624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaa80ccef0 con 0x7fea90004040 2024-09-10T21:47:46.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.602+0000 7feaa4c00640 1 -- 172.21.15.7:0/567001624 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fea9c0035f0 con 0x7fea90004040 2024-09-10T21:47:46.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.602+0000 7feaa4c00640 1 --2- 172.21.15.7:0/567001624 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fea88077fa0 0x7fea8807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:46.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.603+0000 7feaa4c00640 1 -- 172.21.15.7:0/567001624 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fea9c09a780 con 0x7fea90004040 2024-09-10T21:47:46.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.603+0000 7feaa7e00640 1 --2- 172.21.15.7:0/567001624 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fea88077fa0 0x7fea8807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:46.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.604+0000 7feaa7e00640 1 --2- 172.21.15.7:0/567001624 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fea88077fa0 0x7fea8807a460 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fea94033000 tx=0x7feaa815f880 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:46.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.606+0000 7feaa4c00640 1 -- 172.21.15.7:0/567001624 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fea9c063270 con 0x7fea90004040 2024-09-10T21:47:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.835+0000 7feaaf800640 1 -- 172.21.15.7:0/567001624 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feaa8073770 con 0x7fea90004040 2024-09-10T21:47:46.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.835+0000 7feaa4c00640 1 -- 172.21.15.7:0/567001624 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+5233 (secure 0 0 0) 0x7fea9c062b70 con 0x7fea90004040 2024-09-10T21:47:46.837 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:46.837 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":63,"btime":"2024-09-10T21:47:44:249448+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/978763848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":978763848},{"type":"v1","addr":"172.21.15.7:6835","nonce":978763848}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":235775,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/760980985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":760980985},{"type":"v1","addr":"172.21.15.133:6836","nonce":760980985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":235778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3312604545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3312604545},{"type":"v1","addr":"172.21.15.133:6837","nonce":3312604545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:47:40.664091+0000","modified":"2024-09-10T21:47:43.681126+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235781},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235781":{"gid":235781,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6833/1499133391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1499133391},{"type":"v1","addr":"172.21.15.7:6833","nonce":1499133391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235781,"qdb_cluster":[235781]},"id":6}]} 2024-09-10T21:47:46.837 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 63 2024-09-10T21:47:46.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.838+0000 7feaaf800640 1 -- 172.21.15.7:0/567001624 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fea88077fa0 msgr2=0x7fea8807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:46.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.838+0000 7feaaf800640 1 --2- 172.21.15.7:0/567001624 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fea88077fa0 0x7fea8807a460 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fea94033000 tx=0x7feaa815f880 comp rx=0 tx=0).stop 2024-09-10T21:47:46.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.838+0000 7feaaf800640 1 -- 172.21.15.7:0/567001624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90004040 msgr2=0x7feaa8149d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:46.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.838+0000 7feaaf800640 1 --2- 172.21.15.7:0/567001624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90004040 0x7feaa8149d90 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fea9c011b10 tx=0x7fea9c009f10 comp rx=0 tx=0).stop 2024-09-10T21:47:46.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.838+0000 7feaaf800640 1 -- 172.21.15.7:0/567001624 shutdown_connections 2024-09-10T21:47:46.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.838+0000 7feaaf800640 1 --2- 172.21.15.7:0/567001624 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fea88077fa0 0x7fea8807a460 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:46.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.838+0000 7feaaf800640 1 --2- 172.21.15.7:0/567001624 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7feaa815f100 0x7feaa815c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:46.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.838+0000 7feaaf800640 1 --2- 172.21.15.7:0/567001624 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7feaa815ec80 0x7feaa81598e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:46.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.838+0000 7feaaf800640 1 --2- 172.21.15.7:0/567001624 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fea90004040 0x7feaa8149d90 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:46.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.838+0000 7feaaf800640 1 -- 172.21.15.7:0/567001624 >> 172.21.15.7:0/567001624 conn(0x7feaa80b16d0 msgr2=0x7feaa80724f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:46.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.838+0000 7feaaf800640 1 -- 172.21.15.7:0/567001624 shutdown_connections 2024-09-10T21:47:46.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:46.838+0000 7feaaf800640 1 -- 172.21.15.7:0/567001624 wait complete. 2024-09-10T21:47:47.197 INFO:tasks.ceph.mds.a.smithi007.stderr:2024-09-10T21:47:47.196+0000 7f7b02400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:47:47.257 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 63, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:47:40.664091+0000', 'modified': '2024-09-10T21:47:43.681126+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 235781}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_235781': {'gid': 235781, 'name': 'a', 'rank': 0, 'incarnation': 59, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6833/1499133391', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6832', 'nonce': 1499133391}, {'type': 'v1', 'addr': '172.21.15.7:6833', 'nonce': 1499133391}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 235781, 'qdb_cluster': [235781]} 2024-09-10T21:47:47.257 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:47:47.257 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-10T21:47:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.420+0000 7f25bf000640 1 -- 172.21.15.7:0/4248506728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25b806ff50 msgr2=0x7f25b815cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.420+0000 7f25bf000640 1 --2- 172.21.15.7:0/4248506728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25b806ff50 0x7f25b815cbe0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f25b80af560 tx=0x7f25a802a440 comp rx=0 tx=0).stop 2024-09-10T21:47:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.420+0000 7f25bf000640 1 -- 172.21.15.7:0/4248506728 shutdown_connections 2024-09-10T21:47:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.420+0000 7f25bf000640 1 --2- 172.21.15.7:0/4248506728 >> v2:172.21.15.133:3300/0 conn(0x7f25b806e960 0x7f25b8074210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.420+0000 7f25bf000640 1 --2- 172.21.15.7:0/4248506728 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25b806e510 0x7f25b8073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.420+0000 7f25bf000640 1 --2- 172.21.15.7:0/4248506728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25b806ff50 0x7f25b815cbe0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.420+0000 7f25bf000640 1 -- 172.21.15.7:0/4248506728 >> 172.21.15.7:0/4248506728 conn(0x7f25b80b1660 msgr2=0x7f25b80b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:47.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.420+0000 7f25bf000640 1 -- 172.21.15.7:0/4248506728 shutdown_connections 2024-09-10T21:47:47.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.420+0000 7f25bf000640 1 -- 172.21.15.7:0/4248506728 wait complete. 2024-09-10T21:47:47.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.421+0000 7f25bf000640 1 Processor -- start 2024-09-10T21:47:47.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.422+0000 7f25bf000640 1 -- start start 2024-09-10T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.422+0000 7f25bf000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25b806e510 0x7f25b80adb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.422+0000 7f25bf000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25b806e960 0x7f25b80b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.422+0000 7f25bf000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25b806ff50 0x7f25b80b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.422+0000 7f25bf000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f25b815f710 con 0x7f25b806e960 2024-09-10T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.422+0000 7f25bf000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f25b806f930 con 0x7f25b806ff50 2024-09-10T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.422+0000 7f25bf000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f25b815f890 con 0x7f25b806e510 2024-09-10T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.423+0000 7f25afe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25b806e960 0x7f25b80b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.423+0000 7f25afe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25b806e960 0x7f25b80b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57782/0 (socket says 172.21.15.7:57782) 2024-09-10T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.423+0000 7f25afe00640 1 -- 172.21.15.7:0/3162181040 learned_addr learned my addr 172.21.15.7:0/3162181040 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:47.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.423+0000 7f25b7e00640 1 --2- 172.21.15.7:0/3162181040 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25b806e510 0x7f25b80adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.423+0000 7f25bd000640 1 --2- 172.21.15.7:0/3162181040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25b806ff50 0x7f25b80b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.423+0000 7f25afe00640 1 -- 172.21.15.7:0/3162181040 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25b806e510 msgr2=0x7f25b80adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.423+0000 7f25afe00640 1 --2- 172.21.15.7:0/3162181040 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25b806e510 0x7f25b80adb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.423+0000 7f25afe00640 1 -- 172.21.15.7:0/3162181040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25b806ff50 msgr2=0x7f25b80b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.423+0000 7f25afe00640 1 --2- 172.21.15.7:0/3162181040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25b806ff50 0x7f25b80b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.423+0000 7f25afe00640 1 -- 172.21.15.7:0/3162181040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25b80b39b0 con 0x7f25b806e960 2024-09-10T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.423+0000 7f25afe00640 1 --2- 172.21.15.7:0/3162181040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25b806e960 0x7f25b80b4cd0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f25a000c920 tx=0x7f25a000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.424+0000 7f25b4c00640 1 -- 172.21.15.7:0/3162181040 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0007c10 con 0x7f25b806e960 2024-09-10T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.424+0000 7f25b4c00640 1 -- 172.21.15.7:0/3162181040 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25a000f450 con 0x7f25b806e960 2024-09-10T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.424+0000 7f25bf000640 1 -- 172.21.15.7:0/3162181040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25b81f49b0 con 0x7f25b806e960 2024-09-10T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.424+0000 7f25b4c00640 1 -- 172.21.15.7:0/3162181040 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a000f8a0 con 0x7f25b806e960 2024-09-10T21:47:47.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.424+0000 7f25bf000640 1 -- 172.21.15.7:0/3162181040 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25b81f5470 con 0x7f25b806e960 2024-09-10T21:47:47.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.426+0000 7f25b4c00640 1 -- 172.21.15.7:0/3162181040 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f25a0020030 con 0x7f25b806e960 2024-09-10T21:47:47.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.427+0000 7f25b4c00640 1 --2- 172.21.15.7:0/3162181040 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2594077fa0 0x7f259407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.427+0000 7f25b4c00640 1 -- 172.21.15.7:0/3162181040 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f25a009a080 con 0x7f25b806e960 2024-09-10T21:47:47.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.427+0000 7f25c2021740 1 Processor -- start 2024-09-10T21:47:47.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.428+0000 7f25b7e00640 1 --2- 172.21.15.7:0/3162181040 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2594077fa0 0x7f259407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.428+0000 7f25c2021740 1 -- start start 2024-09-10T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.428+0000 7f25b7e00640 1 --2- 172.21.15.7:0/3162181040 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2594077fa0 0x7f259407a460 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f25a8033000 tx=0x7f25a8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.428+0000 7f25c2021740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422f08b0 0x55fa422f0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.429+0000 7f25c2021740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422aba20 0x55fa422ef5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.429+0000 7f25c2021740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422abfa0 0x55fa422efb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.429+0000 7f25c2021740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fa4222e430 con 0x55fa422aba20 2024-09-10T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.429+0000 7f25c2021740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fa4221e5d0 con 0x55fa422f08b0 2024-09-10T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.429+0000 7f25c2021740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55fa4220da00 con 0x55fa422abfa0 2024-09-10T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.429+0000 7f25afe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422aba20 0x55fa422ef5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.429+0000 7f25afe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422aba20 0x55fa422ef5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57784/0 (socket says 172.21.15.7:57784) 2024-09-10T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.429+0000 7f25b7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422f08b0 0x55fa422f0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.429+0000 7f25afe00640 1 -- 172.21.15.7:0/1344126944 learned_addr learned my addr 172.21.15.7:0/1344126944 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:47.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.429+0000 7f25bd000640 1 --2- 172.21.15.7:0/1344126944 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422abfa0 0x55fa422efb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.429+0000 7f25afe00640 1 -- 172.21.15.7:0/1344126944 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422abfa0 msgr2=0x55fa422efb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:47.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.429+0000 7f25afe00640 1 --2- 172.21.15.7:0/1344126944 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422abfa0 0x55fa422efb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.429+0000 7f25afe00640 1 -- 172.21.15.7:0/1344126944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422f08b0 msgr2=0x55fa422f0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:47.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.429+0000 7f25afe00640 1 --2- 172.21.15.7:0/1344126944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422f08b0 0x55fa422f0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.429+0000 7f25afe00640 1 -- 172.21.15.7:0/1344126944 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa421d6520 con 0x55fa422aba20 2024-09-10T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.430+0000 7f25afe00640 1 --2- 172.21.15.7:0/1344126944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422aba20 0x55fa422ef5c0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f25a00624b0 tx=0x7f25a0061c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69849e85dd761937 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.430+0000 7f25acc00640 1 -- 172.21.15.7:0/1344126944 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0049e60 con 0x55fa422aba20 2024-09-10T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.430+0000 7f25acc00640 1 -- 172.21.15.7:0/1344126944 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25a006b2c0 con 0x55fa422aba20 2024-09-10T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.430+0000 7f25acc00640 1 -- 172.21.15.7:0/1344126944 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a0088070 con 0x55fa422aba20 2024-09-10T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.430+0000 7f25c2021740 1 -- 172.21.15.7:0/1344126944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422aba20 msgr2=0x55fa422ef5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.430+0000 7f25c2021740 1 --2- 172.21.15.7:0/1344126944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422aba20 0x55fa422ef5c0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f25a00624b0 tx=0x7f25a0061c90 comp rx=0 tx=0).stop 2024-09-10T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.431+0000 7f25c2021740 1 -- 172.21.15.7:0/1344126944 shutdown_connections 2024-09-10T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.431+0000 7f25c2021740 1 --2- 172.21.15.7:0/1344126944 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422abfa0 0x55fa422efb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.431+0000 7f25c2021740 1 --2- 172.21.15.7:0/1344126944 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422aba20 0x55fa422ef5c0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.431+0000 7f25c2021740 1 --2- 172.21.15.7:0/1344126944 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422f08b0 0x55fa422f0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.431+0000 7f25c2021740 1 -- 172.21.15.7:0/1344126944 >> 172.21.15.7:0/1344126944 conn(0x55fa422a6ff0 msgr2=0x55fa422aaa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:47.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.431+0000 7f25c2021740 1 -- 172.21.15.7:0/1344126944 shutdown_connections 2024-09-10T21:47:47.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.431+0000 7f25c2021740 1 -- 172.21.15.7:0/1344126944 wait complete. 2024-09-10T21:47:47.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.431+0000 7f25c2021740 1 -- start start 2024-09-10T21:47:47.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.431+0000 7f25c2021740 1 Processor -- start 2024-09-10T21:47:47.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.432+0000 7f25c2021740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422abfa0 0x55fa422bf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.432+0000 7f25c2021740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422f08b0 0x55fa422e8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.432+0000 7f25c2021740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422d08c0 0x55fa422e9520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.432+0000 7f25c2021740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fa421b4330 con 0x55fa422abfa0 2024-09-10T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.432+0000 7f25c2021740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fa42211810 con 0x55fa422f08b0 2024-09-10T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.432+0000 7f25c2021740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55fa41f7e620 con 0x55fa422d08c0 2024-09-10T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.432+0000 7f25c2021740 10 client.0 _ll_register_callbacks cb 0x55fa421b9420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.433+0000 7f25b7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422abfa0 0x55fa422bf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.433+0000 7f25b7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422abfa0 0x55fa422bf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57800/0 (socket says 172.21.15.7:57800) 2024-09-10T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.433+0000 7f25b7e00640 1 -- 172.21.15.7:0/188634472 learned_addr learned my addr 172.21.15.7:0/188634472 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.433+0000 7f25afe00640 1 --2- 172.21.15.7:0/188634472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422f08b0 0x55fa422e8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.433+0000 7f25bd000640 1 --2- 172.21.15.7:0/188634472 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422d08c0 0x55fa422e9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.433+0000 7f25b7e00640 1 -- 172.21.15.7:0/188634472 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422d08c0 msgr2=0x55fa422e9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:47.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.433+0000 7f25b7e00640 1 --2- 172.21.15.7:0/188634472 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422d08c0 0x55fa422e9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.433+0000 7f25b7e00640 1 -- 172.21.15.7:0/188634472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422f08b0 msgr2=0x55fa422e8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.433+0000 7f25b7e00640 1 --2- 172.21.15.7:0/188634472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422f08b0 0x55fa422e8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.433+0000 7f25b7e00640 1 -- 172.21.15.7:0/188634472 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa41f6f850 con 0x55fa422abfa0 2024-09-10T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.433+0000 7f25b7e00640 1 --2- 172.21.15.7:0/188634472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422abfa0 0x55fa422bf030 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f25a80099c0 tx=0x7f25a8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc739ab1a1cf06a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.433+0000 7f25a5600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.434+0000 7f25a5600640 1 -- 172.21.15.7:0/188634472 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a804e030 con 0x55fa422abfa0 2024-09-10T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.434+0000 7f25c2021740 1 -- 172.21.15.7:0/188634472 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25a006b2c0 con 0x55fa422abfa0 2024-09-10T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.434+0000 7f25a5600640 1 -- 172.21.15.7:0/188634472 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f25a803cbd0 con 0x55fa422abfa0 2024-09-10T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.434+0000 7f25a5600640 1 -- 172.21.15.7:0/188634472 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f25a8035040 con 0x55fa422abfa0 2024-09-10T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.434+0000 7f25c2021740 10 client.6873 fetch_fsmap learned FSMap version 63 2024-09-10T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.434+0000 7f25c2021740 1 -- 172.21.15.7:0/188634472 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) -- 0x55fa41fe66b0 con 0x55fa422abfa0 2024-09-10T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.434+0000 7f25a5600640 1 -- 172.21.15.7:0/188634472 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25a006b2c0 con 0x55fa422abfa0 2024-09-10T21:47:47.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.434+0000 7f25a5600640 1 -- 172.21.15.7:0/188634472 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 63) ==== 3502+0+0 (secure 0 0 0) 0x7f25a803a040 con 0x55fa422abfa0 2024-09-10T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.434+0000 7f25c2021740 10 client.6873 fetch_fsmap finished waiting for FSMap version 63 2024-09-10T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.434+0000 7f25c2021740 10 client.6873 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.434+0000 7f25c2021740 20 client.6873 populate_metadata read hostname 'smithi007' 2024-09-10T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.434+0000 7f25c2021740 1 --2- 172.21.15.7:0/188634472 >> [v2:172.21.15.7:6832/1499133391,v1:172.21.15.7:6833/1499133391] conn(0x55fa422d6e50 0x55fa422e4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.434+0000 7f25c2021740 4 client.6873 mds_command: new command op to 235781 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.434+0000 7f25c2021740 1 -- 172.21.15.7:0/188634472 --> [v2:172.21.15.7:6832/1499133391,v1:172.21.15.7:6833/1499133391] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa42220520 con 0x55fa422d6e50 2024-09-10T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.435+0000 7f25afe00640 1 --2- 172.21.15.7:0/188634472 >> [v2:172.21.15.7:6832/1499133391,v1:172.21.15.7:6833/1499133391] conn(0x55fa422d6e50 0x55fa422e4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.435+0000 7f25afe00640 1 --2- 172.21.15.7:0/188634472 >> [v2:172.21.15.7:6832/1499133391,v1:172.21.15.7:6833/1499133391] conn(0x55fa422d6e50 0x55fa422e4800 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74f051e54761dd09 server_cookie=6095af0d03d9ec13 in_seq=0 out_seq=0 2024-09-10T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.435+0000 7f25a5600640 10 client.6873 ms_handle_connect on v2:172.21.15.7:6832/1499133391 2024-09-10T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.436+0000 7f25a5600640 1 -- 172.21.15.7:0/188634472 <== mds.0 v2:172.21.15.7:6832/1499133391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55fa42220520 con 0x55fa422d6e50 2024-09-10T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.436+0000 7f25a5600640 10 client.6873 handle_command_reply: tid=0 2024-09-10T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.436+0000 7f25c2021740 1 client.6873 shutdown 2024-09-10T21:47:47.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.436+0000 7f25c2021740 1 -- 172.21.15.7:0/188634472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422abfa0 msgr2=0x55fa422bf030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.436+0000 7f25c2021740 1 --2- 172.21.15.7:0/188634472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422abfa0 0x55fa422bf030 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f25a80099c0 tx=0x7f25a8009260 comp rx=0 tx=0).stop 2024-09-10T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.436+0000 7f25c2021740 1 -- 172.21.15.7:0/188634472 shutdown_connections 2024-09-10T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.436+0000 7f25c2021740 1 --2- 172.21.15.7:0/188634472 >> [v2:172.21.15.7:6832/1499133391,v1:172.21.15.7:6833/1499133391] conn(0x55fa422d6e50 0x55fa422e4800 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.436+0000 7f25c2021740 1 --2- 172.21.15.7:0/188634472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422f08b0 0x55fa422e8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.436+0000 7f25c2021740 1 --2- 172.21.15.7:0/188634472 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422d08c0 0x55fa422e9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.436+0000 7f25c2021740 1 --2- 172.21.15.7:0/188634472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422abfa0 0x55fa422bf030 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.436+0000 7f25a5600640 0 client.6873 ms_handle_reset on v2:172.21.15.7:6832/1499133391 2024-09-10T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.436+0000 7f25c2021740 1 -- 172.21.15.7:0/188634472 >> 172.21.15.7:0/188634472 conn(0x55fa422aba20 msgr2=0x55fa422d1fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.436+0000 7f25c2021740 1 -- 172.21.15.7:0/188634472 shutdown_connections 2024-09-10T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.437+0000 7f25c2021740 1 -- 172.21.15.7:0/188634472 wait complete. 2024-09-10T21:47:47.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.437+0000 7f25c2021740 20 client.6873 trim_cache size 0 max 16384 2024-09-10T21:47:47.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.456+0000 7f25c2021740 1 Processor -- start 2024-09-10T21:47:47.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.456+0000 7f25c2021740 1 -- start start 2024-09-10T21:47:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.457+0000 7f25c2021740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422abfa0 0x55fa422e9640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.457+0000 7f25c2021740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422d08c0 0x55fa4231c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.457+0000 7f25c2021740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422f08b0 0x55fa422d3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.457+0000 7f25c2021740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f25b80aa7c0 con 0x55fa422abfa0 2024-09-10T21:47:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.457+0000 7f25c2021740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f25b81f89c0 con 0x55fa422d08c0 2024-09-10T21:47:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.457+0000 7f25c2021740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f25b80ce1a0 con 0x55fa422f08b0 2024-09-10T21:47:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.457+0000 7f25b7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422abfa0 0x55fa422e9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.457+0000 7f25b7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422abfa0 0x55fa422e9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57806/0 (socket says 172.21.15.7:57806) 2024-09-10T21:47:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.457+0000 7f25b7e00640 1 -- 172.21.15.7:0/509049515 learned_addr learned my addr 172.21.15.7:0/509049515 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.457+0000 7f25afe00640 1 --2- 172.21.15.7:0/509049515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422d08c0 0x55fa4231c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.457+0000 7f25bd000640 1 --2- 172.21.15.7:0/509049515 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422f08b0 0x55fa422d3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.457+0000 7f25b7e00640 1 -- 172.21.15.7:0/509049515 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422f08b0 msgr2=0x55fa422d3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.457+0000 7f25b7e00640 1 --2- 172.21.15.7:0/509049515 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422f08b0 0x55fa422d3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.457+0000 7f25b7e00640 1 -- 172.21.15.7:0/509049515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422d08c0 msgr2=0x55fa4231c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.457+0000 7f25b7e00640 1 --2- 172.21.15.7:0/509049515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422d08c0 0x55fa4231c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.457+0000 7f25b7e00640 1 -- 172.21.15.7:0/509049515 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa422e5990 con 0x55fa422abfa0 2024-09-10T21:47:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.457+0000 7f25bd000640 1 --2- 172.21.15.7:0/509049515 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422f08b0 0x55fa422d3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:47:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.458+0000 7f25b7e00640 1 --2- 172.21.15.7:0/509049515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422abfa0 0x55fa422e9640 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f25a8000c00 tx=0x7f25a8003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d7aa3d22752622 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.458+0000 7f25a7e00640 1 -- 172.21.15.7:0/509049515 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a804e070 con 0x55fa422abfa0 2024-09-10T21:47:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.458+0000 7f25a7e00640 1 -- 172.21.15.7:0/509049515 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25a804f040 con 0x55fa422abfa0 2024-09-10T21:47:47.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.458+0000 7f25a7e00640 1 -- 172.21.15.7:0/509049515 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25a8053460 con 0x55fa422abfa0 2024-09-10T21:47:47.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.459+0000 7f25c2021740 1 -- 172.21.15.7:0/509049515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422abfa0 msgr2=0x55fa422e9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:47.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.459+0000 7f25c2021740 1 --2- 172.21.15.7:0/509049515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422abfa0 0x55fa422e9640 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f25a8000c00 tx=0x7f25a8003f50 comp rx=0 tx=0).stop 2024-09-10T21:47:47.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.459+0000 7f25c2021740 1 -- 172.21.15.7:0/509049515 shutdown_connections 2024-09-10T21:47:47.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.459+0000 7f25c2021740 1 --2- 172.21.15.7:0/509049515 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422d08c0 0x55fa4231c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.459+0000 7f25c2021740 1 --2- 172.21.15.7:0/509049515 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422f08b0 0x55fa422d3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.459+0000 7f25c2021740 1 --2- 172.21.15.7:0/509049515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422abfa0 0x55fa422e9640 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.459+0000 7f25c2021740 1 -- 172.21.15.7:0/509049515 >> 172.21.15.7:0/509049515 conn(0x55fa422aba20 msgr2=0x55fa422d7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:47.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.459+0000 7f25c2021740 1 -- 172.21.15.7:0/509049515 shutdown_connections 2024-09-10T21:47:47.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.459+0000 7f25c2021740 1 -- 172.21.15.7:0/509049515 wait complete. 2024-09-10T21:47:47.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.459+0000 7f25c2021740 1 -- start start 2024-09-10T21:47:47.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.460+0000 7f25c2021740 1 Processor -- start 2024-09-10T21:47:47.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.460+0000 7f25c2021740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422abfa0 0x55fa4231a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.460+0000 7f25c2021740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422d08c0 0x55fa42379ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.460+0000 7f25c2021740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422f08b0 0x55fa42376ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.460+0000 7f25c2021740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55fa422d6cd0 con 0x55fa422f08b0 2024-09-10T21:47:47.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.460+0000 7f25c2021740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55fa422e52a0 con 0x55fa422abfa0 2024-09-10T21:47:47.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.460+0000 7f25c2021740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55fa422d7320 con 0x55fa422d08c0 2024-09-10T21:47:47.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.460+0000 7f25c2021740 10 client.0 _ll_register_callbacks cb 0x7f25a8003a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:47:47.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.461+0000 7f25bd000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422f08b0 0x55fa42376ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.461+0000 7f25bd000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422f08b0 0x55fa42376ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57822/0 (socket says 172.21.15.7:57822) 2024-09-10T21:47:47.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.461+0000 7f25bd000640 1 -- 172.21.15.7:0/2756488011 learned_addr learned my addr 172.21.15.7:0/2756488011 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:47.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.461+0000 7f25b7e00640 1 --2- 172.21.15.7:0/2756488011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422abfa0 0x55fa4231a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.461+0000 7f25afe00640 1 --2- 172.21.15.7:0/2756488011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422d08c0 0x55fa42379ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.461+0000 7f25bd000640 1 -- 172.21.15.7:0/2756488011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422d08c0 msgr2=0x55fa42379ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:47.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.461+0000 7f25bd000640 1 --2- 172.21.15.7:0/2756488011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422d08c0 0x55fa42379ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.461+0000 7f25bd000640 1 -- 172.21.15.7:0/2756488011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422abfa0 msgr2=0x55fa4231a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:47.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.461+0000 7f25bd000640 1 --2- 172.21.15.7:0/2756488011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422abfa0 0x55fa4231a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.461+0000 7f25bd000640 1 -- 172.21.15.7:0/2756488011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa422e48a0 con 0x55fa422f08b0 2024-09-10T21:47:47.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.461+0000 7f25bd000640 1 --2- 172.21.15.7:0/2756488011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422f08b0 0x55fa42376ef0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f25b0010b50 tx=0x7f25b000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bceb0d1ee3c4988 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:47.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.461+0000 7f25a5600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:47:47.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.462+0000 7f25a5600640 1 -- 172.21.15.7:0/2756488011 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b0020060 con 0x55fa422f08b0 2024-09-10T21:47:47.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.462+0000 7f25a5600640 1 -- 172.21.15.7:0/2756488011 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f25b0003b70 con 0x55fa422f08b0 2024-09-10T21:47:47.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.462+0000 7f25c2021740 1 -- 172.21.15.7:0/2756488011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25a804f040 con 0x55fa422f08b0 2024-09-10T21:47:47.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.462+0000 7f25a5600640 1 -- 172.21.15.7:0/2756488011 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=63) ==== 24+0+0 (secure 0 0 0) 0x7f25b0010e50 con 0x55fa422f08b0 2024-09-10T21:47:47.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.462+0000 7f25c2021740 10 client.6879 fetch_fsmap learned FSMap version 63 2024-09-10T21:47:47.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.462+0000 7f25c2021740 1 -- 172.21.15.7:0/2756488011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=63,monmap=2+}) -- 0x55fa4237a280 con 0x55fa422f08b0 2024-09-10T21:47:47.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.462+0000 7f25a5600640 1 -- 172.21.15.7:0/2756488011 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25a804f040 con 0x55fa422f08b0 2024-09-10T21:47:47.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.462+0000 7f25a5600640 1 -- 172.21.15.7:0/2756488011 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 63) ==== 3502+0+0 (secure 0 0 0) 0x7f25b0029020 con 0x55fa422f08b0 2024-09-10T21:47:47.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.462+0000 7f25c2021740 10 client.6879 fetch_fsmap finished waiting for FSMap version 63 2024-09-10T21:47:47.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.462+0000 7f25c2021740 10 client.6879 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-10T21:47:47.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.462+0000 7f25c2021740 20 client.6879 populate_metadata read hostname 'smithi007' 2024-09-10T21:47:47.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.462+0000 7f25c2021740 1 --2- 172.21.15.7:0/2756488011 >> [v2:172.21.15.7:6832/1499133391,v1:172.21.15.7:6833/1499133391] conn(0x55fa422d3f30 0x55fa4237d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.462+0000 7f25c2021740 4 client.6879 mds_command: new command op to 235781 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:47:47.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.462+0000 7f25c2021740 1 -- 172.21.15.7:0/2756488011 --> [v2:172.21.15.7:6832/1499133391,v1:172.21.15.7:6833/1499133391] -- command(tid 0: {"prefix": "status"}) -- 0x55fa4221bfd0 con 0x55fa422d3f30 2024-09-10T21:47:47.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.463+0000 7f25b7e00640 1 --2- 172.21.15.7:0/2756488011 >> [v2:172.21.15.7:6832/1499133391,v1:172.21.15.7:6833/1499133391] conn(0x55fa422d3f30 0x55fa4237d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.463+0000 7f25a5600640 10 client.6879 ms_handle_connect on v2:172.21.15.7:6832/1499133391 2024-09-10T21:47:47.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.463+0000 7f25b7e00640 1 --2- 172.21.15.7:0/2756488011 >> [v2:172.21.15.7:6832/1499133391,v1:172.21.15.7:6833/1499133391] conn(0x55fa422d3f30 0x55fa4237d0b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c07bce14fe41a700 server_cookie=ed8f29d56dce1e07 in_seq=0 out_seq=0 2024-09-10T21:47:47.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.464+0000 7f25a5600640 1 -- 172.21.15.7:0/2756488011 <== mds.0 v2:172.21.15.7:6832/1499133391 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55fa4221bfd0 con 0x55fa422d3f30 2024-09-10T21:47:47.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.464+0000 7f25a5600640 10 client.6879 handle_command_reply: tid=0 2024-09-10T21:47:47.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.464+0000 7f25c2021740 1 client.6879 shutdown 2024-09-10T21:47:47.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.464+0000 7f25c2021740 1 -- 172.21.15.7:0/2756488011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422f08b0 msgr2=0x55fa42376ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:47.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.464+0000 7f25c2021740 1 --2- 172.21.15.7:0/2756488011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422f08b0 0x55fa42376ef0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f25b0010b50 tx=0x7f25b000f7f0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.464+0000 7f25c2021740 1 -- 172.21.15.7:0/2756488011 shutdown_connections 2024-09-10T21:47:47.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.464+0000 7f25c2021740 1 --2- 172.21.15.7:0/2756488011 >> [v2:172.21.15.7:6832/1499133391,v1:172.21.15.7:6833/1499133391] conn(0x55fa422d3f30 0x55fa4237d0b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.464+0000 7f25c2021740 1 --2- 172.21.15.7:0/2756488011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55fa422f08b0 0x55fa42376ef0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.464+0000 7f25c2021740 1 --2- 172.21.15.7:0/2756488011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55fa422d08c0 0x55fa42379ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.464+0000 7f25a5600640 0 client.6879 ms_handle_reset on v2:172.21.15.7:6832/1499133391 2024-09-10T21:47:47.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.464+0000 7f25c2021740 1 --2- 172.21.15.7:0/2756488011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55fa422abfa0 0x55fa4231a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.464+0000 7f25c2021740 1 -- 172.21.15.7:0/2756488011 >> 172.21.15.7:0/2756488011 conn(0x55fa422aba20 msgr2=0x55fa422d7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:47.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.464+0000 7f25c2021740 1 -- 172.21.15.7:0/2756488011 shutdown_connections 2024-09-10T21:47:47.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.464+0000 7f25c2021740 1 -- 172.21.15.7:0/2756488011 wait complete. 2024-09-10T21:47:47.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.464+0000 7f25c2021740 20 client.6879 trim_cache size 0 max 16384 2024-09-10T21:47:47.469 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:47:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2029451a-3c26-48c7-b874-0597197278ac", 2024-09-10T21:47:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-10T21:47:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 235781, 2024-09-10T21:47:47.469 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T21:47:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T21:47:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T21:47:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.2440713859999999, 2024-09-10T21:47:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 63, 2024-09-10T21:47:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 85, 2024-09-10T21:47:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 85, 2024-09-10T21:47:47.470 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 11.528064675 2024-09-10T21:47:47.470 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:47:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.465+0000 7f25bf000640 1 -- 172.21.15.7:0/3162181040 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2594077fa0 msgr2=0x7f259407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.465+0000 7f25bf000640 1 --2- 172.21.15.7:0/3162181040 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2594077fa0 0x7f259407a460 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f25a8033000 tx=0x7f25a8004730 comp rx=0 tx=0).stop 2024-09-10T21:47:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.465+0000 7f25bf000640 1 -- 172.21.15.7:0/3162181040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25b806e960 msgr2=0x7f25b80b4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.465+0000 7f25bf000640 1 --2- 172.21.15.7:0/3162181040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25b806e960 0x7f25b80b4cd0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f25a000c920 tx=0x7f25a000cde0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.465+0000 7f25bf000640 1 -- 172.21.15.7:0/3162181040 shutdown_connections 2024-09-10T21:47:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.465+0000 7f25bf000640 1 --2- 172.21.15.7:0/3162181040 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2594077fa0 0x7f259407a460 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.465+0000 7f25bf000640 1 --2- 172.21.15.7:0/3162181040 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f25b806ff50 0x7f25b80b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.465+0000 7f25bf000640 1 --2- 172.21.15.7:0/3162181040 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f25b806e960 0x7f25b80b4cd0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.465+0000 7f25bf000640 1 --2- 172.21.15.7:0/3162181040 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f25b806e510 0x7f25b80adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.465+0000 7f25bf000640 1 -- 172.21.15.7:0/3162181040 >> 172.21.15.7:0/3162181040 conn(0x7f25b80b1660 msgr2=0x7f25b80b01c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:47.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.465+0000 7f25bf000640 1 -- 172.21.15.7:0/3162181040 shutdown_connections 2024-09-10T21:47:47.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.465+0000 7f25bf000640 1 -- 172.21.15.7:0/3162181040 wait complete. 2024-09-10T21:47:47.481 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:47:47.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.639+0000 7f3952800640 1 -- 172.21.15.7:0/2876363907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c15bc90 msgr2=0x7f394c0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:47.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.639+0000 7f3952800640 1 --2- 172.21.15.7:0/2876363907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c15bc90 0x7f394c0ccef0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f394c0cf860 tx=0x7f393c02a4d0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.639+0000 7f3952800640 1 -- 172.21.15.7:0/2876363907 shutdown_connections 2024-09-10T21:47:47.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.639+0000 7f3952800640 1 --2- 172.21.15.7:0/2876363907 >> v2:172.21.15.133:3300/0 conn(0x7f394c0cd750 0x7f394c073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.639+0000 7f3952800640 1 --2- 172.21.15.7:0/2876363907 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c15bc90 0x7f394c0ccef0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.639+0000 7f3952800640 1 -- 172.21.15.7:0/2876363907 >> 172.21.15.7:0/2876363907 conn(0x7f394c0b16d0 msgr2=0x7f394c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:47.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.639+0000 7f3952800640 1 -- 172.21.15.7:0/2876363907 shutdown_connections 2024-09-10T21:47:47.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.639+0000 7f3952800640 1 -- 172.21.15.7:0/2876363907 wait complete. 2024-09-10T21:47:47.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.641+0000 7f3952800640 1 Processor -- start 2024-09-10T21:47:47.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.641+0000 7f3952800640 1 -- start start 2024-09-10T21:47:47.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.641+0000 7f3952800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c073c70 0x7f394c1f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.642+0000 7f3952800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f394c0cd750 0x7f394c0af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.642+0000 7f3952800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f394c15bc90 0x7f394c1f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.642+0000 7f3952800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f394c06fcf0 con 0x7f394c073c70 2024-09-10T21:47:47.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.642+0000 7f3952800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f394c1606c0 con 0x7f394c15bc90 2024-09-10T21:47:47.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.642+0000 7f3952800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f394c06fe70 con 0x7f394c0cd750 2024-09-10T21:47:47.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.642+0000 7f394b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c073c70 0x7f394c1f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.642+0000 7f394b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c073c70 0x7f394c1f52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57832/0 (socket says 172.21.15.7:57832) 2024-09-10T21:47:47.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.642+0000 7f394b400640 1 -- 172.21.15.7:0/985542868 learned_addr learned my addr 172.21.15.7:0/985542868 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:47.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.642+0000 7f394be00640 1 --2- 172.21.15.7:0/985542868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f394c15bc90 0x7f394c1f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.642+0000 7f394aa00640 1 --2- 172.21.15.7:0/985542868 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f394c0cd750 0x7f394c0af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.642+0000 7f394b400640 1 -- 172.21.15.7:0/985542868 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f394c0cd750 msgr2=0x7f394c0af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:47.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.642+0000 7f394b400640 1 --2- 172.21.15.7:0/985542868 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f394c0cd750 0x7f394c0af4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.642+0000 7f394b400640 1 -- 172.21.15.7:0/985542868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f394c15bc90 msgr2=0x7f394c1f7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:47.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.642+0000 7f394b400640 1 --2- 172.21.15.7:0/985542868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f394c15bc90 0x7f394c1f7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.642+0000 7f394b400640 1 -- 172.21.15.7:0/985542868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f394c0afa30 con 0x7f394c073c70 2024-09-10T21:47:47.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.643+0000 7f394b400640 1 --2- 172.21.15.7:0/985542868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c073c70 0x7f394c1f52f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f393c000c00 tx=0x7f393c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.643+0000 7f3943400640 1 -- 172.21.15.7:0/985542868 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c03e030 con 0x7f394c073c70 2024-09-10T21:47:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.643+0000 7f3943400640 1 -- 172.21.15.7:0/985542868 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f393c003e00 con 0x7f394c073c70 2024-09-10T21:47:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.643+0000 7f3943400640 1 -- 172.21.15.7:0/985542868 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c035040 con 0x7f394c073c70 2024-09-10T21:47:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.643+0000 7f3952800640 1 -- 172.21.15.7:0/985542868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f394c0afcf0 con 0x7f394c073c70 2024-09-10T21:47:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.643+0000 7f3952800640 1 -- 172.21.15.7:0/985542868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f394c0b02b0 con 0x7f394c073c70 2024-09-10T21:47:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.644+0000 7f3952800640 1 -- 172.21.15.7:0/985542868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f394c15f610 con 0x7f394c073c70 2024-09-10T21:47:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.645+0000 7f3943400640 1 -- 172.21.15.7:0/985542868 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f393c0049c0 con 0x7f394c073c70 2024-09-10T21:47:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.646+0000 7f3943400640 1 --2- 172.21.15.7:0/985542868 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f392c077f70 0x7f392c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.646+0000 7f3943400640 1 -- 172.21.15.7:0/985542868 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f393c0ba6a0 con 0x7f394c073c70 2024-09-10T21:47:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.647+0000 7f394aa00640 1 --2- 172.21.15.7:0/985542868 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f392c077f70 0x7f392c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:47.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.647+0000 7f394aa00640 1 --2- 172.21.15.7:0/985542868 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f392c077f70 0x7f392c07a430 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f394c14dfd0 tx=0x7f393800b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:47.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.650+0000 7f3943400640 1 -- 172.21.15.7:0/985542868 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f393c082b40 con 0x7f394c073c70 2024-09-10T21:47:47.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.887+0000 7f3952800640 1 -- 172.21.15.7:0/985542868 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f394c073730 con 0x7f394c073c70 2024-09-10T21:47:47.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.887+0000 7f3943400640 1 -- 172.21.15.7:0/985542868 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v63) ==== 53+0+73 (secure 0 0 0) 0x7f393c087570 con 0x7f394c073c70 2024-09-10T21:47:47.889 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:47:47.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.890+0000 7f3952800640 1 -- 172.21.15.7:0/985542868 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f392c077f70 msgr2=0x7f392c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:47.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.890+0000 7f3952800640 1 --2- 172.21.15.7:0/985542868 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f392c077f70 0x7f392c07a430 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f394c14dfd0 tx=0x7f393800b390 comp rx=0 tx=0).stop 2024-09-10T21:47:47.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.890+0000 7f3952800640 1 -- 172.21.15.7:0/985542868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c073c70 msgr2=0x7f394c1f52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:47.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.890+0000 7f3952800640 1 --2- 172.21.15.7:0/985542868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c073c70 0x7f394c1f52f0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f393c000c00 tx=0x7f393c003910 comp rx=0 tx=0).stop 2024-09-10T21:47:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.890+0000 7f3952800640 1 -- 172.21.15.7:0/985542868 shutdown_connections 2024-09-10T21:47:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.890+0000 7f3952800640 1 --2- 172.21.15.7:0/985542868 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f392c077f70 0x7f392c07a430 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.890+0000 7f3952800640 1 --2- 172.21.15.7:0/985542868 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f394c15bc90 0x7f394c1f7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.890+0000 7f3952800640 1 --2- 172.21.15.7:0/985542868 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f394c0cd750 0x7f394c0af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.890+0000 7f3952800640 1 --2- 172.21.15.7:0/985542868 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f394c073c70 0x7f394c1f52f0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.890+0000 7f3952800640 1 -- 172.21.15.7:0/985542868 >> 172.21.15.7:0/985542868 conn(0x7f394c0b16d0 msgr2=0x7f394c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.890+0000 7f3952800640 1 -- 172.21.15.7:0/985542868 shutdown_connections 2024-09-10T21:47:47.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:47.890+0000 7f3952800640 1 -- 172.21.15.7:0/985542868 wait complete. 2024-09-10T21:47:47.907 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:47:47.907 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:47:47.907 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-10T21:47:47.907 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:47:47.907 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:47:47.907 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:47:47.907 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:47:47.908 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:47:47.908 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:47:47.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:47.908 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-10T21:47:47.944 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:47:47.944 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:47:47.944 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:47:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T21:47:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:47.945 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:47:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T21:47:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-10T21:47:47.945 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:47:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T21:47:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-10T21:47:47.945 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:47:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-10T21:47:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-10T21:47:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T21:47:47.945 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42277sec preferred_lft 42277sec 2024-09-10T21:47:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-10T21:47:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:47.946 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:47:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-10T21:47:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-10T21:47:47.946 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:47:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff 2024-09-10T21:47:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:47:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::3c3d:dbff:fe02:4c15/64 scope link 2024-09-10T21:47:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:47.946 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:47:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether f6:f9:69:5a:86:f5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:47:47.946 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f4f9:69ff:fe5a:86f5/64 scope link 2024-09-10T21:47:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:47.947 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:47:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:47:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::433:5cff:fe44:4a5b/64 scope link 2024-09-10T21:47:47.947 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:47.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:47.947 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:47:48.039 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:47:48.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:48.042 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-10T21:47:48.078 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-10T21:47:48.078 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:47:48.078 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T21:47:48.079 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T21:47:48.079 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:47:48.079 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:47:48.079 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:47:48.079 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:47:48.079 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:47:48.079 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:47:48.079 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:47:48.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:47:48.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:47:48.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:48.280 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:47:48.316 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:47:48.316 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:47:48.316 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:47:48.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:48.316 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-10T21:47:48.412 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:47:48.412 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:47:48.447 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:47:48.447 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:47:48.447 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:47:48.522 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:47:48.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:48.523 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:47:48.651 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:47:48.651 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:47:48.680 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:47:48.680 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:47:48.680 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.133:3300,172.21.15.133:3301" 2024-09-10T21:47:48.680 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@2029451a-3c26-48c7-b874-0597197278ac.cephfs=/ 2024-09-10T21:47:48.680 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.133:3300/172.21.15.133:3301" will pass to kernel 2024-09-10T21:47:48.681 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T21:47:48.681 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:47:48.681 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:47:48.682 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:47:48.682 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:47:48.684 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:47:48.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:48.685 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:47:48.742 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:47:48.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:48 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:47:48.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:48 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:47:48.841 INFO:teuthology.orchestra.run.smithi007.stdout:client6478 2024-09-10T21:47:48.841 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:47:48.841 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:47:48.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:48.878 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir) 2024-09-10T21:47:48.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:48.961 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir/file) 2024-09-10T21:47:49.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:49.040 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir testdir) 2024-09-10T21:47:49.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:49.118 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec ln dir/file testdir/hardlink) 2024-09-10T21:47:49.199 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:47:49.199 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:47:49.199 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:47:49.200 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:47:49.200 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir").st_ino) 2024-09-10T21:47:49.200 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:47:49.344 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627776 2024-09-10T21:47:49.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:47:49.356 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:47:49.356 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:47:49.433 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:47:49.433 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:47:49.509 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:47:49.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.708+0000 7fc331600640 1 -- 172.21.15.7:0/1571208251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc32c0cddd0 msgr2=0x7fc32c0ce260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.708+0000 7fc331600640 1 --2- 172.21.15.7:0/1571208251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc32c0cddd0 0x7fc32c0ce260 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc324005fd0 tx=0x7fc32402a830 comp rx=0 tx=0).stop 2024-09-10T21:47:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.708+0000 7fc331600640 1 -- 172.21.15.7:0/1571208251 shutdown_connections 2024-09-10T21:47:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.708+0000 7fc331600640 1 --2- 172.21.15.7:0/1571208251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc32c0cddd0 0x7fc32c0ce260 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.708+0000 7fc331600640 1 --2- 172.21.15.7:0/1571208251 >> v2:172.21.15.133:3300/0 conn(0x7fc32c15f4d0 0x7fc32c0cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.708+0000 7fc331600640 1 -- 172.21.15.7:0/1571208251 >> 172.21.15.7:0/1571208251 conn(0x7fc32c0b16d0 msgr2=0x7fc32c0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.708+0000 7fc331600640 1 -- 172.21.15.7:0/1571208251 shutdown_connections 2024-09-10T21:47:49.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.708+0000 7fc331600640 1 -- 172.21.15.7:0/1571208251 wait complete. 2024-09-10T21:47:49.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.710+0000 7fc331600640 1 Processor -- start 2024-09-10T21:47:49.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.710+0000 7fc331600640 1 -- start start 2024-09-10T21:47:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.711+0000 7fc331600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc32c0702b0 0x7fc32c14f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.711+0000 7fc331600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc32c0cddd0 0x7fc32c1f5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.711+0000 7fc331600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc32c15f4d0 0x7fc32c1f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.711+0000 7fc331600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c0ce920 con 0x7fc32c0cddd0 2024-09-10T21:47:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.711+0000 7fc331600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c0ce7a0 con 0x7fc32c0702b0 2024-09-10T21:47:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.711+0000 7fc331600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc32c0ceaa0 con 0x7fc32c15f4d0 2024-09-10T21:47:49.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.711+0000 7fc323e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc32c0cddd0 0x7fc32c1f5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.711+0000 7fc323e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc32c0cddd0 0x7fc32c1f5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57846/0 (socket says 172.21.15.7:57846) 2024-09-10T21:47:49.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.711+0000 7fc323e00640 1 -- 172.21.15.7:0/2622376033 learned_addr learned my addr 172.21.15.7:0/2622376033 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:49.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.712+0000 7fc32be00640 1 --2- 172.21.15.7:0/2622376033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc32c0702b0 0x7fc32c14f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:49.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.712+0000 7fc330c00640 1 --2- 172.21.15.7:0/2622376033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc32c15f4d0 0x7fc32c1f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:49.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.712+0000 7fc323e00640 1 -- 172.21.15.7:0/2622376033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc32c15f4d0 msgr2=0x7fc32c1f5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:49.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.712+0000 7fc323e00640 1 --2- 172.21.15.7:0/2622376033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc32c15f4d0 0x7fc32c1f5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:49.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.712+0000 7fc323e00640 1 -- 172.21.15.7:0/2622376033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc32c0702b0 msgr2=0x7fc32c14f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:49.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.712+0000 7fc323e00640 1 --2- 172.21.15.7:0/2622376033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc32c0702b0 0x7fc32c14f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:49.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.712+0000 7fc323e00640 1 -- 172.21.15.7:0/2622376033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32c13e8e0 con 0x7fc32c0cddd0 2024-09-10T21:47:49.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.712+0000 7fc323e00640 1 --2- 172.21.15.7:0/2622376033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc32c0cddd0 0x7fc32c1f5540 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc31400c6e0 tx=0x7fc31400cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.712+0000 7fc328c00640 1 -- 172.21.15.7:0/2622376033 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc314016020 con 0x7fc32c0cddd0 2024-09-10T21:47:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.712+0000 7fc328c00640 1 -- 172.21.15.7:0/2622376033 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc31400a930 con 0x7fc32c0cddd0 2024-09-10T21:47:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.712+0000 7fc328c00640 1 -- 172.21.15.7:0/2622376033 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc31400ad60 con 0x7fc32c0cddd0 2024-09-10T21:47:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.712+0000 7fc331600640 1 -- 172.21.15.7:0/2622376033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32c14f5c0 con 0x7fc32c0cddd0 2024-09-10T21:47:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.713+0000 7fc331600640 1 -- 172.21.15.7:0/2622376033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32c14d7d0 con 0x7fc32c0cddd0 2024-09-10T21:47:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.713+0000 7fc331600640 1 -- 172.21.15.7:0/2622376033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc32c0ce5c0 con 0x7fc32c0cddd0 2024-09-10T21:47:49.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.715+0000 7fc328c00640 1 -- 172.21.15.7:0/2622376033 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc31401b400 con 0x7fc32c0cddd0 2024-09-10T21:47:49.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.716+0000 7fc328c00640 1 --2- 172.21.15.7:0/2622376033 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc308077f70 0x7fc30807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:49.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.716+0000 7fc328c00640 1 -- 172.21.15.7:0/2622376033 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5738+0+0 (secure 0 0 0) 0x7fc31409a1d0 con 0x7fc32c0cddd0 2024-09-10T21:47:49.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.716+0000 7fc32be00640 1 --2- 172.21.15.7:0/2622376033 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc308077f70 0x7fc30807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:49.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.717+0000 7fc32be00640 1 --2- 172.21.15.7:0/2622376033 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc308077f70 0x7fc30807a430 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009ac0 tx=0x7fc31c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:49.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.720+0000 7fc328c00640 1 -- 172.21.15.7:0/2622376033 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc32c0ce5c0 con 0x7fc32c0cddd0 2024-09-10T21:47:49.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.926+0000 7fc331600640 1 -- 172.21.15.7:0/2622376033 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc32c0ce260 con 0x7fc32c0cddd0 2024-09-10T21:47:49.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.927+0000 7fc328c00640 1 -- 172.21.15.7:0/2622376033 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v87) ==== 80+0+0 (secure 0 0 0) 0x7fc314020020 con 0x7fc32c0cddd0 2024-09-10T21:47:49.928 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-10T21:47:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.929+0000 7fc331600640 1 -- 172.21.15.7:0/2622376033 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc308077f70 msgr2=0x7fc30807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.929+0000 7fc331600640 1 --2- 172.21.15.7:0/2622376033 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc308077f70 0x7fc30807a430 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009ac0 tx=0x7fc31c005f90 comp rx=0 tx=0).stop 2024-09-10T21:47:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.929+0000 7fc331600640 1 -- 172.21.15.7:0/2622376033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc32c0cddd0 msgr2=0x7fc32c1f5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:49.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.929+0000 7fc331600640 1 --2- 172.21.15.7:0/2622376033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc32c0cddd0 0x7fc32c1f5540 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fc31400c6e0 tx=0x7fc31400cba0 comp rx=0 tx=0).stop 2024-09-10T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.929+0000 7fc331600640 1 -- 172.21.15.7:0/2622376033 shutdown_connections 2024-09-10T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.929+0000 7fc331600640 1 --2- 172.21.15.7:0/2622376033 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc308077f70 0x7fc30807a430 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.929+0000 7fc331600640 1 --2- 172.21.15.7:0/2622376033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc32c15f4d0 0x7fc32c1f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.929+0000 7fc331600640 1 --2- 172.21.15.7:0/2622376033 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc32c0cddd0 0x7fc32c1f5540 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.929+0000 7fc331600640 1 --2- 172.21.15.7:0/2622376033 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc32c0702b0 0x7fc32c14f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.929+0000 7fc331600640 1 -- 172.21.15.7:0/2622376033 >> 172.21.15.7:0/2622376033 conn(0x7fc32c0b16d0 msgr2=0x7fc32c0cfa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.929+0000 7fc331600640 1 -- 172.21.15.7:0/2622376033 shutdown_connections 2024-09-10T21:47:49.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:49.929+0000 7fc331600640 1 -- 172.21.15.7:0/2622376033 wait complete. 2024-09-10T21:47:49.946 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:47:49.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:49.947 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:47:50.053 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:47:50.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:50.053 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:47:50.093 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:47:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.294+0000 7fa441800640 1 -- 172.21.15.7:0/2468471590 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa43c161f80 msgr2=0x7fa43c0b3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.294+0000 7fa441800640 1 --2- 172.21.15.7:0/2468471590 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa43c161f80 0x7fa43c0b3e70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa42c005fd0 tx=0x7fa42c02a5d0 comp rx=0 tx=0).stop 2024-09-10T21:47:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.294+0000 7fa441800640 1 -- 172.21.15.7:0/2468471590 shutdown_connections 2024-09-10T21:47:50.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.294+0000 7fa441800640 1 --2- 172.21.15.7:0/2468471590 >> v2:172.21.15.133:3300/0 conn(0x7fa43c0b43b0 0x7fa43c0b0c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:50.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.294+0000 7fa441800640 1 --2- 172.21.15.7:0/2468471590 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa43c161f80 0x7fa43c0b3e70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:50.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.294+0000 7fa441800640 1 -- 172.21.15.7:0/2468471590 >> 172.21.15.7:0/2468471590 conn(0x7fa43c09d260 msgr2=0x7fa43c0d3fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:50.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.294+0000 7fa441800640 1 -- 172.21.15.7:0/2468471590 shutdown_connections 2024-09-10T21:47:50.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.294+0000 7fa441800640 1 -- 172.21.15.7:0/2468471590 wait complete. 2024-09-10T21:47:50.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.296+0000 7fa441800640 1 Processor -- start 2024-09-10T21:47:50.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.296+0000 7fa441800640 1 -- start start 2024-09-10T21:47:50.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.297+0000 7fa441800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa43c0b43b0 0x7fa43c1f5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:50.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.297+0000 7fa441800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa43c161f80 0x7fa43c1f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:50.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.297+0000 7fa441800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa43c162940 0x7fa43c1f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:50.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.297+0000 7fa441800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c0b12f0 con 0x7fa43c0b43b0 2024-09-10T21:47:50.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.297+0000 7fa441800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa43c0b1170 con 0x7fa43c162940 2024-09-10T21:47:50.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.297+0000 7fa441800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa43c0b1470 con 0x7fa43c161f80 2024-09-10T21:47:50.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.297+0000 7fa43aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa43c0b43b0 0x7fa43c1f5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:50.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.297+0000 7fa43aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa43c0b43b0 0x7fa43c1f5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57862/0 (socket says 172.21.15.7:57862) 2024-09-10T21:47:50.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.297+0000 7fa43aa00640 1 -- 172.21.15.7:0/1625980404 learned_addr learned my addr 172.21.15.7:0/1625980404 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:50.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.297+0000 7fa43b400640 1 --2- 172.21.15.7:0/1625980404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa43c162940 0x7fa43c1f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:50.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.297+0000 7fa432000640 1 --2- 172.21.15.7:0/1625980404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa43c161f80 0x7fa43c1f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:50.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.298+0000 7fa43aa00640 1 -- 172.21.15.7:0/1625980404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa43c161f80 msgr2=0x7fa43c1f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:50.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.298+0000 7fa43aa00640 1 --2- 172.21.15.7:0/1625980404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa43c161f80 0x7fa43c1f4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:50.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.298+0000 7fa43aa00640 1 -- 172.21.15.7:0/1625980404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa43c162940 msgr2=0x7fa43c1f7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:50.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.298+0000 7fa43aa00640 1 --2- 172.21.15.7:0/1625980404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa43c162940 0x7fa43c1f7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:50.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.298+0000 7fa43aa00640 1 -- 172.21.15.7:0/1625980404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa43c0b81f0 con 0x7fa43c0b43b0 2024-09-10T21:47:50.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.298+0000 7fa43aa00640 1 --2- 172.21.15.7:0/1625980404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa43c0b43b0 0x7fa43c1f5ee0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0099f0 tx=0x7fa42c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:50.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.298+0000 7fa433400640 1 -- 172.21.15.7:0/1625980404 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c03f070 con 0x7fa43c0b43b0 2024-09-10T21:47:50.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.298+0000 7fa433400640 1 -- 172.21.15.7:0/1625980404 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa42c005600 con 0x7fa43c0b43b0 2024-09-10T21:47:50.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.298+0000 7fa441800640 1 -- 172.21.15.7:0/1625980404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa43c1f5370 con 0x7fa43c0b43b0 2024-09-10T21:47:50.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.298+0000 7fa433400640 1 -- 172.21.15.7:0/1625980404 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa42c035030 con 0x7fa43c0b43b0 2024-09-10T21:47:50.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.298+0000 7fa441800640 1 -- 172.21.15.7:0/1625980404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa43c1f5930 con 0x7fa43c0b43b0 2024-09-10T21:47:50.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.299+0000 7fa441800640 1 -- 172.21.15.7:0/1625980404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa43c0b0c30 con 0x7fa43c0b43b0 2024-09-10T21:47:50.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.300+0000 7fa433400640 1 -- 172.21.15.7:0/1625980404 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa42c004a00 con 0x7fa43c0b43b0 2024-09-10T21:47:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.301+0000 7fa433400640 1 --2- 172.21.15.7:0/1625980404 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa418077f70 0x7fa41807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:50.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.301+0000 7fa433400640 1 -- 172.21.15.7:0/1625980404 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5738+0+0 (secure 0 0 0) 0x7fa42c0bac10 con 0x7fa43c0b43b0 2024-09-10T21:47:50.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.302+0000 7fa432000640 1 --2- 172.21.15.7:0/1625980404 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa418077f70 0x7fa41807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:50.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.303+0000 7fa432000640 1 --2- 172.21.15.7:0/1625980404 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa418077f70 0x7fa41807a430 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0d13d0 tx=0x7fa424009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:50.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.305+0000 7fa433400640 1 -- 172.21.15.7:0/1625980404 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fa42c083000 con 0x7fa43c0b43b0 2024-09-10T21:47:50.534 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.533+0000 7fa441800640 1 -- 172.21.15.7:0/1625980404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa43c08d6e0 con 0x7fa43c0b43b0 2024-09-10T21:47:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.534+0000 7fa433400640 1 -- 172.21.15.7:0/1625980404 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+5233 (secure 0 0 0) 0x7fa42c087a30 con 0x7fa43c0b43b0 2024-09-10T21:47:50.535 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:50.535 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":63,"btime":"2024-09-10T21:47:44:249448+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/978763848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":978763848},{"type":"v1","addr":"172.21.15.7:6835","nonce":978763848}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":235775,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/760980985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":760980985},{"type":"v1","addr":"172.21.15.133:6836","nonce":760980985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57},{"gid":235778,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3312604545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3312604545},{"type":"v1","addr":"172.21.15.133:6837","nonce":3312604545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58}],"filesystems":[{"mdsmap":{"epoch":63,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:47:40.664091+0000","modified":"2024-09-10T21:47:43.681126+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235781},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235781":{"gid":235781,"name":"a","rank":0,"incarnation":59,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6833/1499133391","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":1499133391},{"type":"v1","addr":"172.21.15.7:6833","nonce":1499133391}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235781,"qdb_cluster":[235781]},"id":6}]} 2024-09-10T21:47:50.535 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 63 2024-09-10T21:47:50.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.536+0000 7fa441800640 1 -- 172.21.15.7:0/1625980404 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa418077f70 msgr2=0x7fa41807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:50.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.536+0000 7fa441800640 1 --2- 172.21.15.7:0/1625980404 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa418077f70 0x7fa41807a430 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fa43c0d13d0 tx=0x7fa424009040 comp rx=0 tx=0).stop 2024-09-10T21:47:50.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.536+0000 7fa441800640 1 -- 172.21.15.7:0/1625980404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa43c0b43b0 msgr2=0x7fa43c1f5ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:50.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.536+0000 7fa441800640 1 --2- 172.21.15.7:0/1625980404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa43c0b43b0 0x7fa43c1f5ee0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fa42c0099f0 tx=0x7fa42c003820 comp rx=0 tx=0).stop 2024-09-10T21:47:50.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.536+0000 7fa441800640 1 -- 172.21.15.7:0/1625980404 shutdown_connections 2024-09-10T21:47:50.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.536+0000 7fa441800640 1 --2- 172.21.15.7:0/1625980404 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa418077f70 0x7fa41807a430 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:50.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.536+0000 7fa441800640 1 --2- 172.21.15.7:0/1625980404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa43c162940 0x7fa43c1f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:50.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.536+0000 7fa441800640 1 --2- 172.21.15.7:0/1625980404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa43c161f80 0x7fa43c1f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:50.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.536+0000 7fa441800640 1 --2- 172.21.15.7:0/1625980404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa43c0b43b0 0x7fa43c1f5ee0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:50.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.536+0000 7fa441800640 1 -- 172.21.15.7:0/1625980404 >> 172.21.15.7:0/1625980404 conn(0x7fa43c09d260 msgr2=0x7fa43c0d3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:50.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.537+0000 7fa441800640 1 -- 172.21.15.7:0/1625980404 shutdown_connections 2024-09-10T21:47:50.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.537+0000 7fa441800640 1 -- 172.21.15.7:0/1625980404 wait complete. 2024-09-10T21:47:50.553 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-09-10T21:47:50.737 INFO:teuthology.orchestra.run.smithi007.stdout:{"message":"","return_code":0} 2024-09-10T21:47:50.738 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-10T21:47:50.738 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:47:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.900+0000 7f0d56000640 1 -- 172.21.15.7:0/3644391829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d38003a50 msgr2=0x7f0d38003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.900+0000 7f0d56000640 1 --2- 172.21.15.7:0/3644391829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d38003a50 0x7f0d38003e70 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f0d4400c470 tx=0x7f0d38005830 comp rx=0 tx=0).stop 2024-09-10T21:47:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.900+0000 7f0d55600640 1 -- 172.21.15.7:0/3644391829 reap_dead start 2024-09-10T21:47:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.900+0000 7f0d56000640 1 -- 172.21.15.7:0/3644391829 shutdown_connections 2024-09-10T21:47:50.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.900+0000 7f0d56000640 1 -- 172.21.15.7:0/3644391829 >> 172.21.15.7:0/3644391829 conn(0x7f0d500b16d0 msgr2=0x7f0d500b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.900+0000 7f0d56000640 1 -- 172.21.15.7:0/3644391829 shutdown_connections 2024-09-10T21:47:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.900+0000 7f0d56000640 1 -- 172.21.15.7:0/3644391829 wait complete. 2024-09-10T21:47:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.902+0000 7f0d56000640 1 Processor -- start 2024-09-10T21:47:50.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.902+0000 7f0d56000640 1 -- start start 2024-09-10T21:47:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.903+0000 7f0d56000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d38003a50 0x7f0d50153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.903+0000 7f0d56000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d50158160 0x7f0d50141450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.903+0000 7f0d56000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d501585e0 0x7f0d50141990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.903+0000 7f0d56000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d500adfc0 con 0x7f0d50158160 2024-09-10T21:47:50.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.903+0000 7f0d56000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d500ade40 con 0x7f0d501585e0 2024-09-10T21:47:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.903+0000 7f0d56000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f0d500ae140 con 0x7f0d38003a50 2024-09-10T21:47:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.903+0000 7f0d4fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d50158160 0x7f0d50141450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.903+0000 7f0d4fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d50158160 0x7f0d50141450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57882/0 (socket says 172.21.15.7:57882) 2024-09-10T21:47:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.903+0000 7f0d4fe00640 1 -- 172.21.15.7:0/3961748480 learned_addr learned my addr 172.21.15.7:0/3961748480 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.903+0000 7f0d54c00640 1 --2- 172.21.15.7:0/3961748480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d38003a50 0x7f0d50153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.903+0000 7f0d55600640 1 --2- 172.21.15.7:0/3961748480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d501585e0 0x7f0d50141990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.903+0000 7f0d4fe00640 1 -- 172.21.15.7:0/3961748480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d38003a50 msgr2=0x7f0d50153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.903+0000 7f0d4fe00640 1 --2- 172.21.15.7:0/3961748480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d38003a50 0x7f0d50153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.903+0000 7f0d4fe00640 1 -- 172.21.15.7:0/3961748480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d501585e0 msgr2=0x7f0d50141990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.903+0000 7f0d4fe00640 1 --2- 172.21.15.7:0/3961748480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d501585e0 0x7f0d50141990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:50.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.903+0000 7f0d4fe00640 1 -- 172.21.15.7:0/3961748480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d501f5460 con 0x7f0d50158160 2024-09-10T21:47:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.904+0000 7f0d4fe00640 1 --2- 172.21.15.7:0/3961748480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d50158160 0x7f0d50141450 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c011ab0 tx=0x7f0d3c011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:50.906 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.904+0000 7f0d4cc00640 1 -- 172.21.15.7:0/3961748480 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c01e070 con 0x7f0d50158160 2024-09-10T21:47:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.904+0000 7f0d4cc00640 1 -- 172.21.15.7:0/3961748480 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d3c00fbb0 con 0x7f0d50158160 2024-09-10T21:47:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.904+0000 7f0d56000640 1 -- 172.21.15.7:0/3961748480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d50072690 con 0x7f0d50158160 2024-09-10T21:47:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.904+0000 7f0d4cc00640 1 -- 172.21.15.7:0/3961748480 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c019410 con 0x7f0d50158160 2024-09-10T21:47:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.904+0000 7f0d56000640 1 -- 172.21.15.7:0/3961748480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d50072c50 con 0x7f0d50158160 2024-09-10T21:47:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.905+0000 7f0d56000640 1 -- 172.21.15.7:0/3961748480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d5015eb50 con 0x7f0d50158160 2024-09-10T21:47:50.907 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.906+0000 7f0d4cc00640 1 -- 172.21.15.7:0/3961748480 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0d3c0195b0 con 0x7f0d50158160 2024-09-10T21:47:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.907+0000 7f0d4cc00640 1 --2- 172.21.15.7:0/3961748480 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0d30077f70 0x7f0d3007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.907+0000 7f0d4cc00640 1 -- 172.21.15.7:0/3961748480 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 5738+0+0 (secure 0 0 0) 0x7f0d3c09a830 con 0x7f0d50158160 2024-09-10T21:47:50.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.908+0000 7f0d54c00640 1 --2- 172.21.15.7:0/3961748480 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0d30077f70 0x7f0d3007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:50.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.908+0000 7f0d54c00640 1 --2- 172.21.15.7:0/3961748480 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0d30077f70 0x7f0d3007a430 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0d44033000 tx=0x7f0d500b3030 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:50.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:50.911+0000 7f0d4cc00640 1 -- 172.21.15.7:0/3961748480 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0d3c062c20 con 0x7f0d50158160 2024-09-10T21:47:51.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.142+0000 7f0d56000640 1 -- 172.21.15.7:0/3961748480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f0d5015be00 con 0x7f0d50158160 2024-09-10T21:47:51.143 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-10T21:47:51.143+0000 7feb3fa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:47:51.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.145+0000 7f0d4cc00640 1 -- 172.21.15.7:0/3961748480 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v64) ==== 161+0+0 (secure 0 0 0) 0x7f0d3c067650 con 0x7f0d50158160 2024-09-10T21:47:51.146 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:47:51.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.148+0000 7f0d56000640 1 -- 172.21.15.7:0/3961748480 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0d30077f70 msgr2=0x7f0d3007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.148+0000 7f0d56000640 1 --2- 172.21.15.7:0/3961748480 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0d30077f70 0x7f0d3007a430 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f0d44033000 tx=0x7f0d500b3030 comp rx=0 tx=0).stop 2024-09-10T21:47:51.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.148+0000 7f0d56000640 1 -- 172.21.15.7:0/3961748480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d50158160 msgr2=0x7f0d50141450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.148+0000 7f0d56000640 1 --2- 172.21.15.7:0/3961748480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d50158160 0x7f0d50141450 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c011ab0 tx=0x7f0d3c011f70 comp rx=0 tx=0).stop 2024-09-10T21:47:51.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.148+0000 7f0d56000640 1 -- 172.21.15.7:0/3961748480 shutdown_connections 2024-09-10T21:47:51.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.148+0000 7f0d56000640 1 --2- 172.21.15.7:0/3961748480 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0d30077f70 0x7f0d3007a430 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.148+0000 7f0d56000640 1 --2- 172.21.15.7:0/3961748480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d501585e0 0x7f0d50141990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.148+0000 7f0d56000640 1 --2- 172.21.15.7:0/3961748480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d50158160 0x7f0d50141450 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.148+0000 7f0d56000640 1 --2- 172.21.15.7:0/3961748480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d38003a50 0x7f0d50153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.148+0000 7f0d56000640 1 -- 172.21.15.7:0/3961748480 >> 172.21.15.7:0/3961748480 conn(0x7f0d500b16d0 msgr2=0x7f0d5006e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:51.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.148+0000 7f0d56000640 1 -- 172.21.15.7:0/3961748480 shutdown_connections 2024-09-10T21:47:51.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.148+0000 7f0d56000640 1 -- 172.21.15.7:0/3961748480 wait complete. 2024-09-10T21:47:51.161 INFO:tasks.ceph.mds.a.smithi007.stderr: -5530> 2024-09-10T21:47:47.196+0000 7f7b02400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:47:51.165 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rm 10000000000.00000000 2024-09-10T21:47:51.211 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-09-10T21:47:51.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.243+0000 7f72b065bf40 1 Processor -- start 2024-09-10T21:47:51.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.243+0000 7f72b065bf40 1 -- start start 2024-09-10T21:47:51.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.244+0000 7f72b065bf40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55db8da05ae0 0x55db8db67880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.244+0000 7f72b065bf40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db8da52110 0x55db8db6a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.244+0000 7f72b065bf40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8dac2620 0x55db8db69260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.244+0000 7f72b065bf40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55db8da52520 con 0x55db8dac2620 2024-09-10T21:47:51.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.244+0000 7f72b065bf40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55db8d954c60 con 0x55db8da52110 2024-09-10T21:47:51.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.244+0000 7f72b065bf40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55db8dac2a00 con 0x55db8da05ae0 2024-09-10T21:47:51.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.244+0000 7f72ae200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8dac2620 0x55db8db69260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.244+0000 7f72ae200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8dac2620 0x55db8db69260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57898/0 (socket says 172.21.15.7:57898) 2024-09-10T21:47:51.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.244+0000 7f72ae200640 1 -- 172.21.15.7:0/1630192885 learned_addr learned my addr 172.21.15.7:0/1630192885 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:51.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.244+0000 7f72ace00640 1 --2- 172.21.15.7:0/1630192885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db8da52110 0x55db8db6a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.244+0000 7f72ad800640 1 --2- 172.21.15.7:0/1630192885 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55db8da05ae0 0x55db8db67880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.244+0000 7f72ae200640 1 -- 172.21.15.7:0/1630192885 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55db8da05ae0 msgr2=0x55db8db67880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:51.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.244+0000 7f72ae200640 1 --2- 172.21.15.7:0/1630192885 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55db8da05ae0 0x55db8db67880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.244+0000 7f72ae200640 1 -- 172.21.15.7:0/1630192885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db8da52110 msgr2=0x55db8db6a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:51.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.244+0000 7f72ae200640 1 --2- 172.21.15.7:0/1630192885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db8da52110 0x55db8db6a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.244+0000 7f72ae200640 1 -- 172.21.15.7:0/1630192885 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db8d92cf10 con 0x55db8dac2620 2024-09-10T21:47:51.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.244+0000 7f72ace00640 1 --2- 172.21.15.7:0/1630192885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db8da52110 0x55db8db6a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:47:51.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.244+0000 7f72ad800640 1 --2- 172.21.15.7:0/1630192885 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55db8da05ae0 0x55db8db67880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:47:51.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.245+0000 7f72ae200640 1 --2- 172.21.15.7:0/1630192885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8dac2620 0x55db8db69260 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f72a800b670 tx=0x7f72a800bb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff55a59722823c57 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:51.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.245+0000 7f72a7400640 1 -- 172.21.15.7:0/1630192885 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a8009a30 con 0x55db8dac2620 2024-09-10T21:47:51.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.245+0000 7f72a7400640 1 -- 172.21.15.7:0/1630192885 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72a8009cc0 con 0x55db8dac2620 2024-09-10T21:47:51.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.245+0000 7f72a7400640 1 -- 172.21.15.7:0/1630192885 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72a800e6b0 con 0x55db8dac2620 2024-09-10T21:47:51.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.245+0000 7f72b065bf40 1 -- 172.21.15.7:0/1630192885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8dac2620 msgr2=0x55db8db69260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.245+0000 7f72b065bf40 1 --2- 172.21.15.7:0/1630192885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8dac2620 0x55db8db69260 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f72a800b670 tx=0x7f72a800bb30 comp rx=0 tx=0).stop 2024-09-10T21:47:51.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.245+0000 7f72b065bf40 1 -- 172.21.15.7:0/1630192885 shutdown_connections 2024-09-10T21:47:51.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.245+0000 7f72b065bf40 1 --2- 172.21.15.7:0/1630192885 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db8da52110 0x55db8db6a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.245+0000 7f72b065bf40 1 --2- 172.21.15.7:0/1630192885 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8dac2620 0x55db8db69260 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.245+0000 7f72b065bf40 1 --2- 172.21.15.7:0/1630192885 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55db8da05ae0 0x55db8db67880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.245+0000 7f72b065bf40 1 -- 172.21.15.7:0/1630192885 >> 172.21.15.7:0/1630192885 conn(0x55db8da34de0 msgr2=0x55db8da38200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:51.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.246+0000 7f72b065bf40 1 -- 172.21.15.7:0/1630192885 shutdown_connections 2024-09-10T21:47:51.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.246+0000 7f72b065bf40 1 -- 172.21.15.7:0/1630192885 wait complete. 2024-09-10T21:47:51.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.246+0000 7f72b065bf40 1 Processor -- start 2024-09-10T21:47:51.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.246+0000 7f72b065bf40 1 -- start start 2024-09-10T21:47:51.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.247+0000 7f72b065bf40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55db8da05ae0 0x55db8dac0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.247+0000 7f72b065bf40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8da52110 0x55db8dad4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.247+0000 7f72b065bf40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db8dac2620 0x55db8dad6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.247+0000 7f72b065bf40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55db8da37e50 con 0x55db8da52110 2024-09-10T21:47:51.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.247+0000 7f72b065bf40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55db8db67dc0 con 0x55db8dac2620 2024-09-10T21:47:51.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.247+0000 7f72b065bf40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55db8da378b0 con 0x55db8da05ae0 2024-09-10T21:47:51.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.248+0000 7f72ace00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8da52110 0x55db8dad4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.248+0000 7f72ace00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8da52110 0x55db8dad4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57900/0 (socket says 172.21.15.7:57900) 2024-09-10T21:47:51.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.248+0000 7f72ace00640 1 -- 172.21.15.7:0/3959229249 learned_addr learned my addr 172.21.15.7:0/3959229249 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:51.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.248+0000 7f72ad800640 1 --2- 172.21.15.7:0/3959229249 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55db8da05ae0 0x55db8dac0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.248+0000 7f72ae200640 1 --2- 172.21.15.7:0/3959229249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db8dac2620 0x55db8dad6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.248+0000 7f72ace00640 1 -- 172.21.15.7:0/3959229249 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55db8da05ae0 msgr2=0x55db8dac0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.248+0000 7f72ace00640 1 --2- 172.21.15.7:0/3959229249 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55db8da05ae0 0x55db8dac0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.248+0000 7f72ace00640 1 -- 172.21.15.7:0/3959229249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db8dac2620 msgr2=0x55db8dad6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.248+0000 7f72ace00640 1 --2- 172.21.15.7:0/3959229249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db8dac2620 0x55db8dad6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.248+0000 7f72ace00640 1 -- 172.21.15.7:0/3959229249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55db8dad3190 con 0x55db8da52110 2024-09-10T21:47:51.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.248+0000 7f72ae200640 1 --2- 172.21.15.7:0/3959229249 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55db8dac2620 0x55db8dad6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:51.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.248+0000 7f72ace00640 1 --2- 172.21.15.7:0/3959229249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8da52110 0x55db8dad4f10 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f729c025e50 tx=0x7f729c04d7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:51.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.249+0000 7f72a6000640 1 -- 172.21.15.7:0/3959229249 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c066070 con 0x55db8da52110 2024-09-10T21:47:51.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.249+0000 7f72a6000640 1 -- 172.21.15.7:0/3959229249 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f729c056760 con 0x55db8da52110 2024-09-10T21:47:51.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.249+0000 7f72a6000640 1 -- 172.21.15.7:0/3959229249 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c056b90 con 0x55db8da52110 2024-09-10T21:47:51.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.249+0000 7f72b065bf40 1 -- 172.21.15.7:0/3959229249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55db8dad3470 con 0x55db8da52110 2024-09-10T21:47:51.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.249+0000 7f72b065bf40 1 -- 172.21.15.7:0/3959229249 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55db8dad4710 con 0x55db8da52110 2024-09-10T21:47:51.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.251+0000 7f72a6000640 1 -- 172.21.15.7:0/3959229249 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f729c061400 con 0x55db8da52110 2024-09-10T21:47:51.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.252+0000 7f72a6000640 1 --2- 172.21.15.7:0/3959229249 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7290076950 0x7f7290078e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.252+0000 7f72a6000640 1 -- 172.21.15.7:0/3959229249 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f729c0dc580 con 0x55db8da52110 2024-09-10T21:47:51.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.252+0000 7f72b065bf40 1 --2- 172.21.15.7:0/3959229249 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x55db8dad5450 0x55db8db64c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.252+0000 7f72b065bf40 1 -- 172.21.15.7:0/3959229249 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:1 12.10 12:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_force+supports_pool_eio e88) -- 0x55db8db6a350 con 0x55db8dad5450 2024-09-10T21:47:51.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.253+0000 7f72ad800640 1 --2- 172.21.15.7:0/3959229249 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7290076950 0x7f7290078e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.253+0000 7f72ae200640 1 --2- 172.21.15.7:0/3959229249 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x55db8dad5450 0x55db8db64c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.253+0000 7f72ae200640 1 --2- 172.21.15.7:0/3959229249 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x55db8dad5450 0x55db8db64c20 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:51.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.253+0000 7f72ad800640 1 --2- 172.21.15.7:0/3959229249 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7290076950 0x7f7290078e10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f72a0004fd0 tx=0x7f72a0004810 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:51.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.257+0000 7f72ae200640 1 -- 172.21.15.7:0/3959229249 <== osd.0 v2:172.21.15.7:6808/651402908 1 ==== osd_op_reply(1 10000000000.00000000 [delete] v88'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f72a8009cb0 con 0x55db8dad5450 2024-09-10T21:47:51.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.257+0000 7f72b065bf40 1 -- 172.21.15.7:0/3959229249 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x55db8dad5450 msgr2=0x55db8db64c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.257+0000 7f72b065bf40 1 --2- 172.21.15.7:0/3959229249 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x55db8dad5450 0x55db8db64c20 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.257+0000 7f72b065bf40 1 -- 172.21.15.7:0/3959229249 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7290076950 msgr2=0x7f7290078e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.257+0000 7f72b065bf40 1 --2- 172.21.15.7:0/3959229249 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7290076950 0x7f7290078e10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f72a0004fd0 tx=0x7f72a0004810 comp rx=0 tx=0).stop 2024-09-10T21:47:51.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.257+0000 7f72b065bf40 1 -- 172.21.15.7:0/3959229249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8da52110 msgr2=0x55db8dad4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.257+0000 7f72b065bf40 1 --2- 172.21.15.7:0/3959229249 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55db8da52110 0x55db8dad4f10 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f729c025e50 tx=0x7f729c04d7f0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.257+0000 7f72ae200640 1 -- 172.21.15.7:0/3959229249 reap_dead start 2024-09-10T21:47:51.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.257+0000 7f72b065bf40 1 -- 172.21.15.7:0/3959229249 shutdown_connections 2024-09-10T21:47:51.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.257+0000 7f72b065bf40 1 -- 172.21.15.7:0/3959229249 >> 172.21.15.7:0/3959229249 conn(0x55db8da34de0 msgr2=0x55db8da34260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:51.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.257+0000 7f72b065bf40 1 -- 172.21.15.7:0/3959229249 shutdown_connections 2024-09-10T21:47:51.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.257+0000 7f72b065bf40 1 -- 172.21.15.7:0/3959229249 wait complete. 2024-09-10T21:47:51.263 DEBUG:teuthology.orchestra.run.smithi007:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --yes-i-really-really-mean-it 2024-09-10T21:47:51.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.337+0000 7f9b70af2280 1 -- start start 2024-09-10T21:47:51.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.337+0000 7f9b70af2280 1 Processor -- start 2024-09-10T21:47:51.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.338+0000 7f9b70af2280 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f4f2ac00 0x55c5f5cfe000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.338+0000 7f9b70af2280 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f4f2b800 0x55c5f5cfe580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.338+0000 7f9b70af2280 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f4f2bc00 0x55c5f5cfeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.338+0000 7f9b70af2280 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c5f5cd4180 con 0x55c5f4f2bc00 2024-09-10T21:47:51.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.338+0000 7f9b70af2280 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c5f4f8d800 con 0x55c5f4f2b800 2024-09-10T21:47:51.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.338+0000 7f9b70af2280 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55c5f4f8d380 con 0x55c5f4f2ac00 2024-09-10T21:47:51.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.338+0000 7f9b6f600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f4f2bc00 0x55c5f5cfeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.338+0000 7f9b6f600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f4f2bc00 0x55c5f5cfeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57918/0 (socket says 172.21.15.7:57918) 2024-09-10T21:47:51.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.338+0000 7f9b6f600640 1 -- 172.21.15.7:0/85157946 learned_addr learned my addr 172.21.15.7:0/85157946 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:51.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.338+0000 7f9b6e200640 1 --2- 172.21.15.7:0/85157946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f4f2b800 0x55c5f5cfe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.338+0000 7f9b6ec00640 1 --2- 172.21.15.7:0/85157946 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f4f2ac00 0x55c5f5cfe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.339+0000 7f9b6f600640 1 -- 172.21.15.7:0/85157946 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f4f2ac00 msgr2=0x55c5f5cfe000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:51.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.339+0000 7f9b6f600640 1 --2- 172.21.15.7:0/85157946 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f4f2ac00 0x55c5f5cfe000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.339+0000 7f9b6f600640 1 -- 172.21.15.7:0/85157946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f4f2b800 msgr2=0x55c5f5cfe580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:51.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.339+0000 7f9b6f600640 1 --2- 172.21.15.7:0/85157946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f4f2b800 0x55c5f5cfe580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.339+0000 7f9b6f600640 1 -- 172.21.15.7:0/85157946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5f4f62780 con 0x55c5f4f2bc00 2024-09-10T21:47:51.341 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.339+0000 7f9b6e200640 1 --2- 172.21.15.7:0/85157946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f4f2b800 0x55c5f5cfe580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:51.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.339+0000 7f9b6f600640 1 --2- 172.21.15.7:0/85157946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f4f2bc00 0x55c5f5cfeb00 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x55c5f5d75c50 tx=0x55c5f5cc8a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7950ae6d5ad2bde server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:51.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.339+0000 7f9b6ba00640 1 -- 172.21.15.7:0/85157946 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c5f4f50ea0 con 0x55c5f4f2bc00 2024-09-10T21:47:51.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.339+0000 7f9b6ba00640 1 -- 172.21.15.7:0/85157946 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55c5f4f56540 con 0x55c5f4f2bc00 2024-09-10T21:47:51.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.339+0000 7f9b70af2280 1 -- 172.21.15.7:0/85157946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5f4f634a0 con 0x55c5f4f2bc00 2024-09-10T21:47:51.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.339+0000 7f9b70af2280 1 -- 172.21.15.7:0/85157946 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55c5f4f62000 con 0x55c5f4f2bc00 2024-09-10T21:47:51.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.339+0000 7f9b70af2280 4 waiting for MDS map... 2024-09-10T21:47:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.339+0000 7f9b6ba00640 1 -- 172.21.15.7:0/85157946 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c5f4f51380 con 0x55c5f4f2bc00 2024-09-10T21:47:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.340+0000 7f9b6ba00640 1 -- 172.21.15.7:0/85157946 <== mon.0 v2:172.21.15.7:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x55c5f5e24600 con 0x55c5f4f2bc00 2024-09-10T21:47:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.340+0000 7f9b6ba00640 1 -- 172.21.15.7:0/85157946 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 64) ==== 2893+0+0 (secure 0 0 0) 0x55c5f4f85180 con 0x55c5f4f2bc00 2024-09-10T21:47:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.340+0000 7f9b70af2280 4 Got MDS map 64 2024-09-10T21:47:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.340+0000 7f9b70af2280 10 main: JournalTool::main 2024-09-10T21:47:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.340+0000 7f9b70af2280 4 main: JournalTool: connecting to RADOS... 2024-09-10T21:47:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.341+0000 7f9b70af2280 1 Processor -- start 2024-09-10T21:47:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.341+0000 7f9b70af2280 1 -- start start 2024-09-10T21:47:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.341+0000 7f9b70af2280 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e29400 0x55c5f5cff080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.341+0000 7f9b70af2280 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f5e29800 0x55c5f5cff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.341+0000 7f9b70af2280 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f5e29c00 0x55c5f5cffb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.341+0000 7f9b70af2280 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c5f4f8d980 con 0x55c5f5e29400 2024-09-10T21:47:51.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.341+0000 7f9b70af2280 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c5f5cd4d80 con 0x55c5f5e29800 2024-09-10T21:47:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.341+0000 7f9b70af2280 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55c5f5cd4f00 con 0x55c5f5e29c00 2024-09-10T21:47:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.342+0000 7f9b6e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e29400 0x55c5f5cff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.342+0000 7f9b6e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e29400 0x55c5f5cff080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57920/0 (socket says 172.21.15.7:57920) 2024-09-10T21:47:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.342+0000 7f9b6e200640 1 -- 172.21.15.7:0/1804185448 learned_addr learned my addr 172.21.15.7:0/1804185448 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.342+0000 7f9b6ec00640 1 --2- 172.21.15.7:0/1804185448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f5e29800 0x55c5f5cff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.342+0000 7f9b6e200640 1 --2- 172.21.15.7:0/1804185448 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f5e29c00 0x55c5f5cffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.342+0000 7f9b6e200640 1 -- 172.21.15.7:0/1804185448 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f5e29c00 msgr2=0x55c5f5cffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.342+0000 7f9b6e200640 1 --2- 172.21.15.7:0/1804185448 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f5e29c00 0x55c5f5cffb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.342+0000 7f9b6e200640 1 -- 172.21.15.7:0/1804185448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f5e29800 msgr2=0x55c5f5cff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.342+0000 7f9b6e200640 1 --2- 172.21.15.7:0/1804185448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f5e29800 0x55c5f5cff600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.342+0000 7f9b6e200640 1 -- 172.21.15.7:0/1804185448 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5f4f623c0 con 0x55c5f5e29400 2024-09-10T21:47:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.342+0000 7f9b6ec00640 1 --2- 172.21.15.7:0/1804185448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f5e29800 0x55c5f5cff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:51.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.342+0000 7f9b6e200640 1 --2- 172.21.15.7:0/1804185448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e29400 0x55c5f5cff080 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x55c5f5e3d080 tx=0x55c5f5e42c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb09c50a90c1b94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.343+0000 7f9b69200640 1 -- 172.21.15.7:0/1804185448 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c5f4f50ea0 con 0x55c5f5e29400 2024-09-10T21:47:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.343+0000 7f9b69200640 1 -- 172.21.15.7:0/1804185448 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55c5f4f57180 con 0x55c5f5e29400 2024-09-10T21:47:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.343+0000 7f9b69200640 1 -- 172.21.15.7:0/1804185448 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c5f4f51380 con 0x55c5f5e29400 2024-09-10T21:47:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.343+0000 7f9b70af2280 1 -- 172.21.15.7:0/1804185448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e29400 msgr2=0x55c5f5cff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.343+0000 7f9b70af2280 1 --2- 172.21.15.7:0/1804185448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e29400 0x55c5f5cff080 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x55c5f5e3d080 tx=0x55c5f5e42c60 comp rx=0 tx=0).stop 2024-09-10T21:47:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.343+0000 7f9b70af2280 1 -- 172.21.15.7:0/1804185448 shutdown_connections 2024-09-10T21:47:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.343+0000 7f9b70af2280 1 --2- 172.21.15.7:0/1804185448 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f5e29800 0x55c5f5cff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.343+0000 7f9b70af2280 1 --2- 172.21.15.7:0/1804185448 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f5e29c00 0x55c5f5cffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.343+0000 7f9b70af2280 1 --2- 172.21.15.7:0/1804185448 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e29400 0x55c5f5cff080 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.343+0000 7f9b70af2280 1 -- 172.21.15.7:0/1804185448 >> 172.21.15.7:0/1804185448 conn(0x55c5f5e29000 msgr2=0x55c5f5015000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.343+0000 7f9b70af2280 1 -- 172.21.15.7:0/1804185448 shutdown_connections 2024-09-10T21:47:51.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.343+0000 7f9b70af2280 1 -- 172.21.15.7:0/1804185448 wait complete. 2024-09-10T21:47:51.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.344+0000 7f9b70af2280 1 Processor -- start 2024-09-10T21:47:51.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.344+0000 7f9b70af2280 1 -- start start 2024-09-10T21:47:51.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.345+0000 7f9b70af2280 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e29400 0x55c5f5cff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.345+0000 7f9b70af2280 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f5e29800 0x55c5f5cff600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.345+0000 7f9b70af2280 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f5e29c00 0x55c5f5cffb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.345+0000 7f9b70af2280 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c5f5cd5200 con 0x55c5f5e29400 2024-09-10T21:47:51.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.345+0000 7f9b70af2280 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c5f5cd5080 con 0x55c5f5e29800 2024-09-10T21:47:51.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.345+0000 7f9b70af2280 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55c5f5cd5380 con 0x55c5f5e29c00 2024-09-10T21:47:51.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.345+0000 7f9b6e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e29400 0x55c5f5cff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.345+0000 7f9b6e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e29400 0x55c5f5cff080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57924/0 (socket says 172.21.15.7:57924) 2024-09-10T21:47:51.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.345+0000 7f9b6e200640 1 -- 172.21.15.7:0/2629421341 learned_addr learned my addr 172.21.15.7:0/2629421341 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:51.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.345+0000 7f9b6ec00640 1 --2- 172.21.15.7:0/2629421341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f5e29800 0x55c5f5cff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.345+0000 7f9b6e200640 1 --2- 172.21.15.7:0/2629421341 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f5e29c00 0x55c5f5cffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.345+0000 7f9b6e200640 1 -- 172.21.15.7:0/2629421341 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f5e29c00 msgr2=0x55c5f5cffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.345+0000 7f9b6e200640 1 --2- 172.21.15.7:0/2629421341 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f5e29c00 0x55c5f5cffb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.345+0000 7f9b6e200640 1 -- 172.21.15.7:0/2629421341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f5e29800 msgr2=0x55c5f5cff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.345+0000 7f9b6e200640 1 --2- 172.21.15.7:0/2629421341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f5e29800 0x55c5f5cff600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.345+0000 7f9b6e200640 1 -- 172.21.15.7:0/2629421341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5f4f62780 con 0x55c5f5e29400 2024-09-10T21:47:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.346+0000 7f9b6e200640 1 --2- 172.21.15.7:0/2629421341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e29400 0x55c5f5cff080 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x55c5f5e3dd40 tx=0x55c5f5e42e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.346+0000 7f9b67400640 1 -- 172.21.15.7:0/2629421341 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c5f4f51040 con 0x55c5f5e29400 2024-09-10T21:47:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.346+0000 7f9b67400640 1 -- 172.21.15.7:0/2629421341 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55c5f4f56380 con 0x55c5f5e29400 2024-09-10T21:47:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.346+0000 7f9b67400640 1 -- 172.21.15.7:0/2629421341 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c5f4f50ea0 con 0x55c5f5e29400 2024-09-10T21:47:51.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.346+0000 7f9b70af2280 1 -- 172.21.15.7:0/2629421341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c5f4f634a0 con 0x55c5f5e29400 2024-09-10T21:47:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.346+0000 7f9b70af2280 1 -- 172.21.15.7:0/2629421341 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5f4f62960 con 0x55c5f5e29400 2024-09-10T21:47:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.346+0000 7f9b70af2280 4 main: JournalTool: resolving pool 12 2024-09-10T21:47:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.348+0000 7f9b67400640 1 -- 172.21.15.7:0/2629421341 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x55c5f4f85500 con 0x55c5f5e29400 2024-09-10T21:47:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.348+0000 7f9b67400640 1 --2- 172.21.15.7:0/2629421341 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x55c5f5e9ec00 0x55c5f5d00100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.349+0000 7f9b67400640 1 -- 172.21.15.7:0/2629421341 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x55c5f5e98a00 con 0x55c5f5e29400 2024-09-10T21:47:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.349+0000 7f9b70af2280 4 main: JournalTool: creating IoCtx.. 2024-09-10T21:47:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.349+0000 7f9b70af2280 4 main: Executing for rank 0 2024-09-10T21:47:51.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.349+0000 7f9b6ec00640 1 --2- 172.21.15.7:0/2629421341 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x55c5f5e9ec00 0x55c5f5d00100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.349+0000 7f9b70af2280 1 -- start start 2024-09-10T21:47:51.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.350+0000 7f9b70af2280 1 Processor -- start 2024-09-10T21:47:51.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.350+0000 7f9b6ec00640 1 --2- 172.21.15.7:0/2629421341 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x55c5f5e9ec00 0x55c5f5d00100 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55c5f5cd8150 tx=0x55c5f52ecc00 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:51.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.350+0000 7f9b70af2280 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f5e9f400 0x55c5f5d00680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.350+0000 7f9b70af2280 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f5e9f800 0x55c5f5d00c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.350+0000 7f9b70af2280 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e9fc00 0x55c5f5d01180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.350+0000 7f9b70af2280 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c5f5cd5800 con 0x55c5f5e9fc00 2024-09-10T21:47:51.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.350+0000 7f9b70af2280 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c5f5cd5980 con 0x55c5f5e9f800 2024-09-10T21:47:51.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.350+0000 7f9b70af2280 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55c5f5cd5b00 con 0x55c5f5e9f400 2024-09-10T21:47:51.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.351+0000 7f9b6e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e9fc00 0x55c5f5d01180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.351+0000 7f9b6e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e9fc00 0x55c5f5d01180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57932/0 (socket says 172.21.15.7:57932) 2024-09-10T21:47:51.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.351+0000 7f9b6e200640 1 -- 172.21.15.7:0/517850206 learned_addr learned my addr 172.21.15.7:0/517850206 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:51.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.351+0000 7f9b6f600640 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f5e9f400 0x55c5f5d00680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.351+0000 7f9b6ec00640 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f5e9f800 0x55c5f5d00c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.351+0000 7f9b6e200640 1 -- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f5e9f400 msgr2=0x55c5f5d00680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.351+0000 7f9b6e200640 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f5e9f400 0x55c5f5d00680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.351+0000 7f9b6e200640 1 -- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f5e9f800 msgr2=0x55c5f5d00c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.351+0000 7f9b6e200640 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f5e9f800 0x55c5f5d00c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.351+0000 7f9b6e200640 1 -- 172.21.15.7:0/517850206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5f4f63680 con 0x55c5f5e9fc00 2024-09-10T21:47:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.351+0000 7f9b6f600640 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f5e9f400 0x55c5f5d00680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:47:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.351+0000 7f9b6ec00640 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f5e9f800 0x55c5f5d00c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:47:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.352+0000 7f9b6e200640 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e9fc00 0x55c5f5d01180 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x55c5f5e95f50 tx=0x55c5f5e437a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3894024617f33791 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.352+0000 7f9b63800640 1 -- 172.21.15.7:0/517850206 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c5f526eb60 con 0x55c5f5e9fc00 2024-09-10T21:47:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.352+0000 7f9b63800640 1 -- 172.21.15.7:0/517850206 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55c5f4f57180 con 0x55c5f5e9fc00 2024-09-10T21:47:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.352+0000 7f9b63800640 1 -- 172.21.15.7:0/517850206 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55c5f526ed00 con 0x55c5f5e9fc00 2024-09-10T21:47:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.352+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5f4f634a0 con 0x55c5f5e9fc00 2024-09-10T21:47:51.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.352+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55c5f4f62960 con 0x55c5f5e9fc00 2024-09-10T21:47:51.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.352+0000 7f9b70af2280 4 waiting for MDS map... 2024-09-10T21:47:51.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.352+0000 7f9b63800640 1 -- 172.21.15.7:0/517850206 <== mon.0 v2:172.21.15.7:3300/0 4 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x55c5f534c600 con 0x55c5f5e9fc00 2024-09-10T21:47:51.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.352+0000 7f9b63800640 1 -- 172.21.15.7:0/517850206 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 64) ==== 2893+0+0 (secure 0 0 0) 0x55c5f4f85880 con 0x55c5f5e9fc00 2024-09-10T21:47:51.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.352+0000 7f9b70af2280 4 Got MDS map 64 2024-09-10T21:47:51.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.352+0000 7f9b70af2280 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x55c5f5366400 0x55c5f5d01700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.352+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:1 12.f 12:f1f69726:::400.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55c5f5366800 con 0x55c5f5366400 2024-09-10T21:47:51.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.353+0000 7f9b6f600640 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x55c5f5366400 0x55c5f5d01700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.353+0000 7f9b6f600640 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x55c5f5366400 0x55c5f5d01700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=ed7eca4a082e1d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:51.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.357+0000 7f9b6f600640 1 -- 172.21.15.7:0/517850206 <== osd.6 v2:172.21.15.133:6810/3365981160 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x55c5f5cc4000 con 0x55c5f5366400 2024-09-10T21:47:51.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.357+0000 7f9b70af2280 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x55c5f5366c00 0x55c5f5374000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.357+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:2 12.14 12:292cf221:::200.00000000:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e88) -- 0x55c5f5367000 con 0x55c5f5366c00 2024-09-10T21:47:51.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.358+0000 7f9b6ec00640 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x55c5f5366c00 0x55c5f5374000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.358+0000 7f9b6ec00640 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x55c5f5366c00 0x55c5f5374000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=71ca3ee562a75bce server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:51.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.361+0000 7f9b6ec00640 1 -- 172.21.15.7:0/517850206 <== osd.5 v2:172.21.15.133:6818/3838498382 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv20 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x55c5f5cc4500 con 0x55c5f5366c00 2024-09-10T21:47:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.361+0000 7f9b61a00640 1 -- 172.21.15.7:0/517850206 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:3 12.14 12:2e2fa760:::200.00000001:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e88) -- 0x55c5f5367c00 con 0x55c5f5366c00 2024-09-10T21:47:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.361+0000 7f9b61a00640 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x55c5f5d52000 0x55c5f5375b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.361+0000 7f9b61a00640 1 -- 172.21.15.7:0/517850206 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:4 12.1b 12:ddb4e4d7:::200.00000002:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e88) -- 0x55c5f5d52400 con 0x55c5f5d52000 2024-09-10T21:47:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.361+0000 7f9b6ec00640 1 -- 172.21.15.7:0/517850206 <== osd.5 v2:172.21.15.133:6818/3838498382 2 ==== osd_op_reply(3 200.00000001 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 156+0+16 (crc 0 0 0) 0x55c5f5cc4500 con 0x55c5f5366c00 2024-09-10T21:47:51.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.361+0000 7f9b6f600640 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x55c5f5d52000 0x55c5f5375b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.362+0000 7f9b6f600640 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x55c5f5d52000 0x55c5f5375b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=22af4b19ba1a68bf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:51.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.362+0000 7f9b6f600640 1 -- 172.21.15.7:0/517850206 <== osd.4 v2:172.21.15.133:6826/1857710445 1 ==== osd_op_reply(4 200.00000002 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c5f5cc4000 con 0x55c5f5d52000 2024-09-10T21:47:51.363 INFO:teuthology.orchestra.run.smithi007.stdout:old journal was 4212166~2030 2024-09-10T21:47:51.363 INFO:teuthology.orchestra.run.smithi007.stdout:new journal start will be 8388608 (4174412 bytes past old end) 2024-09-10T21:47:51.363 INFO:teuthology.orchestra.run.smithi007.stdout:writing journal head 2024-09-10T21:47:51.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.362+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:5 12.14 12:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x55c5f5367400 con 0x55c5f5366c00 2024-09-10T21:47:51.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.364+0000 7f9b6ec00640 1 -- 172.21.15.7:0/517850206 <== osd.5 v2:172.21.15.133:6818/3838498382 3 ==== osd_op_reply(5 200.00000000 [writefull 0~90] v88'21 uv21 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55c5f5cc4500 con 0x55c5f5366c00 2024-09-10T21:47:51.365 INFO:teuthology.orchestra.run.smithi007.stdout:writing EResetJournal entry 2024-09-10T21:47:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.364+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:6 12.1b 12:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x55c5f5367800 con 0x55c5f5d52000 2024-09-10T21:47:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.364+0000 7f9b70af2280 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55c5f5d52800 0x55c5f5374580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.364+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:7 12.b 12:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x55c5f5d52c00 con 0x55c5f5d52800 2024-09-10T21:47:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.364+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:8 12.1f 12:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x55c5f5d53000 con 0x55c5f5366400 2024-09-10T21:47:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.364+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:9 12.1a 12:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x55c5f5d53400 con 0x55c5f5d52000 2024-09-10T21:47:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.364+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:10 12.1e 12:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x55c5f5d53800 con 0x55c5f5d52000 2024-09-10T21:47:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.364+0000 7f9b70af2280 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x55c5f5d53c00 0x55c5f5374b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.364+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:11 12.1d 12:bec1251c:::200.00000007:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x55c5f5d52400 con 0x55c5f5d53c00 2024-09-10T21:47:51.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.365+0000 7f9b6ec00640 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55c5f5d52800 0x55c5f5374580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.365+0000 7f9b6e200640 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x55c5f5d53c00 0x55c5f5374b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.365+0000 7f9b6ec00640 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55c5f5d52800 0x55c5f5374580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=64862aa2369ae82f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:51.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.366+0000 7f9b6e200640 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x55c5f5d53c00 0x55c5f5374b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=d27ed3dfbf184728 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:51.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.366+0000 7f9b6f600640 1 -- 172.21.15.7:0/517850206 <== osd.6 v2:172.21.15.133:6810/3365981160 2 ==== osd_op_reply(8 200.00000004 [delete] v88'7 uv6 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c5f5cc4000 con 0x55c5f5366400 2024-09-10T21:47:51.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.366+0000 7f9b6f600640 1 -- 172.21.15.7:0/517850206 <== osd.4 v2:172.21.15.133:6826/1857710445 2 ==== osd_op_reply(6 200.00000002 [delete] v88'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c5f5cc4000 con 0x55c5f5d52000 2024-09-10T21:47:51.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.367+0000 7f9b6f600640 1 -- 172.21.15.7:0/517850206 <== osd.4 v2:172.21.15.133:6826/1857710445 3 ==== osd_op_reply(9 200.00000005 [delete] v88'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c5f5cc4000 con 0x55c5f5d52000 2024-09-10T21:47:51.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.367+0000 7f9b6ec00640 1 -- 172.21.15.7:0/517850206 <== osd.3 v2:172.21.15.7:6800/2195665006 1 ==== osd_op_reply(7 200.00000003 [delete] v88'4 uv3 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c5f5cc4500 con 0x55c5f5d52800 2024-09-10T21:47:51.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.367+0000 7f9b61a00640 1 -- 172.21.15.7:0/517850206 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:12 12.1b 12:ddb4e4d7:::200.00000002:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88) -- 0x55c5f5367000 con 0x55c5f5d52000 2024-09-10T21:47:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.367+0000 7f9b6f600640 1 -- 172.21.15.7:0/517850206 <== osd.4 v2:172.21.15.133:6826/1857710445 4 ==== osd_op_reply(10 200.00000006 [delete] v88'2 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c5f5cc4000 con 0x55c5f5d52000 2024-09-10T21:47:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.368+0000 7f9b6e200640 1 -- 172.21.15.7:0/517850206 <== osd.0 v2:172.21.15.7:6808/651402908 1 ==== osd_op_reply(11 200.00000007 [delete] v88'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x55c5f5cc4780 con 0x55c5f5d53c00 2024-09-10T21:47:51.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b6f600640 1 -- 172.21.15.7:0/517850206 <== osd.4 v2:172.21.15.133:6826/1857710445 5 ==== osd_op_reply(12 200.00000002 [write 0~48] v88'3 uv3 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x55c5f5cc4000 con 0x55c5f5d52000 2024-09-10T21:47:51.370 INFO:teuthology.orchestra.run.smithi007.stdout:done 2024-09-10T21:47:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x55c5f5d53c00 msgr2=0x55c5f5374b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b70af2280 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x55c5f5d53c00 0x55c5f5374b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55c5f5d52800 msgr2=0x55c5f5374580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b70af2280 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55c5f5d52800 0x55c5f5374580 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x55c5f5d52000 msgr2=0x55c5f5375b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b70af2280 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x55c5f5d52000 0x55c5f5375b80 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x55c5f5366c00 msgr2=0x55c5f5374000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b70af2280 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x55c5f5366c00 0x55c5f5374000 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x55c5f5366400 msgr2=0x55c5f5d01700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b70af2280 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x55c5f5366400 0x55c5f5d01700 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b6e200640 1 -- 172.21.15.7:0/517850206 reap_dead start 2024-09-10T21:47:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e9fc00 msgr2=0x55c5f5d01180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b70af2280 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e9fc00 0x55c5f5d01180 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x55c5f5e95f50 tx=0x55c5f5e437a0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 shutdown_connections 2024-09-10T21:47:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b70af2280 1 --2- 172.21.15.7:0/517850206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e9fc00 0x55c5f5d01180 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 >> 172.21.15.7:0/517850206 conn(0x55c5f5e9f000 msgr2=0x55c5f5015800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 shutdown_connections 2024-09-10T21:47:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b70af2280 1 -- 172.21.15.7:0/517850206 wait complete. 2024-09-10T21:47:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.369+0000 7f9b70af2280 1 -- 172.21.15.7:0/2629421341 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x55c5f5e9ec00 msgr2=0x55c5f5d00100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.370+0000 7f9b70af2280 1 --2- 172.21.15.7:0/2629421341 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x55c5f5e9ec00 0x55c5f5d00100 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x55c5f5cd8150 tx=0x55c5f52ecc00 comp rx=0 tx=0).stop 2024-09-10T21:47:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.370+0000 7f9b70af2280 1 -- 172.21.15.7:0/2629421341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e29400 msgr2=0x55c5f5cff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.370+0000 7f9b70af2280 1 --2- 172.21.15.7:0/2629421341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e29400 0x55c5f5cff080 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x55c5f5e3dd40 tx=0x55c5f5e42e40 comp rx=0 tx=0).stop 2024-09-10T21:47:51.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.370+0000 7f9b70af2280 1 -- 172.21.15.7:0/2629421341 shutdown_connections 2024-09-10T21:47:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.370+0000 7f9b70af2280 1 --2- 172.21.15.7:0/2629421341 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x55c5f5e9ec00 0x55c5f5d00100 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.370+0000 7f9b70af2280 1 --2- 172.21.15.7:0/2629421341 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f5e29800 0x55c5f5cff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.370+0000 7f9b70af2280 1 --2- 172.21.15.7:0/2629421341 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f5e29c00 0x55c5f5cffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.370+0000 7f9b70af2280 1 --2- 172.21.15.7:0/2629421341 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f5e29400 0x55c5f5cff080 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.370+0000 7f9b70af2280 1 -- 172.21.15.7:0/2629421341 >> 172.21.15.7:0/2629421341 conn(0x55c5f5e29000 msgr2=0x55c5f5015000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.370+0000 7f9b70af2280 1 -- 172.21.15.7:0/2629421341 shutdown_connections 2024-09-10T21:47:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.370+0000 7f9b70af2280 1 -- 172.21.15.7:0/2629421341 wait complete. 2024-09-10T21:47:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.370+0000 7f9b70af2280 1 -- 172.21.15.7:0/85157946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f4f2bc00 msgr2=0x55c5f5cfeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.370+0000 7f9b70af2280 1 --2- 172.21.15.7:0/85157946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f4f2bc00 0x55c5f5cfeb00 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x55c5f5d75c50 tx=0x55c5f5cc8a80 comp rx=0 tx=0).stop 2024-09-10T21:47:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.370+0000 7f9b70af2280 1 -- 172.21.15.7:0/85157946 shutdown_connections 2024-09-10T21:47:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.370+0000 7f9b70af2280 1 --2- 172.21.15.7:0/85157946 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c5f4f2b800 0x55c5f5cfe580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.370+0000 7f9b70af2280 1 --2- 172.21.15.7:0/85157946 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c5f4f2bc00 0x55c5f5cfeb00 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.370+0000 7f9b70af2280 1 --2- 172.21.15.7:0/85157946 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c5f4f2ac00 0x55c5f5cfe000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.370+0000 7f9b70af2280 1 -- 172.21.15.7:0/85157946 >> 172.21.15.7:0/85157946 conn(0x55c5f4f2b000 msgr2=0x55c5f5014800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.370+0000 7f9b70af2280 1 -- 172.21.15.7:0/85157946 shutdown_connections 2024-09-10T21:47:51.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.371+0000 7f9b70af2280 1 -- 172.21.15.7:0/85157946 wait complete. 2024-09-10T21:47:51.376 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--yes-i-really-really-mean-it'] in time 0:00:00.113292, result: old journal was 4212166~2030 new journal start will be 8388608 (4174412 bytes past old end) writing journal head writing EResetJournal entry done 2024-09-10T21:47:51.376 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:47:51.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.538+0000 7fd837200640 1 -- 172.21.15.7:0/3625869157 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8300688e0 msgr2=0x7fd8300d01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.538+0000 7fd837200640 1 --2- 172.21.15.7:0/3625869157 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8300688e0 0x7fd8300d01f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd820009830 tx=0x7fd82002a510 comp rx=0 tx=0).stop 2024-09-10T21:47:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.538+0000 7fd837200640 1 -- 172.21.15.7:0/3625869157 shutdown_connections 2024-09-10T21:47:51.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.538+0000 7fd837200640 1 --2- 172.21.15.7:0/3625869157 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd8300688e0 0x7fd8300d01f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.538+0000 7fd837200640 1 --2- 172.21.15.7:0/3625869157 >> v2:172.21.15.133:3300/0 conn(0x7fd83015bac0 0x7fd8300cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.538+0000 7fd837200640 1 -- 172.21.15.7:0/3625869157 >> 172.21.15.7:0/3625869157 conn(0x7fd830074080 msgr2=0x7fd830074460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:51.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.538+0000 7fd837200640 1 -- 172.21.15.7:0/3625869157 shutdown_connections 2024-09-10T21:47:51.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.538+0000 7fd837200640 1 -- 172.21.15.7:0/3625869157 wait complete. 2024-09-10T21:47:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.540+0000 7fd837200640 1 Processor -- start 2024-09-10T21:47:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.540+0000 7fd837200640 1 -- start start 2024-09-10T21:47:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.541+0000 7fd837200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8300688e0 0x7fd83006f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.541+0000 7fd837200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8300d0730 0x7fd8301ef390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.541+0000 7fd837200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd83015bac0 0x7fd830070840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.541+0000 7fd837200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd8300cf170 con 0x7fd8300688e0 2024-09-10T21:47:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.541+0000 7fd837200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd8300ceff0 con 0x7fd8300d0730 2024-09-10T21:47:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.541+0000 7fd837200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fd8300cf2f0 con 0x7fd83015bac0 2024-09-10T21:47:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.541+0000 7fd835e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8300688e0 0x7fd83006f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.541+0000 7fd835e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8300688e0 0x7fd83006f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57934/0 (socket says 172.21.15.7:57934) 2024-09-10T21:47:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.541+0000 7fd835e00640 1 -- 172.21.15.7:0/2315817557 learned_addr learned my addr 172.21.15.7:0/2315817557 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.541+0000 7fd835400640 1 --2- 172.21.15.7:0/2315817557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8300d0730 0x7fd8301ef390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.541+0000 7fd836800640 1 --2- 172.21.15.7:0/2315817557 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd83015bac0 0x7fd830070840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.542+0000 7fd835e00640 1 -- 172.21.15.7:0/2315817557 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd83015bac0 msgr2=0x7fd830070840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.542+0000 7fd835e00640 1 --2- 172.21.15.7:0/2315817557 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd83015bac0 0x7fd830070840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.542+0000 7fd835e00640 1 -- 172.21.15.7:0/2315817557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8300d0730 msgr2=0x7fd8301ef390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.542+0000 7fd835e00640 1 --2- 172.21.15.7:0/2315817557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8300d0730 0x7fd8301ef390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.542+0000 7fd835e00640 1 -- 172.21.15.7:0/2315817557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8301f9ac0 con 0x7fd8300688e0 2024-09-10T21:47:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.542+0000 7fd835e00640 1 --2- 172.21.15.7:0/2315817557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8300688e0 0x7fd83006f740 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd82400e7c0 tx=0x7fd82400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.542+0000 7fd82a000640 1 -- 172.21.15.7:0/2315817557 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82400ccc0 con 0x7fd8300688e0 2024-09-10T21:47:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.542+0000 7fd82a000640 1 -- 172.21.15.7:0/2315817557 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd824005020 con 0x7fd8300688e0 2024-09-10T21:47:51.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.542+0000 7fd82a000640 1 -- 172.21.15.7:0/2315817557 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd82400ce60 con 0x7fd8300688e0 2024-09-10T21:47:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.542+0000 7fd837200640 1 -- 172.21.15.7:0/2315817557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd830070090 con 0x7fd8300688e0 2024-09-10T21:47:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.543+0000 7fd837200640 1 -- 172.21.15.7:0/2315817557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8300c84b0 con 0x7fd8300688e0 2024-09-10T21:47:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.544+0000 7fd837200640 1 -- 172.21.15.7:0/2315817557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8300d01f0 con 0x7fd8300688e0 2024-09-10T21:47:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.545+0000 7fd82a000640 1 -- 172.21.15.7:0/2315817557 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd824005470 con 0x7fd8300688e0 2024-09-10T21:47:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.547+0000 7fd82a000640 1 --2- 172.21.15.7:0/2315817557 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd814077f70 0x7fd81407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.547+0000 7fd82a000640 1 -- 172.21.15.7:0/2315817557 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fd82409a690 con 0x7fd8300688e0 2024-09-10T21:47:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.547+0000 7fd835400640 1 --2- 172.21.15.7:0/2315817557 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd814077f70 0x7fd81407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.548+0000 7fd835400640 1 --2- 172.21.15.7:0/2315817557 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd814077f70 0x7fd81407a430 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd820034000 tx=0x7fd820004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:51.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.552+0000 7fd82a000640 1 -- 172.21.15.7:0/2315817557 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd824062ba0 con 0x7fd8300688e0 2024-09-10T21:47:51.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:51.783+0000 7fd837200640 1 -- 172.21.15.7:0/2315817557 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fd8300b4cb0 con 0x7fd8300688e0 2024-09-10T21:47:52.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.147+0000 7fd82a000640 1 -- 172.21.15.7:0/2315817557 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v65) ==== 162+0+0 (secure 0 0 0) 0x7fd8240675d0 con 0x7fd8300688e0 2024-09-10T21:47:52.148 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:47:52.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.150+0000 7fd837200640 1 -- 172.21.15.7:0/2315817557 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd814077f70 msgr2=0x7fd81407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:52.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.150+0000 7fd837200640 1 --2- 172.21.15.7:0/2315817557 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd814077f70 0x7fd81407a430 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd820034000 tx=0x7fd820004730 comp rx=0 tx=0).stop 2024-09-10T21:47:52.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.150+0000 7fd837200640 1 -- 172.21.15.7:0/2315817557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8300688e0 msgr2=0x7fd83006f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:52.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.150+0000 7fd837200640 1 --2- 172.21.15.7:0/2315817557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8300688e0 0x7fd83006f740 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd82400e7c0 tx=0x7fd82400ec80 comp rx=0 tx=0).stop 2024-09-10T21:47:52.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.150+0000 7fd837200640 1 -- 172.21.15.7:0/2315817557 shutdown_connections 2024-09-10T21:47:52.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.150+0000 7fd837200640 1 --2- 172.21.15.7:0/2315817557 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd814077f70 0x7fd81407a430 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:52.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.150+0000 7fd837200640 1 --2- 172.21.15.7:0/2315817557 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd8300d0730 0x7fd8301ef390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:52.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.150+0000 7fd837200640 1 --2- 172.21.15.7:0/2315817557 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd83015bac0 0x7fd830070840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:52.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.150+0000 7fd837200640 1 --2- 172.21.15.7:0/2315817557 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd8300688e0 0x7fd83006f740 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:52.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.150+0000 7fd837200640 1 -- 172.21.15.7:0/2315817557 >> 172.21.15.7:0/2315817557 conn(0x7fd830074080 msgr2=0x7fd8300b45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:52.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.150+0000 7fd837200640 1 -- 172.21.15.7:0/2315817557 shutdown_connections 2024-09-10T21:47:52.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.150+0000 7fd837200640 1 -- 172.21.15.7:0/2315817557 wait complete. 2024-09-10T21:47:52.169 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:47:52.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.330+0000 7f4602800640 1 -- 172.21.15.7:0/1121703851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45fc073c70 msgr2=0x7f45fc15fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:52.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.330+0000 7f4602800640 1 --2- 172.21.15.7:0/1121703851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45fc073c70 0x7f45fc15fda0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f45fc06dac0 tx=0x7f45f402a8f0 comp rx=0 tx=0).stop 2024-09-10T21:47:52.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.330+0000 7f4602800640 1 -- 172.21.15.7:0/1121703851 shutdown_connections 2024-09-10T21:47:52.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.330+0000 7f4602800640 1 --2- 172.21.15.7:0/1121703851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45fc073c70 0x7f45fc15fda0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:52.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.330+0000 7f4602800640 1 --2- 172.21.15.7:0/1121703851 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45fc0cd750 0x7f45fc073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:52.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.330+0000 7f4602800640 1 --2- 172.21.15.7:0/1121703851 >> v2:172.21.15.133:3300/0 conn(0x7f45fc15bc90 0x7f45fc0ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:52.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.330+0000 7f4602800640 1 -- 172.21.15.7:0/1121703851 >> 172.21.15.7:0/1121703851 conn(0x7f45fc0b16d0 msgr2=0x7f45fc0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:52.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.330+0000 7f4602800640 1 -- 172.21.15.7:0/1121703851 shutdown_connections 2024-09-10T21:47:52.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.330+0000 7f4602800640 1 -- 172.21.15.7:0/1121703851 wait complete. 2024-09-10T21:47:52.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.332+0000 7f4602800640 1 Processor -- start 2024-09-10T21:47:52.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.332+0000 7f4602800640 1 -- start start 2024-09-10T21:47:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.333+0000 7f4602800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45fc073c70 0x7f45fc0a9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.333+0000 7f4602800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45fc0cd750 0x7f45fc1f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.333+0000 7f4602800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45fc15bc90 0x7f45fc14fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.333+0000 7f4602800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc160670 con 0x7f45fc0cd750 2024-09-10T21:47:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.333+0000 7f4602800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f45fc1604f0 con 0x7f45fc073c70 2024-09-10T21:47:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.333+0000 7f4602800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f45fc06fcf0 con 0x7f45fc15bc90 2024-09-10T21:47:52.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.333+0000 7f45faa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45fc0cd750 0x7f45fc1f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.333+0000 7f45faa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45fc0cd750 0x7f45fc1f2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:57958/0 (socket says 172.21.15.7:57958) 2024-09-10T21:47:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.333+0000 7f45faa00640 1 -- 172.21.15.7:0/3120352345 learned_addr learned my addr 172.21.15.7:0/3120352345 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.333+0000 7f45fb400640 1 --2- 172.21.15.7:0/3120352345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45fc073c70 0x7f45fc0a9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.333+0000 7f45fbe00640 1 --2- 172.21.15.7:0/3120352345 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45fc15bc90 0x7f45fc14fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.333+0000 7f45faa00640 1 -- 172.21.15.7:0/3120352345 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45fc15bc90 msgr2=0x7f45fc14fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.333+0000 7f45faa00640 1 --2- 172.21.15.7:0/3120352345 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45fc15bc90 0x7f45fc14fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.333+0000 7f45faa00640 1 -- 172.21.15.7:0/3120352345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45fc073c70 msgr2=0x7f45fc0a9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.333+0000 7f45faa00640 1 --2- 172.21.15.7:0/3120352345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45fc073c70 0x7f45fc0a9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.333+0000 7f45faa00640 1 -- 172.21.15.7:0/3120352345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45fc1f3430 con 0x7f45fc0cd750 2024-09-10T21:47:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.333+0000 7f45fb400640 1 --2- 172.21.15.7:0/3120352345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45fc073c70 0x7f45fc0a9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.334+0000 7f45faa00640 1 --2- 172.21.15.7:0/3120352345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45fc0cd750 0x7f45fc1f2ce0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f45e800e9f0 tx=0x7f45e800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.334+0000 7f45f3400640 1 -- 172.21.15.7:0/3120352345 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e800cd80 con 0x7f45fc0cd750 2024-09-10T21:47:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.334+0000 7f45f3400640 1 -- 172.21.15.7:0/3120352345 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45e8005020 con 0x7f45fc0cd750 2024-09-10T21:47:52.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.334+0000 7f4602800640 1 -- 172.21.15.7:0/3120352345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45fc1f3710 con 0x7f45fc0cd750 2024-09-10T21:47:52.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.334+0000 7f45f3400640 1 -- 172.21.15.7:0/3120352345 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45e8005450 con 0x7f45fc0cd750 2024-09-10T21:47:52.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.334+0000 7f4602800640 1 -- 172.21.15.7:0/3120352345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45fc1f3cd0 con 0x7f45fc0cd750 2024-09-10T21:47:52.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.335+0000 7f4602800640 1 -- 172.21.15.7:0/3120352345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45fc073730 con 0x7f45fc0cd750 2024-09-10T21:47:52.337 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.336+0000 7f45f3400640 1 -- 172.21.15.7:0/3120352345 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f45e80047a0 con 0x7f45fc0cd750 2024-09-10T21:47:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.337+0000 7f45f3400640 1 --2- 172.21.15.7:0/3120352345 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f45dc077f70 0x7f45dc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.337+0000 7f45f3400640 1 -- 172.21.15.7:0/3120352345 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f45e809a8f0 con 0x7f45fc0cd750 2024-09-10T21:47:52.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.338+0000 7f45fb400640 1 --2- 172.21.15.7:0/3120352345 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f45dc077f70 0x7f45dc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:52.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.338+0000 7f45fb400640 1 --2- 172.21.15.7:0/3120352345 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f45dc077f70 0x7f45dc07a430 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009b00 tx=0x7f45ec005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:52.342 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.341+0000 7f45f3400640 1 -- 172.21.15.7:0/3120352345 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f45e8063450 con 0x7f45fc0cd750 2024-09-10T21:47:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.574+0000 7f4602800640 1 -- 172.21.15.7:0/3120352345 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45fc070810 con 0x7f45fc0cd750 2024-09-10T21:47:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.574+0000 7f45f3400640 1 -- 172.21.15.7:0/3120352345 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) ==== 94+0+5218 (secure 0 0 0) 0x7f45e8062d50 con 0x7f45fc0cd750 2024-09-10T21:47:52.575 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:52.576 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":66,"btime":"2024-09-10T21:47:52:150773+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/978763848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":978763848},{"type":"v1","addr":"172.21.15.7:6835","nonce":978763848}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/269527918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":269527918},{"type":"v1","addr":"172.21.15.7:6833","nonce":269527918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":235775,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/760980985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":760980985},{"type":"v1","addr":"172.21.15.133:6836","nonce":760980985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":66,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:47:40.664091+0000","modified":"2024-09-10T21:47:52.150755+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235778":{"gid":235778,"name":"d","rank":0,"incarnation":66,"state":"up:replay","state_seq":1,"addr":"172.21.15.133:6837/3312604545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3312604545},{"type":"v1","addr":"172.21.15.133:6837","nonce":3312604545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:47:52.576 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 66 2024-09-10T21:47:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.577+0000 7f4602800640 1 -- 172.21.15.7:0/3120352345 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f45dc077f70 msgr2=0x7f45dc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.577+0000 7f4602800640 1 --2- 172.21.15.7:0/3120352345 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f45dc077f70 0x7f45dc07a430 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f45ec009b00 tx=0x7f45ec005f90 comp rx=0 tx=0).stop 2024-09-10T21:47:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.577+0000 7f4602800640 1 -- 172.21.15.7:0/3120352345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45fc0cd750 msgr2=0x7f45fc1f2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.577+0000 7f4602800640 1 --2- 172.21.15.7:0/3120352345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45fc0cd750 0x7f45fc1f2ce0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f45e800e9f0 tx=0x7f45e800edf0 comp rx=0 tx=0).stop 2024-09-10T21:47:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.577+0000 7f4602800640 1 -- 172.21.15.7:0/3120352345 shutdown_connections 2024-09-10T21:47:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.577+0000 7f4602800640 1 --2- 172.21.15.7:0/3120352345 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f45dc077f70 0x7f45dc07a430 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.577+0000 7f4602800640 1 --2- 172.21.15.7:0/3120352345 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f45fc15bc90 0x7f45fc14fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.577+0000 7f4602800640 1 --2- 172.21.15.7:0/3120352345 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f45fc0cd750 0x7f45fc1f2ce0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.577+0000 7f4602800640 1 --2- 172.21.15.7:0/3120352345 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f45fc073c70 0x7f45fc0a9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.577+0000 7f4602800640 1 -- 172.21.15.7:0/3120352345 >> 172.21.15.7:0/3120352345 conn(0x7f45fc0b16d0 msgr2=0x7f45fc070100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.577+0000 7f4602800640 1 -- 172.21.15.7:0/3120352345 shutdown_connections 2024-09-10T21:47:52.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:52.577+0000 7f4602800640 1 -- 172.21.15.7:0/3120352345 wait complete. 2024-09-10T21:47:52.996 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 66, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:47:40.664091+0000', 'modified': '2024-09-10T21:47:52.150755+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 88, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 235778}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_235778': {'gid': 235778, 'name': 'd', 'rank': 0, 'incarnation': 66, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.133:6837/3312604545', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6835', 'nonce': 3312604545}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 3312604545}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:47:52.996 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_235778:up:replay 2024-09-10T21:47:53.997 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:47:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.167+0000 7fd6c4a00640 1 -- 172.21.15.7:0/285399535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6c00b3150 msgr2=0x7fd6c00ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.167+0000 7fd6c4a00640 1 --2- 172.21.15.7:0/285399535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6c00b3150 0x7fd6c00ce6f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00706a0 tx=0x7fd6b802a8c0 comp rx=0 tx=0).stop 2024-09-10T21:47:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.168+0000 7fd6c4a00640 1 -- 172.21.15.7:0/285399535 shutdown_connections 2024-09-10T21:47:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.168+0000 7fd6c4a00640 1 --2- 172.21.15.7:0/285399535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6c00b3150 0x7fd6c00ce6f0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.168+0000 7fd6c4a00640 1 --2- 172.21.15.7:0/285399535 >> v2:172.21.15.133:3300/0 conn(0x7fd6c0145540 0x7fd6c00b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.168+0000 7fd6c4a00640 1 -- 172.21.15.7:0/285399535 >> 172.21.15.7:0/285399535 conn(0x7fd6c00b1d70 msgr2=0x7fd6c00cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.168+0000 7fd6c4a00640 1 -- 172.21.15.7:0/285399535 shutdown_connections 2024-09-10T21:47:54.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.168+0000 7fd6c4a00640 1 -- 172.21.15.7:0/285399535 wait complete. 2024-09-10T21:47:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.169+0000 7fd6c4a00640 1 Processor -- start 2024-09-10T21:47:54.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.169+0000 7fd6c4a00640 1 -- start start 2024-09-10T21:47:54.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.170+0000 7fd6c4a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6c00b3150 0x7fd6c0073670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.170+0000 7fd6c4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6c00b4ac0 0x7fd6c01430f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.170+0000 7fd6c4a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6c0145540 0x7fd6c013bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.170+0000 7fd6c4a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c00cedb0 con 0x7fd6c00b4ac0 2024-09-10T21:47:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.170+0000 7fd6c4a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd6c00cec30 con 0x7fd6c0145540 2024-09-10T21:47:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.170+0000 7fd6c4a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fd6c00cef30 con 0x7fd6c00b3150 2024-09-10T21:47:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.171+0000 7fd6b5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6c00b4ac0 0x7fd6c01430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.171+0000 7fd6b5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6c00b4ac0 0x7fd6c01430f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49406/0 (socket says 172.21.15.7:49406) 2024-09-10T21:47:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.171+0000 7fd6b5e00640 1 -- 172.21.15.7:0/2237537175 learned_addr learned my addr 172.21.15.7:0/2237537175 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.171+0000 7fd6be800640 1 --2- 172.21.15.7:0/2237537175 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6c00b3150 0x7fd6c0073670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:54.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.171+0000 7fd6bf200640 1 --2- 172.21.15.7:0/2237537175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6c0145540 0x7fd6c013bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:54.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.171+0000 7fd6b5e00640 1 -- 172.21.15.7:0/2237537175 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6c00b3150 msgr2=0x7fd6c0073670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:54.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.171+0000 7fd6b5e00640 1 --2- 172.21.15.7:0/2237537175 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6c00b3150 0x7fd6c0073670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.171+0000 7fd6b5e00640 1 -- 172.21.15.7:0/2237537175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6c0145540 msgr2=0x7fd6c013bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.171+0000 7fd6b5e00640 1 --2- 172.21.15.7:0/2237537175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6c0145540 0x7fd6c013bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.171+0000 7fd6b5e00640 1 -- 172.21.15.7:0/2237537175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6c0143ad0 con 0x7fd6c00b4ac0 2024-09-10T21:47:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.171+0000 7fd6b5e00640 1 --2- 172.21.15.7:0/2237537175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6c00b4ac0 0x7fd6c01430f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800c450 tx=0x7fd6a800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.172+0000 7fd6b7400640 1 -- 172.21.15.7:0/2237537175 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8016020 con 0x7fd6c00b4ac0 2024-09-10T21:47:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.172+0000 7fd6b7400640 1 -- 172.21.15.7:0/2237537175 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6a8012920 con 0x7fd6c00b4ac0 2024-09-10T21:47:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.172+0000 7fd6b7400640 1 -- 172.21.15.7:0/2237537175 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a8012d70 con 0x7fd6c00b4ac0 2024-09-10T21:47:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.172+0000 7fd6c4a00640 1 -- 172.21.15.7:0/2237537175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6c013d5b0 con 0x7fd6c00b4ac0 2024-09-10T21:47:54.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.172+0000 7fd6c4a00640 1 -- 172.21.15.7:0/2237537175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6c013e090 con 0x7fd6c00b4ac0 2024-09-10T21:47:54.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.173+0000 7fd6c4a00640 1 -- 172.21.15.7:0/2237537175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6c00ce6f0 con 0x7fd6c00b4ac0 2024-09-10T21:47:54.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.174+0000 7fd6b7400640 1 -- 172.21.15.7:0/2237537175 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd6a801b400 con 0x7fd6c00b4ac0 2024-09-10T21:47:54.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.175+0000 7fd6b7400640 1 --2- 172.21.15.7:0/2237537175 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd69c077f70 0x7fd69c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:54.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.175+0000 7fd6b7400640 1 -- 172.21.15.7:0/2237537175 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fd6a8099f80 con 0x7fd6c00b4ac0 2024-09-10T21:47:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.175+0000 7fd6be800640 1 --2- 172.21.15.7:0/2237537175 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd69c077f70 0x7fd69c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:54.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.176+0000 7fd6be800640 1 --2- 172.21.15.7:0/2237537175 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd69c077f70 0x7fd69c07a430 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd6c01563b0 tx=0x7fd6b000b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:54.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.179+0000 7fd6b7400640 1 -- 172.21.15.7:0/2237537175 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd6a8020020 con 0x7fd6c00b4ac0 2024-09-10T21:47:54.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.413+0000 7fd6c4a00640 1 -- 172.21.15.7:0/2237537175 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd6c015b980 con 0x7fd6c00b4ac0 2024-09-10T21:47:54.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.414+0000 7fd6b7400640 1 -- 172.21.15.7:0/2237537175 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) ==== 94+0+5218 (secure 0 0 0) 0x7fd6a80623d0 con 0x7fd6c00b4ac0 2024-09-10T21:47:54.415 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:54.415 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":68,"btime":"2024-09-10T21:47:54:154644+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/978763848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":978763848},{"type":"v1","addr":"172.21.15.7:6835","nonce":978763848}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/269527918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":269527918},{"type":"v1","addr":"172.21.15.7:6833","nonce":269527918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":235775,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/760980985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":760980985},{"type":"v1","addr":"172.21.15.133:6836","nonce":760980985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:47:40.664091+0000","modified":"2024-09-10T21:47:53.158467+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235778":{"gid":235778,"name":"d","rank":0,"incarnation":66,"state":"up:rejoin","state_seq":7,"addr":"172.21.15.133:6837/3312604545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3312604545},{"type":"v1","addr":"172.21.15.133:6837","nonce":3312604545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:47:54.415 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 68 2024-09-10T21:47:54.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.416+0000 7fd6c4a00640 1 -- 172.21.15.7:0/2237537175 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd69c077f70 msgr2=0x7fd69c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:54.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.416+0000 7fd6c4a00640 1 --2- 172.21.15.7:0/2237537175 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd69c077f70 0x7fd69c07a430 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd6c01563b0 tx=0x7fd6b000b2e0 comp rx=0 tx=0).stop 2024-09-10T21:47:54.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.416+0000 7fd6c4a00640 1 -- 172.21.15.7:0/2237537175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6c00b4ac0 msgr2=0x7fd6c01430f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:54.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.416+0000 7fd6c4a00640 1 --2- 172.21.15.7:0/2237537175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6c00b4ac0 0x7fd6c01430f0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800c450 tx=0x7fd6a800c910 comp rx=0 tx=0).stop 2024-09-10T21:47:54.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.416+0000 7fd6c4a00640 1 -- 172.21.15.7:0/2237537175 shutdown_connections 2024-09-10T21:47:54.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.416+0000 7fd6c4a00640 1 --2- 172.21.15.7:0/2237537175 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd69c077f70 0x7fd69c07a430 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:54.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.416+0000 7fd6c4a00640 1 --2- 172.21.15.7:0/2237537175 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6c0145540 0x7fd6c013bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:54.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.416+0000 7fd6c4a00640 1 --2- 172.21.15.7:0/2237537175 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6c00b4ac0 0x7fd6c01430f0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:54.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.416+0000 7fd6c4a00640 1 --2- 172.21.15.7:0/2237537175 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6c00b3150 0x7fd6c0073670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:54.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.416+0000 7fd6c4a00640 1 -- 172.21.15.7:0/2237537175 >> 172.21.15.7:0/2237537175 conn(0x7fd6c00b1d70 msgr2=0x7fd6c006f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:54.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.416+0000 7fd6c4a00640 1 -- 172.21.15.7:0/2237537175 shutdown_connections 2024-09-10T21:47:54.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:54.416+0000 7fd6c4a00640 1 -- 172.21.15.7:0/2237537175 wait complete. 2024-09-10T21:47:54.835 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:47:40.664091+0000', 'modified': '2024-09-10T21:47:53.158467+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 88, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 235778}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_235778': {'gid': 235778, 'name': 'd', 'rank': 0, 'incarnation': 66, 'state': 'up:rejoin', 'state_seq': 7, 'addr': '172.21.15.133:6837/3312604545', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6835', 'nonce': 3312604545}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 3312604545}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:47:54.836 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_235778:up:rejoin 2024-09-10T21:47:55.837 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:47:56.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.009+0000 7fc9ec000640 1 -- 172.21.15.7:0/558539765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e415fda0 msgr2=0x7fc9e41601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:56.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.009+0000 7fc9ec000640 1 --2- 172.21.15.7:0/558539765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e415fda0 0x7fc9e41601f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc005fd0 tx=0x7fc9dc02a980 comp rx=0 tx=0).stop 2024-09-10T21:47:56.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.009+0000 7fc9ec000640 1 -- 172.21.15.7:0/558539765 shutdown_connections 2024-09-10T21:47:56.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.009+0000 7fc9ec000640 1 --2- 172.21.15.7:0/558539765 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e415fda0 0x7fc9e41601f0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:56.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.009+0000 7fc9ec000640 1 --2- 172.21.15.7:0/558539765 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9e415bc90 0x7fc9e40ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:56.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.009+0000 7fc9ec000640 1 -- 172.21.15.7:0/558539765 >> 172.21.15.7:0/558539765 conn(0x7fc9e40b16d0 msgr2=0x7fc9e40b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:56.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.009+0000 7fc9ec000640 1 -- 172.21.15.7:0/558539765 shutdown_connections 2024-09-10T21:47:56.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.009+0000 7fc9ec000640 1 -- 172.21.15.7:0/558539765 wait complete. 2024-09-10T21:47:56.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.011+0000 7fc9ec000640 1 Processor -- start 2024-09-10T21:47:56.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.011+0000 7fc9ec000640 1 -- start start 2024-09-10T21:47:56.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.012+0000 7fc9ec000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9e40cd750 0x7fc9e4072360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:56.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.012+0000 7fc9ec000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9e415bc90 0x7fc9e40728a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:56.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.012+0000 7fc9ec000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e415fda0 0x7fc9e414f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:56.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.012+0000 7fc9ec000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e406fe70 con 0x7fc9e415fda0 2024-09-10T21:47:56.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.012+0000 7fc9ec000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e406fcf0 con 0x7fc9e415bc90 2024-09-10T21:47:56.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.012+0000 7fc9ec000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc9e406fff0 con 0x7fc9e40cd750 2024-09-10T21:47:56.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.012+0000 7fc9ea000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e415fda0 0x7fc9e414f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:56.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.013+0000 7fc9ea000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e415fda0 0x7fc9e414f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49430/0 (socket says 172.21.15.7:49430) 2024-09-10T21:47:56.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.013+0000 7fc9ea000640 1 -- 172.21.15.7:0/1149998812 learned_addr learned my addr 172.21.15.7:0/1149998812 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:56.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.013+0000 7fc9e9600640 1 --2- 172.21.15.7:0/1149998812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9e40cd750 0x7fc9e4072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:56.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.013+0000 7fc9e8c00640 1 --2- 172.21.15.7:0/1149998812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9e415bc90 0x7fc9e40728a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:56.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.013+0000 7fc9ea000640 1 -- 172.21.15.7:0/1149998812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9e40cd750 msgr2=0x7fc9e4072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:56.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.013+0000 7fc9ea000640 1 --2- 172.21.15.7:0/1149998812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9e40cd750 0x7fc9e4072360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:56.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.013+0000 7fc9ea000640 1 -- 172.21.15.7:0/1149998812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9e415bc90 msgr2=0x7fc9e40728a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:56.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.013+0000 7fc9ea000640 1 --2- 172.21.15.7:0/1149998812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9e415bc90 0x7fc9e40728a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:56.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.013+0000 7fc9ea000640 1 -- 172.21.15.7:0/1149998812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e40c7b60 con 0x7fc9e415fda0 2024-09-10T21:47:56.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.013+0000 7fc9ea000640 1 --2- 172.21.15.7:0/1149998812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e415fda0 0x7fc9e414f9a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc033040 tx=0x7fc9dc009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:56.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.013+0000 7fc9e1600640 1 -- 172.21.15.7:0/1149998812 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc034040 con 0x7fc9e415fda0 2024-09-10T21:47:56.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.014+0000 7fc9e1600640 1 -- 172.21.15.7:0/1149998812 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9dc004540 con 0x7fc9e415fda0 2024-09-10T21:47:56.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.014+0000 7fc9ec000640 1 -- 172.21.15.7:0/1149998812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e41f4700 con 0x7fc9e415fda0 2024-09-10T21:47:56.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.014+0000 7fc9e1600640 1 -- 172.21.15.7:0/1149998812 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc045480 con 0x7fc9e415fda0 2024-09-10T21:47:56.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.014+0000 7fc9ec000640 1 -- 172.21.15.7:0/1149998812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e41f4cc0 con 0x7fc9e415fda0 2024-09-10T21:47:56.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.015+0000 7fc9ec000640 1 -- 172.21.15.7:0/1149998812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e40c7d30 con 0x7fc9e415fda0 2024-09-10T21:47:56.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.016+0000 7fc9e1600640 1 -- 172.21.15.7:0/1149998812 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc9dc04d030 con 0x7fc9e415fda0 2024-09-10T21:47:56.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.017+0000 7fc9e1600640 1 --2- 172.21.15.7:0/1149998812 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc9c4077f70 0x7fc9c407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:56.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.017+0000 7fc9e1600640 1 -- 172.21.15.7:0/1149998812 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fc9dc0bb460 con 0x7fc9e415fda0 2024-09-10T21:47:56.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.018+0000 7fc9e9600640 1 --2- 172.21.15.7:0/1149998812 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc9c4077f70 0x7fc9c407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:56.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.018+0000 7fc9e9600640 1 --2- 172.21.15.7:0/1149998812 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc9c4077f70 0x7fc9c407a430 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8005fd0 tx=0x7fc9d8005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:56.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.021+0000 7fc9e1600640 1 -- 172.21.15.7:0/1149998812 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc9dc083e80 con 0x7fc9e415fda0 2024-09-10T21:47:56.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.255+0000 7fc9ec000640 1 -- 172.21.15.7:0/1149998812 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9e4070510 con 0x7fc9e415fda0 2024-09-10T21:47:56.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.256+0000 7fc9e1600640 1 -- 172.21.15.7:0/1149998812 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) ==== 94+0+5229 (secure 0 0 0) 0x7fc9dc083780 con 0x7fc9e415fda0 2024-09-10T21:47:56.257 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:47:56.257 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":69,"btime":"2024-09-10T21:47:55:156341+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/978763848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":978763848},{"type":"v1","addr":"172.21.15.7:6835","nonce":978763848}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/269527918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":269527918},{"type":"v1","addr":"172.21.15.7:6833","nonce":269527918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":235775,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/760980985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":760980985},{"type":"v1","addr":"172.21.15.133:6836","nonce":760980985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:47:40.664091+0000","modified":"2024-09-10T21:47:55.156336+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235778":{"gid":235778,"name":"d","rank":0,"incarnation":66,"state":"up:active","state_seq":8,"addr":"172.21.15.133:6837/3312604545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3312604545},{"type":"v1","addr":"172.21.15.133:6837","nonce":3312604545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235778,"qdb_cluster":[235778]},"id":6}]} 2024-09-10T21:47:56.257 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 69 2024-09-10T21:47:56.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.258+0000 7fc9ec000640 1 -- 172.21.15.7:0/1149998812 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc9c4077f70 msgr2=0x7fc9c407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:56.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.258+0000 7fc9ec000640 1 --2- 172.21.15.7:0/1149998812 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc9c4077f70 0x7fc9c407a430 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8005fd0 tx=0x7fc9d8005850 comp rx=0 tx=0).stop 2024-09-10T21:47:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.258+0000 7fc9ec000640 1 -- 172.21.15.7:0/1149998812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e415fda0 msgr2=0x7fc9e414f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.258+0000 7fc9ec000640 1 --2- 172.21.15.7:0/1149998812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e415fda0 0x7fc9e414f9a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc033040 tx=0x7fc9dc009360 comp rx=0 tx=0).stop 2024-09-10T21:47:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.258+0000 7fc9ec000640 1 -- 172.21.15.7:0/1149998812 shutdown_connections 2024-09-10T21:47:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.258+0000 7fc9ec000640 1 --2- 172.21.15.7:0/1149998812 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc9c4077f70 0x7fc9c407a430 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.258+0000 7fc9ec000640 1 --2- 172.21.15.7:0/1149998812 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9e415bc90 0x7fc9e40728a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.258+0000 7fc9ec000640 1 --2- 172.21.15.7:0/1149998812 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9e415fda0 0x7fc9e414f9a0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.258+0000 7fc9ec000640 1 --2- 172.21.15.7:0/1149998812 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9e40cd750 0x7fc9e4072360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.258+0000 7fc9ec000640 1 -- 172.21.15.7:0/1149998812 >> 172.21.15.7:0/1149998812 conn(0x7fc9e40b16d0 msgr2=0x7fc9e40b4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.258+0000 7fc9ec000640 1 -- 172.21.15.7:0/1149998812 shutdown_connections 2024-09-10T21:47:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.258+0000 7fc9ec000640 1 -- 172.21.15.7:0/1149998812 wait complete. 2024-09-10T21:47:56.678 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:47:40.664091+0000', 'modified': '2024-09-10T21:47:55.156336+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 88, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 235778}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_235778': {'gid': 235778, 'name': 'd', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.133:6837/3312604545', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6835', 'nonce': 3312604545}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 3312604545}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 235778, 'qdb_cluster': [235778]} 2024-09-10T21:47:56.678 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:47:56.678 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T21:47:56.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.842+0000 7f32b7c00640 1 -- 172.21.15.7:0/219784667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b006ff50 msgr2=0x7f32b015cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:57.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.842+0000 7f32b7c00640 1 --2- 172.21.15.7:0/219784667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b006ff50 0x7f32b015cbe0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f32a0005fd0 tx=0x7f32a002a420 comp rx=0 tx=0).stop 2024-09-10T21:47:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.842+0000 7f32b7c00640 1 -- 172.21.15.7:0/219784667 shutdown_connections 2024-09-10T21:47:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.842+0000 7f32b7c00640 1 --2- 172.21.15.7:0/219784667 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32b006e510 0x7f32b0073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.842+0000 7f32b7c00640 1 --2- 172.21.15.7:0/219784667 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b006ff50 0x7f32b015cbe0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.842+0000 7f32b7c00640 1 -- 172.21.15.7:0/219784667 >> 172.21.15.7:0/219784667 conn(0x7f32b00b1660 msgr2=0x7f32b00b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.842+0000 7f32b7c00640 1 -- 172.21.15.7:0/219784667 shutdown_connections 2024-09-10T21:47:57.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.842+0000 7f32b7c00640 1 -- 172.21.15.7:0/219784667 wait complete. 2024-09-10T21:47:57.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.844+0000 7f32b7c00640 1 Processor -- start 2024-09-10T21:47:57.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.844+0000 7f32b7c00640 1 -- start start 2024-09-10T21:47:57.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.845+0000 7f32b7c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b006e510 0x7f32b01f3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.845+0000 7f32b7c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32b006e960 0x7f32b01f52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.845+0000 7f32b7c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b006ff50 0x7f32b00c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.845+0000 7f32b7c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f32b00745a0 con 0x7f32b006ff50 2024-09-10T21:47:57.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.845+0000 7f32b7c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f32b0074420 con 0x7f32b006e510 2024-09-10T21:47:57.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.845+0000 7f32b7c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f32b0074720 con 0x7f32b006e960 2024-09-10T21:47:57.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.845+0000 7f32b5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b006ff50 0x7f32b00c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.845+0000 7f32b5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b006ff50 0x7f32b00c3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49452/0 (socket says 172.21.15.7:49452) 2024-09-10T21:47:57.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.845+0000 7f32b5c00640 1 -- 172.21.15.7:0/3459279290 learned_addr learned my addr 172.21.15.7:0/3459279290 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:57.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.846+0000 7f32a7e00640 1 --2- 172.21.15.7:0/3459279290 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32b006e960 0x7f32b01f52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.846+0000 7f32b5200640 1 --2- 172.21.15.7:0/3459279290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b006e510 0x7f32b01f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.846+0000 7f32b5c00640 1 -- 172.21.15.7:0/3459279290 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32b006e960 msgr2=0x7f32b01f52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:57.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.846+0000 7f32b5c00640 1 --2- 172.21.15.7:0/3459279290 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32b006e960 0x7f32b01f52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.846+0000 7f32b5c00640 1 -- 172.21.15.7:0/3459279290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b006e510 msgr2=0x7f32b01f3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:57.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.846+0000 7f32b5c00640 1 --2- 172.21.15.7:0/3459279290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b006e510 0x7f32b01f3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.846+0000 7f32b5c00640 1 -- 172.21.15.7:0/3459279290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b0150a70 con 0x7f32b006ff50 2024-09-10T21:47:57.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.846+0000 7f32b5c00640 1 --2- 172.21.15.7:0/3459279290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b006ff50 0x7f32b00c3a30 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f32a800c780 tx=0x7f32a800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:57.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.847+0000 7f32ad600640 1 -- 172.21.15.7:0/3459279290 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8017070 con 0x7f32b006ff50 2024-09-10T21:47:57.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.847+0000 7f32ad600640 1 -- 172.21.15.7:0/3459279290 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32a8003c80 con 0x7f32b006ff50 2024-09-10T21:47:57.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.847+0000 7f32ad600640 1 -- 172.21.15.7:0/3459279290 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a80052d0 con 0x7f32b006ff50 2024-09-10T21:47:57.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.847+0000 7f32b7c00640 1 -- 172.21.15.7:0/3459279290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b0150d30 con 0x7f32b006ff50 2024-09-10T21:47:57.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.847+0000 7f32b7c00640 1 -- 172.21.15.7:0/3459279290 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b0152cb0 con 0x7f32b006ff50 2024-09-10T21:47:57.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.849+0000 7f32ad600640 1 -- 172.21.15.7:0/3459279290 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f32a8005470 con 0x7f32b006ff50 2024-09-10T21:47:57.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.850+0000 7f32ad600640 1 --2- 172.21.15.7:0/3459279290 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f328c077f70 0x7f328c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.850+0000 7f32bac29740 1 Processor -- start 2024-09-10T21:47:57.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.850+0000 7f32ad600640 1 -- 172.21.15.7:0/3459279290 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f32a8099110 con 0x7f32b006ff50 2024-09-10T21:47:57.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.850+0000 7f32bac29740 1 -- start start 2024-09-10T21:47:57.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.851+0000 7f32b5200640 1 --2- 172.21.15.7:0/3459279290 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f328c077f70 0x7f328c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.851+0000 7f32bac29740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4169788b0 0x55d416978c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.851+0000 7f32bac29740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d416933a20 0x55d416977390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.851+0000 7f32bac29740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d416933fa0 0x55d4169778d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.851+0000 7f32bac29740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d4168b6430 con 0x55d416933fa0 2024-09-10T21:47:57.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.851+0000 7f32bac29740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d4168a65d0 con 0x55d4169788b0 2024-09-10T21:47:57.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.851+0000 7f32bac29740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55d416895a00 con 0x55d416933a20 2024-09-10T21:47:57.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.851+0000 7f32b5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d416933fa0 0x55d4169778d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.851+0000 7f32b5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d416933fa0 0x55d4169778d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49458/0 (socket says 172.21.15.7:49458) 2024-09-10T21:47:57.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.851+0000 7f32b5c00640 1 -- 172.21.15.7:0/3002826116 learned_addr learned my addr 172.21.15.7:0/3002826116 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:57.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.851+0000 7f32a7e00640 1 --2- 172.21.15.7:0/3002826116 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d416933a20 0x55d416977390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.851+0000 7f32b5200640 1 --2- 172.21.15.7:0/3459279290 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f328c077f70 0x7f328c07a430 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f32a0033000 tx=0x7f32a0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:57.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.852+0000 7f32b5200640 1 --2- 172.21.15.7:0/3002826116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4169788b0 0x55d416978c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.852+0000 7f32b5c00640 1 -- 172.21.15.7:0/3002826116 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d416933a20 msgr2=0x55d416977390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:57.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.852+0000 7f32b5c00640 1 --2- 172.21.15.7:0/3002826116 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d416933a20 0x55d416977390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.852+0000 7f32b5c00640 1 -- 172.21.15.7:0/3002826116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4169788b0 msgr2=0x55d416978c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:57.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.852+0000 7f32b5c00640 1 --2- 172.21.15.7:0/3002826116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4169788b0 0x55d416978c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.852+0000 7f32b5c00640 1 -- 172.21.15.7:0/3002826116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d41685e520 con 0x55d416933fa0 2024-09-10T21:47:57.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.852+0000 7f32b5c00640 1 --2- 172.21.15.7:0/3002826116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d416933fa0 0x55d4169778d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f32a8097550 tx=0x7f32a8060cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b587f769115a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:57.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.852+0000 7f32a5600640 1 -- 172.21.15.7:0/3002826116 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8061560 con 0x55d416933fa0 2024-09-10T21:47:57.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.852+0000 7f32a5600640 1 -- 172.21.15.7:0/3002826116 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32a8065f90 con 0x55d416933fa0 2024-09-10T21:47:57.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.852+0000 7f32a5600640 1 -- 172.21.15.7:0/3002826116 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a806a9b0 con 0x55d416933fa0 2024-09-10T21:47:57.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.853+0000 7f32bac29740 1 -- 172.21.15.7:0/3002826116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d416933fa0 msgr2=0x55d4169778d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:57.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.853+0000 7f32bac29740 1 --2- 172.21.15.7:0/3002826116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d416933fa0 0x55d4169778d0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f32a8097550 tx=0x7f32a8060cd0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.853+0000 7f32bac29740 1 -- 172.21.15.7:0/3002826116 shutdown_connections 2024-09-10T21:47:57.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.853+0000 7f32bac29740 1 --2- 172.21.15.7:0/3002826116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d416933fa0 0x55d4169778d0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.853+0000 7f32bac29740 1 --2- 172.21.15.7:0/3002826116 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d416933a20 0x55d416977390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.853+0000 7f32bac29740 1 --2- 172.21.15.7:0/3002826116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4169788b0 0x55d416978c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.853+0000 7f32bac29740 1 -- 172.21.15.7:0/3002826116 >> 172.21.15.7:0/3002826116 conn(0x55d41692eff0 msgr2=0x55d416932a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:57.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.853+0000 7f32bac29740 1 -- 172.21.15.7:0/3002826116 shutdown_connections 2024-09-10T21:47:57.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.853+0000 7f32bac29740 1 -- 172.21.15.7:0/3002826116 wait complete. 2024-09-10T21:47:57.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.854+0000 7f32bac29740 1 -- start start 2024-09-10T21:47:57.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.854+0000 7f32bac29740 1 Processor -- start 2024-09-10T21:47:57.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.855+0000 7f32bac29740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d416933fa0 0x55d41693ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.855+0000 7f32bac29740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4169788b0 0x55d41693d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.855+0000 7f32bac29740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41693d8e0 0x55d416957a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.855+0000 7f32bac29740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d41683c330 con 0x55d4169788b0 2024-09-10T21:47:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.855+0000 7f32bac29740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d416899810 con 0x55d41693d8e0 2024-09-10T21:47:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.855+0000 7f32bac29740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55d416606620 con 0x55d416933fa0 2024-09-10T21:47:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.855+0000 7f32bac29740 10 client.0 _ll_register_callbacks cb 0x55d416841420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:47:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.855+0000 7f32a7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4169788b0 0x55d41693d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.855+0000 7f32a7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4169788b0 0x55d41693d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49468/0 (socket says 172.21.15.7:49468) 2024-09-10T21:47:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.855+0000 7f32a7e00640 1 -- 172.21.15.7:0/2834221011 learned_addr learned my addr 172.21.15.7:0/2834221011 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.855+0000 7f32b5200640 1 --2- 172.21.15.7:0/2834221011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d416933fa0 0x55d41693ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.855+0000 7f32b5c00640 1 --2- 172.21.15.7:0/2834221011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41693d8e0 0x55d416957a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.855+0000 7f32a7e00640 1 -- 172.21.15.7:0/2834221011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d416933fa0 msgr2=0x55d41693ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.856+0000 7f32a7e00640 1 --2- 172.21.15.7:0/2834221011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d416933fa0 0x55d41693ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.856+0000 7f32a7e00640 1 -- 172.21.15.7:0/2834221011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41693d8e0 msgr2=0x55d416957a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.856+0000 7f32a7e00640 1 --2- 172.21.15.7:0/2834221011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41693d8e0 0x55d416957a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.856+0000 7f32a7e00640 1 -- 172.21.15.7:0/2834221011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4165f7850 con 0x55d4169788b0 2024-09-10T21:47:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.856+0000 7f32a7e00640 1 --2- 172.21.15.7:0/2834221011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4169788b0 0x55d41693d3a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f32980115d0 tx=0x7f3298011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cc16528a6ad5321 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.856+0000 7f329e000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:47:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.856+0000 7f329e000640 1 -- 172.21.15.7:0/2834221011 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298013670 con 0x55d4169788b0 2024-09-10T21:47:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.856+0000 7f329e000640 1 -- 172.21.15.7:0/2834221011 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f3298013810 con 0x55d4169788b0 2024-09-10T21:47:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.856+0000 7f32bac29740 1 -- 172.21.15.7:0/2834221011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32a8065f90 con 0x55d4169788b0 2024-09-10T21:47:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.856+0000 7f329e000640 1 -- 172.21.15.7:0/2834221011 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f3298025050 con 0x55d4169788b0 2024-09-10T21:47:57.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.856+0000 7f32bac29740 10 client.6972 fetch_fsmap learned FSMap version 69 2024-09-10T21:47:57.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.856+0000 7f32bac29740 1 -- 172.21.15.7:0/2834221011 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x55d41666e6b0 con 0x55d4169788b0 2024-09-10T21:47:57.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.857+0000 7f329e000640 1 -- 172.21.15.7:0/2834221011 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32a8065f90 con 0x55d4169788b0 2024-09-10T21:47:57.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.857+0000 7f329e000640 1 -- 172.21.15.7:0/2834221011 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7f3298028020 con 0x55d4169788b0 2024-09-10T21:47:57.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.857+0000 7f32bac29740 10 client.6972 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:47:57.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.857+0000 7f32bac29740 10 client.6972 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:47:57.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.857+0000 7f32bac29740 20 client.6972 populate_metadata read hostname 'smithi007' 2024-09-10T21:47:57.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.857+0000 7f32bac29740 1 --2- 172.21.15.7:0/2834221011 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55d4169974d0 0x55d4169998c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.857+0000 7f32bac29740 4 client.6972 mds_command: new command op to 235778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:47:57.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.857+0000 7f32bac29740 1 -- 172.21.15.7:0/2834221011 --> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4168a8520 con 0x55d4169974d0 2024-09-10T21:47:57.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.857+0000 7f32b5200640 1 --2- 172.21.15.7:0/2834221011 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55d4169974d0 0x55d4169998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.858+0000 7f32b5200640 1 --2- 172.21.15.7:0/2834221011 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55d4169974d0 0x55d4169998c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=438d600ddf05f84a server_cookie=64100ba9a29cca61 in_seq=0 out_seq=0 2024-09-10T21:47:57.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.858+0000 7f329e000640 10 client.6972 ms_handle_connect on v2:172.21.15.133:6835/3312604545 2024-09-10T21:47:57.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.859+0000 7f329e000640 1 -- 172.21.15.7:0/2834221011 <== mds.0 v2:172.21.15.133:6835/3312604545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d4168a8520 con 0x55d4169974d0 2024-09-10T21:47:57.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.859+0000 7f329e000640 10 client.6972 handle_command_reply: tid=0 2024-09-10T21:47:57.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.859+0000 7f32bac29740 1 client.6972 shutdown 2024-09-10T21:47:57.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.859+0000 7f32bac29740 1 -- 172.21.15.7:0/2834221011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4169788b0 msgr2=0x55d41693d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:57.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.859+0000 7f32bac29740 1 --2- 172.21.15.7:0/2834221011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4169788b0 0x55d41693d3a0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f32980115d0 tx=0x7f3298011a90 comp rx=0 tx=0).stop 2024-09-10T21:47:57.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.859+0000 7f32bac29740 1 -- 172.21.15.7:0/2834221011 shutdown_connections 2024-09-10T21:47:57.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.859+0000 7f32bac29740 1 --2- 172.21.15.7:0/2834221011 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55d4169974d0 0x55d4169998c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.859+0000 7f32bac29740 1 --2- 172.21.15.7:0/2834221011 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d41693d8e0 0x55d416957a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.859+0000 7f32bac29740 1 --2- 172.21.15.7:0/2834221011 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d4169788b0 0x55d41693d3a0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.859+0000 7f32bac29740 1 --2- 172.21.15.7:0/2834221011 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d416933fa0 0x55d41693ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.859+0000 7f329e000640 0 client.6972 ms_handle_reset on v2:172.21.15.133:6835/3312604545 2024-09-10T21:47:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.859+0000 7f32bac29740 1 -- 172.21.15.7:0/2834221011 >> 172.21.15.7:0/2834221011 conn(0x55d416933a20 msgr2=0x55d41692dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.860+0000 7f32bac29740 1 -- 172.21.15.7:0/2834221011 shutdown_connections 2024-09-10T21:47:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.860+0000 7f32bac29740 1 -- 172.21.15.7:0/2834221011 wait complete. 2024-09-10T21:47:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.860+0000 7f32bac29740 20 client.6972 trim_cache size 0 max 16384 2024-09-10T21:47:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.879+0000 7f32bac29740 1 Processor -- start 2024-09-10T21:47:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.879+0000 7f32bac29740 1 -- start start 2024-09-10T21:47:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.880+0000 7f32bac29740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d416933fa0 0x55d41697a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.880+0000 7f32bac29740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d41693d8e0 0x55d416932360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.880+0000 7f32bac29740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4169788b0 0x55d41692be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.880+0000 7f32bac29740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f32b00cdd30 con 0x55d41693d8e0 2024-09-10T21:47:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.880+0000 7f32bac29740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f32b01f6490 con 0x55d4169788b0 2024-09-10T21:47:57.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.880+0000 7f32bac29740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f32b00cd6e0 con 0x55d416933fa0 2024-09-10T21:47:57.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.880+0000 7f32a7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d41693d8e0 0x55d416932360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.880+0000 7f32a7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d41693d8e0 0x55d416932360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49480/0 (socket says 172.21.15.7:49480) 2024-09-10T21:47:57.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.880+0000 7f32a7e00640 1 -- 172.21.15.7:0/3982525563 learned_addr learned my addr 172.21.15.7:0/3982525563 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:57.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.880+0000 7f32b5200640 1 --2- 172.21.15.7:0/3982525563 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d416933fa0 0x55d41697a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.880+0000 7f32b5c00640 1 --2- 172.21.15.7:0/3982525563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4169788b0 0x55d41692be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.880+0000 7f32a7e00640 1 -- 172.21.15.7:0/3982525563 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d416933fa0 msgr2=0x55d41697a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:57.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.880+0000 7f32a7e00640 1 --2- 172.21.15.7:0/3982525563 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d416933fa0 0x55d41697a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.880+0000 7f32a7e00640 1 -- 172.21.15.7:0/3982525563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4169788b0 msgr2=0x55d41692be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:57.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.880+0000 7f32a7e00640 1 --2- 172.21.15.7:0/3982525563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4169788b0 0x55d41692be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.880+0000 7f32a7e00640 1 -- 172.21.15.7:0/3982525563 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d41697a320 con 0x55d41693d8e0 2024-09-10T21:47:57.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.881+0000 7f32a7e00640 1 --2- 172.21.15.7:0/3982525563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d41693d8e0 0x55d416932360 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f3298014010 tx=0x7f3298009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5aba9b1fb8e8c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:57.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.881+0000 7f32a4c00640 1 -- 172.21.15.7:0/3982525563 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298013670 con 0x55d41693d8e0 2024-09-10T21:47:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.881+0000 7f32a4c00640 1 -- 172.21.15.7:0/3982525563 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3298013810 con 0x55d41693d8e0 2024-09-10T21:47:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.881+0000 7f32a4c00640 1 -- 172.21.15.7:0/3982525563 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32980146a0 con 0x55d41693d8e0 2024-09-10T21:47:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.881+0000 7f32bac29740 1 -- 172.21.15.7:0/3982525563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d41693d8e0 msgr2=0x55d416932360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.881+0000 7f32bac29740 1 --2- 172.21.15.7:0/3982525563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d41693d8e0 0x55d416932360 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f3298014010 tx=0x7f3298009f10 comp rx=0 tx=0).stop 2024-09-10T21:47:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.881+0000 7f32bac29740 1 -- 172.21.15.7:0/3982525563 shutdown_connections 2024-09-10T21:47:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.881+0000 7f32bac29740 1 --2- 172.21.15.7:0/3982525563 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4169788b0 0x55d41692be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.881+0000 7f32bac29740 1 --2- 172.21.15.7:0/3982525563 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d41693d8e0 0x55d416932360 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.881+0000 7f32bac29740 1 --2- 172.21.15.7:0/3982525563 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d416933fa0 0x55d41697a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.881+0000 7f32bac29740 1 -- 172.21.15.7:0/3982525563 >> 172.21.15.7:0/3982525563 conn(0x55d416933a20 msgr2=0x55d416979730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.881+0000 7f32bac29740 1 -- 172.21.15.7:0/3982525563 shutdown_connections 2024-09-10T21:47:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.882+0000 7f32bac29740 1 -- 172.21.15.7:0/3982525563 wait complete. 2024-09-10T21:47:57.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.882+0000 7f32bac29740 1 -- start start 2024-09-10T21:47:57.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.882+0000 7f32bac29740 1 Processor -- start 2024-09-10T21:47:57.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.883+0000 7f32bac29740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d416933fa0 0x55d4169577d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.883+0000 7f32bac29740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d41693d8e0 0x55d4169f7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.883+0000 7f32bac29740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4169788b0 0x55d416a00b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.883+0000 7f32bac29740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d4169742f0 con 0x55d416933fa0 2024-09-10T21:47:57.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.883+0000 7f32bac29740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d416973d30 con 0x55d4169788b0 2024-09-10T21:47:57.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.883+0000 7f32bac29740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55d41697bf60 con 0x55d41693d8e0 2024-09-10T21:47:57.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.883+0000 7f32bac29740 10 client.0 _ll_register_callbacks cb 0x7f3298009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:47:57.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.883+0000 7f32b5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d416933fa0 0x55d4169577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.884+0000 7f32b5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d416933fa0 0x55d4169577d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49484/0 (socket says 172.21.15.7:49484) 2024-09-10T21:47:57.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.884+0000 7f32b5200640 1 -- 172.21.15.7:0/339624133 learned_addr learned my addr 172.21.15.7:0/339624133 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.884+0000 7f32b5c00640 1 --2- 172.21.15.7:0/339624133 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4169788b0 0x55d416a00b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.884+0000 7f32a7e00640 1 --2- 172.21.15.7:0/339624133 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d41693d8e0 0x55d4169f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.884+0000 7f32b5200640 1 -- 172.21.15.7:0/339624133 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d41693d8e0 msgr2=0x55d4169f7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.884+0000 7f32b5200640 1 --2- 172.21.15.7:0/339624133 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d41693d8e0 0x55d4169f7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.884+0000 7f32b5200640 1 -- 172.21.15.7:0/339624133 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4169788b0 msgr2=0x55d416a00b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.884+0000 7f32b5200640 1 --2- 172.21.15.7:0/339624133 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4169788b0 0x55d416a00b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.884+0000 7f32b5200640 1 -- 172.21.15.7:0/339624133 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d416931a90 con 0x55d416933fa0 2024-09-10T21:47:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.884+0000 7f32b5200640 1 --2- 172.21.15.7:0/339624133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d416933fa0 0x55d4169577d0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009230 tx=0x7f32a0009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b67ca35287a4224 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.884+0000 7f329e000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:47:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.884+0000 7f329e000640 1 -- 172.21.15.7:0/339624133 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0057060 con 0x55d416933fa0 2024-09-10T21:47:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.884+0000 7f329e000640 1 -- 172.21.15.7:0/339624133 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f32a003c940 con 0x55d416933fa0 2024-09-10T21:47:57.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.884+0000 7f32bac29740 1 -- 172.21.15.7:0/339624133 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3298013810 con 0x55d416933fa0 2024-09-10T21:47:57.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.885+0000 7f329e000640 1 -- 172.21.15.7:0/339624133 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f32a005d050 con 0x55d416933fa0 2024-09-10T21:47:57.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.885+0000 7f32bac29740 10 client.6978 fetch_fsmap learned FSMap version 69 2024-09-10T21:47:57.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.885+0000 7f32bac29740 1 -- 172.21.15.7:0/339624133 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x55d4169f79d0 con 0x55d416933fa0 2024-09-10T21:47:57.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.885+0000 7f329e000640 1 -- 172.21.15.7:0/339624133 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3298013810 con 0x55d416933fa0 2024-09-10T21:47:57.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.885+0000 7f329e000640 1 -- 172.21.15.7:0/339624133 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7f32a0060020 con 0x55d416933fa0 2024-09-10T21:47:57.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.885+0000 7f32bac29740 10 client.6978 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:47:57.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.885+0000 7f32bac29740 10 client.6978 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:47:57.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.885+0000 7f32bac29740 20 client.6978 populate_metadata read hostname 'smithi007' 2024-09-10T21:47:57.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.885+0000 7f32bac29740 1 --2- 172.21.15.7:0/339624133 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55d4169306a0 0x55d416942030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.885+0000 7f32bac29740 4 client.6978 mds_command: new command op to 235778 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:47:57.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.885+0000 7f32bac29740 1 -- 172.21.15.7:0/339624133 --> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] -- command(tid 0: {"prefix": "status"}) -- 0x55d4168a3fd0 con 0x55d4169306a0 2024-09-10T21:47:57.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.886+0000 7f32a7e00640 1 --2- 172.21.15.7:0/339624133 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55d4169306a0 0x55d416942030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.887+0000 7f32a7e00640 1 --2- 172.21.15.7:0/339624133 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55d4169306a0 0x55d416942030 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17b5b4f956225698 server_cookie=bd011397af594100 in_seq=0 out_seq=0 2024-09-10T21:47:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.887+0000 7f329e000640 10 client.6978 ms_handle_connect on v2:172.21.15.133:6835/3312604545 2024-09-10T21:47:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.887+0000 7f329e000640 1 -- 172.21.15.7:0/339624133 <== mds.0 v2:172.21.15.133:6835/3312604545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55d4168a3fd0 con 0x55d4169306a0 2024-09-10T21:47:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.887+0000 7f329e000640 10 client.6978 handle_command_reply: tid=0 2024-09-10T21:47:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.887+0000 7f32bac29740 1 client.6978 shutdown 2024-09-10T21:47:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.887+0000 7f32bac29740 1 -- 172.21.15.7:0/339624133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d416933fa0 msgr2=0x55d4169577d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.887+0000 7f32bac29740 1 --2- 172.21.15.7:0/339624133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d416933fa0 0x55d4169577d0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009230 tx=0x7f32a0009260 comp rx=0 tx=0).stop 2024-09-10T21:47:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.887+0000 7f32bac29740 1 -- 172.21.15.7:0/339624133 shutdown_connections 2024-09-10T21:47:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.887+0000 7f32bac29740 1 --2- 172.21.15.7:0/339624133 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55d4169306a0 0x55d416942030 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.887+0000 7f32bac29740 1 --2- 172.21.15.7:0/339624133 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d4169788b0 0x55d416a00b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.887+0000 7f32bac29740 1 --2- 172.21.15.7:0/339624133 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d41693d8e0 0x55d4169f7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.887+0000 7f329e000640 0 client.6978 ms_handle_reset on v2:172.21.15.133:6835/3312604545 2024-09-10T21:47:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.887+0000 7f32bac29740 1 --2- 172.21.15.7:0/339624133 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d416933fa0 0x55d4169577d0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.887+0000 7f32bac29740 1 -- 172.21.15.7:0/339624133 >> 172.21.15.7:0/339624133 conn(0x55d416933a20 msgr2=0x55d4169713d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.887+0000 7f32bac29740 1 -- 172.21.15.7:0/339624133 shutdown_connections 2024-09-10T21:47:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.887+0000 7f32bac29740 1 -- 172.21.15.7:0/339624133 wait complete. 2024-09-10T21:47:57.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.887+0000 7f32bac29740 20 client.6978 trim_cache size 0 max 16384 2024-09-10T21:47:57.196 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:47:57.196 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2029451a-3c26-48c7-b874-0597197278ac", 2024-09-10T21:47:57.197 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-10T21:47:57.197 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 235778, 2024-09-10T21:47:57.197 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T21:47:57.197 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T21:47:57.197 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T21:47:57.197 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 4.7321282680000003, 2024-09-10T21:47:57.197 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 69, 2024-09-10T21:47:57.197 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 88, 2024-09-10T21:47:57.197 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 88, 2024-09-10T21:47:57.197 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 20.951811545000002 2024-09-10T21:47:57.197 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:47:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.888+0000 7f32b7c00640 1 -- 172.21.15.7:0/3459279290 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f328c077f70 msgr2=0x7f328c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.888+0000 7f32b7c00640 1 --2- 172.21.15.7:0/3459279290 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f328c077f70 0x7f328c07a430 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f32a0033000 tx=0x7f32a0004730 comp rx=0 tx=0).stop 2024-09-10T21:47:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.888+0000 7f32b7c00640 1 -- 172.21.15.7:0/3459279290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b006ff50 msgr2=0x7f32b00c3a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.888+0000 7f32b7c00640 1 --2- 172.21.15.7:0/3459279290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b006ff50 0x7f32b00c3a30 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f32a800c780 tx=0x7f32a800cb80 comp rx=0 tx=0).stop 2024-09-10T21:47:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.888+0000 7f32b7c00640 1 -- 172.21.15.7:0/3459279290 shutdown_connections 2024-09-10T21:47:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.888+0000 7f32b7c00640 1 --2- 172.21.15.7:0/3459279290 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f328c077f70 0x7f328c07a430 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.888+0000 7f32b7c00640 1 --2- 172.21.15.7:0/3459279290 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f32b006ff50 0x7f32b00c3a30 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.888+0000 7f32b7c00640 1 --2- 172.21.15.7:0/3459279290 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f32b006e960 0x7f32b01f52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.888+0000 7f32b7c00640 1 --2- 172.21.15.7:0/3459279290 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f32b006e510 0x7f32b01f3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.888+0000 7f32b7c00640 1 -- 172.21.15.7:0/3459279290 >> 172.21.15.7:0/3459279290 conn(0x7f32b00b1660 msgr2=0x7f32b00afab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:57.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.888+0000 7f32b7c00640 1 -- 172.21.15.7:0/3459279290 shutdown_connections 2024-09-10T21:47:57.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:56.888+0000 7f32b7c00640 1 -- 172.21.15.7:0/3459279290 wait complete. 2024-09-10T21:47:57.200 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:47:57.201 INFO:tasks.ceph.mds.d.smithi133.stderr:2024-09-10T21:47:57.196+0000 7f9011000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:47:57.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.364+0000 7f1d4c600640 1 -- 172.21.15.7:0/1664268270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d2c003a00 msgr2=0x7f1d2c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:57.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.364+0000 7f1d4c600640 1 --2- 172.21.15.7:0/1664268270 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d2c003a00 0x7f1d2c003e20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800c610 tx=0x7f1d2c005830 comp rx=0 tx=0).stop 2024-09-10T21:47:57.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.364+0000 7f1d4a600640 1 -- 172.21.15.7:0/1664268270 reap_dead start 2024-09-10T21:47:57.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.364+0000 7f1d4c600640 1 -- 172.21.15.7:0/1664268270 shutdown_connections 2024-09-10T21:47:57.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.364+0000 7f1d4c600640 1 -- 172.21.15.7:0/1664268270 >> 172.21.15.7:0/1664268270 conn(0x7f1d440b16d0 msgr2=0x7f1d440b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:57.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.364+0000 7f1d4c600640 1 -- 172.21.15.7:0/1664268270 shutdown_connections 2024-09-10T21:47:57.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.364+0000 7f1d4c600640 1 -- 172.21.15.7:0/1664268270 wait complete. 2024-09-10T21:47:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.366+0000 7f1d4c600640 1 Processor -- start 2024-09-10T21:47:57.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.366+0000 7f1d4c600640 1 -- start start 2024-09-10T21:47:57.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.367+0000 7f1d4c600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d2c003a00 0x7f1d441f5630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.367+0000 7f1d4c600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d441f47a0 0x7f1d440c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.367+0000 7f1d4c600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d440afd40 0x7f1d440c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.367+0000 7f1d4c600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1d440cd070 con 0x7f1d440afd40 2024-09-10T21:47:57.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.367+0000 7f1d4c600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1d440ccef0 con 0x7f1d441f47a0 2024-09-10T21:47:57.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.367+0000 7f1d4c600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f1d440cd1f0 con 0x7f1d2c003a00 2024-09-10T21:47:57.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.367+0000 7f1d4a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d440afd40 0x7f1d440c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.367+0000 7f1d4a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d440afd40 0x7f1d440c9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49496/0 (socket says 172.21.15.7:49496) 2024-09-10T21:47:57.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.367+0000 7f1d4a600640 1 -- 172.21.15.7:0/130270674 learned_addr learned my addr 172.21.15.7:0/130270674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:57.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.367+0000 7f1d49200640 1 --2- 172.21.15.7:0/130270674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d441f47a0 0x7f1d440c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.368+0000 7f1d49c00640 1 --2- 172.21.15.7:0/130270674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d2c003a00 0x7f1d441f5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.368+0000 7f1d4a600640 1 -- 172.21.15.7:0/130270674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d2c003a00 msgr2=0x7f1d441f5630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:57.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.368+0000 7f1d4a600640 1 --2- 172.21.15.7:0/130270674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d2c003a00 0x7f1d441f5630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.368+0000 7f1d4a600640 1 -- 172.21.15.7:0/130270674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d441f47a0 msgr2=0x7f1d440c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:57.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.368+0000 7f1d4a600640 1 --2- 172.21.15.7:0/130270674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d441f47a0 0x7f1d440c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.368+0000 7f1d4a600640 1 -- 172.21.15.7:0/130270674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d440aef80 con 0x7f1d440afd40 2024-09-10T21:47:57.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.368+0000 7f1d49c00640 1 --2- 172.21.15.7:0/130270674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d2c003a00 0x7f1d441f5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:57.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.368+0000 7f1d4a600640 1 --2- 172.21.15.7:0/130270674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d440afd40 0x7f1d440c9700 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f1d440c3650 tx=0x7f1d3c011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:57.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.368+0000 7f1d41600640 1 -- 172.21.15.7:0/130270674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c004240 con 0x7f1d440afd40 2024-09-10T21:47:57.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.368+0000 7f1d41600640 1 -- 172.21.15.7:0/130270674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d3c004400 con 0x7f1d440afd40 2024-09-10T21:47:57.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.368+0000 7f1d41600640 1 -- 172.21.15.7:0/130270674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c005280 con 0x7f1d440afd40 2024-09-10T21:47:57.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.368+0000 7f1d4c600640 1 -- 172.21.15.7:0/130270674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d440af260 con 0x7f1d440afd40 2024-09-10T21:47:57.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.369+0000 7f1d4c600640 1 -- 172.21.15.7:0/130270674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d440af820 con 0x7f1d440afd40 2024-09-10T21:47:57.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.370+0000 7f1d4c600640 1 -- 172.21.15.7:0/130270674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d441601f0 con 0x7f1d440afd40 2024-09-10T21:47:57.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.371+0000 7f1d41600640 1 -- 172.21.15.7:0/130270674 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1d3c005420 con 0x7f1d440afd40 2024-09-10T21:47:57.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.372+0000 7f1d41600640 1 --2- 172.21.15.7:0/130270674 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1d24077f70 0x7f1d2407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:57.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.372+0000 7f1d41600640 1 -- 172.21.15.7:0/130270674 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f1d3c09a230 con 0x7f1d440afd40 2024-09-10T21:47:57.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.372+0000 7f1d49c00640 1 --2- 172.21.15.7:0/130270674 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1d24077f70 0x7f1d2407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:57.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.373+0000 7f1d49c00640 1 --2- 172.21.15.7:0/130270674 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1d24077f70 0x7f1d2407a430 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1d38033000 tx=0x7f1d38038040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:57.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.376+0000 7f1d41600640 1 -- 172.21.15.7:0/130270674 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1d3c062680 con 0x7f1d440afd40 2024-09-10T21:47:57.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.607+0000 7f1d4c600640 1 -- 172.21.15.7:0/130270674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f1d440c3430 con 0x7f1d440afd40 2024-09-10T21:47:57.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.608+0000 7f1d41600640 1 -- 172.21.15.7:0/130270674 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v69) ==== 53+0+73 (secure 0 0 0) 0x7f1d3c0670b0 con 0x7f1d440afd40 2024-09-10T21:47:57.609 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:47:57.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.610+0000 7f1d4c600640 1 -- 172.21.15.7:0/130270674 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1d24077f70 msgr2=0x7f1d2407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:57.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.610+0000 7f1d4c600640 1 --2- 172.21.15.7:0/130270674 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1d24077f70 0x7f1d2407a430 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f1d38033000 tx=0x7f1d38038040 comp rx=0 tx=0).stop 2024-09-10T21:47:57.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.610+0000 7f1d4c600640 1 -- 172.21.15.7:0/130270674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d440afd40 msgr2=0x7f1d440c9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:57.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.610+0000 7f1d4c600640 1 --2- 172.21.15.7:0/130270674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d440afd40 0x7f1d440c9700 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f1d440c3650 tx=0x7f1d3c011ae0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.610+0000 7f1d4c600640 1 -- 172.21.15.7:0/130270674 shutdown_connections 2024-09-10T21:47:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.610+0000 7f1d4c600640 1 --2- 172.21.15.7:0/130270674 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1d24077f70 0x7f1d2407a430 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.610+0000 7f1d4c600640 1 --2- 172.21.15.7:0/130270674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d441f47a0 0x7f1d440c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.610+0000 7f1d4c600640 1 --2- 172.21.15.7:0/130270674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d440afd40 0x7f1d440c9700 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.610+0000 7f1d4c600640 1 --2- 172.21.15.7:0/130270674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d2c003a00 0x7f1d441f5630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.610+0000 7f1d4c600640 1 -- 172.21.15.7:0/130270674 >> 172.21.15.7:0/130270674 conn(0x7f1d440b16d0 msgr2=0x7f1d440b4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.610+0000 7f1d4c600640 1 -- 172.21.15.7:0/130270674 shutdown_connections 2024-09-10T21:47:57.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57.610+0000 7f1d4c600640 1 -- 172.21.15.7:0/130270674 wait complete. 2024-09-10T21:47:57.627 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:47:57.627 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:47:57.627 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-10T21:47:57.627 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:47:57.627 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:47:57.628 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:47:57.628 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:47:57.628 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:47:57.628 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:47:57.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:57.628 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-10T21:47:57.664 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:47:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:47:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:47:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T21:47:57.664 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:57.665 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:47:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T21:47:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-10T21:47:57.665 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:47:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T21:47:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-10T21:47:57.665 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:47:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-10T21:47:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-10T21:47:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T21:47:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42267sec preferred_lft 42267sec 2024-09-10T21:47:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-10T21:47:57.665 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:57.666 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:47:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-10T21:47:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-10T21:47:57.666 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:47:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff 2024-09-10T21:47:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:47:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:57.666 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::3c3d:dbff:fe02:4c15/64 scope link 2024-09-10T21:47:57.667 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:57.667 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:47:57.667 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether f6:f9:69:5a:86:f5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:47:57.667 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f4f9:69ff:fe5a:86f5/64 scope link 2024-09-10T21:47:57.667 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:57.667 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:47:57.667 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:47:57.667 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::433:5cff:fe44:4a5b/64 scope link 2024-09-10T21:47:57.667 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:47:57.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:57.667 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:47:57.758 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:47:57.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:57.762 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-10T21:47:57.797 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-10T21:47:57.798 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:47:57.798 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T21:47:57.798 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T21:47:57.798 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:47:57.798 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:47:57.798 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:47:57.798 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:47:57.798 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:47:57.798 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:47:57.798 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:47:57.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:47:57.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:57 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:47:57.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:57.998 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:47:58.034 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:47:58.034 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:47:58.034 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:47:58.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:58.035 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-10T21:47:58.129 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:47:58.130 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:47:58.165 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:47:58.165 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:47:58.166 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:47:58.241 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:47:58.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:58.241 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:47:58.371 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:47:58.371 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:47:58.399 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:47:58.399 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:47:58.399 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.133:3300,172.21.15.133:3301" 2024-09-10T21:47:58.399 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@2029451a-3c26-48c7-b874-0597197278ac.cephfs=/ 2024-09-10T21:47:58.399 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.133:3300/172.21.15.133:3301" will pass to kernel 2024-09-10T21:47:58.400 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T21:47:58.400 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:47:58.400 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:47:58.400 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:47:58.400 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:47:58.403 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:47:58.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:58.404 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:47:58.460 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:47:58.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:47:58.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:47:58.550 INFO:teuthology.orchestra.run.smithi007.stdout:client6571 2024-09-10T21:47:58.550 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:47:58.550 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:47:58.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:58.586 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat testdir/hardlink) 2024-09-10T21:47:58.664 INFO:tasks.ceph.mds.d.smithi133.stderr:2024-09-10T21:47:58.663+0000 7f900c000640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-09-10T21:47:58.665 INFO:teuthology.orchestra.run.smithi007.stderr:stat: cannot statx 'testdir/hardlink': Input/output error 2024-09-10T21:47:58.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:47:58.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:58.666 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive 2024-09-10T21:47:58.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.871+0000 7fe681400640 1 -- 172.21.15.7:0/3378388152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe67c0d1280 msgr2=0x7fe67c0b0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.871+0000 7fe681400640 1 --2- 172.21.15.7:0/3378388152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe67c0d1280 0x7fe67c0b0f40 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fe67c08d170 tx=0x7fe66402a410 comp rx=0 tx=0).stop 2024-09-10T21:47:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.871+0000 7fe681400640 1 -- 172.21.15.7:0/3378388152 shutdown_connections 2024-09-10T21:47:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.871+0000 7fe681400640 1 --2- 172.21.15.7:0/3378388152 >> v2:172.21.15.133:3300/0 conn(0x7fe67c0b1480 0x7fe67c0b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.871+0000 7fe681400640 1 --2- 172.21.15.7:0/3378388152 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe67c0d1280 0x7fe67c0b0f40 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.871+0000 7fe681400640 1 -- 172.21.15.7:0/3378388152 >> 172.21.15.7:0/3378388152 conn(0x7fe67c0d3d80 msgr2=0x7fe67c0d4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.871+0000 7fe681400640 1 -- 172.21.15.7:0/3378388152 shutdown_connections 2024-09-10T21:47:58.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.871+0000 7fe681400640 1 -- 172.21.15.7:0/3378388152 wait complete. 2024-09-10T21:47:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.873+0000 7fe681400640 1 Processor -- start 2024-09-10T21:47:58.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.873+0000 7fe681400640 1 -- start start 2024-09-10T21:47:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.874+0000 7fe681400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe67c0b1480 0x7fe67c1f8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:58.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.874+0000 7fe681400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe67c0d0790 0x7fe67c156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.874+0000 7fe681400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe67c0d1280 0x7fe67c15c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.874+0000 7fe681400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c0d4a20 con 0x7fe67c0d0790 2024-09-10T21:47:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.874+0000 7fe681400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c0d48a0 con 0x7fe67c0d1280 2024-09-10T21:47:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.874+0000 7fe681400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe67c0b1e30 con 0x7fe67c0b1480 2024-09-10T21:47:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.874+0000 7fe673e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe67c0d0790 0x7fe67c156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.874+0000 7fe673e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe67c0d0790 0x7fe67c156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49522/0 (socket says 172.21.15.7:49522) 2024-09-10T21:47:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.874+0000 7fe673e00640 1 -- 172.21.15.7:0/3871066121 learned_addr learned my addr 172.21.15.7:0/3871066121 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.874+0000 7fe680a00640 1 --2- 172.21.15.7:0/3871066121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe67c0d1280 0x7fe67c15c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.874+0000 7fe67be00640 1 --2- 172.21.15.7:0/3871066121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe67c0b1480 0x7fe67c1f8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.875+0000 7fe673e00640 1 -- 172.21.15.7:0/3871066121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe67c0b1480 msgr2=0x7fe67c1f8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.875+0000 7fe673e00640 1 --2- 172.21.15.7:0/3871066121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe67c0b1480 0x7fe67c1f8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.875+0000 7fe673e00640 1 -- 172.21.15.7:0/3871066121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe67c0d1280 msgr2=0x7fe67c15c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.875+0000 7fe673e00640 1 --2- 172.21.15.7:0/3871066121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe67c0d1280 0x7fe67c15c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.875+0000 7fe673e00640 1 -- 172.21.15.7:0/3871066121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe67c15d340 con 0x7fe67c0d0790 2024-09-10T21:47:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.875+0000 7fe67be00640 1 --2- 172.21.15.7:0/3871066121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe67c0b1480 0x7fe67c1f8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.875+0000 7fe673e00640 1 --2- 172.21.15.7:0/3871066121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe67c0d0790 0x7fe67c156560 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fe664009780 tx=0x7fe664033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.875+0000 7fe678c00640 1 -- 172.21.15.7:0/3871066121 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664031d60 con 0x7fe67c0d0790 2024-09-10T21:47:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.875+0000 7fe678c00640 1 -- 172.21.15.7:0/3871066121 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe664033e10 con 0x7fe67c0d0790 2024-09-10T21:47:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.875+0000 7fe678c00640 1 -- 172.21.15.7:0/3871066121 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664005260 con 0x7fe67c0d0790 2024-09-10T21:47:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.875+0000 7fe681400640 1 -- 172.21.15.7:0/3871066121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe67c15d620 con 0x7fe67c0d0790 2024-09-10T21:47:58.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.876+0000 7fe681400640 1 -- 172.21.15.7:0/3871066121 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe67c1f82b0 con 0x7fe67c0d0790 2024-09-10T21:47:58.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.878+0000 7fe678c00640 1 -- 172.21.15.7:0/3871066121 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe664005400 con 0x7fe67c0d0790 2024-09-10T21:47:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.879+0000 7fe678c00640 1 --2- 172.21.15.7:0/3871066121 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe658077fa0 0x7fe65807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.879+0000 7fe678c00640 1 -- 172.21.15.7:0/3871066121 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fe6640baea0 con 0x7fe67c0d0790 2024-09-10T21:47:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.879+0000 7fe685a3d740 1 Processor -- start 2024-09-10T21:47:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.879+0000 7fe685a3d740 1 -- start start 2024-09-10T21:47:58.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.879+0000 7fe67be00640 1 --2- 172.21.15.7:0/3871066121 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe658077fa0 0x7fe65807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:58.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.880+0000 7fe67be00640 1 --2- 172.21.15.7:0/3871066121 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe658077fa0 0x7fe65807a460 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe66c000f60 tx=0x7fe66c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:58.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.880+0000 7fe685a3d740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a0a02960 0x5584a0a02d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:58.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.880+0000 7fe685a3d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09bd980 0x5584a0a013f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:58.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.880+0000 7fe685a3d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09e3dd0 0x5584a0a01930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:58.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.880+0000 7fe685a3d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5584a092f780 con 0x5584a09bd980 2024-09-10T21:47:58.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.880+0000 7fe685a3d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5584a091ebb0 con 0x5584a09e3dd0 2024-09-10T21:47:58.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.880+0000 7fe685a3d740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5584a09229c0 con 0x5584a0a02960 2024-09-10T21:47:58.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.881+0000 7fe673e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09bd980 0x5584a0a013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:58.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.881+0000 7fe673e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09bd980 0x5584a0a013f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49536/0 (socket says 172.21.15.7:49536) 2024-09-10T21:47:58.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.881+0000 7fe67be00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a0a02960 0x5584a0a02d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:58.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.881+0000 7fe673e00640 1 -- 172.21.15.7:0/4227816432 learned_addr learned my addr 172.21.15.7:0/4227816432 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:58.882 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.881+0000 7fe680a00640 1 --2- 172.21.15.7:0/4227816432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09e3dd0 0x5584a0a01930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:58.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.881+0000 7fe673e00640 1 -- 172.21.15.7:0/4227816432 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a0a02960 msgr2=0x5584a0a02d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:58.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.881+0000 7fe673e00640 1 --2- 172.21.15.7:0/4227816432 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a0a02960 0x5584a0a02d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.881+0000 7fe673e00640 1 -- 172.21.15.7:0/4227816432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09e3dd0 msgr2=0x5584a0a01930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:58.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.881+0000 7fe673e00640 1 --2- 172.21.15.7:0/4227816432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09e3dd0 0x5584a0a01930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.881+0000 7fe673e00640 1 -- 172.21.15.7:0/4227816432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584a06f97b0 con 0x5584a09bd980 2024-09-10T21:47:58.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.881+0000 7fe673e00640 1 --2- 172.21.15.7:0/4227816432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09bd980 0x5584a0a013f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe6640831f0 tx=0x7fe6640830f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a882f7e7bad2b96 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:58.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.881+0000 7fe66be00640 1 -- 172.21.15.7:0/4227816432 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664048030 con 0x5584a09bd980 2024-09-10T21:47:58.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.882+0000 7fe66be00640 1 -- 172.21.15.7:0/4227816432 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe664083260 con 0x5584a09bd980 2024-09-10T21:47:58.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.882+0000 7fe66be00640 1 -- 172.21.15.7:0/4227816432 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664087c90 con 0x5584a09bd980 2024-09-10T21:47:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.882+0000 7fe685a3d740 1 -- 172.21.15.7:0/4227816432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09bd980 msgr2=0x5584a0a013f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.882+0000 7fe685a3d740 1 --2- 172.21.15.7:0/4227816432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09bd980 0x5584a0a013f0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fe6640831f0 tx=0x7fe6640830f0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.882+0000 7fe685a3d740 1 -- 172.21.15.7:0/4227816432 shutdown_connections 2024-09-10T21:47:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.882+0000 7fe685a3d740 1 --2- 172.21.15.7:0/4227816432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09e3dd0 0x5584a0a01930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.882+0000 7fe685a3d740 1 --2- 172.21.15.7:0/4227816432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09bd980 0x5584a0a013f0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.882+0000 7fe685a3d740 1 --2- 172.21.15.7:0/4227816432 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a0a02960 0x5584a0a02d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.882+0000 7fe685a3d740 1 -- 172.21.15.7:0/4227816432 >> 172.21.15.7:0/4227816432 conn(0x5584a09b8eb0 msgr2=0x5584a09bc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.882+0000 7fe685a3d740 1 -- 172.21.15.7:0/4227816432 shutdown_connections 2024-09-10T21:47:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.882+0000 7fe685a3d740 1 -- 172.21.15.7:0/4227816432 wait complete. 2024-09-10T21:47:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.883+0000 7fe685a3d740 1 -- start start 2024-09-10T21:47:58.885 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.883+0000 7fe685a3d740 1 Processor -- start 2024-09-10T21:47:58.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.884+0000 7fe685a3d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09e3dd0 0x5584a09f7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.884+0000 7fe685a3d740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a0a02960 0x5584a09de920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.884+0000 7fe685a3d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09dee60 0x5584a0a14c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.884+0000 7fe685a3d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5584a068f6f0 con 0x5584a09dee60 2024-09-10T21:47:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.884+0000 7fe685a3d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5584a08c4590 con 0x5584a09e3dd0 2024-09-10T21:47:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.884+0000 7fe685a3d740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5584a092f780 con 0x5584a0a02960 2024-09-10T21:47:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.884+0000 7fe685a3d740 10 client.0 _ll_register_callbacks cb 0x5584a091b4b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:47:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.885+0000 7fe680a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09dee60 0x5584a0a14c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.885+0000 7fe67be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09e3dd0 0x5584a09f7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.885+0000 7fe680a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09dee60 0x5584a0a14c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49542/0 (socket says 172.21.15.7:49542) 2024-09-10T21:47:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.885+0000 7fe680a00640 1 -- 172.21.15.7:0/2037394956 learned_addr learned my addr 172.21.15.7:0/2037394956 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.885+0000 7fe673e00640 1 --2- 172.21.15.7:0/2037394956 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a0a02960 0x5584a09de920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:58.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.885+0000 7fe680a00640 1 -- 172.21.15.7:0/2037394956 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a0a02960 msgr2=0x5584a09de920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:58.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.885+0000 7fe680a00640 1 --2- 172.21.15.7:0/2037394956 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a0a02960 0x5584a09de920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.885+0000 7fe680a00640 1 -- 172.21.15.7:0/2037394956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09e3dd0 msgr2=0x5584a09f7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:58.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.885+0000 7fe680a00640 1 --2- 172.21.15.7:0/2037394956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09e3dd0 0x5584a09f7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.885+0000 7fe680a00640 1 -- 172.21.15.7:0/2037394956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584a0759f40 con 0x5584a09dee60 2024-09-10T21:47:58.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.885+0000 7fe680a00640 1 --2- 172.21.15.7:0/2037394956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09dee60 0x5584a0a14c70 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fe674013d60 tx=0x7fe6740117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86bd26d73d8f94e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:58.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.885+0000 7fe668c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:47:58.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.886+0000 7fe668c00640 1 -- 172.21.15.7:0/2037394956 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674003b70 con 0x5584a09dee60 2024-09-10T21:47:58.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.886+0000 7fe668c00640 1 -- 172.21.15.7:0/2037394956 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fe674003d10 con 0x5584a09dee60 2024-09-10T21:47:58.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.886+0000 7fe685a3d740 1 -- 172.21.15.7:0/2037394956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe664083260 con 0x5584a09dee60 2024-09-10T21:47:58.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.886+0000 7fe668c00640 1 -- 172.21.15.7:0/2037394956 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fe674004050 con 0x5584a09dee60 2024-09-10T21:47:58.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.886+0000 7fe685a3d740 10 client.6999 fetch_fsmap learned FSMap version 69 2024-09-10T21:47:58.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.886+0000 7fe685a3d740 1 -- 172.21.15.7:0/2037394956 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x5584a065e650 con 0x5584a09dee60 2024-09-10T21:47:58.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.886+0000 7fe668c00640 1 -- 172.21.15.7:0/2037394956 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe664083260 con 0x5584a09dee60 2024-09-10T21:47:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.886+0000 7fe668c00640 1 -- 172.21.15.7:0/2037394956 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7fe674027080 con 0x5584a09dee60 2024-09-10T21:47:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.886+0000 7fe685a3d740 10 client.6999 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:47:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.886+0000 7fe685a3d740 10 client.6999 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-10T21:47:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.886+0000 7fe685a3d740 20 client.6999 populate_metadata read hostname 'smithi007' 2024-09-10T21:47:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.886+0000 7fe685a3d740 1 --2- 172.21.15.7:0/2037394956 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x5584a0a11690 0x5584a0a11aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.886+0000 7fe685a3d740 4 client.6999 mds_command: new command op to 235778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:47:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.886+0000 7fe685a3d740 1 -- 172.21.15.7:0/2037394956 --> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584a09316d0 con 0x5584a0a11690 2024-09-10T21:47:58.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.887+0000 7fe67be00640 1 --2- 172.21.15.7:0/2037394956 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x5584a0a11690 0x5584a0a11aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.888+0000 7fe67be00640 1 --2- 172.21.15.7:0/2037394956 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x5584a0a11690 0x5584a0a11aa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a853e1e014d7a49a server_cookie=8628385edc7afbf8 in_seq=0 out_seq=0 2024-09-10T21:47:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.888+0000 7fe668c00640 10 client.6999 ms_handle_connect on v2:172.21.15.133:6835/3312604545 2024-09-10T21:47:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.888+0000 7fe668c00640 1 -- 172.21.15.7:0/2037394956 <== mds.0 v2:172.21.15.133:6835/3312604545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5584a09316d0 con 0x5584a0a11690 2024-09-10T21:47:58.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.888+0000 7fe668c00640 10 client.6999 handle_command_reply: tid=0 2024-09-10T21:47:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.889+0000 7fe685a3d740 1 client.6999 shutdown 2024-09-10T21:47:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.889+0000 7fe685a3d740 1 -- 172.21.15.7:0/2037394956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09dee60 msgr2=0x5584a0a14c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.889+0000 7fe685a3d740 1 --2- 172.21.15.7:0/2037394956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09dee60 0x5584a0a14c70 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fe674013d60 tx=0x7fe6740117b0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.889+0000 7fe685a3d740 1 -- 172.21.15.7:0/2037394956 shutdown_connections 2024-09-10T21:47:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.889+0000 7fe685a3d740 1 --2- 172.21.15.7:0/2037394956 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x5584a0a11690 0x5584a0a11aa0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.889+0000 7fe685a3d740 1 --2- 172.21.15.7:0/2037394956 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09dee60 0x5584a0a14c70 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.889+0000 7fe685a3d740 1 --2- 172.21.15.7:0/2037394956 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a0a02960 0x5584a09de920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.889+0000 7fe685a3d740 1 --2- 172.21.15.7:0/2037394956 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09e3dd0 0x5584a09f7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.889+0000 7fe668c00640 0 client.6999 ms_handle_reset on v2:172.21.15.133:6835/3312604545 2024-09-10T21:47:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.889+0000 7fe685a3d740 1 -- 172.21.15.7:0/2037394956 >> 172.21.15.7:0/2037394956 conn(0x5584a09bd980 msgr2=0x5584a09b7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.889+0000 7fe685a3d740 1 -- 172.21.15.7:0/2037394956 shutdown_connections 2024-09-10T21:47:58.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.889+0000 7fe685a3d740 1 -- 172.21.15.7:0/2037394956 wait complete. 2024-09-10T21:47:58.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.889+0000 7fe685a3d740 20 client.6999 trim_cache size 0 max 16384 2024-09-10T21:47:58.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.908+0000 7fe685a3d740 1 Processor -- start 2024-09-10T21:47:58.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.908+0000 7fe685a3d740 1 -- start start 2024-09-10T21:47:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.909+0000 7fe685a3d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09dee60 0x5584a09b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.909+0000 7fe685a3d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09e3dd0 0x5584a09b80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.909+0000 7fe685a3d740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a0a02960 0x5584a09bc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.909+0000 7fe685a3d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c200f60 con 0x5584a09dee60 2024-09-10T21:47:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.909+0000 7fe685a3d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c0cbb20 con 0x5584a09e3dd0 2024-09-10T21:47:58.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.909+0000 7fe685a3d740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe67c0cae80 con 0x5584a0a02960 2024-09-10T21:47:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.909+0000 7fe67be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09dee60 0x5584a09b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.909+0000 7fe67be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09dee60 0x5584a09b24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49548/0 (socket says 172.21.15.7:49548) 2024-09-10T21:47:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.909+0000 7fe67be00640 1 -- 172.21.15.7:0/1311816861 learned_addr learned my addr 172.21.15.7:0/1311816861 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.909+0000 7fe680a00640 1 --2- 172.21.15.7:0/1311816861 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a0a02960 0x5584a09bc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.909+0000 7fe673e00640 1 --2- 172.21.15.7:0/1311816861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09e3dd0 0x5584a09b80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.909+0000 7fe67be00640 1 -- 172.21.15.7:0/1311816861 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a0a02960 msgr2=0x5584a09bc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.909+0000 7fe67be00640 1 --2- 172.21.15.7:0/1311816861 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a0a02960 0x5584a09bc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.909+0000 7fe67be00640 1 -- 172.21.15.7:0/1311816861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09e3dd0 msgr2=0x5584a09b80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.909+0000 7fe67be00640 1 --2- 172.21.15.7:0/1311816861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09e3dd0 0x5584a09b80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.909+0000 7fe67be00640 1 -- 172.21.15.7:0/1311816861 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584a0a11e10 con 0x5584a09dee60 2024-09-10T21:47:58.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.910+0000 7fe67be00640 1 --2- 172.21.15.7:0/1311816861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09dee60 0x5584a09b24e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x5584a0a11fe0 tx=0x7fe66c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5017ff2bee849f79 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:58.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.910+0000 7fe66b400640 1 -- 172.21.15.7:0/1311816861 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c00e960 con 0x5584a09dee60 2024-09-10T21:47:58.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.910+0000 7fe66b400640 1 -- 172.21.15.7:0/1311816861 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe66c00ebf0 con 0x5584a09dee60 2024-09-10T21:47:58.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.910+0000 7fe66b400640 1 -- 172.21.15.7:0/1311816861 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c00eda0 con 0x5584a09dee60 2024-09-10T21:47:58.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.910+0000 7fe685a3d740 1 -- 172.21.15.7:0/1311816861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09dee60 msgr2=0x5584a09b24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:58.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.910+0000 7fe685a3d740 1 --2- 172.21.15.7:0/1311816861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09dee60 0x5584a09b24e0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x5584a0a11fe0 tx=0x7fe66c011000 comp rx=0 tx=0).stop 2024-09-10T21:47:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.911+0000 7fe685a3d740 1 -- 172.21.15.7:0/1311816861 shutdown_connections 2024-09-10T21:47:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.911+0000 7fe685a3d740 1 --2- 172.21.15.7:0/1311816861 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09e3dd0 0x5584a09b80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.911+0000 7fe685a3d740 1 --2- 172.21.15.7:0/1311816861 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a0a02960 0x5584a09bc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.911+0000 7fe685a3d740 1 --2- 172.21.15.7:0/1311816861 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a09dee60 0x5584a09b24e0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.911+0000 7fe685a3d740 1 -- 172.21.15.7:0/1311816861 >> 172.21.15.7:0/1311816861 conn(0x5584a09bd980 msgr2=0x5584a0a12090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.911+0000 7fe685a3d740 1 -- 172.21.15.7:0/1311816861 shutdown_connections 2024-09-10T21:47:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.911+0000 7fe685a3d740 1 -- 172.21.15.7:0/1311816861 wait complete. 2024-09-10T21:47:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.911+0000 7fe685a3d740 1 -- start start 2024-09-10T21:47:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.911+0000 7fe685a3d740 1 Processor -- start 2024-09-10T21:47:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.912+0000 7fe685a3d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09dee60 0x5584a09dca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.912+0000 7fe685a3d740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a09e3dd0 0x5584a0a7c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.912+0000 7fe685a3d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a0a02960 0x5584a09bc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:58.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.912+0000 7fe685a3d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe67c0cb4d0 con 0x5584a0a02960 2024-09-10T21:47:58.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.912+0000 7fe685a3d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5584a0a83890 con 0x5584a09dee60 2024-09-10T21:47:58.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.912+0000 7fe685a3d740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5584a09b60e0 con 0x5584a09e3dd0 2024-09-10T21:47:58.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.912+0000 7fe685a3d740 10 client.0 _ll_register_callbacks cb 0x7fe674009b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:47:58.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.913+0000 7fe680a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a0a02960 0x5584a09bc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:58.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.913+0000 7fe67be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09dee60 0x5584a09dca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:58.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.913+0000 7fe67be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09dee60 0x5584a09dca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:33514/0 (socket says 172.21.15.7:33514) 2024-09-10T21:47:58.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.913+0000 7fe67be00640 1 -- 172.21.15.7:0/1299492990 learned_addr learned my addr 172.21.15.7:0/1299492990 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:58.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.913+0000 7fe673e00640 1 --2- 172.21.15.7:0/1299492990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a09e3dd0 0x5584a0a7c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:58.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.913+0000 7fe680a00640 1 -- 172.21.15.7:0/1299492990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a09e3dd0 msgr2=0x5584a0a7c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:58.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.913+0000 7fe680a00640 1 --2- 172.21.15.7:0/1299492990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a09e3dd0 0x5584a0a7c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.913+0000 7fe680a00640 1 -- 172.21.15.7:0/1299492990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09dee60 msgr2=0x5584a09dca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:58.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.913+0000 7fe680a00640 1 --2- 172.21.15.7:0/1299492990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09dee60 0x5584a09dca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.913+0000 7fe680a00640 1 -- 172.21.15.7:0/1299492990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584a09e7490 con 0x5584a0a02960 2024-09-10T21:47:58.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.913+0000 7fe67be00640 1 --2- 172.21.15.7:0/1299492990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09dee60 0x5584a09dca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:47:58.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.913+0000 7fe680a00640 1 --2- 172.21.15.7:0/1299492990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a0a02960 0x5584a09bc510 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fe674004020 tx=0x7fe674011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=837c0359f792586e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:58.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.913+0000 7fe668c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:47:58.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.914+0000 7fe668c00640 1 -- 172.21.15.7:0/1299492990 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674003970 con 0x5584a0a02960 2024-09-10T21:47:58.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.914+0000 7fe668c00640 1 -- 172.21.15.7:0/1299492990 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fe674003b10 con 0x5584a0a02960 2024-09-10T21:47:58.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.914+0000 7fe685a3d740 1 -- 172.21.15.7:0/1299492990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe66c00ebf0 con 0x5584a0a02960 2024-09-10T21:47:58.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.914+0000 7fe668c00640 1 -- 172.21.15.7:0/1299492990 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7fe674037050 con 0x5584a0a02960 2024-09-10T21:47:58.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.914+0000 7fe685a3d740 10 client.7005 fetch_fsmap learned FSMap version 69 2024-09-10T21:47:58.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.914+0000 7fe685a3d740 1 -- 172.21.15.7:0/1299492990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x5584a0a835d0 con 0x5584a0a02960 2024-09-10T21:47:58.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.914+0000 7fe668c00640 1 -- 172.21.15.7:0/1299492990 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe66c00ebf0 con 0x5584a0a02960 2024-09-10T21:47:58.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.914+0000 7fe668c00640 1 -- 172.21.15.7:0/1299492990 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7fe67403a020 con 0x5584a0a02960 2024-09-10T21:47:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.914+0000 7fe685a3d740 10 client.7005 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:47:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.914+0000 7fe685a3d740 10 client.7005 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-10T21:47:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.914+0000 7fe685a3d740 20 client.7005 populate_metadata read hostname 'smithi007' 2024-09-10T21:47:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.914+0000 7fe685a3d740 1 --2- 172.21.15.7:0/1299492990 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x5584a09dd420 0x5584a09bca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.914+0000 7fe685a3d740 4 client.7005 mds_command: new command op to 235778 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}] 2024-09-10T21:47:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.914+0000 7fe685a3d740 1 -- 172.21.15.7:0/1299492990 --> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive"]}) -- 0x5584a09ddcc0 con 0x5584a09dd420 2024-09-10T21:47:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.915+0000 7fe67be00640 1 --2- 172.21.15.7:0/1299492990 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x5584a09dd420 0x5584a09bca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.915+0000 7fe67be00640 1 --2- 172.21.15.7:0/1299492990 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x5584a09dd420 0x5584a09bca50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56aa11c594429315 server_cookie=973abe99c6864d2f in_seq=0 out_seq=0 2024-09-10T21:47:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.915+0000 7fe668c00640 10 client.7005 ms_handle_connect on v2:172.21.15.133:6835/3312604545 2024-09-10T21:47:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.916+0000 7fe668c00640 1 -- 172.21.15.7:0/1299492990 <== mds.0 v2:172.21.15.133:6835/3312604545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5584a09ddcc0 con 0x5584a09dd420 2024-09-10T21:47:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.916+0000 7fe668c00640 10 client.7005 handle_command_reply: tid=0 2024-09-10T21:47:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.916+0000 7fe685a3d740 1 client.7005 shutdown 2024-09-10T21:47:58.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.916+0000 7fe685a3d740 1 -- 172.21.15.7:0/1299492990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a0a02960 msgr2=0x5584a09bc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.916+0000 7fe685a3d740 1 --2- 172.21.15.7:0/1299492990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a0a02960 0x5584a09bc510 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fe674004020 tx=0x7fe674011cb0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.916+0000 7fe685a3d740 1 -- 172.21.15.7:0/1299492990 shutdown_connections 2024-09-10T21:47:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.916+0000 7fe685a3d740 1 --2- 172.21.15.7:0/1299492990 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x5584a09dd420 0x5584a09bca50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.916+0000 7fe685a3d740 1 --2- 172.21.15.7:0/1299492990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584a0a02960 0x5584a09bc510 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.916+0000 7fe685a3d740 1 --2- 172.21.15.7:0/1299492990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584a09e3dd0 0x5584a0a7c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.916+0000 7fe685a3d740 1 --2- 172.21.15.7:0/1299492990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584a09dee60 0x5584a09dca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.916+0000 7fe685a3d740 1 -- 172.21.15.7:0/1299492990 >> 172.21.15.7:0/1299492990 conn(0x5584a09bd980 msgr2=0x5584a09b80f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.916+0000 7fe668c00640 0 client.7005 ms_handle_reset on v2:172.21.15.133:6835/3312604545 2024-09-10T21:47:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.916+0000 7fe685a3d740 1 -- 172.21.15.7:0/1299492990 shutdown_connections 2024-09-10T21:47:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.917+0000 7fe685a3d740 1 -- 172.21.15.7:0/1299492990 wait complete. 2024-09-10T21:47:58.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.917+0000 7fe685a3d740 20 client.7005 trim_cache size 0 max 16384 2024-09-10T21:47:58.922 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:47:58.922 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-09-10T21:47:58.922 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "f38c7999-23d8-4083-82e3-33fb9a7274b3", 2024-09-10T21:47:58.922 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-09-10T21:47:58.922 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:47:58.923 INFO:tasks.ceph.mds.d.smithi133.stderr:2024-09-10T21:47:58.917+0000 7f900c000640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-09-10T21:47:49.036385+0000 1=1+0) n(v0 rc2024-09-10T21:47:49.195383+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x55c0215d4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-09-10T21:47:49.036385+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-09-10T21:47:49.195383+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-09-10T21:47:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.917+0000 7fe681400640 1 -- 172.21.15.7:0/3871066121 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe658077fa0 msgr2=0x7fe65807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.917+0000 7fe681400640 1 --2- 172.21.15.7:0/3871066121 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe658077fa0 0x7fe65807a460 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fe66c000f60 tx=0x7fe66c009040 comp rx=0 tx=0).stop 2024-09-10T21:47:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.917+0000 7fe681400640 1 -- 172.21.15.7:0/3871066121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe67c0d0790 msgr2=0x7fe67c156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.917+0000 7fe681400640 1 --2- 172.21.15.7:0/3871066121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe67c0d0790 0x7fe67c156560 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fe664009780 tx=0x7fe664033c20 comp rx=0 tx=0).stop 2024-09-10T21:47:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.917+0000 7fe681400640 1 -- 172.21.15.7:0/3871066121 shutdown_connections 2024-09-10T21:47:58.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.917+0000 7fe681400640 1 --2- 172.21.15.7:0/3871066121 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe658077fa0 0x7fe65807a460 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.917+0000 7fe681400640 1 --2- 172.21.15.7:0/3871066121 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe67c0d1280 0x7fe67c15c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.917+0000 7fe681400640 1 --2- 172.21.15.7:0/3871066121 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe67c0d0790 0x7fe67c156560 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.917+0000 7fe681400640 1 --2- 172.21.15.7:0/3871066121 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe67c0b1480 0x7fe67c1f8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.917+0000 7fe681400640 1 -- 172.21.15.7:0/3871066121 >> 172.21.15.7:0/3871066121 conn(0x7fe67c0d3d80 msgr2=0x7fe67c15eb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.917+0000 7fe681400640 1 -- 172.21.15.7:0/3871066121 shutdown_connections 2024-09-10T21:47:58.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:58.917+0000 7fe681400640 1 -- 172.21.15.7:0/3871066121 wait complete. 2024-09-10T21:47:58.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:58.934 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-10T21:47:59.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.096+0000 7f7b78600640 1 -- 172.21.15.7:0/1382651777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b7006fcf0 msgr2=0x7f7b70073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:59.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.096+0000 7f7b78600640 1 --2- 172.21.15.7:0/1382651777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b7006fcf0 0x7f7b70073d40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c009830 tx=0x7f7b5c02a510 comp rx=0 tx=0).stop 2024-09-10T21:47:59.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.096+0000 7f7b78600640 1 -- 172.21.15.7:0/1382651777 shutdown_connections 2024-09-10T21:47:59.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.096+0000 7f7b78600640 1 --2- 172.21.15.7:0/1382651777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b7006fcf0 0x7f7b70073d40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.096+0000 7f7b78600640 1 --2- 172.21.15.7:0/1382651777 >> v2:172.21.15.133:3300/0 conn(0x7f7b70155fb0 0x7f7b701600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.096+0000 7f7b78600640 1 -- 172.21.15.7:0/1382651777 >> 172.21.15.7:0/1382651777 conn(0x7f7b700b16d0 msgr2=0x7f7b700b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:59.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.096+0000 7f7b78600640 1 -- 172.21.15.7:0/1382651777 shutdown_connections 2024-09-10T21:47:59.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.096+0000 7f7b78600640 1 -- 172.21.15.7:0/1382651777 wait complete. 2024-09-10T21:47:59.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.098+0000 7f7b78600640 1 Processor -- start 2024-09-10T21:47:59.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.098+0000 7f7b78600640 1 -- start start 2024-09-10T21:47:59.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.099+0000 7f7b78600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b7006fcf0 0x7f7b701f8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:59.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.099+0000 7f7b78600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b70074280 0x7f7b700c9490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:59.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.099+0000 7f7b78600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b70155fb0 0x7f7b701f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:59.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.099+0000 7f7b78600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b70156510 con 0x7f7b70074280 2024-09-10T21:47:59.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.099+0000 7f7b78600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7b70156390 con 0x7f7b7006fcf0 2024-09-10T21:47:59.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.099+0000 7f7b78600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f7b7006ec90 con 0x7f7b70155fb0 2024-09-10T21:47:59.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.099+0000 7f7b75200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b70074280 0x7f7b700c9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:59.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.099+0000 7f7b75200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b70074280 0x7f7b700c9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49556/0 (socket says 172.21.15.7:49556) 2024-09-10T21:47:59.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.099+0000 7f7b75200640 1 -- 172.21.15.7:0/4101668642 learned_addr learned my addr 172.21.15.7:0/4101668642 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:59.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.099+0000 7f7b75c00640 1 --2- 172.21.15.7:0/4101668642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b7006fcf0 0x7f7b701f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:59.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.099+0000 7f7b76600640 1 --2- 172.21.15.7:0/4101668642 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b70155fb0 0x7f7b701f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:59.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.099+0000 7f7b75200640 1 -- 172.21.15.7:0/4101668642 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b70155fb0 msgr2=0x7f7b701f4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:59.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.099+0000 7f7b75200640 1 --2- 172.21.15.7:0/4101668642 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b70155fb0 0x7f7b701f4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.099+0000 7f7b75200640 1 -- 172.21.15.7:0/4101668642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b7006fcf0 msgr2=0x7f7b701f8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:59.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.099+0000 7f7b75200640 1 --2- 172.21.15.7:0/4101668642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b7006fcf0 0x7f7b701f8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.099+0000 7f7b75200640 1 -- 172.21.15.7:0/4101668642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b700b3cd0 con 0x7f7b70074280 2024-09-10T21:47:59.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.099+0000 7f7b76600640 1 --2- 172.21.15.7:0/4101668642 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b70155fb0 0x7f7b701f4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:47:59.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.100+0000 7f7b75200640 1 --2- 172.21.15.7:0/4101668642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b70074280 0x7f7b700c9490 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c000c00 tx=0x7f7b5c032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:59.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.100+0000 7f7b6d600640 1 -- 172.21.15.7:0/4101668642 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c040070 con 0x7f7b70074280 2024-09-10T21:47:59.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.100+0000 7f7b6d600640 1 -- 172.21.15.7:0/4101668642 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b5c0038e0 con 0x7f7b70074280 2024-09-10T21:47:59.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.100+0000 7f7b6d600640 1 -- 172.21.15.7:0/4101668642 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c003d30 con 0x7f7b70074280 2024-09-10T21:47:59.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.100+0000 7f7b78600640 1 -- 172.21.15.7:0/4101668642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b700b3fb0 con 0x7f7b70074280 2024-09-10T21:47:59.102 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.100+0000 7f7b78600640 1 -- 172.21.15.7:0/4101668642 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b701f48e0 con 0x7f7b70074280 2024-09-10T21:47:59.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.103+0000 7f7b6d600640 1 -- 172.21.15.7:0/4101668642 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7b5c04d030 con 0x7f7b70074280 2024-09-10T21:47:59.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.104+0000 7f7b6d600640 1 --2- 172.21.15.7:0/4101668642 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b50077f70 0x7f7b5007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:59.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.104+0000 7f7b6d600640 1 -- 172.21.15.7:0/4101668642 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f7b5c0bb4f0 con 0x7f7b70074280 2024-09-10T21:47:59.105 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.104+0000 7f7b7b5eb740 1 Processor -- start 2024-09-10T21:47:59.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.105+0000 7f7b75c00640 1 --2- 172.21.15.7:0/4101668642 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b50077f70 0x7f7b5007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:59.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.105+0000 7f7b7b5eb740 1 -- start start 2024-09-10T21:47:59.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.105+0000 7f7b75c00640 1 --2- 172.21.15.7:0/4101668642 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b50077f70 0x7f7b5007a430 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7b700d06c0 tx=0x7f7b64005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:59.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.105+0000 7f7b7b5eb740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9358130 0x55f8b9358510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.105+0000 7f7b7b5eb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b93132a0 0x55f8b9356c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.105+0000 7f7b7b5eb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9313820 0x55f8b9357150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.105+0000 7f7b7b5eb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f8b92721c0 con 0x55f8b93132a0 2024-09-10T21:47:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.105+0000 7f7b7b5eb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f8b9285870 con 0x55f8b9313820 2024-09-10T21:47:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.105+0000 7f7b7b5eb740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55f8b9274ca0 con 0x55f8b9358130 2024-09-10T21:47:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.106+0000 7f7b75200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b93132a0 0x55f8b9356c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.106+0000 7f7b75200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b93132a0 0x55f8b9356c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49570/0 (socket says 172.21.15.7:49570) 2024-09-10T21:47:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.106+0000 7f7b75200640 1 -- 172.21.15.7:0/2601675157 learned_addr learned my addr 172.21.15.7:0/2601675157 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.106+0000 7f7b75c00640 1 --2- 172.21.15.7:0/2601675157 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9358130 0x55f8b9358510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.106+0000 7f7b76600640 1 --2- 172.21.15.7:0/2601675157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9313820 0x55f8b9357150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.106+0000 7f7b75200640 1 -- 172.21.15.7:0/2601675157 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9358130 msgr2=0x55f8b9358510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.106+0000 7f7b75200640 1 --2- 172.21.15.7:0/2601675157 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9358130 0x55f8b9358510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.106+0000 7f7b75200640 1 -- 172.21.15.7:0/2601675157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9313820 msgr2=0x55f8b9357150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:59.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.106+0000 7f7b75200640 1 --2- 172.21.15.7:0/2601675157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9313820 0x55f8b9357150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.106+0000 7f7b75200640 1 -- 172.21.15.7:0/2601675157 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8b904d7d0 con 0x55f8b93132a0 2024-09-10T21:47:59.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.106+0000 7f7b76600640 1 --2- 172.21.15.7:0/2601675157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9313820 0x55f8b9357150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:59.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.106+0000 7f7b75200640 1 --2- 172.21.15.7:0/2601675157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b93132a0 0x55f8b9356c10 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c035040 tx=0x7f7b5c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb06df547483070 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:59.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.107+0000 7f7b62000640 1 -- 172.21.15.7:0/2601675157 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c083f00 con 0x55f8b93132a0 2024-09-10T21:47:59.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.107+0000 7f7b62000640 1 -- 172.21.15.7:0/2601675157 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b5c088070 con 0x55f8b93132a0 2024-09-10T21:47:59.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.107+0000 7f7b62000640 1 -- 172.21.15.7:0/2601675157 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c08cad0 con 0x55f8b93132a0 2024-09-10T21:47:59.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.107+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/2601675157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b93132a0 msgr2=0x55f8b9356c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:59.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.107+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/2601675157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b93132a0 0x55f8b9356c10 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c035040 tx=0x7f7b5c005f90 comp rx=0 tx=0).stop 2024-09-10T21:47:59.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.107+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/2601675157 shutdown_connections 2024-09-10T21:47:59.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.107+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/2601675157 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9313820 0x55f8b9357150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.107+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/2601675157 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b93132a0 0x55f8b9356c10 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.107+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/2601675157 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9358130 0x55f8b9358510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.107+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/2601675157 >> 172.21.15.7:0/2601675157 conn(0x55f8b930e870 msgr2=0x55f8b9312290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.108+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/2601675157 shutdown_connections 2024-09-10T21:47:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.108+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/2601675157 wait complete. 2024-09-10T21:47:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.108+0000 7f7b7b5eb740 1 -- start start 2024-09-10T21:47:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.109+0000 7f7b7b5eb740 1 Processor -- start 2024-09-10T21:47:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.109+0000 7f7b7b5eb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9313820 0x55f8b9356a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.109+0000 7f7b7b5eb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9358130 0x55f8b9311c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.110+0000 7f7b7b5eb740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9356fd0 0x55f8b93121d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:59.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.110+0000 7f7b7b5eb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55f8b921b410 con 0x55f8b9358130 2024-09-10T21:47:59.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.110+0000 7f7b7b5eb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f8b9278ab0 con 0x55f8b9313820 2024-09-10T21:47:59.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.110+0000 7f7b7b5eb740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55f8b8fe5740 con 0x55f8b9356fd0 2024-09-10T21:47:59.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.110+0000 7f7b7b5eb740 10 client.0 _ll_register_callbacks cb 0x55f8b91f8360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:47:59.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.110+0000 7f7b75200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9358130 0x55f8b9311c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:59.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.110+0000 7f7b75200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9358130 0x55f8b9311c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49576/0 (socket says 172.21.15.7:49576) 2024-09-10T21:47:59.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.110+0000 7f7b75200640 1 -- 172.21.15.7:0/4214568518 learned_addr learned my addr 172.21.15.7:0/4214568518 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:59.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.110+0000 7f7b75c00640 1 --2- 172.21.15.7:0/4214568518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9313820 0x55f8b9356a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:59.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.110+0000 7f7b76600640 1 --2- 172.21.15.7:0/4214568518 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9356fd0 0x55f8b93121d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:59.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.110+0000 7f7b75200640 1 -- 172.21.15.7:0/4214568518 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9356fd0 msgr2=0x55f8b93121d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:59.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.110+0000 7f7b75200640 1 --2- 172.21.15.7:0/4214568518 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9356fd0 0x55f8b93121d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.110+0000 7f7b75200640 1 -- 172.21.15.7:0/4214568518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9313820 msgr2=0x55f8b9356a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:59.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.110+0000 7f7b75200640 1 --2- 172.21.15.7:0/4214568518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9313820 0x55f8b9356a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.110+0000 7f7b75200640 1 -- 172.21.15.7:0/4214568518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8b8fd6970 con 0x55f8b9358130 2024-09-10T21:47:59.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.111+0000 7f7b75200640 1 --2- 172.21.15.7:0/4214568518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9358130 0x55f8b9311c90 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c035040 tx=0x7f7b5c08cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a30a645b7326d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:59.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.111+0000 7f7b56a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:47:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.111+0000 7f7b56a00640 1 -- 172.21.15.7:0/4214568518 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c0bb2e0 con 0x55f8b9358130 2024-09-10T21:47:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.111+0000 7f7b56a00640 1 -- 172.21.15.7:0/4214568518 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f7b5c084b30 con 0x55f8b9358130 2024-09-10T21:47:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.111+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/4214568518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b5c088070 con 0x55f8b9358130 2024-09-10T21:47:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.111+0000 7f7b56a00640 1 -- 172.21.15.7:0/4214568518 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f7b5c083f00 con 0x55f8b9358130 2024-09-10T21:47:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.111+0000 7f7b7b5eb740 10 client.7014 fetch_fsmap learned FSMap version 69 2024-09-10T21:47:59.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.111+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/4214568518 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x55f8b8fb46a0 con 0x55f8b9358130 2024-09-10T21:47:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.111+0000 7f7b56a00640 1 -- 172.21.15.7:0/4214568518 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b5c088070 con 0x55f8b9358130 2024-09-10T21:47:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.111+0000 7f7b56a00640 1 -- 172.21.15.7:0/4214568518 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7f7b5c09a070 con 0x55f8b9358130 2024-09-10T21:47:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.112+0000 7f7b7b5eb740 10 client.7014 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:47:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.112+0000 7f7b7b5eb740 10 client.7014 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-10T21:47:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.112+0000 7f7b7b5eb740 20 client.7014 populate_metadata read hostname 'smithi007' 2024-09-10T21:47:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.112+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/4214568518 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55f8b9327350 0x55f8b9328f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.112+0000 7f7b7b5eb740 4 client.7014 mds_command: new command op to 235778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:47:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.112+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/4214568518 --> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8b92877c0 con 0x55f8b9327350 2024-09-10T21:47:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.112+0000 7f7b75c00640 1 --2- 172.21.15.7:0/4214568518 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55f8b9327350 0x55f8b9328f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.113+0000 7f7b75c00640 1 --2- 172.21.15.7:0/4214568518 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55f8b9327350 0x55f8b9328f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cdd8518332461d8 server_cookie=edb8f6a4ddb180ad in_seq=0 out_seq=0 2024-09-10T21:47:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.113+0000 7f7b56a00640 10 client.7014 ms_handle_connect on v2:172.21.15.133:6835/3312604545 2024-09-10T21:47:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.113+0000 7f7b56a00640 1 -- 172.21.15.7:0/4214568518 <== mds.0 v2:172.21.15.133:6835/3312604545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f8b92877c0 con 0x55f8b9327350 2024-09-10T21:47:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.113+0000 7f7b56a00640 10 client.7014 handle_command_reply: tid=0 2024-09-10T21:47:59.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.114+0000 7f7b7b5eb740 1 client.7014 shutdown 2024-09-10T21:47:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.114+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/4214568518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9358130 msgr2=0x55f8b9311c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.114+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/4214568518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9358130 0x55f8b9311c90 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c035040 tx=0x7f7b5c08cb20 comp rx=0 tx=0).stop 2024-09-10T21:47:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.114+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/4214568518 shutdown_connections 2024-09-10T21:47:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.114+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/4214568518 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55f8b9327350 0x55f8b9328f30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.114+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/4214568518 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9356fd0 0x55f8b93121d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.114+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/4214568518 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9358130 0x55f8b9311c90 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.114+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/4214568518 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9313820 0x55f8b9356a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.114+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/4214568518 >> 172.21.15.7:0/4214568518 conn(0x55f8b93132a0 msgr2=0x55f8b934feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.114+0000 7f7b56a00640 0 client.7014 ms_handle_reset on v2:172.21.15.133:6835/3312604545 2024-09-10T21:47:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.114+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/4214568518 shutdown_connections 2024-09-10T21:47:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.114+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/4214568518 wait complete. 2024-09-10T21:47:59.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.114+0000 7f7b7b5eb740 20 client.7014 trim_cache size 0 max 16384 2024-09-10T21:47:59.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.133+0000 7f7b7b5eb740 1 Processor -- start 2024-09-10T21:47:59.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.133+0000 7f7b7b5eb740 1 -- start start 2024-09-10T21:47:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.133+0000 7f7b7b5eb740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9313820 0x55f8b933c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.133+0000 7f7b7b5eb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9356fd0 0x55f8b9338290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.133+0000 7f7b7b5eb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9358130 0x55f8b93387d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:59.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.133+0000 7f7b7b5eb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b70201420 con 0x55f8b9358130 2024-09-10T21:47:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.133+0000 7f7b7b5eb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7b70200480 con 0x55f8b9356fd0 2024-09-10T21:47:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.133+0000 7f7b7b5eb740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f7b700c7500 con 0x55f8b9313820 2024-09-10T21:47:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.134+0000 7f7b76600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9358130 0x55f8b93387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.134+0000 7f7b76600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9358130 0x55f8b93387d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:49588/0 (socket says 172.21.15.7:49588) 2024-09-10T21:47:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.134+0000 7f7b76600640 1 -- 172.21.15.7:0/4153260020 learned_addr learned my addr 172.21.15.7:0/4153260020 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.134+0000 7f7b75c00640 1 --2- 172.21.15.7:0/4153260020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9313820 0x55f8b933c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.134+0000 7f7b75200640 1 --2- 172.21.15.7:0/4153260020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9356fd0 0x55f8b9338290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.134+0000 7f7b76600640 1 -- 172.21.15.7:0/4153260020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9313820 msgr2=0x55f8b933c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.134+0000 7f7b76600640 1 --2- 172.21.15.7:0/4153260020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9313820 0x55f8b933c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.134+0000 7f7b76600640 1 -- 172.21.15.7:0/4153260020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9356fd0 msgr2=0x55f8b9338290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.134+0000 7f7b76600640 1 --2- 172.21.15.7:0/4153260020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9356fd0 0x55f8b9338290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.134+0000 7f7b76600640 1 -- 172.21.15.7:0/4153260020 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8b9312e90 con 0x55f8b9358130 2024-09-10T21:47:59.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.134+0000 7f7b75200640 1 --2- 172.21.15.7:0/4153260020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9356fd0 0x55f8b9338290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:59.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.135+0000 7f7b76600640 1 --2- 172.21.15.7:0/4153260020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9358130 0x55f8b93387d0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f7b68002160 tx=0x7f7b68011620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45af11653a830445 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:59.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.135+0000 7f7b61600640 1 -- 172.21.15.7:0/4153260020 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68019070 con 0x55f8b9358130 2024-09-10T21:47:59.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.135+0000 7f7b61600640 1 -- 172.21.15.7:0/4153260020 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b68015430 con 0x55f8b9358130 2024-09-10T21:47:59.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.135+0000 7f7b61600640 1 -- 172.21.15.7:0/4153260020 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b68015840 con 0x55f8b9358130 2024-09-10T21:47:59.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.135+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/4153260020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9358130 msgr2=0x55f8b93387d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:59.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.135+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/4153260020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9358130 0x55f8b93387d0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f7b68002160 tx=0x7f7b68011620 comp rx=0 tx=0).stop 2024-09-10T21:47:59.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.135+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/4153260020 shutdown_connections 2024-09-10T21:47:59.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.135+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/4153260020 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9356fd0 0x55f8b9338290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.135+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/4153260020 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9358130 0x55f8b93387d0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.135+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/4153260020 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9313820 0x55f8b933c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.135+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/4153260020 >> 172.21.15.7:0/4153260020 conn(0x55f8b93132a0 msgr2=0x55f8b9366840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:59.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.136+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/4153260020 shutdown_connections 2024-09-10T21:47:59.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.136+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/4153260020 wait complete. 2024-09-10T21:47:59.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.136+0000 7f7b7b5eb740 1 -- start start 2024-09-10T21:47:59.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.136+0000 7f7b7b5eb740 1 Processor -- start 2024-09-10T21:47:59.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.137+0000 7f7b7b5eb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9313820 0x55f8b93ce970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:59.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.137+0000 7f7b7b5eb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9356fd0 0x55f8b93db680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:59.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.137+0000 7f7b7b5eb740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9358130 0x55f8b93d8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:59.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.137+0000 7f7b7b5eb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b701fff30 con 0x55f8b9356fd0 2024-09-10T21:47:59.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.137+0000 7f7b7b5eb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55f8b9308410 con 0x55f8b9313820 2024-09-10T21:47:59.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.137+0000 7f7b7b5eb740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55f8b93dc170 con 0x55f8b9358130 2024-09-10T21:47:59.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.137+0000 7f7b7b5eb740 10 client.0 _ll_register_callbacks cb 0x7f7b64009730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:47:59.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.138+0000 7f7b75200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9356fd0 0x55f8b93db680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:59.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.138+0000 7f7b75c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9313820 0x55f8b93ce970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:59.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.138+0000 7f7b76600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9358130 0x55f8b93d8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:59.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.138+0000 7f7b75c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9313820 0x55f8b93ce970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:33582/0 (socket says 172.21.15.7:33582) 2024-09-10T21:47:59.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.138+0000 7f7b75c00640 1 -- 172.21.15.7:0/1179889493 learned_addr learned my addr 172.21.15.7:0/1179889493 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:47:59.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.138+0000 7f7b75200640 1 -- 172.21.15.7:0/1179889493 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9358130 msgr2=0x55f8b93d8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:59.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.138+0000 7f7b75200640 1 --2- 172.21.15.7:0/1179889493 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9358130 0x55f8b93d8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.138+0000 7f7b75200640 1 -- 172.21.15.7:0/1179889493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9313820 msgr2=0x55f8b93ce970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:47:59.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.138+0000 7f7b75200640 1 --2- 172.21.15.7:0/1179889493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9313820 0x55f8b93ce970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.138+0000 7f7b75200640 1 -- 172.21.15.7:0/1179889493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8b93080e0 con 0x55f8b9356fd0 2024-09-10T21:47:59.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.138+0000 7f7b75c00640 1 --2- 172.21.15.7:0/1179889493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9313820 0x55f8b93ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:47:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.138+0000 7f7b75200640 1 --2- 172.21.15.7:0/1179889493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9356fd0 0x55f8b93db680 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c0b9870 tx=0x7f7b5c082c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e9fc7669d4f860 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.138+0000 7f7b56a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:47:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.139+0000 7f7b56a00640 1 -- 172.21.15.7:0/1179889493 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c082f70 con 0x55f8b9356fd0 2024-09-10T21:47:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.139+0000 7f7b56a00640 1 -- 172.21.15.7:0/1179889493 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f7b5c0b9300 con 0x55f8b9356fd0 2024-09-10T21:47:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.139+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/1179889493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b68015430 con 0x55f8b9356fd0 2024-09-10T21:47:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.139+0000 7f7b56a00640 1 -- 172.21.15.7:0/1179889493 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) ==== 24+0+0 (secure 0 0 0) 0x7f7b5c08cd00 con 0x55f8b9356fd0 2024-09-10T21:47:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.139+0000 7f7b7b5eb740 10 client.7020 fetch_fsmap learned FSMap version 69 2024-09-10T21:47:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.139+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/1179889493 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) -- 0x55f8b93080e0 con 0x55f8b9356fd0 2024-09-10T21:47:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.139+0000 7f7b56a00640 1 -- 172.21.15.7:0/1179889493 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b68015430 con 0x55f8b9356fd0 2024-09-10T21:47:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.139+0000 7f7b56a00640 1 -- 172.21.15.7:0/1179889493 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 69) ==== 3502+0+0 (secure 0 0 0) 0x7f7b5c09a070 con 0x55f8b9356fd0 2024-09-10T21:47:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.139+0000 7f7b7b5eb740 10 client.7020 fetch_fsmap finished waiting for FSMap version 69 2024-09-10T21:47:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.139+0000 7f7b7b5eb740 10 client.7020 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-10T21:47:59.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.139+0000 7f7b7b5eb740 20 client.7020 populate_metadata read hostname 'smithi007' 2024-09-10T21:47:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.139+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/1179889493 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55f8b9369190 0x55f8b9338290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.139+0000 7f7b7b5eb740 4 client.7020 mds_command: new command op to 235778 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:47:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.139+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/1179889493 --> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f8b9283270 con 0x55f8b9369190 2024-09-10T21:47:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.140+0000 7f7b75c00640 1 --2- 172.21.15.7:0/1179889493 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55f8b9369190 0x55f8b9338290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.140+0000 7f7b75c00640 1 --2- 172.21.15.7:0/1179889493 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55f8b9369190 0x55f8b9338290 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aec5c442b89b832f server_cookie=ff04fcf444e613e6 in_seq=0 out_seq=0 2024-09-10T21:47:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.140+0000 7f7b56a00640 10 client.7020 ms_handle_connect on v2:172.21.15.133:6835/3312604545 2024-09-10T21:47:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.140+0000 7f7b56a00640 1 -- 172.21.15.7:0/1179889493 <== mds.0 v2:172.21.15.133:6835/3312604545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+570 (crc 0 0 0) 0x55f8b9283270 con 0x55f8b9369190 2024-09-10T21:47:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.140+0000 7f7b56a00640 10 client.7020 handle_command_reply: tid=0 2024-09-10T21:47:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.140+0000 7f7b7b5eb740 1 client.7020 shutdown 2024-09-10T21:47:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.141+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/1179889493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9356fd0 msgr2=0x55f8b93db680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.141+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/1179889493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9356fd0 0x55f8b93db680 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c0b9870 tx=0x7f7b5c082c70 comp rx=0 tx=0).stop 2024-09-10T21:47:59.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.141+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/1179889493 shutdown_connections 2024-09-10T21:47:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.141+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/1179889493 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55f8b9369190 0x55f8b9338290 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.141+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/1179889493 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55f8b9358130 0x55f8b93d8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.141+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/1179889493 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55f8b9356fd0 0x55f8b93db680 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.141+0000 7f7b7b5eb740 1 --2- 172.21.15.7:0/1179889493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55f8b9313820 0x55f8b93ce970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.141+0000 7f7b56a00640 0 client.7020 ms_handle_reset on v2:172.21.15.133:6835/3312604545 2024-09-10T21:47:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.141+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/1179889493 >> 172.21.15.7:0/1179889493 conn(0x55f8b93132a0 msgr2=0x55f8b9366840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.141+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/1179889493 shutdown_connections 2024-09-10T21:47:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.141+0000 7f7b7b5eb740 1 -- 172.21.15.7:0/1179889493 wait complete. 2024-09-10T21:47:59.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.141+0000 7f7b7b5eb740 20 client.7020 trim_cache size 0 max 16384 2024-09-10T21:47:59.147 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:47:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:47:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-09-10T21:47:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: "f38c7999-23d8-4083-82e3-33fb9a7274b3": { 2024-09-10T21:47:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/dir", 2024-09-10T21:47:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "f38c7999-23d8-4083-82e3-33fb9a7274b3", 2024-09-10T21:47:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive" 2024-09-10T21:47:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:47:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:47:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: "f38c7999-23d8-4083-82e3-33fb9a7274b3": { 2024-09-10T21:47:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:47:58.916341+0000", 2024-09-10T21:47:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/dir", 2024-09-10T21:47:59.148 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:47:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:47:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:47:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:47:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:59.149 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:47:59.149 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:47:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.142+0000 7f7b78600640 1 -- 172.21.15.7:0/4101668642 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b50077f70 msgr2=0x7f7b5007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.142+0000 7f7b78600640 1 --2- 172.21.15.7:0/4101668642 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b50077f70 0x7f7b5007a430 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f7b700d06c0 tx=0x7f7b64005f90 comp rx=0 tx=0).stop 2024-09-10T21:47:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.142+0000 7f7b78600640 1 -- 172.21.15.7:0/4101668642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b70074280 msgr2=0x7f7b700c9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.142+0000 7f7b78600640 1 --2- 172.21.15.7:0/4101668642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b70074280 0x7f7b700c9490 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c000c00 tx=0x7f7b5c032c10 comp rx=0 tx=0).stop 2024-09-10T21:47:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.142+0000 7f7b78600640 1 -- 172.21.15.7:0/4101668642 shutdown_connections 2024-09-10T21:47:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.142+0000 7f7b78600640 1 --2- 172.21.15.7:0/4101668642 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b50077f70 0x7f7b5007a430 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.142+0000 7f7b78600640 1 --2- 172.21.15.7:0/4101668642 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b70155fb0 0x7f7b701f4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.142+0000 7f7b78600640 1 --2- 172.21.15.7:0/4101668642 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b70074280 0x7f7b700c9490 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.142+0000 7f7b78600640 1 --2- 172.21.15.7:0/4101668642 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b7006fcf0 0x7f7b701f8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.142+0000 7f7b78600640 1 -- 172.21.15.7:0/4101668642 >> 172.21.15.7:0/4101668642 conn(0x7f7b700b16d0 msgr2=0x7f7b7006e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:59.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.142+0000 7f7b78600640 1 -- 172.21.15.7:0/4101668642 shutdown_connections 2024-09-10T21:47:59.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:47:59.142+0000 7f7b78600640 1 -- 172.21.15.7:0/4101668642 wait complete. 2024-09-10T21:47:59.159 INFO:tasks.cephfs.filesystem:scrub status for tag:f38c7999-23d8-4083-82e3-33fb9a7274b3 - {'path': '/dir', 'tag': 'f38c7999-23d8-4083-82e3-33fb9a7274b3', 'options': 'recursive'} 2024-09-10T21:48:00.841 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:48:00.846 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:48:02.184 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-10T21:48:02.183+0000 7feb42c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-09-10T21:48:29.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:48:29.160 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-10T21:48:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.338+0000 7fbf03200640 1 -- 172.21.15.7:0/3240798411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee8004d40 msgr2=0x7fbee80051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.338+0000 7fbf03200640 1 --2- 172.21.15.7:0/3240798411 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee8004d40 0x7fbee80051a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fbefc001dc0 tx=0x7fbee8005830 comp rx=0 tx=0).stop 2024-09-10T21:48:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.338+0000 7fbf02800640 1 -- 172.21.15.7:0/3240798411 reap_dead start 2024-09-10T21:48:29.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.338+0000 7fbf03200640 1 -- 172.21.15.7:0/3240798411 shutdown_connections 2024-09-10T21:48:29.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.338+0000 7fbf03200640 1 -- 172.21.15.7:0/3240798411 >> 172.21.15.7:0/3240798411 conn(0x7fbf0406cd00 msgr2=0x7fbf0406d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:29.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.338+0000 7fbf03200640 1 -- 172.21.15.7:0/3240798411 shutdown_connections 2024-09-10T21:48:29.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.338+0000 7fbf03200640 1 -- 172.21.15.7:0/3240798411 wait complete. 2024-09-10T21:48:29.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.340+0000 7fbf03200640 1 Processor -- start 2024-09-10T21:48:29.340 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.340+0000 7fbf03200640 1 -- start start 2024-09-10T21:48:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.341+0000 7fbf03200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee8004d40 0x7fbf04144480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.341+0000 7fbf03200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf0414c830 0x7fbf0415d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.341+0000 7fbf03200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf04159bb0 0x7fbf0415de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.341+0000 7fbf03200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbf040cff40 con 0x7fbee8004d40 2024-09-10T21:48:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.341+0000 7fbf03200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbf040cfdc0 con 0x7fbf04159bb0 2024-09-10T21:48:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.341+0000 7fbf03200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fbf040d00c0 con 0x7fbf0414c830 2024-09-10T21:48:29.343 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.341+0000 7fbf01e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee8004d40 0x7fbf04144480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.341+0000 7fbf01e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee8004d40 0x7fbf04144480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38822/0 (socket says 172.21.15.7:38822) 2024-09-10T21:48:29.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.341+0000 7fbf01e00640 1 -- 172.21.15.7:0/908646643 learned_addr learned my addr 172.21.15.7:0/908646643 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:29.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.341+0000 7fbf02800640 1 --2- 172.21.15.7:0/908646643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf04159bb0 0x7fbf0415de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.341+0000 7fbf01400640 1 --2- 172.21.15.7:0/908646643 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf0414c830 0x7fbf0415d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.341+0000 7fbf01e00640 1 -- 172.21.15.7:0/908646643 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf0414c830 msgr2=0x7fbf0415d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:29.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.341+0000 7fbf01e00640 1 --2- 172.21.15.7:0/908646643 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf0414c830 0x7fbf0415d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.341+0000 7fbf01e00640 1 -- 172.21.15.7:0/908646643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf04159bb0 msgr2=0x7fbf0415de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:29.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.341+0000 7fbf01e00640 1 --2- 172.21.15.7:0/908646643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf04159bb0 0x7fbf0415de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.341+0000 7fbf01e00640 1 -- 172.21.15.7:0/908646643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0414c2e0 con 0x7fbee8004d40 2024-09-10T21:48:29.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.341+0000 7fbf01400640 1 --2- 172.21.15.7:0/908646643 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf0414c830 0x7fbf0415d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:48:29.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.341+0000 7fbf02800640 1 --2- 172.21.15.7:0/908646643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf04159bb0 0x7fbf0415de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:48:29.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.342+0000 7fbf01e00640 1 --2- 172.21.15.7:0/908646643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee8004d40 0x7fbf04144480 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fbef4000fb0 tx=0x7fbef40039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:29.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.342+0000 7fbefa000640 1 -- 172.21.15.7:0/908646643 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4015070 con 0x7fbee8004d40 2024-09-10T21:48:29.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.342+0000 7fbefa000640 1 -- 172.21.15.7:0/908646643 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbef4011900 con 0x7fbee8004d40 2024-09-10T21:48:29.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.342+0000 7fbefa000640 1 -- 172.21.15.7:0/908646643 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4011d30 con 0x7fbee8004d40 2024-09-10T21:48:29.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.342+0000 7fbf03200640 1 -- 172.21.15.7:0/908646643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0414c580 con 0x7fbee8004d40 2024-09-10T21:48:29.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.342+0000 7fbf03200640 1 -- 172.21.15.7:0/908646643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf040a9850 con 0x7fbee8004d40 2024-09-10T21:48:29.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.344+0000 7fbefa000640 1 -- 172.21.15.7:0/908646643 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fbef40103c0 con 0x7fbee8004d40 2024-09-10T21:48:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.345+0000 7fbefa000640 1 --2- 172.21.15.7:0/908646643 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbee4077f70 0x7fbee407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.345+0000 7fbefa000640 1 -- 172.21.15.7:0/908646643 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fbef409aeb0 con 0x7fbee8004d40 2024-09-10T21:48:29.346 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.345+0000 7fbf0b9f1740 1 Processor -- start 2024-09-10T21:48:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.346+0000 7fbf01400640 1 --2- 172.21.15.7:0/908646643 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbee4077f70 0x7fbee407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.346+0000 7fbf0b9f1740 1 -- start start 2024-09-10T21:48:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.346+0000 7fbf01400640 1 --2- 172.21.15.7:0/908646643 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbee4077f70 0x7fbee407a430 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbf040b4930 tx=0x7fbef0005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.346+0000 7fbf0b9f1740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc5282e0 0x55ebbc5286c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.346+0000 7fbf0b9f1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc4e3450 0x55ebbc526dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.346+0000 7fbf0b9f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc4e39d0 0x55ebbc527300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.347+0000 7fbf0b9f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ebbc4421c0 con 0x55ebbc4e39d0 2024-09-10T21:48:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.347+0000 7fbf0b9f1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ebbc455870 con 0x55ebbc4e3450 2024-09-10T21:48:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.347+0000 7fbf0b9f1740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55ebbc444ca0 con 0x55ebbc5282e0 2024-09-10T21:48:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.347+0000 7fbf02800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc4e39d0 0x55ebbc527300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.349 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.347+0000 7fbf01e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc5282e0 0x55ebbc5286c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.347+0000 7fbf02800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc4e39d0 0x55ebbc527300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38836/0 (socket says 172.21.15.7:38836) 2024-09-10T21:48:29.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.347+0000 7fbf01400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc4e3450 0x55ebbc526dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.347+0000 7fbf02800640 1 -- 172.21.15.7:0/2176660468 learned_addr learned my addr 172.21.15.7:0/2176660468 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:29.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.347+0000 7fbf01e00640 1 -- 172.21.15.7:0/2176660468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc4e3450 msgr2=0x55ebbc526dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:29.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.347+0000 7fbf01e00640 1 --2- 172.21.15.7:0/2176660468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc4e3450 0x55ebbc526dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.347+0000 7fbf01e00640 1 -- 172.21.15.7:0/2176660468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc4e39d0 msgr2=0x55ebbc527300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:29.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.347+0000 7fbf01e00640 1 --2- 172.21.15.7:0/2176660468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc4e39d0 0x55ebbc527300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.347+0000 7fbf01e00640 1 -- 172.21.15.7:0/2176660468 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebbc21d7d0 con 0x55ebbc5282e0 2024-09-10T21:48:29.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.347+0000 7fbf02800640 1 --2- 172.21.15.7:0/2176660468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc4e39d0 0x55ebbc527300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:48:29.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.348+0000 7fbf01e00640 1 --2- 172.21.15.7:0/2176660468 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc5282e0 0x55ebbc5286c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbef4063150 tx=0x7fbef4062cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e4f33a2ea530f83 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:29.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.348+0000 7fbeee600640 1 -- 172.21.15.7:0/2176660468 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef401f040 con 0x55ebbc5282e0 2024-09-10T21:48:29.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.348+0000 7fbeee600640 1 -- 172.21.15.7:0/2176660468 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbef4067ca0 con 0x55ebbc5282e0 2024-09-10T21:48:29.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.348+0000 7fbeee600640 1 -- 172.21.15.7:0/2176660468 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef406c6c0 con 0x55ebbc5282e0 2024-09-10T21:48:29.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.348+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/2176660468 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc5282e0 msgr2=0x55ebbc5286c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.348+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/2176660468 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc5282e0 0x55ebbc5286c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbef4063150 tx=0x7fbef4062cd0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.349+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/2176660468 shutdown_connections 2024-09-10T21:48:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.349+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/2176660468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc4e3450 0x55ebbc526dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.349+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/2176660468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc4e39d0 0x55ebbc527300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.349+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/2176660468 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc5282e0 0x55ebbc5286c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.349+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/2176660468 >> 172.21.15.7:0/2176660468 conn(0x55ebbc4dea20 msgr2=0x55ebbc4e2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.349+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/2176660468 shutdown_connections 2024-09-10T21:48:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.349+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/2176660468 wait complete. 2024-09-10T21:48:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.349+0000 7fbf0b9f1740 1 -- start start 2024-09-10T21:48:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.350+0000 7fbf0b9f1740 1 Processor -- start 2024-09-10T21:48:29.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.351+0000 7fbf0b9f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc4e39d0 0x55ebbc53c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.351+0000 7fbf0b9f1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc5282e0 0x55ebbc53ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.351+0000 7fbf0b9f1740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc53f4b0 0x55ebbc526610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.351+0000 7fbf0b9f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ebbc3eb410 con 0x55ebbc4e39d0 2024-09-10T21:48:29.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.351+0000 7fbf0b9f1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55ebbc448ab0 con 0x55ebbc5282e0 2024-09-10T21:48:29.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.351+0000 7fbf0b9f1740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55ebbc1b5740 con 0x55ebbc53f4b0 2024-09-10T21:48:29.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.351+0000 7fbf0b9f1740 10 client.0 _ll_register_callbacks cb 0x55ebbc3c8360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:48:29.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.351+0000 7fbf01e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc4e39d0 0x55ebbc53c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.351+0000 7fbf01e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc4e39d0 0x55ebbc53c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38848/0 (socket says 172.21.15.7:38848) 2024-09-10T21:48:29.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.351+0000 7fbf01e00640 1 -- 172.21.15.7:0/4076465093 learned_addr learned my addr 172.21.15.7:0/4076465093 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:29.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.351+0000 7fbf02800640 1 --2- 172.21.15.7:0/4076465093 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc53f4b0 0x55ebbc526610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.351+0000 7fbf01400640 1 --2- 172.21.15.7:0/4076465093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc5282e0 0x55ebbc53ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.351+0000 7fbf01e00640 1 -- 172.21.15.7:0/4076465093 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc53f4b0 msgr2=0x55ebbc526610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:29.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.351+0000 7fbf01e00640 1 --2- 172.21.15.7:0/4076465093 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc53f4b0 0x55ebbc526610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.351+0000 7fbf01e00640 1 -- 172.21.15.7:0/4076465093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc5282e0 msgr2=0x55ebbc53ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:29.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.351+0000 7fbf01e00640 1 --2- 172.21.15.7:0/4076465093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc5282e0 0x55ebbc53ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.351+0000 7fbf01e00640 1 -- 172.21.15.7:0/4076465093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebbc1a6970 con 0x55ebbc4e39d0 2024-09-10T21:48:29.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.352+0000 7fbf01e00640 1 --2- 172.21.15.7:0/4076465093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc4e39d0 0x55ebbc53c210 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fbef4001990 tx=0x7fbef406a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25428b0dd29be66 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:29.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.352+0000 7fbee2a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:48:29.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.352+0000 7fbee2a00640 1 -- 172.21.15.7:0/4076465093 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef401f070 con 0x55ebbc4e39d0 2024-09-10T21:48:29.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.352+0000 7fbee2a00640 1 -- 172.21.15.7:0/4076465093 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fbef4062420 con 0x55ebbc4e39d0 2024-09-10T21:48:29.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.352+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/4076465093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbef4067ca0 con 0x55ebbc4e39d0 2024-09-10T21:48:29.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.352+0000 7fbee2a00640 1 -- 172.21.15.7:0/4076465093 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fbef4062620 con 0x55ebbc4e39d0 2024-09-10T21:48:29.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.352+0000 7fbf0b9f1740 10 client.7032 fetch_fsmap learned FSMap version 70 2024-09-10T21:48:29.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.352+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/4076465093 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x55ebbc1846a0 con 0x55ebbc4e39d0 2024-09-10T21:48:29.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.352+0000 7fbee2a00640 1 -- 172.21.15.7:0/4076465093 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbef4067ca0 con 0x55ebbc4e39d0 2024-09-10T21:48:29.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.352+0000 7fbee2a00640 1 -- 172.21.15.7:0/4076465093 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7fbef404ab00 con 0x55ebbc4e39d0 2024-09-10T21:48:29.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.352+0000 7fbf0b9f1740 10 client.7032 fetch_fsmap finished waiting for FSMap version 70 2024-09-10T21:48:29.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.352+0000 7fbf0b9f1740 10 client.7032 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-10T21:48:29.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.353+0000 7fbf0b9f1740 20 client.7032 populate_metadata read hostname 'smithi007' 2024-09-10T21:48:29.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.353+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/4076465093 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55ebbc527020 0x55ebbc4fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.353+0000 7fbf0b9f1740 4 client.7032 mds_command: new command op to 235778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:48:29.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.353+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/4076465093 --> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ebbc4577c0 con 0x55ebbc527020 2024-09-10T21:48:29.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.353+0000 7fbf01400640 1 --2- 172.21.15.7:0/4076465093 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55ebbc527020 0x55ebbc4fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.354+0000 7fbf01400640 1 --2- 172.21.15.7:0/4076465093 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55ebbc527020 0x55ebbc4fcdc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48502abaed0b5bd4 server_cookie=dc9a17209e640550 in_seq=0 out_seq=0 2024-09-10T21:48:29.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.354+0000 7fbee2a00640 10 client.7032 ms_handle_connect on v2:172.21.15.133:6835/3312604545 2024-09-10T21:48:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.354+0000 7fbee2a00640 1 -- 172.21.15.7:0/4076465093 <== mds.0 v2:172.21.15.133:6835/3312604545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ebbc4577c0 con 0x55ebbc527020 2024-09-10T21:48:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.354+0000 7fbee2a00640 10 client.7032 handle_command_reply: tid=0 2024-09-10T21:48:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.354+0000 7fbf0b9f1740 1 client.7032 shutdown 2024-09-10T21:48:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.355+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/4076465093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc4e39d0 msgr2=0x55ebbc53c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.355+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/4076465093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc4e39d0 0x55ebbc53c210 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fbef4001990 tx=0x7fbef406a300 comp rx=0 tx=0).stop 2024-09-10T21:48:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.355+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/4076465093 shutdown_connections 2024-09-10T21:48:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.355+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/4076465093 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55ebbc527020 0x55ebbc4fcdc0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.355+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/4076465093 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc5282e0 0x55ebbc53ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.355+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/4076465093 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc53f4b0 0x55ebbc526610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.355+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/4076465093 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc4e39d0 0x55ebbc53c210 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.355+0000 7fbee2a00640 0 client.7032 ms_handle_reset on v2:172.21.15.133:6835/3312604545 2024-09-10T21:48:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.355+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/4076465093 >> 172.21.15.7:0/4076465093 conn(0x55ebbc4e3450 msgr2=0x55ebbc508b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.355+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/4076465093 shutdown_connections 2024-09-10T21:48:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.355+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/4076465093 wait complete. 2024-09-10T21:48:29.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.355+0000 7fbf0b9f1740 20 client.7032 trim_cache size 0 max 16384 2024-09-10T21:48:29.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.373+0000 7fbf0b9f1740 1 Processor -- start 2024-09-10T21:48:29.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.373+0000 7fbf0b9f1740 1 -- start start 2024-09-10T21:48:29.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.374+0000 7fbf0b9f1740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc4e39d0 0x55ebbc4dbc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.374+0000 7fbf0b9f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc5282e0 0x55ebbc4dd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.374+0000 7fbf0b9f1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc53f4b0 0x55ebbc54f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.374+0000 7fbf0b9f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041fd3c0 con 0x55ebbc5282e0 2024-09-10T21:48:29.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.374+0000 7fbf0b9f1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04073d10 con 0x55ebbc53f4b0 2024-09-10T21:48:29.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.374+0000 7fbf0b9f1740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fbf040ab1e0 con 0x55ebbc4e39d0 2024-09-10T21:48:29.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.374+0000 7fbf01400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc5282e0 0x55ebbc4dd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.374+0000 7fbf01400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc5282e0 0x55ebbc4dd750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38858/0 (socket says 172.21.15.7:38858) 2024-09-10T21:48:29.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.374+0000 7fbf01400640 1 -- 172.21.15.7:0/1506271974 learned_addr learned my addr 172.21.15.7:0/1506271974 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:29.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.375+0000 7fbf01e00640 1 --2- 172.21.15.7:0/1506271974 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc4e39d0 0x55ebbc4dbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.375+0000 7fbf01400640 1 -- 172.21.15.7:0/1506271974 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc4e39d0 msgr2=0x55ebbc4dbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:29.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.375+0000 7fbf02800640 1 --2- 172.21.15.7:0/1506271974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc53f4b0 0x55ebbc54f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.375+0000 7fbf01400640 1 --2- 172.21.15.7:0/1506271974 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc4e39d0 0x55ebbc4dbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.375+0000 7fbf01400640 1 -- 172.21.15.7:0/1506271974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc53f4b0 msgr2=0x55ebbc54f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:29.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.375+0000 7fbf01400640 1 --2- 172.21.15.7:0/1506271974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc53f4b0 0x55ebbc54f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.375+0000 7fbf01400640 1 -- 172.21.15.7:0/1506271974 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ebbc5289a0 con 0x55ebbc5282e0 2024-09-10T21:48:29.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.375+0000 7fbf01e00640 1 --2- 172.21.15.7:0/1506271974 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc4e39d0 0x55ebbc4dbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:48:29.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.375+0000 7fbf02800640 1 --2- 172.21.15.7:0/1506271974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc53f4b0 0x55ebbc54f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:48:29.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.375+0000 7fbf01400640 1 --2- 172.21.15.7:0/1506271974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc5282e0 0x55ebbc4dd750 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x55ebbc52a510 tx=0x7fbef001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f49be5907ba9d2f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:29.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.375+0000 7fbeedc00640 1 -- 172.21.15.7:0/1506271974 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0003ac0 con 0x55ebbc5282e0 2024-09-10T21:48:29.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.375+0000 7fbeedc00640 1 -- 172.21.15.7:0/1506271974 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbef0003c60 con 0x55ebbc5282e0 2024-09-10T21:48:29.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.375+0000 7fbeedc00640 1 -- 172.21.15.7:0/1506271974 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef0005020 con 0x55ebbc5282e0 2024-09-10T21:48:29.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.376+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/1506271974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc5282e0 msgr2=0x55ebbc4dd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:29.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.376+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/1506271974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc5282e0 0x55ebbc4dd750 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x55ebbc52a510 tx=0x7fbef001b000 comp rx=0 tx=0).stop 2024-09-10T21:48:29.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.376+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/1506271974 shutdown_connections 2024-09-10T21:48:29.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.376+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/1506271974 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc53f4b0 0x55ebbc54f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.376+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/1506271974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc5282e0 0x55ebbc4dd750 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.376+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/1506271974 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc4e39d0 0x55ebbc4dbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.376+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/1506271974 >> 172.21.15.7:0/1506271974 conn(0x55ebbc4e3450 msgr2=0x55ebbc4f9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:29.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.376+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/1506271974 shutdown_connections 2024-09-10T21:48:29.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.376+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/1506271974 wait complete. 2024-09-10T21:48:29.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.377+0000 7fbf0b9f1740 1 -- start start 2024-09-10T21:48:29.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.377+0000 7fbf0b9f1740 1 Processor -- start 2024-09-10T21:48:29.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.378+0000 7fbf0b9f1740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc4e39d0 0x55ebbc507b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.378+0000 7fbf0b9f1740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc5282e0 0x55ebbc5b30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.378+0000 7fbf0b9f1740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc53f4b0 0x55ebbc5293a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.378+0000 7fbf0b9f1740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55ebbc528c10 con 0x55ebbc53f4b0 2024-09-10T21:48:29.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.378+0000 7fbf0b9f1740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbf040736c0 con 0x55ebbc4e39d0 2024-09-10T21:48:29.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.378+0000 7fbf0b9f1740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55ebbc4577c0 con 0x55ebbc5282e0 2024-09-10T21:48:29.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.378+0000 7fbf0b9f1740 10 client.0 _ll_register_callbacks cb 0x7fbee80036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:48:29.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.378+0000 7fbf02800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc53f4b0 0x55ebbc5293a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.378+0000 7fbf02800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc53f4b0 0x55ebbc5293a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38860/0 (socket says 172.21.15.7:38860) 2024-09-10T21:48:29.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.378+0000 7fbf02800640 1 -- 172.21.15.7:0/965491687 learned_addr learned my addr 172.21.15.7:0/965491687 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:29.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.378+0000 7fbf01400640 1 --2- 172.21.15.7:0/965491687 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc5282e0 0x55ebbc5b30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.378+0000 7fbf01e00640 1 --2- 172.21.15.7:0/965491687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc4e39d0 0x55ebbc507b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.378+0000 7fbf02800640 1 -- 172.21.15.7:0/965491687 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc5282e0 msgr2=0x55ebbc5b30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.378+0000 7fbf02800640 1 --2- 172.21.15.7:0/965491687 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc5282e0 0x55ebbc5b30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.378+0000 7fbf02800640 1 -- 172.21.15.7:0/965491687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc4e39d0 msgr2=0x55ebbc507b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.378+0000 7fbf02800640 1 --2- 172.21.15.7:0/965491687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc4e39d0 0x55ebbc507b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.378+0000 7fbf02800640 1 -- 172.21.15.7:0/965491687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ebbc525830 con 0x55ebbc53f4b0 2024-09-10T21:48:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.379+0000 7fbf02800640 1 --2- 172.21.15.7:0/965491687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc53f4b0 0x55ebbc5293a0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fbefc02acc0 tx=0x7fbefc001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45a043c7233994c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.379+0000 7fbee2a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:48:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.379+0000 7fbee2a00640 1 -- 172.21.15.7:0/965491687 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc006ca0 con 0x55ebbc53f4b0 2024-09-10T21:48:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.379+0000 7fbee2a00640 1 -- 172.21.15.7:0/965491687 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fbefc0060b0 con 0x55ebbc53f4b0 2024-09-10T21:48:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.379+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/965491687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbef0003c60 con 0x55ebbc53f4b0 2024-09-10T21:48:29.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.379+0000 7fbee2a00640 1 -- 172.21.15.7:0/965491687 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7fbefc04d050 con 0x55ebbc53f4b0 2024-09-10T21:48:29.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.379+0000 7fbf0b9f1740 10 client.7038 fetch_fsmap learned FSMap version 70 2024-09-10T21:48:29.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.379+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/965491687 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x55ebbc4fb600 con 0x55ebbc53f4b0 2024-09-10T21:48:29.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.380+0000 7fbee2a00640 1 -- 172.21.15.7:0/965491687 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbef0003c60 con 0x55ebbc53f4b0 2024-09-10T21:48:29.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.380+0000 7fbee2a00640 1 -- 172.21.15.7:0/965491687 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7fbefc050020 con 0x55ebbc53f4b0 2024-09-10T21:48:29.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.380+0000 7fbf0b9f1740 10 client.7038 fetch_fsmap finished waiting for FSMap version 70 2024-09-10T21:48:29.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.380+0000 7fbf0b9f1740 10 client.7038 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-10T21:48:29.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.380+0000 7fbf0b9f1740 20 client.7038 populate_metadata read hostname 'smithi007' 2024-09-10T21:48:29.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.380+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/965491687 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55ebbc5298e0 0x55ebbc539770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.380+0000 7fbf0b9f1740 4 client.7038 mds_command: new command op to 235778 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:48:29.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.380+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/965491687 --> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ebbc453270 con 0x55ebbc5298e0 2024-09-10T21:48:29.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.380+0000 7fbf01e00640 1 --2- 172.21.15.7:0/965491687 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55ebbc5298e0 0x55ebbc539770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.381+0000 7fbf01e00640 1 --2- 172.21.15.7:0/965491687 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55ebbc5298e0 0x55ebbc539770 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bc7689e322e8ce server_cookie=f90546be2ebdc023 in_seq=0 out_seq=0 2024-09-10T21:48:29.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.381+0000 7fbee2a00640 10 client.7038 ms_handle_connect on v2:172.21.15.133:6835/3312604545 2024-09-10T21:48:29.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.381+0000 7fbee2a00640 1 -- 172.21.15.7:0/965491687 <== mds.0 v2:172.21.15.133:6835/3312604545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+377 (crc 0 0 0) 0x55ebbc453270 con 0x55ebbc5298e0 2024-09-10T21:48:29.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.381+0000 7fbee2a00640 10 client.7038 handle_command_reply: tid=0 2024-09-10T21:48:29.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.381+0000 7fbf0b9f1740 1 client.7038 shutdown 2024-09-10T21:48:29.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.381+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/965491687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc53f4b0 msgr2=0x55ebbc5293a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:29.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.381+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/965491687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc53f4b0 0x55ebbc5293a0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fbefc02acc0 tx=0x7fbefc001510 comp rx=0 tx=0).stop 2024-09-10T21:48:29.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.381+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/965491687 shutdown_connections 2024-09-10T21:48:29.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.381+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/965491687 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x55ebbc5298e0 0x55ebbc539770 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.382+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/965491687 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55ebbc53f4b0 0x55ebbc5293a0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.382+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/965491687 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55ebbc5282e0 0x55ebbc5b30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.382+0000 7fbee2a00640 0 client.7038 ms_handle_reset on v2:172.21.15.133:6835/3312604545 2024-09-10T21:48:29.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.382+0000 7fbf0b9f1740 1 --2- 172.21.15.7:0/965491687 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55ebbc4e39d0 0x55ebbc507b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.382+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/965491687 >> 172.21.15.7:0/965491687 conn(0x55ebbc4e3450 msgr2=0x55ebbc53eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:29.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.382+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/965491687 shutdown_connections 2024-09-10T21:48:29.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.382+0000 7fbf0b9f1740 1 -- 172.21.15.7:0/965491687 wait complete. 2024-09-10T21:48:29.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.382+0000 7fbf0b9f1740 20 client.7038 trim_cache size 0 max 16384 2024-09-10T21:48:29.389 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:48:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-09-10T21:48:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {}, 2024-09-10T21:48:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:48:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "f38c7999-23d8-4083-82e3-33fb9a7274b3": { 2024-09-10T21:48:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:47:58.916341+0000", 2024-09-10T21:48:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/dir", 2024-09-10T21:48:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:48:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:48:29.389 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:48:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:48:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:48:29.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:48:29.390 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:48:29.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.382+0000 7fbf03200640 1 -- 172.21.15.7:0/908646643 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbee4077f70 msgr2=0x7fbee407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.382+0000 7fbf03200640 1 --2- 172.21.15.7:0/908646643 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbee4077f70 0x7fbee407a430 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbf040b4930 tx=0x7fbef0005f90 comp rx=0 tx=0).stop 2024-09-10T21:48:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.382+0000 7fbf03200640 1 -- 172.21.15.7:0/908646643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee8004d40 msgr2=0x7fbf04144480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.382+0000 7fbf03200640 1 --2- 172.21.15.7:0/908646643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee8004d40 0x7fbf04144480 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fbef4000fb0 tx=0x7fbef40039c0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.382+0000 7fbf03200640 1 -- 172.21.15.7:0/908646643 shutdown_connections 2024-09-10T21:48:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.382+0000 7fbf03200640 1 --2- 172.21.15.7:0/908646643 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbee4077f70 0x7fbee407a430 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.382+0000 7fbf03200640 1 --2- 172.21.15.7:0/908646643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbf04159bb0 0x7fbf0415de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.382+0000 7fbf03200640 1 --2- 172.21.15.7:0/908646643 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbf0414c830 0x7fbf0415d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.382+0000 7fbf03200640 1 --2- 172.21.15.7:0/908646643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbee8004d40 0x7fbf04144480 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.382+0000 7fbf03200640 1 -- 172.21.15.7:0/908646643 >> 172.21.15.7:0/908646643 conn(0x7fbf0406cd00 msgr2=0x7fbf0415d150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.383+0000 7fbf03200640 1 -- 172.21.15.7:0/908646643 shutdown_connections 2024-09-10T21:48:29.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.383+0000 7fbf03200640 1 -- 172.21.15.7:0/908646643 wait complete. 2024-09-10T21:48:29.399 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:48:29.399 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:48:29.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.558+0000 7f047d400640 1 -- 172.21.15.7:0/4179250502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f047815bc90 msgr2=0x7f04780ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:29.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.558+0000 7f047d400640 1 --2- 172.21.15.7:0/4179250502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f047815bc90 0x7f04780ccef0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f047815fa90 tx=0x7f046802a470 comp rx=0 tx=0).stop 2024-09-10T21:48:29.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.558+0000 7f047d400640 1 -- 172.21.15.7:0/4179250502 shutdown_connections 2024-09-10T21:48:29.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.558+0000 7f047d400640 1 --2- 172.21.15.7:0/4179250502 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f047815fda0 0x7f04781601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.558+0000 7f047d400640 1 --2- 172.21.15.7:0/4179250502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f047815bc90 0x7f04780ccef0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.558+0000 7f047d400640 1 -- 172.21.15.7:0/4179250502 >> 172.21.15.7:0/4179250502 conn(0x7f04780b16d0 msgr2=0x7f04780b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:29.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.558+0000 7f047d400640 1 -- 172.21.15.7:0/4179250502 shutdown_connections 2024-09-10T21:48:29.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.558+0000 7f047d400640 1 -- 172.21.15.7:0/4179250502 wait complete. 2024-09-10T21:48:29.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.559+0000 7f047d400640 1 Processor -- start 2024-09-10T21:48:29.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.560+0000 7f047d400640 1 -- start start 2024-09-10T21:48:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.561+0000 7f047d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04780cd750 0x7f04781f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.561+0000 7f047d400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f047815bc90 0x7f04780ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.561+0000 7f047d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f047815fda0 0x7f04780c9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.561+0000 7f047d400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f047806fe70 con 0x7f047815fda0 2024-09-10T21:48:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.561+0000 7f047d400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f047806fcf0 con 0x7f04780cd750 2024-09-10T21:48:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.561+0000 7f047d400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f047806fff0 con 0x7f047815bc90 2024-09-10T21:48:29.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.561+0000 7f0477200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f047815fda0 0x7f04780c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.561+0000 7f0477200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f047815fda0 0x7f04780c9410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38870/0 (socket says 172.21.15.7:38870) 2024-09-10T21:48:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.561+0000 7f0477200640 1 -- 172.21.15.7:0/2170578872 learned_addr learned my addr 172.21.15.7:0/2170578872 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.561+0000 7f0476800640 1 --2- 172.21.15.7:0/2170578872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04780cd750 0x7f04781f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.561+0000 7f046fe00640 1 --2- 172.21.15.7:0/2170578872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f047815bc90 0x7f04780ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.561+0000 7f0477200640 1 -- 172.21.15.7:0/2170578872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f047815bc90 msgr2=0x7f04780ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.561+0000 7f0477200640 1 --2- 172.21.15.7:0/2170578872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f047815bc90 0x7f04780ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.561+0000 7f0477200640 1 -- 172.21.15.7:0/2170578872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04780cd750 msgr2=0x7f04781f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.561+0000 7f0477200640 1 --2- 172.21.15.7:0/2170578872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04780cd750 0x7f04781f46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.561+0000 7f0477200640 1 -- 172.21.15.7:0/2170578872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04780c6640 con 0x7f047815fda0 2024-09-10T21:48:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.561+0000 7f046fe00640 1 --2- 172.21.15.7:0/2170578872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f047815bc90 0x7f04780ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:48:29.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.562+0000 7f0477200640 1 --2- 172.21.15.7:0/2170578872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f047815fda0 0x7f04780c9410 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f047000e8b0 tx=0x7f047000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:29.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.562+0000 7f046ea00640 1 -- 172.21.15.7:0/2170578872 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470004d20 con 0x7f047815fda0 2024-09-10T21:48:29.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.562+0000 7f046ea00640 1 -- 172.21.15.7:0/2170578872 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f047000ca60 con 0x7f047815fda0 2024-09-10T21:48:29.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.562+0000 7f047d400640 1 -- 172.21.15.7:0/2170578872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04780ca8d0 con 0x7f047815fda0 2024-09-10T21:48:29.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.562+0000 7f046ea00640 1 -- 172.21.15.7:0/2170578872 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470005020 con 0x7f047815fda0 2024-09-10T21:48:29.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.562+0000 7f047d400640 1 -- 172.21.15.7:0/2170578872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04780cae90 con 0x7f047815fda0 2024-09-10T21:48:29.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.563+0000 7f047d400640 1 -- 172.21.15.7:0/2170578872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04780704a0 con 0x7f047815fda0 2024-09-10T21:48:29.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.564+0000 7f046ea00640 1 -- 172.21.15.7:0/2170578872 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f047000f040 con 0x7f047815fda0 2024-09-10T21:48:29.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.565+0000 7f046ea00640 1 --2- 172.21.15.7:0/2170578872 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0454077f70 0x7f045407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.565+0000 7f046ea00640 1 -- 172.21.15.7:0/2170578872 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f047009a310 con 0x7f047815fda0 2024-09-10T21:48:29.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.566+0000 7f0476800640 1 --2- 172.21.15.7:0/2170578872 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0454077f70 0x7f045407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.566+0000 7f0476800640 1 --2- 172.21.15.7:0/2170578872 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0454077f70 0x7f045407a430 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0468034000 tx=0x7f0468004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:29.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.569+0000 7f046ea00640 1 -- 172.21.15.7:0/2170578872 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0470016030 con 0x7f047815fda0 2024-09-10T21:48:29.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.799+0000 7f047d400640 1 -- 172.21.15.7:0/2170578872 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0478069810 con 0x7f047815fda0 2024-09-10T21:48:29.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.799+0000 7f046ea00640 1 -- 172.21.15.7:0/2170578872 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5230 (secure 0 0 0) 0x7f04700626d0 con 0x7f047815fda0 2024-09-10T21:48:29.800 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:48:29.800 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":70,"btime":"2024-09-10T21:48:02:184202+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/978763848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":978763848},{"type":"v1","addr":"172.21.15.7:6835","nonce":978763848}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/269527918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":269527918},{"type":"v1","addr":"172.21.15.7:6833","nonce":269527918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":235775,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/760980985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":760980985},{"type":"v1","addr":"172.21.15.133:6836","nonce":760980985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:47:40.664091+0000","modified":"2024-09-10T21:48:02.157973+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235778":{"gid":235778,"name":"d","rank":0,"incarnation":66,"state":"up:active","state_seq":10,"addr":"172.21.15.133:6837/3312604545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3312604545},{"type":"v1","addr":"172.21.15.133:6837","nonce":3312604545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235778,"qdb_cluster":[235778]},"id":6}]} 2024-09-10T21:48:29.801 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 70 2024-09-10T21:48:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.801+0000 7f047d400640 1 -- 172.21.15.7:0/2170578872 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0454077f70 msgr2=0x7f045407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.801+0000 7f047d400640 1 --2- 172.21.15.7:0/2170578872 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0454077f70 0x7f045407a430 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0468034000 tx=0x7f0468004730 comp rx=0 tx=0).stop 2024-09-10T21:48:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.802+0000 7f047d400640 1 -- 172.21.15.7:0/2170578872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f047815fda0 msgr2=0x7f04780c9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.802+0000 7f047d400640 1 --2- 172.21.15.7:0/2170578872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f047815fda0 0x7f04780c9410 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f047000e8b0 tx=0x7f047000ed70 comp rx=0 tx=0).stop 2024-09-10T21:48:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.802+0000 7f047d400640 1 -- 172.21.15.7:0/2170578872 shutdown_connections 2024-09-10T21:48:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.802+0000 7f047d400640 1 --2- 172.21.15.7:0/2170578872 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0454077f70 0x7f045407a430 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.802+0000 7f047d400640 1 --2- 172.21.15.7:0/2170578872 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f047815fda0 0x7f04780c9410 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.802+0000 7f047d400640 1 --2- 172.21.15.7:0/2170578872 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f047815bc90 0x7f04780ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.802+0000 7f047d400640 1 --2- 172.21.15.7:0/2170578872 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f04780cd750 0x7f04781f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.802+0000 7f047d400640 1 -- 172.21.15.7:0/2170578872 >> 172.21.15.7:0/2170578872 conn(0x7f04780b16d0 msgr2=0x7f0478072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.802+0000 7f047d400640 1 -- 172.21.15.7:0/2170578872 shutdown_connections 2024-09-10T21:48:29.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.802+0000 7f047d400640 1 -- 172.21.15.7:0/2170578872 wait complete. 2024-09-10T21:48:29.819 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-09-10T21:48:29.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.982+0000 7f75ff600640 1 -- 172.21.15.7:0/3069909515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75dc004d40 msgr2=0x7f75dc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:29.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.982+0000 7f75ff600640 1 --2- 172.21.15.7:0/3069909515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75dc004d40 0x7f75dc0051c0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f75f0001e10 tx=0x7f75dc005830 comp rx=0 tx=0).stop 2024-09-10T21:48:29.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.982+0000 7f75fd600640 1 -- 172.21.15.7:0/3069909515 reap_dead start 2024-09-10T21:48:29.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.982+0000 7f75ff600640 1 -- 172.21.15.7:0/3069909515 shutdown_connections 2024-09-10T21:48:29.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.982+0000 7f75ff600640 1 -- 172.21.15.7:0/3069909515 >> 172.21.15.7:0/3069909515 conn(0x7f75f80b16d0 msgr2=0x7f75f80b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:29.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.982+0000 7f75ff600640 1 -- 172.21.15.7:0/3069909515 shutdown_connections 2024-09-10T21:48:29.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.982+0000 7f75ff600640 1 -- 172.21.15.7:0/3069909515 wait complete. 2024-09-10T21:48:29.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.984+0000 7f75ff600640 1 Processor -- start 2024-09-10T21:48:29.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.984+0000 7f75ff600640 1 -- start start 2024-09-10T21:48:29.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.985+0000 7f75ff600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75dc004d40 0x7f75f80c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.985+0000 7f75ff600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75f80c8050 0x7f75f80b5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.985+0000 7f75ff600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75f81529c0 0x7f75f8152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.985+0000 7f75ff600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f75f8160460 con 0x7f75f81529c0 2024-09-10T21:48:29.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.985+0000 7f75ff600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f75f81602e0 con 0x7f75f80c8050 2024-09-10T21:48:29.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.985+0000 7f75ff600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f75f81605e0 con 0x7f75dc004d40 2024-09-10T21:48:29.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.985+0000 7f75fd600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75f81529c0 0x7f75f8152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.985+0000 7f75fd600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75f81529c0 0x7f75f8152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38890/0 (socket says 172.21.15.7:38890) 2024-09-10T21:48:29.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.985+0000 7f75fd600640 1 -- 172.21.15.7:0/266093251 learned_addr learned my addr 172.21.15.7:0/266093251 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:29.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.985+0000 7f75fcc00640 1 --2- 172.21.15.7:0/266093251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75dc004d40 0x7f75f80c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.985+0000 7f75efe00640 1 --2- 172.21.15.7:0/266093251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75f80c8050 0x7f75f80b5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.986+0000 7f75fd600640 1 -- 172.21.15.7:0/266093251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75dc004d40 msgr2=0x7f75f80c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:29.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.986+0000 7f75fd600640 1 --2- 172.21.15.7:0/266093251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75dc004d40 0x7f75f80c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.986+0000 7f75fd600640 1 -- 172.21.15.7:0/266093251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75f80c8050 msgr2=0x7f75f80b5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:29.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.986+0000 7f75fd600640 1 --2- 172.21.15.7:0/266093251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75f80c8050 0x7f75f80b5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.986+0000 7f75fd600640 1 -- 172.21.15.7:0/266093251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f80b2d20 con 0x7f75f81529c0 2024-09-10T21:48:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.986+0000 7f75fd600640 1 --2- 172.21.15.7:0/266093251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75f81529c0 0x7f75f8152e40 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f75f0001f30 tx=0x7f75f0006c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.986+0000 7f75f5600640 1 -- 172.21.15.7:0/266093251 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f0037040 con 0x7f75f81529c0 2024-09-10T21:48:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.986+0000 7f75f5600640 1 -- 172.21.15.7:0/266093251 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75f0003c30 con 0x7f75f81529c0 2024-09-10T21:48:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.986+0000 7f75f5600640 1 -- 172.21.15.7:0/266093251 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f00062d0 con 0x7f75f81529c0 2024-09-10T21:48:29.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.986+0000 7f75ff600640 1 -- 172.21.15.7:0/266093251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f80b59c0 con 0x7f75f81529c0 2024-09-10T21:48:29.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.986+0000 7f75ff600640 1 -- 172.21.15.7:0/266093251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f81f5c80 con 0x7f75f81529c0 2024-09-10T21:48:29.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.989+0000 7f75f5600640 1 -- 172.21.15.7:0/266093251 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f75f0046070 con 0x7f75f81529c0 2024-09-10T21:48:29.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.990+0000 7f75f5600640 1 --2- 172.21.15.7:0/266093251 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f75d4077fa0 0x7f75d407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.990+0000 7f75f5600640 1 -- 172.21.15.7:0/266093251 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f75f00b9920 con 0x7f75f81529c0 2024-09-10T21:48:29.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.990+0000 7f76024b2740 1 Processor -- start 2024-09-10T21:48:29.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.990+0000 7f76024b2740 1 -- start start 2024-09-10T21:48:29.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.990+0000 7f75fcc00640 1 --2- 172.21.15.7:0/266093251 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f75d4077fa0 0x7f75d407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.991+0000 7f76024b2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a6888a90 0x5578a685fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.991+0000 7f76024b2740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a688c920 0x5578a6860230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.991+0000 7f76024b2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a6888e70 0x5578a6860770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.991+0000 7f76024b2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5578a67d7750 con 0x5578a6888e70 2024-09-10T21:48:29.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.991+0000 7f76024b2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5578a67c6b80 con 0x5578a6888a90 2024-09-10T21:48:29.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.991+0000 7f76024b2740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5578a67ca990 con 0x5578a688c920 2024-09-10T21:48:29.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.991+0000 7f75fcc00640 1 --2- 172.21.15.7:0/266093251 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f75d4077fa0 0x7f75d407a460 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009c40 tx=0x7f75e8005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.991+0000 7f75fcc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a6888a90 0x5578a685fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.991+0000 7f75fd600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a6888e70 0x5578a6860770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.991+0000 7f75fcc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a6888a90 0x5578a685fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:41700/0 (socket says 172.21.15.7:41700) 2024-09-10T21:48:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.991+0000 7f75fcc00640 1 -- 172.21.15.7:0/2994475116 learned_addr learned my addr 172.21.15.7:0/2994475116 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.991+0000 7f75efe00640 1 --2- 172.21.15.7:0/2994475116 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a688c920 0x5578a6860230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.992+0000 7f75fd600640 1 -- 172.21.15.7:0/2994475116 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a688c920 msgr2=0x5578a6860230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.992+0000 7f75fd600640 1 --2- 172.21.15.7:0/2994475116 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a688c920 0x5578a6860230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.992+0000 7f75fd600640 1 -- 172.21.15.7:0/2994475116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a6888a90 msgr2=0x5578a685fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.992+0000 7f75fd600640 1 --2- 172.21.15.7:0/2994475116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a6888a90 0x5578a685fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.992+0000 7f75fd600640 1 -- 172.21.15.7:0/2994475116 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a6754ad0 con 0x5578a6888e70 2024-09-10T21:48:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.992+0000 7f75fcc00640 1 --2- 172.21.15.7:0/2994475116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a6888a90 0x5578a685fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:48:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.992+0000 7f75fd600640 1 --2- 172.21.15.7:0/2994475116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a6888e70 0x5578a6860770 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f75f00b7d30 tx=0x7f75f0081aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb7a2a145511a13 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.992+0000 7f75ed200640 1 -- 172.21.15.7:0/2994475116 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f006d1c0 con 0x5578a6888e70 2024-09-10T21:48:29.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.992+0000 7f75ed200640 1 -- 172.21.15.7:0/2994475116 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75f0081760 con 0x5578a6888e70 2024-09-10T21:48:29.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.992+0000 7f75ed200640 1 -- 172.21.15.7:0/2994475116 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f00810f0 con 0x5578a6888e70 2024-09-10T21:48:29.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.993+0000 7f76024b2740 1 -- 172.21.15.7:0/2994475116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a6888e70 msgr2=0x5578a6860770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.993+0000 7f76024b2740 1 --2- 172.21.15.7:0/2994475116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a6888e70 0x5578a6860770 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f75f00b7d30 tx=0x7f75f0081aa0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.993+0000 7f76024b2740 1 -- 172.21.15.7:0/2994475116 shutdown_connections 2024-09-10T21:48:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.993+0000 7f76024b2740 1 --2- 172.21.15.7:0/2994475116 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a6888e70 0x5578a6860770 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.993+0000 7f76024b2740 1 --2- 172.21.15.7:0/2994475116 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a688c920 0x5578a6860230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.993+0000 7f76024b2740 1 --2- 172.21.15.7:0/2994475116 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a6888a90 0x5578a685fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.993+0000 7f76024b2740 1 -- 172.21.15.7:0/2994475116 >> 172.21.15.7:0/2994475116 conn(0x5578a6861ce0 msgr2=0x5578a6865700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.993+0000 7f76024b2740 1 -- 172.21.15.7:0/2994475116 shutdown_connections 2024-09-10T21:48:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.993+0000 7f76024b2740 1 -- 172.21.15.7:0/2994475116 wait complete. 2024-09-10T21:48:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.994+0000 7f76024b2740 1 -- start start 2024-09-10T21:48:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.994+0000 7f76024b2740 1 Processor -- start 2024-09-10T21:48:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.995+0000 7f76024b2740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a6888e70 0x5578a6878b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.995+0000 7f76024b2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a688c920 0x5578a68a15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.995+0000 7f76024b2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 0x5578a68a1b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.995+0000 7f76024b2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5578a6537700 con 0x5578a68916a0 2024-09-10T21:48:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.995+0000 7f76024b2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5578a676c430 con 0x5578a688c920 2024-09-10T21:48:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.995+0000 7f76024b2740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5578a67c6b80 con 0x5578a6888e70 2024-09-10T21:48:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.995+0000 7f76024b2740 10 client.0 _ll_register_callbacks cb 0x5578a6772600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:48:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.995+0000 7f75fd600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 0x5578a68a1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.995+0000 7f75fd600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 0x5578a68a1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38902/0 (socket says 172.21.15.7:38902) 2024-09-10T21:48:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.995+0000 7f75fd600640 1 -- 172.21.15.7:0/1077950021 learned_addr learned my addr 172.21.15.7:0/1077950021 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.995+0000 7f75fcc00640 1 --2- 172.21.15.7:0/1077950021 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a6888e70 0x5578a6878b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.995+0000 7f75efe00640 1 --2- 172.21.15.7:0/1077950021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a688c920 0x5578a68a15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.996+0000 7f75fd600640 1 -- 172.21.15.7:0/1077950021 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a6888e70 msgr2=0x5578a6878b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.996+0000 7f75fd600640 1 --2- 172.21.15.7:0/1077950021 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a6888e70 0x5578a6878b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.996+0000 7f75fd600640 1 -- 172.21.15.7:0/1077950021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a688c920 msgr2=0x5578a68a15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.996+0000 7f75fd600640 1 --2- 172.21.15.7:0/1077950021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a688c920 0x5578a68a15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.996+0000 7f75fd600640 1 -- 172.21.15.7:0/1077950021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a65a17c0 con 0x5578a68916a0 2024-09-10T21:48:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.996+0000 7f75fd600640 1 --2- 172.21.15.7:0/1077950021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 0x5578a68a1b10 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f75f0081950 tx=0x7f75f007e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c633d87fd2c92f26 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.996+0000 7f75e5600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:48:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.996+0000 7f75e5600640 1 -- 172.21.15.7:0/1077950021 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f0003e20 con 0x5578a68916a0 2024-09-10T21:48:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.996+0000 7f75e5600640 1 -- 172.21.15.7:0/1077950021 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f75f007d5a0 con 0x5578a68916a0 2024-09-10T21:48:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.996+0000 7f76024b2740 1 -- 172.21.15.7:0/1077950021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75f0081760 con 0x5578a68916a0 2024-09-10T21:48:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.996+0000 7f75e5600640 1 -- 172.21.15.7:0/1077950021 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f75f0081c50 con 0x5578a68916a0 2024-09-10T21:48:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.996+0000 7f76024b2740 10 client.7059 fetch_fsmap learned FSMap version 70 2024-09-10T21:48:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.996+0000 7f76024b2740 1 -- 172.21.15.7:0/1077950021 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x5578a6601f50 con 0x5578a68916a0 2024-09-10T21:48:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.996+0000 7f75e5600640 1 -- 172.21.15.7:0/1077950021 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75f0081760 con 0x5578a68916a0 2024-09-10T21:48:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.997+0000 7f75e5600640 1 -- 172.21.15.7:0/1077950021 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f75f007d7a0 con 0x5578a68916a0 2024-09-10T21:48:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.997+0000 7f76024b2740 10 client.7059 fetch_fsmap finished waiting for FSMap version 70 2024-09-10T21:48:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.997+0000 7f76024b2740 10 client.7059 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:48:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.997+0000 7f76024b2740 20 client.7059 populate_metadata read hostname 'smithi007' 2024-09-10T21:48:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.997+0000 7f76024b2740 1 --2- 172.21.15.7:0/1077950021 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x5578a68a3990 0x5578a68a3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.997+0000 7f76024b2740 4 client.7059 mds_command: new command op to 235778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:48:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.997+0000 7f76024b2740 1 -- 172.21.15.7:0/1077950021 --> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5578a67d96a0 con 0x5578a68a3990 2024-09-10T21:48:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.997+0000 7f75fcc00640 1 --2- 172.21.15.7:0/1077950021 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x5578a68a3990 0x5578a68a3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.998+0000 7f75fcc00640 1 --2- 172.21.15.7:0/1077950021 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x5578a68a3990 0x5578a68a3d70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fcd673acd3470ec server_cookie=f349f701accaf69f in_seq=0 out_seq=0 2024-09-10T21:48:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.998+0000 7f75e5600640 10 client.7059 ms_handle_connect on v2:172.21.15.133:6835/3312604545 2024-09-10T21:48:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.999+0000 7f75e5600640 1 -- 172.21.15.7:0/1077950021 <== mds.0 v2:172.21.15.133:6835/3312604545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5578a67d96a0 con 0x5578a68a3990 2024-09-10T21:48:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.999+0000 7f75e5600640 10 client.7059 handle_command_reply: tid=0 2024-09-10T21:48:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.999+0000 7f76024b2740 1 client.7059 shutdown 2024-09-10T21:48:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.999+0000 7f76024b2740 1 -- 172.21.15.7:0/1077950021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 msgr2=0x5578a68a1b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.999+0000 7f76024b2740 1 --2- 172.21.15.7:0/1077950021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 0x5578a68a1b10 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f75f0081950 tx=0x7f75f007e150 comp rx=0 tx=0).stop 2024-09-10T21:48:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.999+0000 7f76024b2740 1 -- 172.21.15.7:0/1077950021 shutdown_connections 2024-09-10T21:48:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.999+0000 7f76024b2740 1 --2- 172.21.15.7:0/1077950021 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x5578a68a3990 0x5578a68a3d70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.999+0000 7f76024b2740 1 --2- 172.21.15.7:0/1077950021 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a688c920 0x5578a68a15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.999+0000 7f76024b2740 1 --2- 172.21.15.7:0/1077950021 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 0x5578a68a1b10 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.999+0000 7f76024b2740 1 --2- 172.21.15.7:0/1077950021 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a6888e70 0x5578a6878b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.999+0000 7f75e5600640 0 client.7059 ms_handle_reset on v2:172.21.15.133:6835/3312604545 2024-09-10T21:48:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.999+0000 7f76024b2740 1 -- 172.21.15.7:0/1077950021 >> 172.21.15.7:0/1077950021 conn(0x5578a6888a90 msgr2=0x5578a68635d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.999+0000 7f76024b2740 1 -- 172.21.15.7:0/1077950021 shutdown_connections 2024-09-10T21:48:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.999+0000 7f76024b2740 1 -- 172.21.15.7:0/1077950021 wait complete. 2024-09-10T21:48:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:29.999+0000 7f76024b2740 20 client.7059 trim_cache size 0 max 16384 2024-09-10T21:48:30.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.013+0000 7f76024b2740 1 Processor -- start 2024-09-10T21:48:30.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.013+0000 7f76024b2740 1 -- start start 2024-09-10T21:48:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.014+0000 7f76024b2740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a6888e70 0x5578a68c0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.014+0000 7f76024b2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a688c920 0x5578a685b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.014+0000 7f76024b2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 0x5578a68a7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.014+0000 7f76024b2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f75f80cf2b0 con 0x5578a68916a0 2024-09-10T21:48:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.014+0000 7f76024b2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f75f81f30b0 con 0x5578a688c920 2024-09-10T21:48:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.014+0000 7f76024b2740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f75f80cec60 con 0x5578a6888e70 2024-09-10T21:48:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.015+0000 7f75fd600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 0x5578a68a7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.015+0000 7f75fd600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 0x5578a68a7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38912/0 (socket says 172.21.15.7:38912) 2024-09-10T21:48:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.015+0000 7f75fd600640 1 -- 172.21.15.7:0/273589283 learned_addr learned my addr 172.21.15.7:0/273589283 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.015+0000 7f75fcc00640 1 --2- 172.21.15.7:0/273589283 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a6888e70 0x5578a68c0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.015+0000 7f75efe00640 1 --2- 172.21.15.7:0/273589283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a688c920 0x5578a685b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.015+0000 7f75fd600640 1 -- 172.21.15.7:0/273589283 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a6888e70 msgr2=0x5578a68c0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.015+0000 7f75fd600640 1 --2- 172.21.15.7:0/273589283 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a6888e70 0x5578a68c0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.015+0000 7f75fd600640 1 -- 172.21.15.7:0/273589283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a688c920 msgr2=0x5578a685b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.015+0000 7f75fd600640 1 --2- 172.21.15.7:0/273589283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a688c920 0x5578a685b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.015+0000 7f75fd600640 1 -- 172.21.15.7:0/273589283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5578a6506660 con 0x5578a68916a0 2024-09-10T21:48:30.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.015+0000 7f75fd600640 1 --2- 172.21.15.7:0/273589283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 0x5578a68a7050 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f75f008b150 tx=0x7f75f0001c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b22179a61eefdfe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:30.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.016+0000 7f75e7e00640 1 -- 172.21.15.7:0/273589283 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f0081760 con 0x5578a68916a0 2024-09-10T21:48:30.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.016+0000 7f75e7e00640 1 -- 172.21.15.7:0/273589283 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75f0003e20 con 0x5578a68916a0 2024-09-10T21:48:30.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.016+0000 7f75e7e00640 1 -- 172.21.15.7:0/273589283 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f0086430 con 0x5578a68916a0 2024-09-10T21:48:30.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.016+0000 7f76024b2740 1 -- 172.21.15.7:0/273589283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 msgr2=0x5578a68a7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.016+0000 7f76024b2740 1 --2- 172.21.15.7:0/273589283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 0x5578a68a7050 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f75f008b150 tx=0x7f75f0001c00 comp rx=0 tx=0).stop 2024-09-10T21:48:30.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.016+0000 7f76024b2740 1 -- 172.21.15.7:0/273589283 shutdown_connections 2024-09-10T21:48:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.016+0000 7f76024b2740 1 --2- 172.21.15.7:0/273589283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a688c920 0x5578a685b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.016+0000 7f76024b2740 1 --2- 172.21.15.7:0/273589283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 0x5578a68a7050 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.016+0000 7f76024b2740 1 --2- 172.21.15.7:0/273589283 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a6888e70 0x5578a68c0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.016+0000 7f76024b2740 1 -- 172.21.15.7:0/273589283 >> 172.21.15.7:0/273589283 conn(0x5578a6888a90 msgr2=0x5578a6887050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.016+0000 7f76024b2740 1 -- 172.21.15.7:0/273589283 shutdown_connections 2024-09-10T21:48:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.016+0000 7f76024b2740 1 -- 172.21.15.7:0/273589283 wait complete. 2024-09-10T21:48:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.017+0000 7f76024b2740 1 -- start start 2024-09-10T21:48:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.017+0000 7f76024b2740 1 Processor -- start 2024-09-10T21:48:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.018+0000 7f76024b2740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a6888e70 0x5578a6878980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.018+0000 7f76024b2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a688c920 0x5578a688d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.018+0000 7f76024b2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 0x5578a6928110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.018+0000 7f76024b2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5578a6887b70 con 0x5578a68916a0 2024-09-10T21:48:30.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.018+0000 7f76024b2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5578a67d7750 con 0x5578a688c920 2024-09-10T21:48:30.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.018+0000 7f76024b2740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5578a6892690 con 0x5578a6888e70 2024-09-10T21:48:30.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.018+0000 7f76024b2740 10 client.0 _ll_register_callbacks cb 0x7f75f0006d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:48:30.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.018+0000 7f75fd600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 0x5578a6928110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.018+0000 7f75fd600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 0x5578a6928110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38914/0 (socket says 172.21.15.7:38914) 2024-09-10T21:48:30.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.018+0000 7f75fd600640 1 -- 172.21.15.7:0/1137898387 learned_addr learned my addr 172.21.15.7:0/1137898387 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:30.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.018+0000 7f75efe00640 1 --2- 172.21.15.7:0/1137898387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a688c920 0x5578a688d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.018+0000 7f75fcc00640 1 --2- 172.21.15.7:0/1137898387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a6888e70 0x5578a6878980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.019+0000 7f75fd600640 1 -- 172.21.15.7:0/1137898387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a6888e70 msgr2=0x5578a6878980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.019+0000 7f75fd600640 1 --2- 172.21.15.7:0/1137898387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a6888e70 0x5578a6878980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.019+0000 7f75fd600640 1 -- 172.21.15.7:0/1137898387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a688c920 msgr2=0x5578a688d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.019+0000 7f75fd600640 1 --2- 172.21.15.7:0/1137898387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a688c920 0x5578a688d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.019+0000 7f75fd600640 1 -- 172.21.15.7:0/1137898387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578a6793630 con 0x5578a68916a0 2024-09-10T21:48:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.019+0000 7f75efe00640 1 --2- 172.21.15.7:0/1137898387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a688c920 0x5578a688d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:48:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.019+0000 7f75fd600640 1 --2- 172.21.15.7:0/1137898387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 0x5578a6928110 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f75f0081be0 tx=0x7f75f0086680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147cec35d8fc4047 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.019+0000 7f75e5600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:48:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.019+0000 7f75e5600640 1 -- 172.21.15.7:0/1137898387 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f0081760 con 0x5578a68916a0 2024-09-10T21:48:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.019+0000 7f75e5600640 1 -- 172.21.15.7:0/1137898387 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f75f009b840 con 0x5578a68916a0 2024-09-10T21:48:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.019+0000 7f76024b2740 1 -- 172.21.15.7:0/1137898387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75f0003e20 con 0x5578a68916a0 2024-09-10T21:48:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.019+0000 7f75e5600640 1 -- 172.21.15.7:0/1137898387 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f75f009bac0 con 0x5578a68916a0 2024-09-10T21:48:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.019+0000 7f76024b2740 10 client.7065 fetch_fsmap learned FSMap version 70 2024-09-10T21:48:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.019+0000 7f76024b2740 1 -- 172.21.15.7:0/1137898387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x5578a6793630 con 0x5578a68916a0 2024-09-10T21:48:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.020+0000 7f75e5600640 1 -- 172.21.15.7:0/1137898387 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75f0003e20 con 0x5578a68916a0 2024-09-10T21:48:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.020+0000 7f75e5600640 1 -- 172.21.15.7:0/1137898387 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f75f0080920 con 0x5578a68916a0 2024-09-10T21:48:30.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.020+0000 7f76024b2740 10 client.7065 fetch_fsmap finished waiting for FSMap version 70 2024-09-10T21:48:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.020+0000 7f76024b2740 10 client.7065 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:48:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.020+0000 7f76024b2740 20 client.7065 populate_metadata read hostname 'smithi007' 2024-09-10T21:48:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.020+0000 7f76024b2740 1 --2- 172.21.15.7:0/1137898387 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x5578a6866a60 0x5578a68c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.020+0000 7f76024b2740 4 client.7065 mds_command: new command op to 235778 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-10T21:48:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.020+0000 7f76024b2740 1 -- 172.21.15.7:0/1137898387 --> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x5578a67d5150 con 0x5578a6866a60 2024-09-10T21:48:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.020+0000 7f75fcc00640 1 --2- 172.21.15.7:0/1137898387 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x5578a6866a60 0x5578a68c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.021+0000 7f75fcc00640 1 --2- 172.21.15.7:0/1137898387 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x5578a6866a60 0x5578a68c01e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aecaa998bb60d010 server_cookie=9980ae77b87c3fb0 in_seq=0 out_seq=0 2024-09-10T21:48:30.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.021+0000 7f75e5600640 10 client.7065 ms_handle_connect on v2:172.21.15.133:6835/3312604545 2024-09-10T21:48:30.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.021+0000 7f75e5600640 1 -- 172.21.15.7:0/1137898387 <== mds.0 v2:172.21.15.133:6835/3312604545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+420 (crc 0 0 0) 0x5578a67d5150 con 0x5578a6866a60 2024-09-10T21:48:30.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.021+0000 7f75e5600640 10 client.7065 handle_command_reply: tid=0 2024-09-10T21:48:30.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.021+0000 7f76024b2740 1 client.7065 shutdown 2024-09-10T21:48:30.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.021+0000 7f76024b2740 1 -- 172.21.15.7:0/1137898387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 msgr2=0x5578a6928110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.021+0000 7f76024b2740 1 --2- 172.21.15.7:0/1137898387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 0x5578a6928110 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f75f0081be0 tx=0x7f75f0086680 comp rx=0 tx=0).stop 2024-09-10T21:48:30.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.021+0000 7f76024b2740 1 -- 172.21.15.7:0/1137898387 shutdown_connections 2024-09-10T21:48:30.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.021+0000 7f76024b2740 1 --2- 172.21.15.7:0/1137898387 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x5578a6866a60 0x5578a68c01e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.021+0000 7f76024b2740 1 --2- 172.21.15.7:0/1137898387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5578a688c920 0x5578a688d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.021+0000 7f76024b2740 1 --2- 172.21.15.7:0/1137898387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5578a68916a0 0x5578a6928110 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.021+0000 7f76024b2740 1 --2- 172.21.15.7:0/1137898387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5578a6888e70 0x5578a6878980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.021+0000 7f75e5600640 0 client.7065 ms_handle_reset on v2:172.21.15.133:6835/3312604545 2024-09-10T21:48:30.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.021+0000 7f76024b2740 1 -- 172.21.15.7:0/1137898387 >> 172.21.15.7:0/1137898387 conn(0x5578a6888a90 msgr2=0x5578a6865550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:30.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.021+0000 7f76024b2740 1 -- 172.21.15.7:0/1137898387 shutdown_connections 2024-09-10T21:48:30.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.021+0000 7f76024b2740 1 -- 172.21.15.7:0/1137898387 wait complete. 2024-09-10T21:48:30.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.022+0000 7f76024b2740 20 client.7065 trim_cache size 0 max 16384 2024-09-10T21:48:30.028 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:48:30.028 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-09-10T21:48:30.028 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:48:30.028 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-09-10T21:48:30.028 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1189945810, 2024-09-10T21:48:30.028 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627777, 2024-09-10T21:48:30.028 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/testdir/hardlink" 2024-09-10T21:48:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:48:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:48:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-09-10T21:48:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3330918035, 2024-09-10T21:48:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627776, 2024-09-10T21:48:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/dir" 2024-09-10T21:48:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:48:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:48:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "dir_frag", 2024-09-10T21:48:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 3801675658, 2024-09-10T21:48:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627776, 2024-09-10T21:48:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: "frag": "*", 2024-09-10T21:48:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/dir" 2024-09-10T21:48:30.029 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:48:30.030 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-09-10T21:48:30.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.022+0000 7f75ff600640 1 -- 172.21.15.7:0/266093251 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f75d4077fa0 msgr2=0x7f75d407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.022+0000 7f75ff600640 1 --2- 172.21.15.7:0/266093251 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f75d4077fa0 0x7f75d407a460 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f75e8009c40 tx=0x7f75e8005f90 comp rx=0 tx=0).stop 2024-09-10T21:48:30.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.022+0000 7f75ff600640 1 -- 172.21.15.7:0/266093251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75f81529c0 msgr2=0x7f75f8152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.022+0000 7f75ff600640 1 --2- 172.21.15.7:0/266093251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75f81529c0 0x7f75f8152e40 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f75f0001f30 tx=0x7f75f0006c40 comp rx=0 tx=0).stop 2024-09-10T21:48:30.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.022+0000 7f75ff600640 1 -- 172.21.15.7:0/266093251 shutdown_connections 2024-09-10T21:48:30.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.022+0000 7f75ff600640 1 --2- 172.21.15.7:0/266093251 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f75d4077fa0 0x7f75d407a460 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.022+0000 7f75ff600640 1 --2- 172.21.15.7:0/266093251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f75f80c8050 0x7f75f80b5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.022+0000 7f75ff600640 1 --2- 172.21.15.7:0/266093251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f75f81529c0 0x7f75f8152e40 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.022+0000 7f75ff600640 1 --2- 172.21.15.7:0/266093251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f75dc004d40 0x7f75f80c97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.022+0000 7f75ff600640 1 -- 172.21.15.7:0/266093251 >> 172.21.15.7:0/266093251 conn(0x7f75f80b16d0 msgr2=0x7f75f815ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:30.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.022+0000 7f75ff600640 1 -- 172.21.15.7:0/266093251 shutdown_connections 2024-09-10T21:48:30.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.022+0000 7f75ff600640 1 -- 172.21.15.7:0/266093251 wait complete. 2024-09-10T21:48:30.038 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:48:30.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.196+0000 7fb4b8400640 1 -- 172.21.15.7:0/495514814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4b015bc90 msgr2=0x7fb4b00ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.196+0000 7fb4b8400640 1 --2- 172.21.15.7:0/495514814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4b015bc90 0x7fb4b00ccef0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4009860 tx=0x7fb4a402a480 comp rx=0 tx=0).stop 2024-09-10T21:48:30.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.197+0000 7fb4b8400640 1 -- 172.21.15.7:0/495514814 shutdown_connections 2024-09-10T21:48:30.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.197+0000 7fb4b8400640 1 --2- 172.21.15.7:0/495514814 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4b015bc90 0x7fb4b00ccef0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.197+0000 7fb4b8400640 1 -- 172.21.15.7:0/495514814 >> 172.21.15.7:0/495514814 conn(0x7fb4b00b16d0 msgr2=0x7fb4b00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:30.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.197+0000 7fb4b8400640 1 -- 172.21.15.7:0/495514814 shutdown_connections 2024-09-10T21:48:30.198 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.197+0000 7fb4b8400640 1 -- 172.21.15.7:0/495514814 wait complete. 2024-09-10T21:48:30.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.198+0000 7fb4b8400640 1 Processor -- start 2024-09-10T21:48:30.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.198+0000 7fb4b8400640 1 -- start start 2024-09-10T21:48:30.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.199+0000 7fb4b8400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4b00cd750 0x7fb4b01f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.199+0000 7fb4b8400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4b015bc90 0x7fb4b00c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.199+0000 7fb4b8400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4b015fda0 0x7fb4b00b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.199+0000 7fb4b8400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b0070610 con 0x7fb4b00cd750 2024-09-10T21:48:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.199+0000 7fb4b8400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b00cf570 con 0x7fb4b015fda0 2024-09-10T21:48:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.199+0000 7fb4b8400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fb4b00c9550 con 0x7fb4b015bc90 2024-09-10T21:48:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.199+0000 7fb4b5a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4b00cd750 0x7fb4b01f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.199+0000 7fb4b5a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4b00cd750 0x7fb4b01f32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38920/0 (socket says 172.21.15.7:38920) 2024-09-10T21:48:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.199+0000 7fb4b5a00640 1 -- 172.21.15.7:0/372956315 learned_addr learned my addr 172.21.15.7:0/372956315 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.200+0000 7fb4b6400640 1 --2- 172.21.15.7:0/372956315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4b015fda0 0x7fb4b00b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.200+0000 7fb4b5000640 1 --2- 172.21.15.7:0/372956315 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4b015bc90 0x7fb4b00c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.200+0000 7fb4b5a00640 1 -- 172.21.15.7:0/372956315 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4b015bc90 msgr2=0x7fb4b00c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.200+0000 7fb4b5a00640 1 --2- 172.21.15.7:0/372956315 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4b015bc90 0x7fb4b00c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.200+0000 7fb4b5a00640 1 -- 172.21.15.7:0/372956315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4b015fda0 msgr2=0x7fb4b00b0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.200+0000 7fb4b5a00640 1 --2- 172.21.15.7:0/372956315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4b015fda0 0x7fb4b00b0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.200+0000 7fb4b5a00640 1 -- 172.21.15.7:0/372956315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b00c81e0 con 0x7fb4b00cd750 2024-09-10T21:48:30.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.200+0000 7fb4b5a00640 1 --2- 172.21.15.7:0/372956315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4b00cd750 0x7fb4b01f32a0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb4a402aa70 tx=0x7fb4a4032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:30.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.200+0000 7fb4ad600640 1 -- 172.21.15.7:0/372956315 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4040070 con 0x7fb4b00cd750 2024-09-10T21:48:30.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.200+0000 7fb4ad600640 1 -- 172.21.15.7:0/372956315 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4a4005310 con 0x7fb4b00cd750 2024-09-10T21:48:30.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.201+0000 7fb4ad600640 1 -- 172.21.15.7:0/372956315 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a40047a0 con 0x7fb4b00cd750 2024-09-10T21:48:30.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.201+0000 7fb4b8400640 1 -- 172.21.15.7:0/372956315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b00c84b0 con 0x7fb4b00cd750 2024-09-10T21:48:30.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.201+0000 7fb4b8400640 1 -- 172.21.15.7:0/372956315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b00c8a70 con 0x7fb4b00cd750 2024-09-10T21:48:30.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.202+0000 7fb4b8400640 1 -- 172.21.15.7:0/372956315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b0160560 con 0x7fb4b00cd750 2024-09-10T21:48:30.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.203+0000 7fb4ad600640 1 -- 172.21.15.7:0/372956315 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb4a404c030 con 0x7fb4b00cd750 2024-09-10T21:48:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.204+0000 7fb4ad600640 1 --2- 172.21.15.7:0/372956315 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb490077f70 0x7fb49007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.204+0000 7fb4ad600640 1 -- 172.21.15.7:0/372956315 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7fb4a40badb0 con 0x7fb4b00cd750 2024-09-10T21:48:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.204+0000 7fb4b5000640 1 --2- 172.21.15.7:0/372956315 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb490077f70 0x7fb49007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.205+0000 7fb4b5000640 1 --2- 172.21.15.7:0/372956315 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb490077f70 0x7fb49007a430 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb49c007fd0 tx=0x7fb49c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:30.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.208+0000 7fb4ad600640 1 -- 172.21.15.7:0/372956315 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb4a40bf070 con 0x7fb4b00cd750 2024-09-10T21:48:30.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.448+0000 7fb4b8400640 1 -- 172.21.15.7:0/372956315 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fb4b015a750 con 0x7fb4b00cd750 2024-09-10T21:48:30.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.448+0000 7fb4ad600640 1 -- 172.21.15.7:0/372956315 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+164 (secure 0 0 0) 0x7fb4a4083170 con 0x7fb4b00cd750 2024-09-10T21:48:30.449 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:48:30.450 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2024-09-10T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.451+0000 7fb4b8400640 1 -- 172.21.15.7:0/372956315 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb490077f70 msgr2=0x7fb49007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.451+0000 7fb4b8400640 1 --2- 172.21.15.7:0/372956315 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb490077f70 0x7fb49007a430 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb49c007fd0 tx=0x7fb49c009040 comp rx=0 tx=0).stop 2024-09-10T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.451+0000 7fb4b8400640 1 -- 172.21.15.7:0/372956315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4b00cd750 msgr2=0x7fb4b01f32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.451+0000 7fb4b8400640 1 --2- 172.21.15.7:0/372956315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4b00cd750 0x7fb4b01f32a0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fb4a402aa70 tx=0x7fb4a4032b90 comp rx=0 tx=0).stop 2024-09-10T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.451+0000 7fb4b8400640 1 -- 172.21.15.7:0/372956315 shutdown_connections 2024-09-10T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.451+0000 7fb4b8400640 1 --2- 172.21.15.7:0/372956315 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb490077f70 0x7fb49007a430 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.451+0000 7fb4b8400640 1 --2- 172.21.15.7:0/372956315 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb4b015fda0 0x7fb4b00b0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.451+0000 7fb4b8400640 1 --2- 172.21.15.7:0/372956315 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb4b015bc90 0x7fb4b00c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.451+0000 7fb4b8400640 1 --2- 172.21.15.7:0/372956315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb4b00cd750 0x7fb4b01f32a0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.451+0000 7fb4b8400640 1 -- 172.21.15.7:0/372956315 >> 172.21.15.7:0/372956315 conn(0x7fb4b00b16d0 msgr2=0x7fb4b0072360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:30.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.451+0000 7fb4b8400640 1 -- 172.21.15.7:0/372956315 shutdown_connections 2024-09-10T21:48:30.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.451+0000 7fb4b8400640 1 -- 172.21.15.7:0/372956315 wait complete. 2024-09-10T21:48:30.470 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:48:30.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:48:30.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-09-10T21:48:30.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.634+0000 7f2f56000640 1 -- 172.21.15.7:0/1568425796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f5015edd0 msgr2=0x7f2f5006cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.634+0000 7f2f56000640 1 --2- 172.21.15.7:0/1568425796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f5015edd0 0x7f2f5006cb20 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f2f38005fd0 tx=0x7f2f3802a4a0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.634+0000 7f2f56000640 1 -- 172.21.15.7:0/1568425796 shutdown_connections 2024-09-10T21:48:30.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.634+0000 7f2f56000640 1 --2- 172.21.15.7:0/1568425796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f5015edd0 0x7f2f5006cb20 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.634+0000 7f2f56000640 1 --2- 172.21.15.7:0/1568425796 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f50158300 0x7f2f500b4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.635+0000 7f2f56000640 1 -- 172.21.15.7:0/1568425796 >> 172.21.15.7:0/1568425796 conn(0x7f2f500b1c10 msgr2=0x7f2f500b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:30.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.635+0000 7f2f56000640 1 -- 172.21.15.7:0/1568425796 shutdown_connections 2024-09-10T21:48:30.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.635+0000 7f2f56000640 1 -- 172.21.15.7:0/1568425796 wait complete. 2024-09-10T21:48:30.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.636+0000 7f2f56000640 1 Processor -- start 2024-09-10T21:48:30.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.636+0000 7f2f56000640 1 -- start start 2024-09-10T21:48:30.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.637+0000 7f2f56000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f5006d060 0x7f2f501f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.637+0000 7f2f56000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f50158300 0x7f2f5014c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.637+0000 7f2f56000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f5015edd0 0x7f2f500cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.637+0000 7f2f56000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2f500acf20 con 0x7f2f5015edd0 2024-09-10T21:48:30.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.637+0000 7f2f56000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2f500acda0 con 0x7f2f50158300 2024-09-10T21:48:30.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.637+0000 7f2f56000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f2f50073f20 con 0x7f2f5006d060 2024-09-10T21:48:30.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.637+0000 7f2f4fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f5015edd0 0x7f2f500cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.638+0000 7f2f4fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f5015edd0 0x7f2f500cdc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38934/0 (socket says 172.21.15.7:38934) 2024-09-10T21:48:30.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.638+0000 7f2f47e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f50158300 0x7f2f5014c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.638+0000 7f2f4fe00640 1 -- 172.21.15.7:0/1270240071 learned_addr learned my addr 172.21.15.7:0/1270240071 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:30.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.638+0000 7f2f4f400640 1 --2- 172.21.15.7:0/1270240071 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f5006d060 0x7f2f501f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.638+0000 7f2f4fe00640 1 -- 172.21.15.7:0/1270240071 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f5006d060 msgr2=0x7f2f501f54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.638+0000 7f2f4fe00640 1 --2- 172.21.15.7:0/1270240071 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f5006d060 0x7f2f501f54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.638+0000 7f2f4fe00640 1 -- 172.21.15.7:0/1270240071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f50158300 msgr2=0x7f2f5014c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.638+0000 7f2f4fe00640 1 --2- 172.21.15.7:0/1270240071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f50158300 0x7f2f5014c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.638+0000 7f2f4fe00640 1 -- 172.21.15.7:0/1270240071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f500ccbf0 con 0x7f2f5015edd0 2024-09-10T21:48:30.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.638+0000 7f2f4fe00640 1 --2- 172.21.15.7:0/1270240071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f5015edd0 0x7f2f500cdc90 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800c780 tx=0x7f2f4800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:30.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.638+0000 7f2f47400640 1 -- 172.21.15.7:0/1270240071 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48004b60 con 0x7f2f5015edd0 2024-09-10T21:48:30.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.638+0000 7f2f47400640 1 -- 172.21.15.7:0/1270240071 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f48004df0 con 0x7f2f5015edd0 2024-09-10T21:48:30.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.638+0000 7f2f56000640 1 -- 172.21.15.7:0/1270240071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f500cced0 con 0x7f2f5015edd0 2024-09-10T21:48:30.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.639+0000 7f2f47400640 1 -- 172.21.15.7:0/1270240071 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4800aa70 con 0x7f2f5015edd0 2024-09-10T21:48:30.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.639+0000 7f2f56000640 1 -- 172.21.15.7:0/1270240071 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f500cd490 con 0x7f2f5015edd0 2024-09-10T21:48:30.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.641+0000 7f2f47400640 1 -- 172.21.15.7:0/1270240071 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2f4800ac10 con 0x7f2f5015edd0 2024-09-10T21:48:30.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.642+0000 7f2f47400640 1 --2- 172.21.15.7:0/1270240071 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f2c077f70 0x7f2f2c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.642+0000 7f2f47400640 1 -- 172.21.15.7:0/1270240071 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f2f48099f80 con 0x7f2f5015edd0 2024-09-10T21:48:30.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.642+0000 7f2f58eda740 1 Processor -- start 2024-09-10T21:48:30.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.642+0000 7f2f58eda740 1 -- start start 2024-09-10T21:48:30.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.642+0000 7f2f4f400640 1 --2- 172.21.15.7:0/1270240071 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f2c077f70 0x7f2f2c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.643+0000 7f2f4f400640 1 --2- 172.21.15.7:0/1270240071 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f2c077f70 0x7f2f2c07a430 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000bbe0 tx=0x7f2f4000b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:30.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.643+0000 7f2f58eda740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05655d10 0x564b056560f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.643+0000 7f2f58eda740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05610e80 0x564b056547f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.643+0000 7f2f58eda740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05611400 0x564b05654d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.643+0000 7f2f58eda740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564b05582f40 con 0x564b05610e80 2024-09-10T21:48:30.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.643+0000 7f2f58eda740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564b05572370 con 0x564b05655d10 2024-09-10T21:48:30.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.643+0000 7f2f58eda740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x564b05576180 con 0x564b05611400 2024-09-10T21:48:30.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.644+0000 7f2f47e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05610e80 0x564b056547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.644+0000 7f2f4f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05655d10 0x564b056560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.644+0000 7f2f47e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05610e80 0x564b056547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38946/0 (socket says 172.21.15.7:38946) 2024-09-10T21:48:30.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.644+0000 7f2f47e00640 1 -- 172.21.15.7:0/4019517189 learned_addr learned my addr 172.21.15.7:0/4019517189 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.644+0000 7f2f4fe00640 1 --2- 172.21.15.7:0/4019517189 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05611400 0x564b05654d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.644+0000 7f2f47e00640 1 -- 172.21.15.7:0/4019517189 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05611400 msgr2=0x564b05654d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.644+0000 7f2f47e00640 1 --2- 172.21.15.7:0/4019517189 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05611400 0x564b05654d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.644+0000 7f2f47e00640 1 -- 172.21.15.7:0/4019517189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05655d10 msgr2=0x564b056560f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.644+0000 7f2f47e00640 1 --2- 172.21.15.7:0/4019517189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05655d10 0x564b056560f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.644+0000 7f2f47e00640 1 -- 172.21.15.7:0/4019517189 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b05264470 con 0x564b05610e80 2024-09-10T21:48:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.644+0000 7f2f47e00640 1 --2- 172.21.15.7:0/4019517189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05610e80 0x564b056547f0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f2f380099f0 tx=0x7f2f380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b55584a92db4af7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.645+0000 7f2f3f400640 1 -- 172.21.15.7:0/4019517189 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f38042070 con 0x564b05610e80 2024-09-10T21:48:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.645+0000 7f2f3f400640 1 -- 172.21.15.7:0/4019517189 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f38005020 con 0x564b05610e80 2024-09-10T21:48:30.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.645+0000 7f2f3f400640 1 -- 172.21.15.7:0/4019517189 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f38035040 con 0x564b05610e80 2024-09-10T21:48:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.645+0000 7f2f58eda740 1 -- 172.21.15.7:0/4019517189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05610e80 msgr2=0x564b056547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.645+0000 7f2f58eda740 1 --2- 172.21.15.7:0/4019517189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05610e80 0x564b056547f0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f2f380099f0 tx=0x7f2f380046c0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.645+0000 7f2f58eda740 1 -- 172.21.15.7:0/4019517189 shutdown_connections 2024-09-10T21:48:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.645+0000 7f2f58eda740 1 --2- 172.21.15.7:0/4019517189 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05611400 0x564b05654d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.645+0000 7f2f58eda740 1 --2- 172.21.15.7:0/4019517189 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05610e80 0x564b056547f0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.645+0000 7f2f58eda740 1 --2- 172.21.15.7:0/4019517189 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05655d10 0x564b056560f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.645+0000 7f2f58eda740 1 -- 172.21.15.7:0/4019517189 >> 172.21.15.7:0/4019517189 conn(0x564b0560c450 msgr2=0x564b0560fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.645+0000 7f2f58eda740 1 -- 172.21.15.7:0/4019517189 shutdown_connections 2024-09-10T21:48:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.645+0000 7f2f58eda740 1 -- 172.21.15.7:0/4019517189 wait complete. 2024-09-10T21:48:30.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.646+0000 7f2f58eda740 1 -- start start 2024-09-10T21:48:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.646+0000 7f2f58eda740 1 Processor -- start 2024-09-10T21:48:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.647+0000 7f2f58eda740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05611400 0x564b05667b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.647+0000 7f2f58eda740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05655d10 0x564b05638a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.647+0000 7f2f58eda740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05638fc0 0x564b0564f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.647+0000 7f2f58eda740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564b052e27c0 con 0x564b05611400 2024-09-10T21:48:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.647+0000 7f2f58eda740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564b05517ae0 con 0x564b05638fc0 2024-09-10T21:48:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.647+0000 7f2f58eda740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x564b0560ae70 con 0x564b05655d10 2024-09-10T21:48:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.647+0000 7f2f58eda740 10 client.0 _ll_register_callbacks cb 0x564b055275b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:48:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.648+0000 7f2f4f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05611400 0x564b05667b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.648+0000 7f2f4f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05611400 0x564b05667b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38950/0 (socket says 172.21.15.7:38950) 2024-09-10T21:48:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.648+0000 7f2f4f400640 1 -- 172.21.15.7:0/754111924 learned_addr learned my addr 172.21.15.7:0/754111924 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:30.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.648+0000 7f2f4fe00640 1 --2- 172.21.15.7:0/754111924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05638fc0 0x564b0564f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.648+0000 7f2f47e00640 1 --2- 172.21.15.7:0/754111924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05655d10 0x564b05638a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.648+0000 7f2f4f400640 1 -- 172.21.15.7:0/754111924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05655d10 msgr2=0x564b05638a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.648+0000 7f2f4f400640 1 --2- 172.21.15.7:0/754111924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05655d10 0x564b05638a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.648+0000 7f2f4f400640 1 -- 172.21.15.7:0/754111924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05638fc0 msgr2=0x564b0564f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.648+0000 7f2f4f400640 1 --2- 172.21.15.7:0/754111924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05638fc0 0x564b0564f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.648+0000 7f2f4f400640 1 -- 172.21.15.7:0/754111924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b053487e0 con 0x564b05611400 2024-09-10T21:48:30.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.648+0000 7f2f4fe00640 1 --2- 172.21.15.7:0/754111924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05638fc0 0x564b0564f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:48:30.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.648+0000 7f2f4f400640 1 --2- 172.21.15.7:0/754111924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05611400 0x564b05667b80 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f2f400117f0 tx=0x7f2f40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87aac6519b04e9d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:30.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.648+0000 7f2f33e00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:48:30.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.649+0000 7f2f33e00640 1 -- 172.21.15.7:0/754111924 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4001fb30 con 0x564b05611400 2024-09-10T21:48:30.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.649+0000 7f2f33e00640 1 -- 172.21.15.7:0/754111924 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f2f4001fcd0 con 0x564b05611400 2024-09-10T21:48:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.649+0000 7f2f58eda740 1 -- 172.21.15.7:0/754111924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f38005020 con 0x564b05611400 2024-09-10T21:48:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.649+0000 7f2f33e00640 1 -- 172.21.15.7:0/754111924 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f2f40028440 con 0x564b05611400 2024-09-10T21:48:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.649+0000 7f2f58eda740 10 client.7083 fetch_fsmap learned FSMap version 70 2024-09-10T21:48:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.649+0000 7f2f58eda740 1 -- 172.21.15.7:0/754111924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x564b053a8f70 con 0x564b05611400 2024-09-10T21:48:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.649+0000 7f2f33e00640 1 -- 172.21.15.7:0/754111924 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f38005020 con 0x564b05611400 2024-09-10T21:48:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.649+0000 7f2f33e00640 1 -- 172.21.15.7:0/754111924 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f2f40026070 con 0x564b05611400 2024-09-10T21:48:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.649+0000 7f2f58eda740 10 client.7083 fetch_fsmap finished waiting for FSMap version 70 2024-09-10T21:48:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.649+0000 7f2f58eda740 10 client.7083 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-10T21:48:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.649+0000 7f2f58eda740 20 client.7083 populate_metadata read hostname 'smithi007' 2024-09-10T21:48:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.649+0000 7f2f58eda740 1 --2- 172.21.15.7:0/754111924 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x564b056683e0 0x564b056687c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.649+0000 7f2f58eda740 4 client.7083 mds_command: new command op to 235778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:48:30.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.649+0000 7f2f58eda740 1 -- 172.21.15.7:0/754111924 --> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b05584e90 con 0x564b056683e0 2024-09-10T21:48:30.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.650+0000 7f2f47e00640 1 --2- 172.21.15.7:0/754111924 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x564b056683e0 0x564b056687c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.650+0000 7f2f47e00640 1 --2- 172.21.15.7:0/754111924 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x564b056683e0 0x564b056687c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2edbb29e77e970ae server_cookie=519428cce5b60c89 in_seq=0 out_seq=0 2024-09-10T21:48:30.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.650+0000 7f2f33e00640 10 client.7083 ms_handle_connect on v2:172.21.15.133:6835/3312604545 2024-09-10T21:48:30.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.651+0000 7f2f33e00640 1 -- 172.21.15.7:0/754111924 <== mds.0 v2:172.21.15.133:6835/3312604545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564b05584e90 con 0x564b056683e0 2024-09-10T21:48:30.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.651+0000 7f2f33e00640 10 client.7083 handle_command_reply: tid=0 2024-09-10T21:48:30.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.652+0000 7f2f58eda740 1 client.7083 shutdown 2024-09-10T21:48:30.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.652+0000 7f2f58eda740 1 -- 172.21.15.7:0/754111924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05611400 msgr2=0x564b05667b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.652+0000 7f2f58eda740 1 --2- 172.21.15.7:0/754111924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05611400 0x564b05667b80 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f2f400117f0 tx=0x7f2f40004730 comp rx=0 tx=0).stop 2024-09-10T21:48:30.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.652+0000 7f2f58eda740 1 -- 172.21.15.7:0/754111924 shutdown_connections 2024-09-10T21:48:30.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.652+0000 7f2f58eda740 1 --2- 172.21.15.7:0/754111924 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x564b056683e0 0x564b056687c0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.652+0000 7f2f58eda740 1 --2- 172.21.15.7:0/754111924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05638fc0 0x564b0564f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.652+0000 7f2f58eda740 1 --2- 172.21.15.7:0/754111924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05655d10 0x564b05638a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.652+0000 7f2f33e00640 0 client.7083 ms_handle_reset on v2:172.21.15.133:6835/3312604545 2024-09-10T21:48:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.652+0000 7f2f58eda740 1 --2- 172.21.15.7:0/754111924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05611400 0x564b05667b80 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.652+0000 7f2f58eda740 1 -- 172.21.15.7:0/754111924 >> 172.21.15.7:0/754111924 conn(0x564b05610e80 msgr2=0x564b05609ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.652+0000 7f2f58eda740 1 -- 172.21.15.7:0/754111924 shutdown_connections 2024-09-10T21:48:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.652+0000 7f2f58eda740 1 -- 172.21.15.7:0/754111924 wait complete. 2024-09-10T21:48:30.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.652+0000 7f2f58eda740 20 client.7083 trim_cache size 0 max 16384 2024-09-10T21:48:30.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.671+0000 7f2f58eda740 1 Processor -- start 2024-09-10T21:48:30.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.671+0000 7f2f58eda740 1 -- start start 2024-09-10T21:48:30.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.672+0000 7f2f58eda740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05611400 0x564b05636eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.672+0000 7f2f58eda740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05638fc0 0x564b0560aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.672+0000 7f2f58eda740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05655d10 0x564b0560b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.672+0000 7f2f58eda740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2f50201350 con 0x564b05655d10 2024-09-10T21:48:30.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.672+0000 7f2f58eda740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2f502003c0 con 0x564b05611400 2024-09-10T21:48:30.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.672+0000 7f2f58eda740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f2f501f8400 con 0x564b05638fc0 2024-09-10T21:48:30.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.672+0000 7f2f4fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05655d10 0x564b0560b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.672+0000 7f2f4fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05655d10 0x564b0560b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38952/0 (socket says 172.21.15.7:38952) 2024-09-10T21:48:30.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.672+0000 7f2f4fe00640 1 -- 172.21.15.7:0/4274167604 learned_addr learned my addr 172.21.15.7:0/4274167604 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:30.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.672+0000 7f2f4fe00640 1 -- 172.21.15.7:0/4274167604 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05638fc0 msgr2=0x564b0560aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.672+0000 7f2f4fe00640 1 --2- 172.21.15.7:0/4274167604 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05638fc0 0x564b0560aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.672+0000 7f2f4fe00640 1 -- 172.21.15.7:0/4274167604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05611400 msgr2=0x564b05636eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.672+0000 7f2f4fe00640 1 --2- 172.21.15.7:0/4274167604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05611400 0x564b05636eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.672+0000 7f2f4fe00640 1 -- 172.21.15.7:0/4274167604 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b052b1720 con 0x564b05655d10 2024-09-10T21:48:30.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.672+0000 7f2f4fe00640 1 --2- 172.21.15.7:0/4274167604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05655d10 0x564b0560b530 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f2f48062360 tx=0x7f2f48069040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdc85cc6638be08 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:30.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.673+0000 7f2f3ea00640 1 -- 172.21.15.7:0/4274167604 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48088070 con 0x564b05655d10 2024-09-10T21:48:30.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.673+0000 7f2f3ea00640 1 -- 172.21.15.7:0/4274167604 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f480623d0 con 0x564b05655d10 2024-09-10T21:48:30.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.673+0000 7f2f3ea00640 1 -- 172.21.15.7:0/4274167604 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f4806b5d0 con 0x564b05655d10 2024-09-10T21:48:30.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.673+0000 7f2f58eda740 1 -- 172.21.15.7:0/4274167604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05655d10 msgr2=0x564b0560b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.673+0000 7f2f58eda740 1 --2- 172.21.15.7:0/4274167604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05655d10 0x564b0560b530 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f2f48062360 tx=0x7f2f48069040 comp rx=0 tx=0).stop 2024-09-10T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.673+0000 7f2f58eda740 1 -- 172.21.15.7:0/4274167604 shutdown_connections 2024-09-10T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.673+0000 7f2f58eda740 1 --2- 172.21.15.7:0/4274167604 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05655d10 0x564b0560b530 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.673+0000 7f2f58eda740 1 --2- 172.21.15.7:0/4274167604 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05638fc0 0x564b0560aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.673+0000 7f2f58eda740 1 --2- 172.21.15.7:0/4274167604 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05611400 0x564b05636eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.673+0000 7f2f58eda740 1 -- 172.21.15.7:0/4274167604 >> 172.21.15.7:0/4274167604 conn(0x564b05610e80 msgr2=0x564b0560fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.673+0000 7f2f58eda740 1 -- 172.21.15.7:0/4274167604 shutdown_connections 2024-09-10T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.673+0000 7f2f58eda740 1 -- 172.21.15.7:0/4274167604 wait complete. 2024-09-10T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.674+0000 7f2f58eda740 1 -- start start 2024-09-10T21:48:30.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.674+0000 7f2f58eda740 1 Processor -- start 2024-09-10T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.675+0000 7f2f58eda740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05611400 0x564b05609810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.675+0000 7f2f58eda740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05638fc0 0x564b056d6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.675+0000 7f2f58eda740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05655d10 0x564b056353c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.675+0000 7f2f58eda740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2f501f8a50 con 0x564b05655d10 2024-09-10T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.675+0000 7f2f58eda740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564b05664a50 con 0x564b05638fc0 2024-09-10T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.675+0000 7f2f58eda740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x564b0560a8f0 con 0x564b05611400 2024-09-10T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.675+0000 7f2f58eda740 10 client.0 _ll_register_callbacks cb 0x7f2f40010c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.675+0000 7f2f4fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05655d10 0x564b056353c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.675+0000 7f2f4fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05655d10 0x564b056353c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38966/0 (socket says 172.21.15.7:38966) 2024-09-10T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.675+0000 7f2f4fe00640 1 -- 172.21.15.7:0/2725537337 learned_addr learned my addr 172.21.15.7:0/2725537337 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.676+0000 7f2f47e00640 1 --2- 172.21.15.7:0/2725537337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05638fc0 0x564b056d6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.676+0000 7f2f4f400640 1 --2- 172.21.15.7:0/2725537337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05611400 0x564b05609810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.676+0000 7f2f4fe00640 1 -- 172.21.15.7:0/2725537337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05611400 msgr2=0x564b05609810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.676+0000 7f2f4fe00640 1 --2- 172.21.15.7:0/2725537337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05611400 0x564b05609810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.676+0000 7f2f4fe00640 1 -- 172.21.15.7:0/2725537337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05638fc0 msgr2=0x564b056d6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.676+0000 7f2f4fe00640 1 --2- 172.21.15.7:0/2725537337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05638fc0 0x564b056d6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.676+0000 7f2f4fe00640 1 -- 172.21.15.7:0/2725537337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b05635f90 con 0x564b05655d10 2024-09-10T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.676+0000 7f2f4f400640 1 --2- 172.21.15.7:0/2725537337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05611400 0x564b05609810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.676+0000 7f2f4fe00640 1 --2- 172.21.15.7:0/2725537337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05655d10 0x564b056353c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800a430 tx=0x7f2f4805e870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626022f7a1b06f32 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.676+0000 7f2f33e00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.676+0000 7f2f33e00640 1 -- 172.21.15.7:0/2725537337 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48095030 con 0x564b05655d10 2024-09-10T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.676+0000 7f2f33e00640 1 -- 172.21.15.7:0/2725537337 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f2f4806b5a0 con 0x564b05655d10 2024-09-10T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.676+0000 7f2f58eda740 1 -- 172.21.15.7:0/2725537337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f480623d0 con 0x564b05655d10 2024-09-10T21:48:30.681 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.676+0000 7f2f33e00640 1 -- 172.21.15.7:0/2725537337 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f2f48088040 con 0x564b05655d10 2024-09-10T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.676+0000 7f2f58eda740 10 client.7089 fetch_fsmap learned FSMap version 70 2024-09-10T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.676+0000 7f2f58eda740 1 -- 172.21.15.7:0/2725537337 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x564b056d65b0 con 0x564b05655d10 2024-09-10T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.677+0000 7f2f33e00640 1 -- 172.21.15.7:0/2725537337 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f480623d0 con 0x564b05655d10 2024-09-10T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.677+0000 7f2f33e00640 1 -- 172.21.15.7:0/2725537337 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f2f480690b0 con 0x564b05655d10 2024-09-10T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.677+0000 7f2f58eda740 10 client.7089 fetch_fsmap finished waiting for FSMap version 70 2024-09-10T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.677+0000 7f2f58eda740 10 client.7089 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-10T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.677+0000 7f2f58eda740 20 client.7089 populate_metadata read hostname 'smithi007' 2024-09-10T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.677+0000 7f2f58eda740 1 --2- 172.21.15.7:0/2725537337 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x564b0560b700 0x564b05635900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.677+0000 7f2f58eda740 4 client.7089 mds_command: new command op to 235778 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-09-10T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.677+0000 7f2f58eda740 1 -- 172.21.15.7:0/2725537337 --> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x564b05580940 con 0x564b0560b700 2024-09-10T21:48:30.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.677+0000 7f2f4f400640 1 --2- 172.21.15.7:0/2725537337 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x564b0560b700 0x564b05635900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.678+0000 7f2f4f400640 1 --2- 172.21.15.7:0/2725537337 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x564b0560b700 0x564b05635900 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25ec6f59f353e837 server_cookie=6d86bd621bdd9ceb in_seq=0 out_seq=0 2024-09-10T21:48:30.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.678+0000 7f2f33e00640 10 client.7089 ms_handle_connect on v2:172.21.15.133:6835/3312604545 2024-09-10T21:48:30.683 INFO:tasks.ceph.mds.d.smithi133.stderr:2024-09-10T21:48:30.678+0000 7f900c000640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv18 ap=3 DIRTYPARENT f(v0 m2024-09-10T21:47:49.036385+0000 1=1+0)->f(v1 m2024-09-10T21:47:49.036385+0000) n(v0 rc2024-09-10T21:47:49.195383+0000 2=1+1)->n(v0 rc2024-09-10T21:47:49.195383+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55c0215d4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-09-10T21:47:49.036385+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-09-10T21:47:49.195383+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-09-10T21:48:30.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.678+0000 7f2f33e00640 1 -- 172.21.15.7:0/2725537337 <== mds.0 v2:172.21.15.133:6835/3312604545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564b05580940 con 0x564b0560b700 2024-09-10T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.678+0000 7f2f33e00640 10 client.7089 handle_command_reply: tid=0 2024-09-10T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.678+0000 7f2f58eda740 1 client.7089 shutdown 2024-09-10T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.679+0000 7f2f58eda740 1 -- 172.21.15.7:0/2725537337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05655d10 msgr2=0x564b056353c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.679+0000 7f2f58eda740 1 --2- 172.21.15.7:0/2725537337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05655d10 0x564b056353c0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800a430 tx=0x7f2f4805e870 comp rx=0 tx=0).stop 2024-09-10T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.679+0000 7f2f58eda740 1 -- 172.21.15.7:0/2725537337 shutdown_connections 2024-09-10T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.679+0000 7f2f58eda740 1 --2- 172.21.15.7:0/2725537337 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x564b0560b700 0x564b05635900 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.679+0000 7f2f58eda740 1 --2- 172.21.15.7:0/2725537337 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564b05638fc0 0x564b056d6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.679+0000 7f2f58eda740 1 --2- 172.21.15.7:0/2725537337 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564b05655d10 0x564b056353c0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.679+0000 7f2f58eda740 1 --2- 172.21.15.7:0/2725537337 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564b05611400 0x564b05609810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.679+0000 7f2f58eda740 1 -- 172.21.15.7:0/2725537337 >> 172.21.15.7:0/2725537337 conn(0x564b05610e80 msgr2=0x564b0560aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.679+0000 7f2f33e00640 0 client.7089 ms_handle_reset on v2:172.21.15.133:6835/3312604545 2024-09-10T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.679+0000 7f2f58eda740 1 -- 172.21.15.7:0/2725537337 shutdown_connections 2024-09-10T21:48:30.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.679+0000 7f2f58eda740 1 -- 172.21.15.7:0/2725537337 wait complete. 2024-09-10T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.679+0000 7f2f58eda740 20 client.7089 trim_cache size 0 max 16384 2024-09-10T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-09-10T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "a1511696-8e50-4b8f-909c-34f5fc84eef4", 2024-09-10T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-09-10T21:48:30.686 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.680+0000 7f2f56000640 1 -- 172.21.15.7:0/1270240071 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f2c077f70 msgr2=0x7f2f2c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.680+0000 7f2f56000640 1 --2- 172.21.15.7:0/1270240071 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f2c077f70 0x7f2f2c07a430 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2f4000bbe0 tx=0x7f2f4000b390 comp rx=0 tx=0).stop 2024-09-10T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.680+0000 7f2f56000640 1 -- 172.21.15.7:0/1270240071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f5015edd0 msgr2=0x7f2f500cdc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.680+0000 7f2f56000640 1 --2- 172.21.15.7:0/1270240071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f5015edd0 0x7f2f500cdc90 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800c780 tx=0x7f2f4800cc40 comp rx=0 tx=0).stop 2024-09-10T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.680+0000 7f2f56000640 1 -- 172.21.15.7:0/1270240071 shutdown_connections 2024-09-10T21:48:30.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.680+0000 7f2f56000640 1 --2- 172.21.15.7:0/1270240071 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2f2c077f70 0x7f2f2c07a430 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.680+0000 7f2f56000640 1 --2- 172.21.15.7:0/1270240071 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2f50158300 0x7f2f5014c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.680+0000 7f2f56000640 1 --2- 172.21.15.7:0/1270240071 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2f5015edd0 0x7f2f500cdc90 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.680+0000 7f2f56000640 1 --2- 172.21.15.7:0/1270240071 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2f5006d060 0x7f2f501f54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.680+0000 7f2f56000640 1 -- 172.21.15.7:0/1270240071 >> 172.21.15.7:0/1270240071 conn(0x7f2f500b1c10 msgr2=0x7f2f50073720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:30.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.680+0000 7f2f56000640 1 -- 172.21.15.7:0/1270240071 shutdown_connections 2024-09-10T21:48:30.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.680+0000 7f2f56000640 1 -- 172.21.15.7:0/1270240071 wait complete. 2024-09-10T21:48:30.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:48:30.697 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-10T21:48:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.858+0000 7f0b20a00640 1 -- 172.21.15.7:0/1912484060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b1c15df70 msgr2=0x7f0b1c0b6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.858+0000 7f0b20a00640 1 --2- 172.21.15.7:0/1912484060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b1c15df70 0x7f0b1c0b6cd0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c0d3980 tx=0x7f0b0c02aa10 comp rx=0 tx=0).stop 2024-09-10T21:48:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.859+0000 7f0b20a00640 1 -- 172.21.15.7:0/1912484060 shutdown_connections 2024-09-10T21:48:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.859+0000 7f0b20a00640 1 --2- 172.21.15.7:0/1912484060 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b1c0b0a10 0x7f0b1c0b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.859+0000 7f0b20a00640 1 --2- 172.21.15.7:0/1912484060 >> v2:172.21.15.133:3300/0 conn(0x7f0b1c0d22b0 0x7f0b1c0d26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.859+0000 7f0b20a00640 1 --2- 172.21.15.7:0/1912484060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b1c15df70 0x7f0b1c0b6cd0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.859+0000 7f0b20a00640 1 -- 172.21.15.7:0/1912484060 >> 172.21.15.7:0/1912484060 conn(0x7f0b1c0d3e80 msgr2=0x7f0b1c0d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:30.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.859+0000 7f0b20a00640 1 -- 172.21.15.7:0/1912484060 shutdown_connections 2024-09-10T21:48:30.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.859+0000 7f0b20a00640 1 -- 172.21.15.7:0/1912484060 wait complete. 2024-09-10T21:48:30.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.860+0000 7f0b20a00640 1 Processor -- start 2024-09-10T21:48:30.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.861+0000 7f0b20a00640 1 -- start start 2024-09-10T21:48:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.861+0000 7f0b20a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b1c0b0a10 0x7f0b1c1606a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.861+0000 7f0b20a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b1c0d22b0 0x7f0b1c15fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.861+0000 7f0b20a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b1c15df70 0x7f0b1c0b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.861+0000 7f0b20a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c15e4d0 con 0x7f0b1c15df70 2024-09-10T21:48:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.861+0000 7f0b20a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c15e350 con 0x7f0b1c0b0a10 2024-09-10T21:48:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.861+0000 7f0b20a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f0b1c0b27e0 con 0x7f0b1c0d22b0 2024-09-10T21:48:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.862+0000 7f0b1b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b1c15df70 0x7f0b1c0b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.862+0000 7f0b1a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b1c0b0a10 0x7f0b1c1606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.862+0000 7f0b19e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b1c0d22b0 0x7f0b1c15fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.862+0000 7f0b1a800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b1c0b0a10 0x7f0b1c1606a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:41780/0 (socket says 172.21.15.7:41780) 2024-09-10T21:48:30.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.862+0000 7f0b1a800640 1 -- 172.21.15.7:0/1961376085 learned_addr learned my addr 172.21.15.7:0/1961376085 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:30.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.862+0000 7f0b1b200640 1 -- 172.21.15.7:0/1961376085 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b1c0d22b0 msgr2=0x7f0b1c15fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.862+0000 7f0b1b200640 1 --2- 172.21.15.7:0/1961376085 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b1c0d22b0 0x7f0b1c15fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.862+0000 7f0b1b200640 1 -- 172.21.15.7:0/1961376085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b1c0b0a10 msgr2=0x7f0b1c1606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.862+0000 7f0b1b200640 1 --2- 172.21.15.7:0/1961376085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b1c0b0a10 0x7f0b1c1606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.862+0000 7f0b1b200640 1 -- 172.21.15.7:0/1961376085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b1c160be0 con 0x7f0b1c15df70 2024-09-10T21:48:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.863+0000 7f0b1b200640 1 --2- 172.21.15.7:0/1961376085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b1c15df70 0x7f0b1c0b2cb0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400c730 tx=0x7f0b1400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.863+0000 7f0b12a00640 1 -- 172.21.15.7:0/1961376085 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14007bb0 con 0x7f0b1c15df70 2024-09-10T21:48:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.863+0000 7f0b12a00640 1 -- 172.21.15.7:0/1961376085 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b1400ce40 con 0x7f0b1c15df70 2024-09-10T21:48:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.863+0000 7f0b20a00640 1 -- 172.21.15.7:0/1961376085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b1c0b4880 con 0x7f0b1c15df70 2024-09-10T21:48:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.863+0000 7f0b12a00640 1 -- 172.21.15.7:0/1961376085 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14007e60 con 0x7f0b1c15df70 2024-09-10T21:48:30.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.863+0000 7f0b20a00640 1 -- 172.21.15.7:0/1961376085 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b1c0b4e40 con 0x7f0b1c15df70 2024-09-10T21:48:30.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.866+0000 7f0b12a00640 1 -- 172.21.15.7:0/1961376085 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0b1400f6b0 con 0x7f0b1c15df70 2024-09-10T21:48:30.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.867+0000 7f0b12a00640 1 --2- 172.21.15.7:0/1961376085 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0afc077f70 0x7f0afc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.867+0000 7f0b12a00640 1 -- 172.21.15.7:0/1961376085 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f0b1409a3f0 con 0x7f0b1c15df70 2024-09-10T21:48:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.868+0000 7f0b1a800640 1 --2- 172.21.15.7:0/1961376085 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0afc077f70 0x7f0afc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.868+0000 7f0b243c2740 1 Processor -- start 2024-09-10T21:48:30.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.868+0000 7f0b243c2740 1 -- start start 2024-09-10T21:48:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.868+0000 7f0b1a800640 1 --2- 172.21.15.7:0/1961376085 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0afc077f70 0x7f0afc07a430 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c000f80 tx=0x7f0b0c03b040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.869+0000 7f0b243c2740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80c172e0 0x563b80c176c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.869+0000 7f0b243c2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80bd2450 0x563b80c15dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.869+0000 7f0b243c2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80bd29d0 0x563b80c16300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.869+0000 7f0b243c2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563b80b311c0 con 0x563b80bd2450 2024-09-10T21:48:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.869+0000 7f0b243c2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563b80b44870 con 0x563b80bd29d0 2024-09-10T21:48:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.869+0000 7f0b243c2740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x563b80b33ca0 con 0x563b80c172e0 2024-09-10T21:48:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.869+0000 7f0b19e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80bd2450 0x563b80c15dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.869+0000 7f0b19e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80bd2450 0x563b80c15dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39002/0 (socket says 172.21.15.7:39002) 2024-09-10T21:48:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.869+0000 7f0b19e00640 1 -- 172.21.15.7:0/2171108443 learned_addr learned my addr 172.21.15.7:0/2171108443 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:30.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.869+0000 7f0b1a800640 1 --2- 172.21.15.7:0/2171108443 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80c172e0 0x563b80c176c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.869+0000 7f0b19e00640 1 -- 172.21.15.7:0/2171108443 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80c172e0 msgr2=0x563b80c176c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.869+0000 7f0b19e00640 1 --2- 172.21.15.7:0/2171108443 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80c172e0 0x563b80c176c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.869+0000 7f0b1b200640 1 --2- 172.21.15.7:0/2171108443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80bd29d0 0x563b80c16300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.869+0000 7f0b19e00640 1 -- 172.21.15.7:0/2171108443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80bd29d0 msgr2=0x563b80c16300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.869+0000 7f0b19e00640 1 --2- 172.21.15.7:0/2171108443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80bd29d0 0x563b80c16300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.869+0000 7f0b19e00640 1 -- 172.21.15.7:0/2171108443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b8090c7d0 con 0x563b80bd2450 2024-09-10T21:48:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.869+0000 7f0b1a800640 1 --2- 172.21.15.7:0/2171108443 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80c172e0 0x563b80c176c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:48:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.870+0000 7f0b19e00640 1 --2- 172.21.15.7:0/2171108443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80bd2450 0x563b80c15dc0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800bcb0 tx=0x7f0b080107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a78ccf737cfabea server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.870+0000 7f0b07000640 1 -- 172.21.15.7:0/2171108443 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08004030 con 0x563b80bd2450 2024-09-10T21:48:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.870+0000 7f0b07000640 1 -- 172.21.15.7:0/2171108443 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b080042c0 con 0x563b80bd2450 2024-09-10T21:48:30.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.870+0000 7f0b07000640 1 -- 172.21.15.7:0/2171108443 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08003850 con 0x563b80bd2450 2024-09-10T21:48:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.870+0000 7f0b243c2740 1 -- 172.21.15.7:0/2171108443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80bd2450 msgr2=0x563b80c15dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.870+0000 7f0b243c2740 1 --2- 172.21.15.7:0/2171108443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80bd2450 0x563b80c15dc0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800bcb0 tx=0x7f0b080107a0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.871+0000 7f0b243c2740 1 -- 172.21.15.7:0/2171108443 shutdown_connections 2024-09-10T21:48:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.871+0000 7f0b243c2740 1 --2- 172.21.15.7:0/2171108443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80bd29d0 0x563b80c16300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.871+0000 7f0b243c2740 1 --2- 172.21.15.7:0/2171108443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80bd2450 0x563b80c15dc0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.871+0000 7f0b243c2740 1 --2- 172.21.15.7:0/2171108443 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80c172e0 0x563b80c176c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.871+0000 7f0b243c2740 1 -- 172.21.15.7:0/2171108443 >> 172.21.15.7:0/2171108443 conn(0x563b80bcda20 msgr2=0x563b80bd1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.871+0000 7f0b243c2740 1 -- 172.21.15.7:0/2171108443 shutdown_connections 2024-09-10T21:48:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.871+0000 7f0b243c2740 1 -- 172.21.15.7:0/2171108443 wait complete. 2024-09-10T21:48:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.871+0000 7f0b243c2740 1 -- start start 2024-09-10T21:48:30.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.872+0000 7f0b243c2740 1 Processor -- start 2024-09-10T21:48:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.873+0000 7f0b243c2740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80bd29d0 0x563b80be6220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.873+0000 7f0b243c2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80c172e0 0x563b80c16020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.873+0000 7f0b243c2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c16560 0x563b80c13e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.873+0000 7f0b243c2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563b80ada410 con 0x563b80c16560 2024-09-10T21:48:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.873+0000 7f0b243c2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563b80b37ab0 con 0x563b80c172e0 2024-09-10T21:48:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.873+0000 7f0b243c2740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x563b808a4740 con 0x563b80bd29d0 2024-09-10T21:48:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.873+0000 7f0b243c2740 10 client.0 _ll_register_callbacks cb 0x563b80ab7360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:48:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.873+0000 7f0b1b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c16560 0x563b80c13e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.873+0000 7f0b1b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c16560 0x563b80c13e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39014/0 (socket says 172.21.15.7:39014) 2024-09-10T21:48:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.873+0000 7f0b1b200640 1 -- 172.21.15.7:0/31991983 learned_addr learned my addr 172.21.15.7:0/31991983 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.873+0000 7f0b19e00640 1 --2- 172.21.15.7:0/31991983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80c172e0 0x563b80c16020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.873+0000 7f0b1a800640 1 --2- 172.21.15.7:0/31991983 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80bd29d0 0x563b80be6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.873+0000 7f0b1b200640 1 -- 172.21.15.7:0/31991983 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80bd29d0 msgr2=0x563b80be6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.873+0000 7f0b1b200640 1 --2- 172.21.15.7:0/31991983 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80bd29d0 0x563b80be6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.873+0000 7f0b1b200640 1 -- 172.21.15.7:0/31991983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80c172e0 msgr2=0x563b80c16020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.873+0000 7f0b1b200640 1 --2- 172.21.15.7:0/31991983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80c172e0 0x563b80c16020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.873+0000 7f0b1b200640 1 -- 172.21.15.7:0/31991983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b80895970 con 0x563b80c16560 2024-09-10T21:48:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.874+0000 7f0b1b200640 1 --2- 172.21.15.7:0/31991983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c16560 0x563b80c13e00 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f0b140987a0 tx=0x7f0b140623f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b311ba35556bf8d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.874+0000 7f0afb400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:48:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.874+0000 7f0afb400640 1 -- 172.21.15.7:0/31991983 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14088020 con 0x563b80c16560 2024-09-10T21:48:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.874+0000 7f0afb400640 1 -- 172.21.15.7:0/31991983 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f0b140883c0 con 0x563b80c16560 2024-09-10T21:48:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.874+0000 7f0b243c2740 1 -- 172.21.15.7:0/31991983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b080042c0 con 0x563b80c16560 2024-09-10T21:48:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.874+0000 7f0afb400640 1 -- 172.21.15.7:0/31991983 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f0b1406bc00 con 0x563b80c16560 2024-09-10T21:48:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.874+0000 7f0b243c2740 10 client.7101 fetch_fsmap learned FSMap version 70 2024-09-10T21:48:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.874+0000 7f0b243c2740 1 -- 172.21.15.7:0/31991983 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x563b808736a0 con 0x563b80c16560 2024-09-10T21:48:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.874+0000 7f0afb400640 1 -- 172.21.15.7:0/31991983 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b080042c0 con 0x563b80c16560 2024-09-10T21:48:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.874+0000 7f0afb400640 1 -- 172.21.15.7:0/31991983 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f0b14061a10 con 0x563b80c16560 2024-09-10T21:48:30.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.874+0000 7f0b243c2740 10 client.7101 fetch_fsmap finished waiting for FSMap version 70 2024-09-10T21:48:30.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.875+0000 7f0b243c2740 10 client.7101 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-10T21:48:30.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.875+0000 7f0b243c2740 20 client.7101 populate_metadata read hostname 'smithi007' 2024-09-10T21:48:30.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.875+0000 7f0b243c2740 1 --2- 172.21.15.7:0/31991983 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x563b80be8ba0 0x563b80be8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.875+0000 7f0b243c2740 4 client.7101 mds_command: new command op to 235778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:48:30.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.875+0000 7f0b243c2740 1 -- 172.21.15.7:0/31991983 --> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b80b467c0 con 0x563b80be8ba0 2024-09-10T21:48:30.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.875+0000 7f0b1a800640 1 --2- 172.21.15.7:0/31991983 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x563b80be8ba0 0x563b80be8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.876+0000 7f0b1a800640 1 --2- 172.21.15.7:0/31991983 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x563b80be8ba0 0x563b80be8fb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f3ccc0583fba55 server_cookie=40fc8c5c04d5f68 in_seq=0 out_seq=0 2024-09-10T21:48:30.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.876+0000 7f0afb400640 10 client.7101 ms_handle_connect on v2:172.21.15.133:6835/3312604545 2024-09-10T21:48:30.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.877+0000 7f0afb400640 1 -- 172.21.15.7:0/31991983 <== mds.0 v2:172.21.15.133:6835/3312604545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563b80b467c0 con 0x563b80be8ba0 2024-09-10T21:48:30.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.877+0000 7f0afb400640 10 client.7101 handle_command_reply: tid=0 2024-09-10T21:48:30.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.877+0000 7f0b243c2740 1 client.7101 shutdown 2024-09-10T21:48:30.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.877+0000 7f0b243c2740 1 -- 172.21.15.7:0/31991983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c16560 msgr2=0x563b80c13e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.877+0000 7f0b243c2740 1 --2- 172.21.15.7:0/31991983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c16560 0x563b80c13e00 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f0b140987a0 tx=0x7f0b140623f0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.877+0000 7f0b243c2740 1 -- 172.21.15.7:0/31991983 shutdown_connections 2024-09-10T21:48:30.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.877+0000 7f0b243c2740 1 --2- 172.21.15.7:0/31991983 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x563b80be8ba0 0x563b80be8fb0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.877+0000 7f0b243c2740 1 --2- 172.21.15.7:0/31991983 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80c172e0 0x563b80c16020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.877+0000 7f0b243c2740 1 --2- 172.21.15.7:0/31991983 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c16560 0x563b80c13e00 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.877+0000 7f0b243c2740 1 --2- 172.21.15.7:0/31991983 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80bd29d0 0x563b80be6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.877+0000 7f0b243c2740 1 -- 172.21.15.7:0/31991983 >> 172.21.15.7:0/31991983 conn(0x563b80bd2450 msgr2=0x563b80bcaff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:30.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.877+0000 7f0afb400640 0 client.7101 ms_handle_reset on v2:172.21.15.133:6835/3312604545 2024-09-10T21:48:30.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.877+0000 7f0b243c2740 1 -- 172.21.15.7:0/31991983 shutdown_connections 2024-09-10T21:48:30.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.877+0000 7f0b243c2740 1 -- 172.21.15.7:0/31991983 wait complete. 2024-09-10T21:48:30.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.877+0000 7f0b243c2740 20 client.7101 trim_cache size 0 max 16384 2024-09-10T21:48:30.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.896+0000 7f0b243c2740 1 Processor -- start 2024-09-10T21:48:30.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.896+0000 7f0b243c2740 1 -- start start 2024-09-10T21:48:30.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.897+0000 7f0b243c2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80bd29d0 0x563b80bcf870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.900 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.897+0000 7f0b243c2740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80c16560 0x563b80bcfdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.897+0000 7f0b243c2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c172e0 0x563b80c39a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.897+0000 7f0b243c2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563b80c294e0 con 0x563b80c172e0 2024-09-10T21:48:30.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.897+0000 7f0b243c2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c2010f0 con 0x563b80bd29d0 2024-09-10T21:48:30.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.897+0000 7f0b243c2740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f0b1c200150 con 0x563b80c16560 2024-09-10T21:48:30.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.897+0000 7f0b1b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c172e0 0x563b80c39a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.897+0000 7f0b1b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c172e0 0x563b80c39a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39028/0 (socket says 172.21.15.7:39028) 2024-09-10T21:48:30.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.897+0000 7f0b1b200640 1 -- 172.21.15.7:0/4173646836 learned_addr learned my addr 172.21.15.7:0/4173646836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:30.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.897+0000 7f0b19e00640 1 --2- 172.21.15.7:0/4173646836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80c16560 0x563b80bcfdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.897+0000 7f0b1a800640 1 --2- 172.21.15.7:0/4173646836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80bd29d0 0x563b80bcf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.897+0000 7f0b1b200640 1 -- 172.21.15.7:0/4173646836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80c16560 msgr2=0x563b80bcfdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.897+0000 7f0b1b200640 1 --2- 172.21.15.7:0/4173646836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80c16560 0x563b80bcfdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.902 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.897+0000 7f0b1b200640 1 -- 172.21.15.7:0/4173646836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80bd29d0 msgr2=0x563b80bcf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.897+0000 7f0b1b200640 1 --2- 172.21.15.7:0/4173646836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80bd29d0 0x563b80bcf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.897+0000 7f0b1b200640 1 -- 172.21.15.7:0/4173646836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b80b44510 con 0x563b80c172e0 2024-09-10T21:48:30.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.897+0000 7f0b1a800640 1 --2- 172.21.15.7:0/4173646836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80bd29d0 0x563b80bcf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:48:30.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.898+0000 7f0b1b200640 1 --2- 172.21.15.7:0/4173646836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c172e0 0x563b80c39a90 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f0b14098770 tx=0x7f0b14061c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f3d922789cd352 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:30.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.898+0000 7f0b06600640 1 -- 172.21.15.7:0/4173646836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b14088070 con 0x563b80c172e0 2024-09-10T21:48:30.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.898+0000 7f0b06600640 1 -- 172.21.15.7:0/4173646836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b14088410 con 0x563b80c172e0 2024-09-10T21:48:30.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.898+0000 7f0b06600640 1 -- 172.21.15.7:0/4173646836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b140695e0 con 0x563b80c172e0 2024-09-10T21:48:30.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.898+0000 7f0b243c2740 1 -- 172.21.15.7:0/4173646836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c172e0 msgr2=0x563b80c39a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.898+0000 7f0b243c2740 1 --2- 172.21.15.7:0/4173646836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c172e0 0x563b80c39a90 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f0b14098770 tx=0x7f0b14061c70 comp rx=0 tx=0).stop 2024-09-10T21:48:30.903 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.898+0000 7f0b243c2740 1 -- 172.21.15.7:0/4173646836 shutdown_connections 2024-09-10T21:48:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.898+0000 7f0b243c2740 1 --2- 172.21.15.7:0/4173646836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c172e0 0x563b80c39a90 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.898+0000 7f0b243c2740 1 --2- 172.21.15.7:0/4173646836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80c16560 0x563b80bcfdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.898+0000 7f0b243c2740 1 --2- 172.21.15.7:0/4173646836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80bd29d0 0x563b80bcf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.898+0000 7f0b243c2740 1 -- 172.21.15.7:0/4173646836 >> 172.21.15.7:0/4173646836 conn(0x563b80bd2450 msgr2=0x563b80be6230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.899+0000 7f0b243c2740 1 -- 172.21.15.7:0/4173646836 shutdown_connections 2024-09-10T21:48:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.899+0000 7f0b243c2740 1 -- 172.21.15.7:0/4173646836 wait complete. 2024-09-10T21:48:30.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.899+0000 7f0b243c2740 1 -- start start 2024-09-10T21:48:30.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.899+0000 7f0b243c2740 1 Processor -- start 2024-09-10T21:48:30.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.900+0000 7f0b243c2740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80bd29d0 0x563b80c963c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.900+0000 7f0b243c2740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c16560 0x563b80c12600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.900+0000 7f0b243c2740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80c172e0 0x563b80c12b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.900+0000 7f0b243c2740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c15f270 con 0x563b80c16560 2024-09-10T21:48:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.900+0000 7f0b243c2740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1c0ccf10 con 0x563b80bd29d0 2024-09-10T21:48:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.900+0000 7f0b243c2740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x563b80bc7360 con 0x563b80c172e0 2024-09-10T21:48:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.900+0000 7f0b243c2740 10 client.0 _ll_register_callbacks cb 0x7f0b1405eea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:48:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.901+0000 7f0b19e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c16560 0x563b80c12600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.901+0000 7f0b19e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c16560 0x563b80c12600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39044/0 (socket says 172.21.15.7:39044) 2024-09-10T21:48:30.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.901+0000 7f0b19e00640 1 -- 172.21.15.7:0/4246101398 learned_addr learned my addr 172.21.15.7:0/4246101398 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:48:30.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.901+0000 7f0b1a800640 1 --2- 172.21.15.7:0/4246101398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80bd29d0 0x563b80c963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.901+0000 7f0b1b200640 1 --2- 172.21.15.7:0/4246101398 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80c172e0 0x563b80c12b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.901+0000 7f0b19e00640 1 -- 172.21.15.7:0/4246101398 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80c172e0 msgr2=0x563b80c12b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.901+0000 7f0b19e00640 1 --2- 172.21.15.7:0/4246101398 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80c172e0 0x563b80c12b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.901+0000 7f0b19e00640 1 -- 172.21.15.7:0/4246101398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80bd29d0 msgr2=0x563b80c963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:48:30.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.901+0000 7f0b19e00640 1 --2- 172.21.15.7:0/4246101398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80bd29d0 0x563b80c963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.901+0000 7f0b19e00640 1 -- 172.21.15.7:0/4246101398 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b80bfbb50 con 0x563b80c16560 2024-09-10T21:48:30.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.901+0000 7f0b19e00640 1 --2- 172.21.15.7:0/4246101398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c16560 0x563b80c12600 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f0b08011010 tx=0x7f0b080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791fda114fcdc20c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:30.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.901+0000 7f0afb400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:48:30.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.901+0000 7f0afb400640 1 -- 172.21.15.7:0/4246101398 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b080037f0 con 0x563b80c16560 2024-09-10T21:48:30.914 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.901+0000 7f0afb400640 1 -- 172.21.15.7:0/4246101398 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f0b08003990 con 0x563b80c16560 2024-09-10T21:48:30.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.902+0000 7f0b243c2740 1 -- 172.21.15.7:0/4246101398 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b14088410 con 0x563b80c16560 2024-09-10T21:48:30.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.902+0000 7f0afb400640 1 -- 172.21.15.7:0/4246101398 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f0b08035050 con 0x563b80c16560 2024-09-10T21:48:30.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.902+0000 7f0b243c2740 10 client.7107 fetch_fsmap learned FSMap version 70 2024-09-10T21:48:30.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.902+0000 7f0b243c2740 1 -- 172.21.15.7:0/4246101398 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x563b80be5bf0 con 0x563b80c16560 2024-09-10T21:48:30.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.902+0000 7f0afb400640 1 -- 172.21.15.7:0/4246101398 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b14088410 con 0x563b80c16560 2024-09-10T21:48:30.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.902+0000 7f0afb400640 1 -- 172.21.15.7:0/4246101398 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f0b08038020 con 0x563b80c16560 2024-09-10T21:48:30.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.902+0000 7f0b243c2740 10 client.7107 fetch_fsmap finished waiting for FSMap version 70 2024-09-10T21:48:30.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.902+0000 7f0b243c2740 10 client.7107 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-10T21:48:30.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.902+0000 7f0b243c2740 20 client.7107 populate_metadata read hostname 'smithi007' 2024-09-10T21:48:30.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.902+0000 7f0b243c2740 1 --2- 172.21.15.7:0/4246101398 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x563b80c96900 0x563b80c9a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:30.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.902+0000 7f0b243c2740 4 client.7107 mds_command: new command op to 235778 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:48:30.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.902+0000 7f0b243c2740 1 -- 172.21.15.7:0/4246101398 --> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563b80b42270 con 0x563b80c96900 2024-09-10T21:48:30.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.902+0000 7f0b1a800640 1 --2- 172.21.15.7:0/4246101398 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x563b80c96900 0x563b80c9a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:30.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.903+0000 7f0b1a800640 1 --2- 172.21.15.7:0/4246101398 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x563b80c96900 0x563b80c9a4f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b899175cb338226d server_cookie=94e1e273202eb280 in_seq=0 out_seq=0 2024-09-10T21:48:30.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.903+0000 7f0afb400640 10 client.7107 ms_handle_connect on v2:172.21.15.133:6835/3312604545 2024-09-10T21:48:30.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.903+0000 7f0afb400640 1 -- 172.21.15.7:0/4246101398 <== mds.0 v2:172.21.15.133:6835/3312604545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+864 (crc 0 0 0) 0x563b80b42270 con 0x563b80c96900 2024-09-10T21:48:30.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.903+0000 7f0afb400640 10 client.7107 handle_command_reply: tid=0 2024-09-10T21:48:30.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.903+0000 7f0b243c2740 1 client.7107 shutdown 2024-09-10T21:48:30.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.904+0000 7f0b243c2740 1 -- 172.21.15.7:0/4246101398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c16560 msgr2=0x563b80c12600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.904+0000 7f0b243c2740 1 --2- 172.21.15.7:0/4246101398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c16560 0x563b80c12600 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f0b08011010 tx=0x7f0b080046c0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.904+0000 7f0b243c2740 1 -- 172.21.15.7:0/4246101398 shutdown_connections 2024-09-10T21:48:30.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.904+0000 7f0b243c2740 1 --2- 172.21.15.7:0/4246101398 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x563b80c96900 0x563b80c9a4f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.904+0000 7f0b243c2740 1 --2- 172.21.15.7:0/4246101398 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x563b80c172e0 0x563b80c12b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.904+0000 7f0b243c2740 1 --2- 172.21.15.7:0/4246101398 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x563b80c16560 0x563b80c12600 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.904+0000 7f0b243c2740 1 --2- 172.21.15.7:0/4246101398 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x563b80bd29d0 0x563b80c963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.904+0000 7f0b243c2740 1 -- 172.21.15.7:0/4246101398 >> 172.21.15.7:0/4246101398 conn(0x563b80bd2450 msgr2=0x563b80bd16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:30.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.904+0000 7f0afb400640 0 client.7107 ms_handle_reset on v2:172.21.15.133:6835/3312604545 2024-09-10T21:48:30.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.904+0000 7f0b243c2740 1 -- 172.21.15.7:0/4246101398 shutdown_connections 2024-09-10T21:48:30.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.904+0000 7f0b243c2740 1 -- 172.21.15.7:0/4246101398 wait complete. 2024-09-10T21:48:30.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.904+0000 7f0b243c2740 20 client.7107 trim_cache size 0 max 16384 2024-09-10T21:48:30.920 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:48:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:48:30.920 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-09-10T21:48:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "a1511696-8e50-4b8f-909c-34f5fc84eef4": { 2024-09-10T21:48:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/dir", 2024-09-10T21:48:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "a1511696-8e50-4b8f-909c-34f5fc84eef4", 2024-09-10T21:48:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,repair" 2024-09-10T21:48:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:48:30.921 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:48:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:48:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "a1511696-8e50-4b8f-909c-34f5fc84eef4": { 2024-09-10T21:48:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:48:30.677722+0000", 2024-09-10T21:48:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/dir", 2024-09-10T21:48:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:48:30.922 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:48:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:48:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:48:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:48:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "f38c7999-23d8-4083-82e3-33fb9a7274b3": { 2024-09-10T21:48:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:47:58.916341+0000", 2024-09-10T21:48:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/dir", 2024-09-10T21:48:30.923 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:48:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:48:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:48:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:48:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:48:30.924 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:48:30.924 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:48:30.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.905+0000 7f0b20a00640 1 -- 172.21.15.7:0/1961376085 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0afc077f70 msgr2=0x7f0afc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.905+0000 7f0b20a00640 1 --2- 172.21.15.7:0/1961376085 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0afc077f70 0x7f0afc07a430 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c000f80 tx=0x7f0b0c03b040 comp rx=0 tx=0).stop 2024-09-10T21:48:30.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.905+0000 7f0b20a00640 1 -- 172.21.15.7:0/1961376085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b1c15df70 msgr2=0x7f0b1c0b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:30.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.905+0000 7f0b20a00640 1 --2- 172.21.15.7:0/1961376085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b1c15df70 0x7f0b1c0b2cb0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400c730 tx=0x7f0b1400cbf0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.905+0000 7f0b20a00640 1 -- 172.21.15.7:0/1961376085 shutdown_connections 2024-09-10T21:48:30.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.905+0000 7f0b20a00640 1 --2- 172.21.15.7:0/1961376085 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0afc077f70 0x7f0afc07a430 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.905+0000 7f0b20a00640 1 --2- 172.21.15.7:0/1961376085 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0b1c15df70 0x7f0b1c0b2cb0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.905+0000 7f0b20a00640 1 --2- 172.21.15.7:0/1961376085 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0b1c0d22b0 0x7f0b1c15fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.905+0000 7f0b20a00640 1 --2- 172.21.15.7:0/1961376085 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0b1c0b0a10 0x7f0b1c1606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:30.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.905+0000 7f0b20a00640 1 -- 172.21.15.7:0/1961376085 >> 172.21.15.7:0/1961376085 conn(0x7f0b1c0d3e80 msgr2=0x7f0b1c15d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:30.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.905+0000 7f0b20a00640 1 -- 172.21.15.7:0/1961376085 shutdown_connections 2024-09-10T21:48:30.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:48:30.905+0000 7f0b20a00640 1 -- 172.21.15.7:0/1961376085 wait complete. 2024-09-10T21:48:30.929 INFO:tasks.cephfs.filesystem:scrub status for tag:a1511696-8e50-4b8f-909c-34f5fc84eef4 - {'path': '/dir', 'tag': 'a1511696-8e50-4b8f-909c-34f5fc84eef4', 'options': 'recursive,repair'} 2024-09-10T21:48:30.938 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:48:30.943 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:49:00.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:49:00.931 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-10T21:49:01.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:49:01.055 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:49:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.106+0000 7f3a27800640 1 -- 172.21.15.7:0/3699030493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a200b04a0 msgr2=0x7f3a20166c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.106+0000 7f3a27800640 1 --2- 172.21.15.7:0/3699030493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a200b04a0 0x7f3a20166c60 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f3a18005fd0 tx=0x7f3a1802a940 comp rx=0 tx=0).stop 2024-09-10T21:49:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.106+0000 7f3a27800640 1 -- 172.21.15.7:0/3699030493 shutdown_connections 2024-09-10T21:49:01.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.106+0000 7f3a27800640 1 --2- 172.21.15.7:0/3699030493 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a200b04a0 0x7f3a20166c60 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.106+0000 7f3a27800640 1 -- 172.21.15.7:0/3699030493 >> 172.21.15.7:0/3699030493 conn(0x7f3a200d1db0 msgr2=0x7f3a200d21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:01.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.106+0000 7f3a27800640 1 -- 172.21.15.7:0/3699030493 shutdown_connections 2024-09-10T21:49:01.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.106+0000 7f3a27800640 1 -- 172.21.15.7:0/3699030493 wait complete. 2024-09-10T21:49:01.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.107+0000 7f3a27800640 1 Processor -- start 2024-09-10T21:49:01.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.108+0000 7f3a27800640 1 -- start start 2024-09-10T21:49:01.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.108+0000 7f3a27800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a200b04a0 0x7f3a201577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.108+0000 7f3a27800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a200b2d70 0x7f3a2006dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.108+0000 7f3a27800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a2015f5b0 0x7f3a2014b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.108+0000 7f3a27800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3a20167320 con 0x7f3a200b04a0 2024-09-10T21:49:01.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.108+0000 7f3a27800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3a201671a0 con 0x7f3a200b2d70 2024-09-10T21:49:01.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.108+0000 7f3a27800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f3a201674a0 con 0x7f3a2015f5b0 2024-09-10T21:49:01.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.109+0000 7f3a24e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a200b04a0 0x7f3a201577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.109+0000 7f3a17e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a200b2d70 0x7f3a2006dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.109+0000 7f3a17e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a200b2d70 0x7f3a2006dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:53006/0 (socket says 172.21.15.7:53006) 2024-09-10T21:49:01.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.109+0000 7f3a17e00640 1 -- 172.21.15.7:0/393591546 learned_addr learned my addr 172.21.15.7:0/393591546 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:01.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.109+0000 7f3a25800640 1 --2- 172.21.15.7:0/393591546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a2015f5b0 0x7f3a2014b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.109+0000 7f3a24e00640 1 -- 172.21.15.7:0/393591546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a2015f5b0 msgr2=0x7f3a2014b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.109+0000 7f3a24e00640 1 --2- 172.21.15.7:0/393591546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a2015f5b0 0x7f3a2014b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.109+0000 7f3a24e00640 1 -- 172.21.15.7:0/393591546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a200b2d70 msgr2=0x7f3a2006dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.109+0000 7f3a24e00640 1 --2- 172.21.15.7:0/393591546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a200b2d70 0x7f3a2006dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.109+0000 7f3a24e00640 1 -- 172.21.15.7:0/393591546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2006ec90 con 0x7f3a200b04a0 2024-09-10T21:49:01.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.110+0000 7f3a24e00640 1 --2- 172.21.15.7:0/393591546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a200b04a0 0x7f3a201577f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000c3c0 tx=0x7f3a1000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:01.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.110+0000 7f3a1d600640 1 -- 172.21.15.7:0/393591546 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10011790 con 0x7f3a200b04a0 2024-09-10T21:49:01.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.110+0000 7f3a27800640 1 -- 172.21.15.7:0/393591546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a20155040 con 0x7f3a200b04a0 2024-09-10T21:49:01.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.110+0000 7f3a1d600640 1 -- 172.21.15.7:0/393591546 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a10017070 con 0x7f3a200b04a0 2024-09-10T21:49:01.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.110+0000 7f3a1d600640 1 -- 172.21.15.7:0/393591546 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10011d20 con 0x7f3a200b04a0 2024-09-10T21:49:01.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.110+0000 7f3a27800640 1 -- 172.21.15.7:0/393591546 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a2006e640 con 0x7f3a200b04a0 2024-09-10T21:49:01.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.112+0000 7f3a1d600640 1 -- 172.21.15.7:0/393591546 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3a1001c460 con 0x7f3a200b04a0 2024-09-10T21:49:01.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.113+0000 7f3a1d600640 1 --2- 172.21.15.7:0/393591546 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f39fc077f70 0x7f39fc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.113+0000 7f3a1d600640 1 -- 172.21.15.7:0/393591546 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f3a10099970 con 0x7f3a200b04a0 2024-09-10T21:49:01.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.114+0000 7f3a17e00640 1 --2- 172.21.15.7:0/393591546 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f39fc077f70 0x7f39fc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.114+0000 7f3a2a65d740 1 Processor -- start 2024-09-10T21:49:01.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.114+0000 7f3a2a65d740 1 -- start start 2024-09-10T21:49:01.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.114+0000 7f3a17e00640 1 --2- 172.21.15.7:0/393591546 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f39fc077f70 0x7f39fc07a430 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3a08000c00 tx=0x7f3a08009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:01.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.115+0000 7f3a2a65d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bc130 0x56391b6bc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.115+0000 7f3a2a65d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6772a0 0x56391b6bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.115+0000 7f3a2a65d740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b677820 0x56391b6bb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.115+0000 7f3a2a65d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56391b5d61c0 con 0x56391b6bc130 2024-09-10T21:49:01.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.115+0000 7f3a2a65d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56391b5e9870 con 0x56391b6772a0 2024-09-10T21:49:01.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.115+0000 7f3a2a65d740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x56391b5d8ca0 con 0x56391b677820 2024-09-10T21:49:01.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.115+0000 7f3a24e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bc130 0x56391b6bc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.115+0000 7f3a24e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bc130 0x56391b6bc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35742/0 (socket says 172.21.15.7:35742) 2024-09-10T21:49:01.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.115+0000 7f3a24e00640 1 -- 172.21.15.7:0/2388635344 learned_addr learned my addr 172.21.15.7:0/2388635344 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:01.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.115+0000 7f3a17e00640 1 --2- 172.21.15.7:0/2388635344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6772a0 0x56391b6bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.115+0000 7f3a25800640 1 --2- 172.21.15.7:0/2388635344 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b677820 0x56391b6bb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.116+0000 7f3a24e00640 1 -- 172.21.15.7:0/2388635344 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b677820 msgr2=0x56391b6bb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:01.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.116+0000 7f3a24e00640 1 --2- 172.21.15.7:0/2388635344 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b677820 0x56391b6bb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.116+0000 7f3a24e00640 1 -- 172.21.15.7:0/2388635344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6772a0 msgr2=0x56391b6bac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:01.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.116+0000 7f3a24e00640 1 --2- 172.21.15.7:0/2388635344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6772a0 0x56391b6bac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.116+0000 7f3a24e00640 1 -- 172.21.15.7:0/2388635344 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56391b3b17d0 con 0x56391b6bc130 2024-09-10T21:49:01.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.116+0000 7f3a17e00640 1 --2- 172.21.15.7:0/2388635344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6772a0 0x56391b6bac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:49:01.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.116+0000 7f3a24e00640 1 --2- 172.21.15.7:0/2388635344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bc130 0x56391b6bc510 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f3a10061a40 tx=0x7f3a10061850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7aef933821c1c94 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:01.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.116+0000 7f3a15600640 1 -- 172.21.15.7:0/2388635344 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10066020 con 0x56391b6bc130 2024-09-10T21:49:01.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.116+0000 7f3a15600640 1 -- 172.21.15.7:0/2388635344 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a10083e50 con 0x56391b6bc130 2024-09-10T21:49:01.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.116+0000 7f3a15600640 1 -- 172.21.15.7:0/2388635344 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10061200 con 0x56391b6bc130 2024-09-10T21:49:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.117+0000 7f3a2a65d740 1 -- 172.21.15.7:0/2388635344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bc130 msgr2=0x56391b6bc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.117+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/2388635344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bc130 0x56391b6bc510 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f3a10061a40 tx=0x7f3a10061850 comp rx=0 tx=0).stop 2024-09-10T21:49:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.117+0000 7f3a2a65d740 1 -- 172.21.15.7:0/2388635344 shutdown_connections 2024-09-10T21:49:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.117+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/2388635344 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6772a0 0x56391b6bac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.117+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/2388635344 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b677820 0x56391b6bb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.117+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/2388635344 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bc130 0x56391b6bc510 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.117+0000 7f3a2a65d740 1 -- 172.21.15.7:0/2388635344 >> 172.21.15.7:0/2388635344 conn(0x56391b672870 msgr2=0x56391b676290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.117+0000 7f3a2a65d740 1 -- 172.21.15.7:0/2388635344 shutdown_connections 2024-09-10T21:49:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.117+0000 7f3a2a65d740 1 -- 172.21.15.7:0/2388635344 wait complete. 2024-09-10T21:49:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.118+0000 7f3a2a65d740 1 -- start start 2024-09-10T21:49:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.118+0000 7f3a2a65d740 1 Processor -- start 2024-09-10T21:49:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.119+0000 7f3a2a65d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b677820 0x56391b6bad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.119+0000 7f3a2a65d740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b6bc130 0x56391b6b3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.119+0000 7f3a2a65d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6bb2d0 0x56391b69fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.119+0000 7f3a2a65d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56391b57f410 con 0x56391b677820 2024-09-10T21:49:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.119+0000 7f3a2a65d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56391b5dcab0 con 0x56391b6bb2d0 2024-09-10T21:49:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.119+0000 7f3a2a65d740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x56391b349740 con 0x56391b6bc130 2024-09-10T21:49:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.119+0000 7f3a2a65d740 10 client.0 _ll_register_callbacks cb 0x56391b55c360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:49:01.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.119+0000 7f3a24e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b677820 0x56391b6bad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.119+0000 7f3a24e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b677820 0x56391b6bad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35756/0 (socket says 172.21.15.7:35756) 2024-09-10T21:49:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.119+0000 7f3a24e00640 1 -- 172.21.15.7:0/3041777306 learned_addr learned my addr 172.21.15.7:0/3041777306 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.119+0000 7f3a17e00640 1 --2- 172.21.15.7:0/3041777306 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b6bc130 0x56391b6b3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.119+0000 7f3a25800640 1 --2- 172.21.15.7:0/3041777306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6bb2d0 0x56391b69fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.120+0000 7f3a24e00640 1 -- 172.21.15.7:0/3041777306 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b6bc130 msgr2=0x56391b6b3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.120+0000 7f3a24e00640 1 --2- 172.21.15.7:0/3041777306 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b6bc130 0x56391b6b3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.120+0000 7f3a24e00640 1 -- 172.21.15.7:0/3041777306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6bb2d0 msgr2=0x56391b69fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.120+0000 7f3a24e00640 1 --2- 172.21.15.7:0/3041777306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6bb2d0 0x56391b69fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.120+0000 7f3a24e00640 1 -- 172.21.15.7:0/3041777306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56391b33a970 con 0x56391b677820 2024-09-10T21:49:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.120+0000 7f3a24e00640 1 --2- 172.21.15.7:0/3041777306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b677820 0x56391b6bad90 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f3a1005e500 tx=0x7f3a10061250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef037afbdef3834 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.120+0000 7f3a0e000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:49:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.120+0000 7f3a0e000640 1 -- 172.21.15.7:0/3041777306 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a10066040 con 0x56391b677820 2024-09-10T21:49:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.120+0000 7f3a2a65d740 1 -- 172.21.15.7:0/3041777306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a10083e50 con 0x56391b677820 2024-09-10T21:49:01.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.120+0000 7f3a0e000640 1 -- 172.21.15.7:0/3041777306 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f3a1005d8a0 con 0x56391b677820 2024-09-10T21:49:01.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.120+0000 7f3a0e000640 1 -- 172.21.15.7:0/3041777306 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f3a10066770 con 0x56391b677820 2024-09-10T21:49:01.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.120+0000 7f3a2a65d740 10 client.7119 fetch_fsmap learned FSMap version 70 2024-09-10T21:49:01.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.120+0000 7f3a2a65d740 1 -- 172.21.15.7:0/3041777306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x56391b3186a0 con 0x56391b677820 2024-09-10T21:49:01.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.121+0000 7f3a0e000640 1 -- 172.21.15.7:0/3041777306 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a10083e50 con 0x56391b677820 2024-09-10T21:49:01.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.121+0000 7f3a0e000640 1 -- 172.21.15.7:0/3041777306 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f3a10082120 con 0x56391b677820 2024-09-10T21:49:01.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.121+0000 7f3a2a65d740 10 client.7119 fetch_fsmap finished waiting for FSMap version 70 2024-09-10T21:49:01.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.121+0000 7f3a2a65d740 10 client.7119 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-10T21:49:01.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.121+0000 7f3a2a65d740 20 client.7119 populate_metadata read hostname 'smithi007' 2024-09-10T21:49:01.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.121+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/3041777306 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x56391b6b40e0 0x56391b6ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.121+0000 7f3a2a65d740 4 client.7119 mds_command: new command op to 235778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:49:01.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.121+0000 7f3a2a65d740 1 -- 172.21.15.7:0/3041777306 --> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56391b5eb7c0 con 0x56391b6b40e0 2024-09-10T21:49:01.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.121+0000 7f3a17e00640 1 --2- 172.21.15.7:0/3041777306 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x56391b6b40e0 0x56391b6ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.122+0000 7f3a17e00640 1 --2- 172.21.15.7:0/3041777306 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x56391b6b40e0 0x56391b6ce370 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9811ce2111dc0fb server_cookie=bad522449227c222 in_seq=0 out_seq=0 2024-09-10T21:49:01.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.122+0000 7f3a0e000640 10 client.7119 ms_handle_connect on v2:172.21.15.133:6835/3312604545 2024-09-10T21:49:01.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.123+0000 7f3a0e000640 1 -- 172.21.15.7:0/3041777306 <== mds.0 v2:172.21.15.133:6835/3312604545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56391b5eb7c0 con 0x56391b6b40e0 2024-09-10T21:49:01.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.123+0000 7f3a0e000640 10 client.7119 handle_command_reply: tid=0 2024-09-10T21:49:01.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.123+0000 7f3a2a65d740 1 client.7119 shutdown 2024-09-10T21:49:01.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.123+0000 7f3a2a65d740 1 -- 172.21.15.7:0/3041777306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b677820 msgr2=0x56391b6bad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.123+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/3041777306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b677820 0x56391b6bad90 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f3a1005e500 tx=0x7f3a10061250 comp rx=0 tx=0).stop 2024-09-10T21:49:01.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.123+0000 7f3a2a65d740 1 -- 172.21.15.7:0/3041777306 shutdown_connections 2024-09-10T21:49:01.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.123+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/3041777306 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x56391b6b40e0 0x56391b6ce370 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.123+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/3041777306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6bb2d0 0x56391b69fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.123+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/3041777306 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b6bc130 0x56391b6b3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.123+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/3041777306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b677820 0x56391b6bad90 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.123+0000 7f3a2a65d740 1 -- 172.21.15.7:0/3041777306 >> 172.21.15.7:0/3041777306 conn(0x56391b6772a0 msgr2=0x56391b671440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:01.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.123+0000 7f3a0e000640 0 client.7119 ms_handle_reset on v2:172.21.15.133:6835/3312604545 2024-09-10T21:49:01.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.123+0000 7f3a2a65d740 1 -- 172.21.15.7:0/3041777306 shutdown_connections 2024-09-10T21:49:01.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.123+0000 7f3a2a65d740 1 -- 172.21.15.7:0/3041777306 wait complete. 2024-09-10T21:49:01.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.123+0000 7f3a2a65d740 20 client.7119 trim_cache size 0 max 16384 2024-09-10T21:49:01.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.142+0000 7f3a2a65d740 1 Processor -- start 2024-09-10T21:49:01.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.142+0000 7f3a2a65d740 1 -- start start 2024-09-10T21:49:01.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.143+0000 7f3a2a65d740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b677820 0x56391b6df180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.143+0000 7f3a2a65d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6bb2d0 0x56391b6713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.143+0000 7f3a2a65d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bc130 0x56391b6718f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.143+0000 7f3a2a65d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3a20205740 con 0x56391b6bc130 2024-09-10T21:49:01.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.143+0000 7f3a2a65d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3a202047a0 con 0x56391b6bb2d0 2024-09-10T21:49:01.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.143+0000 7f3a2a65d740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f3a20157000 con 0x56391b677820 2024-09-10T21:49:01.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.143+0000 7f3a25800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bc130 0x56391b6718f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.143+0000 7f3a25800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bc130 0x56391b6718f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35772/0 (socket says 172.21.15.7:35772) 2024-09-10T21:49:01.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.143+0000 7f3a25800640 1 -- 172.21.15.7:0/694171702 learned_addr learned my addr 172.21.15.7:0/694171702 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:01.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.143+0000 7f3a17e00640 1 --2- 172.21.15.7:0/694171702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6bb2d0 0x56391b6713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.144+0000 7f3a24e00640 1 --2- 172.21.15.7:0/694171702 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b677820 0x56391b6df180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.144+0000 7f3a25800640 1 -- 172.21.15.7:0/694171702 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b677820 msgr2=0x56391b6df180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.144+0000 7f3a25800640 1 --2- 172.21.15.7:0/694171702 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b677820 0x56391b6df180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.144+0000 7f3a25800640 1 -- 172.21.15.7:0/694171702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6bb2d0 msgr2=0x56391b6713b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.144+0000 7f3a25800640 1 --2- 172.21.15.7:0/694171702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6bb2d0 0x56391b6713b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.144+0000 7f3a25800640 1 -- 172.21.15.7:0/694171702 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56391b69b0a0 con 0x56391b6bc130 2024-09-10T21:49:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.144+0000 7f3a17e00640 1 --2- 172.21.15.7:0/694171702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6bb2d0 0x56391b6713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:49:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.144+0000 7f3a25800640 1 --2- 172.21.15.7:0/694171702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bc130 0x56391b6718f0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f3a1802af30 tx=0x7f3a18004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20637228406518 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.145+0000 7f3a14c00640 1 -- 172.21.15.7:0/694171702 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a1804a070 con 0x56391b6bc130 2024-09-10T21:49:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.145+0000 7f3a14c00640 1 -- 172.21.15.7:0/694171702 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a18003e20 con 0x56391b6bc130 2024-09-10T21:49:01.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.145+0000 7f3a14c00640 1 -- 172.21.15.7:0/694171702 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a18004280 con 0x56391b6bc130 2024-09-10T21:49:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.145+0000 7f3a2a65d740 1 -- 172.21.15.7:0/694171702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bc130 msgr2=0x56391b6718f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.145+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/694171702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bc130 0x56391b6718f0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f3a1802af30 tx=0x7f3a18004ab0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.145+0000 7f3a2a65d740 1 -- 172.21.15.7:0/694171702 shutdown_connections 2024-09-10T21:49:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.145+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/694171702 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6bb2d0 0x56391b6713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.145+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/694171702 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bc130 0x56391b6718f0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.145+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/694171702 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b677820 0x56391b6df180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.145+0000 7f3a2a65d740 1 -- 172.21.15.7:0/694171702 >> 172.21.15.7:0/694171702 conn(0x56391b6772a0 msgr2=0x56391b6ba5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.145+0000 7f3a2a65d740 1 -- 172.21.15.7:0/694171702 shutdown_connections 2024-09-10T21:49:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.145+0000 7f3a2a65d740 1 -- 172.21.15.7:0/694171702 wait complete. 2024-09-10T21:49:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.146+0000 7f3a2a65d740 1 -- start start 2024-09-10T21:49:01.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.146+0000 7f3a2a65d740 1 Processor -- start 2024-09-10T21:49:01.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.147+0000 7f3a2a65d740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b677820 0x56391b698ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.147+0000 7f3a2a65d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bb2d0 0x56391b721140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.147+0000 7f3a2a65d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6bc130 0x56391b6b3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.147+0000 7f3a2a65d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3a20204150 con 0x56391b6bb2d0 2024-09-10T21:49:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.147+0000 7f3a2a65d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56391b6ca5b0 con 0x56391b6bc130 2024-09-10T21:49:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.147+0000 7f3a2a65d740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x56391b6701c0 con 0x56391b677820 2024-09-10T21:49:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.147+0000 7f3a2a65d740 10 client.0 _ll_register_callbacks cb 0x7f3a08017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:49:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.147+0000 7f3a17e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bb2d0 0x56391b721140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.147+0000 7f3a17e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bb2d0 0x56391b721140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35786/0 (socket says 172.21.15.7:35786) 2024-09-10T21:49:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.147+0000 7f3a17e00640 1 -- 172.21.15.7:0/79830449 learned_addr learned my addr 172.21.15.7:0/79830449 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:01.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.147+0000 7f3a24e00640 1 --2- 172.21.15.7:0/79830449 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b677820 0x56391b698ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.148+0000 7f3a25800640 1 --2- 172.21.15.7:0/79830449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6bc130 0x56391b6b3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.148+0000 7f3a17e00640 1 -- 172.21.15.7:0/79830449 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b677820 msgr2=0x56391b698ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:01.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.148+0000 7f3a17e00640 1 --2- 172.21.15.7:0/79830449 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b677820 0x56391b698ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.148+0000 7f3a17e00640 1 -- 172.21.15.7:0/79830449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6bc130 msgr2=0x56391b6b3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:01.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.148+0000 7f3a17e00640 1 --2- 172.21.15.7:0/79830449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6bc130 0x56391b6b3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.148+0000 7f3a17e00640 1 -- 172.21.15.7:0/79830449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56391b66ff40 con 0x56391b6bb2d0 2024-09-10T21:49:01.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.148+0000 7f3a24e00640 1 --2- 172.21.15.7:0/79830449 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b677820 0x56391b698ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:01.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.148+0000 7f3a17e00640 1 --2- 172.21.15.7:0/79830449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bb2d0 0x56391b721140 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f3a08007fd0 tx=0x7f3a08012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3c64b28cfcb2efb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:01.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.148+0000 7f3a0e000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:49:01.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.148+0000 7f3a0e000640 1 -- 172.21.15.7:0/79830449 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a08014450 con 0x56391b6bb2d0 2024-09-10T21:49:01.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.148+0000 7f3a0e000640 1 -- 172.21.15.7:0/79830449 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f3a080145f0 con 0x56391b6bb2d0 2024-09-10T21:49:01.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.148+0000 7f3a2a65d740 1 -- 172.21.15.7:0/79830449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3a18003e20 con 0x56391b6bb2d0 2024-09-10T21:49:01.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.149+0000 7f3a0e000640 1 -- 172.21.15.7:0/79830449 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f3a08039050 con 0x56391b6bb2d0 2024-09-10T21:49:01.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.149+0000 7f3a2a65d740 10 client.7125 fetch_fsmap learned FSMap version 70 2024-09-10T21:49:01.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.149+0000 7f3a2a65d740 1 -- 172.21.15.7:0/79830449 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x56391b721740 con 0x56391b6bb2d0 2024-09-10T21:49:01.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.149+0000 7f3a0e000640 1 -- 172.21.15.7:0/79830449 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a18003e20 con 0x56391b6bb2d0 2024-09-10T21:49:01.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.149+0000 7f3a0e000640 1 -- 172.21.15.7:0/79830449 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f3a0803c020 con 0x56391b6bb2d0 2024-09-10T21:49:01.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.149+0000 7f3a2a65d740 10 client.7125 fetch_fsmap finished waiting for FSMap version 70 2024-09-10T21:49:01.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.149+0000 7f3a2a65d740 10 client.7125 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-10T21:49:01.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.149+0000 7f3a2a65d740 20 client.7125 populate_metadata read hostname 'smithi007' 2024-09-10T21:49:01.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.149+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/79830449 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x56391b6ba5a0 0x56391b6b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.149+0000 7f3a2a65d740 4 client.7125 mds_command: new command op to 235778 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:49:01.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.149+0000 7f3a2a65d740 1 -- 172.21.15.7:0/79830449 --> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56391b5e7270 con 0x56391b6ba5a0 2024-09-10T21:49:01.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.150+0000 7f3a24e00640 1 --2- 172.21.15.7:0/79830449 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x56391b6ba5a0 0x56391b6b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.150+0000 7f3a24e00640 1 --2- 172.21.15.7:0/79830449 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x56391b6ba5a0 0x56391b6b3db0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ee090f497191036 server_cookie=b12787f470022d15 in_seq=0 out_seq=0 2024-09-10T21:49:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.151+0000 7f3a0e000640 10 client.7125 ms_handle_connect on v2:172.21.15.133:6835/3312604545 2024-09-10T21:49:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.151+0000 7f3a0e000640 1 -- 172.21.15.7:0/79830449 <== mds.0 v2:172.21.15.133:6835/3312604545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+664 (crc 0 0 0) 0x56391b5e7270 con 0x56391b6ba5a0 2024-09-10T21:49:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.151+0000 7f3a0e000640 10 client.7125 handle_command_reply: tid=0 2024-09-10T21:49:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.151+0000 7f3a2a65d740 1 client.7125 shutdown 2024-09-10T21:49:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.151+0000 7f3a2a65d740 1 -- 172.21.15.7:0/79830449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bb2d0 msgr2=0x56391b721140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.151+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/79830449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bb2d0 0x56391b721140 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f3a08007fd0 tx=0x7f3a08012000 comp rx=0 tx=0).stop 2024-09-10T21:49:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.151+0000 7f3a2a65d740 1 -- 172.21.15.7:0/79830449 shutdown_connections 2024-09-10T21:49:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.151+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/79830449 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x56391b6ba5a0 0x56391b6b3db0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.151+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/79830449 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56391b6bc130 0x56391b6b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.151+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/79830449 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56391b6bb2d0 0x56391b721140 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.151+0000 7f3a0e000640 0 client.7125 ms_handle_reset on v2:172.21.15.133:6835/3312604545 2024-09-10T21:49:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.151+0000 7f3a2a65d740 1 --2- 172.21.15.7:0/79830449 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56391b677820 0x56391b698ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.151+0000 7f3a2a65d740 1 -- 172.21.15.7:0/79830449 >> 172.21.15.7:0/79830449 conn(0x56391b6772a0 msgr2=0x56391b6b2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:01.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.151+0000 7f3a2a65d740 1 -- 172.21.15.7:0/79830449 shutdown_connections 2024-09-10T21:49:01.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.151+0000 7f3a2a65d740 1 -- 172.21.15.7:0/79830449 wait complete. 2024-09-10T21:49:01.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.151+0000 7f3a2a65d740 20 client.7125 trim_cache size 0 max 16384 2024-09-10T21:49:01.156 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:49:01.156 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-09-10T21:49:01.156 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {}, 2024-09-10T21:49:01.156 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:49:01.156 INFO:teuthology.orchestra.run.smithi007.stdout: "a1511696-8e50-4b8f-909c-34f5fc84eef4": { 2024-09-10T21:49:01.156 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:48:30.677722+0000", 2024-09-10T21:49:01.156 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/dir", 2024-09-10T21:49:01.156 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:49:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:49:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:49:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:49:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: "f38c7999-23d8-4083-82e3-33fb9a7274b3": { 2024-09-10T21:49:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:47:58.916341+0000", 2024-09-10T21:49:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/dir", 2024-09-10T21:49:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:49:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:49:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:49:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:49:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:01.157 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:01.157 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:49:01.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.152+0000 7f3a27800640 1 -- 172.21.15.7:0/393591546 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f39fc077f70 msgr2=0x7f39fc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.152+0000 7f3a27800640 1 --2- 172.21.15.7:0/393591546 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f39fc077f70 0x7f39fc07a430 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f3a08000c00 tx=0x7f3a08009040 comp rx=0 tx=0).stop 2024-09-10T21:49:01.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.152+0000 7f3a27800640 1 -- 172.21.15.7:0/393591546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a200b04a0 msgr2=0x7f3a201577f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.152+0000 7f3a27800640 1 --2- 172.21.15.7:0/393591546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a200b04a0 0x7f3a201577f0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000c3c0 tx=0x7f3a1000c880 comp rx=0 tx=0).stop 2024-09-10T21:49:01.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.152+0000 7f3a27800640 1 -- 172.21.15.7:0/393591546 shutdown_connections 2024-09-10T21:49:01.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.152+0000 7f3a27800640 1 --2- 172.21.15.7:0/393591546 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f39fc077f70 0x7f39fc07a430 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.152+0000 7f3a27800640 1 --2- 172.21.15.7:0/393591546 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3a200b2d70 0x7f3a2006dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.152+0000 7f3a27800640 1 --2- 172.21.15.7:0/393591546 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3a2015f5b0 0x7f3a2014b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.152+0000 7f3a27800640 1 --2- 172.21.15.7:0/393591546 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3a200b04a0 0x7f3a201577f0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.152+0000 7f3a27800640 1 -- 172.21.15.7:0/393591546 >> 172.21.15.7:0/393591546 conn(0x7f3a200d1db0 msgr2=0x7f3a200b16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:01.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.152+0000 7f3a27800640 1 -- 172.21.15.7:0/393591546 shutdown_connections 2024-09-10T21:49:01.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.152+0000 7f3a27800640 1 -- 172.21.15.7:0/393591546 wait complete. 2024-09-10T21:49:01.169 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:49:01.169 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:49:01.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.371+0000 7f7045000640 1 -- 172.21.15.7:0/793771681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f704015fda0 msgr2=0x7f7040160180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.371+0000 7f7045000640 1 --2- 172.21.15.7:0/793771681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f704015fda0 0x7f7040160180 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f704006dac0 tx=0x7f703802aa30 comp rx=0 tx=0).stop 2024-09-10T21:49:01.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.371+0000 7f7045000640 1 -- 172.21.15.7:0/793771681 shutdown_connections 2024-09-10T21:49:01.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.371+0000 7f7045000640 1 --2- 172.21.15.7:0/793771681 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f704015fda0 0x7f7040160180 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.371+0000 7f7045000640 1 --2- 172.21.15.7:0/793771681 >> v2:172.21.15.133:3300/0 conn(0x7f704015bc90 0x7f70400ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.371+0000 7f7045000640 1 -- 172.21.15.7:0/793771681 >> 172.21.15.7:0/793771681 conn(0x7f70400b16d0 msgr2=0x7f70400b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:01.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.372+0000 7f7045000640 1 -- 172.21.15.7:0/793771681 shutdown_connections 2024-09-10T21:49:01.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.372+0000 7f7045000640 1 -- 172.21.15.7:0/793771681 wait complete. 2024-09-10T21:49:01.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.373+0000 7f7045000640 1 Processor -- start 2024-09-10T21:49:01.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.373+0000 7f7045000640 1 -- start start 2024-09-10T21:49:01.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.374+0000 7f7045000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70400cd750 0x7f7040153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.374+0000 7f7045000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f704015bc90 0x7f70400c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.374+0000 7f7045000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f704015fda0 0x7f70400c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.374+0000 7f7045000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f704006fe70 con 0x7f704015fda0 2024-09-10T21:49:01.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.374+0000 7f7045000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f704006fcf0 con 0x7f704015bc90 2024-09-10T21:49:01.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.374+0000 7f7045000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f704006fff0 con 0x7f70400cd750 2024-09-10T21:49:01.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.375+0000 7f703e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f704015fda0 0x7f70400c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.375+0000 7f703e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f704015fda0 0x7f70400c89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35796/0 (socket says 172.21.15.7:35796) 2024-09-10T21:49:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.375+0000 7f703e800640 1 -- 172.21.15.7:0/2036735655 learned_addr learned my addr 172.21.15.7:0/2036735655 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.375+0000 7f7035400640 1 --2- 172.21.15.7:0/2036735655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f704015bc90 0x7f70400c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.375+0000 7f703de00640 1 --2- 172.21.15.7:0/2036735655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70400cd750 0x7f7040153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.375+0000 7f703e800640 1 -- 172.21.15.7:0/2036735655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70400cd750 msgr2=0x7f7040153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.375+0000 7f703e800640 1 --2- 172.21.15.7:0/2036735655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70400cd750 0x7f7040153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.375+0000 7f703e800640 1 -- 172.21.15.7:0/2036735655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f704015bc90 msgr2=0x7f70400c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.375+0000 7f703e800640 1 --2- 172.21.15.7:0/2036735655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f704015bc90 0x7f70400c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.375+0000 7f703e800640 1 -- 172.21.15.7:0/2036735655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7040153590 con 0x7f704015fda0 2024-09-10T21:49:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.375+0000 7f703de00640 1 --2- 172.21.15.7:0/2036735655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70400cd750 0x7f7040153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:01.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.375+0000 7f703e800640 1 --2- 172.21.15.7:0/2036735655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f704015fda0 0x7f70400c89c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f7038009d60 tx=0x7f7038003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:01.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.376+0000 7f7036a00640 1 -- 172.21.15.7:0/2036735655 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038033050 con 0x7f704015fda0 2024-09-10T21:49:01.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.376+0000 7f7036a00640 1 -- 172.21.15.7:0/2036735655 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7038004450 con 0x7f704015fda0 2024-09-10T21:49:01.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.376+0000 7f7036a00640 1 -- 172.21.15.7:0/2036735655 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038038450 con 0x7f704015fda0 2024-09-10T21:49:01.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.376+0000 7f7045000640 1 -- 172.21.15.7:0/2036735655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70400c7610 con 0x7f704015fda0 2024-09-10T21:49:01.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.376+0000 7f7045000640 1 -- 172.21.15.7:0/2036735655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70400c7bd0 con 0x7f704015fda0 2024-09-10T21:49:01.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.377+0000 7f7045000640 1 -- 172.21.15.7:0/2036735655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7040070510 con 0x7f704015fda0 2024-09-10T21:49:01.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.378+0000 7f7036a00640 1 -- 172.21.15.7:0/2036735655 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7038040070 con 0x7f704015fda0 2024-09-10T21:49:01.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.379+0000 7f7036a00640 1 --2- 172.21.15.7:0/2036735655 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f701c077f70 0x7f701c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.380+0000 7f7036a00640 1 -- 172.21.15.7:0/2036735655 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f70380bb0c0 con 0x7f704015fda0 2024-09-10T21:49:01.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.380+0000 7f703de00640 1 --2- 172.21.15.7:0/2036735655 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f701c077f70 0x7f701c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.381+0000 7f703de00640 1 --2- 172.21.15.7:0/2036735655 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f701c077f70 0x7f701c07a430 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7030009c20 tx=0x7f7030005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:01.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.384+0000 7f7036a00640 1 -- 172.21.15.7:0/2036735655 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7040153590 con 0x7f704015fda0 2024-09-10T21:49:01.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.613+0000 7f7045000640 1 -- 172.21.15.7:0/2036735655 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70400b33a0 con 0x7f704015fda0 2024-09-10T21:49:01.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.614+0000 7f7036a00640 1 -- 172.21.15.7:0/2036735655 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5230 (secure 0 0 0) 0x7f703808c030 con 0x7f704015fda0 2024-09-10T21:49:01.615 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:01.615 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":70,"btime":"2024-09-10T21:48:02:184202+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/978763848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":978763848},{"type":"v1","addr":"172.21.15.7:6835","nonce":978763848}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/269527918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":269527918},{"type":"v1","addr":"172.21.15.7:6833","nonce":269527918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":235775,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/760980985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":760980985},{"type":"v1","addr":"172.21.15.133:6836","nonce":760980985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:47:40.664091+0000","modified":"2024-09-10T21:48:02.157973+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235778":{"gid":235778,"name":"d","rank":0,"incarnation":66,"state":"up:active","state_seq":10,"addr":"172.21.15.133:6837/3312604545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3312604545},{"type":"v1","addr":"172.21.15.133:6837","nonce":3312604545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235778,"qdb_cluster":[235778]},"id":6}]} 2024-09-10T21:49:01.615 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 70 2024-09-10T21:49:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.616+0000 7f7045000640 1 -- 172.21.15.7:0/2036735655 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f701c077f70 msgr2=0x7f701c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.616+0000 7f7045000640 1 --2- 172.21.15.7:0/2036735655 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f701c077f70 0x7f701c07a430 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f7030009c20 tx=0x7f7030005f90 comp rx=0 tx=0).stop 2024-09-10T21:49:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.616+0000 7f7045000640 1 -- 172.21.15.7:0/2036735655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f704015fda0 msgr2=0x7f70400c89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.616+0000 7f7045000640 1 --2- 172.21.15.7:0/2036735655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f704015fda0 0x7f70400c89c0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f7038009d60 tx=0x7f7038003f10 comp rx=0 tx=0).stop 2024-09-10T21:49:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.616+0000 7f7045000640 1 -- 172.21.15.7:0/2036735655 shutdown_connections 2024-09-10T21:49:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.616+0000 7f7045000640 1 --2- 172.21.15.7:0/2036735655 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f701c077f70 0x7f701c07a430 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.616+0000 7f7045000640 1 --2- 172.21.15.7:0/2036735655 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f704015bc90 0x7f70400c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.616+0000 7f7045000640 1 --2- 172.21.15.7:0/2036735655 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f704015fda0 0x7f70400c89c0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.616+0000 7f7045000640 1 --2- 172.21.15.7:0/2036735655 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f70400cd750 0x7f7040153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.616+0000 7f7045000640 1 -- 172.21.15.7:0/2036735655 >> 172.21.15.7:0/2036735655 conn(0x7f70400b16d0 msgr2=0x7f70400b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.616+0000 7f7045000640 1 -- 172.21.15.7:0/2036735655 shutdown_connections 2024-09-10T21:49:01.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.616+0000 7f7045000640 1 -- 172.21.15.7:0/2036735655 wait complete. 2024-09-10T21:49:01.633 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-09-10T21:49:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.795+0000 7f07cea00640 1 -- 172.21.15.7:0/1548265981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07c8158220 msgr2=0x7f07c80cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.795+0000 7f07cea00640 1 --2- 172.21.15.7:0/1548265981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07c8158220 0x7f07c80cd6f0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f07b8009830 tx=0x7f07b802a480 comp rx=0 tx=0).stop 2024-09-10T21:49:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.795+0000 7f07cea00640 1 -- 172.21.15.7:0/1548265981 shutdown_connections 2024-09-10T21:49:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.795+0000 7f07cea00640 1 --2- 172.21.15.7:0/1548265981 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07c8158220 0x7f07c80cd6f0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.795+0000 7f07cea00640 1 -- 172.21.15.7:0/1548265981 >> 172.21.15.7:0/1548265981 conn(0x7f07c80b16d0 msgr2=0x7f07c80b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.796+0000 7f07cea00640 1 -- 172.21.15.7:0/1548265981 shutdown_connections 2024-09-10T21:49:01.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.796+0000 7f07cea00640 1 -- 172.21.15.7:0/1548265981 wait complete. 2024-09-10T21:49:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.797+0000 7f07cea00640 1 Processor -- start 2024-09-10T21:49:01.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.797+0000 7f07cea00640 1 -- start start 2024-09-10T21:49:01.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.798+0000 7f07cea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07c80ae000 0x7f07c8158a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.798+0000 7f07cea00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07c8158220 0x7f07c80b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.798+0000 7f07cea00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07c8158600 0x7f07c81532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.798+0000 7f07cea00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f07c806fc30 con 0x7f07c80ae000 2024-09-10T21:49:01.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.798+0000 7f07cea00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f07c806fab0 con 0x7f07c8158220 2024-09-10T21:49:01.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.798+0000 7f07cea00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f07c806fdb0 con 0x7f07c8158600 2024-09-10T21:49:01.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.799+0000 7f07c7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07c80ae000 0x7f07c8158a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.799+0000 7f07c7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07c80ae000 0x7f07c8158a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35810/0 (socket says 172.21.15.7:35810) 2024-09-10T21:49:01.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.799+0000 7f07c7e00640 1 -- 172.21.15.7:0/226464295 learned_addr learned my addr 172.21.15.7:0/226464295 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:01.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.799+0000 7f07c7e00640 1 -- 172.21.15.7:0/226464295 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07c8158600 msgr2=0x7f07c81532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.799+0000 7f07c7400640 1 --2- 172.21.15.7:0/226464295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07c8158220 0x7f07c80b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.799+0000 7f07cca00640 1 --2- 172.21.15.7:0/226464295 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07c8158600 0x7f07c81532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.799+0000 7f07c7e00640 1 --2- 172.21.15.7:0/226464295 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07c8158600 0x7f07c81532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.799+0000 7f07c7e00640 1 -- 172.21.15.7:0/226464295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07c8158220 msgr2=0x7f07c80b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.799+0000 7f07c7e00640 1 --2- 172.21.15.7:0/226464295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07c8158220 0x7f07c80b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.799+0000 7f07c7e00640 1 -- 172.21.15.7:0/226464295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07c81576e0 con 0x7f07c80ae000 2024-09-10T21:49:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.799+0000 7f07c7400640 1 --2- 172.21.15.7:0/226464295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07c8158220 0x7f07c80b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:49:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.799+0000 7f07cca00640 1 --2- 172.21.15.7:0/226464295 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07c8158600 0x7f07c81532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:49:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.799+0000 7f07c7e00640 1 --2- 172.21.15.7:0/226464295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07c80ae000 0x7f07c8158a70 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f07b8000c00 tx=0x7f07b8032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.800+0000 7f07bfe00640 1 -- 172.21.15.7:0/226464295 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8040070 con 0x7f07c80ae000 2024-09-10T21:49:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.800+0000 7f07bfe00640 1 -- 172.21.15.7:0/226464295 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07b8003af0 con 0x7f07c80ae000 2024-09-10T21:49:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.800+0000 7f07bfe00640 1 -- 172.21.15.7:0/226464295 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8045460 con 0x7f07c80ae000 2024-09-10T21:49:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.800+0000 7f07cea00640 1 -- 172.21.15.7:0/226464295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07c815c140 con 0x7f07c80ae000 2024-09-10T21:49:01.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.800+0000 7f07cea00640 1 -- 172.21.15.7:0/226464295 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07c815c700 con 0x7f07c80ae000 2024-09-10T21:49:01.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.802+0000 7f07bfe00640 1 -- 172.21.15.7:0/226464295 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f07b804d030 con 0x7f07c80ae000 2024-09-10T21:49:01.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.803+0000 7f07bfe00640 1 --2- 172.21.15.7:0/226464295 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f07a8077fa0 0x7f07a807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.803+0000 7f07bfe00640 1 -- 172.21.15.7:0/226464295 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f07b80bb3e0 con 0x7f07c80ae000 2024-09-10T21:49:01.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.803+0000 7f07d19eb740 1 Processor -- start 2024-09-10T21:49:01.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.804+0000 7f07d19eb740 1 -- start start 2024-09-10T21:49:01.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.804+0000 7f07c7400640 1 --2- 172.21.15.7:0/226464295 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f07a8077fa0 0x7f07a807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.805+0000 7f07d19eb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f8b7a0 0x558565f88640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.805+0000 7f07d19eb740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f9d1c0 0x558565f8bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.805+0000 7f07d19eb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f8c0c0 0x558565f5cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.805+0000 7f07d19eb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558565ed3750 con 0x558565f8b7a0 2024-09-10T21:49:01.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.805+0000 7f07d19eb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558565ec2b80 con 0x558565f8c0c0 2024-09-10T21:49:01.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.805+0000 7f07d19eb740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x558565ec6990 con 0x558565f9d1c0 2024-09-10T21:49:01.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.805+0000 7f07c7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f8b7a0 0x558565f88640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.805+0000 7f07c7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f8b7a0 0x558565f88640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35816/0 (socket says 172.21.15.7:35816) 2024-09-10T21:49:01.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.805+0000 7f07c7e00640 1 -- 172.21.15.7:0/2128281513 learned_addr learned my addr 172.21.15.7:0/2128281513 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:01.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.805+0000 7f07c7400640 1 --2- 172.21.15.7:0/226464295 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f07a8077fa0 0x7f07a807a460 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f07b4007fd0 tx=0x7f07b4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:01.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.805+0000 7f07c7e00640 1 -- 172.21.15.7:0/2128281513 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f9d1c0 msgr2=0x558565f8bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:01.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.805+0000 7f07c7e00640 1 --2- 172.21.15.7:0/2128281513 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f9d1c0 0x558565f8bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.805+0000 7f07c7e00640 1 -- 172.21.15.7:0/2128281513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f8c0c0 msgr2=0x558565f5cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:01.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.805+0000 7f07c7e00640 1 --2- 172.21.15.7:0/2128281513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f8c0c0 0x558565f5cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.805+0000 7f07c7e00640 1 -- 172.21.15.7:0/2128281513 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558565e50ad0 con 0x558565f8b7a0 2024-09-10T21:49:01.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.806+0000 7f07c7e00640 1 --2- 172.21.15.7:0/2128281513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f8b7a0 0x558565f88640 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f07b8083710 tx=0x7f07b8083680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eaf6a8b823c8cba server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:01.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.806+0000 7f07b3e00640 1 -- 172.21.15.7:0/2128281513 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8083e80 con 0x558565f8b7a0 2024-09-10T21:49:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.806+0000 7f07b3e00640 1 -- 172.21.15.7:0/2128281513 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07b8088030 con 0x558565f8b7a0 2024-09-10T21:49:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.806+0000 7f07b3e00640 1 -- 172.21.15.7:0/2128281513 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b8083780 con 0x558565f8b7a0 2024-09-10T21:49:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.806+0000 7f07d19eb740 1 -- 172.21.15.7:0/2128281513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f8b7a0 msgr2=0x558565f88640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.806+0000 7f07d19eb740 1 --2- 172.21.15.7:0/2128281513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f8b7a0 0x558565f88640 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f07b8083710 tx=0x7f07b8083680 comp rx=0 tx=0).stop 2024-09-10T21:49:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.807+0000 7f07d19eb740 1 -- 172.21.15.7:0/2128281513 shutdown_connections 2024-09-10T21:49:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.807+0000 7f07d19eb740 1 --2- 172.21.15.7:0/2128281513 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f8c0c0 0x558565f5cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.807+0000 7f07d19eb740 1 --2- 172.21.15.7:0/2128281513 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f9d1c0 0x558565f8bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.807+0000 7f07d19eb740 1 --2- 172.21.15.7:0/2128281513 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f8b7a0 0x558565f88640 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.807+0000 7f07d19eb740 1 -- 172.21.15.7:0/2128281513 >> 172.21.15.7:0/2128281513 conn(0x558565f5dce0 msgr2=0x558565f61700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.807+0000 7f07d19eb740 1 -- 172.21.15.7:0/2128281513 shutdown_connections 2024-09-10T21:49:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.807+0000 7f07d19eb740 1 -- 172.21.15.7:0/2128281513 wait complete. 2024-09-10T21:49:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.807+0000 7f07d19eb740 1 -- start start 2024-09-10T21:49:01.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.808+0000 7f07d19eb740 1 Processor -- start 2024-09-10T21:49:01.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.808+0000 7f07d19eb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f8c0c0 0x558565fa6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.808+0000 7f07d19eb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 0x558565fa0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.808+0000 7f07d19eb740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f7fe20 0x558565f9fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.808+0000 7f07d19eb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558565c33700 con 0x558565f9d1c0 2024-09-10T21:49:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.808+0000 7f07d19eb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558565e68430 con 0x558565f8c0c0 2024-09-10T21:49:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.808+0000 7f07d19eb740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x558565f62780 con 0x558565f7fe20 2024-09-10T21:49:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.809+0000 7f07d19eb740 10 client.0 _ll_register_callbacks cb 0x558565e6e600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:49:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.809+0000 7f07c7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 0x558565fa0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.809+0000 7f07c7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 0x558565fa0f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35820/0 (socket says 172.21.15.7:35820) 2024-09-10T21:49:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.809+0000 7f07c7400640 1 -- 172.21.15.7:0/2186176943 learned_addr learned my addr 172.21.15.7:0/2186176943 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.809+0000 7f07c7e00640 1 --2- 172.21.15.7:0/2186176943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f8c0c0 0x558565fa6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.809+0000 7f07cca00640 1 --2- 172.21.15.7:0/2186176943 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f7fe20 0x558565f9fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.809+0000 7f07c7400640 1 -- 172.21.15.7:0/2186176943 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f7fe20 msgr2=0x558565f9fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.809+0000 7f07c7400640 1 --2- 172.21.15.7:0/2186176943 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f7fe20 0x558565f9fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.809+0000 7f07c7400640 1 -- 172.21.15.7:0/2186176943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f8c0c0 msgr2=0x558565fa6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:01.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.809+0000 7f07c7400640 1 --2- 172.21.15.7:0/2186176943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f8c0c0 0x558565fa6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.809+0000 7f07c7400640 1 -- 172.21.15.7:0/2186176943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558565c9d7c0 con 0x558565f9d1c0 2024-09-10T21:49:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.809+0000 7f07cca00640 1 --2- 172.21.15.7:0/2186176943 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f7fe20 0x558565f9fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.809+0000 7f07c7400640 1 --2- 172.21.15.7:0/2186176943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 0x558565fa0f90 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f07b400dbf0 tx=0x7f07b4014420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117928550ca97b25 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.809+0000 7f07b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:49:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.810+0000 7f07b0c00640 1 -- 172.21.15.7:0/2186176943 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07b4024060 con 0x558565f9d1c0 2024-09-10T21:49:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.810+0000 7f07b0c00640 1 -- 172.21.15.7:0/2186176943 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f07b401f400 con 0x558565f9d1c0 2024-09-10T21:49:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.810+0000 7f07d19eb740 1 -- 172.21.15.7:0/2186176943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07b8088030 con 0x558565f9d1c0 2024-09-10T21:49:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.810+0000 7f07b0c00640 1 -- 172.21.15.7:0/2186176943 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f07b402a050 con 0x558565f9d1c0 2024-09-10T21:49:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.810+0000 7f07d19eb740 10 client.7149 fetch_fsmap learned FSMap version 70 2024-09-10T21:49:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.810+0000 7f07d19eb740 1 -- 172.21.15.7:0/2186176943 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x558565cfdf50 con 0x558565f9d1c0 2024-09-10T21:49:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.810+0000 7f07b0c00640 1 -- 172.21.15.7:0/2186176943 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07b8088030 con 0x558565f9d1c0 2024-09-10T21:49:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.810+0000 7f07b0c00640 1 -- 172.21.15.7:0/2186176943 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f07b402d020 con 0x558565f9d1c0 2024-09-10T21:49:01.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.810+0000 7f07d19eb740 10 client.7149 fetch_fsmap finished waiting for FSMap version 70 2024-09-10T21:49:01.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.810+0000 7f07d19eb740 10 client.7149 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:49:01.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.810+0000 7f07d19eb740 20 client.7149 populate_metadata read hostname 'smithi007' 2024-09-10T21:49:01.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.810+0000 7f07d19eb740 1 --2- 172.21.15.7:0/2186176943 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x558565fb97e0 0x558565fa35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.811+0000 7f07d19eb740 4 client.7149 mds_command: new command op to 235778 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:49:01.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.811+0000 7f07d19eb740 1 -- 172.21.15.7:0/2186176943 --> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558565ed56a0 con 0x558565fb97e0 2024-09-10T21:49:01.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.811+0000 7f07c7e00640 1 --2- 172.21.15.7:0/2186176943 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x558565fb97e0 0x558565fa35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.811+0000 7f07c7e00640 1 --2- 172.21.15.7:0/2186176943 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x558565fb97e0 0x558565fa35e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c6109215bb926f2 server_cookie=225c935cce2b94c8 in_seq=0 out_seq=0 2024-09-10T21:49:01.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.811+0000 7f07b0c00640 10 client.7149 ms_handle_connect on v2:172.21.15.133:6835/3312604545 2024-09-10T21:49:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.812+0000 7f07b0c00640 1 -- 172.21.15.7:0/2186176943 <== mds.0 v2:172.21.15.133:6835/3312604545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558565ed56a0 con 0x558565fb97e0 2024-09-10T21:49:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.812+0000 7f07b0c00640 10 client.7149 handle_command_reply: tid=0 2024-09-10T21:49:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.812+0000 7f07d19eb740 1 client.7149 shutdown 2024-09-10T21:49:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.813+0000 7f07d19eb740 1 -- 172.21.15.7:0/2186176943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 msgr2=0x558565fa0f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.813+0000 7f07d19eb740 1 --2- 172.21.15.7:0/2186176943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 0x558565fa0f90 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f07b400dbf0 tx=0x7f07b4014420 comp rx=0 tx=0).stop 2024-09-10T21:49:01.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.813+0000 7f07d19eb740 1 -- 172.21.15.7:0/2186176943 shutdown_connections 2024-09-10T21:49:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.813+0000 7f07d19eb740 1 --2- 172.21.15.7:0/2186176943 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x558565fb97e0 0x558565fa35e0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.813+0000 7f07d19eb740 1 --2- 172.21.15.7:0/2186176943 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f7fe20 0x558565f9fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.813+0000 7f07d19eb740 1 --2- 172.21.15.7:0/2186176943 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 0x558565fa0f90 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.813+0000 7f07d19eb740 1 --2- 172.21.15.7:0/2186176943 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f8c0c0 0x558565fa6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.813+0000 7f07b0c00640 0 client.7149 ms_handle_reset on v2:172.21.15.133:6835/3312604545 2024-09-10T21:49:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.813+0000 7f07d19eb740 1 -- 172.21.15.7:0/2186176943 >> 172.21.15.7:0/2186176943 conn(0x558565f8b7a0 msgr2=0x558565f87070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.813+0000 7f07d19eb740 1 -- 172.21.15.7:0/2186176943 shutdown_connections 2024-09-10T21:49:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.813+0000 7f07d19eb740 1 -- 172.21.15.7:0/2186176943 wait complete. 2024-09-10T21:49:01.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.813+0000 7f07d19eb740 20 client.7149 trim_cache size 0 max 16384 2024-09-10T21:49:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.827+0000 7f07d19eb740 1 Processor -- start 2024-09-10T21:49:01.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.827+0000 7f07d19eb740 1 -- start start 2024-09-10T21:49:01.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.828+0000 7f07d19eb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f7fe20 0x558565f9f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.828+0000 7f07d19eb740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f8c0c0 0x558565f9faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.828+0000 7f07d19eb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 0x558565f5ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.828+0000 7f07d19eb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81f9370 con 0x558565f9d1c0 2024-09-10T21:49:01.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.828+0000 7f07d19eb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f07c81f86d0 con 0x558565f7fe20 2024-09-10T21:49:01.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.828+0000 7f07d19eb740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f07c81f8d20 con 0x558565f8c0c0 2024-09-10T21:49:01.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.828+0000 7f07cca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 0x558565f5ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.828+0000 7f07cca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 0x558565f5ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35832/0 (socket says 172.21.15.7:35832) 2024-09-10T21:49:01.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.828+0000 7f07cca00640 1 -- 172.21.15.7:0/170100567 learned_addr learned my addr 172.21.15.7:0/170100567 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:01.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.829+0000 7f07c7e00640 1 --2- 172.21.15.7:0/170100567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f7fe20 0x558565f9f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.829+0000 7f07c7400640 1 --2- 172.21.15.7:0/170100567 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f8c0c0 0x558565f9faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.829+0000 7f07cca00640 1 -- 172.21.15.7:0/170100567 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f8c0c0 msgr2=0x558565f9faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:01.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.829+0000 7f07cca00640 1 --2- 172.21.15.7:0/170100567 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f8c0c0 0x558565f9faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.829+0000 7f07cca00640 1 -- 172.21.15.7:0/170100567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f7fe20 msgr2=0x558565f9f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:01.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.829+0000 7f07cca00640 1 --2- 172.21.15.7:0/170100567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f7fe20 0x558565f9f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.829+0000 7f07cca00640 1 -- 172.21.15.7:0/170100567 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558565c02660 con 0x558565f9d1c0 2024-09-10T21:49:01.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.829+0000 7f07c7e00640 1 --2- 172.21.15.7:0/170100567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f7fe20 0x558565f9f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:01.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.829+0000 7f07cca00640 1 --2- 172.21.15.7:0/170100567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 0x558565f5ccb0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f07c0005410 tx=0x7f07c00058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f191efde81f1fbbe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:01.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.829+0000 7f07b3400640 1 -- 172.21.15.7:0/170100567 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0017020 con 0x558565f9d1c0 2024-09-10T21:49:01.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.829+0000 7f07b3400640 1 -- 172.21.15.7:0/170100567 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07c000f930 con 0x558565f9d1c0 2024-09-10T21:49:01.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.829+0000 7f07b3400640 1 -- 172.21.15.7:0/170100567 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c000fd80 con 0x558565f9d1c0 2024-09-10T21:49:01.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.830+0000 7f07d19eb740 1 -- 172.21.15.7:0/170100567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 msgr2=0x558565f5ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.830+0000 7f07d19eb740 1 --2- 172.21.15.7:0/170100567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 0x558565f5ccb0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f07c0005410 tx=0x7f07c00058d0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.830+0000 7f07d19eb740 1 -- 172.21.15.7:0/170100567 shutdown_connections 2024-09-10T21:49:01.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.830+0000 7f07d19eb740 1 --2- 172.21.15.7:0/170100567 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 0x558565f5ccb0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.830+0000 7f07d19eb740 1 --2- 172.21.15.7:0/170100567 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f8c0c0 0x558565f9faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.830+0000 7f07d19eb740 1 --2- 172.21.15.7:0/170100567 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f7fe20 0x558565f9f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.830+0000 7f07d19eb740 1 -- 172.21.15.7:0/170100567 >> 172.21.15.7:0/170100567 conn(0x558565f8b7a0 msgr2=0x558565f5ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:01.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.830+0000 7f07d19eb740 1 -- 172.21.15.7:0/170100567 shutdown_connections 2024-09-10T21:49:01.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.830+0000 7f07d19eb740 1 -- 172.21.15.7:0/170100567 wait complete. 2024-09-10T21:49:01.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.831+0000 7f07d19eb740 1 -- start start 2024-09-10T21:49:01.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.831+0000 7f07d19eb740 1 Processor -- start 2024-09-10T21:49:01.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.832+0000 7f07d19eb740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f7fe20 0x558565fa3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.832+0000 7f07d19eb740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f8c0c0 0x558565f5ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.832+0000 7f07d19eb740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 0x558565f5d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.832+0000 7f07d19eb740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558565fa1660 con 0x558565f9d1c0 2024-09-10T21:49:01.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.832+0000 7f07d19eb740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558565fa44f0 con 0x558565f7fe20 2024-09-10T21:49:01.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.832+0000 7f07d19eb740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x558565fb9c60 con 0x558565f8c0c0 2024-09-10T21:49:01.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.832+0000 7f07d19eb740 10 client.0 _ll_register_callbacks cb 0x7f07b400d840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:49:01.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.832+0000 7f07cca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 0x558565f5d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.832+0000 7f07cca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 0x558565f5d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35842/0 (socket says 172.21.15.7:35842) 2024-09-10T21:49:01.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.832+0000 7f07cca00640 1 -- 172.21.15.7:0/429412772 learned_addr learned my addr 172.21.15.7:0/429412772 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:01.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.832+0000 7f07c7400640 1 --2- 172.21.15.7:0/429412772 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f8c0c0 0x558565f5ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.832+0000 7f07c7e00640 1 --2- 172.21.15.7:0/429412772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f7fe20 0x558565fa3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.832+0000 7f07cca00640 1 -- 172.21.15.7:0/429412772 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f8c0c0 msgr2=0x558565f5ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:01.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.832+0000 7f07cca00640 1 --2- 172.21.15.7:0/429412772 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f8c0c0 0x558565f5ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.832+0000 7f07cca00640 1 -- 172.21.15.7:0/429412772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f7fe20 msgr2=0x558565fa3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:01.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.832+0000 7f07cca00640 1 --2- 172.21.15.7:0/429412772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f7fe20 0x558565fa3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.832+0000 7f07cca00640 1 -- 172.21.15.7:0/429412772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558565e8f630 con 0x558565f9d1c0 2024-09-10T21:49:01.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.832+0000 7f07c7e00640 1 --2- 172.21.15.7:0/429412772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f7fe20 0x558565fa3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:01.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.833+0000 7f07cca00640 1 --2- 172.21.15.7:0/429412772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 0x558565f5d1f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f07c000f660 tx=0x7f07c0010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632d57565ecdeed5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:01.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.833+0000 7f07b0c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:49:01.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.833+0000 7f07b0c00640 1 -- 172.21.15.7:0/429412772 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c0017070 con 0x558565f9d1c0 2024-09-10T21:49:01.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.833+0000 7f07b0c00640 1 -- 172.21.15.7:0/429412772 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f07c0007450 con 0x558565f9d1c0 2024-09-10T21:49:01.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.833+0000 7f07d19eb740 1 -- 172.21.15.7:0/429412772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07c000f930 con 0x558565f9d1c0 2024-09-10T21:49:01.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.833+0000 7f07b0c00640 1 -- 172.21.15.7:0/429412772 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f07c002c050 con 0x558565f9d1c0 2024-09-10T21:49:01.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.833+0000 7f07d19eb740 10 client.7155 fetch_fsmap learned FSMap version 70 2024-09-10T21:49:01.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.833+0000 7f07d19eb740 1 -- 172.21.15.7:0/429412772 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x558565fb5e20 con 0x558565f9d1c0 2024-09-10T21:49:01.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.834+0000 7f07b0c00640 1 -- 172.21.15.7:0/429412772 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f07c000f930 con 0x558565f9d1c0 2024-09-10T21:49:01.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.834+0000 7f07b0c00640 1 -- 172.21.15.7:0/429412772 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f07c002f020 con 0x558565f9d1c0 2024-09-10T21:49:01.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.834+0000 7f07d19eb740 10 client.7155 fetch_fsmap finished waiting for FSMap version 70 2024-09-10T21:49:01.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.834+0000 7f07d19eb740 10 client.7155 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:49:01.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.834+0000 7f07d19eb740 20 client.7155 populate_metadata read hostname 'smithi007' 2024-09-10T21:49:01.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.834+0000 7f07d19eb740 1 --2- 172.21.15.7:0/429412772 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x558566025e80 0x558565f5b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:01.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.834+0000 7f07d19eb740 4 client.7155 mds_command: new command op to 235778 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-10T21:49:01.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.834+0000 7f07d19eb740 1 -- 172.21.15.7:0/429412772 --> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x558565ed1150 con 0x558566025e80 2024-09-10T21:49:01.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.834+0000 7f07c7e00640 1 --2- 172.21.15.7:0/429412772 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x558566025e80 0x558565f5b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:01.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.835+0000 7f07c7e00640 1 --2- 172.21.15.7:0/429412772 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x558566025e80 0x558565f5b530 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd6e83ec0f779b7d server_cookie=9548f9847ec95279 in_seq=0 out_seq=0 2024-09-10T21:49:01.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.835+0000 7f07b0c00640 10 client.7155 ms_handle_connect on v2:172.21.15.133:6835/3312604545 2024-09-10T21:49:01.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.835+0000 7f07b0c00640 1 -- 172.21.15.7:0/429412772 <== mds.0 v2:172.21.15.133:6835/3312604545 1 ==== command_reply(tid 0: 0 ) ==== 8+0+144 (crc 0 0 0) 0x558565ed1150 con 0x558566025e80 2024-09-10T21:49:01.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.835+0000 7f07b0c00640 10 client.7155 handle_command_reply: tid=0 2024-09-10T21:49:01.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.835+0000 7f07d19eb740 1 client.7155 shutdown 2024-09-10T21:49:01.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.836+0000 7f07d19eb740 1 -- 172.21.15.7:0/429412772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 msgr2=0x558565f5d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.836+0000 7f07d19eb740 1 --2- 172.21.15.7:0/429412772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 0x558565f5d1f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f07c000f660 tx=0x7f07c0010d20 comp rx=0 tx=0).stop 2024-09-10T21:49:01.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.836+0000 7f07d19eb740 1 -- 172.21.15.7:0/429412772 shutdown_connections 2024-09-10T21:49:01.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.836+0000 7f07d19eb740 1 --2- 172.21.15.7:0/429412772 >> [v2:172.21.15.133:6835/3312604545,v1:172.21.15.133:6837/3312604545] conn(0x558566025e80 0x558565f5b530 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.836+0000 7f07d19eb740 1 --2- 172.21.15.7:0/429412772 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x558565f9d1c0 0x558565f5d1f0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.836+0000 7f07d19eb740 1 --2- 172.21.15.7:0/429412772 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x558565f8c0c0 0x558565f5ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.836+0000 7f07d19eb740 1 --2- 172.21.15.7:0/429412772 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x558565f7fe20 0x558565fa3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.836+0000 7f07b0c00640 0 client.7155 ms_handle_reset on v2:172.21.15.133:6835/3312604545 2024-09-10T21:49:01.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.836+0000 7f07d19eb740 1 -- 172.21.15.7:0/429412772 >> 172.21.15.7:0/429412772 conn(0x558565f8b7a0 msgr2=0x558565f5ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:01.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.836+0000 7f07d19eb740 1 -- 172.21.15.7:0/429412772 shutdown_connections 2024-09-10T21:49:01.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.836+0000 7f07d19eb740 1 -- 172.21.15.7:0/429412772 wait complete. 2024-09-10T21:49:01.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.836+0000 7f07d19eb740 20 client.7155 trim_cache size 0 max 16384 2024-09-10T21:49:01.841 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:01.841 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-09-10T21:49:01.841 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:01.841 INFO:teuthology.orchestra.run.smithi007.stdout: "damage_type": "backtrace", 2024-09-10T21:49:01.841 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 1189945810, 2024-09-10T21:49:01.841 INFO:teuthology.orchestra.run.smithi007.stdout: "ino": 1099511627777, 2024-09-10T21:49:01.841 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/testdir/hardlink" 2024-09-10T21:49:01.841 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:01.841 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-09-10T21:49:01.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.837+0000 7f07cea00640 1 -- 172.21.15.7:0/226464295 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f07a8077fa0 msgr2=0x7f07a807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.837+0000 7f07cea00640 1 --2- 172.21.15.7:0/226464295 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f07a8077fa0 0x7f07a807a460 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f07b4007fd0 tx=0x7f07b4009040 comp rx=0 tx=0).stop 2024-09-10T21:49:01.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.837+0000 7f07cea00640 1 -- 172.21.15.7:0/226464295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07c80ae000 msgr2=0x7f07c8158a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:01.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.837+0000 7f07cea00640 1 --2- 172.21.15.7:0/226464295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07c80ae000 0x7f07c8158a70 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f07b8000c00 tx=0x7f07b8032d50 comp rx=0 tx=0).stop 2024-09-10T21:49:01.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.837+0000 7f07cea00640 1 -- 172.21.15.7:0/226464295 shutdown_connections 2024-09-10T21:49:01.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.837+0000 7f07cea00640 1 --2- 172.21.15.7:0/226464295 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f07a8077fa0 0x7f07a807a460 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.837+0000 7f07cea00640 1 --2- 172.21.15.7:0/226464295 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f07c8158220 0x7f07c80b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.837+0000 7f07cea00640 1 --2- 172.21.15.7:0/226464295 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f07c8158600 0x7f07c81532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.837+0000 7f07cea00640 1 --2- 172.21.15.7:0/226464295 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f07c80ae000 0x7f07c8158a70 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.837+0000 7f07cea00640 1 -- 172.21.15.7:0/226464295 >> 172.21.15.7:0/226464295 conn(0x7f07c80b16d0 msgr2=0x7f07c815fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.837+0000 7f07cea00640 1 -- 172.21.15.7:0/226464295 shutdown_connections 2024-09-10T21:49:01.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:01.837+0000 7f07cea00640 1 -- 172.21.15.7:0/226464295 wait complete. 2024-09-10T21:49:01.853 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:49:01.853 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:49:01.888 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:49:01.889 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:49:01.964 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.165+0000 7f847da00640 1 -- 172.21.15.7:0/994407532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84780b4970 msgr2=0x7f847806e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.165+0000 7f847da00640 1 --2- 172.21.15.7:0/994407532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84780b4970 0x7f847806e4a0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f8478076360 tx=0x7f846402a4d0 comp rx=0 tx=0).stop 2024-09-10T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.165+0000 7f847da00640 1 -- 172.21.15.7:0/994407532 shutdown_connections 2024-09-10T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.165+0000 7f847da00640 1 --2- 172.21.15.7:0/994407532 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f84780b4970 0x7f847806e4a0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.165+0000 7f847da00640 1 --2- 172.21.15.7:0/994407532 >> v2:172.21.15.133:3300/0 conn(0x7f847815b6a0 0x7f847815eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.165+0000 7f847da00640 1 -- 172.21.15.7:0/994407532 >> 172.21.15.7:0/994407532 conn(0x7f84780b16d0 msgr2=0x7f84780b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.165+0000 7f847da00640 1 -- 172.21.15.7:0/994407532 shutdown_connections 2024-09-10T21:49:02.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.165+0000 7f847da00640 1 -- 172.21.15.7:0/994407532 wait complete. 2024-09-10T21:49:02.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.166+0000 7f847da00640 1 Processor -- start 2024-09-10T21:49:02.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.166+0000 7f847da00640 1 -- start start 2024-09-10T21:49:02.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.167+0000 7f847da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847806e9e0 0x7f847813e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:02.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.167+0000 7f847da00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84780b4970 0x7f84781f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:02.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.167+0000 7f847da00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f847815b6a0 0x7f847815c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.167+0000 7f847da00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8478161dd0 con 0x7f847806e9e0 2024-09-10T21:49:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.167+0000 7f847da00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f84780aecc0 con 0x7f847815b6a0 2024-09-10T21:49:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.167+0000 7f847da00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f8478161f50 con 0x7f84780b4970 2024-09-10T21:49:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.167+0000 7f8477e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847806e9e0 0x7f847813e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.168+0000 7f8477e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847806e9e0 0x7f847813e1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35866/0 (socket says 172.21.15.7:35866) 2024-09-10T21:49:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.168+0000 7f8477e00640 1 -- 172.21.15.7:0/2359691914 learned_addr learned my addr 172.21.15.7:0/2359691914 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.168+0000 7f8477400640 1 --2- 172.21.15.7:0/2359691914 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84780b4970 0x7f84781f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:02.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.168+0000 7f847d000640 1 --2- 172.21.15.7:0/2359691914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f847815b6a0 0x7f847815c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:02.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.168+0000 7f8477e00640 1 -- 172.21.15.7:0/2359691914 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84780b4970 msgr2=0x7f84781f7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:02.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.168+0000 7f8477e00640 1 --2- 172.21.15.7:0/2359691914 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84780b4970 0x7f84781f7b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:02.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.168+0000 7f8477e00640 1 -- 172.21.15.7:0/2359691914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f847815b6a0 msgr2=0x7f847815c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:02.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.168+0000 7f8477e00640 1 --2- 172.21.15.7:0/2359691914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f847815b6a0 0x7f847815c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:02.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.168+0000 7f8477e00640 1 -- 172.21.15.7:0/2359691914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84781f84f0 con 0x7f847806e9e0 2024-09-10T21:49:02.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.168+0000 7f8477e00640 1 --2- 172.21.15.7:0/2359691914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847806e9e0 0x7f847813e1b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f846800ba50 tx=0x7f846800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:02.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.169+0000 7f846fe00640 1 -- 172.21.15.7:0/2359691914 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8468019070 con 0x7f847806e9e0 2024-09-10T21:49:02.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.169+0000 7f846fe00640 1 -- 172.21.15.7:0/2359691914 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f846800ccf0 con 0x7f847806e9e0 2024-09-10T21:49:02.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.169+0000 7f846fe00640 1 -- 172.21.15.7:0/2359691914 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84680146a0 con 0x7f847806e9e0 2024-09-10T21:49:02.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.169+0000 7f847da00640 1 -- 172.21.15.7:0/2359691914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84781f87d0 con 0x7f847806e9e0 2024-09-10T21:49:02.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.169+0000 7f847da00640 1 -- 172.21.15.7:0/2359691914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f847813f3e0 con 0x7f847806e9e0 2024-09-10T21:49:02.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.170+0000 7f847da00640 1 -- 172.21.15.7:0/2359691914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8478162450 con 0x7f847806e9e0 2024-09-10T21:49:02.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.171+0000 7f846fe00640 1 -- 172.21.15.7:0/2359691914 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f846801d3b0 con 0x7f847806e9e0 2024-09-10T21:49:02.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.172+0000 7f846fe00640 1 --2- 172.21.15.7:0/2359691914 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8458077f70 0x7f845807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:02.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.172+0000 7f846fe00640 1 -- 172.21.15.7:0/2359691914 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f8468099db0 con 0x7f847806e9e0 2024-09-10T21:49:02.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.173+0000 7f8477400640 1 --2- 172.21.15.7:0/2359691914 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8458077f70 0x7f845807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:02.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.173+0000 7f8477400640 1 --2- 172.21.15.7:0/2359691914 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8458077f70 0x7f845807a430 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8464033000 tx=0x7f8464004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:02.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.176+0000 7f846fe00640 1 -- 172.21.15.7:0/2359691914 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f846809e070 con 0x7f847806e9e0 2024-09-10T21:49:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.386+0000 7f847da00640 1 -- 172.21.15.7:0/2359691914 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f847815d8d0 con 0x7f847806e9e0 2024-09-10T21:49:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.387+0000 7f846fe00640 1 -- 172.21.15.7:0/2359691914 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v88) ==== 80+0+120 (secure 0 0 0) 0x7f8468062200 con 0x7f847806e9e0 2024-09-10T21:49:02.388 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6833/1499133391 2024-09-11T21:47:51.143906+0000 2024-09-10T21:49:02.388 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6832/1499133391 2024-09-11T21:47:51.143906+0000 2024-09-10T21:49:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:listed 2 entries 2024-09-10T21:49:02.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.389+0000 7f847da00640 1 -- 172.21.15.7:0/2359691914 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8458077f70 msgr2=0x7f845807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:02.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.389+0000 7f847da00640 1 --2- 172.21.15.7:0/2359691914 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8458077f70 0x7f845807a430 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f8464033000 tx=0x7f8464004730 comp rx=0 tx=0).stop 2024-09-10T21:49:02.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.389+0000 7f847da00640 1 -- 172.21.15.7:0/2359691914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847806e9e0 msgr2=0x7f847813e1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.389+0000 7f847da00640 1 --2- 172.21.15.7:0/2359691914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847806e9e0 0x7f847813e1b0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f846800ba50 tx=0x7f846800bf10 comp rx=0 tx=0).stop 2024-09-10T21:49:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.389+0000 7f847da00640 1 -- 172.21.15.7:0/2359691914 shutdown_connections 2024-09-10T21:49:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.389+0000 7f847da00640 1 --2- 172.21.15.7:0/2359691914 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8458077f70 0x7f845807a430 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.389+0000 7f847da00640 1 --2- 172.21.15.7:0/2359691914 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f847815b6a0 0x7f847815c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.389+0000 7f847da00640 1 --2- 172.21.15.7:0/2359691914 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f84780b4970 0x7f84781f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.389+0000 7f847da00640 1 --2- 172.21.15.7:0/2359691914 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f847806e9e0 0x7f847813e1b0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.389+0000 7f847da00640 1 -- 172.21.15.7:0/2359691914 >> 172.21.15.7:0/2359691914 conn(0x7f84780b16d0 msgr2=0x7f84780cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.389+0000 7f847da00640 1 -- 172.21.15.7:0/2359691914 shutdown_connections 2024-09-10T21:49:02.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.389+0000 7f847da00640 1 -- 172.21.15.7:0/2359691914 wait complete. 2024-09-10T21:49:02.405 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:49:02.405 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:49:02.406 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:49:02.468 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:49:02.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:49:02.468 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:49:02.504 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:49:02.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.702+0000 7f0453200640 1 -- 172.21.15.7:0/4158606536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f044c15bc90 msgr2=0x7f044c0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:02.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.702+0000 7f0453200640 1 --2- 172.21.15.7:0/4158606536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f044c15bc90 0x7f044c0ccef0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f044c0c3650 tx=0x7f043c02a4d0 comp rx=0 tx=0).stop 2024-09-10T21:49:02.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.703+0000 7f0453200640 1 -- 172.21.15.7:0/4158606536 shutdown_connections 2024-09-10T21:49:02.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.703+0000 7f0453200640 1 --2- 172.21.15.7:0/4158606536 >> v2:172.21.15.133:3300/0 conn(0x7f044c15fda0 0x7f044c160180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:02.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.703+0000 7f0453200640 1 --2- 172.21.15.7:0/4158606536 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f044c15bc90 0x7f044c0ccef0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:02.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.703+0000 7f0453200640 1 -- 172.21.15.7:0/4158606536 >> 172.21.15.7:0/4158606536 conn(0x7f044c0b16d0 msgr2=0x7f044c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:02.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.703+0000 7f0453200640 1 -- 172.21.15.7:0/4158606536 shutdown_connections 2024-09-10T21:49:02.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.703+0000 7f0453200640 1 -- 172.21.15.7:0/4158606536 wait complete. 2024-09-10T21:49:02.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.704+0000 7f0453200640 1 Processor -- start 2024-09-10T21:49:02.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.705+0000 7f0453200640 1 -- start start 2024-09-10T21:49:02.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.705+0000 7f0453200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f044c0cd750 0x7f044c158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:02.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.705+0000 7f0453200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f044c15bc90 0x7f044c0723e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:02.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.705+0000 7f0453200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f044c15fda0 0x7f044c072920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:02.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.705+0000 7f0453200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f044c06fe70 con 0x7f044c15fda0 2024-09-10T21:49:02.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.705+0000 7f0453200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f044c06fcf0 con 0x7f044c0cd750 2024-09-10T21:49:02.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.705+0000 7f0453200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f044c06fff0 con 0x7f044c15bc90 2024-09-10T21:49:02.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.706+0000 7f0451200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f044c15fda0 0x7f044c072920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:02.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.706+0000 7f0451200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f044c15fda0 0x7f044c072920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35878/0 (socket says 172.21.15.7:35878) 2024-09-10T21:49:02.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.706+0000 7f0451200640 1 -- 172.21.15.7:0/3064385924 learned_addr learned my addr 172.21.15.7:0/3064385924 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:02.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.706+0000 7f044be00640 1 --2- 172.21.15.7:0/3064385924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f044c0cd750 0x7f044c158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:02.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.706+0000 7f0443e00640 1 --2- 172.21.15.7:0/3064385924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f044c15bc90 0x7f044c0723e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:02.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.706+0000 7f0451200640 1 -- 172.21.15.7:0/3064385924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f044c15bc90 msgr2=0x7f044c0723e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:02.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.706+0000 7f0451200640 1 --2- 172.21.15.7:0/3064385924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f044c15bc90 0x7f044c0723e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:02.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.706+0000 7f0451200640 1 -- 172.21.15.7:0/3064385924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f044c0cd750 msgr2=0x7f044c158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:02.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.706+0000 7f0451200640 1 --2- 172.21.15.7:0/3064385924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f044c0cd750 0x7f044c158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:02.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.706+0000 7f0451200640 1 -- 172.21.15.7:0/3064385924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044c0c6610 con 0x7f044c15fda0 2024-09-10T21:49:02.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.706+0000 7f0451200640 1 --2- 172.21.15.7:0/3064385924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f044c15fda0 0x7f044c072920 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f044400e8b0 tx=0x7f044400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:02.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.707+0000 7f0448c00640 1 -- 172.21.15.7:0/3064385924 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444004d20 con 0x7f044c15fda0 2024-09-10T21:49:02.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.707+0000 7f0448c00640 1 -- 172.21.15.7:0/3064385924 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f044400ca60 con 0x7f044c15fda0 2024-09-10T21:49:02.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.707+0000 7f0453200640 1 -- 172.21.15.7:0/3064385924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044c1f8a60 con 0x7f044c15fda0 2024-09-10T21:49:02.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.707+0000 7f0448c00640 1 -- 172.21.15.7:0/3064385924 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0444005020 con 0x7f044c15fda0 2024-09-10T21:49:02.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.707+0000 7f0453200640 1 -- 172.21.15.7:0/3064385924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f044c1f8260 con 0x7f044c15fda0 2024-09-10T21:49:02.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.708+0000 7f0453200640 1 -- 172.21.15.7:0/3064385924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f044c070510 con 0x7f044c15fda0 2024-09-10T21:49:02.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.709+0000 7f0448c00640 1 -- 172.21.15.7:0/3064385924 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f044400f040 con 0x7f044c15fda0 2024-09-10T21:49:02.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.710+0000 7f0448c00640 1 --2- 172.21.15.7:0/3064385924 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0428077f70 0x7f042807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:02.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.711+0000 7f0448c00640 1 -- 172.21.15.7:0/3064385924 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f044409a360 con 0x7f044c15fda0 2024-09-10T21:49:02.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.711+0000 7f044be00640 1 --2- 172.21.15.7:0/3064385924 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0428077f70 0x7f042807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:02.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.712+0000 7f044be00640 1 --2- 172.21.15.7:0/3064385924 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0428077f70 0x7f042807a430 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f043c033000 tx=0x7f043c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:02.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.715+0000 7f0448c00640 1 -- 172.21.15.7:0/3064385924 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0444016030 con 0x7f044c15fda0 2024-09-10T21:49:02.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.944+0000 7f0453200640 1 -- 172.21.15.7:0/3064385924 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f044c15f4a0 con 0x7f044c15fda0 2024-09-10T21:49:02.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.945+0000 7f0448c00640 1 -- 172.21.15.7:0/3064385924 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5230 (secure 0 0 0) 0x7f0444062720 con 0x7f044c15fda0 2024-09-10T21:49:02.946 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:02.946 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":70,"btime":"2024-09-10T21:48:02:184202+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/978763848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":978763848},{"type":"v1","addr":"172.21.15.7:6835","nonce":978763848}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/269527918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":269527918},{"type":"v1","addr":"172.21.15.7:6833","nonce":269527918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":235775,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/760980985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":760980985},{"type":"v1","addr":"172.21.15.133:6836","nonce":760980985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:47:40.664091+0000","modified":"2024-09-10T21:48:02.157973+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":88,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":235778},"failed":[],"damaged":[],"stopped":[],"info":{"gid_235778":{"gid":235778,"name":"d","rank":0,"incarnation":66,"state":"up:active","state_seq":10,"addr":"172.21.15.133:6837/3312604545","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3312604545},{"type":"v1","addr":"172.21.15.133:6837","nonce":3312604545}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":235778,"qdb_cluster":[235778]},"id":6}]} 2024-09-10T21:49:02.946 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 70 2024-09-10T21:49:02.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.947+0000 7f0453200640 1 -- 172.21.15.7:0/3064385924 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0428077f70 msgr2=0x7f042807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:02.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.947+0000 7f0453200640 1 --2- 172.21.15.7:0/3064385924 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0428077f70 0x7f042807a430 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f043c033000 tx=0x7f043c004730 comp rx=0 tx=0).stop 2024-09-10T21:49:02.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.947+0000 7f0453200640 1 -- 172.21.15.7:0/3064385924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f044c15fda0 msgr2=0x7f044c072920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:02.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.947+0000 7f0453200640 1 --2- 172.21.15.7:0/3064385924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f044c15fda0 0x7f044c072920 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f044400e8b0 tx=0x7f044400ed70 comp rx=0 tx=0).stop 2024-09-10T21:49:02.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.947+0000 7f0453200640 1 -- 172.21.15.7:0/3064385924 shutdown_connections 2024-09-10T21:49:02.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.947+0000 7f0453200640 1 --2- 172.21.15.7:0/3064385924 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0428077f70 0x7f042807a430 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:02.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.947+0000 7f0453200640 1 --2- 172.21.15.7:0/3064385924 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f044c15fda0 0x7f044c072920 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:02.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.947+0000 7f0453200640 1 --2- 172.21.15.7:0/3064385924 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f044c15bc90 0x7f044c0723e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:02.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.947+0000 7f0453200640 1 --2- 172.21.15.7:0/3064385924 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f044c0cd750 0x7f044c158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:02.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.947+0000 7f0453200640 1 -- 172.21.15.7:0/3064385924 >> 172.21.15.7:0/3064385924 conn(0x7f044c0b16d0 msgr2=0x7f044c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:02.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.947+0000 7f0453200640 1 -- 172.21.15.7:0/3064385924 shutdown_connections 2024-09-10T21:49:02.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:02.947+0000 7f0453200640 1 -- 172.21.15.7:0/3064385924 wait complete. 2024-09-10T21:49:02.964 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2024-09-10T21:49:03.156 INFO:teuthology.orchestra.run.smithi133.stdout:{"message":"","return_code":0} 2024-09-10T21:49:03.156 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-10T21:49:03.157 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:49:03.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.318+0000 7f4e8f400640 1 -- 172.21.15.7:0/3512030198 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e88161b10 msgr2=0x7f4e880b3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.318+0000 7f4e8f400640 1 --2- 172.21.15.7:0/3512030198 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e88161b10 0x7f4e880b3910 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4e78005fd0 tx=0x7f4e7802a5d0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.319+0000 7f4e8f400640 1 -- 172.21.15.7:0/3512030198 shutdown_connections 2024-09-10T21:49:03.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.319+0000 7f4e8f400640 1 --2- 172.21.15.7:0/3512030198 >> v2:172.21.15.133:3300/0 conn(0x7f4e880b3e50 0x7f4e88161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.319+0000 7f4e8f400640 1 --2- 172.21.15.7:0/3512030198 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e88161b10 0x7f4e880b3910 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.319+0000 7f4e8f400640 1 -- 172.21.15.7:0/3512030198 >> 172.21.15.7:0/3512030198 conn(0x7f4e880d3fb0 msgr2=0x7f4e880d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:03.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.319+0000 7f4e8f400640 1 -- 172.21.15.7:0/3512030198 shutdown_connections 2024-09-10T21:49:03.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.319+0000 7f4e8f400640 1 -- 172.21.15.7:0/3512030198 wait complete. 2024-09-10T21:49:03.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.320+0000 7f4e8f400640 1 Processor -- start 2024-09-10T21:49:03.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.320+0000 7f4e8f400640 1 -- start start 2024-09-10T21:49:03.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.321+0000 7f4e8f400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e880b3e50 0x7f4e880cccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.321+0000 7f4e8f400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e880d4ad0 0x7f4e880b84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.321+0000 7f4e8f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e88161b10 0x7f4e88160620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.321+0000 7f4e8f400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e880cfe10 con 0x7f4e88161b10 2024-09-10T21:49:03.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.321+0000 7f4e8f400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4e880cfc90 con 0x7f4e880d4ad0 2024-09-10T21:49:03.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.321+0000 7f4e8f400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4e88162430 con 0x7f4e880b3e50 2024-09-10T21:49:03.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.322+0000 7f4e8d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e88161b10 0x7f4e88160620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.322+0000 7f4e8d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e88161b10 0x7f4e88160620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35888/0 (socket says 172.21.15.7:35888) 2024-09-10T21:49:03.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.322+0000 7f4e8ca00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e880b3e50 0x7f4e880cccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.322+0000 7f4e8d400640 1 -- 172.21.15.7:0/2638808976 learned_addr learned my addr 172.21.15.7:0/2638808976 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:03.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.322+0000 7f4e7fe00640 1 --2- 172.21.15.7:0/2638808976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e880d4ad0 0x7f4e880b84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.322+0000 7f4e8d400640 1 -- 172.21.15.7:0/2638808976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e880b3e50 msgr2=0x7f4e880cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.322+0000 7f4e8d400640 1 --2- 172.21.15.7:0/2638808976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e880b3e50 0x7f4e880cccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.322+0000 7f4e8d400640 1 -- 172.21.15.7:0/2638808976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e880d4ad0 msgr2=0x7f4e880b84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.322+0000 7f4e8d400640 1 --2- 172.21.15.7:0/2638808976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e880d4ad0 0x7f4e880b84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.322+0000 7f4e8d400640 1 -- 172.21.15.7:0/2638808976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e881f58a0 con 0x7f4e88161b10 2024-09-10T21:49:03.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.322+0000 7f4e8d400640 1 --2- 172.21.15.7:0/2638808976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e88161b10 0x7f4e88160620 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000eb10 tx=0x7f4e8000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.323+0000 7f4e85600640 1 -- 172.21.15.7:0/2638808976 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80004b20 con 0x7f4e88161b10 2024-09-10T21:49:03.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.323+0000 7f4e85600640 1 -- 172.21.15.7:0/2638808976 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e80004ce0 con 0x7f4e88161b10 2024-09-10T21:49:03.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.323+0000 7f4e8f400640 1 -- 172.21.15.7:0/2638808976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e881f55c0 con 0x7f4e88161b10 2024-09-10T21:49:03.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.323+0000 7f4e85600640 1 -- 172.21.15.7:0/2638808976 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80005280 con 0x7f4e88161b10 2024-09-10T21:49:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.323+0000 7f4e8f400640 1 -- 172.21.15.7:0/2638808976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e880cc4c0 con 0x7f4e88161b10 2024-09-10T21:49:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.324+0000 7f4e8f400640 1 -- 172.21.15.7:0/2638808976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e881628f0 con 0x7f4e88161b10 2024-09-10T21:49:03.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.325+0000 7f4e85600640 1 -- 172.21.15.7:0/2638808976 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4e80005420 con 0x7f4e88161b10 2024-09-10T21:49:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.326+0000 7f4e85600640 1 --2- 172.21.15.7:0/2638808976 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e64077fa0 0x7f4e6407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.326+0000 7f4e85600640 1 -- 172.21.15.7:0/2638808976 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 5824+0+0 (secure 0 0 0) 0x7f4e8009a910 con 0x7f4e88161b10 2024-09-10T21:49:03.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.327+0000 7f4e8ca00640 1 --2- 172.21.15.7:0/2638808976 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e64077fa0 0x7f4e6407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.327+0000 7f4e8ca00640 1 --2- 172.21.15.7:0/2638808976 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e64077fa0 0x7f4e6407a460 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4e78033000 tx=0x7f4e78004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.331 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.330+0000 7f4e85600640 1 -- 172.21.15.7:0/2638808976 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4e80062d60 con 0x7f4e88161b10 2024-09-10T21:49:03.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.561+0000 7f4e8f400640 1 -- 172.21.15.7:0/2638808976 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4e880b5950 con 0x7f4e88161b10 2024-09-10T21:49:03.563 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-10T21:49:03.562+0000 7feb3fa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:49:03.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.564+0000 7f4e85600640 1 -- 172.21.15.7:0/2638808976 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v71) ==== 161+0+0 (secure 0 0 0) 0x7f4e8009a070 con 0x7f4e88161b10 2024-09-10T21:49:03.565 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:49:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.566+0000 7f4e8f400640 1 -- 172.21.15.7:0/2638808976 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e64077fa0 msgr2=0x7f4e6407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.566+0000 7f4e8f400640 1 --2- 172.21.15.7:0/2638808976 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e64077fa0 0x7f4e6407a460 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f4e78033000 tx=0x7f4e78004730 comp rx=0 tx=0).stop 2024-09-10T21:49:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.566+0000 7f4e8f400640 1 -- 172.21.15.7:0/2638808976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e88161b10 msgr2=0x7f4e88160620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.566+0000 7f4e8f400640 1 --2- 172.21.15.7:0/2638808976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e88161b10 0x7f4e88160620 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f4e8000eb10 tx=0x7f4e8000c2d0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.566+0000 7f4e8f400640 1 -- 172.21.15.7:0/2638808976 shutdown_connections 2024-09-10T21:49:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.566+0000 7f4e8f400640 1 --2- 172.21.15.7:0/2638808976 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e64077fa0 0x7f4e6407a460 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.566+0000 7f4e8f400640 1 --2- 172.21.15.7:0/2638808976 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e880d4ad0 0x7f4e880b84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.566+0000 7f4e8f400640 1 --2- 172.21.15.7:0/2638808976 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e88161b10 0x7f4e88160620 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.566+0000 7f4e8f400640 1 --2- 172.21.15.7:0/2638808976 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e880b3e50 0x7f4e880cccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.566+0000 7f4e8f400640 1 -- 172.21.15.7:0/2638808976 >> 172.21.15.7:0/2638808976 conn(0x7f4e880d3fb0 msgr2=0x7f4e8806d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.566+0000 7f4e8f400640 1 -- 172.21.15.7:0/2638808976 shutdown_connections 2024-09-10T21:49:03.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.566+0000 7f4e8f400640 1 -- 172.21.15.7:0/2638808976 wait complete. 2024-09-10T21:49:03.580 INFO:tasks.ceph.mds.d.smithi133.stderr: -4732> 2024-09-10T21:47:57.196+0000 7f9011000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:49:03.581 INFO:tasks.ceph.mds.d.smithi133.stderr: -4304> 2024-09-10T21:47:58.663+0000 7f900c000640 -1 log_channel(cluster) log [ERR] : dir 0x10000000000 object missing on disk; some files may be lost (/dir) 2024-09-10T21:49:03.582 INFO:tasks.ceph.mds.d.smithi133.stderr: -4124> 2024-09-10T21:47:58.917+0000 7f900c000640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 ap=1 f(v0 m2024-09-10T21:47:49.036385+0000 1=1+0) n(v0 rc2024-09-10T21:47:49.195383+0000 2=1+1) | dirfrag=0 authpin=1 scrubqueue=0 0x55c0215d4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-09-10T21:47:49.036385+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-09-10T21:47:49.195383+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":-2} 2024-09-10T21:49:03.583 DEBUG:teuthology.orchestra.run.smithi007:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-09-10T21:49:03.584 INFO:tasks.ceph.mds.d.smithi133.stderr: -2320> 2024-09-10T21:48:30.678+0000 7f900c000640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /dir/ auth v15 pv18 ap=3 DIRTYPARENT f(v0 m2024-09-10T21:47:49.036385+0000 1=1+0)->f(v1 m2024-09-10T21:47:49.036385+0000) n(v0 rc2024-09-10T21:47:49.195383+0000 2=1+1)->n(v0 rc2024-09-10T21:47:49.195383+0000 2=1+1) (inest lock dirty) (ifile lock->sync w=1 flushing) | dirtyscattered=2 lock=1 dirfrag=1 dirtyrstat=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x55c0215d4100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-2,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(12)0x10000000000:[<0x1/dir v15>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 m2024-09-10T21:47:49.036385+0000 1=1+0)","memory_value.rstat":"n(v0 rc2024-09-10T21:47:49.195383+0000 2=1+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-2} 2024-09-10T21:49:03.627 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2024-09-10T21:49:03.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.654+0000 7f543cc20740 1 -- start start 2024-09-10T21:49:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.654+0000 7f543cc20740 1 Processor -- start 2024-09-10T21:49:03.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.655+0000 7f543cc20740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d261fcc00 0x560d26fdc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.655+0000 7f543cc20740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d261fd800 0x560d26fdc580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.655+0000 7f543cc20740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d261fdc00 0x560d26fdcb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.655+0000 7f543cc20740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560d26ff0180 con 0x560d261fdc00 2024-09-10T21:49:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.655+0000 7f543cc20740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560d26ff0300 con 0x560d261fd800 2024-09-10T21:49:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.655+0000 7f543cc20740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x560d26ff0480 con 0x560d261fcc00 2024-09-10T21:49:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.656+0000 7f543aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d261fdc00 0x560d26fdcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.656+0000 7f543aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d261fdc00 0x560d26fdcb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35902/0 (socket says 172.21.15.7:35902) 2024-09-10T21:49:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.656+0000 7f543aa00640 1 -- 172.21.15.7:0/4037144993 learned_addr learned my addr 172.21.15.7:0/4037144993 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.656+0000 7f543a000640 1 --2- 172.21.15.7:0/4037144993 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d261fcc00 0x560d26fdc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.656+0000 7f5439600640 1 --2- 172.21.15.7:0/4037144993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d261fd800 0x560d26fdc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.656+0000 7f543aa00640 1 -- 172.21.15.7:0/4037144993 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d261fcc00 msgr2=0x560d26fdc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:03.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.656+0000 7f543aa00640 1 --2- 172.21.15.7:0/4037144993 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d261fcc00 0x560d26fdc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.656+0000 7f543aa00640 1 -- 172.21.15.7:0/4037144993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d261fd800 msgr2=0x560d26fdc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:03.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.656+0000 7f543aa00640 1 --2- 172.21.15.7:0/4037144993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d261fd800 0x560d26fdc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.656+0000 7f543aa00640 1 -- 172.21.15.7:0/4037144993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d26229680 con 0x560d261fdc00 2024-09-10T21:49:03.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.656+0000 7f543aa00640 1 --2- 172.21.15.7:0/4037144993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d261fdc00 0x560d26fdcb00 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x560d26f7a060 tx=0x560d26fa2a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e18a8af5172f1c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.657+0000 7f5436e00640 1 -- 172.21.15.7:0/4037144993 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560d26222ea0 con 0x560d261fdc00 2024-09-10T21:49:03.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.657+0000 7f5436e00640 1 -- 172.21.15.7:0/4037144993 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x560d26252fc0 con 0x560d261fdc00 2024-09-10T21:49:03.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.657+0000 7f5436e00640 1 -- 172.21.15.7:0/4037144993 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560d26223380 con 0x560d261fdc00 2024-09-10T21:49:03.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.657+0000 7f543cc20740 1 -- 172.21.15.7:0/4037144993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d26229860 con 0x560d261fdc00 2024-09-10T21:49:03.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.657+0000 7f543cc20740 1 -- 172.21.15.7:0/4037144993 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=0}) -- 0x560d262292c0 con 0x560d261fdc00 2024-09-10T21:49:03.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.657+0000 7f543cc20740 4 waiting for MDS map... 2024-09-10T21:49:03.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.657+0000 7f5436e00640 1 -- 172.21.15.7:0/4037144993 <== mon.0 v2:172.21.15.7:3300/0 4 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x560d26f7c600 con 0x560d261fdc00 2024-09-10T21:49:03.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.657+0000 7f5436e00640 1 -- 172.21.15.7:0/4037144993 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 71) ==== 2893+0+0 (secure 0 0 0) 0x560d26257180 con 0x560d261fdc00 2024-09-10T21:49:03.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.657+0000 7f543cc20740 4 Got MDS map 71 2024-09-10T21:49:03.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.657+0000 7f543cc20740 4 datascan.main: Using metadata pool output 2024-09-10T21:49:03.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.657+0000 7f543cc20740 4 datascan.main: connecting to RADOS... 2024-09-10T21:49:03.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.658+0000 7f543cc20740 1 Processor -- start 2024-09-10T21:49:03.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.658+0000 7f543cc20740 1 -- start start 2024-09-10T21:49:03.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.659+0000 7f543cc20740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d27013400 0x560d26fdd080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.659+0000 7f543cc20740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d27013800 0x560d26fdd600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.659+0000 7f543cc20740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d27013c00 0x560d26fddb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.659+0000 7f543cc20740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560d26ff0600 con 0x560d27013800 2024-09-10T21:49:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.659+0000 7f543cc20740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560d26ff0780 con 0x560d27013c00 2024-09-10T21:49:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.659+0000 7f543cc20740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x560d26ff0900 con 0x560d27013400 2024-09-10T21:49:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.659+0000 7f543a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d27013800 0x560d26fdd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.659+0000 7f543a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d27013800 0x560d26fdd600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35906/0 (socket says 172.21.15.7:35906) 2024-09-10T21:49:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.659+0000 7f543a000640 1 -- 172.21.15.7:0/4116756648 learned_addr learned my addr 172.21.15.7:0/4116756648 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.659+0000 7f5439600640 1 --2- 172.21.15.7:0/4116756648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d27013c00 0x560d26fddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.659+0000 7f543a000640 1 -- 172.21.15.7:0/4116756648 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d27013400 msgr2=0x560d26fdd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.659+0000 7f543a000640 1 --2- 172.21.15.7:0/4116756648 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d27013400 0x560d26fdd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.659+0000 7f543a000640 1 -- 172.21.15.7:0/4116756648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d27013c00 msgr2=0x560d26fddb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.659+0000 7f543a000640 1 --2- 172.21.15.7:0/4116756648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d27013c00 0x560d26fddb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.659+0000 7f543a000640 1 -- 172.21.15.7:0/4116756648 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d262294a0 con 0x560d27013800 2024-09-10T21:49:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.659+0000 7f5439600640 1 --2- 172.21.15.7:0/4116756648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d27013c00 0x560d26fddb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:49:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.660+0000 7f543a000640 1 --2- 172.21.15.7:0/4116756648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d27013800 0x560d26fdd600 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x560d26f7aea0 tx=0x560d270f2c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ffbbba47001a48 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.660+0000 7f5434600640 1 -- 172.21.15.7:0/4116756648 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560d26222340 con 0x560d27013800 2024-09-10T21:49:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.660+0000 7f5434600640 1 -- 172.21.15.7:0/4116756648 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x560d26252e00 con 0x560d27013800 2024-09-10T21:49:03.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.660+0000 7f5434600640 1 -- 172.21.15.7:0/4116756648 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560d26222ea0 con 0x560d27013800 2024-09-10T21:49:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.660+0000 7f543cc20740 1 -- 172.21.15.7:0/4116756648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d27013800 msgr2=0x560d26fdd600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.660+0000 7f543cc20740 1 --2- 172.21.15.7:0/4116756648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d27013800 0x560d26fdd600 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x560d26f7aea0 tx=0x560d270f2c00 comp rx=0 tx=0).stop 2024-09-10T21:49:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.660+0000 7f543cc20740 1 -- 172.21.15.7:0/4116756648 shutdown_connections 2024-09-10T21:49:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.660+0000 7f543cc20740 1 --2- 172.21.15.7:0/4116756648 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d27013c00 0x560d26fddb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.660+0000 7f543cc20740 1 --2- 172.21.15.7:0/4116756648 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d27013800 0x560d26fdd600 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.660+0000 7f543cc20740 1 --2- 172.21.15.7:0/4116756648 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d27013400 0x560d26fdd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.660+0000 7f543cc20740 1 -- 172.21.15.7:0/4116756648 >> 172.21.15.7:0/4116756648 conn(0x560d27013000 msgr2=0x560d262ef000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.660+0000 7f543cc20740 1 -- 172.21.15.7:0/4116756648 shutdown_connections 2024-09-10T21:49:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.660+0000 7f543cc20740 1 -- 172.21.15.7:0/4116756648 wait complete. 2024-09-10T21:49:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.661+0000 7f543cc20740 1 Processor -- start 2024-09-10T21:49:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.661+0000 7f543cc20740 1 -- start start 2024-09-10T21:49:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.662+0000 7f543cc20740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d27013400 0x560d26fdd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.662+0000 7f543cc20740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d27013800 0x560d26fdd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.662+0000 7f543cc20740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d27013c00 0x560d26fddb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.662+0000 7f543cc20740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560d26ff1980 con 0x560d27013400 2024-09-10T21:49:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.662+0000 7f543cc20740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560d26ff1800 con 0x560d27013c00 2024-09-10T21:49:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.662+0000 7f543cc20740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x560d26ff1680 con 0x560d27013800 2024-09-10T21:49:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.662+0000 7f5439600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d27013400 0x560d26fdd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.662+0000 7f5439600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d27013400 0x560d26fdd080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35912/0 (socket says 172.21.15.7:35912) 2024-09-10T21:49:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.662+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 learned_addr learned my addr 172.21.15.7:0/3420302251 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.662+0000 7f543a000640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d27013800 0x560d26fdd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.662+0000 7f5439600640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d27013c00 0x560d26fddb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.663+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d27013800 msgr2=0x560d26fdd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.663+0000 7f5439600640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d27013800 0x560d26fdd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.663+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d27013c00 msgr2=0x560d26fddb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.663+0000 7f5439600640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d27013c00 0x560d26fddb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.663+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d26229680 con 0x560d27013400 2024-09-10T21:49:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.663+0000 7f5439600640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d27013400 0x560d26fdd080 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x560d2714e180 tx=0x560d270f2f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.663+0000 7f5432800640 1 -- 172.21.15.7:0/3420302251 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560d26222340 con 0x560d27013400 2024-09-10T21:49:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.663+0000 7f5432800640 1 -- 172.21.15.7:0/3420302251 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x560d26252540 con 0x560d27013400 2024-09-10T21:49:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.663+0000 7f5432800640 1 -- 172.21.15.7:0/3420302251 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x560d26222ea0 con 0x560d27013400 2024-09-10T21:49:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.663+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560d26229860 con 0x560d27013400 2024-09-10T21:49:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.663+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d262290e0 con 0x560d27013400 2024-09-10T21:49:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.663+0000 7f543cc20740 4 datascan.init: resolving metadata pool 12 2024-09-10T21:49:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.665+0000 7f5432800640 1 -- 172.21.15.7:0/3420302251 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x560d26257500 con 0x560d27013400 2024-09-10T21:49:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.666+0000 7f5432800640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x560d2715ac00 0x560d26fde100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.666+0000 7f5432800640 1 -- 172.21.15.7:0/3420302251 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x560d2715c600 con 0x560d27013400 2024-09-10T21:49:03.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.666+0000 7f543cc20740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-09-10T21:49:03.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.666+0000 7f543cc20740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-09-10T21:49:03.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.666+0000 7f543cc20740 4 datascan.main: opening data pool 'cephfs_data' 2024-09-10T21:49:03.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.666+0000 7f543cc20740 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x560d2715b000 0x560d26fde680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.666+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:1 13.0 13:00000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d2715b400 con 0x560d2715b000 2024-09-10T21:49:03.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.666+0000 7f543a000640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x560d2715ac00 0x560d26fde100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.667+0000 7f5439600640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x560d2715b000 0x560d26fde680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.667+0000 7f5439600640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x560d2715b000 0x560d26fde680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.667+0000 7f543a000640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x560d2715ac00 0x560d26fde100 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x560d26f7aea0 tx=0x560d26f9a000 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.669+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.7 v2:172.21.15.133:6802/2638399040 1 ==== osd_op_reply(1 [pgnls start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d2715b000 2024-09-10T21:49:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.670+0000 7f5433c00640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x560d265b2000 0x560d265b8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.670+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:2 13.20 13:04000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265b2400 con 0x560d265b2000 2024-09-10T21:49:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.670+0000 7f543aa00640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x560d265b2000 0x560d265b8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.670+0000 7f543aa00640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x560d265b2000 0x560d265b8000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.671+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.6 v2:172.21.15.133:6810/3365981160 1 ==== osd_op_reply(2 [pgnls start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b2000 2024-09-10T21:49:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.671+0000 7f5434600640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x560d265b2c00 0x560d265b8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.671+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:3 13.10 13:08000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265b3000 con 0x560d265b2c00 2024-09-10T21:49:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.671+0000 7f543a000640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x560d265b2c00 0x560d265b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.672+0000 7f543a000640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x560d265b2c00 0x560d265b8580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.674+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 <== osd.1 v2:172.21.15.7:6824/4175601737 1 ==== osd_op_reply(3 [pgnls start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306780 con 0x560d265b2c00 2024-09-10T21:49:03.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.674+0000 7f5433c00640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x560d265b2800 0x560d265b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.674+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:4 13.30 13:0c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265b3800 con 0x560d265b2800 2024-09-10T21:49:03.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.674+0000 7f5439600640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x560d265b2800 0x560d265b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.675+0000 7f5439600640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x560d265b2800 0x560d265b8b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.675+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.4 v2:172.21.15.133:6826/1857710445 1 ==== osd_op_reply(4 [pgnls start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d265b2800 2024-09-10T21:49:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.675+0000 7f5434600640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x560d265b3400 0x560d265b9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.675+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:5 13.8 13:10000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d0000 con 0x560d265b3400 2024-09-10T21:49:03.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.675+0000 7f543aa00640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x560d265b3400 0x560d265b9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.676+0000 7f543aa00640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x560d265b3400 0x560d265b9600 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.678+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.2 v2:172.21.15.7:6816/4238877366 1 ==== osd_op_reply(5 [pgnls start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b3400 2024-09-10T21:49:03.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.678+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:6 13.28 13:14000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265b3c00 con 0x560d265b3400 2024-09-10T21:49:03.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.678+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.2 v2:172.21.15.7:6816/4238877366 2 ==== osd_op_reply(6 [pgnls start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b3400 2024-09-10T21:49:03.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.679+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:7 13.18 13:18000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d0400 con 0x560d2715b000 2024-09-10T21:49:03.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.679+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.7 v2:172.21.15.133:6802/2638399040 2 ==== osd_op_reply(7 [pgnls start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d2715b000 2024-09-10T21:49:03.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.679+0000 7f5433c00640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x560d265d0c00 0x560d265b9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.679+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:8 13.38 13:1c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d1000 con 0x560d265d0c00 2024-09-10T21:49:03.680 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.679+0000 7f543a000640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x560d265d0c00 0x560d265b9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.680+0000 7f543a000640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x560d265d0c00 0x560d265b9080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.680+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 <== osd.5 v2:172.21.15.133:6818/3838498382 1 ==== osd_op_reply(8 [pgnls start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306780 con 0x560d265d0c00 2024-09-10T21:49:03.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.680+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:9 13.4 13:20000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d0800 con 0x560d265b2000 2024-09-10T21:49:03.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.681+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.6 v2:172.21.15.133:6810/3365981160 2 ==== osd_op_reply(9 [pgnls start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b2000 2024-09-10T21:49:03.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.681+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:10 13.24 13:24000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d1400 con 0x560d265b2000 2024-09-10T21:49:03.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.681+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.6 v2:172.21.15.133:6810/3365981160 3 ==== osd_op_reply(10 [pgnls start_epoch 89 out=49b] v87'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b2000 2024-09-10T21:49:03.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.681+0000 7f5434600640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x560d265d0000 0x560d265b9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.681+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:11 13.14 13:28000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265b2400 con 0x560d265d0000 2024-09-10T21:49:03.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.681+0000 7f5439600640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x560d265d0000 0x560d265b9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.682+0000 7f5439600640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x560d265d0000 0x560d265b9b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.684+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.0 v2:172.21.15.7:6808/651402908 1 ==== osd_op_reply(11 [pgnls start_epoch 89 out=49b] v87'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d265d0000 2024-09-10T21:49:03.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.684+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:12 13.34 13:2c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d1800 con 0x560d265b2000 2024-09-10T21:49:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.685+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.6 v2:172.21.15.133:6810/3365981160 4 ==== osd_op_reply(12 [pgnls start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b2000 2024-09-10T21:49:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.685+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:13 13.c 13:30000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265b3c00 con 0x560d265b2c00 2024-09-10T21:49:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.685+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 <== osd.1 v2:172.21.15.7:6824/4175601737 2 ==== osd_op_reply(13 [pgnls start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306780 con 0x560d265b2c00 2024-09-10T21:49:03.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.685+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:14 13.2c 13:34000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265b3000 con 0x560d265b3400 2024-09-10T21:49:03.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.685+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.2 v2:172.21.15.7:6816/4238877366 3 ==== osd_op_reply(14 [pgnls start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b3400 2024-09-10T21:49:03.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.685+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:15 13.1c 13:38000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d27012800 con 0x560d265b2000 2024-09-10T21:49:03.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.686+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.6 v2:172.21.15.133:6810/3365981160 5 ==== osd_op_reply(15 [pgnls start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b2000 2024-09-10T21:49:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.686+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:16 13.3c 13:3c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265b3c00 con 0x560d265b3400 2024-09-10T21:49:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.686+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.2 v2:172.21.15.7:6816/4238877366 4 ==== osd_op_reply(16 [pgnls start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b3400 2024-09-10T21:49:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.686+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:17 13.2 13:40000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d1000 con 0x560d265d0c00 2024-09-10T21:49:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.687+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 <== osd.5 v2:172.21.15.133:6818/3838498382 2 ==== osd_op_reply(17 [pgnls start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306780 con 0x560d265d0c00 2024-09-10T21:49:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.687+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:18 13.22 13:44000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d2715b400 con 0x560d265b2c00 2024-09-10T21:49:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.687+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 <== osd.1 v2:172.21.15.7:6824/4175601737 3 ==== osd_op_reply(18 [pgnls start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306780 con 0x560d265b2c00 2024-09-10T21:49:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.687+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:19 13.12 13:48000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d0400 con 0x560d265d0000 2024-09-10T21:49:03.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.687+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.0 v2:172.21.15.7:6808/651402908 2 ==== osd_op_reply(19 [pgnls start_epoch 89 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d265d0000 2024-09-10T21:49:03.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.687+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:20 13.32 13:4c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d27001400 con 0x560d265b3400 2024-09-10T21:49:03.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.688+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.2 v2:172.21.15.7:6816/4238877366 5 ==== osd_op_reply(20 [pgnls start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b3400 2024-09-10T21:49:03.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.688+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:21 13.a 13:50000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d0800 con 0x560d265b3400 2024-09-10T21:49:03.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.688+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.2 v2:172.21.15.7:6816/4238877366 6 ==== osd_op_reply(21 [pgnls start_epoch 89 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b3400 2024-09-10T21:49:03.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.688+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:22 13.2a 13:54000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d27001000 con 0x560d265b2c00 2024-09-10T21:49:03.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.688+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 <== osd.1 v2:172.21.15.7:6824/4175601737 4 ==== osd_op_reply(22 [pgnls start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306780 con 0x560d265b2c00 2024-09-10T21:49:03.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.688+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:23 13.1a 13:58000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265b3800 con 0x560d265b2c00 2024-09-10T21:49:03.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.689+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 <== osd.1 v2:172.21.15.7:6824/4175601737 5 ==== osd_op_reply(23 [pgnls start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306780 con 0x560d265b2c00 2024-09-10T21:49:03.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.689+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:24 13.3a 13:5c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d1400 con 0x560d265d0c00 2024-09-10T21:49:03.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.689+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 <== osd.5 v2:172.21.15.133:6818/3838498382 3 ==== osd_op_reply(24 [pgnls start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306780 con 0x560d265d0c00 2024-09-10T21:49:03.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.689+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:25 13.6 13:60000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265b2400 con 0x560d265b2800 2024-09-10T21:49:03.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.690+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.4 v2:172.21.15.133:6826/1857710445 2 ==== osd_op_reply(25 [pgnls start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d265b2800 2024-09-10T21:49:03.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.690+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:26 13.26 13:64000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d1800 con 0x560d2715b000 2024-09-10T21:49:03.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.690+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.7 v2:172.21.15.133:6802/2638399040 3 ==== osd_op_reply(26 [pgnls start_epoch 89 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d2715b000 2024-09-10T21:49:03.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.690+0000 7f5434600640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x560d265d1c00 0x560d265ba100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.690+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:27 13.16 13:68000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265e4000 con 0x560d265d1c00 2024-09-10T21:49:03.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.690+0000 7f543aa00640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x560d265d1c00 0x560d265ba100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.691+0000 7f543aa00640 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x560d265d1c00 0x560d265ba100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.693+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.3 v2:172.21.15.7:6800/2195665006 1 ==== osd_op_reply(27 [pgnls start_epoch 89 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265d1c00 2024-09-10T21:49:03.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.693+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:28 13.36 13:6c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d27001400 con 0x560d265b2000 2024-09-10T21:49:03.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.694+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.6 v2:172.21.15.133:6810/3365981160 6 ==== osd_op_reply(28 [pgnls start_epoch 89 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b2000 2024-09-10T21:49:03.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.694+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:29 13.e 13:70000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265e4400 con 0x560d2715b000 2024-09-10T21:49:03.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.694+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.7 v2:172.21.15.133:6802/2638399040 4 ==== osd_op_reply(29 [pgnls start_epoch 89 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d2715b000 2024-09-10T21:49:03.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.694+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:30 13.2e 13:74000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265b3000 con 0x560d265b3400 2024-09-10T21:49:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.694+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.2 v2:172.21.15.7:6816/4238877366 7 ==== osd_op_reply(30 [pgnls start_epoch 89 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b3400 2024-09-10T21:49:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.694+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:31 13.1e 13:78000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265e4800 con 0x560d265b2800 2024-09-10T21:49:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.695+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.4 v2:172.21.15.133:6826/1857710445 3 ==== osd_op_reply(31 [pgnls start_epoch 89 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d265b2800 2024-09-10T21:49:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.695+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:32 13.3e 13:7c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265b3c00 con 0x560d265b2c00 2024-09-10T21:49:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.695+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 <== osd.1 v2:172.21.15.7:6824/4175601737 6 ==== osd_op_reply(32 [pgnls start_epoch 89 out=49b] v87'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306780 con 0x560d265b2c00 2024-09-10T21:49:03.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.695+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:33 13.1 13:80000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265e4c00 con 0x560d265b2000 2024-09-10T21:49:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.696+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.6 v2:172.21.15.133:6810/3365981160 7 ==== osd_op_reply(33 [pgnls start_epoch 89 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b2000 2024-09-10T21:49:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.696+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:34 13.21 13:84000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d27012800 con 0x560d265b2800 2024-09-10T21:49:03.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.696+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.4 v2:172.21.15.133:6826/1857710445 4 ==== osd_op_reply(34 [pgnls start_epoch 89 out=49b] v87'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d265b2800 2024-09-10T21:49:03.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.697+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:35 13.11 13:88000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265e5000 con 0x560d265b2c00 2024-09-10T21:49:03.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.697+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 <== osd.1 v2:172.21.15.7:6824/4175601737 7 ==== osd_op_reply(35 [pgnls start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306780 con 0x560d265b2c00 2024-09-10T21:49:03.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.697+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:36 13.31 13:8c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d1000 con 0x560d265d1c00 2024-09-10T21:49:03.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.697+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.3 v2:172.21.15.7:6800/2195665006 2 ==== osd_op_reply(36 [pgnls start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265d1c00 2024-09-10T21:49:03.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.697+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:37 13.9 13:90000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265e5400 con 0x560d265d0c00 2024-09-10T21:49:03.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.698+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 <== osd.5 v2:172.21.15.133:6818/3838498382 4 ==== osd_op_reply(37 [pgnls start_epoch 89 out=49b] v87'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306780 con 0x560d265d0c00 2024-09-10T21:49:03.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.698+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:38 13.29 13:94000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d1400 con 0x560d265b2000 2024-09-10T21:49:03.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.698+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.6 v2:172.21.15.133:6810/3365981160 8 ==== osd_op_reply(38 [pgnls start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b2000 2024-09-10T21:49:03.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.698+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:39 13.19 13:98000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265e5800 con 0x560d265d1c00 2024-09-10T21:49:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.698+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.3 v2:172.21.15.7:6800/2195665006 3 ==== osd_op_reply(39 [pgnls start_epoch 89 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265d1c00 2024-09-10T21:49:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.699+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:40 13.39 13:9c000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265b3800 con 0x560d265b3400 2024-09-10T21:49:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.699+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.2 v2:172.21.15.7:6816/4238877366 8 ==== osd_op_reply(40 [pgnls start_epoch 89 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b3400 2024-09-10T21:49:03.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.699+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:41 13.5 13:a0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265e4000 con 0x560d265b2800 2024-09-10T21:49:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.700+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.4 v2:172.21.15.133:6826/1857710445 5 ==== osd_op_reply(41 [pgnls start_epoch 89 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d265b2800 2024-09-10T21:49:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.700+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:42 13.25 13:a4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265b3c00 con 0x560d265b2800 2024-09-10T21:49:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.700+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.4 v2:172.21.15.133:6826/1857710445 6 ==== osd_op_reply(42 [pgnls start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d265b2800 2024-09-10T21:49:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.700+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:43 13.15 13:a8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d0800 con 0x560d265b2800 2024-09-10T21:49:03.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.700+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.4 v2:172.21.15.133:6826/1857710445 7 ==== osd_op_reply(43 [pgnls start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d265b2800 2024-09-10T21:49:03.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.700+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:44 13.35 13:ac000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265b2400 con 0x560d265d1c00 2024-09-10T21:49:03.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.701+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.3 v2:172.21.15.7:6800/2195665006 4 ==== osd_op_reply(44 [pgnls start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265d1c00 2024-09-10T21:49:03.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.701+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:45 13.d 13:b0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d1000 con 0x560d265b2800 2024-09-10T21:49:03.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.701+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.4 v2:172.21.15.133:6826/1857710445 8 ==== osd_op_reply(45 [pgnls start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d265b2800 2024-09-10T21:49:03.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.701+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:46 13.2d 13:b4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d27001000 con 0x560d265d0000 2024-09-10T21:49:03.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.702+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.0 v2:172.21.15.7:6808/651402908 3 ==== osd_op_reply(46 [pgnls start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d265d0000 2024-09-10T21:49:03.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.702+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:47 13.1d 13:b8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d1400 con 0x560d2715b000 2024-09-10T21:49:03.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.702+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.7 v2:172.21.15.133:6802/2638399040 5 ==== osd_op_reply(47 [pgnls start_epoch 89 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d2715b000 2024-09-10T21:49:03.703 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.703+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:48 13.3d 13:bc000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d2715b400 con 0x560d2715b000 2024-09-10T21:49:03.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.703+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.7 v2:172.21.15.133:6802/2638399040 6 ==== osd_op_reply(48 [pgnls start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d2715b000 2024-09-10T21:49:03.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.703+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:49 13.3 13:c0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d0400 con 0x560d265d0000 2024-09-10T21:49:03.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.704+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.0 v2:172.21.15.7:6808/651402908 4 ==== osd_op_reply(49 [pgnls start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d265d0000 2024-09-10T21:49:03.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.704+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:50 13.23 13:c4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265e4000 con 0x560d265b2c00 2024-09-10T21:49:03.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.704+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 <== osd.1 v2:172.21.15.7:6824/4175601737 8 ==== osd_op_reply(50 [pgnls start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306780 con 0x560d265b2c00 2024-09-10T21:49:03.705 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.704+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:51 13.13 13:c8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d1800 con 0x560d2715b000 2024-09-10T21:49:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.705+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.7 v2:172.21.15.133:6802/2638399040 7 ==== osd_op_reply(51 [pgnls start_epoch 89 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d2715b000 2024-09-10T21:49:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.705+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:52 13.33 13:cc000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265e4c00 con 0x560d2715b000 2024-09-10T21:49:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.705+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.7 v2:172.21.15.133:6802/2638399040 8 ==== osd_op_reply(52 [pgnls start_epoch 89 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d2715b000 2024-09-10T21:49:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.705+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:53 13.b 13:d0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265b3000 con 0x560d265d0c00 2024-09-10T21:49:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.706+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 <== osd.5 v2:172.21.15.133:6818/3838498382 5 ==== osd_op_reply(53 [pgnls start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306780 con 0x560d265d0c00 2024-09-10T21:49:03.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.706+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:54 13.2b 13:d4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265e4400 con 0x560d265b3400 2024-09-10T21:49:03.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.706+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.2 v2:172.21.15.7:6816/4238877366 9 ==== osd_op_reply(54 [pgnls start_epoch 89 out=49b] v87'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b3400 2024-09-10T21:49:03.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.706+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:55 13.1b 13:d8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d27001400 con 0x560d265b2800 2024-09-10T21:49:03.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.707+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.4 v2:172.21.15.133:6826/1857710445 9 ==== osd_op_reply(55 [pgnls start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d265b2800 2024-09-10T21:49:03.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.707+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:56 13.3b 13:dc000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d27001000 con 0x560d265b2800 2024-09-10T21:49:03.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.707+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.4 v2:172.21.15.133:6826/1857710445 10 ==== osd_op_reply(56 [pgnls start_epoch 89 out=49b] v87'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d265b2800 2024-09-10T21:49:03.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.707+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:57 13.7 13:e0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265e4c00 con 0x560d2715b000 2024-09-10T21:49:03.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.707+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.7 v2:172.21.15.133:6802/2638399040 9 ==== osd_op_reply(57 [pgnls start_epoch 89 out=81b] v87'6 uv6 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x560d26306000 con 0x560d2715b000 2024-09-10T21:49:03.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.707+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:58 13.7 13:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x560d2715b800 con 0x560d2715b000 2024-09-10T21:49:03.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.708+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.7 v2:172.21.15.133:6802/2638399040 10 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv6 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x560d26306000 con 0x560d2715b000 2024-09-10T21:49:03.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.708+0000 7f543cc20740 10 datascan.operator(): handling object 1099511627777.0 2024-09-10T21:49:03.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.708+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:59 13.7 13:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x560d2715bc00 con 0x560d2715b000 2024-09-10T21:49:03.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.710+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.7 v2:172.21.15.133:6802/2638399040 11 ==== osd_op_reply(59 10000000001.00000000 [call] v89'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x560d26306000 con 0x560d2715b000 2024-09-10T21:49:03.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.710+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:60 13.27 13:e4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265ec000 con 0x560d265b2c00 2024-09-10T21:49:03.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.710+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 <== osd.1 v2:172.21.15.7:6824/4175601737 9 ==== osd_op_reply(60 [pgnls start_epoch 89 out=49b] v87'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306780 con 0x560d265b2c00 2024-09-10T21:49:03.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.710+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:61 13.17 13:e8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265e4800 con 0x560d265d0c00 2024-09-10T21:49:03.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.711+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 <== osd.5 v2:172.21.15.133:6818/3838498382 6 ==== osd_op_reply(61 [pgnls start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306780 con 0x560d265d0c00 2024-09-10T21:49:03.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.711+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:62 13.37 13:ec000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d0800 con 0x560d265d0c00 2024-09-10T21:49:03.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.711+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 <== osd.5 v2:172.21.15.133:6818/3838498382 7 ==== osd_op_reply(62 [pgnls start_epoch 89 out=49b] v87'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306780 con 0x560d265d0c00 2024-09-10T21:49:03.712 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.711+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:63 13.f 13:f0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265e5c00 con 0x560d265b2000 2024-09-10T21:49:03.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.711+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.6 v2:172.21.15.133:6810/3365981160 9 ==== osd_op_reply(63 [pgnls start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b2000 2024-09-10T21:49:03.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.711+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:64 13.2f 13:f4000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265d1000 con 0x560d265b2800 2024-09-10T21:49:03.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.712+0000 7f5439600640 1 -- 172.21.15.7:0/3420302251 <== osd.4 v2:172.21.15.133:6826/1857710445 11 ==== osd_op_reply(64 [pgnls start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306000 con 0x560d265b2800 2024-09-10T21:49:03.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.712+0000 7f5434600640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:65 13.1f 13:f8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d27001400 con 0x560d265b2000 2024-09-10T21:49:03.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.712+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.6 v2:172.21.15.133:6810/3365981160 10 ==== osd_op_reply(65 [pgnls start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b2000 2024-09-10T21:49:03.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.712+0000 7f5433c00640 1 -- 172.21.15.7:0/3420302251 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:66 13.3f 13:fc000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x560d265b3c00 con 0x560d265b2000 2024-09-10T21:49:03.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543aa00640 1 -- 172.21.15.7:0/3420302251 <== osd.6 v2:172.21.15.133:6810/3365981160 11 ==== osd_op_reply(66 [pgnls start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x560d26306500 con 0x560d265b2000 2024-09-10T21:49:03.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x560d265d0000 msgr2=0x560d265b9b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x560d265d0000 0x560d265b9b80 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x560d265b2c00 msgr2=0x560d265b8580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x560d265b2c00 0x560d265b8580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x560d265b3400 msgr2=0x560d265b9600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x560d265b3400 0x560d265b9600 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x560d265d1c00 msgr2=0x560d265ba100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x560d265d1c00 0x560d265ba100 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x560d265b2800 msgr2=0x560d265b8b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x560d265b2800 0x560d265b8b00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x560d265d0c00 msgr2=0x560d265b9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x560d265d0c00 0x560d265b9080 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x560d265b2000 msgr2=0x560d265b8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x560d265b2000 0x560d265b8000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x560d2715b000 msgr2=0x560d26fde680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x560d2715b000 0x560d26fde680 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 reap_dead start 2024-09-10T21:49:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 reap_dead start 2024-09-10T21:49:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 reap_dead start 2024-09-10T21:49:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543a000640 1 -- 172.21.15.7:0/3420302251 reap_dead start 2024-09-10T21:49:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x560d2715ac00 msgr2=0x560d26fde100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x560d2715ac00 0x560d26fde100 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x560d26f7aea0 tx=0x560d26f9a000 comp rx=0 tx=0).stop 2024-09-10T21:49:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d27013400 msgr2=0x560d26fdd080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d27013400 0x560d26fdd080 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x560d2714e180 tx=0x560d270f2f00 comp rx=0 tx=0).stop 2024-09-10T21:49:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 shutdown_connections 2024-09-10T21:49:03.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x560d2715ac00 0x560d26fde100 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 --2- 172.21.15.7:0/3420302251 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d27013400 0x560d26fdd080 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 >> 172.21.15.7:0/3420302251 conn(0x560d27013000 msgr2=0x560d262ef000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 shutdown_connections 2024-09-10T21:49:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.713+0000 7f543cc20740 1 -- 172.21.15.7:0/3420302251 wait complete. 2024-09-10T21:49:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.714+0000 7f543cc20740 1 -- 172.21.15.7:0/4037144993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d261fdc00 msgr2=0x560d26fdcb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.714+0000 7f543cc20740 1 --2- 172.21.15.7:0/4037144993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d261fdc00 0x560d26fdcb00 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x560d26f7a060 tx=0x560d26fa2a80 comp rx=0 tx=0).stop 2024-09-10T21:49:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.714+0000 7f543cc20740 1 -- 172.21.15.7:0/4037144993 shutdown_connections 2024-09-10T21:49:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.714+0000 7f543cc20740 1 --2- 172.21.15.7:0/4037144993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d261fd800 0x560d26fdc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.714+0000 7f543cc20740 1 --2- 172.21.15.7:0/4037144993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d261fdc00 0x560d26fdcb00 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.714+0000 7f543cc20740 1 --2- 172.21.15.7:0/4037144993 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d261fcc00 0x560d26fdc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.714+0000 7f543cc20740 1 -- 172.21.15.7:0/4037144993 >> 172.21.15.7:0/4037144993 conn(0x560d261fd000 msgr2=0x560d262ee800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.714+0000 7f543cc20740 1 -- 172.21.15.7:0/4037144993 shutdown_connections 2024-09-10T21:49:03.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.714+0000 7f543cc20740 1 -- 172.21.15.7:0/4037144993 wait complete. 2024-09-10T21:49:03.721 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.138141, result: 2024-09-10T21:49:03.722 DEBUG:teuthology.orchestra.run.smithi007:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes cephfs_data 2024-09-10T21:49:03.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.793+0000 7f6ff1e13740 1 -- start start 2024-09-10T21:49:03.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.793+0000 7f6ff1e13740 1 Processor -- start 2024-09-10T21:49:03.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.794+0000 7f6ff1e13740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5598f0a92c00 0x5598f0c0e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.794+0000 7f6ff1e13740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598f0a93800 0x5598f0c0e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.794+0000 7f6ff1e13740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598f0a93c00 0x5598f0c0eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.794+0000 7f6ff1e13740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5598f188a180 con 0x5598f0a93c00 2024-09-10T21:49:03.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.794+0000 7f6ff1e13740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5598f188a300 con 0x5598f0a93800 2024-09-10T21:49:03.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.794+0000 7f6ff1e13740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5598f188a480 con 0x5598f0a92c00 2024-09-10T21:49:03.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.794+0000 7f6fefe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598f0a93c00 0x5598f0c0eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.794+0000 7f6fefe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598f0a93c00 0x5598f0c0eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35916/0 (socket says 172.21.15.7:35916) 2024-09-10T21:49:03.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.794+0000 7f6fefe00640 1 -- 172.21.15.7:0/975833554 learned_addr learned my addr 172.21.15.7:0/975833554 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:03.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.795+0000 7f6fef400640 1 --2- 172.21.15.7:0/975833554 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5598f0a92c00 0x5598f0c0e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.795+0000 7f6feea00640 1 --2- 172.21.15.7:0/975833554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598f0a93800 0x5598f0c0e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.795+0000 7f6fefe00640 1 -- 172.21.15.7:0/975833554 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5598f0a92c00 msgr2=0x5598f0c0e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:03.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.795+0000 7f6fefe00640 1 --2- 172.21.15.7:0/975833554 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5598f0a92c00 0x5598f0c0e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.795+0000 7f6fefe00640 1 -- 172.21.15.7:0/975833554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598f0a93800 msgr2=0x5598f0c0e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:03.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.795+0000 7f6fefe00640 1 --2- 172.21.15.7:0/975833554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598f0a93800 0x5598f0c0e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.795+0000 7f6fefe00640 1 -- 172.21.15.7:0/975833554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598f0abf680 con 0x5598f0a93c00 2024-09-10T21:49:03.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.795+0000 7f6fef400640 1 --2- 172.21.15.7:0/975833554 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5598f0a92c00 0x5598f0c0e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:03.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.795+0000 7f6feea00640 1 --2- 172.21.15.7:0/975833554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598f0a93800 0x5598f0c0e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:49:03.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.795+0000 7f6fefe00640 1 --2- 172.21.15.7:0/975833554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598f0a93c00 0x5598f0c0eb00 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x5598f18c3fb0 tx=0x5598f18433e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f7740321e5afcd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.795+0000 7f6fec200640 1 -- 172.21.15.7:0/975833554 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5598f0ab8340 con 0x5598f0a93c00 2024-09-10T21:49:03.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.795+0000 7f6fec200640 1 -- 172.21.15.7:0/975833554 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5598f0ae8fc0 con 0x5598f0a93c00 2024-09-10T21:49:03.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.795+0000 7f6fec200640 1 -- 172.21.15.7:0/975833554 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5598f0ab8ea0 con 0x5598f0a93c00 2024-09-10T21:49:03.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.795+0000 7f6ff1e13740 1 -- 172.21.15.7:0/975833554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598f0abf4a0 con 0x5598f0a93c00 2024-09-10T21:49:03.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.795+0000 7f6ff1e13740 1 -- 172.21.15.7:0/975833554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5598f0abfa40 con 0x5598f0a93c00 2024-09-10T21:49:03.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.795+0000 7f6ff1e13740 4 waiting for MDS map... 2024-09-10T21:49:03.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.796+0000 7f6fec200640 1 -- 172.21.15.7:0/975833554 <== mon.0 v2:172.21.15.7:3300/0 4 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x5598f1974200 con 0x5598f0a93c00 2024-09-10T21:49:03.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.796+0000 7f6fec200640 1 -- 172.21.15.7:0/975833554 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 71) ==== 2893+0+0 (secure 0 0 0) 0x5598f0aed180 con 0x5598f0a93c00 2024-09-10T21:49:03.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.796+0000 7f6ff1e13740 4 Got MDS map 71 2024-09-10T21:49:03.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.796+0000 7f6ff1e13740 4 datascan.main: Using metadata pool output 2024-09-10T21:49:03.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.796+0000 7f6ff1e13740 4 datascan.main: connecting to RADOS... 2024-09-10T21:49:03.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.797+0000 7f6ff1e13740 1 Processor -- start 2024-09-10T21:49:03.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.797+0000 7f6ff1e13740 1 -- start start 2024-09-10T21:49:03.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.797+0000 7f6ff1e13740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5598f19d0000 0x5598f0c0f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.797+0000 7f6ff1e13740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598f19d0400 0x5598f0c0f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.797+0000 7f6ff1e13740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598f19d0800 0x5598f0c0fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.797+0000 7f6ff1e13740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5598f188a600 con 0x5598f19d0800 2024-09-10T21:49:03.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.797+0000 7f6ff1e13740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5598f188a780 con 0x5598f19d0400 2024-09-10T21:49:03.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.797+0000 7f6ff1e13740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5598f188a900 con 0x5598f19d0000 2024-09-10T21:49:03.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.798+0000 7f6fef400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598f19d0400 0x5598f0c0f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.798+0000 7f6feea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598f19d0800 0x5598f0c0fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.798+0000 7f6fef400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598f19d0400 0x5598f0c0f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:53184/0 (socket says 172.21.15.7:53184) 2024-09-10T21:49:03.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.798+0000 7f6fef400640 1 -- 172.21.15.7:0/859895075 learned_addr learned my addr 172.21.15.7:0/859895075 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:03.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.798+0000 7f6feea00640 1 --2- 172.21.15.7:0/859895075 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5598f19d0000 0x5598f0c0f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.798+0000 7f6fef400640 1 -- 172.21.15.7:0/859895075 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5598f19d0000 msgr2=0x5598f0c0f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:03.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.798+0000 7f6fef400640 1 --2- 172.21.15.7:0/859895075 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5598f19d0000 0x5598f0c0f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.798+0000 7f6fef400640 1 -- 172.21.15.7:0/859895075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598f19d0800 msgr2=0x5598f0c0fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:03.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.798+0000 7f6fef400640 1 --2- 172.21.15.7:0/859895075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598f19d0800 0x5598f0c0fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.798+0000 7f6fef400640 1 -- 172.21.15.7:0/859895075 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598f0abf860 con 0x5598f19d0400 2024-09-10T21:49:03.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.798+0000 7f6feea00640 1 --2- 172.21.15.7:0/859895075 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5598f19d0000 0x5598f0c0f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.799+0000 7f6fef400640 1 --2- 172.21.15.7:0/859895075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598f19d0400 0x5598f0c0f600 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x5598f1977260 tx=0x5598f19e4120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f53760a238894db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.799+0000 7f6fe9a00640 1 -- 172.21.15.7:0/859895075 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5598f0ab8340 con 0x5598f19d0400 2024-09-10T21:49:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.799+0000 7f6fe9a00640 1 -- 172.21.15.7:0/859895075 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5598f0ae8c40 con 0x5598f19d0400 2024-09-10T21:49:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.799+0000 7f6fe9a00640 1 -- 172.21.15.7:0/859895075 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5598f0ab8ea0 con 0x5598f19d0400 2024-09-10T21:49:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.799+0000 7f6ff1e13740 1 -- 172.21.15.7:0/859895075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598f19d0400 msgr2=0x5598f0c0f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.799+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/859895075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598f19d0400 0x5598f0c0f600 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x5598f1977260 tx=0x5598f19e4120 comp rx=0 tx=0).stop 2024-09-10T21:49:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.799+0000 7f6ff1e13740 1 -- 172.21.15.7:0/859895075 shutdown_connections 2024-09-10T21:49:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.799+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/859895075 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598f19d0400 0x5598f0c0f600 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.799+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/859895075 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598f19d0800 0x5598f0c0fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.799+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/859895075 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5598f19d0000 0x5598f0c0f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.799+0000 7f6ff1e13740 1 -- 172.21.15.7:0/859895075 >> 172.21.15.7:0/859895075 conn(0x5598f188dc00 msgr2=0x5598f0b85000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.800+0000 7f6ff1e13740 1 -- 172.21.15.7:0/859895075 shutdown_connections 2024-09-10T21:49:03.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.800+0000 7f6ff1e13740 1 -- 172.21.15.7:0/859895075 wait complete. 2024-09-10T21:49:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.800+0000 7f6ff1e13740 1 Processor -- start 2024-09-10T21:49:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.800+0000 7f6ff1e13740 1 -- start start 2024-09-10T21:49:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.801+0000 7f6ff1e13740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598f19d0000 0x5598f0c0f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.801+0000 7f6ff1e13740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5598f19d0400 0x5598f0c0f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.801+0000 7f6ff1e13740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598f19d0800 0x5598f0c0fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.801+0000 7f6ff1e13740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5598f188bc80 con 0x5598f19d0000 2024-09-10T21:49:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.801+0000 7f6ff1e13740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5598f188bb00 con 0x5598f19d0800 2024-09-10T21:49:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.801+0000 7f6ff1e13740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5598f188b980 con 0x5598f19d0400 2024-09-10T21:49:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.801+0000 7f6feea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598f19d0000 0x5598f0c0f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.801+0000 7f6feea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598f19d0000 0x5598f0c0f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35942/0 (socket says 172.21.15.7:35942) 2024-09-10T21:49:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.801+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 learned_addr learned my addr 172.21.15.7:0/1954338988 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.801+0000 7f6fef400640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5598f19d0400 0x5598f0c0f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.805 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.802+0000 7f6feea00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598f19d0800 0x5598f0c0fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.802+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5598f19d0400 msgr2=0x5598f0c0f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.802+0000 7f6feea00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5598f19d0400 0x5598f0c0f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.802+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598f19d0800 msgr2=0x5598f0c0fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.802+0000 7f6feea00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598f19d0800 0x5598f0c0fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.802+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598f0abf2c0 con 0x5598f19d0000 2024-09-10T21:49:03.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.802+0000 7f6feea00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598f19d0000 0x5598f0c0f080 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x5598f1880780 tx=0x5598f1838c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.802+0000 7f6fe7c00640 1 -- 172.21.15.7:0/1954338988 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5598f0ab8340 con 0x5598f19d0000 2024-09-10T21:49:03.808 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.802+0000 7f6fe7c00640 1 -- 172.21.15.7:0/1954338988 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5598f0ae8540 con 0x5598f19d0000 2024-09-10T21:49:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.802+0000 7f6fe7c00640 1 -- 172.21.15.7:0/1954338988 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5598f0ab8ea0 con 0x5598f19d0000 2024-09-10T21:49:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.802+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5598f0abf0e0 con 0x5598f19d0000 2024-09-10T21:49:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.802+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598f0abe3c0 con 0x5598f19d0000 2024-09-10T21:49:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.802+0000 7f6ff1e13740 4 datascan.init: resolving metadata pool 12 2024-09-10T21:49:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.804+0000 7f6fe7c00640 1 -- 172.21.15.7:0/1954338988 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x5598f0aed500 con 0x5598f19d0000 2024-09-10T21:49:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.805+0000 7f6fe7c00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5598f19fcc00 0x5598f0c10100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.805+0000 7f6fe7c00640 1 -- 172.21.15.7:0/1954338988 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x5598f19e7000 con 0x5598f19d0000 2024-09-10T21:49:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.805+0000 7f6ff1e13740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-09-10T21:49:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.805+0000 7f6ff1e13740 4 datascan.main: data pool 'cephfs_data' has ID 13 2024-09-10T21:49:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.805+0000 7f6ff1e13740 4 datascan.main: opening data pool 'cephfs_data' 2024-09-10T21:49:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.805+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5598f19d0c00 0x5598f0c10680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.805+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:1 12.14 12:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x5598f19d1000 con 0x5598f19d0c00 2024-09-10T21:49:03.809 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.805+0000 7f6fef400640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5598f19fcc00 0x5598f0c10100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.806+0000 7f6feea00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5598f19d0c00 0x5598f0c10680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.806+0000 7f6feea00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5598f19d0c00 0x5598f0c10680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.806+0000 7f6fef400640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5598f19fcc00 0x5598f0c10100 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x5598f1977260 tx=0x5598f19e4300 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.806+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.5 v2:172.21.15.133:6818/3838498382 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv25 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f19d0c00 2024-09-10T21:49:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.806+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5598f19fd000 0x5598f0c10c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.806+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:2 12.1f 12:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x5598f19fd400 con 0x5598f19fd000 2024-09-10T21:49:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.807+0000 7f6fefe00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5598f19fd000 0x5598f0c10c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.807+0000 7f6fefe00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5598f19fd000 0x5598f0c10c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.808+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.808+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5598f19fd800 0x5598f0c11180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.808+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:3 13.0 13:00000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19fdc00 con 0x5598f19fd800 2024-09-10T21:49:03.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.808+0000 7f6fef400640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5598f19fd800 0x5598f0c11180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.808+0000 7f6fef400640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5598f19fd800 0x5598f0c11180 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.809+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.7 v2:172.21.15.133:6802/2638399040 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f19fd800 2024-09-10T21:49:03.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.809+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:4 13.20 13:04000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5e000 con 0x5598f19fd000 2024-09-10T21:49:03.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.809+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 2 ==== osd_op_reply(4 [pgnls-filter start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.809+0000 7f6fe9a00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5598f0e5e400 0x5598f0e66000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.809+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:5 13.10 13:08000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5e800 con 0x5598f0e5e400 2024-09-10T21:49:03.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.810+0000 7f6feea00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5598f0e5e400 0x5598f0e66000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.810+0000 7f6feea00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5598f0e5e400 0x5598f0e66000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.810+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.1 v2:172.21.15.7:6824/4175601737 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5e400 2024-09-10T21:49:03.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.810+0000 7f6fe9000640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5598f0e5f000 0x5598f0e66580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.810+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:6 13.30 13:0c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5f400 con 0x5598f0e5f000 2024-09-10T21:49:03.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.811+0000 7f6fefe00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5598f0e5f000 0x5598f0e66580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.811+0000 7f6fefe00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5598f0e5f000 0x5598f0e66580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.812+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.812+0000 7f6fe9a00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5598f0e5ec00 0x5598f0e66b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.812+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:7 13.8 13:10000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5f800 con 0x5598f0e5ec00 2024-09-10T21:49:03.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.812+0000 7f6fef400640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5598f0e5ec00 0x5598f0e66b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.812+0000 7f6fef400640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5598f0e5ec00 0x5598f0e66b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.813+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.2 v2:172.21.15.7:6816/4238877366 1 ==== osd_op_reply(7 [pgnls-filter start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f0e5ec00 2024-09-10T21:49:03.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.813+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:8 13.28 13:14000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74000 con 0x5598f0e5ec00 2024-09-10T21:49:03.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.813+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.2 v2:172.21.15.7:6816/4238877366 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f0e5ec00 2024-09-10T21:49:03.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.813+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:9 13.18 13:18000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5fc00 con 0x5598f19fd800 2024-09-10T21:49:03.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.814+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.7 v2:172.21.15.133:6802/2638399040 2 ==== osd_op_reply(9 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f19fd800 2024-09-10T21:49:03.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.814+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:10 13.38 13:1c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74400 con 0x5598f19d0c00 2024-09-10T21:49:03.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.814+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.5 v2:172.21.15.133:6818/3838498382 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f19d0c00 2024-09-10T21:49:03.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.814+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:11 13.4 13:20000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74000 con 0x5598f19fd000 2024-09-10T21:49:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.815+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 3 ==== osd_op_reply(11 [pgnls-filter start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.815+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:12 13.24 13:24000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74800 con 0x5598f19fd000 2024-09-10T21:49:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.815+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 4 ==== osd_op_reply(12 [pgnls-filter start_epoch 89 out=49b] v87'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.815+0000 7f6fe9a00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5598f0e5f800 0x5598f0e67080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.815+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:13 13.14 13:28000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19fdc00 con 0x5598f0e5f800 2024-09-10T21:49:03.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.816+0000 7f6feea00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5598f0e5f800 0x5598f0e67080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.816+0000 7f6feea00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5598f0e5f800 0x5598f0e67080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.816+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.0 v2:172.21.15.7:6808/651402908 1 ==== osd_op_reply(13 [pgnls-filter start_epoch 89 out=49b] v87'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5f800 2024-09-10T21:49:03.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.816+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:14 13.34 13:2c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5e800 con 0x5598f19fd000 2024-09-10T21:49:03.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.817+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.817+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:15 13.c 13:30000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19d1c00 con 0x5598f0e5e400 2024-09-10T21:49:03.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.817+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.1 v2:172.21.15.7:6824/4175601737 2 ==== osd_op_reply(15 [pgnls-filter start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5e400 2024-09-10T21:49:03.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.817+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:16 13.2c 13:34000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19fdc00 con 0x5598f0e5ec00 2024-09-10T21:49:03.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.818+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.2 v2:172.21.15.7:6816/4238877366 3 ==== osd_op_reply(16 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f0e5ec00 2024-09-10T21:49:03.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.818+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:17 13.1c 13:38000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5e000 con 0x5598f19fd000 2024-09-10T21:49:03.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.818+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 6 ==== osd_op_reply(17 [pgnls-filter start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.818+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:18 13.3c 13:3c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19fd400 con 0x5598f0e5ec00 2024-09-10T21:49:03.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.819+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.2 v2:172.21.15.7:6816/4238877366 4 ==== osd_op_reply(18 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f0e5ec00 2024-09-10T21:49:03.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.819+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:19 13.2 13:40000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5f400 con 0x5598f19d0c00 2024-09-10T21:49:03.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.819+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.5 v2:172.21.15.133:6818/3838498382 3 ==== osd_op_reply(19 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f19d0c00 2024-09-10T21:49:03.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.819+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:20 13.22 13:44000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19d1000 con 0x5598f0e5e400 2024-09-10T21:49:03.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.819+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.1 v2:172.21.15.7:6824/4175601737 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5e400 2024-09-10T21:49:03.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.819+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:21 13.12 13:48000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74000 con 0x5598f0e5f800 2024-09-10T21:49:03.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.820+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.0 v2:172.21.15.7:6808/651402908 2 ==== osd_op_reply(21 [pgnls-filter start_epoch 89 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5f800 2024-09-10T21:49:03.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.820+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:22 13.32 13:4c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e75800 con 0x5598f0e5ec00 2024-09-10T21:49:03.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.820+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.2 v2:172.21.15.7:6816/4238877366 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f0e5ec00 2024-09-10T21:49:03.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.820+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:23 13.a 13:50000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74400 con 0x5598f0e5ec00 2024-09-10T21:49:03.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.820+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.2 v2:172.21.15.7:6816/4238877366 6 ==== osd_op_reply(23 [pgnls-filter start_epoch 89 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f0e5ec00 2024-09-10T21:49:03.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.820+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:24 13.2a 13:54000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e75c00 con 0x5598f0e5e400 2024-09-10T21:49:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.821+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.1 v2:172.21.15.7:6824/4175601737 4 ==== osd_op_reply(24 [pgnls-filter start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5e400 2024-09-10T21:49:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.821+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:25 13.1a 13:58000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74800 con 0x5598f0e5e400 2024-09-10T21:49:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.821+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.1 v2:172.21.15.7:6824/4175601737 5 ==== osd_op_reply(25 [pgnls-filter start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5e400 2024-09-10T21:49:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.821+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:26 13.3a 13:5c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19fdc00 con 0x5598f19d0c00 2024-09-10T21:49:03.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.821+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.5 v2:172.21.15.133:6818/3838498382 4 ==== osd_op_reply(26 [pgnls-filter start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f19d0c00 2024-09-10T21:49:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.821+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:27 13.6 13:60000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74c00 con 0x5598f0e5f000 2024-09-10T21:49:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.822+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 2 ==== osd_op_reply(27 [pgnls-filter start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.822+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:28 13.26 13:64000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19fd400 con 0x5598f19fd800 2024-09-10T21:49:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.822+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.7 v2:172.21.15.133:6802/2638399040 3 ==== osd_op_reply(28 [pgnls-filter start_epoch 89 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f19fd800 2024-09-10T21:49:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.822+0000 7f6fe9a00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5598f0e75000 0x5598f0e67600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.822+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:29 13.16 13:68000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e75400 con 0x5598f0e75000 2024-09-10T21:49:03.824 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.822+0000 7f6fefe00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5598f0e75000 0x5598f0e67600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.823+0000 7f6fefe00640 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5598f0e75000 0x5598f0e67600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.823+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.3 v2:172.21.15.7:6800/2195665006 1 ==== osd_op_reply(29 [pgnls-filter start_epoch 89 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e75000 2024-09-10T21:49:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.823+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:30 13.36 13:6c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5f400 con 0x5598f19fd000 2024-09-10T21:49:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.824+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 7 ==== osd_op_reply(30 [pgnls-filter start_epoch 89 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.824+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:31 13.e 13:70000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74000 con 0x5598f19fd800 2024-09-10T21:49:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.824+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.7 v2:172.21.15.133:6802/2638399040 4 ==== osd_op_reply(31 [pgnls-filter start_epoch 89 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f19fd800 2024-09-10T21:49:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.824+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:32 13.2e 13:74000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5fc00 con 0x5598f0e5ec00 2024-09-10T21:49:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.824+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.2 v2:172.21.15.7:6816/4238877366 7 ==== osd_op_reply(32 [pgnls-filter start_epoch 89 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f0e5ec00 2024-09-10T21:49:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.824+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:33 13.1e 13:78000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e75c00 con 0x5598f0e5f000 2024-09-10T21:49:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.825+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 3 ==== osd_op_reply(33 [pgnls-filter start_epoch 89 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.825+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:34 13.3e 13:7c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19d1c00 con 0x5598f0e5e400 2024-09-10T21:49:03.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.825+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.1 v2:172.21.15.7:6824/4175601737 6 ==== osd_op_reply(34 [pgnls-filter start_epoch 89 out=49b] v87'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5e400 2024-09-10T21:49:03.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.825+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:35 13.1 13:80000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e75800 con 0x5598f19fd000 2024-09-10T21:49:03.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.826+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 8 ==== osd_op_reply(35 [pgnls-filter start_epoch 89 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.826+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:36 13.21 13:84000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19d1000 con 0x5598f0e5f000 2024-09-10T21:49:03.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.826+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 89 out=49b] v87'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.826+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:37 13.11 13:88000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74400 con 0x5598f0e5e400 2024-09-10T21:49:03.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.826+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.1 v2:172.21.15.7:6824/4175601737 7 ==== osd_op_reply(37 [pgnls-filter start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5e400 2024-09-10T21:49:03.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.827+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:38 13.31 13:8c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74c00 con 0x5598f0e75000 2024-09-10T21:49:03.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.827+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.3 v2:172.21.15.7:6800/2195665006 2 ==== osd_op_reply(38 [pgnls-filter start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e75000 2024-09-10T21:49:03.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.827+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:39 13.9 13:90000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e88000 con 0x5598f19d0c00 2024-09-10T21:49:03.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.828+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.5 v2:172.21.15.133:6818/3838498382 5 ==== osd_op_reply(39 [pgnls-filter start_epoch 89 out=49b] v87'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f19d0c00 2024-09-10T21:49:03.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.828+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:40 13.29 13:94000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e75400 con 0x5598f19fd000 2024-09-10T21:49:03.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.828+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 9 ==== osd_op_reply(40 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.828+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:41 13.19 13:98000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e88400 con 0x5598f0e75000 2024-09-10T21:49:03.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.829+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.3 v2:172.21.15.7:6800/2195665006 3 ==== osd_op_reply(41 [pgnls-filter start_epoch 89 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e75000 2024-09-10T21:49:03.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.829+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:42 13.39 13:9c000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e75c00 con 0x5598f0e5ec00 2024-09-10T21:49:03.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.829+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.2 v2:172.21.15.7:6816/4238877366 8 ==== osd_op_reply(42 [pgnls-filter start_epoch 89 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f0e5ec00 2024-09-10T21:49:03.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.829+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:43 13.5 13:a0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e88800 con 0x5598f0e5f000 2024-09-10T21:49:03.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.829+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 5 ==== osd_op_reply(43 [pgnls-filter start_epoch 89 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.829+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:44 13.25 13:a4000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5e800 con 0x5598f0e5f000 2024-09-10T21:49:03.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.830+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 6 ==== osd_op_reply(44 [pgnls-filter start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.830+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:45 13.15 13:a8000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e88c00 con 0x5598f0e5f000 2024-09-10T21:49:03.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.830+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 7 ==== osd_op_reply(45 [pgnls-filter start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.830+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:46 13.35 13:ac000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5e000 con 0x5598f0e75000 2024-09-10T21:49:03.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.830+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.3 v2:172.21.15.7:6800/2195665006 4 ==== osd_op_reply(46 [pgnls-filter start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e75000 2024-09-10T21:49:03.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.830+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:47 13.d 13:b0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e89000 con 0x5598f0e5f000 2024-09-10T21:49:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.831+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 8 ==== osd_op_reply(47 [pgnls-filter start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.831+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:48 13.2d 13:b4000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5f400 con 0x5598f0e5f800 2024-09-10T21:49:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.831+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.0 v2:172.21.15.7:6808/651402908 3 ==== osd_op_reply(48 [pgnls-filter start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5f800 2024-09-10T21:49:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.831+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:49 13.1d 13:b8000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e89400 con 0x5598f19fd800 2024-09-10T21:49:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.832+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.7 v2:172.21.15.133:6802/2638399040 5 ==== osd_op_reply(49 [pgnls-filter start_epoch 89 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f19fd800 2024-09-10T21:49:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.832+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:50 13.3d 13:bc000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74800 con 0x5598f19fd800 2024-09-10T21:49:03.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.832+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.7 v2:172.21.15.133:6802/2638399040 6 ==== osd_op_reply(50 [pgnls-filter start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f19fd800 2024-09-10T21:49:03.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.832+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:51 13.3 13:c0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e89800 con 0x5598f0e5f800 2024-09-10T21:49:03.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.833+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.0 v2:172.21.15.7:6808/651402908 4 ==== osd_op_reply(51 [pgnls-filter start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5f800 2024-09-10T21:49:03.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.833+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:52 13.23 13:c4000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74400 con 0x5598f0e5e400 2024-09-10T21:49:03.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.833+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.1 v2:172.21.15.7:6824/4175601737 8 ==== osd_op_reply(52 [pgnls-filter start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5e400 2024-09-10T21:49:03.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.833+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:53 13.13 13:c8000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e89c00 con 0x5598f19fd800 2024-09-10T21:49:03.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.833+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.7 v2:172.21.15.133:6802/2638399040 7 ==== osd_op_reply(53 [pgnls-filter start_epoch 89 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f19fd800 2024-09-10T21:49:03.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.833+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:54 13.33 13:cc000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e75800 con 0x5598f19fd800 2024-09-10T21:49:03.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.834+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.7 v2:172.21.15.133:6802/2638399040 8 ==== osd_op_reply(54 [pgnls-filter start_epoch 89 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f19fd800 2024-09-10T21:49:03.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.834+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:55 13.b 13:d0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5fc00 con 0x5598f19d0c00 2024-09-10T21:49:03.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.834+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.5 v2:172.21.15.133:6818/3838498382 6 ==== osd_op_reply(55 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f19d0c00 2024-09-10T21:49:03.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.834+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:56 13.2b 13:d4000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74c00 con 0x5598f0e5ec00 2024-09-10T21:49:03.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.834+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.2 v2:172.21.15.7:6816/4238877366 9 ==== osd_op_reply(56 [pgnls-filter start_epoch 89 out=49b] v87'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f0e5ec00 2024-09-10T21:49:03.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.834+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:57 13.1b 13:d8000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e89400 con 0x5598f0e5f000 2024-09-10T21:49:03.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.835+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 9 ==== osd_op_reply(57 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.835+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:58 13.3b 13:dc000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e75400 con 0x5598f0e5f000 2024-09-10T21:49:03.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.835+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 10 ==== osd_op_reply(58 [pgnls-filter start_epoch 89 out=49b] v87'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.835+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:59 13.7 13:e0000000::::head [pgnls-filter start_epoch 89 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e88000 con 0x5598f19fd800 2024-09-10T21:49:03.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.836+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.7 v2:172.21.15.133:6802/2638399040 9 ==== osd_op_reply(59 [pgnls-filter start_epoch 89 out=81b] v89'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f19fd800 2024-09-10T21:49:03.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.836+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:60 13.0 13:00000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5f400 con 0x5598f19fd800 2024-09-10T21:49:03.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.836+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.7 v2:172.21.15.133:6802/2638399040 10 ==== osd_op_reply(60 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f19fd800 2024-09-10T21:49:03.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.836+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:61 13.20 13:04000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e88400 con 0x5598f19fd000 2024-09-10T21:49:03.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.837+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 10 ==== osd_op_reply(61 [pgnls-filter start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.837+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:62 13.10 13:08000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19d1c00 con 0x5598f0e5e400 2024-09-10T21:49:03.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.837+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.1 v2:172.21.15.7:6824/4175601737 9 ==== osd_op_reply(62 [pgnls-filter start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5e400 2024-09-10T21:49:03.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.837+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:63 13.30 13:0c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e88800 con 0x5598f0e5f000 2024-09-10T21:49:03.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.838+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 11 ==== osd_op_reply(63 [pgnls-filter start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.838+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:64 13.8 13:10000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19d1000 con 0x5598f0e5ec00 2024-09-10T21:49:03.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.838+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.2 v2:172.21.15.7:6816/4238877366 10 ==== osd_op_reply(64 [pgnls-filter start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f0e5ec00 2024-09-10T21:49:03.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.838+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:65 13.28 13:14000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74000 con 0x5598f0e5ec00 2024-09-10T21:49:03.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.838+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.2 v2:172.21.15.7:6816/4238877366 11 ==== osd_op_reply(65 [pgnls-filter start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f0e5ec00 2024-09-10T21:49:03.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.838+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:66 13.18 13:18000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19fdc00 con 0x5598f19fd800 2024-09-10T21:49:03.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.839+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.7 v2:172.21.15.133:6802/2638399040 11 ==== osd_op_reply(66 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f19fd800 2024-09-10T21:49:03.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.839+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:67 13.38 13:1c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e75c00 con 0x5598f19d0c00 2024-09-10T21:49:03.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.839+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.5 v2:172.21.15.133:6818/3838498382 7 ==== osd_op_reply(67 [pgnls-filter start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f19d0c00 2024-09-10T21:49:03.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.839+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:68 13.4 13:20000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e75800 con 0x5598f19fd000 2024-09-10T21:49:03.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.839+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 11 ==== osd_op_reply(68 [pgnls-filter start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.839+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:69 13.24 13:24000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74800 con 0x5598f19fd000 2024-09-10T21:49:03.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.840+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 12 ==== osd_op_reply(69 [pgnls-filter start_epoch 89 out=49b] v87'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.840+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:70 13.14 13:28000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74c00 con 0x5598f0e5f800 2024-09-10T21:49:03.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.840+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.0 v2:172.21.15.7:6808/651402908 5 ==== osd_op_reply(70 [pgnls-filter start_epoch 89 out=49b] v87'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5f800 2024-09-10T21:49:03.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.840+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:71 13.34 13:2c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19fd400 con 0x5598f19fd000 2024-09-10T21:49:03.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.841+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 13 ==== osd_op_reply(71 [pgnls-filter start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.841+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:72 13.c 13:30000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74000 con 0x5598f0e5e400 2024-09-10T21:49:03.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.841+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.1 v2:172.21.15.7:6824/4175601737 10 ==== osd_op_reply(72 [pgnls-filter start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5e400 2024-09-10T21:49:03.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.841+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:73 13.2c 13:34000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74400 con 0x5598f0e5ec00 2024-09-10T21:49:03.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.841+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.2 v2:172.21.15.7:6816/4238877366 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f0e5ec00 2024-09-10T21:49:03.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.841+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:74 13.1c 13:38000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19d1000 con 0x5598f19fd000 2024-09-10T21:49:03.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.842+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 14 ==== osd_op_reply(74 [pgnls-filter start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.842+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:75 13.3c 13:3c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e89800 con 0x5598f0e5ec00 2024-09-10T21:49:03.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.842+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.2 v2:172.21.15.7:6816/4238877366 13 ==== osd_op_reply(75 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f0e5ec00 2024-09-10T21:49:03.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.842+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:76 13.2 13:40000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5f400 con 0x5598f19d0c00 2024-09-10T21:49:03.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.842+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.5 v2:172.21.15.133:6818/3838498382 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f19d0c00 2024-09-10T21:49:03.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.842+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:77 13.22 13:44000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e88c00 con 0x5598f0e5e400 2024-09-10T21:49:03.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.843+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.1 v2:172.21.15.7:6824/4175601737 11 ==== osd_op_reply(77 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5e400 2024-09-10T21:49:03.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.843+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:78 13.12 13:48000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e89c00 con 0x5598f0e5f800 2024-09-10T21:49:03.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.843+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.0 v2:172.21.15.7:6808/651402908 6 ==== osd_op_reply(78 [pgnls-filter start_epoch 89 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5f800 2024-09-10T21:49:03.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.843+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:79 13.32 13:4c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e89000 con 0x5598f0e5ec00 2024-09-10T21:49:03.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.843+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.2 v2:172.21.15.7:6816/4238877366 14 ==== osd_op_reply(79 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f0e5ec00 2024-09-10T21:49:03.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.843+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:80 13.a 13:50000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e88000 con 0x5598f0e5ec00 2024-09-10T21:49:03.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.844+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.2 v2:172.21.15.7:6816/4238877366 15 ==== osd_op_reply(80 [pgnls-filter start_epoch 89 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f0e5ec00 2024-09-10T21:49:03.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.844+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:81 13.2a 13:54000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e89400 con 0x5598f0e5e400 2024-09-10T21:49:03.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.844+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.1 v2:172.21.15.7:6824/4175601737 12 ==== osd_op_reply(81 [pgnls-filter start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5e400 2024-09-10T21:49:03.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.844+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:82 13.1a 13:58000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e75c00 con 0x5598f0e5e400 2024-09-10T21:49:03.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.844+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.1 v2:172.21.15.7:6824/4175601737 13 ==== osd_op_reply(82 [pgnls-filter start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5e400 2024-09-10T21:49:03.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.844+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:83 13.3a 13:5c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e88400 con 0x5598f19d0c00 2024-09-10T21:49:03.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.845+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.5 v2:172.21.15.133:6818/3838498382 9 ==== osd_op_reply(83 [pgnls-filter start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f19d0c00 2024-09-10T21:49:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.845+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:84 13.6 13:60000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74c00 con 0x5598f0e5f000 2024-09-10T21:49:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.845+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 12 ==== osd_op_reply(84 [pgnls-filter start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.845+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:85 13.26 13:64000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e88800 con 0x5598f19fd800 2024-09-10T21:49:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.845+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.7 v2:172.21.15.133:6802/2638399040 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 89 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f19fd800 2024-09-10T21:49:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.846+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:86 13.16 13:68000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74000 con 0x5598f0e75000 2024-09-10T21:49:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.846+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.3 v2:172.21.15.7:6800/2195665006 5 ==== osd_op_reply(86 [pgnls-filter start_epoch 89 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e75000 2024-09-10T21:49:03.848 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.846+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:87 13.36 13:6c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5fc00 con 0x5598f19fd000 2024-09-10T21:49:03.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.846+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 15 ==== osd_op_reply(87 [pgnls-filter start_epoch 89 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.846+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:88 13.e 13:70000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e75400 con 0x5598f19fd800 2024-09-10T21:49:03.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.847+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.7 v2:172.21.15.133:6802/2638399040 13 ==== osd_op_reply(88 [pgnls-filter start_epoch 89 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f19fd800 2024-09-10T21:49:03.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.847+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:89 13.2e 13:74000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5e000 con 0x5598f0e5ec00 2024-09-10T21:49:03.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.847+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.2 v2:172.21.15.7:6816/4238877366 16 ==== osd_op_reply(89 [pgnls-filter start_epoch 89 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f0e5ec00 2024-09-10T21:49:03.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.847+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:90 13.1e 13:78000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e75800 con 0x5598f0e5f000 2024-09-10T21:49:03.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.848+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 13 ==== osd_op_reply(90 [pgnls-filter start_epoch 89 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.848+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:91 13.3e 13:7c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19d1c00 con 0x5598f0e5e400 2024-09-10T21:49:03.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.848+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.1 v2:172.21.15.7:6824/4175601737 14 ==== osd_op_reply(91 [pgnls-filter start_epoch 89 out=49b] v87'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5e400 2024-09-10T21:49:03.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.848+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:92 13.1 13:80000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5e000 con 0x5598f19fd000 2024-09-10T21:49:03.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.849+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 16 ==== osd_op_reply(92 [pgnls-filter start_epoch 89 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.849+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:93 13.21 13:84000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e9a000 con 0x5598f0e5f000 2024-09-10T21:49:03.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.849+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 14 ==== osd_op_reply(93 [pgnls-filter start_epoch 89 out=49b] v87'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.849+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:94 13.11 13:88000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74400 con 0x5598f0e5e400 2024-09-10T21:49:03.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.850+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.1 v2:172.21.15.7:6824/4175601737 15 ==== osd_op_reply(94 [pgnls-filter start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5e400 2024-09-10T21:49:03.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.850+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:95 13.31 13:8c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e9a400 con 0x5598f0e75000 2024-09-10T21:49:03.852 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.850+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.3 v2:172.21.15.7:6800/2195665006 6 ==== osd_op_reply(95 [pgnls-filter start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e75000 2024-09-10T21:49:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.850+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:96 13.9 13:90000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e75400 con 0x5598f19d0c00 2024-09-10T21:49:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.851+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.5 v2:172.21.15.133:6818/3838498382 10 ==== osd_op_reply(96 [pgnls-filter start_epoch 89 out=49b] v87'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f19d0c00 2024-09-10T21:49:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.851+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:97 13.29 13:94000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e9a800 con 0x5598f19fd000 2024-09-10T21:49:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.851+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 17 ==== osd_op_reply(97 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.851+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:98 13.19 13:98000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e89800 con 0x5598f0e75000 2024-09-10T21:49:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.851+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.3 v2:172.21.15.7:6800/2195665006 7 ==== osd_op_reply(98 [pgnls-filter start_epoch 89 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e75000 2024-09-10T21:49:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.851+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:99 13.39 13:9c000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e9ac00 con 0x5598f0e5ec00 2024-09-10T21:49:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.852+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.2 v2:172.21.15.7:6816/4238877366 17 ==== osd_op_reply(99 [pgnls-filter start_epoch 89 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f0e5ec00 2024-09-10T21:49:03.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.852+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:100 13.5 13:a0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e89000 con 0x5598f0e5f000 2024-09-10T21:49:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.852+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 15 ==== osd_op_reply(100 [pgnls-filter start_epoch 89 out=49b] v87'5 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.852+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:101 13.25 13:a4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e9b000 con 0x5598f0e5f000 2024-09-10T21:49:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.853+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 16 ==== osd_op_reply(101 [pgnls-filter start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.853+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:102 13.15 13:a8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e88000 con 0x5598f0e5f000 2024-09-10T21:49:03.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.853+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 17 ==== osd_op_reply(102 [pgnls-filter start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.853+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:103 13.35 13:ac000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e9b400 con 0x5598f0e75000 2024-09-10T21:49:03.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.853+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.3 v2:172.21.15.7:6800/2195665006 8 ==== osd_op_reply(103 [pgnls-filter start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e75000 2024-09-10T21:49:03.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.853+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:104 13.d 13:b0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e88800 con 0x5598f0e5f000 2024-09-10T21:49:03.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.854+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 18 ==== osd_op_reply(104 [pgnls-filter start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.854+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:105 13.2d 13:b4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e9b800 con 0x5598f0e5f800 2024-09-10T21:49:03.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.854+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.0 v2:172.21.15.7:6808/651402908 7 ==== osd_op_reply(105 [pgnls-filter start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5f800 2024-09-10T21:49:03.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.854+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:106 13.1d 13:b8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19fdc00 con 0x5598f19fd800 2024-09-10T21:49:03.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.855+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.7 v2:172.21.15.133:6802/2638399040 14 ==== osd_op_reply(106 [pgnls-filter start_epoch 89 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f19fd800 2024-09-10T21:49:03.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.855+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:107 13.3d 13:bc000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e9bc00 con 0x5598f19fd800 2024-09-10T21:49:03.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.855+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.7 v2:172.21.15.133:6802/2638399040 15 ==== osd_op_reply(107 [pgnls-filter start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f19fd800 2024-09-10T21:49:03.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.855+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:108 13.3 13:c0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e75c00 con 0x5598f0e5f800 2024-09-10T21:49:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.855+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.0 v2:172.21.15.7:6808/651402908 8 ==== osd_op_reply(108 [pgnls-filter start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5f800 2024-09-10T21:49:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.855+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:109 13.23 13:c4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e9a000 con 0x5598f0e5e400 2024-09-10T21:49:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.855+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.1 v2:172.21.15.7:6824/4175601737 16 ==== osd_op_reply(109 [pgnls-filter start_epoch 89 out=49b] v87'7 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5e400 2024-09-10T21:49:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.855+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:110 13.13 13:c8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74400 con 0x5598f19fd800 2024-09-10T21:49:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.856+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.7 v2:172.21.15.133:6802/2638399040 16 ==== osd_op_reply(110 [pgnls-filter start_epoch 89 out=49b] v87'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f19fd800 2024-09-10T21:49:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.856+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:111 13.33 13:cc000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5f400 con 0x5598f19fd800 2024-09-10T21:49:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.856+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.7 v2:172.21.15.133:6802/2638399040 17 ==== osd_op_reply(111 [pgnls-filter start_epoch 89 out=49b] v87'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f19fd800 2024-09-10T21:49:03.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.856+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:112 13.b 13:d0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74800 con 0x5598f19d0c00 2024-09-10T21:49:03.858 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.857+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.5 v2:172.21.15.133:6818/3838498382 11 ==== osd_op_reply(112 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f19d0c00 2024-09-10T21:49:03.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.857+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:113 13.2b 13:d4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5fc00 con 0x5598f0e5ec00 2024-09-10T21:49:03.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.857+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.2 v2:172.21.15.7:6816/4238877366 18 ==== osd_op_reply(113 [pgnls-filter start_epoch 89 out=49b] v87'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f0e5ec00 2024-09-10T21:49:03.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.857+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:114 13.1b 13:d8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74c00 con 0x5598f0e5f000 2024-09-10T21:49:03.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.858+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 19 ==== osd_op_reply(114 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.859 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.858+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:115 13.3b 13:dc000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e5e000 con 0x5598f0e5f000 2024-09-10T21:49:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.858+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 20 ==== osd_op_reply(115 [pgnls-filter start_epoch 89 out=49b] v87'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.858+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:116 13.7 13:e0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e74000 con 0x5598f19fd800 2024-09-10T21:49:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.859+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.7 v2:172.21.15.133:6802/2638399040 18 ==== osd_op_reply(116 [pgnls-filter start_epoch 89 out=81b] v89'7 uv7 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f19fd800 2024-09-10T21:49:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.859+0000 7f6ff1e13740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2024-09-10T21:49:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.859+0000 7f6ff1e13740 10 datascan.operator(): handling object 10000000001.0 2024-09-10T21:49:03.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.859+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:117 13.7 13:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e89800 con 0x5598f19fd800 2024-09-10T21:49:03.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.859+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 <== osd.7 v2:172.21.15.133:6802/2638399040 19 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (89) out=89b,getxattr (30) out=30b,getxattr] v0'0 uv7 ondisk = 0) ==== 416+0+157 (crc 0 0 0) 0x5598f0b9c780 con 0x5598f19fd800 2024-09-10T21:49:03.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.859+0000 7f6ff1e13740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2024-09-10T21:49:03.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.859+0000 7f6ff1e13740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/file 2024-09-10T21:49:03.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.859+0000 7f6ff1e13740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=file 2024-09-10T21:49:03.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.859+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:118 12.10 12:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e89000 con 0x5598f0e5f800 2024-09-10T21:49:03.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.860+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.0 v2:172.21.15.7:6808/651402908 9 ==== osd_op_reply(118 10000000000.00000000 [getxattr (59) out=59b] v0'0 uv17 ondisk = 0) ==== 164+0+59 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5f800 2024-09-10T21:49:03.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.860+0000 7f6ff1e13740 20 datascan.get_frag_of: got backtrace parent 0x1/dir 2024-09-10T21:49:03.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.860+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:119 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e88000 con 0x5598f19fd000 2024-09-10T21:49:03.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.861+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 18 ==== osd_op_reply(119 1.00000000 [omap-get-vals-by-keys out=581b] v0'0 uv9 ondisk = 0) ==== 154+0+581 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.861+0000 7f6ff1e13740 20 datascan.read_dentry: oid=1.00000000 dname=dir frag=*, r=0 2024-09-10T21:49:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.861+0000 7f6ff1e13740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-09-10T21:49:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.861+0000 7f6ff1e13740 20 datascan.get_frag_of: frag is * 2024-09-10T21:49:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.861+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:120 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e88800 con 0x5598f0e5f800 2024-09-10T21:49:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.861+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.0 v2:172.21.15.7:6808/651402908 10 ==== osd_op_reply(120 10000000000.00000000 [omap-get-header out=274b] v0'0 uv17 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5f800 2024-09-10T21:49:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.861+0000 7f6ff1e13740 10 datascan.find_or_create_dirfrag: read_version = 17 2024-09-10T21:49:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.861+0000 7f6ff1e13740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000000 * 2024-09-10T21:49:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.861+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:121 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e9a400 con 0x5598f0e5f800 2024-09-10T21:49:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.861+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.0 v2:172.21.15.7:6808/651402908 11 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv17 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5f800 2024-09-10T21:49:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.861+0000 7f6ff1e13740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-09-10T21:49:03.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.861+0000 7f6ff1e13740 20 datascan.read_dentry: file_head not found in result 2024-09-10T21:49:03.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.861+0000 7f6ff1e13740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/file with size=0 bytes 2024-09-10T21:49:03.864 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.861+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:122 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e9a800 con 0x5598f0e5f800 2024-09-10T21:49:03.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.863+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.0 v2:172.21.15.7:6808/651402908 12 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v89'18 uv18 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5f800 2024-09-10T21:49:03.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.863+0000 7f6ff1e13740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-09-10T21:49:03.865 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.863+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:123 13.27 13:e4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e9b000 con 0x5598f0e5e400 2024-09-10T21:49:03.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.863+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.1 v2:172.21.15.7:6824/4175601737 17 ==== osd_op_reply(123 [pgnls-filter start_epoch 89 out=49b] v87'2 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f0e5e400 2024-09-10T21:49:03.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.863+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:124 13.17 13:e8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19d1c00 con 0x5598f19d0c00 2024-09-10T21:49:03.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.864+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.5 v2:172.21.15.133:6818/3838498382 12 ==== osd_op_reply(124 [pgnls-filter start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f19d0c00 2024-09-10T21:49:03.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.864+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:125 13.37 13:ec000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e75800 con 0x5598f19d0c00 2024-09-10T21:49:03.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.864+0000 7f6feea00640 1 -- 172.21.15.7:0/1954338988 <== osd.5 v2:172.21.15.133:6818/3838498382 13 ==== osd_op_reply(125 [pgnls-filter start_epoch 89 out=49b] v87'11 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c000 con 0x5598f19d0c00 2024-09-10T21:49:03.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.864+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:126 13.f 13:f0000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19d1000 con 0x5598f19fd000 2024-09-10T21:49:03.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.864+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 19 ==== osd_op_reply(126 [pgnls-filter start_epoch 89 out=49b] v87'9 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.864+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:127 13.2f 13:f4000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e75400 con 0x5598f0e5f000 2024-09-10T21:49:03.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.865+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.4 v2:172.21.15.133:6826/1857710445 21 ==== osd_op_reply(127 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f0e5f000 2024-09-10T21:49:03.866 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.865+0000 7f6fe9000640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:128 13.1f 13:f8000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f19fd400 con 0x5598f19fd000 2024-09-10T21:49:03.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.865+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 20 ==== osd_op_reply(128 [pgnls-filter start_epoch 89 out=49b] v87'6 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6fe9a00640 1 -- 172.21.15.7:0/1954338988 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:129 13.3f 13:fc000000::::head [pgnls-filter start_epoch 89 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x5598f0e75c00 con 0x5598f19fd000 2024-09-10T21:49:03.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6fefe00640 1 -- 172.21.15.7:0/1954338988 <== osd.6 v2:172.21.15.133:6810/3365981160 21 ==== osd_op_reply(129 [pgnls-filter start_epoch 89 out=49b] v87'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5598f0b9c500 con 0x5598f19fd000 2024-09-10T21:49:03.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5598f0e5f800 msgr2=0x5598f0e67080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5598f0e5f800 0x5598f0e67080 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5598f0e5e400 msgr2=0x5598f0e66000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5598f0e5e400 0x5598f0e66000 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5598f0e5ec00 msgr2=0x5598f0e66b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5598f0e5ec00 0x5598f0e66b00 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5598f0e75000 msgr2=0x5598f0e67600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5598f0e75000 0x5598f0e67600 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 reap_dead start 2024-09-10T21:49:03.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5598f0e5f000 msgr2=0x5598f0e66580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5598f0e5f000 0x5598f0e66580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5598f19d0c00 msgr2=0x5598f0c10680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5598f19d0c00 0x5598f0c10680 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5598f19fd000 msgr2=0x5598f0c10c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5598f19fd000 0x5598f0c10c00 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5598f19fd800 msgr2=0x5598f0c11180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5598f19fd800 0x5598f0c11180 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6fef400640 1 -- 172.21.15.7:0/1954338988 reap_dead start 2024-09-10T21:49:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5598f19fcc00 msgr2=0x5598f0c10100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5598f19fcc00 0x5598f0c10100 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x5598f1977260 tx=0x5598f19e4300 comp rx=0 tx=0).stop 2024-09-10T21:49:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598f19d0000 msgr2=0x5598f0c0f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598f19d0000 0x5598f0c0f080 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x5598f1880780 tx=0x5598f1838c60 comp rx=0 tx=0).stop 2024-09-10T21:49:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 shutdown_connections 2024-09-10T21:49:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5598f19fcc00 0x5598f0c10100 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/1954338988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598f19d0000 0x5598f0c0f080 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.866+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 >> 172.21.15.7:0/1954338988 conn(0x5598f188dc00 msgr2=0x5598f0b85000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.867+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 shutdown_connections 2024-09-10T21:49:03.870 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.867+0000 7f6ff1e13740 1 -- 172.21.15.7:0/1954338988 wait complete. 2024-09-10T21:49:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.867+0000 7f6ff1e13740 1 -- 172.21.15.7:0/975833554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598f0a93c00 msgr2=0x5598f0c0eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.867+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/975833554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598f0a93c00 0x5598f0c0eb00 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x5598f18c3fb0 tx=0x5598f18433e0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.867+0000 7f6ff1e13740 1 -- 172.21.15.7:0/975833554 shutdown_connections 2024-09-10T21:49:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.867+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/975833554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5598f0a93800 0x5598f0c0e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.867+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/975833554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5598f0a93c00 0x5598f0c0eb00 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.867+0000 7f6ff1e13740 1 --2- 172.21.15.7:0/975833554 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5598f0a92c00 0x5598f0c0e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.867+0000 7f6ff1e13740 1 -- 172.21.15.7:0/975833554 >> 172.21.15.7:0/975833554 conn(0x5598f0a93000 msgr2=0x5598f0b84800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.867+0000 7f6ff1e13740 1 -- 172.21.15.7:0/975833554 shutdown_connections 2024-09-10T21:49:03.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.867+0000 7f6ff1e13740 1 -- 172.21.15.7:0/975833554 wait complete. 2024-09-10T21:49:03.873 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', 'cephfs_data'] in time 0:00:00.151849, result: 2024-09-10T21:49:03.874 DEBUG:teuthology.orchestra.run.smithi007:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-09-10T21:49:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.944+0000 7fe1c5622740 1 -- start start 2024-09-10T21:49:03.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.944+0000 7fe1c5622740 1 Processor -- start 2024-09-10T21:49:03.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.945+0000 7fe1c5622740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258b304c00 0x56258b480000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.945+0000 7fe1c5622740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56258b305800 0x56258b480580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.945+0000 7fe1c5622740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56258b305c00 0x56258b480b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.945+0000 7fe1c5622740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56258c1d8180 con 0x56258b304c00 2024-09-10T21:49:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.945+0000 7fe1c5622740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56258c1d8300 con 0x56258b305c00 2024-09-10T21:49:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.945+0000 7fe1c5622740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x56258c1d8480 con 0x56258b305800 2024-09-10T21:49:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.945+0000 7fe1c2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258b304c00 0x56258b480000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.945+0000 7fe1c2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258b304c00 0x56258b480000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52166/0 (socket says 172.21.15.7:52166) 2024-09-10T21:49:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.945+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3536387517 learned_addr learned my addr 172.21.15.7:0/3536387517 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.945+0000 7fe1c3400640 1 --2- 172.21.15.7:0/3536387517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56258b305c00 0x56258b480b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.945+0000 7fe1c2000640 1 --2- 172.21.15.7:0/3536387517 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56258b305800 0x56258b480580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.945+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3536387517 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56258b305800 msgr2=0x56258b480580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:03.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.945+0000 7fe1c2a00640 1 --2- 172.21.15.7:0/3536387517 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56258b305800 0x56258b480580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.945+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3536387517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56258b305c00 msgr2=0x56258b480b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.945+0000 7fe1c2a00640 1 --2- 172.21.15.7:0/3536387517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56258b305c00 0x56258b480b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.945+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3536387517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56258b331680 con 0x56258b304c00 2024-09-10T21:49:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.945+0000 7fe1c3400640 1 --2- 172.21.15.7:0/3536387517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56258b305c00 0x56258b480b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.946+0000 7fe1c2a00640 1 --2- 172.21.15.7:0/3536387517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258b304c00 0x56258b480000 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x56258c1e2060 tx=0x56258c0b5260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f776c7bcd08f439b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.946+0000 7fe1bf800640 1 -- 172.21.15.7:0/3536387517 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56258b32aea0 con 0x56258b304c00 2024-09-10T21:49:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.946+0000 7fe1bf800640 1 -- 172.21.15.7:0/3536387517 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56258b35afc0 con 0x56258b304c00 2024-09-10T21:49:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.946+0000 7fe1bf800640 1 -- 172.21.15.7:0/3536387517 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56258b32b380 con 0x56258b304c00 2024-09-10T21:49:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.946+0000 7fe1c5622740 1 -- 172.21.15.7:0/3536387517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56258b3314a0 con 0x56258b304c00 2024-09-10T21:49:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.946+0000 7fe1c5622740 1 -- 172.21.15.7:0/3536387517 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=0}) -- 0x56258b331a40 con 0x56258b304c00 2024-09-10T21:49:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.946+0000 7fe1c5622740 4 waiting for MDS map... 2024-09-10T21:49:03.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.946+0000 7fe1bf800640 1 -- 172.21.15.7:0/3536387517 <== mon.0 v2:172.21.15.7:3300/0 4 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x56258c1e8600 con 0x56258b304c00 2024-09-10T21:49:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.946+0000 7fe1bf800640 1 -- 172.21.15.7:0/3536387517 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 71) ==== 2893+0+0 (secure 0 0 0) 0x56258b35f180 con 0x56258b304c00 2024-09-10T21:49:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.947+0000 7fe1c5622740 4 Got MDS map 71 2024-09-10T21:49:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.947+0000 7fe1c5622740 4 datascan.main: Using metadata pool output 2024-09-10T21:49:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.947+0000 7fe1c5622740 4 datascan.main: connecting to RADOS... 2024-09-10T21:49:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.947+0000 7fe1c5622740 1 Processor -- start 2024-09-10T21:49:03.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.947+0000 7fe1c5622740 1 -- start start 2024-09-10T21:49:03.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.948+0000 7fe1c5622740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258c1ed400 0x56258b481080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.948+0000 7fe1c5622740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56258c1ed800 0x56258b481600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.948+0000 7fe1c5622740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56258c1edc00 0x56258b481b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.948+0000 7fe1c5622740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56258c1d8600 con 0x56258c1ed400 2024-09-10T21:49:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.948+0000 7fe1c5622740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56258c1d8780 con 0x56258c1edc00 2024-09-10T21:49:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.948+0000 7fe1c5622740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x56258c1d8900 con 0x56258c1ed800 2024-09-10T21:49:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.948+0000 7fe1c3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258c1ed400 0x56258b481080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.948+0000 7fe1c3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258c1ed400 0x56258b481080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52178/0 (socket says 172.21.15.7:52178) 2024-09-10T21:49:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.948+0000 7fe1c3400640 1 -- 172.21.15.7:0/1980973303 learned_addr learned my addr 172.21.15.7:0/1980973303 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.949+0000 7fe1c2000640 1 --2- 172.21.15.7:0/1980973303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56258c1edc00 0x56258b481b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.949+0000 7fe1c2a00640 1 --2- 172.21.15.7:0/1980973303 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56258c1ed800 0x56258b481600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.949+0000 7fe1c3400640 1 -- 172.21.15.7:0/1980973303 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56258c1ed800 msgr2=0x56258b481600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.949+0000 7fe1c3400640 1 --2- 172.21.15.7:0/1980973303 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56258c1ed800 0x56258b481600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.949+0000 7fe1c3400640 1 -- 172.21.15.7:0/1980973303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56258c1edc00 msgr2=0x56258b481b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:03.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.949+0000 7fe1c3400640 1 --2- 172.21.15.7:0/1980973303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56258c1edc00 0x56258b481b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.949+0000 7fe1c3400640 1 -- 172.21.15.7:0/1980973303 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56258b331860 con 0x56258c1ed400 2024-09-10T21:49:03.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.949+0000 7fe1c2a00640 1 --2- 172.21.15.7:0/1980973303 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56258c1ed800 0x56258b481600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:03.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.949+0000 7fe1c3400640 1 --2- 172.21.15.7:0/1980973303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258c1ed400 0x56258b481080 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x56258c1e2ed0 tx=0x56258c204c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f44127b4f22ae600 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.949+0000 7fe1bd000640 1 -- 172.21.15.7:0/1980973303 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56258b32aea0 con 0x56258c1ed400 2024-09-10T21:49:03.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.949+0000 7fe1bd000640 1 -- 172.21.15.7:0/1980973303 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56258b35a540 con 0x56258c1ed400 2024-09-10T21:49:03.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.949+0000 7fe1bd000640 1 -- 172.21.15.7:0/1980973303 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56258b32b380 con 0x56258c1ed400 2024-09-10T21:49:03.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.950+0000 7fe1c5622740 1 -- 172.21.15.7:0/1980973303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258c1ed400 msgr2=0x56258b481080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.950+0000 7fe1c5622740 1 --2- 172.21.15.7:0/1980973303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258c1ed400 0x56258b481080 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x56258c1e2ed0 tx=0x56258c204c00 comp rx=0 tx=0).stop 2024-09-10T21:49:03.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.950+0000 7fe1c5622740 1 -- 172.21.15.7:0/1980973303 shutdown_connections 2024-09-10T21:49:03.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.950+0000 7fe1c5622740 1 --2- 172.21.15.7:0/1980973303 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56258c1edc00 0x56258b481b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.950+0000 7fe1c5622740 1 --2- 172.21.15.7:0/1980973303 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56258c1ed800 0x56258b481600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.950+0000 7fe1c5622740 1 --2- 172.21.15.7:0/1980973303 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258c1ed400 0x56258b481080 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.950+0000 7fe1c5622740 1 -- 172.21.15.7:0/1980973303 >> 172.21.15.7:0/1980973303 conn(0x56258c1ed000 msgr2=0x56258b3f7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:03.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.950+0000 7fe1c5622740 1 -- 172.21.15.7:0/1980973303 shutdown_connections 2024-09-10T21:49:03.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.950+0000 7fe1c5622740 1 -- 172.21.15.7:0/1980973303 wait complete. 2024-09-10T21:49:03.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.951+0000 7fe1c5622740 1 Processor -- start 2024-09-10T21:49:03.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.951+0000 7fe1c5622740 1 -- start start 2024-09-10T21:49:03.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.951+0000 7fe1c5622740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56258c1ed400 0x56258b481080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.951+0000 7fe1c5622740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56258c1ed800 0x56258b481600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.951+0000 7fe1c5622740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258c1edc00 0x56258b481b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.951+0000 7fe1c5622740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56258c1d9c80 con 0x56258c1edc00 2024-09-10T21:49:03.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.951+0000 7fe1c5622740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56258c1d9b00 con 0x56258c1ed400 2024-09-10T21:49:03.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.951+0000 7fe1c5622740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x56258c1d9980 con 0x56258c1ed800 2024-09-10T21:49:03.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.952+0000 7fe1c2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258c1edc00 0x56258b481b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.952+0000 7fe1c2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258c1edc00 0x56258b481b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52190/0 (socket says 172.21.15.7:52190) 2024-09-10T21:49:03.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.952+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 learned_addr learned my addr 172.21.15.7:0/3449413259 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:03.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.952+0000 7fe1c2a00640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56258c1ed800 0x56258b481600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.952+0000 7fe1c3400640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56258c1ed400 0x56258b481080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.952+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56258c1ed800 msgr2=0x56258b481600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.952+0000 7fe1c2000640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56258c1ed800 0x56258b481600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.952+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56258c1ed400 msgr2=0x56258b481080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:03.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.952+0000 7fe1c2000640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56258c1ed400 0x56258b481080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:03.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.952+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56258b331680 con 0x56258c1edc00 2024-09-10T21:49:03.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.952+0000 7fe1c2000640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258c1edc00 0x56258b481b80 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x56258c26e000 tx=0x56258c204f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.953+0000 7fe1bb200640 1 -- 172.21.15.7:0/3449413259 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56258b32a340 con 0x56258c1edc00 2024-09-10T21:49:03.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.953+0000 7fe1bb200640 1 -- 172.21.15.7:0/3449413259 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x56258b35ae00 con 0x56258c1edc00 2024-09-10T21:49:03.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.953+0000 7fe1bb200640 1 -- 172.21.15.7:0/3449413259 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56258b32aea0 con 0x56258c1edc00 2024-09-10T21:49:03.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.953+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56258b3314a0 con 0x56258c1edc00 2024-09-10T21:49:03.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.953+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56258b3312c0 con 0x56258c1edc00 2024-09-10T21:49:03.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.953+0000 7fe1c5622740 4 datascan.init: resolving metadata pool 12 2024-09-10T21:49:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.955+0000 7fe1bb200640 1 -- 172.21.15.7:0/3449413259 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x56258b35f500 con 0x56258c1edc00 2024-09-10T21:49:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.955+0000 7fe1bb200640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x56258c25ec00 0x56258b482100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.956+0000 7fe1bb200640 1 -- 172.21.15.7:0/3449413259 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x56258c264c00 con 0x56258c1edc00 2024-09-10T21:49:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.956+0000 7fe1c5622740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-09-10T21:49:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.956+0000 7fe1c5622740 4 datascan.main: resolving metadata pool 12 2024-09-10T21:49:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.956+0000 7fe1c5622740 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x56258c25f000 0x56258b482680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.956+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:1 12.0 12:00000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258c25f400 con 0x56258c25f000 2024-09-10T21:49:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.956+0000 7fe1c3400640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x56258c25ec00 0x56258b482100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.956+0000 7fe1c2a00640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x56258c25f000 0x56258b482680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.956+0000 7fe1c3400640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x56258c25ec00 0x56258b482100 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x56258c1e2ed0 tx=0x56258c204de0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.957+0000 7fe1c2a00640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x56258c25f000 0x56258b482680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.957+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.4 v2:172.21.15.133:6826/1857710445 1 ==== osd_op_reply(1 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e000 con 0x56258c25f000 2024-09-10T21:49:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.957+0000 7fe1c2a00640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x56258c25f400 0x56258b6bc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.957+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:2 12.10 12:08000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258c1ec800 con 0x56258c25f400 2024-09-10T21:49:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.957+0000 7fe1c3400640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x56258c25f400 0x56258b6bc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.958+0000 7fe1c3400640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x56258c25f400 0x56258b6bc000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.958+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 1 ==== osd_op_reply(2 [pgnls start_epoch 89 out=177b] v89'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.958+0000 7fe1c3400640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x56258c1ec800 0x56258b6bc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.958+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:3 12.8 12:10000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258c25e800 con 0x56258c1ec800 2024-09-10T21:49:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.958+0000 7fe1c2a00640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x56258c1ec800 0x56258b6bc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.959+0000 7fe1c2a00640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x56258c1ec800 0x56258b6bc580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.959+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.3 v2:172.21.15.7:6800/2195665006 1 ==== osd_op_reply(3 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e000 con 0x56258c1ec800 2024-09-10T21:49:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.959+0000 7fe1c2a00640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x56258c25e800 0x56258b6bcb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.959+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:4 12.18 12:18000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cc000 con 0x56258c25e800 2024-09-10T21:49:03.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.960+0000 7fe1c2000640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x56258c25e800 0x56258b6bcb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.960+0000 7fe1c2000640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x56258c25e800 0x56258b6bcb00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.960+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.1 v2:172.21.15.7:6824/4175601737 1 ==== osd_op_reply(4 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e780 con 0x56258c25e800 2024-09-10T21:49:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.960+0000 7fe1c2000640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x56258b6cc000 0x56258b6bd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.960+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:5 12.4 12:20000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258c0fb400 con 0x56258b6cc000 2024-09-10T21:49:03.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.961+0000 7fe1c3400640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x56258b6cc000 0x56258b6bd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.961+0000 7fe1c3400640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x56258b6cc000 0x56258b6bd600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.962+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 1 ==== osd_op_reply(5 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:03.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.962+0000 7fe1c3400640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x56258c0fb400 0x56258b6bdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.962+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:6 12.14 12:28000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cd000 con 0x56258c0fb400 2024-09-10T21:49:03.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.962+0000 7fe1c2a00640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x56258c0fb400 0x56258b6bdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.962+0000 7fe1c2a00640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x56258c0fb400 0x56258b6bdb80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.963+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.5 v2:172.21.15.133:6818/3838498382 1 ==== osd_op_reply(6 [pgnls start_epoch 89 out=149b] v88'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x56258b40e000 con 0x56258c0fb400 2024-09-10T21:49:03.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.963+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:7 12.c 12:30000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cd000 con 0x56258c25f400 2024-09-10T21:49:03.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.963+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 2 ==== osd_op_reply(7 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.963+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:8 12.1c 12:38000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cd000 con 0x56258c25f400 2024-09-10T21:49:03.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.963+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 3 ==== osd_op_reply(8 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.963+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:9 12.2 12:40000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cd000 con 0x56258c25f000 2024-09-10T21:49:03.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.964+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.4 v2:172.21.15.133:6826/1857710445 2 ==== osd_op_reply(9 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e000 con 0x56258c25f000 2024-09-10T21:49:03.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.964+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:10 12.12 12:48000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cd000 con 0x56258c1ec800 2024-09-10T21:49:03.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.964+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.3 v2:172.21.15.7:6800/2195665006 2 ==== osd_op_reply(10 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e000 con 0x56258c1ec800 2024-09-10T21:49:03.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.964+0000 7fe1c2a00640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x56258b6cd000 0x56258b6bd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.964+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:11 12.a 12:50000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cc400 con 0x56258b6cd000 2024-09-10T21:49:03.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.965+0000 7fe1c2000640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x56258b6cd000 0x56258b6bd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.965+0000 7fe1c2000640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x56258b6cd000 0x56258b6bd080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.966+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.2 v2:172.21.15.7:6816/4238877366 1 ==== osd_op_reply(11 [pgnls start_epoch 89 out=81b] v88'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x56258b40e780 con 0x56258b6cd000 2024-09-10T21:49:03.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.966+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:12 12.1a 12:58000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cc400 con 0x56258c25f000 2024-09-10T21:49:03.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.966+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.4 v2:172.21.15.133:6826/1857710445 3 ==== osd_op_reply(12 [pgnls start_epoch 89 out=49b] v88'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e000 con 0x56258c25f000 2024-09-10T21:49:03.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.966+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:13 12.6 12:60000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cc400 con 0x56258c25e800 2024-09-10T21:49:03.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.966+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.1 v2:172.21.15.7:6824/4175601737 2 ==== osd_op_reply(13 [pgnls start_epoch 89 out=77b] v88'7 uv7 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x56258b40e780 con 0x56258c25e800 2024-09-10T21:49:03.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.966+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:14 12.16 12:68000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cc400 con 0x56258b6cc000 2024-09-10T21:49:03.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.967+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 2 ==== osd_op_reply(14 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:03.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.967+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:15 12.e 12:70000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cc400 con 0x56258c25f400 2024-09-10T21:49:03.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.967+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 4 ==== osd_op_reply(15 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.967+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:16 12.1e 12:78000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cc400 con 0x56258c25f000 2024-09-10T21:49:03.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.967+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.4 v2:172.21.15.133:6826/1857710445 4 ==== osd_op_reply(16 [pgnls start_epoch 89 out=49b] v88'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e000 con 0x56258c25f000 2024-09-10T21:49:03.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.967+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:17 12.1 12:80000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cc400 con 0x56258b6cc000 2024-09-10T21:49:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.968+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 3 ==== osd_op_reply(17 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.968+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:18 12.11 12:88000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cc400 con 0x56258c0fb400 2024-09-10T21:49:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.968+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.5 v2:172.21.15.133:6818/3838498382 2 ==== osd_op_reply(18 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e000 con 0x56258c0fb400 2024-09-10T21:49:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.968+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:19 12.9 12:90000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cc400 con 0x56258b6cd000 2024-09-10T21:49:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.969+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.2 v2:172.21.15.7:6816/4238877366 2 ==== osd_op_reply(19 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e780 con 0x56258b6cd000 2024-09-10T21:49:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.969+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:20 12.19 12:98000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cc400 con 0x56258c25f400 2024-09-10T21:49:03.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.969+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 5 ==== osd_op_reply(20 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.969+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:21 12.5 12:a0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cc400 con 0x56258b6cc000 2024-09-10T21:49:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.969+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 4 ==== osd_op_reply(21 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.969+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:22 12.15 12:a8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cc400 con 0x56258b6cc000 2024-09-10T21:49:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.970+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 5 ==== osd_op_reply(22 [pgnls start_epoch 89 out=97b] v88'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.970+0000 7fe1c3400640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x56258b6cc400 0x56258b6be100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.970+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:23 12.d 12:b0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cd400 con 0x56258b6cc400 2024-09-10T21:49:03.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.970+0000 7fe1c3400640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x56258b6cc400 0x56258b6be100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:03.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.971+0000 7fe1c3400640 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x56258b6cc400 0x56258b6be100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:03.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.971+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.7 v2:172.21.15.133:6802/2638399040 1 ==== osd_op_reply(23 [pgnls start_epoch 89 out=74b] v84'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc400 2024-09-10T21:49:03.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.971+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:24 12.1d 12:b8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cd400 con 0x56258c25f400 2024-09-10T21:49:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.972+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 6 ==== osd_op_reply(24 [pgnls start_epoch 89 out=73b] v88'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.972+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:25 12.3 12:c0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cd400 con 0x56258c25f400 2024-09-10T21:49:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.972+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 7 ==== osd_op_reply(25 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.972+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:26 12.13 12:c8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cd400 con 0x56258c25e800 2024-09-10T21:49:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.972+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.1 v2:172.21.15.7:6824/4175601737 3 ==== osd_op_reply(26 [pgnls start_epoch 89 out=98b] v88'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x56258b40e780 con 0x56258c25e800 2024-09-10T21:49:03.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.972+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:27 12.b 12:d0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cd400 con 0x56258c1ec800 2024-09-10T21:49:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.973+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.3 v2:172.21.15.7:6800/2195665006 3 ==== osd_op_reply(27 [pgnls start_epoch 89 out=76b] v88'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x56258b40e000 con 0x56258c1ec800 2024-09-10T21:49:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.973+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:28 12.1b 12:d8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cd400 con 0x56258c25f000 2024-09-10T21:49:03.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.973+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.4 v2:172.21.15.133:6826/1857710445 5 ==== osd_op_reply(28 [pgnls start_epoch 89 out=73b] v88'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56258b40e000 con 0x56258c25f000 2024-09-10T21:49:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.973+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:29 12.7 12:e0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cd400 con 0x56258c25e800 2024-09-10T21:49:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.973+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.1 v2:172.21.15.7:6824/4175601737 4 ==== osd_op_reply(29 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56258b40e780 con 0x56258c25e800 2024-09-10T21:49:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.973+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:30 12.17 12:e8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cd400 con 0x56258b6cc000 2024-09-10T21:49:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.974+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 6 ==== osd_op_reply(30 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.974+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:31 12.f 12:f0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cd400 con 0x56258b6cc000 2024-09-10T21:49:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.974+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 7 ==== osd_op_reply(31 [pgnls start_epoch 89 out=73b] v84'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:03.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.974+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:32 12.1f 12:f8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cd400 con 0x56258b6cc000 2024-09-10T21:49:03.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.974+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 8 ==== osd_op_reply(32 [pgnls start_epoch 89 out=101b] v88'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:03.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.975+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-09-10T21:49:03.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.975+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:33 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258c25f800 con 0x56258c25f400 2024-09-10T21:49:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.975+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.975+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-09-10T21:49:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.975+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:34 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258c25fc00 con 0x56258c25f400 2024-09-10T21:49:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.975+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.976+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-09-10T21:49:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.976+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:35 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cdc00 con 0x56258c25f400 2024-09-10T21:49:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.976+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.976+0000 7fe1c5622740 20 datascan.scan_links: injected ino detected 2024-09-10T21:49:03.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.976+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-09-10T21:49:03.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.976+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:36 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cd800 con 0x56258c25f400 2024-09-10T21:49:03.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.976+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.976+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-09-10T21:49:03.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.976+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:37 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cd400 con 0x56258c25f400 2024-09-10T21:49:03.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.977+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.977+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-09-10T21:49:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.977+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-09-10T21:49:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.977+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-09-10T21:49:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.977+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-09-10T21:49:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.977+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-09-10T21:49:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.977+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-09-10T21:49:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.977+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-09-10T21:49:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.977+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:38 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cdc00 con 0x56258c0fb400 2024-09-10T21:49:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.977+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.5 v2:172.21.15.133:6818/3838498382 3 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e000 con 0x56258c0fb400 2024-09-10T21:49:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.977+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-09-10T21:49:03.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.978+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:39 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258c25f800 con 0x56258c25f400 2024-09-10T21:49:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.978+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.978+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 100000001f6.00000000 2024-09-10T21:49:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.978+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:40 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258c25fc00 con 0x56258b6cd000 2024-09-10T21:49:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.978+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.2 v2:172.21.15.7:6816/4238877366 3 ==== osd_op_reply(40 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x56258b40e780 con 0x56258b6cd000 2024-09-10T21:49:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.978+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-09-10T21:49:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.978+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-09-10T21:49:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.978+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-09-10T21:49:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.978+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:41 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6ea000 con 0x56258c25f400 2024-09-10T21:49:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.979+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.979+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-09-10T21:49:03.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.979+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:42 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6ea400 con 0x56258b6cc000 2024-09-10T21:49:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.979+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.980+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-09-10T21:49:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.980+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-09-10T21:49:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.980+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-09-10T21:49:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.980+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-09-10T21:49:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.980+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-09-10T21:49:03.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.980+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:43 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6ea800 con 0x56258c25f400 2024-09-10T21:49:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.980+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.980+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-09-10T21:49:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.980+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-09-10T21:49:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.980+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-09-10T21:49:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.980+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:44 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eac00 con 0x56258c25e800 2024-09-10T21:49:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.980+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.1 v2:172.21.15.7:6824/4175601737 5 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x56258b40e780 con 0x56258c25e800 2024-09-10T21:49:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.980+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-09-10T21:49:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.980+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-09-10T21:49:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.980+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 200.00000002 2024-09-10T21:49:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.980+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-09-10T21:49:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.980+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-09-10T21:49:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.980+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:45 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb000 con 0x56258c25e800 2024-09-10T21:49:03.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.981+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.1 v2:172.21.15.7:6824/4175601737 6 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e780 con 0x56258c25e800 2024-09-10T21:49:03.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.981+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-09-10T21:49:03.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.981+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-09-10T21:49:03.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.981+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-09-10T21:49:03.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.981+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-09-10T21:49:03.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.981+0000 7fe1c5622740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-09-10T21:49:03.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.981+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:46 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb400 con 0x56258b6cc000 2024-09-10T21:49:03.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.981+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 10 ==== osd_op_reply(46 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) ==== 154+0+1163 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:03.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.981+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:47 12.0 12:00000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c25f000 2024-09-10T21:49:03.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.982+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.4 v2:172.21.15.133:6826/1857710445 6 ==== osd_op_reply(47 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e000 con 0x56258c25f000 2024-09-10T21:49:03.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.982+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:48 12.10 12:08000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c25f400 2024-09-10T21:49:03.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.982+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 16 ==== osd_op_reply(48 [pgnls start_epoch 89 out=177b] v89'18 uv18 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.982+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:49 12.8 12:10000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c1ec800 2024-09-10T21:49:03.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.982+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.3 v2:172.21.15.7:6800/2195665006 4 ==== osd_op_reply(49 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e000 con 0x56258c1ec800 2024-09-10T21:49:03.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.982+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:50 12.18 12:18000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c25e800 2024-09-10T21:49:03.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.983+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.1 v2:172.21.15.7:6824/4175601737 7 ==== osd_op_reply(50 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e780 con 0x56258c25e800 2024-09-10T21:49:03.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.983+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:51 12.4 12:20000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258b6cc000 2024-09-10T21:49:03.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.983+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 11 ==== osd_op_reply(51 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:03.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.983+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:52 12.14 12:28000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c0fb400 2024-09-10T21:49:03.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.984+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.5 v2:172.21.15.133:6818/3838498382 4 ==== osd_op_reply(52 [pgnls start_epoch 89 out=149b] v88'28 uv28 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x56258b40e000 con 0x56258c0fb400 2024-09-10T21:49:03.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.984+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:53 12.c 12:30000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c25f400 2024-09-10T21:49:03.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.984+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 17 ==== osd_op_reply(53 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.984+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:54 12.1c 12:38000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c25f400 2024-09-10T21:49:03.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.984+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 18 ==== osd_op_reply(54 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.984+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:55 12.2 12:40000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c25f000 2024-09-10T21:49:03.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.985+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.4 v2:172.21.15.133:6826/1857710445 7 ==== osd_op_reply(55 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e000 con 0x56258c25f000 2024-09-10T21:49:03.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.985+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:56 12.12 12:48000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c1ec800 2024-09-10T21:49:03.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.985+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.3 v2:172.21.15.7:6800/2195665006 5 ==== osd_op_reply(56 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e000 con 0x56258c1ec800 2024-09-10T21:49:03.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.985+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:57 12.a 12:50000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258b6cd000 2024-09-10T21:49:03.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.985+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.2 v2:172.21.15.7:6816/4238877366 4 ==== osd_op_reply(57 [pgnls start_epoch 89 out=81b] v88'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x56258b40e780 con 0x56258b6cd000 2024-09-10T21:49:03.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.986+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:58 12.1a 12:58000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c25f000 2024-09-10T21:49:03.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.986+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.4 v2:172.21.15.133:6826/1857710445 8 ==== osd_op_reply(58 [pgnls start_epoch 89 out=49b] v88'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e000 con 0x56258c25f000 2024-09-10T21:49:03.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.986+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:59 12.6 12:60000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c25e800 2024-09-10T21:49:03.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.987+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.1 v2:172.21.15.7:6824/4175601737 8 ==== osd_op_reply(59 [pgnls start_epoch 89 out=77b] v88'7 uv7 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x56258b40e780 con 0x56258c25e800 2024-09-10T21:49:03.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.987+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:60 12.16 12:68000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258b6cc000 2024-09-10T21:49:03.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.987+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 12 ==== osd_op_reply(60 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:03.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.987+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:61 12.e 12:70000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c25f400 2024-09-10T21:49:03.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.987+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 19 ==== osd_op_reply(61 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.987+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:62 12.1e 12:78000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c25f000 2024-09-10T21:49:03.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.988+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.4 v2:172.21.15.133:6826/1857710445 9 ==== osd_op_reply(62 [pgnls start_epoch 89 out=49b] v88'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e000 con 0x56258c25f000 2024-09-10T21:49:03.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.988+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:63 12.1 12:80000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258b6cc000 2024-09-10T21:49:03.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.988+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 13 ==== osd_op_reply(63 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:03.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.988+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:64 12.11 12:88000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c0fb400 2024-09-10T21:49:03.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.989+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.5 v2:172.21.15.133:6818/3838498382 5 ==== osd_op_reply(64 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e000 con 0x56258c0fb400 2024-09-10T21:49:03.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.989+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:65 12.9 12:90000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258b6cd000 2024-09-10T21:49:03.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.989+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.2 v2:172.21.15.7:6816/4238877366 5 ==== osd_op_reply(65 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e780 con 0x56258b6cd000 2024-09-10T21:49:03.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.989+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:66 12.19 12:98000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c25f400 2024-09-10T21:49:03.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.989+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 20 ==== osd_op_reply(66 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.989+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:67 12.5 12:a0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258b6cc000 2024-09-10T21:49:03.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.990+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 14 ==== osd_op_reply(67 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:03.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.990+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:68 12.15 12:a8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258b6cc000 2024-09-10T21:49:03.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.991+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 15 ==== osd_op_reply(68 [pgnls start_epoch 89 out=97b] v88'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:03.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.991+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:69 12.d 12:b0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258b6cc400 2024-09-10T21:49:03.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.991+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.7 v2:172.21.15.133:6802/2638399040 2 ==== osd_op_reply(69 [pgnls start_epoch 89 out=74b] v84'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc400 2024-09-10T21:49:03.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.991+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:70 12.1d 12:b8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c25f400 2024-09-10T21:49:03.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.992+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 21 ==== osd_op_reply(70 [pgnls start_epoch 89 out=73b] v88'5 uv5 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.992+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:71 12.3 12:c0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c25f400 2024-09-10T21:49:03.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.992+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 22 ==== osd_op_reply(71 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.992+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:72 12.13 12:c8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c25e800 2024-09-10T21:49:03.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.992+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.1 v2:172.21.15.7:6824/4175601737 9 ==== osd_op_reply(72 [pgnls start_epoch 89 out=98b] v88'4 uv4 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x56258b40e780 con 0x56258c25e800 2024-09-10T21:49:03.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.992+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:73 12.b 12:d0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c1ec800 2024-09-10T21:49:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.993+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.3 v2:172.21.15.7:6800/2195665006 6 ==== osd_op_reply(73 [pgnls start_epoch 89 out=76b] v88'6 uv6 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x56258b40e000 con 0x56258c1ec800 2024-09-10T21:49:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.993+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:74 12.1b 12:d8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c25f000 2024-09-10T21:49:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.993+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.4 v2:172.21.15.133:6826/1857710445 10 ==== osd_op_reply(74 [pgnls start_epoch 89 out=73b] v88'10 uv10 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56258b40e000 con 0x56258c25f000 2024-09-10T21:49:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.993+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:75 12.7 12:e0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258c25e800 2024-09-10T21:49:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.994+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.1 v2:172.21.15.7:6824/4175601737 10 ==== osd_op_reply(75 [pgnls start_epoch 89 out=73b] v88'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56258b40e780 con 0x56258c25e800 2024-09-10T21:49:03.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.994+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:76 12.17 12:e8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258b6cc000 2024-09-10T21:49:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.994+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 16 ==== osd_op_reply(76 [pgnls start_epoch 89 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.994+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:77 12.f 12:f0000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258b6cc000 2024-09-10T21:49:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.994+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 17 ==== osd_op_reply(77 [pgnls start_epoch 89 out=73b] v84'1 uv1 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:03.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.994+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:78 12.1f 12:f8000000::::head [pgnls start_epoch 89 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258b6cc000 2024-09-10T21:49:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.995+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 18 ==== osd_op_reply(78 [pgnls start_epoch 89 out=101b] v88'9 uv9 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.995+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-09-10T21:49:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.995+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:79 12.10 12:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6ebc00 con 0x56258c25f400 2024-09-10T21:49:03.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.995+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 23 ==== osd_op_reply(79 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.995+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-09-10T21:49:03.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.995+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:80 12.10 12:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6ea000 con 0x56258c25f400 2024-09-10T21:49:03.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.996+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 24 ==== osd_op_reply(80 602.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.996+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-09-10T21:49:03.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.996+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:81 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6ea400 con 0x56258c25f400 2024-09-10T21:49:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.996+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 25 ==== osd_op_reply(81 10000000000.00000000 [omap-get-vals out=506b] v0'0 uv18 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.996+0000 7fe1c5622740 20 datascan.scan_links: injected ino detected 2024-09-10T21:49:03.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.996+0000 7fe1c5622740 -1 datascan.scan_links: Bad nlink on 0x10000000001 expected 2 has 1 2024-09-10T21:49:03.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.996+0000 7fe1c5622740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2024-09-10T21:49:03.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.996+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-09-10T21:49:03.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.996+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:82 12.10 12:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6ea800 con 0x56258c25f400 2024-09-10T21:49:03.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.996+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 26 ==== osd_op_reply(82 600.00000000 [omap-get-vals out=5b] v0'0 uv12 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.997+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-09-10T21:49:03.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.997+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:83 12.10 12:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258c25f800 con 0x56258c25f400 2024-09-10T21:49:03.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.997+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 27 ==== osd_op_reply(83 603.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:03.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.997+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-09-10T21:49:03.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.997+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-09-10T21:49:03.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.997+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-09-10T21:49:03.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.997+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-09-10T21:49:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.997+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-09-10T21:49:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.997+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-09-10T21:49:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.997+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-09-10T21:49:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.997+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:84 12.14 12:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cd800 con 0x56258c0fb400 2024-09-10T21:49:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.997+0000 7fe1c2a00640 1 -- 172.21.15.7:0/3449413259 <== osd.5 v2:172.21.15.133:6818/3838498382 6 ==== osd_op_reply(84 606.00000000 [omap-get-vals out=5b] v0'0 uv26 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e000 con 0x56258c0fb400 2024-09-10T21:49:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.997+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-09-10T21:49:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.997+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:85 12.1c 12:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6cd400 con 0x56258c25f400 2024-09-10T21:49:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.998+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 28 ==== osd_op_reply(85 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.998+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 100000001f6.00000000 2024-09-10T21:49:04.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.998+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:86 12.a 12:57fe7535:::100000001f6.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258c25f800 con 0x56258b6cd000 2024-09-10T21:49:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.998+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.2 v2:172.21.15.7:6816/4238877366 6 ==== osd_op_reply(86 100000001f6.00000000 [omap-get-vals out=54b] v0'0 uv3 ondisk = 0) ==== 164+0+54 (crc 0 0 0) 0x56258b40e780 con 0x56258b6cd000 2024-09-10T21:49:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.998+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-09-10T21:49:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.998+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-09-10T21:49:04.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.998+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-09-10T21:49:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.998+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:87 12.e 12:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb400 con 0x56258c25f400 2024-09-10T21:49:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.998+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 29 ==== osd_op_reply(87 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.998+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-09-10T21:49:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.998+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:88 12.15 12:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258b6cc000 2024-09-10T21:49:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.999+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 19 ==== osd_op_reply(88 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.999+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-09-10T21:49:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.999+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-09-10T21:49:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.999+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-09-10T21:49:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.999+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-09-10T21:49:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.999+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-09-10T21:49:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.999+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:89 12.1d 12:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6ebc00 con 0x56258c25f400 2024-09-10T21:49:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.999+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 30 ==== osd_op_reply(89 605.00000000 [omap-get-vals out=5b] v0'0 uv5 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.999+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-09-10T21:49:04.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.999+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-09-10T21:49:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:03.999+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-09-10T21:49:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.000+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:90 12.13 12:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6ea000 con 0x56258c25e800 2024-09-10T21:49:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.000+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.1 v2:172.21.15.7:6824/4175601737 11 ==== osd_op_reply(90 100.00000000 [omap-get-vals out=5195b] v0'0 uv4 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x56258b40e780 con 0x56258c25e800 2024-09-10T21:49:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.000+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-09-10T21:49:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.000+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-09-10T21:49:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.000+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 200.00000002 2024-09-10T21:49:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.000+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000002' 2024-09-10T21:49:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.000+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-09-10T21:49:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.000+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:91 12.7 12:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6ea400 con 0x56258c25e800 2024-09-10T21:49:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.000+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.1 v2:172.21.15.7:6824/4175601737 12 ==== osd_op_reply(91 609.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x56258b40e780 con 0x56258c25e800 2024-09-10T21:49:04.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.000+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-09-10T21:49:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.000+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-09-10T21:49:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.000+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-09-10T21:49:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.000+0000 7fe1c5622740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-09-10T21:49:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.000+0000 7fe1c5622740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-09-10T21:49:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.000+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:92 12.1f 12:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6ea800 con 0x56258b6cc000 2024-09-10T21:49:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.001+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.6 v2:172.21.15.133:6810/3365981160 20 ==== osd_op_reply(92 1.00000000 [omap-get-vals out=1163b] v0'0 uv9 ondisk = 0) ==== 154+0+1163 (crc 0 0 0) 0x56258b40e500 con 0x56258b6cc000 2024-09-10T21:49:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.001+0000 7fe1c5622740 10 datascan.scan_links: processing 0 dup_primaries, 1 remote_links 2024-09-10T21:49:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.001+0000 7fe1c5622740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-09-10T21:49:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.001+0000 7fe1c5622740 10 datascan.scan_links: processing 1 bad_nlink_inos 2024-09-10T21:49:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.001+0000 7fe1c5622740 10 datascan.scan_links: handling bad_nlink_ino 0x10000000001 2024-09-10T21:49:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.001+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:93 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eac00 con 0x56258c25f400 2024-09-10T21:49:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.001+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 31 ==== osd_op_reply(93 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv18 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.001+0000 7fe1c5622740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-09-10T21:49:04.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.001+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:94 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb000 con 0x56258c25f400 2024-09-10T21:49:04.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.003+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 32 ==== osd_op_reply(94 10000000000.00000000 [omap-set-vals] v89'19 uv19 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.003+0000 7fe1c5622740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-09-10T21:49:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.003+0000 7fe1c5622740 10 datascan.scan_links: processing 1 injected_inos 2024-09-10T21:49:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.003+0000 7fe1c5622740 10 datascan.scan_links: handling injected_ino 0x10000000001 2024-09-10T21:49:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.003+0000 7fe1c5622740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:file 2024-09-10T21:49:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.003+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:95 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=17b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6ea000 con 0x56258c25f400 2024-09-10T21:49:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.003+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 33 ==== osd_op_reply(95 10000000000.00000000 [omap-get-vals-by-keys out=505b] v0'0 uv19 ondisk = 0) ==== 164+0+505 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.003+0000 7fe1c5622740 20 datascan.read_dentry: oid=10000000000.00000000 dname=file frag=*, r=0 2024-09-10T21:49:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.003+0000 7fe1c5622740 20 datascan.scan_links: first is now 2 2024-09-10T21:49:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.003+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:96 12.10 12:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=505b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x56258c25fc00 con 0x56258c25f400 2024-09-10T21:49:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.004+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.0 v2:172.21.15.7:6808/651402908 34 ==== osd_op_reply(96 10000000000.00000000 [omap-set-vals] v89'20 uv20 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x56258b40e500 con 0x56258c25f400 2024-09-10T21:49:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.004+0000 7fe1c5622740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/file pointing to 0x0x10000000001 2024-09-10T21:49:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.004+0000 7fe1c5622740 10 datascan.scan_links: updating inotable 2024-09-10T21:49:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.004+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:97 12.13 12:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258c25f800 con 0x56258c25e800 2024-09-10T21:49:04.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.005+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.1 v2:172.21.15.7:6824/4175601737 13 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv3 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x56258b40e780 con 0x56258c25e800 2024-09-10T21:49:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.005+0000 7fe1c5622740 10 mds.0.inotable: skip_inos was [0x10000000002~0x1f4,0x100000001f7~0xfffffffe09] 2024-09-10T21:49:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.005+0000 7fe1c5622740 10 mds.0.inotable: skip_inos now [0x100000001f7~0xfffffffe09] 2024-09-10T21:49:04.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.005+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:98 12.13 12:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb400 con 0x56258c25e800 2024-09-10T21:49:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.006+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 <== osd.1 v2:172.21.15.7:6824/4175601737 14 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v89'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x56258b40e780 con 0x56258c25e800 2024-09-10T21:49:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.006+0000 7fe1c5622740 10 datascan.scan_links: updating snaptable 2024-09-10T21:49:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.006+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:99 12.d 12:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e89) -- 0x56258b6eb800 con 0x56258b6cc400 2024-09-10T21:49:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c3400640 1 -- 172.21.15.7:0/3449413259 <== osd.7 v2:172.21.15.133:6802/2638399040 3 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x56258b40e280 con 0x56258b6cc400 2024-09-10T21:49:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x56258c25f400 msgr2=0x56258b6bc000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:04.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x56258c25f400 0x56258b6bc000 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x56258c25e800 msgr2=0x56258b6bcb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x56258c25e800 0x56258b6bcb00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x56258b6cd000 msgr2=0x56258b6bd080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x56258b6cd000 0x56258b6bd080 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x56258c1ec800 msgr2=0x56258b6bc580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x56258c1ec800 0x56258b6bc580 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x56258c25f000 msgr2=0x56258b482680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x56258c25f000 0x56258b482680 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 reap_dead start 2024-09-10T21:49:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x56258c0fb400 msgr2=0x56258b6bdb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x56258c0fb400 0x56258b6bdb80 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x56258b6cc000 msgr2=0x56258b6bd600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:04.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x56258b6cc000 0x56258b6bd600 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x56258b6cc400 msgr2=0x56258b6be100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x56258b6cc400 0x56258b6be100 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c2000640 1 -- 172.21.15.7:0/3449413259 reap_dead start 2024-09-10T21:49:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x56258c25ec00 msgr2=0x56258b482100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x56258c25ec00 0x56258b482100 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x56258c1e2ed0 tx=0x56258c204de0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258c1edc00 msgr2=0x56258b481b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258c1edc00 0x56258b481b80 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x56258c26e000 tx=0x56258c204f60 comp rx=0 tx=0).stop 2024-09-10T21:49:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 shutdown_connections 2024-09-10T21:49:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x56258c25ec00 0x56258b482100 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 --2- 172.21.15.7:0/3449413259 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258c1edc00 0x56258b481b80 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 >> 172.21.15.7:0/3449413259 conn(0x56258c1ed000 msgr2=0x56258b3f7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 shutdown_connections 2024-09-10T21:49:04.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.007+0000 7fe1c5622740 1 -- 172.21.15.7:0/3449413259 wait complete. 2024-09-10T21:49:04.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.008+0000 7fe1c5622740 1 -- 172.21.15.7:0/3536387517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258b304c00 msgr2=0x56258b480000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:04.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.008+0000 7fe1c5622740 1 --2- 172.21.15.7:0/3536387517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258b304c00 0x56258b480000 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x56258c1e2060 tx=0x56258c0b5260 comp rx=0 tx=0).stop 2024-09-10T21:49:04.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.008+0000 7fe1c5622740 1 -- 172.21.15.7:0/3536387517 shutdown_connections 2024-09-10T21:49:04.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.008+0000 7fe1c5622740 1 --2- 172.21.15.7:0/3536387517 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56258b305c00 0x56258b480b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.008+0000 7fe1c5622740 1 --2- 172.21.15.7:0/3536387517 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56258b305800 0x56258b480580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.008+0000 7fe1c5622740 1 --2- 172.21.15.7:0/3536387517 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56258b304c00 0x56258b480000 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.008+0000 7fe1c5622740 1 -- 172.21.15.7:0/3536387517 >> 172.21.15.7:0/3536387517 conn(0x56258b305000 msgr2=0x56258b3f6800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:04.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.008+0000 7fe1c5622740 1 -- 172.21.15.7:0/3536387517 shutdown_connections 2024-09-10T21:49:04.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.008+0000 7fe1c5622740 1 -- 172.21.15.7:0/3536387517 wait complete. 2024-09-10T21:49:04.015 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.141219, result: 2024-09-10T21:49:04.015 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:49:04.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.177+0000 7f8c8da00640 1 -- 172.21.15.7:0/4233571004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c8815ba00 msgr2=0x7f8c880ad8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:04.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.177+0000 7f8c8da00640 1 --2- 172.21.15.7:0/4233571004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c8815ba00 0x7f8c880ad8f0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f8c78005fd0 tx=0x7f8c7802a540 comp rx=0 tx=0).stop 2024-09-10T21:49:04.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.177+0000 7f8c8da00640 1 -- 172.21.15.7:0/4233571004 shutdown_connections 2024-09-10T21:49:04.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.177+0000 7f8c8da00640 1 --2- 172.21.15.7:0/4233571004 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c880ccc40 0x7f8c8815e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.177+0000 7f8c8da00640 1 --2- 172.21.15.7:0/4233571004 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c8815ba00 0x7f8c880ad8f0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.177+0000 7f8c8da00640 1 -- 172.21.15.7:0/4233571004 >> 172.21.15.7:0/4233571004 conn(0x7f8c880b16d0 msgr2=0x7f8c880b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:04.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.177+0000 7f8c8da00640 1 -- 172.21.15.7:0/4233571004 shutdown_connections 2024-09-10T21:49:04.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.177+0000 7f8c8da00640 1 -- 172.21.15.7:0/4233571004 wait complete. 2024-09-10T21:49:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.179+0000 7f8c8da00640 1 Processor -- start 2024-09-10T21:49:04.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.179+0000 7f8c8da00640 1 -- start start 2024-09-10T21:49:04.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.180+0000 7f8c8da00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c880ccc40 0x7f8c880ca250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:04.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.180+0000 7f8c8da00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c8815ba00 0x7f8c880b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:04.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.180+0000 7f8c8da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c881f4ac0 0x7f8c88159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:04.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.180+0000 7f8c8da00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8c880af280 con 0x7f8c881f4ac0 2024-09-10T21:49:04.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.180+0000 7f8c8da00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8c880af100 con 0x7f8c8815ba00 2024-09-10T21:49:04.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.180+0000 7f8c8da00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f8c880af400 con 0x7f8c880ccc40 2024-09-10T21:49:04.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.180+0000 7f8c87400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c881f4ac0 0x7f8c88159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:04.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.180+0000 7f8c87400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c881f4ac0 0x7f8c88159c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52210/0 (socket says 172.21.15.7:52210) 2024-09-10T21:49:04.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.180+0000 7f8c87400640 1 -- 172.21.15.7:0/908997937 learned_addr learned my addr 172.21.15.7:0/908997937 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:04.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.180+0000 7f8c7e000640 1 --2- 172.21.15.7:0/908997937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c8815ba00 0x7f8c880b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:04.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.180+0000 7f8c86a00640 1 --2- 172.21.15.7:0/908997937 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c880ccc40 0x7f8c880ca250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:04.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.181+0000 7f8c87400640 1 -- 172.21.15.7:0/908997937 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c880ccc40 msgr2=0x7f8c880ca250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:04.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.181+0000 7f8c87400640 1 --2- 172.21.15.7:0/908997937 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c880ccc40 0x7f8c880ca250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.181+0000 7f8c87400640 1 -- 172.21.15.7:0/908997937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c8815ba00 msgr2=0x7f8c880b0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:04.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.181+0000 7f8c87400640 1 --2- 172.21.15.7:0/908997937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c8815ba00 0x7f8c880b0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.181+0000 7f8c87400640 1 -- 172.21.15.7:0/908997937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c880cab10 con 0x7f8c881f4ac0 2024-09-10T21:49:04.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.181+0000 7f8c86a00640 1 --2- 172.21.15.7:0/908997937 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c880ccc40 0x7f8c880ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:04.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.181+0000 7f8c7e000640 1 --2- 172.21.15.7:0/908997937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c8815ba00 0x7f8c880b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:04.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.181+0000 7f8c87400640 1 --2- 172.21.15.7:0/908997937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c881f4ac0 0x7f8c88159c50 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000e9d0 tx=0x7f8c8000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.181+0000 7f8c7f400640 1 -- 172.21.15.7:0/908997937 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8000cd80 con 0x7f8c881f4ac0 2024-09-10T21:49:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.181+0000 7f8c7f400640 1 -- 172.21.15.7:0/908997937 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c80004c30 con 0x7f8c881f4ac0 2024-09-10T21:49:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.181+0000 7f8c7f400640 1 -- 172.21.15.7:0/908997937 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80005280 con 0x7f8c881f4ac0 2024-09-10T21:49:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.181+0000 7f8c8da00640 1 -- 172.21.15.7:0/908997937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c880cadf0 con 0x7f8c881f4ac0 2024-09-10T21:49:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.181+0000 7f8c8da00640 1 -- 172.21.15.7:0/908997937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c880cb3b0 con 0x7f8c881f4ac0 2024-09-10T21:49:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.182+0000 7f8c8da00640 1 -- 172.21.15.7:0/908997937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c880cb5f0 con 0x7f8c881f4ac0 2024-09-10T21:49:04.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.184+0000 7f8c7f400640 1 -- 172.21.15.7:0/908997937 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8c8000f040 con 0x7f8c881f4ac0 2024-09-10T21:49:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.184+0000 7f8c7f400640 1 --2- 172.21.15.7:0/908997937 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8c64077f70 0x7f8c6407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:04.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.185+0000 7f8c7f400640 1 -- 172.21.15.7:0/908997937 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f8c8009a520 con 0x7f8c881f4ac0 2024-09-10T21:49:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.185+0000 7f8c86a00640 1 --2- 172.21.15.7:0/908997937 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8c64077f70 0x7f8c6407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:04.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.185+0000 7f8c86a00640 1 --2- 172.21.15.7:0/908997937 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8c64077f70 0x7f8c6407a430 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8c78033000 tx=0x7f8c78004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:04.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.189+0000 7f8c7f400640 1 -- 172.21.15.7:0/908997937 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8c880cb5f0 con 0x7f8c881f4ac0 2024-09-10T21:49:04.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.423+0000 7f8c8da00640 1 -- 172.21.15.7:0/908997937 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f8c8815e8d0 con 0x7f8c881f4ac0 2024-09-10T21:49:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.571+0000 7f8c7f400640 1 -- 172.21.15.7:0/908997937 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v72) ==== 162+0+0 (secure 0 0 0) 0x7f8c80016030 con 0x7f8c881f4ac0 2024-09-10T21:49:04.572 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:49:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.573+0000 7f8c8da00640 1 -- 172.21.15.7:0/908997937 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8c64077f70 msgr2=0x7f8c6407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.573+0000 7f8c8da00640 1 --2- 172.21.15.7:0/908997937 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8c64077f70 0x7f8c6407a430 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8c78033000 tx=0x7f8c78004730 comp rx=0 tx=0).stop 2024-09-10T21:49:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.573+0000 7f8c8da00640 1 -- 172.21.15.7:0/908997937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c881f4ac0 msgr2=0x7f8c88159c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.573+0000 7f8c8da00640 1 --2- 172.21.15.7:0/908997937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c881f4ac0 0x7f8c88159c50 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000e9d0 tx=0x7f8c8000ee90 comp rx=0 tx=0).stop 2024-09-10T21:49:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.574+0000 7f8c8da00640 1 -- 172.21.15.7:0/908997937 shutdown_connections 2024-09-10T21:49:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.574+0000 7f8c8da00640 1 --2- 172.21.15.7:0/908997937 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8c64077f70 0x7f8c6407a430 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.574+0000 7f8c8da00640 1 --2- 172.21.15.7:0/908997937 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8c8815ba00 0x7f8c880b0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.574+0000 7f8c8da00640 1 --2- 172.21.15.7:0/908997937 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8c881f4ac0 0x7f8c88159c50 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.574+0000 7f8c8da00640 1 --2- 172.21.15.7:0/908997937 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8c880ccc40 0x7f8c880ca250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.574+0000 7f8c8da00640 1 -- 172.21.15.7:0/908997937 >> 172.21.15.7:0/908997937 conn(0x7f8c880b16d0 msgr2=0x7f8c8806fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:04.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.574+0000 7f8c8da00640 1 -- 172.21.15.7:0/908997937 shutdown_connections 2024-09-10T21:49:04.576 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.574+0000 7f8c8da00640 1 -- 172.21.15.7:0/908997937 wait complete. 2024-09-10T21:49:04.591 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:49:04.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.751+0000 7fa492000640 1 -- 172.21.15.7:0/2571919026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c0ae020 msgr2=0x7fa48c0ae4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:04.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.751+0000 7fa492000640 1 --2- 172.21.15.7:0/2571919026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c0ae020 0x7fa48c0ae4d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fa484005fd0 tx=0x7fa48402aaa0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.751+0000 7fa492000640 1 -- 172.21.15.7:0/2571919026 shutdown_connections 2024-09-10T21:49:04.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.751+0000 7fa492000640 1 --2- 172.21.15.7:0/2571919026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c0ae020 0x7fa48c0ae4d0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.751+0000 7fa492000640 1 --2- 172.21.15.7:0/2571919026 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa48c0cbd50 0x7fa48c0cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.751+0000 7fa492000640 1 -- 172.21.15.7:0/2571919026 >> 172.21.15.7:0/2571919026 conn(0x7fa48c0b16d0 msgr2=0x7fa48c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:04.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.751+0000 7fa492000640 1 -- 172.21.15.7:0/2571919026 shutdown_connections 2024-09-10T21:49:04.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.751+0000 7fa492000640 1 -- 172.21.15.7:0/2571919026 wait complete. 2024-09-10T21:49:04.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.753+0000 7fa492000640 1 Processor -- start 2024-09-10T21:49:04.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.753+0000 7fa492000640 1 -- start start 2024-09-10T21:49:04.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.754+0000 7fa492000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa48c0ae020 0x7fa48c155c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:04.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.754+0000 7fa492000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c0cbd50 0x7fa48c0c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:04.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.754+0000 7fa492000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa48c1619e0 0x7fa48c0c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:04.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.754+0000 7fa492000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c161dc0 con 0x7fa48c0cbd50 2024-09-10T21:49:04.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.754+0000 7fa492000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c0cdfd0 con 0x7fa48c1619e0 2024-09-10T21:49:04.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.754+0000 7fa492000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa48c161f40 con 0x7fa48c0ae020 2024-09-10T21:49:04.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.754+0000 7fa482a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c0cbd50 0x7fa48c0c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:04.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.754+0000 7fa482a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c0cbd50 0x7fa48c0c8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52230/0 (socket says 172.21.15.7:52230) 2024-09-10T21:49:04.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.754+0000 7fa482a00640 1 -- 172.21.15.7:0/1646334217 learned_addr learned my addr 172.21.15.7:0/1646334217 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:04.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.754+0000 7fa48b400640 1 --2- 172.21.15.7:0/1646334217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa48c0ae020 0x7fa48c155c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:04.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.755+0000 7fa48be00640 1 --2- 172.21.15.7:0/1646334217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa48c1619e0 0x7fa48c0c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:04.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.755+0000 7fa482a00640 1 -- 172.21.15.7:0/1646334217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa48c0ae020 msgr2=0x7fa48c155c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:04.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.755+0000 7fa482a00640 1 --2- 172.21.15.7:0/1646334217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa48c0ae020 0x7fa48c155c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.755+0000 7fa482a00640 1 -- 172.21.15.7:0/1646334217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa48c1619e0 msgr2=0x7fa48c0c79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:04.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.755+0000 7fa482a00640 1 --2- 172.21.15.7:0/1646334217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa48c1619e0 0x7fa48c0c79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:04.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.755+0000 7fa482a00640 1 -- 172.21.15.7:0/1646334217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c0c6c50 con 0x7fa48c0cbd50 2024-09-10T21:49:04.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.755+0000 7fa48be00640 1 --2- 172.21.15.7:0/1646334217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa48c1619e0 0x7fa48c0c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:49:04.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.755+0000 7fa482a00640 1 --2- 172.21.15.7:0/1646334217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c0cbd50 0x7fa48c0c8fb0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fa47400d770 tx=0x7fa47400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:04.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.755+0000 7fa483e00640 1 -- 172.21.15.7:0/1646334217 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474017070 con 0x7fa48c0cbd50 2024-09-10T21:49:04.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.755+0000 7fa483e00640 1 -- 172.21.15.7:0/1646334217 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa474009d20 con 0x7fa48c0cbd50 2024-09-10T21:49:04.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.755+0000 7fa483e00640 1 -- 172.21.15.7:0/1646334217 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474004280 con 0x7fa48c0cbd50 2024-09-10T21:49:04.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.755+0000 7fa492000640 1 -- 172.21.15.7:0/1646334217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48c0c9920 con 0x7fa48c0cbd50 2024-09-10T21:49:04.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.755+0000 7fa492000640 1 -- 172.21.15.7:0/1646334217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48c0c71d0 con 0x7fa48c0cbd50 2024-09-10T21:49:04.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.756+0000 7fa492000640 1 -- 172.21.15.7:0/1646334217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa48c0ae4d0 con 0x7fa48c0cbd50 2024-09-10T21:49:04.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.758+0000 7fa483e00640 1 -- 172.21.15.7:0/1646334217 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa4740035f0 con 0x7fa48c0cbd50 2024-09-10T21:49:04.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.758+0000 7fa483e00640 1 --2- 172.21.15.7:0/1646334217 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa468077f70 0x7fa46807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:04.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.759+0000 7fa483e00640 1 -- 172.21.15.7:0/1646334217 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7fa474099fe0 con 0x7fa48c0cbd50 2024-09-10T21:49:04.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.759+0000 7fa48b400640 1 --2- 172.21.15.7:0/1646334217 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa468077f70 0x7fa46807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:04.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.759+0000 7fa48b400640 1 --2- 172.21.15.7:0/1646334217 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa468077f70 0x7fa46807a430 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa47c001680 tx=0x7fa47c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:04.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.763+0000 7fa483e00640 1 -- 172.21.15.7:0/1646334217 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fa474062a10 con 0x7fa48c0cbd50 2024-09-10T21:49:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.996+0000 7fa492000640 1 -- 172.21.15.7:0/1646334217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa48c070800 con 0x7fa48c0cbd50 2024-09-10T21:49:04.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.996+0000 7fa483e00640 1 -- 172.21.15.7:0/1646334217 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+5218 (secure 0 0 0) 0x7fa474062310 con 0x7fa48c0cbd50 2024-09-10T21:49:04.998 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:04.998 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":73,"btime":"2024-09-10T21:49:04:574002+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/978763848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":978763848},{"type":"v1","addr":"172.21.15.7:6835","nonce":978763848}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/269527918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":269527918},{"type":"v1","addr":"172.21.15.7:6833","nonce":269527918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":235775,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/760980985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":760980985},{"type":"v1","addr":"172.21.15.133:6836","nonce":760980985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":73,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:47:40.664091+0000","modified":"2024-09-10T21:49:04.573984+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236099":{"gid":236099,"name":"d","rank":0,"incarnation":73,"state":"up:replay","state_seq":1,"addr":"172.21.15.133:6837/4188280820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4188280820},{"type":"v1","addr":"172.21.15.133:6837","nonce":4188280820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:49:04.998 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 73 2024-09-10T21:49:05.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.999+0000 7fa492000640 1 -- 172.21.15.7:0/1646334217 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa468077f70 msgr2=0x7fa46807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:05.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.999+0000 7fa492000640 1 --2- 172.21.15.7:0/1646334217 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa468077f70 0x7fa46807a430 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa47c001680 tx=0x7fa47c009040 comp rx=0 tx=0).stop 2024-09-10T21:49:05.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.999+0000 7fa492000640 1 -- 172.21.15.7:0/1646334217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c0cbd50 msgr2=0x7fa48c0c8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.999+0000 7fa492000640 1 --2- 172.21.15.7:0/1646334217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c0cbd50 0x7fa48c0c8fb0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fa47400d770 tx=0x7fa47400dc30 comp rx=0 tx=0).stop 2024-09-10T21:49:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.999+0000 7fa492000640 1 -- 172.21.15.7:0/1646334217 shutdown_connections 2024-09-10T21:49:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.999+0000 7fa492000640 1 --2- 172.21.15.7:0/1646334217 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa468077f70 0x7fa46807a430 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.999+0000 7fa492000640 1 --2- 172.21.15.7:0/1646334217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa48c1619e0 0x7fa48c0c79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.999+0000 7fa492000640 1 --2- 172.21.15.7:0/1646334217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa48c0cbd50 0x7fa48c0c8fb0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.999+0000 7fa492000640 1 --2- 172.21.15.7:0/1646334217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa48c0ae020 0x7fa48c155c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.999+0000 7fa492000640 1 -- 172.21.15.7:0/1646334217 >> 172.21.15.7:0/1646334217 conn(0x7fa48c0b16d0 msgr2=0x7fa48c157f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.999+0000 7fa492000640 1 -- 172.21.15.7:0/1646334217 shutdown_connections 2024-09-10T21:49:05.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:04.999+0000 7fa492000640 1 -- 172.21.15.7:0/1646334217 wait complete. 2024-09-10T21:49:05.418 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 73, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:47:40.664091+0000', 'modified': '2024-09-10T21:49:04.573984+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 89, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 236099}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_236099': {'gid': 236099, 'name': 'd', 'rank': 0, 'incarnation': 73, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.133:6837/4188280820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6835', 'nonce': 4188280820}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 4188280820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:49:05.418 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_236099:up:replay 2024-09-10T21:49:06.420 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:49:06.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.592+0000 7f74c1a00640 1 -- 172.21.15.7:0/2822745276 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74bc15bc90 msgr2=0x7f74bc0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:06.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.592+0000 7f74c1a00640 1 --2- 172.21.15.7:0/2822745276 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74bc15bc90 0x7f74bc0ccef0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009860 tx=0x7f74ac02a480 comp rx=0 tx=0).stop 2024-09-10T21:49:06.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.592+0000 7f74c1a00640 1 -- 172.21.15.7:0/2822745276 shutdown_connections 2024-09-10T21:49:06.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.592+0000 7f74c1a00640 1 --2- 172.21.15.7:0/2822745276 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74bc15bc90 0x7f74bc0ccef0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:06.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.592+0000 7f74c1a00640 1 -- 172.21.15.7:0/2822745276 >> 172.21.15.7:0/2822745276 conn(0x7f74bc0b16d0 msgr2=0x7f74bc0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.592+0000 7f74c1a00640 1 -- 172.21.15.7:0/2822745276 shutdown_connections 2024-09-10T21:49:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.592+0000 7f74c1a00640 1 -- 172.21.15.7:0/2822745276 wait complete. 2024-09-10T21:49:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.594+0000 7f74c1a00640 1 Processor -- start 2024-09-10T21:49:06.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.594+0000 7f74c1a00640 1 -- start start 2024-09-10T21:49:06.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.595+0000 7f74c1a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc0cd750 0x7f74bc0b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:06.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.595+0000 7f74c1a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74bc15bc90 0x7f74bc1f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:06.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.595+0000 7f74c1a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74bc15fda0 0x7f74bc14c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:06.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.595+0000 7f74c1a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc070610 con 0x7f74bc0cd750 2024-09-10T21:49:06.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.595+0000 7f74c1a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f74bc0cf570 con 0x7f74bc15bc90 2024-09-10T21:49:06.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.595+0000 7f74c1a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f74bc14d640 con 0x7f74bc15fda0 2024-09-10T21:49:06.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.595+0000 7f74baa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc0cd750 0x7f74bc0b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.595+0000 7f74baa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc0cd750 0x7f74bc0b0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52240/0 (socket says 172.21.15.7:52240) 2024-09-10T21:49:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.595+0000 7f74baa00640 1 -- 172.21.15.7:0/3129422136 learned_addr learned my addr 172.21.15.7:0/3129422136 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.595+0000 7f74bb400640 1 --2- 172.21.15.7:0/3129422136 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74bc15fda0 0x7f74bc14c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.595+0000 7f74b3e00640 1 --2- 172.21.15.7:0/3129422136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74bc15bc90 0x7f74bc1f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.595+0000 7f74baa00640 1 -- 172.21.15.7:0/3129422136 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74bc15fda0 msgr2=0x7f74bc14c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.595+0000 7f74baa00640 1 --2- 172.21.15.7:0/3129422136 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74bc15fda0 0x7f74bc14c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.595+0000 7f74baa00640 1 -- 172.21.15.7:0/3129422136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74bc15bc90 msgr2=0x7f74bc1f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.595+0000 7f74baa00640 1 --2- 172.21.15.7:0/3129422136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74bc15bc90 0x7f74bc1f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.595+0000 7f74baa00640 1 -- 172.21.15.7:0/3129422136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74bc1f5740 con 0x7f74bc0cd750 2024-09-10T21:49:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.596+0000 7f74baa00640 1 --2- 172.21.15.7:0/3129422136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc0cd750 0x7f74bc0b0100 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009980 tx=0x7f74ac032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.596+0000 7f74b2a00640 1 -- 172.21.15.7:0/3129422136 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac040070 con 0x7f74bc0cd750 2024-09-10T21:49:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.596+0000 7f74b2a00640 1 -- 172.21.15.7:0/3129422136 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74ac005310 con 0x7f74bc0cd750 2024-09-10T21:49:06.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.596+0000 7f74c1a00640 1 -- 172.21.15.7:0/3129422136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74bc1f5a20 con 0x7f74bc0cd750 2024-09-10T21:49:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.596+0000 7f74b2a00640 1 -- 172.21.15.7:0/3129422136 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac0047a0 con 0x7f74bc0cd750 2024-09-10T21:49:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.596+0000 7f74c1a00640 1 -- 172.21.15.7:0/3129422136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74bc0c6f50 con 0x7f74bc0cd750 2024-09-10T21:49:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.597+0000 7f74c1a00640 1 -- 172.21.15.7:0/3129422136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74bc06f5b0 con 0x7f74bc0cd750 2024-09-10T21:49:06.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.598+0000 7f74b2a00640 1 -- 172.21.15.7:0/3129422136 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f74ac04c030 con 0x7f74bc0cd750 2024-09-10T21:49:06.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.599+0000 7f74b2a00640 1 --2- 172.21.15.7:0/3129422136 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7498077f70 0x7f749807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:06.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.599+0000 7f74b2a00640 1 -- 172.21.15.7:0/3129422136 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f74ac0bad50 con 0x7f74bc0cd750 2024-09-10T21:49:06.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.600+0000 7f74b3e00640 1 --2- 172.21.15.7:0/3129422136 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7498077f70 0x7f749807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:06.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.600+0000 7f74b3e00640 1 --2- 172.21.15.7:0/3129422136 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7498077f70 0x7f749807a430 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f74a4007fd0 tx=0x7f74a4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:06.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.603+0000 7f74b2a00640 1 -- 172.21.15.7:0/3129422136 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f74bc06f5b0 con 0x7f74bc0cd750 2024-09-10T21:49:06.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.841+0000 7f74c1a00640 1 -- 172.21.15.7:0/3129422136 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f74bc160560 con 0x7f74bc0cd750 2024-09-10T21:49:06.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.842+0000 7f74b2a00640 1 -- 172.21.15.7:0/3129422136 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) ==== 94+0+5218 (secure 0 0 0) 0x7f74ac083120 con 0x7f74bc0cd750 2024-09-10T21:49:06.843 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:06.843 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":75,"btime":"2024-09-10T21:49:06:581674+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/978763848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":978763848},{"type":"v1","addr":"172.21.15.7:6835","nonce":978763848}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/269527918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":269527918},{"type":"v1","addr":"172.21.15.7:6833","nonce":269527918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":235775,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/760980985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":760980985},{"type":"v1","addr":"172.21.15.133:6836","nonce":760980985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:47:40.664091+0000","modified":"2024-09-10T21:49:05.584713+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236099":{"gid":236099,"name":"d","rank":0,"incarnation":73,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.133:6837/4188280820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4188280820},{"type":"v1","addr":"172.21.15.133:6837","nonce":4188280820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":6}]} 2024-09-10T21:49:06.843 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 75 2024-09-10T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.845+0000 7f74c1a00640 1 -- 172.21.15.7:0/3129422136 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7498077f70 msgr2=0x7f749807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.845+0000 7f74c1a00640 1 --2- 172.21.15.7:0/3129422136 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7498077f70 0x7f749807a430 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f74a4007fd0 tx=0x7f74a4009040 comp rx=0 tx=0).stop 2024-09-10T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.845+0000 7f74c1a00640 1 -- 172.21.15.7:0/3129422136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc0cd750 msgr2=0x7f74bc0b0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.845+0000 7f74c1a00640 1 --2- 172.21.15.7:0/3129422136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc0cd750 0x7f74bc0b0100 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009980 tx=0x7f74ac032b90 comp rx=0 tx=0).stop 2024-09-10T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.845+0000 7f74c1a00640 1 -- 172.21.15.7:0/3129422136 shutdown_connections 2024-09-10T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.845+0000 7f74c1a00640 1 --2- 172.21.15.7:0/3129422136 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7498077f70 0x7f749807a430 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.845+0000 7f74c1a00640 1 --2- 172.21.15.7:0/3129422136 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74bc15bc90 0x7f74bc1f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.845+0000 7f74c1a00640 1 --2- 172.21.15.7:0/3129422136 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74bc15fda0 0x7f74bc14c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.845+0000 7f74c1a00640 1 --2- 172.21.15.7:0/3129422136 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74bc0cd750 0x7f74bc0b0100 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.845+0000 7f74c1a00640 1 -- 172.21.15.7:0/3129422136 >> 172.21.15.7:0/3129422136 conn(0x7f74bc0b16d0 msgr2=0x7f74bc0b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.845+0000 7f74c1a00640 1 -- 172.21.15.7:0/3129422136 shutdown_connections 2024-09-10T21:49:06.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:06.845+0000 7f74c1a00640 1 -- 172.21.15.7:0/3129422136 wait complete. 2024-09-10T21:49:07.264 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:47:40.664091+0000', 'modified': '2024-09-10T21:49:05.584713+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 89, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 236099}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_236099': {'gid': 236099, 'name': 'd', 'rank': 0, 'incarnation': 73, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.133:6837/4188280820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6835', 'nonce': 4188280820}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 4188280820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:49:07.264 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_236099:up:rejoin 2024-09-10T21:49:08.265 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:49:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.436+0000 7f63b5200640 1 -- 172.21.15.7:0/2315116476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b0073c70 msgr2=0x7f63b015fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:08.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.436+0000 7f63b5200640 1 --2- 172.21.15.7:0/2315116476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b0073c70 0x7f63b015fda0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f63b006dac0 tx=0x7f63a802a8f0 comp rx=0 tx=0).stop 2024-09-10T21:49:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.436+0000 7f63b5200640 1 -- 172.21.15.7:0/2315116476 shutdown_connections 2024-09-10T21:49:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.436+0000 7f63b5200640 1 --2- 172.21.15.7:0/2315116476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b0073c70 0x7f63b015fda0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.436+0000 7f63b5200640 1 --2- 172.21.15.7:0/2315116476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63b00cd750 0x7f63b0073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.436+0000 7f63b5200640 1 --2- 172.21.15.7:0/2315116476 >> v2:172.21.15.133:3300/0 conn(0x7f63b015bc90 0x7f63b00ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.436+0000 7f63b5200640 1 -- 172.21.15.7:0/2315116476 >> 172.21.15.7:0/2315116476 conn(0x7f63b00b16d0 msgr2=0x7f63b00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.436+0000 7f63b5200640 1 -- 172.21.15.7:0/2315116476 shutdown_connections 2024-09-10T21:49:08.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.436+0000 7f63b5200640 1 -- 172.21.15.7:0/2315116476 wait complete. 2024-09-10T21:49:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.438+0000 7f63b5200640 1 Processor -- start 2024-09-10T21:49:08.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.438+0000 7f63b5200640 1 -- start start 2024-09-10T21:49:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.439+0000 7f63b5200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63b0073c70 0x7f63b014afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.439+0000 7f63b5200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b00cd750 0x7f63b00c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:08.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.439+0000 7f63b5200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63b015bc90 0x7f63b00c8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.439+0000 7f63b5200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f63b0160670 con 0x7f63b00cd750 2024-09-10T21:49:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.439+0000 7f63b5200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f63b01604f0 con 0x7f63b0073c70 2024-09-10T21:49:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.439+0000 7f63b5200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f63b006fcf0 con 0x7f63b015bc90 2024-09-10T21:49:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.439+0000 7f63a7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b00cd750 0x7f63b00c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.439+0000 7f63a7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b00cd750 0x7f63b00c7b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52270/0 (socket says 172.21.15.7:52270) 2024-09-10T21:49:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.439+0000 7f63a7e00640 1 -- 172.21.15.7:0/108399231 learned_addr learned my addr 172.21.15.7:0/108399231 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.439+0000 7f63ade00640 1 --2- 172.21.15.7:0/108399231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63b0073c70 0x7f63b014afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.439+0000 7f63ae800640 1 --2- 172.21.15.7:0/108399231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63b015bc90 0x7f63b00c8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.440+0000 7f63a7e00640 1 -- 172.21.15.7:0/108399231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63b015bc90 msgr2=0x7f63b00c8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.440+0000 7f63a7e00640 1 --2- 172.21.15.7:0/108399231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63b015bc90 0x7f63b00c8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.440+0000 7f63a7e00640 1 -- 172.21.15.7:0/108399231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63b0073c70 msgr2=0x7f63b014afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.440+0000 7f63a7e00640 1 --2- 172.21.15.7:0/108399231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63b0073c70 0x7f63b014afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:08.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.440+0000 7f63a7e00640 1 -- 172.21.15.7:0/108399231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63b014b510 con 0x7f63b00cd750 2024-09-10T21:49:08.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.440+0000 7f63a7e00640 1 --2- 172.21.15.7:0/108399231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b00cd750 0x7f63b00c7b70 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f639800e9f0 tx=0x7f639800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:08.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.440+0000 7f63a6000640 1 -- 172.21.15.7:0/108399231 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f639800cd80 con 0x7f63b00cd750 2024-09-10T21:49:08.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.440+0000 7f63a6000640 1 -- 172.21.15.7:0/108399231 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6398005020 con 0x7f63b00cd750 2024-09-10T21:49:08.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.440+0000 7f63a6000640 1 -- 172.21.15.7:0/108399231 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6398005450 con 0x7f63b00cd750 2024-09-10T21:49:08.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.440+0000 7f63b5200640 1 -- 172.21.15.7:0/108399231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63b00c84e0 con 0x7f63b00cd750 2024-09-10T21:49:08.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.440+0000 7f63b5200640 1 -- 172.21.15.7:0/108399231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63b015d6e0 con 0x7f63b00cd750 2024-09-10T21:49:08.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.441+0000 7f63b5200640 1 -- 172.21.15.7:0/108399231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63b0073730 con 0x7f63b00cd750 2024-09-10T21:49:08.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.442+0000 7f63a6000640 1 -- 172.21.15.7:0/108399231 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f63980047a0 con 0x7f63b00cd750 2024-09-10T21:49:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.444+0000 7f63a6000640 1 --2- 172.21.15.7:0/108399231 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f638c077f70 0x7f638c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.444+0000 7f63a6000640 1 -- 172.21.15.7:0/108399231 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f639809a940 con 0x7f63b00cd750 2024-09-10T21:49:08.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.445+0000 7f63ade00640 1 --2- 172.21.15.7:0/108399231 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f638c077f70 0x7f638c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:08.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.445+0000 7f63ade00640 1 --2- 172.21.15.7:0/108399231 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f638c077f70 0x7f638c07a430 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f63b01f55e0 tx=0x7f63a0005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:08.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.449+0000 7f63a6000640 1 -- 172.21.15.7:0/108399231 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f63b014b510 con 0x7f63b00cd750 2024-09-10T21:49:08.678 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.678+0000 7f63b5200640 1 -- 172.21.15.7:0/108399231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f63b0072f30 con 0x7f63b00cd750 2024-09-10T21:49:08.679 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.678+0000 7f63a6000640 1 -- 172.21.15.7:0/108399231 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5229 (secure 0 0 0) 0x7f6398063390 con 0x7f63b00cd750 2024-09-10T21:49:08.679 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:08.679 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":76,"btime":"2024-09-10T21:49:07:587035+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/978763848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":978763848},{"type":"v1","addr":"172.21.15.7:6835","nonce":978763848}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/269527918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":269527918},{"type":"v1","addr":"172.21.15.7:6833","nonce":269527918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":235775,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/760980985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":760980985},{"type":"v1","addr":"172.21.15.133:6836","nonce":760980985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:47:40.664091+0000","modified":"2024-09-10T21:49:07.587028+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236099":{"gid":236099,"name":"d","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6837/4188280820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4188280820},{"type":"v1","addr":"172.21.15.133:6837","nonce":4188280820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":236099,"qdb_cluster":[236099]},"id":6}]} 2024-09-10T21:49:08.680 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 76 2024-09-10T21:49:08.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.681+0000 7f63b5200640 1 -- 172.21.15.7:0/108399231 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f638c077f70 msgr2=0x7f638c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:08.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.681+0000 7f63b5200640 1 --2- 172.21.15.7:0/108399231 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f638c077f70 0x7f638c07a430 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f63b01f55e0 tx=0x7f63a0005f90 comp rx=0 tx=0).stop 2024-09-10T21:49:08.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.681+0000 7f63b5200640 1 -- 172.21.15.7:0/108399231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b00cd750 msgr2=0x7f63b00c7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:08.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.681+0000 7f63b5200640 1 --2- 172.21.15.7:0/108399231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b00cd750 0x7f63b00c7b70 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f639800e9f0 tx=0x7f639800edf0 comp rx=0 tx=0).stop 2024-09-10T21:49:08.682 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.681+0000 7f63b5200640 1 -- 172.21.15.7:0/108399231 shutdown_connections 2024-09-10T21:49:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.681+0000 7f63b5200640 1 --2- 172.21.15.7:0/108399231 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f638c077f70 0x7f638c07a430 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.681+0000 7f63b5200640 1 --2- 172.21.15.7:0/108399231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63b015bc90 0x7f63b00c8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.681+0000 7f63b5200640 1 --2- 172.21.15.7:0/108399231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63b00cd750 0x7f63b00c7b70 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.681+0000 7f63b5200640 1 --2- 172.21.15.7:0/108399231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f63b0073c70 0x7f63b014afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.681+0000 7f63b5200640 1 -- 172.21.15.7:0/108399231 >> 172.21.15.7:0/108399231 conn(0x7f63b00b16d0 msgr2=0x7f63b00740a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.681+0000 7f63b5200640 1 -- 172.21.15.7:0/108399231 shutdown_connections 2024-09-10T21:49:08.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:08.681+0000 7f63b5200640 1 -- 172.21.15.7:0/108399231 wait complete. 2024-09-10T21:49:09.100 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:47:40.664091+0000', 'modified': '2024-09-10T21:49:07.587028+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 89, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 236099}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_236099': {'gid': 236099, 'name': 'd', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.133:6837/4188280820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6835', 'nonce': 4188280820}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 4188280820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 236099, 'qdb_cluster': [236099]} 2024-09-10T21:49:09.101 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:49:09.101 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T21:49:09.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.265+0000 7fc9b3200640 1 -- 172.21.15.7:0/2583987062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9ac0cddd0 msgr2=0x7fc9ac0ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:09.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.265+0000 7fc9b3200640 1 --2- 172.21.15.7:0/2583987062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9ac0cddd0 0x7fc9ac0ce280 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8009be0 tx=0x7fc9a802a8e0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.265+0000 7fc9b3200640 1 -- 172.21.15.7:0/2583987062 shutdown_connections 2024-09-10T21:49:09.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.265+0000 7fc9b3200640 1 --2- 172.21.15.7:0/2583987062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9ac0cddd0 0x7fc9ac0ce280 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.265+0000 7fc9b3200640 1 --2- 172.21.15.7:0/2583987062 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9ac15f4d0 0x7fc9ac0cc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.265+0000 7fc9b3200640 1 -- 172.21.15.7:0/2583987062 >> 172.21.15.7:0/2583987062 conn(0x7fc9ac0b16d0 msgr2=0x7fc9ac0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:09.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.265+0000 7fc9b3200640 1 -- 172.21.15.7:0/2583987062 shutdown_connections 2024-09-10T21:49:09.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.265+0000 7fc9b3200640 1 -- 172.21.15.7:0/2583987062 wait complete. 2024-09-10T21:49:09.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.267+0000 7fc9b3200640 1 Processor -- start 2024-09-10T21:49:09.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.267+0000 7fc9b3200640 1 -- start start 2024-09-10T21:49:09.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.268+0000 7fc9b3200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac0702b0 0x7fc9ac1565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.268+0000 7fc9b3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9ac0cddd0 0x7fc9ac1f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.268+0000 7fc9b3200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9ac15f4d0 0x7fc9ac154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.268+0000 7fc9b3200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac0ce940 con 0x7fc9ac0cddd0 2024-09-10T21:49:09.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.268+0000 7fc9b3200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac0ce7c0 con 0x7fc9ac0702b0 2024-09-10T21:49:09.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.268+0000 7fc9b3200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc9ac0ceac0 con 0x7fc9ac15f4d0 2024-09-10T21:49:09.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.268+0000 7fc9b1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9ac0cddd0 0x7fc9ac1f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:09.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.268+0000 7fc9b1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9ac0cddd0 0x7fc9ac1f4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52286/0 (socket says 172.21.15.7:52286) 2024-09-10T21:49:09.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.268+0000 7fc9b1400640 1 -- 172.21.15.7:0/565843343 learned_addr learned my addr 172.21.15.7:0/565843343 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:09.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.269+0000 7fc9b1400640 1 -- 172.21.15.7:0/565843343 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9ac15f4d0 msgr2=0x7fc9ac154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:09.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.269+0000 7fc9b1400640 1 --2- 172.21.15.7:0/565843343 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9ac15f4d0 0x7fc9ac154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.269+0000 7fc9b1400640 1 -- 172.21.15.7:0/565843343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac0702b0 msgr2=0x7fc9ac1565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:09.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.269+0000 7fc9b1400640 1 --2- 172.21.15.7:0/565843343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac0702b0 0x7fc9ac1565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.269+0000 7fc9b1400640 1 -- 172.21.15.7:0/565843343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ac156c60 con 0x7fc9ac0cddd0 2024-09-10T21:49:09.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.269+0000 7fc9b1400640 1 --2- 172.21.15.7:0/565843343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9ac0cddd0 0x7fc9ac1f4fe0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00c420 tx=0x7fc99c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:09.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.269+0000 7fc9a6000640 1 -- 172.21.15.7:0/565843343 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c011790 con 0x7fc9ac0cddd0 2024-09-10T21:49:09.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.269+0000 7fc9a6000640 1 -- 172.21.15.7:0/565843343 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc99c017070 con 0x7fc9ac0cddd0 2024-09-10T21:49:09.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.269+0000 7fc9b3200640 1 -- 172.21.15.7:0/565843343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9ac1591f0 con 0x7fc9ac0cddd0 2024-09-10T21:49:09.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.269+0000 7fc9a6000640 1 -- 172.21.15.7:0/565843343 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c011c80 con 0x7fc9ac0cddd0 2024-09-10T21:49:09.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.269+0000 7fc9b3200640 1 -- 172.21.15.7:0/565843343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9ac1597b0 con 0x7fc9ac0cddd0 2024-09-10T21:49:09.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.271+0000 7fc9a6000640 1 -- 172.21.15.7:0/565843343 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc99c01c460 con 0x7fc9ac0cddd0 2024-09-10T21:49:09.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.272+0000 7fc9a6000640 1 --2- 172.21.15.7:0/565843343 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc990077f70 0x7fc99007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.273+0000 7fc9a6000640 1 -- 172.21.15.7:0/565843343 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7fc99c099c10 con 0x7fc9ac0cddd0 2024-09-10T21:49:09.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.273+0000 7fc9b1e00640 1 --2- 172.21.15.7:0/565843343 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc990077f70 0x7fc99007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:09.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.273+0000 7fc9b788b740 1 Processor -- start 2024-09-10T21:49:09.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.273+0000 7fc9b788b740 1 -- start start 2024-09-10T21:49:09.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.274+0000 7fc9b1e00640 1 --2- 172.21.15.7:0/565843343 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc990077f70 0x7fc99007a430 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00058c0 tx=0x7fc9a0005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:09.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.274+0000 7fc9b788b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 0x5584177e3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.274+0000 7fc9b788b740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55841779ec10 0x5584177e2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.274+0000 7fc9b788b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55841779f190 0x5584177e2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.274+0000 7fc9b788b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x558417721430 con 0x5584177e3aa0 2024-09-10T21:49:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.274+0000 7fc9b788b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5584177115d0 con 0x55841779f190 2024-09-10T21:49:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.274+0000 7fc9b788b740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x558417700a00 con 0x55841779ec10 2024-09-10T21:49:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.274+0000 7fc9b1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 0x5584177e3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.274+0000 7fc9b1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 0x5584177e3e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52292/0 (socket says 172.21.15.7:52292) 2024-09-10T21:49:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.274+0000 7fc9b1e00640 1 -- 172.21.15.7:0/2627058954 learned_addr learned my addr 172.21.15.7:0/2627058954 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.274+0000 7fc9b1400640 1 --2- 172.21.15.7:0/2627058954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55841779ec10 0x5584177e2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.274+0000 7fc9b2800640 1 --2- 172.21.15.7:0/2627058954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55841779f190 0x5584177e2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.275+0000 7fc9b1e00640 1 -- 172.21.15.7:0/2627058954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55841779ec10 msgr2=0x5584177e2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.275+0000 7fc9b1e00640 1 --2- 172.21.15.7:0/2627058954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55841779ec10 0x5584177e2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.275+0000 7fc9b1e00640 1 -- 172.21.15.7:0/2627058954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55841779f190 msgr2=0x5584177e2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:09.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.275+0000 7fc9b1e00640 1 --2- 172.21.15.7:0/2627058954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55841779f190 0x5584177e2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.275+0000 7fc9b1e00640 1 -- 172.21.15.7:0/2627058954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584176c9520 con 0x5584177e3aa0 2024-09-10T21:49:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.275+0000 7fc9b1400640 1 --2- 172.21.15.7:0/2627058954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55841779ec10 0x5584177e2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.275+0000 7fc9b2800640 1 --2- 172.21.15.7:0/2627058954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55841779f190 0x5584177e2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.275+0000 7fc9b1e00640 1 --2- 172.21.15.7:0/2627058954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 0x5584177e3e80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fc9a000dd30 tx=0x7fc9a0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f2bf1345f5a10f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.275+0000 7fc99a600640 1 -- 172.21.15.7:0/2627058954 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a001d440 con 0x5584177e3aa0 2024-09-10T21:49:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.275+0000 7fc99a600640 1 -- 172.21.15.7:0/2627058954 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9a0022070 con 0x5584177e3aa0 2024-09-10T21:49:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.275+0000 7fc99a600640 1 -- 172.21.15.7:0/2627058954 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a001d970 con 0x5584177e3aa0 2024-09-10T21:49:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.276+0000 7fc9b788b740 1 -- 172.21.15.7:0/2627058954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 msgr2=0x5584177e3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.276+0000 7fc9b788b740 1 --2- 172.21.15.7:0/2627058954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 0x5584177e3e80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fc9a000dd30 tx=0x7fc9a0010000 comp rx=0 tx=0).stop 2024-09-10T21:49:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.276+0000 7fc9b788b740 1 -- 172.21.15.7:0/2627058954 shutdown_connections 2024-09-10T21:49:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.276+0000 7fc9b788b740 1 --2- 172.21.15.7:0/2627058954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55841779f190 0x5584177e2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.276+0000 7fc9b788b740 1 --2- 172.21.15.7:0/2627058954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55841779ec10 0x5584177e2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.276+0000 7fc9b788b740 1 --2- 172.21.15.7:0/2627058954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 0x5584177e3e80 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.276+0000 7fc9b788b740 1 -- 172.21.15.7:0/2627058954 >> 172.21.15.7:0/2627058954 conn(0x55841779a1e0 msgr2=0x55841779dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:09.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.276+0000 7fc9b788b740 1 -- 172.21.15.7:0/2627058954 shutdown_connections 2024-09-10T21:49:09.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.276+0000 7fc9b788b740 1 -- 172.21.15.7:0/2627058954 wait complete. 2024-09-10T21:49:09.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.277+0000 7fc9b788b740 1 -- start start 2024-09-10T21:49:09.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.277+0000 7fc9b788b740 1 Processor -- start 2024-09-10T21:49:09.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.278+0000 7fc9b788b740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55841779f190 0x5584177b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.278+0000 7fc9b788b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 0x5584177b52b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.278+0000 7fc9b788b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584177b57f0 0x5584177970f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.278+0000 7fc9b788b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5584176a7330 con 0x5584177e3aa0 2024-09-10T21:49:09.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.278+0000 7fc9b788b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x558417704810 con 0x5584177b57f0 2024-09-10T21:49:09.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.278+0000 7fc9b788b740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x558417471620 con 0x55841779f190 2024-09-10T21:49:09.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.278+0000 7fc9b788b740 10 client.0 _ll_register_callbacks cb 0x5584176ac420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:49:09.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.278+0000 7fc9b1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 0x5584177b52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:09.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.278+0000 7fc9b1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 0x5584177b52b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52300/0 (socket says 172.21.15.7:52300) 2024-09-10T21:49:09.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.278+0000 7fc9b1400640 1 -- 172.21.15.7:0/1683718370 learned_addr learned my addr 172.21.15.7:0/1683718370 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:09.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.278+0000 7fc9b2800640 1 --2- 172.21.15.7:0/1683718370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584177b57f0 0x5584177970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:09.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.278+0000 7fc9b1e00640 1 --2- 172.21.15.7:0/1683718370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55841779f190 0x5584177b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:09.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.279+0000 7fc9b1400640 1 -- 172.21.15.7:0/1683718370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55841779f190 msgr2=0x5584177b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:09.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.279+0000 7fc9b1400640 1 --2- 172.21.15.7:0/1683718370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55841779f190 0x5584177b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.279+0000 7fc9b1400640 1 -- 172.21.15.7:0/1683718370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584177b57f0 msgr2=0x5584177970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:09.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.279+0000 7fc9b1400640 1 --2- 172.21.15.7:0/1683718370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584177b57f0 0x5584177970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.279+0000 7fc9b1400640 1 -- 172.21.15.7:0/1683718370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558417462850 con 0x5584177e3aa0 2024-09-10T21:49:09.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.279+0000 7fc9b2800640 1 --2- 172.21.15.7:0/1683718370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584177b57f0 0x5584177970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:09.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.279+0000 7fc9b1400640 1 --2- 172.21.15.7:0/1683718370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 0x5584177b52b0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00cde0 tx=0x7fc99c061b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2339f738d8440743 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:09.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.279+0000 7fc98ea00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:49:09.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.279+0000 7fc98ea00640 1 -- 172.21.15.7:0/1683718370 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc99c067da0 con 0x5584177e3aa0 2024-09-10T21:49:09.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.279+0000 7fc98ea00640 1 -- 172.21.15.7:0/1683718370 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7fc99c07b8d0 con 0x5584177e3aa0 2024-09-10T21:49:09.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.279+0000 7fc9b788b740 1 -- 172.21.15.7:0/1683718370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9a0022070 con 0x5584177e3aa0 2024-09-10T21:49:09.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.279+0000 7fc98ea00640 1 -- 172.21.15.7:0/1683718370 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fc99c06b430 con 0x5584177e3aa0 2024-09-10T21:49:09.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.279+0000 7fc9b788b740 10 client.7266 fetch_fsmap learned FSMap version 76 2024-09-10T21:49:09.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.279+0000 7fc9b788b740 1 -- 172.21.15.7:0/1683718370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x5584174d96b0 con 0x5584177e3aa0 2024-09-10T21:49:09.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.280+0000 7fc98ea00640 1 -- 172.21.15.7:0/1683718370 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9a0022070 con 0x5584177e3aa0 2024-09-10T21:49:09.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.280+0000 7fc98ea00640 1 -- 172.21.15.7:0/1683718370 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7fc99c094050 con 0x5584177e3aa0 2024-09-10T21:49:09.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.280+0000 7fc9b788b740 10 client.7266 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T21:49:09.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.280+0000 7fc9b788b740 10 client.7266 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:49:09.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.280+0000 7fc9b788b740 20 client.7266 populate_metadata read hostname 'smithi007' 2024-09-10T21:49:09.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.280+0000 7fc9b788b740 1 --2- 172.21.15.7:0/1683718370 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x5584177e1cc0 0x5584177f6030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.280+0000 7fc9b788b740 4 client.7266 mds_command: new command op to 236099 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:49:09.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.280+0000 7fc9b788b740 1 -- 172.21.15.7:0/1683718370 --> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558417713520 con 0x5584177e1cc0 2024-09-10T21:49:09.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.280+0000 7fc9b1e00640 1 --2- 172.21.15.7:0/1683718370 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x5584177e1cc0 0x5584177f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:09.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.281+0000 7fc9b1e00640 1 --2- 172.21.15.7:0/1683718370 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x5584177e1cc0 0x5584177f6030 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=859393b0213e156f server_cookie=f632412461e050b9 in_seq=0 out_seq=0 2024-09-10T21:49:09.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.281+0000 7fc98ea00640 10 client.7266 ms_handle_connect on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:09.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.282+0000 7fc98ea00640 1 -- 172.21.15.7:0/1683718370 <== mds.0 v2:172.21.15.133:6835/4188280820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558417713520 con 0x5584177e1cc0 2024-09-10T21:49:09.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.282+0000 7fc98ea00640 10 client.7266 handle_command_reply: tid=0 2024-09-10T21:49:09.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.282+0000 7fc9b788b740 1 client.7266 shutdown 2024-09-10T21:49:09.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.283+0000 7fc9b788b740 1 -- 172.21.15.7:0/1683718370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 msgr2=0x5584177b52b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:09.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.283+0000 7fc9b788b740 1 --2- 172.21.15.7:0/1683718370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 0x5584177b52b0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00cde0 tx=0x7fc99c061b60 comp rx=0 tx=0).stop 2024-09-10T21:49:09.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.283+0000 7fc9b788b740 1 -- 172.21.15.7:0/1683718370 shutdown_connections 2024-09-10T21:49:09.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.283+0000 7fc9b788b740 1 --2- 172.21.15.7:0/1683718370 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x5584177e1cc0 0x5584177f6030 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.283+0000 7fc9b788b740 1 --2- 172.21.15.7:0/1683718370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584177b57f0 0x5584177970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.283+0000 7fc9b788b740 1 --2- 172.21.15.7:0/1683718370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 0x5584177b52b0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.283+0000 7fc98ea00640 0 client.7266 ms_handle_reset on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:09.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.283+0000 7fc9b788b740 1 --2- 172.21.15.7:0/1683718370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55841779f190 0x5584177b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.283+0000 7fc9b788b740 1 -- 172.21.15.7:0/1683718370 >> 172.21.15.7:0/1683718370 conn(0x55841779ec10 msgr2=0x558417798db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:09.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.283+0000 7fc9b788b740 1 -- 172.21.15.7:0/1683718370 shutdown_connections 2024-09-10T21:49:09.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.283+0000 7fc9b788b740 1 -- 172.21.15.7:0/1683718370 wait complete. 2024-09-10T21:49:09.288 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.283+0000 7fc9b788b740 20 client.7266 trim_cache size 0 max 16384 2024-09-10T21:49:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.302+0000 7fc9b788b740 1 Processor -- start 2024-09-10T21:49:09.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.302+0000 7fc9b788b740 1 -- start start 2024-09-10T21:49:09.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.303+0000 7fc9b788b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55841779f190 0x558417798d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.303+0000 7fc9b788b740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584177b57f0 0x558417799260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.303+0000 7fc9b788b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584177e3aa0 0x5584177d8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.303+0000 7fc9b788b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac2012e0 con 0x55841779f190 2024-09-10T21:49:09.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.303+0000 7fc9b788b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac200300 con 0x5584177e3aa0 2024-09-10T21:49:09.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.303+0000 7fc9b788b740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc9ac1f8440 con 0x5584177b57f0 2024-09-10T21:49:09.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.303+0000 7fc9b1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55841779f190 0x558417798d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.303+0000 7fc9b1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55841779f190 0x558417798d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52314/0 (socket says 172.21.15.7:52314) 2024-09-10T21:49:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.303+0000 7fc9b1e00640 1 -- 172.21.15.7:0/4162543846 learned_addr learned my addr 172.21.15.7:0/4162543846 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.304+0000 7fc9b1e00640 1 -- 172.21.15.7:0/4162543846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584177b57f0 msgr2=0x558417799260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.304+0000 7fc9b1e00640 1 --2- 172.21.15.7:0/4162543846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584177b57f0 0x558417799260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.304+0000 7fc9b1e00640 1 -- 172.21.15.7:0/4162543846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584177e3aa0 msgr2=0x5584177d8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.304+0000 7fc9b1e00640 1 --2- 172.21.15.7:0/4162543846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584177e3aa0 0x5584177d8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.304+0000 7fc9b1e00640 1 -- 172.21.15.7:0/4162543846 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558417797460 con 0x55841779f190 2024-09-10T21:49:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.304+0000 7fc9b1e00640 1 --2- 172.21.15.7:0/4162543846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55841779f190 0x558417798d20 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc9a001f040 tx=0x7fc9a0005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c86aec2ea5e1558 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.304+0000 7fc999c00640 1 -- 172.21.15.7:0/4162543846 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a001d8d0 con 0x55841779f190 2024-09-10T21:49:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.304+0000 7fc999c00640 1 -- 172.21.15.7:0/4162543846 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9a0022040 con 0x55841779f190 2024-09-10T21:49:09.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.304+0000 7fc999c00640 1 -- 172.21.15.7:0/4162543846 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a001dd50 con 0x55841779f190 2024-09-10T21:49:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.305+0000 7fc9b788b740 1 -- 172.21.15.7:0/4162543846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55841779f190 msgr2=0x558417798d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.305+0000 7fc9b788b740 1 --2- 172.21.15.7:0/4162543846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55841779f190 0x558417798d20 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc9a001f040 tx=0x7fc9a0005c90 comp rx=0 tx=0).stop 2024-09-10T21:49:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.305+0000 7fc9b788b740 1 -- 172.21.15.7:0/4162543846 shutdown_connections 2024-09-10T21:49:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.305+0000 7fc9b788b740 1 --2- 172.21.15.7:0/4162543846 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5584177e3aa0 0x5584177d8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.305+0000 7fc9b788b740 1 --2- 172.21.15.7:0/4162543846 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584177b57f0 0x558417799260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.305+0000 7fc9b788b740 1 --2- 172.21.15.7:0/4162543846 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55841779f190 0x558417798d20 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.305+0000 7fc9b788b740 1 -- 172.21.15.7:0/4162543846 >> 172.21.15.7:0/4162543846 conn(0x55841779ec10 msgr2=0x5584177e3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.305+0000 7fc9b788b740 1 -- 172.21.15.7:0/4162543846 shutdown_connections 2024-09-10T21:49:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.305+0000 7fc9b788b740 1 -- 172.21.15.7:0/4162543846 wait complete. 2024-09-10T21:49:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.306+0000 7fc9b788b740 1 -- start start 2024-09-10T21:49:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.306+0000 7fc9b788b740 1 Processor -- start 2024-09-10T21:49:09.310 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.307+0000 7fc9b788b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55841779f190 0x5584177fc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.307+0000 7fc9b788b740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584177b57f0 0x55841785d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.307+0000 7fc9b788b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 0x558417866d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.307+0000 7fc9b788b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5584177e2990 con 0x5584177e3aa0 2024-09-10T21:49:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.307+0000 7fc9b788b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ac1ffec0 con 0x55841779f190 2024-09-10T21:49:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.307+0000 7fc9b788b740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5584177b60d0 con 0x5584177b57f0 2024-09-10T21:49:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.307+0000 7fc9b788b740 10 client.0 _ll_register_callbacks cb 0x7fc99c061a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:49:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.307+0000 7fc9b2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 0x558417866d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.307+0000 7fc9b2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 0x558417866d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52320/0 (socket says 172.21.15.7:52320) 2024-09-10T21:49:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.307+0000 7fc9b2800640 1 -- 172.21.15.7:0/382913635 learned_addr learned my addr 172.21.15.7:0/382913635 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.307+0000 7fc9b1e00640 1 --2- 172.21.15.7:0/382913635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55841779f190 0x5584177fc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.307+0000 7fc9b1400640 1 --2- 172.21.15.7:0/382913635 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584177b57f0 0x55841785d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:09.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.307+0000 7fc9b2800640 1 -- 172.21.15.7:0/382913635 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584177b57f0 msgr2=0x55841785d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.307+0000 7fc9b2800640 1 --2- 172.21.15.7:0/382913635 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584177b57f0 0x55841785d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.308+0000 7fc9b2800640 1 -- 172.21.15.7:0/382913635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55841779f190 msgr2=0x5584177fc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.308+0000 7fc9b2800640 1 --2- 172.21.15.7:0/382913635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55841779f190 0x5584177fc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.308+0000 7fc9b2800640 1 -- 172.21.15.7:0/382913635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584177f54a0 con 0x5584177e3aa0 2024-09-10T21:49:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.308+0000 7fc9b1e00640 1 --2- 172.21.15.7:0/382913635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55841779f190 0x5584177fc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:49:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.308+0000 7fc9b2800640 1 --2- 172.21.15.7:0/382913635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 0x558417866d40 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8009d00 tx=0x7fc9a8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32bfb32bc41a0613 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.308+0000 7fc98ea00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:49:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.308+0000 7fc98ea00640 1 -- 172.21.15.7:0/382913635 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8004770 con 0x5584177e3aa0 2024-09-10T21:49:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.308+0000 7fc98ea00640 1 -- 172.21.15.7:0/382913635 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7fc9a8045bc0 con 0x5584177e3aa0 2024-09-10T21:49:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.308+0000 7fc9b788b740 1 -- 172.21.15.7:0/382913635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9a0022040 con 0x5584177e3aa0 2024-09-10T21:49:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.308+0000 7fc98ea00640 1 -- 172.21.15.7:0/382913635 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7fc9a8045dc0 con 0x5584177e3aa0 2024-09-10T21:49:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.308+0000 7fc9b788b740 10 client.7272 fetch_fsmap learned FSMap version 76 2024-09-10T21:49:09.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.308+0000 7fc9b788b740 1 -- 172.21.15.7:0/382913635 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x5584177984f0 con 0x5584177e3aa0 2024-09-10T21:49:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.309+0000 7fc98ea00640 1 -- 172.21.15.7:0/382913635 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9a0022040 con 0x5584177e3aa0 2024-09-10T21:49:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.309+0000 7fc98ea00640 1 -- 172.21.15.7:0/382913635 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7fc9a804c050 con 0x5584177e3aa0 2024-09-10T21:49:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.309+0000 7fc9b788b740 10 client.7272 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T21:49:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.309+0000 7fc9b788b740 10 client.7272 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:49:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.309+0000 7fc9b788b740 20 client.7272 populate_metadata read hostname 'smithi007' 2024-09-10T21:49:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.309+0000 7fc9b788b740 1 --2- 172.21.15.7:0/382913635 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x5584177dbca0 0x558417805000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.309+0000 7fc9b788b740 4 client.7272 mds_command: new command op to 236099 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:49:09.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.309+0000 7fc9b788b740 1 -- 172.21.15.7:0/382913635 --> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] -- command(tid 0: {"prefix": "status"}) -- 0x55841770efd0 con 0x5584177dbca0 2024-09-10T21:49:09.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.309+0000 7fc9b1e00640 1 --2- 172.21.15.7:0/382913635 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x5584177dbca0 0x558417805000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.310+0000 7fc9b1e00640 1 --2- 172.21.15.7:0/382913635 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x5584177dbca0 0x558417805000 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dfb298d10de8058 server_cookie=836e6dc480199aa9 in_seq=0 out_seq=0 2024-09-10T21:49:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.310+0000 7fc98ea00640 10 client.7272 ms_handle_connect on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.310+0000 7fc98ea00640 1 -- 172.21.15.7:0/382913635 <== mds.0 v2:172.21.15.133:6835/4188280820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55841770efd0 con 0x5584177dbca0 2024-09-10T21:49:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.310+0000 7fc98ea00640 10 client.7272 handle_command_reply: tid=0 2024-09-10T21:49:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.310+0000 7fc9b788b740 1 client.7272 shutdown 2024-09-10T21:49:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.310+0000 7fc9b788b740 1 -- 172.21.15.7:0/382913635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 msgr2=0x558417866d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.310+0000 7fc9b788b740 1 --2- 172.21.15.7:0/382913635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 0x558417866d40 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc9a8009d00 tx=0x7fc9a8004270 comp rx=0 tx=0).stop 2024-09-10T21:49:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.310+0000 7fc9b788b740 1 -- 172.21.15.7:0/382913635 shutdown_connections 2024-09-10T21:49:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.310+0000 7fc9b788b740 1 --2- 172.21.15.7:0/382913635 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x5584177dbca0 0x558417805000 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.310+0000 7fc9b788b740 1 --2- 172.21.15.7:0/382913635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5584177e3aa0 0x558417866d40 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.310+0000 7fc9b788b740 1 --2- 172.21.15.7:0/382913635 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5584177b57f0 0x55841785d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.310+0000 7fc9b788b740 1 --2- 172.21.15.7:0/382913635 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55841779f190 0x5584177fc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.310+0000 7fc9b788b740 1 -- 172.21.15.7:0/382913635 >> 172.21.15.7:0/382913635 conn(0x55841779ec10 msgr2=0x558417798d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:09.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.310+0000 7fc98ea00640 0 client.7272 ms_handle_reset on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:09.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.311+0000 7fc9b788b740 1 -- 172.21.15.7:0/382913635 shutdown_connections 2024-09-10T21:49:09.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.311+0000 7fc9b788b740 1 -- 172.21.15.7:0/382913635 wait complete. 2024-09-10T21:49:09.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.311+0000 7fc9b788b740 20 client.7272 trim_cache size 0 max 16384 2024-09-10T21:49:09.316 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:49:09.316 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2029451a-3c26-48c7-b874-0597197278ac", 2024-09-10T21:49:09.316 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-10T21:49:09.316 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 236099, 2024-09-10T21:49:09.316 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T21:49:09.316 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T21:49:09.317 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T21:49:09.317 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 4.7295158669999999, 2024-09-10T21:49:09.317 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 76, 2024-09-10T21:49:09.317 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 89, 2024-09-10T21:49:09.317 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 89, 2024-09-10T21:49:09.317 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 5.6806807680000002 2024-09-10T21:49:09.317 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:49:09.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.311+0000 7fc9b3200640 1 -- 172.21.15.7:0/565843343 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc990077f70 msgr2=0x7fc99007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:09.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.311+0000 7fc9b3200640 1 --2- 172.21.15.7:0/565843343 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc990077f70 0x7fc99007a430 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00058c0 tx=0x7fc9a0005850 comp rx=0 tx=0).stop 2024-09-10T21:49:09.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.311+0000 7fc9b3200640 1 -- 172.21.15.7:0/565843343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9ac0cddd0 msgr2=0x7fc9ac1f4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:09.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.311+0000 7fc9b3200640 1 --2- 172.21.15.7:0/565843343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9ac0cddd0 0x7fc9ac1f4fe0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00c420 tx=0x7fc99c00c8e0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.311+0000 7fc9b3200640 1 -- 172.21.15.7:0/565843343 shutdown_connections 2024-09-10T21:49:09.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.311+0000 7fc9b3200640 1 --2- 172.21.15.7:0/565843343 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc990077f70 0x7fc99007a430 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.311+0000 7fc9b3200640 1 --2- 172.21.15.7:0/565843343 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9ac15f4d0 0x7fc9ac154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.311+0000 7fc9b3200640 1 --2- 172.21.15.7:0/565843343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9ac0cddd0 0x7fc9ac1f4fe0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.311+0000 7fc9b3200640 1 --2- 172.21.15.7:0/565843343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9ac0702b0 0x7fc9ac1565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.311+0000 7fc9b3200640 1 -- 172.21.15.7:0/565843343 >> 172.21.15.7:0/565843343 conn(0x7fc9ac0b16d0 msgr2=0x7fc9ac15f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:09.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.312+0000 7fc9b3200640 1 -- 172.21.15.7:0/565843343 shutdown_connections 2024-09-10T21:49:09.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.312+0000 7fc9b3200640 1 -- 172.21.15.7:0/565843343 wait complete. 2024-09-10T21:49:09.328 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:49:09.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.489+0000 7f68bf000640 1 -- 172.21.15.7:0/2979980169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f689c003a50 msgr2=0x7f689c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:09.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.489+0000 7f68bf000640 1 --2- 172.21.15.7:0/2979980169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f689c003a50 0x7f689c003e70 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009cd0 tx=0x7f689c005830 comp rx=0 tx=0).stop 2024-09-10T21:49:09.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.489+0000 7f68bf000640 1 -- 172.21.15.7:0/2979980169 shutdown_connections 2024-09-10T21:49:09.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.489+0000 7f68bf000640 1 --2- 172.21.15.7:0/2979980169 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f689c003a50 0x7f689c003e70 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.489+0000 7f68bf000640 1 -- 172.21.15.7:0/2979980169 >> 172.21.15.7:0/2979980169 conn(0x7f68b80b16d0 msgr2=0x7f68b80b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:09.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.489+0000 7f68bf000640 1 -- 172.21.15.7:0/2979980169 shutdown_connections 2024-09-10T21:49:09.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.489+0000 7f68bf000640 1 -- 172.21.15.7:0/2979980169 wait complete. 2024-09-10T21:49:09.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.491+0000 7f68bf000640 1 Processor -- start 2024-09-10T21:49:09.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.491+0000 7f68bf000640 1 -- start start 2024-09-10T21:49:09.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.492+0000 7f68bf000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f689c003a50 0x7f68b80b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.492+0000 7f68bf000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68b815e840 0x7f68b815cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.492+0000 7f68bf000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b80c7ec0 0x7f68b80c7380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.492+0000 7f68bf000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f68b80cd2d0 con 0x7f689c003a50 2024-09-10T21:49:09.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.492+0000 7f68bf000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f68b80cd150 con 0x7f68b80c7ec0 2024-09-10T21:49:09.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.492+0000 7f68bf000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f68b8160180 con 0x7f68b815e840 2024-09-10T21:49:09.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.492+0000 7f68b7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f689c003a50 0x7f68b80b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:09.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.492+0000 7f68b7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f689c003a50 0x7f68b80b0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52336/0 (socket says 172.21.15.7:52336) 2024-09-10T21:49:09.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.492+0000 7f68b7e00640 1 -- 172.21.15.7:0/329810674 learned_addr learned my addr 172.21.15.7:0/329810674 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:09.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.492+0000 7f68b7400640 1 --2- 172.21.15.7:0/329810674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68b815e840 0x7f68b815cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:09.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.492+0000 7f68bd000640 1 --2- 172.21.15.7:0/329810674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b80c7ec0 0x7f68b80c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:09.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.493+0000 7f68b7e00640 1 -- 172.21.15.7:0/329810674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68b815e840 msgr2=0x7f68b815cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:09.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.493+0000 7f68b7e00640 1 --2- 172.21.15.7:0/329810674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68b815e840 0x7f68b815cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.493+0000 7f68b7e00640 1 -- 172.21.15.7:0/329810674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b80c7ec0 msgr2=0x7f68b80c7380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:09.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.493+0000 7f68b7e00640 1 --2- 172.21.15.7:0/329810674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b80c7ec0 0x7f68b80c7380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.493+0000 7f68b7e00640 1 -- 172.21.15.7:0/329810674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b815ed60 con 0x7f689c003a50 2024-09-10T21:49:09.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.493+0000 7f68bd000640 1 --2- 172.21.15.7:0/329810674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b80c7ec0 0x7f68b80c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:49:09.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.493+0000 7f68b7400640 1 --2- 172.21.15.7:0/329810674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68b815e840 0x7f68b815cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:49:09.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.493+0000 7f68b7e00640 1 --2- 172.21.15.7:0/329810674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f689c003a50 0x7f68b80b0e50 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009df0 tx=0x7f68a8003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:09.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.493+0000 7f68afe00640 1 -- 172.21.15.7:0/329810674 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8038040 con 0x7f689c003a50 2024-09-10T21:49:09.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.493+0000 7f68afe00640 1 -- 172.21.15.7:0/329810674 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68a802b9a0 con 0x7f689c003a50 2024-09-10T21:49:09.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.493+0000 7f68afe00640 1 -- 172.21.15.7:0/329810674 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a8003c70 con 0x7f689c003a50 2024-09-10T21:49:09.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.493+0000 7f68bf000640 1 -- 172.21.15.7:0/329810674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b815f040 con 0x7f689c003a50 2024-09-10T21:49:09.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.493+0000 7f68bf000640 1 -- 172.21.15.7:0/329810674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b80b1390 con 0x7f689c003a50 2024-09-10T21:49:09.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.494+0000 7f68bf000640 1 -- 172.21.15.7:0/329810674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68b815afa0 con 0x7f689c003a50 2024-09-10T21:49:09.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.495+0000 7f68afe00640 1 -- 172.21.15.7:0/329810674 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f68a803b070 con 0x7f689c003a50 2024-09-10T21:49:09.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.496+0000 7f68afe00640 1 --2- 172.21.15.7:0/329810674 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6898077fa0 0x7f689807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:09.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.497+0000 7f68afe00640 1 -- 172.21.15.7:0/329810674 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f68a80ba0d0 con 0x7f689c003a50 2024-09-10T21:49:09.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.497+0000 7f68b7400640 1 --2- 172.21.15.7:0/329810674 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6898077fa0 0x7f689807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:09.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.498+0000 7f68b7400640 1 --2- 172.21.15.7:0/329810674 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6898077fa0 0x7f689807a460 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f68a4006fd0 tx=0x7f68a4006ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:09.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.500+0000 7f68afe00640 1 -- 172.21.15.7:0/329810674 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f68b815afa0 con 0x7f689c003a50 2024-09-10T21:49:09.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.736+0000 7f68bf000640 1 -- 172.21.15.7:0/329810674 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f68b80c6c30 con 0x7f689c003a50 2024-09-10T21:49:09.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.736+0000 7f68afe00640 1 -- 172.21.15.7:0/329810674 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) ==== 53+0+73 (secure 0 0 0) 0x7f68a804a020 con 0x7f689c003a50 2024-09-10T21:49:09.737 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:49:09.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.739+0000 7f68bf000640 1 -- 172.21.15.7:0/329810674 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6898077fa0 msgr2=0x7f689807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:09.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.739+0000 7f68bf000640 1 --2- 172.21.15.7:0/329810674 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6898077fa0 0x7f689807a460 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f68a4006fd0 tx=0x7f68a4006ab0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.739+0000 7f68bf000640 1 -- 172.21.15.7:0/329810674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f689c003a50 msgr2=0x7f68b80b0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:09.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.739+0000 7f68bf000640 1 --2- 172.21.15.7:0/329810674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f689c003a50 0x7f68b80b0e50 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f68a8009df0 tx=0x7f68a8003820 comp rx=0 tx=0).stop 2024-09-10T21:49:09.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.739+0000 7f68bf000640 1 -- 172.21.15.7:0/329810674 shutdown_connections 2024-09-10T21:49:09.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.739+0000 7f68bf000640 1 --2- 172.21.15.7:0/329810674 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6898077fa0 0x7f689807a460 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.739+0000 7f68bf000640 1 --2- 172.21.15.7:0/329810674 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f68b80c7ec0 0x7f68b80c7380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.739+0000 7f68bf000640 1 --2- 172.21.15.7:0/329810674 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f68b815e840 0x7f68b815cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.739+0000 7f68bf000640 1 --2- 172.21.15.7:0/329810674 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f689c003a50 0x7f68b80b0e50 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:09.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.739+0000 7f68bf000640 1 -- 172.21.15.7:0/329810674 >> 172.21.15.7:0/329810674 conn(0x7f68b80b16d0 msgr2=0x7f68b80b2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:09.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.739+0000 7f68bf000640 1 -- 172.21.15.7:0/329810674 shutdown_connections 2024-09-10T21:49:09.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:09.739+0000 7f68bf000640 1 -- 172.21.15.7:0/329810674 wait complete. 2024-09-10T21:49:09.756 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:49:09.756 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:49:09.756 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-10T21:49:09.757 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:49:09.757 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:49:09.757 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:49:09.757 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:49:09.757 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:49:09.757 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:49:09.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:49:09.757 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-10T21:49:09.794 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:49:09.794 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:49:09.794 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:49:09.794 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:49:09.794 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T21:49:09.794 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:49:09.794 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:49:09.794 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T21:49:09.794 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-10T21:49:09.795 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:49:09.795 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T21:49:09.795 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-10T21:49:09.795 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:49:09.795 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-10T21:49:09.795 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-10T21:49:09.795 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T21:49:09.795 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42195sec preferred_lft 42195sec 2024-09-10T21:49:09.795 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-10T21:49:09.795 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:49:09.795 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:49:09.795 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-10T21:49:09.795 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-10T21:49:09.795 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:49:09.796 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff 2024-09-10T21:49:09.796 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:49:09.796 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:49:09.796 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::3c3d:dbff:fe02:4c15/64 scope link 2024-09-10T21:49:09.796 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:49:09.796 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:49:09.796 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether f6:f9:69:5a:86:f5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:49:09.796 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f4f9:69ff:fe5a:86f5/64 scope link 2024-09-10T21:49:09.796 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:49:09.796 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:49:09.796 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:49:09.796 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::433:5cff:fe44:4a5b/64 scope link 2024-09-10T21:49:09.796 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:49:09.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:49:09.797 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:49:09.888 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:49:09.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:49:09.891 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-10T21:49:09.927 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-10T21:49:09.927 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:49:09.927 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T21:49:09.928 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T21:49:09.928 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:49:09.928 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:49:09.928 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:49:09.928 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:49:09.928 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:49:09.928 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:49:09.928 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:49:10.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:49:10.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:49:10.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:49:10.126 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:49:10.162 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:49:10.162 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:49:10.162 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:49:10.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:49:10.162 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-10T21:49:10.257 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:49:10.258 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:49:10.293 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:49:10.293 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:49:10.293 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:49:10.368 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:49:10.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:49:10.369 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:49:10.498 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:49:10.498 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:49:10.525 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:49:10.525 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:49:10.525 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.133:3300,172.21.15.133:3301" 2024-09-10T21:49:10.525 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@2029451a-3c26-48c7-b874-0597197278ac.cephfs=/ 2024-09-10T21:49:10.525 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.133:3300/172.21.15.133:3301" will pass to kernel 2024-09-10T21:49:10.528 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T21:49:10.528 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:49:10.528 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:49:10.528 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:49:10.528 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:49:10.531 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:49:10.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:49:10.531 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:49:10.582 INFO:tasks.ceph.mds.d.smithi133.stderr:2024-09-10T21:49:10.581+0000 7f7358000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:49:10.586 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:49:10.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:49:10.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:49:10.678 INFO:teuthology.orchestra.run.smithi007.stdout:client7281 2024-09-10T21:49:10.678 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:49:10.678 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:49:10.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:49:10.714 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /dir recursive,repair 2024-09-10T21:49:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.918+0000 7f3690200640 1 -- 172.21.15.7:0/2119593945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368815bce0 msgr2=0x7f368806cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.918+0000 7f3690200640 1 --2- 172.21.15.7:0/2119593945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368815bce0 0x7f368806cb20 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f3680005fd0 tx=0x7f368002aa70 comp rx=0 tx=0).stop 2024-09-10T21:49:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.918+0000 7f3690200640 1 -- 172.21.15.7:0/2119593945 shutdown_connections 2024-09-10T21:49:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.918+0000 7f3690200640 1 --2- 172.21.15.7:0/2119593945 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368815bce0 0x7f368806cb20 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.918+0000 7f3690200640 1 --2- 172.21.15.7:0/2119593945 >> v2:172.21.15.133:3300/0 conn(0x7f368815b900 0x7f3688070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.918+0000 7f3690200640 1 -- 172.21.15.7:0/2119593945 >> 172.21.15.7:0/2119593945 conn(0x7f36880b1c10 msgr2=0x7f36880b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.918+0000 7f3690200640 1 -- 172.21.15.7:0/2119593945 shutdown_connections 2024-09-10T21:49:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.918+0000 7f3690200640 1 -- 172.21.15.7:0/2119593945 wait complete. 2024-09-10T21:49:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.919+0000 7f3690200640 1 Processor -- start 2024-09-10T21:49:10.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.920+0000 7f3690200640 1 -- start start 2024-09-10T21:49:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.920+0000 7f3690200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36880ace80 0x7f368814c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.920+0000 7f3690200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f368815b900 0x7f36880740e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.921+0000 7f3690200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368815bce0 0x7f3688156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.921+0000 7f3690200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f36880aed00 con 0x7f368815bce0 2024-09-10T21:49:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.921+0000 7f3690200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f36880aeb80 con 0x7f368815b900 2024-09-10T21:49:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.921+0000 7f3690200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f36880aee80 con 0x7f36880ace80 2024-09-10T21:49:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.921+0000 7f368e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368815bce0 0x7f3688156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.921+0000 7f368e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368815bce0 0x7f3688156800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52350/0 (socket says 172.21.15.7:52350) 2024-09-10T21:49:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.921+0000 7f368e200640 1 -- 172.21.15.7:0/3289815598 learned_addr learned my addr 172.21.15.7:0/3289815598 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.921+0000 7f368d800640 1 --2- 172.21.15.7:0/3289815598 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36880ace80 0x7f368814c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.921+0000 7f368ce00640 1 --2- 172.21.15.7:0/3289815598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f368815b900 0x7f36880740e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.921+0000 7f368e200640 1 -- 172.21.15.7:0/3289815598 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36880ace80 msgr2=0x7f368814c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.921+0000 7f368e200640 1 --2- 172.21.15.7:0/3289815598 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36880ace80 0x7f368814c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.921+0000 7f368e200640 1 -- 172.21.15.7:0/3289815598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f368815b900 msgr2=0x7f36880740e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.921+0000 7f368e200640 1 --2- 172.21.15.7:0/3289815598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f368815b900 0x7f36880740e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.921+0000 7f368e200640 1 -- 172.21.15.7:0/3289815598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36881f4a10 con 0x7f368815bce0 2024-09-10T21:49:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.921+0000 7f368ce00640 1 --2- 172.21.15.7:0/3289815598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f368815b900 0x7f36880740e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.922+0000 7f368e200640 1 --2- 172.21.15.7:0/3289815598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368815bce0 0x7f3688156800 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f3680009d90 tx=0x7f3680003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.922+0000 7f3685600640 1 -- 172.21.15.7:0/3289815598 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680046070 con 0x7f368815bce0 2024-09-10T21:49:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.922+0000 7f3685600640 1 -- 172.21.15.7:0/3289815598 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3680004540 con 0x7f368815bce0 2024-09-10T21:49:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.922+0000 7f3685600640 1 -- 172.21.15.7:0/3289815598 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680033070 con 0x7f368815bce0 2024-09-10T21:49:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.922+0000 7f3690200640 1 -- 172.21.15.7:0/3289815598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36881f4d30 con 0x7f368815bce0 2024-09-10T21:49:10.924 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.922+0000 7f3690200640 1 -- 172.21.15.7:0/3289815598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36881f5570 con 0x7f368815bce0 2024-09-10T21:49:10.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.924+0000 7f3685600640 1 -- 172.21.15.7:0/3289815598 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3680040030 con 0x7f368815bce0 2024-09-10T21:49:10.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.925+0000 7f3685600640 1 --2- 172.21.15.7:0/3289815598 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3668077f70 0x7f366807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:10.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.925+0000 7f3685600640 1 -- 172.21.15.7:0/3289815598 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f36800baf90 con 0x7f368815bce0 2024-09-10T21:49:10.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.926+0000 7f3693221740 1 Processor -- start 2024-09-10T21:49:10.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.926+0000 7f368d800640 1 --2- 172.21.15.7:0/3289815598 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3668077f70 0x7f366807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:10.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.926+0000 7f3693221740 1 -- start start 2024-09-10T21:49:10.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.926+0000 7f368d800640 1 --2- 172.21.15.7:0/3289815598 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3668077f70 0x7f366807a430 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f367c005d00 tx=0x7f367c00b410 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.927+0000 7f3693221740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 0x55a11dcef0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.927+0000 7f3693221740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dca9e80 0x55a11dced7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.927+0000 7f3693221740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dcaa400 0x55a11dcedd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.927+0000 7f3693221740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a11dc1bf40 con 0x55a11dceed10 2024-09-10T21:49:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.927+0000 7f3693221740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a11dc0b370 con 0x55a11dcaa400 2024-09-10T21:49:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.927+0000 7f3693221740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55a11dc0f180 con 0x55a11dca9e80 2024-09-10T21:49:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.927+0000 7f368d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 0x55a11dcef0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.927+0000 7f368d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 0x55a11dcef0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52356/0 (socket says 172.21.15.7:52356) 2024-09-10T21:49:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.927+0000 7f368d800640 1 -- 172.21.15.7:0/2682118701 learned_addr learned my addr 172.21.15.7:0/2682118701 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.927+0000 7f368e200640 1 --2- 172.21.15.7:0/2682118701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dcaa400 0x55a11dcedd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.927+0000 7f368ce00640 1 --2- 172.21.15.7:0/2682118701 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dca9e80 0x55a11dced7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.927+0000 7f368d800640 1 -- 172.21.15.7:0/2682118701 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dca9e80 msgr2=0x55a11dced7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.927+0000 7f368d800640 1 --2- 172.21.15.7:0/2682118701 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dca9e80 0x55a11dced7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.927+0000 7f368d800640 1 -- 172.21.15.7:0/2682118701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dcaa400 msgr2=0x55a11dcedd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.927+0000 7f368d800640 1 --2- 172.21.15.7:0/2682118701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dcaa400 0x55a11dcedd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.927+0000 7f368d800640 1 -- 172.21.15.7:0/2682118701 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a11d8fd470 con 0x55a11dceed10 2024-09-10T21:49:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.928+0000 7f368d800640 1 --2- 172.21.15.7:0/2682118701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 0x55a11dcef0f0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f367c011010 tx=0x7f367c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64acfd451f9d3475 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.928+0000 7f367a000640 1 -- 172.21.15.7:0/2682118701 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c01e620 con 0x55a11dceed10 2024-09-10T21:49:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.928+0000 7f367a000640 1 -- 172.21.15.7:0/2682118701 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f367c01e8b0 con 0x55a11dceed10 2024-09-10T21:49:10.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.928+0000 7f367a000640 1 -- 172.21.15.7:0/2682118701 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c01ec60 con 0x55a11dceed10 2024-09-10T21:49:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.928+0000 7f3693221740 1 -- 172.21.15.7:0/2682118701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 msgr2=0x55a11dcef0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.928+0000 7f3693221740 1 --2- 172.21.15.7:0/2682118701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 0x55a11dcef0f0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f367c011010 tx=0x7f367c013000 comp rx=0 tx=0).stop 2024-09-10T21:49:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.929+0000 7f3693221740 1 -- 172.21.15.7:0/2682118701 shutdown_connections 2024-09-10T21:49:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.929+0000 7f3693221740 1 --2- 172.21.15.7:0/2682118701 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dcaa400 0x55a11dcedd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.929+0000 7f3693221740 1 --2- 172.21.15.7:0/2682118701 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dca9e80 0x55a11dced7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.929+0000 7f3693221740 1 --2- 172.21.15.7:0/2682118701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 0x55a11dcef0f0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.929+0000 7f3693221740 1 -- 172.21.15.7:0/2682118701 >> 172.21.15.7:0/2682118701 conn(0x55a11dca5450 msgr2=0x55a11dca8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:10.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.929+0000 7f3693221740 1 -- 172.21.15.7:0/2682118701 shutdown_connections 2024-09-10T21:49:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.929+0000 7f3693221740 1 -- 172.21.15.7:0/2682118701 wait complete. 2024-09-10T21:49:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.929+0000 7f3693221740 1 -- start start 2024-09-10T21:49:10.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.930+0000 7f3693221740 1 Processor -- start 2024-09-10T21:49:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.931+0000 7f3693221740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dcaa400 0x55a11dca2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.931+0000 7f3693221740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 0x55a11dcedb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.931+0000 7f3693221740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dca2dc0 0x55a11dcbf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.931+0000 7f3693221740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a11d97b7c0 con 0x55a11dceed10 2024-09-10T21:49:10.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.931+0000 7f3693221740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a11dbb0ae0 con 0x55a11dca2dc0 2024-09-10T21:49:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.931+0000 7f3693221740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55a11dca3e70 con 0x55a11dcaa400 2024-09-10T21:49:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.931+0000 7f3693221740 10 client.0 _ll_register_callbacks cb 0x55a11dbc05b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:49:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.931+0000 7f368ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 0x55a11dcedb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.931+0000 7f368ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 0x55a11dcedb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52368/0 (socket says 172.21.15.7:52368) 2024-09-10T21:49:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.931+0000 7f368ce00640 1 -- 172.21.15.7:0/3604284521 learned_addr learned my addr 172.21.15.7:0/3604284521 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.931+0000 7f368d800640 1 --2- 172.21.15.7:0/3604284521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dcaa400 0x55a11dca2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.931+0000 7f368e200640 1 --2- 172.21.15.7:0/3604284521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dca2dc0 0x55a11dcbf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.931+0000 7f368ce00640 1 -- 172.21.15.7:0/3604284521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dcaa400 msgr2=0x55a11dca2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.931+0000 7f368ce00640 1 --2- 172.21.15.7:0/3604284521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dcaa400 0x55a11dca2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.931+0000 7f368ce00640 1 -- 172.21.15.7:0/3604284521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dca2dc0 msgr2=0x55a11dcbf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.931+0000 7f368ce00640 1 --2- 172.21.15.7:0/3604284521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dca2dc0 0x55a11dcbf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.931+0000 7f368ce00640 1 -- 172.21.15.7:0/3604284521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a11d9e17e0 con 0x55a11dceed10 2024-09-10T21:49:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.931+0000 7f368e200640 1 --2- 172.21.15.7:0/3604284521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dca2dc0 0x55a11dcbf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.932+0000 7f368ce00640 1 --2- 172.21.15.7:0/3604284521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 0x55a11dcedb80 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3674003f50 tx=0x7f3674001550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90f0694eeb99535 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:10.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.932+0000 7f366ea00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:49:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.932+0000 7f366ea00640 1 -- 172.21.15.7:0/3604284521 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3674014400 con 0x55a11dceed10 2024-09-10T21:49:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.932+0000 7f366ea00640 1 -- 172.21.15.7:0/3604284521 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f36740145a0 con 0x55a11dceed10 2024-09-10T21:49:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.932+0000 7f3693221740 1 -- 172.21.15.7:0/3604284521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f367c01e8b0 con 0x55a11dceed10 2024-09-10T21:49:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.932+0000 7f366ea00640 1 -- 172.21.15.7:0/3604284521 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f3674021050 con 0x55a11dceed10 2024-09-10T21:49:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.932+0000 7f3693221740 10 client.7293 fetch_fsmap learned FSMap version 76 2024-09-10T21:49:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.932+0000 7f3693221740 1 -- 172.21.15.7:0/3604284521 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55a11da41f70 con 0x55a11dceed10 2024-09-10T21:49:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.932+0000 7f366ea00640 1 -- 172.21.15.7:0/3604284521 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f367c01e8b0 con 0x55a11dceed10 2024-09-10T21:49:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.932+0000 7f366ea00640 1 -- 172.21.15.7:0/3604284521 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7f3674024020 con 0x55a11dceed10 2024-09-10T21:49:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.932+0000 7f3693221740 10 client.7293 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T21:49:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.932+0000 7f3693221740 10 client.7293 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-10T21:49:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.932+0000 7f3693221740 20 client.7293 populate_metadata read hostname 'smithi007' 2024-09-10T21:49:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.932+0000 7f3693221740 1 --2- 172.21.15.7:0/3604284521 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x55a11dcc01a0 0x55a11dceb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:10.936 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.932+0000 7f3693221740 4 client.7293 mds_command: new command op to 236099 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:49:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.932+0000 7f3693221740 1 -- 172.21.15.7:0/3604284521 --> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a11dc1de90 con 0x55a11dcc01a0 2024-09-10T21:49:10.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.933+0000 7f368d800640 1 --2- 172.21.15.7:0/3604284521 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x55a11dcc01a0 0x55a11dceb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.934+0000 7f368d800640 1 --2- 172.21.15.7:0/3604284521 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x55a11dcc01a0 0x55a11dceb680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12d8aff5d4ae1418 server_cookie=9f708c6f2e6815d2 in_seq=0 out_seq=0 2024-09-10T21:49:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.934+0000 7f366ea00640 10 client.7293 ms_handle_connect on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.935+0000 7f366ea00640 1 -- 172.21.15.7:0/3604284521 <== mds.0 v2:172.21.15.133:6835/4188280820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a11dc1de90 con 0x55a11dcc01a0 2024-09-10T21:49:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.935+0000 7f366ea00640 10 client.7293 handle_command_reply: tid=0 2024-09-10T21:49:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.935+0000 7f3693221740 1 client.7293 shutdown 2024-09-10T21:49:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.935+0000 7f3693221740 1 -- 172.21.15.7:0/3604284521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 msgr2=0x55a11dcedb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.935+0000 7f3693221740 1 --2- 172.21.15.7:0/3604284521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 0x55a11dcedb80 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f3674003f50 tx=0x7f3674001550 comp rx=0 tx=0).stop 2024-09-10T21:49:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.935+0000 7f3693221740 1 -- 172.21.15.7:0/3604284521 shutdown_connections 2024-09-10T21:49:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.935+0000 7f3693221740 1 --2- 172.21.15.7:0/3604284521 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x55a11dcc01a0 0x55a11dceb680 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.935+0000 7f3693221740 1 --2- 172.21.15.7:0/3604284521 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dca2dc0 0x55a11dcbf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.935+0000 7f3693221740 1 --2- 172.21.15.7:0/3604284521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 0x55a11dcedb80 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.935+0000 7f3693221740 1 --2- 172.21.15.7:0/3604284521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dcaa400 0x55a11dca2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.935+0000 7f366ea00640 0 client.7293 ms_handle_reset on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.935+0000 7f3693221740 1 -- 172.21.15.7:0/3604284521 >> 172.21.15.7:0/3604284521 conn(0x55a11dca9e80 msgr2=0x55a11dcd2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.935+0000 7f3693221740 1 -- 172.21.15.7:0/3604284521 shutdown_connections 2024-09-10T21:49:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.936+0000 7f3693221740 1 -- 172.21.15.7:0/3604284521 wait complete. 2024-09-10T21:49:10.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.936+0000 7f3693221740 20 client.7293 trim_cache size 0 max 16384 2024-09-10T21:49:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.954+0000 7f3693221740 1 Processor -- start 2024-09-10T21:49:10.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.954+0000 7f3693221740 1 -- start start 2024-09-10T21:49:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.955+0000 7f3693221740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dca2dc0 0x55a11dca8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.955+0000 7f3693221740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dcaa400 0x55a11dca8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.955+0000 7f3693221740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dceed10 0x55a11dccd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.955+0000 7f3693221740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3688201310 con 0x55a11dca2dc0 2024-09-10T21:49:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.955+0000 7f3693221740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3688200380 con 0x55a11dcaa400 2024-09-10T21:49:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.955+0000 7f3693221740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f36881f8ba0 con 0x55a11dceed10 2024-09-10T21:49:10.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.955+0000 7f368d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dca2dc0 0x55a11dca8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.955+0000 7f368d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dca2dc0 0x55a11dca8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52382/0 (socket says 172.21.15.7:52382) 2024-09-10T21:49:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.955+0000 7f368d800640 1 -- 172.21.15.7:0/2612892283 learned_addr learned my addr 172.21.15.7:0/2612892283 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.956+0000 7f368e200640 1 --2- 172.21.15.7:0/2612892283 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dceed10 0x55a11dccd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.956+0000 7f368ce00640 1 --2- 172.21.15.7:0/2612892283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dcaa400 0x55a11dca8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.956+0000 7f368d800640 1 -- 172.21.15.7:0/2612892283 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dceed10 msgr2=0x55a11dccd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.956+0000 7f368d800640 1 --2- 172.21.15.7:0/2612892283 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dceed10 0x55a11dccd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.956+0000 7f368d800640 1 -- 172.21.15.7:0/2612892283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dcaa400 msgr2=0x55a11dca8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.956+0000 7f368d800640 1 --2- 172.21.15.7:0/2612892283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dcaa400 0x55a11dca8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.956+0000 7f368d800640 1 -- 172.21.15.7:0/2612892283 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a11d94a720 con 0x55a11dca2dc0 2024-09-10T21:49:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.956+0000 7f368e200640 1 --2- 172.21.15.7:0/2612892283 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dceed10 0x55a11dccd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.956+0000 7f368d800640 1 --2- 172.21.15.7:0/2612892283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dca2dc0 0x55a11dca8590 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x55a11dcf0390 tx=0x7f367c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d908ca69bc222fc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.956+0000 7f3679600640 1 -- 172.21.15.7:0/2612892283 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c03f070 con 0x55a11dca2dc0 2024-09-10T21:49:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.956+0000 7f3679600640 1 -- 172.21.15.7:0/2612892283 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f367c01eab0 con 0x55a11dca2dc0 2024-09-10T21:49:10.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.956+0000 7f3679600640 1 -- 172.21.15.7:0/2612892283 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c034400 con 0x55a11dca2dc0 2024-09-10T21:49:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.957+0000 7f3693221740 1 -- 172.21.15.7:0/2612892283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dca2dc0 msgr2=0x55a11dca8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.957+0000 7f3693221740 1 --2- 172.21.15.7:0/2612892283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dca2dc0 0x55a11dca8590 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x55a11dcf0390 tx=0x7f367c004730 comp rx=0 tx=0).stop 2024-09-10T21:49:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.957+0000 7f3693221740 1 -- 172.21.15.7:0/2612892283 shutdown_connections 2024-09-10T21:49:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.957+0000 7f3693221740 1 --2- 172.21.15.7:0/2612892283 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dcaa400 0x55a11dca8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.957+0000 7f3693221740 1 --2- 172.21.15.7:0/2612892283 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dceed10 0x55a11dccd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.957+0000 7f3693221740 1 --2- 172.21.15.7:0/2612892283 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dca2dc0 0x55a11dca8590 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.957+0000 7f3693221740 1 -- 172.21.15.7:0/2612892283 >> 172.21.15.7:0/2612892283 conn(0x55a11dca9e80 msgr2=0x55a11dcebfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.957+0000 7f3693221740 1 -- 172.21.15.7:0/2612892283 shutdown_connections 2024-09-10T21:49:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.957+0000 7f3693221740 1 -- 172.21.15.7:0/2612892283 wait complete. 2024-09-10T21:49:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.958+0000 7f3693221740 1 -- start start 2024-09-10T21:49:10.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.958+0000 7f3693221740 1 Processor -- start 2024-09-10T21:49:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.959+0000 7f3693221740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dca2dc0 0x55a11dcffdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.959+0000 7f3693221740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dcaa400 0x55a11dca68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.959+0000 7f3693221740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 0x55a11dca6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.959+0000 7f3693221740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a11dca9aa0 con 0x55a11dceed10 2024-09-10T21:49:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.959+0000 7f3693221740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3688073500 con 0x55a11dcaa400 2024-09-10T21:49:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.959+0000 7f3693221740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55a11dca2760 con 0x55a11dca2dc0 2024-09-10T21:49:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.959+0000 7f3693221740 10 client.0 _ll_register_callbacks cb 0x7f3674005050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:49:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.959+0000 7f368e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 0x55a11dca6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.959+0000 7f368e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 0x55a11dca6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52392/0 (socket says 172.21.15.7:52392) 2024-09-10T21:49:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.959+0000 7f368e200640 1 -- 172.21.15.7:0/3915934310 learned_addr learned my addr 172.21.15.7:0/3915934310 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.959+0000 7f368d800640 1 --2- 172.21.15.7:0/3915934310 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dca2dc0 0x55a11dcffdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.959+0000 7f368ce00640 1 --2- 172.21.15.7:0/3915934310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dcaa400 0x55a11dca68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:10.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.959+0000 7f368e200640 1 -- 172.21.15.7:0/3915934310 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dca2dc0 msgr2=0x55a11dcffdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.959+0000 7f368e200640 1 --2- 172.21.15.7:0/3915934310 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dca2dc0 0x55a11dcffdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.959+0000 7f368e200640 1 -- 172.21.15.7:0/3915934310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dcaa400 msgr2=0x55a11dca68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.959+0000 7f368e200640 1 --2- 172.21.15.7:0/3915934310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dcaa400 0x55a11dca68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.959+0000 7f368e200640 1 -- 172.21.15.7:0/3915934310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a11dca76e0 con 0x55a11dceed10 2024-09-10T21:49:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.959+0000 7f368ce00640 1 --2- 172.21.15.7:0/3915934310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dcaa400 0x55a11dca68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:49:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.960+0000 7f368e200640 1 --2- 172.21.15.7:0/3915934310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 0x55a11dca6e10 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f36800b9350 tx=0x7f3680082c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=132b395728d66d76 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.960+0000 7f366ea00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:49:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.960+0000 7f366ea00640 1 -- 172.21.15.7:0/3915934310 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3680082de0 con 0x55a11dceed10 2024-09-10T21:49:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.960+0000 7f366ea00640 1 -- 172.21.15.7:0/3915934310 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f3680004b40 con 0x55a11dceed10 2024-09-10T21:49:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.960+0000 7f3693221740 1 -- 172.21.15.7:0/3915934310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f367c01eab0 con 0x55a11dceed10 2024-09-10T21:49:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.960+0000 7f366ea00640 1 -- 172.21.15.7:0/3915934310 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f36800832d0 con 0x55a11dceed10 2024-09-10T21:49:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.960+0000 7f3693221740 10 client.7299 fetch_fsmap learned FSMap version 76 2024-09-10T21:49:10.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.960+0000 7f3693221740 1 -- 172.21.15.7:0/3915934310 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x55a11dd6f330 con 0x55a11dceed10 2024-09-10T21:49:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.960+0000 7f366ea00640 1 -- 172.21.15.7:0/3915934310 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f367c01eab0 con 0x55a11dceed10 2024-09-10T21:49:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.960+0000 7f366ea00640 1 -- 172.21.15.7:0/3915934310 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7f368007ec20 con 0x55a11dceed10 2024-09-10T21:49:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.961+0000 7f3693221740 10 client.7299 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T21:49:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.961+0000 7f3693221740 10 client.7299 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-10T21:49:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.961+0000 7f3693221740 20 client.7299 populate_metadata read hostname 'smithi007' 2024-09-10T21:49:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.961+0000 7f3693221740 1 --2- 172.21.15.7:0/3915934310 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x55a11dcecb50 0x55a11dcbe3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.961+0000 7f3693221740 4 client.7299 mds_command: new command op to 236099 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}] 2024-09-10T21:49:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.961+0000 7f3693221740 1 -- 172.21.15.7:0/3915934310 --> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] -- command(tid 0: {"prefix": "scrub start", "path": "/dir", "scrubops": ["recursive", "repair"]}) -- 0x55a11dced3e0 con 0x55a11dcecb50 2024-09-10T21:49:10.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.961+0000 7f368d800640 1 --2- 172.21.15.7:0/3915934310 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x55a11dcecb50 0x55a11dcbe3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:10.966 INFO:tasks.ceph.mds.d.smithi133.stderr:2024-09-10T21:49:10.964+0000 7f7353000640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | request=0 lock=0 dirtyparent=1 authpin=1 scrubqueue=0 0x561bc4424680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-10T21:49:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.962+0000 7f368d800640 1 --2- 172.21.15.7:0/3915934310 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x55a11dcecb50 0x55a11dcbe3d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59cd338df6b3f3c2 server_cookie=3728508192dd180 in_seq=0 out_seq=0 2024-09-10T21:49:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.962+0000 7f366ea00640 10 client.7299 ms_handle_connect on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.962+0000 7f366ea00640 1 -- 172.21.15.7:0/3915934310 <== mds.0 v2:172.21.15.133:6835/4188280820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a11dced3e0 con 0x55a11dcecb50 2024-09-10T21:49:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.962+0000 7f366ea00640 10 client.7299 handle_command_reply: tid=0 2024-09-10T21:49:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.962+0000 7f3693221740 1 client.7299 shutdown 2024-09-10T21:49:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.962+0000 7f3693221740 1 -- 172.21.15.7:0/3915934310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 msgr2=0x55a11dca6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.962+0000 7f3693221740 1 --2- 172.21.15.7:0/3915934310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 0x55a11dca6e10 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f36800b9350 tx=0x7f3680082c20 comp rx=0 tx=0).stop 2024-09-10T21:49:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.962+0000 7f3693221740 1 -- 172.21.15.7:0/3915934310 shutdown_connections 2024-09-10T21:49:10.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.962+0000 7f3693221740 1 --2- 172.21.15.7:0/3915934310 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x55a11dcecb50 0x55a11dcbe3d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.962+0000 7f3693221740 1 --2- 172.21.15.7:0/3915934310 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a11dcaa400 0x55a11dca68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.962+0000 7f3693221740 1 --2- 172.21.15.7:0/3915934310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a11dceed10 0x55a11dca6e10 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.962+0000 7f3693221740 1 --2- 172.21.15.7:0/3915934310 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a11dca2dc0 0x55a11dcffdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.962+0000 7f3693221740 1 -- 172.21.15.7:0/3915934310 >> 172.21.15.7:0/3915934310 conn(0x55a11dca9e80 msgr2=0x55a11dccfb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.962+0000 7f366ea00640 0 client.7299 ms_handle_reset on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.963+0000 7f3693221740 1 -- 172.21.15.7:0/3915934310 shutdown_connections 2024-09-10T21:49:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.963+0000 7f3693221740 1 -- 172.21.15.7:0/3915934310 wait complete. 2024-09-10T21:49:10.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.963+0000 7f3693221740 20 client.7299 trim_cache size 0 max 16384 2024-09-10T21:49:10.968 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:49:10.968 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-09-10T21:49:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "2edd59b4-4331-4f8c-9518-8e5c0af9cba2", 2024-09-10T21:49:10.969 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-09-10T21:49:10.969 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:49:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.963+0000 7f3690200640 1 -- 172.21.15.7:0/3289815598 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3668077f70 msgr2=0x7f366807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:10.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.963+0000 7f3690200640 1 --2- 172.21.15.7:0/3289815598 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3668077f70 0x7f366807a430 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f367c005d00 tx=0x7f367c00b410 comp rx=0 tx=0).stop 2024-09-10T21:49:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.963+0000 7f3690200640 1 -- 172.21.15.7:0/3289815598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368815bce0 msgr2=0x7f3688156800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.963+0000 7f3690200640 1 --2- 172.21.15.7:0/3289815598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368815bce0 0x7f3688156800 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f3680009d90 tx=0x7f3680003f90 comp rx=0 tx=0).stop 2024-09-10T21:49:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.963+0000 7f3690200640 1 -- 172.21.15.7:0/3289815598 shutdown_connections 2024-09-10T21:49:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.963+0000 7f3690200640 1 --2- 172.21.15.7:0/3289815598 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3668077f70 0x7f366807a430 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.963+0000 7f3690200640 1 --2- 172.21.15.7:0/3289815598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f368815b900 0x7f36880740e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.963+0000 7f3690200640 1 --2- 172.21.15.7:0/3289815598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f368815bce0 0x7f3688156800 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.963+0000 7f3690200640 1 --2- 172.21.15.7:0/3289815598 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f36880ace80 0x7f368814c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.963+0000 7f3690200640 1 -- 172.21.15.7:0/3289815598 >> 172.21.15.7:0/3289815598 conn(0x7f36880b1c10 msgr2=0x7f368815a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.964+0000 7f3690200640 1 -- 172.21.15.7:0/3289815598 shutdown_connections 2024-09-10T21:49:10.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:10.964+0000 7f3690200640 1 -- 172.21.15.7:0/3289815598 wait complete. 2024-09-10T21:49:10.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:49:10.980 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-10T21:49:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.143+0000 7fa008a00640 1 -- 172.21.15.7:0/4189153454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa004161ab0 msgr2=0x7fa0040cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.143+0000 7fa008a00640 1 --2- 172.21.15.7:0/4189153454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa004161ab0 0x7fa0040cc530 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9fec009860 tx=0x7f9fec02a580 comp rx=0 tx=0).stop 2024-09-10T21:49:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.143+0000 7fa008a00640 1 -- 172.21.15.7:0/4189153454 shutdown_connections 2024-09-10T21:49:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.143+0000 7fa008a00640 1 --2- 172.21.15.7:0/4189153454 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa004161ab0 0x7fa0040cc530 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.143+0000 7fa008a00640 1 -- 172.21.15.7:0/4189153454 >> 172.21.15.7:0/4189153454 conn(0x7fa0040b16d0 msgr2=0x7fa0040b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.143+0000 7fa008a00640 1 -- 172.21.15.7:0/4189153454 shutdown_connections 2024-09-10T21:49:11.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.143+0000 7fa008a00640 1 -- 172.21.15.7:0/4189153454 wait complete. 2024-09-10T21:49:11.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.145+0000 7fa008a00640 1 Processor -- start 2024-09-10T21:49:11.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.145+0000 7fa008a00640 1 -- start start 2024-09-10T21:49:11.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.146+0000 7fa008a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0040af100 0x7fa0041f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:11.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.146+0000 7fa008a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa00415ba00 0x7fa0040c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:11.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.146+0000 7fa008a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0041f5190 0x7fa004148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:11.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.146+0000 7fa008a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa0040cce00 con 0x7fa00415ba00 2024-09-10T21:49:11.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.146+0000 7fa008a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa0040ccc80 con 0x7fa0041f5190 2024-09-10T21:49:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.146+0000 7fa008a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa0040aef10 con 0x7fa0040af100 2024-09-10T21:49:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.146+0000 7f9ff9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa00415ba00 0x7fa0040c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.146+0000 7f9ff9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa00415ba00 0x7fa0040c6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52400/0 (socket says 172.21.15.7:52400) 2024-09-10T21:49:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.146+0000 7f9ff9400640 1 -- 172.21.15.7:0/2768672783 learned_addr learned my addr 172.21.15.7:0/2768672783 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.146+0000 7fa001e00640 1 --2- 172.21.15.7:0/2768672783 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0040af100 0x7fa0041f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.146+0000 7fa002800640 1 --2- 172.21.15.7:0/2768672783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0041f5190 0x7fa004148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.147+0000 7f9ff9400640 1 -- 172.21.15.7:0/2768672783 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0040af100 msgr2=0x7fa0041f7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.147+0000 7f9ff9400640 1 --2- 172.21.15.7:0/2768672783 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0040af100 0x7fa0041f7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.147+0000 7f9ff9400640 1 -- 172.21.15.7:0/2768672783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0041f5190 msgr2=0x7fa004148990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.147+0000 7f9ff9400640 1 --2- 172.21.15.7:0/2768672783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0041f5190 0x7fa004148990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.147+0000 7f9ff9400640 1 -- 172.21.15.7:0/2768672783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0041f8d80 con 0x7fa00415ba00 2024-09-10T21:49:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.147+0000 7fa001e00640 1 --2- 172.21.15.7:0/2768672783 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0040af100 0x7fa0041f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:11.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.147+0000 7fa002800640 1 --2- 172.21.15.7:0/2768672783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0041f5190 0x7fa004148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.147+0000 7f9ff9400640 1 --2- 172.21.15.7:0/2768672783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa00415ba00 0x7fa0040c6240 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f9fec02ab70 tx=0x7f9fec003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.147+0000 7f9ffaa00640 1 -- 172.21.15.7:0/2768672783 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec005c90 con 0x7fa00415ba00 2024-09-10T21:49:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.147+0000 7f9ffaa00640 1 -- 172.21.15.7:0/2768672783 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9fec005330 con 0x7fa00415ba00 2024-09-10T21:49:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.147+0000 7f9ffaa00640 1 -- 172.21.15.7:0/2768672783 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec0047a0 con 0x7fa00415ba00 2024-09-10T21:49:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.147+0000 7fa008a00640 1 -- 172.21.15.7:0/2768672783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0041f84e0 con 0x7fa00415ba00 2024-09-10T21:49:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.147+0000 7fa008a00640 1 -- 172.21.15.7:0/2768672783 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0041f8aa0 con 0x7fa00415ba00 2024-09-10T21:49:11.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.150+0000 7f9ffaa00640 1 -- 172.21.15.7:0/2768672783 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9fec02ace0 con 0x7fa00415ba00 2024-09-10T21:49:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.150+0000 7f9ffaa00640 1 --2- 172.21.15.7:0/2768672783 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9fe0077fa0 0x7f9fe007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:11.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.151+0000 7f9ffaa00640 1 -- 172.21.15.7:0/2768672783 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f9fec0baa20 con 0x7fa00415ba00 2024-09-10T21:49:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.151+0000 7fa00b8cc740 1 Processor -- start 2024-09-10T21:49:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.151+0000 7fa00b8cc740 1 -- start start 2024-09-10T21:49:11.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.151+0000 7fa001e00640 1 --2- 172.21.15.7:0/2768672783 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9fe0077fa0 0x7f9fe007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.152+0000 7fa00b8cc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b54d130 0x56145b54d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.152+0000 7fa00b8cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b5082a0 0x56145b54bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.152+0000 7fa00b8cc740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b508820 0x56145b54c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.152+0000 7fa00b8cc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56145b4671c0 con 0x56145b54d130 2024-09-10T21:49:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.152+0000 7fa00b8cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56145b47a870 con 0x56145b5082a0 2024-09-10T21:49:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.152+0000 7fa00b8cc740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x56145b469ca0 con 0x56145b508820 2024-09-10T21:49:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.152+0000 7fa001e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b54d130 0x56145b54d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:11.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.152+0000 7fa001e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b54d130 0x56145b54d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52406/0 (socket says 172.21.15.7:52406) 2024-09-10T21:49:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.152+0000 7fa001e00640 1 -- 172.21.15.7:0/272689039 learned_addr learned my addr 172.21.15.7:0/272689039 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.152+0000 7fa001e00640 1 --2- 172.21.15.7:0/2768672783 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9fe0077fa0 0x7f9fe007a460 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4001220 tx=0x7f9ff4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.152+0000 7f9ff9400640 1 --2- 172.21.15.7:0/272689039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b5082a0 0x56145b54bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.152+0000 7fa002800640 1 --2- 172.21.15.7:0/272689039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b508820 0x56145b54c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.153+0000 7fa001e00640 1 -- 172.21.15.7:0/272689039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b508820 msgr2=0x56145b54c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.153+0000 7fa001e00640 1 --2- 172.21.15.7:0/272689039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b508820 0x56145b54c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.153+0000 7fa001e00640 1 -- 172.21.15.7:0/272689039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b5082a0 msgr2=0x56145b54bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.153+0000 7fa001e00640 1 --2- 172.21.15.7:0/272689039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b5082a0 0x56145b54bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.153+0000 7fa001e00640 1 -- 172.21.15.7:0/272689039 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56145b2427d0 con 0x56145b54d130 2024-09-10T21:49:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.153+0000 7fa002800640 1 --2- 172.21.15.7:0/272689039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b508820 0x56145b54c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.153+0000 7fa001e00640 1 --2- 172.21.15.7:0/272689039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b54d130 0x56145b54d510 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400e930 tx=0x7f9ff4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9fdf0e7f5c9e62 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.153+0000 7f9ff2600640 1 -- 172.21.15.7:0/272689039 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff4025070 con 0x56145b54d130 2024-09-10T21:49:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.153+0000 7f9ff2600640 1 -- 172.21.15.7:0/272689039 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ff4015bc0 con 0x56145b54d130 2024-09-10T21:49:11.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.153+0000 7f9ff2600640 1 -- 172.21.15.7:0/272689039 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ff4020400 con 0x56145b54d130 2024-09-10T21:49:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.154+0000 7fa00b8cc740 1 -- 172.21.15.7:0/272689039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b54d130 msgr2=0x56145b54d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.154+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/272689039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b54d130 0x56145b54d510 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f9ff400e930 tx=0x7f9ff4001e70 comp rx=0 tx=0).stop 2024-09-10T21:49:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.154+0000 7fa00b8cc740 1 -- 172.21.15.7:0/272689039 shutdown_connections 2024-09-10T21:49:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.154+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/272689039 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b5082a0 0x56145b54bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.154+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/272689039 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b508820 0x56145b54c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.154+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/272689039 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b54d130 0x56145b54d510 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.154+0000 7fa00b8cc740 1 -- 172.21.15.7:0/272689039 >> 172.21.15.7:0/272689039 conn(0x56145b503870 msgr2=0x56145b507290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.154+0000 7fa00b8cc740 1 -- 172.21.15.7:0/272689039 shutdown_connections 2024-09-10T21:49:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.154+0000 7fa00b8cc740 1 -- 172.21.15.7:0/272689039 wait complete. 2024-09-10T21:49:11.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.155+0000 7fa00b8cc740 1 -- start start 2024-09-10T21:49:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.155+0000 7fa00b8cc740 1 Processor -- start 2024-09-10T21:49:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.156+0000 7fa00b8cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b508820 0x56145b546440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.156+0000 7fa00b8cc740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b54d130 0x56145b54c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.156+0000 7fa00b8cc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b5319c0 0x56145b530fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.156+0000 7fa00b8cc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56145b410410 con 0x56145b5319c0 2024-09-10T21:49:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.156+0000 7fa00b8cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56145b46dab0 con 0x56145b508820 2024-09-10T21:49:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.156+0000 7fa00b8cc740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x56145b1da740 con 0x56145b54d130 2024-09-10T21:49:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.156+0000 7fa00b8cc740 10 client.0 _ll_register_callbacks cb 0x56145b3ed360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:49:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.156+0000 7fa002800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b5319c0 0x56145b530fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.156+0000 7fa002800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b5319c0 0x56145b530fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52416/0 (socket says 172.21.15.7:52416) 2024-09-10T21:49:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.156+0000 7fa002800640 1 -- 172.21.15.7:0/895594439 learned_addr learned my addr 172.21.15.7:0/895594439 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.156+0000 7fa001e00640 1 --2- 172.21.15.7:0/895594439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b508820 0x56145b546440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.156+0000 7f9ff9400640 1 --2- 172.21.15.7:0/895594439 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b54d130 0x56145b54c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:11.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.157+0000 7fa002800640 1 -- 172.21.15.7:0/895594439 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b54d130 msgr2=0x56145b54c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.157+0000 7fa002800640 1 --2- 172.21.15.7:0/895594439 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b54d130 0x56145b54c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.157+0000 7fa002800640 1 -- 172.21.15.7:0/895594439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b508820 msgr2=0x56145b546440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.157+0000 7fa002800640 1 --2- 172.21.15.7:0/895594439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b508820 0x56145b546440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.157+0000 7fa002800640 1 -- 172.21.15.7:0/895594439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56145b1cb970 con 0x56145b5319c0 2024-09-10T21:49:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.157+0000 7fa002800640 1 --2- 172.21.15.7:0/895594439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b5319c0 0x56145b530fd0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc005940 tx=0x7f9ffc001b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a02d66ad88c0e72 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:11.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.157+0000 7f9fe6a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:49:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.157+0000 7f9fe6a00640 1 -- 172.21.15.7:0/895594439 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc012590 con 0x56145b5319c0 2024-09-10T21:49:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.157+0000 7f9fe6a00640 1 -- 172.21.15.7:0/895594439 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f9ffc012730 con 0x56145b5319c0 2024-09-10T21:49:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.157+0000 7fa00b8cc740 1 -- 172.21.15.7:0/895594439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ff4015bc0 con 0x56145b5319c0 2024-09-10T21:49:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.158+0000 7f9fe6a00640 1 -- 172.21.15.7:0/895594439 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f9ffc021050 con 0x56145b5319c0 2024-09-10T21:49:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.158+0000 7fa00b8cc740 10 client.7311 fetch_fsmap learned FSMap version 76 2024-09-10T21:49:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.158+0000 7fa00b8cc740 1 -- 172.21.15.7:0/895594439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x56145b1a96a0 con 0x56145b5319c0 2024-09-10T21:49:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.158+0000 7f9fe6a00640 1 -- 172.21.15.7:0/895594439 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ff4015bc0 con 0x56145b5319c0 2024-09-10T21:49:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.158+0000 7f9fe6a00640 1 -- 172.21.15.7:0/895594439 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7f9ffc024020 con 0x56145b5319c0 2024-09-10T21:49:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.158+0000 7fa00b8cc740 10 client.7311 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T21:49:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.158+0000 7fa00b8cc740 10 client.7311 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-10T21:49:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.158+0000 7fa00b8cc740 20 client.7311 populate_metadata read hostname 'smithi007' 2024-09-10T21:49:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.158+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/895594439 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x56145b51c560 0x56145b545310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.158+0000 7fa00b8cc740 4 client.7311 mds_command: new command op to 236099 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:49:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.158+0000 7fa00b8cc740 1 -- 172.21.15.7:0/895594439 --> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56145b47c7c0 con 0x56145b51c560 2024-09-10T21:49:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.159+0000 7fa001e00640 1 --2- 172.21.15.7:0/895594439 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x56145b51c560 0x56145b545310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:11.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.159+0000 7fa001e00640 1 --2- 172.21.15.7:0/895594439 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x56145b51c560 0x56145b545310 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b74196ec89336f07 server_cookie=e922d055b4afba7d in_seq=0 out_seq=0 2024-09-10T21:49:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.159+0000 7f9fe6a00640 10 client.7311 ms_handle_connect on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.160+0000 7f9fe6a00640 1 -- 172.21.15.7:0/895594439 <== mds.0 v2:172.21.15.133:6835/4188280820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56145b47c7c0 con 0x56145b51c560 2024-09-10T21:49:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.160+0000 7f9fe6a00640 10 client.7311 handle_command_reply: tid=0 2024-09-10T21:49:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.160+0000 7fa00b8cc740 1 client.7311 shutdown 2024-09-10T21:49:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.160+0000 7fa00b8cc740 1 -- 172.21.15.7:0/895594439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b5319c0 msgr2=0x56145b530fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.160+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/895594439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b5319c0 0x56145b530fd0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f9ffc005940 tx=0x7f9ffc001b00 comp rx=0 tx=0).stop 2024-09-10T21:49:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.161+0000 7fa00b8cc740 1 -- 172.21.15.7:0/895594439 shutdown_connections 2024-09-10T21:49:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.161+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/895594439 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x56145b51c560 0x56145b545310 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.161+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/895594439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b5319c0 0x56145b530fd0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.161+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/895594439 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b54d130 0x56145b54c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.161+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/895594439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b508820 0x56145b546440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.161+0000 7fa00b8cc740 1 -- 172.21.15.7:0/895594439 >> 172.21.15.7:0/895594439 conn(0x56145b5082a0 msgr2=0x56145b502440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.161+0000 7f9fe6a00640 0 client.7311 ms_handle_reset on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.161+0000 7fa00b8cc740 1 -- 172.21.15.7:0/895594439 shutdown_connections 2024-09-10T21:49:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.161+0000 7fa00b8cc740 1 -- 172.21.15.7:0/895594439 wait complete. 2024-09-10T21:49:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.161+0000 7fa00b8cc740 20 client.7311 trim_cache size 0 max 16384 2024-09-10T21:49:11.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.179+0000 7fa00b8cc740 1 Processor -- start 2024-09-10T21:49:11.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.179+0000 7fa00b8cc740 1 -- start start 2024-09-10T21:49:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.180+0000 7fa00b8cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b508820 0x56145b501c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.180+0000 7fa00b8cc740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b5319c0 0x56145b5023b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.180+0000 7fa00b8cc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b54d130 0x56145b5028f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.180+0000 7fa00b8cc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa0042012c0 con 0x56145b54d130 2024-09-10T21:49:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.180+0000 7fa00b8cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa004073620 con 0x56145b508820 2024-09-10T21:49:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.180+0000 7fa00b8cc740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa0040b50a0 con 0x56145b5319c0 2024-09-10T21:49:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.181+0000 7fa002800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b54d130 0x56145b5028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.181+0000 7fa002800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b54d130 0x56145b5028f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52430/0 (socket says 172.21.15.7:52430) 2024-09-10T21:49:11.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.181+0000 7fa002800640 1 -- 172.21.15.7:0/1892082796 learned_addr learned my addr 172.21.15.7:0/1892082796 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.181+0000 7fa001e00640 1 --2- 172.21.15.7:0/1892082796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b508820 0x56145b501c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.181+0000 7f9ff9400640 1 --2- 172.21.15.7:0/1892082796 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b5319c0 0x56145b5023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.181+0000 7fa002800640 1 -- 172.21.15.7:0/1892082796 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b5319c0 msgr2=0x56145b5023b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.181+0000 7fa002800640 1 --2- 172.21.15.7:0/1892082796 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b5319c0 0x56145b5023b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.181+0000 7fa002800640 1 -- 172.21.15.7:0/1892082796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b508820 msgr2=0x56145b501c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.181+0000 7fa002800640 1 --2- 172.21.15.7:0/1892082796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b508820 0x56145b501c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.181+0000 7fa002800640 1 -- 172.21.15.7:0/1892082796 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56145b51beb0 con 0x56145b54d130 2024-09-10T21:49:11.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.181+0000 7fa002800640 1 --2- 172.21.15.7:0/1892082796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b54d130 0x56145b5028f0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x56145b542a30 tx=0x7f9ffc003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f92ecb08f97c2ad server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.182+0000 7f9ff1c00640 1 -- 172.21.15.7:0/1892082796 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc029030 con 0x56145b54d130 2024-09-10T21:49:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.182+0000 7f9ff1c00640 1 -- 172.21.15.7:0/1892082796 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ffc019070 con 0x56145b54d130 2024-09-10T21:49:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.182+0000 7f9ff1c00640 1 -- 172.21.15.7:0/1892082796 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ffc0011c0 con 0x56145b54d130 2024-09-10T21:49:11.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.182+0000 7fa00b8cc740 1 -- 172.21.15.7:0/1892082796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b54d130 msgr2=0x56145b5028f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:11.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.182+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/1892082796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b54d130 0x56145b5028f0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x56145b542a30 tx=0x7f9ffc003e70 comp rx=0 tx=0).stop 2024-09-10T21:49:11.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.182+0000 7fa00b8cc740 1 -- 172.21.15.7:0/1892082796 shutdown_connections 2024-09-10T21:49:11.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.182+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/1892082796 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b54d130 0x56145b5028f0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.182+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/1892082796 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b5319c0 0x56145b5023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.182+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/1892082796 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b508820 0x56145b501c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.182+0000 7fa00b8cc740 1 -- 172.21.15.7:0/1892082796 >> 172.21.15.7:0/1892082796 conn(0x56145b5082a0 msgr2=0x56145b541fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:11.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.182+0000 7fa00b8cc740 1 -- 172.21.15.7:0/1892082796 shutdown_connections 2024-09-10T21:49:11.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.182+0000 7fa00b8cc740 1 -- 172.21.15.7:0/1892082796 wait complete. 2024-09-10T21:49:11.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.183+0000 7fa00b8cc740 1 -- start start 2024-09-10T21:49:11.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.183+0000 7fa00b8cc740 1 Processor -- start 2024-09-10T21:49:11.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.184+0000 7fa00b8cc740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b508820 0x56145b5c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:11.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.184+0000 7fa00b8cc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b5319c0 0x56145b56ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:11.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.184+0000 7fa00b8cc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b54d130 0x56145b5007c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:11.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.184+0000 7fa00b8cc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56145b52edb0 con 0x56145b5319c0 2024-09-10T21:49:11.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.184+0000 7fa00b8cc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa0040b56f0 con 0x56145b54d130 2024-09-10T21:49:11.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.184+0000 7fa00b8cc740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x56145b546440 con 0x56145b508820 2024-09-10T21:49:11.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.184+0000 7fa00b8cc740 10 client.0 _ll_register_callbacks cb 0x7f9ffc003bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:49:11.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.184+0000 7f9ff9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b5319c0 0x56145b56ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:11.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.184+0000 7f9ff9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b5319c0 0x56145b56ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:52434/0 (socket says 172.21.15.7:52434) 2024-09-10T21:49:11.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.184+0000 7f9ff9400640 1 -- 172.21.15.7:0/3058968831 learned_addr learned my addr 172.21.15.7:0/3058968831 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:11.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.184+0000 7fa001e00640 1 --2- 172.21.15.7:0/3058968831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b508820 0x56145b5c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:11.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.184+0000 7fa002800640 1 --2- 172.21.15.7:0/3058968831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b54d130 0x56145b5007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:11.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.185+0000 7f9ff9400640 1 -- 172.21.15.7:0/3058968831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b508820 msgr2=0x56145b5c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:11.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.185+0000 7f9ff9400640 1 --2- 172.21.15.7:0/3058968831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b508820 0x56145b5c39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.185+0000 7f9ff9400640 1 -- 172.21.15.7:0/3058968831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b54d130 msgr2=0x56145b5007c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:11.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.185+0000 7f9ff9400640 1 --2- 172.21.15.7:0/3058968831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b54d130 0x56145b5007c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.185+0000 7f9ff9400640 1 -- 172.21.15.7:0/3058968831 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56145b5015b0 con 0x56145b5319c0 2024-09-10T21:49:11.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.185+0000 7fa002800640 1 --2- 172.21.15.7:0/3058968831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b54d130 0x56145b5007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:11.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.185+0000 7f9ff9400640 1 --2- 172.21.15.7:0/3058968831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b5319c0 0x56145b56ecd0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f9fec082cd0 tx=0x7f9fec0832e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a019473136bf5893 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:11.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.185+0000 7f9fe6a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:49:11.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.185+0000 7f9fe6a00640 1 -- 172.21.15.7:0/3058968831 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fec048040 con 0x56145b5319c0 2024-09-10T21:49:11.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.185+0000 7f9fe6a00640 1 -- 172.21.15.7:0/3058968831 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f9fec08bfb0 con 0x56145b5319c0 2024-09-10T21:49:11.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.185+0000 7fa00b8cc740 1 -- 172.21.15.7:0/3058968831 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ffc019070 con 0x56145b5319c0 2024-09-10T21:49:11.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.185+0000 7f9fe6a00640 1 -- 172.21.15.7:0/3058968831 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f9fec082d40 con 0x56145b5319c0 2024-09-10T21:49:11.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.185+0000 7fa00b8cc740 10 client.7317 fetch_fsmap learned FSMap version 76 2024-09-10T21:49:11.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.185+0000 7fa00b8cc740 1 -- 172.21.15.7:0/3058968831 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x56145b5051a0 con 0x56145b5319c0 2024-09-10T21:49:11.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.186+0000 7f9fe6a00640 1 -- 172.21.15.7:0/3058968831 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ffc019070 con 0x56145b5319c0 2024-09-10T21:49:11.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.186+0000 7f9fe6a00640 1 -- 172.21.15.7:0/3058968831 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7f9fec09cbd0 con 0x56145b5319c0 2024-09-10T21:49:11.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.186+0000 7fa00b8cc740 10 client.7317 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T21:49:11.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.186+0000 7fa00b8cc740 10 client.7317 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-10T21:49:11.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.186+0000 7fa00b8cc740 20 client.7317 populate_metadata read hostname 'smithi007' 2024-09-10T21:49:11.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.186+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/3058968831 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x56145b56f210 0x56145b500d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:11.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.186+0000 7fa00b8cc740 4 client.7317 mds_command: new command op to 236099 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:49:11.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.186+0000 7fa00b8cc740 1 -- 172.21.15.7:0/3058968831 --> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56145b478270 con 0x56145b56f210 2024-09-10T21:49:11.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.186+0000 7fa001e00640 1 --2- 172.21.15.7:0/3058968831 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x56145b56f210 0x56145b500d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:11.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.187+0000 7fa001e00640 1 --2- 172.21.15.7:0/3058968831 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x56145b56f210 0x56145b500d00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cea66403e14a599e server_cookie=43b567fe66a85578 in_seq=0 out_seq=0 2024-09-10T21:49:11.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.187+0000 7f9fe6a00640 10 client.7317 ms_handle_connect on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:11.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.187+0000 7f9fe6a00640 1 -- 172.21.15.7:0/3058968831 <== mds.0 v2:172.21.15.133:6835/4188280820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+577 (crc 0 0 0) 0x56145b478270 con 0x56145b56f210 2024-09-10T21:49:11.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.187+0000 7f9fe6a00640 10 client.7317 handle_command_reply: tid=0 2024-09-10T21:49:11.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.187+0000 7fa00b8cc740 1 client.7317 shutdown 2024-09-10T21:49:11.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.187+0000 7fa00b8cc740 1 -- 172.21.15.7:0/3058968831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b5319c0 msgr2=0x56145b56ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:11.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.187+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/3058968831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b5319c0 0x56145b56ecd0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f9fec082cd0 tx=0x7f9fec0832e0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.187+0000 7fa00b8cc740 1 -- 172.21.15.7:0/3058968831 shutdown_connections 2024-09-10T21:49:11.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.187+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/3058968831 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x56145b56f210 0x56145b500d00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.187+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/3058968831 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56145b54d130 0x56145b5007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.187+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/3058968831 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56145b5319c0 0x56145b56ecd0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.187+0000 7fa00b8cc740 1 --2- 172.21.15.7:0/3058968831 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56145b508820 0x56145b5c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.187+0000 7fa00b8cc740 1 -- 172.21.15.7:0/3058968831 >> 172.21.15.7:0/3058968831 conn(0x56145b5082a0 msgr2=0x56145b54c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:11.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.187+0000 7f9fe6a00640 0 client.7317 ms_handle_reset on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:11.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.188+0000 7fa00b8cc740 1 -- 172.21.15.7:0/3058968831 shutdown_connections 2024-09-10T21:49:11.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.188+0000 7fa00b8cc740 1 -- 172.21.15.7:0/3058968831 wait complete. 2024-09-10T21:49:11.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.188+0000 7fa00b8cc740 20 client.7317 trim_cache size 0 max 16384 2024-09-10T21:49:11.193 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:49:11.193 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:49:11.193 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-09-10T21:49:11.193 INFO:teuthology.orchestra.run.smithi007.stdout: "2edd59b4-4331-4f8c-9518-8e5c0af9cba2": { 2024-09-10T21:49:11.193 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/dir", 2024-09-10T21:49:11.193 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "2edd59b4-4331-4f8c-9518-8e5c0af9cba2", 2024-09-10T21:49:11.193 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,repair" 2024-09-10T21:49:11.194 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:11.194 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:11.194 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:49:11.194 INFO:teuthology.orchestra.run.smithi007.stdout: "2edd59b4-4331-4f8c-9518-8e5c0af9cba2": { 2024-09-10T21:49:11.194 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:49:10.961935+0000", 2024-09-10T21:49:11.194 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/dir", 2024-09-10T21:49:11.194 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:49:11.194 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:49:11.194 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:49:11.194 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:49:11.194 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:11.194 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:11.194 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:49:11.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.188+0000 7fa008a00640 1 -- 172.21.15.7:0/2768672783 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9fe0077fa0 msgr2=0x7f9fe007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:11.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.188+0000 7fa008a00640 1 --2- 172.21.15.7:0/2768672783 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9fe0077fa0 0x7f9fe007a460 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9ff4001220 tx=0x7f9ff4009040 comp rx=0 tx=0).stop 2024-09-10T21:49:11.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.188+0000 7fa008a00640 1 -- 172.21.15.7:0/2768672783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa00415ba00 msgr2=0x7fa0040c6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:11.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.188+0000 7fa008a00640 1 --2- 172.21.15.7:0/2768672783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa00415ba00 0x7fa0040c6240 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f9fec02ab70 tx=0x7f9fec003c10 comp rx=0 tx=0).stop 2024-09-10T21:49:11.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.188+0000 7fa008a00640 1 -- 172.21.15.7:0/2768672783 shutdown_connections 2024-09-10T21:49:11.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.188+0000 7fa008a00640 1 --2- 172.21.15.7:0/2768672783 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9fe0077fa0 0x7f9fe007a460 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.188+0000 7fa008a00640 1 --2- 172.21.15.7:0/2768672783 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa0041f5190 0x7fa004148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.188+0000 7fa008a00640 1 --2- 172.21.15.7:0/2768672783 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa00415ba00 0x7fa0040c6240 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.188+0000 7fa008a00640 1 --2- 172.21.15.7:0/2768672783 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa0040af100 0x7fa0041f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:11.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.188+0000 7fa008a00640 1 -- 172.21.15.7:0/2768672783 >> 172.21.15.7:0/2768672783 conn(0x7fa0040b16d0 msgr2=0x7fa00415c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:11.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.189+0000 7fa008a00640 1 -- 172.21.15.7:0/2768672783 shutdown_connections 2024-09-10T21:49:11.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:11.189+0000 7fa008a00640 1 -- 172.21.15.7:0/2768672783 wait complete. 2024-09-10T21:49:11.205 INFO:tasks.cephfs.filesystem:scrub status for tag:2edd59b4-4331-4f8c-9518-8e5c0af9cba2 - {'path': '/dir', 'tag': '2edd59b4-4331-4f8c-9518-8e5c0af9cba2', 'options': 'recursive,repair'} 2024-09-10T21:49:31.190 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:49:31.196 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:49:41.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:49:41.206 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub status 2024-09-10T21:49:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.383+0000 7ff5b4000640 1 -- 172.21.15.7:0/1210265235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5ac15bcc0 msgr2=0x7ff5ac06f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.383+0000 7ff5b4000640 1 --2- 172.21.15.7:0/1210265235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5ac15bcc0 0x7ff5ac06f6b0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac0cf9a0 tx=0x7ff5a002a4d0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.383+0000 7ff5b4000640 1 -- 172.21.15.7:0/1210265235 shutdown_connections 2024-09-10T21:49:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.383+0000 7ff5b4000640 1 --2- 172.21.15.7:0/1210265235 >> v2:172.21.15.133:3300/0 conn(0x7ff5ac0ca170 0x7ff5ac0ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.383+0000 7ff5b4000640 1 --2- 172.21.15.7:0/1210265235 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5ac15bcc0 0x7ff5ac06f6b0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.383+0000 7ff5b4000640 1 -- 172.21.15.7:0/1210265235 >> 172.21.15.7:0/1210265235 conn(0x7ff5ac0b16d0 msgr2=0x7ff5ac0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.384+0000 7ff5b4000640 1 -- 172.21.15.7:0/1210265235 shutdown_connections 2024-09-10T21:49:41.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.384+0000 7ff5b4000640 1 -- 172.21.15.7:0/1210265235 wait complete. 2024-09-10T21:49:41.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.385+0000 7ff5b4000640 1 Processor -- start 2024-09-10T21:49:41.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.385+0000 7ff5b4000640 1 -- start start 2024-09-10T21:49:41.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.386+0000 7ff5b4000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5ac070040 0x7ff5ac14c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.386+0000 7ff5b4000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5ac0ca170 0x7ff5ac0cf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.386+0000 7ff5b4000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff5ac15bcc0 0x7ff5ac0ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.386+0000 7ff5b4000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac0b2d60 con 0x7ff5ac0ca170 2024-09-10T21:49:41.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.386+0000 7ff5b4000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac0b2be0 con 0x7ff5ac070040 2024-09-10T21:49:41.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.386+0000 7ff5b4000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ff5ac0b2ee0 con 0x7ff5ac15bcc0 2024-09-10T21:49:41.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.386+0000 7ff5b0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5ac0ca170 0x7ff5ac0cf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.386+0000 7ff5b0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5ac0ca170 0x7ff5ac0cf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47814/0 (socket says 172.21.15.7:47814) 2024-09-10T21:49:41.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.386+0000 7ff5b0c00640 1 -- 172.21.15.7:0/644225063 learned_addr learned my addr 172.21.15.7:0/644225063 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:41.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.387+0000 7ff5b2000640 1 --2- 172.21.15.7:0/644225063 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff5ac15bcc0 0x7ff5ac0ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.387+0000 7ff5b1600640 1 --2- 172.21.15.7:0/644225063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5ac070040 0x7ff5ac14c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.387+0000 7ff5b0c00640 1 -- 172.21.15.7:0/644225063 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff5ac15bcc0 msgr2=0x7ff5ac0ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:41.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.387+0000 7ff5b0c00640 1 --2- 172.21.15.7:0/644225063 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff5ac15bcc0 0x7ff5ac0ae430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.387+0000 7ff5b0c00640 1 -- 172.21.15.7:0/644225063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5ac070040 msgr2=0x7ff5ac14c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:41.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.387+0000 7ff5b0c00640 1 --2- 172.21.15.7:0/644225063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5ac070040 0x7ff5ac14c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.387+0000 7ff5b0c00640 1 -- 172.21.15.7:0/644225063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5ac156be0 con 0x7ff5ac0ca170 2024-09-10T21:49:41.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.387+0000 7ff5b0c00640 1 --2- 172.21.15.7:0/644225063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5ac0ca170 0x7ff5ac0cf120 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c460 tx=0x7ff59800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:41.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.387+0000 7ff5a9600640 1 -- 172.21.15.7:0/644225063 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598011790 con 0x7ff5ac0ca170 2024-09-10T21:49:41.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.388+0000 7ff5a9600640 1 -- 172.21.15.7:0/644225063 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff598017070 con 0x7ff5ac0ca170 2024-09-10T21:49:41.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.388+0000 7ff5a9600640 1 -- 172.21.15.7:0/644225063 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598011d00 con 0x7ff5ac0ca170 2024-09-10T21:49:41.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.388+0000 7ff5b4000640 1 -- 172.21.15.7:0/644225063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5ac156ec0 con 0x7ff5ac0ca170 2024-09-10T21:49:41.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.388+0000 7ff5b4000640 1 -- 172.21.15.7:0/644225063 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5ac157480 con 0x7ff5ac0ca170 2024-09-10T21:49:41.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.390+0000 7ff5a9600640 1 -- 172.21.15.7:0/644225063 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff59801c460 con 0x7ff5ac0ca170 2024-09-10T21:49:41.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.391+0000 7ff5a9600640 1 --2- 172.21.15.7:0/644225063 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff58c077f70 0x7ff58c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.391+0000 7ff5a9600640 1 -- 172.21.15.7:0/644225063 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7ff598099bc0 con 0x7ff5ac0ca170 2024-09-10T21:49:41.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.391+0000 7ff5b702d740 1 Processor -- start 2024-09-10T21:49:41.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.391+0000 7ff5b702d740 1 -- start start 2024-09-10T21:49:41.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.391+0000 7ff5b1600640 1 --2- 172.21.15.7:0/644225063 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff58c077f70 0x7ff58c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.392+0000 7ff5b702d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c59130 0x564e12c59510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.392+0000 7ff5b1600640 1 --2- 172.21.15.7:0/644225063 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff58c077f70 0x7ff58c07a430 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0033000 tx=0x7ff5a0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:41.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.392+0000 7ff5b702d740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c142a0 0x564e12c57c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.392+0000 7ff5b702d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c14820 0x564e12c58150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.392+0000 7ff5b702d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564e12b731c0 con 0x564e12c59130 2024-09-10T21:49:41.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.392+0000 7ff5b702d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564e12b86870 con 0x564e12c14820 2024-09-10T21:49:41.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.392+0000 7ff5b702d740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x564e12b75ca0 con 0x564e12c142a0 2024-09-10T21:49:41.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.393+0000 7ff5b1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c59130 0x564e12c59510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.393+0000 7ff5b1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c59130 0x564e12c59510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47822/0 (socket says 172.21.15.7:47822) 2024-09-10T21:49:41.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.393+0000 7ff5b1600640 1 -- 172.21.15.7:0/246463621 learned_addr learned my addr 172.21.15.7:0/246463621 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:41.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.393+0000 7ff5b0c00640 1 --2- 172.21.15.7:0/246463621 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c142a0 0x564e12c57c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.393+0000 7ff5b2000640 1 --2- 172.21.15.7:0/246463621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c14820 0x564e12c58150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.393+0000 7ff5b1600640 1 -- 172.21.15.7:0/246463621 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c142a0 msgr2=0x564e12c57c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:41.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.393+0000 7ff5b1600640 1 --2- 172.21.15.7:0/246463621 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c142a0 0x564e12c57c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.393+0000 7ff5b1600640 1 -- 172.21.15.7:0/246463621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c14820 msgr2=0x564e12c58150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:41.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.393+0000 7ff5b1600640 1 --2- 172.21.15.7:0/246463621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c14820 0x564e12c58150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.393+0000 7ff5b1600640 1 -- 172.21.15.7:0/246463621 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e1294e7d0 con 0x564e12c59130 2024-09-10T21:49:41.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.393+0000 7ff5b1600640 1 --2- 172.21.15.7:0/246463621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c59130 0x564e12c59510 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0000c00 tx=0x7ff5a003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0db5c17ff8c3d40 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:41.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.394+0000 7ff59dc00640 1 -- 172.21.15.7:0/246463621 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0035040 con 0x564e12c59130 2024-09-10T21:49:41.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.394+0000 7ff59dc00640 1 -- 172.21.15.7:0/246463621 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5a004e070 con 0x564e12c59130 2024-09-10T21:49:41.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.394+0000 7ff59dc00640 1 -- 172.21.15.7:0/246463621 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a003c6d0 con 0x564e12c59130 2024-09-10T21:49:41.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.394+0000 7ff5b702d740 1 -- 172.21.15.7:0/246463621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c59130 msgr2=0x564e12c59510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:41.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.394+0000 7ff5b702d740 1 --2- 172.21.15.7:0/246463621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c59130 0x564e12c59510 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0000c00 tx=0x7ff5a003c000 comp rx=0 tx=0).stop 2024-09-10T21:49:41.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.394+0000 7ff5b702d740 1 -- 172.21.15.7:0/246463621 shutdown_connections 2024-09-10T21:49:41.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.394+0000 7ff5b702d740 1 --2- 172.21.15.7:0/246463621 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c14820 0x564e12c58150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.394+0000 7ff5b702d740 1 --2- 172.21.15.7:0/246463621 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c142a0 0x564e12c57c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.394+0000 7ff5b702d740 1 --2- 172.21.15.7:0/246463621 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c59130 0x564e12c59510 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.394+0000 7ff5b702d740 1 -- 172.21.15.7:0/246463621 >> 172.21.15.7:0/246463621 conn(0x564e12c0f870 msgr2=0x564e12c13290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:41.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.395+0000 7ff5b702d740 1 -- 172.21.15.7:0/246463621 shutdown_connections 2024-09-10T21:49:41.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.395+0000 7ff5b702d740 1 -- 172.21.15.7:0/246463621 wait complete. 2024-09-10T21:49:41.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.395+0000 7ff5b702d740 1 -- start start 2024-09-10T21:49:41.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.395+0000 7ff5b702d740 1 Processor -- start 2024-09-10T21:49:41.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.396+0000 7ff5b702d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c14820 0x564e12c3d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.396+0000 7ff5b702d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c59130 0x564e12c382b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.396+0000 7ff5b702d740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c3ad80 0x564e12c387f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.396+0000 7ff5b702d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564e12b1c410 con 0x564e12c14820 2024-09-10T21:49:41.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.396+0000 7ff5b702d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564e12b79ab0 con 0x564e12c59130 2024-09-10T21:49:41.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.396+0000 7ff5b702d740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x564e128e6740 con 0x564e12c3ad80 2024-09-10T21:49:41.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.396+0000 7ff5b702d740 10 client.0 _ll_register_callbacks cb 0x564e12af9360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:49:41.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.397+0000 7ff5b1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c14820 0x564e12c3d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.397+0000 7ff5b1600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c14820 0x564e12c3d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47828/0 (socket says 172.21.15.7:47828) 2024-09-10T21:49:41.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.397+0000 7ff5b1600640 1 -- 172.21.15.7:0/4149347232 learned_addr learned my addr 172.21.15.7:0/4149347232 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:41.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.397+0000 7ff5b2000640 1 --2- 172.21.15.7:0/4149347232 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c3ad80 0x564e12c387f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.397+0000 7ff5b0c00640 1 --2- 172.21.15.7:0/4149347232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c59130 0x564e12c382b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.397+0000 7ff5b1600640 1 -- 172.21.15.7:0/4149347232 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c3ad80 msgr2=0x564e12c387f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:41.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.397+0000 7ff5b1600640 1 --2- 172.21.15.7:0/4149347232 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c3ad80 0x564e12c387f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.397+0000 7ff5b1600640 1 -- 172.21.15.7:0/4149347232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c59130 msgr2=0x564e12c382b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:41.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.397+0000 7ff5b1600640 1 --2- 172.21.15.7:0/4149347232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c59130 0x564e12c382b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.397+0000 7ff5b1600640 1 -- 172.21.15.7:0/4149347232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e128d7970 con 0x564e12c14820 2024-09-10T21:49:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.397+0000 7ff5b1600640 1 --2- 172.21.15.7:0/4149347232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c14820 0x564e12c3d910 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0004650 tx=0x7ff5a003c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6bfe32404b6e3a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.397+0000 7ff592000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:49:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.398+0000 7ff592000640 1 -- 172.21.15.7:0/4149347232 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a0056070 con 0x564e12c14820 2024-09-10T21:49:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.398+0000 7ff5b702d740 1 -- 172.21.15.7:0/4149347232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a004e070 con 0x564e12c14820 2024-09-10T21:49:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.398+0000 7ff592000640 1 -- 172.21.15.7:0/4149347232 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7ff5a003caa0 con 0x564e12c14820 2024-09-10T21:49:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.398+0000 7ff592000640 1 -- 172.21.15.7:0/4149347232 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7ff5a0061050 con 0x564e12c14820 2024-09-10T21:49:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.398+0000 7ff5b702d740 10 client.7329 fetch_fsmap learned FSMap version 76 2024-09-10T21:49:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.398+0000 7ff5b702d740 1 -- 172.21.15.7:0/4149347232 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x564e128b56a0 con 0x564e12c14820 2024-09-10T21:49:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.398+0000 7ff592000640 1 -- 172.21.15.7:0/4149347232 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5a004e070 con 0x564e12c14820 2024-09-10T21:49:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.398+0000 7ff592000640 1 -- 172.21.15.7:0/4149347232 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7ff5a0064020 con 0x564e12c14820 2024-09-10T21:49:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.398+0000 7ff5b702d740 10 client.7329 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T21:49:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.398+0000 7ff5b702d740 10 client.7329 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-10T21:49:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.398+0000 7ff5b702d740 20 client.7329 populate_metadata read hostname 'smithi007' 2024-09-10T21:49:41.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.398+0000 7ff5b702d740 1 --2- 172.21.15.7:0/4149347232 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x564e12c53430 0x564e12c6b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.398+0000 7ff5b702d740 4 client.7329 mds_command: new command op to 236099 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:49:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.398+0000 7ff5b702d740 1 -- 172.21.15.7:0/4149347232 --> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e12b887c0 con 0x564e12c53430 2024-09-10T21:49:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.399+0000 7ff5b0c00640 1 --2- 172.21.15.7:0/4149347232 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x564e12c53430 0x564e12c6b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.400+0000 7ff5b0c00640 1 --2- 172.21.15.7:0/4149347232 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x564e12c53430 0x564e12c6b460 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e9c24dc200092d3 server_cookie=51ce5ec6c0289509 in_seq=0 out_seq=0 2024-09-10T21:49:41.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.400+0000 7ff592000640 10 client.7329 ms_handle_connect on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:41.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.400+0000 7ff592000640 1 -- 172.21.15.7:0/4149347232 <== mds.0 v2:172.21.15.133:6835/4188280820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564e12b887c0 con 0x564e12c53430 2024-09-10T21:49:41.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.400+0000 7ff592000640 10 client.7329 handle_command_reply: tid=0 2024-09-10T21:49:41.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.401+0000 7ff5b702d740 1 client.7329 shutdown 2024-09-10T21:49:41.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.401+0000 7ff5b702d740 1 -- 172.21.15.7:0/4149347232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c14820 msgr2=0x564e12c3d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:41.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.401+0000 7ff5b702d740 1 --2- 172.21.15.7:0/4149347232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c14820 0x564e12c3d910 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0004650 tx=0x7ff5a003c6a0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.401+0000 7ff5b702d740 1 -- 172.21.15.7:0/4149347232 shutdown_connections 2024-09-10T21:49:41.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.401+0000 7ff5b702d740 1 --2- 172.21.15.7:0/4149347232 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x564e12c53430 0x564e12c6b460 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.401+0000 7ff5b702d740 1 --2- 172.21.15.7:0/4149347232 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c59130 0x564e12c382b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.401+0000 7ff5b702d740 1 --2- 172.21.15.7:0/4149347232 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c3ad80 0x564e12c387f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.401+0000 7ff5b702d740 1 --2- 172.21.15.7:0/4149347232 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c14820 0x564e12c3d910 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.401+0000 7ff592000640 0 client.7329 ms_handle_reset on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:41.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.401+0000 7ff5b702d740 1 -- 172.21.15.7:0/4149347232 >> 172.21.15.7:0/4149347232 conn(0x564e12c142a0 msgr2=0x564e12c39a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:41.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.401+0000 7ff5b702d740 1 -- 172.21.15.7:0/4149347232 shutdown_connections 2024-09-10T21:49:41.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.401+0000 7ff5b702d740 1 -- 172.21.15.7:0/4149347232 wait complete. 2024-09-10T21:49:41.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.401+0000 7ff5b702d740 20 client.7329 trim_cache size 0 max 16384 2024-09-10T21:49:41.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.419+0000 7ff5b702d740 1 Processor -- start 2024-09-10T21:49:41.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.420+0000 7ff5b702d740 1 -- start start 2024-09-10T21:49:41.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.420+0000 7ff5b702d740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c14820 0x564e12c4d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.420+0000 7ff5b702d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c3ad80 0x564e12c4de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.420+0000 7ff5b702d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c59130 0x564e12c0c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.420+0000 7ff5b702d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac201170 con 0x564e12c59130 2024-09-10T21:49:41.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.420+0000 7ff5b702d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac0ce4b0 con 0x564e12c3ad80 2024-09-10T21:49:41.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.420+0000 7ff5b702d740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ff5ac0cd810 con 0x564e12c14820 2024-09-10T21:49:41.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.421+0000 7ff5b2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c59130 0x564e12c0c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.421+0000 7ff5b2000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c59130 0x564e12c0c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47842/0 (socket says 172.21.15.7:47842) 2024-09-10T21:49:41.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.421+0000 7ff5b2000640 1 -- 172.21.15.7:0/2291720289 learned_addr learned my addr 172.21.15.7:0/2291720289 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:41.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.421+0000 7ff5b1600640 1 --2- 172.21.15.7:0/2291720289 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c14820 0x564e12c4d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.421+0000 7ff5b0c00640 1 --2- 172.21.15.7:0/2291720289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c3ad80 0x564e12c4de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.421+0000 7ff5b2000640 1 -- 172.21.15.7:0/2291720289 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c14820 msgr2=0x564e12c4d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:41.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.421+0000 7ff5b2000640 1 --2- 172.21.15.7:0/2291720289 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c14820 0x564e12c4d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.421+0000 7ff5b2000640 1 -- 172.21.15.7:0/2291720289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c3ad80 msgr2=0x564e12c4de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:41.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.421+0000 7ff5b2000640 1 --2- 172.21.15.7:0/2291720289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c3ad80 0x564e12c4de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.421+0000 7ff5b2000640 1 -- 172.21.15.7:0/2291720289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e12c0dcd0 con 0x564e12c59130 2024-09-10T21:49:41.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.421+0000 7ff5b2000640 1 --2- 172.21.15.7:0/2291720289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c59130 0x564e12c0c7c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40134b0 tx=0x7ff5a4013970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=958638a9e7cdc40f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:41.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.422+0000 7ff59d200640 1 -- 172.21.15.7:0/2291720289 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4003970 con 0x564e12c59130 2024-09-10T21:49:41.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.422+0000 7ff59d200640 1 -- 172.21.15.7:0/2291720289 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5a4003b30 con 0x564e12c59130 2024-09-10T21:49:41.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.422+0000 7ff59d200640 1 -- 172.21.15.7:0/2291720289 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5a4004050 con 0x564e12c59130 2024-09-10T21:49:41.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.422+0000 7ff5b702d740 1 -- 172.21.15.7:0/2291720289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c59130 msgr2=0x564e12c0c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:41.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.422+0000 7ff5b702d740 1 --2- 172.21.15.7:0/2291720289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c59130 0x564e12c0c7c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7ff5a40134b0 tx=0x7ff5a4013970 comp rx=0 tx=0).stop 2024-09-10T21:49:41.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.422+0000 7ff5b702d740 1 -- 172.21.15.7:0/2291720289 shutdown_connections 2024-09-10T21:49:41.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.422+0000 7ff5b702d740 1 --2- 172.21.15.7:0/2291720289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c3ad80 0x564e12c4de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.422+0000 7ff5b702d740 1 --2- 172.21.15.7:0/2291720289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c59130 0x564e12c0c7c0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.422+0000 7ff5b702d740 1 --2- 172.21.15.7:0/2291720289 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c14820 0x564e12c4d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.422+0000 7ff5b702d740 1 -- 172.21.15.7:0/2291720289 >> 172.21.15.7:0/2291720289 conn(0x564e12c142a0 msgr2=0x564e12c6b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:41.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.422+0000 7ff5b702d740 1 -- 172.21.15.7:0/2291720289 shutdown_connections 2024-09-10T21:49:41.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.422+0000 7ff5b702d740 1 -- 172.21.15.7:0/2291720289 wait complete. 2024-09-10T21:49:41.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.423+0000 7ff5b702d740 1 -- start start 2024-09-10T21:49:41.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.423+0000 7ff5b702d740 1 Processor -- start 2024-09-10T21:49:41.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.424+0000 7ff5b702d740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c14820 0x564e12c699f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.424+0000 7ff5b702d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c3ad80 0x564e12c64d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.424+0000 7ff5b702d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c59130 0x564e12cd5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.424+0000 7ff5b702d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564e12c55e40 con 0x564e12c3ad80 2024-09-10T21:49:41.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.424+0000 7ff5b702d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff5ac0cde60 con 0x564e12c59130 2024-09-10T21:49:41.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.424+0000 7ff5b702d740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x564e12c093a0 con 0x564e12c14820 2024-09-10T21:49:41.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.424+0000 7ff5b702d740 10 client.0 _ll_register_callbacks cb 0x7ff5a4009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:49:41.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.424+0000 7ff5b0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c3ad80 0x564e12c64d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.424+0000 7ff5b0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c3ad80 0x564e12c64d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47854/0 (socket says 172.21.15.7:47854) 2024-09-10T21:49:41.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.424+0000 7ff5b0c00640 1 -- 172.21.15.7:0/2327708413 learned_addr learned my addr 172.21.15.7:0/2327708413 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:41.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.425+0000 7ff5b2000640 1 --2- 172.21.15.7:0/2327708413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c59130 0x564e12cd5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.425+0000 7ff5b1600640 1 --2- 172.21.15.7:0/2327708413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c14820 0x564e12c699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.425+0000 7ff5b0c00640 1 -- 172.21.15.7:0/2327708413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c14820 msgr2=0x564e12c699f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:41.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.425+0000 7ff5b0c00640 1 --2- 172.21.15.7:0/2327708413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c14820 0x564e12c699f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.425+0000 7ff5b0c00640 1 -- 172.21.15.7:0/2327708413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c59130 msgr2=0x564e12cd5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:41.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.425+0000 7ff5b0c00640 1 --2- 172.21.15.7:0/2327708413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c59130 0x564e12cd5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.425+0000 7ff5b0c00640 1 -- 172.21.15.7:0/2327708413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e12c3d2b0 con 0x564e12c3ad80 2024-09-10T21:49:41.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.425+0000 7ff5b0c00640 1 --2- 172.21.15.7:0/2327708413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c3ad80 0x564e12c64d10 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff598097f80 tx=0x7ff59805ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802bf4820ad58298 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:41.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.425+0000 7ff592000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:49:41.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.425+0000 7ff592000640 1 -- 172.21.15.7:0/2327708413 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff598061160 con 0x564e12c3ad80 2024-09-10T21:49:41.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.425+0000 7ff592000640 1 -- 172.21.15.7:0/2327708413 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7ff598061300 con 0x564e12c3ad80 2024-09-10T21:49:41.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.425+0000 7ff5b702d740 1 -- 172.21.15.7:0/2327708413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5a4003b30 con 0x564e12c3ad80 2024-09-10T21:49:41.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.426+0000 7ff592000640 1 -- 172.21.15.7:0/2327708413 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7ff59805d850 con 0x564e12c3ad80 2024-09-10T21:49:41.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.426+0000 7ff5b702d740 10 client.7335 fetch_fsmap learned FSMap version 76 2024-09-10T21:49:41.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.426+0000 7ff5b702d740 1 -- 172.21.15.7:0/2327708413 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x564e12c3a340 con 0x564e12c3ad80 2024-09-10T21:49:41.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.426+0000 7ff592000640 1 -- 172.21.15.7:0/2327708413 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5a4003b30 con 0x564e12c3ad80 2024-09-10T21:49:41.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.426+0000 7ff592000640 1 -- 172.21.15.7:0/2327708413 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7ff598096070 con 0x564e12c3ad80 2024-09-10T21:49:41.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.426+0000 7ff5b702d740 10 client.7335 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T21:49:41.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.426+0000 7ff5b702d740 10 client.7335 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.d 2024-09-10T21:49:41.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.426+0000 7ff5b702d740 20 client.7335 populate_metadata read hostname 'smithi007' 2024-09-10T21:49:41.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.426+0000 7ff5b702d740 1 --2- 172.21.15.7:0/2327708413 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x564e12c08ee0 0x564e12c0c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.426+0000 7ff5b702d740 4 client.7335 mds_command: new command op to 236099 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:49:41.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.426+0000 7ff5b702d740 1 -- 172.21.15.7:0/2327708413 --> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564e12b84270 con 0x564e12c08ee0 2024-09-10T21:49:41.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.427+0000 7ff5b1600640 1 --2- 172.21.15.7:0/2327708413 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x564e12c08ee0 0x564e12c0c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.427+0000 7ff5b1600640 1 --2- 172.21.15.7:0/2327708413 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x564e12c08ee0 0x564e12c0c7c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8263ee14d3b9a0b7 server_cookie=9f1c8ef69b6c2727 in_seq=0 out_seq=0 2024-09-10T21:49:41.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.427+0000 7ff592000640 10 client.7335 ms_handle_connect on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:41.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.427+0000 7ff592000640 1 -- 172.21.15.7:0/2327708413 <== mds.0 v2:172.21.15.133:6835/4188280820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+377 (crc 0 0 0) 0x564e12b84270 con 0x564e12c08ee0 2024-09-10T21:49:41.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.427+0000 7ff592000640 10 client.7335 handle_command_reply: tid=0 2024-09-10T21:49:41.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.427+0000 7ff5b702d740 1 client.7335 shutdown 2024-09-10T21:49:41.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.428+0000 7ff5b702d740 1 -- 172.21.15.7:0/2327708413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c3ad80 msgr2=0x564e12c64d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:41.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.428+0000 7ff5b702d740 1 --2- 172.21.15.7:0/2327708413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c3ad80 0x564e12c64d10 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff598097f80 tx=0x7ff59805ded0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.428+0000 7ff5b702d740 1 -- 172.21.15.7:0/2327708413 shutdown_connections 2024-09-10T21:49:41.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.428+0000 7ff5b702d740 1 --2- 172.21.15.7:0/2327708413 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x564e12c08ee0 0x564e12c0c7c0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.428+0000 7ff5b702d740 1 --2- 172.21.15.7:0/2327708413 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564e12c59130 0x564e12cd5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.428+0000 7ff5b702d740 1 --2- 172.21.15.7:0/2327708413 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564e12c3ad80 0x564e12c64d10 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.428+0000 7ff5b702d740 1 --2- 172.21.15.7:0/2327708413 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564e12c14820 0x564e12c699f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.428+0000 7ff5b702d740 1 -- 172.21.15.7:0/2327708413 >> 172.21.15.7:0/2327708413 conn(0x564e12c142a0 msgr2=0x564e12c4d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:41.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.428+0000 7ff592000640 0 client.7335 ms_handle_reset on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:41.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.428+0000 7ff5b702d740 1 -- 172.21.15.7:0/2327708413 shutdown_connections 2024-09-10T21:49:41.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.428+0000 7ff5b702d740 1 -- 172.21.15.7:0/2327708413 wait complete. 2024-09-10T21:49:41.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.428+0000 7ff5b702d740 20 client.7335 trim_cache size 0 max 16384 2024-09-10T21:49:41.433 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:49:41.433 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-09-10T21:49:41.433 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {}, 2024-09-10T21:49:41.433 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:49:41.433 INFO:teuthology.orchestra.run.smithi007.stdout: "2edd59b4-4331-4f8c-9518-8e5c0af9cba2": { 2024-09-10T21:49:41.433 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:49:10.961935+0000", 2024-09-10T21:49:41.433 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/dir", 2024-09-10T21:49:41.433 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:49:41.433 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:49:41.433 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:49:41.433 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:49:41.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:41.434 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:41.434 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:49:41.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.428+0000 7ff5b4000640 1 -- 172.21.15.7:0/644225063 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff58c077f70 msgr2=0x7ff58c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:41.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.428+0000 7ff5b4000640 1 --2- 172.21.15.7:0/644225063 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff58c077f70 0x7ff58c07a430 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7ff5a0033000 tx=0x7ff5a0004730 comp rx=0 tx=0).stop 2024-09-10T21:49:41.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.429+0000 7ff5b4000640 1 -- 172.21.15.7:0/644225063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5ac0ca170 msgr2=0x7ff5ac0cf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:41.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.429+0000 7ff5b4000640 1 --2- 172.21.15.7:0/644225063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5ac0ca170 0x7ff5ac0cf120 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ff59800c460 tx=0x7ff59800c920 comp rx=0 tx=0).stop 2024-09-10T21:49:41.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.429+0000 7ff5b4000640 1 -- 172.21.15.7:0/644225063 shutdown_connections 2024-09-10T21:49:41.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.429+0000 7ff5b4000640 1 --2- 172.21.15.7:0/644225063 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff58c077f70 0x7ff58c07a430 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.429+0000 7ff5b4000640 1 --2- 172.21.15.7:0/644225063 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff5ac15bcc0 0x7ff5ac0ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.429+0000 7ff5b4000640 1 --2- 172.21.15.7:0/644225063 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff5ac0ca170 0x7ff5ac0cf120 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.429+0000 7ff5b4000640 1 --2- 172.21.15.7:0/644225063 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff5ac070040 0x7ff5ac14c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.429+0000 7ff5b4000640 1 -- 172.21.15.7:0/644225063 >> 172.21.15.7:0/644225063 conn(0x7ff5ac0b16d0 msgr2=0x7ff5ac06e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:41.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.429+0000 7ff5b4000640 1 -- 172.21.15.7:0/644225063 shutdown_connections 2024-09-10T21:49:41.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.429+0000 7ff5b4000640 1 -- 172.21.15.7:0/644225063 wait complete. 2024-09-10T21:49:41.445 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:49:41.445 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:49:41.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.604+0000 7f9a0ce00640 1 -- 172.21.15.7:0/2600551680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a0815aa30 msgr2=0x7f9a08073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:41.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.604+0000 7f9a0ce00640 1 --2- 172.21.15.7:0/2600551680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a0815aa30 0x7f9a08073550 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009890 tx=0x7f99f802c750 comp rx=0 tx=0).stop 2024-09-10T21:49:41.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.604+0000 7f9a0ce00640 1 -- 172.21.15.7:0/2600551680 shutdown_connections 2024-09-10T21:49:41.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.604+0000 7f9a0ce00640 1 --2- 172.21.15.7:0/2600551680 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9a080ce860 0x7f9a080ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.604+0000 7f9a0ce00640 1 --2- 172.21.15.7:0/2600551680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a0815aa30 0x7f9a08073550 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.604+0000 7f9a0ce00640 1 -- 172.21.15.7:0/2600551680 >> 172.21.15.7:0/2600551680 conn(0x7f9a080b16d0 msgr2=0x7f9a080b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:41.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.604+0000 7f9a0ce00640 1 -- 172.21.15.7:0/2600551680 shutdown_connections 2024-09-10T21:49:41.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.604+0000 7f9a0ce00640 1 -- 172.21.15.7:0/2600551680 wait complete. 2024-09-10T21:49:41.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.606+0000 7f9a0ce00640 1 Processor -- start 2024-09-10T21:49:41.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.606+0000 7f9a0ce00640 1 -- start start 2024-09-10T21:49:41.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.607+0000 7f9a0ce00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a080ce860 0x7f9a0814c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.607+0000 7f9a0ce00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9a0815aa30 0x7f9a081f7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.607+0000 7f9a0ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a0815e650 0x7f9a081f7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.607+0000 7f9a0ce00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0806ee30 con 0x7f9a0815e650 2024-09-10T21:49:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.607+0000 7f9a0ce00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9a0806ecb0 con 0x7f9a080ce860 2024-09-10T21:49:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.607+0000 7f9a0ce00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f9a080cbcd0 con 0x7f9a0815aa30 2024-09-10T21:49:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.607+0000 7f9a06800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a0815e650 0x7f9a081f7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.607+0000 7f9a06800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a0815e650 0x7f9a081f7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47880/0 (socket says 172.21.15.7:47880) 2024-09-10T21:49:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.607+0000 7f9a06800640 1 -- 172.21.15.7:0/3783861243 learned_addr learned my addr 172.21.15.7:0/3783861243 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.607+0000 7f99fd400640 1 --2- 172.21.15.7:0/3783861243 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9a0815aa30 0x7f9a081f7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.607+0000 7f9a05e00640 1 --2- 172.21.15.7:0/3783861243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a080ce860 0x7f9a0814c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.608+0000 7f9a06800640 1 -- 172.21.15.7:0/3783861243 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9a0815aa30 msgr2=0x7f9a081f7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.608+0000 7f9a06800640 1 --2- 172.21.15.7:0/3783861243 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9a0815aa30 0x7f9a081f7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.608+0000 7f9a06800640 1 -- 172.21.15.7:0/3783861243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a080ce860 msgr2=0x7f9a0814c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.608+0000 7f9a06800640 1 --2- 172.21.15.7:0/3783861243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a080ce860 0x7f9a0814c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.608+0000 7f9a06800640 1 -- 172.21.15.7:0/3783861243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a0814c620 con 0x7f9a0815e650 2024-09-10T21:49:41.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.608+0000 7f99fd400640 1 --2- 172.21.15.7:0/3783861243 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9a0815aa30 0x7f9a081f7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:41.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.608+0000 7f9a06800640 1 --2- 172.21.15.7:0/3783861243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a0815e650 0x7f9a081f7e40 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000caa0 tx=0x7f9a0000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:41.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.608+0000 7f99fea00640 1 -- 172.21.15.7:0/3783861243 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00016020 con 0x7f9a0815e650 2024-09-10T21:49:41.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.608+0000 7f99fea00640 1 -- 172.21.15.7:0/3783861243 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a0000ac30 con 0x7f9a0815e650 2024-09-10T21:49:41.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.608+0000 7f99fea00640 1 -- 172.21.15.7:0/3783861243 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a00012680 con 0x7f9a0815e650 2024-09-10T21:49:41.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.608+0000 7f9a0ce00640 1 -- 172.21.15.7:0/3783861243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a080b10a0 con 0x7f9a0815e650 2024-09-10T21:49:41.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.608+0000 7f9a0ce00640 1 -- 172.21.15.7:0/3783861243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a08152fa0 con 0x7f9a0815e650 2024-09-10T21:49:41.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.610+0000 7f9a0ce00640 1 -- 172.21.15.7:0/3783861243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a080ceca0 con 0x7f9a0815e650 2024-09-10T21:49:41.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.611+0000 7f99fea00640 1 -- 172.21.15.7:0/3783861243 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9a0001b400 con 0x7f9a0815e650 2024-09-10T21:49:41.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.612+0000 7f99fea00640 1 --2- 172.21.15.7:0/3783861243 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f99e4077f70 0x7f99e407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:41.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.612+0000 7f99fea00640 1 -- 172.21.15.7:0/3783861243 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f9a0009a4e0 con 0x7f9a0815e650 2024-09-10T21:49:41.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.612+0000 7f9a05e00640 1 --2- 172.21.15.7:0/3783861243 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f99e4077f70 0x7f99e407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:41.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.613+0000 7f9a05e00640 1 --2- 172.21.15.7:0/3783861243 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f99e4077f70 0x7f99e407a430 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f99f8036000 tx=0x7f99f8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:41.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.616+0000 7f99fea00640 1 -- 172.21.15.7:0/3783861243 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9a00020020 con 0x7f9a0815e650 2024-09-10T21:49:41.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.850+0000 7f9a0ce00640 1 -- 172.21.15.7:0/3783861243 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a08152d60 con 0x7f9a0815e650 2024-09-10T21:49:41.851 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.850+0000 7f99fea00640 1 -- 172.21.15.7:0/3783861243 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5229 (secure 0 0 0) 0x7f9a0814c620 con 0x7f9a0815e650 2024-09-10T21:49:41.851 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:41.851 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":76,"btime":"2024-09-10T21:49:07:587035+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/978763848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":978763848},{"type":"v1","addr":"172.21.15.7:6835","nonce":978763848}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/269527918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":269527918},{"type":"v1","addr":"172.21.15.7:6833","nonce":269527918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":235775,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/760980985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":760980985},{"type":"v1","addr":"172.21.15.133:6836","nonce":760980985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:47:40.664091+0000","modified":"2024-09-10T21:49:07.587028+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236099":{"gid":236099,"name":"d","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6837/4188280820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4188280820},{"type":"v1","addr":"172.21.15.133:6837","nonce":4188280820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":236099,"qdb_cluster":[236099]},"id":6}]} 2024-09-10T21:49:41.852 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 76 2024-09-10T21:49:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.853+0000 7f9a0ce00640 1 -- 172.21.15.7:0/3783861243 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f99e4077f70 msgr2=0x7f99e407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.853+0000 7f9a0ce00640 1 --2- 172.21.15.7:0/3783861243 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f99e4077f70 0x7f99e407a430 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f99f8036000 tx=0x7f99f8004730 comp rx=0 tx=0).stop 2024-09-10T21:49:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.853+0000 7f9a0ce00640 1 -- 172.21.15.7:0/3783861243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a0815e650 msgr2=0x7f9a081f7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.853+0000 7f9a0ce00640 1 --2- 172.21.15.7:0/3783861243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a0815e650 0x7f9a081f7e40 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f9a0000caa0 tx=0x7f9a0000cf60 comp rx=0 tx=0).stop 2024-09-10T21:49:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.853+0000 7f9a0ce00640 1 -- 172.21.15.7:0/3783861243 shutdown_connections 2024-09-10T21:49:41.854 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.853+0000 7f9a0ce00640 1 --2- 172.21.15.7:0/3783861243 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f99e4077f70 0x7f99e407a430 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.853+0000 7f9a0ce00640 1 --2- 172.21.15.7:0/3783861243 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f9a0815e650 0x7f9a081f7e40 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.853+0000 7f9a0ce00640 1 --2- 172.21.15.7:0/3783861243 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f9a0815aa30 0x7f9a081f7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.853+0000 7f9a0ce00640 1 --2- 172.21.15.7:0/3783861243 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f9a080ce860 0x7f9a0814c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:41.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.853+0000 7f9a0ce00640 1 -- 172.21.15.7:0/3783861243 >> 172.21.15.7:0/3783861243 conn(0x7f9a080b16d0 msgr2=0x7f9a080b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:41.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.853+0000 7f9a0ce00640 1 -- 172.21.15.7:0/3783861243 shutdown_connections 2024-09-10T21:49:41.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:41.853+0000 7f9a0ce00640 1 -- 172.21.15.7:0/3783861243 wait complete. 2024-09-10T21:49:41.870 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2024-09-10T21:49:42.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.034+0000 7f8512a00640 1 -- 172.21.15.7:0/1697886331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f850c0d22b0 msgr2=0x7f850c0d26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.034+0000 7f8512a00640 1 --2- 172.21.15.7:0/1697886331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f850c0d22b0 0x7f850c0d26f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f84f8005fd0 tx=0x7f84f802a5d0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.034+0000 7f8512a00640 1 -- 172.21.15.7:0/1697886331 shutdown_connections 2024-09-10T21:49:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.034+0000 7f8512a00640 1 --2- 172.21.15.7:0/1697886331 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f850c0b0a10 0x7f850c0b0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.034+0000 7f8512a00640 1 --2- 172.21.15.7:0/1697886331 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f850c0d22b0 0x7f850c0d26f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.034+0000 7f8512a00640 1 -- 172.21.15.7:0/1697886331 >> 172.21.15.7:0/1697886331 conn(0x7f850c0d3e80 msgr2=0x7f850c0d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.034+0000 7f8512a00640 1 -- 172.21.15.7:0/1697886331 shutdown_connections 2024-09-10T21:49:42.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.034+0000 7f8512a00640 1 -- 172.21.15.7:0/1697886331 wait complete. 2024-09-10T21:49:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.036+0000 7f8512a00640 1 Processor -- start 2024-09-10T21:49:42.037 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.036+0000 7f8512a00640 1 -- start start 2024-09-10T21:49:42.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.037+0000 7f8512a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f850c0b0a10 0x7f850c0b49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.037+0000 7f8512a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f850c0d22b0 0x7f850c1f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.037+0000 7f8512a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f850c15df70 0x7f850c1f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.037+0000 7f8512a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f850c0b2960 con 0x7f850c15df70 2024-09-10T21:49:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.037+0000 7f8512a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f850c0b27e0 con 0x7f850c0b0a10 2024-09-10T21:49:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.037+0000 7f8512a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f850c0b2ae0 con 0x7f850c0d22b0 2024-09-10T21:49:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.037+0000 7f8512000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f850c15df70 0x7f850c1f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.037+0000 7f8512000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f850c15df70 0x7f850c1f7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47888/0 (socket says 172.21.15.7:47888) 2024-09-10T21:49:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.037+0000 7f8512000640 1 -- 172.21.15.7:0/3547999262 learned_addr learned my addr 172.21.15.7:0/3547999262 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.037+0000 7f8511600640 1 --2- 172.21.15.7:0/3547999262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f850c0b0a10 0x7f850c0b49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.037+0000 7f8510c00640 1 --2- 172.21.15.7:0/3547999262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f850c0d22b0 0x7f850c1f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.038+0000 7f8512000640 1 -- 172.21.15.7:0/3547999262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f850c0d22b0 msgr2=0x7f850c1f7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.038+0000 7f8512000640 1 --2- 172.21.15.7:0/3547999262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f850c0d22b0 0x7f850c1f7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.038+0000 7f8512000640 1 -- 172.21.15.7:0/3547999262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f850c0b0a10 msgr2=0x7f850c0b49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.038+0000 7f8512000640 1 --2- 172.21.15.7:0/3547999262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f850c0b0a10 0x7f850c0b49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.038+0000 7f8512000640 1 -- 172.21.15.7:0/3547999262 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f850c0b32e0 con 0x7f850c15df70 2024-09-10T21:49:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.038+0000 7f8512000640 1 --2- 172.21.15.7:0/3547999262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f850c15df70 0x7f850c1f7f90 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f850400efd0 tx=0x7f850400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:42.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.038+0000 7f8509600640 1 -- 172.21.15.7:0/3547999262 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504003b70 con 0x7f850c15df70 2024-09-10T21:49:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.038+0000 7f8509600640 1 -- 172.21.15.7:0/3547999262 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8504003e00 con 0x7f850c15df70 2024-09-10T21:49:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.038+0000 7f8509600640 1 -- 172.21.15.7:0/3547999262 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504004280 con 0x7f850c15df70 2024-09-10T21:49:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.038+0000 7f8512a00640 1 -- 172.21.15.7:0/3547999262 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f850c0c8b80 con 0x7f850c15df70 2024-09-10T21:49:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.039+0000 7f8512a00640 1 -- 172.21.15.7:0/3547999262 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f850c0c9140 con 0x7f850c15df70 2024-09-10T21:49:42.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.041+0000 7f8509600640 1 -- 172.21.15.7:0/3547999262 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8504004420 con 0x7f850c15df70 2024-09-10T21:49:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.042+0000 7f8509600640 1 --2- 172.21.15.7:0/3547999262 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f84ec077f70 0x7f84ec07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.042+0000 7f8509600640 1 -- 172.21.15.7:0/3547999262 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f850409ac10 con 0x7f850c15df70 2024-09-10T21:49:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.042+0000 7f8516eaa740 1 Processor -- start 2024-09-10T21:49:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.042+0000 7f8516eaa740 1 -- start start 2024-09-10T21:49:42.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.042+0000 7f8511600640 1 --2- 172.21.15.7:0/3547999262 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f84ec077f70 0x7f84ec07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:42.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.043+0000 7f8511600640 1 --2- 172.21.15.7:0/3547999262 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f84ec077f70 0x7f84ec07a430 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f850c0698c0 tx=0x7f8500009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.043+0000 7f8516eaa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3de950 0x56157c3db7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.043+0000 7f8516eaa740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56157c3f0370 0x56157c3ded30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.043+0000 7f8516eaa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56157c3df270 0x56157c3afe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.043+0000 7f8516eaa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56157c326750 con 0x56157c3de950 2024-09-10T21:49:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.043+0000 7f8516eaa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56157c315b80 con 0x56157c3df270 2024-09-10T21:49:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.043+0000 7f8516eaa740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x56157c319990 con 0x56157c3f0370 2024-09-10T21:49:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.043+0000 7f8511600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3de950 0x56157c3db7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.043+0000 7f8511600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3de950 0x56157c3db7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47902/0 (socket says 172.21.15.7:47902) 2024-09-10T21:49:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.043+0000 7f8511600640 1 -- 172.21.15.7:0/850659441 learned_addr learned my addr 172.21.15.7:0/850659441 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.044+0000 7f8511600640 1 -- 172.21.15.7:0/850659441 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56157c3f0370 msgr2=0x56157c3ded30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.044+0000 7f8511600640 1 --2- 172.21.15.7:0/850659441 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56157c3f0370 0x56157c3ded30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.044+0000 7f8511600640 1 -- 172.21.15.7:0/850659441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56157c3df270 msgr2=0x56157c3afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:42.045 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.044+0000 7f8511600640 1 --2- 172.21.15.7:0/850659441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56157c3df270 0x56157c3afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.044+0000 7f8511600640 1 -- 172.21.15.7:0/850659441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56157c2a3ad0 con 0x56157c3de950 2024-09-10T21:49:42.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.044+0000 7f8511600640 1 --2- 172.21.15.7:0/850659441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3de950 0x56157c3db7f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f8500007bc0 tx=0x7f8500012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e96aac9f0f52a9b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.044+0000 7f84fdc00640 1 -- 172.21.15.7:0/850659441 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500026070 con 0x56157c3de950 2024-09-10T21:49:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.044+0000 7f84fdc00640 1 -- 172.21.15.7:0/850659441 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f850000f800 con 0x56157c3de950 2024-09-10T21:49:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.044+0000 7f84fdc00640 1 -- 172.21.15.7:0/850659441 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850000fc30 con 0x56157c3de950 2024-09-10T21:49:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.045+0000 7f8516eaa740 1 -- 172.21.15.7:0/850659441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3de950 msgr2=0x56157c3db7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.045+0000 7f8516eaa740 1 --2- 172.21.15.7:0/850659441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3de950 0x56157c3db7f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f8500007bc0 tx=0x7f8500012000 comp rx=0 tx=0).stop 2024-09-10T21:49:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.045+0000 7f8516eaa740 1 -- 172.21.15.7:0/850659441 shutdown_connections 2024-09-10T21:49:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.045+0000 7f8516eaa740 1 --2- 172.21.15.7:0/850659441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56157c3df270 0x56157c3afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.045+0000 7f8516eaa740 1 --2- 172.21.15.7:0/850659441 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56157c3f0370 0x56157c3ded30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.045+0000 7f8516eaa740 1 --2- 172.21.15.7:0/850659441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3de950 0x56157c3db7f0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.045+0000 7f8516eaa740 1 -- 172.21.15.7:0/850659441 >> 172.21.15.7:0/850659441 conn(0x56157c3b0e90 msgr2=0x56157c3b48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:42.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.045+0000 7f8516eaa740 1 -- 172.21.15.7:0/850659441 shutdown_connections 2024-09-10T21:49:42.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.045+0000 7f8516eaa740 1 -- 172.21.15.7:0/850659441 wait complete. 2024-09-10T21:49:42.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.046+0000 7f8516eaa740 1 -- start start 2024-09-10T21:49:42.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.046+0000 7f8516eaa740 1 Processor -- start 2024-09-10T21:49:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.047+0000 7f8516eaa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3df270 0x56157c3f3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.047+0000 7f8516eaa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56157c3f0370 0x56157c3dd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.047+0000 7f8516eaa740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56157c3dddc0 0x56157c40cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.047+0000 7f8516eaa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56157c086700 con 0x56157c3df270 2024-09-10T21:49:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.047+0000 7f8516eaa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56157c2bb430 con 0x56157c3f0370 2024-09-10T21:49:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.047+0000 7f8516eaa740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x56157c3b5930 con 0x56157c3dddc0 2024-09-10T21:49:42.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.047+0000 7f8516eaa740 10 client.0 _ll_register_callbacks cb 0x56157c2c1600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:49:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.047+0000 7f8511600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3df270 0x56157c3f3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.047+0000 7f8511600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3df270 0x56157c3f3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47904/0 (socket says 172.21.15.7:47904) 2024-09-10T21:49:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.047+0000 7f8511600640 1 -- 172.21.15.7:0/1039041103 learned_addr learned my addr 172.21.15.7:0/1039041103 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.047+0000 7f8511600640 1 -- 172.21.15.7:0/1039041103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56157c3dddc0 msgr2=0x56157c40cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.047+0000 7f8511600640 1 --2- 172.21.15.7:0/1039041103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56157c3dddc0 0x56157c40cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.047+0000 7f8511600640 1 -- 172.21.15.7:0/1039041103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56157c3f0370 msgr2=0x56157c3dd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.047+0000 7f8511600640 1 --2- 172.21.15.7:0/1039041103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56157c3f0370 0x56157c3dd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.047+0000 7f8511600640 1 -- 172.21.15.7:0/1039041103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56157c0f07c0 con 0x56157c3df270 2024-09-10T21:49:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.048+0000 7f8511600640 1 --2- 172.21.15.7:0/1039041103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3df270 0x56157c3f3d20 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8500024040 tx=0x7f85000226f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50b2e306dde38e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.048+0000 7f84f2000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:49:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.048+0000 7f84f2000640 1 -- 172.21.15.7:0/1039041103 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8500026040 con 0x56157c3df270 2024-09-10T21:49:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.048+0000 7f84f2000640 1 -- 172.21.15.7:0/1039041103 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f8500022860 con 0x56157c3df270 2024-09-10T21:49:42.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.048+0000 7f8516eaa740 1 -- 172.21.15.7:0/1039041103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f850000f800 con 0x56157c3df270 2024-09-10T21:49:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.048+0000 7f84f2000640 1 -- 172.21.15.7:0/1039041103 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f850000fe60 con 0x56157c3df270 2024-09-10T21:49:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.048+0000 7f8516eaa740 10 client.7353 fetch_fsmap learned FSMap version 76 2024-09-10T21:49:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.048+0000 7f8516eaa740 1 -- 172.21.15.7:0/1039041103 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x56157c150f50 con 0x56157c3df270 2024-09-10T21:49:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.049+0000 7f84f2000640 1 -- 172.21.15.7:0/1039041103 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f850000f800 con 0x56157c3df270 2024-09-10T21:49:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.049+0000 7f84f2000640 1 -- 172.21.15.7:0/1039041103 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7f850003a020 con 0x56157c3df270 2024-09-10T21:49:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.049+0000 7f8516eaa740 10 client.7353 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T21:49:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.049+0000 7f8516eaa740 10 client.7353 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:49:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.049+0000 7f8516eaa740 20 client.7353 populate_metadata read hostname 'smithi007' 2024-09-10T21:49:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.049+0000 7f8516eaa740 1 --2- 172.21.15.7:0/1039041103 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x56157c3f6dc0 0x56157c3f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.049+0000 7f8516eaa740 4 client.7353 mds_command: new command op to 236099 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:49:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.049+0000 7f8516eaa740 1 -- 172.21.15.7:0/1039041103 --> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56157c3286a0 con 0x56157c3f6dc0 2024-09-10T21:49:42.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.049+0000 7f8510c00640 1 --2- 172.21.15.7:0/1039041103 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x56157c3f6dc0 0x56157c3f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.050+0000 7f8510c00640 1 --2- 172.21.15.7:0/1039041103 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x56157c3f6dc0 0x56157c3f71a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7c04439007fad26 server_cookie=2c90bd4272b06dbc in_seq=0 out_seq=0 2024-09-10T21:49:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.050+0000 7f84f2000640 10 client.7353 ms_handle_connect on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.051+0000 7f84f2000640 1 -- 172.21.15.7:0/1039041103 <== mds.0 v2:172.21.15.133:6835/4188280820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56157c3286a0 con 0x56157c3f6dc0 2024-09-10T21:49:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.051+0000 7f84f2000640 10 client.7353 handle_command_reply: tid=0 2024-09-10T21:49:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.051+0000 7f8516eaa740 1 client.7353 shutdown 2024-09-10T21:49:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.051+0000 7f8516eaa740 1 -- 172.21.15.7:0/1039041103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3df270 msgr2=0x56157c3f3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.051+0000 7f8516eaa740 1 --2- 172.21.15.7:0/1039041103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3df270 0x56157c3f3d20 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8500024040 tx=0x7f85000226f0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.052+0000 7f8516eaa740 1 -- 172.21.15.7:0/1039041103 shutdown_connections 2024-09-10T21:49:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.052+0000 7f8516eaa740 1 --2- 172.21.15.7:0/1039041103 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x56157c3f6dc0 0x56157c3f71a0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.052+0000 7f8516eaa740 1 --2- 172.21.15.7:0/1039041103 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56157c3f0370 0x56157c3dd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.052+0000 7f8516eaa740 1 --2- 172.21.15.7:0/1039041103 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56157c3dddc0 0x56157c40cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.052+0000 7f8516eaa740 1 --2- 172.21.15.7:0/1039041103 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3df270 0x56157c3f3d20 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.052+0000 7f84f2000640 0 client.7353 ms_handle_reset on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:42.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.052+0000 7f8516eaa740 1 -- 172.21.15.7:0/1039041103 >> 172.21.15.7:0/1039041103 conn(0x56157c3de950 msgr2=0x56157c3b0310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:42.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.052+0000 7f8516eaa740 1 -- 172.21.15.7:0/1039041103 shutdown_connections 2024-09-10T21:49:42.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.052+0000 7f8516eaa740 1 -- 172.21.15.7:0/1039041103 wait complete. 2024-09-10T21:49:42.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.052+0000 7f8516eaa740 20 client.7353 trim_cache size 0 max 16384 2024-09-10T21:49:42.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.066+0000 7f8516eaa740 1 Processor -- start 2024-09-10T21:49:42.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.066+0000 7f8516eaa740 1 -- start start 2024-09-10T21:49:42.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.067+0000 7f8516eaa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56157c3dddc0 0x56157c3f7390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.067+0000 7f8516eaa740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56157c3df270 0x56157c3d9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.067+0000 7f8516eaa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3f0370 0x56157c3da070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.067+0000 7f8516eaa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f850c200280 con 0x56157c3f0370 2024-09-10T21:49:42.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.067+0000 7f8516eaa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f850c1f4d70 con 0x56157c3dddc0 2024-09-10T21:49:42.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.067+0000 7f8516eaa740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f850c1ffde0 con 0x56157c3df270 2024-09-10T21:49:42.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.067+0000 7f8512000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3f0370 0x56157c3da070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:42.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.067+0000 7f8512000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3f0370 0x56157c3da070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47916/0 (socket says 172.21.15.7:47916) 2024-09-10T21:49:42.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.067+0000 7f8512000640 1 -- 172.21.15.7:0/2594370780 learned_addr learned my addr 172.21.15.7:0/2594370780 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:42.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.067+0000 7f8512000640 1 -- 172.21.15.7:0/2594370780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56157c3df270 msgr2=0x56157c3d9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:42.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.067+0000 7f8512000640 1 --2- 172.21.15.7:0/2594370780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56157c3df270 0x56157c3d9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.067+0000 7f8512000640 1 -- 172.21.15.7:0/2594370780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56157c3dddc0 msgr2=0x56157c3f7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:42.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.067+0000 7f8512000640 1 --2- 172.21.15.7:0/2594370780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56157c3dddc0 0x56157c3f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.067+0000 7f8512000640 1 -- 172.21.15.7:0/2594370780 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56157c055660 con 0x56157c3f0370 2024-09-10T21:49:42.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.068+0000 7f8512000640 1 --2- 172.21.15.7:0/2594370780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3f0370 0x56157c3da070 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f8504062ce0 tx=0x7f85040629f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e465909bfe5f5d82 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:42.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.068+0000 7f84fd200640 1 -- 172.21.15.7:0/2594370780 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850401d040 con 0x56157c3f0370 2024-09-10T21:49:42.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.068+0000 7f84fd200640 1 -- 172.21.15.7:0/2594370780 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8504062fe0 con 0x56157c3f0370 2024-09-10T21:49:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.068+0000 7f84fd200640 1 -- 172.21.15.7:0/2594370780 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504067a10 con 0x56157c3f0370 2024-09-10T21:49:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.068+0000 7f8516eaa740 1 -- 172.21.15.7:0/2594370780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3f0370 msgr2=0x56157c3da070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.068+0000 7f8516eaa740 1 --2- 172.21.15.7:0/2594370780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3f0370 0x56157c3da070 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f8504062ce0 tx=0x7f85040629f0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.068+0000 7f8516eaa740 1 -- 172.21.15.7:0/2594370780 shutdown_connections 2024-09-10T21:49:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.068+0000 7f8516eaa740 1 --2- 172.21.15.7:0/2594370780 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3f0370 0x56157c3da070 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.068+0000 7f8516eaa740 1 --2- 172.21.15.7:0/2594370780 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56157c3df270 0x56157c3d9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.068+0000 7f8516eaa740 1 --2- 172.21.15.7:0/2594370780 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56157c3dddc0 0x56157c3f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.069+0000 7f8516eaa740 1 -- 172.21.15.7:0/2594370780 >> 172.21.15.7:0/2594370780 conn(0x56157c3de950 msgr2=0x56157c3adeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.069+0000 7f8516eaa740 1 -- 172.21.15.7:0/2594370780 shutdown_connections 2024-09-10T21:49:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.069+0000 7f8516eaa740 1 -- 172.21.15.7:0/2594370780 wait complete. 2024-09-10T21:49:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.069+0000 7f8516eaa740 1 -- start start 2024-09-10T21:49:42.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.069+0000 7f8516eaa740 1 Processor -- start 2024-09-10T21:49:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.070+0000 7f8516eaa740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56157c3dddc0 0x56157c473b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.070+0000 7f8516eaa740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56157c3df270 0x56157c41d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.070+0000 7f8516eaa740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3f0370 0x56157c479a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.070+0000 7f8516eaa740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56157c40d8b0 con 0x56157c3f0370 2024-09-10T21:49:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.070+0000 7f8516eaa740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56157c3f4260 con 0x56157c3df270 2024-09-10T21:49:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.070+0000 7f8516eaa740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x56157c3f9a40 con 0x56157c3dddc0 2024-09-10T21:49:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.070+0000 7f8516eaa740 10 client.0 _ll_register_callbacks cb 0x7f850000fd20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:49:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.071+0000 7f8512000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3f0370 0x56157c479a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.071+0000 7f8512000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3f0370 0x56157c479a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47924/0 (socket says 172.21.15.7:47924) 2024-09-10T21:49:42.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.071+0000 7f8512000640 1 -- 172.21.15.7:0/199411856 learned_addr learned my addr 172.21.15.7:0/199411856 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.071+0000 7f8510c00640 1 --2- 172.21.15.7:0/199411856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56157c3df270 0x56157c41d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.071+0000 7f8511600640 1 --2- 172.21.15.7:0/199411856 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56157c3dddc0 0x56157c473b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.071+0000 7f8512000640 1 -- 172.21.15.7:0/199411856 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56157c3dddc0 msgr2=0x56157c473b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.071+0000 7f8512000640 1 --2- 172.21.15.7:0/199411856 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56157c3dddc0 0x56157c473b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.071+0000 7f8512000640 1 -- 172.21.15.7:0/199411856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56157c3df270 msgr2=0x56157c41d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:49:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.071+0000 7f8512000640 1 --2- 172.21.15.7:0/199411856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56157c3df270 0x56157c41d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.071+0000 7f8512000640 1 -- 172.21.15.7:0/199411856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56157c2e2630 con 0x56157c3f0370 2024-09-10T21:49:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.071+0000 7f8512000640 1 --2- 172.21.15.7:0/199411856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3f0370 0x56157c479a60 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f8504062f70 tx=0x7f850406c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4df7ef346bb604 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.071+0000 7f84f2000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:49:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.071+0000 7f84f2000640 1 -- 172.21.15.7:0/199411856 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504081060 con 0x56157c3f0370 2024-09-10T21:49:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.071+0000 7f84f2000640 1 -- 172.21.15.7:0/199411856 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f8504068ba0 con 0x56157c3f0370 2024-09-10T21:49:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.071+0000 7f8516eaa740 1 -- 172.21.15.7:0/199411856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8504062fe0 con 0x56157c3f0370 2024-09-10T21:49:42.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.072+0000 7f84f2000640 1 -- 172.21.15.7:0/199411856 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) ==== 24+0+0 (secure 0 0 0) 0x7f8504067a10 con 0x56157c3f0370 2024-09-10T21:49:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.072+0000 7f8516eaa740 10 client.7359 fetch_fsmap learned FSMap version 76 2024-09-10T21:49:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.072+0000 7f8516eaa740 1 -- 172.21.15.7:0/199411856 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) -- 0x56157c408aa0 con 0x56157c3f0370 2024-09-10T21:49:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.072+0000 7f84f2000640 1 -- 172.21.15.7:0/199411856 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8504062fe0 con 0x56157c3f0370 2024-09-10T21:49:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.072+0000 7f84f2000640 1 -- 172.21.15.7:0/199411856 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 76) ==== 3502+0+0 (secure 0 0 0) 0x7f8504004b40 con 0x56157c3f0370 2024-09-10T21:49:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.072+0000 7f8516eaa740 10 client.7359 fetch_fsmap finished waiting for FSMap version 76 2024-09-10T21:49:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.072+0000 7f8516eaa740 10 client.7359 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:49:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.072+0000 7f8516eaa740 20 client.7359 populate_metadata read hostname 'smithi007' 2024-09-10T21:49:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.072+0000 7f8516eaa740 1 --2- 172.21.15.7:0/199411856 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x56157c3db750 0x56157c3dbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.072+0000 7f8516eaa740 4 client.7359 mds_command: new command op to 236099 tid=0 multi_id=0 [{"prefix": "damage ls", "format": "json-pretty"}] 2024-09-10T21:49:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.072+0000 7f8516eaa740 1 -- 172.21.15.7:0/199411856 --> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) -- 0x56157c3f5770 con 0x56157c3db750 2024-09-10T21:49:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.073+0000 7f8511600640 1 --2- 172.21.15.7:0/199411856 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x56157c3db750 0x56157c3dbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:42.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.073+0000 7f8511600640 1 --2- 172.21.15.7:0/199411856 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x56157c3db750 0x56157c3dbb30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80f87cff6fccee8b server_cookie=1875109ef95885b3 in_seq=0 out_seq=0 2024-09-10T21:49:42.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.073+0000 7f84f2000640 10 client.7359 ms_handle_connect on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:42.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.073+0000 7f84f2000640 1 -- 172.21.15.7:0/199411856 <== mds.0 v2:172.21.15.133:6835/4188280820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56157c3f5770 con 0x56157c3db750 2024-09-10T21:49:42.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.073+0000 7f84f2000640 10 client.7359 handle_command_reply: tid=0 2024-09-10T21:49:42.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.073+0000 7f8516eaa740 1 client.7359 shutdown 2024-09-10T21:49:42.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.074+0000 7f8516eaa740 1 -- 172.21.15.7:0/199411856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3f0370 msgr2=0x56157c479a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:42.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.074+0000 7f8516eaa740 1 --2- 172.21.15.7:0/199411856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3f0370 0x56157c479a60 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f8504062f70 tx=0x7f850406c430 comp rx=0 tx=0).stop 2024-09-10T21:49:42.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.074+0000 7f8516eaa740 1 -- 172.21.15.7:0/199411856 shutdown_connections 2024-09-10T21:49:42.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.074+0000 7f8516eaa740 1 --2- 172.21.15.7:0/199411856 >> [v2:172.21.15.133:6835/4188280820,v1:172.21.15.133:6837/4188280820] conn(0x56157c3db750 0x56157c3dbb30 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.074+0000 7f8516eaa740 1 --2- 172.21.15.7:0/199411856 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56157c3df270 0x56157c41d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.074+0000 7f8516eaa740 1 --2- 172.21.15.7:0/199411856 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56157c3f0370 0x56157c479a60 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.074+0000 7f8516eaa740 1 --2- 172.21.15.7:0/199411856 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56157c3dddc0 0x56157c473b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.074+0000 7f84f2000640 0 client.7359 ms_handle_reset on v2:172.21.15.133:6835/4188280820 2024-09-10T21:49:42.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.074+0000 7f8516eaa740 1 -- 172.21.15.7:0/199411856 >> 172.21.15.7:0/199411856 conn(0x56157c3de950 msgr2=0x56157c3aa520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:42.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.074+0000 7f8516eaa740 1 -- 172.21.15.7:0/199411856 shutdown_connections 2024-09-10T21:49:42.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.074+0000 7f8516eaa740 1 -- 172.21.15.7:0/199411856 wait complete. 2024-09-10T21:49:42.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.074+0000 7f8516eaa740 20 client.7359 trim_cache size 0 max 16384 2024-09-10T21:49:42.079 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:42.079 INFO:teuthology.orchestra.run.smithi007.stdout:[] 2024-09-10T21:49:42.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.075+0000 7f8512a00640 1 -- 172.21.15.7:0/3547999262 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f84ec077f70 msgr2=0x7f84ec07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:42.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.075+0000 7f8512a00640 1 --2- 172.21.15.7:0/3547999262 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f84ec077f70 0x7f84ec07a430 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f850c0698c0 tx=0x7f8500009040 comp rx=0 tx=0).stop 2024-09-10T21:49:42.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.075+0000 7f8512a00640 1 -- 172.21.15.7:0/3547999262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f850c15df70 msgr2=0x7f850c1f7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:42.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.075+0000 7f8512a00640 1 --2- 172.21.15.7:0/3547999262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f850c15df70 0x7f850c1f7f90 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f850400efd0 tx=0x7f850400c6a0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.075+0000 7f8512a00640 1 -- 172.21.15.7:0/3547999262 shutdown_connections 2024-09-10T21:49:42.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.075+0000 7f8512a00640 1 --2- 172.21.15.7:0/3547999262 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f84ec077f70 0x7f84ec07a430 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.075+0000 7f8512a00640 1 --2- 172.21.15.7:0/3547999262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f850c15df70 0x7f850c1f7f90 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.075+0000 7f8512a00640 1 --2- 172.21.15.7:0/3547999262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f850c0d22b0 0x7f850c1f7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.075+0000 7f8512a00640 1 --2- 172.21.15.7:0/3547999262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f850c0b0a10 0x7f850c0b49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.075+0000 7f8512a00640 1 -- 172.21.15.7:0/3547999262 >> 172.21.15.7:0/3547999262 conn(0x7f850c0d3e80 msgr2=0x7f850c06eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:42.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.075+0000 7f8512a00640 1 -- 172.21.15.7:0/3547999262 shutdown_connections 2024-09-10T21:49:42.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.075+0000 7f8512a00640 1 -- 172.21.15.7:0/3547999262 wait complete. 2024-09-10T21:49:42.091 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:49:42.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.248+0000 7fc2f7200640 1 -- 172.21.15.7:0/4076808747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2f80d1e60 msgr2=0x7fc2f80d4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:42.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.248+0000 7fc2f7200640 1 --2- 172.21.15.7:0/4076808747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2f80d1e60 0x7fc2f80d4900 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8009830 tx=0x7fc2e802a5a0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.249+0000 7fc2f7200640 1 -- 172.21.15.7:0/4076808747 shutdown_connections 2024-09-10T21:49:42.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.249+0000 7fc2f7200640 1 --2- 172.21.15.7:0/4076808747 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2f80d2a20 0x7fc2f80d2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.249+0000 7fc2f7200640 1 --2- 172.21.15.7:0/4076808747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2f80d1e60 0x7fc2f80d4900 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.249+0000 7fc2f7200640 1 -- 172.21.15.7:0/4076808747 >> 172.21.15.7:0/4076808747 conn(0x7fc2f80d3de0 msgr2=0x7fc2f80d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:42.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.249+0000 7fc2f7200640 1 -- 172.21.15.7:0/4076808747 shutdown_connections 2024-09-10T21:49:42.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.249+0000 7fc2f7200640 1 -- 172.21.15.7:0/4076808747 wait complete. 2024-09-10T21:49:42.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.250+0000 7fc2f7200640 1 Processor -- start 2024-09-10T21:49:42.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.250+0000 7fc2f7200640 1 -- start start 2024-09-10T21:49:42.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.251+0000 7fc2f7200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2f80d07f0 0x7fc2f81e86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.251+0000 7fc2f7200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f80d1e60 0x7fc2f815a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.251+0000 7fc2f7200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2f80d2a20 0x7fc2f80b8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.251+0000 7fc2f7200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f80d1500 con 0x7fc2f80d2a20 2024-09-10T21:49:42.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.251+0000 7fc2f7200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f80d1380 con 0x7fc2f80d1e60 2024-09-10T21:49:42.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.251+0000 7fc2f7200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc2f80d1680 con 0x7fc2f80d07f0 2024-09-10T21:49:42.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.252+0000 7fc2f6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2f80d2a20 0x7fc2f80b8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:42.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.252+0000 7fc2f5e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2f80d07f0 0x7fc2f81e86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:42.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.252+0000 7fc2f6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2f80d2a20 0x7fc2f80b8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47942/0 (socket says 172.21.15.7:47942) 2024-09-10T21:49:42.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.252+0000 7fc2f6800640 1 -- 172.21.15.7:0/719018820 learned_addr learned my addr 172.21.15.7:0/719018820 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:42.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.252+0000 7fc2f5400640 1 --2- 172.21.15.7:0/719018820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f80d1e60 0x7fc2f815a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:42.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.252+0000 7fc2f6800640 1 -- 172.21.15.7:0/719018820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2f80d07f0 msgr2=0x7fc2f81e86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:42.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.252+0000 7fc2f6800640 1 --2- 172.21.15.7:0/719018820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2f80d07f0 0x7fc2f81e86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.252+0000 7fc2f6800640 1 -- 172.21.15.7:0/719018820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f80d1e60 msgr2=0x7fc2f815a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:42.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.252+0000 7fc2f6800640 1 --2- 172.21.15.7:0/719018820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f80d1e60 0x7fc2f815a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.252+0000 7fc2f6800640 1 -- 172.21.15.7:0/719018820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f81e7340 con 0x7fc2f80d2a20 2024-09-10T21:49:42.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.252+0000 7fc2f6800640 1 --2- 172.21.15.7:0/719018820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2f80d2a20 0x7fc2f80b8240 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000cbc0 tx=0x7fc2f000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:42.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.252+0000 7fc2ee000640 1 -- 172.21.15.7:0/719018820 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0016020 con 0x7fc2f80d2a20 2024-09-10T21:49:42.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.253+0000 7fc2ee000640 1 -- 172.21.15.7:0/719018820 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2f000ae30 con 0x7fc2f80d2a20 2024-09-10T21:49:42.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.253+0000 7fc2ee000640 1 -- 172.21.15.7:0/719018820 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0012660 con 0x7fc2f80d2a20 2024-09-10T21:49:42.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.253+0000 7fc2f7200640 1 -- 172.21.15.7:0/719018820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f81e7620 con 0x7fc2f80d2a20 2024-09-10T21:49:42.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.253+0000 7fc2f7200640 1 -- 172.21.15.7:0/719018820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f81e80d0 con 0x7fc2f80d2a20 2024-09-10T21:49:42.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.254+0000 7fc2f7200640 1 -- 172.21.15.7:0/719018820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2f80751b0 con 0x7fc2f80d2a20 2024-09-10T21:49:42.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.255+0000 7fc2ee000640 1 -- 172.21.15.7:0/719018820 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc2f001a3b0 con 0x7fc2f80d2a20 2024-09-10T21:49:42.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.256+0000 7fc2ee000640 1 --2- 172.21.15.7:0/719018820 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc2d8077fa0 0x7fc2d807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:42.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.256+0000 7fc2ee000640 1 -- 172.21.15.7:0/719018820 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7fc2f009a3f0 con 0x7fc2f80d2a20 2024-09-10T21:49:42.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.256+0000 7fc2f5e00640 1 --2- 172.21.15.7:0/719018820 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc2d8077fa0 0x7fc2d807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:42.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.257+0000 7fc2f5e00640 1 --2- 172.21.15.7:0/719018820 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc2d8077fa0 0x7fc2d807a460 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8034000 tx=0x7fc2e8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:42.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.260+0000 7fc2ee000640 1 -- 172.21.15.7:0/719018820 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc2f0020030 con 0x7fc2f80d2a20 2024-09-10T21:49:42.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.493+0000 7fc2f7200640 1 -- 172.21.15.7:0/719018820 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fc2f80d05d0 con 0x7fc2f80d2a20 2024-09-10T21:49:42.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.493+0000 7fc2ee000640 1 -- 172.21.15.7:0/719018820 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fc2f0062730 con 0x7fc2f80d2a20 2024-09-10T21:49:42.494 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:42.494 INFO:teuthology.orchestra.run.smithi007.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T21:49:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.495+0000 7fc2f7200640 1 -- 172.21.15.7:0/719018820 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc2d8077fa0 msgr2=0x7fc2d807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.495+0000 7fc2f7200640 1 --2- 172.21.15.7:0/719018820 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc2d8077fa0 0x7fc2d807a460 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8034000 tx=0x7fc2e8004730 comp rx=0 tx=0).stop 2024-09-10T21:49:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.495+0000 7fc2f7200640 1 -- 172.21.15.7:0/719018820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2f80d2a20 msgr2=0x7fc2f80b8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.495+0000 7fc2f7200640 1 --2- 172.21.15.7:0/719018820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2f80d2a20 0x7fc2f80b8240 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fc2f000cbc0 tx=0x7fc2f000a430 comp rx=0 tx=0).stop 2024-09-10T21:49:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.495+0000 7fc2f7200640 1 -- 172.21.15.7:0/719018820 shutdown_connections 2024-09-10T21:49:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.495+0000 7fc2f7200640 1 --2- 172.21.15.7:0/719018820 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc2d8077fa0 0x7fc2d807a460 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.495+0000 7fc2f7200640 1 --2- 172.21.15.7:0/719018820 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2f80d1e60 0x7fc2f815a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.495+0000 7fc2f7200640 1 --2- 172.21.15.7:0/719018820 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2f80d2a20 0x7fc2f80b8240 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.495+0000 7fc2f7200640 1 --2- 172.21.15.7:0/719018820 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2f80d07f0 0x7fc2f81e86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.495+0000 7fc2f7200640 1 -- 172.21.15.7:0/719018820 >> 172.21.15.7:0/719018820 conn(0x7fc2f80d3de0 msgr2=0x7fc2f80d41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.495+0000 7fc2f7200640 1 -- 172.21.15.7:0/719018820 shutdown_connections 2024-09-10T21:49:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:42.496+0000 7fc2f7200640 1 -- 172.21.15.7:0/719018820 wait complete. 2024-09-10T21:49:42.512 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-10T21:49:42.512 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:49:42.512 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:49:42.608 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:49:42.705 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:49:42.705 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:49:42.705 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:49:42.705 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:49:42.741 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:49:42.741 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:49:42.816 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:49:43.019 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.018+0000 7ff687c00640 1 -- 172.21.15.7:0/3955013403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6800cc520 msgr2=0x7ff6800cc970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:43.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.018+0000 7ff687c00640 1 --2- 172.21.15.7:0/3955013403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6800cc520 0x7ff6800cc970 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7ff6800cf9e0 tx=0x7ff67802a820 comp rx=0 tx=0).stop 2024-09-10T21:49:43.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.018+0000 7ff687c00640 1 -- 172.21.15.7:0/3955013403 shutdown_connections 2024-09-10T21:49:43.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.018+0000 7ff687c00640 1 --2- 172.21.15.7:0/3955013403 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff6800cc520 0x7ff6800cc970 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:43.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.018+0000 7ff687c00640 1 --2- 172.21.15.7:0/3955013403 >> v2:172.21.15.133:3300/0 conn(0x7ff68015e470 0x7ff6800ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:43.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.018+0000 7ff687c00640 1 -- 172.21.15.7:0/3955013403 >> 172.21.15.7:0/3955013403 conn(0x7ff6800b1660 msgr2=0x7ff6800b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:43.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.018+0000 7ff687c00640 1 -- 172.21.15.7:0/3955013403 shutdown_connections 2024-09-10T21:49:43.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.018+0000 7ff687c00640 1 -- 172.21.15.7:0/3955013403 wait complete. 2024-09-10T21:49:43.020 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.020+0000 7ff687c00640 1 Processor -- start 2024-09-10T21:49:43.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.020+0000 7ff687c00640 1 -- start start 2024-09-10T21:49:43.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.021+0000 7ff687c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6800cc520 0x7ff6801582e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:43.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.021+0000 7ff687c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff68015bc20 0x7ff6800c6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:43.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.021+0000 7ff687c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68015e470 0x7ff6800b2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:43.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.021+0000 7ff687c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff68006f150 con 0x7ff68015e470 2024-09-10T21:49:43.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.021+0000 7ff687c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff68006efd0 con 0x7ff6800cc520 2024-09-10T21:49:43.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.021+0000 7ff687c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ff68006f2d0 con 0x7ff68015bc20 2024-09-10T21:49:43.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.021+0000 7ff685c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68015e470 0x7ff6800b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:43.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.021+0000 7ff685c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68015e470 0x7ff6800b2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47954/0 (socket says 172.21.15.7:47954) 2024-09-10T21:49:43.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.021+0000 7ff685c00640 1 -- 172.21.15.7:0/173979001 learned_addr learned my addr 172.21.15.7:0/173979001 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:43.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.021+0000 7ff685200640 1 --2- 172.21.15.7:0/173979001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6800cc520 0x7ff6801582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:43.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.021+0000 7ff677e00640 1 --2- 172.21.15.7:0/173979001 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff68015bc20 0x7ff6800c6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:43.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.022+0000 7ff685c00640 1 -- 172.21.15.7:0/173979001 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff68015bc20 msgr2=0x7ff6800c6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:43.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.022+0000 7ff685c00640 1 --2- 172.21.15.7:0/173979001 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff68015bc20 0x7ff6800c6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:43.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.022+0000 7ff685c00640 1 -- 172.21.15.7:0/173979001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6800cc520 msgr2=0x7ff6801582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:43.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.022+0000 7ff685c00640 1 --2- 172.21.15.7:0/173979001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6800cc520 0x7ff6801582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:43.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.022+0000 7ff685c00640 1 -- 172.21.15.7:0/173979001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff680072620 con 0x7ff68015e470 2024-09-10T21:49:43.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.022+0000 7ff685200640 1 --2- 172.21.15.7:0/173979001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6800cc520 0x7ff6801582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:43.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.022+0000 7ff685c00640 1 --2- 172.21.15.7:0/173979001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68015e470 0x7ff6800b2960 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7ff678009ca0 tx=0x7ff678032be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:43.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.022+0000 7ff67d600640 1 -- 172.21.15.7:0/173979001 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67803f020 con 0x7ff68015e470 2024-09-10T21:49:43.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.022+0000 7ff67d600640 1 -- 172.21.15.7:0/173979001 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff678004040 con 0x7ff68015e470 2024-09-10T21:49:43.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.022+0000 7ff67d600640 1 -- 172.21.15.7:0/173979001 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff678037040 con 0x7ff68015e470 2024-09-10T21:49:43.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.022+0000 7ff687c00640 1 -- 172.21.15.7:0/173979001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff680072900 con 0x7ff68015e470 2024-09-10T21:49:43.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.022+0000 7ff687c00640 1 -- 172.21.15.7:0/173979001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff680072ec0 con 0x7ff68015e470 2024-09-10T21:49:43.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.024+0000 7ff687c00640 1 -- 172.21.15.7:0/173979001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff68006f780 con 0x7ff68015e470 2024-09-10T21:49:43.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.025+0000 7ff67d600640 1 -- 172.21.15.7:0/173979001 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff67803a030 con 0x7ff68015e470 2024-09-10T21:49:43.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.026+0000 7ff67d600640 1 --2- 172.21.15.7:0/173979001 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff65c077fa0 0x7ff65c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:43.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.026+0000 7ff67d600640 1 -- 172.21.15.7:0/173979001 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7ff6780ba470 con 0x7ff68015e470 2024-09-10T21:49:43.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.026+0000 7ff685200640 1 --2- 172.21.15.7:0/173979001 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff65c077fa0 0x7ff65c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:43.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.027+0000 7ff685200640 1 --2- 172.21.15.7:0/173979001 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff65c077fa0 0x7ff65c07a460 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff6800c69a0 tx=0x7ff670009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:43.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.030+0000 7ff67d600640 1 -- 172.21.15.7:0/173979001 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff678049030 con 0x7ff68015e470 2024-09-10T21:49:43.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.237+0000 7ff687c00640 1 -- 172.21.15.7:0/173979001 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff6800c9d30 con 0x7ff68015e470 2024-09-10T21:49:43.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.238+0000 7ff67d600640 1 -- 172.21.15.7:0/173979001 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v89) ==== 80+0+244 (secure 0 0 0) 0x7ff678082e00 con 0x7ff68015e470 2024-09-10T21:49:43.239 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.133:6837/3312604545 2024-09-11T21:49:03.563183+0000 2024-09-10T21:49:43.239 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.133:6835/3312604545 2024-09-11T21:49:03.563183+0000 2024-09-10T21:49:43.239 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6833/1499133391 2024-09-11T21:47:51.143906+0000 2024-09-10T21:49:43.239 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.7:6832/1499133391 2024-09-11T21:47:51.143906+0000 2024-09-10T21:49:43.240 INFO:teuthology.orchestra.run.smithi007.stderr:listed 4 entries 2024-09-10T21:49:43.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.240+0000 7ff687c00640 1 -- 172.21.15.7:0/173979001 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff65c077fa0 msgr2=0x7ff65c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:43.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.240+0000 7ff687c00640 1 --2- 172.21.15.7:0/173979001 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff65c077fa0 0x7ff65c07a460 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7ff6800c69a0 tx=0x7ff670009040 comp rx=0 tx=0).stop 2024-09-10T21:49:43.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.240+0000 7ff687c00640 1 -- 172.21.15.7:0/173979001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68015e470 msgr2=0x7ff6800b2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:43.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.240+0000 7ff687c00640 1 --2- 172.21.15.7:0/173979001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68015e470 0x7ff6800b2960 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7ff678009ca0 tx=0x7ff678032be0 comp rx=0 tx=0).stop 2024-09-10T21:49:43.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.240+0000 7ff687c00640 1 -- 172.21.15.7:0/173979001 shutdown_connections 2024-09-10T21:49:43.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.240+0000 7ff687c00640 1 --2- 172.21.15.7:0/173979001 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff65c077fa0 0x7ff65c07a460 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:43.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.240+0000 7ff687c00640 1 --2- 172.21.15.7:0/173979001 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff68015e470 0x7ff6800b2960 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:43.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.240+0000 7ff687c00640 1 --2- 172.21.15.7:0/173979001 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff68015bc20 0x7ff6800c6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:43.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.240+0000 7ff687c00640 1 --2- 172.21.15.7:0/173979001 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff6800cc520 0x7ff6801582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:43.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.240+0000 7ff687c00640 1 -- 172.21.15.7:0/173979001 >> 172.21.15.7:0/173979001 conn(0x7ff6800b1660 msgr2=0x7ff6800cb590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:43.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.240+0000 7ff687c00640 1 -- 172.21.15.7:0/173979001 shutdown_connections 2024-09-10T21:49:43.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.240+0000 7ff687c00640 1 -- 172.21.15.7:0/173979001 wait complete. 2024-09-10T21:49:43.258 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:49:43.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:49:43.258 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:49:43.319 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:49:43.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:49:43.320 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:49:43.355 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:49:43.356 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:49:43.356 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:49:43.395 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:49:43.396 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:49:43.396 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:49:43.396 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:49:43.431 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:49:43.431 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:49:43.431 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:49:43.431 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:49:43.475 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:49:43.476 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:49:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.635+0000 7f4927a00640 1 -- 172.21.15.7:0/3139819010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49201619e0 msgr2=0x7f492015e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.635+0000 7f4927a00640 1 --2- 172.21.15.7:0/3139819010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49201619e0 0x7f492015e3c0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f4910005fd0 tx=0x7f491002a5d0 comp rx=0 tx=0).stop 2024-09-10T21:49:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.635+0000 7f4927a00640 1 -- 172.21.15.7:0/3139819010 shutdown_connections 2024-09-10T21:49:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.635+0000 7f4927a00640 1 --2- 172.21.15.7:0/3139819010 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49200cbd50 0x7f49200cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.636+0000 7f4927a00640 1 --2- 172.21.15.7:0/3139819010 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49201619e0 0x7f492015e3c0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:43.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.636+0000 7f4927a00640 1 -- 172.21.15.7:0/3139819010 >> 172.21.15.7:0/3139819010 conn(0x7f49200b16d0 msgr2=0x7f49200b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:43.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.636+0000 7f4927a00640 1 -- 172.21.15.7:0/3139819010 shutdown_connections 2024-09-10T21:49:43.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.636+0000 7f4927a00640 1 -- 172.21.15.7:0/3139819010 wait complete. 2024-09-10T21:49:43.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.637+0000 7f4927a00640 1 Processor -- start 2024-09-10T21:49:43.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.637+0000 7f4927a00640 1 -- start start 2024-09-10T21:49:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.638+0000 7f4927a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49200ae020 0x7f492014f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.638+0000 7f4927a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49200cbd50 0x7f49201f4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.638+0000 7f4927a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49201619e0 0x7f49201f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:43.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.638+0000 7f4927a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49200ae950 con 0x7f49200cbd50 2024-09-10T21:49:43.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.638+0000 7f4927a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f49200ae7d0 con 0x7f49201619e0 2024-09-10T21:49:43.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.638+0000 7f4927a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4920161dc0 con 0x7f49200ae020 2024-09-10T21:49:43.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.638+0000 7f4917e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49200cbd50 0x7f49201f4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:43.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.638+0000 7f4917e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49200cbd50 0x7f49201f4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:47982/0 (socket says 172.21.15.7:47982) 2024-09-10T21:49:43.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.638+0000 7f4917e00640 1 -- 172.21.15.7:0/2520994646 learned_addr learned my addr 172.21.15.7:0/2520994646 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:43.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.639+0000 7f4925000640 1 --2- 172.21.15.7:0/2520994646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49200ae020 0x7f492014f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:43.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.639+0000 7f4925a00640 1 --2- 172.21.15.7:0/2520994646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49201619e0 0x7f49201f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:43.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.639+0000 7f4917e00640 1 -- 172.21.15.7:0/2520994646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49200ae020 msgr2=0x7f492014f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:43.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.639+0000 7f4917e00640 1 --2- 172.21.15.7:0/2520994646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49200ae020 0x7f492014f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:43.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.639+0000 7f4917e00640 1 -- 172.21.15.7:0/2520994646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49201619e0 msgr2=0x7f49201f8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:43.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.639+0000 7f4917e00640 1 --2- 172.21.15.7:0/2520994646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49201619e0 0x7f49201f8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:43.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.639+0000 7f4917e00640 1 -- 172.21.15.7:0/2520994646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49200c3920 con 0x7f49200cbd50 2024-09-10T21:49:43.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.639+0000 7f4925000640 1 --2- 172.21.15.7:0/2520994646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49200ae020 0x7f492014f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:43.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.639+0000 7f4925a00640 1 --2- 172.21.15.7:0/2520994646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49201619e0 0x7f49201f8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:43.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.639+0000 7f4917e00640 1 --2- 172.21.15.7:0/2520994646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49200cbd50 0x7f49201f4ba0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f490800d770 tx=0x7f490800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:43.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.639+0000 7f491d600640 1 -- 172.21.15.7:0/2520994646 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908017070 con 0x7f49200cbd50 2024-09-10T21:49:43.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.639+0000 7f491d600640 1 -- 172.21.15.7:0/2520994646 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4908009d20 con 0x7f49200cbd50 2024-09-10T21:49:43.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.639+0000 7f491d600640 1 -- 172.21.15.7:0/2520994646 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4908004280 con 0x7f49200cbd50 2024-09-10T21:49:43.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.639+0000 7f4927a00640 1 -- 172.21.15.7:0/2520994646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49201f5210 con 0x7f49200cbd50 2024-09-10T21:49:43.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.640+0000 7f4927a00640 1 -- 172.21.15.7:0/2520994646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49201f57d0 con 0x7f49200cbd50 2024-09-10T21:49:43.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.641+0000 7f4927a00640 1 -- 172.21.15.7:0/2520994646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49200cc190 con 0x7f49200cbd50 2024-09-10T21:49:43.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.642+0000 7f491d600640 1 -- 172.21.15.7:0/2520994646 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f49080035f0 con 0x7f49200cbd50 2024-09-10T21:49:43.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.643+0000 7f491d600640 1 --2- 172.21.15.7:0/2520994646 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f48fc077f70 0x7f48fc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:43.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.643+0000 7f491d600640 1 -- 172.21.15.7:0/2520994646 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f4908099f80 con 0x7f49200cbd50 2024-09-10T21:49:43.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.644+0000 7f4925000640 1 --2- 172.21.15.7:0/2520994646 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f48fc077f70 0x7f48fc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:43.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.645+0000 7f4925000640 1 --2- 172.21.15.7:0/2520994646 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f48fc077f70 0x7f48fc07a430 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4910033000 tx=0x7f4910004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:43.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.647+0000 7f491d600640 1 -- 172.21.15.7:0/2520994646 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f49080629a0 con 0x7f49200cbd50 2024-09-10T21:49:43.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.879+0000 7f4927a00640 1 -- 172.21.15.7:0/2520994646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f492014f710 con 0x7f49200cbd50 2024-09-10T21:49:43.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.879+0000 7f491d600640 1 -- 172.21.15.7:0/2520994646 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5229 (secure 0 0 0) 0x7f49080622a0 con 0x7f49200cbd50 2024-09-10T21:49:43.880 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:43.881 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":76,"btime":"2024-09-10T21:49:07:587035+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/978763848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":978763848},{"type":"v1","addr":"172.21.15.7:6835","nonce":978763848}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/269527918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":269527918},{"type":"v1","addr":"172.21.15.7:6833","nonce":269527918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":235775,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/760980985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":760980985},{"type":"v1","addr":"172.21.15.133:6836","nonce":760980985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:47:40.664091+0000","modified":"2024-09-10T21:49:07.587028+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236099":{"gid":236099,"name":"d","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6837/4188280820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4188280820},{"type":"v1","addr":"172.21.15.133:6837","nonce":4188280820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":236099,"qdb_cluster":[236099]},"id":6}]} 2024-09-10T21:49:43.881 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 76 2024-09-10T21:49:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.882+0000 7f4927a00640 1 -- 172.21.15.7:0/2520994646 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f48fc077f70 msgr2=0x7f48fc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.882+0000 7f4927a00640 1 --2- 172.21.15.7:0/2520994646 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f48fc077f70 0x7f48fc07a430 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4910033000 tx=0x7f4910004730 comp rx=0 tx=0).stop 2024-09-10T21:49:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.882+0000 7f4927a00640 1 -- 172.21.15.7:0/2520994646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49200cbd50 msgr2=0x7f49201f4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.882+0000 7f4927a00640 1 --2- 172.21.15.7:0/2520994646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49200cbd50 0x7f49201f4ba0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f490800d770 tx=0x7f490800dc30 comp rx=0 tx=0).stop 2024-09-10T21:49:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.882+0000 7f4927a00640 1 -- 172.21.15.7:0/2520994646 shutdown_connections 2024-09-10T21:49:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.882+0000 7f4927a00640 1 --2- 172.21.15.7:0/2520994646 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f48fc077f70 0x7f48fc07a430 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:43.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.882+0000 7f4927a00640 1 --2- 172.21.15.7:0/2520994646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f49201619e0 0x7f49201f8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.882+0000 7f4927a00640 1 --2- 172.21.15.7:0/2520994646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49200cbd50 0x7f49201f4ba0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.882+0000 7f4927a00640 1 --2- 172.21.15.7:0/2520994646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f49200ae020 0x7f492014f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.882+0000 7f4927a00640 1 -- 172.21.15.7:0/2520994646 >> 172.21.15.7:0/2520994646 conn(0x7f49200b16d0 msgr2=0x7f492015ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.882+0000 7f4927a00640 1 -- 172.21.15.7:0/2520994646 shutdown_connections 2024-09-10T21:49:43.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:43.882+0000 7f4927a00640 1 -- 172.21.15.7:0/2520994646 wait complete. 2024-09-10T21:49:43.899 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:49:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.059+0000 7f249e000640 1 -- 172.21.15.7:0/2053293334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24981604b0 msgr2=0x7f24980ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.059+0000 7f249e000640 1 --2- 172.21.15.7:0/2053293334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24981604b0 0x7f24980ccef0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f2490005fd0 tx=0x7f249002a820 comp rx=0 tx=0).stop 2024-09-10T21:49:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.059+0000 7f249e000640 1 -- 172.21.15.7:0/2053293334 shutdown_connections 2024-09-10T21:49:44.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.059+0000 7f249e000640 1 --2- 172.21.15.7:0/2053293334 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24981604b0 0x7f24980ccef0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.059+0000 7f249e000640 1 -- 172.21.15.7:0/2053293334 >> 172.21.15.7:0/2053293334 conn(0x7f24980b16d0 msgr2=0x7f24980b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:44.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.059+0000 7f249e000640 1 -- 172.21.15.7:0/2053293334 shutdown_connections 2024-09-10T21:49:44.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.059+0000 7f249e000640 1 -- 172.21.15.7:0/2053293334 wait complete. 2024-09-10T21:49:44.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.061+0000 7f249e000640 1 Processor -- start 2024-09-10T21:49:44.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.061+0000 7f249e000640 1 -- start start 2024-09-10T21:49:44.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.062+0000 7f249e000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f24980cd750 0x7f24980ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:44.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.062+0000 7f249e000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f249815bc90 0x7f24981eed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:44.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.062+0000 7f249e000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24981604b0 0x7f2498156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:44.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.062+0000 7f249e000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2498070080 con 0x7f24981604b0 2024-09-10T21:49:44.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.062+0000 7f249e000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f249806ff00 con 0x7f249815bc90 2024-09-10T21:49:44.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.062+0000 7f249e000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f2498070200 con 0x7f24980cd750 2024-09-10T21:49:44.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.062+0000 7f2497e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24981604b0 0x7f2498156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:44.064 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.062+0000 7f2497e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24981604b0 0x7f2498156730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37180/0 (socket says 172.21.15.7:37180) 2024-09-10T21:49:44.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.062+0000 7f2497e00640 1 -- 172.21.15.7:0/4147151725 learned_addr learned my addr 172.21.15.7:0/4147151725 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:44.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.062+0000 7f248ea00640 1 --2- 172.21.15.7:0/4147151725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f249815bc90 0x7f24981eed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:44.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.062+0000 7f2497400640 1 --2- 172.21.15.7:0/4147151725 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f24980cd750 0x7f24980ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:44.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.062+0000 7f2497e00640 1 -- 172.21.15.7:0/4147151725 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f24980cd750 msgr2=0x7f24980ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:44.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.062+0000 7f2497e00640 1 --2- 172.21.15.7:0/4147151725 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f24980cd750 0x7f24980ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.062+0000 7f2497e00640 1 -- 172.21.15.7:0/4147151725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f249815bc90 msgr2=0x7f24981eed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:44.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.062+0000 7f2497e00640 1 --2- 172.21.15.7:0/4147151725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f249815bc90 0x7f24981eed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.062+0000 7f2497e00640 1 -- 172.21.15.7:0/4147151725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24980c87c0 con 0x7f24981604b0 2024-09-10T21:49:44.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.062+0000 7f2497400640 1 --2- 172.21.15.7:0/4147151725 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f24980cd750 0x7f24980ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:44.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.063+0000 7f2497e00640 1 --2- 172.21.15.7:0/4147151725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24981604b0 0x7f2498156730 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f2490006010 tx=0x7f2490032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:44.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.063+0000 7f248fe00640 1 -- 172.21.15.7:0/4147151725 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490037040 con 0x7f24981604b0 2024-09-10T21:49:44.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.063+0000 7f248fe00640 1 -- 172.21.15.7:0/4147151725 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2490042070 con 0x7f24981604b0 2024-09-10T21:49:44.065 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.063+0000 7f249e000640 1 -- 172.21.15.7:0/4147151725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24981571b0 con 0x7f24981604b0 2024-09-10T21:49:44.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.063+0000 7f248fe00640 1 -- 172.21.15.7:0/4147151725 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2490003e60 con 0x7f24981604b0 2024-09-10T21:49:44.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.063+0000 7f249e000640 1 -- 172.21.15.7:0/4147151725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24980c8d40 con 0x7f24981604b0 2024-09-10T21:49:44.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.064+0000 7f249e000640 1 -- 172.21.15.7:0/4147151725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f249815f5e0 con 0x7f24981604b0 2024-09-10T21:49:44.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.065+0000 7f248fe00640 1 -- 172.21.15.7:0/4147151725 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f249003a030 con 0x7f24981604b0 2024-09-10T21:49:44.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.067+0000 7f248fe00640 1 --2- 172.21.15.7:0/4147151725 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2474077f70 0x7f247407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:44.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.067+0000 7f248fe00640 1 -- 172.21.15.7:0/4147151725 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f24900ba760 con 0x7f24981604b0 2024-09-10T21:49:44.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.067+0000 7f2497400640 1 --2- 172.21.15.7:0/4147151725 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2474077f70 0x7f247407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:44.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.068+0000 7f2497400640 1 --2- 172.21.15.7:0/4147151725 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2474077f70 0x7f247407a430 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f24980c3300 tx=0x7f2488009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.071+0000 7f248fe00640 1 -- 172.21.15.7:0/4147151725 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2490083180 con 0x7f24981604b0 2024-09-10T21:49:44.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.301+0000 7f249e000640 1 -- 172.21.15.7:0/4147151725 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f249815fe40 con 0x7f24981604b0 2024-09-10T21:49:44.303 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.302+0000 7f248fe00640 1 -- 172.21.15.7:0/4147151725 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5229 (secure 0 0 0) 0x7f2490082a80 con 0x7f24981604b0 2024-09-10T21:49:44.303 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:44.303 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":76,"btime":"2024-09-10T21:49:07:587035+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/978763848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":978763848},{"type":"v1","addr":"172.21.15.7:6835","nonce":978763848}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/269527918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":269527918},{"type":"v1","addr":"172.21.15.7:6833","nonce":269527918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":235775,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/760980985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":760980985},{"type":"v1","addr":"172.21.15.133:6836","nonce":760980985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:47:40.664091+0000","modified":"2024-09-10T21:49:07.587028+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236099":{"gid":236099,"name":"d","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6837/4188280820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4188280820},{"type":"v1","addr":"172.21.15.133:6837","nonce":4188280820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":236099,"qdb_cluster":[236099]},"id":6}]} 2024-09-10T21:49:44.303 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 76 2024-09-10T21:49:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.304+0000 7f249e000640 1 -- 172.21.15.7:0/4147151725 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2474077f70 msgr2=0x7f247407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.304+0000 7f249e000640 1 --2- 172.21.15.7:0/4147151725 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2474077f70 0x7f247407a430 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f24980c3300 tx=0x7f2488009040 comp rx=0 tx=0).stop 2024-09-10T21:49:44.305 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.304+0000 7f249e000640 1 -- 172.21.15.7:0/4147151725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24981604b0 msgr2=0x7f2498156730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.304+0000 7f249e000640 1 --2- 172.21.15.7:0/4147151725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24981604b0 0x7f2498156730 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f2490006010 tx=0x7f2490032d10 comp rx=0 tx=0).stop 2024-09-10T21:49:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.304+0000 7f249e000640 1 -- 172.21.15.7:0/4147151725 shutdown_connections 2024-09-10T21:49:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.304+0000 7f249e000640 1 --2- 172.21.15.7:0/4147151725 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2474077f70 0x7f247407a430 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.304+0000 7f249e000640 1 --2- 172.21.15.7:0/4147151725 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f249815bc90 0x7f24981eed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.304+0000 7f249e000640 1 --2- 172.21.15.7:0/4147151725 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f24981604b0 0x7f2498156730 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.304+0000 7f249e000640 1 --2- 172.21.15.7:0/4147151725 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f24980cd750 0x7f24980ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.304+0000 7f249e000640 1 -- 172.21.15.7:0/4147151725 >> 172.21.15.7:0/4147151725 conn(0x7f24980b16d0 msgr2=0x7f24980b42f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.305+0000 7f249e000640 1 -- 172.21.15.7:0/4147151725 shutdown_connections 2024-09-10T21:49:44.306 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.305+0000 7f249e000640 1 -- 172.21.15.7:0/4147151725 wait complete. 2024-09-10T21:49:44.322 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:49:44.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.482+0000 7fcb95000640 1 -- 172.21.15.7:0/2246221929 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcb90146c50 msgr2=0x7fcb900723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:44.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.482+0000 7fcb95000640 1 --2- 172.21.15.7:0/2246221929 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcb90146c50 0x7fcb900723b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009830 tx=0x7fcb8002a480 comp rx=0 tx=0).stop 2024-09-10T21:49:44.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.482+0000 7fcb95000640 1 -- 172.21.15.7:0/2246221929 shutdown_connections 2024-09-10T21:49:44.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.482+0000 7fcb95000640 1 --2- 172.21.15.7:0/2246221929 >> v2:172.21.15.133:3300/0 conn(0x7fcb901470b0 0x7fcb900ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.482+0000 7fcb95000640 1 --2- 172.21.15.7:0/2246221929 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcb90146c50 0x7fcb900723b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.482+0000 7fcb95000640 1 -- 172.21.15.7:0/2246221929 >> 172.21.15.7:0/2246221929 conn(0x7fcb900b1d70 msgr2=0x7fcb900cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.482+0000 7fcb95000640 1 -- 172.21.15.7:0/2246221929 shutdown_connections 2024-09-10T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.482+0000 7fcb95000640 1 -- 172.21.15.7:0/2246221929 wait complete. 2024-09-10T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.484+0000 7fcb95000640 1 Processor -- start 2024-09-10T21:49:44.484 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.484+0000 7fcb95000640 1 -- start start 2024-09-10T21:49:44.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.485+0000 7fcb95000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcb9006f590 0x7fcb901455e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:44.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.485+0000 7fcb95000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb90146c50 0x7fcb901f4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:44.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.485+0000 7fcb95000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb901470b0 0x7fcb901e99d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:44.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.485+0000 7fcb95000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcb900ae1b0 con 0x7fcb90146c50 2024-09-10T21:49:44.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.485+0000 7fcb95000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcb900ae030 con 0x7fcb901470b0 2024-09-10T21:49:44.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.485+0000 7fcb95000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fcb900ae330 con 0x7fcb9006f590 2024-09-10T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.485+0000 7fcb86a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb90146c50 0x7fcb901f4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.485+0000 7fcb86a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb90146c50 0x7fcb901f4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37184/0 (socket says 172.21.15.7:37184) 2024-09-10T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.485+0000 7fcb86a00640 1 -- 172.21.15.7:0/3972763404 learned_addr learned my addr 172.21.15.7:0/3972763404 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.485+0000 7fcb8f400640 1 --2- 172.21.15.7:0/3972763404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcb9006f590 0x7fcb901455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.485+0000 7fcb8fe00640 1 --2- 172.21.15.7:0/3972763404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb901470b0 0x7fcb901e99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.485+0000 7fcb86a00640 1 -- 172.21.15.7:0/3972763404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcb9006f590 msgr2=0x7fcb901455e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.485+0000 7fcb86a00640 1 --2- 172.21.15.7:0/3972763404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcb9006f590 0x7fcb901455e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.485+0000 7fcb86a00640 1 -- 172.21.15.7:0/3972763404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb901470b0 msgr2=0x7fcb901e99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.485+0000 7fcb86a00640 1 --2- 172.21.15.7:0/3972763404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb901470b0 0x7fcb901e99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.487 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.485+0000 7fcb86a00640 1 -- 172.21.15.7:0/3972763404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb901e8930 con 0x7fcb90146c50 2024-09-10T21:49:44.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.486+0000 7fcb86a00640 1 --2- 172.21.15.7:0/3972763404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb90146c50 0x7fcb901f4eb0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800c420 tx=0x7fcb7800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:44.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.486+0000 7fcb87e00640 1 -- 172.21.15.7:0/3972763404 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78017070 con 0x7fcb90146c50 2024-09-10T21:49:44.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.486+0000 7fcb87e00640 1 -- 172.21.15.7:0/3972763404 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb780053c0 con 0x7fcb90146c50 2024-09-10T21:49:44.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.486+0000 7fcb95000640 1 -- 172.21.15.7:0/3972763404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb901e8c10 con 0x7fcb90146c50 2024-09-10T21:49:44.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.486+0000 7fcb87e00640 1 -- 172.21.15.7:0/3972763404 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb78003850 con 0x7fcb90146c50 2024-09-10T21:49:44.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.486+0000 7fcb95000640 1 -- 172.21.15.7:0/3972763404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb901e91d0 con 0x7fcb90146c50 2024-09-10T21:49:44.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.487+0000 7fcb95000640 1 -- 172.21.15.7:0/3972763404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb90069fc0 con 0x7fcb90146c50 2024-09-10T21:49:44.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.488+0000 7fcb87e00640 1 -- 172.21.15.7:0/3972763404 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fcb780039f0 con 0x7fcb90146c50 2024-09-10T21:49:44.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.489+0000 7fcb87e00640 1 --2- 172.21.15.7:0/3972763404 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcb6c077f70 0x7fcb6c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:44.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.489+0000 7fcb87e00640 1 -- 172.21.15.7:0/3972763404 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7fcb78099480 con 0x7fcb90146c50 2024-09-10T21:49:44.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.490+0000 7fcb8f400640 1 --2- 172.21.15.7:0/3972763404 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcb6c077f70 0x7fcb6c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:44.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.490+0000 7fcb8f400640 1 --2- 172.21.15.7:0/3972763404 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcb6c077f70 0x7fcb6c07a430 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcb80034000 tx=0x7fcb80004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:44.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.493+0000 7fcb87e00640 1 -- 172.21.15.7:0/3972763404 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fcb90147950 con 0x7fcb90146c50 2024-09-10T21:49:44.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.689+0000 7fcb95000640 1 -- 172.21.15.7:0/3972763404 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcb90161570 con 0x7fcb90146c50 2024-09-10T21:49:44.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.690+0000 7fcb87e00640 1 -- 172.21.15.7:0/3972763404 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16993 (secure 0 0 0) 0x7fcb780617d0 con 0x7fcb90146c50 2024-09-10T21:49:44.692 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:44.692 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:49:03.563205+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:47:38.456723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T21:47:39.629851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6837/3312604545":"2024-09-11T21:49:03.563183+0000","172.21.15.133:6835/3312604545":"2024-09-11T21:49:03.563183+0000","172.21.15.7:6833/1499133391":"2024-09-11T21:47:51.143906+0000","172.21.15.7:6832/1499133391":"2024-09-11T21:47:51.143906+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:49:44.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.693+0000 7fcb95000640 1 -- 172.21.15.7:0/3972763404 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcb6c077f70 msgr2=0x7fcb6c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.693+0000 7fcb95000640 1 --2- 172.21.15.7:0/3972763404 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcb6c077f70 0x7fcb6c07a430 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcb80034000 tx=0x7fcb80004730 comp rx=0 tx=0).stop 2024-09-10T21:49:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.693+0000 7fcb95000640 1 -- 172.21.15.7:0/3972763404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb90146c50 msgr2=0x7fcb901f4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.693+0000 7fcb95000640 1 --2- 172.21.15.7:0/3972763404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb90146c50 0x7fcb901f4eb0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800c420 tx=0x7fcb7800c8e0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.693+0000 7fcb95000640 1 -- 172.21.15.7:0/3972763404 shutdown_connections 2024-09-10T21:49:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.693+0000 7fcb95000640 1 --2- 172.21.15.7:0/3972763404 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcb6c077f70 0x7fcb6c07a430 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.693+0000 7fcb95000640 1 --2- 172.21.15.7:0/3972763404 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcb901470b0 0x7fcb901e99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.693+0000 7fcb95000640 1 --2- 172.21.15.7:0/3972763404 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcb90146c50 0x7fcb901f4eb0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.693+0000 7fcb95000640 1 --2- 172.21.15.7:0/3972763404 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcb9006f590 0x7fcb901455e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.693+0000 7fcb95000640 1 -- 172.21.15.7:0/3972763404 >> 172.21.15.7:0/3972763404 conn(0x7fcb900b1d70 msgr2=0x7fcb90160dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.693+0000 7fcb95000640 1 -- 172.21.15.7:0/3972763404 shutdown_connections 2024-09-10T21:49:44.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.693+0000 7fcb95000640 1 -- 172.21.15.7:0/3972763404 wait complete. 2024-09-10T21:49:44.710 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:49:44.710 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:49:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.871+0000 7f30d5c00640 1 -- 172.21.15.7:0/4144835206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015fda0 msgr2=0x7f30d01601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.871+0000 7f30d5c00640 1 --2- 172.21.15.7:0/4144835206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015fda0 0x7f30d01601f0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f30c8005fd0 tx=0x7f30c802a980 comp rx=0 tx=0).stop 2024-09-10T21:49:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.871+0000 7f30d5c00640 1 -- 172.21.15.7:0/4144835206 shutdown_connections 2024-09-10T21:49:44.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.871+0000 7f30d5c00640 1 --2- 172.21.15.7:0/4144835206 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015fda0 0x7f30d01601f0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.871+0000 7f30d5c00640 1 --2- 172.21.15.7:0/4144835206 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30d015bc90 0x7f30d00ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.871+0000 7f30d5c00640 1 -- 172.21.15.7:0/4144835206 >> 172.21.15.7:0/4144835206 conn(0x7f30d00b16d0 msgr2=0x7f30d00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.871+0000 7f30d5c00640 1 -- 172.21.15.7:0/4144835206 shutdown_connections 2024-09-10T21:49:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.871+0000 7f30d5c00640 1 -- 172.21.15.7:0/4144835206 wait complete. 2024-09-10T21:49:44.873 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.873+0000 7f30d5c00640 1 Processor -- start 2024-09-10T21:49:44.874 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.873+0000 7f30d5c00640 1 -- start start 2024-09-10T21:49:44.875 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.874+0000 7f30d5c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d00cd750 0x7f30d01f88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.874+0000 7f30d5c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015bc90 0x7f30d00b0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.874+0000 7f30d5c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30d015fda0 0x7f30d00c8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.874+0000 7f30d5c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f30d006fe70 con 0x7f30d015bc90 2024-09-10T21:49:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.874+0000 7f30d5c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f30d006fcf0 con 0x7f30d00cd750 2024-09-10T21:49:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.874+0000 7f30d5c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f30d006fff0 con 0x7f30d015fda0 2024-09-10T21:49:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.874+0000 7f30ce000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015bc90 0x7f30d00b0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.874+0000 7f30ce000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015bc90 0x7f30d00b0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37204/0 (socket says 172.21.15.7:37204) 2024-09-10T21:49:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.874+0000 7f30ce000640 1 -- 172.21.15.7:0/2545330261 learned_addr learned my addr 172.21.15.7:0/2545330261 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.874+0000 7f30cea00640 1 --2- 172.21.15.7:0/2545330261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d00cd750 0x7f30d01f88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.874+0000 7f30cf400640 1 --2- 172.21.15.7:0/2545330261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30d015fda0 0x7f30d00c8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:44.876 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.875+0000 7f30ce000640 1 -- 172.21.15.7:0/2545330261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30d015fda0 msgr2=0x7f30d00c8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.875+0000 7f30ce000640 1 --2- 172.21.15.7:0/2545330261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30d015fda0 0x7f30d00c8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.875+0000 7f30ce000640 1 -- 172.21.15.7:0/2545330261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d00cd750 msgr2=0x7f30d01f88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.875+0000 7f30ce000640 1 --2- 172.21.15.7:0/2545330261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d00cd750 0x7f30d01f88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:44.877 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.875+0000 7f30ce000640 1 -- 172.21.15.7:0/2545330261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d01f7e30 con 0x7f30d015bc90 2024-09-10T21:49:44.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.875+0000 7f30ce000640 1 --2- 172.21.15.7:0/2545330261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015bc90 0x7f30d00b0ee0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00c500 tx=0x7f30bc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:44.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.875+0000 7f30c6a00640 1 -- 172.21.15.7:0/2545330261 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc017070 con 0x7f30d015bc90 2024-09-10T21:49:44.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.876+0000 7f30c6a00640 1 -- 172.21.15.7:0/2545330261 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30bc0035f0 con 0x7f30d015bc90 2024-09-10T21:49:44.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.876+0000 7f30c6a00640 1 -- 172.21.15.7:0/2545330261 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30bc003a40 con 0x7f30d015bc90 2024-09-10T21:49:44.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.876+0000 7f30d5c00640 1 -- 172.21.15.7:0/2545330261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d00c9a30 con 0x7f30d015bc90 2024-09-10T21:49:44.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.876+0000 7f30d5c00640 1 -- 172.21.15.7:0/2545330261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d01f80d0 con 0x7f30d015bc90 2024-09-10T21:49:44.878 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.877+0000 7f30d5c00640 1 -- 172.21.15.7:0/2545330261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d0070510 con 0x7f30d015bc90 2024-09-10T21:49:44.879 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.878+0000 7f30c6a00640 1 -- 172.21.15.7:0/2545330261 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f30bc003be0 con 0x7f30d015bc90 2024-09-10T21:49:44.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.879+0000 7f30c6a00640 1 --2- 172.21.15.7:0/2545330261 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f30b0077f70 0x7f30b007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:44.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.879+0000 7f30c6a00640 1 -- 172.21.15.7:0/2545330261 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f30bc099550 con 0x7f30d015bc90 2024-09-10T21:49:44.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.880+0000 7f30cea00640 1 --2- 172.21.15.7:0/2545330261 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f30b0077f70 0x7f30b007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:44.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.880+0000 7f30cea00640 1 --2- 172.21.15.7:0/2545330261 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f30b0077f70 0x7f30b007a430 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f30d00cc020 tx=0x7f30c000b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:44.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:44.885+0000 7f30c6a00640 1 -- 172.21.15.7:0/2545330261 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f30bc061920 con 0x7f30d015bc90 2024-09-10T21:49:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.116+0000 7f30d5c00640 1 -- 172.21.15.7:0/2545330261 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f30d00c3680 con 0x7f30d015bc90 2024-09-10T21:49:45.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.116+0000 7f30c6a00640 1 -- 172.21.15.7:0/2545330261 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5229 (secure 0 0 0) 0x7f30bc066350 con 0x7f30d015bc90 2024-09-10T21:49:45.118 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:45.118 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":76,"btime":"2024-09-10T21:49:07:587035+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/978763848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":978763848},{"type":"v1","addr":"172.21.15.7:6835","nonce":978763848}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/269527918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":269527918},{"type":"v1","addr":"172.21.15.7:6833","nonce":269527918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":235775,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/760980985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":760980985},{"type":"v1","addr":"172.21.15.133:6836","nonce":760980985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:47:40.664091+0000","modified":"2024-09-10T21:49:07.587028+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236099":{"gid":236099,"name":"d","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6837/4188280820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4188280820},{"type":"v1","addr":"172.21.15.133:6837","nonce":4188280820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":236099,"qdb_cluster":[236099]},"id":6}]} 2024-09-10T21:49:45.118 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 76 2024-09-10T21:49:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.119+0000 7f30d5c00640 1 -- 172.21.15.7:0/2545330261 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f30b0077f70 msgr2=0x7f30b007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.119+0000 7f30d5c00640 1 --2- 172.21.15.7:0/2545330261 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f30b0077f70 0x7f30b007a430 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f30d00cc020 tx=0x7f30c000b390 comp rx=0 tx=0).stop 2024-09-10T21:49:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.119+0000 7f30d5c00640 1 -- 172.21.15.7:0/2545330261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015bc90 msgr2=0x7f30d00b0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.119+0000 7f30d5c00640 1 --2- 172.21.15.7:0/2545330261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015bc90 0x7f30d00b0ee0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f30bc00c500 tx=0x7f30bc00c9c0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.119+0000 7f30d5c00640 1 -- 172.21.15.7:0/2545330261 shutdown_connections 2024-09-10T21:49:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.119+0000 7f30d5c00640 1 --2- 172.21.15.7:0/2545330261 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f30b0077f70 0x7f30b007a430 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.119+0000 7f30d5c00640 1 --2- 172.21.15.7:0/2545330261 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f30d015fda0 0x7f30d00c8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.119+0000 7f30d5c00640 1 --2- 172.21.15.7:0/2545330261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f30d015bc90 0x7f30d00b0ee0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.119+0000 7f30d5c00640 1 --2- 172.21.15.7:0/2545330261 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f30d00cd750 0x7f30d01f88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.119+0000 7f30d5c00640 1 -- 172.21.15.7:0/2545330261 >> 172.21.15.7:0/2545330261 conn(0x7f30d00b16d0 msgr2=0x7f30d00c5ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:45.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.119+0000 7f30d5c00640 1 -- 172.21.15.7:0/2545330261 shutdown_connections 2024-09-10T21:49:45.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.119+0000 7f30d5c00640 1 -- 172.21.15.7:0/2545330261 wait complete. 2024-09-10T21:49:45.136 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:49:45.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.295+0000 7f59bee00640 1 -- 172.21.15.7:0/1611565387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b80cd750 msgr2=0x7f59b8073730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:45.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.295+0000 7f59bee00640 1 --2- 172.21.15.7:0/1611565387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b80cd750 0x7f59b8073730 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f59b806dac0 tx=0x7f59a002a560 comp rx=0 tx=0).stop 2024-09-10T21:49:45.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.295+0000 7f59bee00640 1 -- 172.21.15.7:0/1611565387 shutdown_connections 2024-09-10T21:49:45.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.295+0000 7f59bee00640 1 --2- 172.21.15.7:0/1611565387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b80cd750 0x7f59b8073730 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.295+0000 7f59bee00640 1 --2- 172.21.15.7:0/1611565387 >> v2:172.21.15.133:3300/0 conn(0x7f59b815bc90 0x7f59b80ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.295+0000 7f59bee00640 1 -- 172.21.15.7:0/1611565387 >> 172.21.15.7:0/1611565387 conn(0x7f59b80b16d0 msgr2=0x7f59b80b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:45.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.295+0000 7f59bee00640 1 -- 172.21.15.7:0/1611565387 shutdown_connections 2024-09-10T21:49:45.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.295+0000 7f59bee00640 1 -- 172.21.15.7:0/1611565387 wait complete. 2024-09-10T21:49:45.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.296+0000 7f59bee00640 1 Processor -- start 2024-09-10T21:49:45.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.297+0000 7f59bee00640 1 -- start start 2024-09-10T21:49:45.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.297+0000 7f59bee00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59b8073c70 0x7f59b814e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:45.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.297+0000 7f59bee00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b80cd750 0x7f59b80c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:45.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.297+0000 7f59bee00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b815bc90 0x7f59b80724f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:45.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.298+0000 7f59bee00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f59b806fcf0 con 0x7f59b80cd750 2024-09-10T21:49:45.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.298+0000 7f59bee00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f59b81606c0 con 0x7f59b815bc90 2024-09-10T21:49:45.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.298+0000 7f59bee00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f59b806fe70 con 0x7f59b8073c70 2024-09-10T21:49:45.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.298+0000 7f59afe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b80cd750 0x7f59b80c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:45.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.298+0000 7f59afe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b80cd750 0x7f59b80c7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37228/0 (socket says 172.21.15.7:37228) 2024-09-10T21:49:45.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.298+0000 7f59afe00640 1 -- 172.21.15.7:0/1989525898 learned_addr learned my addr 172.21.15.7:0/1989525898 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:45.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.298+0000 7f59b7e00640 1 --2- 172.21.15.7:0/1989525898 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59b8073c70 0x7f59b814e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:45.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.298+0000 7f59bce00640 1 --2- 172.21.15.7:0/1989525898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b815bc90 0x7f59b80724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:45.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.298+0000 7f59afe00640 1 -- 172.21.15.7:0/1989525898 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59b8073c70 msgr2=0x7f59b814e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:45.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.298+0000 7f59afe00640 1 --2- 172.21.15.7:0/1989525898 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59b8073c70 0x7f59b814e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.298+0000 7f59afe00640 1 -- 172.21.15.7:0/1989525898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b815bc90 msgr2=0x7f59b80724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:45.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.298+0000 7f59afe00640 1 --2- 172.21.15.7:0/1989525898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b815bc90 0x7f59b80724f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.298+0000 7f59afe00640 1 -- 172.21.15.7:0/1989525898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59b80c8680 con 0x7f59b80cd750 2024-09-10T21:49:45.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.298+0000 7f59afe00640 1 --2- 172.21.15.7:0/1989525898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b80cd750 0x7f59b80c7ca0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f59a0000c00 tx=0x7f59a00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:45.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.299+0000 7f59b4c00640 1 -- 172.21.15.7:0/1989525898 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a003f070 con 0x7f59b80cd750 2024-09-10T21:49:45.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.299+0000 7f59b4c00640 1 -- 172.21.15.7:0/1989525898 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59a0004890 con 0x7f59b80cd750 2024-09-10T21:49:45.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.299+0000 7f59b4c00640 1 -- 172.21.15.7:0/1989525898 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59a0035030 con 0x7f59b80cd750 2024-09-10T21:49:45.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.299+0000 7f59bee00640 1 -- 172.21.15.7:0/1989525898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59b80c8960 con 0x7f59b80cd750 2024-09-10T21:49:45.301 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.299+0000 7f59bee00640 1 -- 172.21.15.7:0/1989525898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59b814e5d0 con 0x7f59b80cd750 2024-09-10T21:49:45.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.300+0000 7f59bee00640 1 -- 172.21.15.7:0/1989525898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59b8073730 con 0x7f59b80cd750 2024-09-10T21:49:45.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.301+0000 7f59b4c00640 1 -- 172.21.15.7:0/1989525898 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f59a0004ca0 con 0x7f59b80cd750 2024-09-10T21:49:45.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.303+0000 7f59b4c00640 1 --2- 172.21.15.7:0/1989525898 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5994077f70 0x7f599407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:45.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.303+0000 7f59b4c00640 1 -- 172.21.15.7:0/1989525898 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f59a00bb0e0 con 0x7f59b80cd750 2024-09-10T21:49:45.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.303+0000 7f59b7e00640 1 --2- 172.21.15.7:0/1989525898 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5994077f70 0x7f599407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:45.304 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.304+0000 7f59b7e00640 1 --2- 172.21.15.7:0/1989525898 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5994077f70 0x7f599407a430 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f59a8005d70 tx=0x7f59a8005d00 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:45.308 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.307+0000 7f59b4c00640 1 -- 172.21.15.7:0/1989525898 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f59a0083b00 con 0x7f59b80cd750 2024-09-10T21:49:45.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.540+0000 7f59bee00640 1 -- 172.21.15.7:0/1989525898 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59b8160240 con 0x7f59b80cd750 2024-09-10T21:49:45.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.541+0000 7f59b4c00640 1 -- 172.21.15.7:0/1989525898 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) ==== 94+0+5229 (secure 0 0 0) 0x7f59a008c220 con 0x7f59b80cd750 2024-09-10T21:49:45.542 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:45.542 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":76,"btime":"2024-09-10T21:49:07:587035+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/978763848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":978763848},{"type":"v1","addr":"172.21.15.7:6835","nonce":978763848}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/269527918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":269527918},{"type":"v1","addr":"172.21.15.7:6833","nonce":269527918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":235775,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/760980985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":760980985},{"type":"v1","addr":"172.21.15.133:6836","nonce":760980985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:47:40.664091+0000","modified":"2024-09-10T21:49:07.587028+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":89,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236099},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236099":{"gid":236099,"name":"d","rank":0,"incarnation":73,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6837/4188280820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4188280820},{"type":"v1","addr":"172.21.15.133:6837","nonce":4188280820}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":236099,"qdb_cluster":[236099]},"id":6}]} 2024-09-10T21:49:45.542 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 76 2024-09-10T21:49:45.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.543+0000 7f59bee00640 1 -- 172.21.15.7:0/1989525898 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5994077f70 msgr2=0x7f599407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.543+0000 7f59bee00640 1 --2- 172.21.15.7:0/1989525898 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5994077f70 0x7f599407a430 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f59a8005d70 tx=0x7f59a8005d00 comp rx=0 tx=0).stop 2024-09-10T21:49:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.543+0000 7f59bee00640 1 -- 172.21.15.7:0/1989525898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b80cd750 msgr2=0x7f59b80c7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.543+0000 7f59bee00640 1 --2- 172.21.15.7:0/1989525898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b80cd750 0x7f59b80c7ca0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f59a0000c00 tx=0x7f59a00039e0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.543+0000 7f59bee00640 1 -- 172.21.15.7:0/1989525898 shutdown_connections 2024-09-10T21:49:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.543+0000 7f59bee00640 1 --2- 172.21.15.7:0/1989525898 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5994077f70 0x7f599407a430 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.543+0000 7f59bee00640 1 --2- 172.21.15.7:0/1989525898 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f59b815bc90 0x7f59b80724f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.543+0000 7f59bee00640 1 --2- 172.21.15.7:0/1989525898 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f59b80cd750 0x7f59b80c7ca0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.543+0000 7f59bee00640 1 --2- 172.21.15.7:0/1989525898 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f59b8073c70 0x7f59b814e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.543+0000 7f59bee00640 1 -- 172.21.15.7:0/1989525898 >> 172.21.15.7:0/1989525898 conn(0x7f59b80b16d0 msgr2=0x7f59b8070340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.544+0000 7f59bee00640 1 -- 172.21.15.7:0/1989525898 shutdown_connections 2024-09-10T21:49:45.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.544+0000 7f59bee00640 1 -- 172.21.15.7:0/1989525898 wait complete. 2024-09-10T21:49:45.560 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:49:45.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.718+0000 7f7b72e00640 1 -- 172.21.15.7:0/2247808445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b6c15fda0 msgr2=0x7f7b6c160180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:45.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.718+0000 7f7b72e00640 1 --2- 172.21.15.7:0/2247808445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b6c15fda0 0x7f7b6c160180 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0cf860 tx=0x7f7b6402aa30 comp rx=0 tx=0).stop 2024-09-10T21:49:45.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.718+0000 7f7b72e00640 1 -- 172.21.15.7:0/2247808445 shutdown_connections 2024-09-10T21:49:45.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.718+0000 7f7b72e00640 1 --2- 172.21.15.7:0/2247808445 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b6c15fda0 0x7f7b6c160180 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.718+0000 7f7b72e00640 1 --2- 172.21.15.7:0/2247808445 >> v2:172.21.15.133:3300/0 conn(0x7f7b6c0cd750 0x7f7b6c0ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.718+0000 7f7b72e00640 1 -- 172.21.15.7:0/2247808445 >> 172.21.15.7:0/2247808445 conn(0x7f7b6c0b16d0 msgr2=0x7f7b6c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:45.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.718+0000 7f7b72e00640 1 -- 172.21.15.7:0/2247808445 shutdown_connections 2024-09-10T21:49:45.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.718+0000 7f7b72e00640 1 -- 172.21.15.7:0/2247808445 wait complete. 2024-09-10T21:49:45.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.720+0000 7f7b72e00640 1 Processor -- start 2024-09-10T21:49:45.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.720+0000 7f7b72e00640 1 -- start start 2024-09-10T21:49:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.721+0000 7f7b72e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b6c0cd750 0x7f7b6c0c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.721+0000 7f7b72e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b6c15bc90 0x7f7b6c14f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.721+0000 7f7b72e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b6c15fda0 0x7f7b6c1f7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.721+0000 7f7b72e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c06fe70 con 0x7f7b6c0cd750 2024-09-10T21:49:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.721+0000 7f7b72e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6c06fcf0 con 0x7f7b6c15bc90 2024-09-10T21:49:45.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.721+0000 7f7b72e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f7b6c06fff0 con 0x7f7b6c15fda0 2024-09-10T21:49:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.721+0000 7f7b6be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b6c0cd750 0x7f7b6c0c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.721+0000 7f7b6be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b6c0cd750 0x7f7b6c0c49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37238/0 (socket says 172.21.15.7:37238) 2024-09-10T21:49:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.721+0000 7f7b6be00640 1 -- 172.21.15.7:0/2158828980 learned_addr learned my addr 172.21.15.7:0/2158828980 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.721+0000 7f7b63400640 1 --2- 172.21.15.7:0/2158828980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b6c15bc90 0x7f7b6c14f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.721+0000 7f7b70e00640 1 --2- 172.21.15.7:0/2158828980 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b6c15fda0 0x7f7b6c1f7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.722+0000 7f7b6be00640 1 -- 172.21.15.7:0/2158828980 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b6c15fda0 msgr2=0x7f7b6c1f7820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.722+0000 7f7b6be00640 1 --2- 172.21.15.7:0/2158828980 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b6c15fda0 0x7f7b6c1f7820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.722+0000 7f7b6be00640 1 -- 172.21.15.7:0/2158828980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b6c15bc90 msgr2=0x7f7b6c14f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.722+0000 7f7b6be00640 1 --2- 172.21.15.7:0/2158828980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b6c15bc90 0x7f7b6c14f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.722+0000 7f7b6be00640 1 -- 172.21.15.7:0/2158828980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6c0c36b0 con 0x7f7b6c0cd750 2024-09-10T21:49:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.722+0000 7f7b63400640 1 --2- 172.21.15.7:0/2158828980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b6c15bc90 0x7f7b6c14f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.722+0000 7f7b6be00640 1 --2- 172.21.15.7:0/2158828980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b6c0cd750 0x7f7b6c0c49f0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00c450 tx=0x7f7b5c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.722+0000 7f7b68c00640 1 -- 172.21.15.7:0/2158828980 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c0052d0 con 0x7f7b6c0cd750 2024-09-10T21:49:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.722+0000 7f7b68c00640 1 -- 172.21.15.7:0/2158828980 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b5c005470 con 0x7f7b6c0cd750 2024-09-10T21:49:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.722+0000 7f7b68c00640 1 -- 172.21.15.7:0/2158828980 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c012870 con 0x7f7b6c0cd750 2024-09-10T21:49:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.722+0000 7f7b72e00640 1 -- 172.21.15.7:0/2158828980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b6c0c3950 con 0x7f7b6c0cd750 2024-09-10T21:49:45.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.722+0000 7f7b72e00640 1 -- 172.21.15.7:0/2158828980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b6c0c3f10 con 0x7f7b6c0cd750 2024-09-10T21:49:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.723+0000 7f7b72e00640 1 -- 172.21.15.7:0/2158828980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b6c070510 con 0x7f7b6c0cd750 2024-09-10T21:49:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.724+0000 7f7b68c00640 1 -- 172.21.15.7:0/2158828980 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7b5c0035f0 con 0x7f7b6c0cd750 2024-09-10T21:49:45.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.726+0000 7f7b68c00640 1 --2- 172.21.15.7:0/2158828980 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b48077f70 0x7f7b4807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:45.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.726+0000 7f7b68c00640 1 -- 172.21.15.7:0/2158828980 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7f7b5c099a20 con 0x7f7b6c0cd750 2024-09-10T21:49:45.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.726+0000 7f7b63400640 1 --2- 172.21.15.7:0/2158828980 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b48077f70 0x7f7b4807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:45.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.727+0000 7f7b63400640 1 --2- 172.21.15.7:0/2158828980 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b48077f70 0x7f7b4807a430 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7b540058c0 tx=0x7f7b54005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:45.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.730+0000 7f7b68c00640 1 -- 172.21.15.7:0/2158828980 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7b5c0623b0 con 0x7f7b6c0cd750 2024-09-10T21:49:45.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.927+0000 7f7b72e00640 1 -- 172.21.15.7:0/2158828980 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7b6c069810 con 0x7f7b6c0cd750 2024-09-10T21:49:45.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.928+0000 7f7b68c00640 1 -- 172.21.15.7:0/2158828980 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+16993 (secure 0 0 0) 0x7f7b5c061cb0 con 0x7f7b6c0cd750 2024-09-10T21:49:45.929 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:45.929 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":89,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:49:03.563205+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:47:38.456723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-10T21:47:39.629851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.35","mappings":[{"from":1,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6837/3312604545":"2024-09-11T21:49:03.563183+0000","172.21.15.133:6835/3312604545":"2024-09-11T21:49:03.563183+0000","172.21.15.7:6833/1499133391":"2024-09-11T21:47:51.143906+0000","172.21.15.7:6832/1499133391":"2024-09-11T21:47:51.143906+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:49:45.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.931+0000 7f7b72e00640 1 -- 172.21.15.7:0/2158828980 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b48077f70 msgr2=0x7f7b4807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:45.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.931+0000 7f7b72e00640 1 --2- 172.21.15.7:0/2158828980 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b48077f70 0x7f7b4807a430 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7b540058c0 tx=0x7f7b54005850 comp rx=0 tx=0).stop 2024-09-10T21:49:45.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.931+0000 7f7b72e00640 1 -- 172.21.15.7:0/2158828980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b6c0cd750 msgr2=0x7f7b6c0c49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:45.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.931+0000 7f7b72e00640 1 --2- 172.21.15.7:0/2158828980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b6c0cd750 0x7f7b6c0c49f0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c00c450 tx=0x7f7b5c00c910 comp rx=0 tx=0).stop 2024-09-10T21:49:45.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.931+0000 7f7b72e00640 1 -- 172.21.15.7:0/2158828980 shutdown_connections 2024-09-10T21:49:45.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.931+0000 7f7b72e00640 1 --2- 172.21.15.7:0/2158828980 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7b48077f70 0x7f7b4807a430 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.931+0000 7f7b72e00640 1 --2- 172.21.15.7:0/2158828980 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b6c15bc90 0x7f7b6c14f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.931+0000 7f7b72e00640 1 --2- 172.21.15.7:0/2158828980 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b6c15fda0 0x7f7b6c1f7820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.931+0000 7f7b72e00640 1 --2- 172.21.15.7:0/2158828980 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b6c0cd750 0x7f7b6c0c49f0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:45.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.931+0000 7f7b72e00640 1 -- 172.21.15.7:0/2158828980 >> 172.21.15.7:0/2158828980 conn(0x7f7b6c0b16d0 msgr2=0x7f7b6c0b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:45.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.931+0000 7f7b72e00640 1 -- 172.21.15.7:0/2158828980 shutdown_connections 2024-09-10T21:49:45.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:45.931+0000 7f7b72e00640 1 -- 172.21.15.7:0/2158828980 wait complete. 2024-09-10T21:49:45.947 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:49:46.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.108+0000 7fe2cda00640 1 -- 172.21.15.7:0/2924494515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2c806fd30 msgr2=0x7fe2c815f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:46.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.108+0000 7fe2cda00640 1 --2- 172.21.15.7:0/2924494515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2c806fd30 0x7fe2c815f060 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0005fd0 tx=0x7fe2c002ab30 comp rx=0 tx=0).stop 2024-09-10T21:49:46.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.108+0000 7fe2cda00640 1 -- 172.21.15.7:0/2924494515 shutdown_connections 2024-09-10T21:49:46.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.108+0000 7fe2cda00640 1 --2- 172.21.15.7:0/2924494515 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2c806fd30 0x7fe2c815f060 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:46.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.108+0000 7fe2cda00640 1 -- 172.21.15.7:0/2924494515 >> 172.21.15.7:0/2924494515 conn(0x7fe2c80b1660 msgr2=0x7fe2c80b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:46.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.108+0000 7fe2cda00640 1 -- 172.21.15.7:0/2924494515 shutdown_connections 2024-09-10T21:49:46.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.108+0000 7fe2cda00640 1 -- 172.21.15.7:0/2924494515 wait complete. 2024-09-10T21:49:46.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.110+0000 7fe2cda00640 1 Processor -- start 2024-09-10T21:49:46.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.110+0000 7fe2cda00640 1 -- start start 2024-09-10T21:49:46.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.111+0000 7fe2cda00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2c806fd30 0x7fe2c8159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:46.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.111+0000 7fe2cda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2c815a9a0 0x7fe2c80ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:46.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.111+0000 7fe2cda00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2c80b4e90 0x7fe2c80ade30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:46.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.111+0000 7fe2cda00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c80739a0 con 0x7fe2c815a9a0 2024-09-10T21:49:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.111+0000 7fe2cda00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe2c8073820 con 0x7fe2c806fd30 2024-09-10T21:49:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.111+0000 7fe2cda00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe2c8073b20 con 0x7fe2c80b4e90 2024-09-10T21:49:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.111+0000 7fe2be000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2c815a9a0 0x7fe2c80ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.111+0000 7fe2be000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2c815a9a0 0x7fe2c80ad8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37264/0 (socket says 172.21.15.7:37264) 2024-09-10T21:49:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.111+0000 7fe2be000640 1 -- 172.21.15.7:0/2045529490 learned_addr learned my addr 172.21.15.7:0/2045529490 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.111+0000 7fe2c7400640 1 --2- 172.21.15.7:0/2045529490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2c80b4e90 0x7fe2c80ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.111+0000 7fe2be000640 1 -- 172.21.15.7:0/2045529490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2c80b4e90 msgr2=0x7fe2c80ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.111+0000 7fe2be000640 1 --2- 172.21.15.7:0/2045529490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2c80b4e90 0x7fe2c80ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.111+0000 7fe2c6a00640 1 --2- 172.21.15.7:0/2045529490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2c806fd30 0x7fe2c8159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.111+0000 7fe2be000640 1 -- 172.21.15.7:0/2045529490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2c806fd30 msgr2=0x7fe2c8159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.111+0000 7fe2be000640 1 --2- 172.21.15.7:0/2045529490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2c806fd30 0x7fe2c8159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.111+0000 7fe2be000640 1 -- 172.21.15.7:0/2045529490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2c80c9d50 con 0x7fe2c815a9a0 2024-09-10T21:49:46.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.111+0000 7fe2c7400640 1 --2- 172.21.15.7:0/2045529490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2c80b4e90 0x7fe2c80ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:49:46.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.111+0000 7fe2c6a00640 1 --2- 172.21.15.7:0/2045529490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2c806fd30 0x7fe2c8159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:49:46.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.112+0000 7fe2be000640 1 --2- 172.21.15.7:0/2045529490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2c815a9a0 0x7fe2c80ad8f0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000c500 tx=0x7fe2b000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:46.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.112+0000 7fe2bf400640 1 -- 172.21.15.7:0/2045529490 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00052d0 con 0x7fe2c815a9a0 2024-09-10T21:49:46.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.112+0000 7fe2bf400640 1 -- 172.21.15.7:0/2045529490 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2b0005470 con 0x7fe2c815a9a0 2024-09-10T21:49:46.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.112+0000 7fe2cda00640 1 -- 172.21.15.7:0/2045529490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2c81f5450 con 0x7fe2c815a9a0 2024-09-10T21:49:46.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.112+0000 7fe2bf400640 1 -- 172.21.15.7:0/2045529490 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0012870 con 0x7fe2c815a9a0 2024-09-10T21:49:46.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.112+0000 7fe2cda00640 1 -- 172.21.15.7:0/2045529490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2c81f5a10 con 0x7fe2c815a9a0 2024-09-10T21:49:46.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.113+0000 7fe2cda00640 1 -- 172.21.15.7:0/2045529490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2c815f060 con 0x7fe2c815a9a0 2024-09-10T21:49:46.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.114+0000 7fe2bf400640 1 -- 172.21.15.7:0/2045529490 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe2b00035f0 con 0x7fe2c815a9a0 2024-09-10T21:49:46.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.115+0000 7fe2bf400640 1 --2- 172.21.15.7:0/2045529490 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe2a4077f70 0x7fe2a407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:46.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.115+0000 7fe2bf400640 1 -- 172.21.15.7:0/2045529490 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 5910+0+0 (secure 0 0 0) 0x7fe2b00999f0 con 0x7fe2c815a9a0 2024-09-10T21:49:46.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.116+0000 7fe2c6a00640 1 --2- 172.21.15.7:0/2045529490 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe2a4077f70 0x7fe2a407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:46.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.116+0000 7fe2c6a00640 1 --2- 172.21.15.7:0/2045529490 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe2a4077f70 0x7fe2a407a430 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8001740 tx=0x7fe2b8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:46.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.119+0000 7fe2bf400640 1 -- 172.21.15.7:0/2045529490 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe2b0062390 con 0x7fe2c815a9a0 2024-09-10T21:49:46.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.350+0000 7fe2cda00640 1 -- 172.21.15.7:0/2045529490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe2c80b4c70 con 0x7fe2c815a9a0 2024-09-10T21:49:46.352 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-10T21:49:46.351+0000 7feb3fa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:49:46.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.353+0000 7fe2bf400640 1 -- 172.21.15.7:0/2045529490 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v77) ==== 161+0+0 (secure 0 0 0) 0x7fe2b0061c90 con 0x7fe2c815a9a0 2024-09-10T21:49:46.354 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:49:46.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.356+0000 7fe2cda00640 1 -- 172.21.15.7:0/2045529490 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe2a4077f70 msgr2=0x7fe2a407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:46.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.356+0000 7fe2cda00640 1 --2- 172.21.15.7:0/2045529490 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe2a4077f70 0x7fe2a407a430 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8001740 tx=0x7fe2b8009040 comp rx=0 tx=0).stop 2024-09-10T21:49:46.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.356+0000 7fe2cda00640 1 -- 172.21.15.7:0/2045529490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2c815a9a0 msgr2=0x7fe2c80ad8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:46.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.356+0000 7fe2cda00640 1 --2- 172.21.15.7:0/2045529490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2c815a9a0 0x7fe2c80ad8f0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fe2b000c500 tx=0x7fe2b000c9c0 comp rx=0 tx=0).stop 2024-09-10T21:49:46.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.356+0000 7fe2cda00640 1 -- 172.21.15.7:0/2045529490 shutdown_connections 2024-09-10T21:49:46.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.356+0000 7fe2cda00640 1 --2- 172.21.15.7:0/2045529490 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe2a4077f70 0x7fe2a407a430 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.356+0000 7fe2cda00640 1 --2- 172.21.15.7:0/2045529490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2c80b4e90 0x7fe2c80ade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.356+0000 7fe2cda00640 1 --2- 172.21.15.7:0/2045529490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2c815a9a0 0x7fe2c80ad8f0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.356+0000 7fe2cda00640 1 --2- 172.21.15.7:0/2045529490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2c806fd30 0x7fe2c8159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.356+0000 7fe2cda00640 1 -- 172.21.15.7:0/2045529490 >> 172.21.15.7:0/2045529490 conn(0x7fe2c80b1660 msgr2=0x7fe2c80ca970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.356+0000 7fe2cda00640 1 -- 172.21.15.7:0/2045529490 shutdown_connections 2024-09-10T21:49:46.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.357+0000 7fe2cda00640 1 -- 172.21.15.7:0/2045529490 wait complete. 2024-09-10T21:49:46.365 INFO:tasks.ceph.mds.d.smithi133.stderr: -2377> 2024-09-10T21:49:10.581+0000 7f7358000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:49:46.366 INFO:tasks.ceph.mds.d.smithi133.stderr: -1896> 2024-09-10T21:49:10.964+0000 7f7353000640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [...2,head] /dir/file auth v1 ap=1 DIRTYPARENT s=0 nl=2 n() | request=0 lock=0 dirtyparent=1 authpin=1 scrubqueue=0 0x561bc4424680]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(13)0x10000000001:[<0x10000000000/file v8>,<0x1/dir v15>]//[]","memoryvalue":"(13)0x10000000001:[<0x10000000000/file v1>,<0x1/dir v22>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-09-10T21:49:46.373 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:49:46.414 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2024-09-10T21:49:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.537+0000 7fd71be00640 1 -- 172.21.15.7:0/3376348014 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd71406fcf0 msgr2=0x7fd7141600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.537+0000 7fd71be00640 1 --2- 172.21.15.7:0/3376348014 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd71406fcf0 0x7fd7141600d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd700005fd0 tx=0x7fd70002a5d0 comp rx=0 tx=0).stop 2024-09-10T21:49:46.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.537+0000 7fd71be00640 1 -- 172.21.15.7:0/3376348014 shutdown_connections 2024-09-10T21:49:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.537+0000 7fd71be00640 1 --2- 172.21.15.7:0/3376348014 >> v2:172.21.15.133:3300/0 conn(0x7fd714074450 0x7fd7140ca270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.537+0000 7fd71be00640 1 --2- 172.21.15.7:0/3376348014 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd71406fcf0 0x7fd7141600d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.537+0000 7fd71be00640 1 -- 172.21.15.7:0/3376348014 >> 172.21.15.7:0/3376348014 conn(0x7fd7140b16d0 msgr2=0x7fd7140b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.537+0000 7fd71be00640 1 -- 172.21.15.7:0/3376348014 shutdown_connections 2024-09-10T21:49:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.537+0000 7fd71be00640 1 -- 172.21.15.7:0/3376348014 wait complete. 2024-09-10T21:49:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.539+0000 7fd71be00640 1 Processor -- start 2024-09-10T21:49:46.539 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.539+0000 7fd71be00640 1 -- start start 2024-09-10T21:49:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.540+0000 7fd71be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd71406fcf0 0x7fd7140c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.540+0000 7fd71be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd714074450 0x7fd7140c9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.540+0000 7fd71be00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd714155fb0 0x7fd7140b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.540+0000 7fd71be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd7140ca930 con 0x7fd714074450 2024-09-10T21:49:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.540+0000 7fd71be00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd7140ca7b0 con 0x7fd71406fcf0 2024-09-10T21:49:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.540+0000 7fd71be00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fd7140caab0 con 0x7fd714155fb0 2024-09-10T21:49:46.541 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.540+0000 7fd718a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd714074450 0x7fd7140c9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.540+0000 7fd719400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd71406fcf0 0x7fd7140c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.540+0000 7fd718a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd714074450 0x7fd7140c9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37280/0 (socket says 172.21.15.7:37280) 2024-09-10T21:49:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.540+0000 7fd718a00640 1 -- 172.21.15.7:0/3647943682 learned_addr learned my addr 172.21.15.7:0/3647943682 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:46.542 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.540+0000 7fd719e00640 1 --2- 172.21.15.7:0/3647943682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd714155fb0 0x7fd7140b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.541+0000 7fd718a00640 1 -- 172.21.15.7:0/3647943682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd714155fb0 msgr2=0x7fd7140b4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.541+0000 7fd718a00640 1 --2- 172.21.15.7:0/3647943682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd714155fb0 0x7fd7140b4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.541+0000 7fd718a00640 1 -- 172.21.15.7:0/3647943682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd71406fcf0 msgr2=0x7fd7140c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.541+0000 7fd718a00640 1 --2- 172.21.15.7:0/3647943682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd71406fcf0 0x7fd7140c5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.541+0000 7fd718a00640 1 -- 172.21.15.7:0/3647943682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7140c5fe0 con 0x7fd714074450 2024-09-10T21:49:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.541+0000 7fd719400640 1 --2- 172.21.15.7:0/3647943682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd71406fcf0 0x7fd7140c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:49:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.541+0000 7fd719e00640 1 --2- 172.21.15.7:0/3647943682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd714155fb0 0x7fd7140b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:49:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.541+0000 7fd718a00640 1 --2- 172.21.15.7:0/3647943682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd714074450 0x7fd7140c9760 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd700000c00 tx=0x7fd700005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:46.543 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.541+0000 7fd711600640 1 -- 172.21.15.7:0/3647943682 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70003f070 con 0x7fd714074450 2024-09-10T21:49:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.541+0000 7fd711600640 1 -- 172.21.15.7:0/3647943682 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd700003c90 con 0x7fd714074450 2024-09-10T21:49:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.541+0000 7fd711600640 1 -- 172.21.15.7:0/3647943682 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd700035030 con 0x7fd714074450 2024-09-10T21:49:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.541+0000 7fd71be00640 1 -- 172.21.15.7:0/3647943682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7140c62c0 con 0x7fd714074450 2024-09-10T21:49:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.541+0000 7fd71be00640 1 -- 172.21.15.7:0/3647943682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7140c65a0 con 0x7fd714074450 2024-09-10T21:49:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.542+0000 7fd71be00640 1 -- 172.21.15.7:0/3647943682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7141f4870 con 0x7fd714074450 2024-09-10T21:49:46.544 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.544+0000 7fd711600640 1 -- 172.21.15.7:0/3647943682 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd700004a00 con 0x7fd714074450 2024-09-10T21:49:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.545+0000 7fd711600640 1 --2- 172.21.15.7:0/3647943682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd6f4077f70 0x7fd6f407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:46.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.545+0000 7fd711600640 1 -- 172.21.15.7:0/3647943682 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5996+0+0 (secure 0 0 0) 0x7fd7000baac0 con 0x7fd714074450 2024-09-10T21:49:46.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.545+0000 7fd719400640 1 --2- 172.21.15.7:0/3647943682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd6f4077f70 0x7fd6f407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:46.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.546+0000 7fd719400640 1 --2- 172.21.15.7:0/3647943682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd6f4077f70 0x7fd6f407a430 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd708007fd0 tx=0x7fd708009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:46.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.549+0000 7fd711600640 1 -- 172.21.15.7:0/3647943682 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd700082e40 con 0x7fd714074450 2024-09-10T21:49:46.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:46.781+0000 7fd71be00640 1 -- 172.21.15.7:0/3647943682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd7140cafd0 con 0x7fd714074450 2024-09-10T21:49:47.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.355+0000 7fd711600640 1 -- 172.21.15.7:0/3647943682 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v78) ==== 104+0+0 (secure 0 0 0) 0x7fd700087870 con 0x7fd714074450 2024-09-10T21:49:47.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.358+0000 7fd71be00640 1 -- 172.21.15.7:0/3647943682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd6f4077f70 msgr2=0x7fd6f407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:47.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.358+0000 7fd71be00640 1 --2- 172.21.15.7:0/3647943682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd6f4077f70 0x7fd6f407a430 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd708007fd0 tx=0x7fd708009040 comp rx=0 tx=0).stop 2024-09-10T21:49:47.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.358+0000 7fd71be00640 1 -- 172.21.15.7:0/3647943682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd714074450 msgr2=0x7fd7140c9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:47.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.358+0000 7fd71be00640 1 --2- 172.21.15.7:0/3647943682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd714074450 0x7fd7140c9760 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7fd700000c00 tx=0x7fd700005770 comp rx=0 tx=0).stop 2024-09-10T21:49:47.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.358+0000 7fd71be00640 1 -- 172.21.15.7:0/3647943682 shutdown_connections 2024-09-10T21:49:47.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.358+0000 7fd71be00640 1 --2- 172.21.15.7:0/3647943682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd6f4077f70 0x7fd6f407a430 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:47.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.358+0000 7fd71be00640 1 --2- 172.21.15.7:0/3647943682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd714155fb0 0x7fd7140b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:47.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.358+0000 7fd71be00640 1 --2- 172.21.15.7:0/3647943682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd714074450 0x7fd7140c9760 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:47.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.358+0000 7fd71be00640 1 --2- 172.21.15.7:0/3647943682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd71406fcf0 0x7fd7140c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:47.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.358+0000 7fd71be00640 1 -- 172.21.15.7:0/3647943682 >> 172.21.15.7:0/3647943682 conn(0x7fd7140b16d0 msgr2=0x7fd71415b650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:47.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.359+0000 7fd71be00640 1 -- 172.21.15.7:0/3647943682 shutdown_connections 2024-09-10T21:49:47.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.359+0000 7fd71be00640 1 -- 172.21.15.7:0/3647943682 wait complete. 2024-09-10T21:49:47.377 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:49:47.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.543+0000 7ff8e2a00640 1 -- 172.21.15.7:0/1954647671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c0004d40 msgr2=0x7ff8c00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:47.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.543+0000 7ff8e2a00640 1 --2- 172.21.15.7:0/1954647671 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8c0004d40 0x7ff8c00051c0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7ff8d4009ed0 tx=0x7ff8c0005830 comp rx=0 tx=0).stop 2024-09-10T21:49:47.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.543+0000 7ff8e0a00640 1 -- 172.21.15.7:0/1954647671 reap_dead start 2024-09-10T21:49:47.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.544+0000 7ff8e2a00640 1 -- 172.21.15.7:0/1954647671 shutdown_connections 2024-09-10T21:49:47.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.544+0000 7ff8e2a00640 1 -- 172.21.15.7:0/1954647671 >> 172.21.15.7:0/1954647671 conn(0x7ff8dc0b1c10 msgr2=0x7ff8dc0b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:47.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.544+0000 7ff8e2a00640 1 -- 172.21.15.7:0/1954647671 shutdown_connections 2024-09-10T21:49:47.545 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.544+0000 7ff8e2a00640 1 -- 172.21.15.7:0/1954647671 wait complete. 2024-09-10T21:49:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.545+0000 7ff8e2a00640 1 Processor -- start 2024-09-10T21:49:47.546 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.545+0000 7ff8e2a00640 1 -- start start 2024-09-10T21:49:47.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.546+0000 7ff8e2a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c0004d40 0x7ff8dc0cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:47.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.546+0000 7ff8e2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8dc14a730 0x7ff8dc150910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:47.549 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.546+0000 7ff8e2a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff8dc14ab90 0x7ff8dc0ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:47.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.546+0000 7ff8e2a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc15e8d0 con 0x7ff8dc14a730 2024-09-10T21:49:47.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.546+0000 7ff8e2a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff8dc15e750 con 0x7ff8c0004d40 2024-09-10T21:49:47.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.546+0000 7ff8e2a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ff8dc073550 con 0x7ff8dc14ab90 2024-09-10T21:49:47.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.547+0000 7ff8db400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8dc14a730 0x7ff8dc150910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:47.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.547+0000 7ff8db400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8dc14a730 0x7ff8dc150910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37292/0 (socket says 172.21.15.7:37292) 2024-09-10T21:49:47.550 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.547+0000 7ff8db400640 1 -- 172.21.15.7:0/3581300682 learned_addr learned my addr 172.21.15.7:0/3581300682 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:47.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.547+0000 7ff8dbe00640 1 --2- 172.21.15.7:0/3581300682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c0004d40 0x7ff8dc0cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:47.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.547+0000 7ff8e0a00640 1 --2- 172.21.15.7:0/3581300682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff8dc14ab90 0x7ff8dc0ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:47.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.547+0000 7ff8db400640 1 -- 172.21.15.7:0/3581300682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff8dc14ab90 msgr2=0x7ff8dc0ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:47.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.547+0000 7ff8db400640 1 --2- 172.21.15.7:0/3581300682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff8dc14ab90 0x7ff8dc0ca100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:47.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.547+0000 7ff8db400640 1 -- 172.21.15.7:0/3581300682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c0004d40 msgr2=0x7ff8dc0cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:47.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.547+0000 7ff8db400640 1 --2- 172.21.15.7:0/3581300682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c0004d40 0x7ff8dc0cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:47.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.547+0000 7ff8db400640 1 -- 172.21.15.7:0/3581300682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8dc15c1c0 con 0x7ff8dc14a730 2024-09-10T21:49:47.551 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.547+0000 7ff8dbe00640 1 --2- 172.21.15.7:0/3581300682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c0004d40 0x7ff8dc0cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:47.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.547+0000 7ff8db400640 1 --2- 172.21.15.7:0/3581300682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8dc14a730 0x7ff8dc150910 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800e630 tx=0x7ff8c800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:47.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.547+0000 7ff8d3e00640 1 -- 172.21.15.7:0/3581300682 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8015070 con 0x7ff8dc14a730 2024-09-10T21:49:47.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.547+0000 7ff8d3e00640 1 -- 172.21.15.7:0/3581300682 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8c800caa0 con 0x7ff8dc14a730 2024-09-10T21:49:47.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.547+0000 7ff8d3e00640 1 -- 172.21.15.7:0/3581300682 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8c8011460 con 0x7ff8dc14a730 2024-09-10T21:49:47.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.547+0000 7ff8e2a00640 1 -- 172.21.15.7:0/3581300682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8dc15c460 con 0x7ff8dc14a730 2024-09-10T21:49:47.552 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.548+0000 7ff8e2a00640 1 -- 172.21.15.7:0/3581300682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8dc15ca20 con 0x7ff8dc14a730 2024-09-10T21:49:47.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.549+0000 7ff8e2a00640 1 -- 172.21.15.7:0/3581300682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8dc073ab0 con 0x7ff8dc14a730 2024-09-10T21:49:47.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.550+0000 7ff8d3e00640 1 -- 172.21.15.7:0/3581300682 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff8c8011600 con 0x7ff8dc14a730 2024-09-10T21:49:47.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.551+0000 7ff8d3e00640 1 --2- 172.21.15.7:0/3581300682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff8bc077f70 0x7ff8bc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:47.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.551+0000 7ff8d3e00640 1 -- 172.21.15.7:0/3581300682 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 5996+0+0 (secure 0 0 0) 0x7ff8c809a5c0 con 0x7ff8dc14a730 2024-09-10T21:49:47.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.552+0000 7ff8dbe00640 1 --2- 172.21.15.7:0/3581300682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff8bc077f70 0x7ff8bc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:47.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.552+0000 7ff8dbe00640 1 --2- 172.21.15.7:0/3581300682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff8bc077f70 0x7ff8bc07a430 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc000c00 tx=0x7ff8cc006b70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:47.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.555+0000 7ff8d3e00640 1 -- 172.21.15.7:0/3581300682 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff8c8062940 con 0x7ff8dc14a730 2024-09-10T21:49:47.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:47.751+0000 7ff8e2a00640 1 -- 172.21.15.7:0/3581300682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff8dc069a80 con 0x7ff8dc14a730 2024-09-10T21:49:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.357+0000 7ff8d3e00640 1 -- 172.21.15.7:0/3581300682 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) ==== 181+0+0 (secure 0 0 0) 0x7ff8c8067370 con 0x7ff8dc14a730 2024-09-10T21:49:48.358 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:49:48.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.361+0000 7ff8e2a00640 1 -- 172.21.15.7:0/3581300682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff8bc077f70 msgr2=0x7ff8bc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:48.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.361+0000 7ff8e2a00640 1 --2- 172.21.15.7:0/3581300682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff8bc077f70 0x7ff8bc07a430 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc000c00 tx=0x7ff8cc006b70 comp rx=0 tx=0).stop 2024-09-10T21:49:48.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.361+0000 7ff8e2a00640 1 -- 172.21.15.7:0/3581300682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8dc14a730 msgr2=0x7ff8dc150910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:48.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.361+0000 7ff8e2a00640 1 --2- 172.21.15.7:0/3581300682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8dc14a730 0x7ff8dc150910 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ff8c800e630 tx=0x7ff8c800eaf0 comp rx=0 tx=0).stop 2024-09-10T21:49:48.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.361+0000 7ff8e2a00640 1 -- 172.21.15.7:0/3581300682 shutdown_connections 2024-09-10T21:49:48.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.361+0000 7ff8e2a00640 1 --2- 172.21.15.7:0/3581300682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff8bc077f70 0x7ff8bc07a430 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:48.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.361+0000 7ff8e2a00640 1 --2- 172.21.15.7:0/3581300682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff8dc14ab90 0x7ff8dc0ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:48.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.361+0000 7ff8e2a00640 1 --2- 172.21.15.7:0/3581300682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff8dc14a730 0x7ff8dc150910 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:48.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.361+0000 7ff8e2a00640 1 --2- 172.21.15.7:0/3581300682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff8c0004d40 0x7ff8dc0cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:48.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.361+0000 7ff8e2a00640 1 -- 172.21.15.7:0/3581300682 >> 172.21.15.7:0/3581300682 conn(0x7ff8dc0b1c10 msgr2=0x7ff8dc159e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:48.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.362+0000 7ff8e2a00640 1 -- 172.21.15.7:0/3581300682 shutdown_connections 2024-09-10T21:49:48.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.362+0000 7ff8e2a00640 1 -- 172.21.15.7:0/3581300682 wait complete. 2024-09-10T21:49:48.382 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:49:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.552+0000 7f01e3200640 1 -- 172.21.15.7:0/1108416859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01dc06d230 msgr2=0x7f01dc0b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.552+0000 7f01e3200640 1 --2- 172.21.15.7:0/1108416859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01dc06d230 0x7f01dc0b4a30 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009830 tx=0x7f01c402a480 comp rx=0 tx=0).stop 2024-09-10T21:49:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.553+0000 7f01e3200640 1 -- 172.21.15.7:0/1108416859 shutdown_connections 2024-09-10T21:49:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.553+0000 7f01e3200640 1 --2- 172.21.15.7:0/1108416859 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01dc06d230 0x7f01dc0b4a30 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.553+0000 7f01e3200640 1 -- 172.21.15.7:0/1108416859 >> 172.21.15.7:0/1108416859 conn(0x7f01dc0b1c10 msgr2=0x7f01dc0b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.553+0000 7f01e3200640 1 -- 172.21.15.7:0/1108416859 shutdown_connections 2024-09-10T21:49:48.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.553+0000 7f01e3200640 1 -- 172.21.15.7:0/1108416859 wait complete. 2024-09-10T21:49:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.554+0000 7f01e3200640 1 Processor -- start 2024-09-10T21:49:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.554+0000 7f01e3200640 1 -- start start 2024-09-10T21:49:48.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.555+0000 7f01e3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01dc06d230 0x7f01dc1471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.555+0000 7f01e3200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f01dc158300 0x7f01dc09da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.555+0000 7f01e3200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01dc15edd0 0x7f01dc09aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.555+0000 7f01e3200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc0740a0 con 0x7f01dc06d230 2024-09-10T21:49:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.555+0000 7f01e3200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f01dc073f20 con 0x7f01dc15edd0 2024-09-10T21:49:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.555+0000 7f01e3200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f01dc074220 con 0x7f01dc158300 2024-09-10T21:49:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.556+0000 7f01dbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01dc06d230 0x7f01dc1471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.556+0000 7f01dbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01dc06d230 0x7f01dc1471f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37316/0 (socket says 172.21.15.7:37316) 2024-09-10T21:49:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.556+0000 7f01dbe00640 1 -- 172.21.15.7:0/2872767613 learned_addr learned my addr 172.21.15.7:0/2872767613 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.556+0000 7f01d3400640 1 --2- 172.21.15.7:0/2872767613 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f01dc158300 0x7f01dc09da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.556+0000 7f01e1200640 1 --2- 172.21.15.7:0/2872767613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01dc15edd0 0x7f01dc09aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.556+0000 7f01dbe00640 1 -- 172.21.15.7:0/2872767613 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f01dc158300 msgr2=0x7f01dc09da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.556+0000 7f01dbe00640 1 --2- 172.21.15.7:0/2872767613 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f01dc158300 0x7f01dc09da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.556+0000 7f01dbe00640 1 -- 172.21.15.7:0/2872767613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01dc15edd0 msgr2=0x7f01dc09aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.556+0000 7f01dbe00640 1 --2- 172.21.15.7:0/2872767613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01dc15edd0 0x7f01dc09aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.556+0000 7f01dbe00640 1 -- 172.21.15.7:0/2872767613 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01dc14aa60 con 0x7f01dc06d230 2024-09-10T21:49:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.556+0000 7f01e1200640 1 --2- 172.21.15.7:0/2872767613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01dc15edd0 0x7f01dc09aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.556+0000 7f01dbe00640 1 --2- 172.21.15.7:0/2872767613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01dc06d230 0x7f01dc1471f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00c500 tx=0x7f01cc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.557+0000 7f01d8c00640 1 -- 172.21.15.7:0/2872767613 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc017070 con 0x7f01dc06d230 2024-09-10T21:49:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.557+0000 7f01d8c00640 1 -- 172.21.15.7:0/2872767613 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f01cc0035f0 con 0x7f01dc06d230 2024-09-10T21:49:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.557+0000 7f01d8c00640 1 -- 172.21.15.7:0/2872767613 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01cc003a40 con 0x7f01dc06d230 2024-09-10T21:49:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.557+0000 7f01e3200640 1 -- 172.21.15.7:0/2872767613 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01dc09cc70 con 0x7f01dc06d230 2024-09-10T21:49:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.557+0000 7f01e3200640 1 -- 172.21.15.7:0/2872767613 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01dc09d230 con 0x7f01dc06d230 2024-09-10T21:49:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.558+0000 7f01e3200640 1 -- 172.21.15.7:0/2872767613 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01dc0b4a30 con 0x7f01dc06d230 2024-09-10T21:49:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.559+0000 7f01d8c00640 1 -- 172.21.15.7:0/2872767613 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f01cc003be0 con 0x7f01dc06d230 2024-09-10T21:49:48.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.560+0000 7f01d8c00640 1 --2- 172.21.15.7:0/2872767613 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f01b8077f70 0x7f01b807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:48.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.560+0000 7f01d8c00640 1 -- 172.21.15.7:0/2872767613 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 5558+0+0 (secure 0 0 0) 0x7f01cc099390 con 0x7f01dc06d230 2024-09-10T21:49:48.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.560+0000 7f01d3400640 1 --2- 172.21.15.7:0/2872767613 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f01b8077f70 0x7f01b807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.561+0000 7f01d3400640 1 --2- 172.21.15.7:0/2872767613 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f01b8077f70 0x7f01b807a430 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f01c4034000 tx=0x7f01c4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:48.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.564+0000 7f01d8c00640 1 -- 172.21.15.7:0/2872767613 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f01cc0618c0 con 0x7f01dc06d230 2024-09-10T21:49:48.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:48.760+0000 7f01e3200640 1 -- 172.21.15.7:0/2872767613 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f01dc0b2190 con 0x7f01dc06d230 2024-09-10T21:49:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.359+0000 7f01d8c00640 1 -- 172.21.15.7:0/2872767613 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) ==== 169+0+0 (secure 0 0 0) 0x7f01cc0662f0 con 0x7f01dc06d230 2024-09-10T21:49:49.361 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-09-10T21:49:49.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.363+0000 7f01e3200640 1 -- 172.21.15.7:0/2872767613 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f01b8077f70 msgr2=0x7f01b807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:49.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.363+0000 7f01e3200640 1 --2- 172.21.15.7:0/2872767613 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f01b8077f70 0x7f01b807a430 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f01c4034000 tx=0x7f01c4004730 comp rx=0 tx=0).stop 2024-09-10T21:49:49.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.363+0000 7f01e3200640 1 -- 172.21.15.7:0/2872767613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01dc06d230 msgr2=0x7f01dc1471f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:49.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.363+0000 7f01e3200640 1 --2- 172.21.15.7:0/2872767613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01dc06d230 0x7f01dc1471f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00c500 tx=0x7f01cc00c9c0 comp rx=0 tx=0).stop 2024-09-10T21:49:49.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.364+0000 7f01e3200640 1 -- 172.21.15.7:0/2872767613 shutdown_connections 2024-09-10T21:49:49.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.364+0000 7f01e3200640 1 --2- 172.21.15.7:0/2872767613 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f01b8077f70 0x7f01b807a430 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:49.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.364+0000 7f01e3200640 1 --2- 172.21.15.7:0/2872767613 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01dc15edd0 0x7f01dc09aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:49.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.364+0000 7f01e3200640 1 --2- 172.21.15.7:0/2872767613 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f01dc158300 0x7f01dc09da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:49.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.364+0000 7f01e3200640 1 --2- 172.21.15.7:0/2872767613 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01dc06d230 0x7f01dc1471f0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:49.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.364+0000 7f01e3200640 1 -- 172.21.15.7:0/2872767613 >> 172.21.15.7:0/2872767613 conn(0x7f01dc0b1c10 msgr2=0x7f01dc0af8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:49.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.364+0000 7f01e3200640 1 -- 172.21.15.7:0/2872767613 shutdown_connections 2024-09-10T21:49:49.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.364+0000 7f01e3200640 1 -- 172.21.15.7:0/2872767613 wait complete. 2024-09-10T21:49:49.385 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:49:49.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.555+0000 7f6291000640 1 -- 172.21.15.7:0/2996688477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f628c06ff50 msgr2=0x7f628c15cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:49.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.555+0000 7f6291000640 1 --2- 172.21.15.7:0/2996688477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f628c06ff50 0x7f628c15cbe0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f628c073840 tx=0x7f627c02a4d0 comp rx=0 tx=0).stop 2024-09-10T21:49:49.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.555+0000 7f6291000640 1 -- 172.21.15.7:0/2996688477 shutdown_connections 2024-09-10T21:49:49.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.555+0000 7f6291000640 1 --2- 172.21.15.7:0/2996688477 >> v2:172.21.15.133:3300/0 conn(0x7f628c06e510 0x7f628c073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:49.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.555+0000 7f6291000640 1 --2- 172.21.15.7:0/2996688477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f628c06ff50 0x7f628c15cbe0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:49.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.555+0000 7f6291000640 1 -- 172.21.15.7:0/2996688477 >> 172.21.15.7:0/2996688477 conn(0x7f628c0b1660 msgr2=0x7f628c0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:49.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.555+0000 7f6291000640 1 -- 172.21.15.7:0/2996688477 shutdown_connections 2024-09-10T21:49:49.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.555+0000 7f6291000640 1 -- 172.21.15.7:0/2996688477 wait complete. 2024-09-10T21:49:49.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.557+0000 7f6291000640 1 Processor -- start 2024-09-10T21:49:49.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.557+0000 7f6291000640 1 -- start start 2024-09-10T21:49:49.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.558+0000 7f6291000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f628c06e510 0x7f628c156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:49.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.558+0000 7f6291000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f628c06e960 0x7f628c0cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.558+0000 7f6291000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f628c06ff50 0x7f628c0cdc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.558+0000 7f6291000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f628c0745a0 con 0x7f628c06ff50 2024-09-10T21:49:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.558+0000 7f6291000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f628c074420 con 0x7f628c06e960 2024-09-10T21:49:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.558+0000 7f6291000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f628c074720 con 0x7f628c06e510 2024-09-10T21:49:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.558+0000 7f628a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f628c06ff50 0x7f628c0cdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.558+0000 7f628a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f628c06ff50 0x7f628c0cdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37346/0 (socket says 172.21.15.7:37346) 2024-09-10T21:49:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.558+0000 7f628a800640 1 -- 172.21.15.7:0/1439925966 learned_addr learned my addr 172.21.15.7:0/1439925966 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.558+0000 7f6289e00640 1 --2- 172.21.15.7:0/1439925966 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f628c06e510 0x7f628c156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.558+0000 7f6281400640 1 --2- 172.21.15.7:0/1439925966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f628c06e960 0x7f628c0cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.558+0000 7f628a800640 1 -- 172.21.15.7:0/1439925966 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f628c06e510 msgr2=0x7f628c156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.558+0000 7f628a800640 1 --2- 172.21.15.7:0/1439925966 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f628c06e510 0x7f628c156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.558+0000 7f628a800640 1 -- 172.21.15.7:0/1439925966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f628c06e960 msgr2=0x7f628c0cd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.558+0000 7f628a800640 1 --2- 172.21.15.7:0/1439925966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f628c06e960 0x7f628c0cd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:49.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.558+0000 7f628a800640 1 -- 172.21.15.7:0/1439925966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628c1f84b0 con 0x7f628c06ff50 2024-09-10T21:49:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.558+0000 7f6289e00640 1 --2- 172.21.15.7:0/1439925966 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f628c06e510 0x7f628c156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.558+0000 7f6281400640 1 --2- 172.21.15.7:0/1439925966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f628c06e960 0x7f628c0cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.559+0000 7f628a800640 1 --2- 172.21.15.7:0/1439925966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f628c06ff50 0x7f628c0cdc00 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f628400c7b0 tx=0x7f628400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.559+0000 7f6282a00640 1 -- 172.21.15.7:0/1439925966 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6284017070 con 0x7f628c06ff50 2024-09-10T21:49:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.559+0000 7f6282a00640 1 -- 172.21.15.7:0/1439925966 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6284005070 con 0x7f628c06ff50 2024-09-10T21:49:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.559+0000 7f6282a00640 1 -- 172.21.15.7:0/1439925966 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628400ce60 con 0x7f628c06ff50 2024-09-10T21:49:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.559+0000 7f6291000640 1 -- 172.21.15.7:0/1439925966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628c1f43c0 con 0x7f628c06ff50 2024-09-10T21:49:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.559+0000 7f6291000640 1 -- 172.21.15.7:0/1439925966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f628c1573f0 con 0x7f628c06ff50 2024-09-10T21:49:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.560+0000 7f6291000640 1 -- 172.21.15.7:0/1439925966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f628c1f41b0 con 0x7f628c06ff50 2024-09-10T21:49:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.561+0000 7f6282a00640 1 -- 172.21.15.7:0/1439925966 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f62840054c0 con 0x7f628c06ff50 2024-09-10T21:49:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.562+0000 7f6282a00640 1 --2- 172.21.15.7:0/1439925966 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6268077f70 0x7f626807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.562+0000 7f6282a00640 1 -- 172.21.15.7:0/1439925966 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7f6284099470 con 0x7f628c06ff50 2024-09-10T21:49:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.563+0000 7f6289e00640 1 --2- 172.21.15.7:0/1439925966 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6268077f70 0x7f626807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:49.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.563+0000 7f6289e00640 1 --2- 172.21.15.7:0/1439925966 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6268077f70 0x7f626807a430 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f627c033000 tx=0x7f627c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:49.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.566+0000 7f6282a00640 1 -- 172.21.15.7:0/1439925966 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6284062120 con 0x7f628c06ff50 2024-09-10T21:49:49.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.763+0000 7f6291000640 1 -- 172.21.15.7:0/1439925966 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f628c1600d0 con 0x7f628c06ff50 2024-09-10T21:49:49.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.767+0000 7f6282a00640 1 -- 172.21.15.7:0/1439925966 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v37) ==== 71+0+0 (secure 0 0 0) 0x7f6284061a20 con 0x7f628c06ff50 2024-09-10T21:49:49.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.769+0000 7f6291000640 1 -- 172.21.15.7:0/1439925966 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6268077f70 msgr2=0x7f626807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:49.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.769+0000 7f6291000640 1 --2- 172.21.15.7:0/1439925966 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6268077f70 0x7f626807a430 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f627c033000 tx=0x7f627c004730 comp rx=0 tx=0).stop 2024-09-10T21:49:49.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.769+0000 7f6291000640 1 -- 172.21.15.7:0/1439925966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f628c06ff50 msgr2=0x7f628c0cdc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:49.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.769+0000 7f6291000640 1 --2- 172.21.15.7:0/1439925966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f628c06ff50 0x7f628c0cdc00 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f628400c7b0 tx=0x7f628400cc70 comp rx=0 tx=0).stop 2024-09-10T21:49:49.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.769+0000 7f6291000640 1 -- 172.21.15.7:0/1439925966 shutdown_connections 2024-09-10T21:49:49.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.769+0000 7f6291000640 1 --2- 172.21.15.7:0/1439925966 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6268077f70 0x7f626807a430 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:49.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.769+0000 7f6291000640 1 --2- 172.21.15.7:0/1439925966 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f628c06e960 0x7f628c0cd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:49.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.769+0000 7f6291000640 1 --2- 172.21.15.7:0/1439925966 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f628c06ff50 0x7f628c0cdc00 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:49.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.769+0000 7f6291000640 1 --2- 172.21.15.7:0/1439925966 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f628c06e510 0x7f628c156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:49.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.769+0000 7f6291000640 1 -- 172.21.15.7:0/1439925966 >> 172.21.15.7:0/1439925966 conn(0x7f628c0b1660 msgr2=0x7f628c15ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:49.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.769+0000 7f6291000640 1 -- 172.21.15.7:0/1439925966 shutdown_connections 2024-09-10T21:49:49.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.769+0000 7f6291000640 1 -- 172.21.15.7:0/1439925966 wait complete. 2024-09-10T21:49:49.786 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair' 2024-09-10T21:49:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.946+0000 7f12c3800640 1 -- 172.21.15.7:0/1520733559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12a40052e0 msgr2=0x7f12a400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.946+0000 7f12c3800640 1 --2- 172.21.15.7:0/1520733559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12a40052e0 0x7f12a400b9c0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f12b4001d90 tx=0x7f12a4005830 comp rx=0 tx=0).stop 2024-09-10T21:49:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.947+0000 7f12c3800640 1 -- 172.21.15.7:0/1520733559 shutdown_connections 2024-09-10T21:49:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.947+0000 7f12c3800640 1 --2- 172.21.15.7:0/1520733559 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12a40052e0 0x7f12a400b9c0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.947+0000 7f12c3800640 1 -- 172.21.15.7:0/1520733559 >> 172.21.15.7:0/1520733559 conn(0x7f12bc0d1d20 msgr2=0x7f12bc0d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.947+0000 7f12c3800640 1 -- 172.21.15.7:0/1520733559 shutdown_connections 2024-09-10T21:49:49.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.947+0000 7f12c3800640 1 -- 172.21.15.7:0/1520733559 wait complete. 2024-09-10T21:49:49.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.948+0000 7f12c3800640 1 Processor -- start 2024-09-10T21:49:49.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.948+0000 7f12c3800640 1 -- start start 2024-09-10T21:49:49.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.949+0000 7f12c3800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12a40052e0 0x7f12bc0b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:49.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.949+0000 7f12c3800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12bc1f76f0 0x7f12bc0c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:49.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.949+0000 7f12c3800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12bc14af70 0x7f12bc14b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:49.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.949+0000 7f12c3800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc0cce00 con 0x7f12bc14af70 2024-09-10T21:49:49.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.949+0000 7f12c3800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f12bc0ccc80 con 0x7f12a40052e0 2024-09-10T21:49:49.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.949+0000 7f12c3800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f12bc0ccf80 con 0x7f12bc1f76f0 2024-09-10T21:49:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.950+0000 7f12c1800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12bc14af70 0x7f12bc14b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.950+0000 7f12c1800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12bc14af70 0x7f12bc14b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37356/0 (socket says 172.21.15.7:37356) 2024-09-10T21:49:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.950+0000 7f12c1800640 1 -- 172.21.15.7:0/3193125090 learned_addr learned my addr 172.21.15.7:0/3193125090 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.950+0000 7f12c0e00640 1 --2- 172.21.15.7:0/3193125090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12a40052e0 0x7f12bc0b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.950+0000 7f12bbe00640 1 --2- 172.21.15.7:0/3193125090 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12bc1f76f0 0x7f12bc0c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.950+0000 7f12c1800640 1 -- 172.21.15.7:0/3193125090 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12bc1f76f0 msgr2=0x7f12bc0c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.950+0000 7f12c1800640 1 --2- 172.21.15.7:0/3193125090 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12bc1f76f0 0x7f12bc0c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.950+0000 7f12c1800640 1 -- 172.21.15.7:0/3193125090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12a40052e0 msgr2=0x7f12bc0b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.950+0000 7f12c1800640 1 --2- 172.21.15.7:0/3193125090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12a40052e0 0x7f12bc0b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.950+0000 7f12c1800640 1 -- 172.21.15.7:0/3193125090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12bc14b930 con 0x7f12bc14af70 2024-09-10T21:49:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.950+0000 7f12c0e00640 1 --2- 172.21.15.7:0/3193125090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12a40052e0 0x7f12bc0b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.950+0000 7f12bbe00640 1 --2- 172.21.15.7:0/3193125090 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12bc1f76f0 0x7f12bc0c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:49.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.950+0000 7f12c1800640 1 --2- 172.21.15.7:0/3193125090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12bc14af70 0x7f12bc14b3f0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f12b402ac80 tx=0x7f12b4006ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:49.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.951+0000 7f12b8c00640 1 -- 172.21.15.7:0/3193125090 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4003d80 con 0x7f12bc14af70 2024-09-10T21:49:49.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.951+0000 7f12b8c00640 1 -- 172.21.15.7:0/3193125090 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12b4005b80 con 0x7f12bc14af70 2024-09-10T21:49:49.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.951+0000 7f12c3800640 1 -- 172.21.15.7:0/3193125090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12bc0c8150 con 0x7f12bc14af70 2024-09-10T21:49:49.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.951+0000 7f12b8c00640 1 -- 172.21.15.7:0/3193125090 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4005fd0 con 0x7f12bc14af70 2024-09-10T21:49:49.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.951+0000 7f12c3800640 1 -- 172.21.15.7:0/3193125090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12bc0c8710 con 0x7f12bc14af70 2024-09-10T21:49:49.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.952+0000 7f12c3800640 1 -- 172.21.15.7:0/3193125090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12bc1f7ad0 con 0x7f12bc14af70 2024-09-10T21:49:49.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.953+0000 7f12b8c00640 1 -- 172.21.15.7:0/3193125090 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f12b4005030 con 0x7f12bc14af70 2024-09-10T21:49:49.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.954+0000 7f12b8c00640 1 --2- 172.21.15.7:0/3193125090 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f129c077f70 0x7f129c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:49.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.954+0000 7f12b8c00640 1 -- 172.21.15.7:0/3193125090 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7f12b40b9ab0 con 0x7f12bc14af70 2024-09-10T21:49:49.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.955+0000 7f12c0e00640 1 --2- 172.21.15.7:0/3193125090 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f129c077f70 0x7f129c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:49.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.955+0000 7f12c0e00640 1 --2- 172.21.15.7:0/3193125090 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f129c077f70 0x7f129c07a430 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f12b0001eb0 tx=0x7f12b00067a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:49.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:49.958+0000 7f12b8c00640 1 -- 172.21.15.7:0/3193125090 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f12b4043060 con 0x7f12bc14af70 2024-09-10T21:49:50.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.228+0000 7f12c3800640 1 -- 172.21.15.7:0/3193125090 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]} v 0) -- 0x7f12bc0cee40 con 0x7f12bc14af70 2024-09-10T21:49:50.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.371+0000 7f12b8c00640 1 -- 172.21.15.7:0/3193125090 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_dirfrag_repair"]}]=0 v359) ==== 167+0+0 (secure 0 0 0) 0x7f12bc14b930 con 0x7f12bc14af70 2024-09-10T21:49:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.374+0000 7f12c3800640 1 -- 172.21.15.7:0/3193125090 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f129c077f70 msgr2=0x7f129c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.374+0000 7f12c3800640 1 --2- 172.21.15.7:0/3193125090 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f129c077f70 0x7f129c07a430 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f12b0001eb0 tx=0x7f12b00067a0 comp rx=0 tx=0).stop 2024-09-10T21:49:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.374+0000 7f12c3800640 1 -- 172.21.15.7:0/3193125090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12bc14af70 msgr2=0x7f12bc14b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.374+0000 7f12c3800640 1 --2- 172.21.15.7:0/3193125090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12bc14af70 0x7f12bc14b3f0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f12b402ac80 tx=0x7f12b4006ce0 comp rx=0 tx=0).stop 2024-09-10T21:49:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.374+0000 7f12c3800640 1 -- 172.21.15.7:0/3193125090 shutdown_connections 2024-09-10T21:49:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.374+0000 7f12c3800640 1 --2- 172.21.15.7:0/3193125090 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f129c077f70 0x7f129c07a430 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.374+0000 7f12c3800640 1 --2- 172.21.15.7:0/3193125090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f12bc14af70 0x7f12bc14b3f0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.374+0000 7f12c3800640 1 --2- 172.21.15.7:0/3193125090 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f12bc1f76f0 0x7f12bc0c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.374+0000 7f12c3800640 1 --2- 172.21.15.7:0/3193125090 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f12a40052e0 0x7f12bc0b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:50.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.374+0000 7f12c3800640 1 -- 172.21.15.7:0/3193125090 >> 172.21.15.7:0/3193125090 conn(0x7f12bc0d1d20 msgr2=0x7f12bc0b37a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:50.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.374+0000 7f12c3800640 1 -- 172.21.15.7:0/3193125090 shutdown_connections 2024-09-10T21:49:50.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.375+0000 7f12c3800640 1 -- 172.21.15.7:0/3193125090 wait complete. 2024-09-10T21:49:50.391 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ok 2024-09-10T21:49:50.391 INFO:tasks.cephfs_test_runner:Starting test: test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-10T21:49:50.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:49:50.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.552+0000 7fc42fe00640 1 -- 172.21.15.7:0/848975840 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc43015dfd0 msgr2=0x7fc4300d4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:50.553 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.552+0000 7fc42fe00640 1 --2- 172.21.15.7:0/848975840 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc43015dfd0 0x7fc4300d4870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc420009860 tx=0x7fc42002a510 comp rx=0 tx=0).stop 2024-09-10T21:49:50.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.552+0000 7fc42fe00640 1 -- 172.21.15.7:0/848975840 shutdown_connections 2024-09-10T21:49:50.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.552+0000 7fc42fe00640 1 --2- 172.21.15.7:0/848975840 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc43015dfd0 0x7fc4300d4870 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:50.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.552+0000 7fc42fe00640 1 -- 172.21.15.7:0/848975840 >> 172.21.15.7:0/848975840 conn(0x7fc4300d3d50 msgr2=0x7fc4300d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:50.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.552+0000 7fc42fe00640 1 -- 172.21.15.7:0/848975840 shutdown_connections 2024-09-10T21:49:50.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.552+0000 7fc42fe00640 1 -- 172.21.15.7:0/848975840 wait complete. 2024-09-10T21:49:50.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.554+0000 7fc42fe00640 1 Processor -- start 2024-09-10T21:49:50.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.554+0000 7fc42fe00640 1 -- start start 2024-09-10T21:49:50.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.555+0000 7fc42fe00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4300b2b50 0x7fc4300ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:50.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.555+0000 7fc42fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4300d0e70 0x7fc43015a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:50.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.555+0000 7fc42fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc43015dfd0 0x7fc43015b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:50.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.555+0000 7fc42fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc4300d2770 con 0x7fc43015dfd0 2024-09-10T21:49:50.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.555+0000 7fc42fe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc4300d25f0 con 0x7fc4300d0e70 2024-09-10T21:49:50.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.555+0000 7fc42fe00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc4300d28f0 con 0x7fc4300b2b50 2024-09-10T21:49:50.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.555+0000 7fc42f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc43015dfd0 0x7fc43015b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:50.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.555+0000 7fc42f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc43015dfd0 0x7fc43015b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37370/0 (socket says 172.21.15.7:37370) 2024-09-10T21:49:50.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.555+0000 7fc42f400640 1 -- 172.21.15.7:0/684083237 learned_addr learned my addr 172.21.15.7:0/684083237 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:50.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.555+0000 7fc42ea00640 1 --2- 172.21.15.7:0/684083237 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4300b2b50 0x7fc4300ca4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:50.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.555+0000 7fc42e000640 1 --2- 172.21.15.7:0/684083237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4300d0e70 0x7fc43015a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:50.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.555+0000 7fc42f400640 1 -- 172.21.15.7:0/684083237 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4300b2b50 msgr2=0x7fc4300ca4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:50.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.555+0000 7fc42f400640 1 --2- 172.21.15.7:0/684083237 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4300b2b50 0x7fc4300ca4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:50.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.555+0000 7fc42f400640 1 -- 172.21.15.7:0/684083237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4300d0e70 msgr2=0x7fc43015a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:50.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.555+0000 7fc42f400640 1 --2- 172.21.15.7:0/684083237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4300d0e70 0x7fc43015a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:50.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.555+0000 7fc42f400640 1 -- 172.21.15.7:0/684083237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4300c8f60 con 0x7fc43015dfd0 2024-09-10T21:49:50.557 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.556+0000 7fc42f400640 1 --2- 172.21.15.7:0/684083237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc43015dfd0 0x7fc43015b2f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fc42800c730 tx=0x7fc42800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:50.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.556+0000 7fc427400640 1 -- 172.21.15.7:0/684083237 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc428003da0 con 0x7fc43015dfd0 2024-09-10T21:49:50.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.556+0000 7fc427400640 1 -- 172.21.15.7:0/684083237 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc42800ce40 con 0x7fc43015dfd0 2024-09-10T21:49:50.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.556+0000 7fc42fe00640 1 -- 172.21.15.7:0/684083237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4301f7ff0 con 0x7fc43015dfd0 2024-09-10T21:49:50.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.556+0000 7fc427400640 1 -- 172.21.15.7:0/684083237 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4280052d0 con 0x7fc43015dfd0 2024-09-10T21:49:50.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.556+0000 7fc42fe00640 1 -- 172.21.15.7:0/684083237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4301f85b0 con 0x7fc43015dfd0 2024-09-10T21:49:50.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.557+0000 7fc42fe00640 1 -- 172.21.15.7:0/684083237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4300d2d70 con 0x7fc43015dfd0 2024-09-10T21:49:50.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.559+0000 7fc427400640 1 -- 172.21.15.7:0/684083237 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc428005470 con 0x7fc43015dfd0 2024-09-10T21:49:50.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.559+0000 7fc427400640 1 --2- 172.21.15.7:0/684083237 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc410077fa0 0x7fc41007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:50.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.560+0000 7fc427400640 1 -- 172.21.15.7:0/684083237 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7fc428099960 con 0x7fc43015dfd0 2024-09-10T21:49:50.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.560+0000 7fc42ea00640 1 --2- 172.21.15.7:0/684083237 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc410077fa0 0x7fc41007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:50.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.561+0000 7fc42ea00640 1 --2- 172.21.15.7:0/684083237 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc410077fa0 0x7fc41007a460 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc420034000 tx=0x7fc420004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:50.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.563+0000 7fc427400640 1 -- 172.21.15.7:0/684083237 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc428062610 con 0x7fc43015dfd0 2024-09-10T21:49:50.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.760+0000 7fc42fe00640 1 -- 172.21.15.7:0/684083237 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4300d3110 con 0x7fc43015dfd0 2024-09-10T21:49:50.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.761+0000 7fc427400640 1 -- 172.21.15.7:0/684083237 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+13259 (secure 0 0 0) 0x7fc428061f10 con 0x7fc43015dfd0 2024-09-10T21:49:50.761 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:50.762 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":92,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:49:49.358652+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6837/4188280820":"2024-09-11T21:49:46.352223+0000","172.21.15.133:6837/3312604545":"2024-09-11T21:49:03.563183+0000","172.21.15.133:6835/3312604545":"2024-09-11T21:49:03.563183+0000","172.21.15.133:6835/4188280820":"2024-09-11T21:49:46.352223+0000","172.21.15.7:6833/1499133391":"2024-09-11T21:47:51.143906+0000","172.21.15.7:6832/1499133391":"2024-09-11T21:47:51.143906+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:49:50.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.763+0000 7fc42fe00640 1 -- 172.21.15.7:0/684083237 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc410077fa0 msgr2=0x7fc41007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:50.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.763+0000 7fc42fe00640 1 --2- 172.21.15.7:0/684083237 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc410077fa0 0x7fc41007a460 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fc420034000 tx=0x7fc420004730 comp rx=0 tx=0).stop 2024-09-10T21:49:50.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.763+0000 7fc42fe00640 1 -- 172.21.15.7:0/684083237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc43015dfd0 msgr2=0x7fc43015b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:50.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.763+0000 7fc42fe00640 1 --2- 172.21.15.7:0/684083237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc43015dfd0 0x7fc43015b2f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7fc42800c730 tx=0x7fc42800cbf0 comp rx=0 tx=0).stop 2024-09-10T21:49:50.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.763+0000 7fc42fe00640 1 -- 172.21.15.7:0/684083237 shutdown_connections 2024-09-10T21:49:50.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.763+0000 7fc42fe00640 1 --2- 172.21.15.7:0/684083237 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc410077fa0 0x7fc41007a460 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:50.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.763+0000 7fc42fe00640 1 --2- 172.21.15.7:0/684083237 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc4300d0e70 0x7fc43015a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:50.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.763+0000 7fc42fe00640 1 --2- 172.21.15.7:0/684083237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc43015dfd0 0x7fc43015b2f0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:50.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.763+0000 7fc42fe00640 1 --2- 172.21.15.7:0/684083237 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc4300b2b50 0x7fc4300ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:50.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.763+0000 7fc42fe00640 1 -- 172.21.15.7:0/684083237 >> 172.21.15.7:0/684083237 conn(0x7fc4300d3d50 msgr2=0x7fc4300d4130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:50.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.763+0000 7fc42fe00640 1 -- 172.21.15.7:0/684083237 shutdown_connections 2024-09-10T21:49:50.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.763+0000 7fc42fe00640 1 -- 172.21.15.7:0/684083237 wait complete. 2024-09-10T21:49:50.779 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:41:55.889618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:49:50.780 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:49:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.942+0000 7f51f1a00640 1 -- 172.21.15.7:0/4274675659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51ec0cf120 msgr2=0x7f51ec06f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.942+0000 7f51f1a00640 1 --2- 172.21.15.7:0/4274675659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51ec0cf120 0x7f51ec06f240 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f51d4009830 tx=0x7f51d402a480 comp rx=0 tx=0).stop 2024-09-10T21:49:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.942+0000 7f51f1a00640 1 -- 172.21.15.7:0/4274675659 shutdown_connections 2024-09-10T21:49:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.942+0000 7f51f1a00640 1 --2- 172.21.15.7:0/4274675659 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51ec06f780 0x7f51ec15e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.942+0000 7f51f1a00640 1 --2- 172.21.15.7:0/4274675659 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51ec0cf120 0x7f51ec06f240 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.942+0000 7f51f1a00640 1 -- 172.21.15.7:0/4274675659 >> 172.21.15.7:0/4274675659 conn(0x7f51ec0d0b40 msgr2=0x7f51ec0d0f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.942+0000 7f51f1a00640 1 -- 172.21.15.7:0/4274675659 shutdown_connections 2024-09-10T21:49:50.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.942+0000 7f51f1a00640 1 -- 172.21.15.7:0/4274675659 wait complete. 2024-09-10T21:49:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.943+0000 7f51f1a00640 1 Processor -- start 2024-09-10T21:49:50.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.944+0000 7f51f1a00640 1 -- start start 2024-09-10T21:49:50.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.944+0000 7f51f1a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51ec06f780 0x7f51ec1f8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.944+0000 7f51f1a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ec0cf120 0x7f51ec0b6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.944+0000 7f51f1a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51ec15ac30 0x7f51ec156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.944+0000 7f51f1a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec15ed20 con 0x7f51ec06f780 2024-09-10T21:49:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.944+0000 7f51f1a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f51ec15eba0 con 0x7f51ec0cf120 2024-09-10T21:49:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.944+0000 7f51f1a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f51ec15eea0 con 0x7f51ec15ac30 2024-09-10T21:49:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.945+0000 7f51ebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51ec06f780 0x7f51ec1f8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.945+0000 7f51ebe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51ec06f780 0x7f51ec1f8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37378/0 (socket says 172.21.15.7:37378) 2024-09-10T21:49:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.945+0000 7f51ebe00640 1 -- 172.21.15.7:0/614829827 learned_addr learned my addr 172.21.15.7:0/614829827 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.945+0000 7f51f1000640 1 --2- 172.21.15.7:0/614829827 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51ec15ac30 0x7f51ec156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:50.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.945+0000 7f51e3400640 1 --2- 172.21.15.7:0/614829827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ec0cf120 0x7f51ec0b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.945+0000 7f51ebe00640 1 -- 172.21.15.7:0/614829827 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51ec15ac30 msgr2=0x7f51ec156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.945+0000 7f51ebe00640 1 --2- 172.21.15.7:0/614829827 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51ec15ac30 0x7f51ec156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.945+0000 7f51ebe00640 1 -- 172.21.15.7:0/614829827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ec0cf120 msgr2=0x7f51ec0b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.945+0000 7f51ebe00640 1 --2- 172.21.15.7:0/614829827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ec0cf120 0x7f51ec0b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.945+0000 7f51ebe00640 1 -- 172.21.15.7:0/614829827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51ec1f8c60 con 0x7f51ec06f780 2024-09-10T21:49:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.945+0000 7f51e3400640 1 --2- 172.21.15.7:0/614829827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ec0cf120 0x7f51ec0b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.946+0000 7f51ebe00640 1 --2- 172.21.15.7:0/614829827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51ec06f780 0x7f51ec1f8720 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00c3a0 tx=0x7f51dc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.946+0000 7f51e8c00640 1 -- 172.21.15.7:0/614829827 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc011790 con 0x7f51ec06f780 2024-09-10T21:49:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.946+0000 7f51e8c00640 1 -- 172.21.15.7:0/614829827 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51dc017070 con 0x7f51ec06f780 2024-09-10T21:49:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.946+0000 7f51f1a00640 1 -- 172.21.15.7:0/614829827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51ec1f4aa0 con 0x7f51ec06f780 2024-09-10T21:49:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.946+0000 7f51e8c00640 1 -- 172.21.15.7:0/614829827 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51dc011cc0 con 0x7f51ec06f780 2024-09-10T21:49:50.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.946+0000 7f51f1a00640 1 -- 172.21.15.7:0/614829827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51ec1f5340 con 0x7f51ec06f780 2024-09-10T21:49:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.947+0000 7f51f1a00640 1 -- 172.21.15.7:0/614829827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51ec1f4e90 con 0x7f51ec06f780 2024-09-10T21:49:50.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.948+0000 7f51e8c00640 1 -- 172.21.15.7:0/614829827 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f51dc01c460 con 0x7f51ec06f780 2024-09-10T21:49:50.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.949+0000 7f51e8c00640 1 --2- 172.21.15.7:0/614829827 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f51c8077fa0 0x7f51c807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:50.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.949+0000 7f51e8c00640 1 -- 172.21.15.7:0/614829827 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7f51dc099870 con 0x7f51ec06f780 2024-09-10T21:49:50.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.950+0000 7f51e3400640 1 --2- 172.21.15.7:0/614829827 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f51c8077fa0 0x7f51c807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:50.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.950+0000 7f51e3400640 1 --2- 172.21.15.7:0/614829827 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f51c8077fa0 0x7f51c807a460 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f51d4034000 tx=0x7f51d4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:50.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:50.953+0000 7f51e8c00640 1 -- 172.21.15.7:0/614829827 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f51ec1f8c60 con 0x7f51ec06f780 2024-09-10T21:49:51.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.148+0000 7f51f1a00640 1 -- 172.21.15.7:0/614829827 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f51ec15f350 con 0x7f51ec06f780 2024-09-10T21:49:51.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.148+0000 7f51e8c00640 1 -- 172.21.15.7:0/614829827 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f51dc062520 con 0x7f51ec06f780 2024-09-10T21:49:51.149 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-10T21:49:51.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.151+0000 7f51f1a00640 1 -- 172.21.15.7:0/614829827 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f51c8077fa0 msgr2=0x7f51c807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:51.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.151+0000 7f51f1a00640 1 --2- 172.21.15.7:0/614829827 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f51c8077fa0 0x7f51c807a460 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f51d4034000 tx=0x7f51d4004730 comp rx=0 tx=0).stop 2024-09-10T21:49:51.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.151+0000 7f51f1a00640 1 -- 172.21.15.7:0/614829827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51ec06f780 msgr2=0x7f51ec1f8720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:51.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.151+0000 7f51f1a00640 1 --2- 172.21.15.7:0/614829827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51ec06f780 0x7f51ec1f8720 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f51dc00c3a0 tx=0x7f51dc00c860 comp rx=0 tx=0).stop 2024-09-10T21:49:51.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.151+0000 7f51f1a00640 1 -- 172.21.15.7:0/614829827 shutdown_connections 2024-09-10T21:49:51.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.151+0000 7f51f1a00640 1 --2- 172.21.15.7:0/614829827 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f51c8077fa0 0x7f51c807a460 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:51.152 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.151+0000 7f51f1a00640 1 --2- 172.21.15.7:0/614829827 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f51ec0cf120 0x7f51ec0b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:51.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.151+0000 7f51f1a00640 1 --2- 172.21.15.7:0/614829827 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51ec15ac30 0x7f51ec156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:51.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.151+0000 7f51f1a00640 1 --2- 172.21.15.7:0/614829827 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f51ec06f780 0x7f51ec1f8720 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:51.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.151+0000 7f51f1a00640 1 -- 172.21.15.7:0/614829827 >> 172.21.15.7:0/614829827 conn(0x7f51ec0d0b40 msgr2=0x7f51ec159880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:51.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.151+0000 7f51f1a00640 1 -- 172.21.15.7:0/614829827 shutdown_connections 2024-09-10T21:49:51.153 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.151+0000 7f51f1a00640 1 -- 172.21.15.7:0/614829827 wait complete. 2024-09-10T21:49:51.168 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-09-10T21:49:51.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.327+0000 7f43b3200640 1 -- 172.21.15.7:0/4128481810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4398003cb0 msgr2=0x7f43980040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:51.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.327+0000 7f43b3200640 1 --2- 172.21.15.7:0/4128481810 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4398003cb0 0x7f43980040b0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f43a4001860 tx=0x7f4398005830 comp rx=0 tx=0).stop 2024-09-10T21:49:51.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.327+0000 7f43b2800640 1 -- 172.21.15.7:0/4128481810 reap_dead start 2024-09-10T21:49:51.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.327+0000 7f43b3200640 1 -- 172.21.15.7:0/4128481810 shutdown_connections 2024-09-10T21:49:51.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.327+0000 7f43b3200640 1 -- 172.21.15.7:0/4128481810 >> 172.21.15.7:0/4128481810 conn(0x7f43b40b1d70 msgr2=0x7f43b40cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:51.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.327+0000 7f43b3200640 1 -- 172.21.15.7:0/4128481810 shutdown_connections 2024-09-10T21:49:51.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.327+0000 7f43b3200640 1 -- 172.21.15.7:0/4128481810 wait complete. 2024-09-10T21:49:51.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.329+0000 7f43b3200640 1 Processor -- start 2024-09-10T21:49:51.329 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.329+0000 7f43b3200640 1 -- start start 2024-09-10T21:49:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.330+0000 7f43b3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4398003cb0 0x7f43b415d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.330+0000 7f43b3200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43b415d720 0x7f43b4140310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.330+0000 7f43b3200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43b40af6b0 0x7f43b40afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.330+0000 7f43b3200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f43b4166b80 con 0x7f4398003cb0 2024-09-10T21:49:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.330+0000 7f43b3200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f43b406ee10 con 0x7f43b40af6b0 2024-09-10T21:49:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.330+0000 7f43b3200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f43b4166d00 con 0x7f43b415d720 2024-09-10T21:49:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.330+0000 7f43b1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4398003cb0 0x7f43b415d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.330+0000 7f43b1e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4398003cb0 0x7f43b415d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37388/0 (socket says 172.21.15.7:37388) 2024-09-10T21:49:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.330+0000 7f43b1e00640 1 -- 172.21.15.7:0/2526603370 learned_addr learned my addr 172.21.15.7:0/2526603370 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:51.332 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.330+0000 7f43b1400640 1 --2- 172.21.15.7:0/2526603370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43b415d720 0x7f43b4140310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:51.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.330+0000 7f43b2800640 1 --2- 172.21.15.7:0/2526603370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43b40af6b0 0x7f43b40afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:51.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.330+0000 7f43b1e00640 1 -- 172.21.15.7:0/2526603370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43b415d720 msgr2=0x7f43b4140310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:51.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.330+0000 7f43b1e00640 1 --2- 172.21.15.7:0/2526603370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43b415d720 0x7f43b4140310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:51.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.330+0000 7f43b1e00640 1 -- 172.21.15.7:0/2526603370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43b40af6b0 msgr2=0x7f43b40afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:51.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.330+0000 7f43b1e00640 1 --2- 172.21.15.7:0/2526603370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43b40af6b0 0x7f43b40afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:51.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.330+0000 7f43b1e00640 1 -- 172.21.15.7:0/2526603370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b4160770 con 0x7f4398003cb0 2024-09-10T21:49:51.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.330+0000 7f43b1400640 1 --2- 172.21.15.7:0/2526603370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43b415d720 0x7f43b4140310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:51.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.331+0000 7f43b1e00640 1 --2- 172.21.15.7:0/2526603370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4398003cb0 0x7f43b415d120 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f43a4001980 tx=0x7f43a4006060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:51.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.331+0000 7f43aa000640 1 -- 172.21.15.7:0/2526603370 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4043070 con 0x7f4398003cb0 2024-09-10T21:49:51.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.331+0000 7f43aa000640 1 -- 172.21.15.7:0/2526603370 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43a40038e0 con 0x7f4398003cb0 2024-09-10T21:49:51.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.331+0000 7f43aa000640 1 -- 172.21.15.7:0/2526603370 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a4003d30 con 0x7f4398003cb0 2024-09-10T21:49:51.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.331+0000 7f43b3200640 1 -- 172.21.15.7:0/2526603370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b41f7d80 con 0x7f4398003cb0 2024-09-10T21:49:51.333 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.331+0000 7f43b3200640 1 -- 172.21.15.7:0/2526603370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b41f8340 con 0x7f4398003cb0 2024-09-10T21:49:51.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.332+0000 7f43b3200640 1 -- 172.21.15.7:0/2526603370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b40723b0 con 0x7f4398003cb0 2024-09-10T21:49:51.334 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.333+0000 7f43aa000640 1 -- 172.21.15.7:0/2526603370 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f43a4033480 con 0x7f4398003cb0 2024-09-10T21:49:51.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.334+0000 7f43aa000640 1 --2- 172.21.15.7:0/2526603370 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4394077f70 0x7f439407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:51.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.334+0000 7f43aa000640 1 -- 172.21.15.7:0/2526603370 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7f43a40b9a00 con 0x7f4398003cb0 2024-09-10T21:49:51.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.335+0000 7f43b1400640 1 --2- 172.21.15.7:0/2526603370 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4394077f70 0x7f439407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:51.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.335+0000 7f43b1400640 1 --2- 172.21.15.7:0/2526603370 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4394077f70 0x7f439407a430 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f43a00010e0 tx=0x7f43a00018f0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:51.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.338+0000 7f43aa000640 1 -- 172.21.15.7:0/2526603370 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f43a4082750 con 0x7f4398003cb0 2024-09-10T21:49:51.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:51.604+0000 7f43b3200640 1 -- 172.21.15.7:0/2526603370 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f43b4161570 con 0x7f4398003cb0 2024-09-10T21:49:52.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.380+0000 7f43aa000640 1 -- 172.21.15.7:0/2526603370 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v361) ==== 149+0+0 (secure 0 0 0) 0x7f43a4082050 con 0x7f4398003cb0 2024-09-10T21:49:52.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.383+0000 7f43b3200640 1 -- 172.21.15.7:0/2526603370 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4394077f70 msgr2=0x7f439407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:52.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.383+0000 7f43b3200640 1 --2- 172.21.15.7:0/2526603370 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4394077f70 0x7f439407a430 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f43a00010e0 tx=0x7f43a00018f0 comp rx=0 tx=0).stop 2024-09-10T21:49:52.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.383+0000 7f43b3200640 1 -- 172.21.15.7:0/2526603370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4398003cb0 msgr2=0x7f43b415d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:52.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.383+0000 7f43b3200640 1 --2- 172.21.15.7:0/2526603370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4398003cb0 0x7f43b415d120 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f43a4001980 tx=0x7f43a4006060 comp rx=0 tx=0).stop 2024-09-10T21:49:52.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.383+0000 7f43b3200640 1 -- 172.21.15.7:0/2526603370 shutdown_connections 2024-09-10T21:49:52.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.383+0000 7f43b3200640 1 --2- 172.21.15.7:0/2526603370 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4394077f70 0x7f439407a430 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:52.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.383+0000 7f43b3200640 1 --2- 172.21.15.7:0/2526603370 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f43b40af6b0 0x7f43b40afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:52.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.383+0000 7f43b3200640 1 --2- 172.21.15.7:0/2526603370 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f43b415d720 0x7f43b4140310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:52.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.383+0000 7f43b3200640 1 --2- 172.21.15.7:0/2526603370 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4398003cb0 0x7f43b415d120 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:52.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.383+0000 7f43b3200640 1 -- 172.21.15.7:0/2526603370 >> 172.21.15.7:0/2526603370 conn(0x7f43b40b1d70 msgr2=0x7f43b40cf710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:52.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.383+0000 7f43b3200640 1 -- 172.21.15.7:0/2526603370 shutdown_connections 2024-09-10T21:49:52.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.383+0000 7f43b3200640 1 -- 172.21.15.7:0/2526603370 wait complete. 2024-09-10T21:49:52.402 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:49:52.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.565+0000 7fc1c5800640 1 -- 172.21.15.7:0/2652338875 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1c01613f0 msgr2=0x7fc1c00cfe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.565+0000 7fc1c5800640 1 --2- 172.21.15.7:0/2652338875 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1c01613f0 0x7fc1c00cfe80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0009830 tx=0x7fc1b002a480 comp rx=0 tx=0).stop 2024-09-10T21:49:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.565+0000 7fc1c5800640 1 -- 172.21.15.7:0/2652338875 shutdown_connections 2024-09-10T21:49:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.565+0000 7fc1c5800640 1 --2- 172.21.15.7:0/2652338875 >> v2:172.21.15.133:3300/0 conn(0x7fc1c0162460 0x7fc1c00b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.565+0000 7fc1c5800640 1 --2- 172.21.15.7:0/2652338875 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1c01613f0 0x7fc1c00cfe80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.565+0000 7fc1c5800640 1 -- 172.21.15.7:0/2652338875 >> 172.21.15.7:0/2652338875 conn(0x7fc1c00d0430 msgr2=0x7fc1c00d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.565+0000 7fc1c5800640 1 -- 172.21.15.7:0/2652338875 shutdown_connections 2024-09-10T21:49:52.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.565+0000 7fc1c5800640 1 -- 172.21.15.7:0/2652338875 wait complete. 2024-09-10T21:49:52.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.567+0000 7fc1c5800640 1 Processor -- start 2024-09-10T21:49:52.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.567+0000 7fc1c5800640 1 -- start start 2024-09-10T21:49:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.568+0000 7fc1c5800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1c00cce20 0x7fc1c00c7500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.568+0000 7fc1c5800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1c01613f0 0x7fc1c00aee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.568+0000 7fc1c5800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c0162460 0x7fc1c00b5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.568+0000 7fc1c5800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c00cd660 con 0x7fc1c0162460 2024-09-10T21:49:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.568+0000 7fc1c5800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c00cd4e0 con 0x7fc1c01613f0 2024-09-10T21:49:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.568+0000 7fc1c5800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c00afab0 con 0x7fc1c00cce20 2024-09-10T21:49:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.569+0000 7fc1bf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c0162460 0x7fc1c00b5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:52.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.569+0000 7fc1bf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c0162460 0x7fc1c00b5170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37390/0 (socket says 172.21.15.7:37390) 2024-09-10T21:49:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.569+0000 7fc1bf400640 1 -- 172.21.15.7:0/1462769289 learned_addr learned my addr 172.21.15.7:0/1462769289 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.569+0000 7fc1bea00640 1 --2- 172.21.15.7:0/1462769289 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1c00cce20 0x7fc1c00c7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.569+0000 7fc1be000640 1 --2- 172.21.15.7:0/1462769289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1c01613f0 0x7fc1c00aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.569+0000 7fc1bf400640 1 -- 172.21.15.7:0/1462769289 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1c00cce20 msgr2=0x7fc1c00c7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.569+0000 7fc1bf400640 1 --2- 172.21.15.7:0/1462769289 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1c00cce20 0x7fc1c00c7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.569+0000 7fc1bf400640 1 -- 172.21.15.7:0/1462769289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1c01613f0 msgr2=0x7fc1c00aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.569+0000 7fc1bf400640 1 --2- 172.21.15.7:0/1462769289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1c01613f0 0x7fc1c00aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.569+0000 7fc1bf400640 1 -- 172.21.15.7:0/1462769289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c01f4b30 con 0x7fc1c0162460 2024-09-10T21:49:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.569+0000 7fc1be000640 1 --2- 172.21.15.7:0/1462769289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1c01613f0 0x7fc1c00aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:52.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.569+0000 7fc1bf400640 1 --2- 172.21.15.7:0/1462769289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c0162460 0x7fc1c00b5170 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800cbc0 tx=0x7fc1b800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.570+0000 7fc1b6a00640 1 -- 172.21.15.7:0/1462769289 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8016020 con 0x7fc1c0162460 2024-09-10T21:49:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.570+0000 7fc1b6a00640 1 -- 172.21.15.7:0/1462769289 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1b800ae30 con 0x7fc1c0162460 2024-09-10T21:49:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.570+0000 7fc1b6a00640 1 -- 172.21.15.7:0/1462769289 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8012660 con 0x7fc1c0162460 2024-09-10T21:49:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.570+0000 7fc1c5800640 1 -- 172.21.15.7:0/1462769289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c01f8650 con 0x7fc1c0162460 2024-09-10T21:49:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.570+0000 7fc1c5800640 1 -- 172.21.15.7:0/1462769289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c01f8c10 con 0x7fc1c0162460 2024-09-10T21:49:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.571+0000 7fc1c5800640 1 -- 172.21.15.7:0/1462769289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1c00aff60 con 0x7fc1c0162460 2024-09-10T21:49:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.572+0000 7fc1b6a00640 1 -- 172.21.15.7:0/1462769289 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc1b801a3b0 con 0x7fc1c0162460 2024-09-10T21:49:52.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.573+0000 7fc1b6a00640 1 --2- 172.21.15.7:0/1462769289 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc1a0077f70 0x7fc1a007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:52.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.573+0000 7fc1b6a00640 1 -- 172.21.15.7:0/1462769289 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7fc1b809a1d0 con 0x7fc1c0162460 2024-09-10T21:49:52.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.573+0000 7fc1bea00640 1 --2- 172.21.15.7:0/1462769289 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc1a0077f70 0x7fc1a007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:52.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.574+0000 7fc1bea00640 1 --2- 172.21.15.7:0/1462769289 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc1a0077f70 0x7fc1a007a430 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0034000 tx=0x7fc1b0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:52.578 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.577+0000 7fc1b6a00640 1 -- 172.21.15.7:0/1462769289 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc1b8020030 con 0x7fc1c0162460 2024-09-10T21:49:52.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.774+0000 7fc1c5800640 1 -- 172.21.15.7:0/1462769289 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fc1c0160bd0 con 0x7fc1c0162460 2024-09-10T21:49:52.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.774+0000 7fc1b6a00640 1 -- 172.21.15.7:0/1462769289 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+1998 (secure 0 0 0) 0x7fc1b8062e80 con 0x7fc1c0162460 2024-09-10T21:49:52.775 INFO:teuthology.orchestra.run.smithi007.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3770.177547 2024-09-10T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3542.062395 2024-09-10T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3788.037135 2024-09-10T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5808.986321 2024-09-10T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4630.109911 2024-09-10T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4510.255493 2024-09-10T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4792.698527 2024-09-10T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6842.370281 2024-09-10T21:49:52.776 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_mds 20/20 2024-09-10T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_ms 2/2 2024-09-10T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_debug_frag true 2024-09-10T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_verify_scatter true 2024-09-10T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced client_mount_timeout 600 2024-09-10T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced debug_client 20/20 2024-09-10T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced debug_ms 1/1 2024-09-10T21:49:52.777 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:49:52.778 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:49:52.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.777+0000 7fc1c5800640 1 -- 172.21.15.7:0/1462769289 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc1a0077f70 msgr2=0x7fc1a007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.777+0000 7fc1c5800640 1 --2- 172.21.15.7:0/1462769289 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc1a0077f70 0x7fc1a007a430 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0034000 tx=0x7fc1b0004730 comp rx=0 tx=0).stop 2024-09-10T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.777+0000 7fc1c5800640 1 -- 172.21.15.7:0/1462769289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c0162460 msgr2=0x7fc1c00b5170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.777+0000 7fc1c5800640 1 --2- 172.21.15.7:0/1462769289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c0162460 0x7fc1c00b5170 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fc1b800cbc0 tx=0x7fc1b800a430 comp rx=0 tx=0).stop 2024-09-10T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.777+0000 7fc1c5800640 1 -- 172.21.15.7:0/1462769289 shutdown_connections 2024-09-10T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.777+0000 7fc1c5800640 1 --2- 172.21.15.7:0/1462769289 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc1a0077f70 0x7fc1a007a430 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.777+0000 7fc1c5800640 1 --2- 172.21.15.7:0/1462769289 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc1c01613f0 0x7fc1c00aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.777+0000 7fc1c5800640 1 --2- 172.21.15.7:0/1462769289 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc1c0162460 0x7fc1c00b5170 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.777+0000 7fc1c5800640 1 --2- 172.21.15.7:0/1462769289 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc1c00cce20 0x7fc1c00c7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.777+0000 7fc1c5800640 1 -- 172.21.15.7:0/1462769289 >> 172.21.15.7:0/1462769289 conn(0x7fc1c00d0430 msgr2=0x7fc1c006f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.777+0000 7fc1c5800640 1 -- 172.21.15.7:0/1462769289 shutdown_connections 2024-09-10T21:49:52.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.777+0000 7fc1c5800640 1 -- 172.21.15.7:0/1462769289 wait complete. 2024-09-10T21:49:52.793 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:49:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.955+0000 7f4e56a00640 1 -- 172.21.15.7:0/1709939939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5015a5e0 msgr2=0x7f4e5015aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.955+0000 7f4e56a00640 1 --2- 172.21.15.7:0/1709939939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5015a5e0 0x7f4e5015aa70 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f4e500d0bc0 tx=0x7f4e3802a5f0 comp rx=0 tx=0).stop 2024-09-10T21:49:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.956+0000 7f4e56a00640 1 -- 172.21.15.7:0/1709939939 shutdown_connections 2024-09-10T21:49:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.956+0000 7f4e56a00640 1 --2- 172.21.15.7:0/1709939939 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5015a5e0 0x7f4e5015aa70 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.956+0000 7f4e56a00640 1 --2- 172.21.15.7:0/1709939939 >> v2:172.21.15.133:3300/0 conn(0x7f4e50070670 0x7f4e50154370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.956+0000 7f4e56a00640 1 -- 172.21.15.7:0/1709939939 >> 172.21.15.7:0/1709939939 conn(0x7f4e500b16d0 msgr2=0x7f4e500b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.956+0000 7f4e56a00640 1 -- 172.21.15.7:0/1709939939 shutdown_connections 2024-09-10T21:49:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.956+0000 7f4e56a00640 1 -- 172.21.15.7:0/1709939939 wait complete. 2024-09-10T21:49:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.957+0000 7f4e56a00640 1 Processor -- start 2024-09-10T21:49:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.957+0000 7f4e56a00640 1 -- start start 2024-09-10T21:49:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.958+0000 7f4e56a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e50070670 0x7f4e501f73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.958+0000 7f4e56a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e5015a5e0 0x7f4e501f7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.958+0000 7f4e56a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5015d9a0 0x7f4e501f80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.958+0000 7f4e56a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5015e660 con 0x7f4e5015d9a0 2024-09-10T21:49:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.958+0000 7f4e56a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4e5015e4e0 con 0x7f4e5015a5e0 2024-09-10T21:49:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.958+0000 7f4e56a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4e5015e7e0 con 0x7f4e50070670 2024-09-10T21:49:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.959+0000 7f4e54a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5015d9a0 0x7f4e501f80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.959+0000 7f4e54a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5015d9a0 0x7f4e501f80b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37406/0 (socket says 172.21.15.7:37406) 2024-09-10T21:49:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.959+0000 7f4e54a00640 1 -- 172.21.15.7:0/2117842458 learned_addr learned my addr 172.21.15.7:0/2117842458 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.959+0000 7f4e47400640 1 --2- 172.21.15.7:0/2117842458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e5015a5e0 0x7f4e501f7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.959+0000 7f4e4fe00640 1 --2- 172.21.15.7:0/2117842458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e50070670 0x7f4e501f73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.959+0000 7f4e54a00640 1 -- 172.21.15.7:0/2117842458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e50070670 msgr2=0x7f4e501f73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.959+0000 7f4e54a00640 1 --2- 172.21.15.7:0/2117842458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e50070670 0x7f4e501f73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.959+0000 7f4e54a00640 1 -- 172.21.15.7:0/2117842458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e5015a5e0 msgr2=0x7f4e501f7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.959+0000 7f4e54a00640 1 --2- 172.21.15.7:0/2117842458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e5015a5e0 0x7f4e501f7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.959+0000 7f4e54a00640 1 -- 172.21.15.7:0/2117842458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e500c44d0 con 0x7f4e5015d9a0 2024-09-10T21:49:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.959+0000 7f4e47400640 1 --2- 172.21.15.7:0/2117842458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e5015a5e0 0x7f4e501f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.959+0000 7f4e4fe00640 1 --2- 172.21.15.7:0/2117842458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e50070670 0x7f4e501f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:52.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.959+0000 7f4e54a00640 1 --2- 172.21.15.7:0/2117842458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5015d9a0 0x7f4e501f80b0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800c6e0 tx=0x7f4e4800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.960+0000 7f4e4cc00640 1 -- 172.21.15.7:0/2117842458 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48003b70 con 0x7f4e5015d9a0 2024-09-10T21:49:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.960+0000 7f4e4cc00640 1 -- 172.21.15.7:0/2117842458 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e48003d30 con 0x7f4e5015d9a0 2024-09-10T21:49:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.960+0000 7f4e4cc00640 1 -- 172.21.15.7:0/2117842458 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e48005330 con 0x7f4e5015d9a0 2024-09-10T21:49:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.960+0000 7f4e56a00640 1 -- 172.21.15.7:0/2117842458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e501f78e0 con 0x7f4e5015d9a0 2024-09-10T21:49:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.960+0000 7f4e56a00640 1 -- 172.21.15.7:0/2117842458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e501f88b0 con 0x7f4e5015d9a0 2024-09-10T21:49:52.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.961+0000 7f4e56a00640 1 -- 172.21.15.7:0/2117842458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e50154370 con 0x7f4e5015d9a0 2024-09-10T21:49:52.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.962+0000 7f4e4cc00640 1 -- 172.21.15.7:0/2117842458 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4e480054d0 con 0x7f4e5015d9a0 2024-09-10T21:49:52.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.963+0000 7f4e4cc00640 1 --2- 172.21.15.7:0/2117842458 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e2c077f70 0x7f4e2c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:52.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.964+0000 7f4e4cc00640 1 -- 172.21.15.7:0/2117842458 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7f4e48099970 con 0x7f4e5015d9a0 2024-09-10T21:49:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.964+0000 7f4e4fe00640 1 --2- 172.21.15.7:0/2117842458 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e2c077f70 0x7f4e2c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:52.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.965+0000 7f4e4fe00640 1 --2- 172.21.15.7:0/2117842458 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e2c077f70 0x7f4e2c07a430 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4e400058c0 tx=0x7f4e40005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:52.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:52.968+0000 7f4e4cc00640 1 -- 172.21.15.7:0/2117842458 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4e480057f0 con 0x7f4e5015d9a0 2024-09-10T21:49:53.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.163+0000 7f4e56a00640 1 -- 172.21.15.7:0/2117842458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4e50069880 con 0x7f4e5015d9a0 2024-09-10T21:49:53.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.167+0000 7f4e4cc00640 1 -- 172.21.15.7:0/2117842458 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f4e480626b0 con 0x7f4e5015d9a0 2024-09-10T21:49:53.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.170+0000 7f4e56a00640 1 -- 172.21.15.7:0/2117842458 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e2c077f70 msgr2=0x7f4e2c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:53.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.170+0000 7f4e56a00640 1 --2- 172.21.15.7:0/2117842458 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e2c077f70 0x7f4e2c07a430 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4e400058c0 tx=0x7f4e40005850 comp rx=0 tx=0).stop 2024-09-10T21:49:53.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.170+0000 7f4e56a00640 1 -- 172.21.15.7:0/2117842458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5015d9a0 msgr2=0x7f4e501f80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:53.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.170+0000 7f4e56a00640 1 --2- 172.21.15.7:0/2117842458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5015d9a0 0x7f4e501f80b0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f4e4800c6e0 tx=0x7f4e4800cba0 comp rx=0 tx=0).stop 2024-09-10T21:49:53.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.170+0000 7f4e56a00640 1 -- 172.21.15.7:0/2117842458 shutdown_connections 2024-09-10T21:49:53.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.170+0000 7f4e56a00640 1 --2- 172.21.15.7:0/2117842458 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e2c077f70 0x7f4e2c07a430 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:53.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.170+0000 7f4e56a00640 1 --2- 172.21.15.7:0/2117842458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e5015a5e0 0x7f4e501f7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:53.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.170+0000 7f4e56a00640 1 --2- 172.21.15.7:0/2117842458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e5015d9a0 0x7f4e501f80b0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:53.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.170+0000 7f4e56a00640 1 --2- 172.21.15.7:0/2117842458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e50070670 0x7f4e501f73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:53.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.170+0000 7f4e56a00640 1 -- 172.21.15.7:0/2117842458 >> 172.21.15.7:0/2117842458 conn(0x7f4e500b16d0 msgr2=0x7f4e50158e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:53.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.170+0000 7f4e56a00640 1 -- 172.21.15.7:0/2117842458 shutdown_connections 2024-09-10T21:49:53.172 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.170+0000 7f4e56a00640 1 -- 172.21.15.7:0/2117842458 wait complete. 2024-09-10T21:49:53.186 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:49:53.186 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:49:53.277 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:49:53.372 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:49:53.372 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:49:53.372 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:49:53.408 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:49:53.408 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:49:53.408 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:49:53.445 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:49:53.445 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:49:53.446 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:49:53.482 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:49:53.483 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:49:53.483 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:49:53.521 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:49:53.521 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:49:53.521 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:49:53.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.682+0000 7fe65a400640 1 -- 172.21.15.7:0/3694749549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6540d4900 msgr2=0x7fe6540b6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:53.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.682+0000 7fe65a400640 1 --2- 172.21.15.7:0/3694749549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6540d4900 0x7fe6540b6c30 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fe640009830 tx=0x7fe64002a480 comp rx=0 tx=0).stop 2024-09-10T21:49:53.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.682+0000 7fe65a400640 1 -- 172.21.15.7:0/3694749549 shutdown_connections 2024-09-10T21:49:53.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.682+0000 7fe65a400640 1 --2- 172.21.15.7:0/3694749549 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6540d4900 0x7fe6540b6c30 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:53.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.682+0000 7fe65a400640 1 -- 172.21.15.7:0/3694749549 >> 172.21.15.7:0/3694749549 conn(0x7fe6540d3de0 msgr2=0x7fe6540d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:53.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.682+0000 7fe65a400640 1 -- 172.21.15.7:0/3694749549 shutdown_connections 2024-09-10T21:49:53.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.682+0000 7fe65a400640 1 -- 172.21.15.7:0/3694749549 wait complete. 2024-09-10T21:49:53.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.683+0000 7fe65a400640 1 Processor -- start 2024-09-10T21:49:53.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.683+0000 7fe65a400640 1 -- start start 2024-09-10T21:49:53.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.684+0000 7fe65a400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6540b21f0 0x7fe6540cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:53.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.684+0000 7fe65a400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6540d4900 0x7fe6540cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:53.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.684+0000 7fe65a400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe65415dff0 0x7fe6541f8030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:53.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.684+0000 7fe65a400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe6540cfd00 con 0x7fe6540b21f0 2024-09-10T21:49:53.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.684+0000 7fe65a400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe6540cfb80 con 0x7fe65415dff0 2024-09-10T21:49:53.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.684+0000 7fe65a400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe6540cfe80 con 0x7fe6540d4900 2024-09-10T21:49:53.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.685+0000 7fe659000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6540b21f0 0x7fe6540cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:53.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.685+0000 7fe659000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6540b21f0 0x7fe6540cd5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37428/0 (socket says 172.21.15.7:37428) 2024-09-10T21:49:53.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.685+0000 7fe659000640 1 -- 172.21.15.7:0/146069432 learned_addr learned my addr 172.21.15.7:0/146069432 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.685+0000 7fe659a00640 1 --2- 172.21.15.7:0/146069432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe65415dff0 0x7fe6541f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.685+0000 7fe653e00640 1 --2- 172.21.15.7:0/146069432 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6540d4900 0x7fe6540cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.685+0000 7fe659000640 1 -- 172.21.15.7:0/146069432 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6540d4900 msgr2=0x7fe6540cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.685+0000 7fe659000640 1 --2- 172.21.15.7:0/146069432 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6540d4900 0x7fe6540cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.685+0000 7fe659000640 1 -- 172.21.15.7:0/146069432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe65415dff0 msgr2=0x7fe6541f8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.685+0000 7fe659000640 1 --2- 172.21.15.7:0/146069432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe65415dff0 0x7fe6541f8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.685+0000 7fe659000640 1 -- 172.21.15.7:0/146069432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6541f5560 con 0x7fe6540b21f0 2024-09-10T21:49:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.685+0000 7fe653e00640 1 --2- 172.21.15.7:0/146069432 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6540d4900 0x7fe6540cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:53.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.685+0000 7fe659000640 1 --2- 172.21.15.7:0/146069432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6540b21f0 0x7fe6540cd5a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fe64800c3f0 tx=0x7fe64800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:53.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.686+0000 7fe650c00640 1 -- 172.21.15.7:0/146069432 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648011790 con 0x7fe6540b21f0 2024-09-10T21:49:53.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.686+0000 7fe650c00640 1 -- 172.21.15.7:0/146069432 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe648017070 con 0x7fe6540b21f0 2024-09-10T21:49:53.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.686+0000 7fe65a400640 1 -- 172.21.15.7:0/146069432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6541f5820 con 0x7fe6540b21f0 2024-09-10T21:49:53.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.686+0000 7fe650c00640 1 -- 172.21.15.7:0/146069432 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe648011ce0 con 0x7fe6540b21f0 2024-09-10T21:49:53.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.686+0000 7fe65a400640 1 -- 172.21.15.7:0/146069432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe65415a830 con 0x7fe6540b21f0 2024-09-10T21:49:53.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.687+0000 7fe65a400640 1 -- 172.21.15.7:0/146069432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6540b6c30 con 0x7fe6540b21f0 2024-09-10T21:49:53.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.688+0000 7fe650c00640 1 -- 172.21.15.7:0/146069432 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe64801c460 con 0x7fe6540b21f0 2024-09-10T21:49:53.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.689+0000 7fe650c00640 1 --2- 172.21.15.7:0/146069432 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe634077f70 0x7fe63407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:53.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.689+0000 7fe650c00640 1 -- 172.21.15.7:0/146069432 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7fe648099930 con 0x7fe6540b21f0 2024-09-10T21:49:53.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.689+0000 7fe653e00640 1 --2- 172.21.15.7:0/146069432 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe634077f70 0x7fe63407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:53.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.690+0000 7fe653e00640 1 --2- 172.21.15.7:0/146069432 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe634077f70 0x7fe63407a430 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe640034000 tx=0x7fe640004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:53.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.693+0000 7fe650c00640 1 -- 172.21.15.7:0/146069432 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe6480625f0 con 0x7fe6540b21f0 2024-09-10T21:49:53.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.925+0000 7fe65a400640 1 -- 172.21.15.7:0/146069432 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6540d02e0 con 0x7fe6540b21f0 2024-09-10T21:49:53.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.925+0000 7fe650c00640 1 -- 172.21.15.7:0/146069432 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) ==== 94+0+3904 (secure 0 0 0) 0x7fe648061ef0 con 0x7fe6540b21f0 2024-09-10T21:49:53.926 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:53.926 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":78,"btime":"2024-09-10T21:49:47:354515+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6747,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/978763848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6834","nonce":978763848},{"type":"v1","addr":"172.21.15.7:6835","nonce":978763848}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":58},{"gid":6909,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6833/269527918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":269527918},{"type":"v1","addr":"172.21.15.7:6833","nonce":269527918}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6913,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/1062935470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":1062935470},{"type":"v1","addr":"172.21.15.133:6837","nonce":1062935470}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":78},{"gid":235775,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/760980985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":760980985},{"type":"v1","addr":"172.21.15.133:6836","nonce":760980985}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":57}],"filesystems":[]} 2024-09-10T21:49:53.926 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 78 2024-09-10T21:49:53.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.927+0000 7fe65a400640 1 -- 172.21.15.7:0/146069432 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe634077f70 msgr2=0x7fe63407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:53.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.927+0000 7fe65a400640 1 --2- 172.21.15.7:0/146069432 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe634077f70 0x7fe63407a430 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fe640034000 tx=0x7fe640004730 comp rx=0 tx=0).stop 2024-09-10T21:49:53.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.927+0000 7fe65a400640 1 -- 172.21.15.7:0/146069432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6540b21f0 msgr2=0x7fe6540cd5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:53.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.927+0000 7fe65a400640 1 --2- 172.21.15.7:0/146069432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6540b21f0 0x7fe6540cd5a0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fe64800c3f0 tx=0x7fe64800c8b0 comp rx=0 tx=0).stop 2024-09-10T21:49:53.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.927+0000 7fe65a400640 1 -- 172.21.15.7:0/146069432 shutdown_connections 2024-09-10T21:49:53.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.927+0000 7fe65a400640 1 --2- 172.21.15.7:0/146069432 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe634077f70 0x7fe63407a430 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:53.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.927+0000 7fe65a400640 1 --2- 172.21.15.7:0/146069432 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe65415dff0 0x7fe6541f8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:53.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.927+0000 7fe65a400640 1 --2- 172.21.15.7:0/146069432 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe6540d4900 0x7fe6540cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:53.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.927+0000 7fe65a400640 1 --2- 172.21.15.7:0/146069432 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe6540b21f0 0x7fe6540cd5a0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:53.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.927+0000 7fe65a400640 1 -- 172.21.15.7:0/146069432 >> 172.21.15.7:0/146069432 conn(0x7fe6540d3de0 msgr2=0x7fe6540b0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:53.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.928+0000 7fe65a400640 1 -- 172.21.15.7:0/146069432 shutdown_connections 2024-09-10T21:49:53.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:53.928+0000 7fe65a400640 1 -- 172.21.15.7:0/146069432 wait complete. 2024-09-10T21:49:53.945 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:49:53.945 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:49:53.946 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:49:53.946 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:49:53.946 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:49:53.946 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:49:53.946 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:49:53.946 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:49:53.947 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:49:53.947 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:49:53.947 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:49:53.947 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:49:53.947 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:49:53.947 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:49:53.947 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:49:53.947 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:49:53.960 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:49:53.960 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:49:53.961 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:49:53.962 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:49:53.963 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:49:53.964 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:49:53.964 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:49:53.964 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:49:54.007 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:49:54.008 DEBUG:teuthology.parallel:result is None 2024-09-10T21:49:54.008 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:49:54.008 DEBUG:teuthology.parallel:result is None 2024-09-10T21:49:54.010 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:49:54.010 DEBUG:teuthology.parallel:result is None 2024-09-10T21:49:54.011 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:49:54.011 DEBUG:teuthology.parallel:result is None 2024-09-10T21:49:54.012 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:49:54.136 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2024-09-10T21:49:54.137 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2024-09-10T21:49:54.138 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-09-10T21:49:54.138 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-09-10T21:49:54.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.180+0000 7f95cb000640 1 -- 172.21.15.7:0/1645800115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95c40ccc30 msgr2=0x7f95c41649c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:54.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.180+0000 7f95cb000640 1 --2- 172.21.15.7:0/1645800115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95c40ccc30 0x7f95c41649c0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f95bc005fd0 tx=0x7f95bc02a8b0 comp rx=0 tx=0).stop 2024-09-10T21:49:54.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.180+0000 7f95cb000640 1 -- 172.21.15.7:0/1645800115 shutdown_connections 2024-09-10T21:49:54.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.180+0000 7f95cb000640 1 --2- 172.21.15.7:0/1645800115 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95c40ccc30 0x7f95c41649c0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:54.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.180+0000 7f95cb000640 1 -- 172.21.15.7:0/1645800115 >> 172.21.15.7:0/1645800115 conn(0x7f95c40b1660 msgr2=0x7f95c40b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:54.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.180+0000 7f95cb000640 1 -- 172.21.15.7:0/1645800115 shutdown_connections 2024-09-10T21:49:54.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.180+0000 7f95cb000640 1 -- 172.21.15.7:0/1645800115 wait complete. 2024-09-10T21:49:54.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.182+0000 7f95cb000640 1 Processor -- start 2024-09-10T21:49:54.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.182+0000 7f95cb000640 1 -- start start 2024-09-10T21:49:54.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.183+0000 7f95cb000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95c40ccc30 0x7f95c41571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:54.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.183+0000 7f95cb000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95c40cf650 0x7f95c4165550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:54.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.183+0000 7f95cb000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95c41579b0 0x7f95c41688c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:54.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.183+0000 7f95cb000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f95c4164f00 con 0x7f95c41579b0 2024-09-10T21:49:54.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.183+0000 7f95cb000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f95c4070810 con 0x7f95c40cf650 2024-09-10T21:49:54.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.183+0000 7f95cb000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f95c4165080 con 0x7f95c40ccc30 2024-09-10T21:49:54.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.183+0000 7f95c9000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95c41579b0 0x7f95c41688c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:54.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.183+0000 7f95c9000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95c41579b0 0x7f95c41688c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34056/0 (socket says 172.21.15.7:34056) 2024-09-10T21:49:54.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.183+0000 7f95c9000640 1 -- 172.21.15.7:0/3451316580 learned_addr learned my addr 172.21.15.7:0/3451316580 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:54.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.183+0000 7f95c3e00640 1 --2- 172.21.15.7:0/3451316580 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95c40ccc30 0x7f95c41571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:54.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.183+0000 7f95bb400640 1 --2- 172.21.15.7:0/3451316580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95c40cf650 0x7f95c4165550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.184+0000 7f95c9000640 1 -- 172.21.15.7:0/3451316580 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95c40ccc30 msgr2=0x7f95c41571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.184+0000 7f95c9000640 1 --2- 172.21.15.7:0/3451316580 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95c40ccc30 0x7f95c41571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.184+0000 7f95c9000640 1 -- 172.21.15.7:0/3451316580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95c40cf650 msgr2=0x7f95c4165550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.184+0000 7f95c9000640 1 --2- 172.21.15.7:0/3451316580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95c40cf650 0x7f95c4165550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.184+0000 7f95c9000640 1 -- 172.21.15.7:0/3451316580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c41681c0 con 0x7f95c41579b0 2024-09-10T21:49:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.184+0000 7f95c9000640 1 --2- 172.21.15.7:0/3451316580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95c41579b0 0x7f95c41688c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f95bc006010 tx=0x7f95bc003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.184+0000 7f95c0c00640 1 -- 172.21.15.7:0/3451316580 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc040070 con 0x7f95c41579b0 2024-09-10T21:49:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.184+0000 7f95c0c00640 1 -- 172.21.15.7:0/3451316580 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95bc004130 con 0x7f95c41579b0 2024-09-10T21:49:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.184+0000 7f95cb000640 1 -- 172.21.15.7:0/3451316580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c41684a0 con 0x7f95c41579b0 2024-09-10T21:49:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.184+0000 7f95c0c00640 1 -- 172.21.15.7:0/3451316580 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95bc004560 con 0x7f95c41579b0 2024-09-10T21:49:54.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.184+0000 7f95cb000640 1 -- 172.21.15.7:0/3451316580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c40b2fb0 con 0x7f95c41579b0 2024-09-10T21:49:54.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.186+0000 7f95cb000640 1 -- 172.21.15.7:0/3451316580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c40ca140 con 0x7f95c41579b0 2024-09-10T21:49:54.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.186+0000 7f95c0c00640 1 -- 172.21.15.7:0/3451316580 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f95bc03c420 con 0x7f95c41579b0 2024-09-10T21:49:54.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.187+0000 7f95c0c00640 1 --2- 172.21.15.7:0/3451316580 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f95a0077f70 0x7f95a007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:54.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.187+0000 7f95c0c00640 1 -- 172.21.15.7:0/3451316580 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 5115+0+0 (secure 0 0 0) 0x7f95bc0bb430 con 0x7f95c41579b0 2024-09-10T21:49:54.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.188+0000 7f95c3e00640 1 --2- 172.21.15.7:0/3451316580 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f95a0077f70 0x7f95a007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:54.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.188+0000 7f95c3e00640 1 --2- 172.21.15.7:0/3451316580 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f95a0077f70 0x7f95a007a430 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f95c406f6f0 tx=0x7f95b4007eb0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:54.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.191+0000 7f95c0c00640 1 -- 172.21.15.7:0/3451316580 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f95c40ca140 con 0x7f95c41579b0 2024-09-10T21:49:54.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:54.405+0000 7f95cb000640 1 -- 172.21.15.7:0/3451316580 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f95c40b3180 con 0x7f95c41579b0 2024-09-10T21:49:55.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.404+0000 7f95c0c00640 1 -- 172.21.15.7:0/3451316580 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v97) ==== 97+0+0 (secure 0 0 0) 0x7f95bc083a90 con 0x7f95c41579b0 2024-09-10T21:49:55.405 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-09-10T21:49:55.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.407+0000 7f95cb000640 1 -- 172.21.15.7:0/3451316580 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f95a0077f70 msgr2=0x7f95a007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:55.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.407+0000 7f95cb000640 1 --2- 172.21.15.7:0/3451316580 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f95a0077f70 0x7f95a007a430 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f95c406f6f0 tx=0x7f95b4007eb0 comp rx=0 tx=0).stop 2024-09-10T21:49:55.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.407+0000 7f95cb000640 1 -- 172.21.15.7:0/3451316580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95c41579b0 msgr2=0x7f95c41688c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.407+0000 7f95cb000640 1 --2- 172.21.15.7:0/3451316580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95c41579b0 0x7f95c41688c0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f95bc006010 tx=0x7f95bc003970 comp rx=0 tx=0).stop 2024-09-10T21:49:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.407+0000 7f95cb000640 1 -- 172.21.15.7:0/3451316580 shutdown_connections 2024-09-10T21:49:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.407+0000 7f95cb000640 1 --2- 172.21.15.7:0/3451316580 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f95a0077f70 0x7f95a007a430 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.407+0000 7f95cb000640 1 --2- 172.21.15.7:0/3451316580 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f95c40cf650 0x7f95c4165550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.407+0000 7f95cb000640 1 --2- 172.21.15.7:0/3451316580 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f95c41579b0 0x7f95c41688c0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.407+0000 7f95cb000640 1 --2- 172.21.15.7:0/3451316580 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f95c40ccc30 0x7f95c41571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.407+0000 7f95cb000640 1 -- 172.21.15.7:0/3451316580 >> 172.21.15.7:0/3451316580 conn(0x7f95c40b1660 msgr2=0x7f95c4159440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.407+0000 7f95cb000640 1 -- 172.21.15.7:0/3451316580 shutdown_connections 2024-09-10T21:49:55.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.407+0000 7f95cb000640 1 -- 172.21.15.7:0/3451316580 wait complete. 2024-09-10T21:49:55.427 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:49:55.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.592+0000 7ff564200640 1 -- 172.21.15.7:0/2982231511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff55c15a4a0 msgr2=0x7ff55c0cb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:55.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.592+0000 7ff564200640 1 --2- 172.21.15.7:0/2982231511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff55c15a4a0 0x7ff55c0cb0c0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7ff55c074100 tx=0x7ff54802a5f0 comp rx=0 tx=0).stop 2024-09-10T21:49:55.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.592+0000 7ff564200640 1 -- 172.21.15.7:0/2982231511 shutdown_connections 2024-09-10T21:49:55.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.592+0000 7ff564200640 1 --2- 172.21.15.7:0/2982231511 >> v2:172.21.15.133:3300/0 conn(0x7ff55c0ca1d0 0x7ff55c06e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:55.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.592+0000 7ff564200640 1 --2- 172.21.15.7:0/2982231511 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff55c15a4a0 0x7ff55c0cb0c0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:55.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.592+0000 7ff564200640 1 -- 172.21.15.7:0/2982231511 >> 172.21.15.7:0/2982231511 conn(0x7ff55c0b16d0 msgr2=0x7ff55c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.592+0000 7ff564200640 1 -- 172.21.15.7:0/2982231511 shutdown_connections 2024-09-10T21:49:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.592+0000 7ff564200640 1 -- 172.21.15.7:0/2982231511 wait complete. 2024-09-10T21:49:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.593+0000 7ff564200640 1 Processor -- start 2024-09-10T21:49:55.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.594+0000 7ff564200640 1 -- start start 2024-09-10T21:49:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.594+0000 7ff564200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff55c0ca1d0 0x7ff55c1562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.594+0000 7ff564200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff55c15a0c0 0x7ff55c0b44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.594+0000 7ff564200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff55c15a4a0 0x7ff55c152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.594+0000 7ff564200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c0cc600 con 0x7ff55c0ca1d0 2024-09-10T21:49:55.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.594+0000 7ff564200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff55c06ee00 con 0x7ff55c15a0c0 2024-09-10T21:49:55.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.594+0000 7ff564200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ff55c0cc780 con 0x7ff55c15a4a0 2024-09-10T21:49:55.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.595+0000 7ff561800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff55c0ca1d0 0x7ff55c1562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:55.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.595+0000 7ff561800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff55c0ca1d0 0x7ff55c1562b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34078/0 (socket says 172.21.15.7:34078) 2024-09-10T21:49:55.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.595+0000 7ff561800640 1 -- 172.21.15.7:0/896955036 learned_addr learned my addr 172.21.15.7:0/896955036 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:55.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.595+0000 7ff560e00640 1 --2- 172.21.15.7:0/896955036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff55c15a0c0 0x7ff55c0b44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:55.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.595+0000 7ff562200640 1 --2- 172.21.15.7:0/896955036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff55c15a4a0 0x7ff55c152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:55.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.595+0000 7ff561800640 1 -- 172.21.15.7:0/896955036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff55c15a4a0 msgr2=0x7ff55c152860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:55.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.595+0000 7ff561800640 1 --2- 172.21.15.7:0/896955036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff55c15a4a0 0x7ff55c152860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:55.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.595+0000 7ff561800640 1 -- 172.21.15.7:0/896955036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff55c15a0c0 msgr2=0x7ff55c0b44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:55.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.595+0000 7ff561800640 1 --2- 172.21.15.7:0/896955036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff55c15a0c0 0x7ff55c0b44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:55.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.595+0000 7ff561800640 1 -- 172.21.15.7:0/896955036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff55c0c36b0 con 0x7ff55c0ca1d0 2024-09-10T21:49:55.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.596+0000 7ff561800640 1 --2- 172.21.15.7:0/896955036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff55c0ca1d0 0x7ff55c1562b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7ff55000c500 tx=0x7ff55000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:55.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.596+0000 7ff559600640 1 -- 172.21.15.7:0/896955036 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550018070 con 0x7ff55c0ca1d0 2024-09-10T21:49:55.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.596+0000 7ff559600640 1 -- 172.21.15.7:0/896955036 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5500035f0 con 0x7ff55c0ca1d0 2024-09-10T21:49:55.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.596+0000 7ff559600640 1 -- 172.21.15.7:0/896955036 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff550003ac0 con 0x7ff55c0ca1d0 2024-09-10T21:49:55.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.596+0000 7ff564200640 1 -- 172.21.15.7:0/896955036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55c0c3990 con 0x7ff55c0ca1d0 2024-09-10T21:49:55.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.596+0000 7ff564200640 1 -- 172.21.15.7:0/896955036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff55c0c6060 con 0x7ff55c0ca1d0 2024-09-10T21:49:55.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.597+0000 7ff564200640 1 -- 172.21.15.7:0/896955036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff55c0ccba0 con 0x7ff55c0ca1d0 2024-09-10T21:49:55.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.598+0000 7ff559600640 1 -- 172.21.15.7:0/896955036 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff550003c60 con 0x7ff55c0ca1d0 2024-09-10T21:49:55.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.600+0000 7ff559600640 1 --2- 172.21.15.7:0/896955036 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff53c077f70 0x7ff53c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:55.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.600+0000 7ff559600640 1 -- 172.21.15.7:0/896955036 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4857+0+0 (secure 0 0 0) 0x7ff550099080 con 0x7ff55c0ca1d0 2024-09-10T21:49:55.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.600+0000 7ff560e00640 1 --2- 172.21.15.7:0/896955036 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff53c077f70 0x7ff53c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:55.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.601+0000 7ff560e00640 1 --2- 172.21.15.7:0/896955036 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff53c077f70 0x7ff53c07a430 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff548033000 tx=0x7ff548004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:55.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.604+0000 7ff559600640 1 -- 172.21.15.7:0/896955036 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff5500617f0 con 0x7ff55c0ca1d0 2024-09-10T21:49:55.871 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.871+0000 7ff564200640 1 -- 172.21.15.7:0/896955036 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7ff55c0cd020 con 0x7ff55c0ca1d0 2024-09-10T21:49:55.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.871+0000 7ff559600640 1 -- 172.21.15.7:0/896955036 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v37) ==== 80+0+5304 (secure 0 0 0) 0x7ff550066220 con 0x7ff55c0ca1d0 2024-09-10T21:49:55.872 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:55.872 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:49:55.873 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-09-10T21:49:55.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:55.873 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-09-10T21:49:55.873 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmgRMuLBAA88QT12AXv2qf11IxU20Xbg==", 2024-09-10T21:49:55.873 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:49:55.873 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:49:55.873 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:49:55.873 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:49:55.873 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:49:55.873 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.873 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:55.873 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:55.873 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-09-10T21:49:55.873 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmRUM2BBAAwcQM6z+hjZI4ZWBlVrYMwQ==", 2024-09-10T21:49:55.874 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:49:55.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:49:55.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:49:55.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:49:55.874 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:49:55.874 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.874 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:55.874 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:55.874 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-09-10T21:49:55.874 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmnq31NRAAj1eJUAIS986idxppfeXEAA==", 2024-09-10T21:49:55.874 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:49:55.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:49:55.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:49:55.874 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:49:55.875 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:49:55.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:55.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:55.875 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-09-10T21:49:55.875 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmy43uDRAAzdKjLxZMGROuDABrsJ0zgQ==", 2024-09-10T21:49:55.875 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:49:55.875 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:49:55.875 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:49:55.875 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:49:55.875 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:49:55.875 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.875 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:55.875 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:55.876 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-09-10T21:49:55.876 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDwvOBmzfgtOxAAk6K5fU0Uar4VW9nUUIuNdw==", 2024-09-10T21:49:55.876 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:49:55.876 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:49:55.876 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:49:55.876 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:49:55.876 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.876 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:55.876 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:55.876 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-09-10T21:49:55.876 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD0vOBmHcRzChAA3z+CaGgu5NkyUmjpqzmKjw==", 2024-09-10T21:49:55.876 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:49:55.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:49:55.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:49:55.877 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:49:55.877 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:55.877 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:55.877 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-09-10T21:49:55.877 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD3vOBm9sArFBAAGQlyiw6qEcyumEBICWEBKA==", 2024-09-10T21:49:55.877 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:49:55.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:49:55.877 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:49:55.877 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:49:55.877 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.877 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:55.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:55.878 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-09-10T21:49:55.878 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD6vOBmkO3rHxAAW0zPydYZRGUKjGh3yIyPCA==", 2024-09-10T21:49:55.878 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:49:55.878 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:49:55.878 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:49:55.878 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:49:55.878 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.878 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:55.878 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:55.878 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-09-10T21:49:55.878 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAAveBmlPABMxAAeazVOzc1DhsVAwoHn87QIA==", 2024-09-10T21:49:55.878 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:49:55.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:49:55.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:49:55.879 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:49:55.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:55.879 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:55.879 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-09-10T21:49:55.879 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAEveBmL9qzARAApu8nWIY9HVsDl4yUnfhH9w==", 2024-09-10T21:49:55.879 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:49:55.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:49:55.879 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:49:55.879 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:49:55.879 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.879 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:55.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:55.880 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-09-10T21:49:55.880 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAHveBmJ4ofDRAAEiCFN8nGzVSR3u2gKoT+2Q==", 2024-09-10T21:49:55.880 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:49:55.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:49:55.880 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:49:55.880 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:49:55.880 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.880 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:55.880 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:55.880 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-09-10T21:49:55.880 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAKveBmoUGEGBAAhXmkUaniuAQrBiYPWr8Tog==", 2024-09-10T21:49:55.880 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:49:55.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:49:55.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:49:55.881 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:49:55.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.881 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:55.881 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:55.881 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-09-10T21:49:55.881 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBm0aqcFhAAtid2bnzjcpaYAbU7g0JcOA==", 2024-09-10T21:49:55.881 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:49:55.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:49:55.881 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-09-10T21:49:55.881 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:49:55.881 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:55.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:55.882 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-09-10T21:49:55.882 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmqf+MIxAAib5hVnbBsw/bJPqsQszJwg==", 2024-09-10T21:49:55.882 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:49:55.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:49:55.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:49:55.882 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:49:55.882 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:49:55.882 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.882 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:55.882 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:55.882 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-09-10T21:49:55.883 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmybQhHRAAZUwqnIsihEWnHZWQTYr+uQ==", 2024-09-10T21:49:55.883 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:49:55.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:49:55.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:49:55.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:49:55.883 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:49:55.883 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.883 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:55.883 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:55.883 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-09-10T21:49:55.883 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmDbrrKRAAUxb3EBfKIJ4FAGHW3QIsLg==", 2024-09-10T21:49:55.883 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:49:55.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:49:55.883 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:49:55.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:49:55.884 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:49:55.884 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.884 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:55.884 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:55.884 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-09-10T21:49:55.884 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDtvOBmF+eEOBAA3icvNqI0nWDShE49DZ3bvg==", 2024-09-10T21:49:55.884 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:49:55.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:49:55.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:49:55.884 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:49:55.884 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:49:55.885 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:55.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:55.885 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-09-10T21:49:55.885 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmsu6OJRAAhdgDvpeHY/zDKUeUOv6xxA==", 2024-09-10T21:49:55.885 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:49:55.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:49:55.885 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-10T21:49:55.885 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:49:55.885 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.885 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:49:55.885 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:55.885 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-09-10T21:49:55.885 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmKBHLHhAA0W5My19tIhN1v2qev4cBTA==", 2024-09-10T21:49:55.886 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:49:55.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:49:55.886 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-10T21:49:55.886 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:49:55.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.886 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:55.886 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T21:49:55.886 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:49:55.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.873+0000 7ff564200640 1 -- 172.21.15.7:0/896955036 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff53c077f70 msgr2=0x7ff53c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:55.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.873+0000 7ff564200640 1 --2- 172.21.15.7:0/896955036 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff53c077f70 0x7ff53c07a430 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7ff548033000 tx=0x7ff548004730 comp rx=0 tx=0).stop 2024-09-10T21:49:55.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.874+0000 7ff564200640 1 -- 172.21.15.7:0/896955036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff55c0ca1d0 msgr2=0x7ff55c1562b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:55.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.874+0000 7ff564200640 1 --2- 172.21.15.7:0/896955036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff55c0ca1d0 0x7ff55c1562b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7ff55000c500 tx=0x7ff55000c9c0 comp rx=0 tx=0).stop 2024-09-10T21:49:55.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.874+0000 7ff564200640 1 -- 172.21.15.7:0/896955036 shutdown_connections 2024-09-10T21:49:55.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.874+0000 7ff564200640 1 --2- 172.21.15.7:0/896955036 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff53c077f70 0x7ff53c07a430 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:55.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.874+0000 7ff564200640 1 --2- 172.21.15.7:0/896955036 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff55c15a0c0 0x7ff55c0b44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:55.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.874+0000 7ff564200640 1 --2- 172.21.15.7:0/896955036 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff55c15a4a0 0x7ff55c152860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:55.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.874+0000 7ff564200640 1 --2- 172.21.15.7:0/896955036 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff55c0ca1d0 0x7ff55c1562b0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:55.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.874+0000 7ff564200640 1 -- 172.21.15.7:0/896955036 >> 172.21.15.7:0/896955036 conn(0x7ff55c0b16d0 msgr2=0x7ff55c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:55.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.874+0000 7ff564200640 1 -- 172.21.15.7:0/896955036 shutdown_connections 2024-09-10T21:49:55.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:55.874+0000 7ff564200640 1 -- 172.21.15.7:0/896955036 wait complete. 2024-09-10T21:49:55.890 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:49:56.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.051+0000 7fe055e00640 1 -- 172.21.15.7:0/286364324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0500d2690 msgr2=0x7fe05015f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:56.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.051+0000 7fe055e00640 1 --2- 172.21.15.7:0/286364324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0500d2690 0x7fe05015f910 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fe050069850 tx=0x7fe03c02a560 comp rx=0 tx=0).stop 2024-09-10T21:49:56.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.051+0000 7fe055e00640 1 -- 172.21.15.7:0/286364324 shutdown_connections 2024-09-10T21:49:56.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.051+0000 7fe055e00640 1 --2- 172.21.15.7:0/286364324 >> v2:172.21.15.133:3300/0 conn(0x7fe05015f480 0x7fe05015fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:56.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.051+0000 7fe055e00640 1 --2- 172.21.15.7:0/286364324 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0500d2690 0x7fe05015f910 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:56.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.051+0000 7fe055e00640 1 --2- 172.21.15.7:0/286364324 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05015f0a0 0x7fe0500b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:56.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.051+0000 7fe055e00640 1 -- 172.21.15.7:0/286364324 >> 172.21.15.7:0/286364324 conn(0x7fe0500d1ba0 msgr2=0x7fe0500d1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:56.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.051+0000 7fe055e00640 1 -- 172.21.15.7:0/286364324 shutdown_connections 2024-09-10T21:49:56.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.051+0000 7fe055e00640 1 -- 172.21.15.7:0/286364324 wait complete. 2024-09-10T21:49:56.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.052+0000 7fe055e00640 1 Processor -- start 2024-09-10T21:49:56.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.052+0000 7fe055e00640 1 -- start start 2024-09-10T21:49:56.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.053+0000 7fe055e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0500d2690 0x7fe0500c4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:56.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.053+0000 7fe055e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe05015f0a0 0x7fe0500cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:56.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.053+0000 7fe055e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05015f480 0x7fe0500cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:56.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.053+0000 7fe055e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe0500ae970 con 0x7fe05015f0a0 2024-09-10T21:49:56.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.053+0000 7fe055e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe0500b1000 con 0x7fe0500d2690 2024-09-10T21:49:56.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.053+0000 7fe055e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe0500aeaf0 con 0x7fe05015f480 2024-09-10T21:49:56.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.054+0000 7fe04fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe05015f0a0 0x7fe0500cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:56.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.054+0000 7fe04fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe05015f0a0 0x7fe0500cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34110/0 (socket says 172.21.15.7:34110) 2024-09-10T21:49:56.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.054+0000 7fe04fe00640 1 -- 172.21.15.7:0/1075320205 learned_addr learned my addr 172.21.15.7:0/1075320205 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:56.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.054+0000 7fe054a00640 1 --2- 172.21.15.7:0/1075320205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0500d2690 0x7fe0500c4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:56.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.054+0000 7fe055400640 1 --2- 172.21.15.7:0/1075320205 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05015f480 0x7fe0500cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:56.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.054+0000 7fe04fe00640 1 -- 172.21.15.7:0/1075320205 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05015f480 msgr2=0x7fe0500cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:56.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.054+0000 7fe04fe00640 1 --2- 172.21.15.7:0/1075320205 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05015f480 0x7fe0500cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:56.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.054+0000 7fe04fe00640 1 -- 172.21.15.7:0/1075320205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0500d2690 msgr2=0x7fe0500c4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:56.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.054+0000 7fe04fe00640 1 --2- 172.21.15.7:0/1075320205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0500d2690 0x7fe0500c4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:56.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.054+0000 7fe04fe00640 1 -- 172.21.15.7:0/1075320205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe05006faa0 con 0x7fe05015f0a0 2024-09-10T21:49:56.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.054+0000 7fe04fe00640 1 --2- 172.21.15.7:0/1075320205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe05015f0a0 0x7fe0500cb170 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009850 tx=0x7fe03c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:56.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.055+0000 7fe04cc00640 1 -- 172.21.15.7:0/1075320205 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c03f070 con 0x7fe05015f0a0 2024-09-10T21:49:56.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.055+0000 7fe04cc00640 1 -- 172.21.15.7:0/1075320205 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe03c0047a0 con 0x7fe05015f0a0 2024-09-10T21:49:56.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.055+0000 7fe04cc00640 1 -- 172.21.15.7:0/1075320205 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe03c035030 con 0x7fe05015f0a0 2024-09-10T21:49:56.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.055+0000 7fe055e00640 1 -- 172.21.15.7:0/1075320205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0500c9560 con 0x7fe05015f0a0 2024-09-10T21:49:56.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.055+0000 7fe055e00640 1 -- 172.21.15.7:0/1075320205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0500c9b20 con 0x7fe05015f0a0 2024-09-10T21:49:56.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.056+0000 7fe055e00640 1 -- 172.21.15.7:0/1075320205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0500d1420 con 0x7fe05015f0a0 2024-09-10T21:49:56.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.057+0000 7fe04cc00640 1 -- 172.21.15.7:0/1075320205 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe03c004bb0 con 0x7fe05015f0a0 2024-09-10T21:49:56.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.058+0000 7fe04cc00640 1 --2- 172.21.15.7:0/1075320205 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe030077fa0 0x7fe03007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:56.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.058+0000 7fe04cc00640 1 -- 172.21.15.7:0/1075320205 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4857+0+0 (secure 0 0 0) 0x7fe03c0bab20 con 0x7fe05015f0a0 2024-09-10T21:49:56.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.058+0000 7fe054a00640 1 --2- 172.21.15.7:0/1075320205 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe030077fa0 0x7fe03007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:56.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.059+0000 7fe054a00640 1 --2- 172.21.15.7:0/1075320205 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe030077fa0 0x7fe03007a460 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe044005fd0 tx=0x7fe04400b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:56.063 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.062+0000 7fe04cc00640 1 -- 172.21.15.7:0/1075320205 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe0500d1420 con 0x7fe05015f0a0 2024-09-10T21:49:56.260 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.259+0000 7fe055e00640 1 -- 172.21.15.7:0/1075320205 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fe0500b2290 con 0x7fe05015f0a0 2024-09-10T21:49:56.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.260+0000 7fe04cc00640 1 -- 172.21.15.7:0/1075320205 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v97) ==== 84+0+65 (secure 0 0 0) 0x7fe03c083280 con 0x7fe05015f0a0 2024-09-10T21:49:56.261 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:56.261 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-09-10T21:49:56.261 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:49:56.261 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-09-10T21:49:56.261 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-09-10T21:49:56.262 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:49:56.262 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-09-10T21:49:56.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.263+0000 7fe055e00640 1 -- 172.21.15.7:0/1075320205 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe030077fa0 msgr2=0x7fe03007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:56.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.263+0000 7fe055e00640 1 --2- 172.21.15.7:0/1075320205 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe030077fa0 0x7fe03007a460 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fe044005fd0 tx=0x7fe04400b2e0 comp rx=0 tx=0).stop 2024-09-10T21:49:56.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.263+0000 7fe055e00640 1 -- 172.21.15.7:0/1075320205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe05015f0a0 msgr2=0x7fe0500cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:56.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.263+0000 7fe055e00640 1 --2- 172.21.15.7:0/1075320205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe05015f0a0 0x7fe0500cb170 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fe03c009850 tx=0x7fe03c003910 comp rx=0 tx=0).stop 2024-09-10T21:49:56.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.263+0000 7fe055e00640 1 -- 172.21.15.7:0/1075320205 shutdown_connections 2024-09-10T21:49:56.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.263+0000 7fe055e00640 1 --2- 172.21.15.7:0/1075320205 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe030077fa0 0x7fe03007a460 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:56.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.263+0000 7fe055e00640 1 --2- 172.21.15.7:0/1075320205 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe05015f480 0x7fe0500cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:56.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.263+0000 7fe055e00640 1 --2- 172.21.15.7:0/1075320205 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe05015f0a0 0x7fe0500cb170 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:56.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.263+0000 7fe055e00640 1 --2- 172.21.15.7:0/1075320205 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe0500d2690 0x7fe0500c4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:56.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.263+0000 7fe055e00640 1 -- 172.21.15.7:0/1075320205 >> 172.21.15.7:0/1075320205 conn(0x7fe0500d1ba0 msgr2=0x7fe0500714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:56.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.263+0000 7fe055e00640 1 -- 172.21.15.7:0/1075320205 shutdown_connections 2024-09-10T21:49:56.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.263+0000 7fe055e00640 1 -- 172.21.15.7:0/1075320205 wait complete. 2024-09-10T21:49:56.279 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:49:56.279 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:49:56.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.443+0000 7f35b1200640 1 -- 172.21.15.7:0/3927412098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac15bce0 msgr2=0x7f35ac06cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.443+0000 7f35b1200640 1 --2- 172.21.15.7:0/3927412098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac15bce0 0x7f35ac06cb20 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f35a4005fd0 tx=0x7f35a402aa60 comp rx=0 tx=0).stop 2024-09-10T21:49:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.443+0000 7f35b1200640 1 -- 172.21.15.7:0/3927412098 shutdown_connections 2024-09-10T21:49:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.443+0000 7f35b1200640 1 --2- 172.21.15.7:0/3927412098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac15bce0 0x7f35ac06cb20 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.443+0000 7f35b1200640 1 --2- 172.21.15.7:0/3927412098 >> v2:172.21.15.133:3300/0 conn(0x7f35ac15b900 0x7f35ac070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.443+0000 7f35b1200640 1 -- 172.21.15.7:0/3927412098 >> 172.21.15.7:0/3927412098 conn(0x7f35ac0b1c10 msgr2=0x7f35ac0b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.443+0000 7f35b1200640 1 -- 172.21.15.7:0/3927412098 shutdown_connections 2024-09-10T21:49:56.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.443+0000 7f35b1200640 1 -- 172.21.15.7:0/3927412098 wait complete. 2024-09-10T21:49:56.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.445+0000 7f35b1200640 1 Processor -- start 2024-09-10T21:49:56.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.445+0000 7f35b1200640 1 -- start start 2024-09-10T21:49:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.446+0000 7f35b1200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35ac0ace80 0x7f35ac15f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.446+0000 7f35b1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac15b900 0x7f35ac1484c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.446+0000 7f35b1200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35ac15bce0 0x7f35ac1f4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:56.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.446+0000 7f35b1200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac0aed00 con 0x7f35ac15b900 2024-09-10T21:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.446+0000 7f35b1200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac0aeb80 con 0x7f35ac15bce0 2024-09-10T21:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.446+0000 7f35b1200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f35ac0aee80 con 0x7f35ac0ace80 2024-09-10T21:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.446+0000 7f35a1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac15b900 0x7f35ac1484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.446+0000 7f35a1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac15b900 0x7f35ac1484c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34130/0 (socket says 172.21.15.7:34130) 2024-09-10T21:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.446+0000 7f35a1400640 1 -- 172.21.15.7:0/838169032 learned_addr learned my addr 172.21.15.7:0/838169032 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.446+0000 7f35aa800640 1 --2- 172.21.15.7:0/838169032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35ac15bce0 0x7f35ac1f4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.446+0000 7f35a9e00640 1 --2- 172.21.15.7:0/838169032 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35ac0ace80 0x7f35ac15f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.447+0000 7f35a1400640 1 -- 172.21.15.7:0/838169032 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35ac0ace80 msgr2=0x7f35ac15f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.447+0000 7f35a1400640 1 --2- 172.21.15.7:0/838169032 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35ac0ace80 0x7f35ac15f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.447+0000 7f35a1400640 1 -- 172.21.15.7:0/838169032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35ac15bce0 msgr2=0x7f35ac1f4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.447+0000 7f35a1400640 1 --2- 172.21.15.7:0/838169032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35ac15bce0 0x7f35ac1f4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.447+0000 7f35a1400640 1 -- 172.21.15.7:0/838169032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac148a00 con 0x7f35ac15b900 2024-09-10T21:49:56.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.447+0000 7f35a9e00640 1 --2- 172.21.15.7:0/838169032 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35ac0ace80 0x7f35ac15f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:49:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.447+0000 7f35a1400640 1 --2- 172.21.15.7:0/838169032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac15b900 0x7f35ac1484c0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f359400c450 tx=0x7f359400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:56.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.447+0000 7f35a2a00640 1 -- 172.21.15.7:0/838169032 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35940052d0 con 0x7f35ac15b900 2024-09-10T21:49:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.447+0000 7f35a2a00640 1 -- 172.21.15.7:0/838169032 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3594005470 con 0x7f35ac15b900 2024-09-10T21:49:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.447+0000 7f35a2a00640 1 -- 172.21.15.7:0/838169032 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594012870 con 0x7f35ac15b900 2024-09-10T21:49:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.447+0000 7f35b1200640 1 -- 172.21.15.7:0/838169032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ac09c500 con 0x7f35ac15b900 2024-09-10T21:49:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.447+0000 7f35b1200640 1 -- 172.21.15.7:0/838169032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ac09cac0 con 0x7f35ac15b900 2024-09-10T21:49:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.448+0000 7f35b1200640 1 -- 172.21.15.7:0/838169032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ac0af330 con 0x7f35ac15b900 2024-09-10T21:49:56.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.449+0000 7f35a2a00640 1 -- 172.21.15.7:0/838169032 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f35940035f0 con 0x7f35ac15b900 2024-09-10T21:49:56.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.450+0000 7f35a2a00640 1 --2- 172.21.15.7:0/838169032 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3588077f70 0x7f358807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:56.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.450+0000 7f35a2a00640 1 -- 172.21.15.7:0/838169032 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 4857+0+0 (secure 0 0 0) 0x7f3594099510 con 0x7f35ac15b900 2024-09-10T21:49:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.451+0000 7f35a9e00640 1 --2- 172.21.15.7:0/838169032 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3588077f70 0x7f358807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:56.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.452+0000 7f35a9e00640 1 --2- 172.21.15.7:0/838169032 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3588077f70 0x7f358807a430 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f359c0058c0 tx=0x7f359c005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:56.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.455+0000 7f35a2a00640 1 -- 172.21.15.7:0/838169032 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3594062350 con 0x7f35ac15b900 2024-09-10T21:49:56.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:56.647+0000 7f35b1200640 1 -- 172.21.15.7:0/838169032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f35ac06ebe0 con 0x7f35ac15b900 2024-09-10T21:49:57.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.413+0000 7f35a2a00640 1 -- 172.21.15.7:0/838169032 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v98) ==== 138+0+0 (secure 0 0 0) 0x7f3594061c50 con 0x7f35ac15b900 2024-09-10T21:49:57.415 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-09-10T21:49:57.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.417+0000 7f35b1200640 1 -- 172.21.15.7:0/838169032 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3588077f70 msgr2=0x7f358807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:57.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.417+0000 7f35b1200640 1 --2- 172.21.15.7:0/838169032 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3588077f70 0x7f358807a430 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f359c0058c0 tx=0x7f359c005850 comp rx=0 tx=0).stop 2024-09-10T21:49:57.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.417+0000 7f35b1200640 1 -- 172.21.15.7:0/838169032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac15b900 msgr2=0x7f35ac1484c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:57.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.417+0000 7f35b1200640 1 --2- 172.21.15.7:0/838169032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac15b900 0x7f35ac1484c0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f359400c450 tx=0x7f359400c910 comp rx=0 tx=0).stop 2024-09-10T21:49:57.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.417+0000 7f35b1200640 1 -- 172.21.15.7:0/838169032 shutdown_connections 2024-09-10T21:49:57.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.417+0000 7f35b1200640 1 --2- 172.21.15.7:0/838169032 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3588077f70 0x7f358807a430 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:57.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.417+0000 7f35b1200640 1 --2- 172.21.15.7:0/838169032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f35ac15bce0 0x7f35ac1f4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:57.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.417+0000 7f35b1200640 1 --2- 172.21.15.7:0/838169032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35ac15b900 0x7f35ac1484c0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:57.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.417+0000 7f35b1200640 1 --2- 172.21.15.7:0/838169032 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f35ac0ace80 0x7f35ac15f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:57.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.417+0000 7f35b1200640 1 -- 172.21.15.7:0/838169032 >> 172.21.15.7:0/838169032 conn(0x7f35ac0b1c10 msgr2=0x7f35ac14ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:57.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.417+0000 7f35b1200640 1 -- 172.21.15.7:0/838169032 shutdown_connections 2024-09-10T21:49:57.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.417+0000 7f35b1200640 1 -- 172.21.15.7:0/838169032 wait complete. 2024-09-10T21:49:57.442 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:49:57.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.612+0000 7f2abb800640 1 -- 172.21.15.7:0/290292661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ab40af100 msgr2=0x7f2ab40cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.612+0000 7f2abb800640 1 --2- 172.21.15.7:0/290292661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ab40af100 0x7f2ab40cca70 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f2aac005fd0 tx=0x7f2aac02aaa0 comp rx=0 tx=0).stop 2024-09-10T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.612+0000 7f2abb800640 1 -- 172.21.15.7:0/290292661 shutdown_connections 2024-09-10T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.612+0000 7f2abb800640 1 --2- 172.21.15.7:0/290292661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ab40af100 0x7f2ab40cca70 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.612+0000 7f2abb800640 1 --2- 172.21.15.7:0/290292661 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ab4161ab0 0x7f2ab40cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.612+0000 7f2abb800640 1 -- 172.21.15.7:0/290292661 >> 172.21.15.7:0/290292661 conn(0x7f2ab40b16d0 msgr2=0x7f2ab40b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.613+0000 7f2abb800640 1 -- 172.21.15.7:0/290292661 shutdown_connections 2024-09-10T21:49:57.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.613+0000 7f2abb800640 1 -- 172.21.15.7:0/290292661 wait complete. 2024-09-10T21:49:57.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.614+0000 7f2abb800640 1 Processor -- start 2024-09-10T21:49:57.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.614+0000 7f2abb800640 1 -- start start 2024-09-10T21:49:57.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.615+0000 7f2abb800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ab40af100 0x7f2ab41f5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:57.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.615+0000 7f2abb800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ab415ba00 0x7f2ab40c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:57.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.615+0000 7f2abb800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ab40c6f80 0x7f2ab41f8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:57.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.615+0000 7f2abb800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab415ea50 con 0x7f2ab40af100 2024-09-10T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.615+0000 7f2abb800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab415e8d0 con 0x7f2ab40c6f80 2024-09-10T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.615+0000 7f2abb800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f2ab415ebd0 con 0x7f2ab415ba00 2024-09-10T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.615+0000 7f2ab8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ab40af100 0x7f2ab41f5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.615+0000 7f2ab8e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ab40af100 0x7f2ab41f5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34144/0 (socket says 172.21.15.7:34144) 2024-09-10T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.615+0000 7f2ab8e00640 1 -- 172.21.15.7:0/2436423098 learned_addr learned my addr 172.21.15.7:0/2436423098 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.616+0000 7f2aabe00640 1 --2- 172.21.15.7:0/2436423098 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ab415ba00 0x7f2ab40c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.616+0000 7f2ab8e00640 1 -- 172.21.15.7:0/2436423098 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ab415ba00 msgr2=0x7f2ab40c7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.616+0000 7f2ab8e00640 1 --2- 172.21.15.7:0/2436423098 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ab415ba00 0x7f2ab40c7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.616+0000 7f2ab9800640 1 --2- 172.21.15.7:0/2436423098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ab40c6f80 0x7f2ab41f8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.616+0000 7f2ab8e00640 1 -- 172.21.15.7:0/2436423098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ab40c6f80 msgr2=0x7f2ab41f8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.616+0000 7f2ab8e00640 1 --2- 172.21.15.7:0/2436423098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ab40c6f80 0x7f2ab41f8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.616+0000 7f2ab8e00640 1 -- 172.21.15.7:0/2436423098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab40b3ca0 con 0x7f2ab40af100 2024-09-10T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.616+0000 7f2aabe00640 1 --2- 172.21.15.7:0/2436423098 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ab415ba00 0x7f2ab40c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:49:57.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.616+0000 7f2ab9800640 1 --2- 172.21.15.7:0/2436423098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ab40c6f80 0x7f2ab41f8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.616+0000 7f2ab8e00640 1 --2- 172.21.15.7:0/2436423098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ab40af100 0x7f2ab41f5dd0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400c450 tx=0x7f2aa400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.616+0000 7f2ab1600640 1 -- 172.21.15.7:0/2436423098 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4017070 con 0x7f2ab40af100 2024-09-10T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.616+0000 7f2ab1600640 1 -- 172.21.15.7:0/2436423098 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2aa40053c0 con 0x7f2ab40af100 2024-09-10T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.617+0000 7f2abb800640 1 -- 172.21.15.7:0/2436423098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab41f8a50 con 0x7f2ab40af100 2024-09-10T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.617+0000 7f2ab1600640 1 -- 172.21.15.7:0/2436423098 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4003850 con 0x7f2ab40af100 2024-09-10T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.617+0000 7f2abb800640 1 -- 172.21.15.7:0/2436423098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab41f9010 con 0x7f2ab40af100 2024-09-10T21:49:57.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.618+0000 7f2abb800640 1 -- 172.21.15.7:0/2436423098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ab40ce610 con 0x7f2ab40af100 2024-09-10T21:49:57.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.619+0000 7f2ab1600640 1 -- 172.21.15.7:0/2436423098 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2aa40039f0 con 0x7f2ab40af100 2024-09-10T21:49:57.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.620+0000 7f2ab1600640 1 --2- 172.21.15.7:0/2436423098 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2a90077f70 0x7f2a9007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:57.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.620+0000 7f2ab1600640 1 -- 172.21.15.7:0/2436423098 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 5227+0+0 (secure 0 0 0) 0x7f2aa40992f0 con 0x7f2ab40af100 2024-09-10T21:49:57.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.620+0000 7f2aabe00640 1 --2- 172.21.15.7:0/2436423098 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2a90077f70 0x7f2a9007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:57.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.621+0000 7f2aabe00640 1 --2- 172.21.15.7:0/2436423098 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2a90077f70 0x7f2a9007a430 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40b0cb0 tx=0x7f2a9c005db0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:57.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.624+0000 7f2ab1600640 1 -- 172.21.15.7:0/2436423098 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2ab40ce610 con 0x7f2ab40af100 2024-09-10T21:49:57.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:57.818+0000 7f2abb800640 1 -- 172.21.15.7:0/2436423098 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2ab41601a0 con 0x7f2ab40af100 2024-09-10T21:49:58.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.421+0000 7f2ab1600640 1 -- 172.21.15.7:0/2436423098 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v99) ==== 170+0+0 (secure 0 0 0) 0x7f2aa4061f30 con 0x7f2ab40af100 2024-09-10T21:49:58.422 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-09-10T21:49:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.424+0000 7f2abb800640 1 -- 172.21.15.7:0/2436423098 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2a90077f70 msgr2=0x7f2a9007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.424+0000 7f2abb800640 1 --2- 172.21.15.7:0/2436423098 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2a90077f70 0x7f2a9007a430 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40b0cb0 tx=0x7f2a9c005db0 comp rx=0 tx=0).stop 2024-09-10T21:49:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.424+0000 7f2abb800640 1 -- 172.21.15.7:0/2436423098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ab40af100 msgr2=0x7f2ab41f5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:58.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.424+0000 7f2abb800640 1 --2- 172.21.15.7:0/2436423098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ab40af100 0x7f2ab41f5dd0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400c450 tx=0x7f2aa400c910 comp rx=0 tx=0).stop 2024-09-10T21:49:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.425+0000 7f2abb800640 1 -- 172.21.15.7:0/2436423098 shutdown_connections 2024-09-10T21:49:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.425+0000 7f2abb800640 1 --2- 172.21.15.7:0/2436423098 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2a90077f70 0x7f2a9007a430 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.425+0000 7f2abb800640 1 --2- 172.21.15.7:0/2436423098 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2ab40c6f80 0x7f2ab41f8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.425+0000 7f2abb800640 1 --2- 172.21.15.7:0/2436423098 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2ab415ba00 0x7f2ab40c7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.425+0000 7f2abb800640 1 --2- 172.21.15.7:0/2436423098 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2ab40af100 0x7f2ab41f5dd0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:58.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.425+0000 7f2abb800640 1 -- 172.21.15.7:0/2436423098 >> 172.21.15.7:0/2436423098 conn(0x7f2ab40b16d0 msgr2=0x7f2ab40b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.425+0000 7f2abb800640 1 -- 172.21.15.7:0/2436423098 shutdown_connections 2024-09-10T21:49:58.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.425+0000 7f2abb800640 1 -- 172.21.15.7:0/2436423098 wait complete. 2024-09-10T21:49:58.460 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:49:58.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.627+0000 7f47f5e00640 1 -- 172.21.15.7:0/922470930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47d4004d40 msgr2=0x7f47d40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:58.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.627+0000 7f47f5e00640 1 --2- 172.21.15.7:0/922470930 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47d4004d40 0x7f47d40051c0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f47e8009ed0 tx=0x7f47d4005830 comp rx=0 tx=0).stop 2024-09-10T21:49:58.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.627+0000 7f47ef400640 1 -- 172.21.15.7:0/922470930 reap_dead start 2024-09-10T21:49:58.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.627+0000 7f47f5e00640 1 -- 172.21.15.7:0/922470930 shutdown_connections 2024-09-10T21:49:58.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.627+0000 7f47f5e00640 1 -- 172.21.15.7:0/922470930 >> 172.21.15.7:0/922470930 conn(0x7f47f00b16d0 msgr2=0x7f47f00b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:58.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.627+0000 7f47f5e00640 1 -- 172.21.15.7:0/922470930 shutdown_connections 2024-09-10T21:49:58.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.627+0000 7f47f5e00640 1 -- 172.21.15.7:0/922470930 wait complete. 2024-09-10T21:49:58.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.628+0000 7f47f5e00640 1 Processor -- start 2024-09-10T21:49:58.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.629+0000 7f47f5e00640 1 -- start start 2024-09-10T21:49:58.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.629+0000 7f47f5e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47d4004d40 0x7f47f00cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:58.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.629+0000 7f47f5e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47f01f8410 0x7f47f014b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:58.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.629+0000 7f47f5e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47f00cf3c0 0x7f47f00cf870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:58.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.629+0000 7f47f5e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47f00ca3f0 con 0x7f47d4004d40 2024-09-10T21:49:58.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.629+0000 7f47f5e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f47f00ca270 con 0x7f47f01f8410 2024-09-10T21:49:58.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.629+0000 7f47f5e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f47f00ca570 con 0x7f47f00cf3c0 2024-09-10T21:49:58.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.630+0000 7f47eea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47d4004d40 0x7f47f00cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:58.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.630+0000 7f47eea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47d4004d40 0x7f47f00cd6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34172/0 (socket says 172.21.15.7:34172) 2024-09-10T21:49:58.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.630+0000 7f47eea00640 1 -- 172.21.15.7:0/1598951933 learned_addr learned my addr 172.21.15.7:0/1598951933 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:58.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.630+0000 7f47e6000640 1 --2- 172.21.15.7:0/1598951933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47f01f8410 0x7f47f014b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:58.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.630+0000 7f47ef400640 1 --2- 172.21.15.7:0/1598951933 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47f00cf3c0 0x7f47f00cf870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:58.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.630+0000 7f47eea00640 1 -- 172.21.15.7:0/1598951933 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47f00cf3c0 msgr2=0x7f47f00cf870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:58.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.630+0000 7f47eea00640 1 --2- 172.21.15.7:0/1598951933 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47f00cf3c0 0x7f47f00cf870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:58.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.630+0000 7f47eea00640 1 -- 172.21.15.7:0/1598951933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47f01f8410 msgr2=0x7f47f014b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:58.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.630+0000 7f47eea00640 1 --2- 172.21.15.7:0/1598951933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47f01f8410 0x7f47f014b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:58.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.630+0000 7f47eea00640 1 -- 172.21.15.7:0/1598951933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47f00cdfd0 con 0x7f47d4004d40 2024-09-10T21:49:58.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.630+0000 7f47eea00640 1 --2- 172.21.15.7:0/1598951933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47d4004d40 0x7f47f00cd6f0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f47e0004650 tx=0x7f47e000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:58.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.631+0000 7f47e7e00640 1 -- 172.21.15.7:0/1598951933 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e001e070 con 0x7f47d4004d40 2024-09-10T21:49:58.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.631+0000 7f47e7e00640 1 -- 172.21.15.7:0/1598951933 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47e000e7c0 con 0x7f47d4004d40 2024-09-10T21:49:58.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.631+0000 7f47e7e00640 1 -- 172.21.15.7:0/1598951933 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47e000ebd0 con 0x7f47d4004d40 2024-09-10T21:49:58.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.631+0000 7f47f5e00640 1 -- 172.21.15.7:0/1598951933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47f00ce2b0 con 0x7f47d4004d40 2024-09-10T21:49:58.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.631+0000 7f47f5e00640 1 -- 172.21.15.7:0/1598951933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47f00ce870 con 0x7f47d4004d40 2024-09-10T21:49:58.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.632+0000 7f47f5e00640 1 -- 172.21.15.7:0/1598951933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f0160100 con 0x7f47d4004d40 2024-09-10T21:49:58.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.633+0000 7f47e7e00640 1 -- 172.21.15.7:0/1598951933 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f47e0019430 con 0x7f47d4004d40 2024-09-10T21:49:58.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.634+0000 7f47e7e00640 1 --2- 172.21.15.7:0/1598951933 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f47cc077f70 0x7f47cc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:58.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.634+0000 7f47e7e00640 1 -- 172.21.15.7:0/1598951933 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 5609+0+0 (secure 0 0 0) 0x7f47e009b0f0 con 0x7f47d4004d40 2024-09-10T21:49:58.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.634+0000 7f47e6000640 1 --2- 172.21.15.7:0/1598951933 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f47cc077f70 0x7f47cc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:58.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.635+0000 7f47e6000640 1 --2- 172.21.15.7:0/1598951933 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f47cc077f70 0x7f47cc07a430 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f47f00c3510 tx=0x7f47d80067a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:58.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.639+0000 7f47e7e00640 1 -- 172.21.15.7:0/1598951933 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f47e0011e20 con 0x7f47d4004d40 2024-09-10T21:49:58.872 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:58.871+0000 7f47f5e00640 1 -- 172.21.15.7:0/1598951933 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f47f0069880 con 0x7f47d4004d40 2024-09-10T21:49:59.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.423+0000 7f47e7e00640 1 -- 172.21.15.7:0/1598951933 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v81) ==== 174+0+0 (secure 0 0 0) 0x7f47e0063560 con 0x7f47d4004d40 2024-09-10T21:49:59.424 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 14 and data pool 15 2024-09-10T21:49:59.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.428+0000 7f47f5e00640 1 -- 172.21.15.7:0/1598951933 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f47cc077f70 msgr2=0x7f47cc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.428+0000 7f47f5e00640 1 --2- 172.21.15.7:0/1598951933 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f47cc077f70 0x7f47cc07a430 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f47f00c3510 tx=0x7f47d80067a0 comp rx=0 tx=0).stop 2024-09-10T21:49:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.428+0000 7f47f5e00640 1 -- 172.21.15.7:0/1598951933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47d4004d40 msgr2=0x7f47f00cd6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.428+0000 7f47f5e00640 1 --2- 172.21.15.7:0/1598951933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47d4004d40 0x7f47f00cd6f0 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f47e0004650 tx=0x7f47e000f790 comp rx=0 tx=0).stop 2024-09-10T21:49:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.428+0000 7f47f5e00640 1 -- 172.21.15.7:0/1598951933 shutdown_connections 2024-09-10T21:49:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.428+0000 7f47f5e00640 1 --2- 172.21.15.7:0/1598951933 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f47cc077f70 0x7f47cc07a430 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:59.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.428+0000 7f47f5e00640 1 --2- 172.21.15.7:0/1598951933 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f47f01f8410 0x7f47f014b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:59.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.428+0000 7f47f5e00640 1 --2- 172.21.15.7:0/1598951933 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47f00cf3c0 0x7f47f00cf870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:59.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.428+0000 7f47f5e00640 1 --2- 172.21.15.7:0/1598951933 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f47d4004d40 0x7f47f00cd6f0 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:59.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.428+0000 7f47f5e00640 1 -- 172.21.15.7:0/1598951933 >> 172.21.15.7:0/1598951933 conn(0x7f47f00b16d0 msgr2=0x7f47f006ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:59.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.428+0000 7f47f5e00640 1 -- 172.21.15.7:0/1598951933 shutdown_connections 2024-09-10T21:49:59.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.428+0000 7f47f5e00640 1 -- 172.21.15.7:0/1598951933 wait complete. 2024-09-10T21:49:59.471 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:49:59.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.632+0000 7f5b5ee00640 1 -- 172.21.15.7:0/2987449778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b5815fda0 msgr2=0x7f5b58160180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:59.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.632+0000 7f5b5ee00640 1 --2- 172.21.15.7:0/2987449778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b5815fda0 0x7f5b58160180 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f5b5806dac0 tx=0x7f5b5002aa30 comp rx=0 tx=0).stop 2024-09-10T21:49:59.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.632+0000 7f5b5ee00640 1 -- 172.21.15.7:0/2987449778 shutdown_connections 2024-09-10T21:49:59.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.632+0000 7f5b5ee00640 1 --2- 172.21.15.7:0/2987449778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b5815fda0 0x7f5b58160180 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:59.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.632+0000 7f5b5ee00640 1 --2- 172.21.15.7:0/2987449778 >> v2:172.21.15.133:3300/0 conn(0x7f5b5815bc90 0x7f5b580ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:59.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.632+0000 7f5b5ee00640 1 -- 172.21.15.7:0/2987449778 >> 172.21.15.7:0/2987449778 conn(0x7f5b580b16d0 msgr2=0x7f5b580b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:59.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.632+0000 7f5b5ee00640 1 -- 172.21.15.7:0/2987449778 shutdown_connections 2024-09-10T21:49:59.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.632+0000 7f5b5ee00640 1 -- 172.21.15.7:0/2987449778 wait complete. 2024-09-10T21:49:59.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.634+0000 7f5b5ee00640 1 Processor -- start 2024-09-10T21:49:59.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.634+0000 7f5b5ee00640 1 -- start start 2024-09-10T21:49:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.635+0000 7f5b5ee00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b580cd750 0x7f5b580c81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.635+0000 7f5b5ee00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b5815bc90 0x7f5b580c8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.635+0000 7f5b5ee00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b5815fda0 0x7f5b580c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.635+0000 7f5b5ee00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5806fe70 con 0x7f5b580cd750 2024-09-10T21:49:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.635+0000 7f5b5ee00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5b5806fcf0 con 0x7f5b5815bc90 2024-09-10T21:49:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.635+0000 7f5b5ee00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f5b5806fff0 con 0x7f5b5815fda0 2024-09-10T21:49:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.635+0000 7f5b57e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b580cd750 0x7f5b580c81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.635+0000 7f5b57e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b580cd750 0x7f5b580c81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34186/0 (socket says 172.21.15.7:34186) 2024-09-10T21:49:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.635+0000 7f5b57e00640 1 -- 172.21.15.7:0/3013537113 learned_addr learned my addr 172.21.15.7:0/3013537113 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:49:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.635+0000 7f5b4fe00640 1 --2- 172.21.15.7:0/3013537113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b5815bc90 0x7f5b580c8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.635+0000 7f5b5ce00640 1 --2- 172.21.15.7:0/3013537113 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b5815fda0 0x7f5b580c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.636+0000 7f5b57e00640 1 -- 172.21.15.7:0/3013537113 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b5815fda0 msgr2=0x7f5b580c9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.636+0000 7f5b57e00640 1 --2- 172.21.15.7:0/3013537113 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b5815fda0 0x7f5b580c9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.636+0000 7f5b57e00640 1 -- 172.21.15.7:0/3013537113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b5815bc90 msgr2=0x7f5b580c8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.636+0000 7f5b57e00640 1 --2- 172.21.15.7:0/3013537113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b5815bc90 0x7f5b580c8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.636+0000 7f5b57e00640 1 -- 172.21.15.7:0/3013537113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b581f2f00 con 0x7f5b580cd750 2024-09-10T21:49:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.636+0000 7f5b57e00640 1 --2- 172.21.15.7:0/3013537113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b580cd750 0x7f5b580c81a0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800d950 tx=0x7f5b4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.636+0000 7f5b54c00640 1 -- 172.21.15.7:0/3013537113 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48017030 con 0x7f5b580cd750 2024-09-10T21:49:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.636+0000 7f5b54c00640 1 -- 172.21.15.7:0/3013537113 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b48009b20 con 0x7f5b580cd750 2024-09-10T21:49:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.636+0000 7f5b54c00640 1 -- 172.21.15.7:0/3013537113 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b48004020 con 0x7f5b580cd750 2024-09-10T21:49:59.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.636+0000 7f5b5ee00640 1 -- 172.21.15.7:0/3013537113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b581f5960 con 0x7f5b580cd750 2024-09-10T21:49:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.636+0000 7f5b5ee00640 1 -- 172.21.15.7:0/3013537113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b581f5f20 con 0x7f5b580cd750 2024-09-10T21:49:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.637+0000 7f5b5ee00640 1 -- 172.21.15.7:0/3013537113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b580c6480 con 0x7f5b580cd750 2024-09-10T21:49:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.638+0000 7f5b54c00640 1 -- 172.21.15.7:0/3013537113 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5b480041c0 con 0x7f5b580cd750 2024-09-10T21:49:59.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.639+0000 7f5b54c00640 1 --2- 172.21.15.7:0/3013537113 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5b34077f70 0x7f5b3407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:49:59.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.640+0000 7f5b54c00640 1 -- 172.21.15.7:0/3013537113 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f5b48099cf0 con 0x7f5b580cd750 2024-09-10T21:49:59.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.640+0000 7f5b4fe00640 1 --2- 172.21.15.7:0/3013537113 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5b34077f70 0x7f5b3407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:49:59.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.641+0000 7f5b4fe00640 1 --2- 172.21.15.7:0/3013537113 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5b34077f70 0x7f5b3407a430 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5b581f5720 tx=0x7f5b40009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:49:59.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.644+0000 7f5b54c00640 1 -- 172.21.15.7:0/3013537113 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5b580c6480 con 0x7f5b580cd750 2024-09-10T21:49:59.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.839+0000 7f5b5ee00640 1 -- 172.21.15.7:0/3013537113 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b58070510 con 0x7f5b580cd750 2024-09-10T21:49:59.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.840+0000 7f5b54c00640 1 -- 172.21.15.7:0/3013537113 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16648 (secure 0 0 0) 0x7f5b4809e070 con 0x7f5b580cd750 2024-09-10T21:49:59.842 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:49:59.842 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":100,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:49:58.873016+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:49:56.657911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T21:49:57.828979+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:49:59.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.843+0000 7f5b5ee00640 1 -- 172.21.15.7:0/3013537113 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5b34077f70 msgr2=0x7f5b3407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:59.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.843+0000 7f5b5ee00640 1 --2- 172.21.15.7:0/3013537113 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5b34077f70 0x7f5b3407a430 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f5b581f5720 tx=0x7f5b40009040 comp rx=0 tx=0).stop 2024-09-10T21:49:59.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.843+0000 7f5b5ee00640 1 -- 172.21.15.7:0/3013537113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b580cd750 msgr2=0x7f5b580c81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:49:59.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.843+0000 7f5b5ee00640 1 --2- 172.21.15.7:0/3013537113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b580cd750 0x7f5b580c81a0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f5b4800d950 tx=0x7f5b4800de10 comp rx=0 tx=0).stop 2024-09-10T21:49:59.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.843+0000 7f5b5ee00640 1 -- 172.21.15.7:0/3013537113 shutdown_connections 2024-09-10T21:49:59.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.843+0000 7f5b5ee00640 1 --2- 172.21.15.7:0/3013537113 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5b34077f70 0x7f5b3407a430 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:59.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.843+0000 7f5b5ee00640 1 --2- 172.21.15.7:0/3013537113 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5b5815bc90 0x7f5b580c8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:59.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.843+0000 7f5b5ee00640 1 --2- 172.21.15.7:0/3013537113 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5b5815fda0 0x7f5b580c9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:59.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.843+0000 7f5b5ee00640 1 --2- 172.21.15.7:0/3013537113 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5b580cd750 0x7f5b580c81a0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:49:59.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.843+0000 7f5b5ee00640 1 -- 172.21.15.7:0/3013537113 >> 172.21.15.7:0/3013537113 conn(0x7f5b580b16d0 msgr2=0x7f5b580b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:49:59.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.843+0000 7f5b5ee00640 1 -- 172.21.15.7:0/3013537113 shutdown_connections 2024-09-10T21:49:59.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:49:59.843+0000 7f5b5ee00640 1 -- 172.21.15.7:0/3013537113 wait complete. 2024-09-10T21:49:59.861 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:50:00.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.021+0000 7f38ff200640 1 -- 172.21.15.7:0/3689839901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f815aa30 msgr2=0x7f38f8073550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:00.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.021+0000 7f38ff200640 1 --2- 172.21.15.7:0/3689839901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f815aa30 0x7f38f8073550 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009830 tx=0x7f38e802a520 comp rx=0 tx=0).stop 2024-09-10T21:50:00.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.022+0000 7f38ff200640 1 -- 172.21.15.7:0/3689839901 shutdown_connections 2024-09-10T21:50:00.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.022+0000 7f38ff200640 1 --2- 172.21.15.7:0/3689839901 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38f80cbcd0 0x7f38f80ceca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:00.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.022+0000 7f38ff200640 1 --2- 172.21.15.7:0/3689839901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f815aa30 0x7f38f8073550 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:00.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.022+0000 7f38ff200640 1 -- 172.21.15.7:0/3689839901 >> 172.21.15.7:0/3689839901 conn(0x7f38f80b16d0 msgr2=0x7f38f80b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:00.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.022+0000 7f38ff200640 1 -- 172.21.15.7:0/3689839901 shutdown_connections 2024-09-10T21:50:00.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.022+0000 7f38ff200640 1 -- 172.21.15.7:0/3689839901 wait complete. 2024-09-10T21:50:00.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.023+0000 7f38ff200640 1 Processor -- start 2024-09-10T21:50:00.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.023+0000 7f38ff200640 1 -- start start 2024-09-10T21:50:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.024+0000 7f38ff200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38f80cbcd0 0x7f38f80c8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.024+0000 7f38ff200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f80ce860 0x7f38f80c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:00.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.024+0000 7f38ff200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38f815aa30 0x7f38f80c7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:00.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.024+0000 7f38ff200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f38f806ee30 con 0x7f38f80ce860 2024-09-10T21:50:00.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.024+0000 7f38ff200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f38f806ecb0 con 0x7f38f80cbcd0 2024-09-10T21:50:00.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.024+0000 7f38ff200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f38f815ed60 con 0x7f38f815aa30 2024-09-10T21:50:00.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.025+0000 7f38f7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f80ce860 0x7f38f80c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:00.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.025+0000 7f38f7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f80ce860 0x7f38f80c70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34202/0 (socket says 172.21.15.7:34202) 2024-09-10T21:50:00.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.025+0000 7f38f7400640 1 -- 172.21.15.7:0/4280103269 learned_addr learned my addr 172.21.15.7:0/4280103269 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:00.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.025+0000 7f38fd200640 1 --2- 172.21.15.7:0/4280103269 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38f815aa30 0x7f38f80c7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:00.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.025+0000 7f38f7e00640 1 --2- 172.21.15.7:0/4280103269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38f80cbcd0 0x7f38f80c8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:00.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.025+0000 7f38f7400640 1 -- 172.21.15.7:0/4280103269 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38f815aa30 msgr2=0x7f38f80c7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:00.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.025+0000 7f38f7400640 1 --2- 172.21.15.7:0/4280103269 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38f815aa30 0x7f38f80c7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:00.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.025+0000 7f38f7400640 1 -- 172.21.15.7:0/4280103269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38f80cbcd0 msgr2=0x7f38f80c8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:00.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.025+0000 7f38f7400640 1 --2- 172.21.15.7:0/4280103269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38f80cbcd0 0x7f38f80c8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:00.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.025+0000 7f38f7400640 1 -- 172.21.15.7:0/4280103269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38f81f7bd0 con 0x7f38f80ce860 2024-09-10T21:50:00.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.025+0000 7f38f7e00640 1 --2- 172.21.15.7:0/4280103269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38f80cbcd0 0x7f38f80c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:50:00.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.025+0000 7f38f7400640 1 --2- 172.21.15.7:0/4280103269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f80ce860 0x7f38f80c70f0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f38e400c490 tx=0x7f38e400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:00.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.025+0000 7f38efe00640 1 -- 172.21.15.7:0/4280103269 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e4016020 con 0x7f38f80ce860 2024-09-10T21:50:00.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.026+0000 7f38efe00640 1 -- 172.21.15.7:0/4280103269 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38e4012940 con 0x7f38f80ce860 2024-09-10T21:50:00.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.026+0000 7f38efe00640 1 -- 172.21.15.7:0/4280103269 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38e4012d90 con 0x7f38f80ce860 2024-09-10T21:50:00.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.026+0000 7f38ff200640 1 -- 172.21.15.7:0/4280103269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38f81f7eb0 con 0x7f38f80ce860 2024-09-10T21:50:00.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.026+0000 7f38ff200640 1 -- 172.21.15.7:0/4280103269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38f81f8470 con 0x7f38f80ce860 2024-09-10T21:50:00.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.027+0000 7f38ff200640 1 -- 172.21.15.7:0/4280103269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38f815f240 con 0x7f38f80ce860 2024-09-10T21:50:00.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.028+0000 7f38efe00640 1 -- 172.21.15.7:0/4280103269 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f38e401b400 con 0x7f38f80ce860 2024-09-10T21:50:00.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.029+0000 7f38efe00640 1 --2- 172.21.15.7:0/4280103269 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f38d8077f70 0x7f38d807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:00.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.029+0000 7f38efe00640 1 -- 172.21.15.7:0/4280103269 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7f38e409a080 con 0x7f38f80ce860 2024-09-10T21:50:00.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.029+0000 7f38f7e00640 1 --2- 172.21.15.7:0/4280103269 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f38d8077f70 0x7f38d807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:00.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.030+0000 7f38f7e00640 1 --2- 172.21.15.7:0/4280103269 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f38d8077f70 0x7f38d807a430 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f38e8034000 tx=0x7f38e8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:00.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.033+0000 7f38efe00640 1 -- 172.21.15.7:0/4280103269 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f38e4020020 con 0x7f38f80ce860 2024-09-10T21:50:00.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.230+0000 7f38ff200640 1 -- 172.21.15.7:0/4280103269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f38f81f7810 con 0x7f38f80ce860 2024-09-10T21:50:00.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.231+0000 7f38efe00640 1 -- 172.21.15.7:0/4280103269 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+16648 (secure 0 0 0) 0x7f38e4062af0 con 0x7f38f80ce860 2024-09-10T21:50:00.232 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:00.232 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":100,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:49:58.873016+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:49:56.657911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T21:49:57.828979+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:50:00.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.233+0000 7f38ff200640 1 -- 172.21.15.7:0/4280103269 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f38d8077f70 msgr2=0x7f38d807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.233+0000 7f38ff200640 1 --2- 172.21.15.7:0/4280103269 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f38d8077f70 0x7f38d807a430 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f38e8034000 tx=0x7f38e8004730 comp rx=0 tx=0).stop 2024-09-10T21:50:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.233+0000 7f38ff200640 1 -- 172.21.15.7:0/4280103269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f80ce860 msgr2=0x7f38f80c70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.233+0000 7f38ff200640 1 --2- 172.21.15.7:0/4280103269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f80ce860 0x7f38f80c70f0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f38e400c490 tx=0x7f38e400c950 comp rx=0 tx=0).stop 2024-09-10T21:50:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.233+0000 7f38ff200640 1 -- 172.21.15.7:0/4280103269 shutdown_connections 2024-09-10T21:50:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.233+0000 7f38ff200640 1 --2- 172.21.15.7:0/4280103269 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f38d8077f70 0x7f38d807a430 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.233+0000 7f38ff200640 1 --2- 172.21.15.7:0/4280103269 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f38f815aa30 0x7f38f80c7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.233+0000 7f38ff200640 1 --2- 172.21.15.7:0/4280103269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f38f80ce860 0x7f38f80c70f0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.233+0000 7f38ff200640 1 --2- 172.21.15.7:0/4280103269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f38f80cbcd0 0x7f38f80c8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.233+0000 7f38ff200640 1 -- 172.21.15.7:0/4280103269 >> 172.21.15.7:0/4280103269 conn(0x7f38f80b16d0 msgr2=0x7f38f80cacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.234+0000 7f38ff200640 1 -- 172.21.15.7:0/4280103269 shutdown_connections 2024-09-10T21:50:00.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.234+0000 7f38ff200640 1 -- 172.21.15.7:0/4280103269 wait complete. 2024-09-10T21:50:00.250 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:50:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.414+0000 7fb713200640 1 -- 172.21.15.7:0/3069719492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71415f6c0 msgr2=0x7fb71415fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.414+0000 7fb713200640 1 --2- 172.21.15.7:0/3069719492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71415f6c0 0x7fb71415fb50 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fb7140747e0 tx=0x7fb70c02aa10 comp rx=0 tx=0).stop 2024-09-10T21:50:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.414+0000 7fb713200640 1 -- 172.21.15.7:0/3069719492 shutdown_connections 2024-09-10T21:50:00.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.414+0000 7fb713200640 1 --2- 172.21.15.7:0/3069719492 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb71415f6c0 0x7fb71415fb50 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.414+0000 7fb713200640 1 --2- 172.21.15.7:0/3069719492 >> v2:172.21.15.133:3300/0 conn(0x7fb7140cfda0 0x7fb7140d01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.414+0000 7fb713200640 1 --2- 172.21.15.7:0/3069719492 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb71415efe0 0x7fb71406eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.414+0000 7fb713200640 1 -- 172.21.15.7:0/3069719492 >> 172.21.15.7:0/3069719492 conn(0x7fb71406cd00 msgr2=0x7fb71406d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.414+0000 7fb713200640 1 -- 172.21.15.7:0/3069719492 shutdown_connections 2024-09-10T21:50:00.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.414+0000 7fb713200640 1 -- 172.21.15.7:0/3069719492 wait complete. 2024-09-10T21:50:00.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.416+0000 7fb713200640 1 Processor -- start 2024-09-10T21:50:00.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.416+0000 7fb713200640 1 -- start start 2024-09-10T21:50:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.417+0000 7fb713200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7140cfda0 0x7fb7141f8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.417+0000 7fb713200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb71415efe0 0x7fb7141f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.417+0000 7fb713200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb71415f6c0 0x7fb714150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.417+0000 7fb713200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb714160210 con 0x7fb7140cfda0 2024-09-10T21:50:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.417+0000 7fb713200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb714160090 con 0x7fb71415f6c0 2024-09-10T21:50:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.417+0000 7fb713200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fb714160390 con 0x7fb71415efe0 2024-09-10T21:50:00.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.417+0000 7fb711e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7140cfda0 0x7fb7141f8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.417+0000 7fb711e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7140cfda0 0x7fb7141f8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34218/0 (socket says 172.21.15.7:34218) 2024-09-10T21:50:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.417+0000 7fb711e00640 1 -- 172.21.15.7:0/400557391 learned_addr learned my addr 172.21.15.7:0/400557391 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.417+0000 7fb711400640 1 --2- 172.21.15.7:0/400557391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb71415efe0 0x7fb7141f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.417+0000 7fb712800640 1 --2- 172.21.15.7:0/400557391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb71415f6c0 0x7fb714150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.418+0000 7fb711e00640 1 -- 172.21.15.7:0/400557391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb71415efe0 msgr2=0x7fb7141f4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.418+0000 7fb711e00640 1 --2- 172.21.15.7:0/400557391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb71415efe0 0x7fb7141f4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.418+0000 7fb711e00640 1 -- 172.21.15.7:0/400557391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb71415f6c0 msgr2=0x7fb714150790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.418+0000 7fb711e00640 1 --2- 172.21.15.7:0/400557391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb71415f6c0 0x7fb714150790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.418+0000 7fb711e00640 1 -- 172.21.15.7:0/400557391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7141f4c40 con 0x7fb7140cfda0 2024-09-10T21:50:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.418+0000 7fb711e00640 1 --2- 172.21.15.7:0/400557391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7140cfda0 0x7fb7141f8550 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fb70400d7a0 tx=0x7fb70400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.418+0000 7fb70a000640 1 -- 172.21.15.7:0/400557391 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704017070 con 0x7fb7140cfda0 2024-09-10T21:50:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.418+0000 7fb70a000640 1 -- 172.21.15.7:0/400557391 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb704009d20 con 0x7fb7140cfda0 2024-09-10T21:50:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.418+0000 7fb70a000640 1 -- 172.21.15.7:0/400557391 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb704005280 con 0x7fb7140cfda0 2024-09-10T21:50:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.418+0000 7fb713200640 1 -- 172.21.15.7:0/400557391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7141f4f20 con 0x7fb7140cfda0 2024-09-10T21:50:00.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.418+0000 7fb713200640 1 -- 172.21.15.7:0/400557391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb714150270 con 0x7fb7140cfda0 2024-09-10T21:50:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.419+0000 7fb713200640 1 -- 172.21.15.7:0/400557391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb71406eed0 con 0x7fb7140cfda0 2024-09-10T21:50:00.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.420+0000 7fb70a000640 1 -- 172.21.15.7:0/400557391 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb704005420 con 0x7fb7140cfda0 2024-09-10T21:50:00.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.421+0000 7fb70a000640 1 --2- 172.21.15.7:0/400557391 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb6f4077f70 0x7fb6f407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:00.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.421+0000 7fb70a000640 1 -- 172.21.15.7:0/400557391 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5709+0+0 (secure 0 0 0) 0x7fb704099e90 con 0x7fb7140cfda0 2024-09-10T21:50:00.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.422+0000 7fb711400640 1 --2- 172.21.15.7:0/400557391 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb6f4077f70 0x7fb6f407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:00.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.423+0000 7fb711400640 1 --2- 172.21.15.7:0/400557391 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb6f4077f70 0x7fb6f407a430 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb71406f980 tx=0x7fb70000b310 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:00.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.425+0000 7fb70a000640 1 -- 172.21.15.7:0/400557391 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb704062340 con 0x7fb7140cfda0 2024-09-10T21:50:00.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:00.663+0000 7fb713200640 1 -- 172.21.15.7:0/400557391 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb71415e790 con 0x7fb7140cfda0 2024-09-10T21:50:01.295 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:50:01.300 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:50:01.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.447+0000 7fb70a000640 1 -- 172.21.15.7:0/400557391 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v84) ==== 118+0+0 (secure 0 0 0) 0x7fb70406b0b0 con 0x7fb7140cfda0 2024-09-10T21:50:01.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.450+0000 7fb713200640 1 -- 172.21.15.7:0/400557391 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb6f4077f70 msgr2=0x7fb6f407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:01.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.450+0000 7fb713200640 1 --2- 172.21.15.7:0/400557391 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb6f4077f70 0x7fb6f407a430 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fb71406f980 tx=0x7fb70000b310 comp rx=0 tx=0).stop 2024-09-10T21:50:01.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.450+0000 7fb713200640 1 -- 172.21.15.7:0/400557391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7140cfda0 msgr2=0x7fb7141f8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:01.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.450+0000 7fb713200640 1 --2- 172.21.15.7:0/400557391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7140cfda0 0x7fb7141f8550 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fb70400d7a0 tx=0x7fb70400dc60 comp rx=0 tx=0).stop 2024-09-10T21:50:01.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.450+0000 7fb713200640 1 -- 172.21.15.7:0/400557391 shutdown_connections 2024-09-10T21:50:01.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.450+0000 7fb713200640 1 --2- 172.21.15.7:0/400557391 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb6f4077f70 0x7fb6f407a430 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:01.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.450+0000 7fb713200640 1 --2- 172.21.15.7:0/400557391 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb71415f6c0 0x7fb714150790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:01.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.450+0000 7fb713200640 1 --2- 172.21.15.7:0/400557391 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb71415efe0 0x7fb7141f4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:01.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.450+0000 7fb713200640 1 --2- 172.21.15.7:0/400557391 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7140cfda0 0x7fb7141f8550 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:01.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.450+0000 7fb713200640 1 -- 172.21.15.7:0/400557391 >> 172.21.15.7:0/400557391 conn(0x7fb71406cd00 msgr2=0x7fb71415c5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:01.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.451+0000 7fb713200640 1 -- 172.21.15.7:0/400557391 shutdown_connections 2024-09-10T21:50:01.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.451+0000 7fb713200640 1 -- 172.21.15.7:0/400557391 wait complete. 2024-09-10T21:50:01.469 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:50:01.469 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:50:01.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.633+0000 7f3f68200640 1 -- 172.21.15.7:0/2429128661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f60161ab0 msgr2=0x7f3f600cc530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:01.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.633+0000 7f3f68200640 1 --2- 172.21.15.7:0/2429128661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f60161ab0 0x7f3f600cc530 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f3f600cee10 tx=0x7f3f4c02a560 comp rx=0 tx=0).stop 2024-09-10T21:50:01.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.633+0000 7f3f68200640 1 -- 172.21.15.7:0/2429128661 shutdown_connections 2024-09-10T21:50:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.633+0000 7f3f68200640 1 --2- 172.21.15.7:0/2429128661 >> v2:172.21.15.133:3300/0 conn(0x7f3f600af100 0x7f3f600cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.633+0000 7f3f68200640 1 --2- 172.21.15.7:0/2429128661 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f60161ab0 0x7f3f600cc530 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.633+0000 7f3f68200640 1 --2- 172.21.15.7:0/2429128661 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3f6015ba00 0x7f3f600ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.633+0000 7f3f68200640 1 -- 172.21.15.7:0/2429128661 >> 172.21.15.7:0/2429128661 conn(0x7f3f600b16d0 msgr2=0x7f3f600b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.633+0000 7f3f68200640 1 -- 172.21.15.7:0/2429128661 shutdown_connections 2024-09-10T21:50:01.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.633+0000 7f3f68200640 1 -- 172.21.15.7:0/2429128661 wait complete. 2024-09-10T21:50:01.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.635+0000 7f3f68200640 1 Processor -- start 2024-09-10T21:50:01.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.635+0000 7f3f68200640 1 -- start start 2024-09-10T21:50:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.637+0000 7f3f68200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3f600af100 0x7f3f600b56f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.637+0000 7f3f68200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f6015ba00 0x7f3f6014b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.637+0000 7f3f68200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f601f80b0 0x7f3f600cb7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.637+0000 7f3f68200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6015e8d0 con 0x7f3f601f80b0 2024-09-10T21:50:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.637+0000 7f3f68200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3f600aef30 con 0x7f3f6015ba00 2024-09-10T21:50:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.637+0000 7f3f68200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f3f6015ea50 con 0x7f3f600af100 2024-09-10T21:50:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.637+0000 7f3f66200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f601f80b0 0x7f3f600cb7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.637+0000 7f3f66200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f601f80b0 0x7f3f600cb7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34232/0 (socket says 172.21.15.7:34232) 2024-09-10T21:50:01.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.637+0000 7f3f66200640 1 -- 172.21.15.7:0/2226691439 learned_addr learned my addr 172.21.15.7:0/2226691439 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:01.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.637+0000 7f3f65800640 1 --2- 172.21.15.7:0/2226691439 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3f600af100 0x7f3f600b56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:01.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.638+0000 7f3f66200640 1 -- 172.21.15.7:0/2226691439 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3f600af100 msgr2=0x7f3f600b56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:01.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.638+0000 7f3f66200640 1 --2- 172.21.15.7:0/2226691439 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3f600af100 0x7f3f600b56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:01.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.638+0000 7f3f66200640 1 -- 172.21.15.7:0/2226691439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f6015ba00 msgr2=0x7f3f6014b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:01.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.638+0000 7f3f66200640 1 --2- 172.21.15.7:0/2226691439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f6015ba00 0x7f3f6014b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:01.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.638+0000 7f3f66200640 1 -- 172.21.15.7:0/2226691439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f600c6c90 con 0x7f3f601f80b0 2024-09-10T21:50:01.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.638+0000 7f3f66200640 1 --2- 172.21.15.7:0/2226691439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f601f80b0 0x7f3f600cb7a0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f3f5800cce0 tx=0x7f3f5800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:01.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.638+0000 7f3f5d600640 1 -- 172.21.15.7:0/2226691439 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800f450 con 0x7f3f601f80b0 2024-09-10T21:50:01.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.638+0000 7f3f5d600640 1 -- 172.21.15.7:0/2226691439 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f5800f700 con 0x7f3f601f80b0 2024-09-10T21:50:01.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.638+0000 7f3f68200640 1 -- 172.21.15.7:0/2226691439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f600c6f70 con 0x7f3f601f80b0 2024-09-10T21:50:01.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.638+0000 7f3f5d600640 1 -- 172.21.15.7:0/2226691439 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f5800fb50 con 0x7f3f601f80b0 2024-09-10T21:50:01.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.638+0000 7f3f68200640 1 -- 172.21.15.7:0/2226691439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f6014b6b0 con 0x7f3f601f80b0 2024-09-10T21:50:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.639+0000 7f3f68200640 1 -- 172.21.15.7:0/2226691439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f6014a7a0 con 0x7f3f601f80b0 2024-09-10T21:50:01.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.641+0000 7f3f5d600640 1 -- 172.21.15.7:0/2226691439 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3f580213f0 con 0x7f3f601f80b0 2024-09-10T21:50:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.642+0000 7f3f5d600640 1 --2- 172.21.15.7:0/2226691439 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3f40077fa0 0x7f3f4007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.642+0000 7f3f5d600640 1 -- 172.21.15.7:0/2226691439 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f3f5809a990 con 0x7f3f601f80b0 2024-09-10T21:50:01.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.643+0000 7f3f65800640 1 --2- 172.21.15.7:0/2226691439 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3f40077fa0 0x7f3f4007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:01.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.643+0000 7f3f65800640 1 --2- 172.21.15.7:0/2226691439 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3f40077fa0 0x7f3f4007a460 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400cac0 tx=0x7f3f5400c4a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:01.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.646+0000 7f3f5d600640 1 -- 172.21.15.7:0/2226691439 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3f58014030 con 0x7f3f601f80b0 2024-09-10T21:50:01.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:01.882+0000 7f3f68200640 1 -- 172.21.15.7:0/2226691439 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f3f6015dc80 con 0x7f3f601f80b0 2024-09-10T21:50:02.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.453+0000 7f3f5d600640 1 -- 172.21.15.7:0/2226691439 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v85) ==== 122+0+0 (secure 0 0 0) 0x7f3f58063480 con 0x7f3f601f80b0 2024-09-10T21:50:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.455+0000 7f3f68200640 1 -- 172.21.15.7:0/2226691439 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3f40077fa0 msgr2=0x7f3f4007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.455+0000 7f3f68200640 1 --2- 172.21.15.7:0/2226691439 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3f40077fa0 0x7f3f4007a460 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3f5400cac0 tx=0x7f3f5400c4a0 comp rx=0 tx=0).stop 2024-09-10T21:50:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.455+0000 7f3f68200640 1 -- 172.21.15.7:0/2226691439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f601f80b0 msgr2=0x7f3f600cb7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.455+0000 7f3f68200640 1 --2- 172.21.15.7:0/2226691439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f601f80b0 0x7f3f600cb7a0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f3f5800cce0 tx=0x7f3f5800a620 comp rx=0 tx=0).stop 2024-09-10T21:50:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.456+0000 7f3f68200640 1 -- 172.21.15.7:0/2226691439 shutdown_connections 2024-09-10T21:50:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.456+0000 7f3f68200640 1 --2- 172.21.15.7:0/2226691439 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3f40077fa0 0x7f3f4007a460 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:02.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.456+0000 7f3f68200640 1 --2- 172.21.15.7:0/2226691439 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3f6015ba00 0x7f3f6014b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:02.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.456+0000 7f3f68200640 1 --2- 172.21.15.7:0/2226691439 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3f601f80b0 0x7f3f600cb7a0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:02.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.456+0000 7f3f68200640 1 --2- 172.21.15.7:0/2226691439 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3f600af100 0x7f3f600b56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:02.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.456+0000 7f3f68200640 1 -- 172.21.15.7:0/2226691439 >> 172.21.15.7:0/2226691439 conn(0x7f3f600b16d0 msgr2=0x7f3f600ae340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:02.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.456+0000 7f3f68200640 1 -- 172.21.15.7:0/2226691439 shutdown_connections 2024-09-10T21:50:02.458 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.456+0000 7f3f68200640 1 -- 172.21.15.7:0/2226691439 wait complete. 2024-09-10T21:50:02.474 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:50:02.474 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:50:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.636+0000 7fb67e200640 1 -- 172.21.15.7:0/348270090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6780c82d0 msgr2=0x7fb6780c8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.636+0000 7fb67e200640 1 --2- 172.21.15.7:0/348270090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6780c82d0 0x7fb6780c8750 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fb670005fd0 tx=0x7fb67002ab20 comp rx=0 tx=0).stop 2024-09-10T21:50:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.636+0000 7fb67e200640 1 -- 172.21.15.7:0/348270090 shutdown_connections 2024-09-10T21:50:02.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.636+0000 7fb67e200640 1 --2- 172.21.15.7:0/348270090 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6780c82d0 0x7fb6780c8750 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:02.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.636+0000 7fb67e200640 1 --2- 172.21.15.7:0/348270090 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6780d2a20 0x7fb6780d4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:02.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.636+0000 7fb67e200640 1 -- 172.21.15.7:0/348270090 >> 172.21.15.7:0/348270090 conn(0x7fb6780d3de0 msgr2=0x7fb6780d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:02.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.636+0000 7fb67e200640 1 -- 172.21.15.7:0/348270090 shutdown_connections 2024-09-10T21:50:02.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.636+0000 7fb67e200640 1 -- 172.21.15.7:0/348270090 wait complete. 2024-09-10T21:50:02.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.637+0000 7fb67e200640 1 Processor -- start 2024-09-10T21:50:02.638 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.638+0000 7fb67e200640 1 -- start start 2024-09-10T21:50:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.638+0000 7fb67e200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6780c82d0 0x7fb6780cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.638+0000 7fb67e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6780d1e60 0x7fb67814ede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:02.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.639+0000 7fb67e200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6780d2a20 0x7fb67814f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.639+0000 7fb67e200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb6780c8e10 con 0x7fb6780d1e60 2024-09-10T21:50:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.639+0000 7fb67e200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb6780c8c90 con 0x7fb6780d2a20 2024-09-10T21:50:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.639+0000 7fb67e200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fb6780c8f90 con 0x7fb6780c82d0 2024-09-10T21:50:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.639+0000 7fb677e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6780d1e60 0x7fb67814ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.639+0000 7fb677e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6780d1e60 0x7fb67814ede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34246/0 (socket says 172.21.15.7:34246) 2024-09-10T21:50:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.639+0000 7fb677e00640 1 -- 172.21.15.7:0/2644118003 learned_addr learned my addr 172.21.15.7:0/2644118003 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.639+0000 7fb67ce00640 1 --2- 172.21.15.7:0/2644118003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6780c82d0 0x7fb6780cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.639+0000 7fb67d800640 1 --2- 172.21.15.7:0/2644118003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6780d2a20 0x7fb67814f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.639+0000 7fb677e00640 1 -- 172.21.15.7:0/2644118003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6780c82d0 msgr2=0x7fb6780cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.639+0000 7fb677e00640 1 --2- 172.21.15.7:0/2644118003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6780c82d0 0x7fb6780cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.639+0000 7fb677e00640 1 -- 172.21.15.7:0/2644118003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6780d2a20 msgr2=0x7fb67814f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.639+0000 7fb677e00640 1 --2- 172.21.15.7:0/2644118003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6780d2a20 0x7fb67814f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:02.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.639+0000 7fb677e00640 1 -- 172.21.15.7:0/2644118003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb67815b260 con 0x7fb6780d1e60 2024-09-10T21:50:02.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.640+0000 7fb677e00640 1 --2- 172.21.15.7:0/2644118003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6780d1e60 0x7fb67814ede0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fb664009fd0 tx=0x7fb66400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:02.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.640+0000 7fb674c00640 1 -- 172.21.15.7:0/2644118003 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66400ccc0 con 0x7fb6780d1e60 2024-09-10T21:50:02.642 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.640+0000 7fb674c00640 1 -- 172.21.15.7:0/2644118003 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb66400ee70 con 0x7fb6780d1e60 2024-09-10T21:50:02.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.640+0000 7fb674c00640 1 -- 172.21.15.7:0/2644118003 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb66400ce60 con 0x7fb6780d1e60 2024-09-10T21:50:02.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.640+0000 7fb67e200640 1 -- 172.21.15.7:0/2644118003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6780cde70 con 0x7fb6780d1e60 2024-09-10T21:50:02.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.640+0000 7fb67e200640 1 -- 172.21.15.7:0/2644118003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb678160280 con 0x7fb6780d1e60 2024-09-10T21:50:02.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.641+0000 7fb67e200640 1 -- 172.21.15.7:0/2644118003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6780751b0 con 0x7fb6780d1e60 2024-09-10T21:50:02.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.642+0000 7fb674c00640 1 -- 172.21.15.7:0/2644118003 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb664004280 con 0x7fb6780d1e60 2024-09-10T21:50:02.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.643+0000 7fb674c00640 1 --2- 172.21.15.7:0/2644118003 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb658077fa0 0x7fb65807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:02.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.643+0000 7fb674c00640 1 -- 172.21.15.7:0/2644118003 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fb66409a530 con 0x7fb6780d1e60 2024-09-10T21:50:02.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.644+0000 7fb67ce00640 1 --2- 172.21.15.7:0/2644118003 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb658077fa0 0x7fb65807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:02.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.644+0000 7fb67ce00640 1 --2- 172.21.15.7:0/2644118003 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb658077fa0 0x7fb65807a460 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0017c0 tx=0x7fb66c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:02.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.647+0000 7fb674c00640 1 -- 172.21.15.7:0/2644118003 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb6640629d0 con 0x7fb6780d1e60 2024-09-10T21:50:02.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.879+0000 7fb67e200640 1 -- 172.21.15.7:0/2644118003 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6780d05d0 con 0x7fb6780d1e60 2024-09-10T21:50:02.881 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.880+0000 7fb674c00640 1 -- 172.21.15.7:0/2644118003 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5219 (secure 0 0 0) 0x7fb664067400 con 0x7fb6780d1e60 2024-09-10T21:50:02.881 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:02.881 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":85,"btime":"2024-09-10T21:50:02:448373+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/431253133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":431253133},{"type":"v1","addr":"172.21.15.133:6836","nonce":431253133}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7521,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3692199178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3692199178},{"type":"v1","addr":"172.21.15.133:6837","nonce":3692199178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/752989865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":752989865},{"type":"v1","addr":"172.21.15.7:6834","nonce":752989865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:49:58.872982+0000","modified":"2024-09-10T21:50:01.884201+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3075619232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3075619232},{"type":"v1","addr":"172.21.15.7:6835","nonce":3075619232}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7527,"qdb_cluster":[7527]},"id":7}]} 2024-09-10T21:50:02.881 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 85 2024-09-10T21:50:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.882+0000 7fb67e200640 1 -- 172.21.15.7:0/2644118003 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb658077fa0 msgr2=0x7fb65807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:02.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.882+0000 7fb67e200640 1 --2- 172.21.15.7:0/2644118003 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb658077fa0 0x7fb65807a460 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fb66c0017c0 tx=0x7fb66c009040 comp rx=0 tx=0).stop 2024-09-10T21:50:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.882+0000 7fb67e200640 1 -- 172.21.15.7:0/2644118003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6780d1e60 msgr2=0x7fb67814ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.882+0000 7fb67e200640 1 --2- 172.21.15.7:0/2644118003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6780d1e60 0x7fb67814ede0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fb664009fd0 tx=0x7fb66400ec20 comp rx=0 tx=0).stop 2024-09-10T21:50:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.882+0000 7fb67e200640 1 -- 172.21.15.7:0/2644118003 shutdown_connections 2024-09-10T21:50:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.882+0000 7fb67e200640 1 --2- 172.21.15.7:0/2644118003 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb658077fa0 0x7fb65807a460 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.882+0000 7fb67e200640 1 --2- 172.21.15.7:0/2644118003 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb6780d2a20 0x7fb67814f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.882+0000 7fb67e200640 1 --2- 172.21.15.7:0/2644118003 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb6780d1e60 0x7fb67814ede0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.882+0000 7fb67e200640 1 --2- 172.21.15.7:0/2644118003 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb6780c82d0 0x7fb6780cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.882+0000 7fb67e200640 1 -- 172.21.15.7:0/2644118003 >> 172.21.15.7:0/2644118003 conn(0x7fb6780d3de0 msgr2=0x7fb6780b8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.882+0000 7fb67e200640 1 -- 172.21.15.7:0/2644118003 shutdown_connections 2024-09-10T21:50:02.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:02.882+0000 7fb67e200640 1 -- 172.21.15.7:0/2644118003 wait complete. 2024-09-10T21:50:02.899 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:50:03.060 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.059+0000 7f7828a00640 1 -- 172.21.15.7:0/3183594376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78240cd750 msgr2=0x7f78240ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:03.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.059+0000 7f7828a00640 1 --2- 172.21.15.7:0/3183594376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78240cd750 0x7f78240ccef0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f780c009830 tx=0x7f780c02a520 comp rx=0 tx=0).stop 2024-09-10T21:50:03.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.059+0000 7f7828a00640 1 -- 172.21.15.7:0/3183594376 shutdown_connections 2024-09-10T21:50:03.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.059+0000 7f7828a00640 1 --2- 172.21.15.7:0/3183594376 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f78240cd750 0x7f78240ccef0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.059+0000 7f7828a00640 1 -- 172.21.15.7:0/3183594376 >> 172.21.15.7:0/3183594376 conn(0x7f78240b16d0 msgr2=0x7f78240b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:03.061 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.059+0000 7f7828a00640 1 -- 172.21.15.7:0/3183594376 shutdown_connections 2024-09-10T21:50:03.062 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.059+0000 7f7828a00640 1 -- 172.21.15.7:0/3183594376 wait complete. 2024-09-10T21:50:03.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.060+0000 7f7828a00640 1 Processor -- start 2024-09-10T21:50:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.061+0000 7f7828a00640 1 -- start start 2024-09-10T21:50:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.061+0000 7f7828a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78240cd750 0x7f78240c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.061+0000 7f7828a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f782415bc90 0x7f782414ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.061+0000 7f7828a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782415fda0 0x7f78241486c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.062+0000 7f7828a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7824070080 con 0x7f782415bc90 2024-09-10T21:50:03.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.062+0000 7f7828a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f782406ff00 con 0x7f782415fda0 2024-09-10T21:50:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.062+0000 7f7828a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f7824070200 con 0x7f78240cd750 2024-09-10T21:50:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.062+0000 7f7819400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f782415bc90 0x7f782414ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.062+0000 7f7819400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f782415bc90 0x7f782414ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34254/0 (socket says 172.21.15.7:34254) 2024-09-10T21:50:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.062+0000 7f7819400640 1 -- 172.21.15.7:0/1893427585 learned_addr learned my addr 172.21.15.7:0/1893427585 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.062+0000 7f7822800640 1 --2- 172.21.15.7:0/1893427585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782415fda0 0x7f78241486c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:03.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.062+0000 7f7821e00640 1 --2- 172.21.15.7:0/1893427585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78240cd750 0x7f78240c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.062+0000 7f7819400640 1 -- 172.21.15.7:0/1893427585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78240cd750 msgr2=0x7f78240c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.062+0000 7f7819400640 1 --2- 172.21.15.7:0/1893427585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78240cd750 0x7f78240c9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.062+0000 7f7819400640 1 -- 172.21.15.7:0/1893427585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782415fda0 msgr2=0x7f78241486c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.062+0000 7f7819400640 1 --2- 172.21.15.7:0/1893427585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782415fda0 0x7f78241486c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.062+0000 7f7819400640 1 -- 172.21.15.7:0/1893427585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78240c85c0 con 0x7f782415bc90 2024-09-10T21:50:03.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.062+0000 7f7821e00640 1 --2- 172.21.15.7:0/1893427585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78240cd750 0x7f78240c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:50:03.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.063+0000 7f7819400640 1 --2- 172.21.15.7:0/1893427585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f782415bc90 0x7f782414ba30 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f780c009950 tx=0x7f780c032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:03.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.063+0000 7f781aa00640 1 -- 172.21.15.7:0/1893427585 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c040070 con 0x7f782415bc90 2024-09-10T21:50:03.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.063+0000 7f781aa00640 1 -- 172.21.15.7:0/1893427585 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f780c005310 con 0x7f782415bc90 2024-09-10T21:50:03.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.063+0000 7f781aa00640 1 -- 172.21.15.7:0/1893427585 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c005c90 con 0x7f782415bc90 2024-09-10T21:50:03.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.063+0000 7f7828a00640 1 -- 172.21.15.7:0/1893427585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78240c88a0 con 0x7f782415bc90 2024-09-10T21:50:03.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.063+0000 7f7828a00640 1 -- 172.21.15.7:0/1893427585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78240c8e60 con 0x7f782415bc90 2024-09-10T21:50:03.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.064+0000 7f7828a00640 1 -- 172.21.15.7:0/1893427585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78240706b0 con 0x7f782415bc90 2024-09-10T21:50:03.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.065+0000 7f781aa00640 1 -- 172.21.15.7:0/1893427585 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f780c0049b0 con 0x7f782415bc90 2024-09-10T21:50:03.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.066+0000 7f781aa00640 1 --2- 172.21.15.7:0/1893427585 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7800077f70 0x7f780007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:03.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.066+0000 7f781aa00640 1 -- 172.21.15.7:0/1893427585 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f780c0bb380 con 0x7f782415bc90 2024-09-10T21:50:03.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.066+0000 7f7821e00640 1 --2- 172.21.15.7:0/1893427585 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7800077f70 0x7f780007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:03.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.067+0000 7f7821e00640 1 --2- 172.21.15.7:0/1893427585 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7800077f70 0x7f780007a430 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7814001710 tx=0x7f7814009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:03.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.070+0000 7f781aa00640 1 -- 172.21.15.7:0/1893427585 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f780c083de0 con 0x7f782415bc90 2024-09-10T21:50:03.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.266+0000 7f7828a00640 1 -- 172.21.15.7:0/1893427585 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f782414bf70 con 0x7f782415bc90 2024-09-10T21:50:03.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.268+0000 7f781aa00640 1 -- 172.21.15.7:0/1893427585 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) ==== 74+0+16635 (secure 0 0 0) 0x7f780c0836e0 con 0x7f782415bc90 2024-09-10T21:50:03.269 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:03.269 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":101,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:50:00.434986+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:49:56.657911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T21:49:57.828979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:50:03.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.270+0000 7f7828a00640 1 -- 172.21.15.7:0/1893427585 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7800077f70 msgr2=0x7f780007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:03.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.270+0000 7f7828a00640 1 --2- 172.21.15.7:0/1893427585 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7800077f70 0x7f780007a430 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7814001710 tx=0x7f7814009040 comp rx=0 tx=0).stop 2024-09-10T21:50:03.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.270+0000 7f7828a00640 1 -- 172.21.15.7:0/1893427585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f782415bc90 msgr2=0x7f782414ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.270+0000 7f7828a00640 1 --2- 172.21.15.7:0/1893427585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f782415bc90 0x7f782414ba30 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f780c009950 tx=0x7f780c032c10 comp rx=0 tx=0).stop 2024-09-10T21:50:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.270+0000 7f7828a00640 1 -- 172.21.15.7:0/1893427585 shutdown_connections 2024-09-10T21:50:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.270+0000 7f7828a00640 1 --2- 172.21.15.7:0/1893427585 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7800077f70 0x7f780007a430 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.270+0000 7f7828a00640 1 --2- 172.21.15.7:0/1893427585 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f782415fda0 0x7f78241486c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.270+0000 7f7828a00640 1 --2- 172.21.15.7:0/1893427585 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f782415bc90 0x7f782414ba30 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.270+0000 7f7828a00640 1 --2- 172.21.15.7:0/1893427585 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f78240cd750 0x7f78240c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.270+0000 7f7828a00640 1 -- 172.21.15.7:0/1893427585 >> 172.21.15.7:0/1893427585 conn(0x7f78240b16d0 msgr2=0x7f7824072360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.270+0000 7f7828a00640 1 -- 172.21.15.7:0/1893427585 shutdown_connections 2024-09-10T21:50:03.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.270+0000 7f7828a00640 1 -- 172.21.15.7:0/1893427585 wait complete. 2024-09-10T21:50:03.288 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:50:03.288 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:50:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.447+0000 7fc29ec00640 1 -- 172.21.15.7:0/1670505701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc298145540 msgr2=0x7fc2980b4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.447+0000 7fc29ec00640 1 --2- 172.21.15.7:0/1670505701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc298145540 0x7fc2980b4580 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fc28c005fd0 tx=0x7fc28c02a940 comp rx=0 tx=0).stop 2024-09-10T21:50:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.448+0000 7fc29ec00640 1 -- 172.21.15.7:0/1670505701 shutdown_connections 2024-09-10T21:50:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.448+0000 7fc29ec00640 1 --2- 172.21.15.7:0/1670505701 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc298145540 0x7fc2980b4580 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.448+0000 7fc29ec00640 1 -- 172.21.15.7:0/1670505701 >> 172.21.15.7:0/1670505701 conn(0x7fc2980b1d70 msgr2=0x7fc2980cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.448+0000 7fc29ec00640 1 -- 172.21.15.7:0/1670505701 shutdown_connections 2024-09-10T21:50:03.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.448+0000 7fc29ec00640 1 -- 172.21.15.7:0/1670505701 wait complete. 2024-09-10T21:50:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.449+0000 7fc29ec00640 1 Processor -- start 2024-09-10T21:50:03.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.449+0000 7fc29ec00640 1 -- start start 2024-09-10T21:50:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.450+0000 7fc29ec00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2980b3150 0x7fc2980cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.450+0000 7fc29ec00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2980b4ac0 0x7fc2980c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:03.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.450+0000 7fc29ec00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc298145540 0x7fc29815e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.450+0000 7fc29ec00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc2980cef80 con 0x7fc298145540 2024-09-10T21:50:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.450+0000 7fc29ec00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc2980cee00 con 0x7fc2980b4ac0 2024-09-10T21:50:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.450+0000 7fc29ec00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc2980cf100 con 0x7fc2980b3150 2024-09-10T21:50:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.451+0000 7fc29e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc298145540 0x7fc29815e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.451+0000 7fc29e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc298145540 0x7fc29815e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34282/0 (socket says 172.21.15.7:34282) 2024-09-10T21:50:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.451+0000 7fc29e200640 1 -- 172.21.15.7:0/211881490 learned_addr learned my addr 172.21.15.7:0/211881490 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.451+0000 7fc29d800640 1 --2- 172.21.15.7:0/211881490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2980b3150 0x7fc2980cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:03.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.451+0000 7fc29ce00640 1 --2- 172.21.15.7:0/211881490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2980b4ac0 0x7fc2980c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.451+0000 7fc29e200640 1 -- 172.21.15.7:0/211881490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2980b3150 msgr2=0x7fc2980cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.451+0000 7fc29e200640 1 --2- 172.21.15.7:0/211881490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2980b3150 0x7fc2980cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.451+0000 7fc29e200640 1 -- 172.21.15.7:0/211881490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2980b4ac0 msgr2=0x7fc2980c53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.451+0000 7fc29e200640 1 --2- 172.21.15.7:0/211881490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2980b4ac0 0x7fc2980c53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.451+0000 7fc29e200640 1 -- 172.21.15.7:0/211881490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2980c5900 con 0x7fc298145540 2024-09-10T21:50:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.451+0000 7fc29ce00640 1 --2- 172.21.15.7:0/211881490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2980b4ac0 0x7fc2980c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:50:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.451+0000 7fc29e200640 1 --2- 172.21.15.7:0/211881490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc298145540 0x7fc29815e3d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fc29000c430 tx=0x7fc29000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.452+0000 7fc295600640 1 -- 172.21.15.7:0/211881490 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc290017070 con 0x7fc298145540 2024-09-10T21:50:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.452+0000 7fc295600640 1 -- 172.21.15.7:0/211881490 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2900053c0 con 0x7fc298145540 2024-09-10T21:50:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.452+0000 7fc295600640 1 -- 172.21.15.7:0/211881490 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc290003850 con 0x7fc298145540 2024-09-10T21:50:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.452+0000 7fc29ec00640 1 -- 172.21.15.7:0/211881490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2980c5b80 con 0x7fc298145540 2024-09-10T21:50:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.452+0000 7fc29ec00640 1 -- 172.21.15.7:0/211881490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2980c6140 con 0x7fc298145540 2024-09-10T21:50:03.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.453+0000 7fc29ec00640 1 -- 172.21.15.7:0/211881490 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc298144da0 con 0x7fc298145540 2024-09-10T21:50:03.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.454+0000 7fc295600640 1 -- 172.21.15.7:0/211881490 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc2900039f0 con 0x7fc298145540 2024-09-10T21:50:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.455+0000 7fc295600640 1 --2- 172.21.15.7:0/211881490 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc278077f70 0x7fc27807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.455+0000 7fc295600640 1 -- 172.21.15.7:0/211881490 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fc290099430 con 0x7fc298145540 2024-09-10T21:50:03.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.455+0000 7fc29d800640 1 --2- 172.21.15.7:0/211881490 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc278077f70 0x7fc27807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:03.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.456+0000 7fc29d800640 1 --2- 172.21.15.7:0/211881490 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc278077f70 0x7fc27807a430 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc28c034000 tx=0x7fc28c004ab0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:03.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.461+0000 7fc295600640 1 -- 172.21.15.7:0/211881490 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc298144da0 con 0x7fc298145540 2024-09-10T21:50:03.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.656+0000 7fc29ec00640 1 -- 172.21.15.7:0/211881490 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc2980cf5b0 con 0x7fc278077f70 2024-09-10T21:50:03.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.662+0000 7fc295600640 1 -- 172.21.15.7:0/211881490 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+243083 (secure 0 0 0) 0x7fc2980cf5b0 con 0x7fc278077f70 2024-09-10T21:50:03.665 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:03.667 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:50:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.666+0000 7fc29ec00640 1 -- 172.21.15.7:0/211881490 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc278077f70 msgr2=0x7fc27807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.666+0000 7fc29ec00640 1 --2- 172.21.15.7:0/211881490 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc278077f70 0x7fc27807a430 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc28c034000 tx=0x7fc28c004ab0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.666+0000 7fc29ec00640 1 -- 172.21.15.7:0/211881490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc298145540 msgr2=0x7fc29815e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.666+0000 7fc29ec00640 1 --2- 172.21.15.7:0/211881490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc298145540 0x7fc29815e3d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fc29000c430 tx=0x7fc29000c8f0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.666+0000 7fc29ec00640 1 -- 172.21.15.7:0/211881490 shutdown_connections 2024-09-10T21:50:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.666+0000 7fc29ec00640 1 --2- 172.21.15.7:0/211881490 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc278077f70 0x7fc27807a430 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.666+0000 7fc29ec00640 1 --2- 172.21.15.7:0/211881490 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2980b4ac0 0x7fc2980c53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.666+0000 7fc29ec00640 1 --2- 172.21.15.7:0/211881490 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc298145540 0x7fc29815e3d0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.666+0000 7fc29ec00640 1 --2- 172.21.15.7:0/211881490 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2980b3150 0x7fc2980cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.666+0000 7fc29ec00640 1 -- 172.21.15.7:0/211881490 >> 172.21.15.7:0/211881490 conn(0x7fc2980b1d70 msgr2=0x7fc29815f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.666+0000 7fc29ec00640 1 -- 172.21.15.7:0/211881490 shutdown_connections 2024-09-10T21:50:03.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.666+0000 7fc29ec00640 1 -- 172.21.15.7:0/211881490 wait complete. 2024-09-10T21:50:03.682 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-09-10T21:50:01.980950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590480,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":106,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77,"ondisk_log_size":77,"up":100,"acting":100,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":268480,"kb_used_data":10616,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754706240,"statfs":{"total":773094113280,"available":772819189760,"internally_reserved":0,"allocated":10870784,"data_stored":5321650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.565053"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454884+0000","last_change":"2024-09-10T21:49:59.458482+0000","last_active":"2024-09-10T21:50:00.454884+0000","last_peered":"2024-09-10T21:50:00.454884+0000","last_clean":"2024-09-10T21:50:00.454884+0000","last_became_active":"2024-09-10T21:49:59.458124+0000","last_became_peered":"2024-09-10T21:49:59.458124+0000","last_unstale":"2024-09-10T21:50:00.454884+0000","last_undegraded":"2024-09-10T21:50:00.454884+0000","last_fullsized":"2024-09-10T21:50:00.454884+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:01.978617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454347+0000","last_change":"2024-09-10T21:49:59.441978+0000","last_active":"2024-09-10T21:50:00.454347+0000","last_peered":"2024-09-10T21:50:00.454347+0000","last_clean":"2024-09-10T21:50:00.454347+0000","last_became_active":"2024-09-10T21:49:59.441636+0000","last_became_peered":"2024-09-10T21:49:59.441636+0000","last_unstale":"2024-09-10T21:50:00.454347+0000","last_undegraded":"2024-09-10T21:50:00.454347+0000","last_fullsized":"2024-09-10T21:50:00.454347+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:28.828696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454035+0000","last_change":"2024-09-10T21:49:59.445847+0000","last_active":"2024-09-10T21:50:00.454035+0000","last_peered":"2024-09-10T21:50:00.454035+0000","last_clean":"2024-09-10T21:50:00.454035+0000","last_became_active":"2024-09-10T21:49:59.445562+0000","last_became_peered":"2024-09-10T21:49:59.445562+0000","last_unstale":"2024-09-10T21:50:00.454035+0000","last_undegraded":"2024-09-10T21:50:00.454035+0000","last_fullsized":"2024-09-10T21:50:00.454035+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:01.762660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454217+0000","last_change":"2024-09-10T21:49:59.450906+0000","last_active":"2024-09-10T21:50:00.454217+0000","last_peered":"2024-09-10T21:50:00.454217+0000","last_clean":"2024-09-10T21:50:00.454217+0000","last_became_active":"2024-09-10T21:49:59.450618+0000","last_became_peered":"2024-09-10T21:49:59.450618+0000","last_unstale":"2024-09-10T21:50:00.454217+0000","last_undegraded":"2024-09-10T21:50:00.454217+0000","last_fullsized":"2024-09-10T21:50:00.454217+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:31.447063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454658+0000","last_change":"2024-09-10T21:49:59.456269+0000","last_active":"2024-09-10T21:50:00.454658+0000","last_peered":"2024-09-10T21:50:00.454658+0000","last_clean":"2024-09-10T21:50:00.454658+0000","last_became_active":"2024-09-10T21:49:59.456006+0000","last_became_peered":"2024-09-10T21:49:59.456006+0000","last_unstale":"2024-09-10T21:50:00.454658+0000","last_undegraded":"2024-09-10T21:50:00.454658+0000","last_fullsized":"2024-09-10T21:50:00.454658+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:04:57.089080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.453984+0000","last_change":"2024-09-10T21:49:59.467852+0000","last_active":"2024-09-10T21:50:00.453984+0000","last_peered":"2024-09-10T21:50:00.453984+0000","last_clean":"2024-09-10T21:50:00.453984+0000","last_became_active":"2024-09-10T21:49:59.467486+0000","last_became_peered":"2024-09-10T21:49:59.467486+0000","last_unstale":"2024-09-10T21:50:00.453984+0000","last_undegraded":"2024-09-10T21:50:00.453984+0000","last_fullsized":"2024-09-10T21:50:00.453984+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:18.562220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454775+0000","last_change":"2024-09-10T21:49:59.458686+0000","last_active":"2024-09-10T21:50:00.454775+0000","last_peered":"2024-09-10T21:50:00.454775+0000","last_clean":"2024-09-10T21:50:00.454775+0000","last_became_active":"2024-09-10T21:49:59.458359+0000","last_became_peered":"2024-09-10T21:49:59.458359+0000","last_unstale":"2024-09-10T21:50:00.454775+0000","last_undegraded":"2024-09-10T21:50:00.454775+0000","last_fullsized":"2024-09-10T21:50:00.454775+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:25.438731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455007+0000","last_change":"2024-09-10T21:49:59.459014+0000","last_active":"2024-09-10T21:50:00.455007+0000","last_peered":"2024-09-10T21:50:00.455007+0000","last_clean":"2024-09-10T21:50:00.455007+0000","last_became_active":"2024-09-10T21:49:59.458724+0000","last_became_peered":"2024-09-10T21:49:59.458724+0000","last_unstale":"2024-09-10T21:50:00.455007+0000","last_undegraded":"2024-09-10T21:50:00.455007+0000","last_fullsized":"2024-09-10T21:50:00.455007+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:42.911939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454546+0000","last_change":"2024-09-10T21:49:59.469560+0000","last_active":"2024-09-10T21:50:00.454546+0000","last_peered":"2024-09-10T21:50:00.454546+0000","last_clean":"2024-09-10T21:50:00.454546+0000","last_became_active":"2024-09-10T21:49:59.469262+0000","last_became_peered":"2024-09-10T21:49:59.469262+0000","last_unstale":"2024-09-10T21:50:00.454546+0000","last_undegraded":"2024-09-10T21:50:00.454546+0000","last_fullsized":"2024-09-10T21:50:00.454546+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:14.072179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454437+0000","last_change":"2024-09-10T21:49:59.458778+0000","last_active":"2024-09-10T21:50:00.454437+0000","last_peered":"2024-09-10T21:50:00.454437+0000","last_clean":"2024-09-10T21:50:00.454437+0000","last_became_active":"2024-09-10T21:49:59.458043+0000","last_became_peered":"2024-09-10T21:49:59.458043+0000","last_unstale":"2024-09-10T21:50:00.454437+0000","last_undegraded":"2024-09-10T21:50:00.454437+0000","last_fullsized":"2024-09-10T21:50:00.454437+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:23.126640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454037+0000","last_change":"2024-09-10T21:49:59.449917+0000","last_active":"2024-09-10T21:50:00.454037+0000","last_peered":"2024-09-10T21:50:00.454037+0000","last_clean":"2024-09-10T21:50:00.454037+0000","last_became_active":"2024-09-10T21:49:59.449599+0000","last_became_peered":"2024-09-10T21:49:59.449599+0000","last_unstale":"2024-09-10T21:50:00.454037+0000","last_undegraded":"2024-09-10T21:50:00.454037+0000","last_fullsized":"2024-09-10T21:50:00.454037+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:18.368238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.423984+0000","last_change":"2024-09-10T21:49:57.423984+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.423984+0000","last_undegraded":"2024-09-10T21:49:57.423984+0000","last_fullsized":"2024-09-10T21:49:57.423984+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455408+0000","last_change":"2024-09-10T21:49:59.449838+0000","last_active":"2024-09-10T21:50:00.455408+0000","last_peered":"2024-09-10T21:50:00.455408+0000","last_clean":"2024-09-10T21:50:00.455408+0000","last_became_active":"2024-09-10T21:49:59.449158+0000","last_became_peered":"2024-09-10T21:49:59.449158+0000","last_unstale":"2024-09-10T21:50:00.455408+0000","last_undegraded":"2024-09-10T21:50:00.455408+0000","last_fullsized":"2024-09-10T21:50:00.455408+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:52.993443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"100'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454310+0000","last_change":"2024-09-10T21:49:58.445548+0000","last_active":"2024-09-10T21:50:00.454310+0000","last_peered":"2024-09-10T21:50:00.454310+0000","last_clean":"2024-09-10T21:50:00.454310+0000","last_became_active":"2024-09-10T21:49:58.445230+0000","last_became_peered":"2024-09-10T21:49:58.445230+0000","last_unstale":"2024-09-10T21:50:00.454310+0000","last_undegraded":"2024-09-10T21:50:00.454310+0000","last_fullsized":"2024-09-10T21:50:00.454310+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:36.258778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.423732+0000","last_change":"2024-09-10T21:49:57.423732+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.423732+0000","last_undegraded":"2024-09-10T21:49:57.423732+0000","last_fullsized":"2024-09-10T21:49:57.423732+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.423914+0000","last_change":"2024-09-10T21:49:57.423914+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.423914+0000","last_undegraded":"2024-09-10T21:49:57.423914+0000","last_fullsized":"2024-09-10T21:49:57.423914+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.421988+0000","last_change":"2024-09-10T21:49:57.421988+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.421988+0000","last_undegraded":"2024-09-10T21:49:57.421988+0000","last_fullsized":"2024-09-10T21:49:57.421988+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454180+0000","last_change":"2024-09-10T21:49:59.453917+0000","last_active":"2024-09-10T21:50:00.454180+0000","last_peered":"2024-09-10T21:50:00.454180+0000","last_clean":"2024-09-10T21:50:00.454180+0000","last_became_active":"2024-09-10T21:49:59.453588+0000","last_became_peered":"2024-09-10T21:49:59.453588+0000","last_unstale":"2024-09-10T21:50:00.454180+0000","last_undegraded":"2024-09-10T21:50:00.454180+0000","last_fullsized":"2024-09-10T21:50:00.454180+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:25.785285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.423171+0000","last_change":"2024-09-10T21:49:57.423171+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.423171+0000","last_undegraded":"2024-09-10T21:49:57.423171+0000","last_fullsized":"2024-09-10T21:49:57.423171+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.453963+0000","last_change":"2024-09-10T21:49:59.472501+0000","last_active":"2024-09-10T21:50:00.453963+0000","last_peered":"2024-09-10T21:50:00.453963+0000","last_clean":"2024-09-10T21:50:00.453963+0000","last_became_active":"2024-09-10T21:49:59.471592+0000","last_became_peered":"2024-09-10T21:49:59.471592+0000","last_unstale":"2024-09-10T21:50:00.453963+0000","last_undegraded":"2024-09-10T21:50:00.453963+0000","last_fullsized":"2024-09-10T21:50:00.453963+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:18.082636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.419337+0000","last_change":"2024-09-10T21:49:57.419337+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.419337+0000","last_undegraded":"2024-09-10T21:49:57.419337+0000","last_fullsized":"2024-09-10T21:49:57.419337+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.421916+0000","last_change":"2024-09-10T21:49:57.421916+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.421916+0000","last_undegraded":"2024-09-10T21:49:57.421916+0000","last_fullsized":"2024-09-10T21:49:57.421916+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.421508+0000","last_change":"2024-09-10T21:49:57.421508+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.421508+0000","last_undegraded":"2024-09-10T21:49:57.421508+0000","last_fullsized":"2024-09-10T21:49:57.421508+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454917+0000","last_change":"2024-09-10T21:49:58.445035+0000","last_active":"2024-09-10T21:50:00.454917+0000","last_peered":"2024-09-10T21:50:00.454917+0000","last_clean":"2024-09-10T21:50:00.454917+0000","last_became_active":"2024-09-10T21:49:58.444734+0000","last_became_peered":"2024-09-10T21:49:58.444734+0000","last_unstale":"2024-09-10T21:50:00.454917+0000","last_undegraded":"2024-09-10T21:50:00.454917+0000","last_fullsized":"2024-09-10T21:50:00.454917+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:49.874922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"100'3","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454652+0000","last_change":"2024-09-10T21:49:58.442254+0000","last_active":"2024-09-10T21:50:00.454652+0000","last_peered":"2024-09-10T21:50:00.454652+0000","last_clean":"2024-09-10T21:50:00.454652+0000","last_became_active":"2024-09-10T21:49:58.441915+0000","last_became_peered":"2024-09-10T21:49:58.441915+0000","last_unstale":"2024-09-10T21:50:00.454652+0000","last_undegraded":"2024-09-10T21:50:00.454652+0000","last_fullsized":"2024-09-10T21:50:00.454652+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:25.667184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454527+0000","last_change":"2024-09-10T21:49:59.466948+0000","last_active":"2024-09-10T21:50:00.454527+0000","last_peered":"2024-09-10T21:50:00.454527+0000","last_clean":"2024-09-10T21:50:00.454527+0000","last_became_active":"2024-09-10T21:49:59.466340+0000","last_became_peered":"2024-09-10T21:49:59.466340+0000","last_unstale":"2024-09-10T21:50:00.454527+0000","last_undegraded":"2024-09-10T21:50:00.454527+0000","last_fullsized":"2024-09-10T21:50:00.454527+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:12:41.966095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.419806+0000","last_change":"2024-09-10T21:49:57.419806+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.419806+0000","last_undegraded":"2024-09-10T21:49:57.419806+0000","last_fullsized":"2024-09-10T21:49:57.419806+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454373+0000","last_change":"2024-09-10T21:49:59.450770+0000","last_active":"2024-09-10T21:50:00.454373+0000","last_peered":"2024-09-10T21:50:00.454373+0000","last_clean":"2024-09-10T21:50:00.454373+0000","last_became_active":"2024-09-10T21:49:59.450460+0000","last_became_peered":"2024-09-10T21:49:59.450460+0000","last_unstale":"2024-09-10T21:50:00.454373+0000","last_undegraded":"2024-09-10T21:50:00.454373+0000","last_fullsized":"2024-09-10T21:50:00.454373+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:57.481410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.426416+0000","last_change":"2024-09-10T21:49:57.426416+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.426416+0000","last_undegraded":"2024-09-10T21:49:57.426416+0000","last_fullsized":"2024-09-10T21:49:57.426416+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.427865+0000","last_change":"2024-09-10T21:49:57.427865+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.427865+0000","last_undegraded":"2024-09-10T21:49:57.427865+0000","last_fullsized":"2024-09-10T21:49:57.427865+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.418591+0000","last_change":"2024-09-10T21:49:57.418591+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.418591+0000","last_undegraded":"2024-09-10T21:49:57.418591+0000","last_fullsized":"2024-09-10T21:49:57.418591+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.421431+0000","last_change":"2024-09-10T21:49:57.421431+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.421431+0000","last_undegraded":"2024-09-10T21:49:57.421431+0000","last_fullsized":"2024-09-10T21:49:57.421431+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.420114+0000","last_change":"2024-09-10T21:49:57.420114+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.420114+0000","last_undegraded":"2024-09-10T21:49:57.420114+0000","last_fullsized":"2024-09-10T21:49:57.420114+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.418979+0000","last_change":"2024-09-10T21:49:57.418979+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.418979+0000","last_undegraded":"2024-09-10T21:49:57.418979+0000","last_fullsized":"2024-09-10T21:49:57.418979+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454662+0000","last_change":"2024-09-10T21:49:59.454957+0000","last_active":"2024-09-10T21:50:00.454662+0000","last_peered":"2024-09-10T21:50:00.454662+0000","last_clean":"2024-09-10T21:50:00.454662+0000","last_became_active":"2024-09-10T21:49:59.454428+0000","last_became_peered":"2024-09-10T21:49:59.454428+0000","last_unstale":"2024-09-10T21:50:00.454662+0000","last_undegraded":"2024-09-10T21:50:00.454662+0000","last_fullsized":"2024-09-10T21:50:00.454662+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:54.446738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.422779+0000","last_change":"2024-09-10T21:49:57.422779+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.422779+0000","last_undegraded":"2024-09-10T21:49:57.422779+0000","last_fullsized":"2024-09-10T21:49:57.422779+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.424603+0000","last_change":"2024-09-10T21:49:57.424603+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.424603+0000","last_undegraded":"2024-09-10T21:49:57.424603+0000","last_fullsized":"2024-09-10T21:49:57.424603+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.418970+0000","last_change":"2024-09-10T21:49:57.418970+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.418970+0000","last_undegraded":"2024-09-10T21:49:57.418970+0000","last_fullsized":"2024-09-10T21:49:57.418970+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.418934+0000","last_change":"2024-09-10T21:49:57.418934+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.418934+0000","last_undegraded":"2024-09-10T21:49:57.418934+0000","last_fullsized":"2024-09-10T21:49:57.418934+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.422862+0000","last_change":"2024-09-10T21:49:57.422862+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.422862+0000","last_undegraded":"2024-09-10T21:49:57.422862+0000","last_fullsized":"2024-09-10T21:49:57.422862+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.421064+0000","last_change":"2024-09-10T21:49:57.421064+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.421064+0000","last_undegraded":"2024-09-10T21:49:57.421064+0000","last_fullsized":"2024-09-10T21:49:57.421064+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.422852+0000","last_change":"2024-09-10T21:49:57.422852+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.422852+0000","last_undegraded":"2024-09-10T21:49:57.422852+0000","last_fullsized":"2024-09-10T21:49:57.422852+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.424710+0000","last_change":"2024-09-10T21:49:57.424710+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.424710+0000","last_undegraded":"2024-09-10T21:49:57.424710+0000","last_fullsized":"2024-09-10T21:49:57.424710+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.418520+0000","last_change":"2024-09-10T21:49:57.418520+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.418520+0000","last_undegraded":"2024-09-10T21:49:57.418520+0000","last_fullsized":"2024-09-10T21:49:57.418520+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.423268+0000","last_change":"2024-09-10T21:49:57.423268+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.423268+0000","last_undegraded":"2024-09-10T21:49:57.423268+0000","last_fullsized":"2024-09-10T21:49:57.423268+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.418988+0000","last_change":"2024-09-10T21:49:57.418988+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.418988+0000","last_undegraded":"2024-09-10T21:49:57.418988+0000","last_fullsized":"2024-09-10T21:49:57.418988+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.453856+0000","last_change":"2024-09-10T21:49:58.453503+0000","last_active":"2024-09-10T21:50:00.453856+0000","last_peered":"2024-09-10T21:50:00.453856+0000","last_clean":"2024-09-10T21:50:00.453856+0000","last_became_active":"2024-09-10T21:49:58.453200+0000","last_became_peered":"2024-09-10T21:49:58.453200+0000","last_unstale":"2024-09-10T21:50:00.453856+0000","last_undegraded":"2024-09-10T21:50:00.453856+0000","last_fullsized":"2024-09-10T21:50:00.453856+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:14.389900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":1,"reported_epoch":98,"state":"creating+peering","last_fresh":"2024-09-10T21:49:57.426518+0000","last_change":"2024-09-10T21:49:57.426518+0000","last_active":"2024-09-10T21:49:57.406801+0000","last_peered":"2024-09-10T21:49:57.406801+0000","last_clean":"2024-09-10T21:49:57.406801+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:57.426518+0000","last_undegraded":"2024-09-10T21:49:57.426518+0000","last_fullsized":"2024-09-10T21:49:57.426518+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-10T21:49:58.415331+0000","last_change":"2024-09-10T21:49:58.415331+0000","last_active":"2024-09-10T21:49:58.415331+0000","last_peered":"2024-09-10T21:49:58.415331+0000","last_clean":"2024-09-10T21:49:58.415331+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:49:58.415331+0000","last_undegraded":"2024-09-10T21:49:58.415331+0000","last_fullsized":"2024-09-10T21:49:58.415331+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.f","version":"100'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454217+0000","last_change":"2024-09-10T21:49:58.448116+0000","last_active":"2024-09-10T21:50:00.454217+0000","last_peered":"2024-09-10T21:50:00.454217+0000","last_clean":"2024-09-10T21:50:00.454217+0000","last_became_active":"2024-09-10T21:49:58.447745+0000","last_became_peered":"2024-09-10T21:49:58.447745+0000","last_unstale":"2024-09-10T21:50:00.454217+0000","last_undegraded":"2024-09-10T21:50:00.454217+0000","last_fullsized":"2024-09-10T21:50:00.454217+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:58.352646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":258,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454655+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:50:00.454655+0000","last_peered":"2024-09-10T21:50:00.454655+0000","last_clean":"2024-09-10T21:50:00.454655+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:50:00.454655+0000","last_undegraded":"2024-09-10T21:50:00.454655+0000","last_fullsized":"2024-09-10T21:50:00.454655+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":34,"acting":34,"num_store_stats":2},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509540,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1191936,"data_stored":509650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509540,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1191936,"data_stored":509650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32368,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339472,"statfs":{"total":96636764160,"available":96603619328,"internally_reserved":0,"allocated":1191936,"data_stored":509650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509539,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32364,"kb_used_data":1176,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339476,"statfs":{"total":96636764160,"available":96603623424,"internally_reserved":0,"allocated":1204224,"data_stored":515467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509541,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32352,"kb_used_data":1164,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339488,"statfs":{"total":96636764160,"available":96603635712,"internally_reserved":0,"allocated":1191936,"data_stored":509650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509539,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37104,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334736,"statfs":{"total":96636764160,"available":96598769664,"internally_reserved":0,"allocated":1798144,"data_stored":1105835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542243,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1843200,"data_stored":1125297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542244,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36580,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335260,"statfs":{"total":96636764160,"available":96599306240,"internally_reserved":0,"allocated":1257472,"data_stored":536451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:50:03.687 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:50:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.881+0000 7fe5af000640 1 -- 172.21.15.7:0/1928029158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5a80cd750 msgr2=0x7fe5a80ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.881+0000 7fe5af000640 1 --2- 172.21.15.7:0/1928029158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5a80cd750 0x7fe5a80ccef0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fe5a815fa90 tx=0x7fe59002a530 comp rx=0 tx=0).stop 2024-09-10T21:50:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.881+0000 7fe5af000640 1 -- 172.21.15.7:0/1928029158 shutdown_connections 2024-09-10T21:50:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.881+0000 7fe5af000640 1 --2- 172.21.15.7:0/1928029158 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe5a815fda0 0x7fe5a81601d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.881+0000 7fe5af000640 1 --2- 172.21.15.7:0/1928029158 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5a80cd750 0x7fe5a80ccef0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.881+0000 7fe5af000640 1 -- 172.21.15.7:0/1928029158 >> 172.21.15.7:0/1928029158 conn(0x7fe5a80b16d0 msgr2=0x7fe5a80b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.881+0000 7fe5af000640 1 -- 172.21.15.7:0/1928029158 shutdown_connections 2024-09-10T21:50:03.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.881+0000 7fe5af000640 1 -- 172.21.15.7:0/1928029158 wait complete. 2024-09-10T21:50:03.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.883+0000 7fe5af000640 1 Processor -- start 2024-09-10T21:50:03.884 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.883+0000 7fe5af000640 1 -- start start 2024-09-10T21:50:03.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.884+0000 7fe5af000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5a80cd750 0x7fe5a80b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:03.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.884+0000 7fe5af000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe5a815bc90 0x7fe5a80cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:03.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.884+0000 7fe5af000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5a815fda0 0x7fe5a80cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.884+0000 7fe5af000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a806fe70 con 0x7fe5a80cd750 2024-09-10T21:50:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.884+0000 7fe5af000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a806fcf0 con 0x7fe5a815fda0 2024-09-10T21:50:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.884+0000 7fe5af000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe5a806fff0 con 0x7fe5a815bc90 2024-09-10T21:50:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.884+0000 7fe5a7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5a80cd750 0x7fe5a80b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.884+0000 7fe5a7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5a80cd750 0x7fe5a80b3400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58764/0 (socket says 172.21.15.7:58764) 2024-09-10T21:50:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.884+0000 7fe5a7e00640 1 -- 172.21.15.7:0/663871890 learned_addr learned my addr 172.21.15.7:0/663871890 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.885+0000 7fe5ad000640 1 --2- 172.21.15.7:0/663871890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5a815fda0 0x7fe5a80cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.885+0000 7fe59f400640 1 --2- 172.21.15.7:0/663871890 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe5a815bc90 0x7fe5a80cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.885+0000 7fe5a7e00640 1 -- 172.21.15.7:0/663871890 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe5a815bc90 msgr2=0x7fe5a80cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.885+0000 7fe5a7e00640 1 --2- 172.21.15.7:0/663871890 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe5a815bc90 0x7fe5a80cabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.885+0000 7fe5a7e00640 1 -- 172.21.15.7:0/663871890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5a815fda0 msgr2=0x7fe5a80cb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:03.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.885+0000 7fe5a7e00640 1 --2- 172.21.15.7:0/663871890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5a815fda0 0x7fe5a80cb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:03.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.885+0000 7fe5a7e00640 1 -- 172.21.15.7:0/663871890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a80cb810 con 0x7fe5a80cd750 2024-09-10T21:50:03.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.885+0000 7fe5ad000640 1 --2- 172.21.15.7:0/663871890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5a815fda0 0x7fe5a80cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:50:03.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.885+0000 7fe5a7e00640 1 --2- 172.21.15.7:0/663871890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5a80cd750 0x7fe5a80b3400 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fe59800c480 tx=0x7fe59800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:03.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.885+0000 7fe5a4c00640 1 -- 172.21.15.7:0/663871890 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5980052d0 con 0x7fe5a80cd750 2024-09-10T21:50:03.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.885+0000 7fe5a4c00640 1 -- 172.21.15.7:0/663871890 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe598005470 con 0x7fe5a80cd750 2024-09-10T21:50:03.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.885+0000 7fe5a4c00640 1 -- 172.21.15.7:0/663871890 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe598012870 con 0x7fe5a80cd750 2024-09-10T21:50:03.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.885+0000 7fe5af000640 1 -- 172.21.15.7:0/663871890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a815c4b0 con 0x7fe5a80cd750 2024-09-10T21:50:03.888 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.886+0000 7fe5af000640 1 -- 172.21.15.7:0/663871890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a80b3940 con 0x7fe5a80cd750 2024-09-10T21:50:03.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.887+0000 7fe5af000640 1 -- 172.21.15.7:0/663871890 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5a80704a0 con 0x7fe5a80cd750 2024-09-10T21:50:03.889 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.888+0000 7fe5a4c00640 1 -- 172.21.15.7:0/663871890 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe5980035f0 con 0x7fe5a80cd750 2024-09-10T21:50:03.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.889+0000 7fe5a4c00640 1 --2- 172.21.15.7:0/663871890 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe584077f70 0x7fe58407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:03.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.889+0000 7fe5a4c00640 1 -- 172.21.15.7:0/663871890 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fe598099920 con 0x7fe5a80cd750 2024-09-10T21:50:03.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.890+0000 7fe59f400640 1 --2- 172.21.15.7:0/663871890 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe584077f70 0x7fe58407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:03.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.890+0000 7fe59f400640 1 --2- 172.21.15.7:0/663871890 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe584077f70 0x7fe58407a430 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe590033000 tx=0x7fe590004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:03.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:03.893+0000 7fe5a4c00640 1 -- 172.21.15.7:0/663871890 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe5980624b0 con 0x7fe5a80cd750 2024-09-10T21:50:04.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.084+0000 7fe5af000640 1 -- 172.21.15.7:0/663871890 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe5a80c3430 con 0x7fe584077f70 2024-09-10T21:50:04.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.089+0000 7fe5a4c00640 1 -- 172.21.15.7:0/663871890 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252252 (secure 0 0 0) 0x7fe5a80c3430 con 0x7fe584077f70 2024-09-10T21:50:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:04.094 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:50:04.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.093+0000 7fe5af000640 1 -- 172.21.15.7:0/663871890 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe584077f70 msgr2=0x7fe58407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:04.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.093+0000 7fe5af000640 1 --2- 172.21.15.7:0/663871890 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe584077f70 0x7fe58407a430 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fe590033000 tx=0x7fe590004730 comp rx=0 tx=0).stop 2024-09-10T21:50:04.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.093+0000 7fe5af000640 1 -- 172.21.15.7:0/663871890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5a80cd750 msgr2=0x7fe5a80b3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:04.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.093+0000 7fe5af000640 1 --2- 172.21.15.7:0/663871890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5a80cd750 0x7fe5a80b3400 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fe59800c480 tx=0x7fe59800c940 comp rx=0 tx=0).stop 2024-09-10T21:50:04.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.093+0000 7fe5af000640 1 -- 172.21.15.7:0/663871890 shutdown_connections 2024-09-10T21:50:04.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.093+0000 7fe5af000640 1 --2- 172.21.15.7:0/663871890 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe584077f70 0x7fe58407a430 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:04.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.093+0000 7fe5af000640 1 --2- 172.21.15.7:0/663871890 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe5a815fda0 0x7fe5a80cb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:04.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.093+0000 7fe5af000640 1 --2- 172.21.15.7:0/663871890 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe5a815bc90 0x7fe5a80cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:04.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.093+0000 7fe5af000640 1 --2- 172.21.15.7:0/663871890 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe5a80cd750 0x7fe5a80b3400 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:04.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.093+0000 7fe5af000640 1 -- 172.21.15.7:0/663871890 >> 172.21.15.7:0/663871890 conn(0x7fe5a80b16d0 msgr2=0x7fe5a8072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:04.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.093+0000 7fe5af000640 1 -- 172.21.15.7:0/663871890 shutdown_connections 2024-09-10T21:50:04.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.093+0000 7fe5af000640 1 -- 172.21.15.7:0/663871890 wait complete. 2024-09-10T21:50:04.110 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-10T21:50:03.981469+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":268764,"kb_used_data":10920,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754705956,"statfs":{"total":773094113280,"available":772818898944,"internally_reserved":0,"allocated":11182080,"data_stored":5464092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001589"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.443241+0000","last_change":"2024-09-10T21:49:59.466064+0000","last_active":"2024-09-10T21:50:00.443241+0000","last_peered":"2024-09-10T21:50:00.443241+0000","last_clean":"2024-09-10T21:50:00.443241+0000","last_became_active":"2024-09-10T21:49:59.465630+0000","last_became_peered":"2024-09-10T21:49:59.465630+0000","last_unstale":"2024-09-10T21:50:00.443241+0000","last_undegraded":"2024-09-10T21:50:00.443241+0000","last_fullsized":"2024-09-10T21:50:00.443241+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:06.553334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.456341+0000","last_change":"2024-09-10T21:49:59.455977+0000","last_active":"2024-09-10T21:50:00.456341+0000","last_peered":"2024-09-10T21:50:00.456341+0000","last_clean":"2024-09-10T21:50:00.456341+0000","last_became_active":"2024-09-10T21:49:59.455696+0000","last_became_peered":"2024-09-10T21:49:59.455696+0000","last_unstale":"2024-09-10T21:50:00.456341+0000","last_undegraded":"2024-09-10T21:50:00.456341+0000","last_fullsized":"2024-09-10T21:50:00.456341+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:56:40.043097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.443555+0000","last_change":"2024-09-10T21:49:59.465756+0000","last_active":"2024-09-10T21:50:00.443555+0000","last_peered":"2024-09-10T21:50:00.443555+0000","last_clean":"2024-09-10T21:50:00.443555+0000","last_became_active":"2024-09-10T21:49:59.465464+0000","last_became_peered":"2024-09-10T21:49:59.465464+0000","last_unstale":"2024-09-10T21:50:00.443555+0000","last_undegraded":"2024-09-10T21:50:00.443555+0000","last_fullsized":"2024-09-10T21:50:00.443555+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:05:33.089582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454884+0000","last_change":"2024-09-10T21:49:59.458482+0000","last_active":"2024-09-10T21:50:00.454884+0000","last_peered":"2024-09-10T21:50:00.454884+0000","last_clean":"2024-09-10T21:50:00.454884+0000","last_became_active":"2024-09-10T21:49:59.458124+0000","last_became_peered":"2024-09-10T21:49:59.458124+0000","last_unstale":"2024-09-10T21:50:00.454884+0000","last_undegraded":"2024-09-10T21:50:00.454884+0000","last_fullsized":"2024-09-10T21:50:00.454884+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:01.978617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454777+0000","last_change":"2024-09-10T21:49:59.451826+0000","last_active":"2024-09-10T21:50:00.454777+0000","last_peered":"2024-09-10T21:50:00.454777+0000","last_clean":"2024-09-10T21:50:00.454777+0000","last_became_active":"2024-09-10T21:49:59.451519+0000","last_became_peered":"2024-09-10T21:49:59.451519+0000","last_unstale":"2024-09-10T21:50:00.454777+0000","last_undegraded":"2024-09-10T21:50:00.454777+0000","last_fullsized":"2024-09-10T21:50:00.454777+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:10:30.484920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454347+0000","last_change":"2024-09-10T21:49:59.441978+0000","last_active":"2024-09-10T21:50:00.454347+0000","last_peered":"2024-09-10T21:50:00.454347+0000","last_clean":"2024-09-10T21:50:00.454347+0000","last_became_active":"2024-09-10T21:49:59.441636+0000","last_became_peered":"2024-09-10T21:49:59.441636+0000","last_unstale":"2024-09-10T21:50:00.454347+0000","last_undegraded":"2024-09-10T21:50:00.454347+0000","last_fullsized":"2024-09-10T21:50:00.454347+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:32:28.828696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454035+0000","last_change":"2024-09-10T21:49:59.445847+0000","last_active":"2024-09-10T21:50:00.454035+0000","last_peered":"2024-09-10T21:50:00.454035+0000","last_clean":"2024-09-10T21:50:00.454035+0000","last_became_active":"2024-09-10T21:49:59.445562+0000","last_became_peered":"2024-09-10T21:49:59.445562+0000","last_unstale":"2024-09-10T21:50:00.454035+0000","last_undegraded":"2024-09-10T21:50:00.454035+0000","last_fullsized":"2024-09-10T21:50:00.454035+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:01:01.762660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455797+0000","last_change":"2024-09-10T21:49:59.454202+0000","last_active":"2024-09-10T21:50:00.455797+0000","last_peered":"2024-09-10T21:50:00.455797+0000","last_clean":"2024-09-10T21:50:00.455797+0000","last_became_active":"2024-09-10T21:49:59.453879+0000","last_became_peered":"2024-09-10T21:49:59.453879+0000","last_unstale":"2024-09-10T21:50:00.455797+0000","last_undegraded":"2024-09-10T21:50:00.455797+0000","last_fullsized":"2024-09-10T21:50:00.455797+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:32:48.186566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.445500+0000","last_change":"2024-09-10T21:49:59.453232+0000","last_active":"2024-09-10T21:50:00.445500+0000","last_peered":"2024-09-10T21:50:00.445500+0000","last_clean":"2024-09-10T21:50:00.445500+0000","last_became_active":"2024-09-10T21:49:59.451841+0000","last_became_peered":"2024-09-10T21:49:59.451841+0000","last_unstale":"2024-09-10T21:50:00.445500+0000","last_undegraded":"2024-09-10T21:50:00.445500+0000","last_fullsized":"2024-09-10T21:50:00.445500+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:55:27.674516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455298+0000","last_change":"2024-09-10T21:49:59.465467+0000","last_active":"2024-09-10T21:50:00.455298+0000","last_peered":"2024-09-10T21:50:00.455298+0000","last_clean":"2024-09-10T21:50:00.455298+0000","last_became_active":"2024-09-10T21:49:59.465158+0000","last_became_peered":"2024-09-10T21:49:59.465158+0000","last_unstale":"2024-09-10T21:50:00.455298+0000","last_undegraded":"2024-09-10T21:50:00.455298+0000","last_fullsized":"2024-09-10T21:50:00.455298+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:34.456449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454217+0000","last_change":"2024-09-10T21:49:59.450906+0000","last_active":"2024-09-10T21:50:00.454217+0000","last_peered":"2024-09-10T21:50:00.454217+0000","last_clean":"2024-09-10T21:50:00.454217+0000","last_became_active":"2024-09-10T21:49:59.450618+0000","last_became_peered":"2024-09-10T21:49:59.450618+0000","last_unstale":"2024-09-10T21:50:00.454217+0000","last_undegraded":"2024-09-10T21:50:00.454217+0000","last_fullsized":"2024-09-10T21:50:00.454217+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:20:31.447063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.444769+0000","last_change":"2024-09-10T21:49:59.467794+0000","last_active":"2024-09-10T21:50:00.444769+0000","last_peered":"2024-09-10T21:50:00.444769+0000","last_clean":"2024-09-10T21:50:00.444769+0000","last_became_active":"2024-09-10T21:49:59.467506+0000","last_became_peered":"2024-09-10T21:49:59.467506+0000","last_unstale":"2024-09-10T21:50:00.444769+0000","last_undegraded":"2024-09-10T21:50:00.444769+0000","last_fullsized":"2024-09-10T21:50:00.444769+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:16.057848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454658+0000","last_change":"2024-09-10T21:49:59.456269+0000","last_active":"2024-09-10T21:50:00.454658+0000","last_peered":"2024-09-10T21:50:00.454658+0000","last_clean":"2024-09-10T21:50:00.454658+0000","last_became_active":"2024-09-10T21:49:59.456006+0000","last_became_peered":"2024-09-10T21:49:59.456006+0000","last_unstale":"2024-09-10T21:50:00.454658+0000","last_undegraded":"2024-09-10T21:50:00.454658+0000","last_fullsized":"2024-09-10T21:50:00.454658+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:04:57.089080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.456063+0000","last_change":"2024-09-10T21:49:59.453348+0000","last_active":"2024-09-10T21:50:00.456063+0000","last_peered":"2024-09-10T21:50:00.456063+0000","last_clean":"2024-09-10T21:50:00.456063+0000","last_became_active":"2024-09-10T21:49:59.452042+0000","last_became_peered":"2024-09-10T21:49:59.452042+0000","last_unstale":"2024-09-10T21:50:00.456063+0000","last_undegraded":"2024-09-10T21:50:00.456063+0000","last_fullsized":"2024-09-10T21:50:00.456063+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:32:55.218622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.453984+0000","last_change":"2024-09-10T21:49:59.467852+0000","last_active":"2024-09-10T21:50:00.453984+0000","last_peered":"2024-09-10T21:50:00.453984+0000","last_clean":"2024-09-10T21:50:00.453984+0000","last_became_active":"2024-09-10T21:49:59.467486+0000","last_became_peered":"2024-09-10T21:49:59.467486+0000","last_unstale":"2024-09-10T21:50:00.453984+0000","last_undegraded":"2024-09-10T21:50:00.453984+0000","last_fullsized":"2024-09-10T21:50:00.453984+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:11:18.562220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454775+0000","last_change":"2024-09-10T21:49:59.458686+0000","last_active":"2024-09-10T21:50:00.454775+0000","last_peered":"2024-09-10T21:50:00.454775+0000","last_clean":"2024-09-10T21:50:00.454775+0000","last_became_active":"2024-09-10T21:49:59.458359+0000","last_became_peered":"2024-09-10T21:49:59.458359+0000","last_unstale":"2024-09-10T21:50:00.454775+0000","last_undegraded":"2024-09-10T21:50:00.454775+0000","last_fullsized":"2024-09-10T21:50:00.454775+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:18:25.438731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.456210+0000","last_change":"2024-09-10T21:49:59.456147+0000","last_active":"2024-09-10T21:50:00.456210+0000","last_peered":"2024-09-10T21:50:00.456210+0000","last_clean":"2024-09-10T21:50:00.456210+0000","last_became_active":"2024-09-10T21:49:59.455865+0000","last_became_peered":"2024-09-10T21:49:59.455865+0000","last_unstale":"2024-09-10T21:50:00.456210+0000","last_undegraded":"2024-09-10T21:50:00.456210+0000","last_fullsized":"2024-09-10T21:50:00.456210+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:54:58.435997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455007+0000","last_change":"2024-09-10T21:49:59.459014+0000","last_active":"2024-09-10T21:50:00.455007+0000","last_peered":"2024-09-10T21:50:00.455007+0000","last_clean":"2024-09-10T21:50:00.455007+0000","last_became_active":"2024-09-10T21:49:59.458724+0000","last_became_peered":"2024-09-10T21:49:59.458724+0000","last_unstale":"2024-09-10T21:50:00.455007+0000","last_undegraded":"2024-09-10T21:50:00.455007+0000","last_fullsized":"2024-09-10T21:50:00.455007+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:42.911939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455139+0000","last_change":"2024-09-10T21:49:59.452067+0000","last_active":"2024-09-10T21:50:00.455139+0000","last_peered":"2024-09-10T21:50:00.455139+0000","last_clean":"2024-09-10T21:50:00.455139+0000","last_became_active":"2024-09-10T21:49:59.451771+0000","last_became_peered":"2024-09-10T21:49:59.451771+0000","last_unstale":"2024-09-10T21:50:00.455139+0000","last_undegraded":"2024-09-10T21:50:00.455139+0000","last_fullsized":"2024-09-10T21:50:00.455139+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:22.195919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.444789+0000","last_change":"2024-09-10T21:49:59.453610+0000","last_active":"2024-09-10T21:50:00.444789+0000","last_peered":"2024-09-10T21:50:00.444789+0000","last_clean":"2024-09-10T21:50:00.444789+0000","last_became_active":"2024-09-10T21:49:59.452822+0000","last_became_peered":"2024-09-10T21:49:59.452822+0000","last_unstale":"2024-09-10T21:50:00.444789+0000","last_undegraded":"2024-09-10T21:50:00.444789+0000","last_fullsized":"2024-09-10T21:50:00.444789+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:28.856089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455022+0000","last_change":"2024-09-10T21:49:59.438822+0000","last_active":"2024-09-10T21:50:00.455022+0000","last_peered":"2024-09-10T21:50:00.455022+0000","last_clean":"2024-09-10T21:50:00.455022+0000","last_became_active":"2024-09-10T21:49:59.438529+0000","last_became_peered":"2024-09-10T21:49:59.438529+0000","last_unstale":"2024-09-10T21:50:00.455022+0000","last_undegraded":"2024-09-10T21:50:00.455022+0000","last_fullsized":"2024-09-10T21:50:00.455022+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:12.409504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455332+0000","last_change":"2024-09-10T21:49:59.462517+0000","last_active":"2024-09-10T21:50:00.455332+0000","last_peered":"2024-09-10T21:50:00.455332+0000","last_clean":"2024-09-10T21:50:00.455332+0000","last_became_active":"2024-09-10T21:49:59.457686+0000","last_became_peered":"2024-09-10T21:49:59.457686+0000","last_unstale":"2024-09-10T21:50:00.455332+0000","last_undegraded":"2024-09-10T21:50:00.455332+0000","last_fullsized":"2024-09-10T21:50:00.455332+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:02:15.453475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.453984+0000","last_change":"2024-09-10T21:49:59.453005+0000","last_active":"2024-09-10T21:50:00.453984+0000","last_peered":"2024-09-10T21:50:00.453984+0000","last_clean":"2024-09-10T21:50:00.453984+0000","last_became_active":"2024-09-10T21:49:59.452684+0000","last_became_peered":"2024-09-10T21:49:59.452684+0000","last_unstale":"2024-09-10T21:50:00.453984+0000","last_undegraded":"2024-09-10T21:50:00.453984+0000","last_fullsized":"2024-09-10T21:50:00.453984+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:32.127876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454546+0000","last_change":"2024-09-10T21:49:59.469560+0000","last_active":"2024-09-10T21:50:00.454546+0000","last_peered":"2024-09-10T21:50:00.454546+0000","last_clean":"2024-09-10T21:50:00.454546+0000","last_became_active":"2024-09-10T21:49:59.469262+0000","last_became_peered":"2024-09-10T21:49:59.469262+0000","last_unstale":"2024-09-10T21:50:00.454546+0000","last_undegraded":"2024-09-10T21:50:00.454546+0000","last_fullsized":"2024-09-10T21:50:00.454546+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:14.072179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455337+0000","last_change":"2024-09-10T21:49:59.457111+0000","last_active":"2024-09-10T21:50:00.455337+0000","last_peered":"2024-09-10T21:50:00.455337+0000","last_clean":"2024-09-10T21:50:00.455337+0000","last_became_active":"2024-09-10T21:49:59.456808+0000","last_became_peered":"2024-09-10T21:49:59.456808+0000","last_unstale":"2024-09-10T21:50:00.455337+0000","last_undegraded":"2024-09-10T21:50:00.455337+0000","last_fullsized":"2024-09-10T21:50:00.455337+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:12.088646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.456473+0000","last_change":"2024-09-10T21:49:59.463669+0000","last_active":"2024-09-10T21:50:00.456473+0000","last_peered":"2024-09-10T21:50:00.456473+0000","last_clean":"2024-09-10T21:50:00.456473+0000","last_became_active":"2024-09-10T21:49:59.463349+0000","last_became_peered":"2024-09-10T21:49:59.463349+0000","last_unstale":"2024-09-10T21:50:00.456473+0000","last_undegraded":"2024-09-10T21:50:00.456473+0000","last_fullsized":"2024-09-10T21:50:00.456473+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:08:10.561312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454437+0000","last_change":"2024-09-10T21:49:59.458778+0000","last_active":"2024-09-10T21:50:00.454437+0000","last_peered":"2024-09-10T21:50:00.454437+0000","last_clean":"2024-09-10T21:50:00.454437+0000","last_became_active":"2024-09-10T21:49:59.458043+0000","last_became_peered":"2024-09-10T21:49:59.458043+0000","last_unstale":"2024-09-10T21:50:00.454437+0000","last_undegraded":"2024-09-10T21:50:00.454437+0000","last_fullsized":"2024-09-10T21:50:00.454437+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:23.126640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455014+0000","last_change":"2024-09-10T21:49:59.456067+0000","last_active":"2024-09-10T21:50:00.455014+0000","last_peered":"2024-09-10T21:50:00.455014+0000","last_clean":"2024-09-10T21:50:00.455014+0000","last_became_active":"2024-09-10T21:49:59.455569+0000","last_became_peered":"2024-09-10T21:49:59.455569+0000","last_unstale":"2024-09-10T21:50:00.455014+0000","last_undegraded":"2024-09-10T21:50:00.455014+0000","last_fullsized":"2024-09-10T21:50:00.455014+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:23.164745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455739+0000","last_change":"2024-09-10T21:49:59.453249+0000","last_active":"2024-09-10T21:50:00.455739+0000","last_peered":"2024-09-10T21:50:00.455739+0000","last_clean":"2024-09-10T21:50:00.455739+0000","last_became_active":"2024-09-10T21:49:59.452904+0000","last_became_peered":"2024-09-10T21:49:59.452904+0000","last_unstale":"2024-09-10T21:50:00.455739+0000","last_undegraded":"2024-09-10T21:50:00.455739+0000","last_fullsized":"2024-09-10T21:50:00.455739+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:20:28.614828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454553+0000","last_change":"2024-09-10T21:49:59.455597+0000","last_active":"2024-09-10T21:50:00.454553+0000","last_peered":"2024-09-10T21:50:00.454553+0000","last_clean":"2024-09-10T21:50:00.454553+0000","last_became_active":"2024-09-10T21:49:59.454959+0000","last_became_peered":"2024-09-10T21:49:59.454959+0000","last_unstale":"2024-09-10T21:50:00.454553+0000","last_undegraded":"2024-09-10T21:50:00.454553+0000","last_fullsized":"2024-09-10T21:50:00.454553+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:04.908269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454037+0000","last_change":"2024-09-10T21:49:59.449917+0000","last_active":"2024-09-10T21:50:00.454037+0000","last_peered":"2024-09-10T21:50:00.454037+0000","last_clean":"2024-09-10T21:50:00.454037+0000","last_became_active":"2024-09-10T21:49:59.449599+0000","last_became_peered":"2024-09-10T21:49:59.449599+0000","last_unstale":"2024-09-10T21:50:00.454037+0000","last_undegraded":"2024-09-10T21:50:00.454037+0000","last_fullsized":"2024-09-10T21:50:00.454037+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:19:18.368238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.442856+0000","last_change":"2024-09-10T21:49:59.471166+0000","last_active":"2024-09-10T21:50:00.442856+0000","last_peered":"2024-09-10T21:50:00.442856+0000","last_clean":"2024-09-10T21:50:00.442856+0000","last_became_active":"2024-09-10T21:49:59.470620+0000","last_became_peered":"2024-09-10T21:49:59.470620+0000","last_unstale":"2024-09-10T21:50:00.442856+0000","last_undegraded":"2024-09-10T21:50:00.442856+0000","last_fullsized":"2024-09-10T21:50:00.442856+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:49:01.721189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455381+0000","last_change":"2024-09-10T21:49:59.449289+0000","last_active":"2024-09-10T21:50:00.455381+0000","last_peered":"2024-09-10T21:50:00.455381+0000","last_clean":"2024-09-10T21:50:00.455381+0000","last_became_active":"2024-09-10T21:49:59.448976+0000","last_became_peered":"2024-09-10T21:49:59.448976+0000","last_unstale":"2024-09-10T21:50:00.455381+0000","last_undegraded":"2024-09-10T21:50:00.455381+0000","last_fullsized":"2024-09-10T21:50:00.455381+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:01.852370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"100'4","reported_seq":19,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.456717+0000","last_change":"2024-09-10T21:49:58.442123+0000","last_active":"2024-09-10T21:50:00.456717+0000","last_peered":"2024-09-10T21:50:00.456717+0000","last_clean":"2024-09-10T21:50:00.456717+0000","last_became_active":"2024-09-10T21:49:58.441651+0000","last_became_peered":"2024-09-10T21:49:58.441651+0000","last_unstale":"2024-09-10T21:50:00.456717+0000","last_undegraded":"2024-09-10T21:50:00.456717+0000","last_fullsized":"2024-09-10T21:50:00.456717+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:33.447919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455408+0000","last_change":"2024-09-10T21:49:59.449838+0000","last_active":"2024-09-10T21:50:00.455408+0000","last_peered":"2024-09-10T21:50:00.455408+0000","last_clean":"2024-09-10T21:50:00.455408+0000","last_became_active":"2024-09-10T21:49:59.449158+0000","last_became_peered":"2024-09-10T21:49:59.449158+0000","last_unstale":"2024-09-10T21:50:00.455408+0000","last_undegraded":"2024-09-10T21:50:00.455408+0000","last_fullsized":"2024-09-10T21:50:00.455408+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:52.993443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"100'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454310+0000","last_change":"2024-09-10T21:49:58.445548+0000","last_active":"2024-09-10T21:50:00.454310+0000","last_peered":"2024-09-10T21:50:00.454310+0000","last_clean":"2024-09-10T21:50:00.454310+0000","last_became_active":"2024-09-10T21:49:58.445230+0000","last_became_peered":"2024-09-10T21:49:58.445230+0000","last_unstale":"2024-09-10T21:50:00.454310+0000","last_undegraded":"2024-09-10T21:50:00.454310+0000","last_fullsized":"2024-09-10T21:50:00.454310+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:36.258778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.443196+0000","last_change":"2024-09-10T21:49:59.469030+0000","last_active":"2024-09-10T21:50:00.443196+0000","last_peered":"2024-09-10T21:50:00.443196+0000","last_clean":"2024-09-10T21:50:00.443196+0000","last_became_active":"2024-09-10T21:49:59.468600+0000","last_became_peered":"2024-09-10T21:49:59.468600+0000","last_unstale":"2024-09-10T21:50:00.443196+0000","last_undegraded":"2024-09-10T21:50:00.443196+0000","last_fullsized":"2024-09-10T21:50:00.443196+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:03:12.652100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"100'2","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.456121+0000","last_change":"2024-09-10T21:49:58.443132+0000","last_active":"2024-09-10T21:50:00.456121+0000","last_peered":"2024-09-10T21:50:00.456121+0000","last_clean":"2024-09-10T21:50:00.456121+0000","last_became_active":"2024-09-10T21:49:58.442796+0000","last_became_peered":"2024-09-10T21:49:58.442796+0000","last_unstale":"2024-09-10T21:50:00.456121+0000","last_undegraded":"2024-09-10T21:50:00.456121+0000","last_fullsized":"2024-09-10T21:50:00.456121+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:11.791070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.445265+0000","last_change":"2024-09-10T21:49:59.443643+0000","last_active":"2024-09-10T21:50:00.445265+0000","last_peered":"2024-09-10T21:50:00.445265+0000","last_clean":"2024-09-10T21:50:00.445265+0000","last_became_active":"2024-09-10T21:49:59.441671+0000","last_became_peered":"2024-09-10T21:49:59.441671+0000","last_unstale":"2024-09-10T21:50:00.445265+0000","last_undegraded":"2024-09-10T21:50:00.445265+0000","last_fullsized":"2024-09-10T21:50:00.445265+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:13.953427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"100'2","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.453893+0000","last_change":"2024-09-10T21:49:58.441045+0000","last_active":"2024-09-10T21:50:00.453893+0000","last_peered":"2024-09-10T21:50:00.453893+0000","last_clean":"2024-09-10T21:50:00.453893+0000","last_became_active":"2024-09-10T21:49:58.440728+0000","last_became_peered":"2024-09-10T21:49:58.440728+0000","last_unstale":"2024-09-10T21:50:00.453893+0000","last_undegraded":"2024-09-10T21:50:00.453893+0000","last_fullsized":"2024-09-10T21:50:00.453893+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:55.659275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.445100+0000","last_change":"2024-09-10T21:49:59.445219+0000","last_active":"2024-09-10T21:50:00.445100+0000","last_peered":"2024-09-10T21:50:00.445100+0000","last_clean":"2024-09-10T21:50:00.445100+0000","last_became_active":"2024-09-10T21:49:59.444690+0000","last_became_peered":"2024-09-10T21:49:59.444690+0000","last_unstale":"2024-09-10T21:50:00.445100+0000","last_undegraded":"2024-09-10T21:50:00.445100+0000","last_fullsized":"2024-09-10T21:50:00.445100+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:08:20.141576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"100'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.442953+0000","last_change":"2024-09-10T21:49:58.448664+0000","last_active":"2024-09-10T21:50:00.442953+0000","last_peered":"2024-09-10T21:50:00.442953+0000","last_clean":"2024-09-10T21:50:00.442953+0000","last_became_active":"2024-09-10T21:49:58.448412+0000","last_became_peered":"2024-09-10T21:49:58.448412+0000","last_unstale":"2024-09-10T21:50:00.442953+0000","last_undegraded":"2024-09-10T21:50:00.442953+0000","last_fullsized":"2024-09-10T21:50:00.442953+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:21.187389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454180+0000","last_change":"2024-09-10T21:49:59.453917+0000","last_active":"2024-09-10T21:50:00.454180+0000","last_peered":"2024-09-10T21:50:00.454180+0000","last_clean":"2024-09-10T21:50:00.454180+0000","last_became_active":"2024-09-10T21:49:59.453588+0000","last_became_peered":"2024-09-10T21:49:59.453588+0000","last_unstale":"2024-09-10T21:50:00.454180+0000","last_undegraded":"2024-09-10T21:50:00.454180+0000","last_fullsized":"2024-09-10T21:50:00.454180+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:04:25.785285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"100'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455394+0000","last_change":"2024-09-10T21:49:58.444732+0000","last_active":"2024-09-10T21:50:00.455394+0000","last_peered":"2024-09-10T21:50:00.455394+0000","last_clean":"2024-09-10T21:50:00.455394+0000","last_became_active":"2024-09-10T21:49:58.444445+0000","last_became_peered":"2024-09-10T21:49:58.444445+0000","last_unstale":"2024-09-10T21:50:00.455394+0000","last_undegraded":"2024-09-10T21:50:00.455394+0000","last_fullsized":"2024-09-10T21:50:00.455394+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:44.069560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.453963+0000","last_change":"2024-09-10T21:49:59.472501+0000","last_active":"2024-09-10T21:50:00.453963+0000","last_peered":"2024-09-10T21:50:00.453963+0000","last_clean":"2024-09-10T21:50:00.453963+0000","last_became_active":"2024-09-10T21:49:59.471592+0000","last_became_peered":"2024-09-10T21:49:59.471592+0000","last_unstale":"2024-09-10T21:50:00.453963+0000","last_undegraded":"2024-09-10T21:50:00.453963+0000","last_fullsized":"2024-09-10T21:50:00.453963+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:38:18.082636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454330+0000","last_change":"2024-09-10T21:49:58.451985+0000","last_active":"2024-09-10T21:50:00.454330+0000","last_peered":"2024-09-10T21:50:00.454330+0000","last_clean":"2024-09-10T21:50:00.454330+0000","last_became_active":"2024-09-10T21:49:58.451750+0000","last_became_peered":"2024-09-10T21:49:58.451750+0000","last_unstale":"2024-09-10T21:50:00.454330+0000","last_undegraded":"2024-09-10T21:50:00.454330+0000","last_fullsized":"2024-09-10T21:50:00.454330+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:21:04.655941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.444583+0000","last_change":"2024-09-10T21:49:59.440398+0000","last_active":"2024-09-10T21:50:00.444583+0000","last_peered":"2024-09-10T21:50:00.444583+0000","last_clean":"2024-09-10T21:50:00.444583+0000","last_became_active":"2024-09-10T21:49:59.440039+0000","last_became_peered":"2024-09-10T21:49:59.440039+0000","last_unstale":"2024-09-10T21:50:00.444583+0000","last_undegraded":"2024-09-10T21:50:00.444583+0000","last_fullsized":"2024-09-10T21:50:00.444583+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:30:35.245612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.442763+0000","last_change":"2024-09-10T21:49:58.448296+0000","last_active":"2024-09-10T21:50:00.442763+0000","last_peered":"2024-09-10T21:50:00.442763+0000","last_clean":"2024-09-10T21:50:00.442763+0000","last_became_active":"2024-09-10T21:49:58.448029+0000","last_became_peered":"2024-09-10T21:49:58.448029+0000","last_unstale":"2024-09-10T21:50:00.442763+0000","last_undegraded":"2024-09-10T21:50:00.442763+0000","last_fullsized":"2024-09-10T21:50:00.442763+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:06:11.400974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.456528+0000","last_change":"2024-09-10T21:49:59.456906+0000","last_active":"2024-09-10T21:50:00.456528+0000","last_peered":"2024-09-10T21:50:00.456528+0000","last_clean":"2024-09-10T21:50:00.456528+0000","last_became_active":"2024-09-10T21:49:59.456541+0000","last_became_peered":"2024-09-10T21:49:59.456541+0000","last_unstale":"2024-09-10T21:50:00.456528+0000","last_undegraded":"2024-09-10T21:50:00.456528+0000","last_fullsized":"2024-09-10T21:50:00.456528+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:09:29.384993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.443373+0000","last_change":"2024-09-10T21:49:58.449847+0000","last_active":"2024-09-10T21:50:00.443373+0000","last_peered":"2024-09-10T21:50:00.443373+0000","last_clean":"2024-09-10T21:50:00.443373+0000","last_became_active":"2024-09-10T21:49:58.445396+0000","last_became_peered":"2024-09-10T21:49:58.445396+0000","last_unstale":"2024-09-10T21:50:00.443373+0000","last_undegraded":"2024-09-10T21:50:00.443373+0000","last_fullsized":"2024-09-10T21:50:00.443373+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:54.836859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.445437+0000","last_change":"2024-09-10T21:49:59.445168+0000","last_active":"2024-09-10T21:50:00.445437+0000","last_peered":"2024-09-10T21:50:00.445437+0000","last_clean":"2024-09-10T21:50:00.445437+0000","last_became_active":"2024-09-10T21:49:59.442597+0000","last_became_peered":"2024-09-10T21:49:59.442597+0000","last_unstale":"2024-09-10T21:50:00.445437+0000","last_undegraded":"2024-09-10T21:50:00.445437+0000","last_fullsized":"2024-09-10T21:50:00.445437+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:19:58.897928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454917+0000","last_change":"2024-09-10T21:49:58.445035+0000","last_active":"2024-09-10T21:50:00.454917+0000","last_peered":"2024-09-10T21:50:00.454917+0000","last_clean":"2024-09-10T21:50:00.454917+0000","last_became_active":"2024-09-10T21:49:58.444734+0000","last_became_peered":"2024-09-10T21:49:58.444734+0000","last_unstale":"2024-09-10T21:50:00.454917+0000","last_undegraded":"2024-09-10T21:50:00.454917+0000","last_fullsized":"2024-09-10T21:50:00.454917+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:47:49.874922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454948+0000","last_change":"2024-09-10T21:49:59.457123+0000","last_active":"2024-09-10T21:50:00.454948+0000","last_peered":"2024-09-10T21:50:00.454948+0000","last_clean":"2024-09-10T21:50:00.454948+0000","last_became_active":"2024-09-10T21:49:59.456789+0000","last_became_peered":"2024-09-10T21:49:59.456789+0000","last_unstale":"2024-09-10T21:50:00.454948+0000","last_undegraded":"2024-09-10T21:50:00.454948+0000","last_fullsized":"2024-09-10T21:50:00.454948+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:40.587450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"100'3","reported_seq":18,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454652+0000","last_change":"2024-09-10T21:49:58.442254+0000","last_active":"2024-09-10T21:50:00.454652+0000","last_peered":"2024-09-10T21:50:00.454652+0000","last_clean":"2024-09-10T21:50:00.454652+0000","last_became_active":"2024-09-10T21:49:58.441915+0000","last_became_peered":"2024-09-10T21:49:58.441915+0000","last_unstale":"2024-09-10T21:50:00.454652+0000","last_undegraded":"2024-09-10T21:50:00.454652+0000","last_fullsized":"2024-09-10T21:50:00.454652+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:22:25.667184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454527+0000","last_change":"2024-09-10T21:49:59.466948+0000","last_active":"2024-09-10T21:50:00.454527+0000","last_peered":"2024-09-10T21:50:00.454527+0000","last_clean":"2024-09-10T21:50:00.454527+0000","last_became_active":"2024-09-10T21:49:59.466340+0000","last_became_peered":"2024-09-10T21:49:59.466340+0000","last_unstale":"2024-09-10T21:50:00.454527+0000","last_undegraded":"2024-09-10T21:50:00.454527+0000","last_fullsized":"2024-09-10T21:50:00.454527+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:12:41.966095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"100'6","reported_seq":21,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.444924+0000","last_change":"2024-09-10T21:49:58.437418+0000","last_active":"2024-09-10T21:50:00.444924+0000","last_peered":"2024-09-10T21:50:00.444924+0000","last_clean":"2024-09-10T21:50:00.444924+0000","last_became_active":"2024-09-10T21:49:58.437144+0000","last_became_peered":"2024-09-10T21:49:58.437144+0000","last_unstale":"2024-09-10T21:50:00.444924+0000","last_undegraded":"2024-09-10T21:50:00.444924+0000","last_fullsized":"2024-09-10T21:50:00.444924+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:34:56.252995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454373+0000","last_change":"2024-09-10T21:49:59.450770+0000","last_active":"2024-09-10T21:50:00.454373+0000","last_peered":"2024-09-10T21:50:00.454373+0000","last_clean":"2024-09-10T21:50:00.454373+0000","last_became_active":"2024-09-10T21:49:59.450460+0000","last_became_peered":"2024-09-10T21:49:59.450460+0000","last_unstale":"2024-09-10T21:50:00.454373+0000","last_undegraded":"2024-09-10T21:50:00.454373+0000","last_fullsized":"2024-09-10T21:50:00.454373+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:04:57.481410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"100'2","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455150+0000","last_change":"2024-09-10T21:49:58.445507+0000","last_active":"2024-09-10T21:50:00.455150+0000","last_peered":"2024-09-10T21:50:00.455150+0000","last_clean":"2024-09-10T21:50:00.455150+0000","last_became_active":"2024-09-10T21:49:58.445247+0000","last_became_peered":"2024-09-10T21:49:58.445247+0000","last_unstale":"2024-09-10T21:50:00.455150+0000","last_undegraded":"2024-09-10T21:50:00.455150+0000","last_fullsized":"2024-09-10T21:50:00.455150+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:18.676804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.443050+0000","last_change":"2024-09-10T21:49:59.467545+0000","last_active":"2024-09-10T21:50:00.443050+0000","last_peered":"2024-09-10T21:50:00.443050+0000","last_clean":"2024-09-10T21:50:00.443050+0000","last_became_active":"2024-09-10T21:49:59.467178+0000","last_became_peered":"2024-09-10T21:49:59.467178+0000","last_unstale":"2024-09-10T21:50:00.443050+0000","last_undegraded":"2024-09-10T21:50:00.443050+0000","last_fullsized":"2024-09-10T21:50:00.443050+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:33:33.660984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455040+0000","last_change":"2024-09-10T21:49:58.453544+0000","last_active":"2024-09-10T21:50:00.455040+0000","last_peered":"2024-09-10T21:50:00.455040+0000","last_clean":"2024-09-10T21:50:00.455040+0000","last_became_active":"2024-09-10T21:49:58.453136+0000","last_became_peered":"2024-09-10T21:49:58.453136+0000","last_unstale":"2024-09-10T21:50:00.455040+0000","last_undegraded":"2024-09-10T21:50:00.455040+0000","last_fullsized":"2024-09-10T21:50:00.455040+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:06:03.407444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.444246+0000","last_change":"2024-09-10T21:49:59.443246+0000","last_active":"2024-09-10T21:50:00.444246+0000","last_peered":"2024-09-10T21:50:00.444246+0000","last_clean":"2024-09-10T21:50:00.444246+0000","last_became_active":"2024-09-10T21:49:59.442256+0000","last_became_peered":"2024-09-10T21:49:59.442256+0000","last_unstale":"2024-09-10T21:50:00.444246+0000","last_undegraded":"2024-09-10T21:50:00.444246+0000","last_fullsized":"2024-09-10T21:50:00.444246+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:44.399808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.444239+0000","last_change":"2024-09-10T21:49:58.439699+0000","last_active":"2024-09-10T21:50:00.444239+0000","last_peered":"2024-09-10T21:50:00.444239+0000","last_clean":"2024-09-10T21:50:00.444239+0000","last_became_active":"2024-09-10T21:49:58.439394+0000","last_became_peered":"2024-09-10T21:49:58.439394+0000","last_unstale":"2024-09-10T21:50:00.444239+0000","last_undegraded":"2024-09-10T21:50:00.444239+0000","last_fullsized":"2024-09-10T21:50:00.444239+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:53:05.653742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455468+0000","last_change":"2024-09-10T21:49:59.440789+0000","last_active":"2024-09-10T21:50:00.455468+0000","last_peered":"2024-09-10T21:50:00.455468+0000","last_clean":"2024-09-10T21:50:00.455468+0000","last_became_active":"2024-09-10T21:49:59.440510+0000","last_became_peered":"2024-09-10T21:49:59.440510+0000","last_unstale":"2024-09-10T21:50:00.455468+0000","last_undegraded":"2024-09-10T21:50:00.455468+0000","last_fullsized":"2024-09-10T21:50:00.455468+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:09.678502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"100'8","reported_seq":23,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.442882+0000","last_change":"2024-09-10T21:49:58.444081+0000","last_active":"2024-09-10T21:50:00.442882+0000","last_peered":"2024-09-10T21:50:00.442882+0000","last_clean":"2024-09-10T21:50:00.442882+0000","last_became_active":"2024-09-10T21:49:58.443794+0000","last_became_peered":"2024-09-10T21:49:58.443794+0000","last_unstale":"2024-09-10T21:50:00.442882+0000","last_undegraded":"2024-09-10T21:50:00.442882+0000","last_fullsized":"2024-09-10T21:50:00.442882+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:25:15.961314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455438+0000","last_change":"2024-09-10T21:49:59.447210+0000","last_active":"2024-09-10T21:50:00.455438+0000","last_peered":"2024-09-10T21:50:00.455438+0000","last_clean":"2024-09-10T21:50:00.455438+0000","last_became_active":"2024-09-10T21:49:59.446908+0000","last_became_peered":"2024-09-10T21:49:59.446908+0000","last_unstale":"2024-09-10T21:50:00.455438+0000","last_undegraded":"2024-09-10T21:50:00.455438+0000","last_fullsized":"2024-09-10T21:50:00.455438+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:41:44.240016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"100'2","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.445441+0000","last_change":"2024-09-10T21:49:58.448480+0000","last_active":"2024-09-10T21:50:00.445441+0000","last_peered":"2024-09-10T21:50:00.445441+0000","last_clean":"2024-09-10T21:50:00.445441+0000","last_became_active":"2024-09-10T21:49:58.448061+0000","last_became_peered":"2024-09-10T21:49:58.448061+0000","last_unstale":"2024-09-10T21:50:00.445441+0000","last_undegraded":"2024-09-10T21:50:00.445441+0000","last_fullsized":"2024-09-10T21:50:00.445441+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:30.801534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.445904+0000","last_change":"2024-09-10T21:49:59.449502+0000","last_active":"2024-09-10T21:50:00.445904+0000","last_peered":"2024-09-10T21:50:00.445904+0000","last_clean":"2024-09-10T21:50:00.445904+0000","last_became_active":"2024-09-10T21:49:59.448126+0000","last_became_peered":"2024-09-10T21:49:59.448126+0000","last_unstale":"2024-09-10T21:50:00.445904+0000","last_undegraded":"2024-09-10T21:50:00.445904+0000","last_fullsized":"2024-09-10T21:50:00.445904+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:44:00.091689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"100'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455052+0000","last_change":"2024-09-10T21:49:58.438457+0000","last_active":"2024-09-10T21:50:00.455052+0000","last_peered":"2024-09-10T21:50:00.455052+0000","last_clean":"2024-09-10T21:50:00.455052+0000","last_became_active":"2024-09-10T21:49:58.438146+0000","last_became_peered":"2024-09-10T21:49:58.438146+0000","last_unstale":"2024-09-10T21:50:00.455052+0000","last_undegraded":"2024-09-10T21:50:00.455052+0000","last_fullsized":"2024-09-10T21:50:00.455052+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:20:56.914534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454662+0000","last_change":"2024-09-10T21:49:59.454957+0000","last_active":"2024-09-10T21:50:00.454662+0000","last_peered":"2024-09-10T21:50:00.454662+0000","last_clean":"2024-09-10T21:50:00.454662+0000","last_became_active":"2024-09-10T21:49:59.454428+0000","last_became_peered":"2024-09-10T21:49:59.454428+0000","last_unstale":"2024-09-10T21:50:00.454662+0000","last_undegraded":"2024-09-10T21:50:00.454662+0000","last_fullsized":"2024-09-10T21:50:00.454662+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:54.446738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455941+0000","last_change":"2024-09-10T21:49:58.438635+0000","last_active":"2024-09-10T21:50:00.455941+0000","last_peered":"2024-09-10T21:50:00.455941+0000","last_clean":"2024-09-10T21:50:00.455941+0000","last_became_active":"2024-09-10T21:49:58.438159+0000","last_became_peered":"2024-09-10T21:49:58.438159+0000","last_unstale":"2024-09-10T21:50:00.455941+0000","last_undegraded":"2024-09-10T21:50:00.455941+0000","last_fullsized":"2024-09-10T21:50:00.455941+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:22.156215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454765+0000","last_change":"2024-09-10T21:49:59.455200+0000","last_active":"2024-09-10T21:50:00.454765+0000","last_peered":"2024-09-10T21:50:00.454765+0000","last_clean":"2024-09-10T21:50:00.454765+0000","last_became_active":"2024-09-10T21:49:59.454867+0000","last_became_peered":"2024-09-10T21:49:59.454867+0000","last_unstale":"2024-09-10T21:50:00.454765+0000","last_undegraded":"2024-09-10T21:50:00.454765+0000","last_fullsized":"2024-09-10T21:50:00.454765+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:25.226389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"100'2","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455723+0000","last_change":"2024-09-10T21:49:58.448659+0000","last_active":"2024-09-10T21:50:00.455723+0000","last_peered":"2024-09-10T21:50:00.455723+0000","last_clean":"2024-09-10T21:50:00.455723+0000","last_became_active":"2024-09-10T21:49:58.448374+0000","last_became_peered":"2024-09-10T21:49:58.448374+0000","last_unstale":"2024-09-10T21:50:00.455723+0000","last_undegraded":"2024-09-10T21:50:00.455723+0000","last_fullsized":"2024-09-10T21:50:00.455723+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:10:29.476390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455364+0000","last_change":"2024-09-10T21:49:59.461321+0000","last_active":"2024-09-10T21:50:00.455364+0000","last_peered":"2024-09-10T21:50:00.455364+0000","last_clean":"2024-09-10T21:50:00.455364+0000","last_became_active":"2024-09-10T21:49:59.460427+0000","last_became_peered":"2024-09-10T21:49:59.460427+0000","last_unstale":"2024-09-10T21:50:00.455364+0000","last_undegraded":"2024-09-10T21:50:00.455364+0000","last_fullsized":"2024-09-10T21:50:00.455364+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:51:55.845389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.445765+0000","last_change":"2024-09-10T21:49:58.445592+0000","last_active":"2024-09-10T21:50:00.445765+0000","last_peered":"2024-09-10T21:50:00.445765+0000","last_clean":"2024-09-10T21:50:00.445765+0000","last_became_active":"2024-09-10T21:49:58.445223+0000","last_became_peered":"2024-09-10T21:49:58.445223+0000","last_unstale":"2024-09-10T21:50:00.445765+0000","last_undegraded":"2024-09-10T21:50:00.445765+0000","last_fullsized":"2024-09-10T21:50:00.445765+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:29:14.810375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454556+0000","last_change":"2024-09-10T21:49:59.450969+0000","last_active":"2024-09-10T21:50:00.454556+0000","last_peered":"2024-09-10T21:50:00.454556+0000","last_clean":"2024-09-10T21:50:00.454556+0000","last_became_active":"2024-09-10T21:49:59.450682+0000","last_became_peered":"2024-09-10T21:49:59.450682+0000","last_unstale":"2024-09-10T21:50:00.454556+0000","last_undegraded":"2024-09-10T21:50:00.454556+0000","last_fullsized":"2024-09-10T21:50:00.454556+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:53:51.162614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.444489+0000","last_change":"2024-09-10T21:49:58.447337+0000","last_active":"2024-09-10T21:50:00.444489+0000","last_peered":"2024-09-10T21:50:00.444489+0000","last_clean":"2024-09-10T21:50:00.444489+0000","last_became_active":"2024-09-10T21:49:58.447130+0000","last_became_peered":"2024-09-10T21:49:58.447130+0000","last_unstale":"2024-09-10T21:50:00.444489+0000","last_undegraded":"2024-09-10T21:50:00.444489+0000","last_fullsized":"2024-09-10T21:50:00.444489+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:25:44.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454188+0000","last_change":"2024-09-10T21:49:59.465662+0000","last_active":"2024-09-10T21:50:00.454188+0000","last_peered":"2024-09-10T21:50:00.454188+0000","last_clean":"2024-09-10T21:50:00.454188+0000","last_became_active":"2024-09-10T21:49:59.465232+0000","last_became_peered":"2024-09-10T21:49:59.465232+0000","last_unstale":"2024-09-10T21:50:00.454188+0000","last_undegraded":"2024-09-10T21:50:00.454188+0000","last_fullsized":"2024-09-10T21:50:00.454188+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:56.354915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454246+0000","last_change":"2024-09-10T21:49:58.443189+0000","last_active":"2024-09-10T21:50:00.454246+0000","last_peered":"2024-09-10T21:50:00.454246+0000","last_clean":"2024-09-10T21:50:00.454246+0000","last_became_active":"2024-09-10T21:49:58.442849+0000","last_became_peered":"2024-09-10T21:49:58.442849+0000","last_unstale":"2024-09-10T21:50:00.454246+0000","last_undegraded":"2024-09-10T21:50:00.454246+0000","last_fullsized":"2024-09-10T21:50:00.454246+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:55.346385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.456663+0000","last_change":"2024-09-10T21:49:59.455300+0000","last_active":"2024-09-10T21:50:00.456663+0000","last_peered":"2024-09-10T21:50:00.456663+0000","last_clean":"2024-09-10T21:50:00.456663+0000","last_became_active":"2024-09-10T21:49:59.452983+0000","last_became_peered":"2024-09-10T21:49:59.452983+0000","last_unstale":"2024-09-10T21:50:00.456663+0000","last_undegraded":"2024-09-10T21:50:00.456663+0000","last_fullsized":"2024-09-10T21:50:00.456663+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:32.351193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"100'2","reported_seq":17,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.445696+0000","last_change":"2024-09-10T21:49:58.436565+0000","last_active":"2024-09-10T21:50:00.445696+0000","last_peered":"2024-09-10T21:50:00.445696+0000","last_clean":"2024-09-10T21:50:00.445696+0000","last_became_active":"2024-09-10T21:49:58.436266+0000","last_became_peered":"2024-09-10T21:49:58.436266+0000","last_unstale":"2024-09-10T21:50:00.445696+0000","last_undegraded":"2024-09-10T21:50:00.445696+0000","last_fullsized":"2024-09-10T21:50:00.445696+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:15:54.303923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.443342+0000","last_change":"2024-09-10T21:49:59.467462+0000","last_active":"2024-09-10T21:50:00.443342+0000","last_peered":"2024-09-10T21:50:00.443342+0000","last_clean":"2024-09-10T21:50:00.443342+0000","last_became_active":"2024-09-10T21:49:59.467147+0000","last_became_peered":"2024-09-10T21:49:59.467147+0000","last_unstale":"2024-09-10T21:50:00.443342+0000","last_undegraded":"2024-09-10T21:50:00.443342+0000","last_fullsized":"2024-09-10T21:50:00.443342+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:16:00.341360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"101'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.456618+0000","last_change":"2024-09-10T21:49:58.449419+0000","last_active":"2024-09-10T21:50:00.456618+0000","last_peered":"2024-09-10T21:50:00.456618+0000","last_clean":"2024-09-10T21:50:00.456618+0000","last_became_active":"2024-09-10T21:49:58.449102+0000","last_became_peered":"2024-09-10T21:49:58.449102+0000","last_unstale":"2024-09-10T21:50:00.456618+0000","last_undegraded":"2024-09-10T21:50:00.456618+0000","last_fullsized":"2024-09-10T21:50:00.456618+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:17:11.080116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.443235+0000","last_change":"2024-09-10T21:49:59.468086+0000","last_active":"2024-09-10T21:50:00.443235+0000","last_peered":"2024-09-10T21:50:00.443235+0000","last_clean":"2024-09-10T21:50:00.443235+0000","last_became_active":"2024-09-10T21:49:59.464690+0000","last_became_peered":"2024-09-10T21:49:59.464690+0000","last_unstale":"2024-09-10T21:50:00.443235+0000","last_undegraded":"2024-09-10T21:50:00.443235+0000","last_fullsized":"2024-09-10T21:50:00.443235+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:36:51.381318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.456020+0000","last_change":"2024-09-10T21:49:58.439259+0000","last_active":"2024-09-10T21:50:00.456020+0000","last_peered":"2024-09-10T21:50:00.456020+0000","last_clean":"2024-09-10T21:50:00.456020+0000","last_became_active":"2024-09-10T21:49:58.437151+0000","last_became_peered":"2024-09-10T21:49:58.437151+0000","last_unstale":"2024-09-10T21:50:00.456020+0000","last_undegraded":"2024-09-10T21:50:00.456020+0000","last_fullsized":"2024-09-10T21:50:00.456020+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:40:01.242312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454980+0000","last_change":"2024-09-10T21:49:59.455280+0000","last_active":"2024-09-10T21:50:00.454980+0000","last_peered":"2024-09-10T21:50:00.454980+0000","last_clean":"2024-09-10T21:50:00.454980+0000","last_became_active":"2024-09-10T21:49:59.454921+0000","last_became_peered":"2024-09-10T21:49:59.454921+0000","last_unstale":"2024-09-10T21:50:00.454980+0000","last_undegraded":"2024-09-10T21:50:00.454980+0000","last_fullsized":"2024-09-10T21:50:00.454980+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:56.667805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455071+0000","last_change":"2024-09-10T21:49:58.442017+0000","last_active":"2024-09-10T21:50:00.455071+0000","last_peered":"2024-09-10T21:50:00.455071+0000","last_clean":"2024-09-10T21:50:00.455071+0000","last_became_active":"2024-09-10T21:49:58.441660+0000","last_became_peered":"2024-09-10T21:49:58.441660+0000","last_unstale":"2024-09-10T21:50:00.455071+0000","last_undegraded":"2024-09-10T21:50:00.455071+0000","last_fullsized":"2024-09-10T21:50:00.455071+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:39.531629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454974+0000","last_change":"2024-09-10T21:49:59.443060+0000","last_active":"2024-09-10T21:50:00.454974+0000","last_peered":"2024-09-10T21:50:00.454974+0000","last_clean":"2024-09-10T21:50:00.454974+0000","last_became_active":"2024-09-10T21:49:59.440857+0000","last_became_peered":"2024-09-10T21:49:59.440857+0000","last_unstale":"2024-09-10T21:50:00.454974+0000","last_undegraded":"2024-09-10T21:50:00.454974+0000","last_fullsized":"2024-09-10T21:50:00.454974+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:25:16.969844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455799+0000","last_change":"2024-09-10T21:49:58.446284+0000","last_active":"2024-09-10T21:50:00.455799+0000","last_peered":"2024-09-10T21:50:00.455799+0000","last_clean":"2024-09-10T21:50:00.455799+0000","last_became_active":"2024-09-10T21:49:58.445995+0000","last_became_peered":"2024-09-10T21:49:58.445995+0000","last_unstale":"2024-09-10T21:50:00.455799+0000","last_undegraded":"2024-09-10T21:50:00.455799+0000","last_fullsized":"2024-09-10T21:50:00.455799+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:39.578920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.444410+0000","last_change":"2024-09-10T21:49:59.446296+0000","last_active":"2024-09-10T21:50:00.444410+0000","last_peered":"2024-09-10T21:50:00.444410+0000","last_clean":"2024-09-10T21:50:00.444410+0000","last_became_active":"2024-09-10T21:49:59.445980+0000","last_became_peered":"2024-09-10T21:49:59.445980+0000","last_unstale":"2024-09-10T21:50:00.444410+0000","last_undegraded":"2024-09-10T21:50:00.444410+0000","last_fullsized":"2024-09-10T21:50:00.444410+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:27.534373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454207+0000","last_change":"2024-09-10T21:49:58.445347+0000","last_active":"2024-09-10T21:50:00.454207+0000","last_peered":"2024-09-10T21:50:00.454207+0000","last_clean":"2024-09-10T21:50:00.454207+0000","last_became_active":"2024-09-10T21:49:58.445062+0000","last_became_peered":"2024-09-10T21:49:58.445062+0000","last_unstale":"2024-09-10T21:50:00.454207+0000","last_undegraded":"2024-09-10T21:50:00.454207+0000","last_fullsized":"2024-09-10T21:50:00.454207+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:15:54.303923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454840+0000","last_change":"2024-09-10T21:49:59.443456+0000","last_active":"2024-09-10T21:50:00.454840+0000","last_peered":"2024-09-10T21:50:00.454840+0000","last_clean":"2024-09-10T21:50:00.454840+0000","last_became_active":"2024-09-10T21:49:59.443122+0000","last_became_peered":"2024-09-10T21:49:59.443122+0000","last_unstale":"2024-09-10T21:50:00.454840+0000","last_undegraded":"2024-09-10T21:50:00.454840+0000","last_fullsized":"2024-09-10T21:50:00.454840+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:19.811468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.453856+0000","last_change":"2024-09-10T21:49:58.453503+0000","last_active":"2024-09-10T21:50:00.453856+0000","last_peered":"2024-09-10T21:50:00.453856+0000","last_clean":"2024-09-10T21:50:00.453856+0000","last_became_active":"2024-09-10T21:49:58.453200+0000","last_became_peered":"2024-09-10T21:49:58.453200+0000","last_unstale":"2024-09-10T21:50:00.453856+0000","last_undegraded":"2024-09-10T21:50:00.453856+0000","last_fullsized":"2024-09-10T21:50:00.453856+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:58:14.389900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454327+0000","last_change":"2024-09-10T21:49:59.451145+0000","last_active":"2024-09-10T21:50:00.454327+0000","last_peered":"2024-09-10T21:50:00.454327+0000","last_clean":"2024-09-10T21:50:00.454327+0000","last_became_active":"2024-09-10T21:49:59.450642+0000","last_became_peered":"2024-09-10T21:49:59.450642+0000","last_unstale":"2024-09-10T21:50:00.454327+0000","last_undegraded":"2024-09-10T21:50:00.454327+0000","last_fullsized":"2024-09-10T21:50:00.454327+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:45.078090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":15,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454621+0000","last_change":"2024-09-10T21:49:58.452096+0000","last_active":"2024-09-10T21:50:00.454621+0000","last_peered":"2024-09-10T21:50:00.454621+0000","last_clean":"2024-09-10T21:50:00.454621+0000","last_became_active":"2024-09-10T21:49:58.451786+0000","last_became_peered":"2024-09-10T21:49:58.451786+0000","last_unstale":"2024-09-10T21:50:00.454621+0000","last_undegraded":"2024-09-10T21:50:00.454621+0000","last_fullsized":"2024-09-10T21:50:00.454621+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:01:31.470577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":13,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.455586+0000","last_change":"2024-09-10T21:49:59.451494+0000","last_active":"2024-09-10T21:50:00.455586+0000","last_peered":"2024-09-10T21:50:00.455586+0000","last_clean":"2024-09-10T21:50:00.455586+0000","last_became_active":"2024-09-10T21:49:59.450790+0000","last_became_peered":"2024-09-10T21:49:59.450790+0000","last_unstale":"2024-09-10T21:50:00.455586+0000","last_undegraded":"2024-09-10T21:50:00.455586+0000","last_fullsized":"2024-09-10T21:50:00.455586+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":99,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:58.415331+0000","last_clean_scrub_stamp":"2024-09-10T21:49:58.415331+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:27.173201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"100'1","reported_seq":16,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454217+0000","last_change":"2024-09-10T21:49:58.448116+0000","last_active":"2024-09-10T21:50:00.454217+0000","last_peered":"2024-09-10T21:50:00.454217+0000","last_clean":"2024-09-10T21:50:00.454217+0000","last_became_active":"2024-09-10T21:49:58.447745+0000","last_became_peered":"2024-09-10T21:49:58.447745+0000","last_unstale":"2024-09-10T21:50:00.454217+0000","last_undegraded":"2024-09-10T21:50:00.454217+0000","last_fullsized":"2024-09-10T21:50:00.454217+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":98,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:49:57.406801+0000","last_clean_scrub_stamp":"2024-09-10T21:49:57.406801+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:58.352646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":258,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-09-10T21:50:00.454655+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:50:00.454655+0000","last_peered":"2024-09-10T21:50:00.454655+0000","last_clean":"2024-09-10T21:50:00.454655+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:50:00.454655+0000","last_undegraded":"2024-09-10T21:50:00.454655+0000","last_fullsized":"2024-09-10T21:50:00.454655+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32416,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339424,"statfs":{"total":96636764160,"available":96603570176,"internally_reserved":0,"allocated":1249280,"data_stored":535469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509541,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1245184,"data_stored":534887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509540,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32420,"kb_used_data":1232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339420,"statfs":{"total":96636764160,"available":96603566080,"internally_reserved":0,"allocated":1261568,"data_stored":536553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509540,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32404,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339436,"statfs":{"total":96636764160,"available":96603582464,"internally_reserved":0,"allocated":1245184,"data_stored":534851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509542,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32400,"kb_used_data":1212,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339440,"statfs":{"total":96636764160,"available":96603586560,"internally_reserved":0,"allocated":1241088,"data_stored":534817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509540,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37144,"kb_used_data":1796,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334696,"statfs":{"total":96636764160,"available":96598728704,"internally_reserved":0,"allocated":1839104,"data_stored":1125767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542243,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32996,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338844,"statfs":{"total":96636764160,"available":96602976256,"internally_reserved":0,"allocated":1843200,"data_stored":1125297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542244,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36580,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335260,"statfs":{"total":96636764160,"available":96599306240,"internally_reserved":0,"allocated":1257472,"data_stored":536451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:50:04.114 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:50:04.114 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:50:04.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.318+0000 7f9783c00640 1 -- 172.21.15.7:0/4269086844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f977c074420 msgr2=0x7f977c06f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.318+0000 7f9783c00640 1 --2- 172.21.15.7:0/4269086844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f977c074420 0x7f977c06f690 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f977c070a70 tx=0x7f976802a5f0 comp rx=0 tx=0).stop 2024-09-10T21:50:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.318+0000 7f9783c00640 1 -- 172.21.15.7:0/4269086844 shutdown_connections 2024-09-10T21:50:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.318+0000 7f9783c00640 1 --2- 172.21.15.7:0/4269086844 >> v2:172.21.15.133:3300/0 conn(0x7f977c06fbd0 0x7f977c0cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.318+0000 7f9783c00640 1 --2- 172.21.15.7:0/4269086844 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f977c074420 0x7f977c06f690 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:04.320 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.318+0000 7f9783c00640 1 -- 172.21.15.7:0/4269086844 >> 172.21.15.7:0/4269086844 conn(0x7f977c0b16d0 msgr2=0x7f977c0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:04.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.318+0000 7f9783c00640 1 -- 172.21.15.7:0/4269086844 shutdown_connections 2024-09-10T21:50:04.321 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.318+0000 7f9783c00640 1 -- 172.21.15.7:0/4269086844 wait complete. 2024-09-10T21:50:04.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.319+0000 7f9783c00640 1 Processor -- start 2024-09-10T21:50:04.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.320+0000 7f9783c00640 1 -- start start 2024-09-10T21:50:04.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.321+0000 7f9783c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f977c06fbd0 0x7f977c1f4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:04.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.321+0000 7f9783c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f977c074420 0x7f977c1f7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:04.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.321+0000 7f9783c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f977c15e160 0x7f977c148410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:04.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.321+0000 7f9783c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f977c0ce310 con 0x7f977c06fbd0 2024-09-10T21:50:04.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.321+0000 7f9783c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f977c0ce190 con 0x7f977c15e160 2024-09-10T21:50:04.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.321+0000 7f9783c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f977c0ce490 con 0x7f977c074420 2024-09-10T21:50:04.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.321+0000 7f9781200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f977c06fbd0 0x7f977c1f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:04.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.321+0000 7f9781200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f977c06fbd0 0x7f977c1f4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58790/0 (socket says 172.21.15.7:58790) 2024-09-10T21:50:04.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.321+0000 7f9781200640 1 -- 172.21.15.7:0/2770418434 learned_addr learned my addr 172.21.15.7:0/2770418434 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:04.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.321+0000 7f977be00640 1 --2- 172.21.15.7:0/2770418434 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f977c074420 0x7f977c1f7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:04.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.321+0000 7f9781c00640 1 --2- 172.21.15.7:0/2770418434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f977c15e160 0x7f977c148410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:04.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.321+0000 7f9781200640 1 -- 172.21.15.7:0/2770418434 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f977c074420 msgr2=0x7f977c1f7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:04.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.321+0000 7f9781200640 1 --2- 172.21.15.7:0/2770418434 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f977c074420 0x7f977c1f7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:04.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.321+0000 7f9781200640 1 -- 172.21.15.7:0/2770418434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f977c15e160 msgr2=0x7f977c148410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:04.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.321+0000 7f9781200640 1 --2- 172.21.15.7:0/2770418434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f977c15e160 0x7f977c148410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:04.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.321+0000 7f9781200640 1 -- 172.21.15.7:0/2770418434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f977c0c6540 con 0x7f977c06fbd0 2024-09-10T21:50:04.324 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.321+0000 7f9781200640 1 --2- 172.21.15.7:0/2770418434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f977c06fbd0 0x7f977c1f4810 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f977000c3d0 tx=0x7f977000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:04.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.322+0000 7f9778c00640 1 -- 172.21.15.7:0/2770418434 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770017070 con 0x7f977c06fbd0 2024-09-10T21:50:04.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.322+0000 7f9778c00640 1 -- 172.21.15.7:0/2770418434 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97700053c0 con 0x7f977c06fbd0 2024-09-10T21:50:04.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.322+0000 7f9783c00640 1 -- 172.21.15.7:0/2770418434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f977c1f4f10 con 0x7f977c06fbd0 2024-09-10T21:50:04.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.322+0000 7f9778c00640 1 -- 172.21.15.7:0/2770418434 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9770003850 con 0x7f977c06fbd0 2024-09-10T21:50:04.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.322+0000 7f9783c00640 1 -- 172.21.15.7:0/2770418434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f977c1f54d0 con 0x7f977c06fbd0 2024-09-10T21:50:04.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.323+0000 7f9783c00640 1 -- 172.21.15.7:0/2770418434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f977c0ce940 con 0x7f977c06fbd0 2024-09-10T21:50:04.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.324+0000 7f9778c00640 1 -- 172.21.15.7:0/2770418434 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f97700039f0 con 0x7f977c06fbd0 2024-09-10T21:50:04.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.325+0000 7f9778c00640 1 --2- 172.21.15.7:0/2770418434 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f975c077f70 0x7f975c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:04.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.325+0000 7f9778c00640 1 -- 172.21.15.7:0/2770418434 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f97700993c0 con 0x7f977c06fbd0 2024-09-10T21:50:04.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.326+0000 7f977be00640 1 --2- 172.21.15.7:0/2770418434 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f975c077f70 0x7f975c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:04.327 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.326+0000 7f977be00640 1 --2- 172.21.15.7:0/2770418434 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f975c077f70 0x7f975c07a430 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9768033000 tx=0x7f9768004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:04.330 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.329+0000 7f9778c00640 1 -- 172.21.15.7:0/2770418434 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9770061eb0 con 0x7f977c06fbd0 2024-09-10T21:50:04.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.599+0000 7f9783c00640 1 -- 172.21.15.7:0/2770418434 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f977c06f660 con 0x7f977c06fbd0 2024-09-10T21:50:04.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.602+0000 7f9778c00640 1 -- 172.21.15.7:0/2770418434 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v38) ==== 190+0+147 (secure 0 0 0) 0x7f97700617b0 con 0x7f977c06fbd0 2024-09-10T21:50:04.604 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-09-10T21:50:04.605 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDvvOBm0aqcFhAAtid2bnzjcpaYAbU7g0JcOA== 2024-09-10T21:50:04.605 INFO:teuthology.orchestra.run.smithi007.stdout: caps mds = "allow" 2024-09-10T21:50:04.605 INFO:teuthology.orchestra.run.smithi007.stdout: caps mon = "allow r" 2024-09-10T21:50:04.605 INFO:teuthology.orchestra.run.smithi007.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:50:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-09-10T21:50:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.605+0000 7f9783c00640 1 -- 172.21.15.7:0/2770418434 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f975c077f70 msgr2=0x7f975c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.605+0000 7f9783c00640 1 --2- 172.21.15.7:0/2770418434 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f975c077f70 0x7f975c07a430 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f9768033000 tx=0x7f9768004730 comp rx=0 tx=0).stop 2024-09-10T21:50:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.605+0000 7f9783c00640 1 -- 172.21.15.7:0/2770418434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f977c06fbd0 msgr2=0x7f977c1f4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.605+0000 7f9783c00640 1 --2- 172.21.15.7:0/2770418434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f977c06fbd0 0x7f977c1f4810 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f977000c3d0 tx=0x7f977000c890 comp rx=0 tx=0).stop 2024-09-10T21:50:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.605+0000 7f9783c00640 1 -- 172.21.15.7:0/2770418434 shutdown_connections 2024-09-10T21:50:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.605+0000 7f9783c00640 1 --2- 172.21.15.7:0/2770418434 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f975c077f70 0x7f975c07a430 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.605+0000 7f9783c00640 1 --2- 172.21.15.7:0/2770418434 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f977c15e160 0x7f977c148410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.605+0000 7f9783c00640 1 --2- 172.21.15.7:0/2770418434 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f977c074420 0x7f977c1f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.605+0000 7f9783c00640 1 --2- 172.21.15.7:0/2770418434 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f977c06fbd0 0x7f977c1f4810 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.605+0000 7f9783c00640 1 -- 172.21.15.7:0/2770418434 >> 172.21.15.7:0/2770418434 conn(0x7f977c0b16d0 msgr2=0x7f977c0b4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.605+0000 7f9783c00640 1 -- 172.21.15.7:0/2770418434 shutdown_connections 2024-09-10T21:50:04.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.605+0000 7f9783c00640 1 -- 172.21.15.7:0/2770418434 wait complete. 2024-09-10T21:50:04.622 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:50:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.782+0000 7f4e64a00640 1 -- 172.21.15.7:0/1026961600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e6015bc90 msgr2=0x7f4e600ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.782+0000 7f4e64a00640 1 --2- 172.21.15.7:0/1026961600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e6015bc90 0x7f4e600ccef0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f4e600b5560 tx=0x7f4e5002a440 comp rx=0 tx=0).stop 2024-09-10T21:50:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.782+0000 7f4e64a00640 1 -- 172.21.15.7:0/1026961600 shutdown_connections 2024-09-10T21:50:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.782+0000 7f4e64a00640 1 --2- 172.21.15.7:0/1026961600 >> v2:172.21.15.133:3300/0 conn(0x7f4e60073c70 0x7f4e6015fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:04.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.782+0000 7f4e64a00640 1 --2- 172.21.15.7:0/1026961600 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e600cd750 0x7f4e60073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.782+0000 7f4e64a00640 1 --2- 172.21.15.7:0/1026961600 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e6015bc90 0x7f4e600ccef0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.782+0000 7f4e64a00640 1 -- 172.21.15.7:0/1026961600 >> 172.21.15.7:0/1026961600 conn(0x7f4e600b16d0 msgr2=0x7f4e600b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.782+0000 7f4e64a00640 1 -- 172.21.15.7:0/1026961600 shutdown_connections 2024-09-10T21:50:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.782+0000 7f4e64a00640 1 -- 172.21.15.7:0/1026961600 wait complete. 2024-09-10T21:50:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.783+0000 7f4e64a00640 1 Processor -- start 2024-09-10T21:50:04.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.784+0000 7f4e64a00640 1 -- start start 2024-09-10T21:50:04.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.784+0000 7f4e64a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60073c70 0x7f4e601f7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.784+0000 7f4e64a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e600cd750 0x7f4e60152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.785+0000 7f4e64a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e6015bc90 0x7f4e600c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.785+0000 7f4e64a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e60160670 con 0x7f4e60073c70 2024-09-10T21:50:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.785+0000 7f4e64a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4e601604f0 con 0x7f4e6015bc90 2024-09-10T21:50:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.785+0000 7f4e64a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4e6006fcf0 con 0x7f4e600cd750 2024-09-10T21:50:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.785+0000 7f4e5de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60073c70 0x7f4e601f7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.785+0000 7f4e5de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60073c70 0x7f4e601f7e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58818/0 (socket says 172.21.15.7:58818) 2024-09-10T21:50:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.785+0000 7f4e5de00640 1 -- 172.21.15.7:0/1920540826 learned_addr learned my addr 172.21.15.7:0/1920540826 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.785+0000 7f4e5e800640 1 --2- 172.21.15.7:0/1920540826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e6015bc90 0x7f4e600c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.785+0000 7f4e5d400640 1 --2- 172.21.15.7:0/1920540826 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e600cd750 0x7f4e60152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.785+0000 7f4e5de00640 1 -- 172.21.15.7:0/1920540826 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e600cd750 msgr2=0x7f4e60152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.785+0000 7f4e5de00640 1 --2- 172.21.15.7:0/1920540826 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e600cd750 0x7f4e60152920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:04.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.785+0000 7f4e5de00640 1 -- 172.21.15.7:0/1920540826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e6015bc90 msgr2=0x7f4e600c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:04.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.785+0000 7f4e5de00640 1 --2- 172.21.15.7:0/1920540826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e6015bc90 0x7f4e600c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:04.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.785+0000 7f4e5de00640 1 -- 172.21.15.7:0/1920540826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e600ae940 con 0x7f4e60073c70 2024-09-10T21:50:04.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.785+0000 7f4e5de00640 1 --2- 172.21.15.7:0/1920540826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60073c70 0x7f4e601f7e50 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f4e50034040 tx=0x7f4e50003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:04.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.786+0000 7f4e56000640 1 -- 172.21.15.7:0/1920540826 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e5003e030 con 0x7f4e60073c70 2024-09-10T21:50:04.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.786+0000 7f4e56000640 1 -- 172.21.15.7:0/1920540826 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e50003e00 con 0x7f4e60073c70 2024-09-10T21:50:04.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.786+0000 7f4e56000640 1 -- 172.21.15.7:0/1920540826 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e50035040 con 0x7f4e60073c70 2024-09-10T21:50:04.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.786+0000 7f4e64a00640 1 -- 172.21.15.7:0/1920540826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e601f86a0 con 0x7f4e60073c70 2024-09-10T21:50:04.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.786+0000 7f4e64a00640 1 -- 172.21.15.7:0/1920540826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e601f8c60 con 0x7f4e60073c70 2024-09-10T21:50:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.787+0000 7f4e64a00640 1 -- 172.21.15.7:0/1920540826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e60073730 con 0x7f4e60073c70 2024-09-10T21:50:04.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.788+0000 7f4e56000640 1 -- 172.21.15.7:0/1920540826 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4e500049c0 con 0x7f4e60073c70 2024-09-10T21:50:04.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.789+0000 7f4e56000640 1 --2- 172.21.15.7:0/1920540826 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e40077fa0 0x7f4e4007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:04.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.789+0000 7f4e56000640 1 -- 172.21.15.7:0/1920540826 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f4e500ba760 con 0x7f4e60073c70 2024-09-10T21:50:04.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.789+0000 7f4e5d400640 1 --2- 172.21.15.7:0/1920540826 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e40077fa0 0x7f4e4007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:04.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.790+0000 7f4e5d400640 1 --2- 172.21.15.7:0/1920540826 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e40077fa0 0x7f4e4007a460 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c005fd0 tx=0x7f4e4c00b510 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:04.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:04.793+0000 7f4e56000640 1 -- 172.21.15.7:0/1920540826 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4e500831c0 con 0x7f4e60073c70 2024-09-10T21:50:05.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.026+0000 7f4e64a00640 1 -- 172.21.15.7:0/1920540826 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e600b1df0 con 0x7f4e60073c70 2024-09-10T21:50:05.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.026+0000 7f4e56000640 1 -- 172.21.15.7:0/1920540826 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5219 (secure 0 0 0) 0x7f4e50082ac0 con 0x7f4e60073c70 2024-09-10T21:50:05.028 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:05.028 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":85,"btime":"2024-09-10T21:50:02:448373+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/431253133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":431253133},{"type":"v1","addr":"172.21.15.133:6836","nonce":431253133}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7521,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3692199178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3692199178},{"type":"v1","addr":"172.21.15.133:6837","nonce":3692199178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/752989865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":752989865},{"type":"v1","addr":"172.21.15.7:6834","nonce":752989865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:49:58.872982+0000","modified":"2024-09-10T21:50:01.884201+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3075619232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3075619232},{"type":"v1","addr":"172.21.15.7:6835","nonce":3075619232}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7527,"qdb_cluster":[7527]},"id":7}]} 2024-09-10T21:50:05.028 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 85 2024-09-10T21:50:05.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.029+0000 7f4e64a00640 1 -- 172.21.15.7:0/1920540826 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e40077fa0 msgr2=0x7f4e4007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:05.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.029+0000 7f4e64a00640 1 --2- 172.21.15.7:0/1920540826 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e40077fa0 0x7f4e4007a460 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c005fd0 tx=0x7f4e4c00b510 comp rx=0 tx=0).stop 2024-09-10T21:50:05.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.029+0000 7f4e64a00640 1 -- 172.21.15.7:0/1920540826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60073c70 msgr2=0x7f4e601f7e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:05.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.029+0000 7f4e64a00640 1 --2- 172.21.15.7:0/1920540826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60073c70 0x7f4e601f7e50 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f4e50034040 tx=0x7f4e50003910 comp rx=0 tx=0).stop 2024-09-10T21:50:05.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.029+0000 7f4e64a00640 1 -- 172.21.15.7:0/1920540826 shutdown_connections 2024-09-10T21:50:05.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.029+0000 7f4e64a00640 1 --2- 172.21.15.7:0/1920540826 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e40077fa0 0x7f4e4007a460 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.029+0000 7f4e64a00640 1 --2- 172.21.15.7:0/1920540826 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e6015bc90 0x7f4e600c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.029+0000 7f4e64a00640 1 --2- 172.21.15.7:0/1920540826 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e600cd750 0x7f4e60152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.029+0000 7f4e64a00640 1 --2- 172.21.15.7:0/1920540826 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e60073c70 0x7f4e601f7e50 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.029+0000 7f4e64a00640 1 -- 172.21.15.7:0/1920540826 >> 172.21.15.7:0/1920540826 conn(0x7f4e600b16d0 msgr2=0x7f4e600740a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:05.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.029+0000 7f4e64a00640 1 -- 172.21.15.7:0/1920540826 shutdown_connections 2024-09-10T21:50:05.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.029+0000 7f4e64a00640 1 -- 172.21.15.7:0/1920540826 wait complete. 2024-09-10T21:50:05.401 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-09-10T21:50:05.400+0000 7f561f200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:50:05.449 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:49:58.872982+0000', 'modified': '2024-09-10T21:50:01.884201+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7527}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7527': {'gid': 7527, 'name': 'c', 'rank': 0, 'incarnation': 81, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.7:6835/3075619232', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6833', 'nonce': 3075619232}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 3075619232}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7527, 'qdb_cluster': [7527]} 2024-09-10T21:50:05.449 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:50:05.449 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T21:50:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.612+0000 7f372d800640 1 -- 172.21.15.7:0/1602823237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372815a9a0 msgr2=0x7f372815f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.612+0000 7f372d800640 1 --2- 172.21.15.7:0/1602823237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372815a9a0 0x7f372815f060 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f372806db70 tx=0x7f371802aa10 comp rx=0 tx=0).stop 2024-09-10T21:50:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.613+0000 7f372d800640 1 -- 172.21.15.7:0/1602823237 shutdown_connections 2024-09-10T21:50:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.613+0000 7f372d800640 1 --2- 172.21.15.7:0/1602823237 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f372806fb60 0x7f37280cc520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.613+0000 7f372d800640 1 --2- 172.21.15.7:0/1602823237 >> v2:172.21.15.133:3300/0 conn(0x7f37280ce270 0x7f372806f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.613+0000 7f372d800640 1 --2- 172.21.15.7:0/1602823237 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372815a9a0 0x7f372815f060 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.613+0000 7f372d800640 1 -- 172.21.15.7:0/1602823237 >> 172.21.15.7:0/1602823237 conn(0x7f37280b1660 msgr2=0x7f37280b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.613+0000 7f372d800640 1 -- 172.21.15.7:0/1602823237 shutdown_connections 2024-09-10T21:50:05.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.613+0000 7f372d800640 1 -- 172.21.15.7:0/1602823237 wait complete. 2024-09-10T21:50:05.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.614+0000 7f372d800640 1 Processor -- start 2024-09-10T21:50:05.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.614+0000 7f372d800640 1 -- start start 2024-09-10T21:50:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.615+0000 7f372d800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f372806fb60 0x7f3728155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.615+0000 7f372d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372815a9a0 0x7f372814f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.615+0000 7f372d800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3728150110 0x7f37280c8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.616+0000 7f372d800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f37280ccdf0 con 0x7f372815a9a0 2024-09-10T21:50:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.616+0000 7f372d800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f37280ccc70 con 0x7f372806fb60 2024-09-10T21:50:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.616+0000 7f372d800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f37280ccf70 con 0x7f3728150110 2024-09-10T21:50:05.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.616+0000 7f3726000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372815a9a0 0x7f372814f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.616+0000 7f3726000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372815a9a0 0x7f372814f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58846/0 (socket says 172.21.15.7:58846) 2024-09-10T21:50:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.616+0000 7f3726000640 1 -- 172.21.15.7:0/173927441 learned_addr learned my addr 172.21.15.7:0/173927441 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.616+0000 7f3726a00640 1 --2- 172.21.15.7:0/173927441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f372806fb60 0x7f3728155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.616+0000 7f3727400640 1 --2- 172.21.15.7:0/173927441 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3728150110 0x7f37280c8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.616+0000 7f3726000640 1 -- 172.21.15.7:0/173927441 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3728150110 msgr2=0x7f37280c8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:05.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.616+0000 7f3726000640 1 --2- 172.21.15.7:0/173927441 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3728150110 0x7f37280c8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.616+0000 7f3726000640 1 -- 172.21.15.7:0/173927441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f372806fb60 msgr2=0x7f3728155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:05.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.616+0000 7f3726000640 1 --2- 172.21.15.7:0/173927441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f372806fb60 0x7f3728155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.616+0000 7f3726000640 1 -- 172.21.15.7:0/173927441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37280b5180 con 0x7f372815a9a0 2024-09-10T21:50:05.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.616+0000 7f3726000640 1 --2- 172.21.15.7:0/173927441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372815a9a0 0x7f372814f8a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f371400d770 tx=0x7f371400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:05.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.617+0000 7f371ea00640 1 -- 172.21.15.7:0/173927441 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714017070 con 0x7f372815a9a0 2024-09-10T21:50:05.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.617+0000 7f371ea00640 1 -- 172.21.15.7:0/173927441 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3714009d20 con 0x7f372815a9a0 2024-09-10T21:50:05.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.617+0000 7f371ea00640 1 -- 172.21.15.7:0/173927441 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714004280 con 0x7f372815a9a0 2024-09-10T21:50:05.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.617+0000 7f372d800640 1 -- 172.21.15.7:0/173927441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37280b4bc0 con 0x7f372815a9a0 2024-09-10T21:50:05.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.617+0000 7f372d800640 1 -- 172.21.15.7:0/173927441 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37280c6cf0 con 0x7f372815a9a0 2024-09-10T21:50:05.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.619+0000 7f371ea00640 1 -- 172.21.15.7:0/173927441 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f37140035f0 con 0x7f372815a9a0 2024-09-10T21:50:05.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.620+0000 7f371ea00640 1 --2- 172.21.15.7:0/173927441 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3708077f70 0x7f370807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.620+0000 7f371ea00640 1 -- 172.21.15.7:0/173927441 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f3714099e90 con 0x7f372815a9a0 2024-09-10T21:50:05.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.620+0000 7f373087c740 1 Processor -- start 2024-09-10T21:50:05.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.620+0000 7f3726a00640 1 --2- 172.21.15.7:0/173927441 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3708077f70 0x7f370807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.620+0000 7f373087c740 1 -- start start 2024-09-10T21:50:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.621+0000 7f3726a00640 1 --2- 172.21.15.7:0/173927441 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3708077f70 0x7f370807a430 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3718035000 tx=0x7f371803a040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.621+0000 7f373087c740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a733778b0 0x564a73377c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.621+0000 7f373087c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a73332a20 0x564a73376390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.621+0000 7f373087c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a73332fa0 0x564a733768d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.621+0000 7f373087c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564a732b5430 con 0x564a73332a20 2024-09-10T21:50:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.621+0000 7f373087c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564a732a55d0 con 0x564a73332fa0 2024-09-10T21:50:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.621+0000 7f373087c740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x564a73294a00 con 0x564a733778b0 2024-09-10T21:50:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.622+0000 7f3726000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a73332a20 0x564a73376390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.622+0000 7f3726a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a733778b0 0x564a73377c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.622+0000 7f3726a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a733778b0 0x564a73377c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:59402/0 (socket says 172.21.15.7:59402) 2024-09-10T21:50:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.622+0000 7f3726a00640 1 -- 172.21.15.7:0/450781723 learned_addr learned my addr 172.21.15.7:0/450781723 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.622+0000 7f3726000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a73332a20 0x564a73376390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58862/0 (socket says 172.21.15.7:58862) 2024-09-10T21:50:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.622+0000 7f3727400640 1 --2- 172.21.15.7:0/450781723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a73332fa0 0x564a733768d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.622+0000 7f3726000640 1 -- 172.21.15.7:0/450781723 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a733778b0 msgr2=0x564a73377c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:05.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.622+0000 7f3726000640 1 --2- 172.21.15.7:0/450781723 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a733778b0 0x564a73377c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.622+0000 7f3726000640 1 -- 172.21.15.7:0/450781723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a73332fa0 msgr2=0x564a733768d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:05.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.622+0000 7f3726000640 1 --2- 172.21.15.7:0/450781723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a73332fa0 0x564a733768d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.622+0000 7f3726000640 1 -- 172.21.15.7:0/450781723 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a7325d520 con 0x564a73332a20 2024-09-10T21:50:05.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.622+0000 7f3726000640 1 --2- 172.21.15.7:0/450781723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a73332a20 0x564a73376390 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f3714098320 tx=0x7f37140621a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=672283a179576338 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:05.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.623+0000 7f3713400640 1 -- 172.21.15.7:0/450781723 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714020040 con 0x564a73332a20 2024-09-10T21:50:05.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.623+0000 7f3713400640 1 -- 172.21.15.7:0/450781723 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3714066cd0 con 0x564a73332a20 2024-09-10T21:50:05.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.623+0000 7f3713400640 1 -- 172.21.15.7:0/450781723 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3714067e60 con 0x564a73332a20 2024-09-10T21:50:05.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.623+0000 7f373087c740 1 -- 172.21.15.7:0/450781723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a73332a20 msgr2=0x564a73376390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:05.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.623+0000 7f373087c740 1 --2- 172.21.15.7:0/450781723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a73332a20 0x564a73376390 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f3714098320 tx=0x7f37140621a0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.623+0000 7f373087c740 1 -- 172.21.15.7:0/450781723 shutdown_connections 2024-09-10T21:50:05.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.623+0000 7f373087c740 1 --2- 172.21.15.7:0/450781723 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a73332fa0 0x564a733768d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.623+0000 7f373087c740 1 --2- 172.21.15.7:0/450781723 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a73332a20 0x564a73376390 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.631 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.623+0000 7f373087c740 1 --2- 172.21.15.7:0/450781723 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a733778b0 0x564a73377c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.623+0000 7f373087c740 1 -- 172.21.15.7:0/450781723 >> 172.21.15.7:0/450781723 conn(0x564a7332dff0 msgr2=0x564a73331a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.623+0000 7f373087c740 1 -- 172.21.15.7:0/450781723 shutdown_connections 2024-09-10T21:50:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.624+0000 7f373087c740 1 -- 172.21.15.7:0/450781723 wait complete. 2024-09-10T21:50:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.624+0000 7f373087c740 1 -- start start 2024-09-10T21:50:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.624+0000 7f373087c740 1 Processor -- start 2024-09-10T21:50:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.625+0000 7f373087c740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a73332fa0 0x564a7338ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.625+0000 7f373087c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a733778b0 0x564a73376780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.625+0000 7f373087c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a7334d390 0x564a73386330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.625+0000 7f373087c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564a7323b330 con 0x564a7334d390 2024-09-10T21:50:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.625+0000 7f373087c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564a73298810 con 0x564a733778b0 2024-09-10T21:50:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.625+0000 7f373087c740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x564a73005620 con 0x564a73332fa0 2024-09-10T21:50:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.626+0000 7f373087c740 10 client.0 _ll_register_callbacks cb 0x564a73240420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:50:05.632 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.626+0000 7f3727400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a7334d390 0x564a73386330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.626+0000 7f3727400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a7334d390 0x564a73386330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58864/0 (socket says 172.21.15.7:58864) 2024-09-10T21:50:05.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.626+0000 7f3727400640 1 -- 172.21.15.7:0/1102550947 learned_addr learned my addr 172.21.15.7:0/1102550947 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:05.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.626+0000 7f3726a00640 1 --2- 172.21.15.7:0/1102550947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a73332fa0 0x564a7338ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.626+0000 7f3726000640 1 --2- 172.21.15.7:0/1102550947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a733778b0 0x564a73376780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.626+0000 7f3727400640 1 -- 172.21.15.7:0/1102550947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a73332fa0 msgr2=0x564a7338ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:05.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.626+0000 7f3727400640 1 --2- 172.21.15.7:0/1102550947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a73332fa0 0x564a7338ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.626+0000 7f3727400640 1 -- 172.21.15.7:0/1102550947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a733778b0 msgr2=0x564a73376780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:05.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.626+0000 7f3727400640 1 --2- 172.21.15.7:0/1102550947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a733778b0 0x564a73376780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.626+0000 7f3727400640 1 -- 172.21.15.7:0/1102550947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a72ff6850 con 0x564a7334d390 2024-09-10T21:50:05.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.626+0000 7f3726000640 1 --2- 172.21.15.7:0/1102550947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a733778b0 0x564a73376780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:50:05.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.627+0000 7f3727400640 1 --2- 172.21.15.7:0/1102550947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a7334d390 0x564a73386330 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f372000ddd0 tx=0x7f3720012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a9354da3302c3fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:05.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.627+0000 7f3707e00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:50:05.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.627+0000 7f3707e00640 1 -- 172.21.15.7:0/1102550947 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372000f7e0 con 0x564a7334d390 2024-09-10T21:50:05.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.627+0000 7f3707e00640 1 -- 172.21.15.7:0/1102550947 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f372001caf0 con 0x564a7334d390 2024-09-10T21:50:05.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.627+0000 7f373087c740 1 -- 172.21.15.7:0/1102550947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3714066cd0 con 0x564a7334d390 2024-09-10T21:50:05.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.627+0000 7f3707e00640 1 -- 172.21.15.7:0/1102550947 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f372001ccf0 con 0x564a7334d390 2024-09-10T21:50:05.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.627+0000 7f373087c740 10 client.7653 fetch_fsmap learned FSMap version 85 2024-09-10T21:50:05.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.627+0000 7f373087c740 1 -- 172.21.15.7:0/1102550947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x564a7306d6b0 con 0x564a7334d390 2024-09-10T21:50:05.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.627+0000 7f3707e00640 1 -- 172.21.15.7:0/1102550947 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3714066cd0 con 0x564a7334d390 2024-09-10T21:50:05.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.627+0000 7f3707e00640 1 -- 172.21.15.7:0/1102550947 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 85) ==== 3502+0+0 (secure 0 0 0) 0x7f3720023050 con 0x564a7334d390 2024-09-10T21:50:05.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.627+0000 7f373087c740 10 client.7653 fetch_fsmap finished waiting for FSMap version 85 2024-09-10T21:50:05.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.628+0000 7f373087c740 10 client.7653 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:50:05.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.628+0000 7f373087c740 20 client.7653 populate_metadata read hostname 'smithi007' 2024-09-10T21:50:05.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.628+0000 7f373087c740 1 --2- 172.21.15.7:0/1102550947 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x564a7335b200 0x564a73349e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.628+0000 7f373087c740 4 client.7653 mds_command: new command op to 7527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:50:05.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.628+0000 7f373087c740 1 -- 172.21.15.7:0/1102550947 --> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a732a7520 con 0x564a7335b200 2024-09-10T21:50:05.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.628+0000 7f3726a00640 1 --2- 172.21.15.7:0/1102550947 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x564a7335b200 0x564a73349e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.628+0000 7f3726a00640 1 --2- 172.21.15.7:0/1102550947 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x564a7335b200 0x564a73349e20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe9170d382caaee5 server_cookie=54ef52958471bd6f in_seq=0 out_seq=0 2024-09-10T21:50:05.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.628+0000 7f3707e00640 10 client.7653 ms_handle_connect on v2:172.21.15.7:6833/3075619232 2024-09-10T21:50:05.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.629+0000 7f3707e00640 1 -- 172.21.15.7:0/1102550947 <== mds.0 v2:172.21.15.7:6833/3075619232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564a732a7520 con 0x564a7335b200 2024-09-10T21:50:05.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.629+0000 7f3707e00640 10 client.7653 handle_command_reply: tid=0 2024-09-10T21:50:05.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.629+0000 7f373087c740 1 client.7653 shutdown 2024-09-10T21:50:05.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.629+0000 7f373087c740 1 -- 172.21.15.7:0/1102550947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a7334d390 msgr2=0x564a73386330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:05.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.629+0000 7f373087c740 1 --2- 172.21.15.7:0/1102550947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a7334d390 0x564a73386330 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f372000ddd0 tx=0x7f3720012820 comp rx=0 tx=0).stop 2024-09-10T21:50:05.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.630+0000 7f373087c740 1 -- 172.21.15.7:0/1102550947 shutdown_connections 2024-09-10T21:50:05.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.630+0000 7f373087c740 1 --2- 172.21.15.7:0/1102550947 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x564a7335b200 0x564a73349e20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.630+0000 7f373087c740 1 --2- 172.21.15.7:0/1102550947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a733778b0 0x564a73376780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.630+0000 7f373087c740 1 --2- 172.21.15.7:0/1102550947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a7334d390 0x564a73386330 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.630+0000 7f373087c740 1 --2- 172.21.15.7:0/1102550947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a73332fa0 0x564a7338ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.630+0000 7f3707e00640 0 client.7653 ms_handle_reset on v2:172.21.15.7:6833/3075619232 2024-09-10T21:50:05.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.630+0000 7f373087c740 1 -- 172.21.15.7:0/1102550947 >> 172.21.15.7:0/1102550947 conn(0x564a73332a20 msgr2=0x564a7332cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.630+0000 7f373087c740 1 -- 172.21.15.7:0/1102550947 shutdown_connections 2024-09-10T21:50:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.630+0000 7f373087c740 1 -- 172.21.15.7:0/1102550947 wait complete. 2024-09-10T21:50:05.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.630+0000 7f373087c740 20 client.7653 trim_cache size 0 max 16384 2024-09-10T21:50:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.649+0000 7f373087c740 1 Processor -- start 2024-09-10T21:50:05.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.649+0000 7f373087c740 1 -- start start 2024-09-10T21:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.650+0000 7f373087c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a73332fa0 0x564a73376780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.650+0000 7f373087c740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a7334d390 0x564a73357fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.650+0000 7f373087c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a733778b0 0x564a7332cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.650+0000 7f373087c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f37282010c0 con 0x564a73332fa0 2024-09-10T21:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.650+0000 7f373087c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3728154dd0 con 0x564a733778b0 2024-09-10T21:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.650+0000 7f373087c740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f3728154160 con 0x564a7334d390 2024-09-10T21:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.650+0000 7f3726a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a73332fa0 0x564a73376780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.650+0000 7f3726a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a73332fa0 0x564a73376780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58866/0 (socket says 172.21.15.7:58866) 2024-09-10T21:50:05.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.650+0000 7f3726a00640 1 -- 172.21.15.7:0/2074253367 learned_addr learned my addr 172.21.15.7:0/2074253367 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:05.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.650+0000 7f3726a00640 1 -- 172.21.15.7:0/2074253367 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a7334d390 msgr2=0x564a73357fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:05.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.650+0000 7f3726a00640 1 --2- 172.21.15.7:0/2074253367 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a7334d390 0x564a73357fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.650+0000 7f3726a00640 1 -- 172.21.15.7:0/2074253367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a733778b0 msgr2=0x564a7332cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.650+0000 7f3726a00640 1 --2- 172.21.15.7:0/2074253367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a733778b0 0x564a7332cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.650+0000 7f3726a00640 1 -- 172.21.15.7:0/2074253367 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a732a5270 con 0x564a73332fa0 2024-09-10T21:50:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.651+0000 7f3726a00640 1 --2- 172.21.15.7:0/2074253367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a73332fa0 0x564a73376780 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f3718005fd0 tx=0x7f371802aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148c6625c8db4fb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.651+0000 7f3712a00640 1 -- 172.21.15.7:0/2074253367 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718033050 con 0x564a73332fa0 2024-09-10T21:50:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.651+0000 7f3712a00640 1 -- 172.21.15.7:0/2074253367 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37180388d0 con 0x564a73332fa0 2024-09-10T21:50:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.651+0000 7f3712a00640 1 -- 172.21.15.7:0/2074253367 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3718038c80 con 0x564a73332fa0 2024-09-10T21:50:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.652+0000 7f373087c740 1 -- 172.21.15.7:0/2074253367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a73332fa0 msgr2=0x564a73376780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.652+0000 7f373087c740 1 --2- 172.21.15.7:0/2074253367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a73332fa0 0x564a73376780 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f3718005fd0 tx=0x7f371802aa10 comp rx=0 tx=0).stop 2024-09-10T21:50:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.652+0000 7f373087c740 1 -- 172.21.15.7:0/2074253367 shutdown_connections 2024-09-10T21:50:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.652+0000 7f373087c740 1 --2- 172.21.15.7:0/2074253367 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a733778b0 0x564a7332cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.652+0000 7f373087c740 1 --2- 172.21.15.7:0/2074253367 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a7334d390 0x564a73357fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.652+0000 7f373087c740 1 --2- 172.21.15.7:0/2074253367 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a73332fa0 0x564a73376780 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.652+0000 7f373087c740 1 -- 172.21.15.7:0/2074253367 >> 172.21.15.7:0/2074253367 conn(0x564a73332a20 msgr2=0x564a73330a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:05.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.652+0000 7f373087c740 1 -- 172.21.15.7:0/2074253367 shutdown_connections 2024-09-10T21:50:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.652+0000 7f373087c740 1 -- 172.21.15.7:0/2074253367 wait complete. 2024-09-10T21:50:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.652+0000 7f373087c740 1 -- start start 2024-09-10T21:50:05.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.653+0000 7f373087c740 1 Processor -- start 2024-09-10T21:50:05.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.653+0000 7f373087c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a73332fa0 0x564a733fad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.653+0000 7f373087c740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a7334d390 0x564a73331b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.653+0000 7f373087c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a733778b0 0x564a7332cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.654+0000 7f373087c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564a7334b200 con 0x564a733778b0 2024-09-10T21:50:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.654+0000 7f373087c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3728154780 con 0x564a73332fa0 2024-09-10T21:50:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.654+0000 7f373087c740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x564a7335b680 con 0x564a7334d390 2024-09-10T21:50:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.654+0000 7f373087c740 10 client.0 _ll_register_callbacks cb 0x7f372000d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:50:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.654+0000 7f3727400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a733778b0 0x564a7332cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.654+0000 7f3727400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a733778b0 0x564a7332cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58868/0 (socket says 172.21.15.7:58868) 2024-09-10T21:50:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.654+0000 7f3727400640 1 -- 172.21.15.7:0/2523200649 learned_addr learned my addr 172.21.15.7:0/2523200649 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.654+0000 7f3726a00640 1 --2- 172.21.15.7:0/2523200649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a73332fa0 0x564a733fad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.654+0000 7f3726000640 1 --2- 172.21.15.7:0/2523200649 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a7334d390 0x564a73331b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.654+0000 7f3727400640 1 -- 172.21.15.7:0/2523200649 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a7334d390 msgr2=0x564a73331b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.654+0000 7f3727400640 1 --2- 172.21.15.7:0/2523200649 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a7334d390 0x564a73331b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.654+0000 7f3727400640 1 -- 172.21.15.7:0/2523200649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a73332fa0 msgr2=0x564a733fad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:05.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.654+0000 7f3727400640 1 --2- 172.21.15.7:0/2523200649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a73332fa0 0x564a733fad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.654+0000 7f3727400640 1 -- 172.21.15.7:0/2523200649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a73376c00 con 0x564a733778b0 2024-09-10T21:50:05.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.654+0000 7f3726000640 1 --2- 172.21.15.7:0/2523200649 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a7334d390 0x564a73331b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:50:05.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.654+0000 7f3727400640 1 --2- 172.21.15.7:0/2523200649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a733778b0 0x564a7332cd20 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f3720012860 tx=0x7f372000f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d14b4e140c7ac7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:05.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.654+0000 7f3707e00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:50:05.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.655+0000 7f3707e00640 1 -- 172.21.15.7:0/2523200649 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720012d20 con 0x564a733778b0 2024-09-10T21:50:05.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.655+0000 7f373087c740 1 -- 172.21.15.7:0/2523200649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37180388d0 con 0x564a733778b0 2024-09-10T21:50:05.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.655+0000 7f3707e00640 1 -- 172.21.15.7:0/2523200649 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f372000fa00 con 0x564a733778b0 2024-09-10T21:50:05.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.655+0000 7f3707e00640 1 -- 172.21.15.7:0/2523200649 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f3720032050 con 0x564a733778b0 2024-09-10T21:50:05.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.655+0000 7f373087c740 10 client.7659 fetch_fsmap learned FSMap version 85 2024-09-10T21:50:05.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.655+0000 7f373087c740 1 -- 172.21.15.7:0/2523200649 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x564a733320e0 con 0x564a733778b0 2024-09-10T21:50:05.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.655+0000 7f3707e00640 1 -- 172.21.15.7:0/2523200649 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37180388d0 con 0x564a733778b0 2024-09-10T21:50:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.655+0000 7f3707e00640 1 -- 172.21.15.7:0/2523200649 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 85) ==== 3502+0+0 (secure 0 0 0) 0x7f3720035020 con 0x564a733778b0 2024-09-10T21:50:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.655+0000 7f373087c740 10 client.7659 fetch_fsmap finished waiting for FSMap version 85 2024-09-10T21:50:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.655+0000 7f373087c740 10 client.7659 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:50:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.655+0000 7f373087c740 20 client.7659 populate_metadata read hostname 'smithi007' 2024-09-10T21:50:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.655+0000 7f373087c740 1 --2- 172.21.15.7:0/2523200649 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x564a733f7300 0x564a7336e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.655+0000 7f373087c740 4 client.7659 mds_command: new command op to 7527 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:50:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.655+0000 7f373087c740 1 -- 172.21.15.7:0/2523200649 --> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] -- command(tid 0: {"prefix": "status"}) -- 0x564a732a2fd0 con 0x564a733f7300 2024-09-10T21:50:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.656+0000 7f3726a00640 1 --2- 172.21.15.7:0/2523200649 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x564a733f7300 0x564a7336e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.656+0000 7f3726a00640 1 --2- 172.21.15.7:0/2523200649 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x564a733f7300 0x564a7336e510 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dea2767f7d9cf436 server_cookie=2fd1deb0b79f64de in_seq=0 out_seq=0 2024-09-10T21:50:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.656+0000 7f3707e00640 10 client.7659 ms_handle_connect on v2:172.21.15.7:6833/3075619232 2024-09-10T21:50:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.656+0000 7f3707e00640 1 -- 172.21.15.7:0/2523200649 <== mds.0 v2:172.21.15.7:6833/3075619232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x564a732a2fd0 con 0x564a733f7300 2024-09-10T21:50:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.656+0000 7f3707e00640 10 client.7659 handle_command_reply: tid=0 2024-09-10T21:50:05.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.656+0000 7f373087c740 1 client.7659 shutdown 2024-09-10T21:50:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.657+0000 7f373087c740 1 -- 172.21.15.7:0/2523200649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a733778b0 msgr2=0x564a7332cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.657+0000 7f373087c740 1 --2- 172.21.15.7:0/2523200649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a733778b0 0x564a7332cd20 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f3720012860 tx=0x7f372000f810 comp rx=0 tx=0).stop 2024-09-10T21:50:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.657+0000 7f373087c740 1 -- 172.21.15.7:0/2523200649 shutdown_connections 2024-09-10T21:50:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.657+0000 7f373087c740 1 --2- 172.21.15.7:0/2523200649 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x564a733f7300 0x564a7336e510 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.657+0000 7f373087c740 1 --2- 172.21.15.7:0/2523200649 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564a733778b0 0x564a7332cd20 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.657+0000 7f373087c740 1 --2- 172.21.15.7:0/2523200649 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564a7334d390 0x564a73331b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.657+0000 7f373087c740 1 --2- 172.21.15.7:0/2523200649 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564a73332fa0 0x564a733fad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.657+0000 7f3707e00640 0 client.7659 ms_handle_reset on v2:172.21.15.7:6833/3075619232 2024-09-10T21:50:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.657+0000 7f373087c740 1 -- 172.21.15.7:0/2523200649 >> 172.21.15.7:0/2523200649 conn(0x564a73332a20 msgr2=0x564a7336f640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.657+0000 7f373087c740 1 -- 172.21.15.7:0/2523200649 shutdown_connections 2024-09-10T21:50:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.657+0000 7f373087c740 1 -- 172.21.15.7:0/2523200649 wait complete. 2024-09-10T21:50:05.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.657+0000 7f373087c740 20 client.7659 trim_cache size 0 max 16384 2024-09-10T21:50:05.663 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:50:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2029451a-3c26-48c7-b874-0597197278ac", 2024-09-10T21:50:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-10T21:50:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 7527, 2024-09-10T21:50:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T21:50:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T21:50:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T21:50:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 6.2323072789999996, 2024-09-10T21:50:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 85, 2024-09-10T21:50:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 101, 2024-09-10T21:50:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 101, 2024-09-10T21:50:05.663 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 11.512638577000001 2024-09-10T21:50:05.664 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:50:05.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.658+0000 7f372d800640 1 -- 172.21.15.7:0/173927441 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3708077f70 msgr2=0x7f370807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:05.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.658+0000 7f372d800640 1 --2- 172.21.15.7:0/173927441 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3708077f70 0x7f370807a430 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3718035000 tx=0x7f371803a040 comp rx=0 tx=0).stop 2024-09-10T21:50:05.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.658+0000 7f372d800640 1 -- 172.21.15.7:0/173927441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372815a9a0 msgr2=0x7f372814f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:05.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.658+0000 7f372d800640 1 --2- 172.21.15.7:0/173927441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372815a9a0 0x7f372814f8a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f371400d770 tx=0x7f371400dc30 comp rx=0 tx=0).stop 2024-09-10T21:50:05.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.658+0000 7f372d800640 1 -- 172.21.15.7:0/173927441 shutdown_connections 2024-09-10T21:50:05.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.658+0000 7f372d800640 1 --2- 172.21.15.7:0/173927441 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3708077f70 0x7f370807a430 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.658+0000 7f372d800640 1 --2- 172.21.15.7:0/173927441 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3728150110 0x7f37280c8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.658+0000 7f372d800640 1 --2- 172.21.15.7:0/173927441 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f372815a9a0 0x7f372814f8a0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.658+0000 7f372d800640 1 --2- 172.21.15.7:0/173927441 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f372806fb60 0x7f3728155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.658+0000 7f372d800640 1 -- 172.21.15.7:0/173927441 >> 172.21.15.7:0/173927441 conn(0x7f37280b1660 msgr2=0x7f3728072d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:05.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.658+0000 7f372d800640 1 -- 172.21.15.7:0/173927441 shutdown_connections 2024-09-10T21:50:05.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.658+0000 7f372d800640 1 -- 172.21.15.7:0/173927441 wait complete. 2024-09-10T21:50:05.673 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:50:05.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.832+0000 7f52d0a00640 1 -- 172.21.15.7:0/4014247616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc15bc90 msgr2=0x7f52cc0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.832+0000 7f52d0a00640 1 --2- 172.21.15.7:0/4014247616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc15bc90 0x7f52cc0ccef0 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f52cc0cf860 tx=0x7f52bc02a4d0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.832+0000 7f52d0a00640 1 -- 172.21.15.7:0/4014247616 shutdown_connections 2024-09-10T21:50:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.832+0000 7f52d0a00640 1 --2- 172.21.15.7:0/4014247616 >> v2:172.21.15.133:3300/0 conn(0x7f52cc0cd750 0x7f52cc073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.832+0000 7f52d0a00640 1 --2- 172.21.15.7:0/4014247616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc15bc90 0x7f52cc0ccef0 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.832+0000 7f52d0a00640 1 -- 172.21.15.7:0/4014247616 >> 172.21.15.7:0/4014247616 conn(0x7f52cc0b16d0 msgr2=0x7f52cc0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.832+0000 7f52d0a00640 1 -- 172.21.15.7:0/4014247616 shutdown_connections 2024-09-10T21:50:05.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.832+0000 7f52d0a00640 1 -- 172.21.15.7:0/4014247616 wait complete. 2024-09-10T21:50:05.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.834+0000 7f52d0a00640 1 Processor -- start 2024-09-10T21:50:05.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.834+0000 7f52d0a00640 1 -- start start 2024-09-10T21:50:05.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.835+0000 7f52d0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc073c70 0x7f52cc14df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.835+0000 7f52d0a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52cc0cd750 0x7f52cc15c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.835+0000 7f52d0a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52cc15bc90 0x7f52cc0b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.835+0000 7f52d0a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc06fcf0 con 0x7f52cc073c70 2024-09-10T21:50:05.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.835+0000 7f52d0a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f52cc1606c0 con 0x7f52cc15bc90 2024-09-10T21:50:05.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.835+0000 7f52d0a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f52cc06fe70 con 0x7f52cc0cd750 2024-09-10T21:50:05.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.835+0000 7f52c9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc073c70 0x7f52cc14df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.835+0000 7f52c9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc073c70 0x7f52cc14df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58882/0 (socket says 172.21.15.7:58882) 2024-09-10T21:50:05.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.835+0000 7f52c9e00640 1 -- 172.21.15.7:0/506597262 learned_addr learned my addr 172.21.15.7:0/506597262 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:05.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.835+0000 7f52c1400640 1 --2- 172.21.15.7:0/506597262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52cc0cd750 0x7f52cc15c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.835+0000 7f52ca800640 1 --2- 172.21.15.7:0/506597262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52cc15bc90 0x7f52cc0b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.836+0000 7f52c9e00640 1 -- 172.21.15.7:0/506597262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52cc0cd750 msgr2=0x7f52cc15c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:05.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.836+0000 7f52c9e00640 1 --2- 172.21.15.7:0/506597262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52cc0cd750 0x7f52cc15c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.836+0000 7f52c9e00640 1 -- 172.21.15.7:0/506597262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52cc15bc90 msgr2=0x7f52cc0b1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:05.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.836+0000 7f52c9e00640 1 --2- 172.21.15.7:0/506597262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52cc15bc90 0x7f52cc0b1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:05.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.836+0000 7f52c9e00640 1 -- 172.21.15.7:0/506597262 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52cc0c6ee0 con 0x7f52cc073c70 2024-09-10T21:50:05.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.836+0000 7f52c1400640 1 --2- 172.21.15.7:0/506597262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52cc0cd750 0x7f52cc15c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:50:05.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.836+0000 7f52c9e00640 1 --2- 172.21.15.7:0/506597262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc073c70 0x7f52cc14df20 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f52bc034040 tx=0x7f52bc003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:05.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.836+0000 7f52c2a00640 1 -- 172.21.15.7:0/506597262 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc03e030 con 0x7f52cc073c70 2024-09-10T21:50:05.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.836+0000 7f52c2a00640 1 -- 172.21.15.7:0/506597262 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52bc003e00 con 0x7f52cc073c70 2024-09-10T21:50:05.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.836+0000 7f52d0a00640 1 -- 172.21.15.7:0/506597262 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52cc15cda0 con 0x7f52cc073c70 2024-09-10T21:50:05.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.836+0000 7f52c2a00640 1 -- 172.21.15.7:0/506597262 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52bc035040 con 0x7f52cc073c70 2024-09-10T21:50:05.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.836+0000 7f52d0a00640 1 -- 172.21.15.7:0/506597262 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52cc15d360 con 0x7f52cc073c70 2024-09-10T21:50:05.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.838+0000 7f52d0a00640 1 -- 172.21.15.7:0/506597262 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52cc073730 con 0x7f52cc073c70 2024-09-10T21:50:05.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.839+0000 7f52c2a00640 1 -- 172.21.15.7:0/506597262 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f52bc0049c0 con 0x7f52cc073c70 2024-09-10T21:50:05.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.840+0000 7f52c2a00640 1 --2- 172.21.15.7:0/506597262 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f52a8077f70 0x7f52a807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:05.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.840+0000 7f52c2a00640 1 -- 172.21.15.7:0/506597262 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f52bc0ba710 con 0x7f52cc073c70 2024-09-10T21:50:05.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.840+0000 7f52c1400640 1 --2- 172.21.15.7:0/506597262 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f52a8077f70 0x7f52a807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:05.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.841+0000 7f52c1400640 1 --2- 172.21.15.7:0/506597262 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f52a8077f70 0x7f52a807a430 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005fd0 tx=0x7f52b4005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:05.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:05.844+0000 7f52c2a00640 1 -- 172.21.15.7:0/506597262 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f52cc0c6ee0 con 0x7f52cc073c70 2024-09-10T21:50:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:06.079+0000 7f52d0a00640 1 -- 172.21.15.7:0/506597262 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f52cc0c3770 con 0x7f52cc073c70 2024-09-10T21:50:06.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:06.080+0000 7f52c2a00640 1 -- 172.21.15.7:0/506597262 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v85) ==== 53+0+73 (secure 0 0 0) 0x7f52bc082b20 con 0x7f52cc073c70 2024-09-10T21:50:06.081 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:50:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:06.082+0000 7f52d0a00640 1 -- 172.21.15.7:0/506597262 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f52a8077f70 msgr2=0x7f52a807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:06.082+0000 7f52d0a00640 1 --2- 172.21.15.7:0/506597262 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f52a8077f70 0x7f52a807a430 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f52b4005fd0 tx=0x7f52b4005850 comp rx=0 tx=0).stop 2024-09-10T21:50:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:06.082+0000 7f52d0a00640 1 -- 172.21.15.7:0/506597262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc073c70 msgr2=0x7f52cc14df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:06.082+0000 7f52d0a00640 1 --2- 172.21.15.7:0/506597262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc073c70 0x7f52cc14df20 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f52bc034040 tx=0x7f52bc003910 comp rx=0 tx=0).stop 2024-09-10T21:50:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:06.082+0000 7f52d0a00640 1 -- 172.21.15.7:0/506597262 shutdown_connections 2024-09-10T21:50:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:06.082+0000 7f52d0a00640 1 --2- 172.21.15.7:0/506597262 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f52a8077f70 0x7f52a807a430 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:06.082+0000 7f52d0a00640 1 --2- 172.21.15.7:0/506597262 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52cc15bc90 0x7f52cc0b1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:06.082+0000 7f52d0a00640 1 --2- 172.21.15.7:0/506597262 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52cc0cd750 0x7f52cc15c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:06.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:06.082+0000 7f52d0a00640 1 --2- 172.21.15.7:0/506597262 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52cc073c70 0x7f52cc14df20 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:06.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:06.082+0000 7f52d0a00640 1 -- 172.21.15.7:0/506597262 >> 172.21.15.7:0/506597262 conn(0x7f52cc0b16d0 msgr2=0x7f52cc1588e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:06.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:06.082+0000 7f52d0a00640 1 -- 172.21.15.7:0/506597262 shutdown_connections 2024-09-10T21:50:06.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:06.082+0000 7f52d0a00640 1 -- 172.21.15.7:0/506597262 wait complete. 2024-09-10T21:50:06.098 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:50:06.098 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:50:06.098 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-10T21:50:06.098 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:50:06.098 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:50:06.098 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:50:06.098 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:50:06.098 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:50:06.099 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:50:06.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:06.099 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-10T21:50:06.135 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:50:06.135 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:50:06.135 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:50:06.135 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:50:06.135 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T21:50:06.135 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:50:06.135 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:50:06.135 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T21:50:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-10T21:50:06.136 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:50:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T21:50:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-10T21:50:06.136 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:50:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-10T21:50:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-10T21:50:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T21:50:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42139sec preferred_lft 42139sec 2024-09-10T21:50:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-10T21:50:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:50:06.136 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:50:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-10T21:50:06.136 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-10T21:50:06.137 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:50:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff 2024-09-10T21:50:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:50:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:50:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::3c3d:dbff:fe02:4c15/64 scope link 2024-09-10T21:50:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:50:06.137 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:50:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether f6:f9:69:5a:86:f5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:50:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f4f9:69ff:fe5a:86f5/64 scope link 2024-09-10T21:50:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:50:06.137 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:50:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:50:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::433:5cff:fe44:4a5b/64 scope link 2024-09-10T21:50:06.137 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:50:06.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:06.138 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:50:06.228 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:50:06.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:06.231 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-10T21:50:06.267 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-10T21:50:06.267 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:50:06.267 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T21:50:06.267 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T21:50:06.267 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:50:06.268 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:50:06.268 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:50:06.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:50:06.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:50:06.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:50:06.268 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:50:06.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:06 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:50:06.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:50:06.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:06.467 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:50:06.503 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:50:06.503 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:50:06.503 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:50:06.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:06.504 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-10T21:50:06.598 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:50:06.598 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:50:06.633 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:50:06.633 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:50:06.634 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:50:06.708 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:50:06.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:06.709 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:50:06.839 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:50:06.839 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:50:06.867 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:50:06.867 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:50:06.867 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.133:3300,172.21.15.133:3301" 2024-09-10T21:50:06.867 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@2029451a-3c26-48c7-b874-0597197278ac.cephfs=/ 2024-09-10T21:50:06.867 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.133:3300/172.21.15.133:3301" will pass to kernel 2024-09-10T21:50:06.868 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T21:50:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:50:06.868 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:50:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:50:06.869 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:50:06.871 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:50:06.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:06.872 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:50:06.928 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:50:07.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:50:07.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:50:07.025 INFO:teuthology.orchestra.run.smithi007.stdout:client7108 2024-09-10T21:50:07.025 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:50:07.025 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:50:07.062 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:50:07.062 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:50:07.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:07.137 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file1_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-09-10T21:50:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:6+0 records in 2024-09-10T21:50:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:6+0 records out 2024-09-10T21:50:07.302 INFO:teuthology.orchestra.run.smithi007.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0834369 s, 75.4 MB/s 2024-09-10T21:50:07.303 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:50:07.303 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:50:07.338 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:50:07.338 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:50:07.414 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:50:07.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.617+0000 7f1d47c00640 1 -- 172.21.15.7:0/1958377672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d40073cf0 msgr2=0x7f1d40074130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:07.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.617+0000 7f1d47c00640 1 --2- 172.21.15.7:0/1958377672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d40073cf0 0x7f1d40074130 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1d28009830 tx=0x7f1d2802a510 comp rx=0 tx=0).stop 2024-09-10T21:50:07.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.617+0000 7f1d47c00640 1 -- 172.21.15.7:0/1958377672 shutdown_connections 2024-09-10T21:50:07.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.617+0000 7f1d47c00640 1 --2- 172.21.15.7:0/1958377672 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d4015e7b0 0x7f1d4015ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:07.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.617+0000 7f1d47c00640 1 --2- 172.21.15.7:0/1958377672 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d40073cf0 0x7f1d40074130 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:07.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.617+0000 7f1d47c00640 1 -- 172.21.15.7:0/1958377672 >> 172.21.15.7:0/1958377672 conn(0x7f1d400b1660 msgr2=0x7f1d400b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:07.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.618+0000 7f1d47c00640 1 -- 172.21.15.7:0/1958377672 shutdown_connections 2024-09-10T21:50:07.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.618+0000 7f1d47c00640 1 -- 172.21.15.7:0/1958377672 wait complete. 2024-09-10T21:50:07.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.619+0000 7f1d47c00640 1 Processor -- start 2024-09-10T21:50:07.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.619+0000 7f1d47c00640 1 -- start start 2024-09-10T21:50:07.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.620+0000 7f1d47c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d40073cf0 0x7f1d401f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:07.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.620+0000 7f1d47c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d400cc940 0x7f1d400c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:07.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.620+0000 7f1d47c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d4015e7b0 0x7f1d4014e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:07.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.620+0000 7f1d47c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4006f0c0 con 0x7f1d400cc940 2024-09-10T21:50:07.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.620+0000 7f1d47c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1d4015f180 con 0x7f1d4015e7b0 2024-09-10T21:50:07.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.620+0000 7f1d47c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f1d4015f300 con 0x7f1d40073cf0 2024-09-10T21:50:07.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.620+0000 7f1d37e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d400cc940 0x7f1d400c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:07.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.620+0000 7f1d37e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d400cc940 0x7f1d400c5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58884/0 (socket says 172.21.15.7:58884) 2024-09-10T21:50:07.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.620+0000 7f1d37e00640 1 -- 172.21.15.7:0/3347620456 learned_addr learned my addr 172.21.15.7:0/3347620456 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:07.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.621+0000 7f1d45c00640 1 --2- 172.21.15.7:0/3347620456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d4015e7b0 0x7f1d4014e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:07.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.621+0000 7f1d45200640 1 --2- 172.21.15.7:0/3347620456 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d40073cf0 0x7f1d401f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:07.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.621+0000 7f1d37e00640 1 -- 172.21.15.7:0/3347620456 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d40073cf0 msgr2=0x7f1d401f88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:07.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.621+0000 7f1d37e00640 1 --2- 172.21.15.7:0/3347620456 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d40073cf0 0x7f1d401f88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:07.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.621+0000 7f1d37e00640 1 -- 172.21.15.7:0/3347620456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d4015e7b0 msgr2=0x7f1d4014e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:07.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.621+0000 7f1d37e00640 1 --2- 172.21.15.7:0/3347620456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d4015e7b0 0x7f1d4014e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:07.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.621+0000 7f1d37e00640 1 -- 172.21.15.7:0/3347620456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d401f8e20 con 0x7f1d400cc940 2024-09-10T21:50:07.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.621+0000 7f1d37e00640 1 --2- 172.21.15.7:0/3347620456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d400cc940 0x7f1d400c5fd0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f1d2802ab00 tx=0x7f1d28032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:07.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.621+0000 7f1d3d600640 1 -- 172.21.15.7:0/3347620456 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28040070 con 0x7f1d400cc940 2024-09-10T21:50:07.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.622+0000 7f1d3d600640 1 -- 172.21.15.7:0/3347620456 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d28005310 con 0x7f1d400cc940 2024-09-10T21:50:07.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.622+0000 7f1d3d600640 1 -- 172.21.15.7:0/3347620456 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d28005c90 con 0x7f1d400cc940 2024-09-10T21:50:07.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.622+0000 7f1d47c00640 1 -- 172.21.15.7:0/3347620456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d400c6510 con 0x7f1d400cc940 2024-09-10T21:50:07.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.622+0000 7f1d47c00640 1 -- 172.21.15.7:0/3347620456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d400c5210 con 0x7f1d400cc940 2024-09-10T21:50:07.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.623+0000 7f1d47c00640 1 -- 172.21.15.7:0/3347620456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d40074130 con 0x7f1d400cc940 2024-09-10T21:50:07.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.624+0000 7f1d3d600640 1 -- 172.21.15.7:0/3347620456 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1d280047a0 con 0x7f1d400cc940 2024-09-10T21:50:07.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.625+0000 7f1d3d600640 1 --2- 172.21.15.7:0/3347620456 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1d1c077f70 0x7f1d1c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:07.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.625+0000 7f1d3d600640 1 -- 172.21.15.7:0/3347620456 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f1d280bafe0 con 0x7f1d400cc940 2024-09-10T21:50:07.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.625+0000 7f1d45200640 1 --2- 172.21.15.7:0/3347620456 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1d1c077f70 0x7f1d1c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:07.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.626+0000 7f1d45200640 1 --2- 172.21.15.7:0/3347620456 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1d1c077f70 0x7f1d1c07a430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1d300016b0 tx=0x7f1d30009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:07.630 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.629+0000 7f1d3d600640 1 -- 172.21.15.7:0/3347620456 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1d401f8e20 con 0x7f1d400cc940 2024-09-10T21:50:07.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.840+0000 7f1d47c00640 1 -- 172.21.15.7:0/3347620456 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f1d400b1ae0 con 0x7f1d400cc940 2024-09-10T21:50:07.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.840+0000 7f1d3d600640 1 -- 172.21.15.7:0/3347620456 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v101) ==== 80+0+0 (secure 0 0 0) 0x7f1d28083b30 con 0x7f1d400cc940 2024-09-10T21:50:07.841 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-10T21:50:07.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.843+0000 7f1d47c00640 1 -- 172.21.15.7:0/3347620456 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1d1c077f70 msgr2=0x7f1d1c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:07.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.843+0000 7f1d47c00640 1 --2- 172.21.15.7:0/3347620456 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1d1c077f70 0x7f1d1c07a430 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1d300016b0 tx=0x7f1d30009040 comp rx=0 tx=0).stop 2024-09-10T21:50:07.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.843+0000 7f1d47c00640 1 -- 172.21.15.7:0/3347620456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d400cc940 msgr2=0x7f1d400c5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:07.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.843+0000 7f1d47c00640 1 --2- 172.21.15.7:0/3347620456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d400cc940 0x7f1d400c5fd0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f1d2802ab00 tx=0x7f1d28032c10 comp rx=0 tx=0).stop 2024-09-10T21:50:07.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.843+0000 7f1d47c00640 1 -- 172.21.15.7:0/3347620456 shutdown_connections 2024-09-10T21:50:07.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.843+0000 7f1d47c00640 1 --2- 172.21.15.7:0/3347620456 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1d1c077f70 0x7f1d1c07a430 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:07.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.843+0000 7f1d47c00640 1 --2- 172.21.15.7:0/3347620456 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d4015e7b0 0x7f1d4014e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:07.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.843+0000 7f1d47c00640 1 --2- 172.21.15.7:0/3347620456 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d400cc940 0x7f1d400c5fd0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:07.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.843+0000 7f1d47c00640 1 --2- 172.21.15.7:0/3347620456 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d40073cf0 0x7f1d401f88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:07.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.843+0000 7f1d47c00640 1 -- 172.21.15.7:0/3347620456 >> 172.21.15.7:0/3347620456 conn(0x7f1d400b1660 msgr2=0x7f1d400cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:07.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.843+0000 7f1d47c00640 1 -- 172.21.15.7:0/3347620456 shutdown_connections 2024-09-10T21:50:07.845 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:07.843+0000 7f1d47c00640 1 -- 172.21.15.7:0/3347620456 wait complete. 2024-09-10T21:50:07.859 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:50:07.860 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:07.860 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:50:07.926 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:50:07.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:07.926 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:50:07.962 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:50:08.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.162+0000 7f7bdec00640 1 -- 172.21.15.7:0/4162291521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd80ae020 msgr2=0x7f7bd80ae4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:08.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.162+0000 7f7bdec00640 1 --2- 172.21.15.7:0/4162291521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd80ae020 0x7f7bd80ae4d0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f7bd0005fd0 tx=0x7f7bd002a8b0 comp rx=0 tx=0).stop 2024-09-10T21:50:08.163 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.162+0000 7f7bdec00640 1 -- 172.21.15.7:0/4162291521 shutdown_connections 2024-09-10T21:50:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.162+0000 7f7bdec00640 1 --2- 172.21.15.7:0/4162291521 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd80ae020 0x7f7bd80ae4d0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.162+0000 7f7bdec00640 1 --2- 172.21.15.7:0/4162291521 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7bd80cbd50 0x7f7bd80cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.162+0000 7f7bdec00640 1 -- 172.21.15.7:0/4162291521 >> 172.21.15.7:0/4162291521 conn(0x7f7bd80b16d0 msgr2=0x7f7bd80b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.162+0000 7f7bdec00640 1 -- 172.21.15.7:0/4162291521 shutdown_connections 2024-09-10T21:50:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.162+0000 7f7bdec00640 1 -- 172.21.15.7:0/4162291521 wait complete. 2024-09-10T21:50:08.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.164+0000 7f7bdec00640 1 Processor -- start 2024-09-10T21:50:08.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.164+0000 7f7bdec00640 1 -- start start 2024-09-10T21:50:08.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.165+0000 7f7bdec00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd80ae020 0x7f7bd80c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:08.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.165+0000 7f7bdec00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bd80cbd50 0x7f7bd80b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:08.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.165+0000 7f7bdec00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7bd81619e0 0x7f7bd80b0e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:08.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.165+0000 7f7bdec00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd80ce100 con 0x7f7bd80ae020 2024-09-10T21:50:08.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.165+0000 7f7bdec00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7bd80cdf80 con 0x7f7bd80cbd50 2024-09-10T21:50:08.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.165+0000 7f7bdec00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f7bd8161dc0 con 0x7f7bd81619e0 2024-09-10T21:50:08.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.165+0000 7f7bd7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd80ae020 0x7f7bd80c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:08.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.165+0000 7f7bd7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd80ae020 0x7f7bd80c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58906/0 (socket says 172.21.15.7:58906) 2024-09-10T21:50:08.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.165+0000 7f7bd7e00640 1 -- 172.21.15.7:0/989517682 learned_addr learned my addr 172.21.15.7:0/989517682 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:08.167 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.165+0000 7f7bcfe00640 1 --2- 172.21.15.7:0/989517682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bd80cbd50 0x7f7bd80b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.165+0000 7f7bdcc00640 1 --2- 172.21.15.7:0/989517682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7bd81619e0 0x7f7bd80b0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.165+0000 7f7bd7e00640 1 -- 172.21.15.7:0/989517682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7bd81619e0 msgr2=0x7f7bd80b0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.165+0000 7f7bd7e00640 1 --2- 172.21.15.7:0/989517682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7bd81619e0 0x7f7bd80b0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.165+0000 7f7bd7e00640 1 -- 172.21.15.7:0/989517682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bd80cbd50 msgr2=0x7f7bd80b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.165+0000 7f7bd7e00640 1 --2- 172.21.15.7:0/989517682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bd80cbd50 0x7f7bd80b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.165+0000 7f7bd7e00640 1 -- 172.21.15.7:0/989517682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bd80b58d0 con 0x7f7bd80ae020 2024-09-10T21:50:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.165+0000 7f7bcfe00640 1 --2- 172.21.15.7:0/989517682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bd80cbd50 0x7f7bd80b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:50:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.166+0000 7f7bd7e00640 1 --2- 172.21.15.7:0/989517682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd80ae020 0x7f7bd80c7660 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800c4c0 tx=0x7f7bc800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.166+0000 7f7bd4c00640 1 -- 172.21.15.7:0/989517682 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8016020 con 0x7f7bd80ae020 2024-09-10T21:50:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.166+0000 7f7bd4c00640 1 -- 172.21.15.7:0/989517682 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7bc8012940 con 0x7f7bd80ae020 2024-09-10T21:50:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.166+0000 7f7bd4c00640 1 -- 172.21.15.7:0/989517682 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc8012d90 con 0x7f7bd80ae020 2024-09-10T21:50:08.168 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.166+0000 7f7bdec00640 1 -- 172.21.15.7:0/989517682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bd81f83c0 con 0x7f7bd80ae020 2024-09-10T21:50:08.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.166+0000 7f7bdec00640 1 -- 172.21.15.7:0/989517682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bd81f8980 con 0x7f7bd80ae020 2024-09-10T21:50:08.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.167+0000 7f7bdec00640 1 -- 172.21.15.7:0/989517682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bd80ae4d0 con 0x7f7bd80ae020 2024-09-10T21:50:08.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.168+0000 7f7bd4c00640 1 -- 172.21.15.7:0/989517682 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7bc801a3b0 con 0x7f7bd80ae020 2024-09-10T21:50:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.169+0000 7f7bd4c00640 1 --2- 172.21.15.7:0/989517682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7bb4077f70 0x7f7bb407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.169+0000 7f7bd4c00640 1 -- 172.21.15.7:0/989517682 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f7bc8099cc0 con 0x7f7bd80ae020 2024-09-10T21:50:08.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.169+0000 7f7bcfe00640 1 --2- 172.21.15.7:0/989517682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7bb4077f70 0x7f7bb407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:08.171 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.170+0000 7f7bcfe00640 1 --2- 172.21.15.7:0/989517682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7bb4077f70 0x7f7bb407a430 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00058c0 tx=0x7f7bc0005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:08.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.173+0000 7f7bd4c00640 1 -- 172.21.15.7:0/989517682 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7bc80a0080 con 0x7f7bd80ae020 2024-09-10T21:50:08.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.408+0000 7f7bdec00640 1 -- 172.21.15.7:0/989517682 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7bd80c7440 con 0x7f7bd80ae020 2024-09-10T21:50:08.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.409+0000 7f7bd4c00640 1 -- 172.21.15.7:0/989517682 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5219 (secure 0 0 0) 0x7f7bc8020030 con 0x7f7bd80ae020 2024-09-10T21:50:08.410 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:08.410 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":85,"btime":"2024-09-10T21:50:02:448373+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/431253133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":431253133},{"type":"v1","addr":"172.21.15.133:6836","nonce":431253133}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7521,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3692199178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3692199178},{"type":"v1","addr":"172.21.15.133:6837","nonce":3692199178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/752989865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":752989865},{"type":"v1","addr":"172.21.15.7:6834","nonce":752989865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:49:58.872982+0000","modified":"2024-09-10T21:50:01.884201+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3075619232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3075619232},{"type":"v1","addr":"172.21.15.7:6835","nonce":3075619232}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7527,"qdb_cluster":[7527]},"id":7}]} 2024-09-10T21:50:08.410 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 85 2024-09-10T21:50:08.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.411+0000 7f7bdec00640 1 -- 172.21.15.7:0/989517682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7bb4077f70 msgr2=0x7f7bb407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.411+0000 7f7bdec00640 1 --2- 172.21.15.7:0/989517682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7bb4077f70 0x7f7bb407a430 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00058c0 tx=0x7f7bc0005850 comp rx=0 tx=0).stop 2024-09-10T21:50:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.411+0000 7f7bdec00640 1 -- 172.21.15.7:0/989517682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd80ae020 msgr2=0x7f7bd80c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.411+0000 7f7bdec00640 1 --2- 172.21.15.7:0/989517682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd80ae020 0x7f7bd80c7660 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800c4c0 tx=0x7f7bc800c980 comp rx=0 tx=0).stop 2024-09-10T21:50:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.411+0000 7f7bdec00640 1 -- 172.21.15.7:0/989517682 shutdown_connections 2024-09-10T21:50:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.411+0000 7f7bdec00640 1 --2- 172.21.15.7:0/989517682 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7bb4077f70 0x7f7bb407a430 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.411+0000 7f7bdec00640 1 --2- 172.21.15.7:0/989517682 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7bd80cbd50 0x7f7bd80b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.411+0000 7f7bdec00640 1 --2- 172.21.15.7:0/989517682 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7bd81619e0 0x7f7bd80b0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.411+0000 7f7bdec00640 1 --2- 172.21.15.7:0/989517682 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7bd80ae020 0x7f7bd80c7660 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.411+0000 7f7bdec00640 1 -- 172.21.15.7:0/989517682 >> 172.21.15.7:0/989517682 conn(0x7f7bd80b16d0 msgr2=0x7f7bd815c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.411+0000 7f7bdec00640 1 -- 172.21.15.7:0/989517682 shutdown_connections 2024-09-10T21:50:08.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.412+0000 7f7bdec00640 1 -- 172.21.15.7:0/989517682 wait complete. 2024-09-10T21:50:08.429 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-10T21:50:08.614 INFO:teuthology.orchestra.run.smithi007.stdout:{"message":"","return_code":0} 2024-09-10T21:50:08.614 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-10T21:50:08.614 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:50:08.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.775+0000 7f4ecf000640 1 -- 172.21.15.7:0/2372902783 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4ec815dfd0 msgr2=0x7f4ec80d4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:08.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.775+0000 7f4ecf000640 1 --2- 172.21.15.7:0/2372902783 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4ec815dfd0 0x7f4ec80d4870 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc009830 tx=0x7f4ebc02a480 comp rx=0 tx=0).stop 2024-09-10T21:50:08.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.775+0000 7f4ecf000640 1 -- 172.21.15.7:0/2372902783 shutdown_connections 2024-09-10T21:50:08.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.775+0000 7f4ecf000640 1 --2- 172.21.15.7:0/2372902783 >> v2:172.21.15.133:3300/0 conn(0x7f4ec80b2b50 0x7f4ec80d0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:08.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.775+0000 7f4ecf000640 1 --2- 172.21.15.7:0/2372902783 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4ec815dfd0 0x7f4ec80d4870 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:08.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.775+0000 7f4ecf000640 1 -- 172.21.15.7:0/2372902783 >> 172.21.15.7:0/2372902783 conn(0x7f4ec80d3d50 msgr2=0x7f4ec80d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:08.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.775+0000 7f4ecf000640 1 -- 172.21.15.7:0/2372902783 shutdown_connections 2024-09-10T21:50:08.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.775+0000 7f4ecf000640 1 -- 172.21.15.7:0/2372902783 wait complete. 2024-09-10T21:50:08.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.777+0000 7f4ecf000640 1 Processor -- start 2024-09-10T21:50:08.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.777+0000 7f4ecf000640 1 -- start start 2024-09-10T21:50:08.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.778+0000 7f4ecf000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ec80b2b50 0x7f4ec81f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:08.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.778+0000 7f4ecf000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4ec80d0ca0 0x7f4ec814a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:08.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.778+0000 7f4ecf000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec815dfd0 0x7f4ec80b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:08.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.778+0000 7f4ecf000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec80b08c0 con 0x7f4ec80b2b50 2024-09-10T21:50:08.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.778+0000 7f4ecf000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec80d01d0 con 0x7f4ec815dfd0 2024-09-10T21:50:08.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.778+0000 7f4ecf000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec80b0a40 con 0x7f4ec80d0ca0 2024-09-10T21:50:08.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.778+0000 7f4ecdc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ec80b2b50 0x7f4ec81f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:08.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.778+0000 7f4ecdc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ec80b2b50 0x7f4ec81f54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58918/0 (socket says 172.21.15.7:58918) 2024-09-10T21:50:08.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.778+0000 7f4ecdc00640 1 -- 172.21.15.7:0/1185681954 learned_addr learned my addr 172.21.15.7:0/1185681954 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:08.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.778+0000 7f4ecd200640 1 --2- 172.21.15.7:0/1185681954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4ec80d0ca0 0x7f4ec814a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:08.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.778+0000 7f4ece600640 1 --2- 172.21.15.7:0/1185681954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec815dfd0 0x7f4ec80b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:08.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.779+0000 7f4ecdc00640 1 -- 172.21.15.7:0/1185681954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4ec80d0ca0 msgr2=0x7f4ec814a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:08.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.779+0000 7f4ecdc00640 1 --2- 172.21.15.7:0/1185681954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4ec80d0ca0 0x7f4ec814a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:08.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.779+0000 7f4ecdc00640 1 -- 172.21.15.7:0/1185681954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec815dfd0 msgr2=0x7f4ec80b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:08.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.779+0000 7f4ecdc00640 1 --2- 172.21.15.7:0/1185681954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec815dfd0 0x7f4ec80b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:08.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.779+0000 7f4ecdc00640 1 -- 172.21.15.7:0/1185681954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec80cb300 con 0x7f4ec80b2b50 2024-09-10T21:50:08.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.779+0000 7f4ecdc00640 1 --2- 172.21.15.7:0/1185681954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ec80b2b50 0x7f4ec81f54f0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc02aa70 tx=0x7f4ebc033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:08.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.779+0000 7f4ec5600640 1 -- 172.21.15.7:0/1185681954 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc031d60 con 0x7f4ec80b2b50 2024-09-10T21:50:08.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.779+0000 7f4ec5600640 1 -- 172.21.15.7:0/1185681954 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ebc033e10 con 0x7f4ec80b2b50 2024-09-10T21:50:08.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.779+0000 7f4ec5600640 1 -- 172.21.15.7:0/1185681954 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc005260 con 0x7f4ec80b2b50 2024-09-10T21:50:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.779+0000 7f4ecf000640 1 -- 172.21.15.7:0/1185681954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec814d110 con 0x7f4ec80b2b50 2024-09-10T21:50:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.779+0000 7f4ecf000640 1 -- 172.21.15.7:0/1185681954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec80b4bb0 con 0x7f4ec80b2b50 2024-09-10T21:50:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.780+0000 7f4ecf000640 1 -- 172.21.15.7:0/1185681954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ec80d0760 con 0x7f4ec80b2b50 2024-09-10T21:50:08.782 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.782+0000 7f4ec5600640 1 -- 172.21.15.7:0/1185681954 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4ebc005400 con 0x7f4ec80b2b50 2024-09-10T21:50:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.782+0000 7f4ec5600640 1 --2- 172.21.15.7:0/1185681954 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4ea8077f70 0x7f4ea807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:08.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.783+0000 7f4ec5600640 1 -- 172.21.15.7:0/1185681954 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f4ebc0bace0 con 0x7f4ec80b2b50 2024-09-10T21:50:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.783+0000 7f4ecd200640 1 --2- 172.21.15.7:0/1185681954 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4ea8077f70 0x7f4ea807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:08.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.784+0000 7f4ecd200640 1 --2- 172.21.15.7:0/1185681954 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4ea8077f70 0x7f4ea807a430 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80ce1c0 tx=0x7f4eb4005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:08.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:08.787+0000 7f4ec5600640 1 -- 172.21.15.7:0/1185681954 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4ebc03f030 con 0x7f4ec80b2b50 2024-09-10T21:50:09.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.019+0000 7f4ecf000640 1 -- 172.21.15.7:0/1185681954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ec80b9cd0 con 0x7f4ec80b2b50 2024-09-10T21:50:09.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.020+0000 7f4ec5600640 1 -- 172.21.15.7:0/1185681954 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5219 (secure 0 0 0) 0x7f4ebc083180 con 0x7f4ec80b2b50 2024-09-10T21:50:09.021 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:09.021 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":85,"btime":"2024-09-10T21:50:02:448373+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/431253133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":431253133},{"type":"v1","addr":"172.21.15.133:6836","nonce":431253133}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7521,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3692199178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3692199178},{"type":"v1","addr":"172.21.15.133:6837","nonce":3692199178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/752989865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":752989865},{"type":"v1","addr":"172.21.15.7:6834","nonce":752989865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:49:58.872982+0000","modified":"2024-09-10T21:50:01.884201+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3075619232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3075619232},{"type":"v1","addr":"172.21.15.7:6835","nonce":3075619232}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7527,"qdb_cluster":[7527]},"id":7}]} 2024-09-10T21:50:09.021 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 85 2024-09-10T21:50:09.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.022+0000 7f4ecf000640 1 -- 172.21.15.7:0/1185681954 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4ea8077f70 msgr2=0x7f4ea807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:09.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.022+0000 7f4ecf000640 1 --2- 172.21.15.7:0/1185681954 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4ea8077f70 0x7f4ea807a430 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4ec80ce1c0 tx=0x7f4eb4005f90 comp rx=0 tx=0).stop 2024-09-10T21:50:09.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.022+0000 7f4ecf000640 1 -- 172.21.15.7:0/1185681954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ec80b2b50 msgr2=0x7f4ec81f54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:09.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.022+0000 7f4ecf000640 1 --2- 172.21.15.7:0/1185681954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ec80b2b50 0x7f4ec81f54f0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc02aa70 tx=0x7f4ebc033c20 comp rx=0 tx=0).stop 2024-09-10T21:50:09.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.022+0000 7f4ecf000640 1 -- 172.21.15.7:0/1185681954 shutdown_connections 2024-09-10T21:50:09.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.022+0000 7f4ecf000640 1 --2- 172.21.15.7:0/1185681954 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4ea8077f70 0x7f4ea807a430 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.022+0000 7f4ecf000640 1 --2- 172.21.15.7:0/1185681954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4ec815dfd0 0x7f4ec80b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.022+0000 7f4ecf000640 1 --2- 172.21.15.7:0/1185681954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4ec80d0ca0 0x7f4ec814a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.022+0000 7f4ecf000640 1 --2- 172.21.15.7:0/1185681954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4ec80b2b50 0x7f4ec81f54f0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.022+0000 7f4ecf000640 1 -- 172.21.15.7:0/1185681954 >> 172.21.15.7:0/1185681954 conn(0x7f4ec80d3d50 msgr2=0x7f4ec80cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:09.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.022+0000 7f4ecf000640 1 -- 172.21.15.7:0/1185681954 shutdown_connections 2024-09-10T21:50:09.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.022+0000 7f4ecf000640 1 -- 172.21.15.7:0/1185681954 wait complete. 2024-09-10T21:50:09.038 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata get mds0_inotable - 2024-09-10T21:50:09.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.115+0000 7f9391e14f40 1 Processor -- start 2024-09-10T21:50:09.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.115+0000 7f9391e14f40 1 -- start start 2024-09-10T21:50:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.116+0000 7f9391e14f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a389dfeae0 0x55a389eba3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.116+0000 7f9391e14f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a389e4b110 0x55a389eccca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.116+0000 7f9391e14f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a389ebb620 0x55a389e30490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.116+0000 7f9391e14f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a389d4dc60 con 0x55a389ebb620 2024-09-10T21:50:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.116+0000 7f9391e14f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a389dac1d0 con 0x55a389dfeae0 2024-09-10T21:50:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.116+0000 7f9391e14f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55a389e4a1a0 con 0x55a389e4b110 2024-09-10T21:50:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.116+0000 7f938fa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a389ebb620 0x55a389e30490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.116+0000 7f938fa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a389ebb620 0x55a389e30490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58942/0 (socket says 172.21.15.7:58942) 2024-09-10T21:50:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.116+0000 7f938fa00640 1 -- 172.21.15.7:0/3724246715 learned_addr learned my addr 172.21.15.7:0/3724246715 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.116+0000 7f938f000640 1 --2- 172.21.15.7:0/3724246715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a389dfeae0 0x55a389eba3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.116+0000 7f938e600640 1 --2- 172.21.15.7:0/3724246715 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a389e4b110 0x55a389eccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:09.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.116+0000 7f938fa00640 1 -- 172.21.15.7:0/3724246715 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a389e4b110 msgr2=0x55a389eccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:09.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.116+0000 7f938fa00640 1 --2- 172.21.15.7:0/3724246715 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a389e4b110 0x55a389eccca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.116+0000 7f938fa00640 1 -- 172.21.15.7:0/3724246715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a389dfeae0 msgr2=0x55a389eba3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:09.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.116+0000 7f938fa00640 1 --2- 172.21.15.7:0/3724246715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a389dfeae0 0x55a389eba3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.117+0000 7f938fa00640 1 -- 172.21.15.7:0/3724246715 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a389e2ed40 con 0x55a389ebb620 2024-09-10T21:50:09.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.117+0000 7f938e600640 1 --2- 172.21.15.7:0/3724246715 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a389e4b110 0x55a389eccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:50:09.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.117+0000 7f938f000640 1 --2- 172.21.15.7:0/3724246715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a389dfeae0 0x55a389eba3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:50:09.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.117+0000 7f938fa00640 1 --2- 172.21.15.7:0/3724246715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a389ebb620 0x55a389e30490 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f9388001690 tx=0x7f9388001b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d6ef061677272 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:09.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.117+0000 7f938d200640 1 -- 172.21.15.7:0/3724246715 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9388017070 con 0x55a389ebb620 2024-09-10T21:50:09.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.117+0000 7f938d200640 1 -- 172.21.15.7:0/3724246715 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f938800f800 con 0x55a389ebb620 2024-09-10T21:50:09.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.117+0000 7f938d200640 1 -- 172.21.15.7:0/3724246715 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938800fc30 con 0x55a389ebb620 2024-09-10T21:50:09.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.117+0000 7f9391e14f40 1 -- 172.21.15.7:0/3724246715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a389ebb620 msgr2=0x55a389e30490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:09.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.117+0000 7f9391e14f40 1 --2- 172.21.15.7:0/3724246715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a389ebb620 0x55a389e30490 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f9388001690 tx=0x7f9388001b50 comp rx=0 tx=0).stop 2024-09-10T21:50:09.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.118+0000 7f9391e14f40 1 -- 172.21.15.7:0/3724246715 shutdown_connections 2024-09-10T21:50:09.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.118+0000 7f9391e14f40 1 --2- 172.21.15.7:0/3724246715 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a389ebb620 0x55a389e30490 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.118+0000 7f9391e14f40 1 --2- 172.21.15.7:0/3724246715 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a389e4b110 0x55a389eccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.118+0000 7f9391e14f40 1 --2- 172.21.15.7:0/3724246715 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a389dfeae0 0x55a389eba3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.118+0000 7f9391e14f40 1 -- 172.21.15.7:0/3724246715 >> 172.21.15.7:0/3724246715 conn(0x55a389e2dde0 msgr2=0x55a389e31080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.118+0000 7f9391e14f40 1 -- 172.21.15.7:0/3724246715 shutdown_connections 2024-09-10T21:50:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.118+0000 7f9391e14f40 1 -- 172.21.15.7:0/3724246715 wait complete. 2024-09-10T21:50:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.118+0000 7f9391e14f40 1 Processor -- start 2024-09-10T21:50:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.119+0000 7f9391e14f40 1 -- start start 2024-09-10T21:50:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.119+0000 7f9391e14f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a389dfeae0 0x55a389ecbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.120+0000 7f9391e14f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a389e4b110 0x55a389eca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.120+0000 7f9391e14f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a389ebb620 0x55a389f5eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.120+0000 7f9391e14f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a389ebae60 con 0x55a389e4b110 2024-09-10T21:50:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.120+0000 7f9391e14f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a389ecc3f0 con 0x55a389ebb620 2024-09-10T21:50:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.120+0000 7f9391e14f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55a389ed8300 con 0x55a389dfeae0 2024-09-10T21:50:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.120+0000 7f938e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a389e4b110 0x55a389eca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.120+0000 7f938e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a389e4b110 0x55a389eca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58952/0 (socket says 172.21.15.7:58952) 2024-09-10T21:50:09.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.120+0000 7f938e600640 1 -- 172.21.15.7:0/2000776007 learned_addr learned my addr 172.21.15.7:0/2000776007 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:09.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.120+0000 7f938f000640 1 --2- 172.21.15.7:0/2000776007 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a389dfeae0 0x55a389ecbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:09.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.120+0000 7f938fa00640 1 --2- 172.21.15.7:0/2000776007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a389ebb620 0x55a389f5eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:09.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.120+0000 7f938e600640 1 -- 172.21.15.7:0/2000776007 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a389dfeae0 msgr2=0x55a389ecbb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:09.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.120+0000 7f938e600640 1 --2- 172.21.15.7:0/2000776007 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a389dfeae0 0x55a389ecbb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.120+0000 7f938e600640 1 -- 172.21.15.7:0/2000776007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a389ebb620 msgr2=0x55a389f5eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:09.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.120+0000 7f938e600640 1 --2- 172.21.15.7:0/2000776007 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a389ebb620 0x55a389f5eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.120+0000 7f938e600640 1 -- 172.21.15.7:0/2000776007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a389ed85e0 con 0x55a389e4b110 2024-09-10T21:50:09.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.121+0000 7f938e600640 1 --2- 172.21.15.7:0/2000776007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a389e4b110 0x55a389eca8f0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f937c046640 tx=0x7f937c04d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:09.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.121+0000 7f9387400640 1 -- 172.21.15.7:0/2000776007 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c066070 con 0x55a389e4b110 2024-09-10T21:50:09.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.121+0000 7f9387400640 1 -- 172.21.15.7:0/2000776007 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f937c056760 con 0x55a389e4b110 2024-09-10T21:50:09.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.121+0000 7f9387400640 1 -- 172.21.15.7:0/2000776007 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f937c056b90 con 0x55a389e4b110 2024-09-10T21:50:09.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.121+0000 7f9391e14f40 1 -- 172.21.15.7:0/2000776007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a389f5e6d0 con 0x55a389e4b110 2024-09-10T21:50:09.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.121+0000 7f9391e14f40 1 -- 172.21.15.7:0/2000776007 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a389ed88c0 con 0x55a389e4b110 2024-09-10T21:50:09.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.123+0000 7f9387400640 1 -- 172.21.15.7:0/2000776007 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f937c061400 con 0x55a389e4b110 2024-09-10T21:50:09.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.124+0000 7f9387400640 1 --2- 172.21.15.7:0/2000776007 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9370076950 0x7f9370078e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:09.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.124+0000 7f9387400640 1 -- 172.21.15.7:0/2000776007 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f937c0dc6b0 con 0x55a389e4b110 2024-09-10T21:50:09.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.125+0000 7f9391e14f40 1 --2- 172.21.15.7:0/2000776007 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x55a389f63770 0x55a389f60a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:09.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.125+0000 7f9391e14f40 1 -- 172.21.15.7:0/2000776007 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101) -- 0x55a389f60f60 con 0x55a389f63770 2024-09-10T21:50:09.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.125+0000 7f938f000640 1 --2- 172.21.15.7:0/2000776007 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9370076950 0x7f9370078e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:09.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.125+0000 7f938fa00640 1 --2- 172.21.15.7:0/2000776007 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x55a389f63770 0x55a389f60a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:09.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.125+0000 7f938fa00640 1 --2- 172.21.15.7:0/2000776007 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x55a389f63770 0x55a389f60a20 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:09.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.125+0000 7f938f000640 1 --2- 172.21.15.7:0/2000776007 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9370076950 0x7f9370078e10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f938000b710 tx=0x7f9380004880 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:09.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.126+0000 7f938fa00640 1 -- 172.21.15.7:0/2000776007 <== osd.2 v2:172.21.15.7:6816/4238877366 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv3 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x7f9388003030 con 0x55a389f63770 2024-09-10T21:50:09.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.126+0000 7f9391e14f40 1 -- 172.21.15.7:0/2000776007 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x55a389f63770 msgr2=0x55a389f60a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:09.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.126+0000 7f9391e14f40 1 --2- 172.21.15.7:0/2000776007 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x55a389f63770 0x55a389f60a20 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.126+0000 7f9391e14f40 1 -- 172.21.15.7:0/2000776007 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9370076950 msgr2=0x7f9370078e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:09.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.126+0000 7f9391e14f40 1 --2- 172.21.15.7:0/2000776007 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9370076950 0x7f9370078e10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f938000b710 tx=0x7f9380004880 comp rx=0 tx=0).stop 2024-09-10T21:50:09.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.126+0000 7f9391e14f40 1 -- 172.21.15.7:0/2000776007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a389e4b110 msgr2=0x55a389eca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:09.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.126+0000 7f9391e14f40 1 --2- 172.21.15.7:0/2000776007 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a389e4b110 0x55a389eca8f0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f937c046640 tx=0x7f937c04d820 comp rx=0 tx=0).stop 2024-09-10T21:50:09.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.126+0000 7f938fa00640 1 -- 172.21.15.7:0/2000776007 reap_dead start 2024-09-10T21:50:09.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.126+0000 7f9391e14f40 1 -- 172.21.15.7:0/2000776007 shutdown_connections 2024-09-10T21:50:09.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.126+0000 7f9391e14f40 1 -- 172.21.15.7:0/2000776007 >> 172.21.15.7:0/2000776007 conn(0x55a389e2dde0 msgr2=0x55a389e4c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:09.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.126+0000 7f9391e14f40 1 -- 172.21.15.7:0/2000776007 shutdown_connections 2024-09-10T21:50:09.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.126+0000 7f9391e14f40 1 -- 172.21.15.7:0/2000776007 wait complete. 2024-09-10T21:50:09.131 INFO:teuthology.orchestra.run.smithi007.stdout:����� 2024-09-10T21:50:09.131 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:50:09.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.290+0000 7f0926e00640 1 -- 172.21.15.7:0/2474751287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200b2f80 msgr2=0x7f09200ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:09.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.290+0000 7f0926e00640 1 --2- 172.21.15.7:0/2474751287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200b2f80 0x7f09200ce6f0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f0918005fd0 tx=0x7f091802aa90 comp rx=0 tx=0).stop 2024-09-10T21:50:09.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.291+0000 7f0926e00640 1 -- 172.21.15.7:0/2474751287 shutdown_connections 2024-09-10T21:50:09.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.291+0000 7f0926e00640 1 --2- 172.21.15.7:0/2474751287 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200b2f80 0x7f09200ce6f0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.291+0000 7f0926e00640 1 --2- 172.21.15.7:0/2474751287 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f09200b4c90 0x7f09200b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.291+0000 7f0926e00640 1 -- 172.21.15.7:0/2474751287 >> 172.21.15.7:0/2474751287 conn(0x7f09200b1d70 msgr2=0x7f09200cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:09.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.291+0000 7f0926e00640 1 -- 172.21.15.7:0/2474751287 shutdown_connections 2024-09-10T21:50:09.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.291+0000 7f0926e00640 1 -- 172.21.15.7:0/2474751287 wait complete. 2024-09-10T21:50:09.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.292+0000 7f0926e00640 1 Processor -- start 2024-09-10T21:50:09.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.293+0000 7f0926e00640 1 -- start start 2024-09-10T21:50:09.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.293+0000 7f0926e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09200b2f80 0x7f0920142960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.293+0000 7f0926e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200b4c90 0x7f09200ac620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.294+0000 7f0926e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0920145540 0x7f09200ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.294+0000 7f0926e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f09200cedb0 con 0x7f09200b4c90 2024-09-10T21:50:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.294+0000 7f0926e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f09200cec30 con 0x7f09200b2f80 2024-09-10T21:50:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.294+0000 7f0926e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f09200cef30 con 0x7f0920145540 2024-09-10T21:50:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.294+0000 7f0925000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200b4c90 0x7f09200ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.294+0000 7f0925000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200b4c90 0x7f09200ac620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58980/0 (socket says 172.21.15.7:58980) 2024-09-10T21:50:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.294+0000 7f0925000640 1 -- 172.21.15.7:0/21807535 learned_addr learned my addr 172.21.15.7:0/21807535 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.294+0000 7f0925a00640 1 --2- 172.21.15.7:0/21807535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09200b2f80 0x7f0920142960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.294+0000 7f0926400640 1 --2- 172.21.15.7:0/21807535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0920145540 0x7f09200ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.294+0000 7f0925000640 1 -- 172.21.15.7:0/21807535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0920145540 msgr2=0x7f09200ae890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:09.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.294+0000 7f0925000640 1 --2- 172.21.15.7:0/21807535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0920145540 0x7f09200ae890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.294+0000 7f0925000640 1 -- 172.21.15.7:0/21807535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09200b2f80 msgr2=0x7f0920142960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:09.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.294+0000 7f0925000640 1 --2- 172.21.15.7:0/21807535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09200b2f80 0x7f0920142960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.294+0000 7f0925000640 1 -- 172.21.15.7:0/21807535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09200ad350 con 0x7f09200b4c90 2024-09-10T21:50:09.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.294+0000 7f0925a00640 1 --2- 172.21.15.7:0/21807535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09200b2f80 0x7f0920142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:50:09.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.294+0000 7f0926400640 1 --2- 172.21.15.7:0/21807535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0920145540 0x7f09200ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:50:09.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.294+0000 7f0925000640 1 --2- 172.21.15.7:0/21807535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200b4c90 0x7f09200ac620 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f090c00e9f0 tx=0x7f090c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:09.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.295+0000 7f091d600640 1 -- 172.21.15.7:0/21807535 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c00cd80 con 0x7f09200b4c90 2024-09-10T21:50:09.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.295+0000 7f091d600640 1 -- 172.21.15.7:0/21807535 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f090c010430 con 0x7f09200b4c90 2024-09-10T21:50:09.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.295+0000 7f091d600640 1 -- 172.21.15.7:0/21807535 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090c010840 con 0x7f09200b4c90 2024-09-10T21:50:09.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.295+0000 7f0926e00640 1 -- 172.21.15.7:0/21807535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09200ad070 con 0x7f09200b4c90 2024-09-10T21:50:09.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.295+0000 7f0926e00640 1 -- 172.21.15.7:0/21807535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09201f5ae0 con 0x7f09200b4c90 2024-09-10T21:50:09.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.296+0000 7f0926e00640 1 -- 172.21.15.7:0/21807535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09200ce6f0 con 0x7f09200b4c90 2024-09-10T21:50:09.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.297+0000 7f091d600640 1 -- 172.21.15.7:0/21807535 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f090c0109e0 con 0x7f09200b4c90 2024-09-10T21:50:09.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.298+0000 7f091d600640 1 --2- 172.21.15.7:0/21807535 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0900077f70 0x7f090007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:09.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.298+0000 7f091d600640 1 -- 172.21.15.7:0/21807535 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f090c09a900 con 0x7f09200b4c90 2024-09-10T21:50:09.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.298+0000 7f0925a00640 1 --2- 172.21.15.7:0/21807535 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0900077f70 0x7f090007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:09.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.299+0000 7f0925a00640 1 --2- 172.21.15.7:0/21807535 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0900077f70 0x7f090007a430 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f09140016a0 tx=0x7f0914009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:09.302 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.302+0000 7f091d600640 1 -- 172.21.15.7:0/21807535 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f09200ad350 con 0x7f09200b4c90 2024-09-10T21:50:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.534+0000 7f0926e00640 1 -- 172.21.15.7:0/21807535 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f092006f050 con 0x7f09200b4c90 2024-09-10T21:50:09.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.534+0000 7f091d600640 1 -- 172.21.15.7:0/21807535 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v85) ==== 53+0+73 (secure 0 0 0) 0x7f090c09a020 con 0x7f09200b4c90 2024-09-10T21:50:09.535 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:50:09.537 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.536+0000 7f0926e00640 1 -- 172.21.15.7:0/21807535 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0900077f70 msgr2=0x7f090007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.536+0000 7f0926e00640 1 --2- 172.21.15.7:0/21807535 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0900077f70 0x7f090007a430 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f09140016a0 tx=0x7f0914009040 comp rx=0 tx=0).stop 2024-09-10T21:50:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.536+0000 7f0926e00640 1 -- 172.21.15.7:0/21807535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200b4c90 msgr2=0x7f09200ac620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.536+0000 7f0926e00640 1 --2- 172.21.15.7:0/21807535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200b4c90 0x7f09200ac620 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f090c00e9f0 tx=0x7f090c00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.536+0000 7f0926e00640 1 -- 172.21.15.7:0/21807535 shutdown_connections 2024-09-10T21:50:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.536+0000 7f0926e00640 1 --2- 172.21.15.7:0/21807535 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0900077f70 0x7f090007a430 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.536+0000 7f0926e00640 1 --2- 172.21.15.7:0/21807535 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0920145540 0x7f09200ae890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.536+0000 7f0926e00640 1 --2- 172.21.15.7:0/21807535 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f09200b4c90 0x7f09200ac620 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.536+0000 7f0926e00640 1 --2- 172.21.15.7:0/21807535 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f09200b2f80 0x7f0920142960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.536+0000 7f0926e00640 1 -- 172.21.15.7:0/21807535 >> 172.21.15.7:0/21807535 conn(0x7f09200b1d70 msgr2=0x7f092006e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.537+0000 7f0926e00640 1 -- 172.21.15.7:0/21807535 shutdown_connections 2024-09-10T21:50:09.538 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09.537+0000 7f0926e00640 1 -- 172.21.15.7:0/21807535 wait complete. 2024-09-10T21:50:09.553 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:50:09.553 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:50:09.553 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-10T21:50:09.553 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:50:09.553 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:50:09.553 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:50:09.553 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:50:09.553 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:50:09.554 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:50:09.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:09.554 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-10T21:50:09.590 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:50:09.590 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:50:09.590 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:50:09.590 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:50:09.590 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T21:50:09.590 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:50:09.590 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:50:09.590 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T21:50:09.590 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-10T21:50:09.590 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:50:09.590 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T21:50:09.591 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-10T21:50:09.591 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:50:09.591 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-10T21:50:09.591 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-10T21:50:09.591 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T21:50:09.591 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42135sec preferred_lft 42135sec 2024-09-10T21:50:09.591 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-10T21:50:09.591 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:50:09.591 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:50:09.591 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-10T21:50:09.591 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-10T21:50:09.591 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:50:09.591 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff 2024-09-10T21:50:09.592 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:50:09.592 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:50:09.592 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::3c3d:dbff:fe02:4c15/64 scope link 2024-09-10T21:50:09.592 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:50:09.592 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:50:09.592 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether f6:f9:69:5a:86:f5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:50:09.592 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f4f9:69ff:fe5a:86f5/64 scope link 2024-09-10T21:50:09.592 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:50:09.592 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:50:09.592 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:50:09.592 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::433:5cff:fe44:4a5b/64 scope link 2024-09-10T21:50:09.592 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:50:09.593 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:09.593 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:50:09.682 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:50:09.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:09.686 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-10T21:50:09.721 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-10T21:50:09.721 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:50:09.722 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T21:50:09.722 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T21:50:09.722 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:50:09.722 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:50:09.722 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:50:09.724 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:50:09.724 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:50:09.724 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:50:09.724 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:50:09.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:50:09.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:50:09.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:09.921 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:50:09.956 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:50:09.956 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:50:09.957 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:50:09.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:09.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-10T21:50:10.054 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:50:10.054 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:50:10.089 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:50:10.089 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:50:10.090 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:50:10.165 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:50:10.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:10.166 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:50:10.294 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:50:10.294 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:50:10.322 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:50:10.322 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:50:10.323 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.133:3300,172.21.15.133:3301" 2024-09-10T21:50:10.323 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@2029451a-3c26-48c7-b874-0597197278ac.cephfs=/ 2024-09-10T21:50:10.323 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.133:3300/172.21.15.133:3301" will pass to kernel 2024-09-10T21:50:10.325 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T21:50:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:50:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:50:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:50:10.325 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:50:10.328 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:50:10.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:10.328 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:50:10.384 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:50:10.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:50:10.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:50:10.474 INFO:teuthology.orchestra.run.smithi007.stdout:client7141 2024-09-10T21:50:10.474 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:50:10.474 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:50:10.510 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:50:10.510 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:50:10.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:10.586 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file2_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-09-10T21:50:10.734 INFO:teuthology.orchestra.run.smithi007.stderr:6+0 records in 2024-09-10T21:50:10.734 INFO:teuthology.orchestra.run.smithi007.stderr:6+0 records out 2024-09-10T21:50:10.734 INFO:teuthology.orchestra.run.smithi007.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0701537 s, 89.7 MB/s 2024-09-10T21:50:10.735 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:50:10.735 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:50:10.771 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:10.771 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=file3_sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-09-10T21:50:10.915 INFO:teuthology.orchestra.run.smithi007.stderr:6+0 records in 2024-09-10T21:50:10.915 INFO:teuthology.orchestra.run.smithi007.stderr:6+0 records out 2024-09-10T21:50:10.915 INFO:teuthology.orchestra.run.smithi007.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0676142 s, 93.0 MB/s 2024-09-10T21:50:10.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:10.916 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-09-10T21:50:10.953 INFO:teuthology.orchestra.run.smithi007.stdout:./ 2024-09-10T21:50:10.953 INFO:teuthology.orchestra.run.smithi007.stdout:./file2_sixmegs 2024-09-10T21:50:10.953 INFO:teuthology.orchestra.run.smithi007.stdout:./file3_sixmegs 2024-09-10T21:50:10.953 INFO:teuthology.orchestra.run.smithi007.stdout:./file1_sixmegs 2024-09-10T21:50:10.954 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:50:10.954 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:50:10.955 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:50:10.955 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:50:10.955 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-09-10T21:50:10.955 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:50:11.094 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:50:11.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:50:11.106 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:50:11.106 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:50:11.106 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:50:11.106 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:50:11.106 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file2_sixmegs").st_ino) 2024-09-10T21:50:11.106 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:50:11.247 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627777 2024-09-10T21:50:11.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:50:11.258 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:50:11.258 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:50:11.258 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:50:11.258 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:50:11.259 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file3_sixmegs").st_ino) 2024-09-10T21:50:11.259 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:50:11.399 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627778 2024-09-10T21:50:11.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:50:11.410 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:50:11.410 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:50:11.410 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:50:11.410 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:50:11.411 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./file1_sixmegs").st_ino) 2024-09-10T21:50:11.411 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:50:11.551 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627776 2024-09-10T21:50:11.554 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:50:11.562 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:50:11.562 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:50:11.638 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:50:11.638 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:50:11.714 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:50:11.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.914+0000 7fd0e3200640 1 -- 172.21.15.7:0/1179290382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0dc06fd30 msgr2=0x7fd0dc072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:11.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.914+0000 7fd0e3200640 1 --2- 172.21.15.7:0/1179290382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0dc06fd30 0x7fd0dc072df0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc070a20 tx=0x7fd0d402a8b0 comp rx=0 tx=0).stop 2024-09-10T21:50:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.914+0000 7fd0e3200640 1 -- 172.21.15.7:0/1179290382 shutdown_connections 2024-09-10T21:50:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.914+0000 7fd0e3200640 1 --2- 172.21.15.7:0/1179290382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0dc06fd30 0x7fd0dc072df0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.914+0000 7fd0e3200640 1 --2- 172.21.15.7:0/1179290382 >> v2:172.21.15.133:3300/0 conn(0x7fd0dc15a9a0 0x7fd0dc15f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.914+0000 7fd0e3200640 1 -- 172.21.15.7:0/1179290382 >> 172.21.15.7:0/1179290382 conn(0x7fd0dc0b1660 msgr2=0x7fd0dc0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.914+0000 7fd0e3200640 1 -- 172.21.15.7:0/1179290382 shutdown_connections 2024-09-10T21:50:11.916 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.914+0000 7fd0e3200640 1 -- 172.21.15.7:0/1179290382 wait complete. 2024-09-10T21:50:11.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.916+0000 7fd0e3200640 1 Processor -- start 2024-09-10T21:50:11.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.916+0000 7fd0e3200640 1 -- start start 2024-09-10T21:50:11.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.917+0000 7fd0e3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0dc06fd30 0x7fd0dc155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:11.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.917+0000 7fd0e3200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0dc15a9a0 0x7fd0dc1f2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:11.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.917+0000 7fd0e3200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0dc1f2a70 0x7fd0dc0c4340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:11.919 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.917+0000 7fd0e3200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc0736c0 con 0x7fd0dc06fd30 2024-09-10T21:50:11.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.917+0000 7fd0e3200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd0dc073540 con 0x7fd0dc1f2a70 2024-09-10T21:50:11.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.917+0000 7fd0e3200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fd0dc073840 con 0x7fd0dc15a9a0 2024-09-10T21:50:11.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.917+0000 7fd0dbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0dc06fd30 0x7fd0dc155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:11.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.917+0000 7fd0dbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0dc06fd30 0x7fd0dc155720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:58994/0 (socket says 172.21.15.7:58994) 2024-09-10T21:50:11.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.917+0000 7fd0dbe00640 1 -- 172.21.15.7:0/1018206400 learned_addr learned my addr 172.21.15.7:0/1018206400 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:11.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.918+0000 7fd0e1200640 1 --2- 172.21.15.7:0/1018206400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0dc1f2a70 0x7fd0dc0c4340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:11.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.918+0000 7fd0db400640 1 --2- 172.21.15.7:0/1018206400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0dc15a9a0 0x7fd0dc1f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:11.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.918+0000 7fd0dbe00640 1 -- 172.21.15.7:0/1018206400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0dc15a9a0 msgr2=0x7fd0dc1f2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:11.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.918+0000 7fd0dbe00640 1 --2- 172.21.15.7:0/1018206400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0dc15a9a0 0x7fd0dc1f2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:11.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.918+0000 7fd0dbe00640 1 -- 172.21.15.7:0/1018206400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0dc1f2a70 msgr2=0x7fd0dc0c4340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:11.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.918+0000 7fd0dbe00640 1 --2- 172.21.15.7:0/1018206400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0dc1f2a70 0x7fd0dc0c4340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:11.920 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.918+0000 7fd0dbe00640 1 -- 172.21.15.7:0/1018206400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0dc0c50c0 con 0x7fd0dc06fd30 2024-09-10T21:50:11.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.918+0000 7fd0dbe00640 1 --2- 172.21.15.7:0/1018206400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0dc06fd30 0x7fd0dc155720 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00d950 tx=0x7fd0cc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:11.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.918+0000 7fd0d3e00640 1 -- 172.21.15.7:0/1018206400 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc017070 con 0x7fd0dc06fd30 2024-09-10T21:50:11.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.918+0000 7fd0d3e00640 1 -- 172.21.15.7:0/1018206400 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd0cc009d20 con 0x7fd0dc06fd30 2024-09-10T21:50:11.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.918+0000 7fd0d3e00640 1 -- 172.21.15.7:0/1018206400 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0cc0106c0 con 0x7fd0dc06fd30 2024-09-10T21:50:11.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.918+0000 7fd0e3200640 1 -- 172.21.15.7:0/1018206400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0dc0c53a0 con 0x7fd0dc06fd30 2024-09-10T21:50:11.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.919+0000 7fd0e3200640 1 -- 172.21.15.7:0/1018206400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0dc0c5960 con 0x7fd0dc06fd30 2024-09-10T21:50:11.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.920+0000 7fd0e3200640 1 -- 172.21.15.7:0/1018206400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0dc073cf0 con 0x7fd0dc06fd30 2024-09-10T21:50:11.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.921+0000 7fd0d3e00640 1 -- 172.21.15.7:0/1018206400 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd0cc010860 con 0x7fd0dc06fd30 2024-09-10T21:50:11.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.922+0000 7fd0d3e00640 1 --2- 172.21.15.7:0/1018206400 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd0bc077f70 0x7fd0bc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:11.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.922+0000 7fd0d3e00640 1 -- 172.21.15.7:0/1018206400 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fd0cc09a1c0 con 0x7fd0dc06fd30 2024-09-10T21:50:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.922+0000 7fd0db400640 1 --2- 172.21.15.7:0/1018206400 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd0bc077f70 0x7fd0bc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:11.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.923+0000 7fd0db400640 1 --2- 172.21.15.7:0/1018206400 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd0bc077f70 0x7fd0bc07a430 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0b5a60 tx=0x7fd0c8009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:11.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:11.926+0000 7fd0d3e00640 1 -- 172.21.15.7:0/1018206400 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd0cc062cb0 con 0x7fd0dc06fd30 2024-09-10T21:50:12.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.133+0000 7fd0e3200640 1 -- 172.21.15.7:0/1018206400 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd0dc0b2960 con 0x7fd0dc06fd30 2024-09-10T21:50:12.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.134+0000 7fd0d3e00640 1 -- 172.21.15.7:0/1018206400 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v101) ==== 80+0+0 (secure 0 0 0) 0x7fd0cc0625b0 con 0x7fd0dc06fd30 2024-09-10T21:50:12.135 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-10T21:50:12.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.136+0000 7fd0e3200640 1 -- 172.21.15.7:0/1018206400 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd0bc077f70 msgr2=0x7fd0bc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:12.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.136+0000 7fd0e3200640 1 --2- 172.21.15.7:0/1018206400 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd0bc077f70 0x7fd0bc07a430 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc0b5a60 tx=0x7fd0c8009040 comp rx=0 tx=0).stop 2024-09-10T21:50:12.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.136+0000 7fd0e3200640 1 -- 172.21.15.7:0/1018206400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0dc06fd30 msgr2=0x7fd0dc155720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:12.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.136+0000 7fd0e3200640 1 --2- 172.21.15.7:0/1018206400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0dc06fd30 0x7fd0dc155720 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc00d950 tx=0x7fd0cc00de10 comp rx=0 tx=0).stop 2024-09-10T21:50:12.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.136+0000 7fd0e3200640 1 -- 172.21.15.7:0/1018206400 shutdown_connections 2024-09-10T21:50:12.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.136+0000 7fd0e3200640 1 --2- 172.21.15.7:0/1018206400 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd0bc077f70 0x7fd0bc07a430 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:12.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.136+0000 7fd0e3200640 1 --2- 172.21.15.7:0/1018206400 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd0dc1f2a70 0x7fd0dc0c4340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:12.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.136+0000 7fd0e3200640 1 --2- 172.21.15.7:0/1018206400 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd0dc15a9a0 0x7fd0dc1f2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:12.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.136+0000 7fd0e3200640 1 --2- 172.21.15.7:0/1018206400 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd0dc06fd30 0x7fd0dc155720 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:12.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.136+0000 7fd0e3200640 1 -- 172.21.15.7:0/1018206400 >> 172.21.15.7:0/1018206400 conn(0x7fd0dc0b1660 msgr2=0x7fd0dc0b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:12.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.136+0000 7fd0e3200640 1 -- 172.21.15.7:0/1018206400 shutdown_connections 2024-09-10T21:50:12.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.136+0000 7fd0e3200640 1 -- 172.21.15.7:0/1018206400 wait complete. 2024-09-10T21:50:12.155 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:50:12.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:12.156 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:50:12.244 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:50:12.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:12.245 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:50:12.305 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:50:12.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.464+0000 7f6702e00640 1 -- 172.21.15.7:0/1189942395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0cd750 msgr2=0x7f66fc073730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:12.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.464+0000 7f6702e00640 1 --2- 172.21.15.7:0/1189942395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0cd750 0x7f66fc073730 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f66e4005fd0 tx=0x7f66e402a420 comp rx=0 tx=0).stop 2024-09-10T21:50:12.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.464+0000 7f6702e00640 1 -- 172.21.15.7:0/1189942395 shutdown_connections 2024-09-10T21:50:12.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.464+0000 7f6702e00640 1 --2- 172.21.15.7:0/1189942395 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0cd750 0x7f66fc073730 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:12.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.464+0000 7f6702e00640 1 --2- 172.21.15.7:0/1189942395 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66fc15bc90 0x7f66fc0ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:12.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.464+0000 7f6702e00640 1 -- 172.21.15.7:0/1189942395 >> 172.21.15.7:0/1189942395 conn(0x7f66fc0b16d0 msgr2=0x7f66fc0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:12.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.464+0000 7f6702e00640 1 -- 172.21.15.7:0/1189942395 shutdown_connections 2024-09-10T21:50:12.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.464+0000 7f6702e00640 1 -- 172.21.15.7:0/1189942395 wait complete. 2024-09-10T21:50:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.466+0000 7f6702e00640 1 Processor -- start 2024-09-10T21:50:12.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.466+0000 7f6702e00640 1 -- start start 2024-09-10T21:50:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.467+0000 7f6702e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66fc073c70 0x7f66fc0c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.467+0000 7f6702e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0cd750 0x7f66fc156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.467+0000 7f6702e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66fc15bc90 0x7f66fc1f2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.467+0000 7f6702e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc06fcf0 con 0x7f66fc0cd750 2024-09-10T21:50:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.467+0000 7f6702e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f66fc1606c0 con 0x7f66fc15bc90 2024-09-10T21:50:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.467+0000 7f6702e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f66fc06fe70 con 0x7f66fc073c70 2024-09-10T21:50:12.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.467+0000 7f66fb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0cd750 0x7f66fc156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.467+0000 7f66fb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0cd750 0x7f66fc156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59014/0 (socket says 172.21.15.7:59014) 2024-09-10T21:50:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.467+0000 7f66fb400640 1 -- 172.21.15.7:0/3078256029 learned_addr learned my addr 172.21.15.7:0/3078256029 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.467+0000 7f66fb400640 1 -- 172.21.15.7:0/3078256029 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66fc073c70 msgr2=0x7f66fc0c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.467+0000 7f66fbe00640 1 --2- 172.21.15.7:0/3078256029 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66fc073c70 0x7f66fc0c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.467+0000 7f6700e00640 1 --2- 172.21.15.7:0/3078256029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66fc15bc90 0x7f66fc1f2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.467+0000 7f66fb400640 1 --2- 172.21.15.7:0/3078256029 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66fc073c70 0x7f66fc0c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.467+0000 7f66fb400640 1 -- 172.21.15.7:0/3078256029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66fc15bc90 msgr2=0x7f66fc1f2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.467+0000 7f66fb400640 1 --2- 172.21.15.7:0/3078256029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66fc15bc90 0x7f66fc1f2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.467+0000 7f66fb400640 1 -- 172.21.15.7:0/3078256029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66fc1f30e0 con 0x7f66fc0cd750 2024-09-10T21:50:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.467+0000 7f66fbe00640 1 --2- 172.21.15.7:0/3078256029 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66fc073c70 0x7f66fc0c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:50:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.467+0000 7f6700e00640 1 --2- 172.21.15.7:0/3078256029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66fc15bc90 0x7f66fc1f2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:50:12.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.468+0000 7f66fb400640 1 --2- 172.21.15.7:0/3078256029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0cd750 0x7f66fc156410 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f66e4000c00 tx=0x7f66e40039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.468+0000 7f66f3e00640 1 -- 172.21.15.7:0/3078256029 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e403f070 con 0x7f66fc0cd750 2024-09-10T21:50:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.468+0000 7f66f3e00640 1 -- 172.21.15.7:0/3078256029 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66e4004890 con 0x7f66fc0cd750 2024-09-10T21:50:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.468+0000 7f66f3e00640 1 -- 172.21.15.7:0/3078256029 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66e4035030 con 0x7f66fc0cd750 2024-09-10T21:50:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.468+0000 7f6702e00640 1 -- 172.21.15.7:0/3078256029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66fc1f33c0 con 0x7f66fc0cd750 2024-09-10T21:50:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.468+0000 7f6702e00640 1 -- 172.21.15.7:0/3078256029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66fc0c7ba0 con 0x7f66fc0cd750 2024-09-10T21:50:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.469+0000 7f6702e00640 1 -- 172.21.15.7:0/3078256029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66fc073730 con 0x7f66fc0cd750 2024-09-10T21:50:12.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.471+0000 7f66f3e00640 1 -- 172.21.15.7:0/3078256029 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f66e4004ce0 con 0x7f66fc0cd750 2024-09-10T21:50:12.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.471+0000 7f66f3e00640 1 --2- 172.21.15.7:0/3078256029 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f66d8077f70 0x7f66d807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:12.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.471+0000 7f66f3e00640 1 -- 172.21.15.7:0/3078256029 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f66e40bafe0 con 0x7f66fc0cd750 2024-09-10T21:50:12.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.472+0000 7f66fbe00640 1 --2- 172.21.15.7:0/3078256029 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f66d8077f70 0x7f66d807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:12.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.473+0000 7f66fbe00640 1 --2- 172.21.15.7:0/3078256029 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f66d8077f70 0x7f66d807a430 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00ba60 tx=0x7f66ec005cf0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:12.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.475+0000 7f66f3e00640 1 -- 172.21.15.7:0/3078256029 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f66e4083a60 con 0x7f66fc0cd750 2024-09-10T21:50:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.722+0000 7f6702e00640 1 -- 172.21.15.7:0/3078256029 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66fc0c6c30 con 0x7f66fc0cd750 2024-09-10T21:50:12.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.722+0000 7f66f3e00640 1 -- 172.21.15.7:0/3078256029 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) ==== 94+0+5219 (secure 0 0 0) 0x7f66e4083360 con 0x7f66fc0cd750 2024-09-10T21:50:12.724 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:12.724 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":85,"btime":"2024-09-10T21:50:02:448373+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/431253133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":431253133},{"type":"v1","addr":"172.21.15.133:6836","nonce":431253133}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7521,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3692199178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3692199178},{"type":"v1","addr":"172.21.15.133:6837","nonce":3692199178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/752989865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":752989865},{"type":"v1","addr":"172.21.15.7:6834","nonce":752989865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:49:58.872982+0000","modified":"2024-09-10T21:50:01.884201+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7527},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7527":{"gid":7527,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.7:6835/3075619232","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3075619232},{"type":"v1","addr":"172.21.15.7:6835","nonce":3075619232}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7527,"qdb_cluster":[7527]},"id":7}]} 2024-09-10T21:50:12.724 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 85 2024-09-10T21:50:12.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.725+0000 7f6702e00640 1 -- 172.21.15.7:0/3078256029 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f66d8077f70 msgr2=0x7f66d807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:12.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.725+0000 7f6702e00640 1 --2- 172.21.15.7:0/3078256029 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f66d8077f70 0x7f66d807a430 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f66ec00ba60 tx=0x7f66ec005cf0 comp rx=0 tx=0).stop 2024-09-10T21:50:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.725+0000 7f6702e00640 1 -- 172.21.15.7:0/3078256029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0cd750 msgr2=0x7f66fc156410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.725+0000 7f6702e00640 1 --2- 172.21.15.7:0/3078256029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0cd750 0x7f66fc156410 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f66e4000c00 tx=0x7f66e40039e0 comp rx=0 tx=0).stop 2024-09-10T21:50:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.725+0000 7f6702e00640 1 -- 172.21.15.7:0/3078256029 shutdown_connections 2024-09-10T21:50:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.725+0000 7f6702e00640 1 --2- 172.21.15.7:0/3078256029 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f66d8077f70 0x7f66d807a430 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.725+0000 7f6702e00640 1 --2- 172.21.15.7:0/3078256029 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f66fc15bc90 0x7f66fc1f2640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.725+0000 7f6702e00640 1 --2- 172.21.15.7:0/3078256029 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f66fc0cd750 0x7f66fc156410 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.725+0000 7f6702e00640 1 --2- 172.21.15.7:0/3078256029 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f66fc073c70 0x7f66fc0c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.725+0000 7f6702e00640 1 -- 172.21.15.7:0/3078256029 >> 172.21.15.7:0/3078256029 conn(0x7f66fc0b16d0 msgr2=0x7f66fc0b47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.725+0000 7f6702e00640 1 -- 172.21.15.7:0/3078256029 shutdown_connections 2024-09-10T21:50:12.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:12.725+0000 7f6702e00640 1 -- 172.21.15.7:0/3078256029 wait complete. 2024-09-10T21:50:12.742 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-09-10T21:50:12.925 INFO:teuthology.orchestra.run.smithi007.stdout:{"message":"","return_code":0} 2024-09-10T21:50:12.925 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-09-10T21:50:12.925 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:50:12.925 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:50:12.961 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:50:12.961 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:50:12.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:12.961 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:50:13.035 INFO:teuthology.orchestra.run.smithi007.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:50:13.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:50:13.036 DEBUG:teuthology.orchestra.run.smithi007:> sudo daemon-helper kill ceph --cluster ceph -w 2024-09-10T21:50:13.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:13.081 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-09-10T21:50:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.245+0000 7fb77d400640 1 -- 172.21.15.7:0/3593355410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb77815bbd0 msgr2=0x7fb77806cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.245+0000 7fb77d400640 1 --2- 172.21.15.7:0/3593355410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb77815bbd0 0x7fb77806cb20 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fb768005fd0 tx=0x7fb76802a530 comp rx=0 tx=0).stop 2024-09-10T21:50:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.246+0000 7fb77d400640 1 -- 172.21.15.7:0/3593355410 shutdown_connections 2024-09-10T21:50:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.246+0000 7fb77d400640 1 --2- 172.21.15.7:0/3593355410 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb77806f970 0x7fb77806e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.246+0000 7fb77d400640 1 --2- 172.21.15.7:0/3593355410 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb77815bbd0 0x7fb77806cb20 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.246+0000 7fb77d400640 1 -- 172.21.15.7:0/3593355410 >> 172.21.15.7:0/3593355410 conn(0x7fb7780b2b40 msgr2=0x7fb7780b2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.246+0000 7fb77d400640 1 -- 172.21.15.7:0/3593355410 shutdown_connections 2024-09-10T21:50:13.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.246+0000 7fb77d400640 1 -- 172.21.15.7:0/3593355410 wait complete. 2024-09-10T21:50:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.247+0000 7fb77d400640 1 Processor -- start 2024-09-10T21:50:13.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.248+0000 7fb77d400640 1 -- start start 2024-09-10T21:50:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.248+0000 7fb77d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb77806f970 0x7fb7781f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.248+0000 7fb77d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb778073910 0x7fb778148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.248+0000 7fb77d400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb77815bbd0 0x7fb77815eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.248+0000 7fb77d400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb778074150 con 0x7fb778073910 2024-09-10T21:50:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.248+0000 7fb77d400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb778073fd0 con 0x7fb77806f970 2024-09-10T21:50:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.248+0000 7fb77d400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fb7780b01f0 con 0x7fb77815bbd0 2024-09-10T21:50:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.249+0000 7fb76de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb778073910 0x7fb778148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.249+0000 7fb76de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb778073910 0x7fb778148590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59032/0 (socket says 172.21.15.7:59032) 2024-09-10T21:50:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.249+0000 7fb776800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb77806f970 0x7fb7781f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.249+0000 7fb76de00640 1 -- 172.21.15.7:0/2147990221 learned_addr learned my addr 172.21.15.7:0/2147990221 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:13.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.249+0000 7fb777200640 1 --2- 172.21.15.7:0/2147990221 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb77815bbd0 0x7fb77815eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.249+0000 7fb76de00640 1 -- 172.21.15.7:0/2147990221 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb77815bbd0 msgr2=0x7fb77815eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.249+0000 7fb76de00640 1 --2- 172.21.15.7:0/2147990221 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb77815bbd0 0x7fb77815eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.249+0000 7fb76de00640 1 -- 172.21.15.7:0/2147990221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb77806f970 msgr2=0x7fb7781f66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.249+0000 7fb76de00640 1 --2- 172.21.15.7:0/2147990221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb77806f970 0x7fb7781f66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.249+0000 7fb76de00640 1 -- 172.21.15.7:0/2147990221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77809cde0 con 0x7fb778073910 2024-09-10T21:50:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.250+0000 7fb76de00640 1 --2- 172.21.15.7:0/2147990221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb778073910 0x7fb778148590 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fb76000c8f0 tx=0x7fb76000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.250+0000 7fb76f400640 1 -- 172.21.15.7:0/2147990221 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760007bb0 con 0x7fb778073910 2024-09-10T21:50:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.250+0000 7fb76f400640 1 -- 172.21.15.7:0/2147990221 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb760007e40 con 0x7fb778073910 2024-09-10T21:50:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.250+0000 7fb76f400640 1 -- 172.21.15.7:0/2147990221 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb760005280 con 0x7fb778073910 2024-09-10T21:50:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.250+0000 7fb77d400640 1 -- 172.21.15.7:0/2147990221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77809d0c0 con 0x7fb778073910 2024-09-10T21:50:13.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.250+0000 7fb77d400640 1 -- 172.21.15.7:0/2147990221 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb778148ad0 con 0x7fb778073910 2024-09-10T21:50:13.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.254+0000 7fb76f400640 1 -- 172.21.15.7:0/2147990221 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb760020060 con 0x7fb778073910 2024-09-10T21:50:13.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.254+0000 7fb780293740 1 Processor -- start 2024-09-10T21:50:13.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.254+0000 7fb780293740 1 -- start start 2024-09-10T21:50:13.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.255+0000 7fb780293740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec6ad10 0x55773ec6b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.255+0000 7fb780293740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55773ec25e80 0x55773ec697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.255+0000 7fb780293740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ec69d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.255+0000 7fb780293740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55773eb97f40 con 0x55773ec26400 2024-09-10T21:50:13.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.255+0000 7fb780293740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55773eb87370 con 0x55773ec6ad10 2024-09-10T21:50:13.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.255+0000 7fb780293740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55773eb8b180 con 0x55773ec25e80 2024-09-10T21:50:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.255+0000 7fb76f400640 1 --2- 172.21.15.7:0/2147990221 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb754077f70 0x7fb75407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.255+0000 7fb76f400640 1 -- 172.21.15.7:0/2147990221 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fb760099c70 con 0x7fb778073910 2024-09-10T21:50:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.255+0000 7fb777200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ec69d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.255+0000 7fb777200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ec69d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59042/0 (socket says 172.21.15.7:59042) 2024-09-10T21:50:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.255+0000 7fb777200640 1 -- 172.21.15.7:0/2527112443 learned_addr learned my addr 172.21.15.7:0/2527112443 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.255+0000 7fb776800640 1 --2- 172.21.15.7:0/2527112443 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55773ec25e80 0x55773ec697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.255+0000 7fb777200640 1 --2- 172.21.15.7:0/2527112443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec6ad10 0x55773ec6b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.255+0000 7fb76de00640 1 --2- 172.21.15.7:0/2147990221 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb754077f70 0x7fb75407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.256+0000 7fb777200640 1 -- 172.21.15.7:0/2527112443 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55773ec25e80 msgr2=0x55773ec697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.256+0000 7fb777200640 1 --2- 172.21.15.7:0/2527112443 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55773ec25e80 0x55773ec697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.256+0000 7fb777200640 1 -- 172.21.15.7:0/2527112443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec6ad10 msgr2=0x55773ec6b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:13.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.256+0000 7fb777200640 1 --2- 172.21.15.7:0/2527112443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec6ad10 0x55773ec6b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.256+0000 7fb777200640 1 -- 172.21.15.7:0/2527112443 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55773e879470 con 0x55773ec26400 2024-09-10T21:50:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.256+0000 7fb777200640 1 --2- 172.21.15.7:0/2527112443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ec69d30 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fb7700164f0 tx=0x7fb770005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68a7e1e1291f21a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.256+0000 7fb767000640 1 -- 172.21.15.7:0/2527112443 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77001d070 con 0x55773ec26400 2024-09-10T21:50:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.256+0000 7fb76de00640 1 --2- 172.21.15.7:0/2147990221 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb754077f70 0x7fb75407a430 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb760062010 tx=0x7fb7600046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.256+0000 7fb767000640 1 -- 172.21.15.7:0/2527112443 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb770018700 con 0x55773ec26400 2024-09-10T21:50:13.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.256+0000 7fb767000640 1 -- 172.21.15.7:0/2527112443 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770010040 con 0x55773ec26400 2024-09-10T21:50:13.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.257+0000 7fb780293740 1 -- 172.21.15.7:0/2527112443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 msgr2=0x55773ec69d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.257+0000 7fb780293740 1 --2- 172.21.15.7:0/2527112443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ec69d30 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fb7700164f0 tx=0x7fb770005210 comp rx=0 tx=0).stop 2024-09-10T21:50:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.257+0000 7fb780293740 1 -- 172.21.15.7:0/2527112443 shutdown_connections 2024-09-10T21:50:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.257+0000 7fb780293740 1 --2- 172.21.15.7:0/2527112443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ec69d30 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.257+0000 7fb780293740 1 --2- 172.21.15.7:0/2527112443 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55773ec25e80 0x55773ec697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.257+0000 7fb780293740 1 --2- 172.21.15.7:0/2527112443 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec6ad10 0x55773ec6b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.257+0000 7fb780293740 1 -- 172.21.15.7:0/2527112443 >> 172.21.15.7:0/2527112443 conn(0x55773ec21450 msgr2=0x55773ec24e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.257+0000 7fb780293740 1 -- 172.21.15.7:0/2527112443 shutdown_connections 2024-09-10T21:50:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.257+0000 7fb780293740 1 -- 172.21.15.7:0/2527112443 wait complete. 2024-09-10T21:50:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.258+0000 7fb780293740 1 -- start start 2024-09-10T21:50:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.258+0000 7fb780293740 1 Processor -- start 2024-09-10T21:50:13.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.259+0000 7fb780293740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ec65560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.259+0000 7fb780293740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec6ad10 0x55773ec49220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.259+0000 7fb780293740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55773ec4c720 0x55773ec4edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.259+0000 7fb780293740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55773e8f77c0 con 0x55773ec26400 2024-09-10T21:50:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.259+0000 7fb780293740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55773eb2cae0 con 0x55773ec6ad10 2024-09-10T21:50:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.259+0000 7fb780293740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55773ec1fe70 con 0x55773ec4c720 2024-09-10T21:50:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.259+0000 7fb780293740 10 client.0 _ll_register_callbacks cb 0x55773eb3c5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:50:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.259+0000 7fb777200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ec65560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.260+0000 7fb777200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ec65560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59058/0 (socket says 172.21.15.7:59058) 2024-09-10T21:50:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.260+0000 7fb777200640 1 -- 172.21.15.7:0/3064148408 learned_addr learned my addr 172.21.15.7:0/3064148408 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.260+0000 7fb776800640 1 --2- 172.21.15.7:0/3064148408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec6ad10 0x55773ec49220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.260+0000 7fb777200640 1 -- 172.21.15.7:0/3064148408 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55773ec4c720 msgr2=0x55773ec4edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.260+0000 7fb777200640 1 --2- 172.21.15.7:0/3064148408 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55773ec4c720 0x55773ec4edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.260+0000 7fb777200640 1 -- 172.21.15.7:0/3064148408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec6ad10 msgr2=0x55773ec49220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.260+0000 7fb777200640 1 --2- 172.21.15.7:0/3064148408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec6ad10 0x55773ec49220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.260+0000 7fb777200640 1 -- 172.21.15.7:0/3064148408 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55773e95d7e0 con 0x55773ec26400 2024-09-10T21:50:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.260+0000 7fb776800640 1 --2- 172.21.15.7:0/3064148408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec6ad10 0x55773ec49220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:50:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.260+0000 7fb777200640 1 --2- 172.21.15.7:0/3064148408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ec65560 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fb77001b040 tx=0x7fb770007e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc388e469934234 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.260+0000 7fb75b400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:50:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.260+0000 7fb75b400640 1 -- 172.21.15.7:0/3064148408 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77001d040 con 0x55773ec26400 2024-09-10T21:50:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.260+0000 7fb75b400640 1 -- 172.21.15.7:0/3064148408 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fb770006b50 con 0x55773ec26400 2024-09-10T21:50:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.260+0000 7fb780293740 1 -- 172.21.15.7:0/3064148408 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb770018700 con 0x55773ec26400 2024-09-10T21:50:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.261+0000 7fb75b400640 1 -- 172.21.15.7:0/3064148408 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fb770006d50 con 0x55773ec26400 2024-09-10T21:50:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.261+0000 7fb780293740 10 client.7725 fetch_fsmap learned FSMap version 85 2024-09-10T21:50:13.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.261+0000 7fb780293740 1 -- 172.21.15.7:0/3064148408 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x55773e9bdf70 con 0x55773ec26400 2024-09-10T21:50:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.261+0000 7fb75b400640 1 -- 172.21.15.7:0/3064148408 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb770018700 con 0x55773ec26400 2024-09-10T21:50:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.261+0000 7fb75b400640 1 -- 172.21.15.7:0/3064148408 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 85) ==== 3502+0+0 (secure 0 0 0) 0x7fb770018ae0 con 0x55773ec26400 2024-09-10T21:50:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.261+0000 7fb780293740 10 client.7725 fetch_fsmap finished waiting for FSMap version 85 2024-09-10T21:50:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.261+0000 7fb780293740 10 client.7725 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T21:50:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.261+0000 7fb780293740 20 client.7725 populate_metadata read hostname 'smithi007' 2024-09-10T21:50:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.261+0000 7fb780293740 1 --2- 172.21.15.7:0/3064148408 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x55773ec79d40 0x55773ec7a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.261+0000 7fb780293740 4 client.7725 mds_command: new command op to 7527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:50:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.261+0000 7fb780293740 1 -- 172.21.15.7:0/3064148408 --> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55773eb99e90 con 0x55773ec79d40 2024-09-10T21:50:13.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.261+0000 7fb776800640 1 --2- 172.21.15.7:0/3064148408 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x55773ec79d40 0x55773ec7a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.262+0000 7fb776800640 1 --2- 172.21.15.7:0/3064148408 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x55773ec79d40 0x55773ec7a120 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4d205a03d9a8f28 server_cookie=2fc83aa745e55f3b in_seq=0 out_seq=0 2024-09-10T21:50:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.262+0000 7fb75b400640 10 client.7725 ms_handle_connect on v2:172.21.15.7:6833/3075619232 2024-09-10T21:50:13.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.263+0000 7fb75b400640 1 -- 172.21.15.7:0/3064148408 <== mds.0 v2:172.21.15.7:6833/3075619232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55773eb99e90 con 0x55773ec79d40 2024-09-10T21:50:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.263+0000 7fb75b400640 10 client.7725 handle_command_reply: tid=0 2024-09-10T21:50:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.263+0000 7fb780293740 1 client.7725 shutdown 2024-09-10T21:50:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.263+0000 7fb780293740 1 -- 172.21.15.7:0/3064148408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 msgr2=0x55773ec65560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.263+0000 7fb780293740 1 --2- 172.21.15.7:0/3064148408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ec65560 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fb77001b040 tx=0x7fb770007e70 comp rx=0 tx=0).stop 2024-09-10T21:50:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.263+0000 7fb780293740 1 -- 172.21.15.7:0/3064148408 shutdown_connections 2024-09-10T21:50:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.263+0000 7fb780293740 1 --2- 172.21.15.7:0/3064148408 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x55773ec79d40 0x55773ec7a120 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.263+0000 7fb780293740 1 --2- 172.21.15.7:0/3064148408 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec6ad10 0x55773ec49220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.263+0000 7fb780293740 1 --2- 172.21.15.7:0/3064148408 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55773ec4c720 0x55773ec4edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.263+0000 7fb780293740 1 --2- 172.21.15.7:0/3064148408 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ec65560 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.263+0000 7fb780293740 1 -- 172.21.15.7:0/3064148408 >> 172.21.15.7:0/3064148408 conn(0x55773ec25e80 msgr2=0x55773ec1eba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.263+0000 7fb75b400640 0 client.7725 ms_handle_reset on v2:172.21.15.7:6833/3075619232 2024-09-10T21:50:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.263+0000 7fb780293740 1 -- 172.21.15.7:0/3064148408 shutdown_connections 2024-09-10T21:50:13.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.263+0000 7fb780293740 1 -- 172.21.15.7:0/3064148408 wait complete. 2024-09-10T21:50:13.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.264+0000 7fb780293740 20 client.7725 trim_cache size 0 max 16384 2024-09-10T21:50:13.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.270+0000 7fc241200640 1 -- 172.21.15.7:0/3877823643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c06f720 msgr2=0x7fc23c073cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.270+0000 7fc241200640 1 --2- 172.21.15.7:0/3877823643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c06f720 0x7fc23c073cc0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fc22c009830 tx=0x7fc22c02a510 comp rx=0 tx=0).stop 2024-09-10T21:50:13.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.270+0000 7fc241200640 1 -- 172.21.15.7:0/3877823643 shutdown_connections 2024-09-10T21:50:13.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.270+0000 7fc241200640 1 --2- 172.21.15.7:0/3877823643 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc23c06fb00 0x7fc23c15b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.270+0000 7fc241200640 1 --2- 172.21.15.7:0/3877823643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c06f720 0x7fc23c073cc0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.270+0000 7fc241200640 1 -- 172.21.15.7:0/3877823643 >> 172.21.15.7:0/3877823643 conn(0x7fc23c0b1740 msgr2=0x7fc23c0b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:13.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.271+0000 7fc241200640 1 -- 172.21.15.7:0/3877823643 shutdown_connections 2024-09-10T21:50:13.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.271+0000 7fc241200640 1 -- 172.21.15.7:0/3877823643 wait complete. 2024-09-10T21:50:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.272+0000 7fc241200640 1 Processor -- start 2024-09-10T21:50:13.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.272+0000 7fc241200640 1 -- start start 2024-09-10T21:50:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.273+0000 7fc241200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc23c06f720 0x7fc23c14e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.273+0000 7fc241200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc23c06fb00 0x7fc23c0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.273+0000 7fc241200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c15b890 0x7fc23c152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.273+0000 7fc241200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c151e60 con 0x7fc23c15b890 2024-09-10T21:50:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.273+0000 7fc241200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc23c15be90 con 0x7fc23c06f720 2024-09-10T21:50:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.273+0000 7fc241200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc23c151fe0 con 0x7fc23c06fb00 2024-09-10T21:50:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.274+0000 7fc23a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c15b890 0x7fc23c152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.274+0000 7fc23a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c15b890 0x7fc23c152520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59076/0 (socket says 172.21.15.7:59076) 2024-09-10T21:50:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.274+0000 7fc23a800640 1 -- 172.21.15.7:0/3545281989 learned_addr learned my addr 172.21.15.7:0/3545281989 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.274+0000 7fc233e00640 1 --2- 172.21.15.7:0/3545281989 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc23c06fb00 0x7fc23c0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.274+0000 7fc239e00640 1 --2- 172.21.15.7:0/3545281989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc23c06f720 0x7fc23c14e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.274+0000 7fc23a800640 1 -- 172.21.15.7:0/3545281989 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc23c06fb00 msgr2=0x7fc23c0c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.274+0000 7fc23a800640 1 --2- 172.21.15.7:0/3545281989 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc23c06fb00 0x7fc23c0c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.274+0000 7fc23a800640 1 -- 172.21.15.7:0/3545281989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc23c06f720 msgr2=0x7fc23c14e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.274+0000 7fc23a800640 1 --2- 172.21.15.7:0/3545281989 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc23c06f720 0x7fc23c14e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.274+0000 7fc23a800640 1 -- 172.21.15.7:0/3545281989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc23c0c3a70 con 0x7fc23c15b890 2024-09-10T21:50:13.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.274+0000 7fc23a800640 1 --2- 172.21.15.7:0/3545281989 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc23c15b890 0x7fc23c152520 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fc23400c6c0 tx=0x7fc23400cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.275+0000 7fc232000640 1 -- 172.21.15.7:0/3545281989 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc234017070 con 0x7fc23c15b890 2024-09-10T21:50:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.275+0000 7fc232000640 1 -- 172.21.15.7:0/3545281989 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc234003c60 con 0x7fc23c15b890 2024-09-10T21:50:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.275+0000 7fc241200640 1 -- 172.21.15.7:0/3545281989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc23c0c3d50 con 0x7fc23c15b890 2024-09-10T21:50:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.275+0000 7fc232000640 1 -- 172.21.15.7:0/3545281989 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2340052d0 con 0x7fc23c15b890 2024-09-10T21:50:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.275+0000 7fc241200640 1 -- 172.21.15.7:0/3545281989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc23c0c45f0 con 0x7fc23c15b890 2024-09-10T21:50:13.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.275+0000 7fc241200640 1 -- 172.21.15.7:0/3545281989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fc23c0cea80 con 0x7fc23c15b890 2024-09-10T21:50:13.278 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.277+0000 7fc232000640 1 -- 172.21.15.7:0/3545281989 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc234005470 con 0x7fc23c15b890 2024-09-10T21:50:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.278+0000 7fc232000640 1 --2- 172.21.15.7:0/3545281989 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc218077f70 0x7fc21807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.278+0000 7fc232000640 1 -- 172.21.15.7:0/3545281989 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fc234099d50 con 0x7fc23c15b890 2024-09-10T21:50:13.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.278+0000 7fc232000640 1 -- 172.21.15.7:0/3545281989 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+500 (secure 0 0 0) 0x7fc23c0cea80 con 0x7fc23c15b890 2024-09-10T21:50:13.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.278+0000 7fc239e00640 1 --2- 172.21.15.7:0/3545281989 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc218077f70 0x7fc21807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.279+0000 7fc241200640 1 -- 172.21.15.7:0/3545281989 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7fc23c0ce730 con 0x7fc23c15b890 2024-09-10T21:50:13.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.279+0000 7fc239e00640 1 --2- 172.21.15.7:0/3545281989 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc218077f70 0x7fc21807a430 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc22c034000 tx=0x7fc22c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:13.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.285+0000 7fb780293740 1 Processor -- start 2024-09-10T21:50:13.285 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.285+0000 7fb780293740 1 -- start start 2024-09-10T21:50:13.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.285+0000 7fb780293740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ec49350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.285+0000 7fb780293740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec4c720 0x55773ec4e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.285+0000 7fb780293740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55773ec6ad10 0x55773ec66d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.286+0000 7fb780293740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb7781fec90 con 0x55773ec26400 2024-09-10T21:50:13.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.286+0000 7fb780293740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb77814b380 con 0x55773ec4c720 2024-09-10T21:50:13.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.286+0000 7fb780293740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fb77814a6e0 con 0x55773ec6ad10 2024-09-10T21:50:13.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.286+0000 7fb777200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ec49350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.286+0000 7fb777200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ec49350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59082/0 (socket says 172.21.15.7:59082) 2024-09-10T21:50:13.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.286+0000 7fb777200640 1 -- 172.21.15.7:0/371341598 learned_addr learned my addr 172.21.15.7:0/371341598 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:13.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.286+0000 7fb777200640 1 --2- 172.21.15.7:0/371341598 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55773ec6ad10 0x55773ec66d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.287 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.286+0000 7fb776800640 1 --2- 172.21.15.7:0/371341598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec4c720 0x55773ec4e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.286+0000 7fb777200640 1 -- 172.21.15.7:0/371341598 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55773ec6ad10 msgr2=0x55773ec66d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:13.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.286+0000 7fb777200640 1 --2- 172.21.15.7:0/371341598 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55773ec6ad10 0x55773ec66d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.286+0000 7fb777200640 1 -- 172.21.15.7:0/371341598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec4c720 msgr2=0x55773ec4e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:13.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.286+0000 7fb777200640 1 --2- 172.21.15.7:0/371341598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec4c720 0x55773ec4e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.286+0000 7fb777200640 1 -- 172.21.15.7:0/371341598 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55773e8c6720 con 0x55773ec26400 2024-09-10T21:50:13.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.287+0000 7fb777200640 1 --2- 172.21.15.7:0/371341598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ec49350 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fb770005570 tx=0x7fb770018f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e03d76f345a67364 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.287+0000 7fb766600640 1 -- 172.21.15.7:0/371341598 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77001d070 con 0x55773ec26400 2024-09-10T21:50:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.287+0000 7fb766600640 1 -- 172.21.15.7:0/371341598 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7700183e0 con 0x55773ec26400 2024-09-10T21:50:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.288+0000 7fb766600640 1 -- 172.21.15.7:0/371341598 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb770007110 con 0x55773ec26400 2024-09-10T21:50:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.288+0000 7fb780293740 1 -- 172.21.15.7:0/371341598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 msgr2=0x55773ec49350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.288+0000 7fb780293740 1 --2- 172.21.15.7:0/371341598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ec49350 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fb770005570 tx=0x7fb770018f90 comp rx=0 tx=0).stop 2024-09-10T21:50:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.288+0000 7fb780293740 1 -- 172.21.15.7:0/371341598 shutdown_connections 2024-09-10T21:50:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.288+0000 7fb780293740 1 --2- 172.21.15.7:0/371341598 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec4c720 0x55773ec4e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.288+0000 7fb780293740 1 --2- 172.21.15.7:0/371341598 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55773ec6ad10 0x55773ec66d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.288+0000 7fb780293740 1 --2- 172.21.15.7:0/371341598 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ec49350 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.288+0000 7fb780293740 1 -- 172.21.15.7:0/371341598 >> 172.21.15.7:0/371341598 conn(0x55773ec25e80 msgr2=0x55773ec488f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.288+0000 7fb780293740 1 -- 172.21.15.7:0/371341598 shutdown_connections 2024-09-10T21:50:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.288+0000 7fb780293740 1 -- 172.21.15.7:0/371341598 wait complete. 2024-09-10T21:50:13.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.288+0000 7fb780293740 1 -- start start 2024-09-10T21:50:13.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.289+0000 7fb780293740 1 Processor -- start 2024-09-10T21:50:13.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.290+0000 7fb780293740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ecee690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.290+0000 7fb780293740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55773ec4c720 0x55773ec232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.290+0000 7fb780293740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec6ad10 0x55773ec8db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.290+0000 7fb780293740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb77814ad30 con 0x55773ec26400 2024-09-10T21:50:13.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.290+0000 7fb780293740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55773ec4d580 con 0x55773ec6ad10 2024-09-10T21:50:13.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.290+0000 7fb780293740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55773ec24dc0 con 0x55773ec4c720 2024-09-10T21:50:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.290+0000 7fb780293740 10 client.0 _ll_register_callbacks cb 0x7fb77001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:50:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.290+0000 7fb777200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ecee690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.290+0000 7fb777200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ecee690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59092/0 (socket says 172.21.15.7:59092) 2024-09-10T21:50:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.290+0000 7fb777200640 1 -- 172.21.15.7:0/2245415746 learned_addr learned my addr 172.21.15.7:0/2245415746 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.290+0000 7fb776800640 1 --2- 172.21.15.7:0/2245415746 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55773ec4c720 0x55773ec232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.290+0000 7fb777200640 1 --2- 172.21.15.7:0/2245415746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec6ad10 0x55773ec8db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.290+0000 7fb777200640 1 -- 172.21.15.7:0/2245415746 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55773ec4c720 msgr2=0x55773ec232c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.290+0000 7fb777200640 1 --2- 172.21.15.7:0/2245415746 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55773ec4c720 0x55773ec232c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.290+0000 7fb777200640 1 -- 172.21.15.7:0/2245415746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec6ad10 msgr2=0x55773ec8db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.290+0000 7fb777200640 1 --2- 172.21.15.7:0/2245415746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec6ad10 0x55773ec8db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.290+0000 7fb777200640 1 -- 172.21.15.7:0/2245415746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55773ec25010 con 0x55773ec26400 2024-09-10T21:50:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.291+0000 7fb777200640 1 --2- 172.21.15.7:0/2245415746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ecee690 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fb770005060 tx=0x7fb770005210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70410350377ea083 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:13.294 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.291+0000 7fb75b400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:50:13.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.291+0000 7fb75b400640 1 -- 172.21.15.7:0/2245415746 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77001d070 con 0x55773ec26400 2024-09-10T21:50:13.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.291+0000 7fb75b400640 1 -- 172.21.15.7:0/2245415746 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fb7700078d0 con 0x55773ec26400 2024-09-10T21:50:13.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.291+0000 7fb780293740 1 -- 172.21.15.7:0/2245415746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7700183e0 con 0x55773ec26400 2024-09-10T21:50:13.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.291+0000 7fb75b400640 1 -- 172.21.15.7:0/2245415746 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7fb770007ad0 con 0x55773ec26400 2024-09-10T21:50:13.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.291+0000 7fb780293740 10 client.7740 fetch_fsmap learned FSMap version 85 2024-09-10T21:50:13.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.291+0000 7fb780293740 1 -- 172.21.15.7:0/2245415746 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x55773ec49b70 con 0x55773ec26400 2024-09-10T21:50:13.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.292+0000 7fb75b400640 1 -- 172.21.15.7:0/2245415746 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7700183e0 con 0x55773ec26400 2024-09-10T21:50:13.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.292+0000 7fb75b400640 1 -- 172.21.15.7:0/2245415746 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 85) ==== 3502+0+0 (secure 0 0 0) 0x7fb770030450 con 0x55773ec26400 2024-09-10T21:50:13.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.292+0000 7fb780293740 10 client.7740 fetch_fsmap finished waiting for FSMap version 85 2024-09-10T21:50:13.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.292+0000 7fb780293740 10 client.7740 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T21:50:13.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.292+0000 7fb780293740 20 client.7740 populate_metadata read hostname 'smithi007' 2024-09-10T21:50:13.295 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.292+0000 7fb780293740 1 --2- 172.21.15.7:0/2245415746 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x55773ec4ce80 0x55773ec488f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.292+0000 7fb780293740 4 client.7740 mds_command: new command op to 7527 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-09-10T21:50:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.292+0000 7fb780293740 1 -- 172.21.15.7:0/2245415746 --> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55773ec67a50 con 0x55773ec4ce80 2024-09-10T21:50:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.292+0000 7fb776800640 1 --2- 172.21.15.7:0/2245415746 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x55773ec4ce80 0x55773ec488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.293+0000 7fb776800640 1 --2- 172.21.15.7:0/2245415746 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x55773ec4ce80 0x55773ec488f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28e131598d59bd9a server_cookie=cab12b96c087fd67 in_seq=0 out_seq=0 2024-09-10T21:50:13.296 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.293+0000 7fb75b400640 10 client.7740 ms_handle_connect on v2:172.21.15.7:6833/3075619232 2024-09-10T21:50:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.294+0000 7fb75b400640 1 -- 172.21.15.7:0/2245415746 <== mds.0 v2:172.21.15.7:6833/3075619232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55773ec67a50 con 0x55773ec4ce80 2024-09-10T21:50:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.294+0000 7fb75b400640 10 client.7740 handle_command_reply: tid=0 2024-09-10T21:50:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.294+0000 7fb780293740 1 client.7740 shutdown 2024-09-10T21:50:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.294+0000 7fb780293740 1 -- 172.21.15.7:0/2245415746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 msgr2=0x55773ecee690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.294+0000 7fb780293740 1 --2- 172.21.15.7:0/2245415746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ecee690 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fb770005060 tx=0x7fb770005210 comp rx=0 tx=0).stop 2024-09-10T21:50:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.294+0000 7fb780293740 1 -- 172.21.15.7:0/2245415746 shutdown_connections 2024-09-10T21:50:13.297 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.294+0000 7fb780293740 1 --2- 172.21.15.7:0/2245415746 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x55773ec4ce80 0x55773ec488f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.294+0000 7fb780293740 1 --2- 172.21.15.7:0/2245415746 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55773ec6ad10 0x55773ec8db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.294+0000 7fb780293740 1 --2- 172.21.15.7:0/2245415746 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55773ec4c720 0x55773ec232c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.294+0000 7fb780293740 1 --2- 172.21.15.7:0/2245415746 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55773ec26400 0x55773ecee690 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.294+0000 7fb780293740 1 -- 172.21.15.7:0/2245415746 >> 172.21.15.7:0/2245415746 conn(0x55773ec25e80 msgr2=0x55773ec4e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:13.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.294+0000 7fb75b400640 0 client.7740 ms_handle_reset on v2:172.21.15.7:6833/3075619232 2024-09-10T21:50:13.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.295+0000 7fb780293740 1 -- 172.21.15.7:0/2245415746 shutdown_connections 2024-09-10T21:50:13.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.295+0000 7fb780293740 1 -- 172.21.15.7:0/2245415746 wait complete. 2024-09-10T21:50:13.298 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.295+0000 7fb780293740 20 client.7740 trim_cache size 0 max 16384 2024-09-10T21:50:13.298 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:50:13.298 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-09-10T21:50:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "02846736-e3a0-466e-bce1-c3aa27f288c6", 2024-09-10T21:50:13.299 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-09-10T21:50:13.299 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:50:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.295+0000 7fb77d400640 1 -- 172.21.15.7:0/2147990221 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb754077f70 msgr2=0x7fb75407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.299 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.295+0000 7fb77d400640 1 --2- 172.21.15.7:0/2147990221 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb754077f70 0x7fb75407a430 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb760062010 tx=0x7fb7600046c0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.295+0000 7fb77d400640 1 -- 172.21.15.7:0/2147990221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb778073910 msgr2=0x7fb778148590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.295+0000 7fb77d400640 1 --2- 172.21.15.7:0/2147990221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb778073910 0x7fb778148590 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fb76000c8f0 tx=0x7fb76000cdb0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.295+0000 7fb77d400640 1 -- 172.21.15.7:0/2147990221 shutdown_connections 2024-09-10T21:50:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.295+0000 7fb77d400640 1 --2- 172.21.15.7:0/2147990221 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb754077f70 0x7fb75407a430 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.295+0000 7fb77d400640 1 --2- 172.21.15.7:0/2147990221 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb77815bbd0 0x7fb77815eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.295+0000 7fb77d400640 1 --2- 172.21.15.7:0/2147990221 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb778073910 0x7fb778148590 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.295+0000 7fb77d400640 1 --2- 172.21.15.7:0/2147990221 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb77806f970 0x7fb7781f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.295+0000 7fb77d400640 1 -- 172.21.15.7:0/2147990221 >> 172.21.15.7:0/2147990221 conn(0x7fb7780b2b40 msgr2=0x7fb7780b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.296+0000 7fb77d400640 1 -- 172.21.15.7:0/2147990221 shutdown_connections 2024-09-10T21:50:13.300 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.296+0000 7fb77d400640 1 -- 172.21.15.7:0/2147990221 wait complete. 2024-09-10T21:50:13.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:13.312 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-10T21:50:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.473+0000 7ffa41200640 1 -- 172.21.15.7:0/2529670054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3c074450 msgr2=0x7ffa3c0ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.473+0000 7ffa41200640 1 --2- 172.21.15.7:0/2529670054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3c074450 0x7ffa3c0ca270 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0b5c90 tx=0x7ffa3402aa30 comp rx=0 tx=0).stop 2024-09-10T21:50:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.473+0000 7ffa41200640 1 -- 172.21.15.7:0/2529670054 shutdown_connections 2024-09-10T21:50:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.473+0000 7ffa41200640 1 --2- 172.21.15.7:0/2529670054 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3c074450 0x7ffa3c0ca270 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.473+0000 7ffa41200640 1 --2- 172.21.15.7:0/2529670054 >> v2:172.21.15.133:3300/0 conn(0x7ffa3c06fcf0 0x7ffa3c1600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.473+0000 7ffa41200640 1 -- 172.21.15.7:0/2529670054 >> 172.21.15.7:0/2529670054 conn(0x7ffa3c0b16d0 msgr2=0x7ffa3c0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.473+0000 7ffa41200640 1 -- 172.21.15.7:0/2529670054 shutdown_connections 2024-09-10T21:50:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.473+0000 7ffa41200640 1 -- 172.21.15.7:0/2529670054 wait complete. 2024-09-10T21:50:13.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.475+0000 7ffa41200640 1 Processor -- start 2024-09-10T21:50:13.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.475+0000 7ffa41200640 1 -- start start 2024-09-10T21:50:13.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.476+0000 7ffa41200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3c06fcf0 0x7ffa3c152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.476+0000 7ffa41200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa3c074450 0x7ffa3c0b49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.476+0000 7ffa41200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa3c155fb0 0x7ffa3c0b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.476+0000 7ffa41200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c0ca930 con 0x7ffa3c06fcf0 2024-09-10T21:50:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.476+0000 7ffa41200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c0ca7b0 con 0x7ffa3c074450 2024-09-10T21:50:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.476+0000 7ffa41200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ffa3c0caab0 con 0x7ffa3c155fb0 2024-09-10T21:50:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.476+0000 7ffa39e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3c06fcf0 0x7ffa3c152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.476+0000 7ffa39e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3c06fcf0 0x7ffa3c152070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59116/0 (socket says 172.21.15.7:59116) 2024-09-10T21:50:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.476+0000 7ffa39e00640 1 -- 172.21.15.7:0/2528395309 learned_addr learned my addr 172.21.15.7:0/2528395309 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.476+0000 7ffa3a800640 1 --2- 172.21.15.7:0/2528395309 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa3c155fb0 0x7ffa3c0b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.476+0000 7ffa33e00640 1 --2- 172.21.15.7:0/2528395309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa3c074450 0x7ffa3c0b49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.476+0000 7ffa39e00640 1 -- 172.21.15.7:0/2528395309 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa3c155fb0 msgr2=0x7ffa3c0b4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.476+0000 7ffa39e00640 1 --2- 172.21.15.7:0/2528395309 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa3c155fb0 0x7ffa3c0b4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.477+0000 7ffa39e00640 1 -- 172.21.15.7:0/2528395309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa3c074450 msgr2=0x7ffa3c0b49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.477+0000 7ffa39e00640 1 --2- 172.21.15.7:0/2528395309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa3c074450 0x7ffa3c0b49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.477+0000 7ffa39e00640 1 -- 172.21.15.7:0/2528395309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3c1f44e0 con 0x7ffa3c06fcf0 2024-09-10T21:50:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.477+0000 7ffa33e00640 1 --2- 172.21.15.7:0/2528395309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa3c074450 0x7ffa3c0b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:50:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.477+0000 7ffa39e00640 1 --2- 172.21.15.7:0/2528395309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3c06fcf0 0x7ffa3c152070 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00c3a0 tx=0x7ffa2c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:13.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.477+0000 7ffa32000640 1 -- 172.21.15.7:0/2528395309 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c017070 con 0x7ffa3c06fcf0 2024-09-10T21:50:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.477+0000 7ffa32000640 1 -- 172.21.15.7:0/2528395309 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa2c0053c0 con 0x7ffa3c06fcf0 2024-09-10T21:50:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.477+0000 7ffa32000640 1 -- 172.21.15.7:0/2528395309 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c003850 con 0x7ffa3c06fcf0 2024-09-10T21:50:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.477+0000 7ffa41200640 1 -- 172.21.15.7:0/2528395309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3c1f47c0 con 0x7ffa3c06fcf0 2024-09-10T21:50:13.480 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.477+0000 7ffa41200640 1 -- 172.21.15.7:0/2528395309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa3c1f4d80 con 0x7ffa3c06fcf0 2024-09-10T21:50:13.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.481+0000 7ffa32000640 1 -- 172.21.15.7:0/2528395309 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ffa2c0039f0 con 0x7ffa3c06fcf0 2024-09-10T21:50:13.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.482+0000 7ffa32000640 1 --2- 172.21.15.7:0/2528395309 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ffa18077f70 0x7ffa1807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.482+0000 7ffa4427b740 1 Processor -- start 2024-09-10T21:50:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.482+0000 7ffa32000640 1 -- 172.21.15.7:0/2528395309 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7ffa2c0994d0 con 0x7ffa3c06fcf0 2024-09-10T21:50:13.483 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.482+0000 7ffa4427b740 1 -- start start 2024-09-10T21:50:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.482+0000 7ffa33e00640 1 --2- 172.21.15.7:0/2528395309 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ffa18077f70 0x7ffa1807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.483+0000 7ffa4427b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a980a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.483+0000 7ffa4427b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a97c52a0 0x55a9a9808c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.483+0000 7ffa4427b740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a97c5820 0x55a9a9809150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.483+0000 7ffa4427b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a9a97241c0 con 0x55a9a980a130 2024-09-10T21:50:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.483+0000 7ffa4427b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a9a9737870 con 0x55a9a97c52a0 2024-09-10T21:50:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.483+0000 7ffa4427b740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55a9a9726ca0 con 0x55a9a97c5820 2024-09-10T21:50:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.483+0000 7ffa33e00640 1 --2- 172.21.15.7:0/2528395309 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ffa18077f70 0x7ffa1807a430 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0d0020 tx=0x7ffa24005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.483+0000 7ffa39e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a980a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.483+0000 7ffa39e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a980a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59120/0 (socket says 172.21.15.7:59120) 2024-09-10T21:50:13.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.483+0000 7ffa39e00640 1 -- 172.21.15.7:0/2235972680 learned_addr learned my addr 172.21.15.7:0/2235972680 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.483+0000 7ffa33e00640 1 --2- 172.21.15.7:0/2235972680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a97c52a0 0x55a9a9808c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.483+0000 7ffa3a800640 1 --2- 172.21.15.7:0/2235972680 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a97c5820 0x55a9a9809150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.483+0000 7ffa39e00640 1 -- 172.21.15.7:0/2235972680 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a97c5820 msgr2=0x55a9a9809150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.484+0000 7ffa39e00640 1 --2- 172.21.15.7:0/2235972680 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a97c5820 0x55a9a9809150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.484+0000 7ffa39e00640 1 -- 172.21.15.7:0/2235972680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a97c52a0 msgr2=0x55a9a9808c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.484+0000 7ffa39e00640 1 --2- 172.21.15.7:0/2235972680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a97c52a0 0x55a9a9808c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.484+0000 7ffa39e00640 1 -- 172.21.15.7:0/2235972680 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9a94ff7d0 con 0x55a9a980a130 2024-09-10T21:50:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.484+0000 7ffa39e00640 1 --2- 172.21.15.7:0/2235972680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a980a510 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0978a0 tx=0x7ffa2c060fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3e5035fb0f0e610 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.484+0000 7ffa2aa00640 1 -- 172.21.15.7:0/2235972680 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c0617a0 con 0x55a9a980a130 2024-09-10T21:50:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.484+0000 7ffa2aa00640 1 -- 172.21.15.7:0/2235972680 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa2c066280 con 0x55a9a980a130 2024-09-10T21:50:13.486 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.484+0000 7ffa2aa00640 1 -- 172.21.15.7:0/2235972680 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c06ad30 con 0x55a9a980a130 2024-09-10T21:50:13.489 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.485+0000 7ffa4427b740 1 -- 172.21.15.7:0/2235972680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 msgr2=0x55a9a980a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.485+0000 7ffa4427b740 1 --2- 172.21.15.7:0/2235972680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a980a510 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c0978a0 tx=0x7ffa2c060fd0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.485+0000 7ffa4427b740 1 -- 172.21.15.7:0/2235972680 shutdown_connections 2024-09-10T21:50:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.485+0000 7ffa4427b740 1 --2- 172.21.15.7:0/2235972680 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a97c52a0 0x55a9a9808c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.485+0000 7ffa4427b740 1 --2- 172.21.15.7:0/2235972680 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a97c5820 0x55a9a9809150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.485+0000 7ffa4427b740 1 --2- 172.21.15.7:0/2235972680 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a980a510 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.485+0000 7ffa4427b740 1 -- 172.21.15.7:0/2235972680 >> 172.21.15.7:0/2235972680 conn(0x55a9a97c0870 msgr2=0x55a9a97c4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.485+0000 7ffa4427b740 1 -- 172.21.15.7:0/2235972680 shutdown_connections 2024-09-10T21:50:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.485+0000 7ffa4427b740 1 -- 172.21.15.7:0/2235972680 wait complete. 2024-09-10T21:50:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.486+0000 7ffa4427b740 1 -- start start 2024-09-10T21:50:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.486+0000 7ffa4427b740 1 Processor -- start 2024-09-10T21:50:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.487+0000 7ffa4427b740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a97c5820 0x55a9a9803590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.487+0000 7ffa4427b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a981c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.487+0000 7ffa4427b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a9803ad0 0x55a9a981cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.487+0000 7ffa4427b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a9a96cd410 con 0x55a9a980a130 2024-09-10T21:50:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.487+0000 7ffa4427b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a9a972aab0 con 0x55a9a9803ad0 2024-09-10T21:50:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.487+0000 7ffa4427b740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55a9a9497740 con 0x55a9a97c5820 2024-09-10T21:50:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.487+0000 7ffa4427b740 10 client.0 _ll_register_callbacks cb 0x55a9a96aa360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:50:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.487+0000 7ffa33e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a981c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.487+0000 7ffa33e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a981c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59122/0 (socket says 172.21.15.7:59122) 2024-09-10T21:50:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.487+0000 7ffa33e00640 1 -- 172.21.15.7:0/4249941832 learned_addr learned my addr 172.21.15.7:0/4249941832 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.487+0000 7ffa39e00640 1 --2- 172.21.15.7:0/4249941832 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a97c5820 0x55a9a9803590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.487+0000 7ffa3a800640 1 --2- 172.21.15.7:0/4249941832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a9803ad0 0x55a9a981cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.487+0000 7ffa33e00640 1 -- 172.21.15.7:0/4249941832 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a97c5820 msgr2=0x55a9a9803590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:13.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.488+0000 7ffa33e00640 1 --2- 172.21.15.7:0/4249941832 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a97c5820 0x55a9a9803590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.488+0000 7ffa33e00640 1 -- 172.21.15.7:0/4249941832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a9803ad0 msgr2=0x55a9a981cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.488+0000 7ffa33e00640 1 --2- 172.21.15.7:0/4249941832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a9803ad0 0x55a9a981cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.488+0000 7ffa33e00640 1 -- 172.21.15.7:0/4249941832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9a9488970 con 0x55a9a980a130 2024-09-10T21:50:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.488+0000 7ffa3a800640 1 --2- 172.21.15.7:0/4249941832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a9803ad0 0x55a9a981cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:50:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.488+0000 7ffa33e00640 1 --2- 172.21.15.7:0/4249941832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a981c6a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c06fca0 tx=0x7ffa24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a2c32a6cedee36 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.488+0000 7ffa1f400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:50:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.488+0000 7ffa1f400640 1 -- 172.21.15.7:0/4249941832 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2401fb40 con 0x55a9a980a130 2024-09-10T21:50:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.488+0000 7ffa4427b740 1 -- 172.21.15.7:0/4249941832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa2c066280 con 0x55a9a980a130 2024-09-10T21:50:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.488+0000 7ffa1f400640 1 -- 172.21.15.7:0/4249941832 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7ffa2401fce0 con 0x55a9a980a130 2024-09-10T21:50:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.488+0000 7ffa1f400640 1 -- 172.21.15.7:0/4249941832 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7ffa24020430 con 0x55a9a980a130 2024-09-10T21:50:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.488+0000 7ffa4427b740 10 client.7752 fetch_fsmap learned FSMap version 85 2024-09-10T21:50:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.488+0000 7ffa4427b740 1 -- 172.21.15.7:0/4249941832 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x55a9a94666a0 con 0x55a9a980a130 2024-09-10T21:50:13.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.489+0000 7ffa1f400640 1 -- 172.21.15.7:0/4249941832 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa2c066280 con 0x55a9a980a130 2024-09-10T21:50:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.489+0000 7ffa1f400640 1 -- 172.21.15.7:0/4249941832 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 85) ==== 3502+0+0 (secure 0 0 0) 0x7ffa24031050 con 0x55a9a980a130 2024-09-10T21:50:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.489+0000 7ffa4427b740 10 client.7752 fetch_fsmap finished waiting for FSMap version 85 2024-09-10T21:50:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.489+0000 7ffa4427b740 10 client.7752 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T21:50:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.489+0000 7ffa4427b740 20 client.7752 populate_metadata read hostname 'smithi007' 2024-09-10T21:50:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.489+0000 7ffa4427b740 1 --2- 172.21.15.7:0/4249941832 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x55a9a97eea50 0x55a9a981b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.489+0000 7ffa4427b740 4 client.7752 mds_command: new command op to 7527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:50:13.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.489+0000 7ffa4427b740 1 -- 172.21.15.7:0/4249941832 --> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9a97397c0 con 0x55a9a97eea50 2024-09-10T21:50:13.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.489+0000 7ffa39e00640 1 --2- 172.21.15.7:0/4249941832 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x55a9a97eea50 0x55a9a981b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.490+0000 7ffa39e00640 1 --2- 172.21.15.7:0/4249941832 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x55a9a97eea50 0x55a9a981b890 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1140505a65b5dd02 server_cookie=d43fa756ed0b4e5a in_seq=0 out_seq=0 2024-09-10T21:50:13.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.490+0000 7ffa1f400640 10 client.7752 ms_handle_connect on v2:172.21.15.7:6833/3075619232 2024-09-10T21:50:13.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.490+0000 7ffa1f400640 1 -- 172.21.15.7:0/4249941832 <== mds.0 v2:172.21.15.7:6833/3075619232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a9a97397c0 con 0x55a9a97eea50 2024-09-10T21:50:13.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.490+0000 7ffa1f400640 10 client.7752 handle_command_reply: tid=0 2024-09-10T21:50:13.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.490+0000 7ffa4427b740 1 client.7752 shutdown 2024-09-10T21:50:13.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.491+0000 7ffa4427b740 1 -- 172.21.15.7:0/4249941832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 msgr2=0x55a9a981c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.491+0000 7ffa4427b740 1 --2- 172.21.15.7:0/4249941832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a981c6a0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c06fca0 tx=0x7ffa24004730 comp rx=0 tx=0).stop 2024-09-10T21:50:13.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.491+0000 7ffa4427b740 1 -- 172.21.15.7:0/4249941832 shutdown_connections 2024-09-10T21:50:13.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.491+0000 7ffa4427b740 1 --2- 172.21.15.7:0/4249941832 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x55a9a97eea50 0x55a9a981b890 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.491+0000 7ffa4427b740 1 --2- 172.21.15.7:0/4249941832 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a9803ad0 0x55a9a981cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.491+0000 7ffa4427b740 1 --2- 172.21.15.7:0/4249941832 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a981c6a0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.491+0000 7ffa4427b740 1 --2- 172.21.15.7:0/4249941832 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a97c5820 0x55a9a9803590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.491+0000 7ffa1f400640 0 client.7752 ms_handle_reset on v2:172.21.15.7:6833/3075619232 2024-09-10T21:50:13.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.491+0000 7ffa4427b740 1 -- 172.21.15.7:0/4249941832 >> 172.21.15.7:0/4249941832 conn(0x55a9a97c52a0 msgr2=0x55a9a97bf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:13.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.491+0000 7ffa4427b740 1 -- 172.21.15.7:0/4249941832 shutdown_connections 2024-09-10T21:50:13.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.491+0000 7ffa4427b740 1 -- 172.21.15.7:0/4249941832 wait complete. 2024-09-10T21:50:13.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.491+0000 7ffa4427b740 20 client.7752 trim_cache size 0 max 16384 2024-09-10T21:50:13.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.510+0000 7ffa4427b740 1 Processor -- start 2024-09-10T21:50:13.511 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.510+0000 7ffa4427b740 1 -- start start 2024-09-10T21:50:13.512 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.511+0000 7ffa4427b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a97c5820 0x55a9a981b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.511+0000 7ffa4427b740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a9803ad0 0x55a9a97c3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.511+0000 7ffa4427b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a97bf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.511+0000 7ffa4427b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c201260 con 0x55a9a980a130 2024-09-10T21:50:13.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.511+0000 7ffa4427b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c200260 con 0x55a9a97c5820 2024-09-10T21:50:13.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.511+0000 7ffa4427b740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ffa3c15b4c0 con 0x55a9a9803ad0 2024-09-10T21:50:13.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.511+0000 7ffa3a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a97bf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.511+0000 7ffa3a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a97bf3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59138/0 (socket says 172.21.15.7:59138) 2024-09-10T21:50:13.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.511+0000 7ffa3a800640 1 -- 172.21.15.7:0/549945849 learned_addr learned my addr 172.21.15.7:0/549945849 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:13.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.511+0000 7ffa39e00640 1 --2- 172.21.15.7:0/549945849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a97c5820 0x55a9a981b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.511+0000 7ffa33e00640 1 --2- 172.21.15.7:0/549945849 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a9803ad0 0x55a9a97c3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.512+0000 7ffa3a800640 1 -- 172.21.15.7:0/549945849 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a9803ad0 msgr2=0x55a9a97c3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:13.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.512+0000 7ffa3a800640 1 --2- 172.21.15.7:0/549945849 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a9803ad0 0x55a9a97c3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.512+0000 7ffa3a800640 1 -- 172.21.15.7:0/549945849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a97c5820 msgr2=0x55a9a981b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:13.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.512+0000 7ffa3a800640 1 --2- 172.21.15.7:0/549945849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a97c5820 0x55a9a981b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.512+0000 7ffa3a800640 1 -- 172.21.15.7:0/549945849 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9a97eb540 con 0x55a9a980a130 2024-09-10T21:50:13.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.512+0000 7ffa33e00640 1 --2- 172.21.15.7:0/549945849 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a9803ad0 0x55a9a97c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:50:13.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.512+0000 7ffa3a800640 1 --2- 172.21.15.7:0/549945849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a97bf3b0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009d60 tx=0x7ffa34004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9c12c416ba167b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:13.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.512+0000 7ffa2a000640 1 -- 172.21.15.7:0/549945849 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34048070 con 0x55a9a980a130 2024-09-10T21:50:13.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.512+0000 7ffa2a000640 1 -- 172.21.15.7:0/549945849 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa3402f690 con 0x55a9a980a130 2024-09-10T21:50:13.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.512+0000 7ffa2a000640 1 -- 172.21.15.7:0/549945849 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34033050 con 0x55a9a980a130 2024-09-10T21:50:13.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.513+0000 7ffa4427b740 1 -- 172.21.15.7:0/549945849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 msgr2=0x55a9a97bf3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.513+0000 7ffa4427b740 1 --2- 172.21.15.7:0/549945849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a97bf3b0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7ffa34009d60 tx=0x7ffa34004ab0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.513+0000 7ffa4427b740 1 -- 172.21.15.7:0/549945849 shutdown_connections 2024-09-10T21:50:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.513+0000 7ffa4427b740 1 --2- 172.21.15.7:0/549945849 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a97bf3b0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.513+0000 7ffa4427b740 1 --2- 172.21.15.7:0/549945849 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a9803ad0 0x55a9a97c3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.513+0000 7ffa4427b740 1 --2- 172.21.15.7:0/549945849 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a97c5820 0x55a9a981b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.513+0000 7ffa4427b740 1 -- 172.21.15.7:0/549945849 >> 172.21.15.7:0/549945849 conn(0x55a9a97c52a0 msgr2=0x55a9a97b9ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.513+0000 7ffa4427b740 1 -- 172.21.15.7:0/549945849 shutdown_connections 2024-09-10T21:50:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.513+0000 7ffa4427b740 1 -- 172.21.15.7:0/549945849 wait complete. 2024-09-10T21:50:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.514+0000 7ffa4427b740 1 -- start start 2024-09-10T21:50:13.516 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.514+0000 7ffa4427b740 1 Processor -- start 2024-09-10T21:50:13.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.515+0000 7ffa4427b740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a97c5820 0x55a9a97e6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.515+0000 7ffa4427b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a9803ad0 0x55a9a9815ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.515+0000 7ffa4427b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a97d8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.515+0000 7ffa4427b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a9a980aa00 con 0x55a9a980a130 2024-09-10T21:50:13.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.515+0000 7ffa4427b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c1ffc40 con 0x55a9a9803ad0 2024-09-10T21:50:13.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.515+0000 7ffa4427b740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55a9a98038d0 con 0x55a9a97c5820 2024-09-10T21:50:13.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.515+0000 7ffa4427b740 10 client.0 _ll_register_callbacks cb 0x7ffa24011c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:50:13.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.515+0000 7ffa3a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a97d8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.515+0000 7ffa3a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a97d8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:59154/0 (socket says 172.21.15.7:59154) 2024-09-10T21:50:13.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.515+0000 7ffa3a800640 1 -- 172.21.15.7:0/1289772042 learned_addr learned my addr 172.21.15.7:0/1289772042 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:13.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.515+0000 7ffa39e00640 1 --2- 172.21.15.7:0/1289772042 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a97c5820 0x55a9a97e6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.515+0000 7ffa33e00640 1 --2- 172.21.15.7:0/1289772042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a9803ad0 0x55a9a9815ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.515+0000 7ffa3a800640 1 -- 172.21.15.7:0/1289772042 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a97c5820 msgr2=0x55a9a97e6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.516+0000 7ffa3a800640 1 --2- 172.21.15.7:0/1289772042 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a97c5820 0x55a9a97e6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.516+0000 7ffa3a800640 1 -- 172.21.15.7:0/1289772042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a9803ad0 msgr2=0x55a9a9815ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.516+0000 7ffa3a800640 1 --2- 172.21.15.7:0/1289772042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a9803ad0 0x55a9a9815ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.516+0000 7ffa3a800640 1 -- 172.21.15.7:0/1289772042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9a980d200 con 0x55a9a980a130 2024-09-10T21:50:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.516+0000 7ffa33e00640 1 --2- 172.21.15.7:0/1289772042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a9803ad0 0x55a9a9815ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:50:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.516+0000 7ffa3a800640 1 --2- 172.21.15.7:0/1289772042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a97d8150 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7ffa3402b020 tx=0x7ffa3402fe30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d35604201daecd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.516+0000 7ffa1f400640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:50:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.516+0000 7ffa1f400640 1 -- 172.21.15.7:0/1289772042 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa34053020 con 0x55a9a980a130 2024-09-10T21:50:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.516+0000 7ffa1f400640 1 -- 172.21.15.7:0/1289772042 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7ffa3404e670 con 0x55a9a980a130 2024-09-10T21:50:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.516+0000 7ffa4427b740 1 -- 172.21.15.7:0/1289772042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa3402f690 con 0x55a9a980a130 2024-09-10T21:50:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.516+0000 7ffa1f400640 1 -- 172.21.15.7:0/1289772042 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7ffa34033020 con 0x55a9a980a130 2024-09-10T21:50:13.520 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.516+0000 7ffa4427b740 10 client.7758 fetch_fsmap learned FSMap version 85 2024-09-10T21:50:13.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.516+0000 7ffa4427b740 1 -- 172.21.15.7:0/1289772042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x55a9a98155b0 con 0x55a9a980a130 2024-09-10T21:50:13.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.517+0000 7ffa1f400640 1 -- 172.21.15.7:0/1289772042 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa3402f690 con 0x55a9a980a130 2024-09-10T21:50:13.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.517+0000 7ffa1f400640 1 -- 172.21.15.7:0/1289772042 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 85) ==== 3502+0+0 (secure 0 0 0) 0x7ffa34004b40 con 0x55a9a980a130 2024-09-10T21:50:13.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.517+0000 7ffa4427b740 10 client.7758 fetch_fsmap finished waiting for FSMap version 85 2024-09-10T21:50:13.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.517+0000 7ffa4427b740 10 client.7758 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T21:50:13.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.517+0000 7ffa4427b740 20 client.7758 populate_metadata read hostname 'smithi007' 2024-09-10T21:50:13.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.517+0000 7ffa4427b740 1 --2- 172.21.15.7:0/1289772042 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x55a9a97d8690 0x55a9a9889fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:13.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.517+0000 7ffa4427b740 4 client.7758 mds_command: new command op to 7527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:50:13.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.517+0000 7ffa4427b740 1 -- 172.21.15.7:0/1289772042 --> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a9a9735270 con 0x55a9a97d8690 2024-09-10T21:50:13.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.518+0000 7ffa39e00640 1 --2- 172.21.15.7:0/1289772042 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x55a9a97d8690 0x55a9a9889fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:13.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.518+0000 7ffa39e00640 1 --2- 172.21.15.7:0/1289772042 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x55a9a97d8690 0x55a9a9889fe0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=589543297c8507f5 server_cookie=b2a602451728bb62 in_seq=0 out_seq=0 2024-09-10T21:50:13.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.518+0000 7ffa1f400640 10 client.7758 ms_handle_connect on v2:172.21.15.7:6833/3075619232 2024-09-10T21:50:13.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.518+0000 7ffa1f400640 1 -- 172.21.15.7:0/1289772042 <== mds.0 v2:172.21.15.7:6833/3075619232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+571 (crc 0 0 0) 0x55a9a9735270 con 0x55a9a97d8690 2024-09-10T21:50:13.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.518+0000 7ffa1f400640 10 client.7758 handle_command_reply: tid=0 2024-09-10T21:50:13.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.518+0000 7ffa4427b740 1 client.7758 shutdown 2024-09-10T21:50:13.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.518+0000 7ffa4427b740 1 -- 172.21.15.7:0/1289772042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 msgr2=0x55a9a97d8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.518+0000 7ffa4427b740 1 --2- 172.21.15.7:0/1289772042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a97d8150 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7ffa3402b020 tx=0x7ffa3402fe30 comp rx=0 tx=0).stop 2024-09-10T21:50:13.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.518+0000 7ffa4427b740 1 -- 172.21.15.7:0/1289772042 shutdown_connections 2024-09-10T21:50:13.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.518+0000 7ffa4427b740 1 --2- 172.21.15.7:0/1289772042 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x55a9a97d8690 0x55a9a9889fe0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.518+0000 7ffa4427b740 1 --2- 172.21.15.7:0/1289772042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a9a9803ad0 0x55a9a9815ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.518+0000 7ffa4427b740 1 --2- 172.21.15.7:0/1289772042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a9a980a130 0x55a9a97d8150 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.518+0000 7ffa4427b740 1 --2- 172.21.15.7:0/1289772042 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a9a97c5820 0x55a9a97e6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.519+0000 7ffa4427b740 1 -- 172.21.15.7:0/1289772042 >> 172.21.15.7:0/1289772042 conn(0x55a9a97c52a0 msgr2=0x55a9a980c050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:13.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.519+0000 7ffa1f400640 0 client.7758 ms_handle_reset on v2:172.21.15.7:6833/3075619232 2024-09-10T21:50:13.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.519+0000 7ffa4427b740 1 -- 172.21.15.7:0/1289772042 shutdown_connections 2024-09-10T21:50:13.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.519+0000 7ffa4427b740 1 -- 172.21.15.7:0/1289772042 wait complete. 2024-09-10T21:50:13.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.519+0000 7ffa4427b740 20 client.7758 trim_cache size 0 max 16384 2024-09-10T21:50:13.524 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:50:13.524 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:50:13.524 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-09-10T21:50:13.524 INFO:teuthology.orchestra.run.smithi007.stdout: "02846736-e3a0-466e-bce1-c3aa27f288c6": { 2024-09-10T21:50:13.524 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-09-10T21:50:13.524 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "02846736-e3a0-466e-bce1-c3aa27f288c6", 2024-09-10T21:50:13.524 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,repair" 2024-09-10T21:50:13.524 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:50:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:50:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:50:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: "02846736-e3a0-466e-bce1-c3aa27f288c6": { 2024-09-10T21:50:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:50:13.293129+0000", 2024-09-10T21:50:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-09-10T21:50:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:50:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:50:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:50:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:50:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:50:13.525 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:50:13.525 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:50:13.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.519+0000 7ffa41200640 1 -- 172.21.15.7:0/2528395309 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ffa18077f70 msgr2=0x7ffa1807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.519+0000 7ffa41200640 1 --2- 172.21.15.7:0/2528395309 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ffa18077f70 0x7ffa1807a430 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0d0020 tx=0x7ffa24005f90 comp rx=0 tx=0).stop 2024-09-10T21:50:13.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.519+0000 7ffa41200640 1 -- 172.21.15.7:0/2528395309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3c06fcf0 msgr2=0x7ffa3c152070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:13.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.519+0000 7ffa41200640 1 --2- 172.21.15.7:0/2528395309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3c06fcf0 0x7ffa3c152070 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c00c3a0 tx=0x7ffa2c00c860 comp rx=0 tx=0).stop 2024-09-10T21:50:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.520+0000 7ffa41200640 1 -- 172.21.15.7:0/2528395309 shutdown_connections 2024-09-10T21:50:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.520+0000 7ffa41200640 1 --2- 172.21.15.7:0/2528395309 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ffa18077f70 0x7ffa1807a430 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.520+0000 7ffa41200640 1 --2- 172.21.15.7:0/2528395309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ffa3c074450 0x7ffa3c0b49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.520+0000 7ffa41200640 1 --2- 172.21.15.7:0/2528395309 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ffa3c155fb0 0x7ffa3c0b4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.520+0000 7ffa41200640 1 --2- 172.21.15.7:0/2528395309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ffa3c06fcf0 0x7ffa3c152070 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.520+0000 7ffa41200640 1 -- 172.21.15.7:0/2528395309 >> 172.21.15.7:0/2528395309 conn(0x7ffa3c0b16d0 msgr2=0x7ffa3c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.520+0000 7ffa41200640 1 -- 172.21.15.7:0/2528395309 shutdown_connections 2024-09-10T21:50:13.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:13.520+0000 7ffa41200640 1 -- 172.21.15.7:0/2528395309 wait complete. 2024-09-10T21:50:13.536 INFO:tasks.cephfs.filesystem:scrub status for tag:02846736-e3a0-466e-bce1-c3aa27f288c6 - {'path': '/', 'tag': '02846736-e3a0-466e-bce1-c3aa27f288c6', 'options': 'recursive,repair'} 2024-09-10T21:50:14.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:14.504+0000 7fc232000640 1 -- 172.21.15.7:0/3545281989 <== mon.0 v2:172.21.15.7:3300/0 7 ==== log(4 entries from seq 1 at 2024-09-10T21:50:13.294206+0000) ==== 695+0+0 (secure 0 0 0) 0x7fc234005790 con 0x7fc23c15b890 2024-09-10T21:50:14.505 INFO:teuthology.orchestra.run.smithi007.stdout: cluster: 2024-09-10T21:50:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: id: 2029451a-3c26-48c7-b874-0597197278ac 2024-09-10T21:50:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: health: HEALTH_OK 2024-09-10T21:50:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: services: 2024-09-10T21:50:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: mon: 3 daemons, quorum a,b,c (age 8m) 2024-09-10T21:50:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: mgr: x(active, since 8m), standbys: y 2024-09-10T21:50:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: mds: 1/1 daemons up, 3 standby 2024-09-10T21:50:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: osd: 8 osds: 8 up (since 8m), 8 in (since 8m) 2024-09-10T21:50:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: data: 2024-09-10T21:50:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: volumes: 1/1 healthy 2024-09-10T21:50:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: pools: 3 pools, 97 pgs 2024-09-10T21:50:14.506 INFO:teuthology.orchestra.run.smithi007.stdout: objects: 27 objects, 11 MiB 2024-09-10T21:50:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: usage: 284 MiB used, 720 GiB / 720 GiB avail 2024-09-10T21:50:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: pgs: 97 active+clean 2024-09-10T21:50:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: io: 2024-09-10T21:50:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: client: 889 KiB/s wr, 0 op/s rd, 4 op/s wr 2024-09-10T21:50:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:14.507 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:14.507 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:50:13.294206+0000 mds.c [INF] scrub queued for path: / 2024-09-10T21:50:14.507 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:50:13.294210+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-09-10T21:50:14.507 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:50:13.294213+0000 mds.c [INF] scrub summary: active paths [/] 2024-09-10T21:50:14.507 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:50:13.296237+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-09-10T21:50:15.507 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:15.506+0000 7fc232000640 1 -- 172.21.15.7:0/3545281989 <== mon.0 v2:172.21.15.7:3300/0 8 ==== log(1 entries from seq 5 at 2024-09-10T21:50:14.148323+0000) ==== 189+0+0 (secure 0 0 0) 0x7fc2340982f0 con 0x7fc23c15b890 2024-09-10T21:50:15.507 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:50:14.148323+0000 mds.c [INF] scrub summary: idle 2024-09-10T21:50:31.395 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:50:31.400 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:50:43.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:43.537 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-10T21:50:43.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.712+0000 7f77a2800640 1 -- 172.21.15.7:0/2425592635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f779c159680 msgr2=0x7f779c15cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:43.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.712+0000 7f77a2800640 1 --2- 172.21.15.7:0/2425592635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f779c159680 0x7f779c15cc00 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f779c06faf0 tx=0x7f778c02a4d0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.713+0000 7f77a2800640 1 -- 172.21.15.7:0/2425592635 shutdown_connections 2024-09-10T21:50:43.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.713+0000 7f77a2800640 1 --2- 172.21.15.7:0/2425592635 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f779c073e20 0x7f779c0742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.713+0000 7f77a2800640 1 --2- 172.21.15.7:0/2425592635 >> v2:172.21.15.133:3300/0 conn(0x7f779c06e580 0x7f779c0cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.713+0000 7f77a2800640 1 --2- 172.21.15.7:0/2425592635 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f779c159680 0x7f779c15cc00 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.713+0000 7f77a2800640 1 -- 172.21.15.7:0/2425592635 >> 172.21.15.7:0/2425592635 conn(0x7f779c0b16d0 msgr2=0x7f779c0b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.713+0000 7f77a2800640 1 -- 172.21.15.7:0/2425592635 shutdown_connections 2024-09-10T21:50:43.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.713+0000 7f77a2800640 1 -- 172.21.15.7:0/2425592635 wait complete. 2024-09-10T21:50:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.715+0000 7f77a2800640 1 Processor -- start 2024-09-10T21:50:43.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.716+0000 7f77a2800640 1 -- start start 2024-09-10T21:50:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.717+0000 7f77a2800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f779c06e580 0x7f779c14c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.717+0000 7f77a2800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779c073e20 0x7f779c0c7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.717+0000 7f77a2800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f779c159680 0x7f779c0c7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.717+0000 7f77a2800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f779c0ca780 con 0x7f779c159680 2024-09-10T21:50:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.717+0000 7f77a2800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f779c0ca600 con 0x7f779c073e20 2024-09-10T21:50:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.717+0000 7f77a2800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f779c0ca900 con 0x7f779c06e580 2024-09-10T21:50:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.717+0000 7f779be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f779c159680 0x7f779c0c7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.717+0000 7f779be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f779c159680 0x7f779c0c7e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43804/0 (socket says 172.21.15.7:43804) 2024-09-10T21:50:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.717+0000 7f779be00640 1 -- 172.21.15.7:0/319963895 learned_addr learned my addr 172.21.15.7:0/319963895 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.718+0000 7f779b400640 1 --2- 172.21.15.7:0/319963895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f779c06e580 0x7f779c14c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.718+0000 7f7792a00640 1 --2- 172.21.15.7:0/319963895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779c073e20 0x7f779c0c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:43.719 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.718+0000 7f779be00640 1 -- 172.21.15.7:0/319963895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f779c06e580 msgr2=0x7f779c14c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.718+0000 7f779be00640 1 --2- 172.21.15.7:0/319963895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f779c06e580 0x7f779c14c460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.718+0000 7f779be00640 1 -- 172.21.15.7:0/319963895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779c073e20 msgr2=0x7f779c0c7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.718+0000 7f779be00640 1 --2- 172.21.15.7:0/319963895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779c073e20 0x7f779c0c7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.720 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.718+0000 7f779be00640 1 -- 172.21.15.7:0/319963895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f779c0c84a0 con 0x7f779c159680 2024-09-10T21:50:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.718+0000 7f779be00640 1 --2- 172.21.15.7:0/319963895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f779c159680 0x7f779c0c7e90 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f779400eb40 tx=0x7f779400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.718+0000 7f7793e00640 1 -- 172.21.15.7:0/319963895 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794004b20 con 0x7f779c159680 2024-09-10T21:50:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.719+0000 7f7793e00640 1 -- 172.21.15.7:0/319963895 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7794004ce0 con 0x7f779c159680 2024-09-10T21:50:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.719+0000 7f7793e00640 1 -- 172.21.15.7:0/319963895 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794005280 con 0x7f779c159680 2024-09-10T21:50:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.719+0000 7f77a2800640 1 -- 172.21.15.7:0/319963895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f779c0c8780 con 0x7f779c159680 2024-09-10T21:50:43.721 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.719+0000 7f77a2800640 1 -- 172.21.15.7:0/319963895 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f779c0c8d40 con 0x7f779c159680 2024-09-10T21:50:43.722 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.721+0000 7f7793e00640 1 -- 172.21.15.7:0/319963895 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7794005420 con 0x7f779c159680 2024-09-10T21:50:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.722+0000 7f7793e00640 1 --2- 172.21.15.7:0/319963895 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7778077fa0 0x7f777807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.722+0000 7f7793e00640 1 -- 172.21.15.7:0/319963895 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f779409a800 con 0x7f779c159680 2024-09-10T21:50:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.723+0000 7f779b400640 1 --2- 172.21.15.7:0/319963895 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7778077fa0 0x7f777807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.723+0000 7f77a57f0740 1 Processor -- start 2024-09-10T21:50:43.723 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.723+0000 7f77a57f0740 1 -- start start 2024-09-10T21:50:43.725 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.724+0000 7f77a57f0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 0x56377583b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:43.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.724+0000 7f779b400640 1 --2- 172.21.15.7:0/319963895 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7778077fa0 0x7f777807a460 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f778c033000 tx=0x7f778c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:43.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.724+0000 7f77a57f0740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637757f62a0 0x563775839c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:43.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.724+0000 7f77a57f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637757f6820 0x56377583a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:43.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.724+0000 7f77a57f0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5637757551c0 con 0x56377583b130 2024-09-10T21:50:43.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.724+0000 7f77a57f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x563775768870 con 0x5637757f6820 2024-09-10T21:50:43.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.724+0000 7f77a57f0740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x563775757ca0 con 0x5637757f62a0 2024-09-10T21:50:43.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.724+0000 7f779b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 0x56377583b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:43.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.724+0000 7f779b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 0x56377583b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43816/0 (socket says 172.21.15.7:43816) 2024-09-10T21:50:43.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.724+0000 7f779b400640 1 -- 172.21.15.7:0/3599976947 learned_addr learned my addr 172.21.15.7:0/3599976947 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:43.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.724+0000 7f779be00640 1 --2- 172.21.15.7:0/3599976947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637757f6820 0x56377583a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:43.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.724+0000 7f7792a00640 1 --2- 172.21.15.7:0/3599976947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637757f62a0 0x563775839c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:43.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.724+0000 7f779b400640 1 -- 172.21.15.7:0/3599976947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637757f62a0 msgr2=0x563775839c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:43.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.724+0000 7f779b400640 1 --2- 172.21.15.7:0/3599976947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637757f62a0 0x563775839c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.724+0000 7f779b400640 1 -- 172.21.15.7:0/3599976947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637757f6820 msgr2=0x56377583a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:43.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.724+0000 7f779b400640 1 --2- 172.21.15.7:0/3599976947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637757f6820 0x56377583a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.724+0000 7f779b400640 1 -- 172.21.15.7:0/3599976947 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637755307d0 con 0x56377583b130 2024-09-10T21:50:43.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.725+0000 7f779b400640 1 --2- 172.21.15.7:0/3599976947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 0x56377583b510 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f778c000c00 tx=0x7f778c03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7750319487d5ca server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:43.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.725+0000 7f778be00640 1 -- 172.21.15.7:0/3599976947 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c035040 con 0x56377583b130 2024-09-10T21:50:43.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.725+0000 7f778be00640 1 -- 172.21.15.7:0/3599976947 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f778c04e070 con 0x56377583b130 2024-09-10T21:50:43.727 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.725+0000 7f778be00640 1 -- 172.21.15.7:0/3599976947 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778c03c690 con 0x56377583b130 2024-09-10T21:50:43.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.726+0000 7f77a57f0740 1 -- 172.21.15.7:0/3599976947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 msgr2=0x56377583b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:43.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.726+0000 7f77a57f0740 1 --2- 172.21.15.7:0/3599976947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 0x56377583b510 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f778c000c00 tx=0x7f778c03c000 comp rx=0 tx=0).stop 2024-09-10T21:50:43.728 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.726+0000 7f77a57f0740 1 -- 172.21.15.7:0/3599976947 shutdown_connections 2024-09-10T21:50:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.726+0000 7f77a57f0740 1 --2- 172.21.15.7:0/3599976947 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637757f6820 0x56377583a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.726+0000 7f77a57f0740 1 --2- 172.21.15.7:0/3599976947 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637757f62a0 0x563775839c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.726+0000 7f77a57f0740 1 --2- 172.21.15.7:0/3599976947 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 0x56377583b510 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.726+0000 7f77a57f0740 1 -- 172.21.15.7:0/3599976947 >> 172.21.15.7:0/3599976947 conn(0x5637757f1870 msgr2=0x5637757f5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.726+0000 7f77a57f0740 1 -- 172.21.15.7:0/3599976947 shutdown_connections 2024-09-10T21:50:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.726+0000 7f77a57f0740 1 -- 172.21.15.7:0/3599976947 wait complete. 2024-09-10T21:50:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.727+0000 7f77a57f0740 1 -- start start 2024-09-10T21:50:43.729 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.727+0000 7f77a57f0740 1 Processor -- start 2024-09-10T21:50:43.731 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.728+0000 7f77a57f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637757f6820 0x563775833eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.728+0000 7f77a57f0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 0x56377581f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.728+0000 7f77a57f0740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56377584d900 0x5637757f3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.728+0000 7f77a57f0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5637756fe410 con 0x56377583b130 2024-09-10T21:50:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.728+0000 7f77a57f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x56377575bab0 con 0x5637757f6820 2024-09-10T21:50:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.728+0000 7f77a57f0740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5637754c8740 con 0x56377584d900 2024-09-10T21:50:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.728+0000 7f77a57f0740 10 client.0 _ll_register_callbacks cb 0x5637756db360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:50:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.728+0000 7f7792a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 0x56377581f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.728+0000 7f7792a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 0x56377581f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43820/0 (socket says 172.21.15.7:43820) 2024-09-10T21:50:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.728+0000 7f7792a00640 1 -- 172.21.15.7:0/3481003387 learned_addr learned my addr 172.21.15.7:0/3481003387 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.729+0000 7f779be00640 1 --2- 172.21.15.7:0/3481003387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56377584d900 0x5637757f3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:43.732 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.729+0000 7f779b400640 1 --2- 172.21.15.7:0/3481003387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637757f6820 0x563775833eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:43.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.729+0000 7f7792a00640 1 -- 172.21.15.7:0/3481003387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56377584d900 msgr2=0x5637757f3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:43.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.729+0000 7f7792a00640 1 --2- 172.21.15.7:0/3481003387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56377584d900 0x5637757f3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.729+0000 7f7792a00640 1 -- 172.21.15.7:0/3481003387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637757f6820 msgr2=0x563775833eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:43.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.729+0000 7f7792a00640 1 --2- 172.21.15.7:0/3481003387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637757f6820 0x563775833eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.729+0000 7f7792a00640 1 -- 172.21.15.7:0/3481003387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637754b9970 con 0x56377583b130 2024-09-10T21:50:43.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.729+0000 7f779b400640 1 --2- 172.21.15.7:0/3481003387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637757f6820 0x563775833eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:50:43.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.729+0000 7f7792a00640 1 --2- 172.21.15.7:0/3481003387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 0x56377581f100 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f778400bf50 tx=0x7f77840118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168282b64a9d84ff server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:43.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.729+0000 7f7788c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:50:43.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.729+0000 7f7788c00640 1 -- 172.21.15.7:0/3481003387 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784004020 con 0x56377583b130 2024-09-10T21:50:43.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.729+0000 7f7788c00640 1 -- 172.21.15.7:0/3481003387 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f77840041c0 con 0x56377583b130 2024-09-10T21:50:43.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.729+0000 7f77a57f0740 1 -- 172.21.15.7:0/3481003387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f778c04e070 con 0x56377583b130 2024-09-10T21:50:43.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.730+0000 7f7788c00640 1 -- 172.21.15.7:0/3481003387 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f7784023050 con 0x56377583b130 2024-09-10T21:50:43.733 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.730+0000 7f77a57f0740 10 client.7770 fetch_fsmap learned FSMap version 85 2024-09-10T21:50:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.730+0000 7f77a57f0740 1 -- 172.21.15.7:0/3481003387 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x5637754976a0 con 0x56377583b130 2024-09-10T21:50:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.730+0000 7f7788c00640 1 -- 172.21.15.7:0/3481003387 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f778c04e070 con 0x56377583b130 2024-09-10T21:50:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.730+0000 7f7788c00640 1 -- 172.21.15.7:0/3481003387 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 85) ==== 3502+0+0 (secure 0 0 0) 0x7f7784026020 con 0x56377583b130 2024-09-10T21:50:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.730+0000 7f77a57f0740 10 client.7770 fetch_fsmap finished waiting for FSMap version 85 2024-09-10T21:50:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.730+0000 7f77a57f0740 10 client.7770 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T21:50:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.730+0000 7f77a57f0740 20 client.7770 populate_metadata read hostname 'smithi007' 2024-09-10T21:50:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.730+0000 7f77a57f0740 1 --2- 172.21.15.7:0/3481003387 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x563775832190 0x5637758325b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.730+0000 7f77a57f0740 4 client.7770 mds_command: new command op to 7527 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:50:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.730+0000 7f77a57f0740 1 -- 172.21.15.7:0/3481003387 --> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56377576a7c0 con 0x563775832190 2024-09-10T21:50:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.730+0000 7f779b400640 1 --2- 172.21.15.7:0/3481003387 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x563775832190 0x5637758325b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.731+0000 7f779b400640 1 --2- 172.21.15.7:0/3481003387 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x563775832190 0x5637758325b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e61a6cea89633973 server_cookie=816cd8299c1608cc in_seq=0 out_seq=0 2024-09-10T21:50:43.734 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.731+0000 7f7788c00640 10 client.7770 ms_handle_connect on v2:172.21.15.7:6833/3075619232 2024-09-10T21:50:43.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.732+0000 7f7788c00640 1 -- 172.21.15.7:0/3481003387 <== mds.0 v2:172.21.15.7:6833/3075619232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56377576a7c0 con 0x563775832190 2024-09-10T21:50:43.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.732+0000 7f7788c00640 10 client.7770 handle_command_reply: tid=0 2024-09-10T21:50:43.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.732+0000 7f77a57f0740 1 client.7770 shutdown 2024-09-10T21:50:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.732+0000 7f77a57f0740 1 -- 172.21.15.7:0/3481003387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 msgr2=0x56377581f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.732+0000 7f77a57f0740 1 --2- 172.21.15.7:0/3481003387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 0x56377581f100 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f778400bf50 tx=0x7f77840118d0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.732+0000 7f77a57f0740 1 -- 172.21.15.7:0/3481003387 shutdown_connections 2024-09-10T21:50:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.732+0000 7f77a57f0740 1 --2- 172.21.15.7:0/3481003387 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x563775832190 0x5637758325b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.732+0000 7f77a57f0740 1 --2- 172.21.15.7:0/3481003387 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56377584d900 0x5637757f3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.732+0000 7f77a57f0740 1 --2- 172.21.15.7:0/3481003387 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 0x56377581f100 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.732+0000 7f77a57f0740 1 --2- 172.21.15.7:0/3481003387 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637757f6820 0x563775833eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.732+0000 7f7788c00640 0 client.7770 ms_handle_reset on v2:172.21.15.7:6833/3075619232 2024-09-10T21:50:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.732+0000 7f77a57f0740 1 -- 172.21.15.7:0/3481003387 >> 172.21.15.7:0/3481003387 conn(0x5637757f62a0 msgr2=0x5637757ee7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.732+0000 7f77a57f0740 1 -- 172.21.15.7:0/3481003387 shutdown_connections 2024-09-10T21:50:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.732+0000 7f77a57f0740 1 -- 172.21.15.7:0/3481003387 wait complete. 2024-09-10T21:50:43.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.733+0000 7f77a57f0740 20 client.7770 trim_cache size 0 max 16384 2024-09-10T21:50:43.752 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.752+0000 7f77a57f0740 1 Processor -- start 2024-09-10T21:50:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.752+0000 7f77a57f0740 1 -- start start 2024-09-10T21:50:43.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.752+0000 7f77a57f0740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637757f6820 0x56377583a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.753+0000 7f77a57f0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 0x5637757f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.753+0000 7f77a57f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56377584d900 0x5637757f3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.753+0000 7f77a57f0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f779c201180 con 0x56377583b130 2024-09-10T21:50:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.753+0000 7f77a57f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f779c200180 con 0x56377584d900 2024-09-10T21:50:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.753+0000 7f77a57f0740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f779c0b3780 con 0x5637757f6820 2024-09-10T21:50:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.753+0000 7f7792a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 0x5637757f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.753+0000 7f7792a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 0x5637757f2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43834/0 (socket says 172.21.15.7:43834) 2024-09-10T21:50:43.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.753+0000 7f7792a00640 1 -- 172.21.15.7:0/3664577306 learned_addr learned my addr 172.21.15.7:0/3664577306 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.753+0000 7f7792a00640 1 -- 172.21.15.7:0/3664577306 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637757f6820 msgr2=0x56377583a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.753+0000 7f779b400640 1 --2- 172.21.15.7:0/3664577306 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637757f6820 0x56377583a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.753+0000 7f779be00640 1 --2- 172.21.15.7:0/3664577306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56377584d900 0x5637757f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.753+0000 7f7792a00640 1 --2- 172.21.15.7:0/3664577306 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637757f6820 0x56377583a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.753+0000 7f7792a00640 1 -- 172.21.15.7:0/3664577306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56377584d900 msgr2=0x5637757f3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.753+0000 7f7792a00640 1 --2- 172.21.15.7:0/3664577306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56377584d900 0x5637757f3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.753+0000 7f7792a00640 1 -- 172.21.15.7:0/3664577306 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637757f0010 con 0x56377583b130 2024-09-10T21:50:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.753+0000 7f779b400640 1 --2- 172.21.15.7:0/3664577306 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637757f6820 0x56377583a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:50:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.753+0000 7f779be00640 1 --2- 172.21.15.7:0/3664577306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56377584d900 0x5637757f3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:50:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.754+0000 7f7792a00640 1 --2- 172.21.15.7:0/3664577306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 0x5637757f2cf0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f7784006010 tx=0x7f7784004480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da4d6f2d3a49dc11 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.754+0000 7f778b400640 1 -- 172.21.15.7:0/3664577306 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f778402b070 con 0x56377583b130 2024-09-10T21:50:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.754+0000 7f778b400640 1 -- 172.21.15.7:0/3664577306 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7784005160 con 0x56377583b130 2024-09-10T21:50:43.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.754+0000 7f778b400640 1 -- 172.21.15.7:0/3664577306 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77840055b0 con 0x56377583b130 2024-09-10T21:50:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.755+0000 7f77a57f0740 1 -- 172.21.15.7:0/3664577306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 msgr2=0x5637757f2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.755+0000 7f77a57f0740 1 --2- 172.21.15.7:0/3664577306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 0x5637757f2cf0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f7784006010 tx=0x7f7784004480 comp rx=0 tx=0).stop 2024-09-10T21:50:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.755+0000 7f77a57f0740 1 -- 172.21.15.7:0/3664577306 shutdown_connections 2024-09-10T21:50:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.755+0000 7f77a57f0740 1 --2- 172.21.15.7:0/3664577306 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56377584d900 0x5637757f3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.755+0000 7f77a57f0740 1 --2- 172.21.15.7:0/3664577306 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377583b130 0x5637757f2cf0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.755+0000 7f77a57f0740 1 --2- 172.21.15.7:0/3664577306 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637757f6820 0x56377583a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.755+0000 7f77a57f0740 1 -- 172.21.15.7:0/3664577306 >> 172.21.15.7:0/3664577306 conn(0x5637757f62a0 msgr2=0x563775833670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.755+0000 7f77a57f0740 1 -- 172.21.15.7:0/3664577306 shutdown_connections 2024-09-10T21:50:43.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.755+0000 7f77a57f0740 1 -- 172.21.15.7:0/3664577306 wait complete. 2024-09-10T21:50:43.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.756+0000 7f77a57f0740 1 -- start start 2024-09-10T21:50:43.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.756+0000 7f77a57f0740 1 Processor -- start 2024-09-10T21:50:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.757+0000 7f77a57f0740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637757f6820 0x5637758b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.757+0000 7f77a57f0740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56377583b130 0x563775839260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:43.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.757+0000 7f77a57f0740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377584d900 0x5637758397a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:43.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.757+0000 7f77a57f0740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x563775830440 con 0x56377584d900 2024-09-10T21:50:43.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.757+0000 7f77a57f0740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f779c0b3dd0 con 0x5637757f6820 2024-09-10T21:50:43.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.757+0000 7f77a57f0740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x56377581bdd0 con 0x56377583b130 2024-09-10T21:50:43.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.757+0000 7f77a57f0740 10 client.0 _ll_register_callbacks cb 0x7f778400bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:50:43.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.757+0000 7f779be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377584d900 0x5637758397a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:43.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.757+0000 7f779be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377584d900 0x5637758397a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43842/0 (socket says 172.21.15.7:43842) 2024-09-10T21:50:43.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.757+0000 7f779be00640 1 -- 172.21.15.7:0/1545113660 learned_addr learned my addr 172.21.15.7:0/1545113660 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:43.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.757+0000 7f779b400640 1 --2- 172.21.15.7:0/1545113660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637757f6820 0x5637758b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:43.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.757+0000 7f7792a00640 1 --2- 172.21.15.7:0/1545113660 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56377583b130 0x563775839260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:43.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.757+0000 7f779be00640 1 -- 172.21.15.7:0/1545113660 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56377583b130 msgr2=0x563775839260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:43.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.757+0000 7f779be00640 1 --2- 172.21.15.7:0/1545113660 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56377583b130 0x563775839260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.757+0000 7f779be00640 1 -- 172.21.15.7:0/1545113660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637757f6820 msgr2=0x5637758b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:43.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.757+0000 7f779be00640 1 --2- 172.21.15.7:0/1545113660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637757f6820 0x5637758b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.757+0000 7f779be00640 1 -- 172.21.15.7:0/1545113660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637757f33a0 con 0x56377584d900 2024-09-10T21:50:43.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.757+0000 7f7792a00640 1 --2- 172.21.15.7:0/1545113660 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56377583b130 0x563775839260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:50:43.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.758+0000 7f779be00640 1 --2- 172.21.15.7:0/1545113660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377584d900 0x5637758397a0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f7794062cd0 tx=0x7f77940632c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec791bcc546a884 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:43.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.758+0000 7f7788c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:50:43.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.758+0000 7f7788c00640 1 -- 172.21.15.7:0/1545113660 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7794062d40 con 0x56377584d900 2024-09-10T21:50:43.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.758+0000 7f7788c00640 1 -- 172.21.15.7:0/1545113660 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7f7794005950 con 0x56377584d900 2024-09-10T21:50:43.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.758+0000 7f77a57f0740 1 -- 172.21.15.7:0/1545113660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7784005160 con 0x56377584d900 2024-09-10T21:50:43.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.758+0000 7f7788c00640 1 -- 172.21.15.7:0/1545113660 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) ==== 24+0+0 (secure 0 0 0) 0x7f7794083050 con 0x56377584d900 2024-09-10T21:50:43.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.758+0000 7f77a57f0740 10 client.7776 fetch_fsmap learned FSMap version 85 2024-09-10T21:50:43.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.758+0000 7f77a57f0740 1 -- 172.21.15.7:0/1545113660 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) -- 0x563775830c50 con 0x56377584d900 2024-09-10T21:50:43.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.758+0000 7f7788c00640 1 -- 172.21.15.7:0/1545113660 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7784005160 con 0x56377584d900 2024-09-10T21:50:43.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.758+0000 7f7788c00640 1 -- 172.21.15.7:0/1545113660 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 85) ==== 3502+0+0 (secure 0 0 0) 0x7f7794061fa0 con 0x56377584d900 2024-09-10T21:50:43.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.759+0000 7f77a57f0740 10 client.7776 fetch_fsmap finished waiting for FSMap version 85 2024-09-10T21:50:43.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.759+0000 7f77a57f0740 10 client.7776 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T21:50:43.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.759+0000 7f77a57f0740 20 client.7776 populate_metadata read hostname 'smithi007' 2024-09-10T21:50:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.759+0000 7f77a57f0740 1 --2- 172.21.15.7:0/1545113660 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x56377581c6f0 0x5637758bade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.759+0000 7f77a57f0740 4 client.7776 mds_command: new command op to 7527 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:50:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.759+0000 7f77a57f0740 1 -- 172.21.15.7:0/1545113660 --> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563775766270 con 0x56377581c6f0 2024-09-10T21:50:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.759+0000 7f779b400640 1 --2- 172.21.15.7:0/1545113660 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x56377581c6f0 0x5637758bade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.759+0000 7f779b400640 1 --2- 172.21.15.7:0/1545113660 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x56377581c6f0 0x5637758bade0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=456b95dc1eca3366 server_cookie=6077d4a734769878 in_seq=0 out_seq=0 2024-09-10T21:50:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.759+0000 7f7788c00640 10 client.7776 ms_handle_connect on v2:172.21.15.7:6833/3075619232 2024-09-10T21:50:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.760+0000 7f7788c00640 1 -- 172.21.15.7:0/1545113660 <== mds.0 v2:172.21.15.7:6833/3075619232 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x563775766270 con 0x56377581c6f0 2024-09-10T21:50:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.760+0000 7f7788c00640 10 client.7776 handle_command_reply: tid=0 2024-09-10T21:50:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.760+0000 7f77a57f0740 1 client.7776 shutdown 2024-09-10T21:50:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.760+0000 7f77a57f0740 1 -- 172.21.15.7:0/1545113660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377584d900 msgr2=0x5637758397a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.760+0000 7f77a57f0740 1 --2- 172.21.15.7:0/1545113660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377584d900 0x5637758397a0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f7794062cd0 tx=0x7f77940632c0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.760+0000 7f77a57f0740 1 -- 172.21.15.7:0/1545113660 shutdown_connections 2024-09-10T21:50:43.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.760+0000 7f77a57f0740 1 --2- 172.21.15.7:0/1545113660 >> [v2:172.21.15.7:6833/3075619232,v1:172.21.15.7:6835/3075619232] conn(0x56377581c6f0 0x5637758bade0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.760+0000 7f77a57f0740 1 --2- 172.21.15.7:0/1545113660 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56377584d900 0x5637758397a0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.760+0000 7f77a57f0740 1 --2- 172.21.15.7:0/1545113660 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56377583b130 0x563775839260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.760+0000 7f77a57f0740 1 --2- 172.21.15.7:0/1545113660 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637757f6820 0x5637758b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.760+0000 7f7788c00640 0 client.7776 ms_handle_reset on v2:172.21.15.7:6833/3075619232 2024-09-10T21:50:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.760+0000 7f77a57f0740 1 -- 172.21.15.7:0/1545113660 >> 172.21.15.7:0/1545113660 conn(0x5637757f62a0 msgr2=0x5637757f03b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.760+0000 7f77a57f0740 1 -- 172.21.15.7:0/1545113660 shutdown_connections 2024-09-10T21:50:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.760+0000 7f77a57f0740 1 -- 172.21.15.7:0/1545113660 wait complete. 2024-09-10T21:50:43.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.760+0000 7f77a57f0740 20 client.7776 trim_cache size 0 max 16384 2024-09-10T21:50:43.765 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:50:43.765 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-09-10T21:50:43.766 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {}, 2024-09-10T21:50:43.766 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:50:43.766 INFO:teuthology.orchestra.run.smithi007.stdout: "02846736-e3a0-466e-bce1-c3aa27f288c6": { 2024-09-10T21:50:43.766 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:50:13.293129+0000", 2024-09-10T21:50:43.766 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-09-10T21:50:43.766 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:50:43.766 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:50:43.766 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:50:43.766 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:50:43.766 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:50:43.766 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:50:43.766 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:50:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.761+0000 7f77a2800640 1 -- 172.21.15.7:0/319963895 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7778077fa0 msgr2=0x7f777807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.761+0000 7f77a2800640 1 --2- 172.21.15.7:0/319963895 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7778077fa0 0x7f777807a460 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f778c033000 tx=0x7f778c004730 comp rx=0 tx=0).stop 2024-09-10T21:50:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.761+0000 7f77a2800640 1 -- 172.21.15.7:0/319963895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f779c159680 msgr2=0x7f779c0c7e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.761+0000 7f77a2800640 1 --2- 172.21.15.7:0/319963895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f779c159680 0x7f779c0c7e90 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f779400eb40 tx=0x7f779400c2d0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.761+0000 7f77a2800640 1 -- 172.21.15.7:0/319963895 shutdown_connections 2024-09-10T21:50:43.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.761+0000 7f77a2800640 1 --2- 172.21.15.7:0/319963895 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7778077fa0 0x7f777807a460 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.761+0000 7f77a2800640 1 --2- 172.21.15.7:0/319963895 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f779c073e20 0x7f779c0c7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.761+0000 7f77a2800640 1 --2- 172.21.15.7:0/319963895 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f779c159680 0x7f779c0c7e90 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.761+0000 7f77a2800640 1 --2- 172.21.15.7:0/319963895 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f779c06e580 0x7f779c14c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.761+0000 7f77a2800640 1 -- 172.21.15.7:0/319963895 >> 172.21.15.7:0/319963895 conn(0x7f779c0b16d0 msgr2=0x7f779c153bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.761+0000 7f77a2800640 1 -- 172.21.15.7:0/319963895 shutdown_connections 2024-09-10T21:50:43.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:43.761+0000 7f77a2800640 1 -- 172.21.15.7:0/319963895 wait complete. 2024-09-10T21:50:43.776 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:50:43.877 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:50:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.041+0000 7fd40ec00640 1 -- 172.21.15.7:0/4230760770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40815cf30 msgr2=0x7fd40815d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.041+0000 7fd40ec00640 1 --2- 172.21.15.7:0/4230760770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40815cf30 0x7fd40815d380 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fd408051fa0 tx=0x7fd40002a9a0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.041+0000 7fd40ec00640 1 -- 172.21.15.7:0/4230760770 shutdown_connections 2024-09-10T21:50:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.041+0000 7fd40ec00640 1 --2- 172.21.15.7:0/4230760770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40815cf30 0x7fd40815d380 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.041+0000 7fd40ec00640 1 --2- 172.21.15.7:0/4230760770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd40815b120 0x7fd40815b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.041+0000 7fd40ec00640 1 --2- 172.21.15.7:0/4230760770 >> v2:172.21.15.133:3300/0 conn(0x7fd40806fe40 0x7fd408070220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.041+0000 7fd40ec00640 1 -- 172.21.15.7:0/4230760770 >> 172.21.15.7:0/4230760770 conn(0x7fd4080b16d0 msgr2=0x7fd4080b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.041+0000 7fd40ec00640 1 -- 172.21.15.7:0/4230760770 shutdown_connections 2024-09-10T21:50:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.041+0000 7fd40ec00640 1 -- 172.21.15.7:0/4230760770 wait complete. 2024-09-10T21:50:44.043 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.043+0000 7fd40ec00640 1 Processor -- start 2024-09-10T21:50:44.044 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.043+0000 7fd40ec00640 1 -- start start 2024-09-10T21:50:44.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.044+0000 7fd40ec00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40806fe40 0x7fd408148330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.044+0000 7fd40ec00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd40815b120 0x7fd408149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.044+0000 7fd40ec00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd40815cf30 0x7fd408149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.044+0000 7fd40ec00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd40815d8c0 con 0x7fd40806fe40 2024-09-10T21:50:44.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.044+0000 7fd40ec00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd40815cad0 con 0x7fd40815b120 2024-09-10T21:50:44.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.044+0000 7fd40ec00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fd40815da40 con 0x7fd40815cf30 2024-09-10T21:50:44.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.044+0000 7fd40d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40806fe40 0x7fd408148330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.044+0000 7fd40d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40806fe40 0x7fd408148330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36848/0 (socket says 172.21.15.7:36848) 2024-09-10T21:50:44.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.044+0000 7fd40d800640 1 -- 172.21.15.7:0/1555298385 learned_addr learned my addr 172.21.15.7:0/1555298385 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.044+0000 7fd40ce00640 1 --2- 172.21.15.7:0/1555298385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd40815b120 0x7fd408149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.044+0000 7fd40e200640 1 --2- 172.21.15.7:0/1555298385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd40815cf30 0x7fd408149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.044+0000 7fd40d800640 1 -- 172.21.15.7:0/1555298385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd40815cf30 msgr2=0x7fd408149780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.044+0000 7fd40d800640 1 --2- 172.21.15.7:0/1555298385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd40815cf30 0x7fd408149780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.044+0000 7fd40d800640 1 -- 172.21.15.7:0/1555298385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd40815b120 msgr2=0x7fd408149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.044+0000 7fd40d800640 1 --2- 172.21.15.7:0/1555298385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd40815b120 0x7fd408149240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.045+0000 7fd40d800640 1 -- 172.21.15.7:0/1555298385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd408148050 con 0x7fd40806fe40 2024-09-10T21:50:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.045+0000 7fd40d800640 1 --2- 172.21.15.7:0/1555298385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40806fe40 0x7fd408148330 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00e9f0 tx=0x7fd3fc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.045+0000 7fd405600640 1 -- 172.21.15.7:0/1555298385 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc00cc60 con 0x7fd40806fe40 2024-09-10T21:50:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.045+0000 7fd405600640 1 -- 172.21.15.7:0/1555298385 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3fc010430 con 0x7fd40806fe40 2024-09-10T21:50:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.045+0000 7fd405600640 1 -- 172.21.15.7:0/1555298385 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3fc010840 con 0x7fd40806fe40 2024-09-10T21:50:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.045+0000 7fd40ec00640 1 -- 172.21.15.7:0/1555298385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd40814c2e0 con 0x7fd40806fe40 2024-09-10T21:50:44.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.045+0000 7fd40ec00640 1 -- 172.21.15.7:0/1555298385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd408154dd0 con 0x7fd40806fe40 2024-09-10T21:50:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.046+0000 7fd40ec00640 1 -- 172.21.15.7:0/1555298385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd40815dfd0 con 0x7fd40806fe40 2024-09-10T21:50:44.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.048+0000 7fd405600640 1 -- 172.21.15.7:0/1555298385 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd3fc0109e0 con 0x7fd40806fe40 2024-09-10T21:50:44.049 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.048+0000 7fd405600640 1 --2- 172.21.15.7:0/1555298385 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd3e8077f70 0x7fd3e807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.049+0000 7fd405600640 1 -- 172.21.15.7:0/1555298385 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 5709+0+0 (secure 0 0 0) 0x7fd3fc09a7b0 con 0x7fd40806fe40 2024-09-10T21:50:44.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.049+0000 7fd40ce00640 1 --2- 172.21.15.7:0/1555298385 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd3e8077f70 0x7fd3e807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.050+0000 7fd40ce00640 1 --2- 172.21.15.7:0/1555298385 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd3e8077f70 0x7fd3e807a430 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd408151c00 tx=0x7fd3f4005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.053+0000 7fd405600640 1 -- 172.21.15.7:0/1555298385 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd3fc063340 con 0x7fd40806fe40 2024-09-10T21:50:44.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.283+0000 7fd40ec00640 1 -- 172.21.15.7:0/1555298385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fd408072f40 con 0x7fd40806fe40 2024-09-10T21:50:44.284 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-10T21:50:44.284+0000 7feb3fa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:50:44.286 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.286+0000 7fd405600640 1 -- 172.21.15.7:0/1555298385 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) ==== 161+0+0 (secure 0 0 0) 0x7fd3fc062c40 con 0x7fd40806fe40 2024-09-10T21:50:44.286 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:50:44.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.289+0000 7fd40ec00640 1 -- 172.21.15.7:0/1555298385 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd3e8077f70 msgr2=0x7fd3e807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.289+0000 7fd40ec00640 1 --2- 172.21.15.7:0/1555298385 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd3e8077f70 0x7fd3e807a430 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd408151c00 tx=0x7fd3f4005f90 comp rx=0 tx=0).stop 2024-09-10T21:50:44.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.289+0000 7fd40ec00640 1 -- 172.21.15.7:0/1555298385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40806fe40 msgr2=0x7fd408148330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.289+0000 7fd40ec00640 1 --2- 172.21.15.7:0/1555298385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40806fe40 0x7fd408148330 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fd3fc00e9f0 tx=0x7fd3fc00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.290 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.289+0000 7fd40ec00640 1 -- 172.21.15.7:0/1555298385 shutdown_connections 2024-09-10T21:50:44.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.289+0000 7fd40ec00640 1 --2- 172.21.15.7:0/1555298385 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd3e8077f70 0x7fd3e807a430 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.289+0000 7fd40ec00640 1 --2- 172.21.15.7:0/1555298385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd40815b120 0x7fd408149240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.289+0000 7fd40ec00640 1 --2- 172.21.15.7:0/1555298385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd40815cf30 0x7fd408149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.289+0000 7fd40ec00640 1 --2- 172.21.15.7:0/1555298385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd40806fe40 0x7fd408148330 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.289+0000 7fd40ec00640 1 -- 172.21.15.7:0/1555298385 >> 172.21.15.7:0/1555298385 conn(0x7fd4080b16d0 msgr2=0x7fd4080ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:44.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.289+0000 7fd40ec00640 1 -- 172.21.15.7:0/1555298385 shutdown_connections 2024-09-10T21:50:44.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.289+0000 7fd40ec00640 1 -- 172.21.15.7:0/1555298385 wait complete. 2024-09-10T21:50:44.301 INFO:tasks.ceph.mds.c.smithi007.stderr: -7970> 2024-09-10T21:50:05.400+0000 7f561f200640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:50:44.307 DEBUG:teuthology.orchestra.run.smithi007:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627777 summary 2024-09-10T21:50:44.360 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-09-10T21:50:44.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.396+0000 7f0d4f60e280 1 -- start start 2024-09-10T21:50:44.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.396+0000 7f0d4f60e280 1 Processor -- start 2024-09-10T21:50:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.397+0000 7f0d4f60e280 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617c51aac00 0x5617c5f86000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.397+0000 7f0d4f60e280 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617c51ab800 0x5617c5f86580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.397+0000 7f0d4f60e280 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c51abc00 0x5617c5f86b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.397+0000 7f0d4f60e280 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5617c5f90180 con 0x5617c51abc00 2024-09-10T21:50:44.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.397+0000 7f0d4f60e280 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5617c5f90300 con 0x5617c51ab800 2024-09-10T21:50:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.397+0000 7f0d4f60e280 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5617c5f90480 con 0x5617c51aac00 2024-09-10T21:50:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.397+0000 7f0d4d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c51abc00 0x5617c5f86b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.397+0000 7f0d4d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c51abc00 0x5617c5f86b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36900/0 (socket says 172.21.15.7:36900) 2024-09-10T21:50:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.397+0000 7f0d4d400640 1 -- 172.21.15.7:0/1239665186 learned_addr learned my addr 172.21.15.7:0/1239665186 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.397+0000 7f0d4c000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617c51ab800 0x5617c5f86580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.397+0000 7f0d4ca00640 1 --2- 172.21.15.7:0/1239665186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617c51aac00 0x5617c5f86000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.398+0000 7f0d4d400640 1 -- 172.21.15.7:0/1239665186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617c51aac00 msgr2=0x5617c5f86000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.398+0000 7f0d4d400640 1 --2- 172.21.15.7:0/1239665186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617c51aac00 0x5617c5f86000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.398+0000 7f0d4d400640 1 -- 172.21.15.7:0/1239665186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617c51ab800 msgr2=0x5617c5f86580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.398+0000 7f0d4d400640 1 --2- 172.21.15.7:0/1239665186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617c51ab800 0x5617c5f86580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.398+0000 7f0d4d400640 1 -- 172.21.15.7:0/1239665186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617c51e2780 con 0x5617c51abc00 2024-09-10T21:50:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.398+0000 7f0d4c000640 1 --2- 172.21.15.7:0/1239665186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617c51ab800 0x5617c5f86580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:50:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.398+0000 7f0d4d400640 1 --2- 172.21.15.7:0/1239665186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c51abc00 0x5617c5f86b00 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x5617c5fe1b30 tx=0x5617c5f48ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b338825139082516 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.398+0000 7f0d49800640 1 -- 172.21.15.7:0/1239665186 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5617c51d0ea0 con 0x5617c51abc00 2024-09-10T21:50:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.398+0000 7f0d49800640 1 -- 172.21.15.7:0/1239665186 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5617c51d7180 con 0x5617c51abc00 2024-09-10T21:50:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.398+0000 7f0d4f60e280 1 -- 172.21.15.7:0/1239665186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617c51e23c0 con 0x5617c51abc00 2024-09-10T21:50:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.398+0000 7f0d4f60e280 1 -- 172.21.15.7:0/1239665186 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5617c51e3680 con 0x5617c51abc00 2024-09-10T21:50:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.398+0000 7f0d4f60e280 4 waiting for MDS map... 2024-09-10T21:50:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.398+0000 7f0d49800640 1 -- 172.21.15.7:0/1239665186 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5617c51d1380 con 0x5617c51abc00 2024-09-10T21:50:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.399+0000 7f0d49800640 1 -- 172.21.15.7:0/1239665186 <== mon.0 v2:172.21.15.7:3300/0 4 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x5617c5f26600 con 0x5617c51abc00 2024-09-10T21:50:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.399+0000 7f0d49800640 1 -- 172.21.15.7:0/1239665186 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 86) ==== 2893+0+0 (secure 0 0 0) 0x5617c5205180 con 0x5617c51abc00 2024-09-10T21:50:44.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.399+0000 7f0d4f60e280 4 Got MDS map 86 2024-09-10T21:50:44.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.399+0000 7f0d4f60e280 10 main: JournalTool::main 2024-09-10T21:50:44.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.399+0000 7f0d4f60e280 4 main: JournalTool: connecting to RADOS... 2024-09-10T21:50:44.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.399+0000 7f0d4f60e280 1 Processor -- start 2024-09-10T21:50:44.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.400+0000 7f0d4f60e280 1 -- start start 2024-09-10T21:50:44.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.400+0000 7f0d4f60e280 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617c6095c00 0x5617c5f87080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.400+0000 7f0d4f60e280 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617c60dc000 0x5617c5f87600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.400+0000 7f0d4f60e280 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c60dc400 0x5617c5f87b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.400+0000 7f0d4f60e280 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5617c5f90600 con 0x5617c60dc400 2024-09-10T21:50:44.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.400+0000 7f0d4f60e280 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5617c5f90780 con 0x5617c6095c00 2024-09-10T21:50:44.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.400+0000 7f0d4f60e280 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5617c5f90900 con 0x5617c60dc000 2024-09-10T21:50:44.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.401+0000 7f0d4c000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617c6095c00 0x5617c5f87080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.401+0000 7f0d4c000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c60dc400 0x5617c5f87b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.401+0000 7f0d4ca00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617c60dc000 0x5617c5f87600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.401+0000 7f0d4c000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c60dc400 0x5617c5f87b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36908/0 (socket says 172.21.15.7:36908) 2024-09-10T21:50:44.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.401+0000 7f0d4c000640 1 -- 172.21.15.7:0/1081191851 learned_addr learned my addr 172.21.15.7:0/1081191851 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:44.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.401+0000 7f0d4c000640 1 -- 172.21.15.7:0/1081191851 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617c60dc000 msgr2=0x5617c5f87600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:44.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.401+0000 7f0d4c000640 1 --2- 172.21.15.7:0/1081191851 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617c60dc000 0x5617c5f87600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.401+0000 7f0d4c000640 1 -- 172.21.15.7:0/1081191851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617c6095c00 msgr2=0x5617c5f87080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:44.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.401+0000 7f0d4c000640 1 --2- 172.21.15.7:0/1081191851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617c6095c00 0x5617c5f87080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.401+0000 7f0d4c000640 1 -- 172.21.15.7:0/1081191851 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617c51e34a0 con 0x5617c60dc400 2024-09-10T21:50:44.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.401+0000 7f0d4c000640 1 --2- 172.21.15.7:0/1081191851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c60dc400 0x5617c5f87b80 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x5617c5fc4f90 tx=0x5617c60f80c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc3c5003e67fbde server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.402+0000 7f0d47000640 1 -- 172.21.15.7:0/1081191851 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5617c51d0ea0 con 0x5617c60dc400 2024-09-10T21:50:44.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.402+0000 7f0d47000640 1 -- 172.21.15.7:0/1081191851 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5617c51d6380 con 0x5617c60dc400 2024-09-10T21:50:44.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.402+0000 7f0d47000640 1 -- 172.21.15.7:0/1081191851 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5617c51d1380 con 0x5617c60dc400 2024-09-10T21:50:44.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.402+0000 7f0d4f60e280 1 -- 172.21.15.7:0/1081191851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c60dc400 msgr2=0x5617c5f87b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.402+0000 7f0d4f60e280 1 --2- 172.21.15.7:0/1081191851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c60dc400 0x5617c5f87b80 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x5617c5fc4f90 tx=0x5617c60f80c0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.402+0000 7f0d4f60e280 1 -- 172.21.15.7:0/1081191851 shutdown_connections 2024-09-10T21:50:44.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.402+0000 7f0d4f60e280 1 --2- 172.21.15.7:0/1081191851 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c60dc400 0x5617c5f87b80 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.402+0000 7f0d4f60e280 1 --2- 172.21.15.7:0/1081191851 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617c60dc000 0x5617c5f87600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.402+0000 7f0d4f60e280 1 --2- 172.21.15.7:0/1081191851 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617c6095c00 0x5617c5f87080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.402+0000 7f0d4f60e280 1 -- 172.21.15.7:0/1081191851 >> 172.21.15.7:0/1081191851 conn(0x5617c6095800 msgr2=0x5617c52bd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.402+0000 7f0d4f60e280 1 -- 172.21.15.7:0/1081191851 shutdown_connections 2024-09-10T21:50:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.402+0000 7f0d4f60e280 1 -- 172.21.15.7:0/1081191851 wait complete. 2024-09-10T21:50:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.403+0000 7f0d4f60e280 1 Processor -- start 2024-09-10T21:50:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.403+0000 7f0d4f60e280 1 -- start start 2024-09-10T21:50:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.404+0000 7f0d4f60e280 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617c60dcc00 0x5617c5f87080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.404+0000 7f0d4f60e280 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617c60dc800 0x5617c5f87600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.404+0000 7f0d4f60e280 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c60dc400 0x5617c5f87b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.404+0000 7f0d4f60e280 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5617c5f91b00 con 0x5617c60dc400 2024-09-10T21:50:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.404+0000 7f0d4f60e280 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5617c5f91980 con 0x5617c60dcc00 2024-09-10T21:50:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.404+0000 7f0d4f60e280 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5617c5f91800 con 0x5617c60dc800 2024-09-10T21:50:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.404+0000 7f0d4c000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c60dc400 0x5617c5f87b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.404+0000 7f0d4c000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c60dc400 0x5617c5f87b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36916/0 (socket says 172.21.15.7:36916) 2024-09-10T21:50:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.404+0000 7f0d4c000640 1 -- 172.21.15.7:0/753820125 learned_addr learned my addr 172.21.15.7:0/753820125 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.404+0000 7f0d4c000640 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617c60dcc00 0x5617c5f87080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.404+0000 7f0d4ca00640 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617c60dc800 0x5617c5f87600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.405+0000 7f0d4c000640 1 -- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617c60dc800 msgr2=0x5617c5f87600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.405+0000 7f0d4c000640 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617c60dc800 0x5617c5f87600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.405+0000 7f0d4c000640 1 -- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617c60dcc00 msgr2=0x5617c5f87080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.405+0000 7f0d4c000640 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617c60dcc00 0x5617c5f87080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.405+0000 7f0d4c000640 1 -- 172.21.15.7:0/753820125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617c51e2780 con 0x5617c60dc400 2024-09-10T21:50:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.405+0000 7f0d4ca00640 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617c60dc800 0x5617c5f87600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:50:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.405+0000 7f0d4c000640 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c60dc400 0x5617c5f87b80 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x5617c5fc5c50 tx=0x5617c60f8240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.405+0000 7f0d45200640 1 -- 172.21.15.7:0/753820125 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5617c51d1040 con 0x5617c60dc400 2024-09-10T21:50:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.405+0000 7f0d45200640 1 -- 172.21.15.7:0/753820125 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5617c51d6fc0 con 0x5617c60dc400 2024-09-10T21:50:44.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.405+0000 7f0d4f60e280 1 -- 172.21.15.7:0/753820125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5617c51e23c0 con 0x5617c60dc400 2024-09-10T21:50:44.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.405+0000 7f0d45200640 1 -- 172.21.15.7:0/753820125 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5617c51d0ea0 con 0x5617c60dc400 2024-09-10T21:50:44.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.405+0000 7f0d4f60e280 1 -- 172.21.15.7:0/753820125 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617c51e2000 con 0x5617c60dc400 2024-09-10T21:50:44.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.405+0000 7f0d4f60e280 4 main: JournalTool: resolving pool 14 2024-09-10T21:50:44.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.407+0000 7f0d45200640 1 -- 172.21.15.7:0/753820125 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x5617c5205500 con 0x5617c60dc400 2024-09-10T21:50:44.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.407+0000 7f0d45200640 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5617c5554000 0x5617c5f88100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.408+0000 7f0d45200640 1 -- 172.21.15.7:0/753820125 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x5617c60fea00 con 0x5617c60dc400 2024-09-10T21:50:44.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.408+0000 7f0d4f60e280 4 main: JournalTool: creating IoCtx.. 2024-09-10T21:50:44.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.408+0000 7f0d4f60e280 4 main: Executing for rank 0 2024-09-10T21:50:44.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.408+0000 7f0d4f60e280 4 Filtering by inode '1099511627777' 2024-09-10T21:50:44.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.408+0000 7f0d4f60e280 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5617c60dc000 0x5617c5f88680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.408+0000 7f0d4f60e280 1 -- 172.21.15.7:0/753820125 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) -- 0x5617c6095c00 con 0x5617c60dc000 2024-09-10T21:50:44.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.408+0000 7f0d4ca00640 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5617c5554000 0x5617c5f88100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.408+0000 7f0d4c000640 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5617c60dc000 0x5617c5f88680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.409+0000 7f0d4ca00640 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5617c5554000 0x5617c5f88100 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x5617c5f581e0 tx=0x5617c60f82a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.409+0000 7f0d4c000640 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5617c60dc000 0x5617c5f88680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.409+0000 7f0d4c000640 1 -- 172.21.15.7:0/753820125 <== osd.7 v2:172.21.15.133:6802/2638399040 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x5617c5f44000 con 0x5617c60dc000 2024-09-10T21:50:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.409+0000 7f0d4f60e280 4 Pointer 400.00000000 is readable 2024-09-10T21:50:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.409+0000 7f0d4f60e280 4 JournalScanner::scan: reading header object '200.00000000' 2024-09-10T21:50:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.409+0000 7f0d4f60e280 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5617c6095800 0x5617c5f88c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.409+0000 7f0d4f60e280 1 -- 172.21.15.7:0/753820125 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) -- 0x5617c5554400 con 0x5617c6095800 2024-09-10T21:50:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.410+0000 7f0d4d400640 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5617c6095800 0x5617c5f88c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.410+0000 7f0d4d400640 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5617c6095800 0x5617c5f88c00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.411+0000 7f0d4d400640 1 -- 172.21.15.7:0/753820125 <== osd.4 v2:172.21.15.133:6826/1857710445 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x5617c5f44500 con 0x5617c6095800 2024-09-10T21:50:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.411+0000 7f0d4f60e280 10 Header 0x400000 0x407d9f 0x40811b 2024-09-10T21:50:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.411+0000 7f0d4f60e280 10 Starting journal scan from offset 0x407d9f 2024-09-10T21:50:44.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.411+0000 7f0d4f60e280 1 -- 172.21.15.7:0/753820125 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 32159~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) -- 0x5617c5554800 con 0x5617c6095800 2024-09-10T21:50:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.412+0000 7f0d4d400640 1 -- 172.21.15.7:0/753820125 <== osd.4 v2:172.21.15.133:6826/1857710445 2 ==== osd_op_reply(3 200.00000001 [read 32159~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x5617c5f44500 con 0x5617c6095800 2024-09-10T21:50:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.412+0000 7f0d4f60e280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-09-10T21:50:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.412+0000 7f0d4f60e280 10 Parsing data, 0x37c bytes available 2024-09-10T21:50:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.412+0000 7f0d4f60e280 10 Attempting decode at 0x407d9f 2024-09-10T21:50:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.412+0000 7f0d4f60e280 10 Consumed 0x37c bytes 2024-09-10T21:50:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.412+0000 7f0d4f60e280 10 Valid entry at 0x407d9f 2024-09-10T21:50:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.412+0000 7f0d4f60e280 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5617c5554c00 0x5617c5f89180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.412+0000 7f0d4f60e280 1 -- 172.21.15.7:0/753820125 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) -- 0x5617c5555000 con 0x5617c5554c00 2024-09-10T21:50:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.412+0000 7f0d4ca00640 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5617c5554c00 0x5617c5f89180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4ca00640 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5617c5554c00 0x5617c5f89180 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4ca00640 1 -- 172.21.15.7:0/753820125 <== osd.1 v2:172.21.15.7:6824/4175601737 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x5617c5f44780 con 0x5617c5554c00 2024-09-10T21:50:44.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 4 Reached end of journal objects 2024-09-10T21:50:44.416 INFO:teuthology.orchestra.run.smithi007.stdout:Events by type: 2024-09-10T21:50:44.416 INFO:teuthology.orchestra.run.smithi007.stdout:Errors: 0 2024-09-10T21:50:44.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 4 Scanned objects, 0 missing, 1 valid 2024-09-10T21:50:44.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 4 Events scanned, 0 gaps 2024-09-10T21:50:44.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 4 Found 1 valid events 2024-09-10T21:50:44.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 4 Selected 0 events events for processing 2024-09-10T21:50:44.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 4 0 events 2024-09-10T21:50:44.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 1 -- 172.21.15.7:0/753820125 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5617c5554c00 msgr2=0x5617c5f89180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5617c5554c00 0x5617c5f89180 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 1 -- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5617c6095800 msgr2=0x5617c5f88c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5617c6095800 0x5617c5f88c00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 1 -- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5617c60dc000 msgr2=0x5617c5f88680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5617c60dc000 0x5617c5f88680 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4ca00640 1 -- 172.21.15.7:0/753820125 reap_dead start 2024-09-10T21:50:44.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 1 -- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5617c5554000 msgr2=0x5617c5f88100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5617c5554000 0x5617c5f88100 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x5617c5f581e0 tx=0x5617c60f82a0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 1 -- 172.21.15.7:0/753820125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c60dc400 msgr2=0x5617c5f87b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c60dc400 0x5617c5f87b80 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x5617c5fc5c50 tx=0x5617c60f8240 comp rx=0 tx=0).stop 2024-09-10T21:50:44.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 1 -- 172.21.15.7:0/753820125 shutdown_connections 2024-09-10T21:50:44.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5617c5554000 0x5617c5f88100 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 1 --2- 172.21.15.7:0/753820125 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c60dc400 0x5617c5f87b80 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 1 -- 172.21.15.7:0/753820125 >> 172.21.15.7:0/753820125 conn(0x5617c60dd000 msgr2=0x5617c52bd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:44.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 1 -- 172.21.15.7:0/753820125 shutdown_connections 2024-09-10T21:50:44.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.413+0000 7f0d4f60e280 1 -- 172.21.15.7:0/753820125 wait complete. 2024-09-10T21:50:44.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.414+0000 7f0d4f60e280 1 -- 172.21.15.7:0/1239665186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c51abc00 msgr2=0x5617c5f86b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.414+0000 7f0d4f60e280 1 --2- 172.21.15.7:0/1239665186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c51abc00 0x5617c5f86b00 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x5617c5fe1b30 tx=0x5617c5f48ba0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.414+0000 7f0d4f60e280 1 -- 172.21.15.7:0/1239665186 shutdown_connections 2024-09-10T21:50:44.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.414+0000 7f0d4f60e280 1 --2- 172.21.15.7:0/1239665186 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5617c51ab800 0x5617c5f86580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.414+0000 7f0d4f60e280 1 --2- 172.21.15.7:0/1239665186 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5617c51abc00 0x5617c5f86b00 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.414+0000 7f0d4f60e280 1 --2- 172.21.15.7:0/1239665186 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5617c51aac00 0x5617c5f86000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.414+0000 7f0d4f60e280 1 -- 172.21.15.7:0/1239665186 >> 172.21.15.7:0/1239665186 conn(0x5617c51ab000 msgr2=0x5617c52bc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:44.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.414+0000 7f0d4f60e280 1 -- 172.21.15.7:0/1239665186 shutdown_connections 2024-09-10T21:50:44.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.414+0000 7f0d4f60e280 1 -- 172.21.15.7:0/1239665186 wait complete. 2024-09-10T21:50:44.420 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627777', 'summary'] in time 0:00:00.113260, result: Events by type: Errors: 0 2024-09-10T21:50:44.421 DEBUG:teuthology.orchestra.run.smithi007:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 event splice --inode=1099511627778 summary 2024-09-10T21:50:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.496+0000 7f715c6ef280 1 -- start start 2024-09-10T21:50:44.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.496+0000 7f715c6ef280 1 Processor -- start 2024-09-10T21:50:44.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.497+0000 7f715c6ef280 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649d60800 0x562649d45080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.497+0000 7f715c6ef280 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562649d61c00 0x562649d44b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.497+0000 7f715c6ef280 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562649d61400 0x562649d44580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.497+0000 7f715c6ef280 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562649d5c780 con 0x562649d60800 2024-09-10T21:50:44.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.497+0000 7f715c6ef280 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562649d5c600 con 0x562649d61c00 2024-09-10T21:50:44.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.497+0000 7f715c6ef280 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x562648fc7980 con 0x562649d61400 2024-09-10T21:50:44.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.497+0000 7f715a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649d60800 0x562649d45080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.497+0000 7f715a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649d60800 0x562649d45080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36938/0 (socket says 172.21.15.7:36938) 2024-09-10T21:50:44.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.497+0000 7f715a800640 1 -- 172.21.15.7:0/852762963 learned_addr learned my addr 172.21.15.7:0/852762963 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:44.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.497+0000 7f7159e00640 1 --2- 172.21.15.7:0/852762963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562649d61c00 0x562649d44b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.497+0000 7f715b200640 1 --2- 172.21.15.7:0/852762963 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562649d61400 0x562649d44580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.498+0000 7f715a800640 1 -- 172.21.15.7:0/852762963 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562649d61400 msgr2=0x562649d44580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.498+0000 7f715a800640 1 --2- 172.21.15.7:0/852762963 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562649d61400 0x562649d44580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.498+0000 7f715a800640 1 -- 172.21.15.7:0/852762963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562649d61c00 msgr2=0x562649d44b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.498+0000 7f715a800640 1 --2- 172.21.15.7:0/852762963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562649d61c00 0x562649d44b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.498+0000 7f715a800640 1 -- 172.21.15.7:0/852762963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562648f9c780 con 0x562649d60800 2024-09-10T21:50:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.498+0000 7f715a800640 1 --2- 172.21.15.7:0/852762963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649d60800 0x562649d45080 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x562649d9ba70 tx=0x562649d02d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253dec9073f4fcc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.498+0000 7f7157600640 1 -- 172.21.15.7:0/852762963 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562648f8b380 con 0x562649d60800 2024-09-10T21:50:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.498+0000 7f7157600640 1 -- 172.21.15.7:0/852762963 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562648f90fc0 con 0x562649d60800 2024-09-10T21:50:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.498+0000 7f7157600640 1 -- 172.21.15.7:0/852762963 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562648f8b040 con 0x562649d60800 2024-09-10T21:50:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.498+0000 7f715c6ef280 1 -- 172.21.15.7:0/852762963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562648f9d860 con 0x562649d60800 2024-09-10T21:50:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.498+0000 7f715c6ef280 1 -- 172.21.15.7:0/852762963 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=0}) -- 0x562648f9da40 con 0x562649d60800 2024-09-10T21:50:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.498+0000 7f715c6ef280 4 waiting for MDS map... 2024-09-10T21:50:44.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.498+0000 7f7157600640 1 -- 172.21.15.7:0/852762963 <== mon.0 v2:172.21.15.7:3300/0 4 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x562649ce0600 con 0x562649d60800 2024-09-10T21:50:44.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.499+0000 7f7157600640 1 -- 172.21.15.7:0/852762963 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 86) ==== 2893+0+0 (secure 0 0 0) 0x562648fbf180 con 0x562649d60800 2024-09-10T21:50:44.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.499+0000 7f715c6ef280 4 Got MDS map 86 2024-09-10T21:50:44.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.499+0000 7f715c6ef280 10 main: JournalTool::main 2024-09-10T21:50:44.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.499+0000 7f715c6ef280 4 main: JournalTool: connecting to RADOS... 2024-09-10T21:50:44.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.499+0000 7f715c6ef280 1 Processor -- start 2024-09-10T21:50:44.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.499+0000 7f715c6ef280 1 -- start start 2024-09-10T21:50:44.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.500+0000 7f715c6ef280 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649d60400 0x562649d45600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.500+0000 7f715c6ef280 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562649e4d800 0x562649d45b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.500+0000 7f715c6ef280 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562649e4dc00 0x562649d46100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.500+0000 7f715c6ef280 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562648fc7380 con 0x562649d60400 2024-09-10T21:50:44.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.500+0000 7f715c6ef280 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562648fc7800 con 0x562649e4d800 2024-09-10T21:50:44.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.500+0000 7f715c6ef280 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x562649d5d200 con 0x562649e4dc00 2024-09-10T21:50:44.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.500+0000 7f715b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649d60400 0x562649d45600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.500+0000 7f715b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649d60400 0x562649d45600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36948/0 (socket says 172.21.15.7:36948) 2024-09-10T21:50:44.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.500+0000 7f715b200640 1 -- 172.21.15.7:0/4085363350 learned_addr learned my addr 172.21.15.7:0/4085363350 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:44.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.501+0000 7f715a800640 1 --2- 172.21.15.7:0/4085363350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562649e4d800 0x562649d45b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.501+0000 7f7159e00640 1 --2- 172.21.15.7:0/4085363350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562649e4dc00 0x562649d46100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.501+0000 7f715b200640 1 -- 172.21.15.7:0/4085363350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562649e4dc00 msgr2=0x562649d46100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:44.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.501+0000 7f715b200640 1 --2- 172.21.15.7:0/4085363350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562649e4dc00 0x562649d46100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.501+0000 7f715b200640 1 -- 172.21.15.7:0/4085363350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562649e4d800 msgr2=0x562649d45b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.501+0000 7f715b200640 1 --2- 172.21.15.7:0/4085363350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562649e4d800 0x562649d45b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.501+0000 7f715b200640 1 -- 172.21.15.7:0/4085363350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562648f9dc20 con 0x562649d60400 2024-09-10T21:50:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.501+0000 7f7159e00640 1 --2- 172.21.15.7:0/4085363350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562649e4dc00 0x562649d46100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:50:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.501+0000 7f715b200640 1 --2- 172.21.15.7:0/4085363350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649d60400 0x562649d45600 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x562649d126f0 tx=0x562649eae000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c521ca28c63a83e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.501+0000 7f7154e00640 1 -- 172.21.15.7:0/4085363350 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562648f8a340 con 0x562649d60400 2024-09-10T21:50:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.501+0000 7f7154e00640 1 -- 172.21.15.7:0/4085363350 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562648f90380 con 0x562649d60400 2024-09-10T21:50:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.501+0000 7f7154e00640 1 -- 172.21.15.7:0/4085363350 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562648f8b1e0 con 0x562649d60400 2024-09-10T21:50:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.502+0000 7f715c6ef280 1 -- 172.21.15.7:0/4085363350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649d60400 msgr2=0x562649d45600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.502+0000 7f715c6ef280 1 --2- 172.21.15.7:0/4085363350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649d60400 0x562649d45600 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x562649d126f0 tx=0x562649eae000 comp rx=0 tx=0).stop 2024-09-10T21:50:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.502+0000 7f715c6ef280 1 -- 172.21.15.7:0/4085363350 shutdown_connections 2024-09-10T21:50:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.502+0000 7f715c6ef280 1 --2- 172.21.15.7:0/4085363350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562649e4d800 0x562649d45b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.502+0000 7f715c6ef280 1 --2- 172.21.15.7:0/4085363350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562649e4dc00 0x562649d46100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.502+0000 7f715c6ef280 1 --2- 172.21.15.7:0/4085363350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649d60400 0x562649d45600 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.502+0000 7f715c6ef280 1 -- 172.21.15.7:0/4085363350 >> 172.21.15.7:0/4085363350 conn(0x562649d61800 msgr2=0x562649077800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:44.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.502+0000 7f715c6ef280 1 -- 172.21.15.7:0/4085363350 shutdown_connections 2024-09-10T21:50:44.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.502+0000 7f715c6ef280 1 -- 172.21.15.7:0/4085363350 wait complete. 2024-09-10T21:50:44.506 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.503+0000 7f715c6ef280 1 Processor -- start 2024-09-10T21:50:44.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.503+0000 7f715c6ef280 1 -- start start 2024-09-10T21:50:44.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.503+0000 7f715c6ef280 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562649d60400 0x562649d45600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.503+0000 7f715c6ef280 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649e4d800 0x562649d45b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.503+0000 7f715c6ef280 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562649e4dc00 0x562649d46100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.503+0000 7f715c6ef280 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562649d5d500 con 0x562649e4d800 2024-09-10T21:50:44.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.503+0000 7f715c6ef280 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562649d5d380 con 0x562649e4dc00 2024-09-10T21:50:44.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.503+0000 7f715c6ef280 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x562649d5d680 con 0x562649d60400 2024-09-10T21:50:44.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.504+0000 7f715a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649e4d800 0x562649d45b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.504+0000 7f715a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649e4d800 0x562649d45b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36952/0 (socket says 172.21.15.7:36952) 2024-09-10T21:50:44.508 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.504+0000 7f715a800640 1 -- 172.21.15.7:0/2300584298 learned_addr learned my addr 172.21.15.7:0/2300584298 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:44.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.504+0000 7f715b200640 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562649d60400 0x562649d45600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.504+0000 7f7159e00640 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562649e4dc00 0x562649d46100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.504+0000 7f715a800640 1 -- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562649d60400 msgr2=0x562649d45600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.504+0000 7f715a800640 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562649d60400 0x562649d45600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.504+0000 7f715a800640 1 -- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562649e4dc00 msgr2=0x562649d46100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.504+0000 7f715a800640 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562649e4dc00 0x562649d46100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.504+0000 7f715a800640 1 -- 172.21.15.7:0/2300584298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562648f9d680 con 0x562649e4d800 2024-09-10T21:50:44.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.504+0000 7f715a800640 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649e4d800 0x562649d45b80 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x562649e63ce0 tx=0x562649d113e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.505+0000 7f7153000640 1 -- 172.21.15.7:0/2300584298 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562648f8a820 con 0x562649e4d800 2024-09-10T21:50:44.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.505+0000 7f7153000640 1 -- 172.21.15.7:0/2300584298 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x562648f90e00 con 0x562649e4d800 2024-09-10T21:50:44.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.505+0000 7f7153000640 1 -- 172.21.15.7:0/2300584298 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x562648f8a4e0 con 0x562649e4d800 2024-09-10T21:50:44.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.505+0000 7f715c6ef280 1 -- 172.21.15.7:0/2300584298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562648f9d4a0 con 0x562649e4d800 2024-09-10T21:50:44.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.505+0000 7f715c6ef280 1 -- 172.21.15.7:0/2300584298 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562648f9c3c0 con 0x562649e4d800 2024-09-10T21:50:44.509 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.505+0000 7f715c6ef280 4 main: JournalTool: resolving pool 14 2024-09-10T21:50:44.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.506+0000 7f7153000640 1 -- 172.21.15.7:0/2300584298 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x562648fbf500 con 0x562649e4d800 2024-09-10T21:50:44.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.507+0000 7f7153000640 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x562649ebb000 0x562649d46680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.507+0000 7f7153000640 1 -- 172.21.15.7:0/2300584298 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x562649eb8a00 con 0x562649e4d800 2024-09-10T21:50:44.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.507+0000 7f715c6ef280 4 main: JournalTool: creating IoCtx.. 2024-09-10T21:50:44.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.507+0000 7f715c6ef280 4 main: Executing for rank 0 2024-09-10T21:50:44.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.507+0000 7f715c6ef280 4 Filtering by inode '1099511627778' 2024-09-10T21:50:44.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.507+0000 7f715c6ef280 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x562649ebb400 0x562649d46c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.510 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.507+0000 7f715c6ef280 1 -- 172.21.15.7:0/2300584298 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:1 14.f 14:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) -- 0x562649ebb800 con 0x562649ebb400 2024-09-10T21:50:44.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.507+0000 7f715b200640 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x562649ebb000 0x562649d46680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.508+0000 7f7159e00640 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x562649ebb400 0x562649d46c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.508+0000 7f715b200640 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x562649ebb000 0x562649d46680 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x562649d126f0 tx=0x5626490521e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.508+0000 7f7159e00640 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x562649ebb400 0x562649d46c00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.508+0000 7f7159e00640 1 -- 172.21.15.7:0/2300584298 <== osd.7 v2:172.21.15.133:6802/2638399040 1 ==== osd_op_reply(1 400.00000000 [read 0~22 out=22b] v0'0 uv1 ondisk = 0) ==== 156+0+22 (crc 0 0 0) 0x562649cfe000 con 0x562649ebb400 2024-09-10T21:50:44.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.509+0000 7f715c6ef280 4 Pointer 400.00000000 is readable 2024-09-10T21:50:44.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.509+0000 7f715c6ef280 4 JournalScanner::scan: reading header object '200.00000000' 2024-09-10T21:50:44.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.509+0000 7f715c6ef280 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x562649ebbc00 0x562649d47180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.509+0000 7f715c6ef280 1 -- 172.21.15.7:0/2300584298 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:2 14.14 14:292cf221:::200.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) -- 0x562649ebb800 con 0x562649ebbc00 2024-09-10T21:50:44.513 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.509+0000 7f715b200640 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x562649ebbc00 0x562649d47180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.509+0000 7f715b200640 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x562649ebbc00 0x562649d47180 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.510+0000 7f715b200640 1 -- 172.21.15.7:0/2300584298 <== osd.4 v2:172.21.15.133:6826/1857710445 1 ==== osd_op_reply(2 200.00000000 [read 0~90 out=90b] v0'0 uv31 ondisk = 0) ==== 156+0+90 (crc 0 0 0) 0x562649cfe500 con 0x562649ebbc00 2024-09-10T21:50:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.510+0000 7f715c6ef280 10 Header 0x400000 0x407d9f 0x40811b 2024-09-10T21:50:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.510+0000 7f715c6ef280 10 Starting journal scan from offset 0x407d9f 2024-09-10T21:50:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.510+0000 7f715c6ef280 1 -- 172.21.15.7:0/2300584298 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:3 14.14 14:2e2fa760:::200.00000001:head [read 32159~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) -- 0x562648f65800 con 0x562649ebbc00 2024-09-10T21:50:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.510+0000 7f715b200640 1 -- 172.21.15.7:0/2300584298 <== osd.4 v2:172.21.15.133:6826/1857710445 2 ==== osd_op_reply(3 200.00000001 [read 32159~892 out=892b] v0'0 uv28 ondisk = 0) ==== 156+0+892 (crc 0 0 0) 0x562649cfe500 con 0x562649ebbc00 2024-09-10T21:50:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.510+0000 7f715c6ef280 4 Read 0x37c bytes from 200.00000001 gap=0 2024-09-10T21:50:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.510+0000 7f715c6ef280 10 Parsing data, 0x37c bytes available 2024-09-10T21:50:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.510+0000 7f715c6ef280 10 Attempting decode at 0x407d9f 2024-09-10T21:50:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.510+0000 7f715c6ef280 10 Consumed 0x37c bytes 2024-09-10T21:50:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.510+0000 7f715c6ef280 10 Valid entry at 0x407d9f 2024-09-10T21:50:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.510+0000 7f715c6ef280 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x56264931c000 0x562649d47700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.510+0000 7f715c6ef280 1 -- 172.21.15.7:0/2300584298 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:4 14.1b 14:ddb4e4d7:::200.00000002:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e102) -- 0x56264931c400 con 0x56264931c000 2024-09-10T21:50:44.514 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.510+0000 7f715a800640 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x56264931c000 0x562649d47700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.511+0000 7f715a800640 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x56264931c000 0x562649d47700 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.515 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.511+0000 7f715a800640 1 -- 172.21.15.7:0/2300584298 <== osd.1 v2:172.21.15.7:6824/4175601737 1 ==== osd_op_reply(4 200.00000002 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x562649cfe780 con 0x56264931c000 2024-09-10T21:50:44.515 INFO:teuthology.orchestra.run.smithi007.stdout:Events by type: 2024-09-10T21:50:44.515 INFO:teuthology.orchestra.run.smithi007.stdout:Errors: 0 2024-09-10T21:50:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.511+0000 7f715c6ef280 4 Reached end of journal objects 2024-09-10T21:50:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.511+0000 7f715c6ef280 4 Scanned objects, 0 missing, 1 valid 2024-09-10T21:50:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.511+0000 7f715c6ef280 4 Events scanned, 0 gaps 2024-09-10T21:50:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.511+0000 7f715c6ef280 4 Found 1 valid events 2024-09-10T21:50:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.511+0000 7f715c6ef280 4 Selected 0 events events for processing 2024-09-10T21:50:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.511+0000 7f715c6ef280 4 0 events 2024-09-10T21:50:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.511+0000 7f715c6ef280 1 -- 172.21.15.7:0/2300584298 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x56264931c000 msgr2=0x562649d47700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.511+0000 7f715c6ef280 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x56264931c000 0x562649d47700 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.511+0000 7f715c6ef280 1 -- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x562649ebbc00 msgr2=0x562649d47180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.511+0000 7f715c6ef280 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x562649ebbc00 0x562649d47180 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.517 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.511+0000 7f715c6ef280 1 -- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x562649ebb400 msgr2=0x562649d46c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.511+0000 7f715c6ef280 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x562649ebb400 0x562649d46c00 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.511+0000 7f7159e00640 1 -- 172.21.15.7:0/2300584298 reap_dead start 2024-09-10T21:50:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.512+0000 7f715c6ef280 1 -- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x562649ebb000 msgr2=0x562649d46680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.512+0000 7f715c6ef280 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x562649ebb000 0x562649d46680 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x562649d126f0 tx=0x5626490521e0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.512+0000 7f715c6ef280 1 -- 172.21.15.7:0/2300584298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649e4d800 msgr2=0x562649d45b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.512+0000 7f715c6ef280 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649e4d800 0x562649d45b80 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x562649e63ce0 tx=0x562649d113e0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.512+0000 7f715c6ef280 1 -- 172.21.15.7:0/2300584298 shutdown_connections 2024-09-10T21:50:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.512+0000 7f715c6ef280 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x562649ebb000 0x562649d46680 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.512+0000 7f715c6ef280 1 --2- 172.21.15.7:0/2300584298 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649e4d800 0x562649d45b80 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.512+0000 7f715c6ef280 1 -- 172.21.15.7:0/2300584298 >> 172.21.15.7:0/2300584298 conn(0x562649d61800 msgr2=0x562649077800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.512+0000 7f715c6ef280 1 -- 172.21.15.7:0/2300584298 shutdown_connections 2024-09-10T21:50:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.512+0000 7f715c6ef280 1 -- 172.21.15.7:0/2300584298 wait complete. 2024-09-10T21:50:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.512+0000 7f715c6ef280 1 -- 172.21.15.7:0/852762963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649d60800 msgr2=0x562649d45080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.518 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.512+0000 7f715c6ef280 1 --2- 172.21.15.7:0/852762963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649d60800 0x562649d45080 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x562649d9ba70 tx=0x562649d02d20 comp rx=0 tx=0).stop 2024-09-10T21:50:44.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.512+0000 7f715c6ef280 1 -- 172.21.15.7:0/852762963 shutdown_connections 2024-09-10T21:50:44.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.512+0000 7f715c6ef280 1 --2- 172.21.15.7:0/852762963 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562649d61c00 0x562649d44b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.512+0000 7f715c6ef280 1 --2- 172.21.15.7:0/852762963 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562649d61400 0x562649d44580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.512+0000 7f715c6ef280 1 --2- 172.21.15.7:0/852762963 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562649d60800 0x562649d45080 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.512+0000 7f715c6ef280 1 -- 172.21.15.7:0/852762963 >> 172.21.15.7:0/852762963 conn(0x562648f65000 msgr2=0x562649076000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:44.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.512+0000 7f715c6ef280 1 -- 172.21.15.7:0/852762963 shutdown_connections 2024-09-10T21:50:44.519 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.512+0000 7f715c6ef280 1 -- 172.21.15.7:0/852762963 wait complete. 2024-09-10T21:50:44.520 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'event', 'splice', '--inode=1099511627778', 'summary'] in time 0:00:00.099296, result: Events by type: Errors: 0 2024-09-10T21:50:44.520 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata put mds0_inotable - 2024-09-10T21:50:44.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.596+0000 7f6619e2ff40 1 Processor -- start 2024-09-10T21:50:44.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.596+0000 7f6619e2ff40 1 -- start start 2024-09-10T21:50:44.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.597+0000 7f6619e2ff40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d33d564cb0 0x55d33d63e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.597+0000 7f6619e2ff40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d33d5b1110 0x55d33d592920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.597+0000 7f6619e2ff40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d33d621620 0x55d33d61d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.597+0000 7f6619e2ff40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d33d4b3c60 con 0x55d33d5b1110 2024-09-10T21:50:44.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.597+0000 7f6619e2ff40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d33d5121d0 con 0x55d33d564cb0 2024-09-10T21:50:44.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.597+0000 7f6619e2ff40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55d33d5b0440 con 0x55d33d621620 2024-09-10T21:50:44.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.597+0000 7f660fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d33d5b1110 0x55d33d592920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.597+0000 7f660fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d33d5b1110 0x55d33d592920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36966/0 (socket says 172.21.15.7:36966) 2024-09-10T21:50:44.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.597+0000 7f660fe00640 1 -- 172.21.15.7:0/1879943886 learned_addr learned my addr 172.21.15.7:0/1879943886 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:44.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.597+0000 7f6617a00640 1 --2- 172.21.15.7:0/1879943886 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d33d621620 0x55d33d61d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.597+0000 7f6617000640 1 --2- 172.21.15.7:0/1879943886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d33d564cb0 0x55d33d63e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.597+0000 7f660fe00640 1 -- 172.21.15.7:0/1879943886 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d33d621620 msgr2=0x55d33d61d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:44.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.597+0000 7f660fe00640 1 --2- 172.21.15.7:0/1879943886 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d33d621620 0x55d33d61d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.597+0000 7f660fe00640 1 -- 172.21.15.7:0/1879943886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d33d564cb0 msgr2=0x55d33d63e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:44.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.597+0000 7f660fe00640 1 --2- 172.21.15.7:0/1879943886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d33d564cb0 0x55d33d63e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.597+0000 7f660fe00640 1 -- 172.21.15.7:0/1879943886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d33d63e9c0 con 0x55d33d5b1110 2024-09-10T21:50:44.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.597+0000 7f6617000640 1 --2- 172.21.15.7:0/1879943886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d33d564cb0 0x55d33d63e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:50:44.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.597+0000 7f660fe00640 1 --2- 172.21.15.7:0/1879943886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d33d5b1110 0x55d33d592920 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f660000c690 tx=0x7f660000cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bdcd345edfe813 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.598+0000 7f6615c00640 1 -- 172.21.15.7:0/1879943886 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660000adc0 con 0x55d33d5b1110 2024-09-10T21:50:44.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.598+0000 7f6615c00640 1 -- 172.21.15.7:0/1879943886 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6600015440 con 0x55d33d5b1110 2024-09-10T21:50:44.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.598+0000 7f6615c00640 1 -- 172.21.15.7:0/1879943886 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6600015890 con 0x55d33d5b1110 2024-09-10T21:50:44.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.598+0000 7f6619e2ff40 1 -- 172.21.15.7:0/1879943886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d33d5b1110 msgr2=0x55d33d592920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.598+0000 7f6619e2ff40 1 --2- 172.21.15.7:0/1879943886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d33d5b1110 0x55d33d592920 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f660000c690 tx=0x7f660000cb50 comp rx=0 tx=0).stop 2024-09-10T21:50:44.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.598+0000 7f6619e2ff40 1 -- 172.21.15.7:0/1879943886 shutdown_connections 2024-09-10T21:50:44.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.598+0000 7f6619e2ff40 1 --2- 172.21.15.7:0/1879943886 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d33d621620 0x55d33d61d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.598+0000 7f6619e2ff40 1 --2- 172.21.15.7:0/1879943886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d33d5b1110 0x55d33d592920 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.598+0000 7f6619e2ff40 1 --2- 172.21.15.7:0/1879943886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d33d564cb0 0x55d33d63e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.598+0000 7f6619e2ff40 1 -- 172.21.15.7:0/1879943886 >> 172.21.15.7:0/1879943886 conn(0x55d33d593de0 msgr2=0x55d33d594ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.598+0000 7f6619e2ff40 1 -- 172.21.15.7:0/1879943886 shutdown_connections 2024-09-10T21:50:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.598+0000 7f6619e2ff40 1 -- 172.21.15.7:0/1879943886 wait complete. 2024-09-10T21:50:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.599+0000 7f6619e2ff40 1 Processor -- start 2024-09-10T21:50:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.599+0000 7f6619e2ff40 1 -- start start 2024-09-10T21:50:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.600+0000 7f6619e2ff40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d33d564cb0 0x55d33d63e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.600+0000 7f6619e2ff40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d33d5b1110 0x55d33d6d5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.600+0000 7f6619e2ff40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d33d621620 0x55d33d61cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.600+0000 7f6619e2ff40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d33d6318e0 con 0x55d33d5b1110 2024-09-10T21:50:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.600+0000 7f6619e2ff40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d33d595920 con 0x55d33d564cb0 2024-09-10T21:50:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.600+0000 7f6619e2ff40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55d33d5955f0 con 0x55d33d621620 2024-09-10T21:50:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.600+0000 7f660fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d33d5b1110 0x55d33d6d5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.600+0000 7f660fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d33d5b1110 0x55d33d6d5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36970/0 (socket says 172.21.15.7:36970) 2024-09-10T21:50:44.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.600+0000 7f660fe00640 1 -- 172.21.15.7:0/286817056 learned_addr learned my addr 172.21.15.7:0/286817056 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:44.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.601+0000 7f6617000640 1 --2- 172.21.15.7:0/286817056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d33d564cb0 0x55d33d63e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.601+0000 7f6617a00640 1 --2- 172.21.15.7:0/286817056 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d33d621620 0x55d33d61cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.601+0000 7f660fe00640 1 -- 172.21.15.7:0/286817056 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d33d621620 msgr2=0x55d33d61cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.601+0000 7f660fe00640 1 --2- 172.21.15.7:0/286817056 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d33d621620 0x55d33d61cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.601+0000 7f660fe00640 1 -- 172.21.15.7:0/286817056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d33d564cb0 msgr2=0x55d33d63e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.601+0000 7f660fe00640 1 --2- 172.21.15.7:0/286817056 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d33d564cb0 0x55d33d63e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.601+0000 7f660fe00640 1 -- 172.21.15.7:0/286817056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d33d6d42b0 con 0x55d33d5b1110 2024-09-10T21:50:44.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.601+0000 7f660fe00640 1 --2- 172.21.15.7:0/286817056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d33d5b1110 0x55d33d6d5350 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f6600011000 tx=0x7f6600015930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.601+0000 7f660f400640 1 -- 172.21.15.7:0/286817056 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6600022020 con 0x55d33d5b1110 2024-09-10T21:50:44.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.601+0000 7f6619e2ff40 1 -- 172.21.15.7:0/286817056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d33d6d4590 con 0x55d33d5b1110 2024-09-10T21:50:44.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.601+0000 7f660f400640 1 -- 172.21.15.7:0/286817056 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6600015d30 con 0x55d33d5b1110 2024-09-10T21:50:44.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.601+0000 7f660f400640 1 -- 172.21.15.7:0/286817056 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66000086a0 con 0x55d33d5b1110 2024-09-10T21:50:44.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.602+0000 7f6619e2ff40 1 -- 172.21.15.7:0/286817056 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d33d6d4b50 con 0x55d33d5b1110 2024-09-10T21:50:44.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.604+0000 7f660f400640 1 -- 172.21.15.7:0/286817056 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f660001d070 con 0x55d33d5b1110 2024-09-10T21:50:44.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.604+0000 7f660f400640 1 --2- 172.21.15.7:0/286817056 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f65f4076980 0x7f65f4078e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.605+0000 7f660f400640 1 -- 172.21.15.7:0/286817056 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f6600099eb0 con 0x55d33d5b1110 2024-09-10T21:50:44.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.605+0000 7f6619e2ff40 1 --2- 172.21.15.7:0/286817056 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x55d33d6c5180 0x55d33d6c5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.605+0000 7f6619e2ff40 1 -- 172.21.15.7:0/286817056 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102) -- 0x55d33d6d2440 con 0x55d33d6c5180 2024-09-10T21:50:44.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.605+0000 7f6617000640 1 --2- 172.21.15.7:0/286817056 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f65f4076980 0x7f65f4078e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.605+0000 7f6617a00640 1 --2- 172.21.15.7:0/286817056 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x55d33d6c5180 0x55d33d6c5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.606+0000 7f6617a00640 1 --2- 172.21.15.7:0/286817056 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x55d33d6c5180 0x55d33d6c5560 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.606+0000 7f6617000640 1 --2- 172.21.15.7:0/286817056 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f65f4076980 0x7f65f4078e40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6608000c00 tx=0x7f6608001720 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.610+0000 7f6617a00640 1 -- 172.21.15.7:0/286817056 <== osd.2 v2:172.21.15.7:6816/4238877366 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v102'5 uv5 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x7f661000b040 con 0x55d33d6c5180 2024-09-10T21:50:44.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.610+0000 7f6619e2ff40 1 -- 172.21.15.7:0/286817056 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x55d33d6c5180 msgr2=0x55d33d6c5560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.610+0000 7f6619e2ff40 1 --2- 172.21.15.7:0/286817056 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x55d33d6c5180 0x55d33d6c5560 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.610+0000 7f6619e2ff40 1 -- 172.21.15.7:0/286817056 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f65f4076980 msgr2=0x7f65f4078e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.610+0000 7f6619e2ff40 1 --2- 172.21.15.7:0/286817056 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f65f4076980 0x7f65f4078e40 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f6608000c00 tx=0x7f6608001720 comp rx=0 tx=0).stop 2024-09-10T21:50:44.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.610+0000 7f6619e2ff40 1 -- 172.21.15.7:0/286817056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d33d5b1110 msgr2=0x55d33d6d5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.610+0000 7f6619e2ff40 1 --2- 172.21.15.7:0/286817056 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d33d5b1110 0x55d33d6d5350 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f6600011000 tx=0x7f6600015930 comp rx=0 tx=0).stop 2024-09-10T21:50:44.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.610+0000 7f6617a00640 1 -- 172.21.15.7:0/286817056 reap_dead start 2024-09-10T21:50:44.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.610+0000 7f6619e2ff40 1 -- 172.21.15.7:0/286817056 shutdown_connections 2024-09-10T21:50:44.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.610+0000 7f6619e2ff40 1 -- 172.21.15.7:0/286817056 >> 172.21.15.7:0/286817056 conn(0x55d33d593de0 msgr2=0x55d33d61fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:44.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.610+0000 7f6619e2ff40 1 -- 172.21.15.7:0/286817056 shutdown_connections 2024-09-10T21:50:44.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.610+0000 7f6619e2ff40 1 -- 172.21.15.7:0/286817056 wait complete. 2024-09-10T21:50:44.616 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:50:44.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.820+0000 7f4f3fe00640 1 -- 172.21.15.7:0/2180668881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f24004800 msgr2=0x7f4f24004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.820+0000 7f4f3fe00640 1 --2- 172.21.15.7:0/2180668881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f24004800 0x7f4f24004c80 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c006f90 tx=0x7f4f24005830 comp rx=0 tx=0).stop 2024-09-10T21:50:44.821 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.820+0000 7f4f3fe00640 1 -- 172.21.15.7:0/2180668881 shutdown_connections 2024-09-10T21:50:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.820+0000 7f4f3fe00640 1 --2- 172.21.15.7:0/2180668881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f24004800 0x7f4f24004c80 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.820+0000 7f4f3fe00640 1 -- 172.21.15.7:0/2180668881 >> 172.21.15.7:0/2180668881 conn(0x7f4f400d3dd0 msgr2=0x7f4f400d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.820+0000 7f4f3fe00640 1 -- 172.21.15.7:0/2180668881 shutdown_connections 2024-09-10T21:50:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.820+0000 7f4f3fe00640 1 -- 172.21.15.7:0/2180668881 wait complete. 2024-09-10T21:50:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.822+0000 7f4f3fe00640 1 Processor -- start 2024-09-10T21:50:44.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.822+0000 7f4f3fe00640 1 -- start start 2024-09-10T21:50:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.823+0000 7f4f3fe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f24004800 0x7f4f40153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.823+0000 7f4f3fe00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f400b2380 0x7f4f400c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.823+0000 7f4f3fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f400c8e90 0x7f4f400b7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.823+0000 7f4f3fe00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4f400d2860 con 0x7f4f400c8e90 2024-09-10T21:50:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.823+0000 7f4f3fe00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4f400d26e0 con 0x7f4f24004800 2024-09-10T21:50:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.823+0000 7f4f3fe00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4f400d29e0 con 0x7f4f400b2380 2024-09-10T21:50:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.823+0000 7f4f3f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f400c8e90 0x7f4f400b7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.823+0000 7f4f3f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f400c8e90 0x7f4f400b7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36982/0 (socket says 172.21.15.7:36982) 2024-09-10T21:50:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.823+0000 7f4f3f400640 1 -- 172.21.15.7:0/4117750355 learned_addr learned my addr 172.21.15.7:0/4117750355 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.823+0000 7f4f3ea00640 1 --2- 172.21.15.7:0/4117750355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f24004800 0x7f4f40153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.823+0000 7f4f3e000640 1 --2- 172.21.15.7:0/4117750355 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f400b2380 0x7f4f400c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.824+0000 7f4f3f400640 1 -- 172.21.15.7:0/4117750355 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f400b2380 msgr2=0x7f4f400c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.824+0000 7f4f3f400640 1 --2- 172.21.15.7:0/4117750355 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f400b2380 0x7f4f400c8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.824+0000 7f4f3f400640 1 -- 172.21.15.7:0/4117750355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f24004800 msgr2=0x7f4f40153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.824+0000 7f4f3f400640 1 --2- 172.21.15.7:0/4117750355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f24004800 0x7f4f40153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:44.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.824+0000 7f4f3f400640 1 -- 172.21.15.7:0/4117750355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f400b2940 con 0x7f4f400c8e90 2024-09-10T21:50:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.824+0000 7f4f3f400640 1 --2- 172.21.15.7:0/4117750355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f400c8e90 0x7f4f400b7950 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f4f38005fd0 tx=0x7f4f38010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.824+0000 7f4f36a00640 1 -- 172.21.15.7:0/4117750355 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3800ed80 con 0x7f4f400c8e90 2024-09-10T21:50:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.824+0000 7f4f36a00640 1 -- 172.21.15.7:0/4117750355 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f38015070 con 0x7f4f400c8e90 2024-09-10T21:50:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.824+0000 7f4f36a00640 1 -- 172.21.15.7:0/4117750355 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f38003bb0 con 0x7f4f400c8e90 2024-09-10T21:50:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.824+0000 7f4f3fe00640 1 -- 172.21.15.7:0/4117750355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f400b6e70 con 0x7f4f400c8e90 2024-09-10T21:50:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.824+0000 7f4f3fe00640 1 -- 172.21.15.7:0/4117750355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f400b7e90 con 0x7f4f400c8e90 2024-09-10T21:50:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.825+0000 7f4f3fe00640 1 -- 172.21.15.7:0/4117750355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f400d2bc0 con 0x7f4f400c8e90 2024-09-10T21:50:44.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.826+0000 7f4f36a00640 1 -- 172.21.15.7:0/4117750355 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4f38024060 con 0x7f4f400c8e90 2024-09-10T21:50:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.827+0000 7f4f36a00640 1 --2- 172.21.15.7:0/4117750355 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4f20077f70 0x7f4f2007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.827+0000 7f4f36a00640 1 -- 172.21.15.7:0/4117750355 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f4f3809a140 con 0x7f4f400c8e90 2024-09-10T21:50:44.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.828+0000 7f4f3ea00640 1 --2- 172.21.15.7:0/4117750355 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4f20077f70 0x7f4f2007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:44.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.828+0000 7f4f3ea00640 1 --2- 172.21.15.7:0/4117750355 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4f20077f70 0x7f4f2007a430 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4f30000c00 tx=0x7f4f30006b10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:44.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:44.831+0000 7f4f36a00640 1 -- 172.21.15.7:0/4117750355 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4f38062580 con 0x7f4f400c8e90 2024-09-10T21:50:45.066 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.066+0000 7f4f3fe00640 1 -- 172.21.15.7:0/4117750355 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7f4f4008ddf0 con 0x7f4f400c8e90 2024-09-10T21:50:45.289 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.288+0000 7f4f36a00640 1 -- 172.21.15.7:0/4117750355 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v87) ==== 162+0+0 (secure 0 0 0) 0x7f4f38066fb0 con 0x7f4f400c8e90 2024-09-10T21:50:45.289 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:50:45.291 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.290+0000 7f4f3fe00640 1 -- 172.21.15.7:0/4117750355 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4f20077f70 msgr2=0x7f4f2007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:45.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.290+0000 7f4f3fe00640 1 --2- 172.21.15.7:0/4117750355 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4f20077f70 0x7f4f2007a430 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f4f30000c00 tx=0x7f4f30006b10 comp rx=0 tx=0).stop 2024-09-10T21:50:45.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.290+0000 7f4f3fe00640 1 -- 172.21.15.7:0/4117750355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f400c8e90 msgr2=0x7f4f400b7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:45.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.290+0000 7f4f3fe00640 1 --2- 172.21.15.7:0/4117750355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f400c8e90 0x7f4f400b7950 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f4f38005fd0 tx=0x7f4f38010e80 comp rx=0 tx=0).stop 2024-09-10T21:50:45.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.290+0000 7f4f3fe00640 1 -- 172.21.15.7:0/4117750355 shutdown_connections 2024-09-10T21:50:45.292 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.290+0000 7f4f3fe00640 1 --2- 172.21.15.7:0/4117750355 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4f20077f70 0x7f4f2007a430 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:45.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.290+0000 7f4f3fe00640 1 --2- 172.21.15.7:0/4117750355 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4f400c8e90 0x7f4f400b7950 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:45.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.290+0000 7f4f3fe00640 1 --2- 172.21.15.7:0/4117750355 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4f400b2380 0x7f4f400c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:45.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.290+0000 7f4f3fe00640 1 --2- 172.21.15.7:0/4117750355 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4f24004800 0x7f4f40153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:45.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.290+0000 7f4f3fe00640 1 -- 172.21.15.7:0/4117750355 >> 172.21.15.7:0/4117750355 conn(0x7f4f400d3dd0 msgr2=0x7f4f400b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:45.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.290+0000 7f4f3fe00640 1 -- 172.21.15.7:0/4117750355 shutdown_connections 2024-09-10T21:50:45.293 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.290+0000 7f4f3fe00640 1 -- 172.21.15.7:0/4117750355 wait complete. 2024-09-10T21:50:45.312 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:50:45.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.472+0000 7fda83600640 1 -- 172.21.15.7:0/640507921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda7c0cd750 msgr2=0x7fda7c073730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.472+0000 7fda83600640 1 --2- 172.21.15.7:0/640507921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda7c0cd750 0x7fda7c073730 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fda64009830 tx=0x7fda6402a3f0 comp rx=0 tx=0).stop 2024-09-10T21:50:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.472+0000 7fda83600640 1 -- 172.21.15.7:0/640507921 shutdown_connections 2024-09-10T21:50:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.472+0000 7fda83600640 1 --2- 172.21.15.7:0/640507921 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda7c0cd750 0x7fda7c073730 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.472+0000 7fda83600640 1 --2- 172.21.15.7:0/640507921 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda7c15bc90 0x7fda7c0ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.472+0000 7fda83600640 1 -- 172.21.15.7:0/640507921 >> 172.21.15.7:0/640507921 conn(0x7fda7c0b16d0 msgr2=0x7fda7c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.472+0000 7fda83600640 1 -- 172.21.15.7:0/640507921 shutdown_connections 2024-09-10T21:50:45.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.472+0000 7fda83600640 1 -- 172.21.15.7:0/640507921 wait complete. 2024-09-10T21:50:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.474+0000 7fda83600640 1 Processor -- start 2024-09-10T21:50:45.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.474+0000 7fda83600640 1 -- start start 2024-09-10T21:50:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.475+0000 7fda83600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda7c073c70 0x7fda7c152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.475+0000 7fda83600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda7c0cd750 0x7fda7c1f53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.475+0000 7fda83600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7c15bc90 0x7fda7c0c89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.475+0000 7fda83600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c06fcf0 con 0x7fda7c15bc90 2024-09-10T21:50:45.477 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.475+0000 7fda83600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c1606c0 con 0x7fda7c073c70 2024-09-10T21:50:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.475+0000 7fda83600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fda7c06fe70 con 0x7fda7c0cd750 2024-09-10T21:50:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.475+0000 7fda81600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7c15bc90 0x7fda7c0c89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.475+0000 7fda81600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7c15bc90 0x7fda7c0c89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:36984/0 (socket says 172.21.15.7:36984) 2024-09-10T21:50:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.475+0000 7fda81600640 1 -- 172.21.15.7:0/3620244782 learned_addr learned my addr 172.21.15.7:0/3620244782 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.475+0000 7fda73e00640 1 --2- 172.21.15.7:0/3620244782 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda7c0cd750 0x7fda7c1f53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.475+0000 7fda80c00640 1 --2- 172.21.15.7:0/3620244782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda7c073c70 0x7fda7c152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.476+0000 7fda81600640 1 -- 172.21.15.7:0/3620244782 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda7c0cd750 msgr2=0x7fda7c1f53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.476+0000 7fda81600640 1 --2- 172.21.15.7:0/3620244782 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda7c0cd750 0x7fda7c1f53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.476+0000 7fda81600640 1 -- 172.21.15.7:0/3620244782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda7c073c70 msgr2=0x7fda7c152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.476+0000 7fda81600640 1 --2- 172.21.15.7:0/3620244782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda7c073c70 0x7fda7c152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.476+0000 7fda81600640 1 -- 172.21.15.7:0/3620244782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7c153520 con 0x7fda7c15bc90 2024-09-10T21:50:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.476+0000 7fda81600640 1 --2- 172.21.15.7:0/3620244782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7c15bc90 0x7fda7c0c89f0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fda7400cae0 tx=0x7fda7400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.476+0000 7fda79600640 1 -- 172.21.15.7:0/3620244782 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74016020 con 0x7fda7c15bc90 2024-09-10T21:50:45.478 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.476+0000 7fda79600640 1 -- 172.21.15.7:0/3620244782 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda7400acd0 con 0x7fda7c15bc90 2024-09-10T21:50:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.476+0000 7fda83600640 1 -- 172.21.15.7:0/3620244782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7c1f5930 con 0x7fda7c15bc90 2024-09-10T21:50:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.476+0000 7fda79600640 1 -- 172.21.15.7:0/3620244782 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda74012680 con 0x7fda7c15bc90 2024-09-10T21:50:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.476+0000 7fda83600640 1 -- 172.21.15.7:0/3620244782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda7c0c72d0 con 0x7fda7c15bc90 2024-09-10T21:50:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.478+0000 7fda83600640 1 -- 172.21.15.7:0/3620244782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda7c073730 con 0x7fda7c15bc90 2024-09-10T21:50:45.479 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.479+0000 7fda79600640 1 -- 172.21.15.7:0/3620244782 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fda74012820 con 0x7fda7c15bc90 2024-09-10T21:50:45.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.480+0000 7fda79600640 1 --2- 172.21.15.7:0/3620244782 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fda58077f70 0x7fda5807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:45.481 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.480+0000 7fda79600640 1 -- 172.21.15.7:0/3620244782 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7fda7409a620 con 0x7fda7c15bc90 2024-09-10T21:50:45.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.481+0000 7fda80c00640 1 --2- 172.21.15.7:0/3620244782 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fda58077f70 0x7fda5807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:45.482 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.482+0000 7fda80c00640 1 --2- 172.21.15.7:0/3620244782 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fda58077f70 0x7fda5807a430 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009b30 tx=0x7fda6c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:45.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.484+0000 7fda79600640 1 -- 172.21.15.7:0/3620244782 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fda74020020 con 0x7fda7c15bc90 2024-09-10T21:50:45.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.713+0000 7fda83600640 1 -- 172.21.15.7:0/3620244782 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda7c160080 con 0x7fda7c15bc90 2024-09-10T21:50:45.715 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.714+0000 7fda79600640 1 -- 172.21.15.7:0/3620244782 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 88 v88) ==== 94+0+5214 (secure 0 0 0) 0x7fda740629d0 con 0x7fda7c15bc90 2024-09-10T21:50:45.715 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:45.715 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":88,"btime":"2024-09-10T21:50:45:291523+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/431253133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":431253133},{"type":"v1","addr":"172.21.15.133:6836","nonce":431253133}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7521,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3692199178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3692199178},{"type":"v1","addr":"172.21.15.133:6837","nonce":3692199178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/752989865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":752989865},{"type":"v1","addr":"172.21.15.7:6834","nonce":752989865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":88,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:49:58.872982+0000","modified":"2024-09-10T21:50:45.291473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7791},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7791":{"gid":7791,"name":"c","rank":0,"incarnation":88,"state":"up:replay","state_seq":1,"addr":"172.21.15.7:6835/4110623302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":4110623302},{"type":"v1","addr":"172.21.15.7:6835","nonce":4110623302}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-10T21:50:45.715 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 88 2024-09-10T21:50:45.717 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.716+0000 7fda83600640 1 -- 172.21.15.7:0/3620244782 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fda58077f70 msgr2=0x7fda5807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:45.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.716+0000 7fda83600640 1 --2- 172.21.15.7:0/3620244782 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fda58077f70 0x7fda5807a430 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fda6c009b30 tx=0x7fda6c005f90 comp rx=0 tx=0).stop 2024-09-10T21:50:45.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.716+0000 7fda83600640 1 -- 172.21.15.7:0/3620244782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7c15bc90 msgr2=0x7fda7c0c89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:45.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.716+0000 7fda83600640 1 --2- 172.21.15.7:0/3620244782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7c15bc90 0x7fda7c0c89f0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fda7400cae0 tx=0x7fda7400a060 comp rx=0 tx=0).stop 2024-09-10T21:50:45.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.716+0000 7fda83600640 1 -- 172.21.15.7:0/3620244782 shutdown_connections 2024-09-10T21:50:45.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.716+0000 7fda83600640 1 --2- 172.21.15.7:0/3620244782 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fda58077f70 0x7fda5807a430 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:45.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.716+0000 7fda83600640 1 --2- 172.21.15.7:0/3620244782 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fda7c15bc90 0x7fda7c0c89f0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:45.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.716+0000 7fda83600640 1 --2- 172.21.15.7:0/3620244782 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fda7c0cd750 0x7fda7c1f53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:45.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.716+0000 7fda83600640 1 --2- 172.21.15.7:0/3620244782 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fda7c073c70 0x7fda7c152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:45.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.716+0000 7fda83600640 1 -- 172.21.15.7:0/3620244782 >> 172.21.15.7:0/3620244782 conn(0x7fda7c0b16d0 msgr2=0x7fda7c0741a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:45.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.717+0000 7fda83600640 1 -- 172.21.15.7:0/3620244782 shutdown_connections 2024-09-10T21:50:45.718 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:45.717+0000 7fda83600640 1 -- 172.21.15.7:0/3620244782 wait complete. 2024-09-10T21:50:46.136 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 88, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:49:58.872982+0000', 'modified': '2024-09-10T21:50:45.291473+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 102, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7791}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7791': {'gid': 7791, 'name': 'c', 'rank': 0, 'incarnation': 88, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.7:6835/4110623302', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6833', 'nonce': 4110623302}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 4110623302}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:50:46.137 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7791:up:replay 2024-09-10T21:50:47.138 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:50:47.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.309+0000 7fa401800640 1 -- 172.21.15.7:0/782302696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3fc146c50 msgr2=0x7fa3fc0723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:47.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.309+0000 7fa401800640 1 --2- 172.21.15.7:0/782302696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3fc146c50 0x7fa3fc0723b0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc070410 tx=0x7fa3ec02a4a0 comp rx=0 tx=0).stop 2024-09-10T21:50:47.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.310+0000 7fa401800640 1 -- 172.21.15.7:0/782302696 shutdown_connections 2024-09-10T21:50:47.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.310+0000 7fa401800640 1 --2- 172.21.15.7:0/782302696 >> v2:172.21.15.133:3300/0 conn(0x7fa3fc1470b0 0x7fa3fc0ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:47.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.310+0000 7fa401800640 1 --2- 172.21.15.7:0/782302696 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3fc146c50 0x7fa3fc0723b0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:47.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.310+0000 7fa401800640 1 -- 172.21.15.7:0/782302696 >> 172.21.15.7:0/782302696 conn(0x7fa3fc0b1d70 msgr2=0x7fa3fc0cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:47.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.310+0000 7fa401800640 1 -- 172.21.15.7:0/782302696 shutdown_connections 2024-09-10T21:50:47.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.310+0000 7fa401800640 1 -- 172.21.15.7:0/782302696 wait complete. 2024-09-10T21:50:47.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.311+0000 7fa401800640 1 Processor -- start 2024-09-10T21:50:47.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.311+0000 7fa401800640 1 -- start start 2024-09-10T21:50:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.312+0000 7fa401800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3fc06f590 0x7fa3fc1f8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.312+0000 7fa401800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3fc146c50 0x7fa3fc1f3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.312+0000 7fa401800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3fc1470b0 0x7fa3fc13fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.312+0000 7fa401800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa3fc0ade20 con 0x7fa3fc146c50 2024-09-10T21:50:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.312+0000 7fa401800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa3fc073210 con 0x7fa3fc06f590 2024-09-10T21:50:47.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.312+0000 7fa401800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa3fc0adfa0 con 0x7fa3fc1470b0 2024-09-10T21:50:47.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.313+0000 7fa3f3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3fc146c50 0x7fa3fc1f3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:47.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.313+0000 7fa3f3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3fc146c50 0x7fa3fc1f3f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37008/0 (socket says 172.21.15.7:37008) 2024-09-10T21:50:47.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.313+0000 7fa3f3400640 1 -- 172.21.15.7:0/186076505 learned_addr learned my addr 172.21.15.7:0/186076505 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:47.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.313+0000 7fa400e00640 1 --2- 172.21.15.7:0/186076505 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3fc1470b0 0x7fa3fc13fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:47.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.313+0000 7fa3fbe00640 1 --2- 172.21.15.7:0/186076505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3fc06f590 0x7fa3fc1f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:47.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.313+0000 7fa3f3400640 1 -- 172.21.15.7:0/186076505 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3fc1470b0 msgr2=0x7fa3fc13fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:47.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.313+0000 7fa3f3400640 1 --2- 172.21.15.7:0/186076505 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3fc1470b0 0x7fa3fc13fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:47.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.313+0000 7fa3f3400640 1 -- 172.21.15.7:0/186076505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3fc06f590 msgr2=0x7fa3fc1f8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:47.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.313+0000 7fa3f3400640 1 --2- 172.21.15.7:0/186076505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3fc06f590 0x7fa3fc1f8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:47.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.313+0000 7fa3f3400640 1 -- 172.21.15.7:0/186076505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3fc1f8b50 con 0x7fa3fc146c50 2024-09-10T21:50:47.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.313+0000 7fa3f3400640 1 --2- 172.21.15.7:0/186076505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3fc146c50 0x7fa3fc1f3f30 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fa3e400c730 tx=0x7fa3e400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:47.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.314+0000 7fa3f8c00640 1 -- 172.21.15.7:0/186076505 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4017070 con 0x7fa3fc146c50 2024-09-10T21:50:47.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.314+0000 7fa3f8c00640 1 -- 172.21.15.7:0/186076505 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3e4005180 con 0x7fa3fc146c50 2024-09-10T21:50:47.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.314+0000 7fa3f8c00640 1 -- 172.21.15.7:0/186076505 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e400a640 con 0x7fa3fc146c50 2024-09-10T21:50:47.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.314+0000 7fa401800640 1 -- 172.21.15.7:0/186076505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3fc13d5f0 con 0x7fa3fc146c50 2024-09-10T21:50:47.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.314+0000 7fa401800640 1 -- 172.21.15.7:0/186076505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3fc1f4b30 con 0x7fa3fc146c50 2024-09-10T21:50:47.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.315+0000 7fa401800640 1 -- 172.21.15.7:0/186076505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3fc0ad8e0 con 0x7fa3fc146c50 2024-09-10T21:50:47.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.316+0000 7fa3f8c00640 1 -- 172.21.15.7:0/186076505 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa3e400a7e0 con 0x7fa3fc146c50 2024-09-10T21:50:47.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.317+0000 7fa3f8c00640 1 --2- 172.21.15.7:0/186076505 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa3d8077f70 0x7fa3d807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:47.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.317+0000 7fa3f8c00640 1 -- 172.21.15.7:0/186076505 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7fa3e40993a0 con 0x7fa3fc146c50 2024-09-10T21:50:47.318 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.318+0000 7fa3fbe00640 1 --2- 172.21.15.7:0/186076505 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa3d8077f70 0x7fa3d807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:47.319 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.319+0000 7fa3fbe00640 1 --2- 172.21.15.7:0/186076505 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa3d8077f70 0x7fa3d807a430 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec034000 tx=0x7fa3ec004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:47.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.322+0000 7fa3f8c00640 1 -- 172.21.15.7:0/186076505 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fa3fc147950 con 0x7fa3fc146c50 2024-09-10T21:50:47.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.557+0000 7fa401800640 1 -- 172.21.15.7:0/186076505 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3fc161570 con 0x7fa3fc146c50 2024-09-10T21:50:47.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.558+0000 7fa3f8c00640 1 -- 172.21.15.7:0/186076505 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5214 (secure 0 0 0) 0x7fa3e40617f0 con 0x7fa3fc146c50 2024-09-10T21:50:47.559 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:47.559 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":90,"btime":"2024-09-10T21:50:47:295782+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/431253133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":431253133},{"type":"v1","addr":"172.21.15.133:6836","nonce":431253133}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7521,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3692199178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3692199178},{"type":"v1","addr":"172.21.15.133:6837","nonce":3692199178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/752989865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":752989865},{"type":"v1","addr":"172.21.15.7:6834","nonce":752989865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:49:58.872982+0000","modified":"2024-09-10T21:50:46.297538+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7791},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7791":{"gid":7791,"name":"c","rank":0,"incarnation":88,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.7:6835/4110623302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":4110623302},{"type":"v1","addr":"172.21.15.7:6835","nonce":4110623302}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-10T21:50:47.559 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 90 2024-09-10T21:50:47.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.560+0000 7fa401800640 1 -- 172.21.15.7:0/186076505 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa3d8077f70 msgr2=0x7fa3d807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:47.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.560+0000 7fa401800640 1 --2- 172.21.15.7:0/186076505 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa3d8077f70 0x7fa3d807a430 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec034000 tx=0x7fa3ec004730 comp rx=0 tx=0).stop 2024-09-10T21:50:47.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.560+0000 7fa401800640 1 -- 172.21.15.7:0/186076505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3fc146c50 msgr2=0x7fa3fc1f3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:47.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.560+0000 7fa401800640 1 --2- 172.21.15.7:0/186076505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3fc146c50 0x7fa3fc1f3f30 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fa3e400c730 tx=0x7fa3e400cbf0 comp rx=0 tx=0).stop 2024-09-10T21:50:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.560+0000 7fa401800640 1 -- 172.21.15.7:0/186076505 shutdown_connections 2024-09-10T21:50:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.560+0000 7fa401800640 1 --2- 172.21.15.7:0/186076505 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa3d8077f70 0x7fa3d807a430 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.560+0000 7fa401800640 1 --2- 172.21.15.7:0/186076505 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3fc1470b0 0x7fa3fc13fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.560+0000 7fa401800640 1 --2- 172.21.15.7:0/186076505 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3fc146c50 0x7fa3fc1f3f30 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.560+0000 7fa401800640 1 --2- 172.21.15.7:0/186076505 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3fc06f590 0x7fa3fc1f8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.560+0000 7fa401800640 1 -- 172.21.15.7:0/186076505 >> 172.21.15.7:0/186076505 conn(0x7fa3fc0b1d70 msgr2=0x7fa3fc06cb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.560+0000 7fa401800640 1 -- 172.21.15.7:0/186076505 shutdown_connections 2024-09-10T21:50:47.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:47.560+0000 7fa401800640 1 -- 172.21.15.7:0/186076505 wait complete. 2024-09-10T21:50:47.980 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:49:58.872982+0000', 'modified': '2024-09-10T21:50:46.297538+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 102, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7791}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7791': {'gid': 7791, 'name': 'c', 'rank': 0, 'incarnation': 88, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.7:6835/4110623302', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6833', 'nonce': 4110623302}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 4110623302}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:50:47.980 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_7791:up:rejoin 2024-09-10T21:50:48.981 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:50:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.152+0000 7ff4e8800640 1 -- 172.21.15.7:0/1907904739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01604b0 msgr2=0x7ff4e00ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.152+0000 7ff4e8800640 1 --2- 172.21.15.7:0/1907904739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01604b0 0x7ff4e00ccef0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc005fd0 tx=0x7ff4dc02aaa0 comp rx=0 tx=0).stop 2024-09-10T21:50:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.152+0000 7ff4e8800640 1 -- 172.21.15.7:0/1907904739 shutdown_connections 2024-09-10T21:50:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.152+0000 7ff4e8800640 1 --2- 172.21.15.7:0/1907904739 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01604b0 0x7ff4e00ccef0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.152+0000 7ff4e8800640 1 -- 172.21.15.7:0/1907904739 >> 172.21.15.7:0/1907904739 conn(0x7ff4e00b16d0 msgr2=0x7ff4e00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.153+0000 7ff4e8800640 1 -- 172.21.15.7:0/1907904739 shutdown_connections 2024-09-10T21:50:49.154 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.153+0000 7ff4e8800640 1 -- 172.21.15.7:0/1907904739 wait complete. 2024-09-10T21:50:49.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.154+0000 7ff4e8800640 1 Processor -- start 2024-09-10T21:50:49.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.154+0000 7ff4e8800640 1 -- start start 2024-09-10T21:50:49.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.155+0000 7ff4e8800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4e00cd750 0x7ff4e01f4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:49.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.155+0000 7ff4e8800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4e015bc90 0x7ff4e00c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:49.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.155+0000 7ff4e8800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01604b0 0x7ff4e00c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:49.156 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.155+0000 7ff4e8800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e0070080 con 0x7ff4e01604b0 2024-09-10T21:50:49.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.155+0000 7ff4e8800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7ff4e006ff00 con 0x7ff4e015bc90 2024-09-10T21:50:49.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.155+0000 7ff4e8800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7ff4e0070200 con 0x7ff4e00cd750 2024-09-10T21:50:49.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.155+0000 7ff4e6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01604b0 0x7ff4e00c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:49.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.155+0000 7ff4e6800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01604b0 0x7ff4e00c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37024/0 (socket says 172.21.15.7:37024) 2024-09-10T21:50:49.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.156+0000 7ff4e6800640 1 -- 172.21.15.7:0/3164258222 learned_addr learned my addr 172.21.15.7:0/3164258222 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:49.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.156+0000 7ff4e5e00640 1 --2- 172.21.15.7:0/3164258222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4e00cd750 0x7ff4e01f4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:49.157 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.156+0000 7ff4e5400640 1 --2- 172.21.15.7:0/3164258222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4e015bc90 0x7ff4e00c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:49.158 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.156+0000 7ff4e6800640 1 -- 172.21.15.7:0/3164258222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4e00cd750 msgr2=0x7ff4e01f4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:49.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.156+0000 7ff4e6800640 1 --2- 172.21.15.7:0/3164258222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4e00cd750 0x7ff4e01f4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:49.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.156+0000 7ff4e6800640 1 -- 172.21.15.7:0/3164258222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4e015bc90 msgr2=0x7ff4e00c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:49.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.156+0000 7ff4e6800640 1 --2- 172.21.15.7:0/3164258222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4e015bc90 0x7ff4e00c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:49.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.156+0000 7ff4e6800640 1 -- 172.21.15.7:0/3164258222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4e01f54a0 con 0x7ff4e01604b0 2024-09-10T21:50:49.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.156+0000 7ff4e6800640 1 --2- 172.21.15.7:0/3164258222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01604b0 0x7ff4e00c50a0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc006010 tx=0x7ff4dc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:49.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.157+0000 7ff4da000640 1 -- 172.21.15.7:0/3164258222 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc044070 con 0x7ff4e01604b0 2024-09-10T21:50:49.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.157+0000 7ff4da000640 1 -- 172.21.15.7:0/3164258222 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4dc004250 con 0x7ff4e01604b0 2024-09-10T21:50:49.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.157+0000 7ff4da000640 1 -- 172.21.15.7:0/3164258222 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc004680 con 0x7ff4e01604b0 2024-09-10T21:50:49.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.157+0000 7ff4e8800640 1 -- 172.21.15.7:0/3164258222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4e00b1380 con 0x7ff4e01604b0 2024-09-10T21:50:49.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.157+0000 7ff4e8800640 1 -- 172.21.15.7:0/3164258222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4e00ca270 con 0x7ff4e01604b0 2024-09-10T21:50:49.159 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.158+0000 7ff4e8800640 1 -- 172.21.15.7:0/3164258222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4e00706b0 con 0x7ff4e01604b0 2024-09-10T21:50:49.160 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.159+0000 7ff4da000640 1 -- 172.21.15.7:0/3164258222 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7ff4dc02e8d0 con 0x7ff4e01604b0 2024-09-10T21:50:49.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.160+0000 7ff4da000640 1 --2- 172.21.15.7:0/3164258222 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff4c4077f70 0x7ff4c407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:49.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.161+0000 7ff4da000640 1 -- 172.21.15.7:0/3164258222 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7ff4dc0baf60 con 0x7ff4e01604b0 2024-09-10T21:50:49.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.161+0000 7ff4e5e00640 1 --2- 172.21.15.7:0/3164258222 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff4c4077f70 0x7ff4c407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:49.162 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.162+0000 7ff4e5e00640 1 --2- 172.21.15.7:0/3164258222 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff4c4077f70 0x7ff4c407a430 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4000f60 tx=0x7ff4d4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:49.166 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.165+0000 7ff4da000640 1 -- 172.21.15.7:0/3164258222 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7ff4e01f54a0 con 0x7ff4e01604b0 2024-09-10T21:50:49.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.396+0000 7ff4e8800640 1 -- 172.21.15.7:0/3164258222 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff4e00b36a0 con 0x7ff4e01604b0 2024-09-10T21:50:49.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.397+0000 7ff4da000640 1 -- 172.21.15.7:0/3164258222 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) ==== 94+0+5221 (secure 0 0 0) 0x7ff4dc032030 con 0x7ff4e01604b0 2024-09-10T21:50:49.398 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:49.398 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":91,"btime":"2024-09-10T21:50:48:303175+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/431253133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":431253133},{"type":"v1","addr":"172.21.15.133:6836","nonce":431253133}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7521,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3692199178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3692199178},{"type":"v1","addr":"172.21.15.133:6837","nonce":3692199178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/752989865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":752989865},{"type":"v1","addr":"172.21.15.7:6834","nonce":752989865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:49:58.872982+0000","modified":"2024-09-10T21:50:48.303169+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":102,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7791},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7791":{"gid":7791,"name":"c","rank":0,"incarnation":88,"state":"up:active","state_seq":4,"addr":"172.21.15.7:6835/4110623302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":4110623302},{"type":"v1","addr":"172.21.15.7:6835","nonce":4110623302}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7791,"qdb_cluster":[7791]},"id":7}]} 2024-09-10T21:50:49.398 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 91 2024-09-10T21:50:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.399+0000 7ff4e8800640 1 -- 172.21.15.7:0/3164258222 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff4c4077f70 msgr2=0x7ff4c407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.399+0000 7ff4e8800640 1 --2- 172.21.15.7:0/3164258222 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff4c4077f70 0x7ff4c407a430 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff4d4000f60 tx=0x7ff4d4009040 comp rx=0 tx=0).stop 2024-09-10T21:50:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.399+0000 7ff4e8800640 1 -- 172.21.15.7:0/3164258222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01604b0 msgr2=0x7ff4e00c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.399+0000 7ff4e8800640 1 --2- 172.21.15.7:0/3164258222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01604b0 0x7ff4e00c50a0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7ff4dc006010 tx=0x7ff4dc003f10 comp rx=0 tx=0).stop 2024-09-10T21:50:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.399+0000 7ff4e8800640 1 -- 172.21.15.7:0/3164258222 shutdown_connections 2024-09-10T21:50:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.399+0000 7ff4e8800640 1 --2- 172.21.15.7:0/3164258222 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7ff4c4077f70 0x7ff4c407a430 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.399+0000 7ff4e8800640 1 --2- 172.21.15.7:0/3164258222 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7ff4e015bc90 0x7ff4e00c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:49.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.399+0000 7ff4e8800640 1 --2- 172.21.15.7:0/3164258222 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7ff4e01604b0 0x7ff4e00c50a0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:49.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.399+0000 7ff4e8800640 1 --2- 172.21.15.7:0/3164258222 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7ff4e00cd750 0x7ff4e01f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:49.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.399+0000 7ff4e8800640 1 -- 172.21.15.7:0/3164258222 >> 172.21.15.7:0/3164258222 conn(0x7ff4e00b16d0 msgr2=0x7ff4e00b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:49.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.399+0000 7ff4e8800640 1 -- 172.21.15.7:0/3164258222 shutdown_connections 2024-09-10T21:50:49.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.399+0000 7ff4e8800640 1 -- 172.21.15.7:0/3164258222 wait complete. 2024-09-10T21:50:49.818 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 91, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:49:58.872982+0000', 'modified': '2024-09-10T21:50:48.303169+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 102, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7791}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7791': {'gid': 7791, 'name': 'c', 'rank': 0, 'incarnation': 88, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.7:6835/4110623302', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.7:6833', 'nonce': 4110623302}, {'type': 'v1', 'addr': '172.21.15.7:6835', 'nonce': 4110623302}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7791, 'qdb_cluster': [7791]} 2024-09-10T21:50:49.819 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:50:49.819 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-10T21:50:49.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.983+0000 7f1edb200640 1 -- 172.21.15.7:0/1543263182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ebc004800 msgr2=0x7f1ebc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:49.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.983+0000 7f1edb200640 1 --2- 172.21.15.7:0/1543263182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ebc004800 0x7f1ebc004c80 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40016c0 tx=0x7f1ebc005830 comp rx=0 tx=0).stop 2024-09-10T21:50:49.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.983+0000 7f1edb200640 1 -- 172.21.15.7:0/1543263182 shutdown_connections 2024-09-10T21:50:49.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.983+0000 7f1edb200640 1 --2- 172.21.15.7:0/1543263182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ebc004800 0x7f1ebc004c80 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:49.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.983+0000 7f1edb200640 1 -- 172.21.15.7:0/1543263182 >> 172.21.15.7:0/1543263182 conn(0x7f1ed40b16d0 msgr2=0x7f1ed40b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:49.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.983+0000 7f1edb200640 1 -- 172.21.15.7:0/1543263182 shutdown_connections 2024-09-10T21:50:49.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.983+0000 7f1edb200640 1 -- 172.21.15.7:0/1543263182 wait complete. 2024-09-10T21:50:49.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.985+0000 7f1edb200640 1 Processor -- start 2024-09-10T21:50:49.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.985+0000 7f1edb200640 1 -- start start 2024-09-10T21:50:49.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.986+0000 7f1edb200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ebc004800 0x7f1ed41e78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:49.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.986+0000 7f1edb200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed413e140 0x7f1ed41f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:49.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.986+0000 7f1edb200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed41f6200 0x7f1ed41f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:49.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.986+0000 7f1edb200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed415f8b0 con 0x7f1ed413e140 2024-09-10T21:50:49.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.986+0000 7f1edb200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed4073380 con 0x7f1ed41f6200 2024-09-10T21:50:49.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.986+0000 7f1edb200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed415fa30 con 0x7f1ebc004800 2024-09-10T21:50:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.986+0000 7f1ed9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed413e140 0x7f1ed41f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.986+0000 7f1ed9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed413e140 0x7f1ed41f5cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37042/0 (socket says 172.21.15.7:37042) 2024-09-10T21:50:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.986+0000 7f1ed9400640 1 -- 172.21.15.7:0/3387880829 learned_addr learned my addr 172.21.15.7:0/3387880829 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.986+0000 7f1ed9e00640 1 --2- 172.21.15.7:0/3387880829 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ebc004800 0x7f1ed41e78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.986+0000 7f1eda800640 1 --2- 172.21.15.7:0/3387880829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed41f6200 0x7f1ed41f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.986+0000 7f1ed9400640 1 -- 172.21.15.7:0/3387880829 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ebc004800 msgr2=0x7f1ed41e78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.986+0000 7f1ed9400640 1 --2- 172.21.15.7:0/3387880829 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ebc004800 0x7f1ed41e78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.986+0000 7f1ed9400640 1 -- 172.21.15.7:0/3387880829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed41f6200 msgr2=0x7f1ed41f6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.986+0000 7f1ed9400640 1 --2- 172.21.15.7:0/3387880829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed41f6200 0x7f1ed41f6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.986+0000 7f1ed9400640 1 -- 172.21.15.7:0/3387880829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ed41e8000 con 0x7f1ed413e140 2024-09-10T21:50:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.986+0000 7f1eda800640 1 --2- 172.21.15.7:0/3387880829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed41f6200 0x7f1ed41f6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:50:49.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.987+0000 7f1ed9400640 1 --2- 172.21.15.7:0/3387880829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed413e140 0x7f1ed41f5cc0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f1ec402a8e0 tx=0x7f1ec40063a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.987+0000 7f1ece000640 1 -- 172.21.15.7:0/3387880829 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4003a00 con 0x7f1ed413e140 2024-09-10T21:50:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.987+0000 7f1ece000640 1 -- 172.21.15.7:0/3387880829 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ec4039070 con 0x7f1ed413e140 2024-09-10T21:50:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.987+0000 7f1ece000640 1 -- 172.21.15.7:0/3387880829 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4033420 con 0x7f1ed413e140 2024-09-10T21:50:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.987+0000 7f1edb200640 1 -- 172.21.15.7:0/3387880829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ed41e82e0 con 0x7f1ed413e140 2024-09-10T21:50:49.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.987+0000 7f1edb200640 1 -- 172.21.15.7:0/3387880829 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ed413e5c0 con 0x7f1ed413e140 2024-09-10T21:50:49.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.989+0000 7f1ece000640 1 -- 172.21.15.7:0/3387880829 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1ec40335c0 con 0x7f1ed413e140 2024-09-10T21:50:49.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.991+0000 7f1ece000640 1 --2- 172.21.15.7:0/3387880829 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1eb8077f70 0x7f1eb807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:49.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.991+0000 7f1ece000640 1 -- 172.21.15.7:0/3387880829 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f1ec40b97b0 con 0x7f1ed413e140 2024-09-10T21:50:49.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.991+0000 7f1edf885740 1 Processor -- start 2024-09-10T21:50:49.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.991+0000 7f1ed9e00640 1 --2- 172.21.15.7:0/3387880829 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1eb8077f70 0x7f1eb807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:49.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.992+0000 7f1edf885740 1 -- start start 2024-09-10T21:50:49.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.992+0000 7f1ed9e00640 1 --2- 172.21.15.7:0/3387880829 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1eb8077f70 0x7f1eb807a430 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8006bf0 tx=0x7f1ec80067a0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:49.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.992+0000 7f1edf885740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afe5aa0 0x55673afe5e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:49.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.992+0000 7f1edf885740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afa0c10 0x55673afe4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:49.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.992+0000 7f1edf885740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afa1190 0x55673afe4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:49.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.992+0000 7f1edf885740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55673af23430 con 0x55673afa1190 2024-09-10T21:50:49.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.992+0000 7f1edf885740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55673af135d0 con 0x55673afa0c10 2024-09-10T21:50:49.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.992+0000 7f1edf885740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55673af02a00 con 0x55673afe5aa0 2024-09-10T21:50:49.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.993+0000 7f1eda800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afa1190 0x55673afe4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:49.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.993+0000 7f1ed9e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afe5aa0 0x55673afe5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:49.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.993+0000 7f1eda800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afa1190 0x55673afe4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37056/0 (socket says 172.21.15.7:37056) 2024-09-10T21:50:49.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.993+0000 7f1eda800640 1 -- 172.21.15.7:0/2660900862 learned_addr learned my addr 172.21.15.7:0/2660900862 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:49.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.993+0000 7f1ed9400640 1 --2- 172.21.15.7:0/2660900862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afa0c10 0x55673afe4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:49.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.993+0000 7f1eda800640 1 -- 172.21.15.7:0/2660900862 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afe5aa0 msgr2=0x55673afe5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:49.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.993+0000 7f1eda800640 1 --2- 172.21.15.7:0/2660900862 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afe5aa0 0x55673afe5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:49.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.993+0000 7f1eda800640 1 -- 172.21.15.7:0/2660900862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afa0c10 msgr2=0x55673afe4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:49.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.993+0000 7f1eda800640 1 --2- 172.21.15.7:0/2660900862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afa0c10 0x55673afe4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:49.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.993+0000 7f1eda800640 1 -- 172.21.15.7:0/2660900862 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55673aecb520 con 0x55673afa1190 2024-09-10T21:50:49.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.993+0000 7f1eda800640 1 --2- 172.21.15.7:0/2660900862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afa1190 0x55673afe4ac0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40cd2a0 tx=0x7f1ed0012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6448412ff64016fc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:49.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.994+0000 7f1ec2600640 1 -- 172.21.15.7:0/2660900862 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0012070 con 0x55673afa1190 2024-09-10T21:50:49.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.994+0000 7f1ec2600640 1 -- 172.21.15.7:0/2660900862 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ed0012210 con 0x55673afa1190 2024-09-10T21:50:49.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.994+0000 7f1ec2600640 1 -- 172.21.15.7:0/2660900862 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0003b90 con 0x55673afa1190 2024-09-10T21:50:49.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.994+0000 7f1edf885740 1 -- 172.21.15.7:0/2660900862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afa1190 msgr2=0x55673afe4ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:49.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.994+0000 7f1edf885740 1 --2- 172.21.15.7:0/2660900862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afa1190 0x55673afe4ac0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f1ed40cd2a0 tx=0x7f1ed0012820 comp rx=0 tx=0).stop 2024-09-10T21:50:49.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.994+0000 7f1edf885740 1 -- 172.21.15.7:0/2660900862 shutdown_connections 2024-09-10T21:50:49.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.994+0000 7f1edf885740 1 --2- 172.21.15.7:0/2660900862 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afa0c10 0x55673afe4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:49.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.994+0000 7f1edf885740 1 --2- 172.21.15.7:0/2660900862 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afa1190 0x55673afe4ac0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:49.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.994+0000 7f1edf885740 1 --2- 172.21.15.7:0/2660900862 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afe5aa0 0x55673afe5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:49.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.994+0000 7f1edf885740 1 -- 172.21.15.7:0/2660900862 >> 172.21.15.7:0/2660900862 conn(0x55673af9c1e0 msgr2=0x55673af9fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:49.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.995+0000 7f1edf885740 1 -- 172.21.15.7:0/2660900862 shutdown_connections 2024-09-10T21:50:49.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.995+0000 7f1edf885740 1 -- 172.21.15.7:0/2660900862 wait complete. 2024-09-10T21:50:49.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.995+0000 7f1edf885740 1 -- start start 2024-09-10T21:50:49.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.995+0000 7f1edf885740 1 Processor -- start 2024-09-10T21:50:50.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.996+0000 7f1edf885740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afa1190 0x55673afc8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.996+0000 7f1edf885740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afe5aa0 0x55673aff8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.996+0000 7f1edf885740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afc9220 0x55673afb8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.996+0000 7f1edf885740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55673aea9330 con 0x55673afc9220 2024-09-10T21:50:50.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.996+0000 7f1edf885740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55673af06810 con 0x55673afe5aa0 2024-09-10T21:50:50.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.996+0000 7f1edf885740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55673ac73620 con 0x55673afa1190 2024-09-10T21:50:50.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.997+0000 7f1edf885740 10 client.0 _ll_register_callbacks cb 0x55673aeae420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:50:50.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.997+0000 7f1eda800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afc9220 0x55673afb8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.997+0000 7f1ed9e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afa1190 0x55673afc8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.997+0000 7f1eda800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afc9220 0x55673afb8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37072/0 (socket says 172.21.15.7:37072) 2024-09-10T21:50:50.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.997+0000 7f1eda800640 1 -- 172.21.15.7:0/3445634132 learned_addr learned my addr 172.21.15.7:0/3445634132 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:50.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.997+0000 7f1ed9400640 1 --2- 172.21.15.7:0/3445634132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afe5aa0 0x55673aff8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.997+0000 7f1eda800640 1 -- 172.21.15.7:0/3445634132 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afa1190 msgr2=0x55673afc8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.997+0000 7f1eda800640 1 --2- 172.21.15.7:0/3445634132 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afa1190 0x55673afc8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.997+0000 7f1eda800640 1 -- 172.21.15.7:0/3445634132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afe5aa0 msgr2=0x55673aff8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.997+0000 7f1eda800640 1 --2- 172.21.15.7:0/3445634132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afe5aa0 0x55673aff8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.997+0000 7f1eda800640 1 -- 172.21.15.7:0/3445634132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55673ac64850 con 0x55673afc9220 2024-09-10T21:50:50.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.997+0000 7f1eda800640 1 --2- 172.21.15.7:0/3445634132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afc9220 0x55673afb8aa0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x55673afe7120 tx=0x7f1ed0010ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d73bf37b64f00177 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:50.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.997+0000 7f1eb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:50:50.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.998+0000 7f1eb6a00640 1 -- 172.21.15.7:0/3445634132 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0003970 con 0x55673afc9220 2024-09-10T21:50:50.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.998+0000 7f1eb6a00640 1 -- 172.21.15.7:0/3445634132 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f1ed0003b10 con 0x55673afc9220 2024-09-10T21:50:50.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.998+0000 7f1edf885740 1 -- 172.21.15.7:0/3445634132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ed0012210 con 0x55673afc9220 2024-09-10T21:50:50.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.998+0000 7f1eb6a00640 1 -- 172.21.15.7:0/3445634132 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f1ed002c050 con 0x55673afc9220 2024-09-10T21:50:50.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.998+0000 7f1edf885740 10 client.7872 fetch_fsmap learned FSMap version 91 2024-09-10T21:50:50.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.998+0000 7f1edf885740 1 -- 172.21.15.7:0/3445634132 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55673acdb6b0 con 0x55673afc9220 2024-09-10T21:50:50.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.998+0000 7f1eb6a00640 1 -- 172.21.15.7:0/3445634132 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ed0012210 con 0x55673afc9220 2024-09-10T21:50:50.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.998+0000 7f1eb6a00640 1 -- 172.21.15.7:0/3445634132 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 91) ==== 3502+0+0 (secure 0 0 0) 0x7f1ed002f020 con 0x55673afc9220 2024-09-10T21:50:50.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.998+0000 7f1edf885740 10 client.7872 fetch_fsmap finished waiting for FSMap version 91 2024-09-10T21:50:50.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.998+0000 7f1edf885740 10 client.7872 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:50:50.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.998+0000 7f1edf885740 20 client.7872 populate_metadata read hostname 'smithi007' 2024-09-10T21:50:50.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.998+0000 7f1edf885740 1 --2- 172.21.15.7:0/3445634132 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55673afcc0b0 0x55673afbf0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.998+0000 7f1edf885740 4 client.7872 mds_command: new command op to 7791 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:50:50.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.998+0000 7f1edf885740 1 -- 172.21.15.7:0/3445634132 --> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55673af15520 con 0x55673afcc0b0 2024-09-10T21:50:50.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.999+0000 7f1ed9e00640 1 --2- 172.21.15.7:0/3445634132 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55673afcc0b0 0x55673afbf0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.999+0000 7f1ed9e00640 1 --2- 172.21.15.7:0/3445634132 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55673afcc0b0 0x55673afbf0b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9eb30528fb65e72e server_cookie=763857aa8127584b in_seq=0 out_seq=0 2024-09-10T21:50:50.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:49.999+0000 7f1eb6a00640 10 client.7872 ms_handle_connect on v2:172.21.15.7:6833/4110623302 2024-09-10T21:50:50.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.000+0000 7f1eb6a00640 1 -- 172.21.15.7:0/3445634132 <== mds.0 v2:172.21.15.7:6833/4110623302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55673af15520 con 0x55673afcc0b0 2024-09-10T21:50:50.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.000+0000 7f1eb6a00640 10 client.7872 handle_command_reply: tid=0 2024-09-10T21:50:50.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.000+0000 7f1edf885740 1 client.7872 shutdown 2024-09-10T21:50:50.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.000+0000 7f1edf885740 1 -- 172.21.15.7:0/3445634132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afc9220 msgr2=0x55673afb8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.000+0000 7f1edf885740 1 --2- 172.21.15.7:0/3445634132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afc9220 0x55673afb8aa0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x55673afe7120 tx=0x7f1ed0010ad0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.001+0000 7f1edf885740 1 -- 172.21.15.7:0/3445634132 shutdown_connections 2024-09-10T21:50:50.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.001+0000 7f1edf885740 1 --2- 172.21.15.7:0/3445634132 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55673afcc0b0 0x55673afbf0b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.001+0000 7f1edf885740 1 --2- 172.21.15.7:0/3445634132 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afe5aa0 0x55673aff8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.001+0000 7f1edf885740 1 --2- 172.21.15.7:0/3445634132 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afc9220 0x55673afb8aa0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.001+0000 7f1edf885740 1 --2- 172.21.15.7:0/3445634132 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afa1190 0x55673afc8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.001+0000 7f1edf885740 1 -- 172.21.15.7:0/3445634132 >> 172.21.15.7:0/3445634132 conn(0x55673afa0c10 msgr2=0x55673af9adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:50.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.001+0000 7f1eb6a00640 0 client.7872 ms_handle_reset on v2:172.21.15.7:6833/4110623302 2024-09-10T21:50:50.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.001+0000 7f1edf885740 1 -- 172.21.15.7:0/3445634132 shutdown_connections 2024-09-10T21:50:50.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.001+0000 7f1edf885740 1 -- 172.21.15.7:0/3445634132 wait complete. 2024-09-10T21:50:50.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.001+0000 7f1edf885740 20 client.7872 trim_cache size 0 max 16384 2024-09-10T21:50:50.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.020+0000 7f1edf885740 1 Processor -- start 2024-09-10T21:50:50.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.020+0000 7f1edf885740 1 -- start start 2024-09-10T21:50:50.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.021+0000 7f1edf885740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afa1190 0x55673af9f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.021+0000 7f1edf885740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afc9220 0x55673af9fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.021+0000 7f1edf885740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afe5aa0 0x55673af9ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.021+0000 7f1edf885740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed40b4330 con 0x55673afe5aa0 2024-09-10T21:50:50.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.021+0000 7f1edf885740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1ed413fe60 con 0x55673afa1190 2024-09-10T21:50:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.021+0000 7f1edf885740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f1ed40b3ce0 con 0x55673afc9220 2024-09-10T21:50:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.021+0000 7f1eda800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afe5aa0 0x55673af9ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.021+0000 7f1eda800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afe5aa0 0x55673af9ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37088/0 (socket says 172.21.15.7:37088) 2024-09-10T21:50:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.021+0000 7f1eda800640 1 -- 172.21.15.7:0/974189558 learned_addr learned my addr 172.21.15.7:0/974189558 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.022+0000 7f1ed9400640 1 --2- 172.21.15.7:0/974189558 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afc9220 0x55673af9fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.022+0000 7f1ed9e00640 1 --2- 172.21.15.7:0/974189558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afa1190 0x55673af9f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.022+0000 7f1eda800640 1 -- 172.21.15.7:0/974189558 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afc9220 msgr2=0x55673af9fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.022+0000 7f1eda800640 1 --2- 172.21.15.7:0/974189558 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afc9220 0x55673af9fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.022+0000 7f1eda800640 1 -- 172.21.15.7:0/974189558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afa1190 msgr2=0x55673af9f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.022+0000 7f1eda800640 1 --2- 172.21.15.7:0/974189558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afa1190 0x55673af9f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.022+0000 7f1eda800640 1 -- 172.21.15.7:0/974189558 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55673af96180 con 0x55673afe5aa0 2024-09-10T21:50:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.022+0000 7f1ed9e00640 1 --2- 172.21.15.7:0/974189558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afa1190 0x55673af9f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:50:50.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.022+0000 7f1eda800640 1 --2- 172.21.15.7:0/974189558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afe5aa0 0x55673af9ad20 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0009ee0 tx=0x7f1ed0012340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66554d573b3d915c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:50.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.022+0000 7f1ec1c00640 1 -- 172.21.15.7:0/974189558 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0035070 con 0x55673afe5aa0 2024-09-10T21:50:50.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.022+0000 7f1ec1c00640 1 -- 172.21.15.7:0/974189558 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ed0020030 con 0x55673afe5aa0 2024-09-10T21:50:50.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.022+0000 7f1ec1c00640 1 -- 172.21.15.7:0/974189558 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0003bd0 con 0x55673afe5aa0 2024-09-10T21:50:50.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.023+0000 7f1edf885740 1 -- 172.21.15.7:0/974189558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afe5aa0 msgr2=0x55673af9ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.023+0000 7f1edf885740 1 --2- 172.21.15.7:0/974189558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afe5aa0 0x55673af9ad20 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0009ee0 tx=0x7f1ed0012340 comp rx=0 tx=0).stop 2024-09-10T21:50:50.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.023+0000 7f1edf885740 1 -- 172.21.15.7:0/974189558 shutdown_connections 2024-09-10T21:50:50.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.023+0000 7f1edf885740 1 --2- 172.21.15.7:0/974189558 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afe5aa0 0x55673af9ad20 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.023+0000 7f1edf885740 1 --2- 172.21.15.7:0/974189558 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afc9220 0x55673af9fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.023+0000 7f1edf885740 1 --2- 172.21.15.7:0/974189558 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afa1190 0x55673af9f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.023+0000 7f1edf885740 1 -- 172.21.15.7:0/974189558 >> 172.21.15.7:0/974189558 conn(0x55673afa0c10 msgr2=0x55673af990f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.023+0000 7f1edf885740 1 -- 172.21.15.7:0/974189558 shutdown_connections 2024-09-10T21:50:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.023+0000 7f1edf885740 1 -- 172.21.15.7:0/974189558 wait complete. 2024-09-10T21:50:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.024+0000 7f1edf885740 1 -- start start 2024-09-10T21:50:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.024+0000 7f1edf885740 1 Processor -- start 2024-09-10T21:50:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.025+0000 7f1edf885740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afa1190 0x55673b00e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.025+0000 7f1edf885740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afc9220 0x55673af9af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.025+0000 7f1edf885740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afe5aa0 0x55673b06c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.025+0000 7f1edf885740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55673afcbf30 con 0x55673afc9220 2024-09-10T21:50:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.025+0000 7f1edf885740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55673afb9da0 con 0x55673afe5aa0 2024-09-10T21:50:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.025+0000 7f1edf885740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55673afcc550 con 0x55673afa1190 2024-09-10T21:50:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.025+0000 7f1edf885740 10 client.0 _ll_register_callbacks cb 0x7f1ed0010760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:50:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.025+0000 7f1ed9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afc9220 0x55673af9af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.025+0000 7f1ed9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afc9220 0x55673af9af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37090/0 (socket says 172.21.15.7:37090) 2024-09-10T21:50:50.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.025+0000 7f1ed9400640 1 -- 172.21.15.7:0/2257389954 learned_addr learned my addr 172.21.15.7:0/2257389954 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.025+0000 7f1ed9e00640 1 --2- 172.21.15.7:0/2257389954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afa1190 0x55673b00e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.025+0000 7f1eda800640 1 --2- 172.21.15.7:0/2257389954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afe5aa0 0x55673b06c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.026+0000 7f1ed9400640 1 -- 172.21.15.7:0/2257389954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afa1190 msgr2=0x55673b00e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.026+0000 7f1ed9400640 1 --2- 172.21.15.7:0/2257389954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afa1190 0x55673b00e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.026+0000 7f1ed9400640 1 -- 172.21.15.7:0/2257389954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afe5aa0 msgr2=0x55673b06c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.026+0000 7f1ed9400640 1 --2- 172.21.15.7:0/2257389954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afe5aa0 0x55673b06c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.026+0000 7f1ed9400640 1 -- 172.21.15.7:0/2257389954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55673afbc680 con 0x55673afc9220 2024-09-10T21:50:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.026+0000 7f1ed9e00640 1 --2- 172.21.15.7:0/2257389954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afa1190 0x55673b00e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:50:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.026+0000 7f1ed9400640 1 --2- 172.21.15.7:0/2257389954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afc9220 0x55673af9af10 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40016c0 tx=0x7f1ec40814c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14a3863da528e04 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.026+0000 7f1eb6a00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:50:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.026+0000 7f1eb6a00640 1 -- 172.21.15.7:0/2257389954 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ec40b5060 con 0x55673afc9220 2024-09-10T21:50:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.026+0000 7f1eb6a00640 1 -- 172.21.15.7:0/2257389954 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f1ec4081f10 con 0x55673afc9220 2024-09-10T21:50:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.026+0000 7f1edf885740 1 -- 172.21.15.7:0/2257389954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ed0020030 con 0x55673afc9220 2024-09-10T21:50:50.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.026+0000 7f1eb6a00640 1 -- 172.21.15.7:0/2257389954 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f1ec409b030 con 0x55673afc9220 2024-09-10T21:50:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.026+0000 7f1edf885740 10 client.7878 fetch_fsmap learned FSMap version 91 2024-09-10T21:50:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.026+0000 7f1edf885740 1 -- 172.21.15.7:0/2257389954 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55673af9b450 con 0x55673afc9220 2024-09-10T21:50:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.027+0000 7f1eb6a00640 1 -- 172.21.15.7:0/2257389954 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ed0020030 con 0x55673afc9220 2024-09-10T21:50:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.027+0000 7f1eb6a00640 1 -- 172.21.15.7:0/2257389954 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 91) ==== 3502+0+0 (secure 0 0 0) 0x7f1ec408ab20 con 0x55673afc9220 2024-09-10T21:50:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.027+0000 7f1edf885740 10 client.7878 fetch_fsmap finished waiting for FSMap version 91 2024-09-10T21:50:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.027+0000 7f1edf885740 10 client.7878 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-10T21:50:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.027+0000 7f1edf885740 20 client.7878 populate_metadata read hostname 'smithi007' 2024-09-10T21:50:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.027+0000 7f1edf885740 1 --2- 172.21.15.7:0/2257389954 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55673b06d6f0 0x55673afb7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.027+0000 7f1edf885740 4 client.7878 mds_command: new command op to 7791 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:50:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.027+0000 7f1edf885740 1 -- 172.21.15.7:0/2257389954 --> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] -- command(tid 0: {"prefix": "status"}) -- 0x55673af10fd0 con 0x55673b06d6f0 2024-09-10T21:50:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.027+0000 7f1ed9e00640 1 --2- 172.21.15.7:0/2257389954 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55673b06d6f0 0x55673afb7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.027+0000 7f1ed9e00640 1 --2- 172.21.15.7:0/2257389954 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55673b06d6f0 0x55673afb7620 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7f2c6d4c7eef063 server_cookie=ca620a762bb4c831 in_seq=0 out_seq=0 2024-09-10T21:50:50.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.027+0000 7f1eb6a00640 10 client.7878 ms_handle_connect on v2:172.21.15.7:6833/4110623302 2024-09-10T21:50:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.028+0000 7f1eb6a00640 1 -- 172.21.15.7:0/2257389954 <== mds.0 v2:172.21.15.7:6833/4110623302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x55673af10fd0 con 0x55673b06d6f0 2024-09-10T21:50:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.028+0000 7f1eb6a00640 10 client.7878 handle_command_reply: tid=0 2024-09-10T21:50:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.028+0000 7f1edf885740 1 client.7878 shutdown 2024-09-10T21:50:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.028+0000 7f1edf885740 1 -- 172.21.15.7:0/2257389954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afc9220 msgr2=0x55673af9af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.028+0000 7f1edf885740 1 --2- 172.21.15.7:0/2257389954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afc9220 0x55673af9af10 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f1ec40016c0 tx=0x7f1ec40814c0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.028+0000 7f1edf885740 1 -- 172.21.15.7:0/2257389954 shutdown_connections 2024-09-10T21:50:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.028+0000 7f1edf885740 1 --2- 172.21.15.7:0/2257389954 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55673b06d6f0 0x55673afb7620 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.028+0000 7f1edf885740 1 --2- 172.21.15.7:0/2257389954 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55673afe5aa0 0x55673b06c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.028+0000 7f1edf885740 1 --2- 172.21.15.7:0/2257389954 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55673afc9220 0x55673af9af10 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.028+0000 7f1edf885740 1 --2- 172.21.15.7:0/2257389954 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55673afa1190 0x55673b00e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.028+0000 7f1edf885740 1 -- 172.21.15.7:0/2257389954 >> 172.21.15.7:0/2257389954 conn(0x55673afa0c10 msgr2=0x55673afc88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:50.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.028+0000 7f1eb6a00640 0 client.7878 ms_handle_reset on v2:172.21.15.7:6833/4110623302 2024-09-10T21:50:50.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.028+0000 7f1edf885740 1 -- 172.21.15.7:0/2257389954 shutdown_connections 2024-09-10T21:50:50.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.028+0000 7f1edf885740 1 -- 172.21.15.7:0/2257389954 wait complete. 2024-09-10T21:50:50.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.028+0000 7f1edf885740 20 client.7878 trim_cache size 0 max 16384 2024-09-10T21:50:50.034 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:50:50.034 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2029451a-3c26-48c7-b874-0597197278ac", 2024-09-10T21:50:50.034 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-10T21:50:50.034 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 7791, 2024-09-10T21:50:50.034 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T21:50:50.034 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T21:50:50.034 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T21:50:50.034 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 4.7313581400000002, 2024-09-10T21:50:50.035 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 91, 2024-09-10T21:50:50.035 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 102, 2024-09-10T21:50:50.035 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 102, 2024-09-10T21:50:50.035 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 5.6654411109999998 2024-09-10T21:50:50.035 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:50:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.029+0000 7f1edb200640 1 -- 172.21.15.7:0/3387880829 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1eb8077f70 msgr2=0x7f1eb807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.029+0000 7f1edb200640 1 --2- 172.21.15.7:0/3387880829 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1eb8077f70 0x7f1eb807a430 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8006bf0 tx=0x7f1ec80067a0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.029+0000 7f1edb200640 1 -- 172.21.15.7:0/3387880829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed413e140 msgr2=0x7f1ed41f5cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.029+0000 7f1edb200640 1 --2- 172.21.15.7:0/3387880829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed413e140 0x7f1ed41f5cc0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f1ec402a8e0 tx=0x7f1ec40063a0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.029+0000 7f1edb200640 1 -- 172.21.15.7:0/3387880829 shutdown_connections 2024-09-10T21:50:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.029+0000 7f1edb200640 1 --2- 172.21.15.7:0/3387880829 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1eb8077f70 0x7f1eb807a430 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.029+0000 7f1edb200640 1 --2- 172.21.15.7:0/3387880829 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1ed41f6200 0x7f1ed41f6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.029+0000 7f1edb200640 1 --2- 172.21.15.7:0/3387880829 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1ed413e140 0x7f1ed41f5cc0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.029+0000 7f1edb200640 1 --2- 172.21.15.7:0/3387880829 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1ebc004800 0x7f1ed41e78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.029+0000 7f1edb200640 1 -- 172.21.15.7:0/3387880829 >> 172.21.15.7:0/3387880829 conn(0x7f1ed40b16d0 msgr2=0x7f1ed415c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.029+0000 7f1edb200640 1 -- 172.21.15.7:0/3387880829 shutdown_connections 2024-09-10T21:50:50.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.029+0000 7f1edb200640 1 -- 172.21.15.7:0/3387880829 wait complete. 2024-09-10T21:50:50.045 DEBUG:teuthology.orchestra.run.smithi007:> sudo daemon-helper kill ceph --cluster ceph -w 2024-09-10T21:50:50.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:50.049 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start / repair,recursive 2024-09-10T21:50:50.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.213+0000 7f22a6600640 1 -- 172.21.15.7:0/1840617611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a00b4a30 msgr2=0x7f22a015edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.213+0000 7f22a6600640 1 --2- 172.21.15.7:0/1840617611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a00b4a30 0x7f22a015edd0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f2298005fd0 tx=0x7f229802a8b0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.213+0000 7f22a6600640 1 -- 172.21.15.7:0/1840617611 shutdown_connections 2024-09-10T21:50:50.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.213+0000 7f22a6600640 1 --2- 172.21.15.7:0/1840617611 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a00b4a30 0x7f22a015edd0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.213+0000 7f22a6600640 1 -- 172.21.15.7:0/1840617611 >> 172.21.15.7:0/1840617611 conn(0x7f22a00b1c10 msgr2=0x7f22a00b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:50.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.213+0000 7f22a6600640 1 -- 172.21.15.7:0/1840617611 shutdown_connections 2024-09-10T21:50:50.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.213+0000 7f22a6600640 1 -- 172.21.15.7:0/1840617611 wait complete. 2024-09-10T21:50:50.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.215+0000 7f22a6600640 1 Processor -- start 2024-09-10T21:50:50.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.215+0000 7f22a6600640 1 -- start start 2024-09-10T21:50:50.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.216+0000 7f22a6600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a006d230 0x7f22a00cfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.216+0000 7f22a6600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a00b4a30 0x7f22a00d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.216+0000 7f22a6600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a0158300 0x7f22a01f46a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.216+0000 7f22a6600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22a00740a0 con 0x7f22a006d230 2024-09-10T21:50:50.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.216+0000 7f22a6600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f22a0073f20 con 0x7f22a0158300 2024-09-10T21:50:50.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.216+0000 7f22a6600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f22a0074220 con 0x7f22a00b4a30 2024-09-10T21:50:50.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.216+0000 7f229f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a006d230 0x7f22a00cfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.216+0000 7f229f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a006d230 0x7f22a00cfbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37110/0 (socket says 172.21.15.7:37110) 2024-09-10T21:50:50.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.216+0000 7f229f400640 1 -- 172.21.15.7:0/1569147778 learned_addr learned my addr 172.21.15.7:0/1569147778 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:50.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.217+0000 7f229fe00640 1 --2- 172.21.15.7:0/1569147778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a0158300 0x7f22a01f46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.217+0000 7f2296a00640 1 --2- 172.21.15.7:0/1569147778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a00b4a30 0x7f22a00d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.217+0000 7f229f400640 1 -- 172.21.15.7:0/1569147778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a00b4a30 msgr2=0x7f22a00d0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.217+0000 7f229f400640 1 --2- 172.21.15.7:0/1569147778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a00b4a30 0x7f22a00d0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.217+0000 7f229f400640 1 -- 172.21.15.7:0/1569147778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a0158300 msgr2=0x7f22a01f46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.217+0000 7f229f400640 1 --2- 172.21.15.7:0/1569147778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a0158300 0x7f22a01f46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.217+0000 7f229f400640 1 -- 172.21.15.7:0/1569147778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22a014d640 con 0x7f22a006d230 2024-09-10T21:50:50.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.217+0000 7f229fe00640 1 --2- 172.21.15.7:0/1569147778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a0158300 0x7f22a01f46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:50:50.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.217+0000 7f229f400640 1 --2- 172.21.15.7:0/1569147778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a006d230 0x7f22a00cfbc0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f2290001220 tx=0x7f229000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:50.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.217+0000 7f2297e00640 1 -- 172.21.15.7:0/1569147778 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2290011790 con 0x7f22a006d230 2024-09-10T21:50:50.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.218+0000 7f2297e00640 1 -- 172.21.15.7:0/1569147778 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2290017070 con 0x7f22a006d230 2024-09-10T21:50:50.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.218+0000 7f2297e00640 1 -- 172.21.15.7:0/1569147778 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2290011cc0 con 0x7f22a006d230 2024-09-10T21:50:50.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.218+0000 7f22a6600640 1 -- 172.21.15.7:0/1569147778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22a014d920 con 0x7f22a006d230 2024-09-10T21:50:50.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.218+0000 7f22a6600640 1 -- 172.21.15.7:0/1569147778 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22a014dee0 con 0x7f22a006d230 2024-09-10T21:50:50.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.220+0000 7f2297e00640 1 -- 172.21.15.7:0/1569147778 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f229001c460 con 0x7f22a006d230 2024-09-10T21:50:50.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.221+0000 7f2297e00640 1 --2- 172.21.15.7:0/1569147778 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f227c077f70 0x7f227c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.221+0000 7f2297e00640 1 -- 172.21.15.7:0/1569147778 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f2290099c20 con 0x7f22a006d230 2024-09-10T21:50:50.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.221+0000 7f2296a00640 1 --2- 172.21.15.7:0/1569147778 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f227c077f70 0x7f227c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.221+0000 7f22a966d740 1 Processor -- start 2024-09-10T21:50:50.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.222+0000 7f22a966d740 1 -- start start 2024-09-10T21:50:50.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.222+0000 7f2296a00640 1 --2- 172.21.15.7:0/1569147778 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f227c077f70 0x7f227c07a430 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2288000c00 tx=0x7f2288009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:50.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.223+0000 7f22a966d740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a9319d10 0x55d7a931a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.223+0000 7f22a966d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d4e80 0x55d7a93187f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.223+0000 7f22a966d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 0x55d7a9318d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.223+0000 7f22a966d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d7a9246f40 con 0x55d7a92d5400 2024-09-10T21:50:50.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.223+0000 7f22a966d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d7a9236370 con 0x55d7a92d4e80 2024-09-10T21:50:50.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.223+0000 7f22a966d740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55d7a923a180 con 0x55d7a9319d10 2024-09-10T21:50:50.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.223+0000 7f229fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 0x55d7a9318d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.223+0000 7f229fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 0x55d7a9318d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37114/0 (socket says 172.21.15.7:37114) 2024-09-10T21:50:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.223+0000 7f229fe00640 1 -- 172.21.15.7:0/3959209239 learned_addr learned my addr 172.21.15.7:0/3959209239 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.223+0000 7f229f400640 1 --2- 172.21.15.7:0/3959209239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a9319d10 0x55d7a931a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.223+0000 7f2296a00640 1 --2- 172.21.15.7:0/3959209239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d4e80 0x55d7a93187f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.223+0000 7f229fe00640 1 -- 172.21.15.7:0/3959209239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a9319d10 msgr2=0x55d7a931a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.223+0000 7f229fe00640 1 --2- 172.21.15.7:0/3959209239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a9319d10 0x55d7a931a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.223+0000 7f229fe00640 1 -- 172.21.15.7:0/3959209239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d4e80 msgr2=0x55d7a93187f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.223+0000 7f229fe00640 1 --2- 172.21.15.7:0/3959209239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d4e80 0x55d7a93187f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.223+0000 7f229fe00640 1 -- 172.21.15.7:0/3959209239 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7a8f28470 con 0x55d7a92d5400 2024-09-10T21:50:50.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.224+0000 7f229fe00640 1 --2- 172.21.15.7:0/3959209239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 0x55d7a9318d30 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f2298009cd0 tx=0x7f2298004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8366dd561f014f30 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:50.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.224+0000 7f228fe00640 1 -- 172.21.15.7:0/3959209239 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298004040 con 0x55d7a92d5400 2024-09-10T21:50:50.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.224+0000 7f228fe00640 1 -- 172.21.15.7:0/3959209239 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22980041e0 con 0x55d7a92d5400 2024-09-10T21:50:50.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.224+0000 7f228fe00640 1 -- 172.21.15.7:0/3959209239 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22980045d0 con 0x55d7a92d5400 2024-09-10T21:50:50.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.225+0000 7f22a966d740 1 -- 172.21.15.7:0/3959209239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 msgr2=0x55d7a9318d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.225+0000 7f22a966d740 1 --2- 172.21.15.7:0/3959209239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 0x55d7a9318d30 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f2298009cd0 tx=0x7f2298004a40 comp rx=0 tx=0).stop 2024-09-10T21:50:50.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.225+0000 7f22a966d740 1 -- 172.21.15.7:0/3959209239 shutdown_connections 2024-09-10T21:50:50.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.225+0000 7f22a966d740 1 --2- 172.21.15.7:0/3959209239 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d4e80 0x55d7a93187f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.225+0000 7f22a966d740 1 --2- 172.21.15.7:0/3959209239 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 0x55d7a9318d30 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.225+0000 7f22a966d740 1 --2- 172.21.15.7:0/3959209239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a9319d10 0x55d7a931a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.225+0000 7f22a966d740 1 -- 172.21.15.7:0/3959209239 >> 172.21.15.7:0/3959209239 conn(0x55d7a92d0450 msgr2=0x55d7a92d3e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:50.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.225+0000 7f22a966d740 1 -- 172.21.15.7:0/3959209239 shutdown_connections 2024-09-10T21:50:50.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.225+0000 7f22a966d740 1 -- 172.21.15.7:0/3959209239 wait complete. 2024-09-10T21:50:50.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.226+0000 7f22a966d740 1 -- start start 2024-09-10T21:50:50.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.226+0000 7f22a966d740 1 Processor -- start 2024-09-10T21:50:50.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.227+0000 7f22a966d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d5400 0x55d7a9313c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.227+0000 7f22a966d740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a9319d10 0x55d7a92d2180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.227+0000 7f22a966d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d26c0 0x55d7a932a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.227+0000 7f22a966d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d7a8fa67c0 con 0x55d7a92d26c0 2024-09-10T21:50:50.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.227+0000 7f22a966d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55d7a91dbae0 con 0x55d7a92d5400 2024-09-10T21:50:50.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.227+0000 7f22a966d740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55d7a92cee70 con 0x55d7a9319d10 2024-09-10T21:50:50.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.227+0000 7f22a966d740 10 client.0 _ll_register_callbacks cb 0x55d7a91eb5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:50:50.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.228+0000 7f229fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d26c0 0x55d7a932a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.228+0000 7f229fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d26c0 0x55d7a932a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37124/0 (socket says 172.21.15.7:37124) 2024-09-10T21:50:50.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.228+0000 7f229fe00640 1 -- 172.21.15.7:0/1395276017 learned_addr learned my addr 172.21.15.7:0/1395276017 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:50.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.228+0000 7f229f400640 1 --2- 172.21.15.7:0/1395276017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d5400 0x55d7a9313c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.228+0000 7f2296a00640 1 --2- 172.21.15.7:0/1395276017 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a9319d10 0x55d7a92d2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.228+0000 7f229fe00640 1 -- 172.21.15.7:0/1395276017 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a9319d10 msgr2=0x55d7a92d2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.228+0000 7f229fe00640 1 --2- 172.21.15.7:0/1395276017 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a9319d10 0x55d7a92d2180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.228+0000 7f229fe00640 1 -- 172.21.15.7:0/1395276017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d5400 msgr2=0x55d7a9313c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.228+0000 7f229fe00640 1 --2- 172.21.15.7:0/1395276017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d5400 0x55d7a9313c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.228+0000 7f229fe00640 1 -- 172.21.15.7:0/1395276017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7a900c7e0 con 0x55d7a92d26c0 2024-09-10T21:50:50.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.229+0000 7f229fe00640 1 --2- 172.21.15.7:0/1395276017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d26c0 0x55d7a932a0f0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f2298009b30 tx=0x7f229802af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a91b814a751f39b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:50.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.229+0000 7f228cc00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:50:50.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.229+0000 7f228cc00640 1 -- 172.21.15.7:0/1395276017 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298053060 con 0x55d7a92d26c0 2024-09-10T21:50:50.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.229+0000 7f228cc00640 1 -- 172.21.15.7:0/1395276017 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f2298003e50 con 0x55d7a92d26c0 2024-09-10T21:50:50.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.229+0000 7f22a966d740 1 -- 172.21.15.7:0/1395276017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22980041e0 con 0x55d7a92d26c0 2024-09-10T21:50:50.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.229+0000 7f228cc00640 1 -- 172.21.15.7:0/1395276017 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f2298004b40 con 0x55d7a92d26c0 2024-09-10T21:50:50.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.229+0000 7f22a966d740 10 client.7893 fetch_fsmap learned FSMap version 91 2024-09-10T21:50:50.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.229+0000 7f22a966d740 1 -- 172.21.15.7:0/1395276017 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55d7a906cf70 con 0x55d7a92d26c0 2024-09-10T21:50:50.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.229+0000 7f228cc00640 1 -- 172.21.15.7:0/1395276017 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22980041e0 con 0x55d7a92d26c0 2024-09-10T21:50:50.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.230+0000 7f228cc00640 1 -- 172.21.15.7:0/1395276017 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 91) ==== 3502+0+0 (secure 0 0 0) 0x7f2298004ce0 con 0x55d7a92d26c0 2024-09-10T21:50:50.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.230+0000 7f22a966d740 10 client.7893 fetch_fsmap finished waiting for FSMap version 91 2024-09-10T21:50:50.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.230+0000 7f22a966d740 10 client.7893 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T21:50:50.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.230+0000 7f22a966d740 20 client.7893 populate_metadata read hostname 'smithi007' 2024-09-10T21:50:50.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.230+0000 7f22a966d740 1 --2- 172.21.15.7:0/1395276017 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55d7a93299d0 0x55d7a932c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.230+0000 7f22a966d740 4 client.7893 mds_command: new command op to 7791 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:50:50.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.230+0000 7f22a966d740 1 -- 172.21.15.7:0/1395276017 --> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7a9248e90 con 0x55d7a93299d0 2024-09-10T21:50:50.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.230+0000 7f229f400640 1 --2- 172.21.15.7:0/1395276017 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55d7a93299d0 0x55d7a932c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.230+0000 7f229f400640 1 --2- 172.21.15.7:0/1395276017 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55d7a93299d0 0x55d7a932c520 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=603d5b2eb766d1e7 server_cookie=22fe779554c07743 in_seq=0 out_seq=0 2024-09-10T21:50:50.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.230+0000 7f228cc00640 10 client.7893 ms_handle_connect on v2:172.21.15.7:6833/4110623302 2024-09-10T21:50:50.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.231+0000 7f228cc00640 1 -- 172.21.15.7:0/1395276017 <== mds.0 v2:172.21.15.7:6833/4110623302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d7a9248e90 con 0x55d7a93299d0 2024-09-10T21:50:50.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.231+0000 7f228cc00640 10 client.7893 handle_command_reply: tid=0 2024-09-10T21:50:50.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.231+0000 7f22a966d740 1 client.7893 shutdown 2024-09-10T21:50:50.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.231+0000 7f22a966d740 1 -- 172.21.15.7:0/1395276017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d26c0 msgr2=0x55d7a932a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.231+0000 7f22a966d740 1 --2- 172.21.15.7:0/1395276017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d26c0 0x55d7a932a0f0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f2298009b30 tx=0x7f229802af30 comp rx=0 tx=0).stop 2024-09-10T21:50:50.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.232+0000 7f22a966d740 1 -- 172.21.15.7:0/1395276017 shutdown_connections 2024-09-10T21:50:50.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.232+0000 7f22a966d740 1 --2- 172.21.15.7:0/1395276017 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55d7a93299d0 0x55d7a932c520 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.232+0000 7f22a966d740 1 --2- 172.21.15.7:0/1395276017 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d26c0 0x55d7a932a0f0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.232+0000 7f22a966d740 1 --2- 172.21.15.7:0/1395276017 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a9319d10 0x55d7a92d2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.232+0000 7f22a966d740 1 --2- 172.21.15.7:0/1395276017 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d5400 0x55d7a9313c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.232+0000 7f22a966d740 1 -- 172.21.15.7:0/1395276017 >> 172.21.15.7:0/1395276017 conn(0x55d7a92d4e80 msgr2=0x55d7a92d3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:50.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.232+0000 7f228cc00640 0 client.7893 ms_handle_reset on v2:172.21.15.7:6833/4110623302 2024-09-10T21:50:50.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.232+0000 7f22a966d740 1 -- 172.21.15.7:0/1395276017 shutdown_connections 2024-09-10T21:50:50.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.232+0000 7f22a966d740 1 -- 172.21.15.7:0/1395276017 wait complete. 2024-09-10T21:50:50.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.232+0000 7f22a966d740 20 client.7893 trim_cache size 0 max 16384 2024-09-10T21:50:50.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.239+0000 7fa973600640 1 -- 172.21.15.7:0/1194979064 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa96c15e0a0 msgr2=0x7fa96c0ceac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.239+0000 7fa973600640 1 --2- 172.21.15.7:0/1194979064 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa96c15e0a0 0x7fa96c0ceac0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa960009820 tx=0x7fa96002a510 comp rx=0 tx=0).stop 2024-09-10T21:50:50.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.239+0000 7fa973600640 1 -- 172.21.15.7:0/1194979064 shutdown_connections 2024-09-10T21:50:50.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.239+0000 7fa973600640 1 --2- 172.21.15.7:0/1194979064 >> v2:172.21.15.133:3300/0 conn(0x7fa96c0cf710 0x7fa96c0b77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.239+0000 7fa973600640 1 --2- 172.21.15.7:0/1194979064 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa96c15e0a0 0x7fa96c0ceac0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.239+0000 7fa973600640 1 -- 172.21.15.7:0/1194979064 >> 172.21.15.7:0/1194979064 conn(0x7fa96c0d3fb0 msgr2=0x7fa96c0d43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:50.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.239+0000 7fa973600640 1 -- 172.21.15.7:0/1194979064 shutdown_connections 2024-09-10T21:50:50.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.240+0000 7fa973600640 1 -- 172.21.15.7:0/1194979064 wait complete. 2024-09-10T21:50:50.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.242+0000 7fa973600640 1 Processor -- start 2024-09-10T21:50:50.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.242+0000 7fa973600640 1 -- start start 2024-09-10T21:50:50.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.244+0000 7fa973600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa96c0cf710 0x7fa96c0ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.244+0000 7fa973600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa96c15e0a0 0x7fa96c15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.244+0000 7fa973600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa96c15f490 0x7fa96c1f53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.244+0000 7fa973600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c0b7e60 con 0x7fa96c15f490 2024-09-10T21:50:50.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.244+0000 7fa973600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa96c0b7ce0 con 0x7fa96c0cf710 2024-09-10T21:50:50.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.244+0000 7fa973600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa96c0b7fe0 con 0x7fa96c15e0a0 2024-09-10T21:50:50.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.244+0000 7fa971600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa96c15f490 0x7fa96c1f53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.244+0000 7fa971600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa96c15f490 0x7fa96c1f53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37140/0 (socket says 172.21.15.7:37140) 2024-09-10T21:50:50.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.244+0000 7fa971600640 1 -- 172.21.15.7:0/4014916818 learned_addr learned my addr 172.21.15.7:0/4014916818 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:50.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.245+0000 7fa970c00640 1 --2- 172.21.15.7:0/4014916818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa96c0cf710 0x7fa96c0ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.245+0000 7fa96be00640 1 --2- 172.21.15.7:0/4014916818 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa96c15e0a0 0x7fa96c15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.245+0000 7fa971600640 1 -- 172.21.15.7:0/4014916818 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa96c15e0a0 msgr2=0x7fa96c15ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.245+0000 7fa971600640 1 --2- 172.21.15.7:0/4014916818 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa96c15e0a0 0x7fa96c15ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.245+0000 7fa971600640 1 -- 172.21.15.7:0/4014916818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa96c0cf710 msgr2=0x7fa96c0ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.245+0000 7fa971600640 1 --2- 172.21.15.7:0/4014916818 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa96c0cf710 0x7fa96c0ca2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.245+0000 7fa971600640 1 -- 172.21.15.7:0/4014916818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa96c0c9250 con 0x7fa96c15f490 2024-09-10T21:50:50.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.245+0000 7fa971600640 1 --2- 172.21.15.7:0/4014916818 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa96c15f490 0x7fa96c1f53b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7fa96400eb10 tx=0x7fa96400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:50.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.245+0000 7fa968c00640 1 -- 172.21.15.7:0/4014916818 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964004b20 con 0x7fa96c15f490 2024-09-10T21:50:50.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.245+0000 7fa968c00640 1 -- 172.21.15.7:0/4014916818 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa964004ce0 con 0x7fa96c15f490 2024-09-10T21:50:50.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.245+0000 7fa968c00640 1 -- 172.21.15.7:0/4014916818 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa964005280 con 0x7fa96c15f490 2024-09-10T21:50:50.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.246+0000 7fa973600640 1 -- 172.21.15.7:0/4014916818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa96c0c9530 con 0x7fa96c15f490 2024-09-10T21:50:50.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.246+0000 7fa973600640 1 -- 172.21.15.7:0/4014916818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa96c0c9af0 con 0x7fa96c15f490 2024-09-10T21:50:50.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.246+0000 7fa973600640 1 -- 172.21.15.7:0/4014916818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fa96c0b77a0 con 0x7fa96c15f490 2024-09-10T21:50:50.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.248+0000 7fa968c00640 1 -- 172.21.15.7:0/4014916818 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa964005420 con 0x7fa96c15f490 2024-09-10T21:50:50.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.250+0000 7fa968c00640 1 --2- 172.21.15.7:0/4014916818 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa94c077f70 0x7fa94c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.250+0000 7fa968c00640 1 -- 172.21.15.7:0/4014916818 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7fa96409b180 con 0x7fa96c15f490 2024-09-10T21:50:50.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.250+0000 7fa970c00640 1 --2- 172.21.15.7:0/4014916818 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa94c077f70 0x7fa94c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.250+0000 7fa968c00640 1 -- 172.21.15.7:0/4014916818 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+512 (secure 0 0 0) 0x7fa964026020 con 0x7fa96c15f490 2024-09-10T21:50:50.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.251+0000 7fa973600640 1 -- 172.21.15.7:0/4014916818 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({log-info=0+}) -- 0x7fa96c0c7880 con 0x7fa96c15f490 2024-09-10T21:50:50.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.251+0000 7fa968c00640 1 -- 172.21.15.7:0/4014916818 <== mon.0 v2:172.21.15.7:3300/0 7 ==== log(2 entries from seq 924 at 2024-09-10T21:50:48.303236+0000) ==== 389+0+0 (secure 0 0 0) 0x7fa964099740 con 0x7fa96c15f490 2024-09-10T21:50:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: cluster: 2024-09-10T21:50:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: id: 2029451a-3c26-48c7-b874-0597197278ac 2024-09-10T21:50:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: health: HEALTH_OK 2024-09-10T21:50:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: services: 2024-09-10T21:50:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: mon: 3 daemons, quorum a,b,c (age 9m) 2024-09-10T21:50:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: mgr: x(active, since 9m), standbys: y 2024-09-10T21:50:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: mds: 1/1 daemons up, 3 standby 2024-09-10T21:50:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: osd: 8 osds: 8 up (since 8m), 8 in (since 9m) 2024-09-10T21:50:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: data: 2024-09-10T21:50:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: volumes: 1/1 healthy 2024-09-10T21:50:50.252 INFO:teuthology.orchestra.run.smithi007.stdout: pools: 3 pools, 97 pgs 2024-09-10T21:50:50.253 INFO:teuthology.orchestra.run.smithi007.stdout: objects: 30 objects, 19 MiB 2024-09-10T21:50:50.253 INFO:teuthology.orchestra.run.smithi007.stdout: usage: 303 MiB used, 720 GiB / 720 GiB avail 2024-09-10T21:50:50.253 INFO:teuthology.orchestra.run.smithi007.stdout: pgs: 97 active+clean 2024-09-10T21:50:50.253 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:50.253 INFO:teuthology.orchestra.run.smithi007.stdout: io: 2024-09-10T21:50:50.253 INFO:teuthology.orchestra.run.smithi007.stdout: client: 1.8 KiB/s rd, 102 B/s wr, 1 op/s rd, 0 op/s wr 2024-09-10T21:50:50.253 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:50.253 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:50:50.253 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:50:48.303236+0000 mon.a [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T21:50:50.253 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:50:48.303267+0000 mon.a [INF] Cluster is now healthy 2024-09-10T21:50:50.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.251+0000 7fa970c00640 1 --2- 172.21.15.7:0/4014916818 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa94c077f70 0x7fa94c07a430 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa960034000 tx=0x7fa960004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:50.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.258+0000 7f22a966d740 1 Processor -- start 2024-09-10T21:50:50.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.259+0000 7f22a966d740 1 -- start start 2024-09-10T21:50:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.259+0000 7f22a966d740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a92d4e80 0x55d7a93192b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.259+0000 7f22a966d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d26c0 0x55d7a92cd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.259+0000 7f22a966d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 0x55d7a9395760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.259+0000 7f22a966d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f22a0201090 con 0x55d7a92d5400 2024-09-10T21:50:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.259+0000 7f22a966d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f22a02000f0 con 0x55d7a92d26c0 2024-09-10T21:50:50.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.259+0000 7f22a966d740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f22a014c120 con 0x55d7a92d4e80 2024-09-10T21:50:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.260+0000 7f229fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 0x55d7a9395760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.260+0000 7f229fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 0x55d7a9395760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37148/0 (socket says 172.21.15.7:37148) 2024-09-10T21:50:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.260+0000 7f229fe00640 1 -- 172.21.15.7:0/3432239477 learned_addr learned my addr 172.21.15.7:0/3432239477 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.260+0000 7f2296a00640 1 --2- 172.21.15.7:0/3432239477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d26c0 0x55d7a92cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.260+0000 7f229f400640 1 --2- 172.21.15.7:0/3432239477 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a92d4e80 0x55d7a93192b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.260+0000 7f229fe00640 1 -- 172.21.15.7:0/3432239477 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a92d4e80 msgr2=0x55d7a93192b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.260+0000 7f229fe00640 1 --2- 172.21.15.7:0/3432239477 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a92d4e80 0x55d7a93192b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.260+0000 7f229fe00640 1 -- 172.21.15.7:0/3432239477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d26c0 msgr2=0x55d7a92cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.260+0000 7f229fe00640 1 --2- 172.21.15.7:0/3432239477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d26c0 0x55d7a92cd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.260+0000 7f229fe00640 1 -- 172.21.15.7:0/3432239477 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7a8f75720 con 0x55d7a92d5400 2024-09-10T21:50:50.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.261+0000 7f229fe00640 1 --2- 172.21.15.7:0/3432239477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 0x55d7a9395760 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f2298006010 tx=0x7f2298009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddba1a9c538827c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:50.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.261+0000 7f228f400640 1 -- 172.21.15.7:0/3432239477 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298062070 con 0x55d7a92d5400 2024-09-10T21:50:50.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.261+0000 7f228f400640 1 -- 172.21.15.7:0/3432239477 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2298003e50 con 0x55d7a92d5400 2024-09-10T21:50:50.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.261+0000 7f228f400640 1 -- 172.21.15.7:0/3432239477 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22980040a0 con 0x55d7a92d5400 2024-09-10T21:50:50.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.261+0000 7f22a966d740 1 -- 172.21.15.7:0/3432239477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 msgr2=0x55d7a9395760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.261+0000 7f22a966d740 1 --2- 172.21.15.7:0/3432239477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 0x55d7a9395760 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f2298006010 tx=0x7f2298009ac0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.262+0000 7f22a966d740 1 -- 172.21.15.7:0/3432239477 shutdown_connections 2024-09-10T21:50:50.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.262+0000 7f22a966d740 1 --2- 172.21.15.7:0/3432239477 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d26c0 0x55d7a92cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.262+0000 7f22a966d740 1 --2- 172.21.15.7:0/3432239477 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 0x55d7a9395760 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.262+0000 7f22a966d740 1 --2- 172.21.15.7:0/3432239477 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a92d4e80 0x55d7a93192b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.262+0000 7f22a966d740 1 -- 172.21.15.7:0/3432239477 >> 172.21.15.7:0/3432239477 conn(0x55d7a932c740 msgr2=0x55d7a92ced80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:50.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.262+0000 7f22a966d740 1 -- 172.21.15.7:0/3432239477 shutdown_connections 2024-09-10T21:50:50.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.262+0000 7f22a966d740 1 -- 172.21.15.7:0/3432239477 wait complete. 2024-09-10T21:50:50.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.262+0000 7f22a966d740 1 -- start start 2024-09-10T21:50:50.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.263+0000 7f22a966d740 1 Processor -- start 2024-09-10T21:50:50.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.264+0000 7f22a966d740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a92d26c0 0x55d7a932fee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.264+0000 7f22a966d740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d4e80 0x55d7a93918a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.264+0000 7f22a966d740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 0x55d7a939b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.264+0000 7f22a966d740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55d7a93131e0 con 0x55d7a92d5400 2024-09-10T21:50:50.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.264+0000 7f22a966d740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f22a014c770 con 0x55d7a92d4e80 2024-09-10T21:50:50.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.264+0000 7f22a966d740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55d7a9330770 con 0x55d7a92d26c0 2024-09-10T21:50:50.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.264+0000 7f22a966d740 10 client.0 _ll_register_callbacks cb 0x7f229005e5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:50:50.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.264+0000 7f229fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 0x55d7a939b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.264+0000 7f229fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 0x55d7a939b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37164/0 (socket says 172.21.15.7:37164) 2024-09-10T21:50:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.264+0000 7f229fe00640 1 -- 172.21.15.7:0/2771591419 learned_addr learned my addr 172.21.15.7:0/2771591419 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.264+0000 7f229f400640 1 --2- 172.21.15.7:0/2771591419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a92d26c0 0x55d7a932fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.264+0000 7f2296a00640 1 --2- 172.21.15.7:0/2771591419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d4e80 0x55d7a93918a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.264+0000 7f229fe00640 1 -- 172.21.15.7:0/2771591419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a92d26c0 msgr2=0x55d7a932fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.264+0000 7f229fe00640 1 --2- 172.21.15.7:0/2771591419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a92d26c0 0x55d7a932fee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.264+0000 7f229fe00640 1 -- 172.21.15.7:0/2771591419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d4e80 msgr2=0x55d7a93918a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.264+0000 7f229fe00640 1 --2- 172.21.15.7:0/2771591419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d4e80 0x55d7a93918a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.264+0000 7f229fe00640 1 -- 172.21.15.7:0/2771591419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7a939be40 con 0x55d7a92d5400 2024-09-10T21:50:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.264+0000 7f2296a00640 1 --2- 172.21.15.7:0/2771591419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d4e80 0x55d7a93918a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:50:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.265+0000 7f229fe00640 1 --2- 172.21.15.7:0/2771591419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 0x55d7a939b900 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f2298000f80 tx=0x7f2298004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13bc8d3730c48e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.265+0000 7f228cc00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:50:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.265+0000 7f228cc00640 1 -- 172.21.15.7:0/2771591419 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2298062070 con 0x55d7a92d5400 2024-09-10T21:50:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.265+0000 7f228cc00640 1 -- 172.21.15.7:0/2771591419 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f2298002030 con 0x55d7a92d5400 2024-09-10T21:50:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.265+0000 7f22a966d740 1 -- 172.21.15.7:0/2771591419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2298003e50 con 0x55d7a92d5400 2024-09-10T21:50:50.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.265+0000 7f228cc00640 1 -- 172.21.15.7:0/2771591419 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f229806c070 con 0x55d7a92d5400 2024-09-10T21:50:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.265+0000 7f22a966d740 10 client.7905 fetch_fsmap learned FSMap version 91 2024-09-10T21:50:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.265+0000 7f22a966d740 1 -- 172.21.15.7:0/2771591419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55d7a93304b0 con 0x55d7a92d5400 2024-09-10T21:50:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.265+0000 7f228cc00640 1 -- 172.21.15.7:0/2771591419 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2298003e50 con 0x55d7a92d5400 2024-09-10T21:50:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.265+0000 7f228cc00640 1 -- 172.21.15.7:0/2771591419 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 91) ==== 3502+0+0 (secure 0 0 0) 0x7f229806f070 con 0x55d7a92d5400 2024-09-10T21:50:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.265+0000 7f22a966d740 10 client.7905 fetch_fsmap finished waiting for FSMap version 91 2024-09-10T21:50:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.265+0000 7f22a966d740 10 client.7905 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T21:50:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.265+0000 7f22a966d740 20 client.7905 populate_metadata read hostname 'smithi007' 2024-09-10T21:50:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.265+0000 7f22a966d740 1 --2- 172.21.15.7:0/2771591419 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55d7a9319d10 0x55d7a92fde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.265+0000 7f22a966d740 4 client.7905 mds_command: new command op to 7791 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2024-09-10T21:50:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.265+0000 7f22a966d740 1 -- 172.21.15.7:0/2771591419 --> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) -- 0x55d7a93128c0 con 0x55d7a9319d10 2024-09-10T21:50:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.266+0000 7f229f400640 1 --2- 172.21.15.7:0/2771591419 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55d7a9319d10 0x55d7a92fde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.266+0000 7f229f400640 1 --2- 172.21.15.7:0/2771591419 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55d7a9319d10 0x55d7a92fde60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faafee14fdb36459 server_cookie=3b0fbb471f62790d in_seq=0 out_seq=0 2024-09-10T21:50:50.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.266+0000 7f228cc00640 10 client.7905 ms_handle_connect on v2:172.21.15.7:6833/4110623302 2024-09-10T21:50:50.270 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-09-10T21:50:50.268+0000 7f55b5a00640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-09-10T21:50:50.270 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-09-10T21:50:50.268+0000 7f55b5a00640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-09-10T21:50:50.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.268+0000 7f228cc00640 1 -- 172.21.15.7:0/2771591419 <== mds.0 v2:172.21.15.7:6833/4110623302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d7a93128c0 con 0x55d7a9319d10 2024-09-10T21:50:50.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.268+0000 7f228cc00640 10 client.7905 handle_command_reply: tid=0 2024-09-10T21:50:50.271 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-09-10T21:50:50.268+0000 7f55b5a00640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-09-10T21:50:50.271 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-09-10T21:50:50.268+0000 7f55b5a00640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-09-10T21:50:50.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.268+0000 7f22a966d740 1 client.7905 shutdown 2024-09-10T21:50:50.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.268+0000 7f22a966d740 1 -- 172.21.15.7:0/2771591419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 msgr2=0x55d7a939b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.268+0000 7f22a966d740 1 --2- 172.21.15.7:0/2771591419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 0x55d7a939b900 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f2298000f80 tx=0x7f2298004230 comp rx=0 tx=0).stop 2024-09-10T21:50:50.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.268+0000 7f22a966d740 1 -- 172.21.15.7:0/2771591419 shutdown_connections 2024-09-10T21:50:50.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.268+0000 7f22a966d740 1 --2- 172.21.15.7:0/2771591419 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55d7a9319d10 0x55d7a92fde60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.268+0000 7f22a966d740 1 --2- 172.21.15.7:0/2771591419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55d7a92d4e80 0x55d7a93918a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.268+0000 7f22a966d740 1 --2- 172.21.15.7:0/2771591419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55d7a92d5400 0x55d7a939b900 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.268+0000 7f22a966d740 1 --2- 172.21.15.7:0/2771591419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55d7a92d26c0 0x55d7a932fee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.268+0000 7f22a966d740 1 -- 172.21.15.7:0/2771591419 >> 172.21.15.7:0/2771591419 conn(0x55d7a932c740 msgr2=0x55d7a92f9d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:50.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.268+0000 7f228cc00640 0 client.7905 ms_handle_reset on v2:172.21.15.7:6833/4110623302 2024-09-10T21:50:50.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.269+0000 7f22a966d740 1 -- 172.21.15.7:0/2771591419 shutdown_connections 2024-09-10T21:50:50.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.269+0000 7f22a966d740 1 -- 172.21.15.7:0/2771591419 wait complete. 2024-09-10T21:50:50.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.269+0000 7f22a966d740 20 client.7905 trim_cache size 0 max 16384 2024-09-10T21:50:50.272 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:50:50.273 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0, 2024-09-10T21:50:50.273 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_tag": "5b7aaafc-a7c4-4a78-8ebe-9508949e5efb", 2024-09-10T21:50:50.273 INFO:teuthology.orchestra.run.smithi007.stdout: "mode": "asynchronous" 2024-09-10T21:50:50.273 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:50:50.273 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.270+0000 7f22a6600640 1 -- 172.21.15.7:0/1569147778 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f227c077f70 msgr2=0x7f227c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.270+0000 7f22a6600640 1 --2- 172.21.15.7:0/1569147778 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f227c077f70 0x7f227c07a430 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2288000c00 tx=0x7f2288009040 comp rx=0 tx=0).stop 2024-09-10T21:50:50.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.270+0000 7f22a6600640 1 -- 172.21.15.7:0/1569147778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a006d230 msgr2=0x7f22a00cfbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.270+0000 7f22a6600640 1 --2- 172.21.15.7:0/1569147778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a006d230 0x7f22a00cfbc0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f2290001220 tx=0x7f229000c890 comp rx=0 tx=0).stop 2024-09-10T21:50:50.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.270+0000 7f22a6600640 1 -- 172.21.15.7:0/1569147778 shutdown_connections 2024-09-10T21:50:50.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.270+0000 7f22a6600640 1 --2- 172.21.15.7:0/1569147778 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f227c077f70 0x7f227c07a430 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.270+0000 7f22a6600640 1 --2- 172.21.15.7:0/1569147778 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f22a0158300 0x7f22a01f46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.270+0000 7f22a6600640 1 --2- 172.21.15.7:0/1569147778 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f22a00b4a30 0x7f22a00d0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.270+0000 7f22a6600640 1 --2- 172.21.15.7:0/1569147778 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f22a006d230 0x7f22a00cfbc0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.270+0000 7f22a6600640 1 -- 172.21.15.7:0/1569147778 >> 172.21.15.7:0/1569147778 conn(0x7f22a00b1c10 msgr2=0x7f22a0072800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:50.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.270+0000 7f22a6600640 1 -- 172.21.15.7:0/1569147778 shutdown_connections 2024-09-10T21:50:50.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.270+0000 7f22a6600640 1 -- 172.21.15.7:0/1569147778 wait complete. 2024-09-10T21:50:50.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:50:50.287 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-10T21:50:50.299 INFO:tasks.ceph.mds.c.smithi007.stderr:2024-09-10T21:50:50.298+0000 7f55baa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:50:50.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.450+0000 7f4a0de00640 1 -- 172.21.15.7:0/3114948241 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a080b0c70 msgr2=0x7f4a080d0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.450+0000 7f4a0de00640 1 --2- 172.21.15.7:0/3114948241 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a080b0c70 0x7f4a080d0100 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f49f4009860 tx=0x7f49f402a530 comp rx=0 tx=0).stop 2024-09-10T21:50:50.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.451+0000 7f4a0de00640 1 -- 172.21.15.7:0/3114948241 shutdown_connections 2024-09-10T21:50:50.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.451+0000 7f4a0de00640 1 --2- 172.21.15.7:0/3114948241 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a080b0c70 0x7f4a080d0100 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.451+0000 7f4a0de00640 1 -- 172.21.15.7:0/3114948241 >> 172.21.15.7:0/3114948241 conn(0x7f4a080d1c10 msgr2=0x7f4a080d2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:50.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.451+0000 7f4a0de00640 1 -- 172.21.15.7:0/3114948241 shutdown_connections 2024-09-10T21:50:50.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.451+0000 7f4a0de00640 1 -- 172.21.15.7:0/3114948241 wait complete. 2024-09-10T21:50:50.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.452+0000 7f4a0de00640 1 Processor -- start 2024-09-10T21:50:50.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.452+0000 7f4a0de00640 1 -- start start 2024-09-10T21:50:50.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.453+0000 7f4a0de00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a080b0c70 0x7f4a08152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.453+0000 7f4a0de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a080ce690 0x7f4a0806f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.453+0000 7f4a0de00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a080cea70 0x7f4a0815d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.453+0000 7f4a0de00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4a080d0640 con 0x7f4a080ce690 2024-09-10T21:50:50.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.453+0000 7f4a0de00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4a080d2940 con 0x7f4a080cea70 2024-09-10T21:50:50.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.453+0000 7f4a0de00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4a080d07c0 con 0x7f4a080b0c70 2024-09-10T21:50:50.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.454+0000 7f4a07e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a080ce690 0x7f4a0806f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.454+0000 7f4a07e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a080ce690 0x7f4a0806f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37166/0 (socket says 172.21.15.7:37166) 2024-09-10T21:50:50.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.454+0000 7f4a07e00640 1 -- 172.21.15.7:0/1611571309 learned_addr learned my addr 172.21.15.7:0/1611571309 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:50.455 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.454+0000 7f4a0ca00640 1 --2- 172.21.15.7:0/1611571309 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a080b0c70 0x7f4a08152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.454+0000 7f4a0d400640 1 --2- 172.21.15.7:0/1611571309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a080cea70 0x7f4a0815d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.454+0000 7f4a07e00640 1 -- 172.21.15.7:0/1611571309 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a080b0c70 msgr2=0x7f4a08152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.454+0000 7f4a07e00640 1 --2- 172.21.15.7:0/1611571309 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a080b0c70 0x7f4a08152440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.454+0000 7f4a07e00640 1 -- 172.21.15.7:0/1611571309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a080cea70 msgr2=0x7f4a0815d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.454+0000 7f4a07e00640 1 --2- 172.21.15.7:0/1611571309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a080cea70 0x7f4a0815d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.454+0000 7f4a07e00640 1 -- 172.21.15.7:0/1611571309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a0806fbe0 con 0x7f4a080ce690 2024-09-10T21:50:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.455+0000 7f4a07e00640 1 --2- 172.21.15.7:0/1611571309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a080ce690 0x7f4a0806f6a0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f49f402ab20 tx=0x7f49f4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.455+0000 7f4a04c00640 1 -- 172.21.15.7:0/1611571309 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4005c90 con 0x7f4a080ce690 2024-09-10T21:50:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.455+0000 7f4a04c00640 1 -- 172.21.15.7:0/1611571309 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49f4005020 con 0x7f4a080ce690 2024-09-10T21:50:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.455+0000 7f4a0de00640 1 -- 172.21.15.7:0/1611571309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a080c74e0 con 0x7f4a080ce690 2024-09-10T21:50:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.455+0000 7f4a04c00640 1 -- 172.21.15.7:0/1611571309 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4005410 con 0x7f4a080ce690 2024-09-10T21:50:50.457 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.455+0000 7f4a0de00640 1 -- 172.21.15.7:0/1611571309 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a080c7aa0 con 0x7f4a080ce690 2024-09-10T21:50:50.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.458+0000 7f4a04c00640 1 -- 172.21.15.7:0/1611571309 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f49f40317c0 con 0x7f4a080ce690 2024-09-10T21:50:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.459+0000 7f4a04c00640 1 --2- 172.21.15.7:0/1611571309 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f49e8077f70 0x7f49e807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.459+0000 7f4a04c00640 1 -- 172.21.15.7:0/1611571309 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f49f40bafe0 con 0x7f4a080ce690 2024-09-10T21:50:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.459+0000 7f4a12278740 1 Processor -- start 2024-09-10T21:50:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.459+0000 7f4a12278740 1 -- start start 2024-09-10T21:50:50.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.460+0000 7f4a0ca00640 1 --2- 172.21.15.7:0/1611571309 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f49e8077f70 0x7f49e807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.460+0000 7f4a12278740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e63a32e0 0x5587e63a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.460+0000 7f4a12278740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e635e450 0x5587e63a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.460+0000 7f4a12278740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e635e9d0 0x5587e63a2300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.460+0000 7f4a12278740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5587e62bd1c0 con 0x5587e635e450 2024-09-10T21:50:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.460+0000 7f4a12278740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5587e62d0870 con 0x5587e635e9d0 2024-09-10T21:50:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.460+0000 7f4a12278740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5587e62bfca0 con 0x5587e63a32e0 2024-09-10T21:50:50.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.460+0000 7f4a0ca00640 1 --2- 172.21.15.7:0/1611571309 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f49e8077f70 0x7f49e807a430 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f49fc001220 tx=0x7f49fc0016e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.460+0000 7f4a07e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e635e450 0x5587e63a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.461+0000 7f4a07e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e635e450 0x5587e63a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37170/0 (socket says 172.21.15.7:37170) 2024-09-10T21:50:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.461+0000 7f4a0ca00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e63a32e0 0x5587e63a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.461+0000 7f4a07e00640 1 -- 172.21.15.7:0/918282669 learned_addr learned my addr 172.21.15.7:0/918282669 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.461+0000 7f4a0d400640 1 --2- 172.21.15.7:0/918282669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e635e9d0 0x5587e63a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.461+0000 7f4a07e00640 1 -- 172.21.15.7:0/918282669 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e63a32e0 msgr2=0x5587e63a36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.461+0000 7f4a07e00640 1 --2- 172.21.15.7:0/918282669 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e63a32e0 0x5587e63a36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.461+0000 7f4a07e00640 1 -- 172.21.15.7:0/918282669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e635e9d0 msgr2=0x5587e63a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.461+0000 7f4a07e00640 1 --2- 172.21.15.7:0/918282669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e635e9d0 0x5587e63a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.461+0000 7f4a07e00640 1 -- 172.21.15.7:0/918282669 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587e60987d0 con 0x5587e635e450 2024-09-10T21:50:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.461+0000 7f4a0d400640 1 --2- 172.21.15.7:0/918282669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e635e9d0 0x5587e63a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:50:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.461+0000 7f4a07e00640 1 --2- 172.21.15.7:0/918282669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e635e450 0x5587e63a1dc0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f49f40095f0 tx=0x7f49f4082d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd496f93eb6c6da server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:50.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.461+0000 7f49f9200640 1 -- 172.21.15.7:0/918282669 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4049040 con 0x5587e635e450 2024-09-10T21:50:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.461+0000 7f49f9200640 1 -- 172.21.15.7:0/918282669 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49f4082ed0 con 0x5587e635e450 2024-09-10T21:50:50.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.461+0000 7f49f9200640 1 -- 172.21.15.7:0/918282669 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f40833c0 con 0x5587e635e450 2024-09-10T21:50:50.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.462+0000 7f4a12278740 1 -- 172.21.15.7:0/918282669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e635e450 msgr2=0x5587e63a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.462+0000 7f4a12278740 1 --2- 172.21.15.7:0/918282669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e635e450 0x5587e63a1dc0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f49f40095f0 tx=0x7f49f4082d10 comp rx=0 tx=0).stop 2024-09-10T21:50:50.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.462+0000 7f4a12278740 1 -- 172.21.15.7:0/918282669 shutdown_connections 2024-09-10T21:50:50.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.462+0000 7f4a12278740 1 --2- 172.21.15.7:0/918282669 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e635e9d0 0x5587e63a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.462+0000 7f4a12278740 1 --2- 172.21.15.7:0/918282669 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e635e450 0x5587e63a1dc0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.462+0000 7f4a12278740 1 --2- 172.21.15.7:0/918282669 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e63a32e0 0x5587e63a36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.462+0000 7f4a12278740 1 -- 172.21.15.7:0/918282669 >> 172.21.15.7:0/918282669 conn(0x5587e6359a20 msgr2=0x5587e635d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:50.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.462+0000 7f4a12278740 1 -- 172.21.15.7:0/918282669 shutdown_connections 2024-09-10T21:50:50.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.462+0000 7f4a12278740 1 -- 172.21.15.7:0/918282669 wait complete. 2024-09-10T21:50:50.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.463+0000 7f4a12278740 1 -- start start 2024-09-10T21:50:50.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.463+0000 7f4a12278740 1 Processor -- start 2024-09-10T21:50:50.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.464+0000 7f4a12278740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e635e9d0 0x5587e6374f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.464+0000 7f4a12278740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e63a32e0 0x5587e63a2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.464+0000 7f4a12278740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63b22d0 0x5587e635baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.464+0000 7f4a12278740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5587e6266410 con 0x5587e63b22d0 2024-09-10T21:50:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.464+0000 7f4a12278740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5587e62c3ab0 con 0x5587e63a32e0 2024-09-10T21:50:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.464+0000 7f4a12278740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5587e6030740 con 0x5587e635e9d0 2024-09-10T21:50:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.464+0000 7f4a12278740 10 client.0 _ll_register_callbacks cb 0x5587e6243360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:50:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.464+0000 7f4a0d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63b22d0 0x5587e635baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.465+0000 7f4a0ca00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e635e9d0 0x5587e6374f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.465+0000 7f4a0ca00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e635e9d0 0x5587e6374f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:55184/0 (socket says 172.21.15.7:55184) 2024-09-10T21:50:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.465+0000 7f4a0d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63b22d0 0x5587e635baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37180/0 (socket says 172.21.15.7:37180) 2024-09-10T21:50:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.465+0000 7f4a0ca00640 1 -- 172.21.15.7:0/2572327738 learned_addr learned my addr 172.21.15.7:0/2572327738 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.465+0000 7f4a07e00640 1 --2- 172.21.15.7:0/2572327738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e63a32e0 0x5587e63a2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.465+0000 7f4a0ca00640 1 -- 172.21.15.7:0/2572327738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e63a32e0 msgr2=0x5587e63a2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.465+0000 7f4a0ca00640 1 --2- 172.21.15.7:0/2572327738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e63a32e0 0x5587e63a2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.465+0000 7f4a0ca00640 1 -- 172.21.15.7:0/2572327738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63b22d0 msgr2=0x5587e635baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.465+0000 7f4a0ca00640 1 --2- 172.21.15.7:0/2572327738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63b22d0 0x5587e635baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.465+0000 7f4a0ca00640 1 -- 172.21.15.7:0/2572327738 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) -- 0x5587e6021970 con 0x5587e635e9d0 2024-09-10T21:50:50.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.465+0000 7f4a0d400640 1 --2- 172.21.15.7:0/2572327738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63b22d0 0x5587e635baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:50:50.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.465+0000 7f4a0ca00640 1 --2- 172.21.15.7:0/2572327738 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e635e9d0 0x5587e6374f40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00fe40 tx=0x7f49fc004600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8aa343f39f30797 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:50.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.465+0000 7f49ed600640 10 client.0 ms_handle_connect on v2:172.21.15.133:3301/0 2024-09-10T21:50:50.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.466+0000 7f49ed600640 1 -- 172.21.15.7:0/2572327738 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49fc025020 con 0x5587e635e9d0 2024-09-10T21:50:50.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.466+0000 7f49ed600640 1 -- 172.21.15.7:0/2572327738 <== mon.2 v2:172.21.15.133:3301/0 2 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f49fc014660 con 0x5587e635e9d0 2024-09-10T21:50:50.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.466+0000 7f4a12278740 1 -- 172.21.15.7:0/2572327738 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49f4082ed0 con 0x5587e635e9d0 2024-09-10T21:50:50.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.466+0000 7f49ed600640 1 -- 172.21.15.7:0/2572327738 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f49fc02f050 con 0x5587e635e9d0 2024-09-10T21:50:50.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.466+0000 7f4a12278740 10 client.236591 fetch_fsmap learned FSMap version 91 2024-09-10T21:50:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.466+0000 7f4a12278740 1 -- 172.21.15.7:0/2572327738 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x5587e5fff6a0 con 0x5587e635e9d0 2024-09-10T21:50:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.467+0000 7f49ed600640 1 -- 172.21.15.7:0/2572327738 <== mon.2 v2:172.21.15.133:3301/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49f4082ed0 con 0x5587e635e9d0 2024-09-10T21:50:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.467+0000 7f49ed600640 1 -- 172.21.15.7:0/2572327738 <== mon.2 v2:172.21.15.133:3301/0 5 ==== fsmap(e 91) ==== 3502+0+0 (secure 0 0 0) 0x7f49fc032020 con 0x5587e635e9d0 2024-09-10T21:50:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.467+0000 7f4a12278740 10 client.236591 fetch_fsmap finished waiting for FSMap version 91 2024-09-10T21:50:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.467+0000 7f4a12278740 10 client.236591 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T21:50:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.467+0000 7f4a12278740 20 client.236591 populate_metadata read hostname 'smithi007' 2024-09-10T21:50:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.467+0000 7f4a12278740 1 --2- 172.21.15.7:0/2572327738 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x5587e63b6430 0x5587e639f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.467+0000 7f4a12278740 4 client.236591 mds_command: new command op to 7791 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:50:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.467+0000 7f4a12278740 1 -- 172.21.15.7:0/2572327738 --> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5587e62d27c0 con 0x5587e63b6430 2024-09-10T21:50:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.467+0000 7f4a07e00640 1 --2- 172.21.15.7:0/2572327738 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x5587e63b6430 0x5587e639f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.468+0000 7f4a07e00640 1 --2- 172.21.15.7:0/2572327738 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x5587e63b6430 0x5587e639f9a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2617476eec85a28 server_cookie=1f9e832940f08895 in_seq=0 out_seq=0 2024-09-10T21:50:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.468+0000 7f49ed600640 10 client.236591 ms_handle_connect on v2:172.21.15.7:6833/4110623302 2024-09-10T21:50:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.468+0000 7f49ed600640 1 -- 172.21.15.7:0/2572327738 <== mds.0 v2:172.21.15.7:6833/4110623302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5587e62d27c0 con 0x5587e63b6430 2024-09-10T21:50:50.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.468+0000 7f49ed600640 10 client.236591 handle_command_reply: tid=0 2024-09-10T21:50:50.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.468+0000 7f4a12278740 1 client.236591 shutdown 2024-09-10T21:50:50.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.469+0000 7f4a12278740 1 -- 172.21.15.7:0/2572327738 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e635e9d0 msgr2=0x5587e6374f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.469+0000 7f4a12278740 1 --2- 172.21.15.7:0/2572327738 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e635e9d0 0x5587e6374f40 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f49fc00fe40 tx=0x7f49fc004600 comp rx=0 tx=0).stop 2024-09-10T21:50:50.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.469+0000 7f4a12278740 1 -- 172.21.15.7:0/2572327738 shutdown_connections 2024-09-10T21:50:50.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.469+0000 7f4a12278740 1 --2- 172.21.15.7:0/2572327738 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x5587e63b6430 0x5587e639f9a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.469+0000 7f4a12278740 1 --2- 172.21.15.7:0/2572327738 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e63a32e0 0x5587e63a2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.469+0000 7f4a12278740 1 --2- 172.21.15.7:0/2572327738 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63b22d0 0x5587e635baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.469+0000 7f49ed600640 0 client.236591 ms_handle_reset on v2:172.21.15.7:6833/4110623302 2024-09-10T21:50:50.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.469+0000 7f4a12278740 1 --2- 172.21.15.7:0/2572327738 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e635e9d0 0x5587e6374f40 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.469+0000 7f4a12278740 1 -- 172.21.15.7:0/2572327738 >> 172.21.15.7:0/2572327738 conn(0x5587e635e450 msgr2=0x5587e63585f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:50.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.469+0000 7f4a12278740 1 -- 172.21.15.7:0/2572327738 shutdown_connections 2024-09-10T21:50:50.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.469+0000 7f4a12278740 1 -- 172.21.15.7:0/2572327738 wait complete. 2024-09-10T21:50:50.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.469+0000 7f4a12278740 20 client.236591 trim_cache size 0 max 16384 2024-09-10T21:50:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.487+0000 7f4a12278740 1 Processor -- start 2024-09-10T21:50:50.488 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.488+0000 7f4a12278740 1 -- start start 2024-09-10T21:50:50.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.488+0000 7f4a12278740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e635e9d0 0x5587e63570e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.488+0000 7f4a12278740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e63a32e0 0x5587e6398790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.488+0000 7f4a12278740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63b22d0 0x5587e6358750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.488+0000 7f4a12278740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4a08200ee0 con 0x5587e63b22d0 2024-09-10T21:50:50.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.488+0000 7f4a12278740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0806eac0 con 0x5587e635e9d0 2024-09-10T21:50:50.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.488+0000 7f4a12278740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4a0806de20 con 0x5587e63a32e0 2024-09-10T21:50:50.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.489+0000 7f4a0d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63b22d0 0x5587e6358750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.489+0000 7f4a0d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63b22d0 0x5587e6358750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37188/0 (socket says 172.21.15.7:37188) 2024-09-10T21:50:50.490 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.489+0000 7f4a0d400640 1 -- 172.21.15.7:0/4141703935 learned_addr learned my addr 172.21.15.7:0/4141703935 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.489+0000 7f4a07e00640 1 --2- 172.21.15.7:0/4141703935 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e63a32e0 0x5587e6398790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.489+0000 7f4a0ca00640 1 --2- 172.21.15.7:0/4141703935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e635e9d0 0x5587e63570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.489+0000 7f4a0d400640 1 -- 172.21.15.7:0/4141703935 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e63a32e0 msgr2=0x5587e6398790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.489+0000 7f4a0d400640 1 --2- 172.21.15.7:0/4141703935 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e63a32e0 0x5587e6398790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.489+0000 7f4a0d400640 1 -- 172.21.15.7:0/4141703935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e635e9d0 msgr2=0x5587e63570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.489+0000 7f4a0d400640 1 --2- 172.21.15.7:0/4141703935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e635e9d0 0x5587e63570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.489+0000 7f4a0d400640 1 -- 172.21.15.7:0/4141703935 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587e639eea0 con 0x5587e63b22d0 2024-09-10T21:50:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.489+0000 7f4a0ca00640 1 --2- 172.21.15.7:0/4141703935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e635e9d0 0x5587e63570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:50:50.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.489+0000 7f4a07e00640 1 --2- 172.21.15.7:0/4141703935 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e63a32e0 0x5587e6398790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:50:50.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.489+0000 7f4a0d400640 1 --2- 172.21.15.7:0/4141703935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63b22d0 0x5587e6358750 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f4a00005470 tx=0x7f4a00005930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40cff133248a1ad2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:50.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.490+0000 7f49efe00640 1 -- 172.21.15.7:0/4141703935 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00002060 con 0x5587e63b22d0 2024-09-10T21:50:50.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.490+0000 7f49efe00640 1 -- 172.21.15.7:0/4141703935 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a00016720 con 0x5587e63b22d0 2024-09-10T21:50:50.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.490+0000 7f49efe00640 1 -- 172.21.15.7:0/4141703935 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a00016b30 con 0x5587e63b22d0 2024-09-10T21:50:50.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.490+0000 7f4a12278740 1 -- 172.21.15.7:0/4141703935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63b22d0 msgr2=0x5587e6358750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.490+0000 7f4a12278740 1 --2- 172.21.15.7:0/4141703935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63b22d0 0x5587e6358750 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f4a00005470 tx=0x7f4a00005930 comp rx=0 tx=0).stop 2024-09-10T21:50:50.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.490+0000 7f4a12278740 1 -- 172.21.15.7:0/4141703935 shutdown_connections 2024-09-10T21:50:50.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.490+0000 7f4a12278740 1 --2- 172.21.15.7:0/4141703935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63b22d0 0x5587e6358750 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.490+0000 7f4a12278740 1 --2- 172.21.15.7:0/4141703935 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e63a32e0 0x5587e6398790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.490+0000 7f4a12278740 1 --2- 172.21.15.7:0/4141703935 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e635e9d0 0x5587e63570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.490+0000 7f4a12278740 1 -- 172.21.15.7:0/4141703935 >> 172.21.15.7:0/4141703935 conn(0x5587e635e450 msgr2=0x5587e63a2810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:50.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.490+0000 7f4a12278740 1 -- 172.21.15.7:0/4141703935 shutdown_connections 2024-09-10T21:50:50.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.490+0000 7f4a12278740 1 -- 172.21.15.7:0/4141703935 wait complete. 2024-09-10T21:50:50.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.491+0000 7f4a12278740 1 -- start start 2024-09-10T21:50:50.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.491+0000 7f4a12278740 1 Processor -- start 2024-09-10T21:50:50.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.492+0000 7f4a12278740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e635e9d0 0x5587e639c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.492+0000 7f4a12278740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63a32e0 0x5587e641ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.492+0000 7f4a12278740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e63b22d0 0x5587e63c6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.492+0000 7f4a12278740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5587e639b7c0 con 0x5587e63a32e0 2024-09-10T21:50:50.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.492+0000 7f4a12278740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4a0806e470 con 0x5587e63b22d0 2024-09-10T21:50:50.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.492+0000 7f4a12278740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5587e6374fe0 con 0x5587e635e9d0 2024-09-10T21:50:50.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.492+0000 7f4a12278740 10 client.0 _ll_register_callbacks cb 0x7f49fc006890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:50:50.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.492+0000 7f4a07e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63a32e0 0x5587e641ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.492+0000 7f4a07e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63a32e0 0x5587e641ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37196/0 (socket says 172.21.15.7:37196) 2024-09-10T21:50:50.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.492+0000 7f4a07e00640 1 -- 172.21.15.7:0/2571116419 learned_addr learned my addr 172.21.15.7:0/2571116419 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:50:50.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.493+0000 7f4a0ca00640 1 --2- 172.21.15.7:0/2571116419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e635e9d0 0x5587e639c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.493+0000 7f4a0d400640 1 --2- 172.21.15.7:0/2571116419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e63b22d0 0x5587e63c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.493+0000 7f4a07e00640 1 -- 172.21.15.7:0/2571116419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e635e9d0 msgr2=0x5587e639c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.493+0000 7f4a07e00640 1 --2- 172.21.15.7:0/2571116419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e635e9d0 0x5587e639c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.493+0000 7f4a07e00640 1 -- 172.21.15.7:0/2571116419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e63b22d0 msgr2=0x5587e63c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:50:50.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.493+0000 7f4a07e00640 1 --2- 172.21.15.7:0/2571116419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e63b22d0 0x5587e63c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.493+0000 7f4a07e00640 1 -- 172.21.15.7:0/2571116419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5587e6356c00 con 0x5587e63a32e0 2024-09-10T21:50:50.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.493+0000 7f4a0d400640 1 --2- 172.21.15.7:0/2571116419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e63b22d0 0x5587e63c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:50:50.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.493+0000 7f4a07e00640 1 --2- 172.21.15.7:0/2571116419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63a32e0 0x5587e641ffb0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f49f4082e80 tx=0x7f49f4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75252947e10953f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:50:50.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.493+0000 7f49ed600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:50:50.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.493+0000 7f49ed600640 1 -- 172.21.15.7:0/2571116419 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49f4049040 con 0x5587e63a32e0 2024-09-10T21:50:50.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.493+0000 7f49ed600640 1 -- 172.21.15.7:0/2571116419 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f49f407f630 con 0x5587e63a32e0 2024-09-10T21:50:50.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.493+0000 7f4a12278740 1 -- 172.21.15.7:0/2571116419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a00016720 con 0x5587e63a32e0 2024-09-10T21:50:50.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.493+0000 7f49ed600640 1 -- 172.21.15.7:0/2571116419 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f49f4082ed0 con 0x5587e63a32e0 2024-09-10T21:50:50.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.493+0000 7f4a12278740 10 client.7923 fetch_fsmap learned FSMap version 91 2024-09-10T21:50:50.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.493+0000 7f4a12278740 1 -- 172.21.15.7:0/2571116419 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x5587e6357350 con 0x5587e63a32e0 2024-09-10T21:50:50.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.494+0000 7f49ed600640 1 -- 172.21.15.7:0/2571116419 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a00016720 con 0x5587e63a32e0 2024-09-10T21:50:50.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.494+0000 7f49ed600640 1 -- 172.21.15.7:0/2571116419 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 91) ==== 3502+0+0 (secure 0 0 0) 0x7f49f408c380 con 0x5587e63a32e0 2024-09-10T21:50:50.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.494+0000 7f4a12278740 10 client.7923 fetch_fsmap finished waiting for FSMap version 91 2024-09-10T21:50:50.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.494+0000 7f4a12278740 10 client.7923 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T21:50:50.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.494+0000 7f4a12278740 20 client.7923 populate_metadata read hostname 'smithi007' 2024-09-10T21:50:50.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.494+0000 7f4a12278740 1 --2- 172.21.15.7:0/2571116419 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x5587e63820d0 0x5587e63824e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:50:50.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.494+0000 7f4a12278740 4 client.7923 mds_command: new command op to 7791 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:50:50.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.494+0000 7f4a12278740 1 -- 172.21.15.7:0/2571116419 --> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5587e62ce270 con 0x5587e63820d0 2024-09-10T21:50:50.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.494+0000 7f4a0ca00640 1 --2- 172.21.15.7:0/2571116419 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x5587e63820d0 0x5587e63824e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:50:50.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.495+0000 7f4a0ca00640 1 --2- 172.21.15.7:0/2571116419 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x5587e63820d0 0x5587e63824e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b412632a76122faf server_cookie=f2241efdfa50b9ef in_seq=0 out_seq=0 2024-09-10T21:50:50.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.495+0000 7f49ed600640 10 client.7923 ms_handle_connect on v2:172.21.15.7:6833/4110623302 2024-09-10T21:50:50.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.495+0000 7f49ed600640 1 -- 172.21.15.7:0/2571116419 <== mds.0 v2:172.21.15.7:6833/4110623302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+571 (crc 0 0 0) 0x5587e62ce270 con 0x5587e63820d0 2024-09-10T21:50:50.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.495+0000 7f49ed600640 10 client.7923 handle_command_reply: tid=0 2024-09-10T21:50:50.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.495+0000 7f4a12278740 1 client.7923 shutdown 2024-09-10T21:50:50.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.495+0000 7f4a12278740 1 -- 172.21.15.7:0/2571116419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63a32e0 msgr2=0x5587e641ffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.495+0000 7f4a12278740 1 --2- 172.21.15.7:0/2571116419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63a32e0 0x5587e641ffb0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f49f4082e80 tx=0x7f49f4005f90 comp rx=0 tx=0).stop 2024-09-10T21:50:50.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.495+0000 7f4a12278740 1 -- 172.21.15.7:0/2571116419 shutdown_connections 2024-09-10T21:50:50.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.495+0000 7f4a12278740 1 --2- 172.21.15.7:0/2571116419 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x5587e63820d0 0x5587e63824e0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.495+0000 7f4a12278740 1 --2- 172.21.15.7:0/2571116419 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5587e63b22d0 0x5587e63c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.495+0000 7f4a12278740 1 --2- 172.21.15.7:0/2571116419 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5587e63a32e0 0x5587e641ffb0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.495+0000 7f4a12278740 1 --2- 172.21.15.7:0/2571116419 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5587e635e9d0 0x5587e639c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.495+0000 7f49ed600640 0 client.7923 ms_handle_reset on v2:172.21.15.7:6833/4110623302 2024-09-10T21:50:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.495+0000 7f4a12278740 1 -- 172.21.15.7:0/2571116419 >> 172.21.15.7:0/2571116419 conn(0x5587e635e450 msgr2=0x5587e635c2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.495+0000 7f4a12278740 1 -- 172.21.15.7:0/2571116419 shutdown_connections 2024-09-10T21:50:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.495+0000 7f4a12278740 1 -- 172.21.15.7:0/2571116419 wait complete. 2024-09-10T21:50:50.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.495+0000 7f4a12278740 20 client.7923 trim_cache size 0 max 16384 2024-09-10T21:50:50.501 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:50:50.501 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-10T21:50:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": { 2024-09-10T21:50:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "5b7aaafc-a7c4-4a78-8ebe-9508949e5efb": { 2024-09-10T21:50:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-09-10T21:50:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "tag": "5b7aaafc-a7c4-4a78-8ebe-9508949e5efb", 2024-09-10T21:50:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "options": "recursive,repair" 2024-09-10T21:50:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:50:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:50:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:50:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "5b7aaafc-a7c4-4a78-8ebe-9508949e5efb": { 2024-09-10T21:50:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:50:50.266548+0000", 2024-09-10T21:50:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-09-10T21:50:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:50:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:50:50.502 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:50:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:50:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:50:50.503 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:50:50.503 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:50:50.503 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.496+0000 7f4a0de00640 1 -- 172.21.15.7:0/1611571309 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f49e8077f70 msgr2=0x7f49e807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.496+0000 7f4a0de00640 1 --2- 172.21.15.7:0/1611571309 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f49e8077f70 0x7f49e807a430 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f49fc001220 tx=0x7f49fc0016e0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.496+0000 7f4a0de00640 1 -- 172.21.15.7:0/1611571309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a080ce690 msgr2=0x7f4a0806f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:50:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.496+0000 7f4a0de00640 1 --2- 172.21.15.7:0/1611571309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a080ce690 0x7f4a0806f6a0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f49f402ab20 tx=0x7f49f4003c10 comp rx=0 tx=0).stop 2024-09-10T21:50:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.496+0000 7f4a0de00640 1 -- 172.21.15.7:0/1611571309 shutdown_connections 2024-09-10T21:50:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.496+0000 7f4a0de00640 1 --2- 172.21.15.7:0/1611571309 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f49e8077f70 0x7f49e807a430 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.496+0000 7f4a0de00640 1 --2- 172.21.15.7:0/1611571309 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4a080cea70 0x7f4a0815d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.496+0000 7f4a0de00640 1 --2- 172.21.15.7:0/1611571309 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4a080ce690 0x7f4a0806f6a0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.496+0000 7f4a0de00640 1 --2- 172.21.15.7:0/1611571309 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4a080b0c70 0x7f4a08152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:50:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.496+0000 7f4a0de00640 1 -- 172.21.15.7:0/1611571309 >> 172.21.15.7:0/1611571309 conn(0x7f4a080d1c10 msgr2=0x7f4a080cfec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:50:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.496+0000 7f4a0de00640 1 -- 172.21.15.7:0/1611571309 shutdown_connections 2024-09-10T21:50:50.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:50.496+0000 7f4a0de00640 1 -- 172.21.15.7:0/1611571309 wait complete. 2024-09-10T21:50:50.512 INFO:tasks.cephfs.filesystem:scrub status for tag:5b7aaafc-a7c4-4a78-8ebe-9508949e5efb - {'path': '/', 'tag': '5b7aaafc-a7c4-4a78-8ebe-9508949e5efb', 'options': 'recursive,repair'} 2024-09-10T21:50:51.315 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:50:50.267785+0000 mds.c [INF] scrub queued for path: / 2024-09-10T21:50:51.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:51.314+0000 7fa968c00640 1 -- 172.21.15.7:0/4014916818 <== mon.0 v2:172.21.15.7:3300/0 8 ==== log(8 entries from seq 1 at 2024-09-10T21:50:50.267785+0000) ==== 1407+0+0 (secure 0 0 0) 0x7fa964061eb0 con 0x7fa96c15f490 2024-09-10T21:50:51.316 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:50:50.267787+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-09-10T21:50:51.316 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:50:50.267792+0000 mds.c [INF] scrub summary: active paths [/] 2024-09-10T21:50:51.316 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:50:50.269187+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000001 2024-09-10T21:50:51.316 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:50:50.269194+0000 mds.c [ERR] inode table repaired for inode: 0x10000000001 2024-09-10T21:50:51.316 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:50:50.269266+0000 mds.c [ERR] scrub: inode wrongly marked free: 0x10000000002 2024-09-10T21:50:51.316 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:50:50.269276+0000 mds.c [ERR] inode table repaired for inode: 0x10000000002 2024-09-10T21:50:51.316 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:50:50.270040+0000 mds.c [INF] scrub summary: idle+waiting paths [/] 2024-09-10T21:50:56.328 INFO:teuthology.orchestra.run.smithi007.stdout:2024-09-10T21:50:54.366319+0000 mds.c [INF] scrub summary: idle 2024-09-10T21:50:56.328 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:50:56.327+0000 7fa968c00640 1 -- 172.21.15.7:0/4014916818 <== mon.0 v2:172.21.15.7:3300/0 9 ==== log(1 entries from seq 9 at 2024-09-10T21:50:54.366319+0000) ==== 189+0+0 (secure 0 0 0) 0x7fa96405f770 con 0x7fa96c15f490 2024-09-10T21:51:01.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:51:01.527 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:51:20.513 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:20.513 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-09-10T21:51:20.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.688+0000 7fcc6cc00640 1 -- 172.21.15.7:0/417015912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc68074450 msgr2=0x7fcc680ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:20.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.688+0000 7fcc6cc00640 1 --2- 172.21.15.7:0/417015912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc68074450 0x7fcc680ca270 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fcc60005fd0 tx=0x7fcc6002aa00 comp rx=0 tx=0).stop 2024-09-10T21:51:20.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.688+0000 7fcc6cc00640 1 -- 172.21.15.7:0/417015912 shutdown_connections 2024-09-10T21:51:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.688+0000 7fcc6cc00640 1 --2- 172.21.15.7:0/417015912 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc68074450 0x7fcc680ca270 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.688+0000 7fcc6cc00640 1 --2- 172.21.15.7:0/417015912 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc68155fb0 0x7fcc681600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.688+0000 7fcc6cc00640 1 -- 172.21.15.7:0/417015912 >> 172.21.15.7:0/417015912 conn(0x7fcc680b16d0 msgr2=0x7fcc680b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.688+0000 7fcc6cc00640 1 -- 172.21.15.7:0/417015912 shutdown_connections 2024-09-10T21:51:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.688+0000 7fcc6cc00640 1 -- 172.21.15.7:0/417015912 wait complete. 2024-09-10T21:51:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.690+0000 7fcc6cc00640 1 Processor -- start 2024-09-10T21:51:20.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.690+0000 7fcc6cc00640 1 -- start start 2024-09-10T21:51:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.691+0000 7fcc6cc00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc6806fcf0 0x7fcc680b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.691+0000 7fcc6cc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc68074450 0x7fcc6815aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:20.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.691+0000 7fcc6cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc68155fb0 0x7fcc6815b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.691+0000 7fcc6cc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc680ca930 con 0x7fcc68155fb0 2024-09-10T21:51:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.691+0000 7fcc6cc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcc680ca7b0 con 0x7fcc68074450 2024-09-10T21:51:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.691+0000 7fcc6cc00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fcc680caab0 con 0x7fcc6806fcf0 2024-09-10T21:51:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.691+0000 7fcc66800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc68155fb0 0x7fcc6815b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.691+0000 7fcc66800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc68155fb0 0x7fcc6815b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44636/0 (socket says 172.21.15.7:44636) 2024-09-10T21:51:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.691+0000 7fcc66800640 1 -- 172.21.15.7:0/2851046097 learned_addr learned my addr 172.21.15.7:0/2851046097 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.691+0000 7fcc5d400640 1 --2- 172.21.15.7:0/2851046097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc68074450 0x7fcc6815aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.691+0000 7fcc65e00640 1 --2- 172.21.15.7:0/2851046097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc6806fcf0 0x7fcc680b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.692+0000 7fcc66800640 1 -- 172.21.15.7:0/2851046097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc6806fcf0 msgr2=0x7fcc680b4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.692+0000 7fcc66800640 1 --2- 172.21.15.7:0/2851046097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc6806fcf0 0x7fcc680b4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.692+0000 7fcc66800640 1 -- 172.21.15.7:0/2851046097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc68074450 msgr2=0x7fcc6815aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:20.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.692+0000 7fcc66800640 1 --2- 172.21.15.7:0/2851046097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc68074450 0x7fcc6815aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.692+0000 7fcc66800640 1 -- 172.21.15.7:0/2851046097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc680c5020 con 0x7fcc68155fb0 2024-09-10T21:51:20.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.692+0000 7fcc66800640 1 --2- 172.21.15.7:0/2851046097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc68155fb0 0x7fcc6815b000 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fcc60006010 tx=0x7fcc60003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:20.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.692+0000 7fcc5ea00640 1 -- 172.21.15.7:0/2851046097 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc60047070 con 0x7fcc68155fb0 2024-09-10T21:51:20.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.692+0000 7fcc5ea00640 1 -- 172.21.15.7:0/2851046097 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc60004250 con 0x7fcc68155fb0 2024-09-10T21:51:20.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.692+0000 7fcc5ea00640 1 -- 172.21.15.7:0/2851046097 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc60033050 con 0x7fcc68155fb0 2024-09-10T21:51:20.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.692+0000 7fcc6cc00640 1 -- 172.21.15.7:0/2851046097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc680c5300 con 0x7fcc68155fb0 2024-09-10T21:51:20.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.692+0000 7fcc6cc00640 1 -- 172.21.15.7:0/2851046097 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc680b5520 con 0x7fcc68155fb0 2024-09-10T21:51:20.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.695+0000 7fcc5ea00640 1 -- 172.21.15.7:0/2851046097 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fcc600046a0 con 0x7fcc68155fb0 2024-09-10T21:51:20.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.695+0000 7fcc5ea00640 1 --2- 172.21.15.7:0/2851046097 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcc44077f70 0x7fcc4407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:20.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.696+0000 7fcc5ea00640 1 -- 172.21.15.7:0/2851046097 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7fcc600bb3b0 con 0x7fcc68155fb0 2024-09-10T21:51:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.696+0000 7fcc6fa88740 1 Processor -- start 2024-09-10T21:51:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.696+0000 7fcc65e00640 1 --2- 172.21.15.7:0/2851046097 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcc44077f70 0x7fcc4407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:20.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.696+0000 7fcc6fa88740 1 -- start start 2024-09-10T21:51:20.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.697+0000 7fcc65e00640 1 --2- 172.21.15.7:0/2851046097 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcc44077f70 0x7fcc4407a430 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcc580098e0 tx=0x7fcc58009060 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:20.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.697+0000 7fcc6fa88740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d6e130 0x55baa4d6e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:20.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.697+0000 7fcc6fa88740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d292a0 0x55baa4d6cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:20.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.697+0000 7fcc6fa88740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d29820 0x55baa4d6d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:20.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.697+0000 7fcc6fa88740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55baa4c881c0 con 0x55baa4d29820 2024-09-10T21:51:20.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.697+0000 7fcc6fa88740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55baa4c9b870 con 0x55baa4d6e130 2024-09-10T21:51:20.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.697+0000 7fcc6fa88740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55baa4c8aca0 con 0x55baa4d292a0 2024-09-10T21:51:20.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.697+0000 7fcc66800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d29820 0x55baa4d6d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:20.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.697+0000 7fcc66800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d29820 0x55baa4d6d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44644/0 (socket says 172.21.15.7:44644) 2024-09-10T21:51:20.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.697+0000 7fcc66800640 1 -- 172.21.15.7:0/897091066 learned_addr learned my addr 172.21.15.7:0/897091066 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:20.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.697+0000 7fcc5d400640 1 --2- 172.21.15.7:0/897091066 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d292a0 0x55baa4d6cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:20.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.697+0000 7fcc66800640 1 -- 172.21.15.7:0/897091066 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d292a0 msgr2=0x55baa4d6cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:20.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.697+0000 7fcc65e00640 1 --2- 172.21.15.7:0/897091066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d6e130 0x55baa4d6e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:20.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.697+0000 7fcc66800640 1 --2- 172.21.15.7:0/897091066 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d292a0 0x55baa4d6cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.697+0000 7fcc66800640 1 -- 172.21.15.7:0/897091066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d6e130 msgr2=0x55baa4d6e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:20.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.697+0000 7fcc66800640 1 --2- 172.21.15.7:0/897091066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d6e130 0x55baa4d6e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.698+0000 7fcc66800640 1 -- 172.21.15.7:0/897091066 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baa4a637d0 con 0x55baa4d29820 2024-09-10T21:51:20.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.698+0000 7fcc65e00640 1 --2- 172.21.15.7:0/897091066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d6e130 0x55baa4d6e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:51:20.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.698+0000 7fcc66800640 1 --2- 172.21.15.7:0/897091066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d29820 0x55baa4d6d150 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fcc60031040 tx=0x7fcc600831b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9de1515bf33662 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:20.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.698+0000 7fcc56600640 1 -- 172.21.15.7:0/897091066 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc60088190 con 0x55baa4d29820 2024-09-10T21:51:20.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.698+0000 7fcc56600640 1 -- 172.21.15.7:0/897091066 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc600bb0e0 con 0x55baa4d29820 2024-09-10T21:51:20.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.698+0000 7fcc56600640 1 -- 172.21.15.7:0/897091066 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc600829c0 con 0x55baa4d29820 2024-09-10T21:51:20.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.699+0000 7fcc6fa88740 1 -- 172.21.15.7:0/897091066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d29820 msgr2=0x55baa4d6d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:20.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.699+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/897091066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d29820 0x55baa4d6d150 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fcc60031040 tx=0x7fcc600831b0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.699+0000 7fcc6fa88740 1 -- 172.21.15.7:0/897091066 shutdown_connections 2024-09-10T21:51:20.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.699+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/897091066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d29820 0x55baa4d6d150 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.699+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/897091066 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d292a0 0x55baa4d6cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.699+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/897091066 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d6e130 0x55baa4d6e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.699+0000 7fcc6fa88740 1 -- 172.21.15.7:0/897091066 >> 172.21.15.7:0/897091066 conn(0x55baa4d24870 msgr2=0x55baa4d28290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:20.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.699+0000 7fcc6fa88740 1 -- 172.21.15.7:0/897091066 shutdown_connections 2024-09-10T21:51:20.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.699+0000 7fcc6fa88740 1 -- 172.21.15.7:0/897091066 wait complete. 2024-09-10T21:51:20.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.700+0000 7fcc6fa88740 1 -- start start 2024-09-10T21:51:20.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.700+0000 7fcc6fa88740 1 Processor -- start 2024-09-10T21:51:20.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.701+0000 7fcc6fa88740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d29820 0x55baa4d80c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:20.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.701+0000 7fcc6fa88740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d6e130 0x55baa4d4c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:20.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.701+0000 7fcc6fa88740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d52000 0x55baa4d52420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:20.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.701+0000 7fcc6fa88740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55baa4c31410 con 0x55baa4d6e130 2024-09-10T21:51:20.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.701+0000 7fcc6fa88740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55baa4c8eab0 con 0x55baa4d52000 2024-09-10T21:51:20.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.701+0000 7fcc6fa88740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55baa49fb740 con 0x55baa4d29820 2024-09-10T21:51:20.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.701+0000 7fcc6fa88740 10 client.0 _ll_register_callbacks cb 0x55baa4c0e360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:51:20.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.701+0000 7fcc5d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d6e130 0x55baa4d4c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:20.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.701+0000 7fcc5d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d6e130 0x55baa4d4c400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44648/0 (socket says 172.21.15.7:44648) 2024-09-10T21:51:20.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.701+0000 7fcc5d400640 1 -- 172.21.15.7:0/3642990955 learned_addr learned my addr 172.21.15.7:0/3642990955 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:20.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.701+0000 7fcc66800640 1 --2- 172.21.15.7:0/3642990955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d52000 0x55baa4d52420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:20.707 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.701+0000 7fcc65e00640 1 --2- 172.21.15.7:0/3642990955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d29820 0x55baa4d80c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:20.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.701+0000 7fcc5d400640 1 -- 172.21.15.7:0/3642990955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d29820 msgr2=0x55baa4d80c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:20.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.701+0000 7fcc5d400640 1 --2- 172.21.15.7:0/3642990955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d29820 0x55baa4d80c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.701+0000 7fcc5d400640 1 -- 172.21.15.7:0/3642990955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d52000 msgr2=0x55baa4d52420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:20.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.701+0000 7fcc5d400640 1 --2- 172.21.15.7:0/3642990955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d52000 0x55baa4d52420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.701+0000 7fcc5d400640 1 -- 172.21.15.7:0/3642990955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baa49ec970 con 0x55baa4d6e130 2024-09-10T21:51:20.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.701+0000 7fcc65e00640 1 --2- 172.21.15.7:0/3642990955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d29820 0x55baa4d80c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:51:20.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.702+0000 7fcc5d400640 1 --2- 172.21.15.7:0/3642990955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d6e130 0x55baa4d4c400 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fcc50005430 tx=0x7fcc500105b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111b677d77e175fa server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:20.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.702+0000 7fcc4aa00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:51:20.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.702+0000 7fcc4aa00640 1 -- 172.21.15.7:0/3642990955 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc50019020 con 0x55baa4d6e130 2024-09-10T21:51:20.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.702+0000 7fcc6fa88740 1 -- 172.21.15.7:0/3642990955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc600bb0e0 con 0x55baa4d6e130 2024-09-10T21:51:20.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.702+0000 7fcc4aa00640 1 -- 172.21.15.7:0/3642990955 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7fcc50015660 con 0x55baa4d6e130 2024-09-10T21:51:20.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.702+0000 7fcc4aa00640 1 -- 172.21.15.7:0/3642990955 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fcc50023050 con 0x55baa4d6e130 2024-09-10T21:51:20.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.702+0000 7fcc6fa88740 10 client.7935 fetch_fsmap learned FSMap version 91 2024-09-10T21:51:20.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.702+0000 7fcc6fa88740 1 -- 172.21.15.7:0/3642990955 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55baa49ca6a0 con 0x55baa4d6e130 2024-09-10T21:51:20.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.702+0000 7fcc4aa00640 1 -- 172.21.15.7:0/3642990955 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc600bb0e0 con 0x55baa4d6e130 2024-09-10T21:51:20.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.703+0000 7fcc4aa00640 1 -- 172.21.15.7:0/3642990955 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 91) ==== 3502+0+0 (secure 0 0 0) 0x7fcc50026020 con 0x55baa4d6e130 2024-09-10T21:51:20.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.703+0000 7fcc6fa88740 10 client.7935 fetch_fsmap finished waiting for FSMap version 91 2024-09-10T21:51:20.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.703+0000 7fcc6fa88740 10 client.7935 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T21:51:20.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.703+0000 7fcc6fa88740 20 client.7935 populate_metadata read hostname 'smithi007' 2024-09-10T21:51:20.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.703+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/3642990955 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55baa4d67910 0x55baa4d67d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:20.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.703+0000 7fcc6fa88740 4 client.7935 mds_command: new command op to 7791 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:51:20.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.703+0000 7fcc6fa88740 1 -- 172.21.15.7:0/3642990955 --> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55baa4c9d7c0 con 0x55baa4d67910 2024-09-10T21:51:20.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.703+0000 7fcc65e00640 1 --2- 172.21.15.7:0/3642990955 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55baa4d67910 0x55baa4d67d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:20.709 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.703+0000 7fcc65e00640 1 --2- 172.21.15.7:0/3642990955 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55baa4d67910 0x55baa4d67d10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5d87b3b37a5875c server_cookie=f7ebb1d346247214 in_seq=0 out_seq=0 2024-09-10T21:51:20.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.703+0000 7fcc4aa00640 10 client.7935 ms_handle_connect on v2:172.21.15.7:6833/4110623302 2024-09-10T21:51:20.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.704+0000 7fcc4aa00640 1 -- 172.21.15.7:0/3642990955 <== mds.0 v2:172.21.15.7:6833/4110623302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55baa4c9d7c0 con 0x55baa4d67910 2024-09-10T21:51:20.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.704+0000 7fcc4aa00640 10 client.7935 handle_command_reply: tid=0 2024-09-10T21:51:20.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.704+0000 7fcc6fa88740 1 client.7935 shutdown 2024-09-10T21:51:20.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.704+0000 7fcc6fa88740 1 -- 172.21.15.7:0/3642990955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d6e130 msgr2=0x55baa4d4c400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:20.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.704+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/3642990955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d6e130 0x55baa4d4c400 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fcc50005430 tx=0x7fcc500105b0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.705+0000 7fcc6fa88740 1 -- 172.21.15.7:0/3642990955 shutdown_connections 2024-09-10T21:51:20.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.705+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/3642990955 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55baa4d67910 0x55baa4d67d10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.705+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/3642990955 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d52000 0x55baa4d52420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.705+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/3642990955 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d6e130 0x55baa4d4c400 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.705+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/3642990955 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d29820 0x55baa4d80c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.705+0000 7fcc4aa00640 0 client.7935 ms_handle_reset on v2:172.21.15.7:6833/4110623302 2024-09-10T21:51:20.710 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.705+0000 7fcc6fa88740 1 -- 172.21.15.7:0/3642990955 >> 172.21.15.7:0/3642990955 conn(0x55baa4d292a0 msgr2=0x55baa4d21fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:20.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.705+0000 7fcc6fa88740 1 -- 172.21.15.7:0/3642990955 shutdown_connections 2024-09-10T21:51:20.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.705+0000 7fcc6fa88740 1 -- 172.21.15.7:0/3642990955 wait complete. 2024-09-10T21:51:20.711 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.705+0000 7fcc6fa88740 20 client.7935 trim_cache size 0 max 16384 2024-09-10T21:51:20.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.723+0000 7fcc6fa88740 1 Processor -- start 2024-09-10T21:51:20.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.723+0000 7fcc6fa88740 1 -- start start 2024-09-10T21:51:20.735 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.724+0000 7fcc6fa88740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d29820 0x55baa4d22960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:20.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.724+0000 7fcc6fa88740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d52000 0x55baa4d45d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:20.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.724+0000 7fcc6fa88740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d6e130 0x55baa4d27100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:20.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.724+0000 7fcc6fa88740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc68201510 con 0x55baa4d52000 2024-09-10T21:51:20.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.724+0000 7fcc6fa88740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcc681f4ca0 con 0x55baa4d29820 2024-09-10T21:51:20.736 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.724+0000 7fcc6fa88740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fcc680b3790 con 0x55baa4d6e130 2024-09-10T21:51:20.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.724+0000 7fcc5d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d52000 0x55baa4d45d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:20.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.724+0000 7fcc5d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d52000 0x55baa4d45d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44654/0 (socket says 172.21.15.7:44654) 2024-09-10T21:51:20.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.725+0000 7fcc5d400640 1 -- 172.21.15.7:0/3196731878 learned_addr learned my addr 172.21.15.7:0/3196731878 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:20.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.725+0000 7fcc5d400640 1 -- 172.21.15.7:0/3196731878 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d6e130 msgr2=0x55baa4d27100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:20.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.725+0000 7fcc66800640 1 --2- 172.21.15.7:0/3196731878 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d6e130 0x55baa4d27100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:20.737 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.725+0000 7fcc65e00640 1 --2- 172.21.15.7:0/3196731878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d29820 0x55baa4d22960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:20.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.725+0000 7fcc5d400640 1 --2- 172.21.15.7:0/3196731878 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d6e130 0x55baa4d27100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.725+0000 7fcc5d400640 1 -- 172.21.15.7:0/3196731878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d29820 msgr2=0x55baa4d22960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:20.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.725+0000 7fcc5d400640 1 --2- 172.21.15.7:0/3196731878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d29820 0x55baa4d22960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.725+0000 7fcc5d400640 1 -- 172.21.15.7:0/3196731878 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baa4d6fba0 con 0x55baa4d52000 2024-09-10T21:51:20.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.725+0000 7fcc65e00640 1 --2- 172.21.15.7:0/3196731878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d29820 0x55baa4d22960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:51:20.738 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.725+0000 7fcc5d400640 1 --2- 172.21.15.7:0/3196731878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d52000 0x55baa4d45d70 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fcc50000c00 tx=0x7fcc50010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32722e4b02db818 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.725+0000 7fcc55c00640 1 -- 172.21.15.7:0/3196731878 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc50019040 con 0x55baa4d52000 2024-09-10T21:51:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.725+0000 7fcc55c00640 1 -- 172.21.15.7:0/3196731878 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc50015550 con 0x55baa4d52000 2024-09-10T21:51:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.725+0000 7fcc55c00640 1 -- 172.21.15.7:0/3196731878 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc500067d0 con 0x55baa4d52000 2024-09-10T21:51:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.726+0000 7fcc6fa88740 1 -- 172.21.15.7:0/3196731878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d52000 msgr2=0x55baa4d45d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:20.739 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.726+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/3196731878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d52000 0x55baa4d45d70 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fcc50000c00 tx=0x7fcc50010b60 comp rx=0 tx=0).stop 2024-09-10T21:51:20.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.726+0000 7fcc6fa88740 1 -- 172.21.15.7:0/3196731878 shutdown_connections 2024-09-10T21:51:20.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.726+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/3196731878 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d6e130 0x55baa4d27100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.726+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/3196731878 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d52000 0x55baa4d45d70 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.726+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/3196731878 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d29820 0x55baa4d22960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.726+0000 7fcc6fa88740 1 -- 172.21.15.7:0/3196731878 >> 172.21.15.7:0/3196731878 conn(0x55baa4d292a0 msgr2=0x55baa4d69790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:20.740 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.726+0000 7fcc6fa88740 1 -- 172.21.15.7:0/3196731878 shutdown_connections 2024-09-10T21:51:20.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.726+0000 7fcc6fa88740 1 -- 172.21.15.7:0/3196731878 wait complete. 2024-09-10T21:51:20.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.727+0000 7fcc6fa88740 1 -- start start 2024-09-10T21:51:20.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.727+0000 7fcc6fa88740 1 Processor -- start 2024-09-10T21:51:20.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.728+0000 7fcc6fa88740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d29820 0x55baa4df0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:20.741 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.728+0000 7fcc6fa88740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d52000 0x55baa4d23240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.728+0000 7fcc6fa88740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d6e130 0x55baa4d23780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.728+0000 7fcc6fa88740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55baa4d7d4c0 con 0x55baa4d6e130 2024-09-10T21:51:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.728+0000 7fcc6fa88740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcc681f8c50 con 0x55baa4d29820 2024-09-10T21:51:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.728+0000 7fcc6fa88740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55baa4d695f0 con 0x55baa4d52000 2024-09-10T21:51:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.728+0000 7fcc6fa88740 10 client.0 _ll_register_callbacks cb 0x7fcc50004970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:51:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.728+0000 7fcc66800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d6e130 0x55baa4d23780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:20.742 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.728+0000 7fcc66800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d6e130 0x55baa4d23780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44664/0 (socket says 172.21.15.7:44664) 2024-09-10T21:51:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.728+0000 7fcc66800640 1 -- 172.21.15.7:0/768984693 learned_addr learned my addr 172.21.15.7:0/768984693 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.728+0000 7fcc5d400640 1 --2- 172.21.15.7:0/768984693 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d52000 0x55baa4d23240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.728+0000 7fcc65e00640 1 --2- 172.21.15.7:0/768984693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d29820 0x55baa4df0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.729+0000 7fcc66800640 1 -- 172.21.15.7:0/768984693 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d52000 msgr2=0x55baa4d23240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:20.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.729+0000 7fcc66800640 1 --2- 172.21.15.7:0/768984693 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d52000 0x55baa4d23240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.729+0000 7fcc66800640 1 -- 172.21.15.7:0/768984693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d29820 msgr2=0x55baa4df0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:20.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.729+0000 7fcc66800640 1 --2- 172.21.15.7:0/768984693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d29820 0x55baa4df0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.729+0000 7fcc66800640 1 -- 172.21.15.7:0/768984693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baa4d4bdf0 con 0x55baa4d6e130 2024-09-10T21:51:20.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.729+0000 7fcc66800640 1 --2- 172.21.15.7:0/768984693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d6e130 0x55baa4d23780 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fcc60083220 tx=0x7fcc600b9780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27a0179660593f4d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:20.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.729+0000 7fcc4aa00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:51:20.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.729+0000 7fcc4aa00640 1 -- 172.21.15.7:0/768984693 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc600a91e0 con 0x55baa4d6e130 2024-09-10T21:51:20.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.729+0000 7fcc4aa00640 1 -- 172.21.15.7:0/768984693 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7fcc600a9380 con 0x55baa4d6e130 2024-09-10T21:51:20.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.729+0000 7fcc6fa88740 1 -- 172.21.15.7:0/768984693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc50015550 con 0x55baa4d6e130 2024-09-10T21:51:20.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.729+0000 7fcc4aa00640 1 -- 172.21.15.7:0/768984693 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fcc60004b40 con 0x55baa4d6e130 2024-09-10T21:51:20.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.729+0000 7fcc6fa88740 10 client.7941 fetch_fsmap learned FSMap version 91 2024-09-10T21:51:20.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.729+0000 7fcc6fa88740 1 -- 172.21.15.7:0/768984693 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=91,monmap=2+}) -- 0x55baa4d4bdf0 con 0x55baa4d6e130 2024-09-10T21:51:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.730+0000 7fcc4aa00640 1 -- 172.21.15.7:0/768984693 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc50015550 con 0x55baa4d6e130 2024-09-10T21:51:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.730+0000 7fcc4aa00640 1 -- 172.21.15.7:0/768984693 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 91) ==== 3502+0+0 (secure 0 0 0) 0x7fcc60004ce0 con 0x55baa4d6e130 2024-09-10T21:51:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.730+0000 7fcc6fa88740 10 client.7941 fetch_fsmap finished waiting for FSMap version 91 2024-09-10T21:51:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.730+0000 7fcc6fa88740 10 client.7941 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.c 2024-09-10T21:51:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.730+0000 7fcc6fa88740 20 client.7941 populate_metadata read hostname 'smithi007' 2024-09-10T21:51:20.746 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.730+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/768984693 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55baa4d80bd0 0x55baa4d8f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.730+0000 7fcc6fa88740 4 client.7941 mds_command: new command op to 7791 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-10T21:51:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.730+0000 7fcc6fa88740 1 -- 172.21.15.7:0/768984693 --> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55baa4c99270 con 0x55baa4d80bd0 2024-09-10T21:51:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.730+0000 7fcc65e00640 1 --2- 172.21.15.7:0/768984693 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55baa4d80bd0 0x55baa4d8f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.731+0000 7fcc65e00640 1 --2- 172.21.15.7:0/768984693 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55baa4d80bd0 0x55baa4d8f910 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c310b489ef39e4d2 server_cookie=ffbb140ea75b51ca in_seq=0 out_seq=0 2024-09-10T21:51:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.731+0000 7fcc4aa00640 10 client.7941 ms_handle_connect on v2:172.21.15.7:6833/4110623302 2024-09-10T21:51:20.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.731+0000 7fcc4aa00640 1 -- 172.21.15.7:0/768984693 <== mds.0 v2:172.21.15.7:6833/4110623302 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x55baa4c99270 con 0x55baa4d80bd0 2024-09-10T21:51:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.731+0000 7fcc4aa00640 10 client.7941 handle_command_reply: tid=0 2024-09-10T21:51:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.731+0000 7fcc6fa88740 1 client.7941 shutdown 2024-09-10T21:51:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.731+0000 7fcc6fa88740 1 -- 172.21.15.7:0/768984693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d6e130 msgr2=0x55baa4d23780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.731+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/768984693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d6e130 0x55baa4d23780 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fcc60083220 tx=0x7fcc600b9780 comp rx=0 tx=0).stop 2024-09-10T21:51:20.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.731+0000 7fcc6fa88740 1 -- 172.21.15.7:0/768984693 shutdown_connections 2024-09-10T21:51:20.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.731+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/768984693 >> [v2:172.21.15.7:6833/4110623302,v1:172.21.15.7:6835/4110623302] conn(0x55baa4d80bd0 0x55baa4d8f910 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.731+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/768984693 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55baa4d6e130 0x55baa4d23780 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.731+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/768984693 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55baa4d52000 0x55baa4d23240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.731+0000 7fcc6fa88740 1 --2- 172.21.15.7:0/768984693 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55baa4d29820 0x55baa4df0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.731+0000 7fcc6fa88740 1 -- 172.21.15.7:0/768984693 >> 172.21.15.7:0/768984693 conn(0x55baa4d292a0 msgr2=0x55baa4d27d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:20.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.731+0000 7fcc4aa00640 0 client.7941 ms_handle_reset on v2:172.21.15.7:6833/4110623302 2024-09-10T21:51:20.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.731+0000 7fcc6fa88740 1 -- 172.21.15.7:0/768984693 shutdown_connections 2024-09-10T21:51:20.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.731+0000 7fcc6fa88740 1 -- 172.21.15.7:0/768984693 wait complete. 2024-09-10T21:51:20.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.731+0000 7fcc6fa88740 20 client.7941 trim_cache size 0 max 16384 2024-09-10T21:51:20.751 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:51:20.751 INFO:teuthology.orchestra.run.smithi007.stdout: "status": "no active scrubs running", 2024-09-10T21:51:20.751 INFO:teuthology.orchestra.run.smithi007.stdout: "scrubs": {}, 2024-09-10T21:51:20.751 INFO:teuthology.orchestra.run.smithi007.stdout: "scrub_stats": { 2024-09-10T21:51:20.751 INFO:teuthology.orchestra.run.smithi007.stdout: "5b7aaafc-a7c4-4a78-8ebe-9508949e5efb": { 2024-09-10T21:51:20.752 INFO:teuthology.orchestra.run.smithi007.stdout: "start_time": "2024-09-10T21:50:50.266548+0000", 2024-09-10T21:51:20.752 INFO:teuthology.orchestra.run.smithi007.stdout: "path": "/", 2024-09-10T21:51:20.752 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_started": 0, 2024-09-10T21:51:20.752 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_passed": 0, 2024-09-10T21:51:20.752 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_failed": 0, 2024-09-10T21:51:20.752 INFO:teuthology.orchestra.run.smithi007.stdout: "uninline_skipped": 0 2024-09-10T21:51:20.753 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:20.753 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:20.753 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:51:20.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.732+0000 7fcc6cc00640 1 -- 172.21.15.7:0/2851046097 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcc44077f70 msgr2=0x7fcc4407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:20.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.732+0000 7fcc6cc00640 1 --2- 172.21.15.7:0/2851046097 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcc44077f70 0x7fcc4407a430 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fcc580098e0 tx=0x7fcc58009060 comp rx=0 tx=0).stop 2024-09-10T21:51:20.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.732+0000 7fcc6cc00640 1 -- 172.21.15.7:0/2851046097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc68155fb0 msgr2=0x7fcc6815b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:20.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.732+0000 7fcc6cc00640 1 --2- 172.21.15.7:0/2851046097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc68155fb0 0x7fcc6815b000 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fcc60006010 tx=0x7fcc60003f10 comp rx=0 tx=0).stop 2024-09-10T21:51:20.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.732+0000 7fcc6cc00640 1 -- 172.21.15.7:0/2851046097 shutdown_connections 2024-09-10T21:51:20.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.732+0000 7fcc6cc00640 1 --2- 172.21.15.7:0/2851046097 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcc44077f70 0x7fcc4407a430 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.732+0000 7fcc6cc00640 1 --2- 172.21.15.7:0/2851046097 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc68074450 0x7fcc6815aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.732+0000 7fcc6cc00640 1 --2- 172.21.15.7:0/2851046097 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc68155fb0 0x7fcc6815b000 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.732+0000 7fcc6cc00640 1 --2- 172.21.15.7:0/2851046097 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc6806fcf0 0x7fcc680b4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:20.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.732+0000 7fcc6cc00640 1 -- 172.21.15.7:0/2851046097 >> 172.21.15.7:0/2851046097 conn(0x7fcc680b16d0 msgr2=0x7fcc680757f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:20.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.732+0000 7fcc6cc00640 1 -- 172.21.15.7:0/2851046097 shutdown_connections 2024-09-10T21:51:20.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:20.732+0000 7fcc6cc00640 1 -- 172.21.15.7:0/2851046097 wait complete. 2024-09-10T21:51:20.757 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-10T21:51:20.859 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:51:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.020+0000 7f826a600640 1 -- 172.21.15.7:0/1151875079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f826406e510 msgr2=0x7f8264073cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.020+0000 7f826a600640 1 --2- 172.21.15.7:0/1151875079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f826406e510 0x7f8264073cd0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f82640b2620 tx=0x7f824c02a4d0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.021+0000 7f826a600640 1 -- 172.21.15.7:0/1151875079 shutdown_connections 2024-09-10T21:51:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.021+0000 7f826a600640 1 --2- 172.21.15.7:0/1151875079 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f826406e510 0x7f8264073cd0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.021+0000 7f826a600640 1 --2- 172.21.15.7:0/1151875079 >> v2:172.21.15.133:3300/0 conn(0x7f826406ff50 0x7f826415cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.021+0000 7f826a600640 1 -- 172.21.15.7:0/1151875079 >> 172.21.15.7:0/1151875079 conn(0x7f82640b1660 msgr2=0x7f82640b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:21.022 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.021+0000 7f826a600640 1 -- 172.21.15.7:0/1151875079 shutdown_connections 2024-09-10T21:51:21.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.021+0000 7f826a600640 1 -- 172.21.15.7:0/1151875079 wait complete. 2024-09-10T21:51:21.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.022+0000 7f826a600640 1 Processor -- start 2024-09-10T21:51:21.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.022+0000 7f826a600640 1 -- start start 2024-09-10T21:51:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.023+0000 7f826a600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f826406e510 0x7f82640b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.023+0000 7f826a600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f826406e960 0x7f82640b2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.023+0000 7f826a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f826406ff50 0x7f82641f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:21.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.023+0000 7f826a600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f82640745a0 con 0x7f826406ff50 2024-09-10T21:51:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.023+0000 7f826a600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8264074420 con 0x7f826406e510 2024-09-10T21:51:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.023+0000 7f826a600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f8264074720 con 0x7f826406e960 2024-09-10T21:51:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.024+0000 7f8263e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f826406ff50 0x7f82641f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.024+0000 7f8263400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f826406e510 0x7f82640b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.024+0000 7f8263400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f826406e510 0x7f82640b5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:40552/0 (socket says 172.21.15.7:40552) 2024-09-10T21:51:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.024+0000 7f8263400640 1 -- 172.21.15.7:0/2949257770 learned_addr learned my addr 172.21.15.7:0/2949257770 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.024+0000 7f825aa00640 1 --2- 172.21.15.7:0/2949257770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f826406e960 0x7f82640b2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.024+0000 7f8263400640 1 -- 172.21.15.7:0/2949257770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f826406e960 msgr2=0x7f82640b2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.024+0000 7f8263400640 1 --2- 172.21.15.7:0/2949257770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f826406e960 0x7f82640b2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.024+0000 7f8263400640 1 -- 172.21.15.7:0/2949257770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f826406ff50 msgr2=0x7f82641f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.024+0000 7f8263400640 1 --2- 172.21.15.7:0/2949257770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f826406ff50 0x7f82641f40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.026 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.024+0000 7f8263400640 1 -- 172.21.15.7:0/2949257770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8264155370 con 0x7f826406e510 2024-09-10T21:51:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.024+0000 7f8263e00640 1 --2- 172.21.15.7:0/2949257770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f826406ff50 0x7f82641f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:51:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.025+0000 7f8263400640 1 --2- 172.21.15.7:0/2949257770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f826406e510 0x7f82640b5360 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f825400d770 tx=0x7f825400dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.025+0000 7f825be00640 1 -- 172.21.15.7:0/2949257770 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254017070 con 0x7f826406e510 2024-09-10T21:51:21.027 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.025+0000 7f825be00640 1 -- 172.21.15.7:0/2949257770 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8254009d20 con 0x7f826406e510 2024-09-10T21:51:21.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.025+0000 7f826a600640 1 -- 172.21.15.7:0/2949257770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8264155650 con 0x7f826406e510 2024-09-10T21:51:21.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.025+0000 7f825be00640 1 -- 172.21.15.7:0/2949257770 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8254004280 con 0x7f826406e510 2024-09-10T21:51:21.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.025+0000 7f826a600640 1 -- 172.21.15.7:0/2949257770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82640c53e0 con 0x7f826406e510 2024-09-10T21:51:21.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.026+0000 7f826a600640 1 -- 172.21.15.7:0/2949257770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82641600d0 con 0x7f826406e510 2024-09-10T21:51:21.028 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.027+0000 7f825be00640 1 -- 172.21.15.7:0/2949257770 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8254005020 con 0x7f826406e510 2024-09-10T21:51:21.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.028+0000 7f825be00640 1 --2- 172.21.15.7:0/2949257770 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8240077f70 0x7f824007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:21.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.028+0000 7f825be00640 1 -- 172.21.15.7:0/2949257770 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5795+0+0 (secure 0 0 0) 0x7f825409a320 con 0x7f826406e510 2024-09-10T21:51:21.029 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.029+0000 7f825aa00640 1 --2- 172.21.15.7:0/2949257770 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8240077f70 0x7f824007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:21.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.030+0000 7f825aa00640 1 --2- 172.21.15.7:0/2949257770 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8240077f70 0x7f824007a430 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f824c033000 tx=0x7f824c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:21.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.032+0000 7f825be00640 1 -- 172.21.15.7:0/2949257770 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8254020030 con 0x7f826406e510 2024-09-10T21:51:21.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.265+0000 7f826a600640 1 -- 172.21.15.7:0/2949257770 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f82640b5960 con 0x7f826406e510 2024-09-10T21:51:21.267 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-10T21:51:21.267+0000 7feb3fa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:51:21.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.275+0000 7f825be00640 1 -- 172.21.15.7:0/2949257770 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v92) ==== 161+0+0 (secure 0 0 0) 0x7f8254062760 con 0x7f826406e510 2024-09-10T21:51:21.276 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:51:21.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.278+0000 7f826a600640 1 -- 172.21.15.7:0/2949257770 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8240077f70 msgr2=0x7f824007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:21.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.278+0000 7f826a600640 1 --2- 172.21.15.7:0/2949257770 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8240077f70 0x7f824007a430 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f824c033000 tx=0x7f824c004730 comp rx=0 tx=0).stop 2024-09-10T21:51:21.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.278+0000 7f826a600640 1 -- 172.21.15.7:0/2949257770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f826406e510 msgr2=0x7f82640b5360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:21.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.278+0000 7f826a600640 1 --2- 172.21.15.7:0/2949257770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f826406e510 0x7f82640b5360 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f825400d770 tx=0x7f825400dc30 comp rx=0 tx=0).stop 2024-09-10T21:51:21.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.278+0000 7f826a600640 1 -- 172.21.15.7:0/2949257770 shutdown_connections 2024-09-10T21:51:21.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.278+0000 7f826a600640 1 --2- 172.21.15.7:0/2949257770 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8240077f70 0x7f824007a430 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.278+0000 7f826a600640 1 --2- 172.21.15.7:0/2949257770 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f826406ff50 0x7f82641f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.278+0000 7f826a600640 1 --2- 172.21.15.7:0/2949257770 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f826406e960 0x7f82640b2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.278+0000 7f826a600640 1 --2- 172.21.15.7:0/2949257770 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f826406e510 0x7f82640b5360 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.278+0000 7f826a600640 1 -- 172.21.15.7:0/2949257770 >> 172.21.15.7:0/2949257770 conn(0x7f82640b1660 msgr2=0x7f826415dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:21.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.278+0000 7f826a600640 1 -- 172.21.15.7:0/2949257770 shutdown_connections 2024-09-10T21:51:21.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.278+0000 7f826a600640 1 -- 172.21.15.7:0/2949257770 wait complete. 2024-09-10T21:51:21.281 INFO:tasks.ceph.mds.c.smithi007.stderr: -1190> 2024-09-10T21:50:50.268+0000 7f55b5a00640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000001 2024-09-10T21:51:21.281 INFO:tasks.ceph.mds.c.smithi007.stderr: -1186> 2024-09-10T21:50:50.268+0000 7f55b5a00640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000001 2024-09-10T21:51:21.281 INFO:tasks.ceph.mds.c.smithi007.stderr: -1167> 2024-09-10T21:50:50.268+0000 7f55b5a00640 -1 log_channel(cluster) log [ERR] : scrub: inode wrongly marked free: 0x10000000002 2024-09-10T21:51:21.281 INFO:tasks.ceph.mds.c.smithi007.stderr: -1164> 2024-09-10T21:50:50.268+0000 7f55b5a00640 -1 log_channel(cluster) log [ERR] : inode table repaired for inode: 0x10000000002 2024-09-10T21:51:21.281 INFO:tasks.ceph.mds.c.smithi007.stderr: -1067> 2024-09-10T21:50:50.298+0000 7f55baa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:51:21.295 DEBUG:teuthology.orchestra.run.smithi007:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 0 show inode 2024-09-10T21:51:21.324 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-09-10T21:51:21.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.370+0000 7f00edd5b280 1 -- start start 2024-09-10T21:51:21.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.370+0000 7f00edd5b280 1 Processor -- start 2024-09-10T21:51:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.371+0000 7f00edd5b280 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a73dd0ac00 0x55a73eaea000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.371+0000 7f00edd5b280 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73dd0b800 0x55a73eaea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.371+0000 7f00edd5b280 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a73dd0bc00 0x55a73eaeab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.371+0000 7f00edd5b280 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a73eafe600 con 0x55a73dd0b800 2024-09-10T21:51:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.371+0000 7f00edd5b280 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a73eafe480 con 0x55a73dd0ac00 2024-09-10T21:51:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.371+0000 7f00edd5b280 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55a73eafe300 con 0x55a73dd0bc00 2024-09-10T21:51:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.371+0000 7f00eb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73dd0b800 0x55a73eaea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.371+0000 7f00eb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73dd0b800 0x55a73eaea580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44708/0 (socket says 172.21.15.7:44708) 2024-09-10T21:51:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.371+0000 7f00eb400640 1 -- 172.21.15.7:0/3193146860 learned_addr learned my addr 172.21.15.7:0/3193146860 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.371+0000 7f00ec800640 1 --2- 172.21.15.7:0/3193146860 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a73dd0bc00 0x55a73eaeab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:21.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.371+0000 7f00ebe00640 1 --2- 172.21.15.7:0/3193146860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a73dd0ac00 0x55a73eaea000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.371+0000 7f00eb400640 1 -- 172.21.15.7:0/3193146860 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a73dd0bc00 msgr2=0x55a73eaeab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.371+0000 7f00eb400640 1 --2- 172.21.15.7:0/3193146860 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a73dd0bc00 0x55a73eaeab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.371+0000 7f00eb400640 1 -- 172.21.15.7:0/3193146860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a73dd0ac00 msgr2=0x55a73eaea000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.371+0000 7f00eb400640 1 --2- 172.21.15.7:0/3193146860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a73dd0ac00 0x55a73eaea000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.371+0000 7f00eb400640 1 -- 172.21.15.7:0/3193146860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a73dd42780 con 0x55a73dd0b800 2024-09-10T21:51:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.372+0000 7f00eb400640 1 --2- 172.21.15.7:0/3193146860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73dd0b800 0x55a73eaea580 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x55a73eb3b710 tx=0x55a73ddf4120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b552764d9c62a215 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.372+0000 7f00e8c00640 1 -- 172.21.15.7:0/3193146860 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a73dd30340 con 0x55a73dd0b800 2024-09-10T21:51:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.372+0000 7f00e8c00640 1 -- 172.21.15.7:0/3193146860 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55a73dd36fc0 con 0x55a73dd0b800 2024-09-10T21:51:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.372+0000 7f00e8c00640 1 -- 172.21.15.7:0/3193146860 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a73dd30ea0 con 0x55a73dd0b800 2024-09-10T21:51:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.372+0000 7f00edd5b280 1 -- 172.21.15.7:0/3193146860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a73dd434a0 con 0x55a73dd0b800 2024-09-10T21:51:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.372+0000 7f00edd5b280 1 -- 172.21.15.7:0/3193146860 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=0}) -- 0x55a73dd42000 con 0x55a73dd0b800 2024-09-10T21:51:21.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.372+0000 7f00edd5b280 4 waiting for MDS map... 2024-09-10T21:51:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.375+0000 7f00e8c00640 1 -- 172.21.15.7:0/3193146860 <== mon.0 v2:172.21.15.7:3300/0 4 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x55a73ebfa200 con 0x55a73dd0b800 2024-09-10T21:51:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.375+0000 7f00e8c00640 1 -- 172.21.15.7:0/3193146860 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 92) ==== 2893+0+0 (secure 0 0 0) 0x55a73dd65180 con 0x55a73dd0b800 2024-09-10T21:51:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.375+0000 7f00edd5b280 4 Got MDS map 92 2024-09-10T21:51:21.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.375+0000 7f00edd5b280 10 main: main 2024-09-10T21:51:21.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.376+0000 7f00edd5b280 4 main: connecting to RADOS... 2024-09-10T21:51:21.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.376+0000 7f00edd5b280 1 Processor -- start 2024-09-10T21:51:21.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.376+0000 7f00edd5b280 1 -- start start 2024-09-10T21:51:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.377+0000 7f00edd5b280 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a73ec3a000 0x55a73eaeb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.377+0000 7f00edd5b280 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a73ec3a400 0x55a73eaeb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.377+0000 7f00edd5b280 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73ec3a800 0x55a73eaebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.377+0000 7f00edd5b280 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a73eafe180 con 0x55a73ec3a800 2024-09-10T21:51:21.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.377+0000 7f00edd5b280 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a73dd6d380 con 0x55a73ec3a400 2024-09-10T21:51:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.377+0000 7f00edd5b280 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55a73dd6d800 con 0x55a73ec3a000 2024-09-10T21:51:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.377+0000 7f00eb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73ec3a800 0x55a73eaebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.377+0000 7f00ec800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a73ec3a000 0x55a73eaeb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.377+0000 7f00eb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73ec3a800 0x55a73eaebb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44722/0 (socket says 172.21.15.7:44722) 2024-09-10T21:51:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.378+0000 7f00eb400640 1 -- 172.21.15.7:0/3649740147 learned_addr learned my addr 172.21.15.7:0/3649740147 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:21.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.378+0000 7f00ebe00640 1 --2- 172.21.15.7:0/3649740147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a73ec3a400 0x55a73eaeb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:21.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.378+0000 7f00eb400640 1 -- 172.21.15.7:0/3649740147 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a73ec3a000 msgr2=0x55a73eaeb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:21.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.378+0000 7f00eb400640 1 --2- 172.21.15.7:0/3649740147 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a73ec3a000 0x55a73eaeb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.378+0000 7f00eb400640 1 -- 172.21.15.7:0/3649740147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a73ec3a400 msgr2=0x55a73eaeb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.378+0000 7f00eb400640 1 --2- 172.21.15.7:0/3649740147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a73ec3a400 0x55a73eaeb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.378+0000 7f00eb400640 1 -- 172.21.15.7:0/3649740147 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a73dd423c0 con 0x55a73ec3a800 2024-09-10T21:51:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.378+0000 7f00eb400640 1 --2- 172.21.15.7:0/3649740147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73ec3a800 0x55a73eaebb80 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x55a73eb201b0 tx=0x55a73ddf5f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd7de42a322f6fe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.378+0000 7f00e6400640 1 -- 172.21.15.7:0/3649740147 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a73dd31380 con 0x55a73ec3a800 2024-09-10T21:51:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.378+0000 7f00e6400640 1 -- 172.21.15.7:0/3649740147 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55a73dd36380 con 0x55a73ec3a800 2024-09-10T21:51:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.378+0000 7f00e6400640 1 -- 172.21.15.7:0/3649740147 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a73dd311e0 con 0x55a73ec3a800 2024-09-10T21:51:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.379+0000 7f00edd5b280 1 -- 172.21.15.7:0/3649740147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73ec3a800 msgr2=0x55a73eaebb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.379+0000 7f00edd5b280 1 --2- 172.21.15.7:0/3649740147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73ec3a800 0x55a73eaebb80 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x55a73eb201b0 tx=0x55a73ddf5f80 comp rx=0 tx=0).stop 2024-09-10T21:51:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.379+0000 7f00edd5b280 1 -- 172.21.15.7:0/3649740147 shutdown_connections 2024-09-10T21:51:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.379+0000 7f00edd5b280 1 --2- 172.21.15.7:0/3649740147 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a73ec3a400 0x55a73eaeb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.379+0000 7f00edd5b280 1 --2- 172.21.15.7:0/3649740147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73ec3a800 0x55a73eaebb80 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.379+0000 7f00edd5b280 1 --2- 172.21.15.7:0/3649740147 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a73ec3a000 0x55a73eaeb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.379+0000 7f00edd5b280 1 -- 172.21.15.7:0/3649740147 >> 172.21.15.7:0/3649740147 conn(0x55a73eb03c00 msgr2=0x55a73de18800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:21.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.379+0000 7f00edd5b280 1 -- 172.21.15.7:0/3649740147 shutdown_connections 2024-09-10T21:51:21.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.379+0000 7f00edd5b280 1 -- 172.21.15.7:0/3649740147 wait complete. 2024-09-10T21:51:21.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.380+0000 7f00edd5b280 1 Processor -- start 2024-09-10T21:51:21.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.380+0000 7f00edd5b280 1 -- start start 2024-09-10T21:51:21.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.380+0000 7f00edd5b280 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73ec3a000 0x55a73eaeb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.380+0000 7f00edd5b280 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a73ec3a400 0x55a73eaeb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.380+0000 7f00edd5b280 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a73ec3a800 0x55a73eaebb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.380+0000 7f00edd5b280 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a73eaff380 con 0x55a73ec3a000 2024-09-10T21:51:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.381+0000 7f00edd5b280 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a73eaff200 con 0x55a73ec3a400 2024-09-10T21:51:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.381+0000 7f00edd5b280 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55a73eaff500 con 0x55a73ec3a800 2024-09-10T21:51:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.381+0000 7f00ec800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73ec3a000 0x55a73eaeb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.381+0000 7f00ec800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73ec3a000 0x55a73eaeb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44728/0 (socket says 172.21.15.7:44728) 2024-09-10T21:51:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.381+0000 7f00ec800640 1 -- 172.21.15.7:0/3510747224 learned_addr learned my addr 172.21.15.7:0/3510747224 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.381+0000 7f00ebe00640 1 --2- 172.21.15.7:0/3510747224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a73ec3a400 0x55a73eaeb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.381+0000 7f00eb400640 1 --2- 172.21.15.7:0/3510747224 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a73ec3a800 0x55a73eaebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.381+0000 7f00ec800640 1 -- 172.21.15.7:0/3510747224 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a73ec3a800 msgr2=0x55a73eaebb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:21.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.381+0000 7f00ec800640 1 --2- 172.21.15.7:0/3510747224 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a73ec3a800 0x55a73eaebb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.381+0000 7f00ec800640 1 -- 172.21.15.7:0/3510747224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a73ec3a400 msgr2=0x55a73eaeb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.381+0000 7f00ec800640 1 --2- 172.21.15.7:0/3510747224 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a73ec3a400 0x55a73eaeb600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.381+0000 7f00ec800640 1 -- 172.21.15.7:0/3510747224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a73dd42960 con 0x55a73ec3a000 2024-09-10T21:51:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.382+0000 7f00ec800640 1 --2- 172.21.15.7:0/3510747224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73ec3a000 0x55a73eaeb080 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x55a73eb216e0 tx=0x55a73ec427e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.382+0000 7f00e4600640 1 -- 172.21.15.7:0/3510747224 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a73dd31380 con 0x55a73ec3a000 2024-09-10T21:51:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.382+0000 7f00edd5b280 1 -- 172.21.15.7:0/3510747224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a73dd43680 con 0x55a73ec3a000 2024-09-10T21:51:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.382+0000 7f00e4600640 1 -- 172.21.15.7:0/3510747224 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x55a73dd36e00 con 0x55a73ec3a000 2024-09-10T21:51:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.382+0000 7f00e4600640 1 -- 172.21.15.7:0/3510747224 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55a73dd311e0 con 0x55a73ec3a000 2024-09-10T21:51:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.382+0000 7f00edd5b280 1 -- 172.21.15.7:0/3510747224 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a73dd43860 con 0x55a73ec3a000 2024-09-10T21:51:21.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.382+0000 7f00edd5b280 4 main: resolving pool 14 2024-09-10T21:51:21.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.384+0000 7f00e4600640 1 -- 172.21.15.7:0/3510747224 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x55a73dd65500 con 0x55a73ec3a000 2024-09-10T21:51:21.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.384+0000 7f00e4600640 1 --2- 172.21.15.7:0/3510747224 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x55a73ec48c00 0x55a73eaec100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:21.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.384+0000 7f00e4600640 1 -- 172.21.15.7:0/3510747224 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x55a73ec41000 con 0x55a73ec3a000 2024-09-10T21:51:21.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.384+0000 7f00edd5b280 4 main: creating IoCtx.. 2024-09-10T21:51:21.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.384+0000 7f00edd5b280 1 --2- 172.21.15.7:0/3510747224 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x55a73ec3ac00 0x55a73eaec680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:21.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.384+0000 7f00edd5b280 1 -- 172.21.15.7:0/3510747224 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:1 14.13 14:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e103) -- 0x55a73ec3b000 con 0x55a73ec3ac00 2024-09-10T21:51:21.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.384+0000 7f00ebe00640 1 --2- 172.21.15.7:0/3510747224 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x55a73ec48c00 0x55a73eaec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:21.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.385+0000 7f00eb400640 1 --2- 172.21.15.7:0/3510747224 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x55a73ec3ac00 0x55a73eaec680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:21.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.385+0000 7f00ebe00640 1 --2- 172.21.15.7:0/3510747224 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x55a73ec48c00 0x55a73eaec100 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x55a73eaa4e40 tx=0x55a73ec42d20 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:21.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.385+0000 7f00eb400640 1 --2- 172.21.15.7:0/3510747224 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x55a73ec3ac00 0x55a73eaec680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:21.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.388+0000 7f00eb400640 1 -- 172.21.15.7:0/3510747224 <== osd.2 v2:172.21.15.7:6816/4238877366 1 ==== osd_op_reply(1 mds0_inotable [read 0~34 out=34b] v0'0 uv7 ondisk = 0) ==== 157+0+34 (crc 0 0 0) 0x55a73de8c000 con 0x55a73ec3ac00 2024-09-10T21:51:21.389 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:51:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: "0": { 2024-09-10T21:51:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: "data": { 2024-09-10T21:51:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: "version": 5, 2024-09-10T21:51:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: "inotable": { 2024-09-10T21:51:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: "projected_free": [ 2024-09-10T21:51:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: "start": 1099511627779, 2024-09-10T21:51:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: "len": 1099511627773 2024-09-10T21:51:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:21.389 INFO:teuthology.orchestra.run.smithi007.stdout: ], 2024-09-10T21:51:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: "free": [ 2024-09-10T21:51:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: "start": 1099511627779, 2024-09-10T21:51:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: "len": 1099511627773 2024-09-10T21:51:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T21:51:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:51:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: "result": 0 2024-09-10T21:51:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:21.390 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:51:21.390 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.388+0000 7f00edd5b280 1 -- 172.21.15.7:0/3510747224 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x55a73ec3ac00 msgr2=0x55a73eaec680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.388+0000 7f00edd5b280 1 --2- 172.21.15.7:0/3510747224 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x55a73ec3ac00 0x55a73eaec680 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.388+0000 7f00edd5b280 1 -- 172.21.15.7:0/3510747224 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x55a73ec48c00 msgr2=0x55a73eaec100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.388+0000 7f00edd5b280 1 --2- 172.21.15.7:0/3510747224 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x55a73ec48c00 0x55a73eaec100 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x55a73eaa4e40 tx=0x55a73ec42d20 comp rx=0 tx=0).stop 2024-09-10T21:51:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.388+0000 7f00edd5b280 1 -- 172.21.15.7:0/3510747224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73ec3a000 msgr2=0x55a73eaeb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.388+0000 7f00edd5b280 1 --2- 172.21.15.7:0/3510747224 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73ec3a000 0x55a73eaeb080 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x55a73eb216e0 tx=0x55a73ec427e0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.388+0000 7f00ebe00640 1 -- 172.21.15.7:0/3510747224 reap_dead start 2024-09-10T21:51:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.388+0000 7f00edd5b280 1 -- 172.21.15.7:0/3510747224 shutdown_connections 2024-09-10T21:51:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.388+0000 7f00edd5b280 1 -- 172.21.15.7:0/3510747224 >> 172.21.15.7:0/3510747224 conn(0x55a73eb03c00 msgr2=0x55a73de18800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.388+0000 7f00edd5b280 1 -- 172.21.15.7:0/3510747224 shutdown_connections 2024-09-10T21:51:21.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.388+0000 7f00edd5b280 1 -- 172.21.15.7:0/3510747224 wait complete. 2024-09-10T21:51:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.389+0000 7f00edd5b280 1 -- 172.21.15.7:0/3193146860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73dd0b800 msgr2=0x55a73eaea580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.389+0000 7f00edd5b280 1 --2- 172.21.15.7:0/3193146860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73dd0b800 0x55a73eaea580 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x55a73eb3b710 tx=0x55a73ddf4120 comp rx=0 tx=0).stop 2024-09-10T21:51:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.389+0000 7f00edd5b280 1 -- 172.21.15.7:0/3193146860 shutdown_connections 2024-09-10T21:51:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.389+0000 7f00edd5b280 1 --2- 172.21.15.7:0/3193146860 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a73dd0bc00 0x55a73eaeab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.389+0000 7f00edd5b280 1 --2- 172.21.15.7:0/3193146860 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a73dd0b800 0x55a73eaea580 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.389+0000 7f00edd5b280 1 --2- 172.21.15.7:0/3193146860 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a73dd0ac00 0x55a73eaea000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.389+0000 7f00edd5b280 1 -- 172.21.15.7:0/3193146860 >> 172.21.15.7:0/3193146860 conn(0x55a73dd0b000 msgr2=0x55a73de19000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.389+0000 7f00edd5b280 1 -- 172.21.15.7:0/3193146860 shutdown_connections 2024-09-10T21:51:21.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.389+0000 7f00edd5b280 1 -- 172.21.15.7:0/3193146860 wait complete. 2024-09-10T21:51:21.395 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '0', 'show', 'inode'] in time 0:00:00.100144, result: { "0": { "data": { "version": 5, "inotable": { "projected_free": [ { "start": 1099511627779, "len": 1099511627773 } ], "free": [ { "start": 1099511627779, "len": 1099511627773 } ] } }, "result": 0 } } 2024-09-10T21:51:21.395 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:51:21.396 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:51:21.532 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:51:21.625 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:51:21.625 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:51:21.625 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:51:21.625 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:51:21.662 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:51:21.662 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:51:21.662 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:51:21.662 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:51:21.699 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:51:21.699 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:51:21.699 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:51:21.699 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:51:21.736 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:51:21.736 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:51:21.737 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:51:21.737 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:51:21.776 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:51:21.776 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:51:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.936+0000 7f587c600640 1 -- 172.21.15.7:0/3394168310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c0039e0 msgr2=0x7f585c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.936+0000 7f587c600640 1 --2- 172.21.15.7:0/3394168310 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f585c0039e0 0x7f585c003dc0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f586800c260 tx=0x7f585c005830 comp rx=0 tx=0).stop 2024-09-10T21:51:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.936+0000 7f587a600640 1 -- 172.21.15.7:0/3394168310 reap_dead start 2024-09-10T21:51:21.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.936+0000 7f587c600640 1 -- 172.21.15.7:0/3394168310 shutdown_connections 2024-09-10T21:51:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.936+0000 7f587c600640 1 -- 172.21.15.7:0/3394168310 >> 172.21.15.7:0/3394168310 conn(0x7f58740b16d0 msgr2=0x7f58740b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.936+0000 7f587c600640 1 -- 172.21.15.7:0/3394168310 shutdown_connections 2024-09-10T21:51:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.936+0000 7f587c600640 1 -- 172.21.15.7:0/3394168310 wait complete. 2024-09-10T21:51:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.938+0000 7f587c600640 1 Processor -- start 2024-09-10T21:51:21.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.938+0000 7f587c600640 1 -- start start 2024-09-10T21:51:21.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.939+0000 7f587c600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f585c0039e0 0x7f587406f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.939+0000 7f587c600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58741f55a0 0x7f5874157dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.939+0000 7f587c600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58740701c0 0x7f58740733b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.939+0000 7f587c600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f58740ae650 con 0x7f58740701c0 2024-09-10T21:51:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.939+0000 7f587c600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f58740ae4d0 con 0x7f585c0039e0 2024-09-10T21:51:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.939+0000 7f587c600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f58740ae7d0 con 0x7f58741f55a0 2024-09-10T21:51:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.939+0000 7f587a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58740701c0 0x7f58740733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.939+0000 7f587a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58740701c0 0x7f58740733b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44742/0 (socket says 172.21.15.7:44742) 2024-09-10T21:51:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.939+0000 7f587a600640 1 -- 172.21.15.7:0/3227579458 learned_addr learned my addr 172.21.15.7:0/3227579458 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.939+0000 7f5879200640 1 --2- 172.21.15.7:0/3227579458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58741f55a0 0x7f5874157dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.939+0000 7f5879c00640 1 --2- 172.21.15.7:0/3227579458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f585c0039e0 0x7f587406f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.940+0000 7f587a600640 1 -- 172.21.15.7:0/3227579458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58741f55a0 msgr2=0x7f5874157dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.940+0000 7f587a600640 1 --2- 172.21.15.7:0/3227579458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58741f55a0 0x7f5874157dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.940+0000 7f587a600640 1 -- 172.21.15.7:0/3227579458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f585c0039e0 msgr2=0x7f587406f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:21.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.940+0000 7f587a600640 1 --2- 172.21.15.7:0/3227579458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f585c0039e0 0x7f587406f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:21.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.940+0000 7f587a600640 1 -- 172.21.15.7:0/3227579458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f587406e5b0 con 0x7f58740701c0 2024-09-10T21:51:21.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.940+0000 7f5879c00640 1 --2- 172.21.15.7:0/3227579458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f585c0039e0 0x7f587406f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:51:21.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.940+0000 7f587a600640 1 --2- 172.21.15.7:0/3227579458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58740701c0 0x7f58740733b0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f586c001520 tx=0x7f586c001a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:21.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.940+0000 7f5871600640 1 -- 172.21.15.7:0/3227579458 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c0107d0 con 0x7f58740701c0 2024-09-10T21:51:21.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.940+0000 7f5871600640 1 -- 172.21.15.7:0/3227579458 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f586c010970 con 0x7f58740701c0 2024-09-10T21:51:21.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.940+0000 7f5871600640 1 -- 172.21.15.7:0/3227579458 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586c010dc0 con 0x7f58740701c0 2024-09-10T21:51:21.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.940+0000 7f587c600640 1 -- 172.21.15.7:0/3227579458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f587406e870 con 0x7f58740701c0 2024-09-10T21:51:21.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.940+0000 7f587c600640 1 -- 172.21.15.7:0/3227579458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f587406ee30 con 0x7f58740701c0 2024-09-10T21:51:21.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.941+0000 7f587c600640 1 -- 172.21.15.7:0/3227579458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58740cd750 con 0x7f58740701c0 2024-09-10T21:51:21.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.943+0000 7f5871600640 1 -- 172.21.15.7:0/3227579458 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f586c01a400 con 0x7f58740701c0 2024-09-10T21:51:21.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.943+0000 7f5871600640 1 --2- 172.21.15.7:0/3227579458 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5854077f70 0x7f585407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:21.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.944+0000 7f5871600640 1 -- 172.21.15.7:0/3227579458 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7f586c09a500 con 0x7f58740701c0 2024-09-10T21:51:21.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.944+0000 7f5879c00640 1 --2- 172.21.15.7:0/3227579458 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5854077f70 0x7f585407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:21.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.945+0000 7f5879c00640 1 --2- 172.21.15.7:0/3227579458 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5854077f70 0x7f585407a430 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5868033000 tx=0x7f586802d000 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:21.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:21.948+0000 7f5871600640 1 -- 172.21.15.7:0/3227579458 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f586c062eb0 con 0x7f58740701c0 2024-09-10T21:51:22.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.184+0000 7f587c600640 1 -- 172.21.15.7:0/3227579458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58740b5b00 con 0x7f58740701c0 2024-09-10T21:51:22.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.185+0000 7f5871600640 1 -- 172.21.15.7:0/3227579458 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+4384 (secure 0 0 0) 0x7f586c0627b0 con 0x7f58740701c0 2024-09-10T21:51:22.186 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:22.186 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":92,"btime":"2024-09-10T21:51:21:268124+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/431253133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":431253133},{"type":"v1","addr":"172.21.15.133:6836","nonce":431253133}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7521,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3692199178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3692199178},{"type":"v1","addr":"172.21.15.133:6837","nonce":3692199178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/752989865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":752989865},{"type":"v1","addr":"172.21.15.7:6834","nonce":752989865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":92,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:49:58.872982+0000","modified":"2024-09-10T21:51:21.268120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-10T21:51:22.186 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 92 2024-09-10T21:51:22.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.187+0000 7f587c600640 1 -- 172.21.15.7:0/3227579458 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5854077f70 msgr2=0x7f585407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:22.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.187+0000 7f587c600640 1 --2- 172.21.15.7:0/3227579458 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5854077f70 0x7f585407a430 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5868033000 tx=0x7f586802d000 comp rx=0 tx=0).stop 2024-09-10T21:51:22.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.187+0000 7f587c600640 1 -- 172.21.15.7:0/3227579458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58740701c0 msgr2=0x7f58740733b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:22.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.187+0000 7f587c600640 1 --2- 172.21.15.7:0/3227579458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58740701c0 0x7f58740733b0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f586c001520 tx=0x7f586c001a90 comp rx=0 tx=0).stop 2024-09-10T21:51:22.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.188+0000 7f587c600640 1 -- 172.21.15.7:0/3227579458 shutdown_connections 2024-09-10T21:51:22.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.188+0000 7f587c600640 1 --2- 172.21.15.7:0/3227579458 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5854077f70 0x7f585407a430 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:22.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.188+0000 7f587c600640 1 --2- 172.21.15.7:0/3227579458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f58740701c0 0x7f58740733b0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:22.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.188+0000 7f587c600640 1 --2- 172.21.15.7:0/3227579458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f58741f55a0 0x7f5874157dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:22.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.188+0000 7f587c600640 1 --2- 172.21.15.7:0/3227579458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f585c0039e0 0x7f587406f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:22.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.188+0000 7f587c600640 1 -- 172.21.15.7:0/3227579458 >> 172.21.15.7:0/3227579458 conn(0x7f58740b16d0 msgr2=0x7f58741593b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:22.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.188+0000 7f587c600640 1 -- 172.21.15.7:0/3227579458 shutdown_connections 2024-09-10T21:51:22.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.188+0000 7f587c600640 1 -- 172.21.15.7:0/3227579458 wait complete. 2024-09-10T21:51:22.204 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:51:22.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.366+0000 7f4b8b200640 1 -- 172.21.15.7:0/572982239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b8c1470b0 msgr2=0x7f4b8c0ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:22.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.366+0000 7f4b8b200640 1 --2- 172.21.15.7:0/572982239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b8c1470b0 0x7f4b8c0ad8e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4b84005fd0 tx=0x7f4b8402a820 comp rx=0 tx=0).stop 2024-09-10T21:51:22.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.366+0000 7f4b8b200640 1 -- 172.21.15.7:0/572982239 shutdown_connections 2024-09-10T21:51:22.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.366+0000 7f4b8b200640 1 --2- 172.21.15.7:0/572982239 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b8c1470b0 0x7f4b8c0ad8e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:22.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.366+0000 7f4b8b200640 1 --2- 172.21.15.7:0/572982239 >> v2:172.21.15.133:3300/0 conn(0x7f4b8c06f590 0x7f4b8c072ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:22.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.366+0000 7f4b8b200640 1 -- 172.21.15.7:0/572982239 >> 172.21.15.7:0/572982239 conn(0x7f4b8c0b1d70 msgr2=0x7f4b8c0cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:22.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.366+0000 7f4b8b200640 1 -- 172.21.15.7:0/572982239 shutdown_connections 2024-09-10T21:51:22.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.366+0000 7f4b8b200640 1 -- 172.21.15.7:0/572982239 wait complete. 2024-09-10T21:51:22.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.368+0000 7f4b8b200640 1 Processor -- start 2024-09-10T21:51:22.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.368+0000 7f4b8b200640 1 -- start start 2024-09-10T21:51:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.369+0000 7f4b8b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b8c06f590 0x7f4b8c160a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.369+0000 7f4b8b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b8c146c50 0x7f4b8c13e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.369+0000 7f4b8b200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b8c1470b0 0x7f4b8c13ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.369+0000 7f4b8b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c0adfa0 con 0x7f4b8c06f590 2024-09-10T21:51:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.369+0000 7f4b8b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c0ade20 con 0x7f4b8c146c50 2024-09-10T21:51:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.369+0000 7f4b8b200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4b8c0ae120 con 0x7f4b8c1470b0 2024-09-10T21:51:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.369+0000 7f4b89e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b8c06f590 0x7f4b8c160a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.369+0000 7f4b89e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b8c06f590 0x7f4b8c160a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44754/0 (socket says 172.21.15.7:44754) 2024-09-10T21:51:22.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.369+0000 7f4b89e00640 1 -- 172.21.15.7:0/1920780460 learned_addr learned my addr 172.21.15.7:0/1920780460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.369+0000 7f4b83e00640 1 --2- 172.21.15.7:0/1920780460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b8c146c50 0x7f4b8c13e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.369+0000 7f4b8a800640 1 --2- 172.21.15.7:0/1920780460 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b8c1470b0 0x7f4b8c13ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.369+0000 7f4b89e00640 1 -- 172.21.15.7:0/1920780460 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b8c1470b0 msgr2=0x7f4b8c13ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.369+0000 7f4b89e00640 1 --2- 172.21.15.7:0/1920780460 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b8c1470b0 0x7f4b8c13ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.369+0000 7f4b89e00640 1 -- 172.21.15.7:0/1920780460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b8c146c50 msgr2=0x7f4b8c13e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.369+0000 7f4b89e00640 1 --2- 172.21.15.7:0/1920780460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b8c146c50 0x7f4b8c13e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.370+0000 7f4b89e00640 1 -- 172.21.15.7:0/1920780460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8c13ceb0 con 0x7f4b8c06f590 2024-09-10T21:51:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.370+0000 7f4b89e00640 1 --2- 172.21.15.7:0/1920780460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b8c06f590 0x7f4b8c160a90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00c730 tx=0x7f4b7c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.370+0000 7f4b82000640 1 -- 172.21.15.7:0/1920780460 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c016020 con 0x7f4b8c06f590 2024-09-10T21:51:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.370+0000 7f4b82000640 1 -- 172.21.15.7:0/1920780460 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b7c00a930 con 0x7f4b8c06f590 2024-09-10T21:51:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.370+0000 7f4b8b200640 1 -- 172.21.15.7:0/1920780460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8c13d190 con 0x7f4b8c06f590 2024-09-10T21:51:22.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.370+0000 7f4b82000640 1 -- 172.21.15.7:0/1920780460 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7c00ad80 con 0x7f4b8c06f590 2024-09-10T21:51:22.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.370+0000 7f4b8b200640 1 -- 172.21.15.7:0/1920780460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b8c13d750 con 0x7f4b8c06f590 2024-09-10T21:51:22.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.371+0000 7f4b8b200640 1 -- 172.21.15.7:0/1920780460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b8c0ae5d0 con 0x7f4b8c06f590 2024-09-10T21:51:22.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.372+0000 7f4b82000640 1 -- 172.21.15.7:0/1920780460 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4b7c01b400 con 0x7f4b8c06f590 2024-09-10T21:51:22.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.374+0000 7f4b82000640 1 --2- 172.21.15.7:0/1920780460 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4b68077f70 0x7f4b6807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:22.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.374+0000 7f4b82000640 1 -- 172.21.15.7:0/1920780460 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7f4b7c09a0b0 con 0x7f4b8c06f590 2024-09-10T21:51:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.374+0000 7f4b83e00640 1 --2- 172.21.15.7:0/1920780460 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4b68077f70 0x7f4b6807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:22.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.375+0000 7f4b83e00640 1 --2- 172.21.15.7:0/1920780460 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4b68077f70 0x7f4b6807a430 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c161520 tx=0x7f4b74005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:22.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.378+0000 7f4b82000640 1 -- 172.21.15.7:0/1920780460 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4b7c020020 con 0x7f4b8c06f590 2024-09-10T21:51:22.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.609+0000 7f4b8b200640 1 -- 172.21.15.7:0/1920780460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b8c161570 con 0x7f4b8c06f590 2024-09-10T21:51:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.609+0000 7f4b82000640 1 -- 172.21.15.7:0/1920780460 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5206 (secure 0 0 0) 0x7f4b8c147950 con 0x7f4b8c06f590 2024-09-10T21:51:22.610 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:22.610 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":93,"btime":"2024-09-10T21:51:22:269794+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/431253133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":431253133},{"type":"v1","addr":"172.21.15.133:6836","nonce":431253133}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7521,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3692199178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3692199178},{"type":"v1","addr":"172.21.15.133:6837","nonce":3692199178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/752989865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":752989865},{"type":"v1","addr":"172.21.15.7:6834","nonce":752989865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7956,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3804508996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3804508996},{"type":"v1","addr":"172.21.15.7:6835","nonce":3804508996}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":92,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:49:58.872982+0000","modified":"2024-09-10T21:51:21.268120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-10T21:51:22.610 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 93 2024-09-10T21:51:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.612+0000 7f4b8b200640 1 -- 172.21.15.7:0/1920780460 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4b68077f70 msgr2=0x7f4b6807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.612+0000 7f4b8b200640 1 --2- 172.21.15.7:0/1920780460 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4b68077f70 0x7f4b6807a430 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c161520 tx=0x7f4b74005f90 comp rx=0 tx=0).stop 2024-09-10T21:51:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.612+0000 7f4b8b200640 1 -- 172.21.15.7:0/1920780460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b8c06f590 msgr2=0x7f4b8c160a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.612+0000 7f4b8b200640 1 --2- 172.21.15.7:0/1920780460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b8c06f590 0x7f4b8c160a90 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f4b7c00c730 tx=0x7f4b7c00cbf0 comp rx=0 tx=0).stop 2024-09-10T21:51:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.612+0000 7f4b8b200640 1 -- 172.21.15.7:0/1920780460 shutdown_connections 2024-09-10T21:51:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.612+0000 7f4b8b200640 1 --2- 172.21.15.7:0/1920780460 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4b68077f70 0x7f4b6807a430 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.612+0000 7f4b8b200640 1 --2- 172.21.15.7:0/1920780460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4b8c146c50 0x7f4b8c13e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:22.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.612+0000 7f4b8b200640 1 --2- 172.21.15.7:0/1920780460 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4b8c1470b0 0x7f4b8c13ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.612+0000 7f4b8b200640 1 --2- 172.21.15.7:0/1920780460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4b8c06f590 0x7f4b8c160a90 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.612+0000 7f4b8b200640 1 -- 172.21.15.7:0/1920780460 >> 172.21.15.7:0/1920780460 conn(0x7f4b8c0b1d70 msgr2=0x7f4b8c0ae7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.612+0000 7f4b8b200640 1 -- 172.21.15.7:0/1920780460 shutdown_connections 2024-09-10T21:51:22.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.612+0000 7f4b8b200640 1 -- 172.21.15.7:0/1920780460 wait complete. 2024-09-10T21:51:22.628 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:51:22.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.789+0000 7f0d99800640 1 -- 172.21.15.7:0/217114147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d940cd750 msgr2=0x7f0d94073730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:22.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.789+0000 7f0d99800640 1 --2- 172.21.15.7:0/217114147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d940cd750 0x7f0d94073730 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f0d940b5560 tx=0x7f0d7c02a440 comp rx=0 tx=0).stop 2024-09-10T21:51:22.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.789+0000 7f0d99800640 1 -- 172.21.15.7:0/217114147 shutdown_connections 2024-09-10T21:51:22.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.789+0000 7f0d99800640 1 --2- 172.21.15.7:0/217114147 >> v2:172.21.15.133:3300/0 conn(0x7f0d94073c70 0x7f0d9415fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:22.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.789+0000 7f0d99800640 1 --2- 172.21.15.7:0/217114147 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d940cd750 0x7f0d94073730 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:22.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.789+0000 7f0d99800640 1 --2- 172.21.15.7:0/217114147 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d9415bc90 0x7f0d940ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:22.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.789+0000 7f0d99800640 1 -- 172.21.15.7:0/217114147 >> 172.21.15.7:0/217114147 conn(0x7f0d940b16d0 msgr2=0x7f0d940b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:22.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.789+0000 7f0d99800640 1 -- 172.21.15.7:0/217114147 shutdown_connections 2024-09-10T21:51:22.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.789+0000 7f0d99800640 1 -- 172.21.15.7:0/217114147 wait complete. 2024-09-10T21:51:22.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.791+0000 7f0d99800640 1 Processor -- start 2024-09-10T21:51:22.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.791+0000 7f0d99800640 1 -- start start 2024-09-10T21:51:22.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.792+0000 7f0d99800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d94073c70 0x7f0d940a94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:22.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.792+0000 7f0d99800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d940cd750 0x7f0d94151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:22.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.792+0000 7f0d99800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9415bc90 0x7f0d9415e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:22.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.792+0000 7f0d99800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0d94160670 con 0x7f0d9415bc90 2024-09-10T21:51:22.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.792+0000 7f0d99800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0d941604f0 con 0x7f0d94073c70 2024-09-10T21:51:22.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.792+0000 7f0d99800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f0d9406fcf0 con 0x7f0d940cd750 2024-09-10T21:51:22.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.792+0000 7f0d93400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9415bc90 0x7f0d9415e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:22.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.792+0000 7f0d93400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9415bc90 0x7f0d9415e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44770/0 (socket says 172.21.15.7:44770) 2024-09-10T21:51:22.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.792+0000 7f0d93400640 1 -- 172.21.15.7:0/947000747 learned_addr learned my addr 172.21.15.7:0/947000747 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:22.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.792+0000 7f0d8a000640 1 --2- 172.21.15.7:0/947000747 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d940cd750 0x7f0d94151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:22.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.792+0000 7f0d92a00640 1 --2- 172.21.15.7:0/947000747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d94073c70 0x7f0d940a94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:22.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.793+0000 7f0d93400640 1 -- 172.21.15.7:0/947000747 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d940cd750 msgr2=0x7f0d94151030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:22.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.793+0000 7f0d93400640 1 --2- 172.21.15.7:0/947000747 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d940cd750 0x7f0d94151030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:22.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.793+0000 7f0d93400640 1 -- 172.21.15.7:0/947000747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d94073c70 msgr2=0x7f0d940a94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:22.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.793+0000 7f0d93400640 1 --2- 172.21.15.7:0/947000747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d94073c70 0x7f0d940a94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:22.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.793+0000 7f0d93400640 1 -- 172.21.15.7:0/947000747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d940a9be0 con 0x7f0d9415bc90 2024-09-10T21:51:22.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.793+0000 7f0d92a00640 1 --2- 172.21.15.7:0/947000747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d94073c70 0x7f0d940a94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:51:22.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.793+0000 7f0d93400640 1 --2- 172.21.15.7:0/947000747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9415bc90 0x7f0d9415e850 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00e970 tx=0x7f0d8c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:22.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.793+0000 7f0d8b400640 1 -- 172.21.15.7:0/947000747 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c00cab0 con 0x7f0d9415bc90 2024-09-10T21:51:22.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.793+0000 7f0d8b400640 1 -- 172.21.15.7:0/947000747 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d8c00cc50 con 0x7f0d9415bc90 2024-09-10T21:51:22.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.793+0000 7f0d8b400640 1 -- 172.21.15.7:0/947000747 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d8c005280 con 0x7f0d9415bc90 2024-09-10T21:51:22.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.793+0000 7f0d99800640 1 -- 172.21.15.7:0/947000747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d940a9ec0 con 0x7f0d9415bc90 2024-09-10T21:51:22.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.793+0000 7f0d99800640 1 -- 172.21.15.7:0/947000747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d940aa480 con 0x7f0d9415bc90 2024-09-10T21:51:22.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.794+0000 7f0d99800640 1 -- 172.21.15.7:0/947000747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d94073730 con 0x7f0d9415bc90 2024-09-10T21:51:22.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.795+0000 7f0d8b400640 1 -- 172.21.15.7:0/947000747 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0d8c0047a0 con 0x7f0d9415bc90 2024-09-10T21:51:22.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.796+0000 7f0d8b400640 1 --2- 172.21.15.7:0/947000747 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0d70077fa0 0x7f0d7007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:22.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.797+0000 7f0d8b400640 1 -- 172.21.15.7:0/947000747 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7f0d8c09a960 con 0x7f0d9415bc90 2024-09-10T21:51:22.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.797+0000 7f0d92a00640 1 --2- 172.21.15.7:0/947000747 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0d70077fa0 0x7f0d7007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:22.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.797+0000 7f0d92a00640 1 --2- 172.21.15.7:0/947000747 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0d70077fa0 0x7f0d7007a460 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0d84005fd0 tx=0x7f0d84009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:22.801 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.800+0000 7f0d8b400640 1 -- 172.21.15.7:0/947000747 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0d8c063310 con 0x7f0d9415bc90 2024-09-10T21:51:22.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.997+0000 7f0d99800640 1 -- 172.21.15.7:0/947000747 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0d940c9d80 con 0x7f0d9415bc90 2024-09-10T21:51:22.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:22.998+0000 7f0d8b400640 1 -- 172.21.15.7:0/947000747 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16890 (secure 0 0 0) 0x7f0d8c00ce20 con 0x7f0d9415bc90 2024-09-10T21:51:22.999 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:22.999 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":103,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:51:21.267960+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:49:56.657911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T21:49:57.828979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6835/4110623302":"2024-09-11T21:51:21.267938+0000","172.21.15.7:6833/4110623302":"2024-09-11T21:51:21.267938+0000","172.21.15.7:6835/3075619232":"2024-09-11T21:50:44.284783+0000","172.21.15.7:6833/3075619232":"2024-09-11T21:50:44.284783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:51:23.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.000+0000 7f0d99800640 1 -- 172.21.15.7:0/947000747 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0d70077fa0 msgr2=0x7f0d7007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:23.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.000+0000 7f0d99800640 1 --2- 172.21.15.7:0/947000747 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0d70077fa0 0x7f0d7007a460 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0d84005fd0 tx=0x7f0d84009500 comp rx=0 tx=0).stop 2024-09-10T21:51:23.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.000+0000 7f0d99800640 1 -- 172.21.15.7:0/947000747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9415bc90 msgr2=0x7f0d9415e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:23.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.000+0000 7f0d99800640 1 --2- 172.21.15.7:0/947000747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9415bc90 0x7f0d9415e850 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f0d8c00e970 tx=0x7f0d8c00ee30 comp rx=0 tx=0).stop 2024-09-10T21:51:23.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.000+0000 7f0d99800640 1 -- 172.21.15.7:0/947000747 shutdown_connections 2024-09-10T21:51:23.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.000+0000 7f0d99800640 1 --2- 172.21.15.7:0/947000747 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0d70077fa0 0x7f0d7007a460 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.000+0000 7f0d99800640 1 --2- 172.21.15.7:0/947000747 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0d9415bc90 0x7f0d9415e850 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.000+0000 7f0d99800640 1 --2- 172.21.15.7:0/947000747 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0d940cd750 0x7f0d94151030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.000+0000 7f0d99800640 1 --2- 172.21.15.7:0/947000747 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0d94073c70 0x7f0d940a94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.000+0000 7f0d99800640 1 -- 172.21.15.7:0/947000747 >> 172.21.15.7:0/947000747 conn(0x7f0d940b16d0 msgr2=0x7f0d940b4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:23.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.000+0000 7f0d99800640 1 -- 172.21.15.7:0/947000747 shutdown_connections 2024-09-10T21:51:23.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.000+0000 7f0d99800640 1 -- 172.21.15.7:0/947000747 wait complete. 2024-09-10T21:51:23.018 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:51:23.018 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:51:23.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.178+0000 7f9831800640 1 -- 172.21.15.7:0/4046593410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f982c0d4900 msgr2=0x7f982c0b6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:23.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.178+0000 7f9831800640 1 --2- 172.21.15.7:0/4046593410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f982c0d4900 0x7f982c0b6c30 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9818009830 tx=0x7f981802a520 comp rx=0 tx=0).stop 2024-09-10T21:51:23.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.178+0000 7f9831800640 1 -- 172.21.15.7:0/4046593410 shutdown_connections 2024-09-10T21:51:23.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.178+0000 7f9831800640 1 --2- 172.21.15.7:0/4046593410 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f982c0d4900 0x7f982c0b6c30 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.178+0000 7f9831800640 1 -- 172.21.15.7:0/4046593410 >> 172.21.15.7:0/4046593410 conn(0x7f982c0d3de0 msgr2=0x7f982c0d41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:23.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.178+0000 7f9831800640 1 -- 172.21.15.7:0/4046593410 shutdown_connections 2024-09-10T21:51:23.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.178+0000 7f9831800640 1 -- 172.21.15.7:0/4046593410 wait complete. 2024-09-10T21:51:23.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.180+0000 7f9831800640 1 Processor -- start 2024-09-10T21:51:23.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.180+0000 7f9831800640 1 -- start start 2024-09-10T21:51:23.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.181+0000 7f9831800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f982c0b21f0 0x7f982c1f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:23.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.181+0000 7f9831800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f982c0d4900 0x7f982c1f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:23.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.181+0000 7f9831800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f982c15dff0 0x7f982c0cda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:23.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.181+0000 7f9831800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f982c0cfd00 con 0x7f982c15dff0 2024-09-10T21:51:23.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.181+0000 7f9831800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f982c0cfb80 con 0x7f982c0b21f0 2024-09-10T21:51:23.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.181+0000 7f9831800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f982c0cfe80 con 0x7f982c0d4900 2024-09-10T21:51:23.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.181+0000 7f9830e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f982c15dff0 0x7f982c0cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:23.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.181+0000 7f9830e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f982c15dff0 0x7f982c0cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44772/0 (socket says 172.21.15.7:44772) 2024-09-10T21:51:23.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.181+0000 7f9830e00640 1 -- 172.21.15.7:0/2337605163 learned_addr learned my addr 172.21.15.7:0/2337605163 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:23.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.181+0000 7f982be00640 1 --2- 172.21.15.7:0/2337605163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f982c0b21f0 0x7f982c1f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:23.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.182+0000 7f982b400640 1 --2- 172.21.15.7:0/2337605163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f982c0d4900 0x7f982c1f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:23.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.182+0000 7f9830e00640 1 -- 172.21.15.7:0/2337605163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f982c0d4900 msgr2=0x7f982c1f80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:23.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.182+0000 7f9830e00640 1 --2- 172.21.15.7:0/2337605163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f982c0d4900 0x7f982c1f80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.182+0000 7f9830e00640 1 -- 172.21.15.7:0/2337605163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f982c0b21f0 msgr2=0x7f982c1f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:23.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.182+0000 7f9830e00640 1 --2- 172.21.15.7:0/2337605163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f982c0b21f0 0x7f982c1f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.182+0000 7f9830e00640 1 -- 172.21.15.7:0/2337605163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c15b770 con 0x7f982c15dff0 2024-09-10T21:51:23.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.182+0000 7f9830e00640 1 --2- 172.21.15.7:0/2337605163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f982c15dff0 0x7f982c0cda10 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f982400cb70 tx=0x7f982400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.182+0000 7f9823e00640 1 -- 172.21.15.7:0/2337605163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824016020 con 0x7f982c15dff0 2024-09-10T21:51:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.182+0000 7f9823e00640 1 -- 172.21.15.7:0/2337605163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f982400ae50 con 0x7f982c15dff0 2024-09-10T21:51:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.182+0000 7f9823e00640 1 -- 172.21.15.7:0/2337605163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9824012660 con 0x7f982c15dff0 2024-09-10T21:51:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.182+0000 7f9831800640 1 -- 172.21.15.7:0/2337605163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982c1f51d0 con 0x7f982c15dff0 2024-09-10T21:51:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.182+0000 7f9831800640 1 -- 172.21.15.7:0/2337605163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f982c1f4a50 con 0x7f982c15dff0 2024-09-10T21:51:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.183+0000 7f9831800640 1 -- 172.21.15.7:0/2337605163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f982c0b6c30 con 0x7f982c15dff0 2024-09-10T21:51:23.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.185+0000 7f9823e00640 1 -- 172.21.15.7:0/2337605163 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f9824012800 con 0x7f982c15dff0 2024-09-10T21:51:23.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.186+0000 7f9823e00640 1 --2- 172.21.15.7:0/2337605163 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f980c077f70 0x7f980c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:23.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.186+0000 7f9823e00640 1 -- 172.21.15.7:0/2337605163 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7f982409a7e0 con 0x7f982c15dff0 2024-09-10T21:51:23.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.187+0000 7f982be00640 1 --2- 172.21.15.7:0/2337605163 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f980c077f70 0x7f980c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:23.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.187+0000 7f982be00640 1 --2- 172.21.15.7:0/2337605163 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f980c077f70 0x7f980c07a430 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f981c007fd0 tx=0x7f981c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:23.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.191+0000 7f9823e00640 1 -- 172.21.15.7:0/2337605163 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9824020020 con 0x7f982c15dff0 2024-09-10T21:51:23.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.425+0000 7f9831800640 1 -- 172.21.15.7:0/2337605163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f982c076010 con 0x7f982c15dff0 2024-09-10T21:51:23.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.426+0000 7f9823e00640 1 -- 172.21.15.7:0/2337605163 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5206 (secure 0 0 0) 0x7f9824063220 con 0x7f982c15dff0 2024-09-10T21:51:23.427 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:23.427 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":93,"btime":"2024-09-10T21:51:22:269794+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/431253133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":431253133},{"type":"v1","addr":"172.21.15.133:6836","nonce":431253133}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7521,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3692199178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3692199178},{"type":"v1","addr":"172.21.15.133:6837","nonce":3692199178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/752989865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":752989865},{"type":"v1","addr":"172.21.15.7:6834","nonce":752989865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7956,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3804508996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3804508996},{"type":"v1","addr":"172.21.15.7:6835","nonce":3804508996}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":92,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:49:58.872982+0000","modified":"2024-09-10T21:51:21.268120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-10T21:51:23.427 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 93 2024-09-10T21:51:23.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.428+0000 7f9831800640 1 -- 172.21.15.7:0/2337605163 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f980c077f70 msgr2=0x7f980c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:23.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.428+0000 7f9831800640 1 --2- 172.21.15.7:0/2337605163 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f980c077f70 0x7f980c07a430 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f981c007fd0 tx=0x7f981c009040 comp rx=0 tx=0).stop 2024-09-10T21:51:23.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.428+0000 7f9831800640 1 -- 172.21.15.7:0/2337605163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f982c15dff0 msgr2=0x7f982c0cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:23.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.428+0000 7f9831800640 1 --2- 172.21.15.7:0/2337605163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f982c15dff0 0x7f982c0cda10 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f982400cb70 tx=0x7f982400a430 comp rx=0 tx=0).stop 2024-09-10T21:51:23.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.428+0000 7f9831800640 1 -- 172.21.15.7:0/2337605163 shutdown_connections 2024-09-10T21:51:23.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.428+0000 7f9831800640 1 --2- 172.21.15.7:0/2337605163 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f980c077f70 0x7f980c07a430 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.429+0000 7f9831800640 1 --2- 172.21.15.7:0/2337605163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f982c15dff0 0x7f982c0cda10 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.429+0000 7f9831800640 1 --2- 172.21.15.7:0/2337605163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f982c0d4900 0x7f982c1f80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.429+0000 7f9831800640 1 --2- 172.21.15.7:0/2337605163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f982c0b21f0 0x7f982c1f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.429+0000 7f9831800640 1 -- 172.21.15.7:0/2337605163 >> 172.21.15.7:0/2337605163 conn(0x7f982c0d3de0 msgr2=0x7f982c0b8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:23.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.429+0000 7f9831800640 1 -- 172.21.15.7:0/2337605163 shutdown_connections 2024-09-10T21:51:23.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.429+0000 7f9831800640 1 -- 172.21.15.7:0/2337605163 wait complete. 2024-09-10T21:51:23.446 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:51:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.606+0000 7fbbe0a00640 1 -- 172.21.15.7:0/1603733630 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbdc15fda0 msgr2=0x7fbbdc1601d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.606+0000 7fbbe0a00640 1 --2- 172.21.15.7:0/1603733630 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbdc15fda0 0x7fbbdc1601d0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc15fa90 tx=0x7fbbd402a800 comp rx=0 tx=0).stop 2024-09-10T21:51:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.606+0000 7fbbe0a00640 1 -- 172.21.15.7:0/1603733630 shutdown_connections 2024-09-10T21:51:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.606+0000 7fbbe0a00640 1 --2- 172.21.15.7:0/1603733630 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbdc15fda0 0x7fbbdc1601d0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.606+0000 7fbbe0a00640 1 --2- 172.21.15.7:0/1603733630 >> v2:172.21.15.133:3300/0 conn(0x7fbbdc0cd750 0x7fbbdc0ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.606+0000 7fbbe0a00640 1 -- 172.21.15.7:0/1603733630 >> 172.21.15.7:0/1603733630 conn(0x7fbbdc0b16d0 msgr2=0x7fbbdc0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.606+0000 7fbbe0a00640 1 -- 172.21.15.7:0/1603733630 shutdown_connections 2024-09-10T21:51:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.606+0000 7fbbe0a00640 1 -- 172.21.15.7:0/1603733630 wait complete. 2024-09-10T21:51:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.608+0000 7fbbe0a00640 1 Processor -- start 2024-09-10T21:51:23.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.608+0000 7fbbe0a00640 1 -- start start 2024-09-10T21:51:23.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.609+0000 7fbbe0a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbdc0cd750 0x7fbbdc15dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:23.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.609+0000 7fbbe0a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbdc15bc90 0x7fbbdc1f1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:23.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.609+0000 7fbbe0a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbdc15fda0 0x7fbbdc072360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:23.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.609+0000 7fbbe0a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc06fe70 con 0x7fbbdc15fda0 2024-09-10T21:51:23.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.609+0000 7fbbe0a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbbdc06fcf0 con 0x7fbbdc0cd750 2024-09-10T21:51:23.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.609+0000 7fbbe0a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fbbdc06fff0 con 0x7fbbdc15bc90 2024-09-10T21:51:23.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.609+0000 7fbbda800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbdc15fda0 0x7fbbdc072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.609+0000 7fbbda800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbdc15fda0 0x7fbbdc072360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44788/0 (socket says 172.21.15.7:44788) 2024-09-10T21:51:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.609+0000 7fbbda800640 1 -- 172.21.15.7:0/881997962 learned_addr learned my addr 172.21.15.7:0/881997962 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.609+0000 7fbbd9e00640 1 --2- 172.21.15.7:0/881997962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbdc0cd750 0x7fbbdc15dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.609+0000 7fbbd9400640 1 --2- 172.21.15.7:0/881997962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbdc15bc90 0x7fbbdc1f1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.609+0000 7fbbda800640 1 -- 172.21.15.7:0/881997962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbdc15bc90 msgr2=0x7fbbdc1f1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.609+0000 7fbbda800640 1 --2- 172.21.15.7:0/881997962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbdc15bc90 0x7fbbdc1f1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.609+0000 7fbbda800640 1 -- 172.21.15.7:0/881997962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbdc0cd750 msgr2=0x7fbbdc15dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.609+0000 7fbbda800640 1 --2- 172.21.15.7:0/881997962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbdc0cd750 0x7fbbdc15dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.609+0000 7fbbda800640 1 -- 172.21.15.7:0/881997962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbdc15e200 con 0x7fbbdc15fda0 2024-09-10T21:51:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.610+0000 7fbbda800640 1 --2- 172.21.15.7:0/881997962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbdc15fda0 0x7fbbdc072360 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4035040 tx=0x7fbbd4033dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.610+0000 7fbbd2000640 1 -- 172.21.15.7:0/881997962 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4042020 con 0x7fbbdc15fda0 2024-09-10T21:51:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.610+0000 7fbbd2000640 1 -- 172.21.15.7:0/881997962 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbd4004040 con 0x7fbbdc15fda0 2024-09-10T21:51:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.610+0000 7fbbd2000640 1 -- 172.21.15.7:0/881997962 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4037040 con 0x7fbbdc15fda0 2024-09-10T21:51:23.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.610+0000 7fbbe0a00640 1 -- 172.21.15.7:0/881997962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbdc0c70a0 con 0x7fbbdc15fda0 2024-09-10T21:51:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.610+0000 7fbbe0a00640 1 -- 172.21.15.7:0/881997962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbdc0c7660 con 0x7fbbdc15fda0 2024-09-10T21:51:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.611+0000 7fbbe0a00640 1 -- 172.21.15.7:0/881997962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbdc0704a0 con 0x7fbbdc15fda0 2024-09-10T21:51:23.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.612+0000 7fbbd2000640 1 -- 172.21.15.7:0/881997962 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fbbd4031070 con 0x7fbbdc15fda0 2024-09-10T21:51:23.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.613+0000 7fbbd2000640 1 --2- 172.21.15.7:0/881997962 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbbbc077f70 0x7fbbbc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:23.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.614+0000 7fbbd2000640 1 -- 172.21.15.7:0/881997962 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7fbbd40baa20 con 0x7fbbdc15fda0 2024-09-10T21:51:23.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.614+0000 7fbbd9e00640 1 --2- 172.21.15.7:0/881997962 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbbbc077f70 0x7fbbbc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:23.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.615+0000 7fbbd9e00640 1 --2- 172.21.15.7:0/881997962 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbbbc077f70 0x7fbbbc07a430 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0016b0 tx=0x7fbbcc009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:23.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.617+0000 7fbbd2000640 1 -- 172.21.15.7:0/881997962 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fbbd404c070 con 0x7fbbdc15fda0 2024-09-10T21:51:23.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.852+0000 7fbbe0a00640 1 -- 172.21.15.7:0/881997962 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbdc15f4a0 con 0x7fbbdc15fda0 2024-09-10T21:51:23.853 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.853+0000 7fbbd2000640 1 -- 172.21.15.7:0/881997962 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 93 v93) ==== 94+0+5206 (secure 0 0 0) 0x7fbbd402f070 con 0x7fbbdc15fda0 2024-09-10T21:51:23.853 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:23.854 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":93,"btime":"2024-09-10T21:51:22:269794+0000","default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/431253133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":431253133},{"type":"v1","addr":"172.21.15.133:6836","nonce":431253133}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7521,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3692199178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3692199178},{"type":"v1","addr":"172.21.15.133:6837","nonce":3692199178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/752989865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":752989865},{"type":"v1","addr":"172.21.15.7:6834","nonce":752989865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7956,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3804508996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3804508996},{"type":"v1","addr":"172.21.15.7:6835","nonce":3804508996}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":92,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:49:58.872982+0000","modified":"2024-09-10T21:51:21.268120+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":103,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":7}]} 2024-09-10T21:51:23.854 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 93 2024-09-10T21:51:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.855+0000 7fbbe0a00640 1 -- 172.21.15.7:0/881997962 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbbbc077f70 msgr2=0x7fbbbc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.855+0000 7fbbe0a00640 1 --2- 172.21.15.7:0/881997962 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbbbc077f70 0x7fbbbc07a430 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbbcc0016b0 tx=0x7fbbcc009040 comp rx=0 tx=0).stop 2024-09-10T21:51:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.855+0000 7fbbe0a00640 1 -- 172.21.15.7:0/881997962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbdc15fda0 msgr2=0x7fbbdc072360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.855+0000 7fbbe0a00640 1 --2- 172.21.15.7:0/881997962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbdc15fda0 0x7fbbdc072360 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fbbd4035040 tx=0x7fbbd4033dd0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.855+0000 7fbbe0a00640 1 -- 172.21.15.7:0/881997962 shutdown_connections 2024-09-10T21:51:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.855+0000 7fbbe0a00640 1 --2- 172.21.15.7:0/881997962 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbbbc077f70 0x7fbbbc07a430 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.855+0000 7fbbe0a00640 1 --2- 172.21.15.7:0/881997962 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbbdc15fda0 0x7fbbdc072360 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.855+0000 7fbbe0a00640 1 --2- 172.21.15.7:0/881997962 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbbdc15bc90 0x7fbbdc1f1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.855+0000 7fbbe0a00640 1 --2- 172.21.15.7:0/881997962 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbbdc0cd750 0x7fbbdc15dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.855+0000 7fbbe0a00640 1 -- 172.21.15.7:0/881997962 >> 172.21.15.7:0/881997962 conn(0x7fbbdc0b16d0 msgr2=0x7fbbdc0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:23.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.855+0000 7fbbe0a00640 1 -- 172.21.15.7:0/881997962 shutdown_connections 2024-09-10T21:51:23.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:23.855+0000 7fbbe0a00640 1 -- 172.21.15.7:0/881997962 wait complete. 2024-09-10T21:51:23.872 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:51:24.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.031+0000 7fca42000640 1 -- 172.21.15.7:0/670072319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca3c0cfe30 msgr2=0x7fca3c0cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:24.032 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.031+0000 7fca42000640 1 --2- 172.21.15.7:0/670072319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca3c0cfe30 0x7fca3c0cd3d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fca28009830 tx=0x7fca2802a3f0 comp rx=0 tx=0).stop 2024-09-10T21:51:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.031+0000 7fca42000640 1 -- 172.21.15.7:0/670072319 shutdown_connections 2024-09-10T21:51:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.031+0000 7fca42000640 1 --2- 172.21.15.7:0/670072319 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca3c0cfe30 0x7fca3c0cd3d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.031+0000 7fca42000640 1 --2- 172.21.15.7:0/670072319 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca3c15f290 0x7fca3c0cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.031+0000 7fca42000640 1 -- 172.21.15.7:0/670072319 >> 172.21.15.7:0/670072319 conn(0x7fca3c0d1d20 msgr2=0x7fca3c0d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.031+0000 7fca42000640 1 -- 172.21.15.7:0/670072319 shutdown_connections 2024-09-10T21:51:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.031+0000 7fca42000640 1 -- 172.21.15.7:0/670072319 wait complete. 2024-09-10T21:51:24.033 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.033+0000 7fca42000640 1 Processor -- start 2024-09-10T21:51:24.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.033+0000 7fca42000640 1 -- start start 2024-09-10T21:51:24.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.034+0000 7fca42000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca3c0cd910 0x7fca3c1f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:24.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.034+0000 7fca42000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca3c0cfe30 0x7fca3c15c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:24.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.034+0000 7fca42000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca3c15f290 0x7fca3c0c68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.034+0000 7fca42000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c15f7f0 con 0x7fca3c0cfe30 2024-09-10T21:51:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.034+0000 7fca42000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fca3c15f670 con 0x7fca3c15f290 2024-09-10T21:51:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.034+0000 7fca42000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fca3c0ce3c0 con 0x7fca3c0cd910 2024-09-10T21:51:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.034+0000 7fca3aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca3c0cfe30 0x7fca3c15c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.034+0000 7fca3aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca3c0cfe30 0x7fca3c15c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48712/0 (socket says 172.21.15.7:48712) 2024-09-10T21:51:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.034+0000 7fca3aa00640 1 -- 172.21.15.7:0/4253644228 learned_addr learned my addr 172.21.15.7:0/4253644228 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.034+0000 7fca3b400640 1 --2- 172.21.15.7:0/4253644228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca3c0cd910 0x7fca3c1f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.034+0000 7fca3be00640 1 --2- 172.21.15.7:0/4253644228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca3c15f290 0x7fca3c0c68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.035+0000 7fca3aa00640 1 -- 172.21.15.7:0/4253644228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca3c0cd910 msgr2=0x7fca3c1f31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.035+0000 7fca3aa00640 1 --2- 172.21.15.7:0/4253644228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca3c0cd910 0x7fca3c1f31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.035+0000 7fca3aa00640 1 -- 172.21.15.7:0/4253644228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca3c15f290 msgr2=0x7fca3c0c68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.035+0000 7fca3aa00640 1 --2- 172.21.15.7:0/4253644228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca3c15f290 0x7fca3c0c68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.035+0000 7fca3aa00640 1 -- 172.21.15.7:0/4253644228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca3c0c7010 con 0x7fca3c0cfe30 2024-09-10T21:51:24.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.035+0000 7fca3b400640 1 --2- 172.21.15.7:0/4253644228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca3c0cd910 0x7fca3c1f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:51:24.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.035+0000 7fca3aa00640 1 --2- 172.21.15.7:0/4253644228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca3c0cfe30 0x7fca3c15c360 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fca2802a9e0 tx=0x7fca28032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:24.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.035+0000 7fca33400640 1 -- 172.21.15.7:0/4253644228 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28040070 con 0x7fca3c0cfe30 2024-09-10T21:51:24.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.035+0000 7fca42000640 1 -- 172.21.15.7:0/4253644228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca3c0c5470 con 0x7fca3c0cfe30 2024-09-10T21:51:24.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.035+0000 7fca33400640 1 -- 172.21.15.7:0/4253644228 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca28005310 con 0x7fca3c0cfe30 2024-09-10T21:51:24.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.035+0000 7fca33400640 1 -- 172.21.15.7:0/4253644228 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca28005c90 con 0x7fca3c0cfe30 2024-09-10T21:51:24.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.035+0000 7fca42000640 1 -- 172.21.15.7:0/4253644228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca3c0c5a30 con 0x7fca3c0cfe30 2024-09-10T21:51:24.038 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.037+0000 7fca42000640 1 -- 172.21.15.7:0/4253644228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca3c0c5c00 con 0x7fca3c0cfe30 2024-09-10T21:51:24.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.038+0000 7fca33400640 1 -- 172.21.15.7:0/4253644228 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fca280049b0 con 0x7fca3c0cfe30 2024-09-10T21:51:24.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.039+0000 7fca33400640 1 --2- 172.21.15.7:0/4253644228 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fca1c077fa0 0x7fca1c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:24.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.039+0000 7fca33400640 1 -- 172.21.15.7:0/4253644228 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7fca280bb700 con 0x7fca3c0cfe30 2024-09-10T21:51:24.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.040+0000 7fca3b400640 1 --2- 172.21.15.7:0/4253644228 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fca1c077fa0 0x7fca1c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:24.041 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.040+0000 7fca3b400640 1 --2- 172.21.15.7:0/4253644228 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fca1c077fa0 0x7fca1c07a460 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0ced20 tx=0x7fca2c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:24.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.042+0000 7fca33400640 1 -- 172.21.15.7:0/4253644228 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fca28084260 con 0x7fca3c0cfe30 2024-09-10T21:51:24.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.237+0000 7fca42000640 1 -- 172.21.15.7:0/4253644228 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fca3c0b1030 con 0x7fca3c0cfe30 2024-09-10T21:51:24.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.238+0000 7fca33400640 1 -- 172.21.15.7:0/4253644228 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) ==== 74+0+16890 (secure 0 0 0) 0x7fca28083b60 con 0x7fca3c0cfe30 2024-09-10T21:51:24.239 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:24.239 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":103,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:51:21.267960+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:49:56.657911+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"100","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2024-09-10T21:49:57.828979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6835/4110623302":"2024-09-11T21:51:21.267938+0000","172.21.15.7:6833/4110623302":"2024-09-11T21:51:21.267938+0000","172.21.15.7:6835/3075619232":"2024-09-11T21:50:44.284783+0000","172.21.15.7:6833/3075619232":"2024-09-11T21:50:44.284783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:51:24.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.241+0000 7fca42000640 1 -- 172.21.15.7:0/4253644228 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fca1c077fa0 msgr2=0x7fca1c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:24.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.241+0000 7fca42000640 1 --2- 172.21.15.7:0/4253644228 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fca1c077fa0 0x7fca1c07a460 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fca3c0ced20 tx=0x7fca2c005f90 comp rx=0 tx=0).stop 2024-09-10T21:51:24.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.241+0000 7fca42000640 1 -- 172.21.15.7:0/4253644228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca3c0cfe30 msgr2=0x7fca3c15c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:24.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.241+0000 7fca42000640 1 --2- 172.21.15.7:0/4253644228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca3c0cfe30 0x7fca3c15c360 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7fca2802a9e0 tx=0x7fca28032c10 comp rx=0 tx=0).stop 2024-09-10T21:51:24.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.241+0000 7fca42000640 1 -- 172.21.15.7:0/4253644228 shutdown_connections 2024-09-10T21:51:24.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.241+0000 7fca42000640 1 --2- 172.21.15.7:0/4253644228 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fca1c077fa0 0x7fca1c07a460 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:24.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.241+0000 7fca42000640 1 --2- 172.21.15.7:0/4253644228 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca3c15f290 0x7fca3c0c68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:24.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.241+0000 7fca42000640 1 --2- 172.21.15.7:0/4253644228 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca3c0cfe30 0x7fca3c15c360 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:24.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.241+0000 7fca42000640 1 --2- 172.21.15.7:0/4253644228 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca3c0cd910 0x7fca3c1f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:24.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.241+0000 7fca42000640 1 -- 172.21.15.7:0/4253644228 >> 172.21.15.7:0/4253644228 conn(0x7fca3c0d1d20 msgr2=0x7fca3c071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:24.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.241+0000 7fca42000640 1 -- 172.21.15.7:0/4253644228 shutdown_connections 2024-09-10T21:51:24.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.241+0000 7fca42000640 1 -- 172.21.15.7:0/4253644228 wait complete. 2024-09-10T21:51:24.258 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:51:24.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.422+0000 7f4d9ac00640 1 -- 172.21.15.7:0/1846190472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d9406e510 msgr2=0x7f4d94073cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:24.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.422+0000 7f4d9ac00640 1 --2- 172.21.15.7:0/1846190472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d9406e510 0x7f4d94073cd0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c009830 tx=0x7f4d7c02a3f0 comp rx=0 tx=0).stop 2024-09-10T21:51:24.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.422+0000 7f4d9ac00640 1 -- 172.21.15.7:0/1846190472 shutdown_connections 2024-09-10T21:51:24.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.422+0000 7f4d9ac00640 1 --2- 172.21.15.7:0/1846190472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d9406e510 0x7f4d94073cd0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:24.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.422+0000 7f4d9ac00640 1 --2- 172.21.15.7:0/1846190472 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d9406ff50 0x7f4d9415cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:24.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.422+0000 7f4d9ac00640 1 -- 172.21.15.7:0/1846190472 >> 172.21.15.7:0/1846190472 conn(0x7f4d940b1660 msgr2=0x7f4d940b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:24.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.422+0000 7f4d9ac00640 1 -- 172.21.15.7:0/1846190472 shutdown_connections 2024-09-10T21:51:24.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.422+0000 7f4d9ac00640 1 -- 172.21.15.7:0/1846190472 wait complete. 2024-09-10T21:51:24.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.423+0000 7f4d9ac00640 1 Processor -- start 2024-09-10T21:51:24.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.424+0000 7f4d9ac00640 1 -- start start 2024-09-10T21:51:24.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.425+0000 7f4d9ac00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d9406e510 0x7f4d94154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:24.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.425+0000 7f4d9ac00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d9406e930 0x7f4d941f17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:24.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.425+0000 7f4d9ac00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d9406ff50 0x7f4d940b4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:24.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.425+0000 7f4d9ac00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4d940745a0 con 0x7f4d9406e510 2024-09-10T21:51:24.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.425+0000 7f4d9ac00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4d94074420 con 0x7f4d9406ff50 2024-09-10T21:51:24.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.425+0000 7f4d9ac00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4d94074720 con 0x7f4d9406e930 2024-09-10T21:51:24.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.425+0000 7f4d93e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d9406e510 0x7f4d94154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:24.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.425+0000 7f4d93e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d9406e510 0x7f4d94154720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48738/0 (socket says 172.21.15.7:48738) 2024-09-10T21:51:24.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.425+0000 7f4d93e00640 1 -- 172.21.15.7:0/1874554046 learned_addr learned my addr 172.21.15.7:0/1874554046 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:24.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.425+0000 7f4d8b400640 1 --2- 172.21.15.7:0/1874554046 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d9406e930 0x7f4d941f17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:24.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.425+0000 7f4d98c00640 1 --2- 172.21.15.7:0/1874554046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d9406ff50 0x7f4d940b4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:24.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.425+0000 7f4d93e00640 1 -- 172.21.15.7:0/1874554046 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d9406e930 msgr2=0x7f4d941f17d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:24.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.425+0000 7f4d93e00640 1 --2- 172.21.15.7:0/1874554046 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d9406e930 0x7f4d941f17d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:24.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.425+0000 7f4d93e00640 1 -- 172.21.15.7:0/1874554046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d9406ff50 msgr2=0x7f4d940b4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:24.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.425+0000 7f4d93e00640 1 --2- 172.21.15.7:0/1874554046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d9406ff50 0x7f4d940b4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:24.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.425+0000 7f4d93e00640 1 -- 172.21.15.7:0/1874554046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d940b2ed0 con 0x7f4d9406e510 2024-09-10T21:51:24.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.425+0000 7f4d98c00640 1 --2- 172.21.15.7:0/1874554046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d9406ff50 0x7f4d940b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:51:24.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.426+0000 7f4d93e00640 1 --2- 172.21.15.7:0/1874554046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d9406e510 0x7f4d94154720 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400d770 tx=0x7f4d8400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:24.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.426+0000 7f4d90c00640 1 -- 172.21.15.7:0/1874554046 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84017070 con 0x7f4d9406e510 2024-09-10T21:51:24.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.426+0000 7f4d90c00640 1 -- 172.21.15.7:0/1874554046 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d84009d20 con 0x7f4d9406e510 2024-09-10T21:51:24.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.426+0000 7f4d9ac00640 1 -- 172.21.15.7:0/1874554046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d940b31b0 con 0x7f4d9406e510 2024-09-10T21:51:24.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.426+0000 7f4d90c00640 1 -- 172.21.15.7:0/1874554046 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d84004280 con 0x7f4d9406e510 2024-09-10T21:51:24.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.426+0000 7f4d9ac00640 1 -- 172.21.15.7:0/1874554046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d940b3770 con 0x7f4d9406e510 2024-09-10T21:51:24.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.427+0000 7f4d9ac00640 1 -- 172.21.15.7:0/1874554046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d94160130 con 0x7f4d9406e510 2024-09-10T21:51:24.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.428+0000 7f4d90c00640 1 -- 172.21.15.7:0/1874554046 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4d84005020 con 0x7f4d9406e510 2024-09-10T21:51:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.429+0000 7f4d90c00640 1 --2- 172.21.15.7:0/1874554046 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4d70077f70 0x7f4d7007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.429+0000 7f4d90c00640 1 -- 172.21.15.7:0/1874554046 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7f4d8409a1b0 con 0x7f4d9406e510 2024-09-10T21:51:24.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.430+0000 7f4d8b400640 1 --2- 172.21.15.7:0/1874554046 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4d70077f70 0x7f4d7007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:24.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.430+0000 7f4d8b400640 1 --2- 172.21.15.7:0/1874554046 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4d70077f70 0x7f4d7007a430 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c034000 tx=0x7f4d7c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:24.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.433+0000 7f4d90c00640 1 -- 172.21.15.7:0/1874554046 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4d84020020 con 0x7f4d9406e510 2024-09-10T21:51:24.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:24.669+0000 7f4d9ac00640 1 -- 172.21.15.7:0/1874554046 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4d94073da0 con 0x7f4d9406e510 2024-09-10T21:51:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.397+0000 7f4d90c00640 1 -- 172.21.15.7:0/1874554046 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v94) ==== 161+0+0 (secure 0 0 0) 0x7f4d84062bf0 con 0x7f4d9406e510 2024-09-10T21:51:25.399 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:51:25.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.400+0000 7f4d9ac00640 1 -- 172.21.15.7:0/1874554046 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4d70077f70 msgr2=0x7f4d7007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.400+0000 7f4d9ac00640 1 --2- 172.21.15.7:0/1874554046 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4d70077f70 0x7f4d7007a430 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c034000 tx=0x7f4d7c004730 comp rx=0 tx=0).stop 2024-09-10T21:51:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.400+0000 7f4d9ac00640 1 -- 172.21.15.7:0/1874554046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d9406e510 msgr2=0x7f4d94154720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.400+0000 7f4d9ac00640 1 --2- 172.21.15.7:0/1874554046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d9406e510 0x7f4d94154720 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f4d8400d770 tx=0x7f4d8400dc30 comp rx=0 tx=0).stop 2024-09-10T21:51:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.400+0000 7f4d9ac00640 1 -- 172.21.15.7:0/1874554046 shutdown_connections 2024-09-10T21:51:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.400+0000 7f4d9ac00640 1 --2- 172.21.15.7:0/1874554046 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4d70077f70 0x7f4d7007a430 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.400+0000 7f4d9ac00640 1 --2- 172.21.15.7:0/1874554046 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4d9406ff50 0x7f4d940b4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.400+0000 7f4d9ac00640 1 --2- 172.21.15.7:0/1874554046 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4d9406e930 0x7f4d941f17d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.400+0000 7f4d9ac00640 1 --2- 172.21.15.7:0/1874554046 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4d9406e510 0x7f4d94154720 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.400+0000 7f4d9ac00640 1 -- 172.21.15.7:0/1874554046 >> 172.21.15.7:0/1874554046 conn(0x7f4d940b1660 msgr2=0x7f4d940b1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.401+0000 7f4d9ac00640 1 -- 172.21.15.7:0/1874554046 shutdown_connections 2024-09-10T21:51:25.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.401+0000 7f4d9ac00640 1 -- 172.21.15.7:0/1874554046 wait complete. 2024-09-10T21:51:25.421 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:51:25.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.587+0000 7fb81a600640 1 -- 172.21.15.7:0/71100167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f8005910 msgr2=0x7fb7f8005d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:25.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.587+0000 7fb81a600640 1 --2- 172.21.15.7:0/71100167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f8005910 0x7fb7f8005d90 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc009b50 tx=0x7fb7f8006940 comp rx=0 tx=0).stop 2024-09-10T21:51:25.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.587+0000 7fb81a600640 1 -- 172.21.15.7:0/71100167 shutdown_connections 2024-09-10T21:51:25.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.587+0000 7fb81a600640 1 --2- 172.21.15.7:0/71100167 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f8005910 0x7fb7f8005d90 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:25.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.587+0000 7fb81a600640 1 -- 172.21.15.7:0/71100167 >> 172.21.15.7:0/71100167 conn(0x7fb8140b16d0 msgr2=0x7fb8140b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:25.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.587+0000 7fb81a600640 1 -- 172.21.15.7:0/71100167 shutdown_connections 2024-09-10T21:51:25.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.587+0000 7fb81a600640 1 -- 172.21.15.7:0/71100167 wait complete. 2024-09-10T21:51:25.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.589+0000 7fb81a600640 1 Processor -- start 2024-09-10T21:51:25.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.589+0000 7fb81a600640 1 -- start start 2024-09-10T21:51:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.590+0000 7fb81a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f8005910 0x7fb8141f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.590+0000 7fb81a600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb81415f930 0x7fb8141f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.590+0000 7fb81a600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8140b4eb0 0x7fb8140b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.590+0000 7fb81a600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fb814154810 con 0x7fb7f8005910 2024-09-10T21:51:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.590+0000 7fb81a600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fb814154690 con 0x7fb8140b4eb0 2024-09-10T21:51:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.590+0000 7fb81a600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fb81415a5e0 con 0x7fb81415f930 2024-09-10T21:51:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.590+0000 7fb813400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f8005910 0x7fb8141f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.590+0000 7fb813400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f8005910 0x7fb8141f56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48748/0 (socket says 172.21.15.7:48748) 2024-09-10T21:51:25.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.590+0000 7fb813400640 1 -- 172.21.15.7:0/129973734 learned_addr learned my addr 172.21.15.7:0/129973734 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:25.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.590+0000 7fb80aa00640 1 --2- 172.21.15.7:0/129973734 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb81415f930 0x7fb8141f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:25.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.590+0000 7fb813e00640 1 --2- 172.21.15.7:0/129973734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8140b4eb0 0x7fb8140b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:25.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.591+0000 7fb813400640 1 -- 172.21.15.7:0/129973734 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb81415f930 msgr2=0x7fb8141f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:25.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.591+0000 7fb813400640 1 --2- 172.21.15.7:0/129973734 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb81415f930 0x7fb8141f7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:25.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.591+0000 7fb813400640 1 -- 172.21.15.7:0/129973734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8140b4eb0 msgr2=0x7fb8140b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:25.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.591+0000 7fb813400640 1 --2- 172.21.15.7:0/129973734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8140b4eb0 0x7fb8140b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:25.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.591+0000 7fb813400640 1 -- 172.21.15.7:0/129973734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8141f5c10 con 0x7fb7f8005910 2024-09-10T21:51:25.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.591+0000 7fb813e00640 1 --2- 172.21.15.7:0/129973734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8140b4eb0 0x7fb8140b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:51:25.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.591+0000 7fb813400640 1 --2- 172.21.15.7:0/129973734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f8005910 0x7fb8141f56d0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fb804004c80 tx=0x7fb804001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:25.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.591+0000 7fb80be00640 1 -- 172.21.15.7:0/129973734 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804015020 con 0x7fb7f8005910 2024-09-10T21:51:25.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.591+0000 7fb80be00640 1 -- 172.21.15.7:0/129973734 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb804011980 con 0x7fb7f8005910 2024-09-10T21:51:25.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.591+0000 7fb80be00640 1 -- 172.21.15.7:0/129973734 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804011dd0 con 0x7fb7f8005910 2024-09-10T21:51:25.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.591+0000 7fb81a600640 1 -- 172.21.15.7:0/129973734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8141f7ff0 con 0x7fb7f8005910 2024-09-10T21:51:25.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.591+0000 7fb81a600640 1 -- 172.21.15.7:0/129973734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8140c5780 con 0x7fb7f8005910 2024-09-10T21:51:25.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.592+0000 7fb81a600640 1 -- 172.21.15.7:0/129973734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8140c6350 con 0x7fb7f8005910 2024-09-10T21:51:25.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.593+0000 7fb80be00640 1 -- 172.21.15.7:0/129973734 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fb80401a400 con 0x7fb7f8005910 2024-09-10T21:51:25.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.594+0000 7fb80be00640 1 --2- 172.21.15.7:0/129973734 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb7f0077fa0 0x7fb7f007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:25.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.595+0000 7fb80be00640 1 -- 172.21.15.7:0/129973734 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7fb80409aad0 con 0x7fb7f8005910 2024-09-10T21:51:25.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.595+0000 7fb80aa00640 1 --2- 172.21.15.7:0/129973734 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb7f0077fa0 0x7fb7f007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:25.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.596+0000 7fb80aa00640 1 --2- 172.21.15.7:0/129973734 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb7f0077fa0 0x7fb7f007a460 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc035000 tx=0x7fb7fc0046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:25.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.598+0000 7fb80be00640 1 -- 172.21.15.7:0/129973734 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fb80401f020 con 0x7fb7f8005910 2024-09-10T21:51:25.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:25.830+0000 7fb81a600640 1 -- 172.21.15.7:0/129973734 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb8140c3230 con 0x7fb7f8005910 2024-09-10T21:51:26.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.403+0000 7fb80be00640 1 -- 172.21.15.7:0/129973734 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v95) ==== 104+0+0 (secure 0 0 0) 0x7fb8141f7ff0 con 0x7fb7f8005910 2024-09-10T21:51:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.406+0000 7fb81a600640 1 -- 172.21.15.7:0/129973734 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb7f0077fa0 msgr2=0x7fb7f007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.406+0000 7fb81a600640 1 --2- 172.21.15.7:0/129973734 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb7f0077fa0 0x7fb7f007a460 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc035000 tx=0x7fb7fc0046c0 comp rx=0 tx=0).stop 2024-09-10T21:51:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.406+0000 7fb81a600640 1 -- 172.21.15.7:0/129973734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f8005910 msgr2=0x7fb8141f56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.406+0000 7fb81a600640 1 --2- 172.21.15.7:0/129973734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f8005910 0x7fb8141f56d0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fb804004c80 tx=0x7fb804001800 comp rx=0 tx=0).stop 2024-09-10T21:51:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.407+0000 7fb81a600640 1 -- 172.21.15.7:0/129973734 shutdown_connections 2024-09-10T21:51:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.407+0000 7fb81a600640 1 --2- 172.21.15.7:0/129973734 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fb7f0077fa0 0x7fb7f007a460 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.407+0000 7fb81a600640 1 --2- 172.21.15.7:0/129973734 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fb8140b4eb0 0x7fb8140b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.407+0000 7fb81a600640 1 --2- 172.21.15.7:0/129973734 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fb81415f930 0x7fb8141f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.407+0000 7fb81a600640 1 --2- 172.21.15.7:0/129973734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fb7f8005910 0x7fb8141f56d0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.407+0000 7fb81a600640 1 -- 172.21.15.7:0/129973734 >> 172.21.15.7:0/129973734 conn(0x7fb8140b16d0 msgr2=0x7fb814160430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.407+0000 7fb81a600640 1 -- 172.21.15.7:0/129973734 shutdown_connections 2024-09-10T21:51:26.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.407+0000 7fb81a600640 1 -- 172.21.15.7:0/129973734 wait complete. 2024-09-10T21:51:26.427 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:51:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.594+0000 7fbe64a00640 1 -- 172.21.15.7:0/2684385188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe6015bce0 msgr2=0x7fbe6006cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.594+0000 7fbe64a00640 1 --2- 172.21.15.7:0/2684385188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe6015bce0 0x7fbe6006cb20 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fbe58005fd0 tx=0x7fbe5802aa00 comp rx=0 tx=0).stop 2024-09-10T21:51:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.595+0000 7fbe64a00640 1 -- 172.21.15.7:0/2684385188 shutdown_connections 2024-09-10T21:51:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.595+0000 7fbe64a00640 1 --2- 172.21.15.7:0/2684385188 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe6015bce0 0x7fbe6006cb20 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.595+0000 7fbe64a00640 1 --2- 172.21.15.7:0/2684385188 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe600ace80 0x7fbe60070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.595+0000 7fbe64a00640 1 -- 172.21.15.7:0/2684385188 >> 172.21.15.7:0/2684385188 conn(0x7fbe600b1c10 msgr2=0x7fbe600b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.595+0000 7fbe64a00640 1 -- 172.21.15.7:0/2684385188 shutdown_connections 2024-09-10T21:51:26.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.595+0000 7fbe64a00640 1 -- 172.21.15.7:0/2684385188 wait complete. 2024-09-10T21:51:26.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.596+0000 7fbe64a00640 1 Processor -- start 2024-09-10T21:51:26.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.596+0000 7fbe64a00640 1 -- start start 2024-09-10T21:51:26.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.597+0000 7fbe64a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe600ace80 0x7fbe60154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:26.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.597+0000 7fbe64a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe6015b900 0x7fbe6009c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.597+0000 7fbe64a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe6015bce0 0x7fbe6009ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.597+0000 7fbe64a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fbe600aed00 con 0x7fbe600ace80 2024-09-10T21:51:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.597+0000 7fbe64a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fbe600aeb80 con 0x7fbe6015bce0 2024-09-10T21:51:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.597+0000 7fbe64a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fbe600aee80 con 0x7fbe6015b900 2024-09-10T21:51:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.598+0000 7fbe5de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe600ace80 0x7fbe60154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.598+0000 7fbe5de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe600ace80 0x7fbe60154ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48772/0 (socket says 172.21.15.7:48772) 2024-09-10T21:51:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.598+0000 7fbe5de00640 1 -- 172.21.15.7:0/731610990 learned_addr learned my addr 172.21.15.7:0/731610990 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.598+0000 7fbe5e800640 1 --2- 172.21.15.7:0/731610990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe6015bce0 0x7fbe6009ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:26.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.598+0000 7fbe57e00640 1 --2- 172.21.15.7:0/731610990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe6015b900 0x7fbe6009c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.598+0000 7fbe5de00640 1 -- 172.21.15.7:0/731610990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe6015b900 msgr2=0x7fbe6009c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.598+0000 7fbe5de00640 1 --2- 172.21.15.7:0/731610990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe6015b900 0x7fbe6009c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.598+0000 7fbe5de00640 1 -- 172.21.15.7:0/731610990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe6015bce0 msgr2=0x7fbe6009ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.598+0000 7fbe5de00640 1 --2- 172.21.15.7:0/731610990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe6015bce0 0x7fbe6009ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:26.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.598+0000 7fbe5de00640 1 -- 172.21.15.7:0/731610990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe60155400 con 0x7fbe600ace80 2024-09-10T21:51:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.598+0000 7fbe5de00640 1 --2- 172.21.15.7:0/731610990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe600ace80 0x7fbe60154ec0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000c450 tx=0x7fbe5000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.599+0000 7fbe56000640 1 -- 172.21.15.7:0/731610990 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe500052d0 con 0x7fbe600ace80 2024-09-10T21:51:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.599+0000 7fbe56000640 1 -- 172.21.15.7:0/731610990 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbe50005470 con 0x7fbe600ace80 2024-09-10T21:51:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.599+0000 7fbe56000640 1 -- 172.21.15.7:0/731610990 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe50012870 con 0x7fbe600ace80 2024-09-10T21:51:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.599+0000 7fbe64a00640 1 -- 172.21.15.7:0/731610990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe60073d10 con 0x7fbe600ace80 2024-09-10T21:51:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.599+0000 7fbe64a00640 1 -- 172.21.15.7:0/731610990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe60152bc0 con 0x7fbe600ace80 2024-09-10T21:51:26.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.600+0000 7fbe64a00640 1 -- 172.21.15.7:0/731610990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe600af330 con 0x7fbe600ace80 2024-09-10T21:51:26.603 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.601+0000 7fbe56000640 1 -- 172.21.15.7:0/731610990 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fbe500035f0 con 0x7fbe600ace80 2024-09-10T21:51:26.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.602+0000 7fbe56000640 1 --2- 172.21.15.7:0/731610990 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbe3c077fa0 0x7fbe3c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:26.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.602+0000 7fbe56000640 1 -- 172.21.15.7:0/731610990 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5881+0+0 (secure 0 0 0) 0x7fbe50099820 con 0x7fbe600ace80 2024-09-10T21:51:26.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.603+0000 7fbe57e00640 1 --2- 172.21.15.7:0/731610990 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbe3c077fa0 0x7fbe3c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:26.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.603+0000 7fbe57e00640 1 --2- 172.21.15.7:0/731610990 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbe3c077fa0 0x7fbe3c07a460 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbe600b2520 tx=0x7fbe4800b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:26.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.606+0000 7fbe56000640 1 -- 172.21.15.7:0/731610990 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fbe50061c10 con 0x7fbe600ace80 2024-09-10T21:51:26.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:26.802+0000 7fbe64a00640 1 -- 172.21.15.7:0/731610990 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbe6015a6d0 con 0x7fbe600ace80 2024-09-10T21:51:27.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.406+0000 7fbe56000640 1 -- 172.21.15.7:0/731610990 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) ==== 181+0+0 (secure 0 0 0) 0x7fbe50066640 con 0x7fbe600ace80 2024-09-10T21:51:27.408 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:51:27.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.408+0000 7fbe64a00640 1 -- 172.21.15.7:0/731610990 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbe3c077fa0 msgr2=0x7fbe3c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:27.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.408+0000 7fbe64a00640 1 --2- 172.21.15.7:0/731610990 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbe3c077fa0 0x7fbe3c07a460 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fbe600b2520 tx=0x7fbe4800b390 comp rx=0 tx=0).stop 2024-09-10T21:51:27.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.408+0000 7fbe64a00640 1 -- 172.21.15.7:0/731610990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe600ace80 msgr2=0x7fbe60154ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:27.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.408+0000 7fbe64a00640 1 --2- 172.21.15.7:0/731610990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe600ace80 0x7fbe60154ec0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000c450 tx=0x7fbe5000c910 comp rx=0 tx=0).stop 2024-09-10T21:51:27.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.408+0000 7fbe64a00640 1 -- 172.21.15.7:0/731610990 shutdown_connections 2024-09-10T21:51:27.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.408+0000 7fbe64a00640 1 --2- 172.21.15.7:0/731610990 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fbe3c077fa0 0x7fbe3c07a460 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:27.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.408+0000 7fbe64a00640 1 --2- 172.21.15.7:0/731610990 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fbe6015bce0 0x7fbe6009ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:27.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.408+0000 7fbe64a00640 1 --2- 172.21.15.7:0/731610990 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fbe6015b900 0x7fbe6009c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:27.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.408+0000 7fbe64a00640 1 --2- 172.21.15.7:0/731610990 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fbe600ace80 0x7fbe60154ec0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:27.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.408+0000 7fbe64a00640 1 -- 172.21.15.7:0/731610990 >> 172.21.15.7:0/731610990 conn(0x7fbe600b1c10 msgr2=0x7fbe6015abf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:27.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.409+0000 7fbe64a00640 1 -- 172.21.15.7:0/731610990 shutdown_connections 2024-09-10T21:51:27.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.409+0000 7fbe64a00640 1 -- 172.21.15.7:0/731610990 wait complete. 2024-09-10T21:51:27.428 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:51:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.598+0000 7f08cc800640 1 -- 172.21.15.7:0/2852780451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08c406fdb0 msgr2=0x7f08c406d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.598+0000 7f08cc800640 1 --2- 172.21.15.7:0/2852780451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08c406fdb0 0x7f08c406d230 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009be0 tx=0x7f08c002a8d0 comp rx=0 tx=0).stop 2024-09-10T21:51:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.598+0000 7f08cc800640 1 -- 172.21.15.7:0/2852780451 shutdown_connections 2024-09-10T21:51:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.598+0000 7f08cc800640 1 --2- 172.21.15.7:0/2852780451 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08c406fdb0 0x7f08c406d230 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.598+0000 7f08cc800640 1 --2- 172.21.15.7:0/2852780451 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08c4151a10 0x7f08c415ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.598+0000 7f08cc800640 1 -- 172.21.15.7:0/2852780451 >> 172.21.15.7:0/2852780451 conn(0x7f08c40b1c10 msgr2=0x7f08c40b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.598+0000 7f08cc800640 1 -- 172.21.15.7:0/2852780451 shutdown_connections 2024-09-10T21:51:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.598+0000 7f08cc800640 1 -- 172.21.15.7:0/2852780451 wait complete. 2024-09-10T21:51:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.600+0000 7f08cc800640 1 Processor -- start 2024-09-10T21:51:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.600+0000 7f08cc800640 1 -- start start 2024-09-10T21:51:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.601+0000 7f08cc800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08c406f970 0x7f08c414a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.601+0000 7f08cc800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08c406fdb0 0x7f08c4152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.601+0000 7f08cc800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08c4151a10 0x7f08c40aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.601+0000 7f08cc800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08c415e8d0 con 0x7f08c406fdb0 2024-09-10T21:51:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.601+0000 7f08cc800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f08c415e750 con 0x7f08c4151a10 2024-09-10T21:51:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.601+0000 7f08cc800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f08c4073e50 con 0x7f08c406f970 2024-09-10T21:51:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.601+0000 7f08c9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08c406fdb0 0x7f08c4152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.601+0000 7f08c9400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08c406fdb0 0x7f08c4152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48788/0 (socket says 172.21.15.7:48788) 2024-09-10T21:51:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.601+0000 7f08c9400640 1 -- 172.21.15.7:0/2347933049 learned_addr learned my addr 172.21.15.7:0/2347933049 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.602+0000 7f08c9e00640 1 --2- 172.21.15.7:0/2347933049 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08c406f970 0x7f08c414a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.602+0000 7f08ca800640 1 --2- 172.21.15.7:0/2347933049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08c4151a10 0x7f08c40aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.602+0000 7f08c9400640 1 -- 172.21.15.7:0/2347933049 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08c406f970 msgr2=0x7f08c414a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.602+0000 7f08c9400640 1 --2- 172.21.15.7:0/2347933049 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08c406f970 0x7f08c414a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.602+0000 7f08c9400640 1 -- 172.21.15.7:0/2347933049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08c4151a10 msgr2=0x7f08c40aff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.602+0000 7f08c9400640 1 --2- 172.21.15.7:0/2347933049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08c4151a10 0x7f08c40aff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.602+0000 7f08c9400640 1 -- 172.21.15.7:0/2347933049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08c40cc480 con 0x7f08c406fdb0 2024-09-10T21:51:27.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.602+0000 7f08c9400640 1 --2- 172.21.15.7:0/2347933049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08c406fdb0 0x7f08c4152a20 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f08b400c460 tx=0x7f08b400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.602+0000 7f08be000640 1 -- 172.21.15.7:0/2347933049 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b4016020 con 0x7f08c406fdb0 2024-09-10T21:51:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.603+0000 7f08be000640 1 -- 172.21.15.7:0/2347933049 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08b4012920 con 0x7f08c406fdb0 2024-09-10T21:51:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.603+0000 7f08be000640 1 -- 172.21.15.7:0/2347933049 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b4012d50 con 0x7f08c406fdb0 2024-09-10T21:51:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.603+0000 7f08cc800640 1 -- 172.21.15.7:0/2347933049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08c4152f60 con 0x7f08c406fdb0 2024-09-10T21:51:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.603+0000 7f08cc800640 1 -- 172.21.15.7:0/2347933049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08c41534e0 con 0x7f08c406fdb0 2024-09-10T21:51:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.604+0000 7f08cc800640 1 -- 172.21.15.7:0/2347933049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08c406d8e0 con 0x7f08c406fdb0 2024-09-10T21:51:27.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.605+0000 7f08be000640 1 -- 172.21.15.7:0/2347933049 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f08b401a3b0 con 0x7f08c406fdb0 2024-09-10T21:51:27.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.606+0000 7f08be000640 1 --2- 172.21.15.7:0/2347933049 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f08a8077f70 0x7f08a807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:27.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.606+0000 7f08be000640 1 -- 172.21.15.7:0/2347933049 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5443+0+0 (secure 0 0 0) 0x7f08b4099af0 con 0x7f08c406fdb0 2024-09-10T21:51:27.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.606+0000 7f08c9e00640 1 --2- 172.21.15.7:0/2347933049 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f08a8077f70 0x7f08a807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:27.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.607+0000 7f08c9e00640 1 --2- 172.21.15.7:0/2347933049 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f08a8077f70 0x7f08a807a430 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009b30 tx=0x7f08b8005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:27.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.610+0000 7f08be000640 1 -- 172.21.15.7:0/2347933049 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f08b4020030 con 0x7f08c406fdb0 2024-09-10T21:51:27.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:27.811+0000 7f08cc800640 1 -- 172.21.15.7:0/2347933049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f08c415b8b0 con 0x7f08c406fdb0 2024-09-10T21:51:28.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.418+0000 7f08be000640 1 -- 172.21.15.7:0/2347933049 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) ==== 169+0+0 (secure 0 0 0) 0x7f08c40cc480 con 0x7f08c406fdb0 2024-09-10T21:51:28.420 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-09-10T21:51:28.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.421+0000 7f08cc800640 1 -- 172.21.15.7:0/2347933049 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f08a8077f70 msgr2=0x7f08a807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:28.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.421+0000 7f08cc800640 1 --2- 172.21.15.7:0/2347933049 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f08a8077f70 0x7f08a807a430 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009b30 tx=0x7f08b8005f90 comp rx=0 tx=0).stop 2024-09-10T21:51:28.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.421+0000 7f08cc800640 1 -- 172.21.15.7:0/2347933049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08c406fdb0 msgr2=0x7f08c4152a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:28.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.421+0000 7f08cc800640 1 --2- 172.21.15.7:0/2347933049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08c406fdb0 0x7f08c4152a20 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f08b400c460 tx=0x7f08b400c920 comp rx=0 tx=0).stop 2024-09-10T21:51:28.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.421+0000 7f08cc800640 1 -- 172.21.15.7:0/2347933049 shutdown_connections 2024-09-10T21:51:28.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.421+0000 7f08cc800640 1 --2- 172.21.15.7:0/2347933049 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f08a8077f70 0x7f08a807a430 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:28.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.421+0000 7f08cc800640 1 --2- 172.21.15.7:0/2347933049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08c4151a10 0x7f08c40aff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:28.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.421+0000 7f08cc800640 1 --2- 172.21.15.7:0/2347933049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08c406fdb0 0x7f08c4152a20 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:28.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.421+0000 7f08cc800640 1 --2- 172.21.15.7:0/2347933049 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08c406f970 0x7f08c414a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:28.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.422+0000 7f08cc800640 1 -- 172.21.15.7:0/2347933049 >> 172.21.15.7:0/2347933049 conn(0x7f08c40b1c10 msgr2=0x7f08c4072e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:28.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.422+0000 7f08cc800640 1 -- 172.21.15.7:0/2347933049 shutdown_connections 2024-09-10T21:51:28.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.422+0000 7f08cc800640 1 -- 172.21.15.7:0/2347933049 wait complete. 2024-09-10T21:51:28.442 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:51:28.608 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.606+0000 7fe902c00640 1 -- 172.21.15.7:0/601581953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc06e960 msgr2=0x7fe8fc074210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.606+0000 7fe902c00640 1 --2- 172.21.15.7:0/601581953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc06e960 0x7fe8fc074210 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0b2620 tx=0x7fe8f402a8f0 comp rx=0 tx=0).stop 2024-09-10T21:51:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.606+0000 7fe902c00640 1 -- 172.21.15.7:0/601581953 shutdown_connections 2024-09-10T21:51:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.606+0000 7fe902c00640 1 --2- 172.21.15.7:0/601581953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc06e960 0x7fe8fc074210 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.606+0000 7fe902c00640 1 --2- 172.21.15.7:0/601581953 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe8fc06e510 0x7fe8fc073cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.606+0000 7fe902c00640 1 --2- 172.21.15.7:0/601581953 >> v2:172.21.15.133:3300/0 conn(0x7fe8fc06ff50 0x7fe8fc15cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.606+0000 7fe902c00640 1 -- 172.21.15.7:0/601581953 >> 172.21.15.7:0/601581953 conn(0x7fe8fc0b1660 msgr2=0x7fe8fc0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.606+0000 7fe902c00640 1 -- 172.21.15.7:0/601581953 shutdown_connections 2024-09-10T21:51:28.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.606+0000 7fe902c00640 1 -- 172.21.15.7:0/601581953 wait complete. 2024-09-10T21:51:28.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.608+0000 7fe902c00640 1 Processor -- start 2024-09-10T21:51:28.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.608+0000 7fe902c00640 1 -- start start 2024-09-10T21:51:28.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.609+0000 7fe902c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe8fc06e510 0x7fe8fc151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:28.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.609+0000 7fe902c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8fc06e960 0x7fe8fc151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:28.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.609+0000 7fe902c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc06ff50 0x7fe8fc0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:28.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.609+0000 7fe902c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc15f710 con 0x7fe8fc06ff50 2024-09-10T21:51:28.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.609+0000 7fe902c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe8fc06f930 con 0x7fe8fc06e960 2024-09-10T21:51:28.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.609+0000 7fe902c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe8fc15f890 con 0x7fe8fc06e510 2024-09-10T21:51:28.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.609+0000 7fe900c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc06ff50 0x7fe8fc0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:28.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.609+0000 7fe8fbe00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe8fc06e510 0x7fe8fc151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:28.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.609+0000 7fe8fbe00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe8fc06e510 0x7fe8fc151830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:59034/0 (socket says 172.21.15.7:59034) 2024-09-10T21:51:28.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.609+0000 7fe8fbe00640 1 -- 172.21.15.7:0/1958659044 learned_addr learned my addr 172.21.15.7:0/1958659044 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:28.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.609+0000 7fe8f3400640 1 --2- 172.21.15.7:0/1958659044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8fc06e960 0x7fe8fc151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:28.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.610+0000 7fe900c00640 1 -- 172.21.15.7:0/1958659044 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe8fc06e510 msgr2=0x7fe8fc151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:28.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.610+0000 7fe900c00640 1 --2- 172.21.15.7:0/1958659044 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe8fc06e510 0x7fe8fc151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:28.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.610+0000 7fe900c00640 1 -- 172.21.15.7:0/1958659044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8fc06e960 msgr2=0x7fe8fc151f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:28.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.610+0000 7fe900c00640 1 --2- 172.21.15.7:0/1958659044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8fc06e960 0x7fe8fc151f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:28.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.610+0000 7fe900c00640 1 -- 172.21.15.7:0/1958659044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8fc0b2a00 con 0x7fe8fc06ff50 2024-09-10T21:51:28.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.610+0000 7fe8fbe00640 1 --2- 172.21.15.7:0/1958659044 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe8fc06e510 0x7fe8fc151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:51:28.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.610+0000 7fe900c00640 1 --2- 172.21.15.7:0/1958659044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc06ff50 0x7fe8fc0af9c0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0749c0 tx=0x7fe8f4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:28.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.610+0000 7fe8f8c00640 1 -- 172.21.15.7:0/1958659044 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4004040 con 0x7fe8fc06ff50 2024-09-10T21:51:28.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.610+0000 7fe8f8c00640 1 -- 172.21.15.7:0/1958659044 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8f402d070 con 0x7fe8fc06ff50 2024-09-10T21:51:28.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.610+0000 7fe902c00640 1 -- 172.21.15.7:0/1958659044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8fc1f8710 con 0x7fe8fc06ff50 2024-09-10T21:51:28.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.610+0000 7fe8f8c00640 1 -- 172.21.15.7:0/1958659044 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4004440 con 0x7fe8fc06ff50 2024-09-10T21:51:28.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.610+0000 7fe902c00640 1 -- 172.21.15.7:0/1958659044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8fc0b5490 con 0x7fe8fc06ff50 2024-09-10T21:51:28.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.611+0000 7fe902c00640 1 -- 172.21.15.7:0/1958659044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8fc15cbe0 con 0x7fe8fc06ff50 2024-09-10T21:51:28.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.613+0000 7fe8f8c00640 1 -- 172.21.15.7:0/1958659044 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe8f40045e0 con 0x7fe8fc06ff50 2024-09-10T21:51:28.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.613+0000 7fe8f8c00640 1 --2- 172.21.15.7:0/1958659044 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe8d8077fa0 0x7fe8d807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:28.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.614+0000 7fe8f8c00640 1 -- 172.21.15.7:0/1958659044 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5029+0+0 (secure 0 0 0) 0x7fe8f40bb350 con 0x7fe8fc06ff50 2024-09-10T21:51:28.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.614+0000 7fe8fbe00640 1 --2- 172.21.15.7:0/1958659044 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe8d8077fa0 0x7fe8d807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:28.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.615+0000 7fe8fbe00640 1 --2- 172.21.15.7:0/1958659044 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe8d8077fa0 0x7fe8d807a460 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0058c0 tx=0x7fe8ec00b410 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:28.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.618+0000 7fe8f8c00640 1 -- 172.21.15.7:0/1958659044 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe8f4083a90 con 0x7fe8fc06ff50 2024-09-10T21:51:28.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.813+0000 7fe902c00640 1 -- 172.21.15.7:0/1958659044 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7fe8fc15f450 con 0x7fe8fc06ff50 2024-09-10T21:51:28.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.817+0000 7fe8f8c00640 1 -- 172.21.15.7:0/1958659044 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v39) ==== 71+0+0 (secure 0 0 0) 0x7fe8f40884c0 con 0x7fe8fc06ff50 2024-09-10T21:51:28.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.819+0000 7fe902c00640 1 -- 172.21.15.7:0/1958659044 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe8d8077fa0 msgr2=0x7fe8d807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:28.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.819+0000 7fe902c00640 1 --2- 172.21.15.7:0/1958659044 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe8d8077fa0 0x7fe8d807a460 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec0058c0 tx=0x7fe8ec00b410 comp rx=0 tx=0).stop 2024-09-10T21:51:28.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.819+0000 7fe902c00640 1 -- 172.21.15.7:0/1958659044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc06ff50 msgr2=0x7fe8fc0af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:28.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.819+0000 7fe902c00640 1 --2- 172.21.15.7:0/1958659044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc06ff50 0x7fe8fc0af9c0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0749c0 tx=0x7fe8f4003ba0 comp rx=0 tx=0).stop 2024-09-10T21:51:28.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.819+0000 7fe902c00640 1 -- 172.21.15.7:0/1958659044 shutdown_connections 2024-09-10T21:51:28.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.819+0000 7fe902c00640 1 --2- 172.21.15.7:0/1958659044 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe8d8077fa0 0x7fe8d807a460 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:28.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.819+0000 7fe902c00640 1 --2- 172.21.15.7:0/1958659044 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe8fc06e960 0x7fe8fc151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:28.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.819+0000 7fe902c00640 1 --2- 172.21.15.7:0/1958659044 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe8fc06ff50 0x7fe8fc0af9c0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:28.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.819+0000 7fe902c00640 1 --2- 172.21.15.7:0/1958659044 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe8fc06e510 0x7fe8fc151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:28.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.820+0000 7fe902c00640 1 -- 172.21.15.7:0/1958659044 >> 172.21.15.7:0/1958659044 conn(0x7fe8fc0b1660 msgr2=0x7fe8fc0cc780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:28.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.820+0000 7fe902c00640 1 -- 172.21.15.7:0/1958659044 shutdown_connections 2024-09-10T21:51:28.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.820+0000 7fe902c00640 1 -- 172.21.15.7:0/1958659044 wait complete. 2024-09-10T21:51:28.838 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync' 2024-09-10T21:51:29.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.999+0000 7f512f000640 1 -- 172.21.15.7:0/1182173750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512815fda0 msgr2=0x7f51281601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:29.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.999+0000 7f512f000640 1 --2- 172.21.15.7:0/1182173750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512815fda0 0x7f51281601f0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f5120005fd0 tx=0x7f512002a780 comp rx=0 tx=0).stop 2024-09-10T21:51:29.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.999+0000 7f512f000640 1 -- 172.21.15.7:0/1182173750 shutdown_connections 2024-09-10T21:51:29.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.999+0000 7f512f000640 1 --2- 172.21.15.7:0/1182173750 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512815fda0 0x7f51281601f0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:29.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.999+0000 7f512f000640 1 --2- 172.21.15.7:0/1182173750 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51280cd750 0x7f51280ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:29.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.999+0000 7f512f000640 1 -- 172.21.15.7:0/1182173750 >> 172.21.15.7:0/1182173750 conn(0x7f51280b16d0 msgr2=0x7f51280b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:29.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.999+0000 7f512f000640 1 -- 172.21.15.7:0/1182173750 shutdown_connections 2024-09-10T21:51:29.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:28.999+0000 7f512f000640 1 -- 172.21.15.7:0/1182173750 wait complete. 2024-09-10T21:51:29.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.001+0000 7f512f000640 1 Processor -- start 2024-09-10T21:51:29.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.001+0000 7f512f000640 1 -- start start 2024-09-10T21:51:29.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.002+0000 7f512f000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51280cd750 0x7f51280c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:29.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.002+0000 7f512f000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f512815bc90 0x7f51281f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:29.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.002+0000 7f512f000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512815fda0 0x7f51280aee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:29.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.002+0000 7f512f000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f512806fe70 con 0x7f512815fda0 2024-09-10T21:51:29.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.002+0000 7f512f000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f512806fcf0 con 0x7f512815bc90 2024-09-10T21:51:29.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.002+0000 7f512f000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f512806fff0 con 0x7f51280cd750 2024-09-10T21:51:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.002+0000 7f512d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512815fda0 0x7f51280aee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.002+0000 7f512d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512815fda0 0x7f51280aee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48820/0 (socket says 172.21.15.7:48820) 2024-09-10T21:51:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.002+0000 7f512d000640 1 -- 172.21.15.7:0/2294038185 learned_addr learned my addr 172.21.15.7:0/2294038185 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.003+0000 7f511f400640 1 --2- 172.21.15.7:0/2294038185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f512815bc90 0x7f51281f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.003+0000 7f5127e00640 1 --2- 172.21.15.7:0/2294038185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51280cd750 0x7f51280c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.003+0000 7f512d000640 1 -- 172.21.15.7:0/2294038185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51280cd750 msgr2=0x7f51280c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.003+0000 7f512d000640 1 --2- 172.21.15.7:0/2294038185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51280cd750 0x7f51280c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.003+0000 7f512d000640 1 -- 172.21.15.7:0/2294038185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f512815bc90 msgr2=0x7f51281f4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.003+0000 7f512d000640 1 --2- 172.21.15.7:0/2294038185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f512815bc90 0x7f51281f4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.003+0000 7f512d000640 1 -- 172.21.15.7:0/2294038185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51280c97e0 con 0x7f512815fda0 2024-09-10T21:51:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.003+0000 7f511f400640 1 --2- 172.21.15.7:0/2294038185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f512815bc90 0x7f51281f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:51:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.003+0000 7f512d000640 1 --2- 172.21.15.7:0/2294038185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512815fda0 0x7f51280aee80 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f5120034040 tx=0x7f5120003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.003+0000 7f5124c00640 1 -- 172.21.15.7:0/2294038185 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5120040070 con 0x7f512815fda0 2024-09-10T21:51:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.003+0000 7f5124c00640 1 -- 172.21.15.7:0/2294038185 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5120004330 con 0x7f512815fda0 2024-09-10T21:51:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.003+0000 7f5124c00640 1 -- 172.21.15.7:0/2294038185 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5120004740 con 0x7f512815fda0 2024-09-10T21:51:29.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.003+0000 7f512f000640 1 -- 172.21.15.7:0/2294038185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51280c7190 con 0x7f512815fda0 2024-09-10T21:51:29.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.004+0000 7f512f000640 1 -- 172.21.15.7:0/2294038185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51281f4a00 con 0x7f512815fda0 2024-09-10T21:51:29.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.005+0000 7f512f000640 1 -- 172.21.15.7:0/2294038185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5128070510 con 0x7f512815fda0 2024-09-10T21:51:29.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.006+0000 7f5124c00640 1 -- 172.21.15.7:0/2294038185 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f512003c420 con 0x7f512815fda0 2024-09-10T21:51:29.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.007+0000 7f5124c00640 1 --2- 172.21.15.7:0/2294038185 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5104077f70 0x7f510407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:29.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.007+0000 7f5124c00640 1 -- 172.21.15.7:0/2294038185 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5029+0+0 (secure 0 0 0) 0x7f51200baf50 con 0x7f512815fda0 2024-09-10T21:51:29.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.008+0000 7f5127e00640 1 --2- 172.21.15.7:0/2294038185 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5104077f70 0x7f510407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:29.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.008+0000 7f5127e00640 1 --2- 172.21.15.7:0/2294038185 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5104077f70 0x7f510407a430 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f51280c59e0 tx=0x7f5118009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:29.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.011+0000 7f5124c00640 1 -- 172.21.15.7:0/2294038185 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5120083cf0 con 0x7f512815fda0 2024-09-10T21:51:29.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.281+0000 7f512f000640 1 -- 172.21.15.7:0/2294038185 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]} v 0) -- 0x7f512815a730 con 0x7f512815fda0 2024-09-10T21:51:29.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.421+0000 7f5124c00640 1 -- 172.21.15.7:0/2294038185 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_inotable_sync"]}]=0 v429) ==== 146+0+0 (secure 0 0 0) 0x7f51200835f0 con 0x7f512815fda0 2024-09-10T21:51:29.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.424+0000 7f512f000640 1 -- 172.21.15.7:0/2294038185 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5104077f70 msgr2=0x7f510407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:29.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.424+0000 7f512f000640 1 --2- 172.21.15.7:0/2294038185 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5104077f70 0x7f510407a430 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f51280c59e0 tx=0x7f5118009040 comp rx=0 tx=0).stop 2024-09-10T21:51:29.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.424+0000 7f512f000640 1 -- 172.21.15.7:0/2294038185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512815fda0 msgr2=0x7f51280aee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:29.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.424+0000 7f512f000640 1 --2- 172.21.15.7:0/2294038185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512815fda0 0x7f51280aee80 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f5120034040 tx=0x7f5120003ba0 comp rx=0 tx=0).stop 2024-09-10T21:51:29.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.424+0000 7f512f000640 1 -- 172.21.15.7:0/2294038185 shutdown_connections 2024-09-10T21:51:29.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.424+0000 7f512f000640 1 --2- 172.21.15.7:0/2294038185 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5104077f70 0x7f510407a430 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:29.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.424+0000 7f512f000640 1 --2- 172.21.15.7:0/2294038185 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f512815bc90 0x7f51281f4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:29.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.424+0000 7f512f000640 1 --2- 172.21.15.7:0/2294038185 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f512815fda0 0x7f51280aee80 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:29.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.424+0000 7f512f000640 1 --2- 172.21.15.7:0/2294038185 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f51280cd750 0x7f51280c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:29.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.424+0000 7f512f000640 1 -- 172.21.15.7:0/2294038185 >> 172.21.15.7:0/2294038185 conn(0x7f51280b16d0 msgr2=0x7f5128072360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:29.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.425+0000 7f512f000640 1 -- 172.21.15.7:0/2294038185 shutdown_connections 2024-09-10T21:51:29.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.425+0000 7f512f000640 1 -- 172.21.15.7:0/2294038185 wait complete. 2024-09-10T21:51:29.444 INFO:tasks.cephfs_test_runner:test_inotable_sync (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-09-10T21:51:29.444 INFO:tasks.cephfs_test_runner:Starting test: test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2024-09-10T21:51:29.444 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:51:29.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.603+0000 7fcda6a00640 1 -- 172.21.15.7:0/1041449927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00cd750 msgr2=0x7fcda0073730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:29.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.603+0000 7fcda6a00640 1 --2- 172.21.15.7:0/1041449927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00cd750 0x7fcda0073730 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fcda006dac0 tx=0x7fcd8c02a560 comp rx=0 tx=0).stop 2024-09-10T21:51:29.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.603+0000 7fcda6a00640 1 -- 172.21.15.7:0/1041449927 shutdown_connections 2024-09-10T21:51:29.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.603+0000 7fcda6a00640 1 --2- 172.21.15.7:0/1041449927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00cd750 0x7fcda0073730 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:29.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.603+0000 7fcda6a00640 1 --2- 172.21.15.7:0/1041449927 >> v2:172.21.15.133:3300/0 conn(0x7fcda015bc90 0x7fcda00ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:29.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.603+0000 7fcda6a00640 1 -- 172.21.15.7:0/1041449927 >> 172.21.15.7:0/1041449927 conn(0x7fcda00b16d0 msgr2=0x7fcda00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:29.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.603+0000 7fcda6a00640 1 -- 172.21.15.7:0/1041449927 shutdown_connections 2024-09-10T21:51:29.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.603+0000 7fcda6a00640 1 -- 172.21.15.7:0/1041449927 wait complete. 2024-09-10T21:51:29.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.606+0000 7fcda6a00640 1 Processor -- start 2024-09-10T21:51:29.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.606+0000 7fcda6a00640 1 -- start start 2024-09-10T21:51:29.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcda6a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcda0073c70 0x7fcda015c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:29.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcda6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00cd750 0x7fcda015e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:29.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcda6a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcda015bc90 0x7fcda014ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:29.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcda6a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcda006fcf0 con 0x7fcda00cd750 2024-09-10T21:51:29.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcda6a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcda01606c0 con 0x7fcda0073c70 2024-09-10T21:51:29.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcda6a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fcda006fe70 con 0x7fcda015bc90 2024-09-10T21:51:29.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcd9f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00cd750 0x7fcda015e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:29.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcd9f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00cd750 0x7fcda015e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48842/0 (socket says 172.21.15.7:48842) 2024-09-10T21:51:29.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcd9f400640 1 -- 172.21.15.7:0/278352958 learned_addr learned my addr 172.21.15.7:0/278352958 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:29.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcd9fe00640 1 --2- 172.21.15.7:0/278352958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcda0073c70 0x7fcda015c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:29.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcda4a00640 1 --2- 172.21.15.7:0/278352958 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcda015bc90 0x7fcda014ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:29.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcd9f400640 1 -- 172.21.15.7:0/278352958 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcda015bc90 msgr2=0x7fcda014ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:29.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcd9f400640 1 --2- 172.21.15.7:0/278352958 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcda015bc90 0x7fcda014ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:29.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcd9f400640 1 -- 172.21.15.7:0/278352958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcda0073c70 msgr2=0x7fcda015c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:29.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcd9f400640 1 --2- 172.21.15.7:0/278352958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcda0073c70 0x7fcda015c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:29.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcd9f400640 1 -- 172.21.15.7:0/278352958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcda015c8b0 con 0x7fcda00cd750 2024-09-10T21:51:29.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcd9f400640 1 --2- 172.21.15.7:0/278352958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00cd750 0x7fcda015e0f0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009850 tx=0x7fcd8c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:29.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcd97e00640 1 -- 172.21.15.7:0/278352958 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c03f070 con 0x7fcda00cd750 2024-09-10T21:51:29.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcd97e00640 1 -- 172.21.15.7:0/278352958 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd8c0047a0 con 0x7fcda00cd750 2024-09-10T21:51:29.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcd97e00640 1 -- 172.21.15.7:0/278352958 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c035030 con 0x7fcda00cd750 2024-09-10T21:51:29.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.607+0000 7fcda6a00640 1 -- 172.21.15.7:0/278352958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcda015ea10 con 0x7fcda00cd750 2024-09-10T21:51:29.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.608+0000 7fcda6a00640 1 -- 172.21.15.7:0/278352958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcda015efd0 con 0x7fcda00cd750 2024-09-10T21:51:29.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.609+0000 7fcda6a00640 1 -- 172.21.15.7:0/278352958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcda0073730 con 0x7fcda00cd750 2024-09-10T21:51:29.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.610+0000 7fcd97e00640 1 -- 172.21.15.7:0/278352958 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fcd8c004bb0 con 0x7fcda00cd750 2024-09-10T21:51:29.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.610+0000 7fcd97e00640 1 --2- 172.21.15.7:0/278352958 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcd80077fa0 0x7fcd8007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:29.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.611+0000 7fcd97e00640 1 -- 172.21.15.7:0/278352958 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5029+0+0 (secure 0 0 0) 0x7fcd8c0bac50 con 0x7fcda00cd750 2024-09-10T21:51:29.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.611+0000 7fcd9fe00640 1 --2- 172.21.15.7:0/278352958 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcd80077fa0 0x7fcd8007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:29.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.612+0000 7fcd9fe00640 1 --2- 172.21.15.7:0/278352958 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcd80077fa0 0x7fcd8007a460 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcda00c6620 tx=0x7fcd9000b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:29.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.615+0000 7fcd97e00640 1 -- 172.21.15.7:0/278352958 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fcd8c083280 con 0x7fcda00cd750 2024-09-10T21:51:29.813 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.811+0000 7fcda6a00640 1 -- 172.21.15.7:0/278352958 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcda015ff40 con 0x7fcda00cd750 2024-09-10T21:51:29.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.811+0000 7fcd97e00640 1 -- 172.21.15.7:0/278352958 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+13124 (secure 0 0 0) 0x7fcd8c087cb0 con 0x7fcda00cd750 2024-09-10T21:51:29.814 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:29.814 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":105,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:51:28.414821+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.7:6835/4110623302":"2024-09-11T21:51:21.267938+0000","172.21.15.7:6833/4110623302":"2024-09-11T21:51:21.267938+0000","172.21.15.7:6835/3075619232":"2024-09-11T21:50:44.284783+0000","172.21.15.7:6833/3075619232":"2024-09-11T21:50:44.284783+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:51:29.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.814+0000 7fcda6a00640 1 -- 172.21.15.7:0/278352958 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcd80077fa0 msgr2=0x7fcd8007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:29.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.814+0000 7fcda6a00640 1 --2- 172.21.15.7:0/278352958 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcd80077fa0 0x7fcd8007a460 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fcda00c6620 tx=0x7fcd9000b2e0 comp rx=0 tx=0).stop 2024-09-10T21:51:29.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.814+0000 7fcda6a00640 1 -- 172.21.15.7:0/278352958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00cd750 msgr2=0x7fcda015e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:29.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.814+0000 7fcda6a00640 1 --2- 172.21.15.7:0/278352958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00cd750 0x7fcda015e0f0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c009850 tx=0x7fcd8c003910 comp rx=0 tx=0).stop 2024-09-10T21:51:29.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.814+0000 7fcda6a00640 1 -- 172.21.15.7:0/278352958 shutdown_connections 2024-09-10T21:51:29.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.814+0000 7fcda6a00640 1 --2- 172.21.15.7:0/278352958 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcd80077fa0 0x7fcd8007a460 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:29.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.814+0000 7fcda6a00640 1 --2- 172.21.15.7:0/278352958 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcda015bc90 0x7fcda014ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:29.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.814+0000 7fcda6a00640 1 --2- 172.21.15.7:0/278352958 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcda00cd750 0x7fcda015e0f0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:29.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.814+0000 7fcda6a00640 1 --2- 172.21.15.7:0/278352958 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcda0073c70 0x7fcda015c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:29.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.814+0000 7fcda6a00640 1 -- 172.21.15.7:0/278352958 >> 172.21.15.7:0/278352958 conn(0x7fcda00b16d0 msgr2=0x7fcda0072d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:29.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.814+0000 7fcda6a00640 1 -- 172.21.15.7:0/278352958 shutdown_connections 2024-09-10T21:51:29.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.814+0000 7fcda6a00640 1 -- 172.21.15.7:0/278352958 wait complete. 2024-09-10T21:51:29.833 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:41:55.889618+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T21:51:29.834 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:51:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.996+0000 7f673d400640 1 -- 172.21.15.7:0/3888784196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c003c30 msgr2=0x7f671c004030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.996+0000 7f673d400640 1 --2- 172.21.15.7:0/3888784196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c003c30 0x7f671c004030 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f6728006f70 tx=0x7f671c005830 comp rx=0 tx=0).stop 2024-09-10T21:51:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.996+0000 7f673d400640 1 -- 172.21.15.7:0/3888784196 shutdown_connections 2024-09-10T21:51:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.996+0000 7f673d400640 1 --2- 172.21.15.7:0/3888784196 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c003c30 0x7f671c004030 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.996+0000 7f673d400640 1 -- 172.21.15.7:0/3888784196 >> 172.21.15.7:0/3888784196 conn(0x7f67380b16d0 msgr2=0x7f67380b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.997+0000 7f673d400640 1 -- 172.21.15.7:0/3888784196 shutdown_connections 2024-09-10T21:51:29.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.997+0000 7f673d400640 1 -- 172.21.15.7:0/3888784196 wait complete. 2024-09-10T21:51:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.998+0000 7f673d400640 1 Processor -- start 2024-09-10T21:51:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.998+0000 7f673d400640 1 -- start start 2024-09-10T21:51:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.999+0000 7f673d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c003c30 0x7f67380c4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.999+0000 7f673d400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67381f4ad0 0x7f67380b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.999+0000 7f673d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67380c3650 0x7f6738153ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.999+0000 7f673d400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6738160490 con 0x7f671c003c30 2024-09-10T21:51:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.999+0000 7f673d400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6738160310 con 0x7f67380c3650 2024-09-10T21:51:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:29.999+0000 7f673d400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f6738073d40 con 0x7f67381f4ad0 2024-09-10T21:51:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.000+0000 7f6736800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c003c30 0x7f67380c4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.000+0000 7f6736800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c003c30 0x7f67380c4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48848/0 (socket says 172.21.15.7:48848) 2024-09-10T21:51:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.000+0000 7f6736800640 1 -- 172.21.15.7:0/2359477393 learned_addr learned my addr 172.21.15.7:0/2359477393 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.000+0000 7f6735e00640 1 --2- 172.21.15.7:0/2359477393 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67381f4ad0 0x7f67380b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.000+0000 7f6737200640 1 --2- 172.21.15.7:0/2359477393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67380c3650 0x7f6738153ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.000+0000 7f6736800640 1 -- 172.21.15.7:0/2359477393 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67381f4ad0 msgr2=0x7f67380b41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:30.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.000+0000 7f6736800640 1 --2- 172.21.15.7:0/2359477393 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67381f4ad0 0x7f67380b41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.000+0000 7f6736800640 1 -- 172.21.15.7:0/2359477393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67380c3650 msgr2=0x7f6738153ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.000+0000 7f6736800640 1 --2- 172.21.15.7:0/2359477393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67380c3650 0x7f6738153ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.000+0000 7f6736800640 1 -- 172.21.15.7:0/2359477393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67380b4710 con 0x7f671c003c30 2024-09-10T21:51:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.000+0000 7f6736800640 1 --2- 172.21.15.7:0/2359477393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c003c30 0x7f67380c4820 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f6728031040 tx=0x7f6728003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.001+0000 7f672ea00640 1 -- 172.21.15.7:0/2359477393 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6728001c30 con 0x7f671c003c30 2024-09-10T21:51:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.001+0000 7f672ea00640 1 -- 172.21.15.7:0/2359477393 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67280060d0 con 0x7f671c003c30 2024-09-10T21:51:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.001+0000 7f672ea00640 1 -- 172.21.15.7:0/2359477393 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f672802e6a0 con 0x7f671c003c30 2024-09-10T21:51:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.001+0000 7f673d400640 1 -- 172.21.15.7:0/2359477393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67381f5a50 con 0x7f671c003c30 2024-09-10T21:51:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.001+0000 7f673d400640 1 -- 172.21.15.7:0/2359477393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67381f5490 con 0x7f671c003c30 2024-09-10T21:51:30.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.002+0000 7f673d400640 1 -- 172.21.15.7:0/2359477393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f673806ed40 con 0x7f671c003c30 2024-09-10T21:51:30.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.004+0000 7f672ea00640 1 -- 172.21.15.7:0/2359477393 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f672802e840 con 0x7f671c003c30 2024-09-10T21:51:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.004+0000 7f672ea00640 1 --2- 172.21.15.7:0/2359477393 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6718077fa0 0x7f671807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.004+0000 7f672ea00640 1 -- 172.21.15.7:0/2359477393 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5029+0+0 (secure 0 0 0) 0x7f67280b9670 con 0x7f671c003c30 2024-09-10T21:51:30.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.005+0000 7f6735e00640 1 --2- 172.21.15.7:0/2359477393 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6718077fa0 0x7f671807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:30.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.005+0000 7f6735e00640 1 --2- 172.21.15.7:0/2359477393 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6718077fa0 0x7f671807a460 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6724000c00 tx=0x7f6724006b10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:30.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.008+0000 7f672ea00640 1 -- 172.21.15.7:0/2359477393 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f672803e070 con 0x7f671c003c30 2024-09-10T21:51:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.205+0000 7f673d400640 1 -- 172.21.15.7:0/2359477393 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f67380b1b50 con 0x7f671c003c30 2024-09-10T21:51:30.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.206+0000 7f672ea00640 1 -- 172.21.15.7:0/2359477393 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v105) ==== 93+0+10 (secure 0 0 0) 0x7f6728081db0 con 0x7f671c003c30 2024-09-10T21:51:30.207 INFO:teuthology.orchestra.run.smithi007.stdout:pg_num: 1 2024-09-10T21:51:30.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.208+0000 7f673d400640 1 -- 172.21.15.7:0/2359477393 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6718077fa0 msgr2=0x7f671807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:30.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.208+0000 7f673d400640 1 --2- 172.21.15.7:0/2359477393 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6718077fa0 0x7f671807a460 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f6724000c00 tx=0x7f6724006b10 comp rx=0 tx=0).stop 2024-09-10T21:51:30.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.208+0000 7f673d400640 1 -- 172.21.15.7:0/2359477393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c003c30 msgr2=0x7f67380c4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.208+0000 7f673d400640 1 --2- 172.21.15.7:0/2359477393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c003c30 0x7f67380c4820 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f6728031040 tx=0x7f6728003e80 comp rx=0 tx=0).stop 2024-09-10T21:51:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.208+0000 7f673d400640 1 -- 172.21.15.7:0/2359477393 shutdown_connections 2024-09-10T21:51:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.208+0000 7f673d400640 1 --2- 172.21.15.7:0/2359477393 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6718077fa0 0x7f671807a460 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.208+0000 7f673d400640 1 --2- 172.21.15.7:0/2359477393 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67380c3650 0x7f6738153ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.208+0000 7f673d400640 1 --2- 172.21.15.7:0/2359477393 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67381f4ad0 0x7f67380b41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.208+0000 7f673d400640 1 --2- 172.21.15.7:0/2359477393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f671c003c30 0x7f67380c4820 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.208+0000 7f673d400640 1 -- 172.21.15.7:0/2359477393 >> 172.21.15.7:0/2359477393 conn(0x7f67380b16d0 msgr2=0x7f673815da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.208+0000 7f673d400640 1 -- 172.21.15.7:0/2359477393 shutdown_connections 2024-09-10T21:51:30.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.208+0000 7f673d400640 1 -- 172.21.15.7:0/2359477393 wait complete. 2024-09-10T21:51:30.225 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-09-10T21:51:30.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.383+0000 7f52a4a00640 1 -- 172.21.15.7:0/1451381393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52a015fda0 msgr2=0x7f52a01601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:30.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.383+0000 7f52a4a00640 1 --2- 172.21.15.7:0/1451381393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52a015fda0 0x7f52a01601f0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f5298005fd0 tx=0x7f529802a8f0 comp rx=0 tx=0).stop 2024-09-10T21:51:30.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.383+0000 7f52a4a00640 1 -- 172.21.15.7:0/1451381393 shutdown_connections 2024-09-10T21:51:30.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.383+0000 7f52a4a00640 1 --2- 172.21.15.7:0/1451381393 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52a015fda0 0x7f52a01601f0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:30.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.383+0000 7f52a4a00640 1 --2- 172.21.15.7:0/1451381393 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52a00cd750 0x7f52a00ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:30.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.383+0000 7f52a4a00640 1 -- 172.21.15.7:0/1451381393 >> 172.21.15.7:0/1451381393 conn(0x7f52a00b16d0 msgr2=0x7f52a00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:30.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.384+0000 7f52a4a00640 1 -- 172.21.15.7:0/1451381393 shutdown_connections 2024-09-10T21:51:30.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.384+0000 7f52a4a00640 1 -- 172.21.15.7:0/1451381393 wait complete. 2024-09-10T21:51:30.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.385+0000 7f52a4a00640 1 Processor -- start 2024-09-10T21:51:30.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.385+0000 7f52a4a00640 1 -- start start 2024-09-10T21:51:30.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.386+0000 7f52a4a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52a00cd750 0x7f52a0158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:30.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.386+0000 7f52a4a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52a015bc90 0x7f52a00ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:30.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.386+0000 7f52a4a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52a015fda0 0x7f52a0150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:30.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.386+0000 7f52a4a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f52a006fe70 con 0x7f52a015fda0 2024-09-10T21:51:30.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.386+0000 7f52a4a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f52a006fcf0 con 0x7f52a015bc90 2024-09-10T21:51:30.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.386+0000 7f52a4a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f52a006fff0 con 0x7f52a00cd750 2024-09-10T21:51:30.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.386+0000 7f529e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52a015fda0 0x7f52a0150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:30.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.387+0000 7f529e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52a015fda0 0x7f52a0150e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48870/0 (socket says 172.21.15.7:48870) 2024-09-10T21:51:30.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.387+0000 7f529e800640 1 -- 172.21.15.7:0/1662114988 learned_addr learned my addr 172.21.15.7:0/1662114988 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:30.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.387+0000 7f5297e00640 1 --2- 172.21.15.7:0/1662114988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52a015bc90 0x7f52a00ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:30.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.387+0000 7f529de00640 1 --2- 172.21.15.7:0/1662114988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52a00cd750 0x7f52a0158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:30.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.387+0000 7f529e800640 1 -- 172.21.15.7:0/1662114988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52a00cd750 msgr2=0x7f52a0158380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:30.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.387+0000 7f529e800640 1 --2- 172.21.15.7:0/1662114988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52a00cd750 0x7f52a0158380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.387+0000 7f529e800640 1 -- 172.21.15.7:0/1662114988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52a015bc90 msgr2=0x7f52a00ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.387+0000 7f529e800640 1 --2- 172.21.15.7:0/1662114988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52a015bc90 0x7f52a00ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.387+0000 7f529e800640 1 -- 172.21.15.7:0/1662114988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a00caa00 con 0x7f52a015fda0 2024-09-10T21:51:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.387+0000 7f529e800640 1 --2- 172.21.15.7:0/1662114988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52a015fda0 0x7f52a0150e00 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f5298033040 tx=0x7f5298003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.388+0000 7f5296000640 1 -- 172.21.15.7:0/1662114988 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298004040 con 0x7f52a015fda0 2024-09-10T21:51:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.388+0000 7f5296000640 1 -- 172.21.15.7:0/1662114988 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f529802d070 con 0x7f52a015fda0 2024-09-10T21:51:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.388+0000 7f52a4a00640 1 -- 172.21.15.7:0/1662114988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a00cace0 con 0x7f52a015fda0 2024-09-10T21:51:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.388+0000 7f5296000640 1 -- 172.21.15.7:0/1662114988 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298004420 con 0x7f52a015fda0 2024-09-10T21:51:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.388+0000 7f52a4a00640 1 -- 172.21.15.7:0/1662114988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a00cb2a0 con 0x7f52a015fda0 2024-09-10T21:51:30.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.389+0000 7f52a4a00640 1 -- 172.21.15.7:0/1662114988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52a0070570 con 0x7f52a015fda0 2024-09-10T21:51:30.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.390+0000 7f5296000640 1 -- 172.21.15.7:0/1662114988 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f52980045c0 con 0x7f52a015fda0 2024-09-10T21:51:30.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.391+0000 7f5296000640 1 --2- 172.21.15.7:0/1662114988 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f527c077fa0 0x7f527c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:30.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.391+0000 7f5296000640 1 -- 172.21.15.7:0/1662114988 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5029+0+0 (secure 0 0 0) 0x7f52980bb3b0 con 0x7f52a015fda0 2024-09-10T21:51:30.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.391+0000 7f529de00640 1 --2- 172.21.15.7:0/1662114988 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f527c077fa0 0x7f527c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:30.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.392+0000 7f529de00640 1 --2- 172.21.15.7:0/1662114988 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f527c077fa0 0x7f527c07a460 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5290004f60 tx=0x7f5290007e30 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:30.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.395+0000 7f5296000640 1 -- 172.21.15.7:0/1662114988 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f52980840b0 con 0x7f52a015fda0 2024-09-10T21:51:30.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:30.664+0000 7f52a4a00640 1 -- 172.21.15.7:0/1662114988 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f52a00b1df0 con 0x7f52a015fda0 2024-09-10T21:51:31.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.429+0000 7f5296000640 1 -- 172.21.15.7:0/1662114988 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v431) ==== 147+0+0 (secure 0 0 0) 0x7f52980839b0 con 0x7f52a015fda0 2024-09-10T21:51:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.433+0000 7f52a4a00640 1 -- 172.21.15.7:0/1662114988 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f527c077fa0 msgr2=0x7f527c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.433+0000 7f52a4a00640 1 --2- 172.21.15.7:0/1662114988 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f527c077fa0 0x7f527c07a460 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f5290004f60 tx=0x7f5290007e30 comp rx=0 tx=0).stop 2024-09-10T21:51:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.433+0000 7f52a4a00640 1 -- 172.21.15.7:0/1662114988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52a015fda0 msgr2=0x7f52a0150e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.433+0000 7f52a4a00640 1 --2- 172.21.15.7:0/1662114988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52a015fda0 0x7f52a0150e00 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7f5298033040 tx=0x7f5298003ba0 comp rx=0 tx=0).stop 2024-09-10T21:51:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.433+0000 7f52a4a00640 1 -- 172.21.15.7:0/1662114988 shutdown_connections 2024-09-10T21:51:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.433+0000 7f52a4a00640 1 --2- 172.21.15.7:0/1662114988 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f527c077fa0 0x7f527c07a460 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.433+0000 7f52a4a00640 1 --2- 172.21.15.7:0/1662114988 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f52a015bc90 0x7f52a00ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.433+0000 7f52a4a00640 1 --2- 172.21.15.7:0/1662114988 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f52a015fda0 0x7f52a0150e00 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.433+0000 7f52a4a00640 1 --2- 172.21.15.7:0/1662114988 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f52a00cd750 0x7f52a0158380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.433+0000 7f52a4a00640 1 -- 172.21.15.7:0/1662114988 >> 172.21.15.7:0/1662114988 conn(0x7f52a00b16d0 msgr2=0x7f52a00b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:31.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.433+0000 7f52a4a00640 1 -- 172.21.15.7:0/1662114988 shutdown_connections 2024-09-10T21:51:31.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.433+0000 7f52a4a00640 1 -- 172.21.15.7:0/1662114988 wait complete. 2024-09-10T21:51:31.452 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-10T21:51:31.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.616+0000 7f4850800640 1 -- 172.21.15.7:0/81564610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f484815bc90 msgr2=0x7f48480ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:31.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.616+0000 7f4850800640 1 --2- 172.21.15.7:0/81564610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f484815bc90 0x7f48480ccef0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f483c009830 tx=0x7f483c02a3f0 comp rx=0 tx=0).stop 2024-09-10T21:51:31.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.616+0000 7f4850800640 1 -- 172.21.15.7:0/81564610 shutdown_connections 2024-09-10T21:51:31.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.616+0000 7f4850800640 1 --2- 172.21.15.7:0/81564610 >> v2:172.21.15.133:3300/0 conn(0x7f484815fda0 0x7f48481601f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:31.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.616+0000 7f4850800640 1 --2- 172.21.15.7:0/81564610 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f484815bc90 0x7f48480ccef0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:31.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.616+0000 7f4850800640 1 -- 172.21.15.7:0/81564610 >> 172.21.15.7:0/81564610 conn(0x7f48480b16d0 msgr2=0x7f48480b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:31.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.616+0000 7f4850800640 1 -- 172.21.15.7:0/81564610 shutdown_connections 2024-09-10T21:51:31.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.616+0000 7f4850800640 1 -- 172.21.15.7:0/81564610 wait complete. 2024-09-10T21:51:31.619 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:51:31.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.618+0000 7f4850800640 1 Processor -- start 2024-09-10T21:51:31.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.618+0000 7f4850800640 1 -- start start 2024-09-10T21:51:31.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.619+0000 7f4850800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48480cd750 0x7f484815a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:31.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.619+0000 7f4850800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f484815bc90 0x7f48481f4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:31.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.619+0000 7f4850800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f484815fda0 0x7f48480b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:31.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.619+0000 7f4850800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f484806fe70 con 0x7f484815fda0 2024-09-10T21:51:31.620 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.619+0000 7f4850800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f484806fcf0 con 0x7f484815bc90 2024-09-10T21:51:31.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.619+0000 7f4850800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f484806fff0 con 0x7f48480cd750 2024-09-10T21:51:31.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.619+0000 7f484e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f484815fda0 0x7f48480b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:31.623 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:51:31.623 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.619+0000 7f484e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f484815fda0 0x7f48480b39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48874/0 (socket says 172.21.15.7:48874) 2024-09-10T21:51:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.619+0000 7f484e800640 1 -- 172.21.15.7:0/1424118620 learned_addr learned my addr 172.21.15.7:0/1424118620 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.620+0000 7f484de00640 1 --2- 172.21.15.7:0/1424118620 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48480cd750 0x7f484815a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.619+0000 7f484d400640 1 --2- 172.21.15.7:0/1424118620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f484815bc90 0x7f48481f4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.620+0000 7f484e800640 1 -- 172.21.15.7:0/1424118620 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48480cd750 msgr2=0x7f484815a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.620+0000 7f484e800640 1 --2- 172.21.15.7:0/1424118620 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48480cd750 0x7f484815a1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.620+0000 7f484e800640 1 -- 172.21.15.7:0/1424118620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f484815bc90 msgr2=0x7f48481f4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.620+0000 7f484e800640 1 --2- 172.21.15.7:0/1424118620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f484815bc90 0x7f48481f4d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.620+0000 7f484e800640 1 -- 172.21.15.7:0/1424118620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48481f52c0 con 0x7f484815fda0 2024-09-10T21:51:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.620+0000 7f484d400640 1 --2- 172.21.15.7:0/1424118620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f484815bc90 0x7f48481f4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:51:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.620+0000 7f484e800640 1 --2- 172.21.15.7:0/1424118620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f484815fda0 0x7f48480b39e0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f484400eb70 tx=0x7f484400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.620+0000 7f4842000640 1 -- 172.21.15.7:0/1424118620 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844003b70 con 0x7f484815fda0 2024-09-10T21:51:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.621+0000 7f4842000640 1 -- 172.21.15.7:0/1424118620 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4844003e00 con 0x7f484815fda0 2024-09-10T21:51:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.621+0000 7f4850800640 1 -- 172.21.15.7:0/1424118620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f484815e160 con 0x7f484815fda0 2024-09-10T21:51:31.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.621+0000 7f4842000640 1 -- 172.21.15.7:0/1424118620 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844004280 con 0x7f484815fda0 2024-09-10T21:51:31.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.621+0000 7f4850800640 1 -- 172.21.15.7:0/1424118620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48480adbd0 con 0x7f484815fda0 2024-09-10T21:51:31.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.622+0000 7f4850800640 1 -- 172.21.15.7:0/1424118620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4848070510 con 0x7f484815fda0 2024-09-10T21:51:31.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.623+0000 7f4842000640 1 -- 172.21.15.7:0/1424118620 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4844004420 con 0x7f484815fda0 2024-09-10T21:51:31.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.625+0000 7f4842000640 1 --2- 172.21.15.7:0/1424118620 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f482c077f70 0x7f482c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:31.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.625+0000 7f4842000640 1 -- 172.21.15.7:0/1424118620 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5029+0+0 (secure 0 0 0) 0x7f484409a8d0 con 0x7f484815fda0 2024-09-10T21:51:31.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.625+0000 7f484de00640 1 --2- 172.21.15.7:0/1424118620 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f482c077f70 0x7f482c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:31.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.626+0000 7f484de00640 1 --2- 172.21.15.7:0/1424118620 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f482c077f70 0x7f482c07a430 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f483c034000 tx=0x7f483c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:31.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.628+0000 7f4842000640 1 -- 172.21.15.7:0/1424118620 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4844063550 con 0x7f484815fda0 2024-09-10T21:51:31.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.825+0000 7f4850800640 1 -- 172.21.15.7:0/1424118620 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f48480733f0 con 0x7f484815fda0 2024-09-10T21:51:31.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.826+0000 7f4842000640 1 -- 172.21.15.7:0/1424118620 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v39) ==== 59+0+1998 (secure 0 0 0) 0x7f4844062e50 con 0x7f484815fda0 2024-09-10T21:51:31.827 INFO:teuthology.orchestra.run.smithi007.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-10T21:51:31.827 INFO:teuthology.orchestra.run.smithi007.stdout:mon advanced mon_op_complaint_time 120 2024-09-10T21:51:31.827 INFO:teuthology.orchestra.run.smithi007.stdout:mon dev mon_warn_on_pool_no_app false 2024-09-10T21:51:31.827 INFO:teuthology.orchestra.run.smithi007.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-10T21:51:31.827 INFO:teuthology.orchestra.run.smithi007.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3770.177547 2024-09-10T21:51:31.827 INFO:teuthology.orchestra.run.smithi007.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3542.062395 2024-09-10T21:51:31.827 INFO:teuthology.orchestra.run.smithi007.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3788.037135 2024-09-10T21:51:31.827 INFO:teuthology.orchestra.run.smithi007.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 5808.986321 2024-09-10T21:51:31.828 INFO:teuthology.orchestra.run.smithi007.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4630.109911 2024-09-10T21:51:31.828 INFO:teuthology.orchestra.run.smithi007.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4510.255493 2024-09-10T21:51:31.828 INFO:teuthology.orchestra.run.smithi007.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4792.698527 2024-09-10T21:51:31.828 INFO:teuthology.orchestra.run.smithi007.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 6842.370281 2024-09-10T21:51:31.828 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_mds 20/20 2024-09-10T21:51:31.828 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_mds_balancer 20/20 2024-09-10T21:51:31.828 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced debug_ms 2/2 2024-09-10T21:51:31.828 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_debug_frag true 2024-09-10T21:51:31.828 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_debug_scatterstat true 2024-09-10T21:51:31.828 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-10T21:51:31.828 INFO:teuthology.orchestra.run.smithi007.stdout:mds dev mds_verify_scatter true 2024-09-10T21:51:31.828 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-10T21:51:31.828 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced rados_mon_op_timeout 900 2024-09-10T21:51:31.828 INFO:teuthology.orchestra.run.smithi007.stdout:mds advanced rados_osd_op_timeout 900 2024-09-10T21:51:31.829 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced client_mount_timeout 600 2024-09-10T21:51:31.829 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced debug_client 20/20 2024-09-10T21:51:31.829 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced debug_ms 1/1 2024-09-10T21:51:31.829 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced rados_mon_op_timeout 900 2024-09-10T21:51:31.829 INFO:teuthology.orchestra.run.smithi007.stdout:client advanced rados_osd_op_timeout 900 2024-09-10T21:51:31.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.828+0000 7f4850800640 1 -- 172.21.15.7:0/1424118620 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f482c077f70 msgr2=0x7f482c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:31.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.828+0000 7f4850800640 1 --2- 172.21.15.7:0/1424118620 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f482c077f70 0x7f482c07a430 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f483c034000 tx=0x7f483c004730 comp rx=0 tx=0).stop 2024-09-10T21:51:31.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.828+0000 7f4850800640 1 -- 172.21.15.7:0/1424118620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f484815fda0 msgr2=0x7f48480b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:31.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.828+0000 7f4850800640 1 --2- 172.21.15.7:0/1424118620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f484815fda0 0x7f48480b39e0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f484400eb70 tx=0x7f484400c6a0 comp rx=0 tx=0).stop 2024-09-10T21:51:31.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.829+0000 7f4850800640 1 -- 172.21.15.7:0/1424118620 shutdown_connections 2024-09-10T21:51:31.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.829+0000 7f4850800640 1 --2- 172.21.15.7:0/1424118620 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f482c077f70 0x7f482c07a430 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:31.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.829+0000 7f4850800640 1 --2- 172.21.15.7:0/1424118620 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f484815bc90 0x7f48481f4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:31.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.829+0000 7f4850800640 1 --2- 172.21.15.7:0/1424118620 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f484815fda0 0x7f48480b39e0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:31.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.829+0000 7f4850800640 1 --2- 172.21.15.7:0/1424118620 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f48480cd750 0x7f484815a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:31.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.829+0000 7f4850800640 1 -- 172.21.15.7:0/1424118620 >> 172.21.15.7:0/1424118620 conn(0x7f48480b16d0 msgr2=0x7f4848072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:31.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.829+0000 7f4850800640 1 -- 172.21.15.7:0/1424118620 shutdown_connections 2024-09-10T21:51:31.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:31.829+0000 7f4850800640 1 -- 172.21.15.7:0/1424118620 wait complete. 2024-09-10T21:51:31.845 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-10T21:51:32.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.009+0000 7f71da000640 1 -- 172.21.15.7:0/2963133569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d4151960 msgr2=0x7f71d40cb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:32.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.009+0000 7f71da000640 1 --2- 172.21.15.7:0/2963133569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d4151960 0x7f71d40cb190 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f71c4005fd0 tx=0x7f71c402a530 comp rx=0 tx=0).stop 2024-09-10T21:51:32.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.009+0000 7f71da000640 1 -- 172.21.15.7:0/2963133569 shutdown_connections 2024-09-10T21:51:32.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.009+0000 7f71da000640 1 --2- 172.21.15.7:0/2963133569 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71d406f690 0x7f71d406fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:32.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.009+0000 7f71da000640 1 --2- 172.21.15.7:0/2963133569 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d4151960 0x7f71d40cb190 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:32.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.009+0000 7f71da000640 1 -- 172.21.15.7:0/2963133569 >> 172.21.15.7:0/2963133569 conn(0x7f71d40b16d0 msgr2=0x7f71d40b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:32.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.009+0000 7f71da000640 1 -- 172.21.15.7:0/2963133569 shutdown_connections 2024-09-10T21:51:32.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.009+0000 7f71da000640 1 -- 172.21.15.7:0/2963133569 wait complete. 2024-09-10T21:51:32.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.011+0000 7f71da000640 1 Processor -- start 2024-09-10T21:51:32.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.011+0000 7f71da000640 1 -- start start 2024-09-10T21:51:32.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.012+0000 7f71da000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d406f690 0x7f71d414d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:32.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.012+0000 7f71da000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71d4151960 0x7f71d40cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:32.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.012+0000 7f71da000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71d415e650 0x7f71d40d0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:32.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.012+0000 7f71da000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f71d40cb870 con 0x7f71d406f690 2024-09-10T21:51:32.014 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.012+0000 7f71da000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f71d4151f50 con 0x7f71d4151960 2024-09-10T21:51:32.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.012+0000 7f71da000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f71d415f1d0 con 0x7f71d415e650 2024-09-10T21:51:32.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.012+0000 7f71d3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d406f690 0x7f71d414d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:32.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.012+0000 7f71d3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d406f690 0x7f71d414d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48886/0 (socket says 172.21.15.7:48886) 2024-09-10T21:51:32.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.012+0000 7f71d3400640 1 -- 172.21.15.7:0/1716924294 learned_addr learned my addr 172.21.15.7:0/1716924294 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:32.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.013+0000 7f71d3400640 1 -- 172.21.15.7:0/1716924294 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71d415e650 msgr2=0x7f71d40d0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:32.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.013+0000 7f71caa00640 1 --2- 172.21.15.7:0/1716924294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71d4151960 0x7f71d40cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:32.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.013+0000 7f71d3e00640 1 --2- 172.21.15.7:0/1716924294 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71d415e650 0x7f71d40d0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:32.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.013+0000 7f71d3400640 1 --2- 172.21.15.7:0/1716924294 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71d415e650 0x7f71d40d0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:32.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.013+0000 7f71d3400640 1 -- 172.21.15.7:0/1716924294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71d4151960 msgr2=0x7f71d40cfb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:32.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.013+0000 7f71d3400640 1 --2- 172.21.15.7:0/1716924294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71d4151960 0x7f71d40cfb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:32.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.013+0000 7f71d3400640 1 -- 172.21.15.7:0/1716924294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71d41f2320 con 0x7f71d406f690 2024-09-10T21:51:32.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.013+0000 7f71d3e00640 1 --2- 172.21.15.7:0/1716924294 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71d415e650 0x7f71d40d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:51:32.015 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.013+0000 7f71caa00640 1 --2- 172.21.15.7:0/1716924294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71d4151960 0x7f71d40cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:51:32.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.013+0000 7f71d3400640 1 --2- 172.21.15.7:0/1716924294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d406f690 0x7f71d414d2c0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009850 tx=0x7f71c40039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:32.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.013+0000 7f71cbe00640 1 -- 172.21.15.7:0/1716924294 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c403f070 con 0x7f71d406f690 2024-09-10T21:51:32.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.013+0000 7f71cbe00640 1 -- 172.21.15.7:0/1716924294 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71c4004890 con 0x7f71d406f690 2024-09-10T21:51:32.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.013+0000 7f71cbe00640 1 -- 172.21.15.7:0/1716924294 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71c4035030 con 0x7f71d406f690 2024-09-10T21:51:32.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.013+0000 7f71da000640 1 -- 172.21.15.7:0/1716924294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71d41f66b0 con 0x7f71d406f690 2024-09-10T21:51:32.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.014+0000 7f71da000640 1 -- 172.21.15.7:0/1716924294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71d41f2880 con 0x7f71d406f690 2024-09-10T21:51:32.016 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.014+0000 7f71da000640 1 -- 172.21.15.7:0/1716924294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71d4073d80 con 0x7f71d406f690 2024-09-10T21:51:32.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.016+0000 7f71cbe00640 1 -- 172.21.15.7:0/1716924294 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f71c4004c60 con 0x7f71d406f690 2024-09-10T21:51:32.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.017+0000 7f71cbe00640 1 --2- 172.21.15.7:0/1716924294 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f71b0077f70 0x7f71b007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:32.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.017+0000 7f71cbe00640 1 -- 172.21.15.7:0/1716924294 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5029+0+0 (secure 0 0 0) 0x7f71c40bac70 con 0x7f71d406f690 2024-09-10T21:51:32.017 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.017+0000 7f71caa00640 1 --2- 172.21.15.7:0/1716924294 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f71b0077f70 0x7f71b007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:32.018 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.018+0000 7f71caa00640 1 --2- 172.21.15.7:0/1716924294 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f71b0077f70 0x7f71b007a430 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f71bc006fd0 tx=0x7f71bc008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:32.021 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.021+0000 7f71cbe00640 1 -- 172.21.15.7:0/1716924294 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f71d41f2320 con 0x7f71d406f690 2024-09-10T21:51:32.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.216+0000 7f71da000640 1 -- 172.21.15.7:0/1716924294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f71d4069880 con 0x7f71d406f690 2024-09-10T21:51:32.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.220+0000 7f71cbe00640 1 -- 172.21.15.7:0/1716924294 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7f71c4083a00 con 0x7f71d406f690 2024-09-10T21:51:32.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.222+0000 7f71da000640 1 -- 172.21.15.7:0/1716924294 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f71b0077f70 msgr2=0x7f71b007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:32.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.222+0000 7f71da000640 1 --2- 172.21.15.7:0/1716924294 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f71b0077f70 0x7f71b007a430 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f71bc006fd0 tx=0x7f71bc008040 comp rx=0 tx=0).stop 2024-09-10T21:51:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.222+0000 7f71da000640 1 -- 172.21.15.7:0/1716924294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d406f690 msgr2=0x7f71d414d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.222+0000 7f71da000640 1 --2- 172.21.15.7:0/1716924294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d406f690 0x7f71d414d2c0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009850 tx=0x7f71c40039e0 comp rx=0 tx=0).stop 2024-09-10T21:51:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.222+0000 7f71da000640 1 -- 172.21.15.7:0/1716924294 shutdown_connections 2024-09-10T21:51:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.222+0000 7f71da000640 1 --2- 172.21.15.7:0/1716924294 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f71b0077f70 0x7f71b007a430 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.222+0000 7f71da000640 1 --2- 172.21.15.7:0/1716924294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f71d4151960 0x7f71d40cfb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.222+0000 7f71da000640 1 --2- 172.21.15.7:0/1716924294 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f71d415e650 0x7f71d40d0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.223+0000 7f71da000640 1 --2- 172.21.15.7:0/1716924294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f71d406f690 0x7f71d414d2c0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.223+0000 7f71da000640 1 -- 172.21.15.7:0/1716924294 >> 172.21.15.7:0/1716924294 conn(0x7f71d40b16d0 msgr2=0x7f71d4073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.223+0000 7f71da000640 1 -- 172.21.15.7:0/1716924294 shutdown_connections 2024-09-10T21:51:32.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.223+0000 7f71da000640 1 -- 172.21.15.7:0/1716924294 wait complete. 2024-09-10T21:51:32.239 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:51:32.240 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:51:32.333 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:51:32.433 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:51:32.433 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:51:32.433 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:51:32.468 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:51:32.469 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:51:32.469 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:51:32.506 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:51:32.506 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:51:32.506 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:51:32.543 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:51:32.544 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:51:32.544 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:51:32.582 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:51:32.582 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-10T21:51:32.582 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:51:32.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.742+0000 7f082c800640 1 -- 172.21.15.7:0/544151233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08240ae020 msgr2=0x7f08240ae4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:32.743 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.742+0000 7f082c800640 1 --2- 172.21.15.7:0/544151233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08240ae020 0x7f08240ae4d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0820005fd0 tx=0x7f082002a8b0 comp rx=0 tx=0).stop 2024-09-10T21:51:32.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.742+0000 7f082c800640 1 -- 172.21.15.7:0/544151233 shutdown_connections 2024-09-10T21:51:32.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.742+0000 7f082c800640 1 --2- 172.21.15.7:0/544151233 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08240ae020 0x7f08240ae4d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:32.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.742+0000 7f082c800640 1 --2- 172.21.15.7:0/544151233 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08240cbd50 0x7f08240cc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:32.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.742+0000 7f082c800640 1 -- 172.21.15.7:0/544151233 >> 172.21.15.7:0/544151233 conn(0x7f08240b16d0 msgr2=0x7f08240b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:32.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.742+0000 7f082c800640 1 -- 172.21.15.7:0/544151233 shutdown_connections 2024-09-10T21:51:32.744 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.742+0000 7f082c800640 1 -- 172.21.15.7:0/544151233 wait complete. 2024-09-10T21:51:32.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.744+0000 7f082c800640 1 Processor -- start 2024-09-10T21:51:32.745 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.744+0000 7f082c800640 1 -- start start 2024-09-10T21:51:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.745+0000 7f082c800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08240ae020 0x7f08240c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.745+0000 7f082c800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08240cbd50 0x7f08240c9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.745+0000 7f082c800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08241619e0 0x7f082406e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.745+0000 7f082c800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f08240ce100 con 0x7f08240ae020 2024-09-10T21:51:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.745+0000 7f082c800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f08240cdf80 con 0x7f08241619e0 2024-09-10T21:51:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.745+0000 7f082c800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f0824161dc0 con 0x7f08240cbd50 2024-09-10T21:51:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.745+0000 7f0829e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08240ae020 0x7f08240c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.745+0000 7f0829400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08240cbd50 0x7f08240c9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.745+0000 7f0829e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08240ae020 0x7f08240c9520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48896/0 (socket says 172.21.15.7:48896) 2024-09-10T21:51:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.745+0000 7f0829e00640 1 -- 172.21.15.7:0/3818205686 learned_addr learned my addr 172.21.15.7:0/3818205686 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.745+0000 7f082a800640 1 --2- 172.21.15.7:0/3818205686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08241619e0 0x7f082406e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:32.747 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.746+0000 7f0829e00640 1 -- 172.21.15.7:0/3818205686 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08240cbd50 msgr2=0x7f08240c9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:32.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.746+0000 7f0829e00640 1 --2- 172.21.15.7:0/3818205686 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08240cbd50 0x7f08240c9a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:32.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.746+0000 7f0829e00640 1 -- 172.21.15.7:0/3818205686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08241619e0 msgr2=0x7f082406e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:32.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.746+0000 7f0829e00640 1 --2- 172.21.15.7:0/3818205686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08241619e0 0x7f082406e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:32.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.746+0000 7f0829e00640 1 -- 172.21.15.7:0/3818205686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f082406f3b0 con 0x7f08240ae020 2024-09-10T21:51:32.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.746+0000 7f0829400640 1 --2- 172.21.15.7:0/3818205686 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08240cbd50 0x7f08240c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:51:32.748 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.746+0000 7f0829e00640 1 --2- 172.21.15.7:0/3818205686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08240ae020 0x7f08240c9520 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f081800c4c0 tx=0x7f081800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:32.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.746+0000 7f081e000640 1 -- 172.21.15.7:0/3818205686 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08180052d0 con 0x7f08240ae020 2024-09-10T21:51:32.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.746+0000 7f081e000640 1 -- 172.21.15.7:0/3818205686 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0818005490 con 0x7f08240ae020 2024-09-10T21:51:32.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.746+0000 7f081e000640 1 -- 172.21.15.7:0/3818205686 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0818012890 con 0x7f08240ae020 2024-09-10T21:51:32.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.746+0000 7f082c800640 1 -- 172.21.15.7:0/3818205686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f082406f690 con 0x7f08240ae020 2024-09-10T21:51:32.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.746+0000 7f082c800640 1 -- 172.21.15.7:0/3818205686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f082406fc50 con 0x7f08240ae020 2024-09-10T21:51:32.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.747+0000 7f082c800640 1 -- 172.21.15.7:0/3818205686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08240ae4d0 con 0x7f08240ae020 2024-09-10T21:51:32.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.749+0000 7f081e000640 1 -- 172.21.15.7:0/3818205686 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0818012a30 con 0x7f08240ae020 2024-09-10T21:51:32.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.749+0000 7f081e000640 1 --2- 172.21.15.7:0/3818205686 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0808077fa0 0x7f080807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:32.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.750+0000 7f081e000640 1 -- 172.21.15.7:0/3818205686 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5029+0+0 (secure 0 0 0) 0x7f0818099490 con 0x7f08240ae020 2024-09-10T21:51:32.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.750+0000 7f0829400640 1 --2- 172.21.15.7:0/3818205686 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0808077fa0 0x7f080807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:32.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.751+0000 7f0829400640 1 --2- 172.21.15.7:0/3818205686 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0808077fa0 0x7f080807a460 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f08140096c0 tx=0x7f0814009260 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:32.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.754+0000 7f081e000640 1 -- 172.21.15.7:0/3818205686 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0818061bd0 con 0x7f08240ae020 2024-09-10T21:51:32.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.985+0000 7f082c800640 1 -- 172.21.15.7:0/3818205686 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f08240b5b00 con 0x7f08240ae020 2024-09-10T21:51:32.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.985+0000 7f081e000640 1 -- 172.21.15.7:0/3818205686 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 95 v95) ==== 94+0+3905 (secure 0 0 0) 0x7f0818066600 con 0x7f08240ae020 2024-09-10T21:51:32.986 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:32.986 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":95,"btime":"2024-09-10T21:51:26:399322+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6994,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/431253133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":431253133},{"type":"v1","addr":"172.21.15.133:6836","nonce":431253133}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":79},{"gid":7521,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/3692199178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":3692199178},{"type":"v1","addr":"172.21.15.133:6837","nonce":3692199178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7524,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/752989865","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":752989865},{"type":"v1","addr":"172.21.15.7:6834","nonce":752989865}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":80},{"gid":7956,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/3804508996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":3804508996},{"type":"v1","addr":"172.21.15.7:6835","nonce":3804508996}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[]} 2024-09-10T21:51:32.987 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 95 2024-09-10T21:51:32.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.987+0000 7f082c800640 1 -- 172.21.15.7:0/3818205686 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0808077fa0 msgr2=0x7f080807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:32.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.987+0000 7f082c800640 1 --2- 172.21.15.7:0/3818205686 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0808077fa0 0x7f080807a460 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f08140096c0 tx=0x7f0814009260 comp rx=0 tx=0).stop 2024-09-10T21:51:32.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.987+0000 7f082c800640 1 -- 172.21.15.7:0/3818205686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08240ae020 msgr2=0x7f08240c9520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:32.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.987+0000 7f082c800640 1 --2- 172.21.15.7:0/3818205686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08240ae020 0x7f08240c9520 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f081800c4c0 tx=0x7f081800c980 comp rx=0 tx=0).stop 2024-09-10T21:51:32.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.987+0000 7f082c800640 1 -- 172.21.15.7:0/3818205686 shutdown_connections 2024-09-10T21:51:32.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.987+0000 7f082c800640 1 --2- 172.21.15.7:0/3818205686 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0808077fa0 0x7f080807a460 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:32.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.987+0000 7f082c800640 1 --2- 172.21.15.7:0/3818205686 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f08241619e0 0x7f082406e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:32.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.987+0000 7f082c800640 1 --2- 172.21.15.7:0/3818205686 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f08240cbd50 0x7f08240c9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:32.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.987+0000 7f082c800640 1 --2- 172.21.15.7:0/3818205686 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f08240ae020 0x7f08240c9520 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:32.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.987+0000 7f082c800640 1 -- 172.21.15.7:0/3818205686 >> 172.21.15.7:0/3818205686 conn(0x7f08240b16d0 msgr2=0x7f082415d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:32.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.988+0000 7f082c800640 1 -- 172.21.15.7:0/3818205686 shutdown_connections 2024-09-10T21:51:32.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:32.988+0000 7f082c800640 1 -- 172.21.15.7:0/3818205686 wait complete. 2024-09-10T21:51:33.004 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:51:33.004 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-10T21:51:33.004 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:51:33.004 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:51:33.005 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:51:33.005 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-10T21:51:33.005 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:51:33.005 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:51:33.005 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:51:33.005 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-10T21:51:33.005 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:51:33.005 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:51:33.006 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:51:33.006 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-10T21:51:33.006 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:51:33.006 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:51:33.018 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:51:33.018 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:51:33.019 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:51:33.019 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:51:33.022 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:51:33.022 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:51:33.022 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:51:33.023 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:51:33.064 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:51:33.064 DEBUG:teuthology.parallel:result is None 2024-09-10T21:51:33.066 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:51:33.066 DEBUG:teuthology.parallel:result is None 2024-09-10T21:51:33.067 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:51:33.067 DEBUG:teuthology.parallel:result is None 2024-09-10T21:51:33.069 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:51:33.069 DEBUG:teuthology.parallel:result is None 2024-09-10T21:51:33.069 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-10T21:51:33.197 INFO:tasks.ceph.mds.a.smithi007.stdout:starting mds.a at 2024-09-10T21:51:33.198 INFO:tasks.ceph.mds.c.smithi007.stdout:starting mds.c at 2024-09-10T21:51:33.200 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2024-09-10T21:51:33.200 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2024-09-10T21:51:33.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.238+0000 7fcfc3200640 1 -- 172.21.15.7:0/1635728703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfbc0cf650 msgr2=0x7fcfbc0cc520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:33.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.238+0000 7fcfc3200640 1 --2- 172.21.15.7:0/1635728703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfbc0cf650 0x7fcfbc0cc520 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc0cc4f0 tx=0x7fcfa402a540 comp rx=0 tx=0).stop 2024-09-10T21:51:33.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.238+0000 7fcfc3200640 1 -- 172.21.15.7:0/1635728703 shutdown_connections 2024-09-10T21:51:33.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.238+0000 7fcfc3200640 1 --2- 172.21.15.7:0/1635728703 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfbc0cf650 0x7fcfbc0cc520 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:33.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.238+0000 7fcfc3200640 1 --2- 172.21.15.7:0/1635728703 >> v2:172.21.15.133:3300/0 conn(0x7fcfbc070600 0x7fcfbc1579b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:33.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.238+0000 7fcfc3200640 1 -- 172.21.15.7:0/1635728703 >> 172.21.15.7:0/1635728703 conn(0x7fcfbc0b1660 msgr2=0x7fcfbc0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:33.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.238+0000 7fcfc3200640 1 -- 172.21.15.7:0/1635728703 shutdown_connections 2024-09-10T21:51:33.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.238+0000 7fcfc3200640 1 -- 172.21.15.7:0/1635728703 wait complete. 2024-09-10T21:51:33.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.240+0000 7fcfc3200640 1 Processor -- start 2024-09-10T21:51:33.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.240+0000 7fcfc3200640 1 -- start start 2024-09-10T21:51:33.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.241+0000 7fcfc3200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfbc070600 0x7fcfbc0b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:33.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.241+0000 7fcfc3200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcfbc0cca60 0x7fcfbc0c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:33.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.241+0000 7fcfc3200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfbc0cf650 0x7fcfbc0cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:33.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.241+0000 7fcfc3200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc0af090 con 0x7fcfbc070600 2024-09-10T21:51:33.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.241+0000 7fcfc3200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcfbc0aef10 con 0x7fcfbc0cf650 2024-09-10T21:51:33.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.241+0000 7fcfc3200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fcfbc0af210 con 0x7fcfbc0cca60 2024-09-10T21:51:33.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.241+0000 7fcfbbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfbc070600 0x7fcfbc0b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:33.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.241+0000 7fcfbbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfbc070600 0x7fcfbc0b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:48932/0 (socket says 172.21.15.7:48932) 2024-09-10T21:51:33.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.241+0000 7fcfbbe00640 1 -- 172.21.15.7:0/2309627917 learned_addr learned my addr 172.21.15.7:0/2309627917 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:33.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.241+0000 7fcfc1200640 1 --2- 172.21.15.7:0/2309627917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfbc0cf650 0x7fcfbc0cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:33.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.241+0000 7fcfb3400640 1 --2- 172.21.15.7:0/2309627917 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcfbc0cca60 0x7fcfbc0c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:33.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.241+0000 7fcfbbe00640 1 -- 172.21.15.7:0/2309627917 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcfbc0cca60 msgr2=0x7fcfbc0c61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:33.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.241+0000 7fcfbbe00640 1 --2- 172.21.15.7:0/2309627917 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcfbc0cca60 0x7fcfbc0c61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:33.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.241+0000 7fcfbbe00640 1 -- 172.21.15.7:0/2309627917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfbc0cf650 msgr2=0x7fcfbc0cdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:33.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.241+0000 7fcfbbe00640 1 --2- 172.21.15.7:0/2309627917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfbc0cf650 0x7fcfbc0cdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:33.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.241+0000 7fcfbbe00640 1 -- 172.21.15.7:0/2309627917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfbc0b49b0 con 0x7fcfbc070600 2024-09-10T21:51:33.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.242+0000 7fcfbbe00640 1 --2- 172.21.15.7:0/2309627917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfbc070600 0x7fcfbc0b4080 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00e9f0 tx=0x7fcfac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:33.244 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.242+0000 7fcfb8c00640 1 -- 172.21.15.7:0/2309627917 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac00cd80 con 0x7fcfbc070600 2024-09-10T21:51:33.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.242+0000 7fcfb8c00640 1 -- 172.21.15.7:0/2309627917 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfac010430 con 0x7fcfbc070600 2024-09-10T21:51:33.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.242+0000 7fcfb8c00640 1 -- 172.21.15.7:0/2309627917 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfac010840 con 0x7fcfbc070600 2024-09-10T21:51:33.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.242+0000 7fcfc3200640 1 -- 172.21.15.7:0/2309627917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfbc1f6050 con 0x7fcfbc070600 2024-09-10T21:51:33.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.243+0000 7fcfc3200640 1 -- 172.21.15.7:0/2309627917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfbc1f6610 con 0x7fcfbc070600 2024-09-10T21:51:33.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.243+0000 7fcfc3200640 1 -- 172.21.15.7:0/2309627917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfbc06a1f0 con 0x7fcfbc070600 2024-09-10T21:51:33.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.245+0000 7fcfb8c00640 1 -- 172.21.15.7:0/2309627917 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fcfac0109e0 con 0x7fcfbc070600 2024-09-10T21:51:33.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.246+0000 7fcfb8c00640 1 --2- 172.21.15.7:0/2309627917 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcf98077f70 0x7fcf9807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:33.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.246+0000 7fcfb8c00640 1 -- 172.21.15.7:0/2309627917 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5029+0+0 (secure 0 0 0) 0x7fcfac09a5d0 con 0x7fcfbc070600 2024-09-10T21:51:33.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.246+0000 7fcfb3400640 1 --2- 172.21.15.7:0/2309627917 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcf98077f70 0x7fcf9807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:33.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.247+0000 7fcfb3400640 1 --2- 172.21.15.7:0/2309627917 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcf98077f70 0x7fcf9807a430 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4034000 tx=0x7fcfa4004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:33.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.250+0000 7fcfb8c00640 1 -- 172.21.15.7:0/2309627917 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fcfbc06a1f0 con 0x7fcfbc070600 2024-09-10T21:51:33.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:33.460+0000 7fcfc3200640 1 -- 172.21.15.7:0/2309627917 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fcfbc1579b0 con 0x7fcfbc070600 2024-09-10T21:51:34.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.461+0000 7fcfb8c00640 1 -- 172.21.15.7:0/2309627917 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v110) ==== 97+0+0 (secure 0 0 0) 0x7fcfac062e70 con 0x7fcfbc070600 2024-09-10T21:51:34.463 INFO:teuthology.orchestra.run.smithi007.stderr: removed all blocklist entries 2024-09-10T21:51:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.465+0000 7fcfc3200640 1 -- 172.21.15.7:0/2309627917 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcf98077f70 msgr2=0x7fcf9807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.465+0000 7fcfc3200640 1 --2- 172.21.15.7:0/2309627917 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcf98077f70 0x7fcf9807a430 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fcfa4034000 tx=0x7fcfa4004730 comp rx=0 tx=0).stop 2024-09-10T21:51:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.465+0000 7fcfc3200640 1 -- 172.21.15.7:0/2309627917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfbc070600 msgr2=0x7fcfbc0b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.465+0000 7fcfc3200640 1 --2- 172.21.15.7:0/2309627917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfbc070600 0x7fcfbc0b4080 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00e9f0 tx=0x7fcfac00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:51:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.465+0000 7fcfc3200640 1 -- 172.21.15.7:0/2309627917 shutdown_connections 2024-09-10T21:51:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.465+0000 7fcfc3200640 1 --2- 172.21.15.7:0/2309627917 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcf98077f70 0x7fcf9807a430 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.465+0000 7fcfc3200640 1 --2- 172.21.15.7:0/2309627917 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcfbc0cf650 0x7fcfbc0cdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.465+0000 7fcfc3200640 1 --2- 172.21.15.7:0/2309627917 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcfbc0cca60 0x7fcfbc0c61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.465+0000 7fcfc3200640 1 --2- 172.21.15.7:0/2309627917 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcfbc070600 0x7fcfbc0b4080 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.465+0000 7fcfc3200640 1 -- 172.21.15.7:0/2309627917 >> 172.21.15.7:0/2309627917 conn(0x7fcfbc0b1660 msgr2=0x7fcfbc0af640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:34.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.465+0000 7fcfc3200640 1 -- 172.21.15.7:0/2309627917 shutdown_connections 2024-09-10T21:51:34.467 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.465+0000 7fcfc3200640 1 -- 172.21.15.7:0/2309627917 wait complete. 2024-09-10T21:51:34.484 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-10T21:51:34.652 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.651+0000 7f014d200640 1 -- 172.21.15.7:0/959146996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01480b2f80 msgr2=0x7f01480ce6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:34.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.651+0000 7f014d200640 1 --2- 172.21.15.7:0/959146996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01480b2f80 0x7f01480ce6f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0140005fd0 tx=0x7f014002a8b0 comp rx=0 tx=0).stop 2024-09-10T21:51:34.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.651+0000 7f014d200640 1 -- 172.21.15.7:0/959146996 shutdown_connections 2024-09-10T21:51:34.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.651+0000 7f014d200640 1 --2- 172.21.15.7:0/959146996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01480b2f80 0x7f01480ce6f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:34.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.651+0000 7f014d200640 1 --2- 172.21.15.7:0/959146996 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f01480b4c90 0x7f01480b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:34.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.651+0000 7f014d200640 1 -- 172.21.15.7:0/959146996 >> 172.21.15.7:0/959146996 conn(0x7f01480b1d70 msgr2=0x7f01480cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:34.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.651+0000 7f014d200640 1 -- 172.21.15.7:0/959146996 shutdown_connections 2024-09-10T21:51:34.653 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.651+0000 7f014d200640 1 -- 172.21.15.7:0/959146996 wait complete. 2024-09-10T21:51:34.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.653+0000 7f014d200640 1 Processor -- start 2024-09-10T21:51:34.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.653+0000 7f014d200640 1 -- start start 2024-09-10T21:51:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.654+0000 7f014d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01480b2f80 0x7f01481430c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.654+0000 7f014d200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01480b4c90 0x7f014813d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.654+0000 7f014d200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0148145540 0x7f014813e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.654+0000 7f014d200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f01480cedb0 con 0x7f01480b2f80 2024-09-10T21:51:34.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.654+0000 7f014d200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f01480cec30 con 0x7f01480b4c90 2024-09-10T21:51:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.654+0000 7f014d200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f01480cef30 con 0x7f0148145540 2024-09-10T21:51:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.654+0000 7f0147400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01480b2f80 0x7f01481430c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.654+0000 7f0147400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01480b2f80 0x7f01481430c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44800/0 (socket says 172.21.15.7:44800) 2024-09-10T21:51:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.654+0000 7f0147400640 1 -- 172.21.15.7:0/2457860217 learned_addr learned my addr 172.21.15.7:0/2457860217 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.654+0000 7f013ea00640 1 --2- 172.21.15.7:0/2457860217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01480b4c90 0x7f014813d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:34.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.654+0000 7f0147e00640 1 --2- 172.21.15.7:0/2457860217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0148145540 0x7f014813e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:34.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.654+0000 7f0147400640 1 -- 172.21.15.7:0/2457860217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0148145540 msgr2=0x7f014813e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:34.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.654+0000 7f0147400640 1 --2- 172.21.15.7:0/2457860217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0148145540 0x7f014813e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:34.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.654+0000 7f0147400640 1 -- 172.21.15.7:0/2457860217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01480b4c90 msgr2=0x7f014813d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:34.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.654+0000 7f0147400640 1 --2- 172.21.15.7:0/2457860217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01480b4c90 0x7f014813d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:34.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.654+0000 7f0147400640 1 -- 172.21.15.7:0/2457860217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f014813eb50 con 0x7f01480b2f80 2024-09-10T21:51:34.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.655+0000 7f0147400640 1 --2- 172.21.15.7:0/2457860217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01480b2f80 0x7f01481430c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f013800c3a0 tx=0x7f013800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:34.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.655+0000 7f013fe00640 1 -- 172.21.15.7:0/2457860217 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138011790 con 0x7f01480b2f80 2024-09-10T21:51:34.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.655+0000 7f013fe00640 1 -- 172.21.15.7:0/2457860217 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0138017070 con 0x7f01480b2f80 2024-09-10T21:51:34.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.655+0000 7f013fe00640 1 -- 172.21.15.7:0/2457860217 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0138011cc0 con 0x7f01480b2f80 2024-09-10T21:51:34.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.655+0000 7f014d200640 1 -- 172.21.15.7:0/2457860217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f014813edd0 con 0x7f01480b2f80 2024-09-10T21:51:34.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.655+0000 7f014d200640 1 -- 172.21.15.7:0/2457860217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f014813f390 con 0x7f01480b2f80 2024-09-10T21:51:34.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.656+0000 7f014d200640 1 -- 172.21.15.7:0/2457860217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0148069fc0 con 0x7f01480b2f80 2024-09-10T21:51:34.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.657+0000 7f013fe00640 1 -- 172.21.15.7:0/2457860217 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f013801c460 con 0x7f01480b2f80 2024-09-10T21:51:34.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.658+0000 7f013fe00640 1 --2- 172.21.15.7:0/2457860217 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0124077f70 0x7f012407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:34.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.658+0000 7f013fe00640 1 -- 172.21.15.7:0/2457860217 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4857+0+0 (secure 0 0 0) 0x7f01380996d0 con 0x7f01480b2f80 2024-09-10T21:51:34.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.659+0000 7f013ea00640 1 --2- 172.21.15.7:0/2457860217 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0124077f70 0x7f012407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:34.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.660+0000 7f013ea00640 1 --2- 172.21.15.7:0/2457860217 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0124077f70 0x7f012407a430 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f014815bd80 tx=0x7f0130005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:34.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.663+0000 7f013fe00640 1 -- 172.21.15.7:0/2457860217 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0148069fc0 con 0x7f01480b2f80 2024-09-10T21:51:34.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.927+0000 7f014d200640 1 -- 172.21.15.7:0/2457860217 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f01480ce6f0 con 0x7f01480b2f80 2024-09-10T21:51:34.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.928+0000 7f013fe00640 1 -- 172.21.15.7:0/2457860217 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v38) ==== 80+0+5304 (secure 0 0 0) 0x7f014813eb50 con 0x7f01480b2f80 2024-09-10T21:51:34.929 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:34.930 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:51:34.930 INFO:teuthology.orchestra.run.smithi007.stdout: "auth_dump": [ 2024-09-10T21:51:34.930 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:34.930 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.a", 2024-09-10T21:51:34.930 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmgRMuLBAA88QT12AXv2qf11IxU20Xbg==", 2024-09-10T21:51:34.931 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:51:34.931 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:51:34.931 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:51:34.931 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:51:34.931 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:51:34.931 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.932 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:51:34.932 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:34.932 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.b", 2024-09-10T21:51:34.932 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmRUM2BBAAwcQM6z+hjZI4ZWBlVrYMwQ==", 2024-09-10T21:51:34.932 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:51:34.933 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:51:34.933 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:51:34.933 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:51:34.933 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:51:34.933 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.933 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:51:34.934 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:34.934 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.c", 2024-09-10T21:51:34.934 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmnq31NRAAj1eJUAIS986idxppfeXEAA==", 2024-09-10T21:51:34.934 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:51:34.934 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:51:34.935 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:51:34.935 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:51:34.935 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:51:34.935 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.935 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:51:34.935 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:34.936 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mds.d", 2024-09-10T21:51:34.936 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmy43uDRAAzdKjLxZMGROuDABrsJ0zgQ==", 2024-09-10T21:51:34.936 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:51:34.936 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:51:34.936 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:51:34.936 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:51:34.937 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:51:34.937 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.937 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:51:34.937 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:34.937 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.0", 2024-09-10T21:51:34.937 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDwvOBmzfgtOxAAk6K5fU0Uar4VW9nUUIuNdw==", 2024-09-10T21:51:34.938 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:51:34.938 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:51:34.938 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:51:34.938 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:51:34.938 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.938 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:51:34.939 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:34.939 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.1", 2024-09-10T21:51:34.939 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD0vOBmHcRzChAA3z+CaGgu5NkyUmjpqzmKjw==", 2024-09-10T21:51:34.939 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:51:34.939 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:51:34.939 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:51:34.940 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:51:34.940 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.940 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:51:34.940 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:34.940 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.2", 2024-09-10T21:51:34.940 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD3vOBm9sArFBAAGQlyiw6qEcyumEBICWEBKA==", 2024-09-10T21:51:34.941 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:51:34.941 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:51:34.941 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:51:34.941 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:51:34.941 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.941 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:51:34.942 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:34.942 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.3", 2024-09-10T21:51:34.942 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQD6vOBmkO3rHxAAW0zPydYZRGUKjGh3yIyPCA==", 2024-09-10T21:51:34.942 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:51:34.942 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:51:34.942 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:51:34.943 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:51:34.943 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.943 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:51:34.943 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:34.943 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.4", 2024-09-10T21:51:34.943 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAAveBmlPABMxAAeazVOzc1DhsVAwoHn87QIA==", 2024-09-10T21:51:34.944 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:51:34.944 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:51:34.944 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:51:34.944 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:51:34.944 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.944 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:51:34.945 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:34.945 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.5", 2024-09-10T21:51:34.945 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAEveBmL9qzARAApu8nWIY9HVsDl4yUnfhH9w==", 2024-09-10T21:51:34.945 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:51:34.945 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:51:34.945 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:51:34.946 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:51:34.946 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.946 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:51:34.946 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:34.946 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.6", 2024-09-10T21:51:34.946 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAHveBmJ4ofDRAAEiCFN8nGzVSR3u2gKoT+2Q==", 2024-09-10T21:51:34.947 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:51:34.947 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:51:34.947 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:51:34.947 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:51:34.948 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.948 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:51:34.948 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:34.948 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "osd.7", 2024-09-10T21:51:34.948 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQAKveBmoUGEGBAAhXmkUaniuAQrBiYPWr8Tog==", 2024-09-10T21:51:34.949 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:51:34.949 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow profile osd", 2024-09-10T21:51:34.949 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile osd", 2024-09-10T21:51:34.949 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:51:34.949 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.949 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:51:34.950 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:34.950 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.0", 2024-09-10T21:51:34.950 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBm0aqcFhAAtid2bnzjcpaYAbU7g0JcOA==", 2024-09-10T21:51:34.950 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:51:34.950 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:51:34.950 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow r", 2024-09-10T21:51:34.950 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-10T21:51:34.951 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.951 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:51:34.951 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:34.951 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.1", 2024-09-10T21:51:34.951 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmqf+MIxAAib5hVnbBsw/bJPqsQszJwg==", 2024-09-10T21:51:34.952 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:51:34.952 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:51:34.952 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:51:34.952 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:51:34.952 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:51:34.952 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.953 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:51:34.953 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:34.953 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.2", 2024-09-10T21:51:34.953 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmybQhHRAAZUwqnIsihEWnHZWQTYr+uQ==", 2024-09-10T21:51:34.953 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:51:34.954 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:51:34.954 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:51:34.954 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:51:34.954 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:51:34.955 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.955 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:51:34.955 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:34.955 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.3", 2024-09-10T21:51:34.955 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDvvOBmDbrrKRAAUxb3EBfKIJ4FAGHW3QIsLg==", 2024-09-10T21:51:34.955 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:51:34.956 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow", 2024-09-10T21:51:34.956 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow r", 2024-09-10T21:51:34.956 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow rw", 2024-09-10T21:51:34.956 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow rwx" 2024-09-10T21:51:34.956 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.956 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:51:34.956 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:34.957 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "client.admin", 2024-09-10T21:51:34.957 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDtvOBmF+eEOBAA3icvNqI0nWDShE49DZ3bvg==", 2024-09-10T21:51:34.957 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:51:34.957 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:51:34.957 INFO:teuthology.orchestra.run.smithi007.stdout: "mgr": "allow *", 2024-09-10T21:51:34.957 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow *", 2024-09-10T21:51:34.958 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:51:34.958 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.958 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:51:34.958 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:34.958 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.x", 2024-09-10T21:51:34.959 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmsu6OJRAAhdgDvpeHY/zDKUeUOv6xxA==", 2024-09-10T21:51:34.959 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:51:34.959 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:51:34.959 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-10T21:51:34.959 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:51:34.959 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.960 INFO:teuthology.orchestra.run.smithi007.stdout: }, 2024-09-10T21:51:34.960 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:34.960 INFO:teuthology.orchestra.run.smithi007.stdout: "entity": "mgr.y", 2024-09-10T21:51:34.960 INFO:teuthology.orchestra.run.smithi007.stdout: "key": "AQDuvOBmKBHLHhAA0W5My19tIhN1v2qev4cBTA==", 2024-09-10T21:51:34.960 INFO:teuthology.orchestra.run.smithi007.stdout: "caps": { 2024-09-10T21:51:34.961 INFO:teuthology.orchestra.run.smithi007.stdout: "mds": "allow *", 2024-09-10T21:51:34.961 INFO:teuthology.orchestra.run.smithi007.stdout: "mon": "allow profile mgr", 2024-09-10T21:51:34.961 INFO:teuthology.orchestra.run.smithi007.stdout: "osd": "allow *" 2024-09-10T21:51:34.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.961 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:34.961 INFO:teuthology.orchestra.run.smithi007.stdout: ] 2024-09-10T21:51:34.962 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:51:34.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.930+0000 7f014d200640 1 -- 172.21.15.7:0/2457860217 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0124077f70 msgr2=0x7f012407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:34.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.930+0000 7f014d200640 1 --2- 172.21.15.7:0/2457860217 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0124077f70 0x7f012407a430 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f014815bd80 tx=0x7f0130005f90 comp rx=0 tx=0).stop 2024-09-10T21:51:34.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.930+0000 7f014d200640 1 -- 172.21.15.7:0/2457860217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01480b2f80 msgr2=0x7f01481430c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.930+0000 7f014d200640 1 --2- 172.21.15.7:0/2457860217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01480b2f80 0x7f01481430c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f013800c3a0 tx=0x7f013800c860 comp rx=0 tx=0).stop 2024-09-10T21:51:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.930+0000 7f014d200640 1 -- 172.21.15.7:0/2457860217 shutdown_connections 2024-09-10T21:51:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.930+0000 7f014d200640 1 --2- 172.21.15.7:0/2457860217 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0124077f70 0x7f012407a430 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.930+0000 7f014d200640 1 --2- 172.21.15.7:0/2457860217 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f01480b4c90 0x7f014813d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.930+0000 7f014d200640 1 --2- 172.21.15.7:0/2457860217 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0148145540 0x7f014813e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.930+0000 7f014d200640 1 --2- 172.21.15.7:0/2457860217 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f01480b2f80 0x7f01481430c0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.930+0000 7f014d200640 1 -- 172.21.15.7:0/2457860217 >> 172.21.15.7:0/2457860217 conn(0x7f01480b1d70 msgr2=0x7f01480b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.930+0000 7f014d200640 1 -- 172.21.15.7:0/2457860217 shutdown_connections 2024-09-10T21:51:34.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:34.930+0000 7f014d200640 1 -- 172.21.15.7:0/2457860217 wait complete. 2024-09-10T21:51:34.966 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:51:35.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.125+0000 7f3eb7e00640 1 -- 172.21.15.7:0/2328077857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3eb00cd750 msgr2=0x7f3eb00ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.125+0000 7f3eb7e00640 1 --2- 172.21.15.7:0/2328077857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3eb00cd750 0x7f3eb00ccef0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c009830 tx=0x7f3e9c02a520 comp rx=0 tx=0).stop 2024-09-10T21:51:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.125+0000 7f3eb7e00640 1 -- 172.21.15.7:0/2328077857 shutdown_connections 2024-09-10T21:51:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.125+0000 7f3eb7e00640 1 --2- 172.21.15.7:0/2328077857 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3eb00cd750 0x7f3eb00ccef0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.125+0000 7f3eb7e00640 1 -- 172.21.15.7:0/2328077857 >> 172.21.15.7:0/2328077857 conn(0x7f3eb00b16d0 msgr2=0x7f3eb00b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.125+0000 7f3eb7e00640 1 -- 172.21.15.7:0/2328077857 shutdown_connections 2024-09-10T21:51:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.126+0000 7f3eb7e00640 1 -- 172.21.15.7:0/2328077857 wait complete. 2024-09-10T21:51:35.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.127+0000 7f3eb7e00640 1 Processor -- start 2024-09-10T21:51:35.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.127+0000 7f3eb7e00640 1 -- start start 2024-09-10T21:51:35.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.128+0000 7f3eb7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eb00cd750 0x7f3eb00c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:35.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.128+0000 7f3eb7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3eb015bc90 0x7f3eb01f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:35.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.128+0000 7f3eb7e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3eb015fda0 0x7f3eb00af860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:35.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.128+0000 7f3eb7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb0070080 con 0x7f3eb00cd750 2024-09-10T21:51:35.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.128+0000 7f3eb7e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb006ff00 con 0x7f3eb015bc90 2024-09-10T21:51:35.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.128+0000 7f3eb7e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f3eb0070200 con 0x7f3eb015fda0 2024-09-10T21:51:35.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.128+0000 7f3eb5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eb00cd750 0x7f3eb00c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:35.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.129+0000 7f3eb5400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eb00cd750 0x7f3eb00c8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44802/0 (socket says 172.21.15.7:44802) 2024-09-10T21:51:35.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.129+0000 7f3eb5400640 1 -- 172.21.15.7:0/4079204599 learned_addr learned my addr 172.21.15.7:0/4079204599 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:35.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.129+0000 7f3eb4a00640 1 --2- 172.21.15.7:0/4079204599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3eb015bc90 0x7f3eb01f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:35.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.129+0000 7f3eb5e00640 1 --2- 172.21.15.7:0/4079204599 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3eb015fda0 0x7f3eb00af860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:35.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.129+0000 7f3eb5400640 1 -- 172.21.15.7:0/4079204599 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3eb015fda0 msgr2=0x7f3eb00af860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:35.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.129+0000 7f3eb5400640 1 --2- 172.21.15.7:0/4079204599 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3eb015fda0 0x7f3eb00af860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:35.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.129+0000 7f3eb5400640 1 -- 172.21.15.7:0/4079204599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3eb015bc90 msgr2=0x7f3eb01f5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:35.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.129+0000 7f3eb5400640 1 --2- 172.21.15.7:0/4079204599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3eb015bc90 0x7f3eb01f5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:35.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.129+0000 7f3eb5400640 1 -- 172.21.15.7:0/4079204599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb00c8e80 con 0x7f3eb00cd750 2024-09-10T21:51:35.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.129+0000 7f3eb4a00640 1 --2- 172.21.15.7:0/4079204599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3eb015bc90 0x7f3eb01f5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:51:35.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.129+0000 7f3eb5400640 1 --2- 172.21.15.7:0/4079204599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eb00cd750 0x7f3eb00c8940 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400cb90 tx=0x7f3ea400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:35.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.130+0000 7f3ead600640 1 -- 172.21.15.7:0/4079204599 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4016020 con 0x7f3eb00cd750 2024-09-10T21:51:35.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.130+0000 7f3eb7e00640 1 -- 172.21.15.7:0/4079204599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb00adc20 con 0x7f3eb00cd750 2024-09-10T21:51:35.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.130+0000 7f3ead600640 1 -- 172.21.15.7:0/4079204599 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ea400ae30 con 0x7f3eb00cd750 2024-09-10T21:51:35.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.130+0000 7f3ead600640 1 -- 172.21.15.7:0/4079204599 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea4012660 con 0x7f3eb00cd750 2024-09-10T21:51:35.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.130+0000 7f3eb7e00640 1 -- 172.21.15.7:0/4079204599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb00ae1e0 con 0x7f3eb00cd750 2024-09-10T21:51:35.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.131+0000 7f3eb7e00640 1 -- 172.21.15.7:0/4079204599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb00c9f20 con 0x7f3eb00cd750 2024-09-10T21:51:35.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.132+0000 7f3ead600640 1 -- 172.21.15.7:0/4079204599 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3ea401a3b0 con 0x7f3eb00cd750 2024-09-10T21:51:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.133+0000 7f3ead600640 1 --2- 172.21.15.7:0/4079204599 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3e90077f70 0x7f3e9007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.133+0000 7f3ead600640 1 -- 172.21.15.7:0/4079204599 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4857+0+0 (secure 0 0 0) 0x7f3ea409a130 con 0x7f3eb00cd750 2024-09-10T21:51:35.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.134+0000 7f3eb4a00640 1 --2- 172.21.15.7:0/4079204599 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3e90077f70 0x7f3e9007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:35.135 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.134+0000 7f3eb4a00640 1 --2- 172.21.15.7:0/4079204599 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3e90077f70 0x7f3e9007a430 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c034000 tx=0x7f3e9c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:35.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.137+0000 7f3ead600640 1 -- 172.21.15.7:0/4079204599 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3eb00c9f20 con 0x7f3eb00cd750 2024-09-10T21:51:35.335 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.335+0000 7f3eb7e00640 1 -- 172.21.15.7:0/4079204599 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f3eb00706b0 con 0x7f3eb00cd750 2024-09-10T21:51:35.336 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.335+0000 7f3ead600640 1 -- 172.21.15.7:0/4079204599 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v110) ==== 84+0+65 (secure 0 0 0) 0x7f3ea4020030 con 0x7f3eb00cd750 2024-09-10T21:51:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:35.336 INFO:teuthology.orchestra.run.smithi007.stdout:[ 2024-09-10T21:51:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: { 2024-09-10T21:51:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: "poolnum": 1, 2024-09-10T21:51:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: "poolname": ".mgr" 2024-09-10T21:51:35.336 INFO:teuthology.orchestra.run.smithi007.stdout: } 2024-09-10T21:51:35.337 INFO:teuthology.orchestra.run.smithi007.stdout:] 2024-09-10T21:51:35.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.337+0000 7f3eb7e00640 1 -- 172.21.15.7:0/4079204599 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3e90077f70 msgr2=0x7f3e9007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:35.338 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.337+0000 7f3eb7e00640 1 --2- 172.21.15.7:0/4079204599 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3e90077f70 0x7f3e9007a430 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c034000 tx=0x7f3e9c004730 comp rx=0 tx=0).stop 2024-09-10T21:51:35.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.337+0000 7f3eb7e00640 1 -- 172.21.15.7:0/4079204599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eb00cd750 msgr2=0x7f3eb00c8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:35.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.337+0000 7f3eb7e00640 1 --2- 172.21.15.7:0/4079204599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eb00cd750 0x7f3eb00c8940 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f3ea400cb90 tx=0x7f3ea400a430 comp rx=0 tx=0).stop 2024-09-10T21:51:35.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.337+0000 7f3eb7e00640 1 -- 172.21.15.7:0/4079204599 shutdown_connections 2024-09-10T21:51:35.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.337+0000 7f3eb7e00640 1 --2- 172.21.15.7:0/4079204599 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3e90077f70 0x7f3e9007a430 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:35.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.337+0000 7f3eb7e00640 1 --2- 172.21.15.7:0/4079204599 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3eb015bc90 0x7f3eb01f5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:35.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.337+0000 7f3eb7e00640 1 --2- 172.21.15.7:0/4079204599 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3eb015fda0 0x7f3eb00af860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:35.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.337+0000 7f3eb7e00640 1 --2- 172.21.15.7:0/4079204599 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3eb00cd750 0x7f3eb00c8940 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:35.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.337+0000 7f3eb7e00640 1 -- 172.21.15.7:0/4079204599 >> 172.21.15.7:0/4079204599 conn(0x7f3eb00b16d0 msgr2=0x7f3eb0072360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:35.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.337+0000 7f3eb7e00640 1 -- 172.21.15.7:0/4079204599 shutdown_connections 2024-09-10T21:51:35.339 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.338+0000 7f3eb7e00640 1 -- 172.21.15.7:0/4079204599 wait complete. 2024-09-10T21:51:35.354 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:51:35.354 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:51:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.519+0000 7f6840e00640 1 -- 172.21.15.7:0/1234268469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f683c06fdb0 msgr2=0x7f683c06d230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.519+0000 7f6840e00640 1 --2- 172.21.15.7:0/1234268469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f683c06fdb0 0x7f683c06d230 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f683c06dbc0 tx=0x7f683402aab0 comp rx=0 tx=0).stop 2024-09-10T21:51:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.520+0000 7f6840e00640 1 -- 172.21.15.7:0/1234268469 shutdown_connections 2024-09-10T21:51:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.520+0000 7f6840e00640 1 --2- 172.21.15.7:0/1234268469 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f683c06fdb0 0x7f683c06d230 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.520+0000 7f6840e00640 1 --2- 172.21.15.7:0/1234268469 >> v2:172.21.15.133:3300/0 conn(0x7f683c151a10 0x7f683c15ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.520+0000 7f6840e00640 1 -- 172.21.15.7:0/1234268469 >> 172.21.15.7:0/1234268469 conn(0x7f683c0b1c10 msgr2=0x7f683c0b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.520+0000 7f6840e00640 1 -- 172.21.15.7:0/1234268469 shutdown_connections 2024-09-10T21:51:35.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.520+0000 7f6840e00640 1 -- 172.21.15.7:0/1234268469 wait complete. 2024-09-10T21:51:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.521+0000 7f6840e00640 1 Processor -- start 2024-09-10T21:51:35.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.521+0000 7f6840e00640 1 -- start start 2024-09-10T21:51:35.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.522+0000 7f6840e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f683c06f970 0x7f683c1443f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:35.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.522+0000 7f6840e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f683c06fdb0 0x7f683c1f8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:35.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.522+0000 7f6840e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f683c151a10 0x7f683c0afbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:35.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.522+0000 7f6840e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f683c15e8d0 con 0x7f683c06fdb0 2024-09-10T21:51:35.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.522+0000 7f6840e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f683c15e750 con 0x7f683c151a10 2024-09-10T21:51:35.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.522+0000 7f6840e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f683c073e50 con 0x7f683c06f970 2024-09-10T21:51:35.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.522+0000 7f6831400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f683c06fdb0 0x7f683c1f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:35.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.523+0000 7f6831400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f683c06fdb0 0x7f683c1f8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44808/0 (socket says 172.21.15.7:44808) 2024-09-10T21:51:35.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.523+0000 7f6831400640 1 -- 172.21.15.7:0/3517282214 learned_addr learned my addr 172.21.15.7:0/3517282214 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:35.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.523+0000 7f6839e00640 1 --2- 172.21.15.7:0/3517282214 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f683c06f970 0x7f683c1443f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:35.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.523+0000 7f683a800640 1 --2- 172.21.15.7:0/3517282214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f683c151a10 0x7f683c0afbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:35.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.523+0000 7f6831400640 1 -- 172.21.15.7:0/3517282214 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f683c06f970 msgr2=0x7f683c1443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:35.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.523+0000 7f6831400640 1 --2- 172.21.15.7:0/3517282214 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f683c06f970 0x7f683c1443f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:35.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.523+0000 7f6831400640 1 -- 172.21.15.7:0/3517282214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f683c151a10 msgr2=0x7f683c0afbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:35.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.523+0000 7f6831400640 1 --2- 172.21.15.7:0/3517282214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f683c151a10 0x7f683c0afbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:35.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.523+0000 7f6831400640 1 -- 172.21.15.7:0/3517282214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683c1f8c70 con 0x7f683c06fdb0 2024-09-10T21:51:35.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.523+0000 7f6831400640 1 --2- 172.21.15.7:0/3517282214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f683c06fdb0 0x7f683c1f8300 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f682400c480 tx=0x7f682400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:35.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.523+0000 7f6832a00640 1 -- 172.21.15.7:0/3517282214 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68240052d0 con 0x7f683c06fdb0 2024-09-10T21:51:35.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.523+0000 7f6832a00640 1 -- 172.21.15.7:0/3517282214 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6824005470 con 0x7f683c06fdb0 2024-09-10T21:51:35.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.524+0000 7f6832a00640 1 -- 172.21.15.7:0/3517282214 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6824012870 con 0x7f683c06fdb0 2024-09-10T21:51:35.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.524+0000 7f6840e00640 1 -- 172.21.15.7:0/3517282214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683c1f8f50 con 0x7f683c06fdb0 2024-09-10T21:51:35.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.524+0000 7f6840e00640 1 -- 172.21.15.7:0/3517282214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f683c14dfe0 con 0x7f683c06fdb0 2024-09-10T21:51:35.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.525+0000 7f6840e00640 1 -- 172.21.15.7:0/3517282214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f683c1f9190 con 0x7f683c06fdb0 2024-09-10T21:51:35.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.526+0000 7f6832a00640 1 -- 172.21.15.7:0/3517282214 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f68240035f0 con 0x7f683c06fdb0 2024-09-10T21:51:35.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.527+0000 7f6832a00640 1 --2- 172.21.15.7:0/3517282214 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6818077f70 0x7f681807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:35.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.527+0000 7f6832a00640 1 -- 172.21.15.7:0/3517282214 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 4857+0+0 (secure 0 0 0) 0x7f6824099640 con 0x7f683c06fdb0 2024-09-10T21:51:35.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.527+0000 7f6839e00640 1 --2- 172.21.15.7:0/3517282214 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6818077f70 0x7f681807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:35.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.528+0000 7f6839e00640 1 --2- 172.21.15.7:0/3517282214 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6818077f70 0x7f681807a430 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f682c005fd0 tx=0x7f682c005850 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:35.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.531+0000 7f6832a00640 1 -- 172.21.15.7:0/3517282214 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f683c1f9190 con 0x7f683c06fdb0 2024-09-10T21:51:35.726 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:35.726+0000 7f6840e00640 1 -- 172.21.15.7:0/3517282214 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f683c0b2240 con 0x7f683c06fdb0 2024-09-10T21:51:36.456 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.455+0000 7f6832a00640 1 -- 172.21.15.7:0/3517282214 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v111) ==== 138+0+0 (secure 0 0 0) 0x7f6824062480 con 0x7f683c06fdb0 2024-09-10T21:51:36.457 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' created 2024-09-10T21:51:36.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.459+0000 7f6840e00640 1 -- 172.21.15.7:0/3517282214 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6818077f70 msgr2=0x7f681807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:36.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.459+0000 7f6840e00640 1 --2- 172.21.15.7:0/3517282214 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6818077f70 0x7f681807a430 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f682c005fd0 tx=0x7f682c005850 comp rx=0 tx=0).stop 2024-09-10T21:51:36.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.459+0000 7f6840e00640 1 -- 172.21.15.7:0/3517282214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f683c06fdb0 msgr2=0x7f683c1f8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:36.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.459+0000 7f6840e00640 1 --2- 172.21.15.7:0/3517282214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f683c06fdb0 0x7f683c1f8300 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f682400c480 tx=0x7f682400c940 comp rx=0 tx=0).stop 2024-09-10T21:51:36.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.459+0000 7f6840e00640 1 -- 172.21.15.7:0/3517282214 shutdown_connections 2024-09-10T21:51:36.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.459+0000 7f6840e00640 1 --2- 172.21.15.7:0/3517282214 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6818077f70 0x7f681807a430 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:36.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.459+0000 7f6840e00640 1 --2- 172.21.15.7:0/3517282214 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f683c151a10 0x7f683c0afbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:36.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.459+0000 7f6840e00640 1 --2- 172.21.15.7:0/3517282214 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f683c06fdb0 0x7f683c1f8300 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:36.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.459+0000 7f6840e00640 1 --2- 172.21.15.7:0/3517282214 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f683c06f970 0x7f683c1443f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:36.460 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.459+0000 7f6840e00640 1 -- 172.21.15.7:0/3517282214 >> 172.21.15.7:0/3517282214 conn(0x7f683c0b1c10 msgr2=0x7f683c14f770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:36.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.459+0000 7f6840e00640 1 -- 172.21.15.7:0/3517282214 shutdown_connections 2024-09-10T21:51:36.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.459+0000 7f6840e00640 1 -- 172.21.15.7:0/3517282214 wait complete. 2024-09-10T21:51:36.484 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:51:36.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.655+0000 7f0247c00640 1 -- 172.21.15.7:0/2353660008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0240159680 msgr2=0x7f024015cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:36.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.655+0000 7f0247c00640 1 --2- 172.21.15.7:0/2353660008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0240159680 0x7f024015cc00 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f024006faf0 tx=0x7f023402a560 comp rx=0 tx=0).stop 2024-09-10T21:51:36.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.655+0000 7f0247c00640 1 -- 172.21.15.7:0/2353660008 shutdown_connections 2024-09-10T21:51:36.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.655+0000 7f0247c00640 1 --2- 172.21.15.7:0/2353660008 >> v2:172.21.15.133:3300/0 conn(0x7f024006e580 0x7f02400cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:36.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.655+0000 7f0247c00640 1 --2- 172.21.15.7:0/2353660008 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0240159680 0x7f024015cc00 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:36.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.655+0000 7f0247c00640 1 -- 172.21.15.7:0/2353660008 >> 172.21.15.7:0/2353660008 conn(0x7f02400b16d0 msgr2=0x7f02400b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:36.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.655+0000 7f0247c00640 1 -- 172.21.15.7:0/2353660008 shutdown_connections 2024-09-10T21:51:36.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.655+0000 7f0247c00640 1 -- 172.21.15.7:0/2353660008 wait complete. 2024-09-10T21:51:36.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.657+0000 7f0247c00640 1 Processor -- start 2024-09-10T21:51:36.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.657+0000 7f0247c00640 1 -- start start 2024-09-10T21:51:36.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.658+0000 7f0247c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f024006e580 0x7f0240157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:36.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.658+0000 7f0247c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0240073e20 0x7f02400b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:36.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.658+0000 7f0247c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0240159680 0x7f02401520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:36.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.658+0000 7f0247c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02400cb0a0 con 0x7f0240159680 2024-09-10T21:51:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.658+0000 7f0247c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f02400caf20 con 0x7f0240073e20 2024-09-10T21:51:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.658+0000 7f0247c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f0240159b10 con 0x7f024006e580 2024-09-10T21:51:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.658+0000 7f0245c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0240159680 0x7f02401520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.658+0000 7f0245c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0240159680 0x7f02401520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44824/0 (socket says 172.21.15.7:44824) 2024-09-10T21:51:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.658+0000 7f0245c00640 1 -- 172.21.15.7:0/2370671566 learned_addr learned my addr 172.21.15.7:0/2370671566 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.658+0000 7f023fe00640 1 --2- 172.21.15.7:0/2370671566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0240073e20 0x7f02400b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.658+0000 7f0245200640 1 --2- 172.21.15.7:0/2370671566 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f024006e580 0x7f0240157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.658+0000 7f0245c00640 1 -- 172.21.15.7:0/2370671566 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f024006e580 msgr2=0x7f0240157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.658+0000 7f0245c00640 1 --2- 172.21.15.7:0/2370671566 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f024006e580 0x7f0240157340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.658+0000 7f0245c00640 1 -- 172.21.15.7:0/2370671566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0240073e20 msgr2=0x7f02400b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.658+0000 7f0245c00640 1 --2- 172.21.15.7:0/2370671566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0240073e20 0x7f02400b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.658+0000 7f0245c00640 1 -- 172.21.15.7:0/2370671566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0240157a40 con 0x7f0240159680 2024-09-10T21:51:36.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.659+0000 7f0245c00640 1 --2- 172.21.15.7:0/2370671566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0240159680 0x7f02401520e0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f023800c7d0 tx=0x7f023800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:36.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.659+0000 7f023cc00640 1 -- 172.21.15.7:0/2370671566 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0238003b70 con 0x7f0240159680 2024-09-10T21:51:36.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.659+0000 7f023cc00640 1 -- 172.21.15.7:0/2370671566 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0238003d10 con 0x7f0240159680 2024-09-10T21:51:36.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.659+0000 7f023cc00640 1 -- 172.21.15.7:0/2370671566 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f023800aab0 con 0x7f0240159680 2024-09-10T21:51:36.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.659+0000 7f0247c00640 1 -- 172.21.15.7:0/2370671566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0240157d20 con 0x7f0240159680 2024-09-10T21:51:36.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.659+0000 7f0247c00640 1 -- 172.21.15.7:0/2370671566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0240152620 con 0x7f0240159680 2024-09-10T21:51:36.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.660+0000 7f0247c00640 1 -- 172.21.15.7:0/2370671566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0240074500 con 0x7f0240159680 2024-09-10T21:51:36.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.662+0000 7f023cc00640 1 -- 172.21.15.7:0/2370671566 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f023800ac50 con 0x7f0240159680 2024-09-10T21:51:36.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.663+0000 7f023cc00640 1 --2- 172.21.15.7:0/2370671566 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0220077f70 0x7f022007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:36.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.663+0000 7f023cc00640 1 -- 172.21.15.7:0/2370671566 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5227+0+0 (secure 0 0 0) 0x7f023809a3a0 con 0x7f0240159680 2024-09-10T21:51:36.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.663+0000 7f0245200640 1 --2- 172.21.15.7:0/2370671566 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0220077f70 0x7f022007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:36.665 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.664+0000 7f0245200640 1 --2- 172.21.15.7:0/2370671566 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0220077f70 0x7f022007a430 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0234033000 tx=0x7f0234004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:36.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.666+0000 7f023cc00640 1 -- 172.21.15.7:0/2370671566 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0240072830 con 0x7f0240159680 2024-09-10T21:51:36.863 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:36.863+0000 7f0247c00640 1 -- 172.21.15.7:0/2370671566 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f024015c490 con 0x7f0240159680 2024-09-10T21:51:37.459 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.458+0000 7f023cc00640 1 -- 172.21.15.7:0/2370671566 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v112) ==== 170+0+0 (secure 0 0 0) 0x7f0238062a10 con 0x7f0240159680 2024-09-10T21:51:37.460 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' created 2024-09-10T21:51:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.462+0000 7f0247c00640 1 -- 172.21.15.7:0/2370671566 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0220077f70 msgr2=0x7f022007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.462+0000 7f0247c00640 1 --2- 172.21.15.7:0/2370671566 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0220077f70 0x7f022007a430 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0234033000 tx=0x7f0234004730 comp rx=0 tx=0).stop 2024-09-10T21:51:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.462+0000 7f0247c00640 1 -- 172.21.15.7:0/2370671566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0240159680 msgr2=0x7f02401520e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.462+0000 7f0247c00640 1 --2- 172.21.15.7:0/2370671566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0240159680 0x7f02401520e0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f023800c7d0 tx=0x7f023800cc90 comp rx=0 tx=0).stop 2024-09-10T21:51:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.462+0000 7f0247c00640 1 -- 172.21.15.7:0/2370671566 shutdown_connections 2024-09-10T21:51:37.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.462+0000 7f0247c00640 1 --2- 172.21.15.7:0/2370671566 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0220077f70 0x7f022007a430 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.462+0000 7f0247c00640 1 --2- 172.21.15.7:0/2370671566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0240073e20 0x7f02400b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.462+0000 7f0247c00640 1 --2- 172.21.15.7:0/2370671566 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0240159680 0x7f02401520e0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.462+0000 7f0247c00640 1 --2- 172.21.15.7:0/2370671566 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f024006e580 0x7f0240157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.462+0000 7f0247c00640 1 -- 172.21.15.7:0/2370671566 >> 172.21.15.7:0/2370671566 conn(0x7f02400b16d0 msgr2=0x7f02400cb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.462+0000 7f0247c00640 1 -- 172.21.15.7:0/2370671566 shutdown_connections 2024-09-10T21:51:37.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.463+0000 7f0247c00640 1 -- 172.21.15.7:0/2370671566 wait complete. 2024-09-10T21:51:37.500 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:51:37.667 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.666+0000 7f539b200640 1 -- 172.21.15.7:0/3794266558 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f539c0b1120 msgr2=0x7f539c0d26f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:37.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.666+0000 7f539b200640 1 --2- 172.21.15.7:0/3794266558 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f539c0b1120 0x7f539c0d26f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f5394009be0 tx=0x7f539402a920 comp rx=0 tx=0).stop 2024-09-10T21:51:37.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.666+0000 7f539b200640 1 -- 172.21.15.7:0/3794266558 shutdown_connections 2024-09-10T21:51:37.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.666+0000 7f539b200640 1 --2- 172.21.15.7:0/3794266558 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f539c0b1120 0x7f539c0d26f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:37.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.666+0000 7f539b200640 1 -- 172.21.15.7:0/3794266558 >> 172.21.15.7:0/3794266558 conn(0x7f539c0d3e80 msgr2=0x7f539c0d4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:37.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.666+0000 7f539b200640 1 -- 172.21.15.7:0/3794266558 shutdown_connections 2024-09-10T21:51:37.668 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.666+0000 7f539b200640 1 -- 172.21.15.7:0/3794266558 wait complete. 2024-09-10T21:51:37.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.668+0000 7f539b200640 1 Processor -- start 2024-09-10T21:51:37.669 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.668+0000 7f539b200640 1 -- start start 2024-09-10T21:51:37.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.669+0000 7f539b200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f539c0b1120 0x7f539c1f3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:37.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.669+0000 7f539b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f539c0d22b0 0x7f539c15beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:37.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.669+0000 7f539b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f539c15df70 0x7f539c15c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:37.670 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.669+0000 7f539b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f539c0b2b70 con 0x7f539c0d22b0 2024-09-10T21:51:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.669+0000 7f539b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f539c0b29f0 con 0x7f539c15df70 2024-09-10T21:51:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.669+0000 7f539b200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f539c0b2cf0 con 0x7f539c0b1120 2024-09-10T21:51:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.669+0000 7f5391400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f539c0d22b0 0x7f539c15beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.669+0000 7f5391400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f539c0d22b0 0x7f539c15beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44836/0 (socket says 172.21.15.7:44836) 2024-09-10T21:51:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.670+0000 7f5391400640 1 -- 172.21.15.7:0/3248581042 learned_addr learned my addr 172.21.15.7:0/3248581042 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.670+0000 7f5399e00640 1 --2- 172.21.15.7:0/3248581042 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f539c0b1120 0x7f539c1f3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:37.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.670+0000 7f539a800640 1 --2- 172.21.15.7:0/3248581042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f539c15df70 0x7f539c15c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.670+0000 7f5391400640 1 -- 172.21.15.7:0/3248581042 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f539c0b1120 msgr2=0x7f539c1f3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.670+0000 7f5391400640 1 --2- 172.21.15.7:0/3248581042 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f539c0b1120 0x7f539c1f3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.670+0000 7f5391400640 1 -- 172.21.15.7:0/3248581042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f539c15df70 msgr2=0x7f539c15c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.670+0000 7f5391400640 1 --2- 172.21.15.7:0/3248581042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f539c15df70 0x7f539c15c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.670+0000 7f5391400640 1 -- 172.21.15.7:0/3248581042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c1f4490 con 0x7f539c0d22b0 2024-09-10T21:51:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.670+0000 7f5391400640 1 --2- 172.21.15.7:0/3248581042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f539c0d22b0 0x7f539c15beb0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f538400c4d0 tx=0x7f538400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:37.672 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.670+0000 7f5392a00640 1 -- 172.21.15.7:0/3248581042 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384016020 con 0x7f539c0d22b0 2024-09-10T21:51:37.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.671+0000 7f5392a00640 1 -- 172.21.15.7:0/3248581042 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5384012920 con 0x7f539c0d22b0 2024-09-10T21:51:37.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.671+0000 7f5392a00640 1 -- 172.21.15.7:0/3248581042 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384012d70 con 0x7f539c0d22b0 2024-09-10T21:51:37.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.671+0000 7f539b200640 1 -- 172.21.15.7:0/3248581042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539c0ce240 con 0x7f539c0d22b0 2024-09-10T21:51:37.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.671+0000 7f539b200640 1 -- 172.21.15.7:0/3248581042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539c1f7f80 con 0x7f539c0d22b0 2024-09-10T21:51:37.673 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.672+0000 7f539b200640 1 -- 172.21.15.7:0/3248581042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f539c0b3210 con 0x7f539c0d22b0 2024-09-10T21:51:37.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.673+0000 7f5392a00640 1 -- 172.21.15.7:0/3248581042 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5384005020 con 0x7f539c0d22b0 2024-09-10T21:51:37.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.674+0000 7f5392a00640 1 --2- 172.21.15.7:0/3248581042 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5378077fa0 0x7f537807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:37.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.674+0000 7f5392a00640 1 -- 172.21.15.7:0/3248581042 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 5609+0+0 (secure 0 0 0) 0x7f5384099f80 con 0x7f539c0d22b0 2024-09-10T21:51:37.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.675+0000 7f5399e00640 1 --2- 172.21.15.7:0/3248581042 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5378077fa0 0x7f537807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:37.676 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.675+0000 7f5399e00640 1 --2- 172.21.15.7:0/3248581042 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5378077fa0 0x7f537807a460 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f538c001120 tx=0x7f538c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:37.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.682+0000 7f5392a00640 1 -- 172.21.15.7:0/3248581042 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5384020030 con 0x7f539c0d22b0 2024-09-10T21:51:37.913 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:37.912+0000 7f539b200640 1 -- 172.21.15.7:0/3248581042 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f539c06da80 con 0x7f539c0d22b0 2024-09-10T21:51:38.485 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.484+0000 7f5392a00640 1 -- 172.21.15.7:0/3248581042 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v98) ==== 174+0+0 (secure 0 0 0) 0x7f5384062530 con 0x7f539c0d22b0 2024-09-10T21:51:38.486 INFO:teuthology.orchestra.run.smithi007.stderr:new fs with metadata pool 16 and data pool 17 2024-09-10T21:51:38.491 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.490+0000 7f539b200640 1 -- 172.21.15.7:0/3248581042 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5378077fa0 msgr2=0x7f537807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.490+0000 7f539b200640 1 --2- 172.21.15.7:0/3248581042 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5378077fa0 0x7f537807a460 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f538c001120 tx=0x7f538c009040 comp rx=0 tx=0).stop 2024-09-10T21:51:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.490+0000 7f539b200640 1 -- 172.21.15.7:0/3248581042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f539c0d22b0 msgr2=0x7f539c15beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:38.492 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.490+0000 7f539b200640 1 --2- 172.21.15.7:0/3248581042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f539c0d22b0 0x7f539c15beb0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f538400c4d0 tx=0x7f538400c990 comp rx=0 tx=0).stop 2024-09-10T21:51:38.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.491+0000 7f539b200640 1 -- 172.21.15.7:0/3248581042 shutdown_connections 2024-09-10T21:51:38.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.491+0000 7f539b200640 1 --2- 172.21.15.7:0/3248581042 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5378077fa0 0x7f537807a460 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:38.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.491+0000 7f539b200640 1 --2- 172.21.15.7:0/3248581042 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f539c15df70 0x7f539c15c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:38.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.491+0000 7f539b200640 1 --2- 172.21.15.7:0/3248581042 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f539c0d22b0 0x7f539c15beb0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:38.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.491+0000 7f539b200640 1 --2- 172.21.15.7:0/3248581042 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f539c0b1120 0x7f539c1f3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:38.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.491+0000 7f539b200640 1 -- 172.21.15.7:0/3248581042 >> 172.21.15.7:0/3248581042 conn(0x7f539c0d3e80 msgr2=0x7f539c0b78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:38.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.492+0000 7f539b200640 1 -- 172.21.15.7:0/3248581042 shutdown_connections 2024-09-10T21:51:38.495 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.492+0000 7f539b200640 1 -- 172.21.15.7:0/3248581042 wait complete. 2024-09-10T21:51:38.522 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:51:38.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.682+0000 7f623aa00640 1 -- 172.21.15.7:0/3514292748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62340cd750 msgr2=0x7f62340b29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:38.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.682+0000 7f623aa00640 1 --2- 172.21.15.7:0/3514292748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62340cd750 0x7f62340b29d0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f6220005fd0 tx=0x7f622002a540 comp rx=0 tx=0).stop 2024-09-10T21:51:38.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.682+0000 7f623aa00640 1 -- 172.21.15.7:0/3514292748 shutdown_connections 2024-09-10T21:51:38.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.682+0000 7f623aa00640 1 --2- 172.21.15.7:0/3514292748 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f62340cd750 0x7f62340b29d0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:38.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.682+0000 7f623aa00640 1 --2- 172.21.15.7:0/3514292748 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f623415bc00 0x7f62340ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:38.683 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.682+0000 7f623aa00640 1 -- 172.21.15.7:0/3514292748 >> 172.21.15.7:0/3514292748 conn(0x7f62340b16d0 msgr2=0x7f62340b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:38.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.682+0000 7f623aa00640 1 -- 172.21.15.7:0/3514292748 shutdown_connections 2024-09-10T21:51:38.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.682+0000 7f623aa00640 1 -- 172.21.15.7:0/3514292748 wait complete. 2024-09-10T21:51:38.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.684+0000 7f623aa00640 1 Processor -- start 2024-09-10T21:51:38.684 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.684+0000 7f623aa00640 1 -- start start 2024-09-10T21:51:38.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.685+0000 7f623aa00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f623406fcf0 0x7f62340b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:38.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.685+0000 7f623aa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62340cd750 0x7f623414f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:38.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.685+0000 7f623aa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623415bc00 0x7f62341f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:38.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.685+0000 7f623aa00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6234073e70 con 0x7f623415bc00 2024-09-10T21:51:38.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.685+0000 7f623aa00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6234073cf0 con 0x7f62340cd750 2024-09-10T21:51:38.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.685+0000 7f623aa00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f6234073ff0 con 0x7f623406fcf0 2024-09-10T21:51:38.686 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.685+0000 7f6238a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623415bc00 0x7f62341f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:38.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.685+0000 7f6238a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623415bc00 0x7f62341f5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44852/0 (socket says 172.21.15.7:44852) 2024-09-10T21:51:38.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.685+0000 7f6238a00640 1 -- 172.21.15.7:0/1875596644 learned_addr learned my addr 172.21.15.7:0/1875596644 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:38.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.685+0000 7f6233e00640 1 --2- 172.21.15.7:0/1875596644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f623406fcf0 0x7f62340b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:38.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.685+0000 7f6233400640 1 --2- 172.21.15.7:0/1875596644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62340cd750 0x7f623414f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:38.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.685+0000 7f6238a00640 1 -- 172.21.15.7:0/1875596644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f623406fcf0 msgr2=0x7f62340b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:38.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.685+0000 7f6238a00640 1 --2- 172.21.15.7:0/1875596644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f623406fcf0 0x7f62340b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:38.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.685+0000 7f6238a00640 1 -- 172.21.15.7:0/1875596644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62340cd750 msgr2=0x7f623414f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:38.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.685+0000 7f6238a00640 1 --2- 172.21.15.7:0/1875596644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62340cd750 0x7f623414f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:38.687 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.685+0000 7f6238a00640 1 -- 172.21.15.7:0/1875596644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62340b4ef0 con 0x7f623415bc00 2024-09-10T21:51:38.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.686+0000 7f6238a00640 1 --2- 172.21.15.7:0/1875596644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623415bc00 0x7f62341f5080 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f622c00c760 tx=0x7f622c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:38.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.686+0000 7f622be00640 1 -- 172.21.15.7:0/1875596644 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c003b70 con 0x7f623415bc00 2024-09-10T21:51:38.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.686+0000 7f622be00640 1 -- 172.21.15.7:0/1875596644 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f622c003d30 con 0x7f623415bc00 2024-09-10T21:51:38.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.686+0000 7f623aa00640 1 -- 172.21.15.7:0/1875596644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62340b51d0 con 0x7f623415bc00 2024-09-10T21:51:38.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.686+0000 7f622be00640 1 -- 172.21.15.7:0/1875596644 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622c00aab0 con 0x7f623415bc00 2024-09-10T21:51:38.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.686+0000 7f623aa00640 1 -- 172.21.15.7:0/1875596644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62341f4b60 con 0x7f623415bc00 2024-09-10T21:51:38.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.687+0000 7f623aa00640 1 -- 172.21.15.7:0/1875596644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6234074480 con 0x7f623415bc00 2024-09-10T21:51:38.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.689+0000 7f622be00640 1 -- 172.21.15.7:0/1875596644 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f622c00ac50 con 0x7f623415bc00 2024-09-10T21:51:38.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.689+0000 7f622be00640 1 --2- 172.21.15.7:0/1875596644 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6214077f70 0x7f621407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:38.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.690+0000 7f622be00640 1 -- 172.21.15.7:0/1875596644 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7f622c099a50 con 0x7f623415bc00 2024-09-10T21:51:38.691 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.690+0000 7f6233e00640 1 --2- 172.21.15.7:0/1875596644 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6214077f70 0x7f621407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:38.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.691+0000 7f6233e00640 1 --2- 172.21.15.7:0/1875596644 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6214077f70 0x7f621407a430 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f622400bb50 tx=0x7f622400b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:38.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.694+0000 7f622be00640 1 -- 172.21.15.7:0/1875596644 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f622c061e60 con 0x7f623415bc00 2024-09-10T21:51:38.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.890+0000 7f623aa00640 1 -- 172.21.15.7:0/1875596644 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6234072a20 con 0x7f623415bc00 2024-09-10T21:51:38.892 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.891+0000 7f622be00640 1 -- 172.21.15.7:0/1875596644 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16664 (secure 0 0 0) 0x7f622c066890 con 0x7f623415bc00 2024-09-10T21:51:38.893 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:38.893 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":113,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:51:37.913991+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:51:35.736733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T21:51:36.873305+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:51:38.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.894+0000 7f623aa00640 1 -- 172.21.15.7:0/1875596644 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6214077f70 msgr2=0x7f621407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:38.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.894+0000 7f623aa00640 1 --2- 172.21.15.7:0/1875596644 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6214077f70 0x7f621407a430 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f622400bb50 tx=0x7f622400b2e0 comp rx=0 tx=0).stop 2024-09-10T21:51:38.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.894+0000 7f623aa00640 1 -- 172.21.15.7:0/1875596644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623415bc00 msgr2=0x7f62341f5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:38.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.894+0000 7f623aa00640 1 --2- 172.21.15.7:0/1875596644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623415bc00 0x7f62341f5080 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f622c00c760 tx=0x7f622c00cc20 comp rx=0 tx=0).stop 2024-09-10T21:51:38.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.894+0000 7f623aa00640 1 -- 172.21.15.7:0/1875596644 shutdown_connections 2024-09-10T21:51:38.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.894+0000 7f623aa00640 1 --2- 172.21.15.7:0/1875596644 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6214077f70 0x7f621407a430 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:38.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.894+0000 7f623aa00640 1 --2- 172.21.15.7:0/1875596644 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f62340cd750 0x7f623414f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:38.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.894+0000 7f623aa00640 1 --2- 172.21.15.7:0/1875596644 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f623415bc00 0x7f62341f5080 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:38.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.894+0000 7f623aa00640 1 --2- 172.21.15.7:0/1875596644 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f623406fcf0 0x7f62340b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:38.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.894+0000 7f623aa00640 1 -- 172.21.15.7:0/1875596644 >> 172.21.15.7:0/1875596644 conn(0x7f62340b16d0 msgr2=0x7f62341595a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:38.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.894+0000 7f623aa00640 1 -- 172.21.15.7:0/1875596644 shutdown_connections 2024-09-10T21:51:38.896 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:38.894+0000 7f623aa00640 1 -- 172.21.15.7:0/1875596644 wait complete. 2024-09-10T21:51:38.911 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:51:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.070+0000 7fd99f200640 1 -- 172.21.15.7:0/2123679388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd998146c50 msgr2=0x7fd9980723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.070+0000 7fd99f200640 1 --2- 172.21.15.7:0/2123679388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd998146c50 0x7fd9980723b0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fd998073600 tx=0x7fd98c02a4d0 comp rx=0 tx=0).stop 2024-09-10T21:51:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.071+0000 7fd99f200640 1 -- 172.21.15.7:0/2123679388 shutdown_connections 2024-09-10T21:51:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.071+0000 7fd99f200640 1 --2- 172.21.15.7:0/2123679388 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9981470b0 0x7fd998072e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.071+0000 7fd99f200640 1 --2- 172.21.15.7:0/2123679388 >> v2:172.21.15.133:3300/0 conn(0x7fd99806f590 0x7fd9980728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.071+0000 7fd99f200640 1 --2- 172.21.15.7:0/2123679388 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd998146c50 0x7fd9980723b0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.071+0000 7fd99f200640 1 -- 172.21.15.7:0/2123679388 >> 172.21.15.7:0/2123679388 conn(0x7fd9980b1d70 msgr2=0x7fd9980cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.071+0000 7fd99f200640 1 -- 172.21.15.7:0/2123679388 shutdown_connections 2024-09-10T21:51:39.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.071+0000 7fd99f200640 1 -- 172.21.15.7:0/2123679388 wait complete. 2024-09-10T21:51:39.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.072+0000 7fd99f200640 1 Processor -- start 2024-09-10T21:51:39.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.072+0000 7fd99f200640 1 -- start start 2024-09-10T21:51:39.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.073+0000 7fd99f200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99806f590 0x7fd99813fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:39.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.073+0000 7fd99f200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd998146c50 0x7fd99813c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:39.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.073+0000 7fd99f200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9981470b0 0x7fd99815fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:39.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.073+0000 7fd99f200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd998161c50 con 0x7fd99806f590 2024-09-10T21:51:39.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.073+0000 7fd99f200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd998161ad0 con 0x7fd998146c50 2024-09-10T21:51:39.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.073+0000 7fd99f200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fd998161dd0 con 0x7fd9981470b0 2024-09-10T21:51:39.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.074+0000 7fd99de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99806f590 0x7fd99813fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:39.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.074+0000 7fd99de00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99806f590 0x7fd99813fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44878/0 (socket says 172.21.15.7:44878) 2024-09-10T21:51:39.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.074+0000 7fd99de00640 1 -- 172.21.15.7:0/4121995502 learned_addr learned my addr 172.21.15.7:0/4121995502 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:39.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.074+0000 7fd99e800640 1 --2- 172.21.15.7:0/4121995502 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9981470b0 0x7fd99815fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:39.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.074+0000 7fd99d400640 1 --2- 172.21.15.7:0/4121995502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd998146c50 0x7fd99813c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:39.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.074+0000 7fd99de00640 1 -- 172.21.15.7:0/4121995502 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9981470b0 msgr2=0x7fd99815fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:39.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.074+0000 7fd99de00640 1 --2- 172.21.15.7:0/4121995502 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9981470b0 0x7fd99815fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:39.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.074+0000 7fd99de00640 1 -- 172.21.15.7:0/4121995502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd998146c50 msgr2=0x7fd99813c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:39.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.074+0000 7fd99de00640 1 --2- 172.21.15.7:0/4121995502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd998146c50 0x7fd99813c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:39.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.074+0000 7fd99de00640 1 -- 172.21.15.7:0/4121995502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99813d780 con 0x7fd99806f590 2024-09-10T21:51:39.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.074+0000 7fd99de00640 1 --2- 172.21.15.7:0/4121995502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99806f590 0x7fd99813fa60 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fd98c0099c0 tx=0x7fd98c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:39.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.075+0000 7fd992000640 1 -- 172.21.15.7:0/4121995502 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c03f070 con 0x7fd99806f590 2024-09-10T21:51:39.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.075+0000 7fd992000640 1 -- 172.21.15.7:0/4121995502 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd98c0047a0 con 0x7fd99806f590 2024-09-10T21:51:39.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.075+0000 7fd99f200640 1 -- 172.21.15.7:0/4121995502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd998140140 con 0x7fd99806f590 2024-09-10T21:51:39.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.075+0000 7fd992000640 1 -- 172.21.15.7:0/4121995502 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c035030 con 0x7fd99806f590 2024-09-10T21:51:39.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.075+0000 7fd99f200640 1 -- 172.21.15.7:0/4121995502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9981f8370 con 0x7fd99806f590 2024-09-10T21:51:39.077 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.076+0000 7fd99f200640 1 -- 172.21.15.7:0/4121995502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd99813cd10 con 0x7fd99806f590 2024-09-10T21:51:39.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.077+0000 7fd992000640 1 -- 172.21.15.7:0/4121995502 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd98c004bf0 con 0x7fd99806f590 2024-09-10T21:51:39.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.079+0000 7fd992000640 1 --2- 172.21.15.7:0/4121995502 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd97c077f70 0x7fd97c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:39.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.079+0000 7fd992000640 1 -- 172.21.15.7:0/4121995502 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7fd98c0bb1d0 con 0x7fd99806f590 2024-09-10T21:51:39.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.079+0000 7fd99d400640 1 --2- 172.21.15.7:0/4121995502 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd97c077f70 0x7fd97c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:39.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.080+0000 7fd99d400640 1 --2- 172.21.15.7:0/4121995502 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd97c077f70 0x7fd97c07a430 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd9980b0ad0 tx=0x7fd98800b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:39.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.083+0000 7fd992000640 1 -- 172.21.15.7:0/4121995502 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd98c083670 con 0x7fd99806f590 2024-09-10T21:51:39.279 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.278+0000 7fd99f200640 1 -- 172.21.15.7:0/4121995502 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd998162360 con 0x7fd99806f590 2024-09-10T21:51:39.280 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.279+0000 7fd992000640 1 -- 172.21.15.7:0/4121995502 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) ==== 74+0+16664 (secure 0 0 0) 0x7fd98c08c3e0 con 0x7fd99806f590 2024-09-10T21:51:39.280 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:39.281 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":113,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:51:37.913991+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:51:35.736733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T21:51:36.873305+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:51:39.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.282+0000 7fd99f200640 1 -- 172.21.15.7:0/4121995502 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd97c077f70 msgr2=0x7fd97c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:39.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.282+0000 7fd99f200640 1 --2- 172.21.15.7:0/4121995502 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd97c077f70 0x7fd97c07a430 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd9980b0ad0 tx=0x7fd98800b390 comp rx=0 tx=0).stop 2024-09-10T21:51:39.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.282+0000 7fd99f200640 1 -- 172.21.15.7:0/4121995502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99806f590 msgr2=0x7fd99813fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:39.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.282+0000 7fd99f200640 1 --2- 172.21.15.7:0/4121995502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99806f590 0x7fd99813fa60 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fd98c0099c0 tx=0x7fd98c0038a0 comp rx=0 tx=0).stop 2024-09-10T21:51:39.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.282+0000 7fd99f200640 1 -- 172.21.15.7:0/4121995502 shutdown_connections 2024-09-10T21:51:39.283 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.282+0000 7fd99f200640 1 --2- 172.21.15.7:0/4121995502 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd97c077f70 0x7fd97c07a430 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:39.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.282+0000 7fd99f200640 1 --2- 172.21.15.7:0/4121995502 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd998146c50 0x7fd99813c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:39.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.282+0000 7fd99f200640 1 --2- 172.21.15.7:0/4121995502 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd9981470b0 0x7fd99815fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:39.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.282+0000 7fd99f200640 1 --2- 172.21.15.7:0/4121995502 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd99806f590 0x7fd99813fa60 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:39.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.282+0000 7fd99f200640 1 -- 172.21.15.7:0/4121995502 >> 172.21.15.7:0/4121995502 conn(0x7fd9980b1d70 msgr2=0x7fd9980b1000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:39.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.282+0000 7fd99f200640 1 -- 172.21.15.7:0/4121995502 shutdown_connections 2024-09-10T21:51:39.284 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.282+0000 7fd99f200640 1 -- 172.21.15.7:0/4121995502 wait complete. 2024-09-10T21:51:39.299 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:51:39.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.464+0000 7fd129c00640 1 -- 172.21.15.7:0/235190643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd12406fcf0 msgr2=0x7fd124073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:39.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.464+0000 7fd129c00640 1 --2- 172.21.15.7:0/235190643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd12406fcf0 0x7fd124073d40 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fd110005fd0 tx=0x7fd11002a5d0 comp rx=0 tx=0).stop 2024-09-10T21:51:39.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.464+0000 7fd129c00640 1 -- 172.21.15.7:0/235190643 shutdown_connections 2024-09-10T21:51:39.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.464+0000 7fd129c00640 1 --2- 172.21.15.7:0/235190643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd12406fcf0 0x7fd124073d40 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:39.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.464+0000 7fd129c00640 1 --2- 172.21.15.7:0/235190643 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd124156090 0x7fd124160100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:39.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.464+0000 7fd129c00640 1 -- 172.21.15.7:0/235190643 >> 172.21.15.7:0/235190643 conn(0x7fd1240b16d0 msgr2=0x7fd1240b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:39.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.464+0000 7fd129c00640 1 -- 172.21.15.7:0/235190643 shutdown_connections 2024-09-10T21:51:39.465 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.464+0000 7fd129c00640 1 -- 172.21.15.7:0/235190643 wait complete. 2024-09-10T21:51:39.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.465+0000 7fd129c00640 1 Processor -- start 2024-09-10T21:51:39.466 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.466+0000 7fd129c00640 1 -- start start 2024-09-10T21:51:39.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.466+0000 7fd129c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd12406fcf0 0x7fd1240c4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:39.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.466+0000 7fd129c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd124074280 0x7fd1240c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:39.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.466+0000 7fd129c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd124156090 0x7fd124156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:39.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.466+0000 7fd129c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd12406ec90 con 0x7fd124156090 2024-09-10T21:51:39.468 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.466+0000 7fd129c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd1240cbb40 con 0x7fd124074280 2024-09-10T21:51:39.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.467+0000 7fd129c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fd12406ee10 con 0x7fd12406fcf0 2024-09-10T21:51:39.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.467+0000 7fd123400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd124156090 0x7fd124156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:39.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.467+0000 7fd122a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd12406fcf0 0x7fd1240c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:39.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.467+0000 7fd123400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd124156090 0x7fd124156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44900/0 (socket says 172.21.15.7:44900) 2024-09-10T21:51:39.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.467+0000 7fd123400640 1 -- 172.21.15.7:0/150455706 learned_addr learned my addr 172.21.15.7:0/150455706 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:39.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.467+0000 7fd122000640 1 --2- 172.21.15.7:0/150455706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd124074280 0x7fd1240c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:39.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.467+0000 7fd123400640 1 -- 172.21.15.7:0/150455706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd12406fcf0 msgr2=0x7fd1240c4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:39.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.467+0000 7fd123400640 1 --2- 172.21.15.7:0/150455706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd12406fcf0 0x7fd1240c4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:39.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.467+0000 7fd123400640 1 -- 172.21.15.7:0/150455706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd124074280 msgr2=0x7fd1240c54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:39.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.467+0000 7fd123400640 1 --2- 172.21.15.7:0/150455706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd124074280 0x7fd1240c54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:39.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.467+0000 7fd123400640 1 -- 172.21.15.7:0/150455706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1241f26d0 con 0x7fd124156090 2024-09-10T21:51:39.469 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.468+0000 7fd123400640 1 --2- 172.21.15.7:0/150455706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd124156090 0x7fd124156680 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00c7a0 tx=0x7fd11c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:39.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.468+0000 7fd11aa00640 1 -- 172.21.15.7:0/150455706 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c003b70 con 0x7fd124156090 2024-09-10T21:51:39.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.468+0000 7fd11aa00640 1 -- 172.21.15.7:0/150455706 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd11c003d10 con 0x7fd124156090 2024-09-10T21:51:39.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.468+0000 7fd11aa00640 1 -- 172.21.15.7:0/150455706 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd11c00aab0 con 0x7fd124156090 2024-09-10T21:51:39.470 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.468+0000 7fd129c00640 1 -- 172.21.15.7:0/150455706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1240c4a10 con 0x7fd124156090 2024-09-10T21:51:39.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.468+0000 7fd129c00640 1 -- 172.21.15.7:0/150455706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1240c4cb0 con 0x7fd124156090 2024-09-10T21:51:39.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.469+0000 7fd129c00640 1 -- 172.21.15.7:0/150455706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd12406f2c0 con 0x7fd124156090 2024-09-10T21:51:39.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.470+0000 7fd11aa00640 1 -- 172.21.15.7:0/150455706 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd11c00ac50 con 0x7fd124156090 2024-09-10T21:51:39.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.471+0000 7fd11aa00640 1 --2- 172.21.15.7:0/150455706 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd104077f70 0x7fd10407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:39.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.471+0000 7fd11aa00640 1 -- 172.21.15.7:0/150455706 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 5709+0+0 (secure 0 0 0) 0x7fd11c099af0 con 0x7fd124156090 2024-09-10T21:51:39.472 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.472+0000 7fd122a00640 1 --2- 172.21.15.7:0/150455706 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd104077f70 0x7fd10407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:39.473 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.472+0000 7fd122a00640 1 --2- 172.21.15.7:0/150455706 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd104077f70 0x7fd10407a430 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd114004590 tx=0x7fd11400b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:39.476 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.475+0000 7fd11aa00640 1 -- 172.21.15.7:0/150455706 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd11c061f90 con 0x7fd124156090 2024-09-10T21:51:39.708 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:39.708+0000 7fd129c00640 1 -- 172.21.15.7:0/150455706 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fd12406e620 con 0x7fd124156090 2024-09-10T21:51:40.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.498+0000 7fd11aa00640 1 -- 172.21.15.7:0/150455706 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v101) ==== 118+0+0 (secure 0 0 0) 0x7fd11c0669c0 con 0x7fd124156090 2024-09-10T21:51:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.503+0000 7fd129c00640 1 -- 172.21.15.7:0/150455706 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd104077f70 msgr2=0x7fd10407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.503+0000 7fd129c00640 1 --2- 172.21.15.7:0/150455706 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd104077f70 0x7fd10407a430 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd114004590 tx=0x7fd11400b390 comp rx=0 tx=0).stop 2024-09-10T21:51:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.503+0000 7fd129c00640 1 -- 172.21.15.7:0/150455706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd124156090 msgr2=0x7fd124156680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.503+0000 7fd129c00640 1 --2- 172.21.15.7:0/150455706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd124156090 0x7fd124156680 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fd11c00c7a0 tx=0x7fd11c00cc60 comp rx=0 tx=0).stop 2024-09-10T21:51:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.503+0000 7fd129c00640 1 -- 172.21.15.7:0/150455706 shutdown_connections 2024-09-10T21:51:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.503+0000 7fd129c00640 1 --2- 172.21.15.7:0/150455706 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd104077f70 0x7fd10407a430 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:40.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.503+0000 7fd129c00640 1 --2- 172.21.15.7:0/150455706 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd124074280 0x7fd1240c54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:40.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.503+0000 7fd129c00640 1 --2- 172.21.15.7:0/150455706 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd124156090 0x7fd124156680 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:40.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.503+0000 7fd129c00640 1 --2- 172.21.15.7:0/150455706 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd12406fcf0 0x7fd1240c4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:40.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.503+0000 7fd129c00640 1 -- 172.21.15.7:0/150455706 >> 172.21.15.7:0/150455706 conn(0x7fd1240b16d0 msgr2=0x7fd12415f070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:40.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.503+0000 7fd129c00640 1 -- 172.21.15.7:0/150455706 shutdown_connections 2024-09-10T21:51:40.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.503+0000 7fd129c00640 1 -- 172.21.15.7:0/150455706 wait complete. 2024-09-10T21:51:40.522 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-10T21:51:40.523 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:51:40.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.692+0000 7f7d2d200640 1 -- 172.21.15.7:0/1856471667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d28161f50 msgr2=0x7f7d280b5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:40.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.692+0000 7f7d2d200640 1 --2- 172.21.15.7:0/1856471667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d28161f50 0x7f7d280b5350 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f7d18009830 tx=0x7f7d1802a520 comp rx=0 tx=0).stop 2024-09-10T21:51:40.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.692+0000 7f7d2d200640 1 -- 172.21.15.7:0/1856471667 shutdown_connections 2024-09-10T21:51:40.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.692+0000 7f7d2d200640 1 --2- 172.21.15.7:0/1856471667 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d28162770 0x7f7d28161060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:40.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.692+0000 7f7d2d200640 1 --2- 172.21.15.7:0/1856471667 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d28161f50 0x7f7d280b5350 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:40.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.692+0000 7f7d2d200640 1 -- 172.21.15.7:0/1856471667 >> 172.21.15.7:0/1856471667 conn(0x7f7d280d3dd0 msgr2=0x7f7d280d41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:40.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.692+0000 7f7d2d200640 1 -- 172.21.15.7:0/1856471667 shutdown_connections 2024-09-10T21:51:40.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.692+0000 7f7d2d200640 1 -- 172.21.15.7:0/1856471667 wait complete. 2024-09-10T21:51:40.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.694+0000 7f7d2d200640 1 Processor -- start 2024-09-10T21:51:40.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.694+0000 7f7d2d200640 1 -- start start 2024-09-10T21:51:40.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.695+0000 7f7d2d200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d28161f50 0x7f7d280b9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:40.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.695+0000 7f7d2d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d28162330 0x7f7d28156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:40.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.695+0000 7f7d2d200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d28162770 0x7f7d280b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:40.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.695+0000 7f7d2d200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7d281619e0 con 0x7f7d28162330 2024-09-10T21:51:40.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.695+0000 7f7d2d200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7d28161860 con 0x7f7d28161f50 2024-09-10T21:51:40.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.695+0000 7f7d2d200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f7d280b14d0 con 0x7f7d28162770 2024-09-10T21:51:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.695+0000 7f7d1ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d28162330 0x7f7d28156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.695+0000 7f7d1ea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d28162330 0x7f7d28156320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44908/0 (socket says 172.21.15.7:44908) 2024-09-10T21:51:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.695+0000 7f7d1ea00640 1 -- 172.21.15.7:0/1145536968 learned_addr learned my addr 172.21.15.7:0/1145536968 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.695+0000 7f7d27400640 1 --2- 172.21.15.7:0/1145536968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d28161f50 0x7f7d280b9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.695+0000 7f7d27e00640 1 --2- 172.21.15.7:0/1145536968 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d28162770 0x7f7d280b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.696+0000 7f7d1ea00640 1 -- 172.21.15.7:0/1145536968 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d28162770 msgr2=0x7f7d280b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.696+0000 7f7d1ea00640 1 --2- 172.21.15.7:0/1145536968 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d28162770 0x7f7d280b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.696+0000 7f7d1ea00640 1 -- 172.21.15.7:0/1145536968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d28161f50 msgr2=0x7f7d280b9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.696+0000 7f7d1ea00640 1 --2- 172.21.15.7:0/1145536968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d28161f50 0x7f7d280b9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.696+0000 7f7d1ea00640 1 -- 172.21.15.7:0/1145536968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d280b7fc0 con 0x7f7d28162330 2024-09-10T21:51:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.696+0000 7f7d27e00640 1 --2- 172.21.15.7:0/1145536968 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d28162770 0x7f7d280b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:51:40.698 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.696+0000 7f7d1ea00640 1 --2- 172.21.15.7:0/1145536968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d28162330 0x7f7d28156320 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000c490 tx=0x7f7d1000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:40.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.696+0000 7f7d1fe00640 1 -- 172.21.15.7:0/1145536968 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10016020 con 0x7f7d28162330 2024-09-10T21:51:40.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.696+0000 7f7d1fe00640 1 -- 172.21.15.7:0/1145536968 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d10012940 con 0x7f7d28162330 2024-09-10T21:51:40.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.696+0000 7f7d1fe00640 1 -- 172.21.15.7:0/1145536968 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10012d90 con 0x7f7d28162330 2024-09-10T21:51:40.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.696+0000 7f7d2d200640 1 -- 172.21.15.7:0/1145536968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d280b82a0 con 0x7f7d28162330 2024-09-10T21:51:40.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.696+0000 7f7d2d200640 1 -- 172.21.15.7:0/1145536968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d280b8860 con 0x7f7d28162330 2024-09-10T21:51:40.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.697+0000 7f7d2d200640 1 -- 172.21.15.7:0/1145536968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d280b5350 con 0x7f7d28162330 2024-09-10T21:51:40.699 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.698+0000 7f7d1fe00640 1 -- 172.21.15.7:0/1145536968 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7d1001a3b0 con 0x7f7d28162330 2024-09-10T21:51:40.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.699+0000 7f7d1fe00640 1 --2- 172.21.15.7:0/1145536968 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7d04077fa0 0x7f7d0407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:40.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.699+0000 7f7d1fe00640 1 -- 172.21.15.7:0/1145536968 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5709+0+0 (secure 0 0 0) 0x7f7d10099c50 con 0x7f7d28162330 2024-09-10T21:51:40.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.700+0000 7f7d27400640 1 --2- 172.21.15.7:0/1145536968 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7d04077fa0 0x7f7d0407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:40.701 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.701+0000 7f7d27400640 1 --2- 172.21.15.7:0/1145536968 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7d04077fa0 0x7f7d0407a460 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7d18034000 tx=0x7f7d18004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:40.704 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.703+0000 7f7d1fe00640 1 -- 172.21.15.7:0/1145536968 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7d10020030 con 0x7f7d28162330 2024-09-10T21:51:40.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:40.934+0000 7f7d2d200640 1 -- 172.21.15.7:0/1145536968 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f7d2808ddf0 con 0x7f7d28162330 2024-09-10T21:51:41.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.500+0000 7f7d1fe00640 1 -- 172.21.15.7:0/1145536968 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v102) ==== 122+0+0 (secure 0 0 0) 0x7f7d10062740 con 0x7f7d28162330 2024-09-10T21:51:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.503+0000 7f7d2d200640 1 -- 172.21.15.7:0/1145536968 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7d04077fa0 msgr2=0x7f7d0407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.503+0000 7f7d2d200640 1 --2- 172.21.15.7:0/1145536968 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7d04077fa0 0x7f7d0407a460 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7d18034000 tx=0x7f7d18004730 comp rx=0 tx=0).stop 2024-09-10T21:51:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.503+0000 7f7d2d200640 1 -- 172.21.15.7:0/1145536968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d28162330 msgr2=0x7f7d28156320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.503+0000 7f7d2d200640 1 --2- 172.21.15.7:0/1145536968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d28162330 0x7f7d28156320 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000c490 tx=0x7f7d1000c950 comp rx=0 tx=0).stop 2024-09-10T21:51:41.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.503+0000 7f7d2d200640 1 -- 172.21.15.7:0/1145536968 shutdown_connections 2024-09-10T21:51:41.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.503+0000 7f7d2d200640 1 --2- 172.21.15.7:0/1145536968 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7d04077fa0 0x7f7d0407a460 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:41.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.503+0000 7f7d2d200640 1 --2- 172.21.15.7:0/1145536968 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7d28162770 0x7f7d280b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:41.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.503+0000 7f7d2d200640 1 --2- 172.21.15.7:0/1145536968 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7d28162330 0x7f7d28156320 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:41.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.503+0000 7f7d2d200640 1 --2- 172.21.15.7:0/1145536968 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7d28161f50 0x7f7d280b9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:41.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.503+0000 7f7d2d200640 1 -- 172.21.15.7:0/1145536968 >> 172.21.15.7:0/1145536968 conn(0x7f7d280d3dd0 msgr2=0x7f7d2815f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:41.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.503+0000 7f7d2d200640 1 -- 172.21.15.7:0/1145536968 shutdown_connections 2024-09-10T21:51:41.505 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.503+0000 7f7d2d200640 1 -- 172.21.15.7:0/1145536968 wait complete. 2024-09-10T21:51:41.523 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:51:41.523 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:51:41.688 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.687+0000 7f7a3d200640 1 -- 172.21.15.7:0/2554245825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3815fda0 msgr2=0x7f7a381601f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:41.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.687+0000 7f7a3d200640 1 --2- 172.21.15.7:0/2554245825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3815fda0 0x7f7a381601f0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f7a30005fd0 tx=0x7f7a3002a780 comp rx=0 tx=0).stop 2024-09-10T21:51:41.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.687+0000 7f7a3d200640 1 -- 172.21.15.7:0/2554245825 shutdown_connections 2024-09-10T21:51:41.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.687+0000 7f7a3d200640 1 --2- 172.21.15.7:0/2554245825 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3815fda0 0x7f7a381601f0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:41.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.687+0000 7f7a3d200640 1 --2- 172.21.15.7:0/2554245825 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a3815bc90 0x7f7a380ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:41.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.687+0000 7f7a3d200640 1 -- 172.21.15.7:0/2554245825 >> 172.21.15.7:0/2554245825 conn(0x7f7a380b16d0 msgr2=0x7f7a380b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:41.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.687+0000 7f7a3d200640 1 -- 172.21.15.7:0/2554245825 shutdown_connections 2024-09-10T21:51:41.689 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.687+0000 7f7a3d200640 1 -- 172.21.15.7:0/2554245825 wait complete. 2024-09-10T21:51:41.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.689+0000 7f7a3d200640 1 Processor -- start 2024-09-10T21:51:41.690 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.689+0000 7f7a3d200640 1 -- start start 2024-09-10T21:51:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.690+0000 7f7a3d200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a380cd750 0x7f7a381f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.690+0000 7f7a3d200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3815bc90 0x7f7a380c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:41.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.690+0000 7f7a3d200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a3815fda0 0x7f7a381f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.690+0000 7f7a3d200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3806fe70 con 0x7f7a3815bc90 2024-09-10T21:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.690+0000 7f7a3d200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7a3806fcf0 con 0x7f7a380cd750 2024-09-10T21:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.690+0000 7f7a3d200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f7a3806fff0 con 0x7f7a3815fda0 2024-09-10T21:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.690+0000 7f7a2d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3815bc90 0x7f7a380c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.690+0000 7f7a2d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3815bc90 0x7f7a380c6c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44920/0 (socket says 172.21.15.7:44920) 2024-09-10T21:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.690+0000 7f7a2d400640 1 -- 172.21.15.7:0/2139704231 learned_addr learned my addr 172.21.15.7:0/2139704231 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.691+0000 7f7a36800640 1 --2- 172.21.15.7:0/2139704231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a3815fda0 0x7f7a381f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.691+0000 7f7a35e00640 1 --2- 172.21.15.7:0/2139704231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a380cd750 0x7f7a381f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.691+0000 7f7a2d400640 1 -- 172.21.15.7:0/2139704231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a3815fda0 msgr2=0x7f7a381f1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.691+0000 7f7a2d400640 1 --2- 172.21.15.7:0/2139704231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a3815fda0 0x7f7a381f1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.691+0000 7f7a2d400640 1 -- 172.21.15.7:0/2139704231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a380cd750 msgr2=0x7f7a381f5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.691+0000 7f7a2d400640 1 --2- 172.21.15.7:0/2139704231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a380cd750 0x7f7a381f5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.691+0000 7f7a2d400640 1 -- 172.21.15.7:0/2139704231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a380c8500 con 0x7f7a3815bc90 2024-09-10T21:51:41.693 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.691+0000 7f7a35e00640 1 --2- 172.21.15.7:0/2139704231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a380cd750 0x7f7a381f5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:51:41.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.691+0000 7f7a2d400640 1 --2- 172.21.15.7:0/2139704231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3815bc90 0x7f7a380c6c50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000c500 tx=0x7f7a2000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:41.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.691+0000 7f7a2ea00640 1 -- 172.21.15.7:0/2139704231 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20017070 con 0x7f7a3815bc90 2024-09-10T21:51:41.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.691+0000 7f7a2ea00640 1 -- 172.21.15.7:0/2139704231 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a200035f0 con 0x7f7a3815bc90 2024-09-10T21:51:41.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.691+0000 7f7a3d200640 1 -- 172.21.15.7:0/2139704231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a380c7190 con 0x7f7a3815bc90 2024-09-10T21:51:41.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.692+0000 7f7a2ea00640 1 -- 172.21.15.7:0/2139704231 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20003a40 con 0x7f7a3815bc90 2024-09-10T21:51:41.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.692+0000 7f7a3d200640 1 -- 172.21.15.7:0/2139704231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a380c7740 con 0x7f7a3815bc90 2024-09-10T21:51:41.694 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.693+0000 7f7a3d200640 1 -- 172.21.15.7:0/2139704231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a3806e730 con 0x7f7a3815bc90 2024-09-10T21:51:41.695 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.694+0000 7f7a2ea00640 1 -- 172.21.15.7:0/2139704231 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7a20003be0 con 0x7f7a3815bc90 2024-09-10T21:51:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.695+0000 7f7a2ea00640 1 --2- 172.21.15.7:0/2139704231 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7a14077f70 0x7f7a1407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.695+0000 7f7a2ea00640 1 -- 172.21.15.7:0/2139704231 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5709+0+0 (secure 0 0 0) 0x7f7a200995a0 con 0x7f7a3815bc90 2024-09-10T21:51:41.696 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.696+0000 7f7a35e00640 1 --2- 172.21.15.7:0/2139704231 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7a14077f70 0x7f7a1407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:41.697 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.696+0000 7f7a35e00640 1 --2- 172.21.15.7:0/2139704231 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7a14077f70 0x7f7a1407a430 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7a380c95f0 tx=0x7f7a28005e70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:41.700 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.699+0000 7f7a2ea00640 1 -- 172.21.15.7:0/2139704231 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7a3806e730 con 0x7f7a3815bc90 2024-09-10T21:51:41.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.927+0000 7f7a3d200640 1 -- 172.21.15.7:0/2139704231 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7a380b1b50 con 0x7f7a3815bc90 2024-09-10T21:51:41.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.927+0000 7f7a2ea00640 1 -- 172.21.15.7:0/2139704231 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5233 (secure 0 0 0) 0x7f7a20062090 con 0x7f7a3815bc90 2024-09-10T21:51:41.928 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:41.928 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":102,"btime":"2024-09-10T21:51:41:499376+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3012572644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3012572644},{"type":"v1","addr":"172.21.15.7:6834","nonce":3012572644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/148356862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":148356862},{"type":"v1","addr":"172.21.15.7:6835","nonce":148356862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":236723,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/772257983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":772257983},{"type":"v1","addr":"172.21.15.133:6836","nonce":772257983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:51:37.913965+0000","modified":"2024-09-10T21:51:40.935708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236726},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236726":{"gid":236726,"name":"b","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3764354952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3764354952},{"type":"v1","addr":"172.21.15.133:6837","nonce":3764354952}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":236726,"qdb_cluster":[236726]},"id":8}]} 2024-09-10T21:51:41.929 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 102 2024-09-10T21:51:41.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.930+0000 7f7a3d200640 1 -- 172.21.15.7:0/2139704231 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7a14077f70 msgr2=0x7f7a1407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:41.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.930+0000 7f7a3d200640 1 --2- 172.21.15.7:0/2139704231 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7a14077f70 0x7f7a1407a430 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f7a380c95f0 tx=0x7f7a28005e70 comp rx=0 tx=0).stop 2024-09-10T21:51:41.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.930+0000 7f7a3d200640 1 -- 172.21.15.7:0/2139704231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3815bc90 msgr2=0x7f7a380c6c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:41.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.930+0000 7f7a3d200640 1 --2- 172.21.15.7:0/2139704231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3815bc90 0x7f7a380c6c50 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000c500 tx=0x7f7a2000c9c0 comp rx=0 tx=0).stop 2024-09-10T21:51:41.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.930+0000 7f7a3d200640 1 -- 172.21.15.7:0/2139704231 shutdown_connections 2024-09-10T21:51:41.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.930+0000 7f7a3d200640 1 --2- 172.21.15.7:0/2139704231 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7a14077f70 0x7f7a1407a430 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:41.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.930+0000 7f7a3d200640 1 --2- 172.21.15.7:0/2139704231 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7a3815fda0 0x7f7a381f1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:41.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.930+0000 7f7a3d200640 1 --2- 172.21.15.7:0/2139704231 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7a3815bc90 0x7f7a380c6c50 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:41.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.930+0000 7f7a3d200640 1 --2- 172.21.15.7:0/2139704231 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7a380cd750 0x7f7a381f5390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:41.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.930+0000 7f7a3d200640 1 -- 172.21.15.7:0/2139704231 >> 172.21.15.7:0/2139704231 conn(0x7f7a380b16d0 msgr2=0x7f7a380b29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:41.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.930+0000 7f7a3d200640 1 -- 172.21.15.7:0/2139704231 shutdown_connections 2024-09-10T21:51:41.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:41.930+0000 7f7a3d200640 1 -- 172.21.15.7:0/2139704231 wait complete. 2024-09-10T21:51:41.946 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:51:42.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.106+0000 7f1d78c00640 1 -- 172.21.15.7:0/2199875582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d7415bc90 msgr2=0x7f1d740ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:42.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.106+0000 7f1d78c00640 1 --2- 172.21.15.7:0/2199875582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d7415bc90 0x7f1d740ccef0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1d64009830 tx=0x7f1d6402a3f0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.106+0000 7f1d78c00640 1 -- 172.21.15.7:0/2199875582 shutdown_connections 2024-09-10T21:51:42.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.106+0000 7f1d78c00640 1 --2- 172.21.15.7:0/2199875582 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d740cd750 0x7f1d74073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.106+0000 7f1d78c00640 1 --2- 172.21.15.7:0/2199875582 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d7415bc90 0x7f1d740ccef0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.106+0000 7f1d78c00640 1 -- 172.21.15.7:0/2199875582 >> 172.21.15.7:0/2199875582 conn(0x7f1d740b16d0 msgr2=0x7f1d740b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:42.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.106+0000 7f1d78c00640 1 -- 172.21.15.7:0/2199875582 shutdown_connections 2024-09-10T21:51:42.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.106+0000 7f1d78c00640 1 -- 172.21.15.7:0/2199875582 wait complete. 2024-09-10T21:51:42.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.107+0000 7f1d78c00640 1 Processor -- start 2024-09-10T21:51:42.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.108+0000 7f1d78c00640 1 -- start start 2024-09-10T21:51:42.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.109+0000 7f1d78c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d74073c70 0x7f1d7415a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:42.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.109+0000 7f1d78c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d740cd750 0x7f1d740c8500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:42.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.109+0000 7f1d78c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d7415bc90 0x7f1d740c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:42.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.109+0000 7f1d78c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1d7406fcf0 con 0x7f1d74073c70 2024-09-10T21:51:42.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.109+0000 7f1d78c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1d741606c0 con 0x7f1d740cd750 2024-09-10T21:51:42.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.109+0000 7f1d78c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f1d7406fe70 con 0x7f1d7415bc90 2024-09-10T21:51:42.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.109+0000 7f1d71e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d74073c70 0x7f1d7415a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:42.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.109+0000 7f1d71e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d74073c70 0x7f1d7415a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44924/0 (socket says 172.21.15.7:44924) 2024-09-10T21:51:42.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.109+0000 7f1d71e00640 1 -- 172.21.15.7:0/3038443677 learned_addr learned my addr 172.21.15.7:0/3038443677 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:42.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.109+0000 7f1d6be00640 1 --2- 172.21.15.7:0/3038443677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d740cd750 0x7f1d740c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:42.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.109+0000 7f1d72800640 1 --2- 172.21.15.7:0/3038443677 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d7415bc90 0x7f1d740c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:42.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.109+0000 7f1d71e00640 1 -- 172.21.15.7:0/3038443677 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d7415bc90 msgr2=0x7f1d740c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:42.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.109+0000 7f1d71e00640 1 --2- 172.21.15.7:0/3038443677 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d7415bc90 0x7f1d740c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.109+0000 7f1d71e00640 1 -- 172.21.15.7:0/3038443677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d740cd750 msgr2=0x7f1d740c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:42.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.109+0000 7f1d71e00640 1 --2- 172.21.15.7:0/3038443677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d740cd750 0x7f1d740c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.109+0000 7f1d71e00640 1 -- 172.21.15.7:0/3038443677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d740c7700 con 0x7f1d74073c70 2024-09-10T21:51:42.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.110+0000 7f1d71e00640 1 --2- 172.21.15.7:0/3038443677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d74073c70 0x7f1d7415a140 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f1d64000c00 tx=0x7f1d64033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:42.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.110+0000 7f1d6a000640 1 -- 172.21.15.7:0/3038443677 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d64005c90 con 0x7f1d74073c70 2024-09-10T21:51:42.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.110+0000 7f1d6a000640 1 -- 172.21.15.7:0/3038443677 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d64031e50 con 0x7f1d74073c70 2024-09-10T21:51:42.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.110+0000 7f1d6a000640 1 -- 172.21.15.7:0/3038443677 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d640332f0 con 0x7f1d74073c70 2024-09-10T21:51:42.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.110+0000 7f1d78c00640 1 -- 172.21.15.7:0/3038443677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d740cb7e0 con 0x7f1d74073c70 2024-09-10T21:51:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.110+0000 7f1d78c00640 1 -- 172.21.15.7:0/3038443677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d740c7970 con 0x7f1d74073c70 2024-09-10T21:51:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.111+0000 7f1d78c00640 1 -- 172.21.15.7:0/3038443677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d74073730 con 0x7f1d74073c70 2024-09-10T21:51:42.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.112+0000 7f1d6a000640 1 -- 172.21.15.7:0/3038443677 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1d6404b030 con 0x7f1d74073c70 2024-09-10T21:51:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.113+0000 7f1d6a000640 1 --2- 172.21.15.7:0/3038443677 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1d50077f70 0x7f1d5007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.113+0000 7f1d6a000640 1 -- 172.21.15.7:0/3038443677 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5709+0+0 (secure 0 0 0) 0x7f1d640bac80 con 0x7f1d74073c70 2024-09-10T21:51:42.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.114+0000 7f1d6be00640 1 --2- 172.21.15.7:0/3038443677 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1d50077f70 0x7f1d5007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:42.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.114+0000 7f1d6be00640 1 --2- 172.21.15.7:0/3038443677 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1d50077f70 0x7f1d5007a430 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1d740cb5b0 tx=0x7f1d5c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:42.118 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.117+0000 7f1d6a000640 1 -- 172.21.15.7:0/3038443677 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1d6403f030 con 0x7f1d74073c70 2024-09-10T21:51:42.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.312+0000 7f1d78c00640 1 -- 172.21.15.7:0/3038443677 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1d7415fe40 con 0x7f1d74073c70 2024-09-10T21:51:42.313 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.313+0000 7f1d6a000640 1 -- 172.21.15.7:0/3038443677 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v114) ==== 74+0+16651 (secure 0 0 0) 0x7f1d64083090 con 0x7f1d74073c70 2024-09-10T21:51:42.314 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:42.314 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":114,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:51:39.484414+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:51:35.736733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T21:51:36.873305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:51:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.315+0000 7f1d78c00640 1 -- 172.21.15.7:0/3038443677 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1d50077f70 msgr2=0x7f1d5007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.315+0000 7f1d78c00640 1 --2- 172.21.15.7:0/3038443677 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1d50077f70 0x7f1d5007a430 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1d740cb5b0 tx=0x7f1d5c005f90 comp rx=0 tx=0).stop 2024-09-10T21:51:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.315+0000 7f1d78c00640 1 -- 172.21.15.7:0/3038443677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d74073c70 msgr2=0x7f1d7415a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:42.316 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.315+0000 7f1d78c00640 1 --2- 172.21.15.7:0/3038443677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d74073c70 0x7f1d7415a140 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f1d64000c00 tx=0x7f1d64033c20 comp rx=0 tx=0).stop 2024-09-10T21:51:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.315+0000 7f1d78c00640 1 -- 172.21.15.7:0/3038443677 shutdown_connections 2024-09-10T21:51:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.315+0000 7f1d78c00640 1 --2- 172.21.15.7:0/3038443677 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1d50077f70 0x7f1d5007a430 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.315+0000 7f1d78c00640 1 --2- 172.21.15.7:0/3038443677 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1d740cd750 0x7f1d740c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.315+0000 7f1d78c00640 1 --2- 172.21.15.7:0/3038443677 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1d7415bc90 0x7f1d740c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.315+0000 7f1d78c00640 1 --2- 172.21.15.7:0/3038443677 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1d74073c70 0x7f1d7415a140 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.315+0000 7f1d78c00640 1 -- 172.21.15.7:0/3038443677 >> 172.21.15.7:0/3038443677 conn(0x7f1d740b16d0 msgr2=0x7f1d740ca1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.315+0000 7f1d78c00640 1 -- 172.21.15.7:0/3038443677 shutdown_connections 2024-09-10T21:51:42.317 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.315+0000 7f1d78c00640 1 -- 172.21.15.7:0/3038443677 wait complete. 2024-09-10T21:51:42.332 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:51:42.332 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:51:42.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.491+0000 7f9802000640 1 -- 172.21.15.7:0/3475464076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97fc1470b0 msgr2=0x7f97fc0ad8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:42.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.491+0000 7f9802000640 1 --2- 172.21.15.7:0/3475464076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97fc1470b0 0x7f97fc0ad8e0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f97fc070410 tx=0x7f97f402a820 comp rx=0 tx=0).stop 2024-09-10T21:51:42.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.492+0000 7f9802000640 1 -- 172.21.15.7:0/3475464076 shutdown_connections 2024-09-10T21:51:42.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.492+0000 7f9802000640 1 --2- 172.21.15.7:0/3475464076 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97fc1470b0 0x7f97fc0ad8e0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.492+0000 7f9802000640 1 --2- 172.21.15.7:0/3475464076 >> v2:172.21.15.133:3300/0 conn(0x7f97fc146c50 0x7f97fc0723b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.492+0000 7f9802000640 1 -- 172.21.15.7:0/3475464076 >> 172.21.15.7:0/3475464076 conn(0x7f97fc0b1d70 msgr2=0x7f97fc0cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:42.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.492+0000 7f9802000640 1 -- 172.21.15.7:0/3475464076 shutdown_connections 2024-09-10T21:51:42.493 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.492+0000 7f9802000640 1 -- 172.21.15.7:0/3475464076 wait complete. 2024-09-10T21:51:42.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.494+0000 7f9802000640 1 Processor -- start 2024-09-10T21:51:42.494 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.494+0000 7f9802000640 1 -- start start 2024-09-10T21:51:42.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.495+0000 7f9802000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97fc06f590 0x7f97fc1393a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:42.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.495+0000 7f9802000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97fc146c50 0x7f97fc140970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:42.496 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.495+0000 7f9802000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97fc1470b0 0x7f97fc140eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.495+0000 7f9802000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc0ade20 con 0x7f97fc146c50 2024-09-10T21:51:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.495+0000 7f9802000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc073210 con 0x7f97fc06f590 2024-09-10T21:51:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.495+0000 7f9802000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f97fc0adfa0 con 0x7f97fc1470b0 2024-09-10T21:51:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.495+0000 7f97f3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97fc146c50 0x7f97fc140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.495+0000 7f97f3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97fc146c50 0x7f97fc140970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44940/0 (socket says 172.21.15.7:44940) 2024-09-10T21:51:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.495+0000 7f97f3e00640 1 -- 172.21.15.7:0/3937536463 learned_addr learned my addr 172.21.15.7:0/3937536463 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.495+0000 7f9800c00640 1 --2- 172.21.15.7:0/3937536463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97fc06f590 0x7f97fc1393a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.495+0000 7f9801600640 1 --2- 172.21.15.7:0/3937536463 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97fc1470b0 0x7f97fc140eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.495+0000 7f97f3e00640 1 -- 172.21.15.7:0/3937536463 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97fc1470b0 msgr2=0x7f97fc140eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.495+0000 7f97f3e00640 1 --2- 172.21.15.7:0/3937536463 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97fc1470b0 0x7f97fc140eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.495+0000 7f97f3e00640 1 -- 172.21.15.7:0/3937536463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97fc06f590 msgr2=0x7f97fc1393a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.495+0000 7f97f3e00640 1 --2- 172.21.15.7:0/3937536463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97fc06f590 0x7f97fc1393a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.497 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.495+0000 7f97f3e00640 1 -- 172.21.15.7:0/3937536463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97fc1f8240 con 0x7f97fc146c50 2024-09-10T21:51:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.496+0000 7f97f3e00640 1 --2- 172.21.15.7:0/3937536463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97fc146c50 0x7f97fc140970 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f97e400c730 tx=0x7f97e400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.496+0000 7f97f9600640 1 -- 172.21.15.7:0/3937536463 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4017070 con 0x7f97fc146c50 2024-09-10T21:51:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.496+0000 7f97f9600640 1 -- 172.21.15.7:0/3937536463 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f97e4005180 con 0x7f97fc146c50 2024-09-10T21:51:42.498 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.496+0000 7f97f9600640 1 -- 172.21.15.7:0/3937536463 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e400a640 con 0x7f97fc146c50 2024-09-10T21:51:42.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.496+0000 7f9802000640 1 -- 172.21.15.7:0/3937536463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97fc1f8520 con 0x7f97fc146c50 2024-09-10T21:51:42.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.496+0000 7f9802000640 1 -- 172.21.15.7:0/3937536463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97fc1f8ae0 con 0x7f97fc146c50 2024-09-10T21:51:42.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.497+0000 7f9802000640 1 -- 172.21.15.7:0/3937536463 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97fc0ad8e0 con 0x7f97fc146c50 2024-09-10T21:51:42.499 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.499+0000 7f97f9600640 1 -- 172.21.15.7:0/3937536463 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f97e400a7e0 con 0x7f97fc146c50 2024-09-10T21:51:42.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.499+0000 7f97f9600640 1 --2- 172.21.15.7:0/3937536463 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f97d8077f70 0x7f97d807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:42.500 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.500+0000 7f97f9600640 1 -- 172.21.15.7:0/3937536463 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5709+0+0 (secure 0 0 0) 0x7f97e40992b0 con 0x7f97fc146c50 2024-09-10T21:51:42.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.500+0000 7f9800c00640 1 --2- 172.21.15.7:0/3937536463 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f97d8077f70 0x7f97d807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:42.501 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.501+0000 7f9800c00640 1 --2- 172.21.15.7:0/3937536463 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f97d8077f70 0x7f97d807a430 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f97ec005d00 tx=0x7f97ec00b410 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:42.504 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.503+0000 7f97f9600640 1 -- 172.21.15.7:0/3937536463 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f97fc147950 con 0x7f97fc146c50 2024-09-10T21:51:42.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.701+0000 7f9802000640 1 -- 172.21.15.7:0/3937536463 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f97fc161570 con 0x7f97d8077f70 2024-09-10T21:51:42.706 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.706+0000 7f97f9600640 1 -- 172.21.15.7:0/3937536463 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248776 (secure 0 0 0) 0x7f97fc161570 con 0x7f97d8077f70 2024-09-10T21:51:42.710 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:42.711 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:51:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.710+0000 7f9802000640 1 -- 172.21.15.7:0/3937536463 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f97d8077f70 msgr2=0x7f97d807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.710+0000 7f9802000640 1 --2- 172.21.15.7:0/3937536463 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f97d8077f70 0x7f97d807a430 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f97ec005d00 tx=0x7f97ec00b410 comp rx=0 tx=0).stop 2024-09-10T21:51:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.710+0000 7f9802000640 1 -- 172.21.15.7:0/3937536463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97fc146c50 msgr2=0x7f97fc140970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.710+0000 7f9802000640 1 --2- 172.21.15.7:0/3937536463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97fc146c50 0x7f97fc140970 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f97e400c730 tx=0x7f97e400cbf0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.710+0000 7f9802000640 1 -- 172.21.15.7:0/3937536463 shutdown_connections 2024-09-10T21:51:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.710+0000 7f9802000640 1 --2- 172.21.15.7:0/3937536463 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f97d8077f70 0x7f97d807a430 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.710+0000 7f9802000640 1 --2- 172.21.15.7:0/3937536463 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f97fc1470b0 0x7f97fc140eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.710+0000 7f9802000640 1 --2- 172.21.15.7:0/3937536463 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f97fc146c50 0x7f97fc140970 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.713 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.710+0000 7f9802000640 1 --2- 172.21.15.7:0/3937536463 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f97fc06f590 0x7f97fc1393a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.710+0000 7f9802000640 1 -- 172.21.15.7:0/3937536463 >> 172.21.15.7:0/3937536463 conn(0x7f97fc0b1d70 msgr2=0x7f97fc0b2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:42.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.710+0000 7f9802000640 1 -- 172.21.15.7:0/3937536463 shutdown_connections 2024-09-10T21:51:42.714 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.710+0000 7f9802000640 1 -- 172.21.15.7:0/3937536463 wait complete. 2024-09-10T21:51:42.726 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-09-10T21:51:42.008379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":274004,"kb_used_data":12000,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754700716,"statfs":{"total":773094113280,"available":772813533184,"internally_reserved":0,"allocated":12288000,"data_stored":5960164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.550209"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.485770+0000","last_change":"2024-09-10T21:51:37.485770+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.485770+0000","last_undegraded":"2024-09-10T21:51:37.485770+0000","last_fullsized":"2024-09-10T21:51:37.485770+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.467394+0000","last_change":"2024-09-10T21:51:37.467394+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.467394+0000","last_undegraded":"2024-09-10T21:51:37.467394+0000","last_fullsized":"2024-09-10T21:51:37.467394+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966452+0000","last_change":"2024-09-10T21:51:38.505264+0000","last_active":"2024-09-10T21:51:39.966452+0000","last_peered":"2024-09-10T21:51:39.966452+0000","last_clean":"2024-09-10T21:51:39.966452+0000","last_became_active":"2024-09-10T21:51:38.504121+0000","last_became_peered":"2024-09-10T21:51:38.504121+0000","last_unstale":"2024-09-10T21:51:39.966452+0000","last_undegraded":"2024-09-10T21:51:39.966452+0000","last_fullsized":"2024-09-10T21:51:39.966452+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:32.797143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.489196+0000","last_change":"2024-09-10T21:51:37.489196+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.489196+0000","last_undegraded":"2024-09-10T21:51:37.489196+0000","last_fullsized":"2024-09-10T21:51:37.489196+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.476187+0000","last_change":"2024-09-10T21:51:37.476187+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.476187+0000","last_undegraded":"2024-09-10T21:51:37.476187+0000","last_fullsized":"2024-09-10T21:51:37.476187+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.469812+0000","last_change":"2024-09-10T21:51:37.469812+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.469812+0000","last_undegraded":"2024-09-10T21:51:37.469812+0000","last_fullsized":"2024-09-10T21:51:37.469812+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.465592+0000","last_change":"2024-09-10T21:51:37.465592+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.465592+0000","last_undegraded":"2024-09-10T21:51:37.465592+0000","last_fullsized":"2024-09-10T21:51:37.465592+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966436+0000","last_change":"2024-09-10T21:51:38.498466+0000","last_active":"2024-09-10T21:51:39.966436+0000","last_peered":"2024-09-10T21:51:39.966436+0000","last_clean":"2024-09-10T21:51:39.966436+0000","last_became_active":"2024-09-10T21:51:38.498041+0000","last_became_peered":"2024-09-10T21:51:38.498041+0000","last_unstale":"2024-09-10T21:51:39.966436+0000","last_undegraded":"2024-09-10T21:51:39.966436+0000","last_fullsized":"2024-09-10T21:51:39.966436+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:19.954387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.482918+0000","last_change":"2024-09-10T21:51:37.482918+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.482918+0000","last_undegraded":"2024-09-10T21:51:37.482918+0000","last_fullsized":"2024-09-10T21:51:37.482918+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.482427+0000","last_change":"2024-09-10T21:51:37.482427+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.482427+0000","last_undegraded":"2024-09-10T21:51:37.482427+0000","last_fullsized":"2024-09-10T21:51:37.482427+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.468175+0000","last_change":"2024-09-10T21:51:37.468175+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.468175+0000","last_undegraded":"2024-09-10T21:51:37.468175+0000","last_fullsized":"2024-09-10T21:51:37.468175+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.485246+0000","last_change":"2024-09-10T21:51:37.485246+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.485246+0000","last_undegraded":"2024-09-10T21:51:37.485246+0000","last_fullsized":"2024-09-10T21:51:37.485246+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966404+0000","last_change":"2024-09-10T21:51:38.503338+0000","last_active":"2024-09-10T21:51:39.966404+0000","last_peered":"2024-09-10T21:51:39.966404+0000","last_clean":"2024-09-10T21:51:39.966404+0000","last_became_active":"2024-09-10T21:51:38.501614+0000","last_became_peered":"2024-09-10T21:51:38.501614+0000","last_unstale":"2024-09-10T21:51:39.966404+0000","last_undegraded":"2024-09-10T21:51:39.966404+0000","last_fullsized":"2024-09-10T21:51:39.966404+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:10.529060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966266+0000","last_change":"2024-09-10T21:51:38.498967+0000","last_active":"2024-09-10T21:51:39.966266+0000","last_peered":"2024-09-10T21:51:39.966266+0000","last_clean":"2024-09-10T21:51:39.966266+0000","last_became_active":"2024-09-10T21:51:38.498641+0000","last_became_peered":"2024-09-10T21:51:38.498641+0000","last_unstale":"2024-09-10T21:51:39.966266+0000","last_undegraded":"2024-09-10T21:51:39.966266+0000","last_fullsized":"2024-09-10T21:51:39.966266+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:41.424015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.469393+0000","last_change":"2024-09-10T21:51:37.469393+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.469393+0000","last_undegraded":"2024-09-10T21:51:37.469393+0000","last_fullsized":"2024-09-10T21:51:37.469393+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.481943+0000","last_change":"2024-09-10T21:51:37.481943+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.481943+0000","last_undegraded":"2024-09-10T21:51:37.481943+0000","last_fullsized":"2024-09-10T21:51:37.481943+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.468661+0000","last_change":"2024-09-10T21:51:37.468661+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.468661+0000","last_undegraded":"2024-09-10T21:51:37.468661+0000","last_fullsized":"2024-09-10T21:51:37.468661+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.479101+0000","last_change":"2024-09-10T21:51:37.479101+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.479101+0000","last_undegraded":"2024-09-10T21:51:37.479101+0000","last_fullsized":"2024-09-10T21:51:37.479101+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966459+0000","last_change":"2024-09-10T21:51:38.506319+0000","last_active":"2024-09-10T21:51:39.966459+0000","last_peered":"2024-09-10T21:51:39.966459+0000","last_clean":"2024-09-10T21:51:39.966459+0000","last_became_active":"2024-09-10T21:51:38.506041+0000","last_became_peered":"2024-09-10T21:51:38.506041+0000","last_unstale":"2024-09-10T21:51:39.966459+0000","last_undegraded":"2024-09-10T21:51:39.966459+0000","last_fullsized":"2024-09-10T21:51:39.966459+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:33.484489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.467188+0000","last_change":"2024-09-10T21:51:37.467188+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.467188+0000","last_undegraded":"2024-09-10T21:51:37.467188+0000","last_fullsized":"2024-09-10T21:51:37.467188+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.469235+0000","last_change":"2024-09-10T21:51:37.469235+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.469235+0000","last_undegraded":"2024-09-10T21:51:37.469235+0000","last_fullsized":"2024-09-10T21:51:37.469235+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965676+0000","last_change":"2024-09-10T21:51:38.495309+0000","last_active":"2024-09-10T21:51:39.965676+0000","last_peered":"2024-09-10T21:51:39.965676+0000","last_clean":"2024-09-10T21:51:39.965676+0000","last_became_active":"2024-09-10T21:51:38.494998+0000","last_became_peered":"2024-09-10T21:51:38.494998+0000","last_unstale":"2024-09-10T21:51:39.965676+0000","last_undegraded":"2024-09-10T21:51:39.965676+0000","last_fullsized":"2024-09-10T21:51:39.965676+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:41.427352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.466079+0000","last_change":"2024-09-10T21:51:37.466079+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.466079+0000","last_undegraded":"2024-09-10T21:51:37.466079+0000","last_fullsized":"2024-09-10T21:51:37.466079+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965229+0000","last_change":"2024-09-10T21:51:38.505466+0000","last_active":"2024-09-10T21:51:39.965229+0000","last_peered":"2024-09-10T21:51:39.965229+0000","last_clean":"2024-09-10T21:51:39.965229+0000","last_became_active":"2024-09-10T21:51:38.505053+0000","last_became_peered":"2024-09-10T21:51:38.505053+0000","last_unstale":"2024-09-10T21:51:39.965229+0000","last_undegraded":"2024-09-10T21:51:39.965229+0000","last_fullsized":"2024-09-10T21:51:39.965229+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:36:29.403651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.467357+0000","last_change":"2024-09-10T21:51:37.467357+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.467357+0000","last_undegraded":"2024-09-10T21:51:37.467357+0000","last_fullsized":"2024-09-10T21:51:37.467357+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.478229+0000","last_change":"2024-09-10T21:51:37.478229+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.478229+0000","last_undegraded":"2024-09-10T21:51:37.478229+0000","last_fullsized":"2024-09-10T21:51:37.478229+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.473593+0000","last_change":"2024-09-10T21:51:37.473593+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.473593+0000","last_undegraded":"2024-09-10T21:51:37.473593+0000","last_fullsized":"2024-09-10T21:51:37.473593+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965771+0000","last_change":"2024-09-10T21:51:38.496708+0000","last_active":"2024-09-10T21:51:39.965771+0000","last_peered":"2024-09-10T21:51:39.965771+0000","last_clean":"2024-09-10T21:51:39.965771+0000","last_became_active":"2024-09-10T21:51:38.496402+0000","last_became_peered":"2024-09-10T21:51:38.496402+0000","last_unstale":"2024-09-10T21:51:39.965771+0000","last_undegraded":"2024-09-10T21:51:39.965771+0000","last_fullsized":"2024-09-10T21:51:39.965771+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:40.305224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.474613+0000","last_change":"2024-09-10T21:51:37.474613+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.474613+0000","last_undegraded":"2024-09-10T21:51:37.474613+0000","last_fullsized":"2024-09-10T21:51:37.474613+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.463487+0000","last_change":"2024-09-10T21:51:37.463487+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.463487+0000","last_undegraded":"2024-09-10T21:51:37.463487+0000","last_fullsized":"2024-09-10T21:51:37.463487+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.478186+0000","last_change":"2024-09-10T21:51:37.478186+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.478186+0000","last_undegraded":"2024-09-10T21:51:37.478186+0000","last_fullsized":"2024-09-10T21:51:37.478186+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.468954+0000","last_change":"2024-09-10T21:51:37.468954+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.468954+0000","last_undegraded":"2024-09-10T21:51:37.468954+0000","last_fullsized":"2024-09-10T21:51:37.468954+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.477280+0000","last_change":"2024-09-10T21:51:37.477280+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.477280+0000","last_undegraded":"2024-09-10T21:51:37.477280+0000","last_fullsized":"2024-09-10T21:51:37.477280+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.474881+0000","last_change":"2024-09-10T21:51:37.472999+0000","last_active":"2024-09-10T21:51:37.474881+0000","last_peered":"2024-09-10T21:51:37.474881+0000","last_clean":"2024-09-10T21:51:37.474881+0000","last_became_active":"2024-09-10T21:51:37.472471+0000","last_became_peered":"2024-09-10T21:51:37.472471+0000","last_unstale":"2024-09-10T21:51:37.474881+0000","last_undegraded":"2024-09-10T21:51:37.474881+0000","last_fullsized":"2024-09-10T21:51:37.474881+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:30.555990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.480358+0000","last_change":"2024-09-10T21:51:37.480358+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.480358+0000","last_undegraded":"2024-09-10T21:51:37.480358+0000","last_fullsized":"2024-09-10T21:51:37.480358+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966096+0000","last_change":"2024-09-10T21:51:37.481369+0000","last_active":"2024-09-10T21:51:39.966096+0000","last_peered":"2024-09-10T21:51:39.966096+0000","last_clean":"2024-09-10T21:51:39.966096+0000","last_became_active":"2024-09-10T21:51:37.481061+0000","last_became_peered":"2024-09-10T21:51:37.481061+0000","last_unstale":"2024-09-10T21:51:39.966096+0000","last_undegraded":"2024-09-10T21:51:39.966096+0000","last_fullsized":"2024-09-10T21:51:39.966096+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:48.767875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.473226+0000","last_change":"2024-09-10T21:51:37.473226+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.473226+0000","last_undegraded":"2024-09-10T21:51:37.473226+0000","last_fullsized":"2024-09-10T21:51:37.473226+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.497767+0000","last_change":"2024-09-10T21:51:37.495401+0000","last_active":"2024-09-10T21:51:37.497767+0000","last_peered":"2024-09-10T21:51:37.497767+0000","last_clean":"2024-09-10T21:51:37.497767+0000","last_became_active":"2024-09-10T21:51:37.495129+0000","last_became_peered":"2024-09-10T21:51:37.495129+0000","last_unstale":"2024-09-10T21:51:37.497767+0000","last_undegraded":"2024-09-10T21:51:37.497767+0000","last_fullsized":"2024-09-10T21:51:37.497767+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:58.176252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966641+0000","last_change":"2024-09-10T21:51:38.507168+0000","last_active":"2024-09-10T21:51:39.966641+0000","last_peered":"2024-09-10T21:51:39.966641+0000","last_clean":"2024-09-10T21:51:39.966641+0000","last_became_active":"2024-09-10T21:51:38.506850+0000","last_became_peered":"2024-09-10T21:51:38.506850+0000","last_unstale":"2024-09-10T21:51:39.966641+0000","last_undegraded":"2024-09-10T21:51:39.966641+0000","last_fullsized":"2024-09-10T21:51:39.966641+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:40.305224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.497488+0000","last_change":"2024-09-10T21:51:37.495072+0000","last_active":"2024-09-10T21:51:37.497488+0000","last_peered":"2024-09-10T21:51:37.497488+0000","last_clean":"2024-09-10T21:51:37.497488+0000","last_became_active":"2024-09-10T21:51:37.494778+0000","last_became_peered":"2024-09-10T21:51:37.494778+0000","last_unstale":"2024-09-10T21:51:37.497488+0000","last_undegraded":"2024-09-10T21:51:37.497488+0000","last_fullsized":"2024-09-10T21:51:37.497488+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:18.574835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.476588+0000","last_change":"2024-09-10T21:51:37.476588+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.476588+0000","last_undegraded":"2024-09-10T21:51:37.476588+0000","last_fullsized":"2024-09-10T21:51:37.476588+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.476242+0000","last_change":"2024-09-10T21:51:37.474801+0000","last_active":"2024-09-10T21:51:37.476242+0000","last_peered":"2024-09-10T21:51:37.476242+0000","last_clean":"2024-09-10T21:51:37.476242+0000","last_became_active":"2024-09-10T21:51:37.474455+0000","last_became_peered":"2024-09-10T21:51:37.474455+0000","last_unstale":"2024-09-10T21:51:37.476242+0000","last_undegraded":"2024-09-10T21:51:37.476242+0000","last_fullsized":"2024-09-10T21:51:37.476242+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:00.575937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.474186+0000","last_change":"2024-09-10T21:51:37.474186+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.474186+0000","last_undegraded":"2024-09-10T21:51:37.474186+0000","last_fullsized":"2024-09-10T21:51:37.474186+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965968+0000","last_change":"2024-09-10T21:51:37.481177+0000","last_active":"2024-09-10T21:51:39.965968+0000","last_peered":"2024-09-10T21:51:39.965968+0000","last_clean":"2024-09-10T21:51:39.965968+0000","last_became_active":"2024-09-10T21:51:37.480878+0000","last_became_peered":"2024-09-10T21:51:37.480878+0000","last_unstale":"2024-09-10T21:51:39.965968+0000","last_undegraded":"2024-09-10T21:51:39.965968+0000","last_fullsized":"2024-09-10T21:51:39.965968+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:39.644340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.477970+0000","last_change":"2024-09-10T21:51:37.477970+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.477970+0000","last_undegraded":"2024-09-10T21:51:37.477970+0000","last_fullsized":"2024-09-10T21:51:37.477970+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.487128+0000","last_change":"2024-09-10T21:51:37.486466+0000","last_active":"2024-09-10T21:51:37.487128+0000","last_peered":"2024-09-10T21:51:37.487128+0000","last_clean":"2024-09-10T21:51:37.487128+0000","last_became_active":"2024-09-10T21:51:37.486160+0000","last_became_peered":"2024-09-10T21:51:37.486160+0000","last_unstale":"2024-09-10T21:51:37.487128+0000","last_undegraded":"2024-09-10T21:51:37.487128+0000","last_fullsized":"2024-09-10T21:51:37.487128+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:38.793976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.463935+0000","last_change":"2024-09-10T21:51:37.463935+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.463935+0000","last_undegraded":"2024-09-10T21:51:37.463935+0000","last_fullsized":"2024-09-10T21:51:37.463935+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.491594+0000","last_change":"2024-09-10T21:51:37.489582+0000","last_active":"2024-09-10T21:51:37.491594+0000","last_peered":"2024-09-10T21:51:37.491594+0000","last_clean":"2024-09-10T21:51:37.491594+0000","last_became_active":"2024-09-10T21:51:37.487889+0000","last_became_peered":"2024-09-10T21:51:37.487889+0000","last_unstale":"2024-09-10T21:51:37.491594+0000","last_undegraded":"2024-09-10T21:51:37.491594+0000","last_fullsized":"2024-09-10T21:51:37.491594+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:53.102794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.463971+0000","last_change":"2024-09-10T21:51:37.463971+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.463971+0000","last_undegraded":"2024-09-10T21:51:37.463971+0000","last_fullsized":"2024-09-10T21:51:37.463971+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.496341+0000","last_change":"2024-09-10T21:51:37.493056+0000","last_active":"2024-09-10T21:51:37.496341+0000","last_peered":"2024-09-10T21:51:37.496341+0000","last_clean":"2024-09-10T21:51:37.496341+0000","last_became_active":"2024-09-10T21:51:37.492756+0000","last_became_peered":"2024-09-10T21:51:37.492756+0000","last_unstale":"2024-09-10T21:51:37.496341+0000","last_undegraded":"2024-09-10T21:51:37.496341+0000","last_fullsized":"2024-09-10T21:51:37.496341+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:33.347129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.473160+0000","last_change":"2024-09-10T21:51:37.473160+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.473160+0000","last_undegraded":"2024-09-10T21:51:37.473160+0000","last_fullsized":"2024-09-10T21:51:37.473160+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966160+0000","last_change":"2024-09-10T21:51:37.487117+0000","last_active":"2024-09-10T21:51:39.966160+0000","last_peered":"2024-09-10T21:51:39.966160+0000","last_clean":"2024-09-10T21:51:39.966160+0000","last_became_active":"2024-09-10T21:51:37.484685+0000","last_became_peered":"2024-09-10T21:51:37.484685+0000","last_unstale":"2024-09-10T21:51:39.966160+0000","last_undegraded":"2024-09-10T21:51:39.966160+0000","last_fullsized":"2024-09-10T21:51:39.966160+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:13.540834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965991+0000","last_change":"2024-09-10T21:51:38.500232+0000","last_active":"2024-09-10T21:51:39.965991+0000","last_peered":"2024-09-10T21:51:39.965991+0000","last_clean":"2024-09-10T21:51:39.965991+0000","last_became_active":"2024-09-10T21:51:38.499959+0000","last_became_peered":"2024-09-10T21:51:38.499959+0000","last_unstale":"2024-09-10T21:51:39.965991+0000","last_undegraded":"2024-09-10T21:51:39.965991+0000","last_fullsized":"2024-09-10T21:51:39.965991+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:35.171265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.484868+0000","last_change":"2024-09-10T21:51:37.483939+0000","last_active":"2024-09-10T21:51:37.484868+0000","last_peered":"2024-09-10T21:51:37.484868+0000","last_clean":"2024-09-10T21:51:37.484868+0000","last_became_active":"2024-09-10T21:51:37.483023+0000","last_became_peered":"2024-09-10T21:51:37.483023+0000","last_unstale":"2024-09-10T21:51:37.484868+0000","last_undegraded":"2024-09-10T21:51:37.484868+0000","last_fullsized":"2024-09-10T21:51:37.484868+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:42.259904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.464945+0000","last_change":"2024-09-10T21:51:37.464945+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.464945+0000","last_undegraded":"2024-09-10T21:51:37.464945+0000","last_fullsized":"2024-09-10T21:51:37.464945+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.492264+0000","last_change":"2024-09-10T21:51:37.490837+0000","last_active":"2024-09-10T21:51:37.492264+0000","last_peered":"2024-09-10T21:51:37.492264+0000","last_clean":"2024-09-10T21:51:37.492264+0000","last_became_active":"2024-09-10T21:51:37.489661+0000","last_became_peered":"2024-09-10T21:51:37.489661+0000","last_unstale":"2024-09-10T21:51:37.492264+0000","last_undegraded":"2024-09-10T21:51:37.492264+0000","last_fullsized":"2024-09-10T21:51:37.492264+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:58.486726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966089+0000","last_change":"2024-09-10T21:51:38.499244+0000","last_active":"2024-09-10T21:51:39.966089+0000","last_peered":"2024-09-10T21:51:39.966089+0000","last_clean":"2024-09-10T21:51:39.966089+0000","last_became_active":"2024-09-10T21:51:38.498965+0000","last_became_peered":"2024-09-10T21:51:38.498965+0000","last_unstale":"2024-09-10T21:51:39.966089+0000","last_undegraded":"2024-09-10T21:51:39.966089+0000","last_fullsized":"2024-09-10T21:51:39.966089+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:12.629720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.488507+0000","last_change":"2024-09-10T21:51:37.480818+0000","last_active":"2024-09-10T21:51:37.488507+0000","last_peered":"2024-09-10T21:51:37.488507+0000","last_clean":"2024-09-10T21:51:37.488507+0000","last_became_active":"2024-09-10T21:51:37.480562+0000","last_became_peered":"2024-09-10T21:51:37.480562+0000","last_unstale":"2024-09-10T21:51:37.488507+0000","last_undegraded":"2024-09-10T21:51:37.488507+0000","last_fullsized":"2024-09-10T21:51:37.488507+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:05.569050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965893+0000","last_change":"2024-09-10T21:51:38.504740+0000","last_active":"2024-09-10T21:51:39.965893+0000","last_peered":"2024-09-10T21:51:39.965893+0000","last_clean":"2024-09-10T21:51:39.965893+0000","last_became_active":"2024-09-10T21:51:38.504214+0000","last_became_peered":"2024-09-10T21:51:38.504214+0000","last_unstale":"2024-09-10T21:51:39.965893+0000","last_undegraded":"2024-09-10T21:51:39.965893+0000","last_fullsized":"2024-09-10T21:51:39.965893+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:47.805058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.492594+0000","last_change":"2024-09-10T21:51:37.491755+0000","last_active":"2024-09-10T21:51:37.492594+0000","last_peered":"2024-09-10T21:51:37.492594+0000","last_clean":"2024-09-10T21:51:37.492594+0000","last_became_active":"2024-09-10T21:51:37.491472+0000","last_became_peered":"2024-09-10T21:51:37.491472+0000","last_unstale":"2024-09-10T21:51:37.492594+0000","last_undegraded":"2024-09-10T21:51:37.492594+0000","last_fullsized":"2024-09-10T21:51:37.492594+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:09.126767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965395+0000","last_change":"2024-09-10T21:51:38.500521+0000","last_active":"2024-09-10T21:51:39.965395+0000","last_peered":"2024-09-10T21:51:39.965395+0000","last_clean":"2024-09-10T21:51:39.965395+0000","last_became_active":"2024-09-10T21:51:38.500234+0000","last_became_peered":"2024-09-10T21:51:38.500234+0000","last_unstale":"2024-09-10T21:51:39.965395+0000","last_undegraded":"2024-09-10T21:51:39.965395+0000","last_fullsized":"2024-09-10T21:51:39.965395+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:28.278479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.496082+0000","last_change":"2024-09-10T21:51:37.492464+0000","last_active":"2024-09-10T21:51:37.496082+0000","last_peered":"2024-09-10T21:51:37.496082+0000","last_clean":"2024-09-10T21:51:37.496082+0000","last_became_active":"2024-09-10T21:51:37.492162+0000","last_became_peered":"2024-09-10T21:51:37.492162+0000","last_unstale":"2024-09-10T21:51:37.496082+0000","last_undegraded":"2024-09-10T21:51:37.496082+0000","last_fullsized":"2024-09-10T21:51:37.496082+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:43.699147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.475624+0000","last_change":"2024-09-10T21:51:37.475624+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.475624+0000","last_undegraded":"2024-09-10T21:51:37.475624+0000","last_fullsized":"2024-09-10T21:51:37.475624+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.491463+0000","last_change":"2024-09-10T21:51:37.468073+0000","last_active":"2024-09-10T21:51:37.491463+0000","last_peered":"2024-09-10T21:51:37.491463+0000","last_clean":"2024-09-10T21:51:37.491463+0000","last_became_active":"2024-09-10T21:51:37.467716+0000","last_became_peered":"2024-09-10T21:51:37.467716+0000","last_unstale":"2024-09-10T21:51:37.491463+0000","last_undegraded":"2024-09-10T21:51:37.491463+0000","last_fullsized":"2024-09-10T21:51:37.491463+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:36.082631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.473936+0000","last_change":"2024-09-10T21:51:37.473936+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.473936+0000","last_undegraded":"2024-09-10T21:51:37.473936+0000","last_fullsized":"2024-09-10T21:51:37.473936+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.496000+0000","last_change":"2024-09-10T21:51:37.485673+0000","last_active":"2024-09-10T21:51:37.496000+0000","last_peered":"2024-09-10T21:51:37.496000+0000","last_clean":"2024-09-10T21:51:37.496000+0000","last_became_active":"2024-09-10T21:51:37.485132+0000","last_became_peered":"2024-09-10T21:51:37.485132+0000","last_unstale":"2024-09-10T21:51:37.496000+0000","last_undegraded":"2024-09-10T21:51:37.496000+0000","last_fullsized":"2024-09-10T21:51:37.496000+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:30:53.853582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966440+0000","last_change":"2024-09-10T21:51:38.505872+0000","last_active":"2024-09-10T21:51:39.966440+0000","last_peered":"2024-09-10T21:51:39.966440+0000","last_clean":"2024-09-10T21:51:39.966440+0000","last_became_active":"2024-09-10T21:51:38.505488+0000","last_became_peered":"2024-09-10T21:51:38.505488+0000","last_unstale":"2024-09-10T21:51:39.966440+0000","last_undegraded":"2024-09-10T21:51:39.966440+0000","last_fullsized":"2024-09-10T21:51:39.966440+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:29.689596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.485861+0000","last_change":"2024-09-10T21:51:37.485228+0000","last_active":"2024-09-10T21:51:37.485861+0000","last_peered":"2024-09-10T21:51:37.485861+0000","last_clean":"2024-09-10T21:51:37.485861+0000","last_became_active":"2024-09-10T21:51:37.484980+0000","last_became_peered":"2024-09-10T21:51:37.484980+0000","last_unstale":"2024-09-10T21:51:37.485861+0000","last_undegraded":"2024-09-10T21:51:37.485861+0000","last_fullsized":"2024-09-10T21:51:37.485861+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:24.448659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966710+0000","last_change":"2024-09-10T21:51:38.504879+0000","last_active":"2024-09-10T21:51:39.966710+0000","last_peered":"2024-09-10T21:51:39.966710+0000","last_clean":"2024-09-10T21:51:39.966710+0000","last_became_active":"2024-09-10T21:51:38.504560+0000","last_became_peered":"2024-09-10T21:51:38.504560+0000","last_unstale":"2024-09-10T21:51:39.966710+0000","last_undegraded":"2024-09-10T21:51:39.966710+0000","last_fullsized":"2024-09-10T21:51:39.966710+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:47.805058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.473267+0000","last_change":"2024-09-10T21:51:37.469638+0000","last_active":"2024-09-10T21:51:37.473267+0000","last_peered":"2024-09-10T21:51:37.473267+0000","last_clean":"2024-09-10T21:51:37.473267+0000","last_became_active":"2024-09-10T21:51:37.467556+0000","last_became_peered":"2024-09-10T21:51:37.467556+0000","last_unstale":"2024-09-10T21:51:37.473267+0000","last_undegraded":"2024-09-10T21:51:37.473267+0000","last_fullsized":"2024-09-10T21:51:37.473267+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:11.474396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966529+0000","last_change":"2024-09-10T21:51:38.504149+0000","last_active":"2024-09-10T21:51:39.966529+0000","last_peered":"2024-09-10T21:51:39.966529+0000","last_clean":"2024-09-10T21:51:39.966529+0000","last_became_active":"2024-09-10T21:51:38.503873+0000","last_became_peered":"2024-09-10T21:51:38.503873+0000","last_unstale":"2024-09-10T21:51:39.966529+0000","last_undegraded":"2024-09-10T21:51:39.966529+0000","last_fullsized":"2024-09-10T21:51:39.966529+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:38.756786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.492556+0000","last_change":"2024-09-10T21:51:37.488302+0000","last_active":"2024-09-10T21:51:37.492556+0000","last_peered":"2024-09-10T21:51:37.492556+0000","last_clean":"2024-09-10T21:51:37.492556+0000","last_became_active":"2024-09-10T21:51:37.488054+0000","last_became_peered":"2024-09-10T21:51:37.488054+0000","last_unstale":"2024-09-10T21:51:37.492556+0000","last_undegraded":"2024-09-10T21:51:37.492556+0000","last_fullsized":"2024-09-10T21:51:37.492556+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:13.280288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.471734+0000","last_change":"2024-09-10T21:51:37.471734+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.471734+0000","last_undegraded":"2024-09-10T21:51:37.471734+0000","last_fullsized":"2024-09-10T21:51:37.471734+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.497783+0000","last_change":"2024-09-10T21:51:37.495655+0000","last_active":"2024-09-10T21:51:37.497783+0000","last_peered":"2024-09-10T21:51:37.497783+0000","last_clean":"2024-09-10T21:51:37.497783+0000","last_became_active":"2024-09-10T21:51:37.495383+0000","last_became_peered":"2024-09-10T21:51:37.495383+0000","last_unstale":"2024-09-10T21:51:37.497783+0000","last_undegraded":"2024-09-10T21:51:37.497783+0000","last_fullsized":"2024-09-10T21:51:37.497783+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:38.126366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966363+0000","last_change":"2024-09-10T21:51:38.498184+0000","last_active":"2024-09-10T21:51:39.966363+0000","last_peered":"2024-09-10T21:51:39.966363+0000","last_clean":"2024-09-10T21:51:39.966363+0000","last_became_active":"2024-09-10T21:51:38.497840+0000","last_became_peered":"2024-09-10T21:51:38.497840+0000","last_unstale":"2024-09-10T21:51:39.966363+0000","last_undegraded":"2024-09-10T21:51:39.966363+0000","last_fullsized":"2024-09-10T21:51:39.966363+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:50.132177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.495778+0000","last_change":"2024-09-10T21:51:37.493629+0000","last_active":"2024-09-10T21:51:37.495778+0000","last_peered":"2024-09-10T21:51:37.495778+0000","last_clean":"2024-09-10T21:51:37.495778+0000","last_became_active":"2024-09-10T21:51:37.493347+0000","last_became_peered":"2024-09-10T21:51:37.493347+0000","last_unstale":"2024-09-10T21:51:37.495778+0000","last_undegraded":"2024-09-10T21:51:37.495778+0000","last_fullsized":"2024-09-10T21:51:37.495778+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:54.092525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965539+0000","last_change":"2024-09-10T21:51:38.497499+0000","last_active":"2024-09-10T21:51:39.965539+0000","last_peered":"2024-09-10T21:51:39.965539+0000","last_clean":"2024-09-10T21:51:39.965539+0000","last_became_active":"2024-09-10T21:51:38.494172+0000","last_became_peered":"2024-09-10T21:51:38.494172+0000","last_unstale":"2024-09-10T21:51:39.965539+0000","last_undegraded":"2024-09-10T21:51:39.965539+0000","last_fullsized":"2024-09-10T21:51:39.965539+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:38.756786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.491642+0000","last_change":"2024-09-10T21:51:37.490268+0000","last_active":"2024-09-10T21:51:37.491642+0000","last_peered":"2024-09-10T21:51:37.491642+0000","last_clean":"2024-09-10T21:51:37.491642+0000","last_became_active":"2024-09-10T21:51:37.488772+0000","last_became_peered":"2024-09-10T21:51:37.488772+0000","last_unstale":"2024-09-10T21:51:37.491642+0000","last_undegraded":"2024-09-10T21:51:37.491642+0000","last_fullsized":"2024-09-10T21:51:37.491642+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:18.953120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.470749+0000","last_change":"2024-09-10T21:51:37.470749+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.470749+0000","last_undegraded":"2024-09-10T21:51:37.470749+0000","last_fullsized":"2024-09-10T21:51:37.470749+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966234+0000","last_change":"2024-09-10T21:51:37.493424+0000","last_active":"2024-09-10T21:51:39.966234+0000","last_peered":"2024-09-10T21:51:39.966234+0000","last_clean":"2024-09-10T21:51:39.966234+0000","last_became_active":"2024-09-10T21:51:37.492964+0000","last_became_peered":"2024-09-10T21:51:37.492964+0000","last_unstale":"2024-09-10T21:51:39.966234+0000","last_undegraded":"2024-09-10T21:51:39.966234+0000","last_fullsized":"2024-09-10T21:51:39.966234+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:56.310705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.472197+0000","last_change":"2024-09-10T21:51:37.472197+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.472197+0000","last_undegraded":"2024-09-10T21:51:37.472197+0000","last_fullsized":"2024-09-10T21:51:37.472197+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.482551+0000","last_change":"2024-09-10T21:51:37.481469+0000","last_active":"2024-09-10T21:51:37.482551+0000","last_peered":"2024-09-10T21:51:37.482551+0000","last_clean":"2024-09-10T21:51:37.482551+0000","last_became_active":"2024-09-10T21:51:37.481196+0000","last_became_peered":"2024-09-10T21:51:37.481196+0000","last_unstale":"2024-09-10T21:51:37.482551+0000","last_undegraded":"2024-09-10T21:51:37.482551+0000","last_fullsized":"2024-09-10T21:51:37.482551+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:10.385869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.471647+0000","last_change":"2024-09-10T21:51:37.471647+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.471647+0000","last_undegraded":"2024-09-10T21:51:37.471647+0000","last_fullsized":"2024-09-10T21:51:37.471647+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.492284+0000","last_change":"2024-09-10T21:51:37.490280+0000","last_active":"2024-09-10T21:51:37.492284+0000","last_peered":"2024-09-10T21:51:37.492284+0000","last_clean":"2024-09-10T21:51:37.492284+0000","last_became_active":"2024-09-10T21:51:37.489979+0000","last_became_peered":"2024-09-10T21:51:37.489979+0000","last_unstale":"2024-09-10T21:51:37.492284+0000","last_undegraded":"2024-09-10T21:51:37.492284+0000","last_fullsized":"2024-09-10T21:51:37.492284+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:09:51.282814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.475986+0000","last_change":"2024-09-10T21:51:37.475986+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.475986+0000","last_undegraded":"2024-09-10T21:51:37.475986+0000","last_fullsized":"2024-09-10T21:51:37.475986+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966131+0000","last_change":"2024-09-10T21:51:37.496547+0000","last_active":"2024-09-10T21:51:39.966131+0000","last_peered":"2024-09-10T21:51:39.966131+0000","last_clean":"2024-09-10T21:51:39.966131+0000","last_became_active":"2024-09-10T21:51:37.495965+0000","last_became_peered":"2024-09-10T21:51:37.495965+0000","last_unstale":"2024-09-10T21:51:39.966131+0000","last_undegraded":"2024-09-10T21:51:39.966131+0000","last_fullsized":"2024-09-10T21:51:39.966131+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:45.517218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.466542+0000","last_change":"2024-09-10T21:51:37.466542+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.466542+0000","last_undegraded":"2024-09-10T21:51:37.466542+0000","last_fullsized":"2024-09-10T21:51:37.466542+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.478377+0000","last_change":"2024-09-10T21:51:37.477655+0000","last_active":"2024-09-10T21:51:37.478377+0000","last_peered":"2024-09-10T21:51:37.478377+0000","last_clean":"2024-09-10T21:51:37.478377+0000","last_became_active":"2024-09-10T21:51:37.477156+0000","last_became_peered":"2024-09-10T21:51:37.477156+0000","last_unstale":"2024-09-10T21:51:37.478377+0000","last_undegraded":"2024-09-10T21:51:37.478377+0000","last_fullsized":"2024-09-10T21:51:37.478377+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:07.719047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.473775+0000","last_change":"2024-09-10T21:51:37.473775+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.473775+0000","last_undegraded":"2024-09-10T21:51:37.473775+0000","last_fullsized":"2024-09-10T21:51:37.473775+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.480768+0000","last_change":"2024-09-10T21:51:37.478410+0000","last_active":"2024-09-10T21:51:37.480768+0000","last_peered":"2024-09-10T21:51:37.480768+0000","last_clean":"2024-09-10T21:51:37.480768+0000","last_became_active":"2024-09-10T21:51:37.478141+0000","last_became_peered":"2024-09-10T21:51:37.478141+0000","last_unstale":"2024-09-10T21:51:37.480768+0000","last_undegraded":"2024-09-10T21:51:37.480768+0000","last_fullsized":"2024-09-10T21:51:37.480768+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:31:40.813722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966600+0000","last_change":"2024-09-10T21:51:38.501070+0000","last_active":"2024-09-10T21:51:39.966600+0000","last_peered":"2024-09-10T21:51:39.966600+0000","last_clean":"2024-09-10T21:51:39.966600+0000","last_became_active":"2024-09-10T21:51:38.500807+0000","last_became_peered":"2024-09-10T21:51:38.500807+0000","last_unstale":"2024-09-10T21:51:39.966600+0000","last_undegraded":"2024-09-10T21:51:39.966600+0000","last_fullsized":"2024-09-10T21:51:39.966600+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:41.427352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.496386+0000","last_change":"2024-09-10T21:51:37.494039+0000","last_active":"2024-09-10T21:51:37.496386+0000","last_peered":"2024-09-10T21:51:37.496386+0000","last_clean":"2024-09-10T21:51:37.496386+0000","last_became_active":"2024-09-10T21:51:37.493739+0000","last_became_peered":"2024-09-10T21:51:37.493739+0000","last_unstale":"2024-09-10T21:51:37.496386+0000","last_undegraded":"2024-09-10T21:51:37.496386+0000","last_fullsized":"2024-09-10T21:51:37.496386+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:35.957741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.474336+0000","last_change":"2024-09-10T21:51:37.474336+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.474336+0000","last_undegraded":"2024-09-10T21:51:37.474336+0000","last_fullsized":"2024-09-10T21:51:37.474336+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.491621+0000","last_change":"2024-09-10T21:51:37.489957+0000","last_active":"2024-09-10T21:51:37.491621+0000","last_peered":"2024-09-10T21:51:37.491621+0000","last_clean":"2024-09-10T21:51:37.491621+0000","last_became_active":"2024-09-10T21:51:37.488416+0000","last_became_peered":"2024-09-10T21:51:37.488416+0000","last_unstale":"2024-09-10T21:51:37.491621+0000","last_undegraded":"2024-09-10T21:51:37.491621+0000","last_fullsized":"2024-09-10T21:51:37.491621+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:29.190750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965291+0000","last_change":"2024-09-10T21:51:38.495176+0000","last_active":"2024-09-10T21:51:39.965291+0000","last_peered":"2024-09-10T21:51:39.965291+0000","last_clean":"2024-09-10T21:51:39.965291+0000","last_became_active":"2024-09-10T21:51:38.494857+0000","last_became_peered":"2024-09-10T21:51:38.494857+0000","last_unstale":"2024-09-10T21:51:39.965291+0000","last_undegraded":"2024-09-10T21:51:39.965291+0000","last_fullsized":"2024-09-10T21:51:39.965291+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:32.797143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.473311+0000","last_change":"2024-09-10T21:51:37.467889+0000","last_active":"2024-09-10T21:51:37.473311+0000","last_peered":"2024-09-10T21:51:37.473311+0000","last_clean":"2024-09-10T21:51:37.473311+0000","last_became_active":"2024-09-10T21:51:37.467603+0000","last_became_peered":"2024-09-10T21:51:37.467603+0000","last_unstale":"2024-09-10T21:51:37.473311+0000","last_undegraded":"2024-09-10T21:51:37.473311+0000","last_fullsized":"2024-09-10T21:51:37.473311+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:43:22.274692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":281,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966593+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:51:39.966593+0000","last_peered":"2024-09-10T21:51:39.966593+0000","last_clean":"2024-09-10T21:51:39.966593+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:51:39.966593+0000","last_undegraded":"2024-09-10T21:51:39.966593+0000","last_fullsized":"2024-09-10T21:51:39.966593+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509560,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1372160,"data_stored":590735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1372160,"data_stored":590735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509559,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1372160,"data_stored":590735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509559,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1372160,"data_stored":590735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509561,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":1384448,"data_stored":597010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509559,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37280,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334560,"statfs":{"total":96636764160,"available":96598589440,"internally_reserved":0,"allocated":1978368,"data_stored":1187378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542263,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2011136,"data_stored":1201069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542264,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36744,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335096,"statfs":{"total":96636764160,"available":96599138304,"internally_reserved":0,"allocated":1425408,"data_stored":611767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:51:42.731 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:51:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.925+0000 7f8792800640 1 -- 172.21.15.7:0/848917091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f878c15bc90 msgr2=0x7f878c0ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.925+0000 7f8792800640 1 --2- 172.21.15.7:0/848917091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f878c15bc90 0x7f878c0ccef0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f878c0b5560 tx=0x7f877c02a440 comp rx=0 tx=0).stop 2024-09-10T21:51:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.926+0000 7f8792800640 1 -- 172.21.15.7:0/848917091 shutdown_connections 2024-09-10T21:51:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.926+0000 7f8792800640 1 --2- 172.21.15.7:0/848917091 >> v2:172.21.15.133:3300/0 conn(0x7f878c073c70 0x7f878c15fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.926+0000 7f8792800640 1 --2- 172.21.15.7:0/848917091 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f878c0cd750 0x7f878c073730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.926+0000 7f8792800640 1 --2- 172.21.15.7:0/848917091 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f878c15bc90 0x7f878c0ccef0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.926+0000 7f8792800640 1 -- 172.21.15.7:0/848917091 >> 172.21.15.7:0/848917091 conn(0x7f878c0b16d0 msgr2=0x7f878c0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.926+0000 7f8792800640 1 -- 172.21.15.7:0/848917091 shutdown_connections 2024-09-10T21:51:42.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.926+0000 7f8792800640 1 -- 172.21.15.7:0/848917091 wait complete. 2024-09-10T21:51:42.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.927+0000 7f8792800640 1 Processor -- start 2024-09-10T21:51:42.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.927+0000 7f8792800640 1 -- start start 2024-09-10T21:51:42.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.928+0000 7f8792800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f878c073c70 0x7f878c148610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:42.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.928+0000 7f8792800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f878c0cd750 0x7f878c145190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:42.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.928+0000 7f8792800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f878c15bc90 0x7f878c157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:42.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.928+0000 7f8792800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f878c160670 con 0x7f878c073c70 2024-09-10T21:51:42.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.928+0000 7f8792800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f878c1604f0 con 0x7f878c15bc90 2024-09-10T21:51:42.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.928+0000 7f8792800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f878c06fcf0 con 0x7f878c0cd750 2024-09-10T21:51:42.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.928+0000 7f878b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f878c073c70 0x7f878c148610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:42.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.929+0000 7f878b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f878c073c70 0x7f878c148610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44952/0 (socket says 172.21.15.7:44952) 2024-09-10T21:51:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.929+0000 7f878b400640 1 -- 172.21.15.7:0/3281985110 learned_addr learned my addr 172.21.15.7:0/3281985110 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.929+0000 7f878be00640 1 --2- 172.21.15.7:0/3281985110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f878c15bc90 0x7f878c157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.929+0000 7f878aa00640 1 --2- 172.21.15.7:0/3281985110 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f878c0cd750 0x7f878c145190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.929+0000 7f878b400640 1 -- 172.21.15.7:0/3281985110 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f878c0cd750 msgr2=0x7f878c145190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.929+0000 7f878b400640 1 --2- 172.21.15.7:0/3281985110 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f878c0cd750 0x7f878c145190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.929+0000 7f878b400640 1 -- 172.21.15.7:0/3281985110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f878c15bc90 msgr2=0x7f878c157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.929+0000 7f878b400640 1 --2- 172.21.15.7:0/3281985110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f878c15bc90 0x7f878c157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.929+0000 7f878b400640 1 -- 172.21.15.7:0/3281985110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f878c157bd0 con 0x7f878c073c70 2024-09-10T21:51:42.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.929+0000 7f878b400640 1 --2- 172.21.15.7:0/3281985110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f878c073c70 0x7f878c148610 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f877c034040 tx=0x7f877c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:42.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.930+0000 7f8783400640 1 -- 172.21.15.7:0/3281985110 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c03e030 con 0x7f878c073c70 2024-09-10T21:51:42.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.930+0000 7f8783400640 1 -- 172.21.15.7:0/3281985110 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f877c003e00 con 0x7f878c073c70 2024-09-10T21:51:42.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.930+0000 7f8783400640 1 -- 172.21.15.7:0/3281985110 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f877c035040 con 0x7f878c073c70 2024-09-10T21:51:42.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.930+0000 7f8792800640 1 -- 172.21.15.7:0/3281985110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f878c157eb0 con 0x7f878c073c70 2024-09-10T21:51:42.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.930+0000 7f8792800640 1 -- 172.21.15.7:0/3281985110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f878c158470 con 0x7f878c073c70 2024-09-10T21:51:42.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.931+0000 7f8792800640 1 -- 172.21.15.7:0/3281985110 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f878c073730 con 0x7f878c073c70 2024-09-10T21:51:42.932 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.932+0000 7f8783400640 1 -- 172.21.15.7:0/3281985110 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f877c0049c0 con 0x7f878c073c70 2024-09-10T21:51:42.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.933+0000 7f8783400640 1 --2- 172.21.15.7:0/3281985110 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f876c077fa0 0x7f876c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.933+0000 7f8783400640 1 -- 172.21.15.7:0/3281985110 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5709+0+0 (secure 0 0 0) 0x7f877c0ba700 con 0x7f878c073c70 2024-09-10T21:51:42.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.933+0000 7f878aa00640 1 --2- 172.21.15.7:0/3281985110 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f876c077fa0 0x7f876c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:42.935 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.934+0000 7f878aa00640 1 --2- 172.21.15.7:0/3281985110 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f876c077fa0 0x7f876c07a460 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f878c0c3330 tx=0x7f877800b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:42.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:42.937+0000 7f8783400640 1 -- 172.21.15.7:0/3281985110 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f877c082b10 con 0x7f878c073c70 2024-09-10T21:51:43.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.128+0000 7f8792800640 1 -- 172.21.15.7:0/3281985110 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f878c073310 con 0x7f876c077fa0 2024-09-10T21:51:43.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.133+0000 7f8783400640 1 -- 172.21.15.7:0/3281985110 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248776 (secure 0 0 0) 0x7f878c073310 con 0x7f876c077fa0 2024-09-10T21:51:43.137 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:43.138 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:51:43.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.137+0000 7f8792800640 1 -- 172.21.15.7:0/3281985110 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f876c077fa0 msgr2=0x7f876c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:43.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.137+0000 7f8792800640 1 --2- 172.21.15.7:0/3281985110 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f876c077fa0 0x7f876c07a460 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f878c0c3330 tx=0x7f877800b390 comp rx=0 tx=0).stop 2024-09-10T21:51:43.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.137+0000 7f8792800640 1 -- 172.21.15.7:0/3281985110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f878c073c70 msgr2=0x7f878c148610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:43.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.137+0000 7f8792800640 1 --2- 172.21.15.7:0/3281985110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f878c073c70 0x7f878c148610 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f877c034040 tx=0x7f877c003910 comp rx=0 tx=0).stop 2024-09-10T21:51:43.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.137+0000 7f8792800640 1 -- 172.21.15.7:0/3281985110 shutdown_connections 2024-09-10T21:51:43.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.137+0000 7f8792800640 1 --2- 172.21.15.7:0/3281985110 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f876c077fa0 0x7f876c07a460 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:43.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.138+0000 7f8792800640 1 --2- 172.21.15.7:0/3281985110 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f878c15bc90 0x7f878c157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:43.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.138+0000 7f8792800640 1 --2- 172.21.15.7:0/3281985110 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f878c0cd750 0x7f878c145190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:43.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.138+0000 7f8792800640 1 --2- 172.21.15.7:0/3281985110 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f878c073c70 0x7f878c148610 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:43.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.138+0000 7f8792800640 1 -- 172.21.15.7:0/3281985110 >> 172.21.15.7:0/3281985110 conn(0x7f878c0b16d0 msgr2=0x7f878c0c3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:43.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.138+0000 7f8792800640 1 -- 172.21.15.7:0/3281985110 shutdown_connections 2024-09-10T21:51:43.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.138+0000 7f8792800640 1 -- 172.21.15.7:0/3281985110 wait complete. 2024-09-10T21:51:43.154 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-09-10T21:51:42.008379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":274004,"kb_used_data":12000,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754700716,"statfs":{"total":773094113280,"available":772813533184,"internally_reserved":0,"allocated":12288000,"data_stored":5960164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.550209"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.485770+0000","last_change":"2024-09-10T21:51:37.485770+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.485770+0000","last_undegraded":"2024-09-10T21:51:37.485770+0000","last_fullsized":"2024-09-10T21:51:37.485770+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.467394+0000","last_change":"2024-09-10T21:51:37.467394+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.467394+0000","last_undegraded":"2024-09-10T21:51:37.467394+0000","last_fullsized":"2024-09-10T21:51:37.467394+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966452+0000","last_change":"2024-09-10T21:51:38.505264+0000","last_active":"2024-09-10T21:51:39.966452+0000","last_peered":"2024-09-10T21:51:39.966452+0000","last_clean":"2024-09-10T21:51:39.966452+0000","last_became_active":"2024-09-10T21:51:38.504121+0000","last_became_peered":"2024-09-10T21:51:38.504121+0000","last_unstale":"2024-09-10T21:51:39.966452+0000","last_undegraded":"2024-09-10T21:51:39.966452+0000","last_fullsized":"2024-09-10T21:51:39.966452+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:32.797143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.489196+0000","last_change":"2024-09-10T21:51:37.489196+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.489196+0000","last_undegraded":"2024-09-10T21:51:37.489196+0000","last_fullsized":"2024-09-10T21:51:37.489196+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.476187+0000","last_change":"2024-09-10T21:51:37.476187+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.476187+0000","last_undegraded":"2024-09-10T21:51:37.476187+0000","last_fullsized":"2024-09-10T21:51:37.476187+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.469812+0000","last_change":"2024-09-10T21:51:37.469812+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.469812+0000","last_undegraded":"2024-09-10T21:51:37.469812+0000","last_fullsized":"2024-09-10T21:51:37.469812+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.465592+0000","last_change":"2024-09-10T21:51:37.465592+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.465592+0000","last_undegraded":"2024-09-10T21:51:37.465592+0000","last_fullsized":"2024-09-10T21:51:37.465592+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966436+0000","last_change":"2024-09-10T21:51:38.498466+0000","last_active":"2024-09-10T21:51:39.966436+0000","last_peered":"2024-09-10T21:51:39.966436+0000","last_clean":"2024-09-10T21:51:39.966436+0000","last_became_active":"2024-09-10T21:51:38.498041+0000","last_became_peered":"2024-09-10T21:51:38.498041+0000","last_unstale":"2024-09-10T21:51:39.966436+0000","last_undegraded":"2024-09-10T21:51:39.966436+0000","last_fullsized":"2024-09-10T21:51:39.966436+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:19.954387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.482918+0000","last_change":"2024-09-10T21:51:37.482918+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.482918+0000","last_undegraded":"2024-09-10T21:51:37.482918+0000","last_fullsized":"2024-09-10T21:51:37.482918+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.482427+0000","last_change":"2024-09-10T21:51:37.482427+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.482427+0000","last_undegraded":"2024-09-10T21:51:37.482427+0000","last_fullsized":"2024-09-10T21:51:37.482427+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.468175+0000","last_change":"2024-09-10T21:51:37.468175+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.468175+0000","last_undegraded":"2024-09-10T21:51:37.468175+0000","last_fullsized":"2024-09-10T21:51:37.468175+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.485246+0000","last_change":"2024-09-10T21:51:37.485246+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.485246+0000","last_undegraded":"2024-09-10T21:51:37.485246+0000","last_fullsized":"2024-09-10T21:51:37.485246+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966404+0000","last_change":"2024-09-10T21:51:38.503338+0000","last_active":"2024-09-10T21:51:39.966404+0000","last_peered":"2024-09-10T21:51:39.966404+0000","last_clean":"2024-09-10T21:51:39.966404+0000","last_became_active":"2024-09-10T21:51:38.501614+0000","last_became_peered":"2024-09-10T21:51:38.501614+0000","last_unstale":"2024-09-10T21:51:39.966404+0000","last_undegraded":"2024-09-10T21:51:39.966404+0000","last_fullsized":"2024-09-10T21:51:39.966404+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:10.529060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966266+0000","last_change":"2024-09-10T21:51:38.498967+0000","last_active":"2024-09-10T21:51:39.966266+0000","last_peered":"2024-09-10T21:51:39.966266+0000","last_clean":"2024-09-10T21:51:39.966266+0000","last_became_active":"2024-09-10T21:51:38.498641+0000","last_became_peered":"2024-09-10T21:51:38.498641+0000","last_unstale":"2024-09-10T21:51:39.966266+0000","last_undegraded":"2024-09-10T21:51:39.966266+0000","last_fullsized":"2024-09-10T21:51:39.966266+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:41.424015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.469393+0000","last_change":"2024-09-10T21:51:37.469393+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.469393+0000","last_undegraded":"2024-09-10T21:51:37.469393+0000","last_fullsized":"2024-09-10T21:51:37.469393+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.481943+0000","last_change":"2024-09-10T21:51:37.481943+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.481943+0000","last_undegraded":"2024-09-10T21:51:37.481943+0000","last_fullsized":"2024-09-10T21:51:37.481943+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.468661+0000","last_change":"2024-09-10T21:51:37.468661+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.468661+0000","last_undegraded":"2024-09-10T21:51:37.468661+0000","last_fullsized":"2024-09-10T21:51:37.468661+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.479101+0000","last_change":"2024-09-10T21:51:37.479101+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.479101+0000","last_undegraded":"2024-09-10T21:51:37.479101+0000","last_fullsized":"2024-09-10T21:51:37.479101+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966459+0000","last_change":"2024-09-10T21:51:38.506319+0000","last_active":"2024-09-10T21:51:39.966459+0000","last_peered":"2024-09-10T21:51:39.966459+0000","last_clean":"2024-09-10T21:51:39.966459+0000","last_became_active":"2024-09-10T21:51:38.506041+0000","last_became_peered":"2024-09-10T21:51:38.506041+0000","last_unstale":"2024-09-10T21:51:39.966459+0000","last_undegraded":"2024-09-10T21:51:39.966459+0000","last_fullsized":"2024-09-10T21:51:39.966459+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:33.484489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.467188+0000","last_change":"2024-09-10T21:51:37.467188+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.467188+0000","last_undegraded":"2024-09-10T21:51:37.467188+0000","last_fullsized":"2024-09-10T21:51:37.467188+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.469235+0000","last_change":"2024-09-10T21:51:37.469235+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.469235+0000","last_undegraded":"2024-09-10T21:51:37.469235+0000","last_fullsized":"2024-09-10T21:51:37.469235+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965676+0000","last_change":"2024-09-10T21:51:38.495309+0000","last_active":"2024-09-10T21:51:39.965676+0000","last_peered":"2024-09-10T21:51:39.965676+0000","last_clean":"2024-09-10T21:51:39.965676+0000","last_became_active":"2024-09-10T21:51:38.494998+0000","last_became_peered":"2024-09-10T21:51:38.494998+0000","last_unstale":"2024-09-10T21:51:39.965676+0000","last_undegraded":"2024-09-10T21:51:39.965676+0000","last_fullsized":"2024-09-10T21:51:39.965676+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:41.427352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.466079+0000","last_change":"2024-09-10T21:51:37.466079+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.466079+0000","last_undegraded":"2024-09-10T21:51:37.466079+0000","last_fullsized":"2024-09-10T21:51:37.466079+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965229+0000","last_change":"2024-09-10T21:51:38.505466+0000","last_active":"2024-09-10T21:51:39.965229+0000","last_peered":"2024-09-10T21:51:39.965229+0000","last_clean":"2024-09-10T21:51:39.965229+0000","last_became_active":"2024-09-10T21:51:38.505053+0000","last_became_peered":"2024-09-10T21:51:38.505053+0000","last_unstale":"2024-09-10T21:51:39.965229+0000","last_undegraded":"2024-09-10T21:51:39.965229+0000","last_fullsized":"2024-09-10T21:51:39.965229+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:36:29.403651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.467357+0000","last_change":"2024-09-10T21:51:37.467357+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.467357+0000","last_undegraded":"2024-09-10T21:51:37.467357+0000","last_fullsized":"2024-09-10T21:51:37.467357+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.478229+0000","last_change":"2024-09-10T21:51:37.478229+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.478229+0000","last_undegraded":"2024-09-10T21:51:37.478229+0000","last_fullsized":"2024-09-10T21:51:37.478229+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.473593+0000","last_change":"2024-09-10T21:51:37.473593+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.473593+0000","last_undegraded":"2024-09-10T21:51:37.473593+0000","last_fullsized":"2024-09-10T21:51:37.473593+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965771+0000","last_change":"2024-09-10T21:51:38.496708+0000","last_active":"2024-09-10T21:51:39.965771+0000","last_peered":"2024-09-10T21:51:39.965771+0000","last_clean":"2024-09-10T21:51:39.965771+0000","last_became_active":"2024-09-10T21:51:38.496402+0000","last_became_peered":"2024-09-10T21:51:38.496402+0000","last_unstale":"2024-09-10T21:51:39.965771+0000","last_undegraded":"2024-09-10T21:51:39.965771+0000","last_fullsized":"2024-09-10T21:51:39.965771+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:40.305224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.474613+0000","last_change":"2024-09-10T21:51:37.474613+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.474613+0000","last_undegraded":"2024-09-10T21:51:37.474613+0000","last_fullsized":"2024-09-10T21:51:37.474613+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.463487+0000","last_change":"2024-09-10T21:51:37.463487+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.463487+0000","last_undegraded":"2024-09-10T21:51:37.463487+0000","last_fullsized":"2024-09-10T21:51:37.463487+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.478186+0000","last_change":"2024-09-10T21:51:37.478186+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.478186+0000","last_undegraded":"2024-09-10T21:51:37.478186+0000","last_fullsized":"2024-09-10T21:51:37.478186+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.468954+0000","last_change":"2024-09-10T21:51:37.468954+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.468954+0000","last_undegraded":"2024-09-10T21:51:37.468954+0000","last_fullsized":"2024-09-10T21:51:37.468954+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.477280+0000","last_change":"2024-09-10T21:51:37.477280+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.477280+0000","last_undegraded":"2024-09-10T21:51:37.477280+0000","last_fullsized":"2024-09-10T21:51:37.477280+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.474881+0000","last_change":"2024-09-10T21:51:37.472999+0000","last_active":"2024-09-10T21:51:37.474881+0000","last_peered":"2024-09-10T21:51:37.474881+0000","last_clean":"2024-09-10T21:51:37.474881+0000","last_became_active":"2024-09-10T21:51:37.472471+0000","last_became_peered":"2024-09-10T21:51:37.472471+0000","last_unstale":"2024-09-10T21:51:37.474881+0000","last_undegraded":"2024-09-10T21:51:37.474881+0000","last_fullsized":"2024-09-10T21:51:37.474881+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:30.555990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.480358+0000","last_change":"2024-09-10T21:51:37.480358+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.480358+0000","last_undegraded":"2024-09-10T21:51:37.480358+0000","last_fullsized":"2024-09-10T21:51:37.480358+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966096+0000","last_change":"2024-09-10T21:51:37.481369+0000","last_active":"2024-09-10T21:51:39.966096+0000","last_peered":"2024-09-10T21:51:39.966096+0000","last_clean":"2024-09-10T21:51:39.966096+0000","last_became_active":"2024-09-10T21:51:37.481061+0000","last_became_peered":"2024-09-10T21:51:37.481061+0000","last_unstale":"2024-09-10T21:51:39.966096+0000","last_undegraded":"2024-09-10T21:51:39.966096+0000","last_fullsized":"2024-09-10T21:51:39.966096+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:48.767875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.473226+0000","last_change":"2024-09-10T21:51:37.473226+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.473226+0000","last_undegraded":"2024-09-10T21:51:37.473226+0000","last_fullsized":"2024-09-10T21:51:37.473226+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.497767+0000","last_change":"2024-09-10T21:51:37.495401+0000","last_active":"2024-09-10T21:51:37.497767+0000","last_peered":"2024-09-10T21:51:37.497767+0000","last_clean":"2024-09-10T21:51:37.497767+0000","last_became_active":"2024-09-10T21:51:37.495129+0000","last_became_peered":"2024-09-10T21:51:37.495129+0000","last_unstale":"2024-09-10T21:51:37.497767+0000","last_undegraded":"2024-09-10T21:51:37.497767+0000","last_fullsized":"2024-09-10T21:51:37.497767+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:58.176252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966641+0000","last_change":"2024-09-10T21:51:38.507168+0000","last_active":"2024-09-10T21:51:39.966641+0000","last_peered":"2024-09-10T21:51:39.966641+0000","last_clean":"2024-09-10T21:51:39.966641+0000","last_became_active":"2024-09-10T21:51:38.506850+0000","last_became_peered":"2024-09-10T21:51:38.506850+0000","last_unstale":"2024-09-10T21:51:39.966641+0000","last_undegraded":"2024-09-10T21:51:39.966641+0000","last_fullsized":"2024-09-10T21:51:39.966641+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:40.305224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.497488+0000","last_change":"2024-09-10T21:51:37.495072+0000","last_active":"2024-09-10T21:51:37.497488+0000","last_peered":"2024-09-10T21:51:37.497488+0000","last_clean":"2024-09-10T21:51:37.497488+0000","last_became_active":"2024-09-10T21:51:37.494778+0000","last_became_peered":"2024-09-10T21:51:37.494778+0000","last_unstale":"2024-09-10T21:51:37.497488+0000","last_undegraded":"2024-09-10T21:51:37.497488+0000","last_fullsized":"2024-09-10T21:51:37.497488+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:18.574835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.476588+0000","last_change":"2024-09-10T21:51:37.476588+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.476588+0000","last_undegraded":"2024-09-10T21:51:37.476588+0000","last_fullsized":"2024-09-10T21:51:37.476588+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.476242+0000","last_change":"2024-09-10T21:51:37.474801+0000","last_active":"2024-09-10T21:51:37.476242+0000","last_peered":"2024-09-10T21:51:37.476242+0000","last_clean":"2024-09-10T21:51:37.476242+0000","last_became_active":"2024-09-10T21:51:37.474455+0000","last_became_peered":"2024-09-10T21:51:37.474455+0000","last_unstale":"2024-09-10T21:51:37.476242+0000","last_undegraded":"2024-09-10T21:51:37.476242+0000","last_fullsized":"2024-09-10T21:51:37.476242+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:00.575937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.474186+0000","last_change":"2024-09-10T21:51:37.474186+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.474186+0000","last_undegraded":"2024-09-10T21:51:37.474186+0000","last_fullsized":"2024-09-10T21:51:37.474186+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965968+0000","last_change":"2024-09-10T21:51:37.481177+0000","last_active":"2024-09-10T21:51:39.965968+0000","last_peered":"2024-09-10T21:51:39.965968+0000","last_clean":"2024-09-10T21:51:39.965968+0000","last_became_active":"2024-09-10T21:51:37.480878+0000","last_became_peered":"2024-09-10T21:51:37.480878+0000","last_unstale":"2024-09-10T21:51:39.965968+0000","last_undegraded":"2024-09-10T21:51:39.965968+0000","last_fullsized":"2024-09-10T21:51:39.965968+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:39.644340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.477970+0000","last_change":"2024-09-10T21:51:37.477970+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.477970+0000","last_undegraded":"2024-09-10T21:51:37.477970+0000","last_fullsized":"2024-09-10T21:51:37.477970+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.487128+0000","last_change":"2024-09-10T21:51:37.486466+0000","last_active":"2024-09-10T21:51:37.487128+0000","last_peered":"2024-09-10T21:51:37.487128+0000","last_clean":"2024-09-10T21:51:37.487128+0000","last_became_active":"2024-09-10T21:51:37.486160+0000","last_became_peered":"2024-09-10T21:51:37.486160+0000","last_unstale":"2024-09-10T21:51:37.487128+0000","last_undegraded":"2024-09-10T21:51:37.487128+0000","last_fullsized":"2024-09-10T21:51:37.487128+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:38.793976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.463935+0000","last_change":"2024-09-10T21:51:37.463935+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.463935+0000","last_undegraded":"2024-09-10T21:51:37.463935+0000","last_fullsized":"2024-09-10T21:51:37.463935+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.491594+0000","last_change":"2024-09-10T21:51:37.489582+0000","last_active":"2024-09-10T21:51:37.491594+0000","last_peered":"2024-09-10T21:51:37.491594+0000","last_clean":"2024-09-10T21:51:37.491594+0000","last_became_active":"2024-09-10T21:51:37.487889+0000","last_became_peered":"2024-09-10T21:51:37.487889+0000","last_unstale":"2024-09-10T21:51:37.491594+0000","last_undegraded":"2024-09-10T21:51:37.491594+0000","last_fullsized":"2024-09-10T21:51:37.491594+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:53.102794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.463971+0000","last_change":"2024-09-10T21:51:37.463971+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.463971+0000","last_undegraded":"2024-09-10T21:51:37.463971+0000","last_fullsized":"2024-09-10T21:51:37.463971+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.496341+0000","last_change":"2024-09-10T21:51:37.493056+0000","last_active":"2024-09-10T21:51:37.496341+0000","last_peered":"2024-09-10T21:51:37.496341+0000","last_clean":"2024-09-10T21:51:37.496341+0000","last_became_active":"2024-09-10T21:51:37.492756+0000","last_became_peered":"2024-09-10T21:51:37.492756+0000","last_unstale":"2024-09-10T21:51:37.496341+0000","last_undegraded":"2024-09-10T21:51:37.496341+0000","last_fullsized":"2024-09-10T21:51:37.496341+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:33.347129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.473160+0000","last_change":"2024-09-10T21:51:37.473160+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.473160+0000","last_undegraded":"2024-09-10T21:51:37.473160+0000","last_fullsized":"2024-09-10T21:51:37.473160+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966160+0000","last_change":"2024-09-10T21:51:37.487117+0000","last_active":"2024-09-10T21:51:39.966160+0000","last_peered":"2024-09-10T21:51:39.966160+0000","last_clean":"2024-09-10T21:51:39.966160+0000","last_became_active":"2024-09-10T21:51:37.484685+0000","last_became_peered":"2024-09-10T21:51:37.484685+0000","last_unstale":"2024-09-10T21:51:39.966160+0000","last_undegraded":"2024-09-10T21:51:39.966160+0000","last_fullsized":"2024-09-10T21:51:39.966160+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:13.540834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965991+0000","last_change":"2024-09-10T21:51:38.500232+0000","last_active":"2024-09-10T21:51:39.965991+0000","last_peered":"2024-09-10T21:51:39.965991+0000","last_clean":"2024-09-10T21:51:39.965991+0000","last_became_active":"2024-09-10T21:51:38.499959+0000","last_became_peered":"2024-09-10T21:51:38.499959+0000","last_unstale":"2024-09-10T21:51:39.965991+0000","last_undegraded":"2024-09-10T21:51:39.965991+0000","last_fullsized":"2024-09-10T21:51:39.965991+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:35.171265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.484868+0000","last_change":"2024-09-10T21:51:37.483939+0000","last_active":"2024-09-10T21:51:37.484868+0000","last_peered":"2024-09-10T21:51:37.484868+0000","last_clean":"2024-09-10T21:51:37.484868+0000","last_became_active":"2024-09-10T21:51:37.483023+0000","last_became_peered":"2024-09-10T21:51:37.483023+0000","last_unstale":"2024-09-10T21:51:37.484868+0000","last_undegraded":"2024-09-10T21:51:37.484868+0000","last_fullsized":"2024-09-10T21:51:37.484868+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:42.259904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.464945+0000","last_change":"2024-09-10T21:51:37.464945+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.464945+0000","last_undegraded":"2024-09-10T21:51:37.464945+0000","last_fullsized":"2024-09-10T21:51:37.464945+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.492264+0000","last_change":"2024-09-10T21:51:37.490837+0000","last_active":"2024-09-10T21:51:37.492264+0000","last_peered":"2024-09-10T21:51:37.492264+0000","last_clean":"2024-09-10T21:51:37.492264+0000","last_became_active":"2024-09-10T21:51:37.489661+0000","last_became_peered":"2024-09-10T21:51:37.489661+0000","last_unstale":"2024-09-10T21:51:37.492264+0000","last_undegraded":"2024-09-10T21:51:37.492264+0000","last_fullsized":"2024-09-10T21:51:37.492264+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:58.486726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966089+0000","last_change":"2024-09-10T21:51:38.499244+0000","last_active":"2024-09-10T21:51:39.966089+0000","last_peered":"2024-09-10T21:51:39.966089+0000","last_clean":"2024-09-10T21:51:39.966089+0000","last_became_active":"2024-09-10T21:51:38.498965+0000","last_became_peered":"2024-09-10T21:51:38.498965+0000","last_unstale":"2024-09-10T21:51:39.966089+0000","last_undegraded":"2024-09-10T21:51:39.966089+0000","last_fullsized":"2024-09-10T21:51:39.966089+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:12.629720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.488507+0000","last_change":"2024-09-10T21:51:37.480818+0000","last_active":"2024-09-10T21:51:37.488507+0000","last_peered":"2024-09-10T21:51:37.488507+0000","last_clean":"2024-09-10T21:51:37.488507+0000","last_became_active":"2024-09-10T21:51:37.480562+0000","last_became_peered":"2024-09-10T21:51:37.480562+0000","last_unstale":"2024-09-10T21:51:37.488507+0000","last_undegraded":"2024-09-10T21:51:37.488507+0000","last_fullsized":"2024-09-10T21:51:37.488507+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:05.569050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965893+0000","last_change":"2024-09-10T21:51:38.504740+0000","last_active":"2024-09-10T21:51:39.965893+0000","last_peered":"2024-09-10T21:51:39.965893+0000","last_clean":"2024-09-10T21:51:39.965893+0000","last_became_active":"2024-09-10T21:51:38.504214+0000","last_became_peered":"2024-09-10T21:51:38.504214+0000","last_unstale":"2024-09-10T21:51:39.965893+0000","last_undegraded":"2024-09-10T21:51:39.965893+0000","last_fullsized":"2024-09-10T21:51:39.965893+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:47.805058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.492594+0000","last_change":"2024-09-10T21:51:37.491755+0000","last_active":"2024-09-10T21:51:37.492594+0000","last_peered":"2024-09-10T21:51:37.492594+0000","last_clean":"2024-09-10T21:51:37.492594+0000","last_became_active":"2024-09-10T21:51:37.491472+0000","last_became_peered":"2024-09-10T21:51:37.491472+0000","last_unstale":"2024-09-10T21:51:37.492594+0000","last_undegraded":"2024-09-10T21:51:37.492594+0000","last_fullsized":"2024-09-10T21:51:37.492594+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:09.126767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965395+0000","last_change":"2024-09-10T21:51:38.500521+0000","last_active":"2024-09-10T21:51:39.965395+0000","last_peered":"2024-09-10T21:51:39.965395+0000","last_clean":"2024-09-10T21:51:39.965395+0000","last_became_active":"2024-09-10T21:51:38.500234+0000","last_became_peered":"2024-09-10T21:51:38.500234+0000","last_unstale":"2024-09-10T21:51:39.965395+0000","last_undegraded":"2024-09-10T21:51:39.965395+0000","last_fullsized":"2024-09-10T21:51:39.965395+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:28.278479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.496082+0000","last_change":"2024-09-10T21:51:37.492464+0000","last_active":"2024-09-10T21:51:37.496082+0000","last_peered":"2024-09-10T21:51:37.496082+0000","last_clean":"2024-09-10T21:51:37.496082+0000","last_became_active":"2024-09-10T21:51:37.492162+0000","last_became_peered":"2024-09-10T21:51:37.492162+0000","last_unstale":"2024-09-10T21:51:37.496082+0000","last_undegraded":"2024-09-10T21:51:37.496082+0000","last_fullsized":"2024-09-10T21:51:37.496082+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:43.699147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.475624+0000","last_change":"2024-09-10T21:51:37.475624+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.475624+0000","last_undegraded":"2024-09-10T21:51:37.475624+0000","last_fullsized":"2024-09-10T21:51:37.475624+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.491463+0000","last_change":"2024-09-10T21:51:37.468073+0000","last_active":"2024-09-10T21:51:37.491463+0000","last_peered":"2024-09-10T21:51:37.491463+0000","last_clean":"2024-09-10T21:51:37.491463+0000","last_became_active":"2024-09-10T21:51:37.467716+0000","last_became_peered":"2024-09-10T21:51:37.467716+0000","last_unstale":"2024-09-10T21:51:37.491463+0000","last_undegraded":"2024-09-10T21:51:37.491463+0000","last_fullsized":"2024-09-10T21:51:37.491463+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:36.082631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.473936+0000","last_change":"2024-09-10T21:51:37.473936+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.473936+0000","last_undegraded":"2024-09-10T21:51:37.473936+0000","last_fullsized":"2024-09-10T21:51:37.473936+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.496000+0000","last_change":"2024-09-10T21:51:37.485673+0000","last_active":"2024-09-10T21:51:37.496000+0000","last_peered":"2024-09-10T21:51:37.496000+0000","last_clean":"2024-09-10T21:51:37.496000+0000","last_became_active":"2024-09-10T21:51:37.485132+0000","last_became_peered":"2024-09-10T21:51:37.485132+0000","last_unstale":"2024-09-10T21:51:37.496000+0000","last_undegraded":"2024-09-10T21:51:37.496000+0000","last_fullsized":"2024-09-10T21:51:37.496000+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:30:53.853582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966440+0000","last_change":"2024-09-10T21:51:38.505872+0000","last_active":"2024-09-10T21:51:39.966440+0000","last_peered":"2024-09-10T21:51:39.966440+0000","last_clean":"2024-09-10T21:51:39.966440+0000","last_became_active":"2024-09-10T21:51:38.505488+0000","last_became_peered":"2024-09-10T21:51:38.505488+0000","last_unstale":"2024-09-10T21:51:39.966440+0000","last_undegraded":"2024-09-10T21:51:39.966440+0000","last_fullsized":"2024-09-10T21:51:39.966440+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:29.689596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.485861+0000","last_change":"2024-09-10T21:51:37.485228+0000","last_active":"2024-09-10T21:51:37.485861+0000","last_peered":"2024-09-10T21:51:37.485861+0000","last_clean":"2024-09-10T21:51:37.485861+0000","last_became_active":"2024-09-10T21:51:37.484980+0000","last_became_peered":"2024-09-10T21:51:37.484980+0000","last_unstale":"2024-09-10T21:51:37.485861+0000","last_undegraded":"2024-09-10T21:51:37.485861+0000","last_fullsized":"2024-09-10T21:51:37.485861+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:24.448659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966710+0000","last_change":"2024-09-10T21:51:38.504879+0000","last_active":"2024-09-10T21:51:39.966710+0000","last_peered":"2024-09-10T21:51:39.966710+0000","last_clean":"2024-09-10T21:51:39.966710+0000","last_became_active":"2024-09-10T21:51:38.504560+0000","last_became_peered":"2024-09-10T21:51:38.504560+0000","last_unstale":"2024-09-10T21:51:39.966710+0000","last_undegraded":"2024-09-10T21:51:39.966710+0000","last_fullsized":"2024-09-10T21:51:39.966710+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:47.805058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.473267+0000","last_change":"2024-09-10T21:51:37.469638+0000","last_active":"2024-09-10T21:51:37.473267+0000","last_peered":"2024-09-10T21:51:37.473267+0000","last_clean":"2024-09-10T21:51:37.473267+0000","last_became_active":"2024-09-10T21:51:37.467556+0000","last_became_peered":"2024-09-10T21:51:37.467556+0000","last_unstale":"2024-09-10T21:51:37.473267+0000","last_undegraded":"2024-09-10T21:51:37.473267+0000","last_fullsized":"2024-09-10T21:51:37.473267+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:11.474396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966529+0000","last_change":"2024-09-10T21:51:38.504149+0000","last_active":"2024-09-10T21:51:39.966529+0000","last_peered":"2024-09-10T21:51:39.966529+0000","last_clean":"2024-09-10T21:51:39.966529+0000","last_became_active":"2024-09-10T21:51:38.503873+0000","last_became_peered":"2024-09-10T21:51:38.503873+0000","last_unstale":"2024-09-10T21:51:39.966529+0000","last_undegraded":"2024-09-10T21:51:39.966529+0000","last_fullsized":"2024-09-10T21:51:39.966529+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:38.756786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.492556+0000","last_change":"2024-09-10T21:51:37.488302+0000","last_active":"2024-09-10T21:51:37.492556+0000","last_peered":"2024-09-10T21:51:37.492556+0000","last_clean":"2024-09-10T21:51:37.492556+0000","last_became_active":"2024-09-10T21:51:37.488054+0000","last_became_peered":"2024-09-10T21:51:37.488054+0000","last_unstale":"2024-09-10T21:51:37.492556+0000","last_undegraded":"2024-09-10T21:51:37.492556+0000","last_fullsized":"2024-09-10T21:51:37.492556+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:13.280288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.471734+0000","last_change":"2024-09-10T21:51:37.471734+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.471734+0000","last_undegraded":"2024-09-10T21:51:37.471734+0000","last_fullsized":"2024-09-10T21:51:37.471734+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.497783+0000","last_change":"2024-09-10T21:51:37.495655+0000","last_active":"2024-09-10T21:51:37.497783+0000","last_peered":"2024-09-10T21:51:37.497783+0000","last_clean":"2024-09-10T21:51:37.497783+0000","last_became_active":"2024-09-10T21:51:37.495383+0000","last_became_peered":"2024-09-10T21:51:37.495383+0000","last_unstale":"2024-09-10T21:51:37.497783+0000","last_undegraded":"2024-09-10T21:51:37.497783+0000","last_fullsized":"2024-09-10T21:51:37.497783+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:38.126366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966363+0000","last_change":"2024-09-10T21:51:38.498184+0000","last_active":"2024-09-10T21:51:39.966363+0000","last_peered":"2024-09-10T21:51:39.966363+0000","last_clean":"2024-09-10T21:51:39.966363+0000","last_became_active":"2024-09-10T21:51:38.497840+0000","last_became_peered":"2024-09-10T21:51:38.497840+0000","last_unstale":"2024-09-10T21:51:39.966363+0000","last_undegraded":"2024-09-10T21:51:39.966363+0000","last_fullsized":"2024-09-10T21:51:39.966363+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:50.132177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.495778+0000","last_change":"2024-09-10T21:51:37.493629+0000","last_active":"2024-09-10T21:51:37.495778+0000","last_peered":"2024-09-10T21:51:37.495778+0000","last_clean":"2024-09-10T21:51:37.495778+0000","last_became_active":"2024-09-10T21:51:37.493347+0000","last_became_peered":"2024-09-10T21:51:37.493347+0000","last_unstale":"2024-09-10T21:51:37.495778+0000","last_undegraded":"2024-09-10T21:51:37.495778+0000","last_fullsized":"2024-09-10T21:51:37.495778+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:54.092525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965539+0000","last_change":"2024-09-10T21:51:38.497499+0000","last_active":"2024-09-10T21:51:39.965539+0000","last_peered":"2024-09-10T21:51:39.965539+0000","last_clean":"2024-09-10T21:51:39.965539+0000","last_became_active":"2024-09-10T21:51:38.494172+0000","last_became_peered":"2024-09-10T21:51:38.494172+0000","last_unstale":"2024-09-10T21:51:39.965539+0000","last_undegraded":"2024-09-10T21:51:39.965539+0000","last_fullsized":"2024-09-10T21:51:39.965539+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:38.756786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.491642+0000","last_change":"2024-09-10T21:51:37.490268+0000","last_active":"2024-09-10T21:51:37.491642+0000","last_peered":"2024-09-10T21:51:37.491642+0000","last_clean":"2024-09-10T21:51:37.491642+0000","last_became_active":"2024-09-10T21:51:37.488772+0000","last_became_peered":"2024-09-10T21:51:37.488772+0000","last_unstale":"2024-09-10T21:51:37.491642+0000","last_undegraded":"2024-09-10T21:51:37.491642+0000","last_fullsized":"2024-09-10T21:51:37.491642+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:18.953120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.470749+0000","last_change":"2024-09-10T21:51:37.470749+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.470749+0000","last_undegraded":"2024-09-10T21:51:37.470749+0000","last_fullsized":"2024-09-10T21:51:37.470749+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966234+0000","last_change":"2024-09-10T21:51:37.493424+0000","last_active":"2024-09-10T21:51:39.966234+0000","last_peered":"2024-09-10T21:51:39.966234+0000","last_clean":"2024-09-10T21:51:39.966234+0000","last_became_active":"2024-09-10T21:51:37.492964+0000","last_became_peered":"2024-09-10T21:51:37.492964+0000","last_unstale":"2024-09-10T21:51:39.966234+0000","last_undegraded":"2024-09-10T21:51:39.966234+0000","last_fullsized":"2024-09-10T21:51:39.966234+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:56.310705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.472197+0000","last_change":"2024-09-10T21:51:37.472197+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.472197+0000","last_undegraded":"2024-09-10T21:51:37.472197+0000","last_fullsized":"2024-09-10T21:51:37.472197+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.482551+0000","last_change":"2024-09-10T21:51:37.481469+0000","last_active":"2024-09-10T21:51:37.482551+0000","last_peered":"2024-09-10T21:51:37.482551+0000","last_clean":"2024-09-10T21:51:37.482551+0000","last_became_active":"2024-09-10T21:51:37.481196+0000","last_became_peered":"2024-09-10T21:51:37.481196+0000","last_unstale":"2024-09-10T21:51:37.482551+0000","last_undegraded":"2024-09-10T21:51:37.482551+0000","last_fullsized":"2024-09-10T21:51:37.482551+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:10.385869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.471647+0000","last_change":"2024-09-10T21:51:37.471647+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.471647+0000","last_undegraded":"2024-09-10T21:51:37.471647+0000","last_fullsized":"2024-09-10T21:51:37.471647+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.492284+0000","last_change":"2024-09-10T21:51:37.490280+0000","last_active":"2024-09-10T21:51:37.492284+0000","last_peered":"2024-09-10T21:51:37.492284+0000","last_clean":"2024-09-10T21:51:37.492284+0000","last_became_active":"2024-09-10T21:51:37.489979+0000","last_became_peered":"2024-09-10T21:51:37.489979+0000","last_unstale":"2024-09-10T21:51:37.492284+0000","last_undegraded":"2024-09-10T21:51:37.492284+0000","last_fullsized":"2024-09-10T21:51:37.492284+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:09:51.282814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.475986+0000","last_change":"2024-09-10T21:51:37.475986+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.475986+0000","last_undegraded":"2024-09-10T21:51:37.475986+0000","last_fullsized":"2024-09-10T21:51:37.475986+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966131+0000","last_change":"2024-09-10T21:51:37.496547+0000","last_active":"2024-09-10T21:51:39.966131+0000","last_peered":"2024-09-10T21:51:39.966131+0000","last_clean":"2024-09-10T21:51:39.966131+0000","last_became_active":"2024-09-10T21:51:37.495965+0000","last_became_peered":"2024-09-10T21:51:37.495965+0000","last_unstale":"2024-09-10T21:51:39.966131+0000","last_undegraded":"2024-09-10T21:51:39.966131+0000","last_fullsized":"2024-09-10T21:51:39.966131+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:45.517218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.466542+0000","last_change":"2024-09-10T21:51:37.466542+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.466542+0000","last_undegraded":"2024-09-10T21:51:37.466542+0000","last_fullsized":"2024-09-10T21:51:37.466542+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.478377+0000","last_change":"2024-09-10T21:51:37.477655+0000","last_active":"2024-09-10T21:51:37.478377+0000","last_peered":"2024-09-10T21:51:37.478377+0000","last_clean":"2024-09-10T21:51:37.478377+0000","last_became_active":"2024-09-10T21:51:37.477156+0000","last_became_peered":"2024-09-10T21:51:37.477156+0000","last_unstale":"2024-09-10T21:51:37.478377+0000","last_undegraded":"2024-09-10T21:51:37.478377+0000","last_fullsized":"2024-09-10T21:51:37.478377+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:07.719047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.473775+0000","last_change":"2024-09-10T21:51:37.473775+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.473775+0000","last_undegraded":"2024-09-10T21:51:37.473775+0000","last_fullsized":"2024-09-10T21:51:37.473775+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.480768+0000","last_change":"2024-09-10T21:51:37.478410+0000","last_active":"2024-09-10T21:51:37.480768+0000","last_peered":"2024-09-10T21:51:37.480768+0000","last_clean":"2024-09-10T21:51:37.480768+0000","last_became_active":"2024-09-10T21:51:37.478141+0000","last_became_peered":"2024-09-10T21:51:37.478141+0000","last_unstale":"2024-09-10T21:51:37.480768+0000","last_undegraded":"2024-09-10T21:51:37.480768+0000","last_fullsized":"2024-09-10T21:51:37.480768+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:31:40.813722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966600+0000","last_change":"2024-09-10T21:51:38.501070+0000","last_active":"2024-09-10T21:51:39.966600+0000","last_peered":"2024-09-10T21:51:39.966600+0000","last_clean":"2024-09-10T21:51:39.966600+0000","last_became_active":"2024-09-10T21:51:38.500807+0000","last_became_peered":"2024-09-10T21:51:38.500807+0000","last_unstale":"2024-09-10T21:51:39.966600+0000","last_undegraded":"2024-09-10T21:51:39.966600+0000","last_fullsized":"2024-09-10T21:51:39.966600+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:41.427352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.496386+0000","last_change":"2024-09-10T21:51:37.494039+0000","last_active":"2024-09-10T21:51:37.496386+0000","last_peered":"2024-09-10T21:51:37.496386+0000","last_clean":"2024-09-10T21:51:37.496386+0000","last_became_active":"2024-09-10T21:51:37.493739+0000","last_became_peered":"2024-09-10T21:51:37.493739+0000","last_unstale":"2024-09-10T21:51:37.496386+0000","last_undegraded":"2024-09-10T21:51:37.496386+0000","last_fullsized":"2024-09-10T21:51:37.496386+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:35.957741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.474336+0000","last_change":"2024-09-10T21:51:37.474336+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.474336+0000","last_undegraded":"2024-09-10T21:51:37.474336+0000","last_fullsized":"2024-09-10T21:51:37.474336+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.491621+0000","last_change":"2024-09-10T21:51:37.489957+0000","last_active":"2024-09-10T21:51:37.491621+0000","last_peered":"2024-09-10T21:51:37.491621+0000","last_clean":"2024-09-10T21:51:37.491621+0000","last_became_active":"2024-09-10T21:51:37.488416+0000","last_became_peered":"2024-09-10T21:51:37.488416+0000","last_unstale":"2024-09-10T21:51:37.491621+0000","last_undegraded":"2024-09-10T21:51:37.491621+0000","last_fullsized":"2024-09-10T21:51:37.491621+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:29.190750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965291+0000","last_change":"2024-09-10T21:51:38.495176+0000","last_active":"2024-09-10T21:51:39.965291+0000","last_peered":"2024-09-10T21:51:39.965291+0000","last_clean":"2024-09-10T21:51:39.965291+0000","last_became_active":"2024-09-10T21:51:38.494857+0000","last_became_peered":"2024-09-10T21:51:38.494857+0000","last_unstale":"2024-09-10T21:51:39.965291+0000","last_undegraded":"2024-09-10T21:51:39.965291+0000","last_fullsized":"2024-09-10T21:51:39.965291+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:32.797143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.473311+0000","last_change":"2024-09-10T21:51:37.467889+0000","last_active":"2024-09-10T21:51:37.473311+0000","last_peered":"2024-09-10T21:51:37.473311+0000","last_clean":"2024-09-10T21:51:37.473311+0000","last_became_active":"2024-09-10T21:51:37.467603+0000","last_became_peered":"2024-09-10T21:51:37.467603+0000","last_unstale":"2024-09-10T21:51:37.473311+0000","last_undegraded":"2024-09-10T21:51:37.473311+0000","last_fullsized":"2024-09-10T21:51:37.473311+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:43:22.274692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":281,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966593+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:51:39.966593+0000","last_peered":"2024-09-10T21:51:39.966593+0000","last_clean":"2024-09-10T21:51:39.966593+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:51:39.966593+0000","last_undegraded":"2024-09-10T21:51:39.966593+0000","last_fullsized":"2024-09-10T21:51:39.966593+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509560,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1372160,"data_stored":590735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1372160,"data_stored":590735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509559,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1372160,"data_stored":590735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509559,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1372160,"data_stored":590735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509561,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":1384448,"data_stored":597010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509559,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37280,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334560,"statfs":{"total":96636764160,"available":96598589440,"internally_reserved":0,"allocated":1978368,"data_stored":1187378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542263,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2011136,"data_stored":1201069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542264,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36744,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335096,"statfs":{"total":96636764160,"available":96599138304,"internally_reserved":0,"allocated":1425408,"data_stored":611767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:51:43.158 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:51:43.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.354+0000 7f6ebe800640 1 -- 172.21.15.7:0/3970707269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ea0005210 msgr2=0x7f6ea000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.354+0000 7f6ebe800640 1 --2- 172.21.15.7:0/3970707269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ea0005210 0x7f6ea000b9c0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f6eb0001f30 tx=0x7f6ea0005830 comp rx=0 tx=0).stop 2024-09-10T21:51:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.354+0000 7f6ebe800640 1 -- 172.21.15.7:0/3970707269 shutdown_connections 2024-09-10T21:51:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.354+0000 7f6ebe800640 1 --2- 172.21.15.7:0/3970707269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ea0005210 0x7f6ea000b9c0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.354+0000 7f6ebe800640 1 -- 172.21.15.7:0/3970707269 >> 172.21.15.7:0/3970707269 conn(0x7f6eb80b1d70 msgr2=0x7f6eb80cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.354+0000 7f6ebe800640 1 -- 172.21.15.7:0/3970707269 shutdown_connections 2024-09-10T21:51:43.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.355+0000 7f6ebe800640 1 -- 172.21.15.7:0/3970707269 wait complete. 2024-09-10T21:51:43.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.356+0000 7f6ebe800640 1 Processor -- start 2024-09-10T21:51:43.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.356+0000 7f6ebe800640 1 -- start start 2024-09-10T21:51:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.357+0000 7f6ebe800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ea0005210 0x7f6eb8139da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.357+0000 7f6ebe800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6eb81e55b0 0x7f6eb813d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.357+0000 7f6ebe800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6eb813dd70 0x7f6eb815acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.357+0000 7f6ebe800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb80b2bc0 con 0x7f6ea0005210 2024-09-10T21:51:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.357+0000 7f6ebe800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6eb80b2a40 con 0x7f6eb813dd70 2024-09-10T21:51:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.357+0000 7f6ebe800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f6eb80b2d40 con 0x7f6eb81e55b0 2024-09-10T21:51:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.358+0000 7f6ebd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ea0005210 0x7f6eb8139da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.358+0000 7f6ebd400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ea0005210 0x7f6eb8139da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:44970/0 (socket says 172.21.15.7:44970) 2024-09-10T21:51:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.358+0000 7f6ebd400640 1 -- 172.21.15.7:0/4046594128 learned_addr learned my addr 172.21.15.7:0/4046594128 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.358+0000 7f6ebca00640 1 --2- 172.21.15.7:0/4046594128 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6eb81e55b0 0x7f6eb813d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.358+0000 7f6ebde00640 1 --2- 172.21.15.7:0/4046594128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6eb813dd70 0x7f6eb815acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:43.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.358+0000 7f6ebd400640 1 -- 172.21.15.7:0/4046594128 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6eb81e55b0 msgr2=0x7f6eb813d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.358+0000 7f6ebd400640 1 --2- 172.21.15.7:0/4046594128 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6eb81e55b0 0x7f6eb813d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.358+0000 7f6ebd400640 1 -- 172.21.15.7:0/4046594128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6eb813dd70 msgr2=0x7f6eb815acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.358+0000 7f6ebd400640 1 --2- 172.21.15.7:0/4046594128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6eb813dd70 0x7f6eb815acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.358+0000 7f6ebd400640 1 -- 172.21.15.7:0/4046594128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6eb81f7a50 con 0x7f6ea0005210 2024-09-10T21:51:43.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.358+0000 7f6ebde00640 1 --2- 172.21.15.7:0/4046594128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6eb813dd70 0x7f6eb815acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:51:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.358+0000 7f6ebd400640 1 --2- 172.21.15.7:0/4046594128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ea0005210 0x7f6eb8139da0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00e550 tx=0x7f6eac00e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.359+0000 7f6eb5600640 1 -- 172.21.15.7:0/4046594128 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac015020 con 0x7f6ea0005210 2024-09-10T21:51:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.359+0000 7f6eb5600640 1 -- 172.21.15.7:0/4046594128 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6eac010980 con 0x7f6ea0005210 2024-09-10T21:51:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.359+0000 7f6eb5600640 1 -- 172.21.15.7:0/4046594128 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6eac010dd0 con 0x7f6ea0005210 2024-09-10T21:51:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.359+0000 7f6ebe800640 1 -- 172.21.15.7:0/4046594128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6eb81f7cf0 con 0x7f6ea0005210 2024-09-10T21:51:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.359+0000 7f6ebe800640 1 -- 172.21.15.7:0/4046594128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6eb81f82b0 con 0x7f6ea0005210 2024-09-10T21:51:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.360+0000 7f6ebe800640 1 -- 172.21.15.7:0/4046594128 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6eb80b4800 con 0x7f6ea0005210 2024-09-10T21:51:43.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.361+0000 7f6eb5600640 1 -- 172.21.15.7:0/4046594128 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6eac011460 con 0x7f6ea0005210 2024-09-10T21:51:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.362+0000 7f6eb5600640 1 --2- 172.21.15.7:0/4046594128 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e98077fa0 0x7f6e9807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.362+0000 7f6eb5600640 1 -- 172.21.15.7:0/4046594128 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5709+0+0 (secure 0 0 0) 0x7f6eac09a570 con 0x7f6ea0005210 2024-09-10T21:51:43.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.363+0000 7f6ebca00640 1 --2- 172.21.15.7:0/4046594128 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e98077fa0 0x7f6e9807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:43.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.363+0000 7f6ebca00640 1 --2- 172.21.15.7:0/4046594128 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e98077fa0 0x7f6e9807a460 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6eb813d5f0 tx=0x7f6ea4005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:43.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.366+0000 7f6eb5600640 1 -- 172.21.15.7:0/4046594128 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6eac01f020 con 0x7f6ea0005210 2024-09-10T21:51:43.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.559+0000 7f6ebe800640 1 -- 172.21.15.7:0/4046594128 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6eb8144760 con 0x7f6e98077fa0 2024-09-10T21:51:43.564 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.564+0000 7f6eb5600640 1 -- 172.21.15.7:0/4046594128 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+248776 (secure 0 0 0) 0x7f6eb8144760 con 0x7f6e98077fa0 2024-09-10T21:51:43.567 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:43.568 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:51:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.568+0000 7f6ebe800640 1 -- 172.21.15.7:0/4046594128 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e98077fa0 msgr2=0x7f6e9807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.568+0000 7f6ebe800640 1 --2- 172.21.15.7:0/4046594128 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e98077fa0 0x7f6e9807a460 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f6eb813d5f0 tx=0x7f6ea4005f90 comp rx=0 tx=0).stop 2024-09-10T21:51:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.568+0000 7f6ebe800640 1 -- 172.21.15.7:0/4046594128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ea0005210 msgr2=0x7f6eb8139da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.568+0000 7f6ebe800640 1 --2- 172.21.15.7:0/4046594128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ea0005210 0x7f6eb8139da0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f6eac00e550 tx=0x7f6eac00e950 comp rx=0 tx=0).stop 2024-09-10T21:51:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.568+0000 7f6ebe800640 1 -- 172.21.15.7:0/4046594128 shutdown_connections 2024-09-10T21:51:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.568+0000 7f6ebe800640 1 --2- 172.21.15.7:0/4046594128 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e98077fa0 0x7f6e9807a460 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.568+0000 7f6ebe800640 1 --2- 172.21.15.7:0/4046594128 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6eb813dd70 0x7f6eb815acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:43.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.568+0000 7f6ebe800640 1 --2- 172.21.15.7:0/4046594128 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6eb81e55b0 0x7f6eb813d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:43.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.568+0000 7f6ebe800640 1 --2- 172.21.15.7:0/4046594128 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6ea0005210 0x7f6eb8139da0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:43.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.568+0000 7f6ebe800640 1 -- 172.21.15.7:0/4046594128 >> 172.21.15.7:0/4046594128 conn(0x7f6eb80b1d70 msgr2=0x7f6eb80b0200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:43.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.568+0000 7f6ebe800640 1 -- 172.21.15.7:0/4046594128 shutdown_connections 2024-09-10T21:51:43.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.568+0000 7f6ebe800640 1 -- 172.21.15.7:0/4046594128 wait complete. 2024-09-10T21:51:43.584 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-09-10T21:51:42.008379+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":104,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":87,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":274004,"kb_used_data":12000,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754700716,"statfs":{"total":773094113280,"available":772813533184,"internally_reserved":0,"allocated":12288000,"data_stored":5960164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.550209"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.485770+0000","last_change":"2024-09-10T21:51:37.485770+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.485770+0000","last_undegraded":"2024-09-10T21:51:37.485770+0000","last_fullsized":"2024-09-10T21:51:37.485770+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.467394+0000","last_change":"2024-09-10T21:51:37.467394+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.467394+0000","last_undegraded":"2024-09-10T21:51:37.467394+0000","last_fullsized":"2024-09-10T21:51:37.467394+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966452+0000","last_change":"2024-09-10T21:51:38.505264+0000","last_active":"2024-09-10T21:51:39.966452+0000","last_peered":"2024-09-10T21:51:39.966452+0000","last_clean":"2024-09-10T21:51:39.966452+0000","last_became_active":"2024-09-10T21:51:38.504121+0000","last_became_peered":"2024-09-10T21:51:38.504121+0000","last_unstale":"2024-09-10T21:51:39.966452+0000","last_undegraded":"2024-09-10T21:51:39.966452+0000","last_fullsized":"2024-09-10T21:51:39.966452+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:32.797143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.489196+0000","last_change":"2024-09-10T21:51:37.489196+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.489196+0000","last_undegraded":"2024-09-10T21:51:37.489196+0000","last_fullsized":"2024-09-10T21:51:37.489196+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.476187+0000","last_change":"2024-09-10T21:51:37.476187+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.476187+0000","last_undegraded":"2024-09-10T21:51:37.476187+0000","last_fullsized":"2024-09-10T21:51:37.476187+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.469812+0000","last_change":"2024-09-10T21:51:37.469812+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.469812+0000","last_undegraded":"2024-09-10T21:51:37.469812+0000","last_fullsized":"2024-09-10T21:51:37.469812+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.465592+0000","last_change":"2024-09-10T21:51:37.465592+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.465592+0000","last_undegraded":"2024-09-10T21:51:37.465592+0000","last_fullsized":"2024-09-10T21:51:37.465592+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966436+0000","last_change":"2024-09-10T21:51:38.498466+0000","last_active":"2024-09-10T21:51:39.966436+0000","last_peered":"2024-09-10T21:51:39.966436+0000","last_clean":"2024-09-10T21:51:39.966436+0000","last_became_active":"2024-09-10T21:51:38.498041+0000","last_became_peered":"2024-09-10T21:51:38.498041+0000","last_unstale":"2024-09-10T21:51:39.966436+0000","last_undegraded":"2024-09-10T21:51:39.966436+0000","last_fullsized":"2024-09-10T21:51:39.966436+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:19.954387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.482918+0000","last_change":"2024-09-10T21:51:37.482918+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.482918+0000","last_undegraded":"2024-09-10T21:51:37.482918+0000","last_fullsized":"2024-09-10T21:51:37.482918+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.482427+0000","last_change":"2024-09-10T21:51:37.482427+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.482427+0000","last_undegraded":"2024-09-10T21:51:37.482427+0000","last_fullsized":"2024-09-10T21:51:37.482427+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.468175+0000","last_change":"2024-09-10T21:51:37.468175+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.468175+0000","last_undegraded":"2024-09-10T21:51:37.468175+0000","last_fullsized":"2024-09-10T21:51:37.468175+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.485246+0000","last_change":"2024-09-10T21:51:37.485246+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.485246+0000","last_undegraded":"2024-09-10T21:51:37.485246+0000","last_fullsized":"2024-09-10T21:51:37.485246+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966404+0000","last_change":"2024-09-10T21:51:38.503338+0000","last_active":"2024-09-10T21:51:39.966404+0000","last_peered":"2024-09-10T21:51:39.966404+0000","last_clean":"2024-09-10T21:51:39.966404+0000","last_became_active":"2024-09-10T21:51:38.501614+0000","last_became_peered":"2024-09-10T21:51:38.501614+0000","last_unstale":"2024-09-10T21:51:39.966404+0000","last_undegraded":"2024-09-10T21:51:39.966404+0000","last_fullsized":"2024-09-10T21:51:39.966404+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:10.529060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966266+0000","last_change":"2024-09-10T21:51:38.498967+0000","last_active":"2024-09-10T21:51:39.966266+0000","last_peered":"2024-09-10T21:51:39.966266+0000","last_clean":"2024-09-10T21:51:39.966266+0000","last_became_active":"2024-09-10T21:51:38.498641+0000","last_became_peered":"2024-09-10T21:51:38.498641+0000","last_unstale":"2024-09-10T21:51:39.966266+0000","last_undegraded":"2024-09-10T21:51:39.966266+0000","last_fullsized":"2024-09-10T21:51:39.966266+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:41.424015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.469393+0000","last_change":"2024-09-10T21:51:37.469393+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.469393+0000","last_undegraded":"2024-09-10T21:51:37.469393+0000","last_fullsized":"2024-09-10T21:51:37.469393+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.481943+0000","last_change":"2024-09-10T21:51:37.481943+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.481943+0000","last_undegraded":"2024-09-10T21:51:37.481943+0000","last_fullsized":"2024-09-10T21:51:37.481943+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.468661+0000","last_change":"2024-09-10T21:51:37.468661+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.468661+0000","last_undegraded":"2024-09-10T21:51:37.468661+0000","last_fullsized":"2024-09-10T21:51:37.468661+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.479101+0000","last_change":"2024-09-10T21:51:37.479101+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.479101+0000","last_undegraded":"2024-09-10T21:51:37.479101+0000","last_fullsized":"2024-09-10T21:51:37.479101+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966459+0000","last_change":"2024-09-10T21:51:38.506319+0000","last_active":"2024-09-10T21:51:39.966459+0000","last_peered":"2024-09-10T21:51:39.966459+0000","last_clean":"2024-09-10T21:51:39.966459+0000","last_became_active":"2024-09-10T21:51:38.506041+0000","last_became_peered":"2024-09-10T21:51:38.506041+0000","last_unstale":"2024-09-10T21:51:39.966459+0000","last_undegraded":"2024-09-10T21:51:39.966459+0000","last_fullsized":"2024-09-10T21:51:39.966459+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:33.484489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.467188+0000","last_change":"2024-09-10T21:51:37.467188+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.467188+0000","last_undegraded":"2024-09-10T21:51:37.467188+0000","last_fullsized":"2024-09-10T21:51:37.467188+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.469235+0000","last_change":"2024-09-10T21:51:37.469235+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.469235+0000","last_undegraded":"2024-09-10T21:51:37.469235+0000","last_fullsized":"2024-09-10T21:51:37.469235+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965676+0000","last_change":"2024-09-10T21:51:38.495309+0000","last_active":"2024-09-10T21:51:39.965676+0000","last_peered":"2024-09-10T21:51:39.965676+0000","last_clean":"2024-09-10T21:51:39.965676+0000","last_became_active":"2024-09-10T21:51:38.494998+0000","last_became_peered":"2024-09-10T21:51:38.494998+0000","last_unstale":"2024-09-10T21:51:39.965676+0000","last_undegraded":"2024-09-10T21:51:39.965676+0000","last_fullsized":"2024-09-10T21:51:39.965676+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:41.427352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.466079+0000","last_change":"2024-09-10T21:51:37.466079+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.466079+0000","last_undegraded":"2024-09-10T21:51:37.466079+0000","last_fullsized":"2024-09-10T21:51:37.466079+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965229+0000","last_change":"2024-09-10T21:51:38.505466+0000","last_active":"2024-09-10T21:51:39.965229+0000","last_peered":"2024-09-10T21:51:39.965229+0000","last_clean":"2024-09-10T21:51:39.965229+0000","last_became_active":"2024-09-10T21:51:38.505053+0000","last_became_peered":"2024-09-10T21:51:38.505053+0000","last_unstale":"2024-09-10T21:51:39.965229+0000","last_undegraded":"2024-09-10T21:51:39.965229+0000","last_fullsized":"2024-09-10T21:51:39.965229+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:36:29.403651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.467357+0000","last_change":"2024-09-10T21:51:37.467357+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.467357+0000","last_undegraded":"2024-09-10T21:51:37.467357+0000","last_fullsized":"2024-09-10T21:51:37.467357+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.478229+0000","last_change":"2024-09-10T21:51:37.478229+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.478229+0000","last_undegraded":"2024-09-10T21:51:37.478229+0000","last_fullsized":"2024-09-10T21:51:37.478229+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.473593+0000","last_change":"2024-09-10T21:51:37.473593+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.473593+0000","last_undegraded":"2024-09-10T21:51:37.473593+0000","last_fullsized":"2024-09-10T21:51:37.473593+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965771+0000","last_change":"2024-09-10T21:51:38.496708+0000","last_active":"2024-09-10T21:51:39.965771+0000","last_peered":"2024-09-10T21:51:39.965771+0000","last_clean":"2024-09-10T21:51:39.965771+0000","last_became_active":"2024-09-10T21:51:38.496402+0000","last_became_peered":"2024-09-10T21:51:38.496402+0000","last_unstale":"2024-09-10T21:51:39.965771+0000","last_undegraded":"2024-09-10T21:51:39.965771+0000","last_fullsized":"2024-09-10T21:51:39.965771+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:40.305224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.474613+0000","last_change":"2024-09-10T21:51:37.474613+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.474613+0000","last_undegraded":"2024-09-10T21:51:37.474613+0000","last_fullsized":"2024-09-10T21:51:37.474613+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.463487+0000","last_change":"2024-09-10T21:51:37.463487+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.463487+0000","last_undegraded":"2024-09-10T21:51:37.463487+0000","last_fullsized":"2024-09-10T21:51:37.463487+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.478186+0000","last_change":"2024-09-10T21:51:37.478186+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.478186+0000","last_undegraded":"2024-09-10T21:51:37.478186+0000","last_fullsized":"2024-09-10T21:51:37.478186+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.468954+0000","last_change":"2024-09-10T21:51:37.468954+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.468954+0000","last_undegraded":"2024-09-10T21:51:37.468954+0000","last_fullsized":"2024-09-10T21:51:37.468954+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.477280+0000","last_change":"2024-09-10T21:51:37.477280+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.477280+0000","last_undegraded":"2024-09-10T21:51:37.477280+0000","last_fullsized":"2024-09-10T21:51:37.477280+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.474881+0000","last_change":"2024-09-10T21:51:37.472999+0000","last_active":"2024-09-10T21:51:37.474881+0000","last_peered":"2024-09-10T21:51:37.474881+0000","last_clean":"2024-09-10T21:51:37.474881+0000","last_became_active":"2024-09-10T21:51:37.472471+0000","last_became_peered":"2024-09-10T21:51:37.472471+0000","last_unstale":"2024-09-10T21:51:37.474881+0000","last_undegraded":"2024-09-10T21:51:37.474881+0000","last_fullsized":"2024-09-10T21:51:37.474881+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:30.555990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.480358+0000","last_change":"2024-09-10T21:51:37.480358+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.480358+0000","last_undegraded":"2024-09-10T21:51:37.480358+0000","last_fullsized":"2024-09-10T21:51:37.480358+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966096+0000","last_change":"2024-09-10T21:51:37.481369+0000","last_active":"2024-09-10T21:51:39.966096+0000","last_peered":"2024-09-10T21:51:39.966096+0000","last_clean":"2024-09-10T21:51:39.966096+0000","last_became_active":"2024-09-10T21:51:37.481061+0000","last_became_peered":"2024-09-10T21:51:37.481061+0000","last_unstale":"2024-09-10T21:51:39.966096+0000","last_undegraded":"2024-09-10T21:51:39.966096+0000","last_fullsized":"2024-09-10T21:51:39.966096+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:48.767875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.473226+0000","last_change":"2024-09-10T21:51:37.473226+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.473226+0000","last_undegraded":"2024-09-10T21:51:37.473226+0000","last_fullsized":"2024-09-10T21:51:37.473226+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.497767+0000","last_change":"2024-09-10T21:51:37.495401+0000","last_active":"2024-09-10T21:51:37.497767+0000","last_peered":"2024-09-10T21:51:37.497767+0000","last_clean":"2024-09-10T21:51:37.497767+0000","last_became_active":"2024-09-10T21:51:37.495129+0000","last_became_peered":"2024-09-10T21:51:37.495129+0000","last_unstale":"2024-09-10T21:51:37.497767+0000","last_undegraded":"2024-09-10T21:51:37.497767+0000","last_fullsized":"2024-09-10T21:51:37.497767+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:09:58.176252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966641+0000","last_change":"2024-09-10T21:51:38.507168+0000","last_active":"2024-09-10T21:51:39.966641+0000","last_peered":"2024-09-10T21:51:39.966641+0000","last_clean":"2024-09-10T21:51:39.966641+0000","last_became_active":"2024-09-10T21:51:38.506850+0000","last_became_peered":"2024-09-10T21:51:38.506850+0000","last_unstale":"2024-09-10T21:51:39.966641+0000","last_undegraded":"2024-09-10T21:51:39.966641+0000","last_fullsized":"2024-09-10T21:51:39.966641+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:40.305224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.497488+0000","last_change":"2024-09-10T21:51:37.495072+0000","last_active":"2024-09-10T21:51:37.497488+0000","last_peered":"2024-09-10T21:51:37.497488+0000","last_clean":"2024-09-10T21:51:37.497488+0000","last_became_active":"2024-09-10T21:51:37.494778+0000","last_became_peered":"2024-09-10T21:51:37.494778+0000","last_unstale":"2024-09-10T21:51:37.497488+0000","last_undegraded":"2024-09-10T21:51:37.497488+0000","last_fullsized":"2024-09-10T21:51:37.497488+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:40:18.574835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.476588+0000","last_change":"2024-09-10T21:51:37.476588+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.476588+0000","last_undegraded":"2024-09-10T21:51:37.476588+0000","last_fullsized":"2024-09-10T21:51:37.476588+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.476242+0000","last_change":"2024-09-10T21:51:37.474801+0000","last_active":"2024-09-10T21:51:37.476242+0000","last_peered":"2024-09-10T21:51:37.476242+0000","last_clean":"2024-09-10T21:51:37.476242+0000","last_became_active":"2024-09-10T21:51:37.474455+0000","last_became_peered":"2024-09-10T21:51:37.474455+0000","last_unstale":"2024-09-10T21:51:37.476242+0000","last_undegraded":"2024-09-10T21:51:37.476242+0000","last_fullsized":"2024-09-10T21:51:37.476242+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:00.575937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.474186+0000","last_change":"2024-09-10T21:51:37.474186+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.474186+0000","last_undegraded":"2024-09-10T21:51:37.474186+0000","last_fullsized":"2024-09-10T21:51:37.474186+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965968+0000","last_change":"2024-09-10T21:51:37.481177+0000","last_active":"2024-09-10T21:51:39.965968+0000","last_peered":"2024-09-10T21:51:39.965968+0000","last_clean":"2024-09-10T21:51:39.965968+0000","last_became_active":"2024-09-10T21:51:37.480878+0000","last_became_peered":"2024-09-10T21:51:37.480878+0000","last_unstale":"2024-09-10T21:51:39.965968+0000","last_undegraded":"2024-09-10T21:51:39.965968+0000","last_fullsized":"2024-09-10T21:51:39.965968+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:39.644340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.477970+0000","last_change":"2024-09-10T21:51:37.477970+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.477970+0000","last_undegraded":"2024-09-10T21:51:37.477970+0000","last_fullsized":"2024-09-10T21:51:37.477970+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.487128+0000","last_change":"2024-09-10T21:51:37.486466+0000","last_active":"2024-09-10T21:51:37.487128+0000","last_peered":"2024-09-10T21:51:37.487128+0000","last_clean":"2024-09-10T21:51:37.487128+0000","last_became_active":"2024-09-10T21:51:37.486160+0000","last_became_peered":"2024-09-10T21:51:37.486160+0000","last_unstale":"2024-09-10T21:51:37.487128+0000","last_undegraded":"2024-09-10T21:51:37.487128+0000","last_fullsized":"2024-09-10T21:51:37.487128+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:38.793976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.463935+0000","last_change":"2024-09-10T21:51:37.463935+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.463935+0000","last_undegraded":"2024-09-10T21:51:37.463935+0000","last_fullsized":"2024-09-10T21:51:37.463935+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.491594+0000","last_change":"2024-09-10T21:51:37.489582+0000","last_active":"2024-09-10T21:51:37.491594+0000","last_peered":"2024-09-10T21:51:37.491594+0000","last_clean":"2024-09-10T21:51:37.491594+0000","last_became_active":"2024-09-10T21:51:37.487889+0000","last_became_peered":"2024-09-10T21:51:37.487889+0000","last_unstale":"2024-09-10T21:51:37.491594+0000","last_undegraded":"2024-09-10T21:51:37.491594+0000","last_fullsized":"2024-09-10T21:51:37.491594+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:53.102794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.463971+0000","last_change":"2024-09-10T21:51:37.463971+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.463971+0000","last_undegraded":"2024-09-10T21:51:37.463971+0000","last_fullsized":"2024-09-10T21:51:37.463971+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.496341+0000","last_change":"2024-09-10T21:51:37.493056+0000","last_active":"2024-09-10T21:51:37.496341+0000","last_peered":"2024-09-10T21:51:37.496341+0000","last_clean":"2024-09-10T21:51:37.496341+0000","last_became_active":"2024-09-10T21:51:37.492756+0000","last_became_peered":"2024-09-10T21:51:37.492756+0000","last_unstale":"2024-09-10T21:51:37.496341+0000","last_undegraded":"2024-09-10T21:51:37.496341+0000","last_fullsized":"2024-09-10T21:51:37.496341+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:33.347129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.473160+0000","last_change":"2024-09-10T21:51:37.473160+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.473160+0000","last_undegraded":"2024-09-10T21:51:37.473160+0000","last_fullsized":"2024-09-10T21:51:37.473160+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966160+0000","last_change":"2024-09-10T21:51:37.487117+0000","last_active":"2024-09-10T21:51:39.966160+0000","last_peered":"2024-09-10T21:51:39.966160+0000","last_clean":"2024-09-10T21:51:39.966160+0000","last_became_active":"2024-09-10T21:51:37.484685+0000","last_became_peered":"2024-09-10T21:51:37.484685+0000","last_unstale":"2024-09-10T21:51:39.966160+0000","last_undegraded":"2024-09-10T21:51:39.966160+0000","last_fullsized":"2024-09-10T21:51:39.966160+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:13.540834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965991+0000","last_change":"2024-09-10T21:51:38.500232+0000","last_active":"2024-09-10T21:51:39.965991+0000","last_peered":"2024-09-10T21:51:39.965991+0000","last_clean":"2024-09-10T21:51:39.965991+0000","last_became_active":"2024-09-10T21:51:38.499959+0000","last_became_peered":"2024-09-10T21:51:38.499959+0000","last_unstale":"2024-09-10T21:51:39.965991+0000","last_undegraded":"2024-09-10T21:51:39.965991+0000","last_fullsized":"2024-09-10T21:51:39.965991+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:35.171265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.484868+0000","last_change":"2024-09-10T21:51:37.483939+0000","last_active":"2024-09-10T21:51:37.484868+0000","last_peered":"2024-09-10T21:51:37.484868+0000","last_clean":"2024-09-10T21:51:37.484868+0000","last_became_active":"2024-09-10T21:51:37.483023+0000","last_became_peered":"2024-09-10T21:51:37.483023+0000","last_unstale":"2024-09-10T21:51:37.484868+0000","last_undegraded":"2024-09-10T21:51:37.484868+0000","last_fullsized":"2024-09-10T21:51:37.484868+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:42.259904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.464945+0000","last_change":"2024-09-10T21:51:37.464945+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.464945+0000","last_undegraded":"2024-09-10T21:51:37.464945+0000","last_fullsized":"2024-09-10T21:51:37.464945+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.492264+0000","last_change":"2024-09-10T21:51:37.490837+0000","last_active":"2024-09-10T21:51:37.492264+0000","last_peered":"2024-09-10T21:51:37.492264+0000","last_clean":"2024-09-10T21:51:37.492264+0000","last_became_active":"2024-09-10T21:51:37.489661+0000","last_became_peered":"2024-09-10T21:51:37.489661+0000","last_unstale":"2024-09-10T21:51:37.492264+0000","last_undegraded":"2024-09-10T21:51:37.492264+0000","last_fullsized":"2024-09-10T21:51:37.492264+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:58.486726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966089+0000","last_change":"2024-09-10T21:51:38.499244+0000","last_active":"2024-09-10T21:51:39.966089+0000","last_peered":"2024-09-10T21:51:39.966089+0000","last_clean":"2024-09-10T21:51:39.966089+0000","last_became_active":"2024-09-10T21:51:38.498965+0000","last_became_peered":"2024-09-10T21:51:38.498965+0000","last_unstale":"2024-09-10T21:51:39.966089+0000","last_undegraded":"2024-09-10T21:51:39.966089+0000","last_fullsized":"2024-09-10T21:51:39.966089+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:12.629720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.488507+0000","last_change":"2024-09-10T21:51:37.480818+0000","last_active":"2024-09-10T21:51:37.488507+0000","last_peered":"2024-09-10T21:51:37.488507+0000","last_clean":"2024-09-10T21:51:37.488507+0000","last_became_active":"2024-09-10T21:51:37.480562+0000","last_became_peered":"2024-09-10T21:51:37.480562+0000","last_unstale":"2024-09-10T21:51:37.488507+0000","last_undegraded":"2024-09-10T21:51:37.488507+0000","last_fullsized":"2024-09-10T21:51:37.488507+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:05.569050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965893+0000","last_change":"2024-09-10T21:51:38.504740+0000","last_active":"2024-09-10T21:51:39.965893+0000","last_peered":"2024-09-10T21:51:39.965893+0000","last_clean":"2024-09-10T21:51:39.965893+0000","last_became_active":"2024-09-10T21:51:38.504214+0000","last_became_peered":"2024-09-10T21:51:38.504214+0000","last_unstale":"2024-09-10T21:51:39.965893+0000","last_undegraded":"2024-09-10T21:51:39.965893+0000","last_fullsized":"2024-09-10T21:51:39.965893+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:47.805058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.492594+0000","last_change":"2024-09-10T21:51:37.491755+0000","last_active":"2024-09-10T21:51:37.492594+0000","last_peered":"2024-09-10T21:51:37.492594+0000","last_clean":"2024-09-10T21:51:37.492594+0000","last_became_active":"2024-09-10T21:51:37.491472+0000","last_became_peered":"2024-09-10T21:51:37.491472+0000","last_unstale":"2024-09-10T21:51:37.492594+0000","last_undegraded":"2024-09-10T21:51:37.492594+0000","last_fullsized":"2024-09-10T21:51:37.492594+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:05:09.126767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965395+0000","last_change":"2024-09-10T21:51:38.500521+0000","last_active":"2024-09-10T21:51:39.965395+0000","last_peered":"2024-09-10T21:51:39.965395+0000","last_clean":"2024-09-10T21:51:39.965395+0000","last_became_active":"2024-09-10T21:51:38.500234+0000","last_became_peered":"2024-09-10T21:51:38.500234+0000","last_unstale":"2024-09-10T21:51:39.965395+0000","last_undegraded":"2024-09-10T21:51:39.965395+0000","last_fullsized":"2024-09-10T21:51:39.965395+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:28.278479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.496082+0000","last_change":"2024-09-10T21:51:37.492464+0000","last_active":"2024-09-10T21:51:37.496082+0000","last_peered":"2024-09-10T21:51:37.496082+0000","last_clean":"2024-09-10T21:51:37.496082+0000","last_became_active":"2024-09-10T21:51:37.492162+0000","last_became_peered":"2024-09-10T21:51:37.492162+0000","last_unstale":"2024-09-10T21:51:37.496082+0000","last_undegraded":"2024-09-10T21:51:37.496082+0000","last_fullsized":"2024-09-10T21:51:37.496082+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:22:43.699147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.475624+0000","last_change":"2024-09-10T21:51:37.475624+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.475624+0000","last_undegraded":"2024-09-10T21:51:37.475624+0000","last_fullsized":"2024-09-10T21:51:37.475624+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.491463+0000","last_change":"2024-09-10T21:51:37.468073+0000","last_active":"2024-09-10T21:51:37.491463+0000","last_peered":"2024-09-10T21:51:37.491463+0000","last_clean":"2024-09-10T21:51:37.491463+0000","last_became_active":"2024-09-10T21:51:37.467716+0000","last_became_peered":"2024-09-10T21:51:37.467716+0000","last_unstale":"2024-09-10T21:51:37.491463+0000","last_undegraded":"2024-09-10T21:51:37.491463+0000","last_fullsized":"2024-09-10T21:51:37.491463+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:36.082631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.473936+0000","last_change":"2024-09-10T21:51:37.473936+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.473936+0000","last_undegraded":"2024-09-10T21:51:37.473936+0000","last_fullsized":"2024-09-10T21:51:37.473936+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.496000+0000","last_change":"2024-09-10T21:51:37.485673+0000","last_active":"2024-09-10T21:51:37.496000+0000","last_peered":"2024-09-10T21:51:37.496000+0000","last_clean":"2024-09-10T21:51:37.496000+0000","last_became_active":"2024-09-10T21:51:37.485132+0000","last_became_peered":"2024-09-10T21:51:37.485132+0000","last_unstale":"2024-09-10T21:51:37.496000+0000","last_undegraded":"2024-09-10T21:51:37.496000+0000","last_fullsized":"2024-09-10T21:51:37.496000+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:30:53.853582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966440+0000","last_change":"2024-09-10T21:51:38.505872+0000","last_active":"2024-09-10T21:51:39.966440+0000","last_peered":"2024-09-10T21:51:39.966440+0000","last_clean":"2024-09-10T21:51:39.966440+0000","last_became_active":"2024-09-10T21:51:38.505488+0000","last_became_peered":"2024-09-10T21:51:38.505488+0000","last_unstale":"2024-09-10T21:51:39.966440+0000","last_undegraded":"2024-09-10T21:51:39.966440+0000","last_fullsized":"2024-09-10T21:51:39.966440+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:29.689596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.485861+0000","last_change":"2024-09-10T21:51:37.485228+0000","last_active":"2024-09-10T21:51:37.485861+0000","last_peered":"2024-09-10T21:51:37.485861+0000","last_clean":"2024-09-10T21:51:37.485861+0000","last_became_active":"2024-09-10T21:51:37.484980+0000","last_became_peered":"2024-09-10T21:51:37.484980+0000","last_unstale":"2024-09-10T21:51:37.485861+0000","last_undegraded":"2024-09-10T21:51:37.485861+0000","last_fullsized":"2024-09-10T21:51:37.485861+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:24.448659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966710+0000","last_change":"2024-09-10T21:51:38.504879+0000","last_active":"2024-09-10T21:51:39.966710+0000","last_peered":"2024-09-10T21:51:39.966710+0000","last_clean":"2024-09-10T21:51:39.966710+0000","last_became_active":"2024-09-10T21:51:38.504560+0000","last_became_peered":"2024-09-10T21:51:38.504560+0000","last_unstale":"2024-09-10T21:51:39.966710+0000","last_undegraded":"2024-09-10T21:51:39.966710+0000","last_fullsized":"2024-09-10T21:51:39.966710+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:47.805058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.473267+0000","last_change":"2024-09-10T21:51:37.469638+0000","last_active":"2024-09-10T21:51:37.473267+0000","last_peered":"2024-09-10T21:51:37.473267+0000","last_clean":"2024-09-10T21:51:37.473267+0000","last_became_active":"2024-09-10T21:51:37.467556+0000","last_became_peered":"2024-09-10T21:51:37.467556+0000","last_unstale":"2024-09-10T21:51:37.473267+0000","last_undegraded":"2024-09-10T21:51:37.473267+0000","last_fullsized":"2024-09-10T21:51:37.473267+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:11.474396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966529+0000","last_change":"2024-09-10T21:51:38.504149+0000","last_active":"2024-09-10T21:51:39.966529+0000","last_peered":"2024-09-10T21:51:39.966529+0000","last_clean":"2024-09-10T21:51:39.966529+0000","last_became_active":"2024-09-10T21:51:38.503873+0000","last_became_peered":"2024-09-10T21:51:38.503873+0000","last_unstale":"2024-09-10T21:51:39.966529+0000","last_undegraded":"2024-09-10T21:51:39.966529+0000","last_fullsized":"2024-09-10T21:51:39.966529+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:38.756786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.492556+0000","last_change":"2024-09-10T21:51:37.488302+0000","last_active":"2024-09-10T21:51:37.492556+0000","last_peered":"2024-09-10T21:51:37.492556+0000","last_clean":"2024-09-10T21:51:37.492556+0000","last_became_active":"2024-09-10T21:51:37.488054+0000","last_became_peered":"2024-09-10T21:51:37.488054+0000","last_unstale":"2024-09-10T21:51:37.492556+0000","last_undegraded":"2024-09-10T21:51:37.492556+0000","last_fullsized":"2024-09-10T21:51:37.492556+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:13.280288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.471734+0000","last_change":"2024-09-10T21:51:37.471734+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.471734+0000","last_undegraded":"2024-09-10T21:51:37.471734+0000","last_fullsized":"2024-09-10T21:51:37.471734+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.497783+0000","last_change":"2024-09-10T21:51:37.495655+0000","last_active":"2024-09-10T21:51:37.497783+0000","last_peered":"2024-09-10T21:51:37.497783+0000","last_clean":"2024-09-10T21:51:37.497783+0000","last_became_active":"2024-09-10T21:51:37.495383+0000","last_became_peered":"2024-09-10T21:51:37.495383+0000","last_unstale":"2024-09-10T21:51:37.497783+0000","last_undegraded":"2024-09-10T21:51:37.497783+0000","last_fullsized":"2024-09-10T21:51:37.497783+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:45:38.126366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966363+0000","last_change":"2024-09-10T21:51:38.498184+0000","last_active":"2024-09-10T21:51:39.966363+0000","last_peered":"2024-09-10T21:51:39.966363+0000","last_clean":"2024-09-10T21:51:39.966363+0000","last_became_active":"2024-09-10T21:51:38.497840+0000","last_became_peered":"2024-09-10T21:51:38.497840+0000","last_unstale":"2024-09-10T21:51:39.966363+0000","last_undegraded":"2024-09-10T21:51:39.966363+0000","last_fullsized":"2024-09-10T21:51:39.966363+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:50.132177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.495778+0000","last_change":"2024-09-10T21:51:37.493629+0000","last_active":"2024-09-10T21:51:37.495778+0000","last_peered":"2024-09-10T21:51:37.495778+0000","last_clean":"2024-09-10T21:51:37.495778+0000","last_became_active":"2024-09-10T21:51:37.493347+0000","last_became_peered":"2024-09-10T21:51:37.493347+0000","last_unstale":"2024-09-10T21:51:37.495778+0000","last_undegraded":"2024-09-10T21:51:37.495778+0000","last_fullsized":"2024-09-10T21:51:37.495778+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:47:54.092525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965539+0000","last_change":"2024-09-10T21:51:38.497499+0000","last_active":"2024-09-10T21:51:39.965539+0000","last_peered":"2024-09-10T21:51:39.965539+0000","last_clean":"2024-09-10T21:51:39.965539+0000","last_became_active":"2024-09-10T21:51:38.494172+0000","last_became_peered":"2024-09-10T21:51:38.494172+0000","last_unstale":"2024-09-10T21:51:39.965539+0000","last_undegraded":"2024-09-10T21:51:39.965539+0000","last_fullsized":"2024-09-10T21:51:39.965539+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:38.756786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.491642+0000","last_change":"2024-09-10T21:51:37.490268+0000","last_active":"2024-09-10T21:51:37.491642+0000","last_peered":"2024-09-10T21:51:37.491642+0000","last_clean":"2024-09-10T21:51:37.491642+0000","last_became_active":"2024-09-10T21:51:37.488772+0000","last_became_peered":"2024-09-10T21:51:37.488772+0000","last_unstale":"2024-09-10T21:51:37.491642+0000","last_undegraded":"2024-09-10T21:51:37.491642+0000","last_fullsized":"2024-09-10T21:51:37.491642+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:58:18.953120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.470749+0000","last_change":"2024-09-10T21:51:37.470749+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.470749+0000","last_undegraded":"2024-09-10T21:51:37.470749+0000","last_fullsized":"2024-09-10T21:51:37.470749+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966234+0000","last_change":"2024-09-10T21:51:37.493424+0000","last_active":"2024-09-10T21:51:39.966234+0000","last_peered":"2024-09-10T21:51:39.966234+0000","last_clean":"2024-09-10T21:51:39.966234+0000","last_became_active":"2024-09-10T21:51:37.492964+0000","last_became_peered":"2024-09-10T21:51:37.492964+0000","last_unstale":"2024-09-10T21:51:39.966234+0000","last_undegraded":"2024-09-10T21:51:39.966234+0000","last_fullsized":"2024-09-10T21:51:39.966234+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:56.310705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.472197+0000","last_change":"2024-09-10T21:51:37.472197+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.472197+0000","last_undegraded":"2024-09-10T21:51:37.472197+0000","last_fullsized":"2024-09-10T21:51:37.472197+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.482551+0000","last_change":"2024-09-10T21:51:37.481469+0000","last_active":"2024-09-10T21:51:37.482551+0000","last_peered":"2024-09-10T21:51:37.482551+0000","last_clean":"2024-09-10T21:51:37.482551+0000","last_became_active":"2024-09-10T21:51:37.481196+0000","last_became_peered":"2024-09-10T21:51:37.481196+0000","last_unstale":"2024-09-10T21:51:37.482551+0000","last_undegraded":"2024-09-10T21:51:37.482551+0000","last_fullsized":"2024-09-10T21:51:37.482551+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:10.385869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.471647+0000","last_change":"2024-09-10T21:51:37.471647+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.471647+0000","last_undegraded":"2024-09-10T21:51:37.471647+0000","last_fullsized":"2024-09-10T21:51:37.471647+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.492284+0000","last_change":"2024-09-10T21:51:37.490280+0000","last_active":"2024-09-10T21:51:37.492284+0000","last_peered":"2024-09-10T21:51:37.492284+0000","last_clean":"2024-09-10T21:51:37.492284+0000","last_became_active":"2024-09-10T21:51:37.489979+0000","last_became_peered":"2024-09-10T21:51:37.489979+0000","last_unstale":"2024-09-10T21:51:37.492284+0000","last_undegraded":"2024-09-10T21:51:37.492284+0000","last_fullsized":"2024-09-10T21:51:37.492284+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:09:51.282814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.475986+0000","last_change":"2024-09-10T21:51:37.475986+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.475986+0000","last_undegraded":"2024-09-10T21:51:37.475986+0000","last_fullsized":"2024-09-10T21:51:37.475986+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966131+0000","last_change":"2024-09-10T21:51:37.496547+0000","last_active":"2024-09-10T21:51:39.966131+0000","last_peered":"2024-09-10T21:51:39.966131+0000","last_clean":"2024-09-10T21:51:39.966131+0000","last_became_active":"2024-09-10T21:51:37.495965+0000","last_became_peered":"2024-09-10T21:51:37.495965+0000","last_unstale":"2024-09-10T21:51:39.966131+0000","last_undegraded":"2024-09-10T21:51:39.966131+0000","last_fullsized":"2024-09-10T21:51:39.966131+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:45.517218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.466542+0000","last_change":"2024-09-10T21:51:37.466542+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.466542+0000","last_undegraded":"2024-09-10T21:51:37.466542+0000","last_fullsized":"2024-09-10T21:51:37.466542+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.478377+0000","last_change":"2024-09-10T21:51:37.477655+0000","last_active":"2024-09-10T21:51:37.478377+0000","last_peered":"2024-09-10T21:51:37.478377+0000","last_clean":"2024-09-10T21:51:37.478377+0000","last_became_active":"2024-09-10T21:51:37.477156+0000","last_became_peered":"2024-09-10T21:51:37.477156+0000","last_unstale":"2024-09-10T21:51:37.478377+0000","last_undegraded":"2024-09-10T21:51:37.478377+0000","last_fullsized":"2024-09-10T21:51:37.478377+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:07.719047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.473775+0000","last_change":"2024-09-10T21:51:37.473775+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.473775+0000","last_undegraded":"2024-09-10T21:51:37.473775+0000","last_fullsized":"2024-09-10T21:51:37.473775+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.480768+0000","last_change":"2024-09-10T21:51:37.478410+0000","last_active":"2024-09-10T21:51:37.480768+0000","last_peered":"2024-09-10T21:51:37.480768+0000","last_clean":"2024-09-10T21:51:37.480768+0000","last_became_active":"2024-09-10T21:51:37.478141+0000","last_became_peered":"2024-09-10T21:51:37.478141+0000","last_unstale":"2024-09-10T21:51:37.480768+0000","last_undegraded":"2024-09-10T21:51:37.480768+0000","last_fullsized":"2024-09-10T21:51:37.480768+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:31:40.813722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966600+0000","last_change":"2024-09-10T21:51:38.501070+0000","last_active":"2024-09-10T21:51:39.966600+0000","last_peered":"2024-09-10T21:51:39.966600+0000","last_clean":"2024-09-10T21:51:39.966600+0000","last_became_active":"2024-09-10T21:51:38.500807+0000","last_became_peered":"2024-09-10T21:51:38.500807+0000","last_unstale":"2024-09-10T21:51:39.966600+0000","last_undegraded":"2024-09-10T21:51:39.966600+0000","last_fullsized":"2024-09-10T21:51:39.966600+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:41.427352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.496386+0000","last_change":"2024-09-10T21:51:37.494039+0000","last_active":"2024-09-10T21:51:37.496386+0000","last_peered":"2024-09-10T21:51:37.496386+0000","last_clean":"2024-09-10T21:51:37.496386+0000","last_became_active":"2024-09-10T21:51:37.493739+0000","last_became_peered":"2024-09-10T21:51:37.493739+0000","last_unstale":"2024-09-10T21:51:37.496386+0000","last_undegraded":"2024-09-10T21:51:37.496386+0000","last_fullsized":"2024-09-10T21:51:37.496386+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:22:35.957741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":1,"reported_epoch":112,"state":"creating+peering","last_fresh":"2024-09-10T21:51:37.474336+0000","last_change":"2024-09-10T21:51:37.474336+0000","last_active":"2024-09-10T21:51:37.456491+0000","last_peered":"2024-09-10T21:51:37.456491+0000","last_clean":"2024-09-10T21:51:37.456491+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-10T21:51:37.474336+0000","last_undegraded":"2024-09-10T21:51:37.474336+0000","last_fullsized":"2024-09-10T21:51:37.474336+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.491621+0000","last_change":"2024-09-10T21:51:37.489957+0000","last_active":"2024-09-10T21:51:37.491621+0000","last_peered":"2024-09-10T21:51:37.491621+0000","last_clean":"2024-09-10T21:51:37.491621+0000","last_became_active":"2024-09-10T21:51:37.488416+0000","last_became_peered":"2024-09-10T21:51:37.488416+0000","last_unstale":"2024-09-10T21:51:37.491621+0000","last_undegraded":"2024-09-10T21:51:37.491621+0000","last_fullsized":"2024-09-10T21:51:37.491621+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:16:29.190750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965291+0000","last_change":"2024-09-10T21:51:38.495176+0000","last_active":"2024-09-10T21:51:39.965291+0000","last_peered":"2024-09-10T21:51:39.965291+0000","last_clean":"2024-09-10T21:51:39.965291+0000","last_became_active":"2024-09-10T21:51:38.494857+0000","last_became_peered":"2024-09-10T21:51:38.494857+0000","last_unstale":"2024-09-10T21:51:39.965291+0000","last_undegraded":"2024-09-10T21:51:39.965291+0000","last_fullsized":"2024-09-10T21:51:39.965291+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:32.797143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":10,"reported_epoch":112,"state":"active+clean","last_fresh":"2024-09-10T21:51:37.473311+0000","last_change":"2024-09-10T21:51:37.467889+0000","last_active":"2024-09-10T21:51:37.473311+0000","last_peered":"2024-09-10T21:51:37.473311+0000","last_clean":"2024-09-10T21:51:37.473311+0000","last_became_active":"2024-09-10T21:51:37.467603+0000","last_became_peered":"2024-09-10T21:51:37.467603+0000","last_unstale":"2024-09-10T21:51:37.473311+0000","last_undegraded":"2024-09-10T21:51:37.473311+0000","last_fullsized":"2024-09-10T21:51:37.473311+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:43:22.274692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":281,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966593+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:51:39.966593+0000","last_peered":"2024-09-10T21:51:39.966593+0000","last_clean":"2024-09-10T21:51:39.966593+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:51:39.966593+0000","last_undegraded":"2024-09-10T21:51:39.966593+0000","last_fullsized":"2024-09-10T21:51:39.966593+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509560,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1372160,"data_stored":590735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509560,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1372160,"data_stored":590735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509559,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32528,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339312,"statfs":{"total":96636764160,"available":96603455488,"internally_reserved":0,"allocated":1372160,"data_stored":590735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509559,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32532,"kb_used_data":1340,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339308,"statfs":{"total":96636764160,"available":96603451392,"internally_reserved":0,"allocated":1372160,"data_stored":590735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509561,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36704,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335136,"statfs":{"total":96636764160,"available":96599179264,"internally_reserved":0,"allocated":1384448,"data_stored":597010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509559,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37280,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334560,"statfs":{"total":96636764160,"available":96598589440,"internally_reserved":0,"allocated":1978368,"data_stored":1187378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542263,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2011136,"data_stored":1201069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542264,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36744,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335096,"statfs":{"total":96636764160,"available":96599138304,"internally_reserved":0,"allocated":1425408,"data_stored":611767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:51:43.588 INFO:tasks.ceph.ceph_manager.ceph:PG 17.3f is not active+clean 2024-09-10T21:51:43.588 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.485770+0000', 'last_change': '2024-09-10T21:51:37.485770+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.485770+0000', 'last_undegraded': '2024-09-10T21:51:37.485770+0000', 'last_fullsized': '2024-09-10T21:51:37.485770+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:51:43.588 INFO:tasks.ceph.ceph_manager.ceph:PG 17.3e is not active+clean 2024-09-10T21:51:43.588 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.467394+0000', 'last_change': '2024-09-10T21:51:37.467394+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.467394+0000', 'last_undegraded': '2024-09-10T21:51:37.467394+0000', 'last_fullsized': '2024-09-10T21:51:37.467394+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:51:43.588 INFO:tasks.ceph.ceph_manager.ceph:PG 17.3c is not active+clean 2024-09-10T21:51:43.588 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.489196+0000', 'last_change': '2024-09-10T21:51:37.489196+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.489196+0000', 'last_undegraded': '2024-09-10T21:51:37.489196+0000', 'last_fullsized': '2024-09-10T21:51:37.489196+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:51:43.588 INFO:tasks.ceph.ceph_manager.ceph:PG 17.3b is not active+clean 2024-09-10T21:51:43.588 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.476187+0000', 'last_change': '2024-09-10T21:51:37.476187+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.476187+0000', 'last_undegraded': '2024-09-10T21:51:37.476187+0000', 'last_fullsized': '2024-09-10T21:51:37.476187+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:51:43.589 INFO:tasks.ceph.ceph_manager.ceph:PG 17.3a is not active+clean 2024-09-10T21:51:43.589 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.469812+0000', 'last_change': '2024-09-10T21:51:37.469812+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.469812+0000', 'last_undegraded': '2024-09-10T21:51:37.469812+0000', 'last_fullsized': '2024-09-10T21:51:37.469812+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:51:43.589 INFO:tasks.ceph.ceph_manager.ceph:PG 17.39 is not active+clean 2024-09-10T21:51:43.589 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.465592+0000', 'last_change': '2024-09-10T21:51:37.465592+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.465592+0000', 'last_undegraded': '2024-09-10T21:51:37.465592+0000', 'last_fullsized': '2024-09-10T21:51:37.465592+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:51:43.589 INFO:tasks.ceph.ceph_manager.ceph:PG 17.37 is not active+clean 2024-09-10T21:51:43.589 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.482918+0000', 'last_change': '2024-09-10T21:51:37.482918+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.482918+0000', 'last_undegraded': '2024-09-10T21:51:37.482918+0000', 'last_fullsized': '2024-09-10T21:51:37.482918+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:51:43.589 INFO:tasks.ceph.ceph_manager.ceph:PG 17.36 is not active+clean 2024-09-10T21:51:43.589 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.482427+0000', 'last_change': '2024-09-10T21:51:37.482427+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.482427+0000', 'last_undegraded': '2024-09-10T21:51:37.482427+0000', 'last_fullsized': '2024-09-10T21:51:37.482427+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:51:43.589 INFO:tasks.ceph.ceph_manager.ceph:PG 17.35 is not active+clean 2024-09-10T21:51:43.589 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.468175+0000', 'last_change': '2024-09-10T21:51:37.468175+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.468175+0000', 'last_undegraded': '2024-09-10T21:51:37.468175+0000', 'last_fullsized': '2024-09-10T21:51:37.468175+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:51:43.589 INFO:tasks.ceph.ceph_manager.ceph:PG 17.34 is not active+clean 2024-09-10T21:51:43.589 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.485246+0000', 'last_change': '2024-09-10T21:51:37.485246+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.485246+0000', 'last_undegraded': '2024-09-10T21:51:37.485246+0000', 'last_fullsized': '2024-09-10T21:51:37.485246+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:51:43.590 INFO:tasks.ceph.ceph_manager.ceph:PG 17.31 is not active+clean 2024-09-10T21:51:43.590 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.469393+0000', 'last_change': '2024-09-10T21:51:37.469393+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.469393+0000', 'last_undegraded': '2024-09-10T21:51:37.469393+0000', 'last_fullsized': '2024-09-10T21:51:37.469393+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:51:43.590 INFO:tasks.ceph.ceph_manager.ceph:PG 17.30 is not active+clean 2024-09-10T21:51:43.590 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.481943+0000', 'last_change': '2024-09-10T21:51:37.481943+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.481943+0000', 'last_undegraded': '2024-09-10T21:51:37.481943+0000', 'last_fullsized': '2024-09-10T21:51:37.481943+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:51:43.590 INFO:tasks.ceph.ceph_manager.ceph:PG 17.2f is not active+clean 2024-09-10T21:51:43.590 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.468661+0000', 'last_change': '2024-09-10T21:51:37.468661+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.468661+0000', 'last_undegraded': '2024-09-10T21:51:37.468661+0000', 'last_fullsized': '2024-09-10T21:51:37.468661+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:51:43.590 INFO:tasks.ceph.ceph_manager.ceph:PG 17.2e is not active+clean 2024-09-10T21:51:43.590 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.479101+0000', 'last_change': '2024-09-10T21:51:37.479101+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.479101+0000', 'last_undegraded': '2024-09-10T21:51:37.479101+0000', 'last_fullsized': '2024-09-10T21:51:37.479101+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:51:43.590 INFO:tasks.ceph.ceph_manager.ceph:PG 17.2c is not active+clean 2024-09-10T21:51:43.590 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.467188+0000', 'last_change': '2024-09-10T21:51:37.467188+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.467188+0000', 'last_undegraded': '2024-09-10T21:51:37.467188+0000', 'last_fullsized': '2024-09-10T21:51:37.467188+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:51:43.590 INFO:tasks.ceph.ceph_manager.ceph:PG 17.2b is not active+clean 2024-09-10T21:51:43.590 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.469235+0000', 'last_change': '2024-09-10T21:51:37.469235+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.469235+0000', 'last_undegraded': '2024-09-10T21:51:37.469235+0000', 'last_fullsized': '2024-09-10T21:51:37.469235+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:51:43.591 INFO:tasks.ceph.ceph_manager.ceph:PG 17.29 is not active+clean 2024-09-10T21:51:43.591 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.466079+0000', 'last_change': '2024-09-10T21:51:37.466079+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.466079+0000', 'last_undegraded': '2024-09-10T21:51:37.466079+0000', 'last_fullsized': '2024-09-10T21:51:37.466079+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:51:43.591 INFO:tasks.ceph.ceph_manager.ceph:PG 17.27 is not active+clean 2024-09-10T21:51:43.591 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.467357+0000', 'last_change': '2024-09-10T21:51:37.467357+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.467357+0000', 'last_undegraded': '2024-09-10T21:51:37.467357+0000', 'last_fullsized': '2024-09-10T21:51:37.467357+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:51:43.591 INFO:tasks.ceph.ceph_manager.ceph:PG 17.26 is not active+clean 2024-09-10T21:51:43.591 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.478229+0000', 'last_change': '2024-09-10T21:51:37.478229+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.478229+0000', 'last_undegraded': '2024-09-10T21:51:37.478229+0000', 'last_fullsized': '2024-09-10T21:51:37.478229+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:51:43.591 INFO:tasks.ceph.ceph_manager.ceph:PG 17.25 is not active+clean 2024-09-10T21:51:43.591 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.473593+0000', 'last_change': '2024-09-10T21:51:37.473593+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.473593+0000', 'last_undegraded': '2024-09-10T21:51:37.473593+0000', 'last_fullsized': '2024-09-10T21:51:37.473593+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:51:43.591 INFO:tasks.ceph.ceph_manager.ceph:PG 17.23 is not active+clean 2024-09-10T21:51:43.591 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.474613+0000', 'last_change': '2024-09-10T21:51:37.474613+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.474613+0000', 'last_undegraded': '2024-09-10T21:51:37.474613+0000', 'last_fullsized': '2024-09-10T21:51:37.474613+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:51:43.591 INFO:tasks.ceph.ceph_manager.ceph:PG 17.22 is not active+clean 2024-09-10T21:51:43.591 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.463487+0000', 'last_change': '2024-09-10T21:51:37.463487+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.463487+0000', 'last_undegraded': '2024-09-10T21:51:37.463487+0000', 'last_fullsized': '2024-09-10T21:51:37.463487+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:51:43.592 INFO:tasks.ceph.ceph_manager.ceph:PG 17.21 is not active+clean 2024-09-10T21:51:43.592 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.478186+0000', 'last_change': '2024-09-10T21:51:37.478186+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.478186+0000', 'last_undegraded': '2024-09-10T21:51:37.478186+0000', 'last_fullsized': '2024-09-10T21:51:37.478186+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:51:43.592 INFO:tasks.ceph.ceph_manager.ceph:PG 17.20 is not active+clean 2024-09-10T21:51:43.592 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.468954+0000', 'last_change': '2024-09-10T21:51:37.468954+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.468954+0000', 'last_undegraded': '2024-09-10T21:51:37.468954+0000', 'last_fullsized': '2024-09-10T21:51:37.468954+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:51:43.592 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1e is not active+clean 2024-09-10T21:51:43.592 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.477280+0000', 'last_change': '2024-09-10T21:51:37.477280+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.477280+0000', 'last_undegraded': '2024-09-10T21:51:37.477280+0000', 'last_fullsized': '2024-09-10T21:51:37.477280+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:51:43.592 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1f is not active+clean 2024-09-10T21:51:43.592 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.480358+0000', 'last_change': '2024-09-10T21:51:37.480358+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.480358+0000', 'last_undegraded': '2024-09-10T21:51:37.480358+0000', 'last_fullsized': '2024-09-10T21:51:37.480358+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:51:43.592 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1c is not active+clean 2024-09-10T21:51:43.592 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.473226+0000', 'last_change': '2024-09-10T21:51:37.473226+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.473226+0000', 'last_undegraded': '2024-09-10T21:51:37.473226+0000', 'last_fullsized': '2024-09-10T21:51:37.473226+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:51:43.592 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1a is not active+clean 2024-09-10T21:51:43.593 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.476588+0000', 'last_change': '2024-09-10T21:51:37.476588+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.476588+0000', 'last_undegraded': '2024-09-10T21:51:37.476588+0000', 'last_fullsized': '2024-09-10T21:51:37.476588+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:51:43.593 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1b is not active+clean 2024-09-10T21:51:43.593 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.474186+0000', 'last_change': '2024-09-10T21:51:37.474186+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.474186+0000', 'last_undegraded': '2024-09-10T21:51:37.474186+0000', 'last_fullsized': '2024-09-10T21:51:37.474186+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:51:43.593 INFO:tasks.ceph.ceph_manager.ceph:PG 17.18 is not active+clean 2024-09-10T21:51:43.593 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.477970+0000', 'last_change': '2024-09-10T21:51:37.477970+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.477970+0000', 'last_undegraded': '2024-09-10T21:51:37.477970+0000', 'last_fullsized': '2024-09-10T21:51:37.477970+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:51:43.593 INFO:tasks.ceph.ceph_manager.ceph:PG 17.19 is not active+clean 2024-09-10T21:51:43.593 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.463935+0000', 'last_change': '2024-09-10T21:51:37.463935+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.463935+0000', 'last_undegraded': '2024-09-10T21:51:37.463935+0000', 'last_fullsized': '2024-09-10T21:51:37.463935+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:51:43.593 INFO:tasks.ceph.ceph_manager.ceph:PG 17.16 is not active+clean 2024-09-10T21:51:43.593 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.463971+0000', 'last_change': '2024-09-10T21:51:37.463971+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.463971+0000', 'last_undegraded': '2024-09-10T21:51:37.463971+0000', 'last_fullsized': '2024-09-10T21:51:37.463971+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:51:43.593 INFO:tasks.ceph.ceph_manager.ceph:PG 17.17 is not active+clean 2024-09-10T21:51:43.593 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.473160+0000', 'last_change': '2024-09-10T21:51:37.473160+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.473160+0000', 'last_undegraded': '2024-09-10T21:51:37.473160+0000', 'last_fullsized': '2024-09-10T21:51:37.473160+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:51:43.593 INFO:tasks.ceph.ceph_manager.ceph:PG 17.15 is not active+clean 2024-09-10T21:51:43.594 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.464945+0000', 'last_change': '2024-09-10T21:51:37.464945+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.464945+0000', 'last_undegraded': '2024-09-10T21:51:37.464945+0000', 'last_fullsized': '2024-09-10T21:51:37.464945+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-10T21:51:43.594 INFO:tasks.ceph.ceph_manager.ceph:PG 17.e is not active+clean 2024-09-10T21:51:43.594 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.475624+0000', 'last_change': '2024-09-10T21:51:37.475624+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.475624+0000', 'last_undegraded': '2024-09-10T21:51:37.475624+0000', 'last_fullsized': '2024-09-10T21:51:37.475624+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-10T21:51:43.594 INFO:tasks.ceph.ceph_manager.ceph:PG 17.f is not active+clean 2024-09-10T21:51:43.594 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.473936+0000', 'last_change': '2024-09-10T21:51:37.473936+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.473936+0000', 'last_undegraded': '2024-09-10T21:51:37.473936+0000', 'last_fullsized': '2024-09-10T21:51:37.473936+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-10T21:51:43.594 INFO:tasks.ceph.ceph_manager.ceph:PG 17.b is not active+clean 2024-09-10T21:51:43.594 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.471734+0000', 'last_change': '2024-09-10T21:51:37.471734+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.471734+0000', 'last_undegraded': '2024-09-10T21:51:37.471734+0000', 'last_fullsized': '2024-09-10T21:51:37.471734+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:51:43.594 INFO:tasks.ceph.ceph_manager.ceph:PG 17.6 is not active+clean 2024-09-10T21:51:43.594 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.470749+0000', 'last_change': '2024-09-10T21:51:37.470749+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.470749+0000', 'last_undegraded': '2024-09-10T21:51:37.470749+0000', 'last_fullsized': '2024-09-10T21:51:37.470749+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:51:43.594 INFO:tasks.ceph.ceph_manager.ceph:PG 17.7 is not active+clean 2024-09-10T21:51:43.594 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.472197+0000', 'last_change': '2024-09-10T21:51:37.472197+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.472197+0000', 'last_undegraded': '2024-09-10T21:51:37.472197+0000', 'last_fullsized': '2024-09-10T21:51:37.472197+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:51:43.594 INFO:tasks.ceph.ceph_manager.ceph:PG 17.4 is not active+clean 2024-09-10T21:51:43.595 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.471647+0000', 'last_change': '2024-09-10T21:51:37.471647+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.471647+0000', 'last_undegraded': '2024-09-10T21:51:37.471647+0000', 'last_fullsized': '2024-09-10T21:51:37.471647+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:51:43.595 INFO:tasks.ceph.ceph_manager.ceph:PG 17.5 is not active+clean 2024-09-10T21:51:43.595 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.475986+0000', 'last_change': '2024-09-10T21:51:37.475986+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.475986+0000', 'last_undegraded': '2024-09-10T21:51:37.475986+0000', 'last_fullsized': '2024-09-10T21:51:37.475986+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-10T21:51:43.595 INFO:tasks.ceph.ceph_manager.ceph:PG 17.2 is not active+clean 2024-09-10T21:51:43.595 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.466542+0000', 'last_change': '2024-09-10T21:51:37.466542+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.466542+0000', 'last_undegraded': '2024-09-10T21:51:37.466542+0000', 'last_fullsized': '2024-09-10T21:51:37.466542+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:51:43.595 INFO:tasks.ceph.ceph_manager.ceph:PG 17.3 is not active+clean 2024-09-10T21:51:43.595 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.473775+0000', 'last_change': '2024-09-10T21:51:37.473775+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.473775+0000', 'last_undegraded': '2024-09-10T21:51:37.473775+0000', 'last_fullsized': '2024-09-10T21:51:37.473775+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-10T21:51:43.595 INFO:tasks.ceph.ceph_manager.ceph:PG 17.1 is not active+clean 2024-09-10T21:51:43.595 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '17.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 112, 'state': 'creating+peering', 'last_fresh': '2024-09-10T21:51:37.474336+0000', 'last_change': '2024-09-10T21:51:37.474336+0000', 'last_active': '2024-09-10T21:51:37.456491+0000', 'last_peered': '2024-09-10T21:51:37.456491+0000', 'last_clean': '2024-09-10T21:51:37.456491+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-10T21:51:37.474336+0000', 'last_undegraded': '2024-09-10T21:51:37.474336+0000', 'last_fullsized': '2024-09-10T21:51:37.474336+0000', 'mapping_epoch': 112, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 112, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'last_clean_scrub_stamp': '2024-09-10T21:51:37.456491+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-10T21:51:43.596 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-10T21:51:43.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.781+0000 7f1145800640 1 -- 172.21.15.7:0/2870699759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1140146c50 msgr2=0x7f11400723b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:43.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.781+0000 7f1145800640 1 --2- 172.21.15.7:0/2870699759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1140146c50 0x7f11400723b0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f1130005fd0 tx=0x7f113002a540 comp rx=0 tx=0).stop 2024-09-10T21:51:43.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.782+0000 7f1145800640 1 -- 172.21.15.7:0/2870699759 shutdown_connections 2024-09-10T21:51:43.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.782+0000 7f1145800640 1 --2- 172.21.15.7:0/2870699759 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f114006f590 0x7f11400728f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:43.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.782+0000 7f1145800640 1 --2- 172.21.15.7:0/2870699759 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1140146c50 0x7f11400723b0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:43.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.782+0000 7f1145800640 1 -- 172.21.15.7:0/2870699759 >> 172.21.15.7:0/2870699759 conn(0x7f11400b1d70 msgr2=0x7f11400cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:43.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.782+0000 7f1145800640 1 -- 172.21.15.7:0/2870699759 shutdown_connections 2024-09-10T21:51:43.783 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.782+0000 7f1145800640 1 -- 172.21.15.7:0/2870699759 wait complete. 2024-09-10T21:51:43.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.783+0000 7f1145800640 1 Processor -- start 2024-09-10T21:51:43.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.784+0000 7f1145800640 1 -- start start 2024-09-10T21:51:43.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.784+0000 7f1145800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f114006f590 0x7f1140143110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:43.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.784+0000 7f1145800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1140146c50 0x7f1140145e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:43.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.784+0000 7f1145800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11401470b0 0x7f114013c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:43.786 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.784+0000 7f1145800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1140161f10 con 0x7f1140146c50 2024-09-10T21:51:43.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.784+0000 7f1145800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f11400731c0 con 0x7f114006f590 2024-09-10T21:51:43.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.784+0000 7f1145800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f1140162090 con 0x7f11401470b0 2024-09-10T21:51:43.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.785+0000 7f1137400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1140146c50 0x7f1140145e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:43.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.785+0000 7f1137400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1140146c50 0x7f1140145e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:45000/0 (socket says 172.21.15.7:45000) 2024-09-10T21:51:43.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.785+0000 7f1137400640 1 -- 172.21.15.7:0/108880442 learned_addr learned my addr 172.21.15.7:0/108880442 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:43.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.785+0000 7f113fe00640 1 --2- 172.21.15.7:0/108880442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f114006f590 0x7f1140143110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:43.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.785+0000 7f1137400640 1 -- 172.21.15.7:0/108880442 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11401470b0 msgr2=0x7f114013c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:43.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.785+0000 7f1137400640 1 --2- 172.21.15.7:0/108880442 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11401470b0 0x7f114013c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:43.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.785+0000 7f1137400640 1 -- 172.21.15.7:0/108880442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f114006f590 msgr2=0x7f1140143110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:43.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.785+0000 7f1137400640 1 --2- 172.21.15.7:0/108880442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f114006f590 0x7f1140143110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:43.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.785+0000 7f1137400640 1 -- 172.21.15.7:0/108880442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11401f8560 con 0x7f1140146c50 2024-09-10T21:51:43.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.785+0000 7f113fe00640 1 --2- 172.21.15.7:0/108880442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f114006f590 0x7f1140143110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:51:43.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.785+0000 7f1137400640 1 --2- 172.21.15.7:0/108880442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1140146c50 0x7f1140145e90 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f1128005fd0 tx=0x7f112800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:43.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.786+0000 7f113cc00640 1 -- 172.21.15.7:0/108880442 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1128017070 con 0x7f1140146c50 2024-09-10T21:51:43.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.786+0000 7f113cc00640 1 -- 172.21.15.7:0/108880442 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1128009d20 con 0x7f1140146c50 2024-09-10T21:51:43.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.786+0000 7f113cc00640 1 -- 172.21.15.7:0/108880442 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1128004280 con 0x7f1140146c50 2024-09-10T21:51:43.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.786+0000 7f1145800640 1 -- 172.21.15.7:0/108880442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11401f8840 con 0x7f1140146c50 2024-09-10T21:51:43.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.786+0000 7f1145800640 1 -- 172.21.15.7:0/108880442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11401f8e00 con 0x7f1140146c50 2024-09-10T21:51:43.788 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.787+0000 7f1145800640 1 -- 172.21.15.7:0/108880442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1140162540 con 0x7f1140146c50 2024-09-10T21:51:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.788+0000 7f113cc00640 1 -- 172.21.15.7:0/108880442 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f11280035f0 con 0x7f1140146c50 2024-09-10T21:51:43.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.789+0000 7f113cc00640 1 --2- 172.21.15.7:0/108880442 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f111c077f70 0x7f111c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:43.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.789+0000 7f113cc00640 1 -- 172.21.15.7:0/108880442 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5709+0+0 (secure 0 0 0) 0x7f112809a170 con 0x7f1140146c50 2024-09-10T21:51:43.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.789+0000 7f113fe00640 1 --2- 172.21.15.7:0/108880442 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f111c077f70 0x7f111c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:43.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.790+0000 7f113fe00640 1 --2- 172.21.15.7:0/108880442 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f111c077f70 0x7f111c07a430 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1130033000 tx=0x7f1130004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:43.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:43.793+0000 7f113cc00640 1 -- 172.21.15.7:0/108880442 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1128020030 con 0x7f1140146c50 2024-09-10T21:51:44.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.068+0000 7f1145800640 1 -- 172.21.15.7:0/108880442 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f11400af230 con 0x7f1140146c50 2024-09-10T21:51:44.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.068+0000 7f113cc00640 1 -- 172.21.15.7:0/108880442 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1142 (secure 0 0 0) 0x7f1128062c60 con 0x7f1140146c50 2024-09-10T21:51:44.069 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:44.069 INFO:teuthology.orchestra.run.smithi007.stdout:{"fsid":"2029451a-3c26-48c7-b874-0597197278ac","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":600,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":114,"num_osds":8,"num_up_osds":8,"osd_up_since":1726004516,"num_in_osds":8,"osd_in_since":1726004507,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":53},{"state_name":"creating+peering","count":44}],"num_pgs":97,"num_pools":3,"num_objects":3,"data_bytes":590368,"bytes_used":280580096,"bytes_avail":772813533184,"bytes_total":773094113280,"inactive_pgs_ratio":0.45360824465751648,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":102,"btime":"2024-09-10T21:51:41:499376+0000","id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"b","status":"up:active","gid":236726}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":24,"modified":"2024-09-10T21:51:22.003621+0000","services":{}},"progress_events":{}} 2024-09-10T21:51:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.071+0000 7f1145800640 1 -- 172.21.15.7:0/108880442 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f111c077f70 msgr2=0x7f111c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.071+0000 7f1145800640 1 --2- 172.21.15.7:0/108880442 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f111c077f70 0x7f111c07a430 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1130033000 tx=0x7f1130004730 comp rx=0 tx=0).stop 2024-09-10T21:51:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.071+0000 7f1145800640 1 -- 172.21.15.7:0/108880442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1140146c50 msgr2=0x7f1140145e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.071+0000 7f1145800640 1 --2- 172.21.15.7:0/108880442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1140146c50 0x7f1140145e90 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f1128005fd0 tx=0x7f112800dde0 comp rx=0 tx=0).stop 2024-09-10T21:51:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.071+0000 7f1145800640 1 -- 172.21.15.7:0/108880442 shutdown_connections 2024-09-10T21:51:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.071+0000 7f1145800640 1 --2- 172.21.15.7:0/108880442 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f111c077f70 0x7f111c07a430 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.071+0000 7f1145800640 1 --2- 172.21.15.7:0/108880442 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f11401470b0 0x7f114013c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.071+0000 7f1145800640 1 --2- 172.21.15.7:0/108880442 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1140146c50 0x7f1140145e90 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.071+0000 7f1145800640 1 --2- 172.21.15.7:0/108880442 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f114006f590 0x7f1140143110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.071+0000 7f1145800640 1 -- 172.21.15.7:0/108880442 >> 172.21.15.7:0/108880442 conn(0x7f11400b1d70 msgr2=0x7f11400b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.071+0000 7f1145800640 1 -- 172.21.15.7:0/108880442 shutdown_connections 2024-09-10T21:51:44.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.071+0000 7f1145800640 1 -- 172.21.15.7:0/108880442 wait complete. 2024-09-10T21:51:44.087 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-10T21:51:44.087 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:51:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.246+0000 7f631cc00640 1 -- 172.21.15.7:0/3036963066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63180b4ac0 msgr2=0x7f63180b2a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.246+0000 7f631cc00640 1 --2- 172.21.15.7:0/3036963066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63180b4ac0 0x7f63180b2a40 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f63180af640 tx=0x7f630402a560 comp rx=0 tx=0).stop 2024-09-10T21:51:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.246+0000 7f631cc00640 1 -- 172.21.15.7:0/3036963066 shutdown_connections 2024-09-10T21:51:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.246+0000 7f631cc00640 1 --2- 172.21.15.7:0/3036963066 >> v2:172.21.15.133:3300/0 conn(0x7f63180b2f80 0x7f631815f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.246+0000 7f631cc00640 1 --2- 172.21.15.7:0/3036963066 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63180b4ac0 0x7f63180b2a40 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:44.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.246+0000 7f631cc00640 1 --2- 172.21.15.7:0/3036963066 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6318145540 0x7f63180b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.246+0000 7f631cc00640 1 -- 172.21.15.7:0/3036963066 >> 172.21.15.7:0/3036963066 conn(0x7f63180b1d70 msgr2=0x7f63180cfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.246+0000 7f631cc00640 1 -- 172.21.15.7:0/3036963066 shutdown_connections 2024-09-10T21:51:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.246+0000 7f631cc00640 1 -- 172.21.15.7:0/3036963066 wait complete. 2024-09-10T21:51:44.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.248+0000 7f631cc00640 1 Processor -- start 2024-09-10T21:51:44.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.248+0000 7f631cc00640 1 -- start start 2024-09-10T21:51:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.248+0000 7f631cc00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63180b2f80 0x7f631815e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.249+0000 7f631cc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63180b4ac0 0x7f63181f2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.249+0000 7f631cc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6318145540 0x7f631815a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.249+0000 7f631cc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f631815fe90 con 0x7f63180b4ac0 2024-09-10T21:51:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.249+0000 7f631cc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f631815fd10 con 0x7f6318145540 2024-09-10T21:51:44.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.249+0000 7f631cc00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f6318160010 con 0x7f63180b2f80 2024-09-10T21:51:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.249+0000 7f6316a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63180b4ac0 0x7f63181f2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.249+0000 7f6316a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63180b4ac0 0x7f63181f2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43758/0 (socket says 172.21.15.7:43758) 2024-09-10T21:51:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.249+0000 7f6316a00640 1 -- 172.21.15.7:0/909014246 learned_addr learned my addr 172.21.15.7:0/909014246 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.249+0000 7f6317400640 1 --2- 172.21.15.7:0/909014246 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63180b2f80 0x7f631815e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.249+0000 7f6317e00640 1 --2- 172.21.15.7:0/909014246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6318145540 0x7f631815a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.249+0000 7f6316a00640 1 -- 172.21.15.7:0/909014246 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63180b2f80 msgr2=0x7f631815e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.249+0000 7f6316a00640 1 --2- 172.21.15.7:0/909014246 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63180b2f80 0x7f631815e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.249+0000 7f6316a00640 1 -- 172.21.15.7:0/909014246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6318145540 msgr2=0x7f631815a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.249+0000 7f6316a00640 1 --2- 172.21.15.7:0/909014246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6318145540 0x7f631815a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.249+0000 7f6316a00640 1 -- 172.21.15.7:0/909014246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f631815b4f0 con 0x7f63180b4ac0 2024-09-10T21:51:44.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.250+0000 7f6316a00640 1 --2- 172.21.15.7:0/909014246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63180b4ac0 0x7f63181f2d40 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f6304009850 tx=0x7f63040038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:44.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.250+0000 7f630f400640 1 -- 172.21.15.7:0/909014246 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f630403f070 con 0x7f63180b4ac0 2024-09-10T21:51:44.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.250+0000 7f630f400640 1 -- 172.21.15.7:0/909014246 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63040047a0 con 0x7f63180b4ac0 2024-09-10T21:51:44.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.250+0000 7f631cc00640 1 -- 172.21.15.7:0/909014246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f631815ac50 con 0x7f63180b4ac0 2024-09-10T21:51:44.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.250+0000 7f630f400640 1 -- 172.21.15.7:0/909014246 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6304035030 con 0x7f63180b4ac0 2024-09-10T21:51:44.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.250+0000 7f631cc00640 1 -- 172.21.15.7:0/909014246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f631815dbb0 con 0x7f63180b4ac0 2024-09-10T21:51:44.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.251+0000 7f631cc00640 1 -- 172.21.15.7:0/909014246 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63180cd3f0 con 0x7f63180b4ac0 2024-09-10T21:51:44.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.253+0000 7f630f400640 1 -- 172.21.15.7:0/909014246 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6304004bb0 con 0x7f63180b4ac0 2024-09-10T21:51:44.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.254+0000 7f630f400640 1 --2- 172.21.15.7:0/909014246 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f62f8077f70 0x7f62f807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:44.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.254+0000 7f630f400640 1 -- 172.21.15.7:0/909014246 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5709+0+0 (secure 0 0 0) 0x7f63040bac60 con 0x7f63180b4ac0 2024-09-10T21:51:44.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.255+0000 7f6317400640 1 --2- 172.21.15.7:0/909014246 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f62f8077f70 0x7f62f807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:44.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.255+0000 7f6317400640 1 --2- 172.21.15.7:0/909014246 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f62f8077f70 0x7f62f807a430 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6308009b30 tx=0x7f6308005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:44.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.258+0000 7f630f400640 1 -- 172.21.15.7:0/909014246 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f63180cd3f0 con 0x7f63180b4ac0 2024-09-10T21:51:44.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.447+0000 7f631cc00640 1 -- 172.21.15.7:0/909014246 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f63181604c0 con 0x7f62f8077f70 2024-09-10T21:51:44.453 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.452+0000 7f630f400640 1 -- 172.21.15.7:0/909014246 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252472 (secure 0 0 0) 0x7f63181604c0 con 0x7f62f8077f70 2024-09-10T21:51:44.457 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:44.459 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:51:44.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.456+0000 7f631cc00640 1 -- 172.21.15.7:0/909014246 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f62f8077f70 msgr2=0x7f62f807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:44.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.456+0000 7f631cc00640 1 --2- 172.21.15.7:0/909014246 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f62f8077f70 0x7f62f807a430 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f6308009b30 tx=0x7f6308005f90 comp rx=0 tx=0).stop 2024-09-10T21:51:44.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.456+0000 7f631cc00640 1 -- 172.21.15.7:0/909014246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63180b4ac0 msgr2=0x7f63181f2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:44.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.456+0000 7f631cc00640 1 --2- 172.21.15.7:0/909014246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63180b4ac0 0x7f63181f2d40 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f6304009850 tx=0x7f63040038a0 comp rx=0 tx=0).stop 2024-09-10T21:51:44.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.456+0000 7f631cc00640 1 -- 172.21.15.7:0/909014246 shutdown_connections 2024-09-10T21:51:44.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.456+0000 7f631cc00640 1 --2- 172.21.15.7:0/909014246 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f62f8077f70 0x7f62f807a430 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:44.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.456+0000 7f631cc00640 1 --2- 172.21.15.7:0/909014246 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6318145540 0x7f631815a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:44.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.456+0000 7f631cc00640 1 --2- 172.21.15.7:0/909014246 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f63180b4ac0 0x7f63181f2d40 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:44.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.456+0000 7f631cc00640 1 --2- 172.21.15.7:0/909014246 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63180b2f80 0x7f631815e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:44.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.456+0000 7f631cc00640 1 -- 172.21.15.7:0/909014246 >> 172.21.15.7:0/909014246 conn(0x7f63180b1d70 msgr2=0x7f63180b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:44.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.457+0000 7f631cc00640 1 -- 172.21.15.7:0/909014246 shutdown_connections 2024-09-10T21:51:44.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:44.457+0000 7f631cc00640 1 -- 172.21.15.7:0/909014246 wait complete. 2024-09-10T21:51:44.464 INFO:tasks.ceph.mds.b.smithi133.stderr:2024-09-10T21:51:44.455+0000 7f1d73400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:51:44.474 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":401,"stamp":"2024-09-10T21:51:44.008990+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":274240,"kb_used_data":12232,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754700480,"statfs":{"total":773094113280,"available":772813291520,"internally_reserved":0,"allocated":12525568,"data_stored":6066236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.547539"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494583+0000","last_change":"2024-09-10T21:51:38.494481+0000","last_active":"2024-09-10T21:51:39.494583+0000","last_peered":"2024-09-10T21:51:39.494583+0000","last_clean":"2024-09-10T21:51:39.494583+0000","last_became_active":"2024-09-10T21:51:38.494199+0000","last_became_peered":"2024-09-10T21:51:38.494199+0000","last_unstale":"2024-09-10T21:51:39.494583+0000","last_undegraded":"2024-09-10T21:51:39.494583+0000","last_fullsized":"2024-09-10T21:51:39.494583+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:40.667423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496904+0000","last_change":"2024-09-10T21:51:38.493007+0000","last_active":"2024-09-10T21:51:39.496904+0000","last_peered":"2024-09-10T21:51:39.496904+0000","last_clean":"2024-09-10T21:51:39.496904+0000","last_became_active":"2024-09-10T21:51:38.492740+0000","last_became_peered":"2024-09-10T21:51:38.492740+0000","last_unstale":"2024-09-10T21:51:39.496904+0000","last_undegraded":"2024-09-10T21:51:39.496904+0000","last_fullsized":"2024-09-10T21:51:39.496904+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:18.728391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966452+0000","last_change":"2024-09-10T21:51:38.505264+0000","last_active":"2024-09-10T21:51:39.966452+0000","last_peered":"2024-09-10T21:51:39.966452+0000","last_clean":"2024-09-10T21:51:39.966452+0000","last_became_active":"2024-09-10T21:51:38.504121+0000","last_became_peered":"2024-09-10T21:51:38.504121+0000","last_unstale":"2024-09-10T21:51:39.966452+0000","last_undegraded":"2024-09-10T21:51:39.966452+0000","last_fullsized":"2024-09-10T21:51:39.966452+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:32.797143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495139+0000","last_change":"2024-09-10T21:51:38.492127+0000","last_active":"2024-09-10T21:51:39.495139+0000","last_peered":"2024-09-10T21:51:39.495139+0000","last_clean":"2024-09-10T21:51:39.495139+0000","last_became_active":"2024-09-10T21:51:38.491848+0000","last_became_peered":"2024-09-10T21:51:38.491848+0000","last_unstale":"2024-09-10T21:51:39.495139+0000","last_undegraded":"2024-09-10T21:51:39.495139+0000","last_fullsized":"2024-09-10T21:51:39.495139+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:34.188188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965536+0000","last_change":"2024-09-10T21:51:38.489365+0000","last_active":"2024-09-10T21:51:39.965536+0000","last_peered":"2024-09-10T21:51:39.965536+0000","last_clean":"2024-09-10T21:51:39.965536+0000","last_became_active":"2024-09-10T21:51:38.489084+0000","last_became_peered":"2024-09-10T21:51:38.489084+0000","last_unstale":"2024-09-10T21:51:39.965536+0000","last_undegraded":"2024-09-10T21:51:39.965536+0000","last_fullsized":"2024-09-10T21:51:39.965536+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:09.526080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.836686+0000","last_change":"2024-09-10T21:51:38.495769+0000","last_active":"2024-09-10T21:51:39.836686+0000","last_peered":"2024-09-10T21:51:39.836686+0000","last_clean":"2024-09-10T21:51:39.836686+0000","last_became_active":"2024-09-10T21:51:38.495508+0000","last_became_peered":"2024-09-10T21:51:38.495508+0000","last_unstale":"2024-09-10T21:51:39.836686+0000","last_undegraded":"2024-09-10T21:51:39.836686+0000","last_fullsized":"2024-09-10T21:51:39.836686+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:12.480879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495331+0000","last_change":"2024-09-10T21:51:38.484984+0000","last_active":"2024-09-10T21:51:39.495331+0000","last_peered":"2024-09-10T21:51:39.495331+0000","last_clean":"2024-09-10T21:51:39.495331+0000","last_became_active":"2024-09-10T21:51:38.483932+0000","last_became_peered":"2024-09-10T21:51:38.483932+0000","last_unstale":"2024-09-10T21:51:39.495331+0000","last_undegraded":"2024-09-10T21:51:39.495331+0000","last_fullsized":"2024-09-10T21:51:39.495331+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:24.626861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966436+0000","last_change":"2024-09-10T21:51:38.498466+0000","last_active":"2024-09-10T21:51:39.966436+0000","last_peered":"2024-09-10T21:51:39.966436+0000","last_clean":"2024-09-10T21:51:39.966436+0000","last_became_active":"2024-09-10T21:51:38.498041+0000","last_became_peered":"2024-09-10T21:51:38.498041+0000","last_unstale":"2024-09-10T21:51:39.966436+0000","last_undegraded":"2024-09-10T21:51:39.966436+0000","last_fullsized":"2024-09-10T21:51:39.966436+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:19.954387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494640+0000","last_change":"2024-09-10T21:51:38.489094+0000","last_active":"2024-09-10T21:51:39.494640+0000","last_peered":"2024-09-10T21:51:39.494640+0000","last_clean":"2024-09-10T21:51:39.494640+0000","last_became_active":"2024-09-10T21:51:38.488756+0000","last_became_peered":"2024-09-10T21:51:38.488756+0000","last_unstale":"2024-09-10T21:51:39.494640+0000","last_undegraded":"2024-09-10T21:51:39.494640+0000","last_fullsized":"2024-09-10T21:51:39.494640+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:28.335704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494779+0000","last_change":"2024-09-10T21:51:38.499151+0000","last_active":"2024-09-10T21:51:39.494779+0000","last_peered":"2024-09-10T21:51:39.494779+0000","last_clean":"2024-09-10T21:51:39.494779+0000","last_became_active":"2024-09-10T21:51:38.498858+0000","last_became_peered":"2024-09-10T21:51:38.498858+0000","last_unstale":"2024-09-10T21:51:39.494779+0000","last_undegraded":"2024-09-10T21:51:39.494779+0000","last_fullsized":"2024-09-10T21:51:39.494779+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:11.108209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496669+0000","last_change":"2024-09-10T21:51:38.491484+0000","last_active":"2024-09-10T21:51:39.496669+0000","last_peered":"2024-09-10T21:51:39.496669+0000","last_clean":"2024-09-10T21:51:39.496669+0000","last_became_active":"2024-09-10T21:51:38.491193+0000","last_became_peered":"2024-09-10T21:51:38.491193+0000","last_unstale":"2024-09-10T21:51:39.496669+0000","last_undegraded":"2024-09-10T21:51:39.496669+0000","last_fullsized":"2024-09-10T21:51:39.496669+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:28.006935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494629+0000","last_change":"2024-09-10T21:51:38.491409+0000","last_active":"2024-09-10T21:51:39.494629+0000","last_peered":"2024-09-10T21:51:39.494629+0000","last_clean":"2024-09-10T21:51:39.494629+0000","last_became_active":"2024-09-10T21:51:38.490377+0000","last_became_peered":"2024-09-10T21:51:38.490377+0000","last_unstale":"2024-09-10T21:51:39.494629+0000","last_undegraded":"2024-09-10T21:51:39.494629+0000","last_fullsized":"2024-09-10T21:51:39.494629+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:33.606508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966404+0000","last_change":"2024-09-10T21:51:38.503338+0000","last_active":"2024-09-10T21:51:39.966404+0000","last_peered":"2024-09-10T21:51:39.966404+0000","last_clean":"2024-09-10T21:51:39.966404+0000","last_became_active":"2024-09-10T21:51:38.501614+0000","last_became_peered":"2024-09-10T21:51:38.501614+0000","last_unstale":"2024-09-10T21:51:39.966404+0000","last_undegraded":"2024-09-10T21:51:39.966404+0000","last_fullsized":"2024-09-10T21:51:39.966404+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:10.529060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966266+0000","last_change":"2024-09-10T21:51:38.498967+0000","last_active":"2024-09-10T21:51:39.966266+0000","last_peered":"2024-09-10T21:51:39.966266+0000","last_clean":"2024-09-10T21:51:39.966266+0000","last_became_active":"2024-09-10T21:51:38.498641+0000","last_became_peered":"2024-09-10T21:51:38.498641+0000","last_unstale":"2024-09-10T21:51:39.966266+0000","last_undegraded":"2024-09-10T21:51:39.966266+0000","last_fullsized":"2024-09-10T21:51:39.966266+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:41.424015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.836623+0000","last_change":"2024-09-10T21:51:38.494202+0000","last_active":"2024-09-10T21:51:39.836623+0000","last_peered":"2024-09-10T21:51:39.836623+0000","last_clean":"2024-09-10T21:51:39.836623+0000","last_became_active":"2024-09-10T21:51:38.493912+0000","last_became_peered":"2024-09-10T21:51:38.493912+0000","last_unstale":"2024-09-10T21:51:39.836623+0000","last_undegraded":"2024-09-10T21:51:39.836623+0000","last_fullsized":"2024-09-10T21:51:39.836623+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:51.129806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494544+0000","last_change":"2024-09-10T21:51:38.502368+0000","last_active":"2024-09-10T21:51:39.494544+0000","last_peered":"2024-09-10T21:51:39.494544+0000","last_clean":"2024-09-10T21:51:39.494544+0000","last_became_active":"2024-09-10T21:51:38.502112+0000","last_became_peered":"2024-09-10T21:51:38.502112+0000","last_unstale":"2024-09-10T21:51:39.494544+0000","last_undegraded":"2024-09-10T21:51:39.494544+0000","last_fullsized":"2024-09-10T21:51:39.494544+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:03.034899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496859+0000","last_change":"2024-09-10T21:51:38.495199+0000","last_active":"2024-09-10T21:51:39.496859+0000","last_peered":"2024-09-10T21:51:39.496859+0000","last_clean":"2024-09-10T21:51:39.496859+0000","last_became_active":"2024-09-10T21:51:38.494943+0000","last_became_peered":"2024-09-10T21:51:38.494943+0000","last_unstale":"2024-09-10T21:51:39.496859+0000","last_undegraded":"2024-09-10T21:51:39.496859+0000","last_fullsized":"2024-09-10T21:51:39.496859+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:05.034431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.499881+0000","last_change":"2024-09-10T21:51:38.487185+0000","last_active":"2024-09-10T21:51:39.499881+0000","last_peered":"2024-09-10T21:51:39.499881+0000","last_clean":"2024-09-10T21:51:39.499881+0000","last_became_active":"2024-09-10T21:51:38.486876+0000","last_became_peered":"2024-09-10T21:51:38.486876+0000","last_unstale":"2024-09-10T21:51:39.499881+0000","last_undegraded":"2024-09-10T21:51:39.499881+0000","last_fullsized":"2024-09-10T21:51:39.499881+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:30.506461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966459+0000","last_change":"2024-09-10T21:51:38.506319+0000","last_active":"2024-09-10T21:51:39.966459+0000","last_peered":"2024-09-10T21:51:39.966459+0000","last_clean":"2024-09-10T21:51:39.966459+0000","last_became_active":"2024-09-10T21:51:38.506041+0000","last_became_peered":"2024-09-10T21:51:38.506041+0000","last_unstale":"2024-09-10T21:51:39.966459+0000","last_undegraded":"2024-09-10T21:51:39.966459+0000","last_fullsized":"2024-09-10T21:51:39.966459+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:33.484489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496585+0000","last_change":"2024-09-10T21:51:38.499791+0000","last_active":"2024-09-10T21:51:39.496585+0000","last_peered":"2024-09-10T21:51:39.496585+0000","last_clean":"2024-09-10T21:51:39.496585+0000","last_became_active":"2024-09-10T21:51:38.499505+0000","last_became_peered":"2024-09-10T21:51:38.499505+0000","last_unstale":"2024-09-10T21:51:39.496585+0000","last_undegraded":"2024-09-10T21:51:39.496585+0000","last_fullsized":"2024-09-10T21:51:39.496585+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:04.591632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496782+0000","last_change":"2024-09-10T21:51:38.502073+0000","last_active":"2024-09-10T21:51:39.496782+0000","last_peered":"2024-09-10T21:51:39.496782+0000","last_clean":"2024-09-10T21:51:39.496782+0000","last_became_active":"2024-09-10T21:51:38.501750+0000","last_became_peered":"2024-09-10T21:51:38.501750+0000","last_unstale":"2024-09-10T21:51:39.496782+0000","last_undegraded":"2024-09-10T21:51:39.496782+0000","last_fullsized":"2024-09-10T21:51:39.496782+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:24.265846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965676+0000","last_change":"2024-09-10T21:51:38.495309+0000","last_active":"2024-09-10T21:51:39.965676+0000","last_peered":"2024-09-10T21:51:39.965676+0000","last_clean":"2024-09-10T21:51:39.965676+0000","last_became_active":"2024-09-10T21:51:38.494998+0000","last_became_peered":"2024-09-10T21:51:38.494998+0000","last_unstale":"2024-09-10T21:51:39.965676+0000","last_undegraded":"2024-09-10T21:51:39.965676+0000","last_fullsized":"2024-09-10T21:51:39.965676+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:41.427352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495913+0000","last_change":"2024-09-10T21:51:38.499369+0000","last_active":"2024-09-10T21:51:39.495913+0000","last_peered":"2024-09-10T21:51:39.495913+0000","last_clean":"2024-09-10T21:51:39.495913+0000","last_became_active":"2024-09-10T21:51:38.499094+0000","last_became_peered":"2024-09-10T21:51:38.499094+0000","last_unstale":"2024-09-10T21:51:39.495913+0000","last_undegraded":"2024-09-10T21:51:39.495913+0000","last_fullsized":"2024-09-10T21:51:39.495913+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:36.192408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965229+0000","last_change":"2024-09-10T21:51:38.505466+0000","last_active":"2024-09-10T21:51:39.965229+0000","last_peered":"2024-09-10T21:51:39.965229+0000","last_clean":"2024-09-10T21:51:39.965229+0000","last_became_active":"2024-09-10T21:51:38.505053+0000","last_became_peered":"2024-09-10T21:51:38.505053+0000","last_unstale":"2024-09-10T21:51:39.965229+0000","last_undegraded":"2024-09-10T21:51:39.965229+0000","last_fullsized":"2024-09-10T21:51:39.965229+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:36:29.403651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.497048+0000","last_change":"2024-09-10T21:51:38.494581+0000","last_active":"2024-09-10T21:51:39.497048+0000","last_peered":"2024-09-10T21:51:39.497048+0000","last_clean":"2024-09-10T21:51:39.497048+0000","last_became_active":"2024-09-10T21:51:38.493441+0000","last_became_peered":"2024-09-10T21:51:38.493441+0000","last_unstale":"2024-09-10T21:51:39.497048+0000","last_undegraded":"2024-09-10T21:51:39.497048+0000","last_fullsized":"2024-09-10T21:51:39.497048+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:29.395227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494942+0000","last_change":"2024-09-10T21:51:38.493643+0000","last_active":"2024-09-10T21:51:39.494942+0000","last_peered":"2024-09-10T21:51:39.494942+0000","last_clean":"2024-09-10T21:51:39.494942+0000","last_became_active":"2024-09-10T21:51:38.493374+0000","last_became_peered":"2024-09-10T21:51:38.493374+0000","last_unstale":"2024-09-10T21:51:39.494942+0000","last_undegraded":"2024-09-10T21:51:39.494942+0000","last_fullsized":"2024-09-10T21:51:39.494942+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:53.810720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.836711+0000","last_change":"2024-09-10T21:51:38.491652+0000","last_active":"2024-09-10T21:51:39.836711+0000","last_peered":"2024-09-10T21:51:39.836711+0000","last_clean":"2024-09-10T21:51:39.836711+0000","last_became_active":"2024-09-10T21:51:38.488308+0000","last_became_peered":"2024-09-10T21:51:38.488308+0000","last_unstale":"2024-09-10T21:51:39.836711+0000","last_undegraded":"2024-09-10T21:51:39.836711+0000","last_fullsized":"2024-09-10T21:51:39.836711+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:43.939923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965771+0000","last_change":"2024-09-10T21:51:38.496708+0000","last_active":"2024-09-10T21:51:39.965771+0000","last_peered":"2024-09-10T21:51:39.965771+0000","last_clean":"2024-09-10T21:51:39.965771+0000","last_became_active":"2024-09-10T21:51:38.496402+0000","last_became_peered":"2024-09-10T21:51:38.496402+0000","last_unstale":"2024-09-10T21:51:39.965771+0000","last_undegraded":"2024-09-10T21:51:39.965771+0000","last_fullsized":"2024-09-10T21:51:39.965771+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:40.305224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496429+0000","last_change":"2024-09-10T21:51:38.499760+0000","last_active":"2024-09-10T21:51:39.496429+0000","last_peered":"2024-09-10T21:51:39.496429+0000","last_clean":"2024-09-10T21:51:39.496429+0000","last_became_active":"2024-09-10T21:51:38.499426+0000","last_became_peered":"2024-09-10T21:51:38.499426+0000","last_unstale":"2024-09-10T21:51:39.496429+0000","last_undegraded":"2024-09-10T21:51:39.496429+0000","last_fullsized":"2024-09-10T21:51:39.496429+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:38.181706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495415+0000","last_change":"2024-09-10T21:51:38.500997+0000","last_active":"2024-09-10T21:51:39.495415+0000","last_peered":"2024-09-10T21:51:39.495415+0000","last_clean":"2024-09-10T21:51:39.495415+0000","last_became_active":"2024-09-10T21:51:38.500708+0000","last_became_peered":"2024-09-10T21:51:38.500708+0000","last_unstale":"2024-09-10T21:51:39.495415+0000","last_undegraded":"2024-09-10T21:51:39.495415+0000","last_fullsized":"2024-09-10T21:51:39.495415+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:02.532116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.493986+0000","last_change":"2024-09-10T21:51:38.486559+0000","last_active":"2024-09-10T21:51:39.493986+0000","last_peered":"2024-09-10T21:51:39.493986+0000","last_clean":"2024-09-10T21:51:39.493986+0000","last_became_active":"2024-09-10T21:51:38.486293+0000","last_became_peered":"2024-09-10T21:51:38.486293+0000","last_unstale":"2024-09-10T21:51:39.493986+0000","last_undegraded":"2024-09-10T21:51:39.493986+0000","last_fullsized":"2024-09-10T21:51:39.493986+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:52.862206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.836657+0000","last_change":"2024-09-10T21:51:38.501466+0000","last_active":"2024-09-10T21:51:39.836657+0000","last_peered":"2024-09-10T21:51:39.836657+0000","last_clean":"2024-09-10T21:51:39.836657+0000","last_became_active":"2024-09-10T21:51:38.501158+0000","last_became_peered":"2024-09-10T21:51:38.501158+0000","last_unstale":"2024-09-10T21:51:39.836657+0000","last_undegraded":"2024-09-10T21:51:39.836657+0000","last_fullsized":"2024-09-10T21:51:39.836657+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:43:23.281176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.836993+0000","last_change":"2024-09-10T21:51:38.488631+0000","last_active":"2024-09-10T21:51:39.836993+0000","last_peered":"2024-09-10T21:51:39.836993+0000","last_clean":"2024-09-10T21:51:39.836993+0000","last_became_active":"2024-09-10T21:51:38.487728+0000","last_became_peered":"2024-09-10T21:51:38.487728+0000","last_unstale":"2024-09-10T21:51:39.836993+0000","last_undegraded":"2024-09-10T21:51:39.836993+0000","last_fullsized":"2024-09-10T21:51:39.836993+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:08.725531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"113'4","reported_seq":19,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496069+0000","last_change":"2024-09-10T21:51:37.472999+0000","last_active":"2024-09-10T21:51:39.496069+0000","last_peered":"2024-09-10T21:51:39.496069+0000","last_clean":"2024-09-10T21:51:39.496069+0000","last_became_active":"2024-09-10T21:51:37.472471+0000","last_became_peered":"2024-09-10T21:51:37.472471+0000","last_unstale":"2024-09-10T21:51:39.496069+0000","last_undegraded":"2024-09-10T21:51:39.496069+0000","last_fullsized":"2024-09-10T21:51:39.496069+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:10.162552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.500153+0000","last_change":"2024-09-10T21:51:38.493621+0000","last_active":"2024-09-10T21:51:39.500153+0000","last_peered":"2024-09-10T21:51:39.500153+0000","last_clean":"2024-09-10T21:51:39.500153+0000","last_became_active":"2024-09-10T21:51:38.492451+0000","last_became_peered":"2024-09-10T21:51:38.492451+0000","last_unstale":"2024-09-10T21:51:39.500153+0000","last_undegraded":"2024-09-10T21:51:39.500153+0000","last_fullsized":"2024-09-10T21:51:39.500153+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:29.822792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966096+0000","last_change":"2024-09-10T21:51:37.481369+0000","last_active":"2024-09-10T21:51:39.966096+0000","last_peered":"2024-09-10T21:51:39.966096+0000","last_clean":"2024-09-10T21:51:39.966096+0000","last_became_active":"2024-09-10T21:51:37.481061+0000","last_became_peered":"2024-09-10T21:51:37.481061+0000","last_unstale":"2024-09-10T21:51:39.966096+0000","last_undegraded":"2024-09-10T21:51:39.966096+0000","last_fullsized":"2024-09-10T21:51:39.966096+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:48.767875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.836894+0000","last_change":"2024-09-10T21:51:38.501458+0000","last_active":"2024-09-10T21:51:39.836894+0000","last_peered":"2024-09-10T21:51:39.836894+0000","last_clean":"2024-09-10T21:51:39.836894+0000","last_became_active":"2024-09-10T21:51:38.500856+0000","last_became_peered":"2024-09-10T21:51:38.500856+0000","last_unstale":"2024-09-10T21:51:39.836894+0000","last_undegraded":"2024-09-10T21:51:39.836894+0000","last_fullsized":"2024-09-10T21:51:39.836894+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:01.582421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965735+0000","last_change":"2024-09-10T21:51:37.495401+0000","last_active":"2024-09-10T21:51:39.965735+0000","last_peered":"2024-09-10T21:51:39.965735+0000","last_clean":"2024-09-10T21:51:39.965735+0000","last_became_active":"2024-09-10T21:51:37.495129+0000","last_became_peered":"2024-09-10T21:51:37.495129+0000","last_unstale":"2024-09-10T21:51:39.965735+0000","last_undegraded":"2024-09-10T21:51:39.965735+0000","last_fullsized":"2024-09-10T21:51:39.965735+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:36:50.834277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966641+0000","last_change":"2024-09-10T21:51:38.507168+0000","last_active":"2024-09-10T21:51:39.966641+0000","last_peered":"2024-09-10T21:51:39.966641+0000","last_clean":"2024-09-10T21:51:39.966641+0000","last_became_active":"2024-09-10T21:51:38.506850+0000","last_became_peered":"2024-09-10T21:51:38.506850+0000","last_unstale":"2024-09-10T21:51:39.966641+0000","last_undegraded":"2024-09-10T21:51:39.966641+0000","last_fullsized":"2024-09-10T21:51:39.966641+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:40.305224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965580+0000","last_change":"2024-09-10T21:51:37.495072+0000","last_active":"2024-09-10T21:51:39.965580+0000","last_peered":"2024-09-10T21:51:39.965580+0000","last_clean":"2024-09-10T21:51:39.965580+0000","last_became_active":"2024-09-10T21:51:37.494778+0000","last_became_peered":"2024-09-10T21:51:37.494778+0000","last_unstale":"2024-09-10T21:51:39.965580+0000","last_undegraded":"2024-09-10T21:51:39.965580+0000","last_fullsized":"2024-09-10T21:51:39.965580+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:02.253882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494058+0000","last_change":"2024-09-10T21:51:38.492903+0000","last_active":"2024-09-10T21:51:39.494058+0000","last_peered":"2024-09-10T21:51:39.494058+0000","last_clean":"2024-09-10T21:51:39.494058+0000","last_became_active":"2024-09-10T21:51:38.491312+0000","last_became_peered":"2024-09-10T21:51:38.491312+0000","last_unstale":"2024-09-10T21:51:39.494058+0000","last_undegraded":"2024-09-10T21:51:39.494058+0000","last_fullsized":"2024-09-10T21:51:39.494058+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:45.871229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495488+0000","last_change":"2024-09-10T21:51:37.474801+0000","last_active":"2024-09-10T21:51:39.495488+0000","last_peered":"2024-09-10T21:51:39.495488+0000","last_clean":"2024-09-10T21:51:39.495488+0000","last_became_active":"2024-09-10T21:51:37.474455+0000","last_became_peered":"2024-09-10T21:51:37.474455+0000","last_unstale":"2024-09-10T21:51:39.495488+0000","last_undegraded":"2024-09-10T21:51:39.495488+0000","last_fullsized":"2024-09-10T21:51:39.495488+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:59.589239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496464+0000","last_change":"2024-09-10T21:51:38.496785+0000","last_active":"2024-09-10T21:51:39.496464+0000","last_peered":"2024-09-10T21:51:39.496464+0000","last_clean":"2024-09-10T21:51:39.496464+0000","last_became_active":"2024-09-10T21:51:38.496207+0000","last_became_peered":"2024-09-10T21:51:38.496207+0000","last_unstale":"2024-09-10T21:51:39.496464+0000","last_undegraded":"2024-09-10T21:51:39.496464+0000","last_fullsized":"2024-09-10T21:51:39.496464+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:16.990097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965968+0000","last_change":"2024-09-10T21:51:37.481177+0000","last_active":"2024-09-10T21:51:39.965968+0000","last_peered":"2024-09-10T21:51:39.965968+0000","last_clean":"2024-09-10T21:51:39.965968+0000","last_became_active":"2024-09-10T21:51:37.480878+0000","last_became_peered":"2024-09-10T21:51:37.480878+0000","last_unstale":"2024-09-10T21:51:39.965968+0000","last_undegraded":"2024-09-10T21:51:39.965968+0000","last_fullsized":"2024-09-10T21:51:39.965968+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:39.644340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494080+0000","last_change":"2024-09-10T21:51:38.495605+0000","last_active":"2024-09-10T21:51:39.494080+0000","last_peered":"2024-09-10T21:51:39.494080+0000","last_clean":"2024-09-10T21:51:39.494080+0000","last_became_active":"2024-09-10T21:51:38.495338+0000","last_became_peered":"2024-09-10T21:51:38.495338+0000","last_unstale":"2024-09-10T21:51:39.494080+0000","last_undegraded":"2024-09-10T21:51:39.494080+0000","last_fullsized":"2024-09-10T21:51:39.494080+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:04.658227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496179+0000","last_change":"2024-09-10T21:51:37.486466+0000","last_active":"2024-09-10T21:51:39.496179+0000","last_peered":"2024-09-10T21:51:39.496179+0000","last_clean":"2024-09-10T21:51:39.496179+0000","last_became_active":"2024-09-10T21:51:37.486160+0000","last_became_peered":"2024-09-10T21:51:37.486160+0000","last_unstale":"2024-09-10T21:51:39.496179+0000","last_undegraded":"2024-09-10T21:51:39.496179+0000","last_fullsized":"2024-09-10T21:51:39.496179+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:10.863565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495164+0000","last_change":"2024-09-10T21:51:38.493500+0000","last_active":"2024-09-10T21:51:39.495164+0000","last_peered":"2024-09-10T21:51:39.495164+0000","last_clean":"2024-09-10T21:51:39.495164+0000","last_became_active":"2024-09-10T21:51:38.491267+0000","last_became_peered":"2024-09-10T21:51:38.491267+0000","last_unstale":"2024-09-10T21:51:39.495164+0000","last_undegraded":"2024-09-10T21:51:39.495164+0000","last_fullsized":"2024-09-10T21:51:39.495164+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:04.908941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494247+0000","last_change":"2024-09-10T21:51:37.489582+0000","last_active":"2024-09-10T21:51:39.494247+0000","last_peered":"2024-09-10T21:51:39.494247+0000","last_clean":"2024-09-10T21:51:39.494247+0000","last_became_active":"2024-09-10T21:51:37.487889+0000","last_became_peered":"2024-09-10T21:51:37.487889+0000","last_unstale":"2024-09-10T21:51:39.494247+0000","last_undegraded":"2024-09-10T21:51:39.494247+0000","last_fullsized":"2024-09-10T21:51:39.494247+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:22.505773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496017+0000","last_change":"2024-09-10T21:51:38.489067+0000","last_active":"2024-09-10T21:51:39.496017+0000","last_peered":"2024-09-10T21:51:39.496017+0000","last_clean":"2024-09-10T21:51:39.496017+0000","last_became_active":"2024-09-10T21:51:38.487546+0000","last_became_peered":"2024-09-10T21:51:38.487546+0000","last_unstale":"2024-09-10T21:51:39.496017+0000","last_undegraded":"2024-09-10T21:51:39.496017+0000","last_fullsized":"2024-09-10T21:51:39.496017+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:56.724163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965825+0000","last_change":"2024-09-10T21:51:37.493056+0000","last_active":"2024-09-10T21:51:39.965825+0000","last_peered":"2024-09-10T21:51:39.965825+0000","last_clean":"2024-09-10T21:51:39.965825+0000","last_became_active":"2024-09-10T21:51:37.492756+0000","last_became_peered":"2024-09-10T21:51:37.492756+0000","last_unstale":"2024-09-10T21:51:39.965825+0000","last_undegraded":"2024-09-10T21:51:39.965825+0000","last_fullsized":"2024-09-10T21:51:39.965825+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:49.945201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.500224+0000","last_change":"2024-09-10T21:51:38.485917+0000","last_active":"2024-09-10T21:51:39.500224+0000","last_peered":"2024-09-10T21:51:39.500224+0000","last_clean":"2024-09-10T21:51:39.500224+0000","last_became_active":"2024-09-10T21:51:38.485409+0000","last_became_peered":"2024-09-10T21:51:38.485409+0000","last_unstale":"2024-09-10T21:51:39.500224+0000","last_undegraded":"2024-09-10T21:51:39.500224+0000","last_fullsized":"2024-09-10T21:51:39.500224+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:14.773637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966160+0000","last_change":"2024-09-10T21:51:37.487117+0000","last_active":"2024-09-10T21:51:39.966160+0000","last_peered":"2024-09-10T21:51:39.966160+0000","last_clean":"2024-09-10T21:51:39.966160+0000","last_became_active":"2024-09-10T21:51:37.484685+0000","last_became_peered":"2024-09-10T21:51:37.484685+0000","last_unstale":"2024-09-10T21:51:39.966160+0000","last_undegraded":"2024-09-10T21:51:39.966160+0000","last_fullsized":"2024-09-10T21:51:39.966160+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:13.540834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965991+0000","last_change":"2024-09-10T21:51:38.500232+0000","last_active":"2024-09-10T21:51:39.965991+0000","last_peered":"2024-09-10T21:51:39.965991+0000","last_clean":"2024-09-10T21:51:39.965991+0000","last_became_active":"2024-09-10T21:51:38.499959+0000","last_became_peered":"2024-09-10T21:51:38.499959+0000","last_unstale":"2024-09-10T21:51:39.965991+0000","last_undegraded":"2024-09-10T21:51:39.965991+0000","last_fullsized":"2024-09-10T21:51:39.965991+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:35.171265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"113'3","reported_seq":18,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494324+0000","last_change":"2024-09-10T21:51:37.483939+0000","last_active":"2024-09-10T21:51:39.494324+0000","last_peered":"2024-09-10T21:51:39.494324+0000","last_clean":"2024-09-10T21:51:39.494324+0000","last_became_active":"2024-09-10T21:51:37.483023+0000","last_became_peered":"2024-09-10T21:51:37.483023+0000","last_unstale":"2024-09-10T21:51:39.494324+0000","last_undegraded":"2024-09-10T21:51:39.494324+0000","last_fullsized":"2024-09-10T21:51:39.494324+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:28.514952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495638+0000","last_change":"2024-09-10T21:51:38.496905+0000","last_active":"2024-09-10T21:51:39.495638+0000","last_peered":"2024-09-10T21:51:39.495638+0000","last_clean":"2024-09-10T21:51:39.495638+0000","last_became_active":"2024-09-10T21:51:38.496075+0000","last_became_peered":"2024-09-10T21:51:38.496075+0000","last_unstale":"2024-09-10T21:51:39.495638+0000","last_undegraded":"2024-09-10T21:51:39.495638+0000","last_fullsized":"2024-09-10T21:51:39.495638+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:35.518282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"113'6","reported_seq":21,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495060+0000","last_change":"2024-09-10T21:51:37.490837+0000","last_active":"2024-09-10T21:51:39.495060+0000","last_peered":"2024-09-10T21:51:39.495060+0000","last_clean":"2024-09-10T21:51:39.495060+0000","last_became_active":"2024-09-10T21:51:37.489661+0000","last_became_peered":"2024-09-10T21:51:37.489661+0000","last_unstale":"2024-09-10T21:51:39.495060+0000","last_undegraded":"2024-09-10T21:51:39.495060+0000","last_fullsized":"2024-09-10T21:51:39.495060+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:53.621980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966089+0000","last_change":"2024-09-10T21:51:38.499244+0000","last_active":"2024-09-10T21:51:39.966089+0000","last_peered":"2024-09-10T21:51:39.966089+0000","last_clean":"2024-09-10T21:51:39.966089+0000","last_became_active":"2024-09-10T21:51:38.498965+0000","last_became_peered":"2024-09-10T21:51:38.498965+0000","last_unstale":"2024-09-10T21:51:39.966089+0000","last_undegraded":"2024-09-10T21:51:39.966089+0000","last_fullsized":"2024-09-10T21:51:39.966089+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:12.629720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.836736+0000","last_change":"2024-09-10T21:51:37.480818+0000","last_active":"2024-09-10T21:51:39.836736+0000","last_peered":"2024-09-10T21:51:39.836736+0000","last_clean":"2024-09-10T21:51:39.836736+0000","last_became_active":"2024-09-10T21:51:37.480562+0000","last_became_peered":"2024-09-10T21:51:37.480562+0000","last_unstale":"2024-09-10T21:51:39.836736+0000","last_undegraded":"2024-09-10T21:51:39.836736+0000","last_fullsized":"2024-09-10T21:51:39.836736+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:36.932605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965893+0000","last_change":"2024-09-10T21:51:38.504740+0000","last_active":"2024-09-10T21:51:39.965893+0000","last_peered":"2024-09-10T21:51:39.965893+0000","last_clean":"2024-09-10T21:51:39.965893+0000","last_became_active":"2024-09-10T21:51:38.504214+0000","last_became_peered":"2024-09-10T21:51:38.504214+0000","last_unstale":"2024-09-10T21:51:39.965893+0000","last_undegraded":"2024-09-10T21:51:39.965893+0000","last_fullsized":"2024-09-10T21:51:39.965893+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:47.805058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.499813+0000","last_change":"2024-09-10T21:51:37.491755+0000","last_active":"2024-09-10T21:51:39.499813+0000","last_peered":"2024-09-10T21:51:39.499813+0000","last_clean":"2024-09-10T21:51:39.499813+0000","last_became_active":"2024-09-10T21:51:37.491472+0000","last_became_peered":"2024-09-10T21:51:37.491472+0000","last_unstale":"2024-09-10T21:51:39.499813+0000","last_undegraded":"2024-09-10T21:51:39.499813+0000","last_fullsized":"2024-09-10T21:51:39.499813+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:15.025130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965395+0000","last_change":"2024-09-10T21:51:38.500521+0000","last_active":"2024-09-10T21:51:39.965395+0000","last_peered":"2024-09-10T21:51:39.965395+0000","last_clean":"2024-09-10T21:51:39.965395+0000","last_became_active":"2024-09-10T21:51:38.500234+0000","last_became_peered":"2024-09-10T21:51:38.500234+0000","last_unstale":"2024-09-10T21:51:39.965395+0000","last_undegraded":"2024-09-10T21:51:39.965395+0000","last_fullsized":"2024-09-10T21:51:39.965395+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:28.278479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"113'8","reported_seq":23,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965294+0000","last_change":"2024-09-10T21:51:37.492464+0000","last_active":"2024-09-10T21:51:39.965294+0000","last_peered":"2024-09-10T21:51:39.965294+0000","last_clean":"2024-09-10T21:51:39.965294+0000","last_became_active":"2024-09-10T21:51:37.492162+0000","last_became_peered":"2024-09-10T21:51:37.492162+0000","last_unstale":"2024-09-10T21:51:39.965294+0000","last_undegraded":"2024-09-10T21:51:39.965294+0000","last_fullsized":"2024-09-10T21:51:39.965294+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:40.052415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965974+0000","last_change":"2024-09-10T21:51:38.487213+0000","last_active":"2024-09-10T21:51:39.965974+0000","last_peered":"2024-09-10T21:51:39.965974+0000","last_clean":"2024-09-10T21:51:39.965974+0000","last_became_active":"2024-09-10T21:51:38.486561+0000","last_became_peered":"2024-09-10T21:51:38.486561+0000","last_unstale":"2024-09-10T21:51:39.965974+0000","last_undegraded":"2024-09-10T21:51:39.965974+0000","last_fullsized":"2024-09-10T21:51:39.965974+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:19.628610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494803+0000","last_change":"2024-09-10T21:51:37.468073+0000","last_active":"2024-09-10T21:51:39.494803+0000","last_peered":"2024-09-10T21:51:39.494803+0000","last_clean":"2024-09-10T21:51:39.494803+0000","last_became_active":"2024-09-10T21:51:37.467716+0000","last_became_peered":"2024-09-10T21:51:37.467716+0000","last_unstale":"2024-09-10T21:51:39.494803+0000","last_undegraded":"2024-09-10T21:51:39.494803+0000","last_fullsized":"2024-09-10T21:51:39.494803+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:27.988074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.837019+0000","last_change":"2024-09-10T21:51:38.492359+0000","last_active":"2024-09-10T21:51:39.837019+0000","last_peered":"2024-09-10T21:51:39.837019+0000","last_clean":"2024-09-10T21:51:39.837019+0000","last_became_active":"2024-09-10T21:51:38.490060+0000","last_became_peered":"2024-09-10T21:51:38.490060+0000","last_unstale":"2024-09-10T21:51:39.837019+0000","last_undegraded":"2024-09-10T21:51:39.837019+0000","last_fullsized":"2024-09-10T21:51:39.837019+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:18:00.595722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965984+0000","last_change":"2024-09-10T21:51:37.485673+0000","last_active":"2024-09-10T21:51:39.965984+0000","last_peered":"2024-09-10T21:51:39.965984+0000","last_clean":"2024-09-10T21:51:39.965984+0000","last_became_active":"2024-09-10T21:51:37.485132+0000","last_became_peered":"2024-09-10T21:51:37.485132+0000","last_unstale":"2024-09-10T21:51:39.965984+0000","last_undegraded":"2024-09-10T21:51:39.965984+0000","last_fullsized":"2024-09-10T21:51:39.965984+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:02.339465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966440+0000","last_change":"2024-09-10T21:51:38.505872+0000","last_active":"2024-09-10T21:51:39.966440+0000","last_peered":"2024-09-10T21:51:39.966440+0000","last_clean":"2024-09-10T21:51:39.966440+0000","last_became_active":"2024-09-10T21:51:38.505488+0000","last_became_peered":"2024-09-10T21:51:38.505488+0000","last_unstale":"2024-09-10T21:51:39.966440+0000","last_undegraded":"2024-09-10T21:51:39.966440+0000","last_fullsized":"2024-09-10T21:51:39.966440+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:29.689596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.500122+0000","last_change":"2024-09-10T21:51:37.485228+0000","last_active":"2024-09-10T21:51:39.500122+0000","last_peered":"2024-09-10T21:51:39.500122+0000","last_clean":"2024-09-10T21:51:39.500122+0000","last_became_active":"2024-09-10T21:51:37.484980+0000","last_became_peered":"2024-09-10T21:51:37.484980+0000","last_unstale":"2024-09-10T21:51:39.500122+0000","last_undegraded":"2024-09-10T21:51:39.500122+0000","last_fullsized":"2024-09-10T21:51:39.500122+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:32.265998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966710+0000","last_change":"2024-09-10T21:51:38.504879+0000","last_active":"2024-09-10T21:51:39.966710+0000","last_peered":"2024-09-10T21:51:39.966710+0000","last_clean":"2024-09-10T21:51:39.966710+0000","last_became_active":"2024-09-10T21:51:38.504560+0000","last_became_peered":"2024-09-10T21:51:38.504560+0000","last_unstale":"2024-09-10T21:51:39.966710+0000","last_undegraded":"2024-09-10T21:51:39.966710+0000","last_fullsized":"2024-09-10T21:51:39.966710+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:47.805058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495226+0000","last_change":"2024-09-10T21:51:37.469638+0000","last_active":"2024-09-10T21:51:39.495226+0000","last_peered":"2024-09-10T21:51:39.495226+0000","last_clean":"2024-09-10T21:51:39.495226+0000","last_became_active":"2024-09-10T21:51:37.467556+0000","last_became_peered":"2024-09-10T21:51:37.467556+0000","last_unstale":"2024-09-10T21:51:39.495226+0000","last_undegraded":"2024-09-10T21:51:39.495226+0000","last_fullsized":"2024-09-10T21:51:39.495226+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:13.973508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966529+0000","last_change":"2024-09-10T21:51:38.504149+0000","last_active":"2024-09-10T21:51:39.966529+0000","last_peered":"2024-09-10T21:51:39.966529+0000","last_clean":"2024-09-10T21:51:39.966529+0000","last_became_active":"2024-09-10T21:51:38.503873+0000","last_became_peered":"2024-09-10T21:51:38.503873+0000","last_unstale":"2024-09-10T21:51:39.966529+0000","last_undegraded":"2024-09-10T21:51:39.966529+0000","last_fullsized":"2024-09-10T21:51:39.966529+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:38.756786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.836754+0000","last_change":"2024-09-10T21:51:37.488302+0000","last_active":"2024-09-10T21:51:39.836754+0000","last_peered":"2024-09-10T21:51:39.836754+0000","last_clean":"2024-09-10T21:51:39.836754+0000","last_became_active":"2024-09-10T21:51:37.488054+0000","last_became_peered":"2024-09-10T21:51:37.488054+0000","last_unstale":"2024-09-10T21:51:39.836754+0000","last_undegraded":"2024-09-10T21:51:39.836754+0000","last_fullsized":"2024-09-10T21:51:39.836754+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:51.988103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494166+0000","last_change":"2024-09-10T21:51:38.500053+0000","last_active":"2024-09-10T21:51:39.494166+0000","last_peered":"2024-09-10T21:51:39.494166+0000","last_clean":"2024-09-10T21:51:39.494166+0000","last_became_active":"2024-09-10T21:51:38.499702+0000","last_became_peered":"2024-09-10T21:51:38.499702+0000","last_unstale":"2024-09-10T21:51:39.494166+0000","last_undegraded":"2024-09-10T21:51:39.494166+0000","last_fullsized":"2024-09-10T21:51:39.494166+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:23.077848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965745+0000","last_change":"2024-09-10T21:51:37.495655+0000","last_active":"2024-09-10T21:51:39.965745+0000","last_peered":"2024-09-10T21:51:39.965745+0000","last_clean":"2024-09-10T21:51:39.965745+0000","last_became_active":"2024-09-10T21:51:37.495383+0000","last_became_peered":"2024-09-10T21:51:37.495383+0000","last_unstale":"2024-09-10T21:51:39.965745+0000","last_undegraded":"2024-09-10T21:51:39.965745+0000","last_fullsized":"2024-09-10T21:51:39.965745+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:57.108755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966363+0000","last_change":"2024-09-10T21:51:38.498184+0000","last_active":"2024-09-10T21:51:39.966363+0000","last_peered":"2024-09-10T21:51:39.966363+0000","last_clean":"2024-09-10T21:51:39.966363+0000","last_became_active":"2024-09-10T21:51:38.497840+0000","last_became_peered":"2024-09-10T21:51:38.497840+0000","last_unstale":"2024-09-10T21:51:39.966363+0000","last_undegraded":"2024-09-10T21:51:39.966363+0000","last_fullsized":"2024-09-10T21:51:39.966363+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:50.132177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.836599+0000","last_change":"2024-09-10T21:51:37.493629+0000","last_active":"2024-09-10T21:51:39.836599+0000","last_peered":"2024-09-10T21:51:39.836599+0000","last_clean":"2024-09-10T21:51:39.836599+0000","last_became_active":"2024-09-10T21:51:37.493347+0000","last_became_peered":"2024-09-10T21:51:37.493347+0000","last_unstale":"2024-09-10T21:51:39.836599+0000","last_undegraded":"2024-09-10T21:51:39.836599+0000","last_fullsized":"2024-09-10T21:51:39.836599+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:06.890765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965539+0000","last_change":"2024-09-10T21:51:38.497499+0000","last_active":"2024-09-10T21:51:39.965539+0000","last_peered":"2024-09-10T21:51:39.965539+0000","last_clean":"2024-09-10T21:51:39.965539+0000","last_became_active":"2024-09-10T21:51:38.494172+0000","last_became_peered":"2024-09-10T21:51:38.494172+0000","last_unstale":"2024-09-10T21:51:39.965539+0000","last_undegraded":"2024-09-10T21:51:39.965539+0000","last_fullsized":"2024-09-10T21:51:39.965539+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:38.756786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494370+0000","last_change":"2024-09-10T21:51:37.490268+0000","last_active":"2024-09-10T21:51:39.494370+0000","last_peered":"2024-09-10T21:51:39.494370+0000","last_clean":"2024-09-10T21:51:39.494370+0000","last_became_active":"2024-09-10T21:51:37.488772+0000","last_became_peered":"2024-09-10T21:51:37.488772+0000","last_unstale":"2024-09-10T21:51:39.494370+0000","last_undegraded":"2024-09-10T21:51:39.494370+0000","last_fullsized":"2024-09-10T21:51:39.494370+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:47.135421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.499991+0000","last_change":"2024-09-10T21:51:38.485773+0000","last_active":"2024-09-10T21:51:39.499991+0000","last_peered":"2024-09-10T21:51:39.499991+0000","last_clean":"2024-09-10T21:51:39.499991+0000","last_became_active":"2024-09-10T21:51:38.485453+0000","last_became_peered":"2024-09-10T21:51:38.485453+0000","last_unstale":"2024-09-10T21:51:39.499991+0000","last_undegraded":"2024-09-10T21:51:39.499991+0000","last_fullsized":"2024-09-10T21:51:39.499991+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:18.728391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966234+0000","last_change":"2024-09-10T21:51:37.493424+0000","last_active":"2024-09-10T21:51:39.966234+0000","last_peered":"2024-09-10T21:51:39.966234+0000","last_clean":"2024-09-10T21:51:39.966234+0000","last_became_active":"2024-09-10T21:51:37.492964+0000","last_became_peered":"2024-09-10T21:51:37.492964+0000","last_unstale":"2024-09-10T21:51:39.966234+0000","last_undegraded":"2024-09-10T21:51:39.966234+0000","last_fullsized":"2024-09-10T21:51:39.966234+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:56.310705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494784+0000","last_change":"2024-09-10T21:51:38.486288+0000","last_active":"2024-09-10T21:51:39.494784+0000","last_peered":"2024-09-10T21:51:39.494784+0000","last_clean":"2024-09-10T21:51:39.494784+0000","last_became_active":"2024-09-10T21:51:38.486013+0000","last_became_peered":"2024-09-10T21:51:38.486013+0000","last_unstale":"2024-09-10T21:51:39.494784+0000","last_undegraded":"2024-09-10T21:51:39.494784+0000","last_fullsized":"2024-09-10T21:51:39.494784+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:38.040724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"114'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.500176+0000","last_change":"2024-09-10T21:51:37.481469+0000","last_active":"2024-09-10T21:51:39.500176+0000","last_peered":"2024-09-10T21:51:39.500176+0000","last_clean":"2024-09-10T21:51:39.500176+0000","last_became_active":"2024-09-10T21:51:37.481196+0000","last_became_peered":"2024-09-10T21:51:37.481196+0000","last_unstale":"2024-09-10T21:51:39.500176+0000","last_undegraded":"2024-09-10T21:51:39.500176+0000","last_fullsized":"2024-09-10T21:51:39.500176+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:35.185924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494858+0000","last_change":"2024-09-10T21:51:38.493151+0000","last_active":"2024-09-10T21:51:39.494858+0000","last_peered":"2024-09-10T21:51:39.494858+0000","last_clean":"2024-09-10T21:51:39.494858+0000","last_became_active":"2024-09-10T21:51:38.492219+0000","last_became_peered":"2024-09-10T21:51:38.492219+0000","last_unstale":"2024-09-10T21:51:39.494858+0000","last_undegraded":"2024-09-10T21:51:39.494858+0000","last_fullsized":"2024-09-10T21:51:39.494858+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:05.522186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494373+0000","last_change":"2024-09-10T21:51:37.490280+0000","last_active":"2024-09-10T21:51:39.494373+0000","last_peered":"2024-09-10T21:51:39.494373+0000","last_clean":"2024-09-10T21:51:39.494373+0000","last_became_active":"2024-09-10T21:51:37.489979+0000","last_became_peered":"2024-09-10T21:51:37.489979+0000","last_unstale":"2024-09-10T21:51:39.494373+0000","last_undegraded":"2024-09-10T21:51:39.494373+0000","last_fullsized":"2024-09-10T21:51:39.494373+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:24.259852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.500006+0000","last_change":"2024-09-10T21:51:38.484059+0000","last_active":"2024-09-10T21:51:39.500006+0000","last_peered":"2024-09-10T21:51:39.500006+0000","last_clean":"2024-09-10T21:51:39.500006+0000","last_became_active":"2024-09-10T21:51:38.483600+0000","last_became_peered":"2024-09-10T21:51:38.483600+0000","last_unstale":"2024-09-10T21:51:39.500006+0000","last_undegraded":"2024-09-10T21:51:39.500006+0000","last_fullsized":"2024-09-10T21:51:39.500006+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:55.459385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966131+0000","last_change":"2024-09-10T21:51:37.496547+0000","last_active":"2024-09-10T21:51:39.966131+0000","last_peered":"2024-09-10T21:51:39.966131+0000","last_clean":"2024-09-10T21:51:39.966131+0000","last_became_active":"2024-09-10T21:51:37.495965+0000","last_became_peered":"2024-09-10T21:51:37.495965+0000","last_unstale":"2024-09-10T21:51:39.966131+0000","last_undegraded":"2024-09-10T21:51:39.966131+0000","last_fullsized":"2024-09-10T21:51:39.966131+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:45.517218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496291+0000","last_change":"2024-09-10T21:51:38.496844+0000","last_active":"2024-09-10T21:51:39.496291+0000","last_peered":"2024-09-10T21:51:39.496291+0000","last_clean":"2024-09-10T21:51:39.496291+0000","last_became_active":"2024-09-10T21:51:38.495814+0000","last_became_peered":"2024-09-10T21:51:38.495814+0000","last_unstale":"2024-09-10T21:51:39.496291+0000","last_undegraded":"2024-09-10T21:51:39.496291+0000","last_fullsized":"2024-09-10T21:51:39.496291+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:13.139951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495600+0000","last_change":"2024-09-10T21:51:37.477655+0000","last_active":"2024-09-10T21:51:39.495600+0000","last_peered":"2024-09-10T21:51:39.495600+0000","last_clean":"2024-09-10T21:51:39.495600+0000","last_became_active":"2024-09-10T21:51:37.477156+0000","last_became_peered":"2024-09-10T21:51:37.477156+0000","last_unstale":"2024-09-10T21:51:39.495600+0000","last_undegraded":"2024-09-10T21:51:39.495600+0000","last_fullsized":"2024-09-10T21:51:39.495600+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:26.947972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.497006+0000","last_change":"2024-09-10T21:51:38.488657+0000","last_active":"2024-09-10T21:51:39.497006+0000","last_peered":"2024-09-10T21:51:39.497006+0000","last_clean":"2024-09-10T21:51:39.497006+0000","last_became_active":"2024-09-10T21:51:38.488017+0000","last_became_peered":"2024-09-10T21:51:38.488017+0000","last_unstale":"2024-09-10T21:51:39.497006+0000","last_undegraded":"2024-09-10T21:51:39.497006+0000","last_fullsized":"2024-09-10T21:51:39.497006+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.698033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496329+0000","last_change":"2024-09-10T21:51:37.478410+0000","last_active":"2024-09-10T21:51:39.496329+0000","last_peered":"2024-09-10T21:51:39.496329+0000","last_clean":"2024-09-10T21:51:39.496329+0000","last_became_active":"2024-09-10T21:51:37.478141+0000","last_became_peered":"2024-09-10T21:51:37.478141+0000","last_unstale":"2024-09-10T21:51:39.496329+0000","last_undegraded":"2024-09-10T21:51:39.496329+0000","last_fullsized":"2024-09-10T21:51:39.496329+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:52.334651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966600+0000","last_change":"2024-09-10T21:51:38.501070+0000","last_active":"2024-09-10T21:51:39.966600+0000","last_peered":"2024-09-10T21:51:39.966600+0000","last_clean":"2024-09-10T21:51:39.966600+0000","last_became_active":"2024-09-10T21:51:38.500807+0000","last_became_peered":"2024-09-10T21:51:38.500807+0000","last_unstale":"2024-09-10T21:51:39.966600+0000","last_undegraded":"2024-09-10T21:51:39.966600+0000","last_fullsized":"2024-09-10T21:51:39.966600+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:41.427352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965553+0000","last_change":"2024-09-10T21:51:37.494039+0000","last_active":"2024-09-10T21:51:39.965553+0000","last_peered":"2024-09-10T21:51:39.965553+0000","last_clean":"2024-09-10T21:51:39.965553+0000","last_became_active":"2024-09-10T21:51:37.493739+0000","last_became_peered":"2024-09-10T21:51:37.493739+0000","last_unstale":"2024-09-10T21:51:39.965553+0000","last_undegraded":"2024-09-10T21:51:39.965553+0000","last_fullsized":"2024-09-10T21:51:39.965553+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:41:40.285518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494056+0000","last_change":"2024-09-10T21:51:38.496313+0000","last_active":"2024-09-10T21:51:39.494056+0000","last_peered":"2024-09-10T21:51:39.494056+0000","last_clean":"2024-09-10T21:51:39.494056+0000","last_became_active":"2024-09-10T21:51:38.495521+0000","last_became_peered":"2024-09-10T21:51:38.495521+0000","last_unstale":"2024-09-10T21:51:39.494056+0000","last_undegraded":"2024-09-10T21:51:39.494056+0000","last_fullsized":"2024-09-10T21:51:39.494056+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:51.059341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494498+0000","last_change":"2024-09-10T21:51:37.489957+0000","last_active":"2024-09-10T21:51:39.494498+0000","last_peered":"2024-09-10T21:51:39.494498+0000","last_clean":"2024-09-10T21:51:39.494498+0000","last_became_active":"2024-09-10T21:51:37.488416+0000","last_became_peered":"2024-09-10T21:51:37.488416+0000","last_unstale":"2024-09-10T21:51:39.494498+0000","last_undegraded":"2024-09-10T21:51:39.494498+0000","last_fullsized":"2024-09-10T21:51:39.494498+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:37.929683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965291+0000","last_change":"2024-09-10T21:51:38.495176+0000","last_active":"2024-09-10T21:51:39.965291+0000","last_peered":"2024-09-10T21:51:39.965291+0000","last_clean":"2024-09-10T21:51:39.965291+0000","last_became_active":"2024-09-10T21:51:38.494857+0000","last_became_peered":"2024-09-10T21:51:38.494857+0000","last_unstale":"2024-09-10T21:51:39.965291+0000","last_undegraded":"2024-09-10T21:51:39.965291+0000","last_fullsized":"2024-09-10T21:51:39.965291+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:32.797143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495810+0000","last_change":"2024-09-10T21:51:37.467889+0000","last_active":"2024-09-10T21:51:39.495810+0000","last_peered":"2024-09-10T21:51:39.495810+0000","last_clean":"2024-09-10T21:51:39.495810+0000","last_became_active":"2024-09-10T21:51:37.467603+0000","last_became_peered":"2024-09-10T21:51:37.467603+0000","last_unstale":"2024-09-10T21:51:39.495810+0000","last_undegraded":"2024-09-10T21:51:39.495810+0000","last_fullsized":"2024-09-10T21:51:39.495810+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:38.793976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":281,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966593+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:51:39.966593+0000","last_peered":"2024-09-10T21:51:39.966593+0000","last_clean":"2024-09-10T21:51:39.966593+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:51:39.966593+0000","last_undegraded":"2024-09-10T21:51:39.966593+0000","last_fullsized":"2024-09-10T21:51:39.966593+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509561,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1413120,"data_stored":610155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509561,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1417216,"data_stored":610689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1409024,"data_stored":610085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509560,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1409024,"data_stored":610085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509562,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36732,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335108,"statfs":{"total":96636764160,"available":96599150592,"internally_reserved":0,"allocated":1417216,"data_stored":610209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509560,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37332,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334508,"statfs":{"total":96636764160,"available":96598536192,"internally_reserved":0,"allocated":2023424,"data_stored":1202177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542263,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2011136,"data_stored":1201069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542264,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36744,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335096,"statfs":{"total":96636764160,"available":96599138304,"internally_reserved":0,"allocated":1425408,"data_stored":611767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:51:47.482 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:51:47.655 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.654+0000 7f8150400640 1 -- 172.21.15.7:0/4215289277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81481604b0 msgr2=0x7f81480ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:47.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.654+0000 7f8150400640 1 --2- 172.21.15.7:0/4215289277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81481604b0 0x7f81480ccef0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f8140005fd0 tx=0x7f814002a8b0 comp rx=0 tx=0).stop 2024-09-10T21:51:47.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.654+0000 7f8150400640 1 -- 172.21.15.7:0/4215289277 shutdown_connections 2024-09-10T21:51:47.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.654+0000 7f8150400640 1 --2- 172.21.15.7:0/4215289277 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f81481604b0 0x7f81480ccef0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:47.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.654+0000 7f8150400640 1 -- 172.21.15.7:0/4215289277 >> 172.21.15.7:0/4215289277 conn(0x7f81480b16d0 msgr2=0x7f81480b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:47.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.654+0000 7f8150400640 1 -- 172.21.15.7:0/4215289277 shutdown_connections 2024-09-10T21:51:47.656 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.654+0000 7f8150400640 1 -- 172.21.15.7:0/4215289277 wait complete. 2024-09-10T21:51:47.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.656+0000 7f8150400640 1 Processor -- start 2024-09-10T21:51:47.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.656+0000 7f8150400640 1 -- start start 2024-09-10T21:51:47.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.657+0000 7f8150400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81480cd750 0x7f8148159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:47.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.657+0000 7f8150400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f814815bc90 0x7f81481f3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:47.659 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.657+0000 7f8150400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81481604b0 0x7f8148072360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.657+0000 7f8150400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8148070080 con 0x7f814815bc90 2024-09-10T21:51:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.657+0000 7f8150400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f814806ff00 con 0x7f81480cd750 2024-09-10T21:51:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.657+0000 7f8150400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f8148070200 con 0x7f81481604b0 2024-09-10T21:51:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.657+0000 7f814d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f814815bc90 0x7f81481f3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.657+0000 7f814d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f814815bc90 0x7f81481f3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43784/0 (socket says 172.21.15.7:43784) 2024-09-10T21:51:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.657+0000 7f814d000640 1 -- 172.21.15.7:0/580749112 learned_addr learned my addr 172.21.15.7:0/580749112 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.657+0000 7f814da00640 1 --2- 172.21.15.7:0/580749112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81480cd750 0x7f8148159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.658+0000 7f814e400640 1 --2- 172.21.15.7:0/580749112 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81481604b0 0x7f8148072360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.658+0000 7f814d000640 1 -- 172.21.15.7:0/580749112 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81481604b0 msgr2=0x7f8148072360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.658+0000 7f814d000640 1 --2- 172.21.15.7:0/580749112 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81481604b0 0x7f8148072360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.658+0000 7f814d000640 1 -- 172.21.15.7:0/580749112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81480cd750 msgr2=0x7f8148159a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.658+0000 7f814d000640 1 --2- 172.21.15.7:0/580749112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81480cd750 0x7f8148159a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:47.660 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.658+0000 7f814d000640 1 -- 172.21.15.7:0/580749112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8148159f80 con 0x7f814815bc90 2024-09-10T21:51:47.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.658+0000 7f814d000640 1 --2- 172.21.15.7:0/580749112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f814815bc90 0x7f81481f3cf0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f813400c440 tx=0x7f813400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:47.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.658+0000 7f8145600640 1 -- 172.21.15.7:0/580749112 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134017070 con 0x7f814815bc90 2024-09-10T21:51:47.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.658+0000 7f8145600640 1 -- 172.21.15.7:0/580749112 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81340035f0 con 0x7f814815bc90 2024-09-10T21:51:47.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.658+0000 7f8145600640 1 -- 172.21.15.7:0/580749112 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8134003a40 con 0x7f814815bc90 2024-09-10T21:51:47.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.659+0000 7f8150400640 1 -- 172.21.15.7:0/580749112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81481f46a0 con 0x7f814815bc90 2024-09-10T21:51:47.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.659+0000 7f8150400640 1 -- 172.21.15.7:0/580749112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81481f4c60 con 0x7f814815bc90 2024-09-10T21:51:47.661 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.660+0000 7f8150400640 1 -- 172.21.15.7:0/580749112 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81480706b0 con 0x7f814815bc90 2024-09-10T21:51:47.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.661+0000 7f8145600640 1 -- 172.21.15.7:0/580749112 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8134003be0 con 0x7f814815bc90 2024-09-10T21:51:47.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.662+0000 7f8145600640 1 --2- 172.21.15.7:0/580749112 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8128077f70 0x7f812807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:47.662 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.662+0000 7f8145600640 1 -- 172.21.15.7:0/580749112 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5709+0+0 (secure 0 0 0) 0x7f8134099330 con 0x7f814815bc90 2024-09-10T21:51:47.663 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.662+0000 7f814da00640 1 --2- 172.21.15.7:0/580749112 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8128077f70 0x7f812807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:47.664 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.663+0000 7f814da00640 1 --2- 172.21.15.7:0/580749112 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8128077f70 0x7f812807a430 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f813c000f60 tx=0x7f813c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:47.666 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.666+0000 7f8145600640 1 -- 172.21.15.7:0/580749112 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f8134061740 con 0x7f814815bc90 2024-09-10T21:51:47.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.856+0000 7f8150400640 1 -- 172.21.15.7:0/580749112 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8148069810 con 0x7f8128077f70 2024-09-10T21:51:47.862 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.861+0000 7f8145600640 1 -- 172.21.15.7:0/580749112 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252472 (secure 0 0 0) 0x7f8148069810 con 0x7f8128077f70 2024-09-10T21:51:47.865 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:47.866 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:51:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.866+0000 7f8150400640 1 -- 172.21.15.7:0/580749112 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8128077f70 msgr2=0x7f812807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.866+0000 7f8150400640 1 --2- 172.21.15.7:0/580749112 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8128077f70 0x7f812807a430 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f813c000f60 tx=0x7f813c009040 comp rx=0 tx=0).stop 2024-09-10T21:51:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.866+0000 7f8150400640 1 -- 172.21.15.7:0/580749112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f814815bc90 msgr2=0x7f81481f3cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.866+0000 7f8150400640 1 --2- 172.21.15.7:0/580749112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f814815bc90 0x7f81481f3cf0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f813400c440 tx=0x7f813400c900 comp rx=0 tx=0).stop 2024-09-10T21:51:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.866+0000 7f8150400640 1 -- 172.21.15.7:0/580749112 shutdown_connections 2024-09-10T21:51:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.866+0000 7f8150400640 1 --2- 172.21.15.7:0/580749112 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8128077f70 0x7f812807a430 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.866+0000 7f8150400640 1 --2- 172.21.15.7:0/580749112 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f81481604b0 0x7f8148072360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.866+0000 7f8150400640 1 --2- 172.21.15.7:0/580749112 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f814815bc90 0x7f81481f3cf0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.866+0000 7f8150400640 1 --2- 172.21.15.7:0/580749112 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f81480cd750 0x7f8148159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.866+0000 7f8150400640 1 -- 172.21.15.7:0/580749112 >> 172.21.15.7:0/580749112 conn(0x7f81480b16d0 msgr2=0x7f81480b42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:47.868 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.866+0000 7f8150400640 1 -- 172.21.15.7:0/580749112 shutdown_connections 2024-09-10T21:51:47.869 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:47.866+0000 7f8150400640 1 -- 172.21.15.7:0/580749112 wait complete. 2024-09-10T21:51:47.883 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-09-10T21:51:46.009550+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":274240,"kb_used_data":12232,"kb_used_omap":12,"kb_used_meta":261811,"kb_avail":754700480,"statfs":{"total":773094113280,"available":772813291520,"internally_reserved":0,"allocated":12525568,"data_stored":6066236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":268095025},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002357"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494583+0000","last_change":"2024-09-10T21:51:38.494481+0000","last_active":"2024-09-10T21:51:39.494583+0000","last_peered":"2024-09-10T21:51:39.494583+0000","last_clean":"2024-09-10T21:51:39.494583+0000","last_became_active":"2024-09-10T21:51:38.494199+0000","last_became_peered":"2024-09-10T21:51:38.494199+0000","last_unstale":"2024-09-10T21:51:39.494583+0000","last_undegraded":"2024-09-10T21:51:39.494583+0000","last_fullsized":"2024-09-10T21:51:39.494583+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:15:40.667423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496904+0000","last_change":"2024-09-10T21:51:38.493007+0000","last_active":"2024-09-10T21:51:39.496904+0000","last_peered":"2024-09-10T21:51:39.496904+0000","last_clean":"2024-09-10T21:51:39.496904+0000","last_became_active":"2024-09-10T21:51:38.492740+0000","last_became_peered":"2024-09-10T21:51:38.492740+0000","last_unstale":"2024-09-10T21:51:39.496904+0000","last_undegraded":"2024-09-10T21:51:39.496904+0000","last_fullsized":"2024-09-10T21:51:39.496904+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:18.728391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966452+0000","last_change":"2024-09-10T21:51:38.505264+0000","last_active":"2024-09-10T21:51:39.966452+0000","last_peered":"2024-09-10T21:51:39.966452+0000","last_clean":"2024-09-10T21:51:39.966452+0000","last_became_active":"2024-09-10T21:51:38.504121+0000","last_became_peered":"2024-09-10T21:51:38.504121+0000","last_unstale":"2024-09-10T21:51:39.966452+0000","last_undegraded":"2024-09-10T21:51:39.966452+0000","last_fullsized":"2024-09-10T21:51:39.966452+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:32.797143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495139+0000","last_change":"2024-09-10T21:51:38.492127+0000","last_active":"2024-09-10T21:51:39.495139+0000","last_peered":"2024-09-10T21:51:39.495139+0000","last_clean":"2024-09-10T21:51:39.495139+0000","last_became_active":"2024-09-10T21:51:38.491848+0000","last_became_peered":"2024-09-10T21:51:38.491848+0000","last_unstale":"2024-09-10T21:51:39.495139+0000","last_undegraded":"2024-09-10T21:51:39.495139+0000","last_fullsized":"2024-09-10T21:51:39.495139+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:35:34.188188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965536+0000","last_change":"2024-09-10T21:51:38.489365+0000","last_active":"2024-09-10T21:51:39.965536+0000","last_peered":"2024-09-10T21:51:39.965536+0000","last_clean":"2024-09-10T21:51:39.965536+0000","last_became_active":"2024-09-10T21:51:38.489084+0000","last_became_peered":"2024-09-10T21:51:38.489084+0000","last_unstale":"2024-09-10T21:51:39.965536+0000","last_undegraded":"2024-09-10T21:51:39.965536+0000","last_fullsized":"2024-09-10T21:51:39.965536+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:12:09.526080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.836686+0000","last_change":"2024-09-10T21:51:38.495769+0000","last_active":"2024-09-10T21:51:39.836686+0000","last_peered":"2024-09-10T21:51:39.836686+0000","last_clean":"2024-09-10T21:51:39.836686+0000","last_became_active":"2024-09-10T21:51:38.495508+0000","last_became_peered":"2024-09-10T21:51:38.495508+0000","last_unstale":"2024-09-10T21:51:39.836686+0000","last_undegraded":"2024-09-10T21:51:39.836686+0000","last_fullsized":"2024-09-10T21:51:39.836686+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:12.480879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495331+0000","last_change":"2024-09-10T21:51:38.484984+0000","last_active":"2024-09-10T21:51:39.495331+0000","last_peered":"2024-09-10T21:51:39.495331+0000","last_clean":"2024-09-10T21:51:39.495331+0000","last_became_active":"2024-09-10T21:51:38.483932+0000","last_became_peered":"2024-09-10T21:51:38.483932+0000","last_unstale":"2024-09-10T21:51:39.495331+0000","last_undegraded":"2024-09-10T21:51:39.495331+0000","last_fullsized":"2024-09-10T21:51:39.495331+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:07:24.626861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966436+0000","last_change":"2024-09-10T21:51:38.498466+0000","last_active":"2024-09-10T21:51:39.966436+0000","last_peered":"2024-09-10T21:51:39.966436+0000","last_clean":"2024-09-10T21:51:39.966436+0000","last_became_active":"2024-09-10T21:51:38.498041+0000","last_became_peered":"2024-09-10T21:51:38.498041+0000","last_unstale":"2024-09-10T21:51:39.966436+0000","last_undegraded":"2024-09-10T21:51:39.966436+0000","last_fullsized":"2024-09-10T21:51:39.966436+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:19.954387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494640+0000","last_change":"2024-09-10T21:51:38.489094+0000","last_active":"2024-09-10T21:51:39.494640+0000","last_peered":"2024-09-10T21:51:39.494640+0000","last_clean":"2024-09-10T21:51:39.494640+0000","last_became_active":"2024-09-10T21:51:38.488756+0000","last_became_peered":"2024-09-10T21:51:38.488756+0000","last_unstale":"2024-09-10T21:51:39.494640+0000","last_undegraded":"2024-09-10T21:51:39.494640+0000","last_fullsized":"2024-09-10T21:51:39.494640+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:28.335704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494779+0000","last_change":"2024-09-10T21:51:38.499151+0000","last_active":"2024-09-10T21:51:39.494779+0000","last_peered":"2024-09-10T21:51:39.494779+0000","last_clean":"2024-09-10T21:51:39.494779+0000","last_became_active":"2024-09-10T21:51:38.498858+0000","last_became_peered":"2024-09-10T21:51:38.498858+0000","last_unstale":"2024-09-10T21:51:39.494779+0000","last_undegraded":"2024-09-10T21:51:39.494779+0000","last_fullsized":"2024-09-10T21:51:39.494779+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:39:11.108209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496669+0000","last_change":"2024-09-10T21:51:38.491484+0000","last_active":"2024-09-10T21:51:39.496669+0000","last_peered":"2024-09-10T21:51:39.496669+0000","last_clean":"2024-09-10T21:51:39.496669+0000","last_became_active":"2024-09-10T21:51:38.491193+0000","last_became_peered":"2024-09-10T21:51:38.491193+0000","last_unstale":"2024-09-10T21:51:39.496669+0000","last_undegraded":"2024-09-10T21:51:39.496669+0000","last_fullsized":"2024-09-10T21:51:39.496669+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:28.006935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494629+0000","last_change":"2024-09-10T21:51:38.491409+0000","last_active":"2024-09-10T21:51:39.494629+0000","last_peered":"2024-09-10T21:51:39.494629+0000","last_clean":"2024-09-10T21:51:39.494629+0000","last_became_active":"2024-09-10T21:51:38.490377+0000","last_became_peered":"2024-09-10T21:51:38.490377+0000","last_unstale":"2024-09-10T21:51:39.494629+0000","last_undegraded":"2024-09-10T21:51:39.494629+0000","last_fullsized":"2024-09-10T21:51:39.494629+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:33.606508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966404+0000","last_change":"2024-09-10T21:51:38.503338+0000","last_active":"2024-09-10T21:51:39.966404+0000","last_peered":"2024-09-10T21:51:39.966404+0000","last_clean":"2024-09-10T21:51:39.966404+0000","last_became_active":"2024-09-10T21:51:38.501614+0000","last_became_peered":"2024-09-10T21:51:38.501614+0000","last_unstale":"2024-09-10T21:51:39.966404+0000","last_undegraded":"2024-09-10T21:51:39.966404+0000","last_fullsized":"2024-09-10T21:51:39.966404+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:30:10.529060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966266+0000","last_change":"2024-09-10T21:51:38.498967+0000","last_active":"2024-09-10T21:51:39.966266+0000","last_peered":"2024-09-10T21:51:39.966266+0000","last_clean":"2024-09-10T21:51:39.966266+0000","last_became_active":"2024-09-10T21:51:38.498641+0000","last_became_peered":"2024-09-10T21:51:38.498641+0000","last_unstale":"2024-09-10T21:51:39.966266+0000","last_undegraded":"2024-09-10T21:51:39.966266+0000","last_fullsized":"2024-09-10T21:51:39.966266+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:41.424015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.836623+0000","last_change":"2024-09-10T21:51:38.494202+0000","last_active":"2024-09-10T21:51:39.836623+0000","last_peered":"2024-09-10T21:51:39.836623+0000","last_clean":"2024-09-10T21:51:39.836623+0000","last_became_active":"2024-09-10T21:51:38.493912+0000","last_became_peered":"2024-09-10T21:51:38.493912+0000","last_unstale":"2024-09-10T21:51:39.836623+0000","last_undegraded":"2024-09-10T21:51:39.836623+0000","last_fullsized":"2024-09-10T21:51:39.836623+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:18:51.129806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494544+0000","last_change":"2024-09-10T21:51:38.502368+0000","last_active":"2024-09-10T21:51:39.494544+0000","last_peered":"2024-09-10T21:51:39.494544+0000","last_clean":"2024-09-10T21:51:39.494544+0000","last_became_active":"2024-09-10T21:51:38.502112+0000","last_became_peered":"2024-09-10T21:51:38.502112+0000","last_unstale":"2024-09-10T21:51:39.494544+0000","last_undegraded":"2024-09-10T21:51:39.494544+0000","last_fullsized":"2024-09-10T21:51:39.494544+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:14:03.034899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496859+0000","last_change":"2024-09-10T21:51:38.495199+0000","last_active":"2024-09-10T21:51:39.496859+0000","last_peered":"2024-09-10T21:51:39.496859+0000","last_clean":"2024-09-10T21:51:39.496859+0000","last_became_active":"2024-09-10T21:51:38.494943+0000","last_became_peered":"2024-09-10T21:51:38.494943+0000","last_unstale":"2024-09-10T21:51:39.496859+0000","last_undegraded":"2024-09-10T21:51:39.496859+0000","last_fullsized":"2024-09-10T21:51:39.496859+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:05.034431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.499881+0000","last_change":"2024-09-10T21:51:38.487185+0000","last_active":"2024-09-10T21:51:39.499881+0000","last_peered":"2024-09-10T21:51:39.499881+0000","last_clean":"2024-09-10T21:51:39.499881+0000","last_became_active":"2024-09-10T21:51:38.486876+0000","last_became_peered":"2024-09-10T21:51:38.486876+0000","last_unstale":"2024-09-10T21:51:39.499881+0000","last_undegraded":"2024-09-10T21:51:39.499881+0000","last_fullsized":"2024-09-10T21:51:39.499881+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:28:30.506461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966459+0000","last_change":"2024-09-10T21:51:38.506319+0000","last_active":"2024-09-10T21:51:39.966459+0000","last_peered":"2024-09-10T21:51:39.966459+0000","last_clean":"2024-09-10T21:51:39.966459+0000","last_became_active":"2024-09-10T21:51:38.506041+0000","last_became_peered":"2024-09-10T21:51:38.506041+0000","last_unstale":"2024-09-10T21:51:39.966459+0000","last_undegraded":"2024-09-10T21:51:39.966459+0000","last_fullsized":"2024-09-10T21:51:39.966459+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:33.484489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496585+0000","last_change":"2024-09-10T21:51:38.499791+0000","last_active":"2024-09-10T21:51:39.496585+0000","last_peered":"2024-09-10T21:51:39.496585+0000","last_clean":"2024-09-10T21:51:39.496585+0000","last_became_active":"2024-09-10T21:51:38.499505+0000","last_became_peered":"2024-09-10T21:51:38.499505+0000","last_unstale":"2024-09-10T21:51:39.496585+0000","last_undegraded":"2024-09-10T21:51:39.496585+0000","last_fullsized":"2024-09-10T21:51:39.496585+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:18:04.591632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496782+0000","last_change":"2024-09-10T21:51:38.502073+0000","last_active":"2024-09-10T21:51:39.496782+0000","last_peered":"2024-09-10T21:51:39.496782+0000","last_clean":"2024-09-10T21:51:39.496782+0000","last_became_active":"2024-09-10T21:51:38.501750+0000","last_became_peered":"2024-09-10T21:51:38.501750+0000","last_unstale":"2024-09-10T21:51:39.496782+0000","last_undegraded":"2024-09-10T21:51:39.496782+0000","last_fullsized":"2024-09-10T21:51:39.496782+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:24.265846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965676+0000","last_change":"2024-09-10T21:51:38.495309+0000","last_active":"2024-09-10T21:51:39.965676+0000","last_peered":"2024-09-10T21:51:39.965676+0000","last_clean":"2024-09-10T21:51:39.965676+0000","last_became_active":"2024-09-10T21:51:38.494998+0000","last_became_peered":"2024-09-10T21:51:38.494998+0000","last_unstale":"2024-09-10T21:51:39.965676+0000","last_undegraded":"2024-09-10T21:51:39.965676+0000","last_fullsized":"2024-09-10T21:51:39.965676+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:41.427352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495913+0000","last_change":"2024-09-10T21:51:38.499369+0000","last_active":"2024-09-10T21:51:39.495913+0000","last_peered":"2024-09-10T21:51:39.495913+0000","last_clean":"2024-09-10T21:51:39.495913+0000","last_became_active":"2024-09-10T21:51:38.499094+0000","last_became_peered":"2024-09-10T21:51:38.499094+0000","last_unstale":"2024-09-10T21:51:39.495913+0000","last_undegraded":"2024-09-10T21:51:39.495913+0000","last_fullsized":"2024-09-10T21:51:39.495913+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:36.192408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965229+0000","last_change":"2024-09-10T21:51:38.505466+0000","last_active":"2024-09-10T21:51:39.965229+0000","last_peered":"2024-09-10T21:51:39.965229+0000","last_clean":"2024-09-10T21:51:39.965229+0000","last_became_active":"2024-09-10T21:51:38.505053+0000","last_became_peered":"2024-09-10T21:51:38.505053+0000","last_unstale":"2024-09-10T21:51:39.965229+0000","last_undegraded":"2024-09-10T21:51:39.965229+0000","last_fullsized":"2024-09-10T21:51:39.965229+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:36:29.403651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.497048+0000","last_change":"2024-09-10T21:51:38.494581+0000","last_active":"2024-09-10T21:51:39.497048+0000","last_peered":"2024-09-10T21:51:39.497048+0000","last_clean":"2024-09-10T21:51:39.497048+0000","last_became_active":"2024-09-10T21:51:38.493441+0000","last_became_peered":"2024-09-10T21:51:38.493441+0000","last_unstale":"2024-09-10T21:51:39.497048+0000","last_undegraded":"2024-09-10T21:51:39.497048+0000","last_fullsized":"2024-09-10T21:51:39.497048+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:57:29.395227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494942+0000","last_change":"2024-09-10T21:51:38.493643+0000","last_active":"2024-09-10T21:51:39.494942+0000","last_peered":"2024-09-10T21:51:39.494942+0000","last_clean":"2024-09-10T21:51:39.494942+0000","last_became_active":"2024-09-10T21:51:38.493374+0000","last_became_peered":"2024-09-10T21:51:38.493374+0000","last_unstale":"2024-09-10T21:51:39.494942+0000","last_undegraded":"2024-09-10T21:51:39.494942+0000","last_fullsized":"2024-09-10T21:51:39.494942+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:37:53.810720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.836711+0000","last_change":"2024-09-10T21:51:38.491652+0000","last_active":"2024-09-10T21:51:39.836711+0000","last_peered":"2024-09-10T21:51:39.836711+0000","last_clean":"2024-09-10T21:51:39.836711+0000","last_became_active":"2024-09-10T21:51:38.488308+0000","last_became_peered":"2024-09-10T21:51:38.488308+0000","last_unstale":"2024-09-10T21:51:39.836711+0000","last_undegraded":"2024-09-10T21:51:39.836711+0000","last_fullsized":"2024-09-10T21:51:39.836711+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:43.939923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965771+0000","last_change":"2024-09-10T21:51:38.496708+0000","last_active":"2024-09-10T21:51:39.965771+0000","last_peered":"2024-09-10T21:51:39.965771+0000","last_clean":"2024-09-10T21:51:39.965771+0000","last_became_active":"2024-09-10T21:51:38.496402+0000","last_became_peered":"2024-09-10T21:51:38.496402+0000","last_unstale":"2024-09-10T21:51:39.965771+0000","last_undegraded":"2024-09-10T21:51:39.965771+0000","last_fullsized":"2024-09-10T21:51:39.965771+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:40.305224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496429+0000","last_change":"2024-09-10T21:51:38.499760+0000","last_active":"2024-09-10T21:51:39.496429+0000","last_peered":"2024-09-10T21:51:39.496429+0000","last_clean":"2024-09-10T21:51:39.496429+0000","last_became_active":"2024-09-10T21:51:38.499426+0000","last_became_peered":"2024-09-10T21:51:38.499426+0000","last_unstale":"2024-09-10T21:51:39.496429+0000","last_undegraded":"2024-09-10T21:51:39.496429+0000","last_fullsized":"2024-09-10T21:51:39.496429+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:38.181706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495415+0000","last_change":"2024-09-10T21:51:38.500997+0000","last_active":"2024-09-10T21:51:39.495415+0000","last_peered":"2024-09-10T21:51:39.495415+0000","last_clean":"2024-09-10T21:51:39.495415+0000","last_became_active":"2024-09-10T21:51:38.500708+0000","last_became_peered":"2024-09-10T21:51:38.500708+0000","last_unstale":"2024-09-10T21:51:39.495415+0000","last_undegraded":"2024-09-10T21:51:39.495415+0000","last_fullsized":"2024-09-10T21:51:39.495415+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:02.532116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.493986+0000","last_change":"2024-09-10T21:51:38.486559+0000","last_active":"2024-09-10T21:51:39.493986+0000","last_peered":"2024-09-10T21:51:39.493986+0000","last_clean":"2024-09-10T21:51:39.493986+0000","last_became_active":"2024-09-10T21:51:38.486293+0000","last_became_peered":"2024-09-10T21:51:38.486293+0000","last_unstale":"2024-09-10T21:51:39.493986+0000","last_undegraded":"2024-09-10T21:51:39.493986+0000","last_fullsized":"2024-09-10T21:51:39.493986+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:52.862206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.836657+0000","last_change":"2024-09-10T21:51:38.501466+0000","last_active":"2024-09-10T21:51:39.836657+0000","last_peered":"2024-09-10T21:51:39.836657+0000","last_clean":"2024-09-10T21:51:39.836657+0000","last_became_active":"2024-09-10T21:51:38.501158+0000","last_became_peered":"2024-09-10T21:51:38.501158+0000","last_unstale":"2024-09-10T21:51:39.836657+0000","last_undegraded":"2024-09-10T21:51:39.836657+0000","last_fullsized":"2024-09-10T21:51:39.836657+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:43:23.281176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.836993+0000","last_change":"2024-09-10T21:51:38.488631+0000","last_active":"2024-09-10T21:51:39.836993+0000","last_peered":"2024-09-10T21:51:39.836993+0000","last_clean":"2024-09-10T21:51:39.836993+0000","last_became_active":"2024-09-10T21:51:38.487728+0000","last_became_peered":"2024-09-10T21:51:38.487728+0000","last_unstale":"2024-09-10T21:51:39.836993+0000","last_undegraded":"2024-09-10T21:51:39.836993+0000","last_fullsized":"2024-09-10T21:51:39.836993+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:36:08.725531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"113'4","reported_seq":19,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496069+0000","last_change":"2024-09-10T21:51:37.472999+0000","last_active":"2024-09-10T21:51:39.496069+0000","last_peered":"2024-09-10T21:51:39.496069+0000","last_clean":"2024-09-10T21:51:39.496069+0000","last_became_active":"2024-09-10T21:51:37.472471+0000","last_became_peered":"2024-09-10T21:51:37.472471+0000","last_unstale":"2024-09-10T21:51:39.496069+0000","last_undegraded":"2024-09-10T21:51:39.496069+0000","last_fullsized":"2024-09-10T21:51:39.496069+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:23:10.162552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.500153+0000","last_change":"2024-09-10T21:51:38.493621+0000","last_active":"2024-09-10T21:51:39.500153+0000","last_peered":"2024-09-10T21:51:39.500153+0000","last_clean":"2024-09-10T21:51:39.500153+0000","last_became_active":"2024-09-10T21:51:38.492451+0000","last_became_peered":"2024-09-10T21:51:38.492451+0000","last_unstale":"2024-09-10T21:51:39.500153+0000","last_undegraded":"2024-09-10T21:51:39.500153+0000","last_fullsized":"2024-09-10T21:51:39.500153+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:29.822792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966096+0000","last_change":"2024-09-10T21:51:37.481369+0000","last_active":"2024-09-10T21:51:39.966096+0000","last_peered":"2024-09-10T21:51:39.966096+0000","last_clean":"2024-09-10T21:51:39.966096+0000","last_became_active":"2024-09-10T21:51:37.481061+0000","last_became_peered":"2024-09-10T21:51:37.481061+0000","last_unstale":"2024-09-10T21:51:39.966096+0000","last_undegraded":"2024-09-10T21:51:39.966096+0000","last_fullsized":"2024-09-10T21:51:39.966096+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:07:48.767875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.836894+0000","last_change":"2024-09-10T21:51:38.501458+0000","last_active":"2024-09-10T21:51:39.836894+0000","last_peered":"2024-09-10T21:51:39.836894+0000","last_clean":"2024-09-10T21:51:39.836894+0000","last_became_active":"2024-09-10T21:51:38.500856+0000","last_became_peered":"2024-09-10T21:51:38.500856+0000","last_unstale":"2024-09-10T21:51:39.836894+0000","last_undegraded":"2024-09-10T21:51:39.836894+0000","last_fullsized":"2024-09-10T21:51:39.836894+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:32:01.582421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965735+0000","last_change":"2024-09-10T21:51:37.495401+0000","last_active":"2024-09-10T21:51:39.965735+0000","last_peered":"2024-09-10T21:51:39.965735+0000","last_clean":"2024-09-10T21:51:39.965735+0000","last_became_active":"2024-09-10T21:51:37.495129+0000","last_became_peered":"2024-09-10T21:51:37.495129+0000","last_unstale":"2024-09-10T21:51:39.965735+0000","last_undegraded":"2024-09-10T21:51:39.965735+0000","last_fullsized":"2024-09-10T21:51:39.965735+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:36:50.834277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966641+0000","last_change":"2024-09-10T21:51:38.507168+0000","last_active":"2024-09-10T21:51:39.966641+0000","last_peered":"2024-09-10T21:51:39.966641+0000","last_clean":"2024-09-10T21:51:39.966641+0000","last_became_active":"2024-09-10T21:51:38.506850+0000","last_became_peered":"2024-09-10T21:51:38.506850+0000","last_unstale":"2024-09-10T21:51:39.966641+0000","last_undegraded":"2024-09-10T21:51:39.966641+0000","last_fullsized":"2024-09-10T21:51:39.966641+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:40.305224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965580+0000","last_change":"2024-09-10T21:51:37.495072+0000","last_active":"2024-09-10T21:51:39.965580+0000","last_peered":"2024-09-10T21:51:39.965580+0000","last_clean":"2024-09-10T21:51:39.965580+0000","last_became_active":"2024-09-10T21:51:37.494778+0000","last_became_peered":"2024-09-10T21:51:37.494778+0000","last_unstale":"2024-09-10T21:51:39.965580+0000","last_undegraded":"2024-09-10T21:51:39.965580+0000","last_fullsized":"2024-09-10T21:51:39.965580+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:02.253882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494058+0000","last_change":"2024-09-10T21:51:38.492903+0000","last_active":"2024-09-10T21:51:39.494058+0000","last_peered":"2024-09-10T21:51:39.494058+0000","last_clean":"2024-09-10T21:51:39.494058+0000","last_became_active":"2024-09-10T21:51:38.491312+0000","last_became_peered":"2024-09-10T21:51:38.491312+0000","last_unstale":"2024-09-10T21:51:39.494058+0000","last_undegraded":"2024-09-10T21:51:39.494058+0000","last_fullsized":"2024-09-10T21:51:39.494058+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:45.871229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495488+0000","last_change":"2024-09-10T21:51:37.474801+0000","last_active":"2024-09-10T21:51:39.495488+0000","last_peered":"2024-09-10T21:51:39.495488+0000","last_clean":"2024-09-10T21:51:39.495488+0000","last_became_active":"2024-09-10T21:51:37.474455+0000","last_became_peered":"2024-09-10T21:51:37.474455+0000","last_unstale":"2024-09-10T21:51:39.495488+0000","last_undegraded":"2024-09-10T21:51:39.495488+0000","last_fullsized":"2024-09-10T21:51:39.495488+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:59.589239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496464+0000","last_change":"2024-09-10T21:51:38.496785+0000","last_active":"2024-09-10T21:51:39.496464+0000","last_peered":"2024-09-10T21:51:39.496464+0000","last_clean":"2024-09-10T21:51:39.496464+0000","last_became_active":"2024-09-10T21:51:38.496207+0000","last_became_peered":"2024-09-10T21:51:38.496207+0000","last_unstale":"2024-09-10T21:51:39.496464+0000","last_undegraded":"2024-09-10T21:51:39.496464+0000","last_fullsized":"2024-09-10T21:51:39.496464+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:16.990097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965968+0000","last_change":"2024-09-10T21:51:37.481177+0000","last_active":"2024-09-10T21:51:39.965968+0000","last_peered":"2024-09-10T21:51:39.965968+0000","last_clean":"2024-09-10T21:51:39.965968+0000","last_became_active":"2024-09-10T21:51:37.480878+0000","last_became_peered":"2024-09-10T21:51:37.480878+0000","last_unstale":"2024-09-10T21:51:39.965968+0000","last_undegraded":"2024-09-10T21:51:39.965968+0000","last_fullsized":"2024-09-10T21:51:39.965968+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:39.644340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494080+0000","last_change":"2024-09-10T21:51:38.495605+0000","last_active":"2024-09-10T21:51:39.494080+0000","last_peered":"2024-09-10T21:51:39.494080+0000","last_clean":"2024-09-10T21:51:39.494080+0000","last_became_active":"2024-09-10T21:51:38.495338+0000","last_became_peered":"2024-09-10T21:51:38.495338+0000","last_unstale":"2024-09-10T21:51:39.494080+0000","last_undegraded":"2024-09-10T21:51:39.494080+0000","last_fullsized":"2024-09-10T21:51:39.494080+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:16:04.658227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496179+0000","last_change":"2024-09-10T21:51:37.486466+0000","last_active":"2024-09-10T21:51:39.496179+0000","last_peered":"2024-09-10T21:51:39.496179+0000","last_clean":"2024-09-10T21:51:39.496179+0000","last_became_active":"2024-09-10T21:51:37.486160+0000","last_became_peered":"2024-09-10T21:51:37.486160+0000","last_unstale":"2024-09-10T21:51:39.496179+0000","last_undegraded":"2024-09-10T21:51:39.496179+0000","last_fullsized":"2024-09-10T21:51:39.496179+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:10.863565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495164+0000","last_change":"2024-09-10T21:51:38.493500+0000","last_active":"2024-09-10T21:51:39.495164+0000","last_peered":"2024-09-10T21:51:39.495164+0000","last_clean":"2024-09-10T21:51:39.495164+0000","last_became_active":"2024-09-10T21:51:38.491267+0000","last_became_peered":"2024-09-10T21:51:38.491267+0000","last_unstale":"2024-09-10T21:51:39.495164+0000","last_undegraded":"2024-09-10T21:51:39.495164+0000","last_fullsized":"2024-09-10T21:51:39.495164+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:50:04.908941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494247+0000","last_change":"2024-09-10T21:51:37.489582+0000","last_active":"2024-09-10T21:51:39.494247+0000","last_peered":"2024-09-10T21:51:39.494247+0000","last_clean":"2024-09-10T21:51:39.494247+0000","last_became_active":"2024-09-10T21:51:37.487889+0000","last_became_peered":"2024-09-10T21:51:37.487889+0000","last_unstale":"2024-09-10T21:51:39.494247+0000","last_undegraded":"2024-09-10T21:51:39.494247+0000","last_fullsized":"2024-09-10T21:51:39.494247+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:22.505773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496017+0000","last_change":"2024-09-10T21:51:38.489067+0000","last_active":"2024-09-10T21:51:39.496017+0000","last_peered":"2024-09-10T21:51:39.496017+0000","last_clean":"2024-09-10T21:51:39.496017+0000","last_became_active":"2024-09-10T21:51:38.487546+0000","last_became_peered":"2024-09-10T21:51:38.487546+0000","last_unstale":"2024-09-10T21:51:39.496017+0000","last_undegraded":"2024-09-10T21:51:39.496017+0000","last_fullsized":"2024-09-10T21:51:39.496017+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:56.724163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965825+0000","last_change":"2024-09-10T21:51:37.493056+0000","last_active":"2024-09-10T21:51:39.965825+0000","last_peered":"2024-09-10T21:51:39.965825+0000","last_clean":"2024-09-10T21:51:39.965825+0000","last_became_active":"2024-09-10T21:51:37.492756+0000","last_became_peered":"2024-09-10T21:51:37.492756+0000","last_unstale":"2024-09-10T21:51:39.965825+0000","last_undegraded":"2024-09-10T21:51:39.965825+0000","last_fullsized":"2024-09-10T21:51:39.965825+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:49.945201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.500224+0000","last_change":"2024-09-10T21:51:38.485917+0000","last_active":"2024-09-10T21:51:39.500224+0000","last_peered":"2024-09-10T21:51:39.500224+0000","last_clean":"2024-09-10T21:51:39.500224+0000","last_became_active":"2024-09-10T21:51:38.485409+0000","last_became_peered":"2024-09-10T21:51:38.485409+0000","last_unstale":"2024-09-10T21:51:39.500224+0000","last_undegraded":"2024-09-10T21:51:39.500224+0000","last_fullsized":"2024-09-10T21:51:39.500224+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:32:14.773637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966160+0000","last_change":"2024-09-10T21:51:37.487117+0000","last_active":"2024-09-10T21:51:39.966160+0000","last_peered":"2024-09-10T21:51:39.966160+0000","last_clean":"2024-09-10T21:51:39.966160+0000","last_became_active":"2024-09-10T21:51:37.484685+0000","last_became_peered":"2024-09-10T21:51:37.484685+0000","last_unstale":"2024-09-10T21:51:39.966160+0000","last_undegraded":"2024-09-10T21:51:39.966160+0000","last_fullsized":"2024-09-10T21:51:39.966160+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:43:13.540834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965991+0000","last_change":"2024-09-10T21:51:38.500232+0000","last_active":"2024-09-10T21:51:39.965991+0000","last_peered":"2024-09-10T21:51:39.965991+0000","last_clean":"2024-09-10T21:51:39.965991+0000","last_became_active":"2024-09-10T21:51:38.499959+0000","last_became_peered":"2024-09-10T21:51:38.499959+0000","last_unstale":"2024-09-10T21:51:39.965991+0000","last_undegraded":"2024-09-10T21:51:39.965991+0000","last_fullsized":"2024-09-10T21:51:39.965991+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:57:35.171265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"113'3","reported_seq":18,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494324+0000","last_change":"2024-09-10T21:51:37.483939+0000","last_active":"2024-09-10T21:51:39.494324+0000","last_peered":"2024-09-10T21:51:39.494324+0000","last_clean":"2024-09-10T21:51:39.494324+0000","last_became_active":"2024-09-10T21:51:37.483023+0000","last_became_peered":"2024-09-10T21:51:37.483023+0000","last_unstale":"2024-09-10T21:51:39.494324+0000","last_undegraded":"2024-09-10T21:51:39.494324+0000","last_fullsized":"2024-09-10T21:51:39.494324+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:50:28.514952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495638+0000","last_change":"2024-09-10T21:51:38.496905+0000","last_active":"2024-09-10T21:51:39.495638+0000","last_peered":"2024-09-10T21:51:39.495638+0000","last_clean":"2024-09-10T21:51:39.495638+0000","last_became_active":"2024-09-10T21:51:38.496075+0000","last_became_peered":"2024-09-10T21:51:38.496075+0000","last_unstale":"2024-09-10T21:51:39.495638+0000","last_undegraded":"2024-09-10T21:51:39.495638+0000","last_fullsized":"2024-09-10T21:51:39.495638+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:14:35.518282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"113'6","reported_seq":21,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495060+0000","last_change":"2024-09-10T21:51:37.490837+0000","last_active":"2024-09-10T21:51:39.495060+0000","last_peered":"2024-09-10T21:51:39.495060+0000","last_clean":"2024-09-10T21:51:39.495060+0000","last_became_active":"2024-09-10T21:51:37.489661+0000","last_became_peered":"2024-09-10T21:51:37.489661+0000","last_unstale":"2024-09-10T21:51:39.495060+0000","last_undegraded":"2024-09-10T21:51:39.495060+0000","last_fullsized":"2024-09-10T21:51:39.495060+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:53.621980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966089+0000","last_change":"2024-09-10T21:51:38.499244+0000","last_active":"2024-09-10T21:51:39.966089+0000","last_peered":"2024-09-10T21:51:39.966089+0000","last_clean":"2024-09-10T21:51:39.966089+0000","last_became_active":"2024-09-10T21:51:38.498965+0000","last_became_peered":"2024-09-10T21:51:38.498965+0000","last_unstale":"2024-09-10T21:51:39.966089+0000","last_undegraded":"2024-09-10T21:51:39.966089+0000","last_fullsized":"2024-09-10T21:51:39.966089+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:10:12.629720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.836736+0000","last_change":"2024-09-10T21:51:37.480818+0000","last_active":"2024-09-10T21:51:39.836736+0000","last_peered":"2024-09-10T21:51:39.836736+0000","last_clean":"2024-09-10T21:51:39.836736+0000","last_became_active":"2024-09-10T21:51:37.480562+0000","last_became_peered":"2024-09-10T21:51:37.480562+0000","last_unstale":"2024-09-10T21:51:39.836736+0000","last_undegraded":"2024-09-10T21:51:39.836736+0000","last_fullsized":"2024-09-10T21:51:39.836736+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:36.932605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965893+0000","last_change":"2024-09-10T21:51:38.504740+0000","last_active":"2024-09-10T21:51:39.965893+0000","last_peered":"2024-09-10T21:51:39.965893+0000","last_clean":"2024-09-10T21:51:39.965893+0000","last_became_active":"2024-09-10T21:51:38.504214+0000","last_became_peered":"2024-09-10T21:51:38.504214+0000","last_unstale":"2024-09-10T21:51:39.965893+0000","last_undegraded":"2024-09-10T21:51:39.965893+0000","last_fullsized":"2024-09-10T21:51:39.965893+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:47.805058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.499813+0000","last_change":"2024-09-10T21:51:37.491755+0000","last_active":"2024-09-10T21:51:39.499813+0000","last_peered":"2024-09-10T21:51:39.499813+0000","last_clean":"2024-09-10T21:51:39.499813+0000","last_became_active":"2024-09-10T21:51:37.491472+0000","last_became_peered":"2024-09-10T21:51:37.491472+0000","last_unstale":"2024-09-10T21:51:39.499813+0000","last_undegraded":"2024-09-10T21:51:39.499813+0000","last_fullsized":"2024-09-10T21:51:39.499813+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:04:15.025130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965395+0000","last_change":"2024-09-10T21:51:38.500521+0000","last_active":"2024-09-10T21:51:39.965395+0000","last_peered":"2024-09-10T21:51:39.965395+0000","last_clean":"2024-09-10T21:51:39.965395+0000","last_became_active":"2024-09-10T21:51:38.500234+0000","last_became_peered":"2024-09-10T21:51:38.500234+0000","last_unstale":"2024-09-10T21:51:39.965395+0000","last_undegraded":"2024-09-10T21:51:39.965395+0000","last_fullsized":"2024-09-10T21:51:39.965395+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:28.278479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"113'8","reported_seq":23,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965294+0000","last_change":"2024-09-10T21:51:37.492464+0000","last_active":"2024-09-10T21:51:39.965294+0000","last_peered":"2024-09-10T21:51:39.965294+0000","last_clean":"2024-09-10T21:51:39.965294+0000","last_became_active":"2024-09-10T21:51:37.492162+0000","last_became_peered":"2024-09-10T21:51:37.492162+0000","last_unstale":"2024-09-10T21:51:39.965294+0000","last_undegraded":"2024-09-10T21:51:39.965294+0000","last_fullsized":"2024-09-10T21:51:39.965294+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:11:40.052415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965974+0000","last_change":"2024-09-10T21:51:38.487213+0000","last_active":"2024-09-10T21:51:39.965974+0000","last_peered":"2024-09-10T21:51:39.965974+0000","last_clean":"2024-09-10T21:51:39.965974+0000","last_became_active":"2024-09-10T21:51:38.486561+0000","last_became_peered":"2024-09-10T21:51:38.486561+0000","last_unstale":"2024-09-10T21:51:39.965974+0000","last_undegraded":"2024-09-10T21:51:39.965974+0000","last_fullsized":"2024-09-10T21:51:39.965974+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:19.628610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494803+0000","last_change":"2024-09-10T21:51:37.468073+0000","last_active":"2024-09-10T21:51:39.494803+0000","last_peered":"2024-09-10T21:51:39.494803+0000","last_clean":"2024-09-10T21:51:39.494803+0000","last_became_active":"2024-09-10T21:51:37.467716+0000","last_became_peered":"2024-09-10T21:51:37.467716+0000","last_unstale":"2024-09-10T21:51:39.494803+0000","last_undegraded":"2024-09-10T21:51:39.494803+0000","last_fullsized":"2024-09-10T21:51:39.494803+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:33:27.988074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.837019+0000","last_change":"2024-09-10T21:51:38.492359+0000","last_active":"2024-09-10T21:51:39.837019+0000","last_peered":"2024-09-10T21:51:39.837019+0000","last_clean":"2024-09-10T21:51:39.837019+0000","last_became_active":"2024-09-10T21:51:38.490060+0000","last_became_peered":"2024-09-10T21:51:38.490060+0000","last_unstale":"2024-09-10T21:51:39.837019+0000","last_undegraded":"2024-09-10T21:51:39.837019+0000","last_fullsized":"2024-09-10T21:51:39.837019+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:18:00.595722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965984+0000","last_change":"2024-09-10T21:51:37.485673+0000","last_active":"2024-09-10T21:51:39.965984+0000","last_peered":"2024-09-10T21:51:39.965984+0000","last_clean":"2024-09-10T21:51:39.965984+0000","last_became_active":"2024-09-10T21:51:37.485132+0000","last_became_peered":"2024-09-10T21:51:37.485132+0000","last_unstale":"2024-09-10T21:51:39.965984+0000","last_undegraded":"2024-09-10T21:51:39.965984+0000","last_fullsized":"2024-09-10T21:51:39.965984+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:51:02.339465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966440+0000","last_change":"2024-09-10T21:51:38.505872+0000","last_active":"2024-09-10T21:51:39.966440+0000","last_peered":"2024-09-10T21:51:39.966440+0000","last_clean":"2024-09-10T21:51:39.966440+0000","last_became_active":"2024-09-10T21:51:38.505488+0000","last_became_peered":"2024-09-10T21:51:38.505488+0000","last_unstale":"2024-09-10T21:51:39.966440+0000","last_undegraded":"2024-09-10T21:51:39.966440+0000","last_fullsized":"2024-09-10T21:51:39.966440+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:29.689596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"113'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.500122+0000","last_change":"2024-09-10T21:51:37.485228+0000","last_active":"2024-09-10T21:51:39.500122+0000","last_peered":"2024-09-10T21:51:39.500122+0000","last_clean":"2024-09-10T21:51:39.500122+0000","last_became_active":"2024-09-10T21:51:37.484980+0000","last_became_peered":"2024-09-10T21:51:37.484980+0000","last_unstale":"2024-09-10T21:51:39.500122+0000","last_undegraded":"2024-09-10T21:51:39.500122+0000","last_fullsized":"2024-09-10T21:51:39.500122+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:32.265998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966710+0000","last_change":"2024-09-10T21:51:38.504879+0000","last_active":"2024-09-10T21:51:39.966710+0000","last_peered":"2024-09-10T21:51:39.966710+0000","last_clean":"2024-09-10T21:51:39.966710+0000","last_became_active":"2024-09-10T21:51:38.504560+0000","last_became_peered":"2024-09-10T21:51:38.504560+0000","last_unstale":"2024-09-10T21:51:39.966710+0000","last_undegraded":"2024-09-10T21:51:39.966710+0000","last_fullsized":"2024-09-10T21:51:39.966710+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:46:47.805058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495226+0000","last_change":"2024-09-10T21:51:37.469638+0000","last_active":"2024-09-10T21:51:39.495226+0000","last_peered":"2024-09-10T21:51:39.495226+0000","last_clean":"2024-09-10T21:51:39.495226+0000","last_became_active":"2024-09-10T21:51:37.467556+0000","last_became_peered":"2024-09-10T21:51:37.467556+0000","last_unstale":"2024-09-10T21:51:39.495226+0000","last_undegraded":"2024-09-10T21:51:39.495226+0000","last_fullsized":"2024-09-10T21:51:39.495226+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:03:13.973508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966529+0000","last_change":"2024-09-10T21:51:38.504149+0000","last_active":"2024-09-10T21:51:39.966529+0000","last_peered":"2024-09-10T21:51:39.966529+0000","last_clean":"2024-09-10T21:51:39.966529+0000","last_became_active":"2024-09-10T21:51:38.503873+0000","last_became_peered":"2024-09-10T21:51:38.503873+0000","last_unstale":"2024-09-10T21:51:39.966529+0000","last_undegraded":"2024-09-10T21:51:39.966529+0000","last_fullsized":"2024-09-10T21:51:39.966529+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:38.756786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.836754+0000","last_change":"2024-09-10T21:51:37.488302+0000","last_active":"2024-09-10T21:51:39.836754+0000","last_peered":"2024-09-10T21:51:39.836754+0000","last_clean":"2024-09-10T21:51:39.836754+0000","last_became_active":"2024-09-10T21:51:37.488054+0000","last_became_peered":"2024-09-10T21:51:37.488054+0000","last_unstale":"2024-09-10T21:51:39.836754+0000","last_undegraded":"2024-09-10T21:51:39.836754+0000","last_fullsized":"2024-09-10T21:51:39.836754+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:51:51.988103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494166+0000","last_change":"2024-09-10T21:51:38.500053+0000","last_active":"2024-09-10T21:51:39.494166+0000","last_peered":"2024-09-10T21:51:39.494166+0000","last_clean":"2024-09-10T21:51:39.494166+0000","last_became_active":"2024-09-10T21:51:38.499702+0000","last_became_peered":"2024-09-10T21:51:38.499702+0000","last_unstale":"2024-09-10T21:51:39.494166+0000","last_undegraded":"2024-09-10T21:51:39.494166+0000","last_fullsized":"2024-09-10T21:51:39.494166+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:23.077848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965745+0000","last_change":"2024-09-10T21:51:37.495655+0000","last_active":"2024-09-10T21:51:39.965745+0000","last_peered":"2024-09-10T21:51:39.965745+0000","last_clean":"2024-09-10T21:51:39.965745+0000","last_became_active":"2024-09-10T21:51:37.495383+0000","last_became_peered":"2024-09-10T21:51:37.495383+0000","last_unstale":"2024-09-10T21:51:39.965745+0000","last_undegraded":"2024-09-10T21:51:39.965745+0000","last_fullsized":"2024-09-10T21:51:39.965745+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:43:57.108755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966363+0000","last_change":"2024-09-10T21:51:38.498184+0000","last_active":"2024-09-10T21:51:39.966363+0000","last_peered":"2024-09-10T21:51:39.966363+0000","last_clean":"2024-09-10T21:51:39.966363+0000","last_became_active":"2024-09-10T21:51:38.497840+0000","last_became_peered":"2024-09-10T21:51:38.497840+0000","last_unstale":"2024-09-10T21:51:39.966363+0000","last_undegraded":"2024-09-10T21:51:39.966363+0000","last_fullsized":"2024-09-10T21:51:39.966363+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:50.132177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.836599+0000","last_change":"2024-09-10T21:51:37.493629+0000","last_active":"2024-09-10T21:51:39.836599+0000","last_peered":"2024-09-10T21:51:39.836599+0000","last_clean":"2024-09-10T21:51:39.836599+0000","last_became_active":"2024-09-10T21:51:37.493347+0000","last_became_peered":"2024-09-10T21:51:37.493347+0000","last_unstale":"2024-09-10T21:51:39.836599+0000","last_undegraded":"2024-09-10T21:51:39.836599+0000","last_fullsized":"2024-09-10T21:51:39.836599+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:06.890765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965539+0000","last_change":"2024-09-10T21:51:38.497499+0000","last_active":"2024-09-10T21:51:39.965539+0000","last_peered":"2024-09-10T21:51:39.965539+0000","last_clean":"2024-09-10T21:51:39.965539+0000","last_became_active":"2024-09-10T21:51:38.494172+0000","last_became_peered":"2024-09-10T21:51:38.494172+0000","last_unstale":"2024-09-10T21:51:39.965539+0000","last_undegraded":"2024-09-10T21:51:39.965539+0000","last_fullsized":"2024-09-10T21:51:39.965539+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:23:38.756786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494370+0000","last_change":"2024-09-10T21:51:37.490268+0000","last_active":"2024-09-10T21:51:39.494370+0000","last_peered":"2024-09-10T21:51:39.494370+0000","last_clean":"2024-09-10T21:51:39.494370+0000","last_became_active":"2024-09-10T21:51:37.488772+0000","last_became_peered":"2024-09-10T21:51:37.488772+0000","last_unstale":"2024-09-10T21:51:39.494370+0000","last_undegraded":"2024-09-10T21:51:39.494370+0000","last_fullsized":"2024-09-10T21:51:39.494370+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:47.135421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.499991+0000","last_change":"2024-09-10T21:51:38.485773+0000","last_active":"2024-09-10T21:51:39.499991+0000","last_peered":"2024-09-10T21:51:39.499991+0000","last_clean":"2024-09-10T21:51:39.499991+0000","last_became_active":"2024-09-10T21:51:38.485453+0000","last_became_peered":"2024-09-10T21:51:38.485453+0000","last_unstale":"2024-09-10T21:51:39.499991+0000","last_undegraded":"2024-09-10T21:51:39.499991+0000","last_fullsized":"2024-09-10T21:51:39.499991+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:25:18.728391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"113'2","reported_seq":17,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966234+0000","last_change":"2024-09-10T21:51:37.493424+0000","last_active":"2024-09-10T21:51:39.966234+0000","last_peered":"2024-09-10T21:51:39.966234+0000","last_clean":"2024-09-10T21:51:39.966234+0000","last_became_active":"2024-09-10T21:51:37.492964+0000","last_became_peered":"2024-09-10T21:51:37.492964+0000","last_unstale":"2024-09-10T21:51:39.966234+0000","last_undegraded":"2024-09-10T21:51:39.966234+0000","last_fullsized":"2024-09-10T21:51:39.966234+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:55:56.310705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494784+0000","last_change":"2024-09-10T21:51:38.486288+0000","last_active":"2024-09-10T21:51:39.494784+0000","last_peered":"2024-09-10T21:51:39.494784+0000","last_clean":"2024-09-10T21:51:39.494784+0000","last_became_active":"2024-09-10T21:51:38.486013+0000","last_became_peered":"2024-09-10T21:51:38.486013+0000","last_unstale":"2024-09-10T21:51:39.494784+0000","last_undegraded":"2024-09-10T21:51:39.494784+0000","last_fullsized":"2024-09-10T21:51:39.494784+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:38.040724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"114'1","reported_seq":16,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.500176+0000","last_change":"2024-09-10T21:51:37.481469+0000","last_active":"2024-09-10T21:51:39.500176+0000","last_peered":"2024-09-10T21:51:39.500176+0000","last_clean":"2024-09-10T21:51:39.500176+0000","last_became_active":"2024-09-10T21:51:37.481196+0000","last_became_peered":"2024-09-10T21:51:37.481196+0000","last_unstale":"2024-09-10T21:51:39.500176+0000","last_undegraded":"2024-09-10T21:51:39.500176+0000","last_fullsized":"2024-09-10T21:51:39.500176+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:08:35.185924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494858+0000","last_change":"2024-09-10T21:51:38.493151+0000","last_active":"2024-09-10T21:51:39.494858+0000","last_peered":"2024-09-10T21:51:39.494858+0000","last_clean":"2024-09-10T21:51:39.494858+0000","last_became_active":"2024-09-10T21:51:38.492219+0000","last_became_peered":"2024-09-10T21:51:38.492219+0000","last_unstale":"2024-09-10T21:51:39.494858+0000","last_undegraded":"2024-09-10T21:51:39.494858+0000","last_fullsized":"2024-09-10T21:51:39.494858+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:05.522186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494373+0000","last_change":"2024-09-10T21:51:37.490280+0000","last_active":"2024-09-10T21:51:39.494373+0000","last_peered":"2024-09-10T21:51:39.494373+0000","last_clean":"2024-09-10T21:51:39.494373+0000","last_became_active":"2024-09-10T21:51:37.489979+0000","last_became_peered":"2024-09-10T21:51:37.489979+0000","last_unstale":"2024-09-10T21:51:39.494373+0000","last_undegraded":"2024-09-10T21:51:39.494373+0000","last_fullsized":"2024-09-10T21:51:39.494373+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:24.259852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.500006+0000","last_change":"2024-09-10T21:51:38.484059+0000","last_active":"2024-09-10T21:51:39.500006+0000","last_peered":"2024-09-10T21:51:39.500006+0000","last_clean":"2024-09-10T21:51:39.500006+0000","last_became_active":"2024-09-10T21:51:38.483600+0000","last_became_peered":"2024-09-10T21:51:38.483600+0000","last_unstale":"2024-09-10T21:51:39.500006+0000","last_undegraded":"2024-09-10T21:51:39.500006+0000","last_fullsized":"2024-09-10T21:51:39.500006+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:14:55.459385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966131+0000","last_change":"2024-09-10T21:51:37.496547+0000","last_active":"2024-09-10T21:51:39.966131+0000","last_peered":"2024-09-10T21:51:39.966131+0000","last_clean":"2024-09-10T21:51:39.966131+0000","last_became_active":"2024-09-10T21:51:37.495965+0000","last_became_peered":"2024-09-10T21:51:37.495965+0000","last_unstale":"2024-09-10T21:51:39.966131+0000","last_undegraded":"2024-09-10T21:51:39.966131+0000","last_fullsized":"2024-09-10T21:51:39.966131+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:45.517218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496291+0000","last_change":"2024-09-10T21:51:38.496844+0000","last_active":"2024-09-10T21:51:39.496291+0000","last_peered":"2024-09-10T21:51:39.496291+0000","last_clean":"2024-09-10T21:51:39.496291+0000","last_became_active":"2024-09-10T21:51:38.495814+0000","last_became_peered":"2024-09-10T21:51:38.495814+0000","last_unstale":"2024-09-10T21:51:39.496291+0000","last_undegraded":"2024-09-10T21:51:39.496291+0000","last_fullsized":"2024-09-10T21:51:39.496291+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:13.139951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495600+0000","last_change":"2024-09-10T21:51:37.477655+0000","last_active":"2024-09-10T21:51:39.495600+0000","last_peered":"2024-09-10T21:51:39.495600+0000","last_clean":"2024-09-10T21:51:39.495600+0000","last_became_active":"2024-09-10T21:51:37.477156+0000","last_became_peered":"2024-09-10T21:51:37.477156+0000","last_unstale":"2024-09-10T21:51:39.495600+0000","last_undegraded":"2024-09-10T21:51:39.495600+0000","last_fullsized":"2024-09-10T21:51:39.495600+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:01:26.947972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.497006+0000","last_change":"2024-09-10T21:51:38.488657+0000","last_active":"2024-09-10T21:51:39.497006+0000","last_peered":"2024-09-10T21:51:39.497006+0000","last_clean":"2024-09-10T21:51:39.497006+0000","last_became_active":"2024-09-10T21:51:38.488017+0000","last_became_peered":"2024-09-10T21:51:38.488017+0000","last_unstale":"2024-09-10T21:51:39.497006+0000","last_undegraded":"2024-09-10T21:51:39.497006+0000","last_fullsized":"2024-09-10T21:51:39.497006+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:44:49.698033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.496329+0000","last_change":"2024-09-10T21:51:37.478410+0000","last_active":"2024-09-10T21:51:39.496329+0000","last_peered":"2024-09-10T21:51:39.496329+0000","last_clean":"2024-09-10T21:51:39.496329+0000","last_became_active":"2024-09-10T21:51:37.478141+0000","last_became_peered":"2024-09-10T21:51:37.478141+0000","last_unstale":"2024-09-10T21:51:39.496329+0000","last_undegraded":"2024-09-10T21:51:39.496329+0000","last_fullsized":"2024-09-10T21:51:39.496329+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:52.334651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966600+0000","last_change":"2024-09-10T21:51:38.501070+0000","last_active":"2024-09-10T21:51:39.966600+0000","last_peered":"2024-09-10T21:51:39.966600+0000","last_clean":"2024-09-10T21:51:39.966600+0000","last_became_active":"2024-09-10T21:51:38.500807+0000","last_became_peered":"2024-09-10T21:51:38.500807+0000","last_unstale":"2024-09-10T21:51:39.966600+0000","last_undegraded":"2024-09-10T21:51:39.966600+0000","last_fullsized":"2024-09-10T21:51:39.966600+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:03:41.427352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965553+0000","last_change":"2024-09-10T21:51:37.494039+0000","last_active":"2024-09-10T21:51:39.965553+0000","last_peered":"2024-09-10T21:51:39.965553+0000","last_clean":"2024-09-10T21:51:39.965553+0000","last_became_active":"2024-09-10T21:51:37.493739+0000","last_became_peered":"2024-09-10T21:51:37.493739+0000","last_unstale":"2024-09-10T21:51:39.965553+0000","last_undegraded":"2024-09-10T21:51:39.965553+0000","last_fullsized":"2024-09-10T21:51:39.965553+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:41:40.285518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494056+0000","last_change":"2024-09-10T21:51:38.496313+0000","last_active":"2024-09-10T21:51:39.494056+0000","last_peered":"2024-09-10T21:51:39.494056+0000","last_clean":"2024-09-10T21:51:39.494056+0000","last_became_active":"2024-09-10T21:51:38.495521+0000","last_became_peered":"2024-09-10T21:51:38.495521+0000","last_unstale":"2024-09-10T21:51:39.494056+0000","last_undegraded":"2024-09-10T21:51:39.494056+0000","last_fullsized":"2024-09-10T21:51:39.494056+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:51.059341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.494498+0000","last_change":"2024-09-10T21:51:37.489957+0000","last_active":"2024-09-10T21:51:39.494498+0000","last_peered":"2024-09-10T21:51:39.494498+0000","last_clean":"2024-09-10T21:51:39.494498+0000","last_became_active":"2024-09-10T21:51:37.488416+0000","last_became_peered":"2024-09-10T21:51:37.488416+0000","last_unstale":"2024-09-10T21:51:39.494498+0000","last_undegraded":"2024-09-10T21:51:39.494498+0000","last_fullsized":"2024-09-10T21:51:39.494498+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:37.929683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.965291+0000","last_change":"2024-09-10T21:51:38.495176+0000","last_active":"2024-09-10T21:51:39.965291+0000","last_peered":"2024-09-10T21:51:39.965291+0000","last_clean":"2024-09-10T21:51:39.965291+0000","last_became_active":"2024-09-10T21:51:38.494857+0000","last_became_peered":"2024-09-10T21:51:38.494857+0000","last_unstale":"2024-09-10T21:51:39.965291+0000","last_undegraded":"2024-09-10T21:51:39.965291+0000","last_fullsized":"2024-09-10T21:51:39.965291+0000","mapping_epoch":112,"log_start":"0'0","ondisk_log_start":"0'0","created":112,"last_epoch_clean":113,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:37.456491+0000","last_clean_scrub_stamp":"2024-09-10T21:51:37.456491+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:32.797143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.495810+0000","last_change":"2024-09-10T21:51:37.467889+0000","last_active":"2024-09-10T21:51:39.495810+0000","last_peered":"2024-09-10T21:51:39.495810+0000","last_clean":"2024-09-10T21:51:39.495810+0000","last_became_active":"2024-09-10T21:51:37.467603+0000","last_became_peered":"2024-09-10T21:51:37.467603+0000","last_unstale":"2024-09-10T21:51:39.495810+0000","last_undegraded":"2024-09-10T21:51:39.495810+0000","last_fullsized":"2024-09-10T21:51:39.495810+0000","mapping_epoch":111,"log_start":"0'0","ondisk_log_start":"0'0","created":111,"last_epoch_clean":112,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:51:36.450007+0000","last_clean_scrub_stamp":"2024-09-10T21:51:36.450007+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:52:38.793976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":281,"reported_epoch":114,"state":"active+clean","last_fresh":"2024-09-10T21:51:39.966593+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:51:39.966593+0000","last_peered":"2024-09-10T21:51:39.966593+0000","last_clean":"2024-09-10T21:51:39.966593+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:51:39.966593+0000","last_undegraded":"2024-09-10T21:51:39.966593+0000","last_fullsized":"2024-09-10T21:51:39.966593+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509561,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1413120,"data_stored":610155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509561,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32572,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339268,"statfs":{"total":96636764160,"available":96603410432,"internally_reserved":0,"allocated":1417216,"data_stored":610689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1409024,"data_stored":610085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509560,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32564,"kb_used_data":1376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339276,"statfs":{"total":96636764160,"available":96603418624,"internally_reserved":0,"allocated":1409024,"data_stored":610085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509562,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36732,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335108,"statfs":{"total":96636764160,"available":96599150592,"internally_reserved":0,"allocated":1417216,"data_stored":610209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509560,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37332,"kb_used_data":1976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334508,"statfs":{"total":96636764160,"available":96598536192,"internally_reserved":0,"allocated":2023424,"data_stored":1202177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542263,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33156,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338684,"statfs":{"total":96636764160,"available":96602812416,"internally_reserved":0,"allocated":2011136,"data_stored":1201069,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542264,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36744,"kb_used_data":1392,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335096,"statfs":{"total":96636764160,"available":96599138304,"internally_reserved":0,"allocated":1425408,"data_stored":611767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:51:47.890 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:51:47.890 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-10T21:51:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.091+0000 7f96fd600640 1 -- 172.21.15.7:0/1158544129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96f8155fb0 msgr2=0x7f96f81600d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.091+0000 7f96fd600640 1 --2- 172.21.15.7:0/1158544129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96f8155fb0 0x7f96f81600d0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f96f80b5c60 tx=0x7f96e802aa10 comp rx=0 tx=0).stop 2024-09-10T21:51:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.091+0000 7f96fd600640 1 -- 172.21.15.7:0/1158544129 shutdown_connections 2024-09-10T21:51:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.091+0000 7f96fd600640 1 --2- 172.21.15.7:0/1158544129 >> v2:172.21.15.133:3300/0 conn(0x7f96f8074280 0x7f96f806e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.091+0000 7f96fd600640 1 --2- 172.21.15.7:0/1158544129 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96f806fcf0 0x7f96f8073d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.091+0000 7f96fd600640 1 --2- 172.21.15.7:0/1158544129 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96f8155fb0 0x7f96f81600d0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.091+0000 7f96fd600640 1 -- 172.21.15.7:0/1158544129 >> 172.21.15.7:0/1158544129 conn(0x7f96f80b16d0 msgr2=0x7f96f80b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.091+0000 7f96fd600640 1 -- 172.21.15.7:0/1158544129 shutdown_connections 2024-09-10T21:51:48.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.091+0000 7f96fd600640 1 -- 172.21.15.7:0/1158544129 wait complete. 2024-09-10T21:51:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.093+0000 7f96fd600640 1 Processor -- start 2024-09-10T21:51:48.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.093+0000 7f96fd600640 1 -- start start 2024-09-10T21:51:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.094+0000 7f96fd600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96f806fcf0 0x7f96f80c33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.094+0000 7f96fd600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96f8074280 0x7f96f814d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.094+0000 7f96fd600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96f8155fb0 0x7f96f81f8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.094+0000 7f96fd600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f96f80cbc50 con 0x7f96f8155fb0 2024-09-10T21:51:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.094+0000 7f96fd600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f96f806eac0 con 0x7f96f806fcf0 2024-09-10T21:51:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.094+0000 7f96fd600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f96f806ec40 con 0x7f96f8074280 2024-09-10T21:51:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.094+0000 7f96f7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96f8155fb0 0x7f96f81f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.094+0000 7f96f6a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96f806fcf0 0x7f96f80c33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.094+0000 7f96f7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96f8155fb0 0x7f96f81f8e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43814/0 (socket says 172.21.15.7:43814) 2024-09-10T21:51:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.095+0000 7f96f7400640 1 -- 172.21.15.7:0/2918427049 learned_addr learned my addr 172.21.15.7:0/2918427049 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.095+0000 7f96f6000640 1 --2- 172.21.15.7:0/2918427049 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96f8074280 0x7f96f814d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:48.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.095+0000 7f96f7400640 1 -- 172.21.15.7:0/2918427049 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96f8074280 msgr2=0x7f96f814d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.095+0000 7f96f7400640 1 --2- 172.21.15.7:0/2918427049 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96f8074280 0x7f96f814d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.095+0000 7f96f7400640 1 -- 172.21.15.7:0/2918427049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96f806fcf0 msgr2=0x7f96f80c33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.095+0000 7f96f7400640 1 --2- 172.21.15.7:0/2918427049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96f806fcf0 0x7f96f80c33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.095+0000 7f96f7400640 1 -- 172.21.15.7:0/2918427049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f81525c0 con 0x7f96f8155fb0 2024-09-10T21:51:48.097 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.095+0000 7f96f7400640 1 --2- 172.21.15.7:0/2918427049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96f8155fb0 0x7f96f81f8e20 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009fd0 tx=0x7f96f000b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.095+0000 7f96eea00640 1 -- 172.21.15.7:0/2918427049 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0015020 con 0x7f96f8155fb0 2024-09-10T21:51:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.095+0000 7f96eea00640 1 -- 172.21.15.7:0/2918427049 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96f0011920 con 0x7f96f8155fb0 2024-09-10T21:51:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.095+0000 7f96eea00640 1 -- 172.21.15.7:0/2918427049 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0011d50 con 0x7f96f8155fb0 2024-09-10T21:51:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.095+0000 7f96fd600640 1 -- 172.21.15.7:0/2918427049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f8153000 con 0x7f96f8155fb0 2024-09-10T21:51:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.096+0000 7f96fd600640 1 -- 172.21.15.7:0/2918427049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f81535c0 con 0x7f96f8155fb0 2024-09-10T21:51:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.097+0000 7f96fd600640 1 -- 172.21.15.7:0/2918427049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96f806f140 con 0x7f96f8155fb0 2024-09-10T21:51:48.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.098+0000 7f96eea00640 1 -- 172.21.15.7:0/2918427049 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f96f00193b0 con 0x7f96f8155fb0 2024-09-10T21:51:48.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.099+0000 7f96eea00640 1 --2- 172.21.15.7:0/2918427049 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f96d8077f70 0x7f96d807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:48.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.099+0000 7f96eea00640 1 -- 172.21.15.7:0/2918427049 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 5709+0+0 (secure 0 0 0) 0x7f96f0099be0 con 0x7f96f8155fb0 2024-09-10T21:51:48.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.100+0000 7f96f6a00640 1 --2- 172.21.15.7:0/2918427049 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f96d8077f70 0x7f96d807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:48.101 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.100+0000 7f96f6a00640 1 --2- 172.21.15.7:0/2918427049 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f96d8077f70 0x7f96d807a430 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f96e8035000 tx=0x7f96e803b040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:48.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.104+0000 7f96eea00640 1 -- 172.21.15.7:0/2918427049 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f96f001f030 con 0x7f96f8155fb0 2024-09-10T21:51:48.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.370+0000 7f96fd600640 1 -- 172.21.15.7:0/2918427049 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f96f8153790 con 0x7f96f8155fb0 2024-09-10T21:51:48.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.372+0000 7f96eea00640 1 -- 172.21.15.7:0/2918427049 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v39) ==== 190+0+147 (secure 0 0 0) 0x7f96f00626d0 con 0x7f96f8155fb0 2024-09-10T21:51:48.373 INFO:teuthology.orchestra.run.smithi007.stdout:[client.0] 2024-09-10T21:51:48.373 INFO:teuthology.orchestra.run.smithi007.stdout: key = AQDvvOBm0aqcFhAAtid2bnzjcpaYAbU7g0JcOA== 2024-09-10T21:51:48.373 INFO:teuthology.orchestra.run.smithi007.stdout: caps mds = "allow" 2024-09-10T21:51:48.373 INFO:teuthology.orchestra.run.smithi007.stdout: caps mon = "allow r" 2024-09-10T21:51:48.373 INFO:teuthology.orchestra.run.smithi007.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-10T21:51:48.374 INFO:teuthology.orchestra.run.smithi007.stderr:updated caps for client.0 2024-09-10T21:51:48.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.375+0000 7f96fd600640 1 -- 172.21.15.7:0/2918427049 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f96d8077f70 msgr2=0x7f96d807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:48.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.375+0000 7f96fd600640 1 --2- 172.21.15.7:0/2918427049 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f96d8077f70 0x7f96d807a430 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f96e8035000 tx=0x7f96e803b040 comp rx=0 tx=0).stop 2024-09-10T21:51:48.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.375+0000 7f96fd600640 1 -- 172.21.15.7:0/2918427049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96f8155fb0 msgr2=0x7f96f81f8e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:48.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.375+0000 7f96fd600640 1 --2- 172.21.15.7:0/2918427049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96f8155fb0 0x7f96f81f8e20 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f96f0009fd0 tx=0x7f96f000b820 comp rx=0 tx=0).stop 2024-09-10T21:51:48.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.375+0000 7f96fd600640 1 -- 172.21.15.7:0/2918427049 shutdown_connections 2024-09-10T21:51:48.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.375+0000 7f96fd600640 1 --2- 172.21.15.7:0/2918427049 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f96d8077f70 0x7f96d807a430 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:48.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.375+0000 7f96fd600640 1 --2- 172.21.15.7:0/2918427049 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f96f8155fb0 0x7f96f81f8e20 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:48.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.375+0000 7f96fd600640 1 --2- 172.21.15.7:0/2918427049 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f96f8074280 0x7f96f814d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:48.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.375+0000 7f96fd600640 1 --2- 172.21.15.7:0/2918427049 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f96f806fcf0 0x7f96f80c33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:48.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.375+0000 7f96fd600640 1 -- 172.21.15.7:0/2918427049 >> 172.21.15.7:0/2918427049 conn(0x7f96f80b16d0 msgr2=0x7f96f80cfa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:48.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.375+0000 7f96fd600640 1 -- 172.21.15.7:0/2918427049 shutdown_connections 2024-09-10T21:51:48.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.375+0000 7f96fd600640 1 -- 172.21.15.7:0/2918427049 wait complete. 2024-09-10T21:51:48.392 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:51:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.554+0000 7f2655000640 1 -- 172.21.15.7:0/2067734881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f265015bc90 msgr2=0x7f26500ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.554+0000 7f2655000640 1 --2- 172.21.15.7:0/2067734881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f265015bc90 0x7f26500ccef0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x7f2640009830 tx=0x7f264002a480 comp rx=0 tx=0).stop 2024-09-10T21:51:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.554+0000 7f2655000640 1 -- 172.21.15.7:0/2067734881 shutdown_connections 2024-09-10T21:51:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.554+0000 7f2655000640 1 --2- 172.21.15.7:0/2067734881 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f265015bc90 0x7f26500ccef0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.554+0000 7f2655000640 1 -- 172.21.15.7:0/2067734881 >> 172.21.15.7:0/2067734881 conn(0x7f26500b16d0 msgr2=0x7f26500b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.554+0000 7f2655000640 1 -- 172.21.15.7:0/2067734881 shutdown_connections 2024-09-10T21:51:48.555 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.554+0000 7f2655000640 1 -- 172.21.15.7:0/2067734881 wait complete. 2024-09-10T21:51:48.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.555+0000 7f2655000640 1 Processor -- start 2024-09-10T21:51:48.556 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.555+0000 7f2655000640 1 -- start start 2024-09-10T21:51:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.556+0000 7f2655000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26500cd750 0x7f26501f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.556+0000 7f2655000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f265015bc90 0x7f26500c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.556+0000 7f2655000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f265015fda0 0x7f265015ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.556+0000 7f2655000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2650070080 con 0x7f265015bc90 2024-09-10T21:51:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.556+0000 7f2655000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f265006ff00 con 0x7f26500cd750 2024-09-10T21:51:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.556+0000 7f2655000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f2650070200 con 0x7f265015fda0 2024-09-10T21:51:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.557+0000 7f264d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f265015bc90 0x7f26500c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.557+0000 7f264d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f265015bc90 0x7f26500c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43830/0 (socket says 172.21.15.7:43830) 2024-09-10T21:51:48.558 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.557+0000 7f264d400640 1 -- 172.21.15.7:0/989352264 learned_addr learned my addr 172.21.15.7:0/989352264 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.557+0000 7f264de00640 1 --2- 172.21.15.7:0/989352264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26500cd750 0x7f26501f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.557+0000 7f264e800640 1 --2- 172.21.15.7:0/989352264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f265015fda0 0x7f265015ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.557+0000 7f264d400640 1 -- 172.21.15.7:0/989352264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f265015fda0 msgr2=0x7f265015ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.557+0000 7f264d400640 1 --2- 172.21.15.7:0/989352264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f265015fda0 0x7f265015ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.557+0000 7f264d400640 1 -- 172.21.15.7:0/989352264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26500cd750 msgr2=0x7f26501f5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.557+0000 7f264d400640 1 --2- 172.21.15.7:0/989352264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26500cd750 0x7f26501f5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.557+0000 7f264d400640 1 -- 172.21.15.7:0/989352264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26500ca1d0 con 0x7f265015bc90 2024-09-10T21:51:48.559 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.557+0000 7f264d400640 1 --2- 172.21.15.7:0/989352264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f265015bc90 0x7f26500c95b0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f263c00c450 tx=0x7f263c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.558+0000 7f2646000640 1 -- 172.21.15.7:0/989352264 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c016020 con 0x7f265015bc90 2024-09-10T21:51:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.558+0000 7f2646000640 1 -- 172.21.15.7:0/989352264 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f263c012920 con 0x7f265015bc90 2024-09-10T21:51:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.558+0000 7f2646000640 1 -- 172.21.15.7:0/989352264 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c012d70 con 0x7f265015bc90 2024-09-10T21:51:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.558+0000 7f2655000640 1 -- 172.21.15.7:0/989352264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f265015f3e0 con 0x7f265015bc90 2024-09-10T21:51:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.558+0000 7f2655000640 1 -- 172.21.15.7:0/989352264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26500ca750 con 0x7f265015bc90 2024-09-10T21:51:48.560 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.559+0000 7f2655000640 1 -- 172.21.15.7:0/989352264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26500706f0 con 0x7f265015bc90 2024-09-10T21:51:48.561 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.560+0000 7f2646000640 1 -- 172.21.15.7:0/989352264 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f263c01b400 con 0x7f265015bc90 2024-09-10T21:51:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.561+0000 7f2646000640 1 --2- 172.21.15.7:0/989352264 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2630077fa0 0x7f263007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.561+0000 7f2646000640 1 -- 172.21.15.7:0/989352264 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5767+0+0 (secure 0 0 0) 0x7f263c099e20 con 0x7f265015bc90 2024-09-10T21:51:48.562 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.561+0000 7f264de00640 1 --2- 172.21.15.7:0/989352264 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2630077fa0 0x7f263007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:48.563 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.562+0000 7f264de00640 1 --2- 172.21.15.7:0/989352264 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2630077fa0 0x7f263007a460 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2640034000 tx=0x7f2640004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:48.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.565+0000 7f2646000640 1 -- 172.21.15.7:0/989352264 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f263c020030 con 0x7f265015bc90 2024-09-10T21:51:48.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.798+0000 7f2655000640 1 -- 172.21.15.7:0/989352264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2650069810 con 0x7f265015bc90 2024-09-10T21:51:48.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.798+0000 7f2646000640 1 -- 172.21.15.7:0/989352264 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+5233 (secure 0 0 0) 0x7f263c062840 con 0x7f265015bc90 2024-09-10T21:51:48.799 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:48.800 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":102,"btime":"2024-09-10T21:51:41:499376+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3012572644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3012572644},{"type":"v1","addr":"172.21.15.7:6834","nonce":3012572644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":8094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/148356862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":148356862},{"type":"v1","addr":"172.21.15.7:6835","nonce":148356862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97},{"gid":236723,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/772257983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":772257983},{"type":"v1","addr":"172.21.15.133:6836","nonce":772257983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:51:37.913965+0000","modified":"2024-09-10T21:51:40.935708+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236726},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236726":{"gid":236726,"name":"b","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.133:6837/3764354952","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":3764354952},{"type":"v1","addr":"172.21.15.133:6837","nonce":3764354952}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":236726,"qdb_cluster":[236726]},"id":8}]} 2024-09-10T21:51:48.800 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 102 2024-09-10T21:51:48.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.801+0000 7f2655000640 1 -- 172.21.15.7:0/989352264 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2630077fa0 msgr2=0x7f263007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:48.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.801+0000 7f2655000640 1 --2- 172.21.15.7:0/989352264 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2630077fa0 0x7f263007a460 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2640034000 tx=0x7f2640004730 comp rx=0 tx=0).stop 2024-09-10T21:51:48.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.801+0000 7f2655000640 1 -- 172.21.15.7:0/989352264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f265015bc90 msgr2=0x7f26500c95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:48.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.801+0000 7f2655000640 1 --2- 172.21.15.7:0/989352264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f265015bc90 0x7f26500c95b0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f263c00c450 tx=0x7f263c00c910 comp rx=0 tx=0).stop 2024-09-10T21:51:48.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.801+0000 7f2655000640 1 -- 172.21.15.7:0/989352264 shutdown_connections 2024-09-10T21:51:48.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.801+0000 7f2655000640 1 --2- 172.21.15.7:0/989352264 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2630077fa0 0x7f263007a460 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:48.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.801+0000 7f2655000640 1 --2- 172.21.15.7:0/989352264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f265015fda0 0x7f265015ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:48.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.801+0000 7f2655000640 1 --2- 172.21.15.7:0/989352264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f265015bc90 0x7f26500c95b0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:48.802 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.801+0000 7f2655000640 1 --2- 172.21.15.7:0/989352264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f26500cd750 0x7f26501f5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:48.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.801+0000 7f2655000640 1 -- 172.21.15.7:0/989352264 >> 172.21.15.7:0/989352264 conn(0x7f26500b16d0 msgr2=0x7f2650072b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:48.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.801+0000 7f2655000640 1 -- 172.21.15.7:0/989352264 shutdown_connections 2024-09-10T21:51:48.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:48.801+0000 7f2655000640 1 -- 172.21.15.7:0/989352264 wait complete. 2024-09-10T21:51:49.220 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:51:37.913965+0000', 'modified': '2024-09-10T21:51:40.935708+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 236726}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_236726': {'gid': 236726, 'name': 'b', 'rank': 0, 'incarnation': 98, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.133:6837/3764354952', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6834', 'nonce': 3764354952}, {'type': 'v1', 'addr': '172.21.15.133:6837', 'nonce': 3764354952}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 236726, 'qdb_cluster': [236726]} 2024-09-10T21:51:49.220 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:51:49.220 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-10T21:51:49.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.384+0000 7f2306000640 1 -- 172.21.15.7:0/3658147061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f230015cbe0 msgr2=0x7f2300073cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:49.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.384+0000 7f2306000640 1 --2- 172.21.15.7:0/3658147061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f230015cbe0 0x7f2300073cd0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f22ec005fd0 tx=0x7f22ec02a410 comp rx=0 tx=0).stop 2024-09-10T21:51:49.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.384+0000 7f2306000640 1 -- 172.21.15.7:0/3658147061 shutdown_connections 2024-09-10T21:51:49.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.384+0000 7f2306000640 1 --2- 172.21.15.7:0/3658147061 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2300074210 0x7f230015f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.384+0000 7f2306000640 1 --2- 172.21.15.7:0/3658147061 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f230015cbe0 0x7f2300073cd0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.384+0000 7f2306000640 1 -- 172.21.15.7:0/3658147061 >> 172.21.15.7:0/3658147061 conn(0x7f23000b1660 msgr2=0x7f23000b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:49.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.384+0000 7f2306000640 1 -- 172.21.15.7:0/3658147061 shutdown_connections 2024-09-10T21:51:49.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.384+0000 7f2306000640 1 -- 172.21.15.7:0/3658147061 wait complete. 2024-09-10T21:51:49.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.385+0000 7f2306000640 1 Processor -- start 2024-09-10T21:51:49.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.386+0000 7f2306000640 1 -- start start 2024-09-10T21:51:49.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.386+0000 7f2306000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f230006ff50 0x7f2300154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.386+0000 7f2306000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2300074210 0x7f23001f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.386+0000 7f2306000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f230015cbe0 0x7f2300155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.386+0000 7f2306000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f230006f8a0 con 0x7f230006ff50 2024-09-10T21:51:49.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.386+0000 7f2306000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f230006f720 con 0x7f2300074210 2024-09-10T21:51:49.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.386+0000 7f2306000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f230015fc50 con 0x7f230015cbe0 2024-09-10T21:51:49.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.387+0000 7f22ff400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f230006ff50 0x7f2300154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.387+0000 7f22ff400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f230006ff50 0x7f2300154bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43852/0 (socket says 172.21.15.7:43852) 2024-09-10T21:51:49.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.387+0000 7f22ff400640 1 -- 172.21.15.7:0/3428361025 learned_addr learned my addr 172.21.15.7:0/3428361025 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:49.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.387+0000 7f22fea00640 1 --2- 172.21.15.7:0/3428361025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2300074210 0x7f23001f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.387+0000 7f22ffe00640 1 --2- 172.21.15.7:0/3428361025 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f230015cbe0 0x7f2300155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.387+0000 7f22ff400640 1 -- 172.21.15.7:0/3428361025 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f230015cbe0 msgr2=0x7f2300155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:49.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.387+0000 7f22ff400640 1 --2- 172.21.15.7:0/3428361025 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f230015cbe0 0x7f2300155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.387+0000 7f22ff400640 1 -- 172.21.15.7:0/3428361025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2300074210 msgr2=0x7f23001f34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:49.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.387+0000 7f22ff400640 1 --2- 172.21.15.7:0/3428361025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2300074210 0x7f23001f34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.387+0000 7f22ff400640 1 -- 172.21.15.7:0/3428361025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23000adbd0 con 0x7f230006ff50 2024-09-10T21:51:49.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.387+0000 7f22ffe00640 1 --2- 172.21.15.7:0/3428361025 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f230015cbe0 0x7f2300155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:51:49.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.387+0000 7f22ff400640 1 --2- 172.21.15.7:0/3428361025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f230006ff50 0x7f2300154bb0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f22f000c430 tx=0x7f22f000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:49.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.388+0000 7f22f7400640 1 -- 172.21.15.7:0/3428361025 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0011790 con 0x7f230006ff50 2024-09-10T21:51:49.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.388+0000 7f22f7400640 1 -- 172.21.15.7:0/3428361025 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22f0017070 con 0x7f230006ff50 2024-09-10T21:51:49.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.388+0000 7f22f7400640 1 -- 172.21.15.7:0/3428361025 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0011d00 con 0x7f230006ff50 2024-09-10T21:51:49.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.388+0000 7f2306000640 1 -- 172.21.15.7:0/3428361025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23000adeb0 con 0x7f230006ff50 2024-09-10T21:51:49.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.388+0000 7f2306000640 1 -- 172.21.15.7:0/3428361025 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23000ae470 con 0x7f230006ff50 2024-09-10T21:51:49.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.390+0000 7f22f7400640 1 -- 172.21.15.7:0/3428361025 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f22f001c460 con 0x7f230006ff50 2024-09-10T21:51:49.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.391+0000 7f22f7400640 1 --2- 172.21.15.7:0/3428361025 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f22e0077fa0 0x7f22e007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.391+0000 7f22f7400640 1 -- 172.21.15.7:0/3428361025 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 5767+0+0 (secure 0 0 0) 0x7f22f0099b80 con 0x7f230006ff50 2024-09-10T21:51:49.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.391+0000 7f2308e7b740 1 Processor -- start 2024-09-10T21:51:49.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.391+0000 7f2308e7b740 1 -- start start 2024-09-10T21:51:49.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.392+0000 7f22fea00640 1 --2- 172.21.15.7:0/3428361025 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f22e0077fa0 0x7f22e007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.392+0000 7f2308e7b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d938b0 0x565097d93c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.392+0000 7f22fea00640 1 --2- 172.21.15.7:0/3428361025 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f22e0077fa0 0x7f22e007a460 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f22ec033000 tx=0x7f22ec004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:49.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.392+0000 7f2308e7b740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d4ea20 0x565097d92390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.392+0000 7f2308e7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d4efa0 0x565097d928d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.392+0000 7f2308e7b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x565097cd1430 con 0x565097d938b0 2024-09-10T21:51:49.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.392+0000 7f2308e7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x565097cc15d0 con 0x565097d4efa0 2024-09-10T21:51:49.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.392+0000 7f2308e7b740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x565097cb0a00 con 0x565097d4ea20 2024-09-10T21:51:49.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.393+0000 7f22ff400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d938b0 0x565097d93c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.393+0000 7f22fea00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d4ea20 0x565097d92390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.393+0000 7f22ff400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d938b0 0x565097d93c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43862/0 (socket says 172.21.15.7:43862) 2024-09-10T21:51:49.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.393+0000 7f22ff400640 1 -- 172.21.15.7:0/370964751 learned_addr learned my addr 172.21.15.7:0/370964751 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:49.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.393+0000 7f22ffe00640 1 --2- 172.21.15.7:0/370964751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d4efa0 0x565097d928d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.393+0000 7f22ff400640 1 -- 172.21.15.7:0/370964751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d4ea20 msgr2=0x565097d92390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:49.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.393+0000 7f22ff400640 1 --2- 172.21.15.7:0/370964751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d4ea20 0x565097d92390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.393+0000 7f22ff400640 1 -- 172.21.15.7:0/370964751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d4efa0 msgr2=0x565097d928d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:49.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.393+0000 7f22ff400640 1 --2- 172.21.15.7:0/370964751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d4efa0 0x565097d928d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.393+0000 7f22ff400640 1 -- 172.21.15.7:0/370964751 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565097c79520 con 0x565097d938b0 2024-09-10T21:51:49.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.393+0000 7f22ff400640 1 --2- 172.21.15.7:0/370964751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d938b0 0x565097d93c90 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f22f005f1a0 tx=0x7f22f0061720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63260d71a1e2a541 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:49.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.394+0000 7f22eb400640 1 -- 172.21.15.7:0/370964751 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0062630 con 0x565097d938b0 2024-09-10T21:51:49.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.394+0000 7f22eb400640 1 -- 172.21.15.7:0/370964751 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22f0061f30 con 0x565097d938b0 2024-09-10T21:51:49.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.394+0000 7f22eb400640 1 -- 172.21.15.7:0/370964751 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f0067dc0 con 0x565097d938b0 2024-09-10T21:51:49.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.394+0000 7f2308e7b740 1 -- 172.21.15.7:0/370964751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d938b0 msgr2=0x565097d93c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:49.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.394+0000 7f2308e7b740 1 --2- 172.21.15.7:0/370964751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d938b0 0x565097d93c90 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f22f005f1a0 tx=0x7f22f0061720 comp rx=0 tx=0).stop 2024-09-10T21:51:49.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.394+0000 7f2308e7b740 1 -- 172.21.15.7:0/370964751 shutdown_connections 2024-09-10T21:51:49.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.394+0000 7f2308e7b740 1 --2- 172.21.15.7:0/370964751 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d4efa0 0x565097d928d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.394+0000 7f2308e7b740 1 --2- 172.21.15.7:0/370964751 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d4ea20 0x565097d92390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.394+0000 7f2308e7b740 1 --2- 172.21.15.7:0/370964751 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d938b0 0x565097d93c90 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.394+0000 7f2308e7b740 1 -- 172.21.15.7:0/370964751 >> 172.21.15.7:0/370964751 conn(0x565097d49ff0 msgr2=0x565097d4da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.394+0000 7f2308e7b740 1 -- 172.21.15.7:0/370964751 shutdown_connections 2024-09-10T21:51:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.394+0000 7f2308e7b740 1 -- 172.21.15.7:0/370964751 wait complete. 2024-09-10T21:51:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.395+0000 7f2308e7b740 1 -- start start 2024-09-10T21:51:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.395+0000 7f2308e7b740 1 Processor -- start 2024-09-10T21:51:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.396+0000 7f2308e7b740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d4efa0 0x565097d92b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.396+0000 7f2308e7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d938b0 0x565097d93070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.396+0000 7f2308e7b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 0x565097da5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.396+0000 7f2308e7b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x565097c57330 con 0x565097d65350 2024-09-10T21:51:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.396+0000 7f2308e7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x565097cb4810 con 0x565097d938b0 2024-09-10T21:51:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.396+0000 7f2308e7b740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x565097a21620 con 0x565097d4efa0 2024-09-10T21:51:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.396+0000 7f2308e7b740 10 client.0 _ll_register_callbacks cb 0x565097c5c420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:51:49.400 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.396+0000 7f22ffe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 0x565097da5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.397+0000 7f22ffe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 0x565097da5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43870/0 (socket says 172.21.15.7:43870) 2024-09-10T21:51:49.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.397+0000 7f22ffe00640 1 -- 172.21.15.7:0/4088222728 learned_addr learned my addr 172.21.15.7:0/4088222728 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:49.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.397+0000 7f22ff400640 1 --2- 172.21.15.7:0/4088222728 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d4efa0 0x565097d92b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.397+0000 7f22fea00640 1 --2- 172.21.15.7:0/4088222728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d938b0 0x565097d93070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.397+0000 7f22ffe00640 1 -- 172.21.15.7:0/4088222728 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d4efa0 msgr2=0x565097d92b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:49.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.397+0000 7f22ffe00640 1 --2- 172.21.15.7:0/4088222728 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d4efa0 0x565097d92b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.397+0000 7f22ffe00640 1 -- 172.21.15.7:0/4088222728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d938b0 msgr2=0x565097d93070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:49.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.397+0000 7f22ffe00640 1 --2- 172.21.15.7:0/4088222728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d938b0 0x565097d93070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.397+0000 7f22ffe00640 1 -- 172.21.15.7:0/4088222728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565097a12850 con 0x565097d65350 2024-09-10T21:51:49.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.397+0000 7f22ff400640 1 --2- 172.21.15.7:0/4088222728 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d4efa0 0x565097d92b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:51:49.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.397+0000 7f22ffe00640 1 --2- 172.21.15.7:0/4088222728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 0x565097da5dc0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f22f800e760 tx=0x7f22f800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45eb1d2b85e647b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:49.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.397+0000 7f22dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:51:49.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.397+0000 7f22dfe00640 1 -- 172.21.15.7:0/4088222728 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22f8021060 con 0x565097d65350 2024-09-10T21:51:49.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.397+0000 7f22dfe00640 1 -- 172.21.15.7:0/4088222728 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 5767+0+0 (secure 0 0 0) 0x7f22f801c480 con 0x565097d65350 2024-09-10T21:51:49.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.398+0000 7f2308e7b740 1 -- 172.21.15.7:0/4088222728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22f0061f30 con 0x565097d65350 2024-09-10T21:51:49.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.398+0000 7f22dfe00640 1 -- 172.21.15.7:0/4088222728 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f22f8027050 con 0x565097d65350 2024-09-10T21:51:49.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.398+0000 7f2308e7b740 10 client.8247 fetch_fsmap learned FSMap version 102 2024-09-10T21:51:49.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.398+0000 7f2308e7b740 1 -- 172.21.15.7:0/4088222728 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) -- 0x565097a896b0 con 0x565097d65350 2024-09-10T21:51:49.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.398+0000 7f22dfe00640 1 -- 172.21.15.7:0/4088222728 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22f0061f30 con 0x565097d65350 2024-09-10T21:51:49.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.398+0000 7f22dfe00640 1 -- 172.21.15.7:0/4088222728 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 102) ==== 3502+0+0 (secure 0 0 0) 0x7f22f802a020 con 0x565097d65350 2024-09-10T21:51:49.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.398+0000 7f2308e7b740 10 client.8247 fetch_fsmap finished waiting for FSMap version 102 2024-09-10T21:51:49.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.398+0000 7f2308e7b740 10 client.8247 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:51:49.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.398+0000 7f2308e7b740 20 client.8247 populate_metadata read hostname 'smithi007' 2024-09-10T21:51:49.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.398+0000 7f2308e7b740 1 --2- 172.21.15.7:0/4088222728 >> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] conn(0x565097d8af30 0x565097d8b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.398+0000 7f2308e7b740 4 client.8247 mds_command: new command op to 236726 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:51:49.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.398+0000 7f2308e7b740 1 -- 172.21.15.7:0/4088222728 --> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565097cc3520 con 0x565097d8af30 2024-09-10T21:51:49.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.399+0000 7f22ff400640 1 --2- 172.21.15.7:0/4088222728 >> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] conn(0x565097d8af30 0x565097d8b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.399+0000 7f22ff400640 1 --2- 172.21.15.7:0/4088222728 >> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] conn(0x565097d8af30 0x565097d8b310 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52227b57631a1e2 server_cookie=2d652d946592dc4a in_seq=0 out_seq=0 2024-09-10T21:51:49.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.399+0000 7f22dfe00640 10 client.8247 ms_handle_connect on v2:172.21.15.133:6834/3764354952 2024-09-10T21:51:49.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.400+0000 7f22dfe00640 1 -- 172.21.15.7:0/4088222728 <== mds.0 v2:172.21.15.133:6834/3764354952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x565097cc3520 con 0x565097d8af30 2024-09-10T21:51:49.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.400+0000 7f22dfe00640 10 client.8247 handle_command_reply: tid=0 2024-09-10T21:51:49.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.401+0000 7f2308e7b740 1 client.8247 shutdown 2024-09-10T21:51:49.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.401+0000 7f2308e7b740 1 -- 172.21.15.7:0/4088222728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 msgr2=0x565097da5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:49.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.401+0000 7f2308e7b740 1 --2- 172.21.15.7:0/4088222728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 0x565097da5dc0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f22f800e760 tx=0x7f22f800ec20 comp rx=0 tx=0).stop 2024-09-10T21:51:49.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.401+0000 7f2308e7b740 1 -- 172.21.15.7:0/4088222728 shutdown_connections 2024-09-10T21:51:49.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.401+0000 7f2308e7b740 1 --2- 172.21.15.7:0/4088222728 >> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] conn(0x565097d8af30 0x565097d8b310 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.401+0000 7f2308e7b740 1 --2- 172.21.15.7:0/4088222728 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d938b0 0x565097d93070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.401+0000 7f22dfe00640 0 client.8247 ms_handle_reset on v2:172.21.15.133:6834/3764354952 2024-09-10T21:51:49.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.401+0000 7f2308e7b740 1 --2- 172.21.15.7:0/4088222728 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 0x565097da5dc0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.401+0000 7f2308e7b740 1 --2- 172.21.15.7:0/4088222728 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d4efa0 0x565097d92b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.401+0000 7f2308e7b740 1 -- 172.21.15.7:0/4088222728 >> 172.21.15.7:0/4088222728 conn(0x565097d4ea20 msgr2=0x565097d48bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:49.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.401+0000 7f2308e7b740 1 -- 172.21.15.7:0/4088222728 shutdown_connections 2024-09-10T21:51:49.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.401+0000 7f2308e7b740 1 -- 172.21.15.7:0/4088222728 wait complete. 2024-09-10T21:51:49.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.401+0000 7f2308e7b740 20 client.8247 trim_cache size 0 max 16384 2024-09-10T21:51:49.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.420+0000 7f2308e7b740 1 Processor -- start 2024-09-10T21:51:49.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.420+0000 7f2308e7b740 1 -- start start 2024-09-10T21:51:49.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.421+0000 7f2308e7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d4efa0 0x565097d48080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.421+0000 7f2308e7b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 0x565097da4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.421+0000 7f2308e7b740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d938b0 0x565097e0a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.421+0000 7f2308e7b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f23000cef40 con 0x565097d65350 2024-09-10T21:51:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.421+0000 7f2308e7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f23001f6220 con 0x565097d4efa0 2024-09-10T21:51:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.421+0000 7f2308e7b740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f23000ce8f0 con 0x565097d938b0 2024-09-10T21:51:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.421+0000 7f22fea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 0x565097da4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.421+0000 7f22fea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 0x565097da4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43882/0 (socket says 172.21.15.7:43882) 2024-09-10T21:51:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.421+0000 7f22fea00640 1 -- 172.21.15.7:0/561390571 learned_addr learned my addr 172.21.15.7:0/561390571 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.422+0000 7f22ffe00640 1 --2- 172.21.15.7:0/561390571 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d938b0 0x565097e0a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.422+0000 7f22ff400640 1 --2- 172.21.15.7:0/561390571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d4efa0 0x565097d48080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.422+0000 7f22fea00640 1 -- 172.21.15.7:0/561390571 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d938b0 msgr2=0x565097e0a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:49.423 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.422+0000 7f22fea00640 1 --2- 172.21.15.7:0/561390571 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d938b0 0x565097e0a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.422+0000 7f22fea00640 1 -- 172.21.15.7:0/561390571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d4efa0 msgr2=0x565097d48080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.422+0000 7f22fea00640 1 --2- 172.21.15.7:0/561390571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d4efa0 0x565097d48080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.422+0000 7f22fea00640 1 -- 172.21.15.7:0/561390571 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565097d4ca80 con 0x565097d65350 2024-09-10T21:51:49.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.422+0000 7f22ff400640 1 --2- 172.21.15.7:0/561390571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d4efa0 0x565097d48080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:51:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.422+0000 7f22fea00640 1 --2- 172.21.15.7:0/561390571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 0x565097da4fa0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f22ec000c00 tx=0x7f22ec03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab368d18e7fb510a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.422+0000 7f22eaa00640 1 -- 172.21.15.7:0/561390571 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec04f020 con 0x565097d65350 2024-09-10T21:51:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.422+0000 7f22eaa00640 1 -- 172.21.15.7:0/561390571 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22ec043040 con 0x565097d65350 2024-09-10T21:51:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.422+0000 7f22eaa00640 1 -- 172.21.15.7:0/561390571 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec035040 con 0x565097d65350 2024-09-10T21:51:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.423+0000 7f2308e7b740 1 -- 172.21.15.7:0/561390571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 msgr2=0x565097da4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:49.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.423+0000 7f2308e7b740 1 --2- 172.21.15.7:0/561390571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 0x565097da4fa0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f22ec000c00 tx=0x7f22ec03c000 comp rx=0 tx=0).stop 2024-09-10T21:51:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.423+0000 7f2308e7b740 1 -- 172.21.15.7:0/561390571 shutdown_connections 2024-09-10T21:51:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.423+0000 7f2308e7b740 1 --2- 172.21.15.7:0/561390571 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d938b0 0x565097e0a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.423+0000 7f2308e7b740 1 --2- 172.21.15.7:0/561390571 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 0x565097da4fa0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.423+0000 7f2308e7b740 1 --2- 172.21.15.7:0/561390571 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d4efa0 0x565097d48080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.423+0000 7f2308e7b740 1 -- 172.21.15.7:0/561390571 >> 172.21.15.7:0/561390571 conn(0x565097d4ea20 msgr2=0x565097d48b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.423+0000 7f2308e7b740 1 -- 172.21.15.7:0/561390571 shutdown_connections 2024-09-10T21:51:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.423+0000 7f2308e7b740 1 -- 172.21.15.7:0/561390571 wait complete. 2024-09-10T21:51:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.424+0000 7f2308e7b740 1 -- start start 2024-09-10T21:51:49.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.424+0000 7f2308e7b740 1 Processor -- start 2024-09-10T21:51:49.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.425+0000 7f2308e7b740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d4efa0 0x565097d95770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.425+0000 7f2308e7b740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 0x565097d72dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.425+0000 7f2308e7b740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d938b0 0x565097e03bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.425+0000 7f2308e7b740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x565097d705c0 con 0x565097d65350 2024-09-10T21:51:49.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.425+0000 7f2308e7b740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x565097d8d180 con 0x565097d4efa0 2024-09-10T21:51:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.425+0000 7f2308e7b740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x565097d77a70 con 0x565097d938b0 2024-09-10T21:51:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.425+0000 7f2308e7b740 10 client.0 _ll_register_callbacks cb 0x7f22f8010cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:51:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.425+0000 7f22fea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 0x565097d72dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.425+0000 7f22fea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 0x565097d72dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43894/0 (socket says 172.21.15.7:43894) 2024-09-10T21:51:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.425+0000 7f22fea00640 1 -- 172.21.15.7:0/3793525641 learned_addr learned my addr 172.21.15.7:0/3793525641 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.425+0000 7f22fea00640 1 -- 172.21.15.7:0/3793525641 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d938b0 msgr2=0x565097e03bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.425+0000 7f22fea00640 1 --2- 172.21.15.7:0/3793525641 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d938b0 0x565097e03bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.425+0000 7f22fea00640 1 -- 172.21.15.7:0/3793525641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d4efa0 msgr2=0x565097d95770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.425+0000 7f22fea00640 1 --2- 172.21.15.7:0/3793525641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d4efa0 0x565097d95770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.425+0000 7f22fea00640 1 -- 172.21.15.7:0/3793525641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565097d95cb0 con 0x565097d65350 2024-09-10T21:51:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.426+0000 7f22fea00640 1 --2- 172.21.15.7:0/3793525641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 0x565097d72dc0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f22ec006010 tx=0x7f22ec04bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16f40ab904081dac server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:49.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.426+0000 7f22dfe00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:51:49.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.426+0000 7f22dfe00640 1 -- 172.21.15.7:0/3793525641 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22ec04f040 con 0x565097d65350 2024-09-10T21:51:49.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.426+0000 7f22dfe00640 1 -- 172.21.15.7:0/3793525641 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(115..115 src has 1..115) ==== 5767+0+0 (secure 0 0 0) 0x7f22ec03c470 con 0x565097d65350 2024-09-10T21:51:49.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.426+0000 7f2308e7b740 1 -- 172.21.15.7:0/3793525641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22ec043040 con 0x565097d65350 2024-09-10T21:51:49.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.426+0000 7f22dfe00640 1 -- 172.21.15.7:0/3793525641 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7f22ec063050 con 0x565097d65350 2024-09-10T21:51:49.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.426+0000 7f2308e7b740 10 client.8253 fetch_fsmap learned FSMap version 102 2024-09-10T21:51:49.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.426+0000 7f2308e7b740 1 -- 172.21.15.7:0/3793525641 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) -- 0x565097d64f10 con 0x565097d65350 2024-09-10T21:51:49.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.426+0000 7f22dfe00640 1 -- 172.21.15.7:0/3793525641 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22ec043040 con 0x565097d65350 2024-09-10T21:51:49.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.427+0000 7f22dfe00640 1 -- 172.21.15.7:0/3793525641 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 102) ==== 3502+0+0 (secure 0 0 0) 0x7f22ec066020 con 0x565097d65350 2024-09-10T21:51:49.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.427+0000 7f2308e7b740 10 client.8253 fetch_fsmap finished waiting for FSMap version 102 2024-09-10T21:51:49.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.427+0000 7f2308e7b740 10 client.8253 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-10T21:51:49.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.427+0000 7f2308e7b740 20 client.8253 populate_metadata read hostname 'smithi007' 2024-09-10T21:51:49.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.427+0000 7f2308e7b740 1 --2- 172.21.15.7:0/3793525641 >> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] conn(0x565097e094a0 0x565097e07860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.427+0000 7f2308e7b740 4 client.8253 mds_command: new command op to 236726 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:51:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.427+0000 7f2308e7b740 1 -- 172.21.15.7:0/3793525641 --> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] -- command(tid 0: {"prefix": "status"}) -- 0x565097cbefd0 con 0x565097e094a0 2024-09-10T21:51:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.427+0000 7f22ff400640 1 --2- 172.21.15.7:0/3793525641 >> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] conn(0x565097e094a0 0x565097e07860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.428+0000 7f22ff400640 1 --2- 172.21.15.7:0/3793525641 >> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] conn(0x565097e094a0 0x565097e07860 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b39ceccc388bde0b server_cookie=bd8f9bd84845a5b7 in_seq=0 out_seq=0 2024-09-10T21:51:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.428+0000 7f22dfe00640 10 client.8253 ms_handle_connect on v2:172.21.15.133:6834/3764354952 2024-09-10T21:51:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.428+0000 7f22dfe00640 1 -- 172.21.15.7:0/3793525641 <== mds.0 v2:172.21.15.133:6834/3764354952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+336 (crc 0 0 0) 0x565097cbefd0 con 0x565097e094a0 2024-09-10T21:51:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.428+0000 7f22dfe00640 10 client.8253 handle_command_reply: tid=0 2024-09-10T21:51:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.428+0000 7f2308e7b740 1 client.8253 shutdown 2024-09-10T21:51:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.428+0000 7f2308e7b740 1 -- 172.21.15.7:0/3793525641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 msgr2=0x565097d72dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.428+0000 7f2308e7b740 1 --2- 172.21.15.7:0/3793525641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 0x565097d72dc0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f22ec006010 tx=0x7f22ec04bbc0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.428+0000 7f2308e7b740 1 -- 172.21.15.7:0/3793525641 shutdown_connections 2024-09-10T21:51:49.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.428+0000 7f2308e7b740 1 --2- 172.21.15.7:0/3793525641 >> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] conn(0x565097e094a0 0x565097e07860 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.428+0000 7f2308e7b740 1 --2- 172.21.15.7:0/3793525641 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x565097d938b0 0x565097e03bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.428+0000 7f2308e7b740 1 --2- 172.21.15.7:0/3793525641 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x565097d65350 0x565097d72dc0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.428+0000 7f2308e7b740 1 --2- 172.21.15.7:0/3793525641 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x565097d4efa0 0x565097d95770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.428+0000 7f2308e7b740 1 -- 172.21.15.7:0/3793525641 >> 172.21.15.7:0/3793525641 conn(0x565097d4ea20 msgr2=0x565097da4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.428+0000 7f22dfe00640 0 client.8253 ms_handle_reset on v2:172.21.15.133:6834/3764354952 2024-09-10T21:51:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.428+0000 7f2308e7b740 1 -- 172.21.15.7:0/3793525641 shutdown_connections 2024-09-10T21:51:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.428+0000 7f2308e7b740 1 -- 172.21.15.7:0/3793525641 wait complete. 2024-09-10T21:51:49.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.429+0000 7f2308e7b740 20 client.8253 trim_cache size 0 max 16384 2024-09-10T21:51:49.433 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:51:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2029451a-3c26-48c7-b874-0597197278ac", 2024-09-10T21:51:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-10T21:51:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 236726, 2024-09-10T21:51:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T21:51:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T21:51:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T21:51:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 10.941632160999999, 2024-09-10T21:51:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 102, 2024-09-10T21:51:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 115, 2024-09-10T21:51:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 114, 2024-09-10T21:51:49.434 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 16.224804819999999 2024-09-10T21:51:49.434 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:51:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.429+0000 7f2306000640 1 -- 172.21.15.7:0/3428361025 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f22e0077fa0 msgr2=0x7f22e007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.429+0000 7f2306000640 1 --2- 172.21.15.7:0/3428361025 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f22e0077fa0 0x7f22e007a460 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f22ec033000 tx=0x7f22ec004730 comp rx=0 tx=0).stop 2024-09-10T21:51:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.429+0000 7f2306000640 1 -- 172.21.15.7:0/3428361025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f230006ff50 msgr2=0x7f2300154bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.429+0000 7f2306000640 1 --2- 172.21.15.7:0/3428361025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f230006ff50 0x7f2300154bb0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f22f000c430 tx=0x7f22f000c8f0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.429+0000 7f2306000640 1 -- 172.21.15.7:0/3428361025 shutdown_connections 2024-09-10T21:51:49.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.429+0000 7f2306000640 1 --2- 172.21.15.7:0/3428361025 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f22e0077fa0 0x7f22e007a460 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.429+0000 7f2306000640 1 --2- 172.21.15.7:0/3428361025 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2300074210 0x7f23001f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.429+0000 7f2306000640 1 --2- 172.21.15.7:0/3428361025 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f230015cbe0 0x7f2300155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.429+0000 7f2306000640 1 --2- 172.21.15.7:0/3428361025 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f230006ff50 0x7f2300154bb0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.429+0000 7f2306000640 1 -- 172.21.15.7:0/3428361025 >> 172.21.15.7:0/3428361025 conn(0x7f23000b1660 msgr2=0x7f23000affd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.429+0000 7f2306000640 1 -- 172.21.15.7:0/3428361025 shutdown_connections 2024-09-10T21:51:49.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.429+0000 7f2306000640 1 -- 172.21.15.7:0/3428361025 wait complete. 2024-09-10T21:51:49.446 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:51:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.610+0000 7f72d3c00640 1 -- 172.21.15.7:0/1921273833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72cc0cbd50 msgr2=0x7f72cc0cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.610+0000 7f72d3c00640 1 --2- 172.21.15.7:0/1921273833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72cc0cbd50 0x7f72cc0cc1c0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f72b8005fd0 tx=0x7f72b802a5d0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.610+0000 7f72d3c00640 1 -- 172.21.15.7:0/1921273833 shutdown_connections 2024-09-10T21:51:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.610+0000 7f72d3c00640 1 --2- 172.21.15.7:0/1921273833 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72cc0ae020 0x7f72cc0ae4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.610+0000 7f72d3c00640 1 --2- 172.21.15.7:0/1921273833 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72cc0cbd50 0x7f72cc0cc1c0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.610+0000 7f72d3c00640 1 -- 172.21.15.7:0/1921273833 >> 172.21.15.7:0/1921273833 conn(0x7f72cc0b16d0 msgr2=0x7f72cc0b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.610+0000 7f72d3c00640 1 -- 172.21.15.7:0/1921273833 shutdown_connections 2024-09-10T21:51:49.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.610+0000 7f72d3c00640 1 -- 172.21.15.7:0/1921273833 wait complete. 2024-09-10T21:51:49.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.612+0000 7f72d3c00640 1 Processor -- start 2024-09-10T21:51:49.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.612+0000 7f72d3c00640 1 -- start start 2024-09-10T21:51:49.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.613+0000 7f72d3c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72cc0ae020 0x7f72cc0b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.613+0000 7f72d3c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72cc0cbd50 0x7f72cc14af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.613+0000 7f72d3c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72cc1619e0 0x7f72cc0733b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.613+0000 7f72d3c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc0ce0d0 con 0x7f72cc0ae020 2024-09-10T21:51:49.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.613+0000 7f72d3c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f72cc0cdf50 con 0x7f72cc0cbd50 2024-09-10T21:51:49.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.613+0000 7f72d3c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f72cc161dc0 con 0x7f72cc1619e0 2024-09-10T21:51:49.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.613+0000 7f72d1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72cc0ae020 0x7f72cc0b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.613+0000 7f72d1200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72cc0ae020 0x7f72cc0b29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43918/0 (socket says 172.21.15.7:43918) 2024-09-10T21:51:49.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.613+0000 7f72d1200640 1 -- 172.21.15.7:0/590067589 learned_addr learned my addr 172.21.15.7:0/590067589 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:49.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.613+0000 7f72cbe00640 1 --2- 172.21.15.7:0/590067589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72cc0cbd50 0x7f72cc14af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.613+0000 7f72d1c00640 1 --2- 172.21.15.7:0/590067589 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72cc1619e0 0x7f72cc0733b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.613+0000 7f72d1200640 1 -- 172.21.15.7:0/590067589 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72cc1619e0 msgr2=0x7f72cc0733b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:49.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.613+0000 7f72d1200640 1 --2- 172.21.15.7:0/590067589 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72cc1619e0 0x7f72cc0733b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.613+0000 7f72d1200640 1 -- 172.21.15.7:0/590067589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72cc0cbd50 msgr2=0x7f72cc14af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:49.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.613+0000 7f72d1200640 1 --2- 172.21.15.7:0/590067589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72cc0cbd50 0x7f72cc14af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.613+0000 7f72d1200640 1 -- 172.21.15.7:0/590067589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72cc0b3200 con 0x7f72cc0ae020 2024-09-10T21:51:49.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.614+0000 7f72d1200640 1 --2- 172.21.15.7:0/590067589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72cc0ae020 0x7f72cc0b29d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f72c000c430 tx=0x7f72c000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:49.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.614+0000 7f72c8c00640 1 -- 172.21.15.7:0/590067589 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0017070 con 0x7f72cc0ae020 2024-09-10T21:51:49.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.614+0000 7f72c8c00640 1 -- 172.21.15.7:0/590067589 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f72c00053c0 con 0x7f72cc0ae020 2024-09-10T21:51:49.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.614+0000 7f72c8c00640 1 -- 172.21.15.7:0/590067589 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72c0003850 con 0x7f72cc0ae020 2024-09-10T21:51:49.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.614+0000 7f72d3c00640 1 -- 172.21.15.7:0/590067589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72cc0b34e0 con 0x7f72cc0ae020 2024-09-10T21:51:49.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.614+0000 7f72d3c00640 1 -- 172.21.15.7:0/590067589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72cc0c8070 con 0x7f72cc0ae020 2024-09-10T21:51:49.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.615+0000 7f72d3c00640 1 -- 172.21.15.7:0/590067589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72cc0ae4b0 con 0x7f72cc0ae020 2024-09-10T21:51:49.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.617+0000 7f72c8c00640 1 -- 172.21.15.7:0/590067589 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f72c00039f0 con 0x7f72cc0ae020 2024-09-10T21:51:49.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.618+0000 7f72c8c00640 1 --2- 172.21.15.7:0/590067589 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f72ac077f70 0x7f72ac07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:49.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.618+0000 7f72c8c00640 1 -- 172.21.15.7:0/590067589 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5767+0+0 (secure 0 0 0) 0x7f72c00998d0 con 0x7f72cc0ae020 2024-09-10T21:51:49.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.618+0000 7f72cbe00640 1 --2- 172.21.15.7:0/590067589 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f72ac077f70 0x7f72ac07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:49.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.619+0000 7f72cbe00640 1 --2- 172.21.15.7:0/590067589 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f72ac077f70 0x7f72ac07a430 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f72b8033000 tx=0x7f72b8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:49.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.621+0000 7f72c8c00640 1 -- 172.21.15.7:0/590067589 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f72c0021020 con 0x7f72cc0ae020 2024-09-10T21:51:49.856 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.855+0000 7f72d3c00640 1 -- 172.21.15.7:0/590067589 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f72cc0b0ab0 con 0x7f72cc0ae020 2024-09-10T21:51:49.857 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.856+0000 7f72c8c00640 1 -- 172.21.15.7:0/590067589 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v102) ==== 53+0+73 (secure 0 0 0) 0x7f72c0062400 con 0x7f72cc0ae020 2024-09-10T21:51:49.857 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:51:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.858+0000 7f72d3c00640 1 -- 172.21.15.7:0/590067589 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f72ac077f70 msgr2=0x7f72ac07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.858+0000 7f72d3c00640 1 --2- 172.21.15.7:0/590067589 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f72ac077f70 0x7f72ac07a430 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f72b8033000 tx=0x7f72b8004730 comp rx=0 tx=0).stop 2024-09-10T21:51:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.858+0000 7f72d3c00640 1 -- 172.21.15.7:0/590067589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72cc0ae020 msgr2=0x7f72cc0b29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.858+0000 7f72d3c00640 1 --2- 172.21.15.7:0/590067589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72cc0ae020 0x7f72cc0b29d0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f72c000c430 tx=0x7f72c000c8f0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.859+0000 7f72d3c00640 1 -- 172.21.15.7:0/590067589 shutdown_connections 2024-09-10T21:51:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.859+0000 7f72d3c00640 1 --2- 172.21.15.7:0/590067589 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f72ac077f70 0x7f72ac07a430 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.859+0000 7f72d3c00640 1 --2- 172.21.15.7:0/590067589 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f72cc0cbd50 0x7f72cc14af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.859+0000 7f72d3c00640 1 --2- 172.21.15.7:0/590067589 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f72cc1619e0 0x7f72cc0733b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.859+0000 7f72d3c00640 1 --2- 172.21.15.7:0/590067589 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f72cc0ae020 0x7f72cc0b29d0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.859+0000 7f72d3c00640 1 -- 172.21.15.7:0/590067589 >> 172.21.15.7:0/590067589 conn(0x7f72cc0b16d0 msgr2=0x7f72cc15f520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:49.860 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.859+0000 7f72d3c00640 1 -- 172.21.15.7:0/590067589 shutdown_connections 2024-09-10T21:51:49.861 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:49.859+0000 7f72d3c00640 1 -- 172.21.15.7:0/590067589 wait complete. 2024-09-10T21:51:49.875 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:51:49.875 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:51:49.875 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-10T21:51:49.875 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:51:49.875 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:51:49.876 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:51:49.876 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:51:49.876 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:51:49.876 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:51:49.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:49.876 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-10T21:51:49.912 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:51:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:51:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:51:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:51:49.912 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T21:51:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:51:49.913 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:51:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T21:51:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-10T21:51:49.913 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:51:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T21:51:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-10T21:51:49.913 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:51:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-10T21:51:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-10T21:51:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T21:51:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42035sec preferred_lft 42035sec 2024-09-10T21:51:49.913 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-10T21:51:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:51:49.914 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:51:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-10T21:51:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-10T21:51:49.914 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:51:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff 2024-09-10T21:51:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:51:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:51:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::3c3d:dbff:fe02:4c15/64 scope link 2024-09-10T21:51:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:51:49.914 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:51:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether f6:f9:69:5a:86:f5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:51:49.914 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f4f9:69ff:fe5a:86f5/64 scope link 2024-09-10T21:51:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:51:49.915 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:51:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:51:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::433:5cff:fe44:4a5b/64 scope link 2024-09-10T21:51:49.915 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:51:49.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:49.915 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:51:50.006 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:51:50.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:50.009 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-10T21:51:50.045 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-10T21:51:50.046 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:51:50.046 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T21:51:50.046 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T21:51:50.046 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:51:50.046 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:51:50.046 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:51:50.046 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:51:50.046 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:51:50.046 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:51:50.046 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:51:50.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:50 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:51:50.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:51:50.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:50.246 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:51:50.281 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:51:50.281 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:51:50.281 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:51:50.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:50.281 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-10T21:51:50.376 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:51:50.377 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:51:50.412 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:51:50.412 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:51:50.412 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:51:50.487 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:51:50.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:50.488 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:51:50.616 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:51:50.616 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:51:50.644 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:51:50.644 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:51:50.644 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.133:3300,172.21.15.133:3301" 2024-09-10T21:51:50.644 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@2029451a-3c26-48c7-b874-0597197278ac.cephfs=/ 2024-09-10T21:51:50.644 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.133:3300/172.21.15.133:3301" will pass to kernel 2024-09-10T21:51:50.646 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T21:51:50.646 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:51:50.646 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:51:50.646 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:51:50.646 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:51:50.649 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:51:50.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:50.649 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:51:50.705 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:51:50.787 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:51:50.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:51:50.804 INFO:teuthology.orchestra.run.smithi007.stdout:client7612 2024-09-10T21:51:50.804 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:51:50.804 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:51:50.839 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:51:50.839 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:51:50.840 DEBUG:teuthology.orchestra.run.smithi007:> mount_point = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T21:51:50.840 DEBUG:teuthology.orchestra.run.smithi007:> parent = os.path.join(mount_point, "parent") 2024-09-10T21:51:50.840 DEBUG:teuthology.orchestra.run.smithi007:> os.mkdir(parent) 2024-09-10T21:51:50.840 DEBUG:teuthology.orchestra.run.smithi007:> flushed = os.path.join(parent, "flushed") 2024-09-10T21:51:50.840 DEBUG:teuthology.orchestra.run.smithi007:> os.mkdir(flushed) 2024-09-10T21:51:50.840 DEBUG:teuthology.orchestra.run.smithi007:> for f in ["alpha", "bravo", "charlie"]: 2024-09-10T21:51:50.840 DEBUG:teuthology.orchestra.run.smithi007:> open(os.path.join(flushed, f), '"'"'w'"'"').write(f) 2024-09-10T21:51:50.840 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:51:50.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:51:51.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:51.005 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-09-10T21:51:51.088 INFO:teuthology.orchestra.run.smithi007.stdout:./ 2024-09-10T21:51:51.088 INFO:teuthology.orchestra.run.smithi007.stdout:./parent 2024-09-10T21:51:51.088 INFO:teuthology.orchestra.run.smithi007.stdout:./parent/flushed 2024-09-10T21:51:51.088 INFO:teuthology.orchestra.run.smithi007.stdout:./parent/flushed/charlie 2024-09-10T21:51:51.088 INFO:teuthology.orchestra.run.smithi007.stdout:./parent/flushed/alpha 2024-09-10T21:51:51.088 INFO:teuthology.orchestra.run.smithi007.stdout:./parent/flushed/bravo 2024-09-10T21:51:51.089 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:51:51.089 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:51:51.090 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:51:51.090 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:51:51.090 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-09-10T21:51:51.090 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:51:51.229 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:51:51.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:51:51.241 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:51:51.241 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:51:51.241 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:51:51.241 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:51:51.241 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-09-10T21:51:51.242 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:51:51.380 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627776 2024-09-10T21:51:51.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:51:51.392 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:51:51.392 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:51:51.392 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:51:51.392 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:51:51.392 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-09-10T21:51:51.392 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:51:51.532 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627777 2024-09-10T21:51:51.535 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:51:51.543 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:51:51.543 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:51:51.543 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:51:51.543 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:51:51.543 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-09-10T21:51:51.543 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:51:51.682 INFO:teuthology.orchestra.run.smithi007.stdout:1099511628280 2024-09-10T21:51:51.685 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:51:51.693 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:51:51.694 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:51:51.694 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:51:51.694 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:51:51.694 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-09-10T21:51:51.694 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:51:51.834 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627778 2024-09-10T21:51:51.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:51:51.845 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:51:51.845 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:51:51.845 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:51:51.845 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:51:51.846 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-09-10T21:51:51.846 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:51:51.985 INFO:teuthology.orchestra.run.smithi007.stdout:1099511628279 2024-09-10T21:51:51.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:51 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:51:51.996 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:51:51.997 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:51:52.072 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:51:52.072 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:51:52.148 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:51:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.348+0000 7f20a1000640 1 -- 172.21.15.7:0/165301033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f209c06ff50 msgr2=0x7f209c15cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.348+0000 7f20a1000640 1 --2- 172.21.15.7:0/165301033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f209c06ff50 0x7f209c15cbe0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f208c009830 tx=0x7f208c02a3f0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.348+0000 7f20a1000640 1 -- 172.21.15.7:0/165301033 shutdown_connections 2024-09-10T21:51:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.348+0000 7f20a1000640 1 --2- 172.21.15.7:0/165301033 >> v2:172.21.15.133:3300/0 conn(0x7f209c0743e0 0x7f209c15f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.349+0000 7f20a1000640 1 --2- 172.21.15.7:0/165301033 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f209c06ff50 0x7f209c15cbe0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.349+0000 7f20a1000640 1 -- 172.21.15.7:0/165301033 >> 172.21.15.7:0/165301033 conn(0x7f209c0b1660 msgr2=0x7f209c0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.349+0000 7f20a1000640 1 -- 172.21.15.7:0/165301033 shutdown_connections 2024-09-10T21:51:52.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.349+0000 7f20a1000640 1 -- 172.21.15.7:0/165301033 wait complete. 2024-09-10T21:51:52.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.350+0000 7f20a1000640 1 Processor -- start 2024-09-10T21:51:52.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.350+0000 7f20a1000640 1 -- start start 2024-09-10T21:51:52.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.351+0000 7f20a1000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f209c06e510 0x7f209c15df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.351+0000 7f20a1000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f209c06ff50 0x7f209c0b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.351+0000 7f20a1000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f209c0743e0 0x7f209c149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.351+0000 7f20a1000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f209c06eb40 con 0x7f209c06e510 2024-09-10T21:51:52.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.351+0000 7f20a1000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f209c15fc50 con 0x7f209c0743e0 2024-09-10T21:51:52.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.351+0000 7f20a1000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f209c15e4b0 con 0x7f209c06ff50 2024-09-10T21:51:52.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.352+0000 7f2099e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f209c06e510 0x7f209c15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.352+0000 7f2099e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f209c06e510 0x7f209c15df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43924/0 (socket says 172.21.15.7:43924) 2024-09-10T21:51:52.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.352+0000 7f2099e00640 1 -- 172.21.15.7:0/1022776965 learned_addr learned my addr 172.21.15.7:0/1022776965 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:52.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.352+0000 7f2099e00640 1 -- 172.21.15.7:0/1022776965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f209c06ff50 msgr2=0x7f209c0b4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:52.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.352+0000 7f2099e00640 1 --2- 172.21.15.7:0/1022776965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f209c06ff50 0x7f209c0b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.352+0000 7f2099e00640 1 -- 172.21.15.7:0/1022776965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f209c0743e0 msgr2=0x7f209c149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:52.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.352+0000 7f2099e00640 1 --2- 172.21.15.7:0/1022776965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f209c0743e0 0x7f209c149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.352+0000 7f2099e00640 1 -- 172.21.15.7:0/1022776965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209c14bd60 con 0x7f209c06e510 2024-09-10T21:51:52.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.352+0000 7f2099e00640 1 --2- 172.21.15.7:0/1022776965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f209c06e510 0x7f209c15df70 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f208c009950 tx=0x7f208c032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:52.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.353+0000 7f2092a00640 1 -- 172.21.15.7:0/1022776965 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c040070 con 0x7f209c06e510 2024-09-10T21:51:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.353+0000 7f2092a00640 1 -- 172.21.15.7:0/1022776965 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f208c005310 con 0x7f209c06e510 2024-09-10T21:51:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.353+0000 7f2092a00640 1 -- 172.21.15.7:0/1022776965 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208c005c90 con 0x7f209c06e510 2024-09-10T21:51:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.353+0000 7f20a1000640 1 -- 172.21.15.7:0/1022776965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209c14c040 con 0x7f209c06e510 2024-09-10T21:51:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.353+0000 7f20a1000640 1 -- 172.21.15.7:0/1022776965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f209c14c600 con 0x7f209c06e510 2024-09-10T21:51:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.354+0000 7f20a1000640 1 -- 172.21.15.7:0/1022776965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f209c1600e0 con 0x7f209c06e510 2024-09-10T21:51:52.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.355+0000 7f2092a00640 1 -- 172.21.15.7:0/1022776965 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f208c04c030 con 0x7f209c06e510 2024-09-10T21:51:52.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.356+0000 7f2092a00640 1 --2- 172.21.15.7:0/1022776965 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2078077f70 0x7f207807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.356+0000 7f2092a00640 1 -- 172.21.15.7:0/1022776965 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5767+0+0 (secure 0 0 0) 0x7f208c0bad60 con 0x7f209c06e510 2024-09-10T21:51:52.357 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.356+0000 7f2091400640 1 --2- 172.21.15.7:0/1022776965 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2078077f70 0x7f207807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.358 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.357+0000 7f2091400640 1 --2- 172.21.15.7:0/1022776965 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2078077f70 0x7f207807a430 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2084007fd0 tx=0x7f2084007900 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:52.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.360+0000 7f2092a00640 1 -- 172.21.15.7:0/1022776965 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f208c083810 con 0x7f209c06e510 2024-09-10T21:51:52.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.568+0000 7f20a1000640 1 -- 172.21.15.7:0/1022776965 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f209c0b1be0 con 0x7f209c06e510 2024-09-10T21:51:52.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.568+0000 7f2092a00640 1 -- 172.21.15.7:0/1022776965 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v116) ==== 80+0+0 (secure 0 0 0) 0x7f208c083110 con 0x7f209c06e510 2024-09-10T21:51:52.569 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-10T21:51:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.570+0000 7f20a1000640 1 -- 172.21.15.7:0/1022776965 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2078077f70 msgr2=0x7f207807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.570+0000 7f20a1000640 1 --2- 172.21.15.7:0/1022776965 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2078077f70 0x7f207807a430 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2084007fd0 tx=0x7f2084007900 comp rx=0 tx=0).stop 2024-09-10T21:51:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.570+0000 7f20a1000640 1 -- 172.21.15.7:0/1022776965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f209c06e510 msgr2=0x7f209c15df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.570+0000 7f20a1000640 1 --2- 172.21.15.7:0/1022776965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f209c06e510 0x7f209c15df70 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f208c009950 tx=0x7f208c032c10 comp rx=0 tx=0).stop 2024-09-10T21:51:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.571+0000 7f20a1000640 1 -- 172.21.15.7:0/1022776965 shutdown_connections 2024-09-10T21:51:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.571+0000 7f20a1000640 1 --2- 172.21.15.7:0/1022776965 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2078077f70 0x7f207807a430 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.571+0000 7f20a1000640 1 --2- 172.21.15.7:0/1022776965 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f209c0743e0 0x7f209c149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.571+0000 7f20a1000640 1 --2- 172.21.15.7:0/1022776965 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f209c06ff50 0x7f209c0b4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.571+0000 7f20a1000640 1 --2- 172.21.15.7:0/1022776965 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f209c06e510 0x7f209c15df70 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.571+0000 7f20a1000640 1 -- 172.21.15.7:0/1022776965 >> 172.21.15.7:0/1022776965 conn(0x7f209c0b1660 msgr2=0x7f209c0afcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.571+0000 7f20a1000640 1 -- 172.21.15.7:0/1022776965 shutdown_connections 2024-09-10T21:51:52.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.571+0000 7f20a1000640 1 -- 172.21.15.7:0/1022776965 wait complete. 2024-09-10T21:51:52.588 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:51:52.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:52.588 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:51:52.695 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:51:52.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:52.696 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:51:52.736 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-09-10T21:51:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.938+0000 7fa3fc400640 1 -- 172.21.15.7:0/54377981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3f40ae000 msgr2=0x7fa3f40cdc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.938+0000 7fa3fc400640 1 --2- 172.21.15.7:0/54377981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3f40ae000 0x7fa3f40cdc30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec005fd0 tx=0x7fa3ec02a820 comp rx=0 tx=0).stop 2024-09-10T21:51:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.938+0000 7fa3fc400640 1 -- 172.21.15.7:0/54377981 shutdown_connections 2024-09-10T21:51:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.938+0000 7fa3fc400640 1 --2- 172.21.15.7:0/54377981 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3f40ae000 0x7fa3f40cdc30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.938+0000 7fa3fc400640 1 --2- 172.21.15.7:0/54377981 >> v2:172.21.15.133:3300/0 conn(0x7fa3f4158220 0x7fa3f40cd6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.938+0000 7fa3fc400640 1 -- 172.21.15.7:0/54377981 >> 172.21.15.7:0/54377981 conn(0x7fa3f40b16d0 msgr2=0x7fa3f40b1ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:52.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.938+0000 7fa3fc400640 1 -- 172.21.15.7:0/54377981 shutdown_connections 2024-09-10T21:51:52.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.938+0000 7fa3fc400640 1 -- 172.21.15.7:0/54377981 wait complete. 2024-09-10T21:51:52.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.940+0000 7fa3fc400640 1 Processor -- start 2024-09-10T21:51:52.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.940+0000 7fa3fc400640 1 -- start start 2024-09-10T21:51:52.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.941+0000 7fa3fc400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3f40ae000 0x7fa3f40c4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.943 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.941+0000 7fa3fc400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3f4158220 0x7fa3f41f7e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.941+0000 7fa3fc400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3f4158600 0x7fa3f41f8390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.941+0000 7fa3fc400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f406f9c0 con 0x7fa3f40ae000 2024-09-10T21:51:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.941+0000 7fa3fc400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f406f840 con 0x7fa3f4158220 2024-09-10T21:51:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.941+0000 7fa3fc400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f406fb40 con 0x7fa3f4158600 2024-09-10T21:51:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.941+0000 7fa3f9a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3f40ae000 0x7fa3f40c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.941+0000 7fa3f9a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3f40ae000 0x7fa3f40c4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43930/0 (socket says 172.21.15.7:43930) 2024-09-10T21:51:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.941+0000 7fa3f9a00640 1 -- 172.21.15.7:0/210915264 learned_addr learned my addr 172.21.15.7:0/210915264 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.942+0000 7fa3f9000640 1 --2- 172.21.15.7:0/210915264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3f4158220 0x7fa3f41f7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.942+0000 7fa3fa400640 1 --2- 172.21.15.7:0/210915264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3f4158600 0x7fa3f41f8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.942+0000 7fa3f9a00640 1 -- 172.21.15.7:0/210915264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3f4158600 msgr2=0x7fa3f41f8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:52.944 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.942+0000 7fa3f9a00640 1 --2- 172.21.15.7:0/210915264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3f4158600 0x7fa3f41f8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.942+0000 7fa3f9a00640 1 -- 172.21.15.7:0/210915264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3f4158220 msgr2=0x7fa3f41f7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.942+0000 7fa3f9a00640 1 --2- 172.21.15.7:0/210915264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3f4158220 0x7fa3f41f7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.942+0000 7fa3f9a00640 1 -- 172.21.15.7:0/210915264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f40743e0 con 0x7fa3f40ae000 2024-09-10T21:51:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.942+0000 7fa3fa400640 1 --2- 172.21.15.7:0/210915264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3f4158600 0x7fa3f41f8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:51:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.942+0000 7fa3f9a00640 1 --2- 172.21.15.7:0/210915264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3f40ae000 0x7fa3f40c4210 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800ea60 tx=0x7fa3e800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.942+0000 7fa3f1600640 1 -- 172.21.15.7:0/210915264 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e800cde0 con 0x7fa3f40ae000 2024-09-10T21:51:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.942+0000 7fa3f1600640 1 -- 172.21.15.7:0/210915264 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3e8004020 con 0x7fa3f40ae000 2024-09-10T21:51:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.942+0000 7fa3f1600640 1 -- 172.21.15.7:0/210915264 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8004450 con 0x7fa3f40ae000 2024-09-10T21:51:52.945 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.942+0000 7fa3fc400640 1 -- 172.21.15.7:0/210915264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f40c3280 con 0x7fa3f40ae000 2024-09-10T21:51:52.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.943+0000 7fa3fc400640 1 -- 172.21.15.7:0/210915264 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f40c2f70 con 0x7fa3f40ae000 2024-09-10T21:51:52.946 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.945+0000 7fa3f1600640 1 -- 172.21.15.7:0/210915264 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa3e80035f0 con 0x7fa3f40ae000 2024-09-10T21:51:52.947 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.946+0000 7fa3f1600640 1 --2- 172.21.15.7:0/210915264 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa3d4077f70 0x7fa3d407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.947+0000 7fa3f1600640 1 -- 172.21.15.7:0/210915264 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5767+0+0 (secure 0 0 0) 0x7fa3e809a790 con 0x7fa3f40ae000 2024-09-10T21:51:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.947+0000 7fa3ff482740 1 Processor -- start 2024-09-10T21:51:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.947+0000 7fa3f9000640 1 --2- 172.21.15.7:0/210915264 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa3d4077f70 0x7fa3d407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.947+0000 7fa3ff482740 1 -- start start 2024-09-10T21:51:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.948+0000 7fa3f9000640 1 --2- 172.21.15.7:0/210915264 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa3d4077f70 0x7fa3d407a430 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0007fd0 tx=0x7fa3e0001370 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:52.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.948+0000 7fa3ff482740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa0a29130 0x561aa0a29510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.948+0000 7fa3ff482740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e42a0 0x561aa0a27c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.948+0000 7fa3ff482740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa09e4820 0x561aa0a28150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.948+0000 7fa3ff482740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561aa09431c0 con 0x561aa09e42a0 2024-09-10T21:51:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.948+0000 7fa3ff482740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561aa0956870 con 0x561aa09e4820 2024-09-10T21:51:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.948+0000 7fa3ff482740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x561aa0945ca0 con 0x561aa0a29130 2024-09-10T21:51:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.948+0000 7fa3f9000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e42a0 0x561aa0a27c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.948+0000 7fa3f9000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e42a0 0x561aa0a27c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43938/0 (socket says 172.21.15.7:43938) 2024-09-10T21:51:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.948+0000 7fa3f9000640 1 -- 172.21.15.7:0/3647264915 learned_addr learned my addr 172.21.15.7:0/3647264915 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.949+0000 7fa3f9a00640 1 --2- 172.21.15.7:0/3647264915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa0a29130 0x561aa0a29510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.949+0000 7fa3fa400640 1 --2- 172.21.15.7:0/3647264915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa09e4820 0x561aa0a28150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.950 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.949+0000 7fa3f9000640 1 -- 172.21.15.7:0/3647264915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa0a29130 msgr2=0x561aa0a29510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.949+0000 7fa3f9000640 1 --2- 172.21.15.7:0/3647264915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa0a29130 0x561aa0a29510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.949+0000 7fa3f9000640 1 -- 172.21.15.7:0/3647264915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa09e4820 msgr2=0x561aa0a28150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.949+0000 7fa3f9000640 1 --2- 172.21.15.7:0/3647264915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa09e4820 0x561aa0a28150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.951 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.949+0000 7fa3f9000640 1 -- 172.21.15.7:0/3647264915 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aa071e7d0 con 0x561aa09e42a0 2024-09-10T21:51:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.949+0000 7fa3f9000640 1 --2- 172.21.15.7:0/3647264915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e42a0 0x561aa0a27c10 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000f9d0 tx=0x7fa3e0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d690f430335e48 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.949+0000 7fa3e6000640 1 -- 172.21.15.7:0/3647264915 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e001b050 con 0x561aa09e42a0 2024-09-10T21:51:52.952 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.950+0000 7fa3e6000640 1 -- 172.21.15.7:0/3647264915 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3e0025070 con 0x561aa09e42a0 2024-09-10T21:51:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.950+0000 7fa3e6000640 1 -- 172.21.15.7:0/3647264915 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0014de0 con 0x561aa09e42a0 2024-09-10T21:51:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.950+0000 7fa3ff482740 1 -- 172.21.15.7:0/3647264915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e42a0 msgr2=0x561aa0a27c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.950+0000 7fa3ff482740 1 --2- 172.21.15.7:0/3647264915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e42a0 0x561aa0a27c10 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000f9d0 tx=0x7fa3e0012000 comp rx=0 tx=0).stop 2024-09-10T21:51:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.950+0000 7fa3ff482740 1 -- 172.21.15.7:0/3647264915 shutdown_connections 2024-09-10T21:51:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.950+0000 7fa3ff482740 1 --2- 172.21.15.7:0/3647264915 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa09e4820 0x561aa0a28150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.950+0000 7fa3ff482740 1 --2- 172.21.15.7:0/3647264915 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e42a0 0x561aa0a27c10 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.950+0000 7fa3ff482740 1 --2- 172.21.15.7:0/3647264915 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa0a29130 0x561aa0a29510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.950+0000 7fa3ff482740 1 -- 172.21.15.7:0/3647264915 >> 172.21.15.7:0/3647264915 conn(0x561aa09df870 msgr2=0x561aa09e3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.950+0000 7fa3ff482740 1 -- 172.21.15.7:0/3647264915 shutdown_connections 2024-09-10T21:51:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.950+0000 7fa3ff482740 1 -- 172.21.15.7:0/3647264915 wait complete. 2024-09-10T21:51:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.951+0000 7fa3ff482740 1 -- start start 2024-09-10T21:51:52.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.951+0000 7fa3ff482740 1 Processor -- start 2024-09-10T21:51:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.952+0000 7fa3ff482740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 0x561aa09f7f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.952+0000 7fa3ff482740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa0a29130 0x561aa0a3b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.952+0000 7fa3ff482740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa0a3b910 0x561aa09e1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.952+0000 7fa3ff482740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561aa08ec410 con 0x561aa09e4820 2024-09-10T21:51:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.952+0000 7fa3ff482740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x561aa0949ab0 con 0x561aa0a29130 2024-09-10T21:51:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.952+0000 7fa3ff482740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x561aa06b6740 con 0x561aa0a3b910 2024-09-10T21:51:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.952+0000 7fa3ff482740 10 client.0 _ll_register_callbacks cb 0x561aa08c9360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:51:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.952+0000 7fa3f9a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 0x561aa09f7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.952+0000 7fa3f9a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 0x561aa09f7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43942/0 (socket says 172.21.15.7:43942) 2024-09-10T21:51:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.952+0000 7fa3f9a00640 1 -- 172.21.15.7:0/4117922163 learned_addr learned my addr 172.21.15.7:0/4117922163 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.953+0000 7fa3f9000640 1 --2- 172.21.15.7:0/4117922163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa0a29130 0x561aa0a3b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.953+0000 7fa3fa400640 1 --2- 172.21.15.7:0/4117922163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa0a3b910 0x561aa09e1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.953+0000 7fa3f9a00640 1 -- 172.21.15.7:0/4117922163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa0a3b910 msgr2=0x561aa09e1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.953+0000 7fa3f9a00640 1 --2- 172.21.15.7:0/4117922163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa0a3b910 0x561aa09e1550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.953+0000 7fa3f9a00640 1 -- 172.21.15.7:0/4117922163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa0a29130 msgr2=0x561aa0a3b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.953+0000 7fa3f9a00640 1 --2- 172.21.15.7:0/4117922163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa0a29130 0x561aa0a3b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.953+0000 7fa3f9a00640 1 -- 172.21.15.7:0/4117922163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aa06a7970 con 0x561aa09e4820 2024-09-10T21:51:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.953+0000 7fa3fa400640 1 --2- 172.21.15.7:0/4117922163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa0a3b910 0x561aa09e1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:51:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.953+0000 7fa3f9a00640 1 --2- 172.21.15.7:0/4117922163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 0x561aa09f7f50 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8098b50 tx=0x7fa3e805e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83dd7dd7756df999 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.953+0000 7fa3daa00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:51:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.953+0000 7fa3daa00640 1 -- 172.21.15.7:0/4117922163 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8096060 con 0x561aa09e4820 2024-09-10T21:51:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.953+0000 7fa3daa00640 1 -- 172.21.15.7:0/4117922163 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5767+0+0 (secure 0 0 0) 0x7fa3e80670c0 con 0x561aa09e4820 2024-09-10T21:51:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.953+0000 7fa3ff482740 1 -- 172.21.15.7:0/4117922163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3e0025070 con 0x561aa09e4820 2024-09-10T21:51:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.953+0000 7fa3daa00640 1 -- 172.21.15.7:0/4117922163 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7fa3e807c070 con 0x561aa09e4820 2024-09-10T21:51:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.954+0000 7fa3ff482740 10 client.8274 fetch_fsmap learned FSMap version 102 2024-09-10T21:51:52.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.954+0000 7fa3ff482740 1 -- 172.21.15.7:0/4117922163 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) -- 0x561aa06856a0 con 0x561aa09e4820 2024-09-10T21:51:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.954+0000 7fa3daa00640 1 -- 172.21.15.7:0/4117922163 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3e0025070 con 0x561aa09e4820 2024-09-10T21:51:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.954+0000 7fa3daa00640 1 -- 172.21.15.7:0/4117922163 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 102) ==== 3502+0+0 (secure 0 0 0) 0x7fa3e8061d10 con 0x561aa09e4820 2024-09-10T21:51:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.954+0000 7fa3ff482740 10 client.8274 fetch_fsmap finished waiting for FSMap version 102 2024-09-10T21:51:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.954+0000 7fa3ff482740 10 client.8274 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-09-10T21:51:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.954+0000 7fa3ff482740 20 client.8274 populate_metadata read hostname 'smithi007' 2024-09-10T21:51:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.954+0000 7fa3ff482740 1 --2- 172.21.15.7:0/4117922163 >> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] conn(0x561aa09fb670 0x561aa0a27530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.954+0000 7fa3ff482740 4 client.8274 mds_command: new command op to 236726 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:51:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.954+0000 7fa3ff482740 1 -- 172.21.15.7:0/4117922163 --> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561aa09587c0 con 0x561aa09fb670 2024-09-10T21:51:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.954+0000 7fa3f9000640 1 --2- 172.21.15.7:0/4117922163 >> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] conn(0x561aa09fb670 0x561aa0a27530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.955+0000 7fa3f9000640 1 --2- 172.21.15.7:0/4117922163 >> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] conn(0x561aa09fb670 0x561aa0a27530 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e0817e6b294395c server_cookie=60e593b5fcf4803f in_seq=0 out_seq=0 2024-09-10T21:51:52.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.955+0000 7fa3daa00640 10 client.8274 ms_handle_connect on v2:172.21.15.133:6834/3764354952 2024-09-10T21:51:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.956+0000 7fa3daa00640 1 -- 172.21.15.7:0/4117922163 <== mds.0 v2:172.21.15.133:6834/3764354952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561aa09587c0 con 0x561aa09fb670 2024-09-10T21:51:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.956+0000 7fa3daa00640 10 client.8274 handle_command_reply: tid=0 2024-09-10T21:51:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.956+0000 7fa3ff482740 1 client.8274 shutdown 2024-09-10T21:51:52.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.956+0000 7fa3ff482740 1 -- 172.21.15.7:0/4117922163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 msgr2=0x561aa09f7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.956+0000 7fa3ff482740 1 --2- 172.21.15.7:0/4117922163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 0x561aa09f7f50 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8098b50 tx=0x7fa3e805e950 comp rx=0 tx=0).stop 2024-09-10T21:51:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.956+0000 7fa3ff482740 1 -- 172.21.15.7:0/4117922163 shutdown_connections 2024-09-10T21:51:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.956+0000 7fa3ff482740 1 --2- 172.21.15.7:0/4117922163 >> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] conn(0x561aa09fb670 0x561aa0a27530 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.956+0000 7fa3ff482740 1 --2- 172.21.15.7:0/4117922163 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa0a29130 0x561aa0a3b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.956+0000 7fa3ff482740 1 --2- 172.21.15.7:0/4117922163 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa0a3b910 0x561aa09e1550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.956+0000 7fa3ff482740 1 --2- 172.21.15.7:0/4117922163 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 0x561aa09f7f50 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.956+0000 7fa3daa00640 0 client.8274 ms_handle_reset on v2:172.21.15.133:6834/3764354952 2024-09-10T21:51:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.956+0000 7fa3ff482740 1 -- 172.21.15.7:0/4117922163 >> 172.21.15.7:0/4117922163 conn(0x561aa09e42a0 msgr2=0x561aa0a286b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.956+0000 7fa3ff482740 1 -- 172.21.15.7:0/4117922163 shutdown_connections 2024-09-10T21:51:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.957+0000 7fa3ff482740 1 -- 172.21.15.7:0/4117922163 wait complete. 2024-09-10T21:51:52.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.957+0000 7fa3ff482740 20 client.8274 trim_cache size 0 max 16384 2024-09-10T21:51:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.972+0000 7fa3ff482740 1 Processor -- start 2024-09-10T21:51:52.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.972+0000 7fa3ff482740 1 -- start start 2024-09-10T21:51:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.973+0000 7fa3ff482740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa09e42a0 0x561aa09f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.973+0000 7fa3ff482740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 0x561aa09d9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.973+0000 7fa3ff482740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa0a29130 0x561aa0a283e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.973+0000 7fa3ff482740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f4201100 con 0x561aa09e4820 2024-09-10T21:51:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.973+0000 7fa3ff482740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f42001e0 con 0x561aa0a29130 2024-09-10T21:51:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.973+0000 7fa3ff482740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa3f4073070 con 0x561aa09e42a0 2024-09-10T21:51:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.973+0000 7fa3f9000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 0x561aa09d9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.974 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.973+0000 7fa3f9000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 0x561aa09d9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43944/0 (socket says 172.21.15.7:43944) 2024-09-10T21:51:52.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.973+0000 7fa3f9000640 1 -- 172.21.15.7:0/1509338617 learned_addr learned my addr 172.21.15.7:0/1509338617 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.974+0000 7fa3fa400640 1 --2- 172.21.15.7:0/1509338617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa0a29130 0x561aa0a283e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.974+0000 7fa3f9a00640 1 --2- 172.21.15.7:0/1509338617 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa09e42a0 0x561aa09f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.974+0000 7fa3f9000640 1 -- 172.21.15.7:0/1509338617 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa09e42a0 msgr2=0x561aa09f7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.974+0000 7fa3f9000640 1 --2- 172.21.15.7:0/1509338617 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa09e42a0 0x561aa09f7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.974+0000 7fa3f9000640 1 -- 172.21.15.7:0/1509338617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa0a29130 msgr2=0x561aa0a283e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.974+0000 7fa3f9000640 1 --2- 172.21.15.7:0/1509338617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa0a29130 0x561aa0a283e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.974+0000 7fa3f9000640 1 -- 172.21.15.7:0/1509338617 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561aa09de050 con 0x561aa09e4820 2024-09-10T21:51:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.974+0000 7fa3fa400640 1 --2- 172.21.15.7:0/1509338617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa0a29130 0x561aa0a283e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:51:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.974+0000 7fa3f9a00640 1 --2- 172.21.15.7:0/1509338617 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa09e42a0 0x561aa09f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:51:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.974+0000 7fa3f9000640 1 --2- 172.21.15.7:0/1509338617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 0x561aa09d9080 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0022040 tx=0x7fa3e0007ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=559c89cc59c4d907 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.974+0000 7fa3e5600640 1 -- 172.21.15.7:0/1509338617 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e001b040 con 0x561aa09e4820 2024-09-10T21:51:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.974+0000 7fa3e5600640 1 -- 172.21.15.7:0/1509338617 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3e0025050 con 0x561aa09e4820 2024-09-10T21:51:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.974+0000 7fa3e5600640 1 -- 172.21.15.7:0/1509338617 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0014e60 con 0x561aa09e4820 2024-09-10T21:51:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.975+0000 7fa3ff482740 1 -- 172.21.15.7:0/1509338617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 msgr2=0x561aa09d9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.975+0000 7fa3ff482740 1 --2- 172.21.15.7:0/1509338617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 0x561aa09d9080 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0022040 tx=0x7fa3e0007ce0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.975+0000 7fa3ff482740 1 -- 172.21.15.7:0/1509338617 shutdown_connections 2024-09-10T21:51:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.975+0000 7fa3ff482740 1 --2- 172.21.15.7:0/1509338617 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa0a29130 0x561aa0a283e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.975+0000 7fa3ff482740 1 --2- 172.21.15.7:0/1509338617 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 0x561aa09d9080 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.975+0000 7fa3ff482740 1 --2- 172.21.15.7:0/1509338617 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa09e42a0 0x561aa09f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.975+0000 7fa3ff482740 1 -- 172.21.15.7:0/1509338617 >> 172.21.15.7:0/1509338617 conn(0x561aa09e3540 msgr2=0x561aa0a09d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.975+0000 7fa3ff482740 1 -- 172.21.15.7:0/1509338617 shutdown_connections 2024-09-10T21:51:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.975+0000 7fa3ff482740 1 -- 172.21.15.7:0/1509338617 wait complete. 2024-09-10T21:51:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.976+0000 7fa3ff482740 1 -- start start 2024-09-10T21:51:52.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.976+0000 7fa3ff482740 1 Processor -- start 2024-09-10T21:51:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.977+0000 7fa3ff482740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa09e42a0 0x561aa09f7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.977+0000 7fa3ff482740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 0x561aa09dccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.977+0000 7fa3ff482740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa0a29130 0x561aa0aa81d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.977+0000 7fa3ff482740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x561aa09f5190 con 0x561aa09e4820 2024-09-10T21:51:52.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.977+0000 7fa3ff482740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f40736c0 con 0x561aa0a29130 2024-09-10T21:51:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.977+0000 7fa3ff482740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x561aa0a3a410 con 0x561aa09e42a0 2024-09-10T21:51:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.977+0000 7fa3ff482740 10 client.0 _ll_register_callbacks cb 0x7fa3e800e9b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:51:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.977+0000 7fa3f9000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 0x561aa09dccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.977+0000 7fa3f9000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 0x561aa09dccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43952/0 (socket says 172.21.15.7:43952) 2024-09-10T21:51:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.977+0000 7fa3f9000640 1 -- 172.21.15.7:0/3052038867 learned_addr learned my addr 172.21.15.7:0/3052038867 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.977+0000 7fa3f9a00640 1 --2- 172.21.15.7:0/3052038867 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa09e42a0 0x561aa09f7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.977+0000 7fa3fa400640 1 --2- 172.21.15.7:0/3052038867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa0a29130 0x561aa0aa81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.977+0000 7fa3f9000640 1 -- 172.21.15.7:0/3052038867 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa09e42a0 msgr2=0x561aa09f7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.977+0000 7fa3f9000640 1 --2- 172.21.15.7:0/3052038867 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa09e42a0 0x561aa09f7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.977+0000 7fa3f9000640 1 -- 172.21.15.7:0/3052038867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa0a29130 msgr2=0x561aa0aa81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:52.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.977+0000 7fa3f9000640 1 --2- 172.21.15.7:0/3052038867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa0a29130 0x561aa0aa81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.977+0000 7fa3f9000640 1 -- 172.21.15.7:0/3052038867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561aa09d9220 con 0x561aa09e4820 2024-09-10T21:51:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.978+0000 7fa3f9000640 1 --2- 172.21.15.7:0/3052038867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 0x561aa09dccd0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000f9a0 tx=0x7fa3e00079a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ffdab2d6e13105 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.978+0000 7fa3daa00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:51:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.978+0000 7fa3daa00640 1 -- 172.21.15.7:0/3052038867 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e001b030 con 0x561aa09e4820 2024-09-10T21:51:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.978+0000 7fa3daa00640 1 -- 172.21.15.7:0/3052038867 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 5767+0+0 (secure 0 0 0) 0x7fa3e003daf0 con 0x561aa09e4820 2024-09-10T21:51:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.978+0000 7fa3ff482740 1 -- 172.21.15.7:0/3052038867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3e0025050 con 0x561aa09e4820 2024-09-10T21:51:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.978+0000 7fa3daa00640 1 -- 172.21.15.7:0/3052038867 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) ==== 24+0+0 (secure 0 0 0) 0x7fa3e0044050 con 0x561aa09e4820 2024-09-10T21:51:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.978+0000 7fa3ff482740 10 client.8280 fetch_fsmap learned FSMap version 102 2024-09-10T21:51:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.978+0000 7fa3ff482740 1 -- 172.21.15.7:0/3052038867 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) -- 0x561aa0aa8710 con 0x561aa09e4820 2024-09-10T21:51:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.978+0000 7fa3daa00640 1 -- 172.21.15.7:0/3052038867 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3e0025050 con 0x561aa09e4820 2024-09-10T21:51:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.979+0000 7fa3daa00640 1 -- 172.21.15.7:0/3052038867 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 102) ==== 3502+0+0 (secure 0 0 0) 0x7fa3e0047020 con 0x561aa09e4820 2024-09-10T21:51:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.979+0000 7fa3ff482740 10 client.8280 fetch_fsmap finished waiting for FSMap version 102 2024-09-10T21:51:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.979+0000 7fa3ff482740 10 client.8280 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.b 2024-09-10T21:51:52.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.979+0000 7fa3ff482740 20 client.8280 populate_metadata read hostname 'smithi007' 2024-09-10T21:51:52.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.979+0000 7fa3ff482740 1 --2- 172.21.15.7:0/3052038867 >> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] conn(0x561aa0a3b910 0x561aa0a07b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:52.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.979+0000 7fa3ff482740 4 client.8280 mds_command: new command op to 236726 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T21:51:52.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.979+0000 7fa3ff482740 1 -- 172.21.15.7:0/3052038867 --> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561aa0954270 con 0x561aa0a3b910 2024-09-10T21:51:52.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.979+0000 7fa3f9a00640 1 --2- 172.21.15.7:0/3052038867 >> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] conn(0x561aa0a3b910 0x561aa0a07b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:52.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.980+0000 7fa3f9a00640 1 --2- 172.21.15.7:0/3052038867 >> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] conn(0x561aa0a3b910 0x561aa0a07b80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=663854625f783b6c server_cookie=33f4e73fd7215118 in_seq=0 out_seq=0 2024-09-10T21:51:52.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.980+0000 7fa3daa00640 10 client.8280 ms_handle_connect on v2:172.21.15.133:6834/3764354952 2024-09-10T21:51:52.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.991+0000 7fa3daa00640 1 -- 172.21.15.7:0/3052038867 <== mds.0 v2:172.21.15.133:6834/3764354952 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561aa0954270 con 0x561aa0a3b910 2024-09-10T21:51:52.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.991+0000 7fa3daa00640 10 client.8280 handle_command_reply: tid=0 2024-09-10T21:51:52.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.991+0000 7fa3ff482740 1 client.8280 shutdown 2024-09-10T21:51:52.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.991+0000 7fa3ff482740 1 -- 172.21.15.7:0/3052038867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 msgr2=0x561aa09dccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:52.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.991+0000 7fa3ff482740 1 --2- 172.21.15.7:0/3052038867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 0x561aa09dccd0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000f9a0 tx=0x7fa3e00079a0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.991+0000 7fa3ff482740 1 -- 172.21.15.7:0/3052038867 shutdown_connections 2024-09-10T21:51:52.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.991+0000 7fa3ff482740 1 --2- 172.21.15.7:0/3052038867 >> [v2:172.21.15.133:6834/3764354952,v1:172.21.15.133:6837/3764354952] conn(0x561aa0a3b910 0x561aa0a07b80 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.991+0000 7fa3ff482740 1 --2- 172.21.15.7:0/3052038867 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x561aa0a29130 0x561aa0aa81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.991+0000 7fa3ff482740 1 --2- 172.21.15.7:0/3052038867 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x561aa09e4820 0x561aa09dccd0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.991+0000 7fa3daa00640 0 client.8280 ms_handle_reset on v2:172.21.15.133:6834/3764354952 2024-09-10T21:51:52.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.991+0000 7fa3ff482740 1 --2- 172.21.15.7:0/3052038867 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x561aa09e42a0 0x561aa09f7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.991+0000 7fa3ff482740 1 -- 172.21.15.7:0/3052038867 >> 172.21.15.7:0/3052038867 conn(0x561aa09e3540 msgr2=0x561aa0a4be50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:52.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.991+0000 7fa3ff482740 1 -- 172.21.15.7:0/3052038867 shutdown_connections 2024-09-10T21:51:52.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.991+0000 7fa3ff482740 1 -- 172.21.15.7:0/3052038867 wait complete. 2024-09-10T21:51:52.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.991+0000 7fa3ff482740 20 client.8280 trim_cache size 0 max 16384 2024-09-10T21:51:52.993 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:51:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "", 2024-09-10T21:51:52.993 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0 2024-09-10T21:51:52.994 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:51:52.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.992+0000 7fa3fc400640 1 -- 172.21.15.7:0/210915264 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa3d4077f70 msgr2=0x7fa3d407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:52.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.992+0000 7fa3fc400640 1 --2- 172.21.15.7:0/210915264 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa3d4077f70 0x7fa3d407a430 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0007fd0 tx=0x7fa3e0001370 comp rx=0 tx=0).stop 2024-09-10T21:51:52.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.992+0000 7fa3fc400640 1 -- 172.21.15.7:0/210915264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3f40ae000 msgr2=0x7fa3f40c4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:52.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.992+0000 7fa3fc400640 1 --2- 172.21.15.7:0/210915264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3f40ae000 0x7fa3f40c4210 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800ea60 tx=0x7fa3e800ef20 comp rx=0 tx=0).stop 2024-09-10T21:51:52.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.992+0000 7fa3fc400640 1 -- 172.21.15.7:0/210915264 shutdown_connections 2024-09-10T21:51:52.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.992+0000 7fa3fc400640 1 --2- 172.21.15.7:0/210915264 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa3d4077f70 0x7fa3d407a430 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.992+0000 7fa3fc400640 1 --2- 172.21.15.7:0/210915264 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa3f4158220 0x7fa3f41f7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.992+0000 7fa3fc400640 1 --2- 172.21.15.7:0/210915264 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa3f4158600 0x7fa3f41f8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.992+0000 7fa3fc400640 1 --2- 172.21.15.7:0/210915264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa3f40ae000 0x7fa3f40c4210 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:52.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.992+0000 7fa3fc400640 1 -- 172.21.15.7:0/210915264 >> 172.21.15.7:0/210915264 conn(0x7fa3f40b16d0 msgr2=0x7fa3f415fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:52.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.992+0000 7fa3fc400640 1 -- 172.21.15.7:0/210915264 shutdown_connections 2024-09-10T21:51:52.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:52.992+0000 7fa3fc400640 1 -- 172.21.15.7:0/210915264 wait complete. 2024-09-10T21:51:53.008 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:51:53.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.168+0000 7f0feda00640 1 -- 172.21.15.7:0/230204726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fcc004d90 msgr2=0x7f0fcc005210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:53.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.168+0000 7f0feda00640 1 --2- 172.21.15.7:0/230204726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fcc004d90 0x7f0fcc005210 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0001f00 tx=0x7f0fcc005830 comp rx=0 tx=0).stop 2024-09-10T21:51:53.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.168+0000 7f0fe7400640 1 -- 172.21.15.7:0/230204726 reap_dead start 2024-09-10T21:51:53.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.168+0000 7f0feda00640 1 -- 172.21.15.7:0/230204726 shutdown_connections 2024-09-10T21:51:53.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.168+0000 7f0feda00640 1 -- 172.21.15.7:0/230204726 >> 172.21.15.7:0/230204726 conn(0x7f0fe80b16d0 msgr2=0x7f0fe80b1ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:53.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.168+0000 7f0feda00640 1 -- 172.21.15.7:0/230204726 shutdown_connections 2024-09-10T21:51:53.169 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.168+0000 7f0feda00640 1 -- 172.21.15.7:0/230204726 wait complete. 2024-09-10T21:51:53.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.169+0000 7f0feda00640 1 Processor -- start 2024-09-10T21:51:53.170 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.170+0000 7f0feda00640 1 -- start start 2024-09-10T21:51:53.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.170+0000 7f0feda00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fcc004d90 0x7f0fe80affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:53.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.170+0000 7f0feda00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fe81f50d0 0x7f0fe815dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:53.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.170+0000 7f0feda00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe815e130 0x7f0fe80aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:53.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.171+0000 7f0feda00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe8070610 con 0x7f0fe815e130 2024-09-10T21:51:53.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.171+0000 7f0feda00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f0fe80cf570 con 0x7f0fe81f50d0 2024-09-10T21:51:53.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.171+0000 7f0feda00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f0fe81f4630 con 0x7f0fcc004d90 2024-09-10T21:51:53.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.171+0000 7f0fe7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe815e130 0x7f0fe80aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:53.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.171+0000 7f0fe7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe815e130 0x7f0fe80aed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:43974/0 (socket says 172.21.15.7:43974) 2024-09-10T21:51:53.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.171+0000 7f0fe7400640 1 -- 172.21.15.7:0/3016020574 learned_addr learned my addr 172.21.15.7:0/3016020574 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:53.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.171+0000 7f0fe6a00640 1 --2- 172.21.15.7:0/3016020574 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fcc004d90 0x7f0fe80affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:53.173 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.171+0000 7f0fe6000640 1 --2- 172.21.15.7:0/3016020574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fe81f50d0 0x7f0fe815dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:53.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.171+0000 7f0fe7400640 1 -- 172.21.15.7:0/3016020574 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fcc004d90 msgr2=0x7f0fe80affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:53.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.171+0000 7f0fe7400640 1 --2- 172.21.15.7:0/3016020574 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fcc004d90 0x7f0fe80affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:53.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.171+0000 7f0fe7400640 1 -- 172.21.15.7:0/3016020574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fe81f50d0 msgr2=0x7f0fe815dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:53.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.171+0000 7f0fe7400640 1 --2- 172.21.15.7:0/3016020574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fe81f50d0 0x7f0fe815dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:53.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.171+0000 7f0fe7400640 1 -- 172.21.15.7:0/3016020574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fe815e600 con 0x7f0fe815e130 2024-09-10T21:51:53.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.171+0000 7f0fe6000640 1 --2- 172.21.15.7:0/3016020574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fe81f50d0 0x7f0fe815dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:51:53.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.172+0000 7f0fe7400640 1 --2- 172.21.15.7:0/3016020574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe815e130 0x7f0fe80aed40 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0002020 tx=0x7f0fe0001c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:53.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.172+0000 7f0fdea00640 1 -- 172.21.15.7:0/3016020574 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0037030 con 0x7f0fe815e130 2024-09-10T21:51:53.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.172+0000 7f0fdea00640 1 -- 172.21.15.7:0/3016020574 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0fe0003d40 con 0x7f0fe815e130 2024-09-10T21:51:53.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.172+0000 7f0fdea00640 1 -- 172.21.15.7:0/3016020574 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0006c40 con 0x7f0fe815e130 2024-09-10T21:51:53.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.172+0000 7f0feda00640 1 -- 172.21.15.7:0/3016020574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fe81f48d0 con 0x7f0fe815e130 2024-09-10T21:51:53.174 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.172+0000 7f0feda00640 1 -- 172.21.15.7:0/3016020574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fe81f87a0 con 0x7f0fe815e130 2024-09-10T21:51:53.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.173+0000 7f0feda00640 1 -- 172.21.15.7:0/3016020574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fe80cd280 con 0x7f0fe815e130 2024-09-10T21:51:53.175 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.174+0000 7f0fdea00640 1 -- 172.21.15.7:0/3016020574 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f0fe0006500 con 0x7f0fe815e130 2024-09-10T21:51:53.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.175+0000 7f0fdea00640 1 --2- 172.21.15.7:0/3016020574 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0fc8077f70 0x7f0fc807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:53.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.175+0000 7f0fdea00640 1 -- 172.21.15.7:0/3016020574 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5767+0+0 (secure 0 0 0) 0x7f0fe00b9820 con 0x7f0fe815e130 2024-09-10T21:51:53.176 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.176+0000 7f0fe6a00640 1 --2- 172.21.15.7:0/3016020574 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0fc8077f70 0x7f0fc807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:53.177 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.176+0000 7f0fe6a00640 1 --2- 172.21.15.7:0/3016020574 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0fc8077f70 0x7f0fc807a430 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80afd00 tx=0x7f0fd8001020 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:53.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.180+0000 7f0fdea00640 1 -- 172.21.15.7:0/3016020574 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f0fe0006820 con 0x7f0fe815e130 2024-09-10T21:51:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.415+0000 7f0feda00640 1 -- 172.21.15.7:0/3016020574 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f0fe80b1d00 con 0x7f0fe815e130 2024-09-10T21:51:53.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.416+0000 7f0fdea00640 1 -- 172.21.15.7:0/3016020574 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v102) ==== 53+0+73 (secure 0 0 0) 0x7f0fe0081c80 con 0x7f0fe815e130 2024-09-10T21:51:53.417 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:51:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.418+0000 7f0feda00640 1 -- 172.21.15.7:0/3016020574 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0fc8077f70 msgr2=0x7f0fc807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.418+0000 7f0feda00640 1 --2- 172.21.15.7:0/3016020574 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0fc8077f70 0x7f0fc807a430 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80afd00 tx=0x7f0fd8001020 comp rx=0 tx=0).stop 2024-09-10T21:51:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.418+0000 7f0feda00640 1 -- 172.21.15.7:0/3016020574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe815e130 msgr2=0x7f0fe80aed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.418+0000 7f0feda00640 1 --2- 172.21.15.7:0/3016020574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe815e130 0x7f0fe80aed40 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0002020 tx=0x7f0fe0001c00 comp rx=0 tx=0).stop 2024-09-10T21:51:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.418+0000 7f0feda00640 1 -- 172.21.15.7:0/3016020574 shutdown_connections 2024-09-10T21:51:53.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.418+0000 7f0feda00640 1 --2- 172.21.15.7:0/3016020574 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0fc8077f70 0x7f0fc807a430 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:53.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.418+0000 7f0feda00640 1 --2- 172.21.15.7:0/3016020574 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f0fe81f50d0 0x7f0fe815dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:53.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.418+0000 7f0feda00640 1 --2- 172.21.15.7:0/3016020574 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f0fe815e130 0x7f0fe80aed40 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:53.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.418+0000 7f0feda00640 1 --2- 172.21.15.7:0/3016020574 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f0fcc004d90 0x7f0fe80affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:53.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.418+0000 7f0feda00640 1 -- 172.21.15.7:0/3016020574 >> 172.21.15.7:0/3016020574 conn(0x7f0fe80b16d0 msgr2=0x7f0fe80b2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:53.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.418+0000 7f0feda00640 1 -- 172.21.15.7:0/3016020574 shutdown_connections 2024-09-10T21:51:53.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53.418+0000 7f0feda00640 1 -- 172.21.15.7:0/3016020574 wait complete. 2024-09-10T21:51:53.435 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:51:53.435 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:51:53.435 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-10T21:51:53.435 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:51:53.435 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:51:53.435 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:51:53.435 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:51:53.435 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:51:53.435 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:51:53.435 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:53.435 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-10T21:51:53.471 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:51:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:51:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:51:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:51:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T21:51:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:51:53.472 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:51:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T21:51:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-10T21:51:53.472 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:51:53.472 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T21:51:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-10T21:51:53.473 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:51:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-10T21:51:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-10T21:51:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T21:51:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42032sec preferred_lft 42032sec 2024-09-10T21:51:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-10T21:51:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:51:53.473 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:51:53.473 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-10T21:51:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-10T21:51:53.474 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:51:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff 2024-09-10T21:51:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:51:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:51:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::3c3d:dbff:fe02:4c15/64 scope link 2024-09-10T21:51:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:51:53.474 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:51:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether f6:f9:69:5a:86:f5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:51:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f4f9:69ff:fe5a:86f5/64 scope link 2024-09-10T21:51:53.474 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:51:53.475 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:51:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:51:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::433:5cff:fe44:4a5b/64 scope link 2024-09-10T21:51:53.475 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:51:53.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:53.475 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:51:53.567 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:51:53.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:53.570 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-10T21:51:53.606 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-10T21:51:53.606 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:51:53.606 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T21:51:53.606 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T21:51:53.606 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:51:53.606 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:51:53.606 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:51:53.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:51:53.607 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:51:53.607 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:51:53.607 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:51:53.724 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:51:53.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:53 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:51:53.804 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:53.804 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:51:53.839 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:51:53.839 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:51:53.839 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:51:53.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:53.839 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-10T21:51:53.936 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:51:53.936 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:51:53.971 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:51:53.971 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:51:53.971 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:51:54.046 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:51:54.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:54.046 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:51:54.175 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:51:54.176 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:51:54.205 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:51:54.205 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:51:54.205 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.133:3300,172.21.15.133:3301" 2024-09-10T21:51:54.205 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@2029451a-3c26-48c7-b874-0597197278ac.cephfs=/ 2024-09-10T21:51:54.205 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.133:3300/172.21.15.133:3301" will pass to kernel 2024-09-10T21:51:54.207 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T21:51:54.207 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:51:54.207 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:51:54.207 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:51:54.207 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:51:54.210 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:51:54.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:54.210 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:51:54.266 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:51:54.344 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:54 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:51:54.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:51:54.356 INFO:teuthology.orchestra.run.smithi007.stdout:client7642 2024-09-10T21:51:54.356 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:51:54.356 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:51:54.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:54.392 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/unflushed) 2024-09-10T21:51:54.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:54.473 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=./parent/unflushed/jfile bs=1M count=8) 2024-09-10T21:51:54.597 INFO:teuthology.orchestra.run.smithi007.stderr:8+0 records in 2024-09-10T21:51:54.597 INFO:teuthology.orchestra.run.smithi007.stderr:8+0 records out 2024-09-10T21:51:54.597 INFO:teuthology.orchestra.run.smithi007.stderr:8388608 bytes (8.4 MB, 8.0 MiB) copied, 0.0458076 s, 183 MB/s 2024-09-10T21:51:54.598 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:51:54.598 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:51:54.598 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:51:54.598 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:51:54.598 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-09-10T21:51:54.598 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:51:54.699 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627779 2024-09-10T21:51:54.702 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:51:54.710 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:51:54.710 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:51:54.710 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:51:54.711 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:51:54.711 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-09-10T21:51:54.711 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:51:54.851 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627780 2024-09-10T21:51:54.855 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:51:54.863 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:51:54.863 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:51:54.939 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:51:54.939 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:51:55.014 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:51:55.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.216+0000 7f7b20a00640 1 -- 172.21.15.7:0/4010560264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b00004db0 msgr2=0x7f7b00005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:55.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.216+0000 7f7b20a00640 1 --2- 172.21.15.7:0/4010560264 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b00004db0 0x7f7b00005230 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f7b1400c590 tx=0x7f7b00005830 comp rx=0 tx=0).stop 2024-09-10T21:51:55.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.216+0000 7f7b1a800640 1 -- 172.21.15.7:0/4010560264 reap_dead start 2024-09-10T21:51:55.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.216+0000 7f7b20a00640 1 -- 172.21.15.7:0/4010560264 shutdown_connections 2024-09-10T21:51:55.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.216+0000 7f7b20a00640 1 -- 172.21.15.7:0/4010560264 >> 172.21.15.7:0/4010560264 conn(0x7f7b1c0b1660 msgr2=0x7f7b1c0b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:55.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.216+0000 7f7b20a00640 1 -- 172.21.15.7:0/4010560264 shutdown_connections 2024-09-10T21:51:55.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.216+0000 7f7b20a00640 1 -- 172.21.15.7:0/4010560264 wait complete. 2024-09-10T21:51:55.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.218+0000 7f7b20a00640 1 Processor -- start 2024-09-10T21:51:55.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.218+0000 7f7b20a00640 1 -- start start 2024-09-10T21:51:55.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.219+0000 7f7b20a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b00004db0 0x7f7b1c0b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:55.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.219+0000 7f7b20a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b1c148650 0x7f7b1c1f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:55.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.219+0000 7f7b20a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b1c148ad0 0x7f7b1c1f8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:55.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.219+0000 7f7b20a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c06f8a0 con 0x7f7b1c148650 2024-09-10T21:51:55.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.219+0000 7f7b20a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1c06f720 con 0x7f7b1c148ad0 2024-09-10T21:51:55.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.219+0000 7f7b20a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f7b1c073cd0 con 0x7f7b00004db0 2024-09-10T21:51:55.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.219+0000 7f7b11400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b1c148650 0x7f7b1c1f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:55.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.219+0000 7f7b11400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b1c148650 0x7f7b1c1f7f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53476/0 (socket says 172.21.15.7:53476) 2024-09-10T21:51:55.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.219+0000 7f7b11400640 1 -- 172.21.15.7:0/2042486462 learned_addr learned my addr 172.21.15.7:0/2042486462 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:55.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.219+0000 7f7b1a800640 1 --2- 172.21.15.7:0/2042486462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b1c148ad0 0x7f7b1c1f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:55.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.219+0000 7f7b19e00640 1 --2- 172.21.15.7:0/2042486462 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b00004db0 0x7f7b1c0b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:55.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.220+0000 7f7b11400640 1 -- 172.21.15.7:0/2042486462 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b00004db0 msgr2=0x7f7b1c0b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:55.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.220+0000 7f7b11400640 1 --2- 172.21.15.7:0/2042486462 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b00004db0 0x7f7b1c0b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:55.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.220+0000 7f7b11400640 1 -- 172.21.15.7:0/2042486462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b1c148ad0 msgr2=0x7f7b1c1f8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:55.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.220+0000 7f7b11400640 1 --2- 172.21.15.7:0/2042486462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b1c148ad0 0x7f7b1c1f8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:55.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.220+0000 7f7b11400640 1 -- 172.21.15.7:0/2042486462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b1c1f5560 con 0x7f7b1c148650 2024-09-10T21:51:55.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.220+0000 7f7b1a800640 1 --2- 172.21.15.7:0/2042486462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b1c148ad0 0x7f7b1c1f8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:51:55.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.220+0000 7f7b11400640 1 --2- 172.21.15.7:0/2042486462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b1c148650 0x7f7b1c1f7f00 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f7b04011ae0 tx=0x7f7b04009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:55.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.220+0000 7f7b12a00640 1 -- 172.21.15.7:0/2042486462 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0401e070 con 0x7f7b1c148650 2024-09-10T21:51:55.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.220+0000 7f7b12a00640 1 -- 172.21.15.7:0/2042486462 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b0400fe70 con 0x7f7b1c148650 2024-09-10T21:51:55.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.220+0000 7f7b12a00640 1 -- 172.21.15.7:0/2042486462 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b04004290 con 0x7f7b1c148650 2024-09-10T21:51:55.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.220+0000 7f7b20a00640 1 -- 172.21.15.7:0/2042486462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b1c1f5840 con 0x7f7b1c148650 2024-09-10T21:51:55.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.220+0000 7f7b20a00640 1 -- 172.21.15.7:0/2042486462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b1c0c52a0 con 0x7f7b1c148650 2024-09-10T21:51:55.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.221+0000 7f7b20a00640 1 -- 172.21.15.7:0/2042486462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b1c06e510 con 0x7f7b1c148650 2024-09-10T21:51:55.223 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.222+0000 7f7b12a00640 1 -- 172.21.15.7:0/2042486462 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7b040035f0 con 0x7f7b1c148650 2024-09-10T21:51:55.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.223+0000 7f7b12a00640 1 --2- 172.21.15.7:0/2042486462 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7af8077f70 0x7f7af807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:55.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.223+0000 7f7b12a00640 1 -- 172.21.15.7:0/2042486462 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5767+0+0 (secure 0 0 0) 0x7f7b0409aaf0 con 0x7f7b1c148650 2024-09-10T21:51:55.224 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.224+0000 7f7b19e00640 1 --2- 172.21.15.7:0/2042486462 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7af8077f70 0x7f7af807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:55.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.225+0000 7f7b19e00640 1 --2- 172.21.15.7:0/2042486462 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7af8077f70 0x7f7af807a430 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c1560a0 tx=0x7f7b0c0069e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:55.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.228+0000 7f7b12a00640 1 -- 172.21.15.7:0/2042486462 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7b04063010 con 0x7f7b1c148650 2024-09-10T21:51:55.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.434+0000 7f7b20a00640 1 -- 172.21.15.7:0/2042486462 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7b1c0b12a0 con 0x7f7b1c148650 2024-09-10T21:51:55.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.435+0000 7f7b12a00640 1 -- 172.21.15.7:0/2042486462 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v116) ==== 80+0+0 (secure 0 0 0) 0x7f7b04067a40 con 0x7f7b1c148650 2024-09-10T21:51:55.436 INFO:teuthology.orchestra.run.smithi007.stderr:listed 0 entries 2024-09-10T21:51:55.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.437+0000 7f7b20a00640 1 -- 172.21.15.7:0/2042486462 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7af8077f70 msgr2=0x7f7af807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:55.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.437+0000 7f7b20a00640 1 --2- 172.21.15.7:0/2042486462 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7af8077f70 0x7f7af807a430 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c1560a0 tx=0x7f7b0c0069e0 comp rx=0 tx=0).stop 2024-09-10T21:51:55.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.438+0000 7f7b20a00640 1 -- 172.21.15.7:0/2042486462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b1c148650 msgr2=0x7f7b1c1f7f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:55.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.438+0000 7f7b20a00640 1 --2- 172.21.15.7:0/2042486462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b1c148650 0x7f7b1c1f7f00 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f7b04011ae0 tx=0x7f7b04009e90 comp rx=0 tx=0).stop 2024-09-10T21:51:55.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.438+0000 7f7b20a00640 1 -- 172.21.15.7:0/2042486462 shutdown_connections 2024-09-10T21:51:55.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.438+0000 7f7b20a00640 1 --2- 172.21.15.7:0/2042486462 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7af8077f70 0x7f7af807a430 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:55.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.438+0000 7f7b20a00640 1 --2- 172.21.15.7:0/2042486462 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f7b1c148ad0 0x7f7b1c1f8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:55.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.438+0000 7f7b20a00640 1 --2- 172.21.15.7:0/2042486462 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f7b1c148650 0x7f7b1c1f7f00 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:55.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.438+0000 7f7b20a00640 1 --2- 172.21.15.7:0/2042486462 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f7b00004db0 0x7f7b1c0b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:55.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.438+0000 7f7b20a00640 1 -- 172.21.15.7:0/2042486462 >> 172.21.15.7:0/2042486462 conn(0x7f7b1c0b1660 msgr2=0x7f7b1c0afed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:55.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.438+0000 7f7b20a00640 1 -- 172.21.15.7:0/2042486462 shutdown_connections 2024-09-10T21:51:55.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.438+0000 7f7b20a00640 1 -- 172.21.15.7:0/2042486462 wait complete. 2024-09-10T21:51:55.455 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:51:55.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:55.455 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:51:55.598 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:51:55.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:55.598 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:51:55.648 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:51:55.648 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:51:55.683 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:51:55.683 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:51:55.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:51:55.684 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:51:55.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:51:55.759 INFO:teuthology.orchestra.run.smithi007.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T21:51:55.760 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:51:55.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.959+0000 7fcc16400640 1 -- 172.21.15.7:0/2030407931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc100b2b50 msgr2=0x7fcc100d1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:55.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.959+0000 7fcc16400640 1 --2- 172.21.15.7:0/2030407931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc100b2b50 0x7fcc100d1ee0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7fcc100cadb0 tx=0x7fcc0802a820 comp rx=0 tx=0).stop 2024-09-10T21:51:55.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.960+0000 7fcc16400640 1 -- 172.21.15.7:0/2030407931 shutdown_connections 2024-09-10T21:51:55.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.960+0000 7fcc16400640 1 --2- 172.21.15.7:0/2030407931 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc100b2b50 0x7fcc100d1ee0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:55.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.960+0000 7fcc16400640 1 --2- 172.21.15.7:0/2030407931 >> v2:172.21.15.133:3300/0 conn(0x7fcc100d0e70 0x7fcc100d4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:55.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.960+0000 7fcc16400640 1 -- 172.21.15.7:0/2030407931 >> 172.21.15.7:0/2030407931 conn(0x7fcc100d3d50 msgr2=0x7fcc100d4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:55.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.960+0000 7fcc16400640 1 -- 172.21.15.7:0/2030407931 shutdown_connections 2024-09-10T21:51:55.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.960+0000 7fcc16400640 1 -- 172.21.15.7:0/2030407931 wait complete. 2024-09-10T21:51:55.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.961+0000 7fcc16400640 1 Processor -- start 2024-09-10T21:51:55.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.961+0000 7fcc16400640 1 -- start start 2024-09-10T21:51:55.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.962+0000 7fcc16400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc100b2b50 0x7fcc100b4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:55.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.962+0000 7fcc16400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc100d0e70 0x7fcc100a0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:55.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.962+0000 7fcc16400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1015dfd0 0x7fcc100a14e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:55.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.962+0000 7fcc16400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fcc100d25a0 con 0x7fcc1015dfd0 2024-09-10T21:51:55.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.962+0000 7fcc16400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fcc100d2420 con 0x7fcc100b2b50 2024-09-10T21:51:55.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.962+0000 7fcc16400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fcc100d2720 con 0x7fcc100d0e70 2024-09-10T21:51:55.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.962+0000 7fcc15a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1015dfd0 0x7fcc100a14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:55.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.963+0000 7fcc15a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1015dfd0 0x7fcc100a14e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53494/0 (socket says 172.21.15.7:53494) 2024-09-10T21:51:55.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.963+0000 7fcc15a00640 1 -- 172.21.15.7:0/459817495 learned_addr learned my addr 172.21.15.7:0/459817495 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:55.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.963+0000 7fcc15000640 1 --2- 172.21.15.7:0/459817495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc100b2b50 0x7fcc100b4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:55.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.963+0000 7fcc0fe00640 1 --2- 172.21.15.7:0/459817495 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc100d0e70 0x7fcc100a0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:55.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.963+0000 7fcc15a00640 1 -- 172.21.15.7:0/459817495 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc100d0e70 msgr2=0x7fcc100a0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:55.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.963+0000 7fcc15a00640 1 --2- 172.21.15.7:0/459817495 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc100d0e70 0x7fcc100a0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:55.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.963+0000 7fcc15a00640 1 -- 172.21.15.7:0/459817495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc100b2b50 msgr2=0x7fcc100b4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:55.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.963+0000 7fcc15a00640 1 --2- 172.21.15.7:0/459817495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc100b2b50 0x7fcc100b4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:55.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.963+0000 7fcc15a00640 1 -- 172.21.15.7:0/459817495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc101f7be0 con 0x7fcc1015dfd0 2024-09-10T21:51:55.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.963+0000 7fcc15a00640 1 --2- 172.21.15.7:0/459817495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1015dfd0 0x7fcc100a14e0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009b30 tx=0x7fcc08032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:55.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.964+0000 7fcc0cc00640 1 -- 172.21.15.7:0/459817495 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc08037040 con 0x7fcc1015dfd0 2024-09-10T21:51:55.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.964+0000 7fcc0cc00640 1 -- 172.21.15.7:0/459817495 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc08042070 con 0x7fcc1015dfd0 2024-09-10T21:51:55.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.964+0000 7fcc16400640 1 -- 172.21.15.7:0/459817495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc101f7ec0 con 0x7fcc1015dfd0 2024-09-10T21:51:55.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.964+0000 7fcc0cc00640 1 -- 172.21.15.7:0/459817495 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc080042a0 con 0x7fcc1015dfd0 2024-09-10T21:51:55.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.964+0000 7fcc16400640 1 -- 172.21.15.7:0/459817495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc101f8760 con 0x7fcc1015dfd0 2024-09-10T21:51:55.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.965+0000 7fcc16400640 1 -- 172.21.15.7:0/459817495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc101f82b0 con 0x7fcc1015dfd0 2024-09-10T21:51:55.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.966+0000 7fcc0cc00640 1 -- 172.21.15.7:0/459817495 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fcc0803a030 con 0x7fcc1015dfd0 2024-09-10T21:51:55.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.967+0000 7fcc0cc00640 1 --2- 172.21.15.7:0/459817495 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcbf0077fa0 0x7fcbf007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:55.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.968+0000 7fcc0cc00640 1 -- 172.21.15.7:0/459817495 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 5767+0+0 (secure 0 0 0) 0x7fcc080ba5e0 con 0x7fcc1015dfd0 2024-09-10T21:51:55.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.968+0000 7fcc15000640 1 --2- 172.21.15.7:0/459817495 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcbf0077fa0 0x7fcbf007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:55.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.969+0000 7fcc15000640 1 --2- 172.21.15.7:0/459817495 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcbf0077fa0 0x7fcbf007a460 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcc04007fd0 tx=0x7fcc04001370 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:55.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:55.972+0000 7fcc0cc00640 1 -- 172.21.15.7:0/459817495 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fcc08082a40 con 0x7fcc1015dfd0 2024-09-10T21:51:56.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.205+0000 7fcc16400640 1 -- 172.21.15.7:0/459817495 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcc100b9cd0 con 0x7fcc1015dfd0 2024-09-10T21:51:56.206 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-10T21:51:56.206+0000 7feb3fa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:51:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.208+0000 7fcc0cc00640 1 -- 172.21.15.7:0/459817495 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v103) ==== 161+0+0 (secure 0 0 0) 0x7fcc08087470 con 0x7fcc1015dfd0 2024-09-10T21:51:56.209 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:51:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.211+0000 7fcc16400640 1 -- 172.21.15.7:0/459817495 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcbf0077fa0 msgr2=0x7fcbf007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.211+0000 7fcc16400640 1 --2- 172.21.15.7:0/459817495 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcbf0077fa0 0x7fcbf007a460 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fcc04007fd0 tx=0x7fcc04001370 comp rx=0 tx=0).stop 2024-09-10T21:51:56.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.211+0000 7fcc16400640 1 -- 172.21.15.7:0/459817495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1015dfd0 msgr2=0x7fcc100a14e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.211+0000 7fcc16400640 1 --2- 172.21.15.7:0/459817495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1015dfd0 0x7fcc100a14e0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fcc08009b30 tx=0x7fcc08032c90 comp rx=0 tx=0).stop 2024-09-10T21:51:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.211+0000 7fcc16400640 1 -- 172.21.15.7:0/459817495 shutdown_connections 2024-09-10T21:51:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.211+0000 7fcc16400640 1 --2- 172.21.15.7:0/459817495 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcbf0077fa0 0x7fcbf007a460 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.211+0000 7fcc16400640 1 --2- 172.21.15.7:0/459817495 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fcc1015dfd0 0x7fcc100a14e0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.211+0000 7fcc16400640 1 --2- 172.21.15.7:0/459817495 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fcc100d0e70 0x7fcc100a0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.211+0000 7fcc16400640 1 --2- 172.21.15.7:0/459817495 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fcc100b2b50 0x7fcc100b4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.211+0000 7fcc16400640 1 -- 172.21.15.7:0/459817495 >> 172.21.15.7:0/459817495 conn(0x7fcc100d3d50 msgr2=0x7fcc1006d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.211+0000 7fcc16400640 1 -- 172.21.15.7:0/459817495 shutdown_connections 2024-09-10T21:51:56.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.211+0000 7fcc16400640 1 -- 172.21.15.7:0/459817495 wait complete. 2024-09-10T21:51:56.223 INFO:tasks.ceph.mds.b.smithi133.stderr: -9719> 2024-09-10T21:51:44.455+0000 7f1d73400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:51:56.229 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T21:51:56.234 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T21:51:56.280 INFO:tasks.ceph.mds.b.smithi133.stdout:starting mds.b at 2024-09-10T21:51:56.329 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-09-10T21:51:56.329 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-09-10T21:51:56.329 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T21:51:56.329 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-09-10T21:51:56.329 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.329 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-09-10T21:51:56.329 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.329 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-09-10T21:51:56.329 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-09-10T21:51:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-09-10T21:51:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-09-10T21:51:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-09-10T21:51:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T21:51:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-09-10T21:51:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T21:51:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T21:51:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-09-10T21:51:56.330 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-09-10T21:51:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-09-10T21:51:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-09-10T21:51:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-09-10T21:51:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-09-10T21:51:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-09-10T21:51:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-09-10T21:51:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T21:51:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T21:51:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-09-10T21:51:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T21:51:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.331 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-09-10T21:51:56.332 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.332 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-09-10T21:51:56.332 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.332 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-09-10T21:51:56.332 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-09-10T21:51:56.332 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-09-10T21:51:56.332 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2024-09-10T21:51:56.332 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on skipped message = False 2024-09-10T21:51:56.332 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 2029451a-3c26-48c7-b874-0597197278ac 2024-09-10T21:51:56.332 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.332 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-09-10T21:51:56.332 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-09-10T21:51:56.332 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.333 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-09-10T21:51:56.333 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2024-09-10T21:51:56.333 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-09-10T21:51:56.333 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.333 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-09-10T21:51:56.333 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-09-10T21:51:56.333 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.333 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-09-10T21:51:56.333 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-09-10T21:51:56.333 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-09-10T21:51:56.333 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug trim objects = true 2024-09-10T21:51:56.333 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.333 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-09-10T21:51:56.333 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-09-10T21:51:56.334 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.334 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-09-10T21:51:56.334 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.334 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-09-10T21:51:56.334 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.334 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-09-10T21:51:56.334 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.334 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-09-10T21:51:56.334 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-09-10T21:51:56.334 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.334 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-09-10T21:51:56.334 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-09-10T21:51:56.334 INFO:teuthology.orchestra.run.smithi007.stdout: bdev async discard = True 2024-09-10T21:51:56.335 INFO:teuthology.orchestra.run.smithi007.stdout: bdev enable discard = True 2024-09-10T21:51:56.335 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore allocator = bitmap 2024-09-10T21:51:56.335 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-09-10T21:51:56.335 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-09-10T21:51:56.335 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 1/20 2024-09-10T21:51:56.335 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 1/20 2024-09-10T21:51:56.335 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-10T21:51:56.335 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-09-10T21:51:56.335 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 4/10 2024-09-10T21:51:56.335 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T21:51:56.335 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-09-10T21:51:56.335 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-09-10T21:51:56.335 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2024-09-10T21:51:56.335 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2024-09-10T21:51:56.335 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.336 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-09-10T21:51:56.336 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-10T21:51:56.336 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-09-10T21:51:56.336 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-10T21:51:56.336 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-10T21:51:56.336 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:51:56.336 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:51:56.336 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-09-10T21:51:56.336 INFO:teuthology.orchestra.run.smithi007.stdout: client mount timeout = 30 2024-09-10T21:51:56.336 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-09-10T21:51:56.336 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.336 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-09-10T21:51:56.336 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-10T21:51:56.336 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-10T21:51:56.337 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-09-10T21:51:56.337 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-10T21:51:56.337 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-09-10T21:51:56.337 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-09-10T21:51:56.337 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:51:56.337 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-09-10T21:51:56.337 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-09-10T21:51:56.337 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:51:56.337 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.337 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T21:51:56.337 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T21:51:56.337 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T21:51:56.337 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.338 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T21:51:56.338 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T21:51:56.338 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T21:51:56.338 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.338 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-09-10T21:51:56.338 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-09-10T21:51:56.338 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.338 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-09-10T21:51:56.338 INFO:teuthology.orchestra.run.smithi007.stdout: mon down mkfs grace = 300 2024-09-10T21:51:56.338 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.338 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-09-10T21:51:56.338 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-09-10T21:51:56.338 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-09-10T21:51:56.338 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-09-10T21:51:56.338 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T21:51:56.339 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T21:51:56.339 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = "" 2024-09-10T21:51:56.339 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-09-10T21:51:56.339 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-09-10T21:51:56.339 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-09-10T21:51:56.339 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-09-10T21:51:56.339 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-09-10T21:51:56.339 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-09-10T21:51:56.339 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-09-10T21:51:56.339 INFO:teuthology.orchestra.run.smithi007.stdout:[mds] 2024-09-10T21:51:56.339 INFO:teuthology.orchestra.run.smithi007.stdout: mds verify scatter = False 2024-09-10T21:51:56.340 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2024-09-10T21:51:56.340 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2024-09-10T21:51:56.340 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T21:51:56.340 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2024-09-10T21:51:56.340 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.340 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2024-09-10T21:51:56.340 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.340 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2024-09-10T21:51:56.340 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.340 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2024-09-10T21:51:56.340 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2024-09-10T21:51:56.340 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.340 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2024-09-10T21:51:56.341 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2024-09-10T21:51:56.341 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.341 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T21:51:56.341 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2024-09-10T21:51:56.341 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.341 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T21:51:56.341 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T21:51:56.341 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.341 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2024-09-10T21:51:56.341 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.341 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2024-09-10T21:51:56.341 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2024-09-10T21:51:56.341 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2024-09-10T21:51:56.341 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2024-09-10T21:51:56.342 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2024-09-10T21:51:56.342 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2024-09-10T21:51:56.342 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2024-09-10T21:51:56.342 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T21:51:56.342 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T21:51:56.342 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2024-09-10T21:51:56.342 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.342 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T21:51:56.342 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.342 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2024-09-10T21:51:56.342 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.342 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2024-09-10T21:51:56.342 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.342 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2024-09-10T21:51:56.342 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2024-09-10T21:51:56.343 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2024-09-10T21:51:56.343 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.7,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2024-09-10T21:51:56.343 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on skipped message = False 2024-09-10T21:51:56.343 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 2029451a-3c26-48c7-b874-0597197278ac 2024-09-10T21:51:56.343 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.343 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2024-09-10T21:51:56.343 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2024-09-10T21:51:56.343 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.343 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2024-09-10T21:51:56.343 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2024-09-10T21:51:56.343 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2024-09-10T21:51:56.343 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.343 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2024-09-10T21:51:56.344 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2024-09-10T21:51:56.344 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.344 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2024-09-10T21:51:56.344 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2024-09-10T21:51:56.344 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2024-09-10T21:51:56.344 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug trim objects = true 2024-09-10T21:51:56.344 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.344 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2024-09-10T21:51:56.344 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2024-09-10T21:51:56.344 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.344 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2024-09-10T21:51:56.344 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.344 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2024-09-10T21:51:56.344 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.345 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2024-09-10T21:51:56.345 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.345 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2024-09-10T21:51:56.345 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2024-09-10T21:51:56.345 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.345 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2024-09-10T21:51:56.345 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2024-09-10T21:51:56.345 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2024-09-10T21:51:56.345 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2024-09-10T21:51:56.345 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = bitmap 2024-09-10T21:51:56.345 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2024-09-10T21:51:56.345 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2024-09-10T21:51:56.345 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 1/20 2024-09-10T21:51:56.345 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 1/20 2024-09-10T21:51:56.346 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-10T21:51:56.346 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2024-09-10T21:51:56.346 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 4/10 2024-09-10T21:51:56.346 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T21:51:56.346 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2024-09-10T21:51:56.346 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2024-09-10T21:51:56.346 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2024-09-10T21:51:56.346 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2024-09-10T21:51:56.346 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.346 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2024-09-10T21:51:56.346 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-10T21:51:56.346 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2024-09-10T21:51:56.346 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-09-10T21:51:56.346 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-09-10T21:51:56.347 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:51:56.347 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:51:56.347 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2024-09-10T21:51:56.347 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 30 2024-09-10T21:51:56.347 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2024-09-10T21:51:56.347 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.347 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2024-09-10T21:51:56.347 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-10T21:51:56.347 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-09-10T21:51:56.347 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2024-09-10T21:51:56.347 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-09-10T21:51:56.347 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2024-09-10T21:51:56.347 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2024-09-10T21:51:56.348 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:51:56.348 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2024-09-10T21:51:56.348 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2024-09-10T21:51:56.348 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:51:56.348 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.348 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T21:51:56.348 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T21:51:56.348 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T21:51:56.348 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.348 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T21:51:56.348 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T21:51:56.348 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T21:51:56.348 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.348 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2024-09-10T21:51:56.349 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2024-09-10T21:51:56.349 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.349 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2024-09-10T21:51:56.349 INFO:teuthology.orchestra.run.smithi133.stdout: mon down mkfs grace = 300 2024-09-10T21:51:56.349 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.349 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2024-09-10T21:51:56.349 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2024-09-10T21:51:56.349 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2024-09-10T21:51:56.349 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2024-09-10T21:51:56.349 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T21:51:56.349 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T21:51:56.349 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2024-09-10T21:51:56.349 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2024-09-10T21:51:56.349 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2024-09-10T21:51:56.350 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2024-09-10T21:51:56.350 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-09-10T21:51:56.350 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-09-10T21:51:56.350 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-09-10T21:51:56.350 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2024-09-10T21:51:56.350 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2024-09-10T21:51:56.350 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = False 2024-09-10T21:51:56.363 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T21:51:56.366 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T21:51:56.461 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-09-10T21:51:56.461 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-09-10T21:51:56.461 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T21:51:56.461 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-09-10T21:51:56.461 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.461 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-09-10T21:51:56.461 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.461 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-09-10T21:51:56.461 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.461 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-09-10T21:51:56.461 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-09-10T21:51:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-09-10T21:51:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-09-10T21:51:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T21:51:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-09-10T21:51:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T21:51:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T21:51:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-09-10T21:51:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-09-10T21:51:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-09-10T21:51:56.462 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-09-10T21:51:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-09-10T21:51:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-09-10T21:51:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-09-10T21:51:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-09-10T21:51:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T21:51:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T21:51:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-09-10T21:51:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T21:51:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-09-10T21:51:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.463 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-09-10T21:51:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-09-10T21:51:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-09-10T21:51:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-09-10T21:51:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2024-09-10T21:51:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on skipped message = False 2024-09-10T21:51:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 2029451a-3c26-48c7-b874-0597197278ac 2024-09-10T21:51:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.464 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-09-10T21:51:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-09-10T21:51:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-09-10T21:51:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2024-09-10T21:51:56.464 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-09-10T21:51:56.465 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-09-10T21:51:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-09-10T21:51:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-09-10T21:51:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-09-10T21:51:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-09-10T21:51:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug trim objects = true 2024-09-10T21:51:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-09-10T21:51:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-09-10T21:51:56.468 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-09-10T21:51:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-09-10T21:51:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-09-10T21:51:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-09-10T21:51:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-09-10T21:51:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-09-10T21:51:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-09-10T21:51:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: bdev async discard = True 2024-09-10T21:51:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: bdev enable discard = True 2024-09-10T21:51:56.469 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore allocator = bitmap 2024-09-10T21:51:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-09-10T21:51:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-09-10T21:51:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 1/20 2024-09-10T21:51:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 1/20 2024-09-10T21:51:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-10T21:51:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-09-10T21:51:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 4/10 2024-09-10T21:51:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T21:51:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-09-10T21:51:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-09-10T21:51:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2024-09-10T21:51:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2024-09-10T21:51:56.470 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.470 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-09-10T21:51:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-10T21:51:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-09-10T21:51:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-10T21:51:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-10T21:51:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:51:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:51:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-09-10T21:51:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: client mount timeout = 30 2024-09-10T21:51:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-09-10T21:51:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.471 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-09-10T21:51:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-10T21:51:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-10T21:51:56.471 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-09-10T21:51:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-10T21:51:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-09-10T21:51:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-09-10T21:51:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:51:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-09-10T21:51:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-09-10T21:51:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:51:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T21:51:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T21:51:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T21:51:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T21:51:56.472 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T21:51:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T21:51:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-09-10T21:51:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-09-10T21:51:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-09-10T21:51:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: mon down mkfs grace = 300 2024-09-10T21:51:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:56.473 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-09-10T21:51:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-09-10T21:51:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-09-10T21:51:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-09-10T21:51:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T21:51:56.473 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T21:51:56.474 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = "" 2024-09-10T21:51:56.474 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-09-10T21:51:56.474 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-09-10T21:51:56.474 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-09-10T21:51:56.474 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-09-10T21:51:56.474 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-09-10T21:51:56.474 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-09-10T21:51:56.474 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-09-10T21:51:56.474 INFO:teuthology.orchestra.run.smithi007.stdout:[mds] 2024-09-10T21:51:56.474 INFO:teuthology.orchestra.run.smithi007.stdout: mds verify scatter = False 2024-09-10T21:51:56.474 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug scatterstat = False 2024-09-10T21:51:56.475 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2024-09-10T21:51:56.475 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2024-09-10T21:51:56.475 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T21:51:56.475 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2024-09-10T21:51:56.475 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.475 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2024-09-10T21:51:56.475 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.475 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2024-09-10T21:51:56.475 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.475 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2024-09-10T21:51:56.475 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2024-09-10T21:51:56.475 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.476 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2024-09-10T21:51:56.476 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2024-09-10T21:51:56.476 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.476 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T21:51:56.476 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2024-09-10T21:51:56.476 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.476 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T21:51:56.476 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T21:51:56.476 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.476 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2024-09-10T21:51:56.476 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.476 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2024-09-10T21:51:56.476 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2024-09-10T21:51:56.477 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2024-09-10T21:51:56.477 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2024-09-10T21:51:56.477 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2024-09-10T21:51:56.477 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2024-09-10T21:51:56.477 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2024-09-10T21:51:56.477 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T21:51:56.477 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T21:51:56.477 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2024-09-10T21:51:56.477 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.477 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T21:51:56.477 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.477 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2024-09-10T21:51:56.477 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.477 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2024-09-10T21:51:56.478 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.478 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2024-09-10T21:51:56.478 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2024-09-10T21:51:56.478 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2024-09-10T21:51:56.478 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.7,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2024-09-10T21:51:56.478 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on skipped message = False 2024-09-10T21:51:56.478 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 2029451a-3c26-48c7-b874-0597197278ac 2024-09-10T21:51:56.478 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.478 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2024-09-10T21:51:56.478 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2024-09-10T21:51:56.478 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.478 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2024-09-10T21:51:56.478 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2024-09-10T21:51:56.479 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2024-09-10T21:51:56.479 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.479 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2024-09-10T21:51:56.479 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2024-09-10T21:51:56.479 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.479 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2024-09-10T21:51:56.479 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2024-09-10T21:51:56.479 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2024-09-10T21:51:56.479 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug trim objects = true 2024-09-10T21:51:56.479 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.479 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2024-09-10T21:51:56.479 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2024-09-10T21:51:56.479 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.479 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2024-09-10T21:51:56.480 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.480 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2024-09-10T21:51:56.480 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.480 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2024-09-10T21:51:56.480 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.480 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2024-09-10T21:51:56.480 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2024-09-10T21:51:56.480 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.480 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2024-09-10T21:51:56.480 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2024-09-10T21:51:56.480 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2024-09-10T21:51:56.480 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2024-09-10T21:51:56.480 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = bitmap 2024-09-10T21:51:56.480 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2024-09-10T21:51:56.481 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2024-09-10T21:51:56.481 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 1/20 2024-09-10T21:51:56.481 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 1/20 2024-09-10T21:51:56.481 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-10T21:51:56.481 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2024-09-10T21:51:56.481 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 4/10 2024-09-10T21:51:56.481 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T21:51:56.481 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2024-09-10T21:51:56.481 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2024-09-10T21:51:56.481 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2024-09-10T21:51:56.481 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2024-09-10T21:51:56.481 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.481 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2024-09-10T21:51:56.482 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-10T21:51:56.482 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2024-09-10T21:51:56.482 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-09-10T21:51:56.482 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-09-10T21:51:56.482 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:51:56.482 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:51:56.482 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2024-09-10T21:51:56.482 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 30 2024-09-10T21:51:56.482 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2024-09-10T21:51:56.482 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.482 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2024-09-10T21:51:56.482 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-10T21:51:56.482 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-09-10T21:51:56.482 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2024-09-10T21:51:56.483 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-09-10T21:51:56.483 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2024-09-10T21:51:56.483 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2024-09-10T21:51:56.483 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:51:56.483 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2024-09-10T21:51:56.483 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2024-09-10T21:51:56.483 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:51:56.483 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.483 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T21:51:56.483 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T21:51:56.483 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T21:51:56.483 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.483 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T21:51:56.483 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T21:51:56.484 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T21:51:56.484 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.484 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2024-09-10T21:51:56.484 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2024-09-10T21:51:56.484 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.484 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2024-09-10T21:51:56.484 INFO:teuthology.orchestra.run.smithi133.stdout: mon down mkfs grace = 300 2024-09-10T21:51:56.484 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:51:56.484 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2024-09-10T21:51:56.484 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2024-09-10T21:51:56.484 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2024-09-10T21:51:56.484 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2024-09-10T21:51:56.484 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T21:51:56.484 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T21:51:56.485 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2024-09-10T21:51:56.485 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2024-09-10T21:51:56.485 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2024-09-10T21:51:56.485 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2024-09-10T21:51:56.485 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-09-10T21:51:56.485 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-09-10T21:51:56.485 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-09-10T21:51:56.485 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2024-09-10T21:51:56.485 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2024-09-10T21:51:56.485 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = False 2024-09-10T21:51:56.485 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = False 2024-09-10T21:51:56.488 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000001.00000000 bravo_head 2024-09-10T21:51:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.564+0000 7fcfc1688f40 1 Processor -- start 2024-09-10T21:51:56.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.564+0000 7fcfc1688f40 1 -- start start 2024-09-10T21:51:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.565+0000 7fcfc1688f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e875af9e10 0x55e875bad650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.565+0000 7fcfc1688f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e875b278b0 0x55e875b23230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.565+0000 7fcfc1688f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e875b27c90 0x55e875b21d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:56.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.565+0000 7fcfc1688f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e875af31b0 con 0x55e875af9e10 2024-09-10T21:51:56.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.565+0000 7fcfc1688f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e875a43c60 con 0x55e875b278b0 2024-09-10T21:51:56.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.565+0000 7fcfc1688f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55e875bceff0 con 0x55e875b27c90 2024-09-10T21:51:56.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.565+0000 7fcfbe800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e875af9e10 0x55e875bad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:56.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.565+0000 7fcfbe800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e875af9e10 0x55e875bad650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53506/0 (socket says 172.21.15.7:53506) 2024-09-10T21:51:56.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.565+0000 7fcfbe800640 1 -- 172.21.15.7:0/197665294 learned_addr learned my addr 172.21.15.7:0/197665294 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:56.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.566+0000 7fcfbe800640 1 -- 172.21.15.7:0/197665294 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e875b27c90 msgr2=0x55e875b21d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:56.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.566+0000 7fcfbe800640 1 --2- 172.21.15.7:0/197665294 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e875b27c90 0x55e875b21d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:56.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.566+0000 7fcfbe800640 1 -- 172.21.15.7:0/197665294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e875b278b0 msgr2=0x55e875b23230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:51:56.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.566+0000 7fcfbe800640 1 --2- 172.21.15.7:0/197665294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e875b278b0 0x55e875b23230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:56.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.566+0000 7fcfbe800640 1 -- 172.21.15.7:0/197665294 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e875a1bf10 con 0x55e875af9e10 2024-09-10T21:51:56.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.566+0000 7fcfbe800640 1 --2- 172.21.15.7:0/197665294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e875af9e10 0x55e875bad650 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000b790 tx=0x7fcfb0004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769fe02899ed4721 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:56.568 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.566+0000 7fcfbca00640 1 -- 172.21.15.7:0/197665294 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb000ac20 con 0x55e875af9e10 2024-09-10T21:51:56.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.566+0000 7fcfbca00640 1 -- 172.21.15.7:0/197665294 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfb000adc0 con 0x55e875af9e10 2024-09-10T21:51:56.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.566+0000 7fcfbca00640 1 -- 172.21.15.7:0/197665294 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb00086b0 con 0x55e875af9e10 2024-09-10T21:51:56.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.567+0000 7fcfc1688f40 1 -- 172.21.15.7:0/197665294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e875af9e10 msgr2=0x55e875bad650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:56.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.567+0000 7fcfc1688f40 1 --2- 172.21.15.7:0/197665294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e875af9e10 0x55e875bad650 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000b790 tx=0x7fcfb0004f90 comp rx=0 tx=0).stop 2024-09-10T21:51:56.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.567+0000 7fcfc1688f40 1 -- 172.21.15.7:0/197665294 shutdown_connections 2024-09-10T21:51:56.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.567+0000 7fcfc1688f40 1 --2- 172.21.15.7:0/197665294 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e875b278b0 0x55e875b23230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:56.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.567+0000 7fcfc1688f40 1 --2- 172.21.15.7:0/197665294 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e875b27c90 0x55e875b21d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:56.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.567+0000 7fcfc1688f40 1 --2- 172.21.15.7:0/197665294 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e875af9e10 0x55e875bad650 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:56.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.567+0000 7fcfc1688f40 1 -- 172.21.15.7:0/197665294 >> 172.21.15.7:0/197665294 conn(0x55e875b421c0 msgr2=0x55e875b425a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:56.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.567+0000 7fcfc1688f40 1 -- 172.21.15.7:0/197665294 shutdown_connections 2024-09-10T21:51:56.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.567+0000 7fcfc1688f40 1 -- 172.21.15.7:0/197665294 wait complete. 2024-09-10T21:51:56.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.568+0000 7fcfc1688f40 1 Processor -- start 2024-09-10T21:51:56.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.568+0000 7fcfc1688f40 1 -- start start 2024-09-10T21:51:56.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.569+0000 7fcfc1688f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e875af9e10 0x55e875bc4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:56.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.569+0000 7fcfc1688f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e875b278b0 0x55e875b25a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:56.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.569+0000 7fcfc1688f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e875b27c90 0x55e875b25fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:56.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.569+0000 7fcfc1688f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55e875b23770 con 0x55e875b27c90 2024-09-10T21:51:56.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.569+0000 7fcfc1688f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55e875badfe0 con 0x55e875b278b0 2024-09-10T21:51:56.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.569+0000 7fcfc1688f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55e875b26b00 con 0x55e875af9e10 2024-09-10T21:51:56.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.569+0000 7fcfbf200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e875b27c90 0x55e875b25fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:56.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.569+0000 7fcfbf200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e875b27c90 0x55e875b25fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53508/0 (socket says 172.21.15.7:53508) 2024-09-10T21:51:56.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.569+0000 7fcfbf200640 1 -- 172.21.15.7:0/180124468 learned_addr learned my addr 172.21.15.7:0/180124468 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:56.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.569+0000 7fcfbde00640 1 --2- 172.21.15.7:0/180124468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e875b278b0 0x55e875b25a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:56.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.569+0000 7fcfbe800640 1 --2- 172.21.15.7:0/180124468 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e875af9e10 0x55e875bc4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:56.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.569+0000 7fcfbf200640 1 -- 172.21.15.7:0/180124468 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e875af9e10 msgr2=0x55e875bc4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:56.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.569+0000 7fcfbf200640 1 --2- 172.21.15.7:0/180124468 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e875af9e10 0x55e875bc4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:56.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.569+0000 7fcfbf200640 1 -- 172.21.15.7:0/180124468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e875b278b0 msgr2=0x55e875b25a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:56.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.569+0000 7fcfbf200640 1 --2- 172.21.15.7:0/180124468 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55e875b278b0 0x55e875b25a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:56.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.569+0000 7fcfbf200640 1 -- 172.21.15.7:0/180124468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e875bb4bf0 con 0x55e875b27c90 2024-09-10T21:51:56.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.569+0000 7fcfbe800640 1 --2- 172.21.15.7:0/180124468 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55e875af9e10 0x55e875bc4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:51:56.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.570+0000 7fcfbf200640 1 --2- 172.21.15.7:0/180124468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e875b27c90 0x55e875b25fa0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800d5d0 tx=0x7fcfb800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:56.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.570+0000 7fcfb7400640 1 -- 172.21.15.7:0/180124468 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb800f900 con 0x55e875b27c90 2024-09-10T21:51:56.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.570+0000 7fcfb7400640 1 -- 172.21.15.7:0/180124468 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfb800fb90 con 0x55e875b27c90 2024-09-10T21:51:56.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.570+0000 7fcfb7400640 1 -- 172.21.15.7:0/180124468 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfb800e420 con 0x55e875b27c90 2024-09-10T21:51:56.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.570+0000 7fcfc1688f40 1 -- 172.21.15.7:0/180124468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e875bc8ff0 con 0x55e875b27c90 2024-09-10T21:51:56.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.570+0000 7fcfc1688f40 1 -- 172.21.15.7:0/180124468 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e875bc9ad0 con 0x55e875b27c90 2024-09-10T21:51:56.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.572+0000 7fcfb7400640 1 -- 172.21.15.7:0/180124468 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fcfb800e5c0 con 0x55e875b27c90 2024-09-10T21:51:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.573+0000 7fcfb7400640 1 --2- 172.21.15.7:0/180124468 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcfa0076950 0x7fcfa0078e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.573+0000 7fcfb7400640 1 -- 172.21.15.7:0/180124468 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5853+0+0 (secure 0 0 0) 0x7fcfb8099bc0 con 0x55e875b27c90 2024-09-10T21:51:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.573+0000 7fcfc1688f40 1 --2- 172.21.15.7:0/180124468 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55e875bc7590 0x55e875bc7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.573+0000 7fcfc1688f40 1 -- 172.21.15.7:0/180124468 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:1 16.7 16:e2e07930:::10000000001.00000000:head [omap-rm-keys in=18b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e117) -- 0x55e875c62a20 con 0x55e875bc7590 2024-09-10T21:51:56.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.573+0000 7fcfbe800640 1 --2- 172.21.15.7:0/180124468 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcfa0076950 0x7fcfa0078e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:56.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.574+0000 7fcfbde00640 1 --2- 172.21.15.7:0/180124468 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55e875bc7590 0x55e875bc7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:56.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.574+0000 7fcfbe800640 1 --2- 172.21.15.7:0/180124468 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcfa0076950 0x7fcfa0078e10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0002040 tx=0x7fcfb0002fb0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:56.575 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.574+0000 7fcfbde00640 1 --2- 172.21.15.7:0/180124468 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55e875bc7590 0x55e875bc7970 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:56.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.578+0000 7fcfbde00640 1 -- 172.21.15.7:0/180124468 <== osd.3 v2:172.21.15.7:6800/2195665006 1 ==== osd_op_reply(1 10000000001.00000000 [omap-rm-keys] v117'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fcfac04d740 con 0x55e875bc7590 2024-09-10T21:51:56.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.578+0000 7fcfc1688f40 1 -- 172.21.15.7:0/180124468 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55e875bc7590 msgr2=0x55e875bc7970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:56.579 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.578+0000 7fcfc1688f40 1 --2- 172.21.15.7:0/180124468 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55e875bc7590 0x55e875bc7970 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:56.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.578+0000 7fcfc1688f40 1 -- 172.21.15.7:0/180124468 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcfa0076950 msgr2=0x7fcfa0078e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:56.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.578+0000 7fcfc1688f40 1 --2- 172.21.15.7:0/180124468 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fcfa0076950 0x7fcfa0078e10 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fcfb0002040 tx=0x7fcfb0002fb0 comp rx=0 tx=0).stop 2024-09-10T21:51:56.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.578+0000 7fcfc1688f40 1 -- 172.21.15.7:0/180124468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e875b27c90 msgr2=0x55e875b25fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:56.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.578+0000 7fcfc1688f40 1 --2- 172.21.15.7:0/180124468 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55e875b27c90 0x55e875b25fa0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fcfb800d5d0 tx=0x7fcfb800da90 comp rx=0 tx=0).stop 2024-09-10T21:51:56.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.578+0000 7fcfbf200640 1 -- 172.21.15.7:0/180124468 reap_dead start 2024-09-10T21:51:56.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.578+0000 7fcfc1688f40 1 -- 172.21.15.7:0/180124468 shutdown_connections 2024-09-10T21:51:56.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.578+0000 7fcfc1688f40 1 -- 172.21.15.7:0/180124468 >> 172.21.15.7:0/180124468 conn(0x55e875b421c0 msgr2=0x55e875b425a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:56.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.578+0000 7fcfc1688f40 1 -- 172.21.15.7:0/180124468 shutdown_connections 2024-09-10T21:51:56.580 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.579+0000 7fcfc1688f40 1 -- 172.21.15.7:0/180124468 wait complete. 2024-09-10T21:51:56.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:51:56.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.748+0000 7fe969a00640 1 -- 172.21.15.7:0/465070060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964142260 msgr2=0x7fe96415c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:56.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.748+0000 7fe969a00640 1 --2- 172.21.15.7:0/465070060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964142260 0x7fe96415c460 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fe964070dc0 tx=0x7fe94c02a560 comp rx=0 tx=0).stop 2024-09-10T21:51:56.749 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.748+0000 7fe969a00640 1 -- 172.21.15.7:0/465070060 shutdown_connections 2024-09-10T21:51:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.748+0000 7fe969a00640 1 --2- 172.21.15.7:0/465070060 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe96415c9a0 0x7fe96406e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.748+0000 7fe969a00640 1 --2- 172.21.15.7:0/465070060 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964142260 0x7fe96415c460 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.748+0000 7fe969a00640 1 --2- 172.21.15.7:0/465070060 >> v2:172.21.15.133:3300/0 conn(0x7fe9640b5170 0x7fe9640b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.748+0000 7fe969a00640 1 -- 172.21.15.7:0/465070060 >> 172.21.15.7:0/465070060 conn(0x7fe9640b1b60 msgr2=0x7fe9640b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.748+0000 7fe969a00640 1 -- 172.21.15.7:0/465070060 shutdown_connections 2024-09-10T21:51:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.748+0000 7fe969a00640 1 -- 172.21.15.7:0/465070060 wait complete. 2024-09-10T21:51:56.750 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.750+0000 7fe969a00640 1 Processor -- start 2024-09-10T21:51:56.751 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.750+0000 7fe969a00640 1 -- start start 2024-09-10T21:51:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.751+0000 7fe969a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9640b5170 0x7fe9641f8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.751+0000 7fe969a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964142260 0x7fe9641f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.751+0000 7fe969a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe96415c9a0 0x7fe9640b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.751+0000 7fe969a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe964160940 con 0x7fe964142260 2024-09-10T21:51:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.751+0000 7fe969a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe9641607c0 con 0x7fe9640b5170 2024-09-10T21:51:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.751+0000 7fe969a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe96406ec40 con 0x7fe96415c9a0 2024-09-10T21:51:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.751+0000 7fe95a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964142260 0x7fe9641f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.751+0000 7fe95a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964142260 0x7fe9641f51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53530/0 (socket says 172.21.15.7:53530) 2024-09-10T21:51:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.751+0000 7fe95a000640 1 -- 172.21.15.7:0/4189931561 learned_addr learned my addr 172.21.15.7:0/4189931561 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.751+0000 7fe963400640 1 --2- 172.21.15.7:0/4189931561 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe96415c9a0 0x7fe9640b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.751+0000 7fe962a00640 1 --2- 172.21.15.7:0/4189931561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9640b5170 0x7fe9641f8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:56.753 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.752+0000 7fe95a000640 1 -- 172.21.15.7:0/4189931561 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe96415c9a0 msgr2=0x7fe9640b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.752+0000 7fe95a000640 1 --2- 172.21.15.7:0/4189931561 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe96415c9a0 0x7fe9640b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.752+0000 7fe95a000640 1 -- 172.21.15.7:0/4189931561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9640b5170 msgr2=0x7fe9641f8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.752+0000 7fe95a000640 1 --2- 172.21.15.7:0/4189931561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9640b5170 0x7fe9641f8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.752+0000 7fe95a000640 1 -- 172.21.15.7:0/4189931561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe964156e30 con 0x7fe964142260 2024-09-10T21:51:56.754 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.752+0000 7fe95a000640 1 --2- 172.21.15.7:0/4189931561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964142260 0x7fe9641f51a0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009850 tx=0x7fe94c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.752+0000 7fe95b400640 1 -- 172.21.15.7:0/4189931561 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c03f070 con 0x7fe964142260 2024-09-10T21:51:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.752+0000 7fe95b400640 1 -- 172.21.15.7:0/4189931561 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe94c0047a0 con 0x7fe964142260 2024-09-10T21:51:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.752+0000 7fe95b400640 1 -- 172.21.15.7:0/4189931561 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94c035030 con 0x7fe964142260 2024-09-10T21:51:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.752+0000 7fe969a00640 1 -- 172.21.15.7:0/4189931561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9640b48b0 con 0x7fe964142260 2024-09-10T21:51:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.753+0000 7fe969a00640 1 -- 172.21.15.7:0/4189931561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9640af5d0 con 0x7fe964142260 2024-09-10T21:51:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.754+0000 7fe969a00640 1 -- 172.21.15.7:0/4189931561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe96415c460 con 0x7fe964142260 2024-09-10T21:51:56.755 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.755+0000 7fe95b400640 1 -- 172.21.15.7:0/4189931561 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe94c004bb0 con 0x7fe964142260 2024-09-10T21:51:56.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.756+0000 7fe95b400640 1 --2- 172.21.15.7:0/4189931561 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe940077fa0 0x7fe94007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:56.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.756+0000 7fe95b400640 1 -- 172.21.15.7:0/4189931561 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5853+0+0 (secure 0 0 0) 0x7fe94c0baf40 con 0x7fe964142260 2024-09-10T21:51:56.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.756+0000 7fe962a00640 1 --2- 172.21.15.7:0/4189931561 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe940077fa0 0x7fe94007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:56.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.757+0000 7fe962a00640 1 --2- 172.21.15.7:0/4189931561 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe940077fa0 0x7fe94007a460 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe96415b390 tx=0x7fe954005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:56.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.760+0000 7fe95b400640 1 -- 172.21.15.7:0/4189931561 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe964156e30 con 0x7fe964142260 2024-09-10T21:51:56.994 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:56.993+0000 7fe969a00640 1 -- 172.21.15.7:0/4189931561 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fe9640b2800 con 0x7fe964142260 2024-09-10T21:51:57.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.209+0000 7fe95b400640 1 -- 172.21.15.7:0/4189931561 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v104) ==== 162+0+0 (secure 0 0 0) 0x7fe94c0832c0 con 0x7fe964142260 2024-09-10T21:51:57.210 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:51:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.212+0000 7fe969a00640 1 -- 172.21.15.7:0/4189931561 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe940077fa0 msgr2=0x7fe94007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.212+0000 7fe969a00640 1 --2- 172.21.15.7:0/4189931561 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe940077fa0 0x7fe94007a460 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fe96415b390 tx=0x7fe954005f90 comp rx=0 tx=0).stop 2024-09-10T21:51:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.212+0000 7fe969a00640 1 -- 172.21.15.7:0/4189931561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964142260 msgr2=0x7fe9641f51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.212+0000 7fe969a00640 1 --2- 172.21.15.7:0/4189931561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964142260 0x7fe9641f51a0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009850 tx=0x7fe94c0038a0 comp rx=0 tx=0).stop 2024-09-10T21:51:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.212+0000 7fe969a00640 1 -- 172.21.15.7:0/4189931561 shutdown_connections 2024-09-10T21:51:57.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.212+0000 7fe969a00640 1 --2- 172.21.15.7:0/4189931561 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe940077fa0 0x7fe94007a460 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.212+0000 7fe969a00640 1 --2- 172.21.15.7:0/4189931561 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe96415c9a0 0x7fe9640b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.212+0000 7fe969a00640 1 --2- 172.21.15.7:0/4189931561 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe964142260 0x7fe9641f51a0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.212+0000 7fe969a00640 1 --2- 172.21.15.7:0/4189931561 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe9640b5170 0x7fe9641f8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.212+0000 7fe969a00640 1 -- 172.21.15.7:0/4189931561 >> 172.21.15.7:0/4189931561 conn(0x7fe9640b1b60 msgr2=0x7fe964074090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.212+0000 7fe969a00640 1 -- 172.21.15.7:0/4189931561 shutdown_connections 2024-09-10T21:51:57.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.212+0000 7fe969a00640 1 -- 172.21.15.7:0/4189931561 wait complete. 2024-09-10T21:51:57.231 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:51:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.389+0000 7f4228000640 1 -- 172.21.15.7:0/1084494551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4220159910 msgr2=0x7f422015d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.389+0000 7f4228000640 1 --2- 172.21.15.7:0/1084494551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4220159910 0x7f422015d090 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f42200cfed0 tx=0x7f421402a5f0 comp rx=0 tx=0).stop 2024-09-10T21:51:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.389+0000 7f4228000640 1 -- 172.21.15.7:0/1084494551 shutdown_connections 2024-09-10T21:51:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.389+0000 7f4228000640 1 --2- 172.21.15.7:0/1084494551 >> v2:172.21.15.133:3300/0 conn(0x7f422006e810 0x7f42200cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.389+0000 7f4228000640 1 --2- 172.21.15.7:0/1084494551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4220159910 0x7f422015d090 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.389+0000 7f4228000640 1 -- 172.21.15.7:0/1084494551 >> 172.21.15.7:0/1084494551 conn(0x7f422006ca60 msgr2=0x7f422006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.389+0000 7f4228000640 1 -- 172.21.15.7:0/1084494551 shutdown_connections 2024-09-10T21:51:57.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.389+0000 7f4228000640 1 -- 172.21.15.7:0/1084494551 wait complete. 2024-09-10T21:51:57.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.390+0000 7f4228000640 1 Processor -- start 2024-09-10T21:51:57.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.391+0000 7f4228000640 1 -- start start 2024-09-10T21:51:57.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.391+0000 7f4228000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f422006e810 0x7f42200b4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:57.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.392+0000 7f4228000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42200cdbf0 0x7f42200ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:57.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.392+0000 7f4228000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4220159910 0x7f42200add90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:57.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.392+0000 7f4228000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f42200740c0 con 0x7f4220159910 2024-09-10T21:51:57.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.392+0000 7f4228000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4220073f40 con 0x7f422006e810 2024-09-10T21:51:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.392+0000 7f4228000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4220074240 con 0x7f42200cdbf0 2024-09-10T21:51:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.392+0000 7f4226000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4220159910 0x7f42200add90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.392+0000 7f4226000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4220159910 0x7f42200add90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53546/0 (socket says 172.21.15.7:53546) 2024-09-10T21:51:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.392+0000 7f4226000640 1 -- 172.21.15.7:0/677433059 learned_addr learned my addr 172.21.15.7:0/677433059 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.392+0000 7f4225600640 1 --2- 172.21.15.7:0/677433059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f422006e810 0x7f42200b4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.392+0000 7f4224c00640 1 --2- 172.21.15.7:0/677433059 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42200cdbf0 0x7f42200ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.392+0000 7f4226000640 1 -- 172.21.15.7:0/677433059 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42200cdbf0 msgr2=0x7f42200ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.392+0000 7f4226000640 1 --2- 172.21.15.7:0/677433059 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42200cdbf0 0x7f42200ad850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.392+0000 7f4226000640 1 -- 172.21.15.7:0/677433059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f422006e810 msgr2=0x7f42200b4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.392+0000 7f4226000640 1 --2- 172.21.15.7:0/677433059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f422006e810 0x7f42200b4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.392+0000 7f4226000640 1 -- 172.21.15.7:0/677433059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422014f180 con 0x7f4220159910 2024-09-10T21:51:57.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.393+0000 7f4226000640 1 --2- 172.21.15.7:0/677433059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4220159910 0x7f42200add90 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f421800c7a0 tx=0x7f421800cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.393+0000 7f421d600640 1 -- 172.21.15.7:0/677433059 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218003b70 con 0x7f4220159910 2024-09-10T21:51:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.393+0000 7f421d600640 1 -- 172.21.15.7:0/677433059 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4218003d30 con 0x7f4220159910 2024-09-10T21:51:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.393+0000 7f421d600640 1 -- 172.21.15.7:0/677433059 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218005330 con 0x7f4220159910 2024-09-10T21:51:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.393+0000 7f4228000640 1 -- 172.21.15.7:0/677433059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42201f49d0 con 0x7f4220159910 2024-09-10T21:51:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.393+0000 7f4228000640 1 -- 172.21.15.7:0/677433059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42201f4f90 con 0x7f4220159910 2024-09-10T21:51:57.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.394+0000 7f4228000640 1 -- 172.21.15.7:0/677433059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42200cd6b0 con 0x7f4220159910 2024-09-10T21:51:57.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.395+0000 7f421d600640 1 -- 172.21.15.7:0/677433059 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f42180054d0 con 0x7f4220159910 2024-09-10T21:51:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.397+0000 7f421d600640 1 --2- 172.21.15.7:0/677433059 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4200077f70 0x7f420007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.397+0000 7f421d600640 1 -- 172.21.15.7:0/677433059 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5853+0+0 (secure 0 0 0) 0x7f4218099b20 con 0x7f4220159910 2024-09-10T21:51:57.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.397+0000 7f4225600640 1 --2- 172.21.15.7:0/677433059 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4200077f70 0x7f420007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:57.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.398+0000 7f4225600640 1 --2- 172.21.15.7:0/677433059 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4200077f70 0x7f420007a430 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4214033000 tx=0x7f4214004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:57.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.401+0000 7f421d600640 1 -- 172.21.15.7:0/677433059 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f42180057f0 con 0x7f4220159910 2024-09-10T21:51:57.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.635+0000 7f4228000640 1 -- 172.21.15.7:0/677433059 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4220075930 con 0x7f4220159910 2024-09-10T21:51:57.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.636+0000 7f421d600640 1 -- 172.21.15.7:0/677433059 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 105 v105) ==== 95+0+5224 (secure 0 0 0) 0x7f42180624f0 con 0x7f4220159910 2024-09-10T21:51:57.637 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:57.637 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":105,"btime":"2024-09-10T21:51:57:213302+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3012572644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3012572644},{"type":"v1","addr":"172.21.15.7:6834","nonce":3012572644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7660,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2492331385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2492331385},{"type":"v1","addr":"172.21.15.133:6837","nonce":2492331385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":8094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/148356862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":148356862},{"type":"v1","addr":"172.21.15.7:6835","nonce":148356862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":105,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:51:37.913965+0000","modified":"2024-09-10T21:51:57.213285+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236723},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236723":{"gid":236723,"name":"d","rank":0,"incarnation":105,"state":"up:replay","state_seq":1,"addr":"172.21.15.133:6836/772257983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":772257983},{"type":"v1","addr":"172.21.15.133:6836","nonce":772257983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-09-10T21:51:57.637 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 105 2024-09-10T21:51:57.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.638+0000 7f4228000640 1 -- 172.21.15.7:0/677433059 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4200077f70 msgr2=0x7f420007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.638+0000 7f4228000640 1 --2- 172.21.15.7:0/677433059 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4200077f70 0x7f420007a430 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4214033000 tx=0x7f4214004730 comp rx=0 tx=0).stop 2024-09-10T21:51:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.638+0000 7f4228000640 1 -- 172.21.15.7:0/677433059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4220159910 msgr2=0x7f42200add90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.638+0000 7f4228000640 1 --2- 172.21.15.7:0/677433059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4220159910 0x7f42200add90 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f421800c7a0 tx=0x7f421800cba0 comp rx=0 tx=0).stop 2024-09-10T21:51:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.638+0000 7f4228000640 1 -- 172.21.15.7:0/677433059 shutdown_connections 2024-09-10T21:51:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.638+0000 7f4228000640 1 --2- 172.21.15.7:0/677433059 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4200077f70 0x7f420007a430 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.638+0000 7f4228000640 1 --2- 172.21.15.7:0/677433059 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4220159910 0x7f42200add90 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.638+0000 7f4228000640 1 --2- 172.21.15.7:0/677433059 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f42200cdbf0 0x7f42200ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.638+0000 7f4228000640 1 --2- 172.21.15.7:0/677433059 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f422006e810 0x7f42200b4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.638+0000 7f4228000640 1 -- 172.21.15.7:0/677433059 >> 172.21.15.7:0/677433059 conn(0x7f422006ca60 msgr2=0x7f4220151630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.639+0000 7f4228000640 1 -- 172.21.15.7:0/677433059 shutdown_connections 2024-09-10T21:51:57.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:57.639+0000 7f4228000640 1 -- 172.21.15.7:0/677433059 wait complete. 2024-09-10T21:51:58.057 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 105, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:51:37.913965+0000', 'modified': '2024-09-10T21:51:57.213285+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 117, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 236723}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_236723': {'gid': 236723, 'name': 'd', 'rank': 0, 'incarnation': 105, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.133:6836/772257983', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6835', 'nonce': 772257983}, {'type': 'v1', 'addr': '172.21.15.133:6836', 'nonce': 772257983}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:51:58.058 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_236723:up:replay 2024-09-10T21:51:59.059 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:51:59.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.231+0000 7fc965800640 1 -- 172.21.15.7:0/1426441721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc944005230 msgr2=0x7fc94400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:59.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.231+0000 7fc965800640 1 --2- 172.21.15.7:0/1426441721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc944005230 0x7fc94400b9c0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fc958001800 tx=0x7fc944005830 comp rx=0 tx=0).stop 2024-09-10T21:51:59.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.231+0000 7fc965800640 1 -- 172.21.15.7:0/1426441721 shutdown_connections 2024-09-10T21:51:59.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.231+0000 7fc965800640 1 --2- 172.21.15.7:0/1426441721 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc944005230 0x7fc94400b9c0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:59.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.231+0000 7fc965800640 1 -- 172.21.15.7:0/1426441721 >> 172.21.15.7:0/1426441721 conn(0x7fc96006ca60 msgr2=0x7fc96006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:59.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.231+0000 7fc965800640 1 -- 172.21.15.7:0/1426441721 shutdown_connections 2024-09-10T21:51:59.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.231+0000 7fc965800640 1 -- 172.21.15.7:0/1426441721 wait complete. 2024-09-10T21:51:59.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.233+0000 7fc965800640 1 Processor -- start 2024-09-10T21:51:59.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.233+0000 7fc965800640 1 -- start start 2024-09-10T21:51:59.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.234+0000 7fc965800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc944005230 0x7fc960155eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:59.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.234+0000 7fc965800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9601f69b0 0x7fc96015c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:59.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.234+0000 7fc965800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9601f3190 0x7fc9600a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:59.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.234+0000 7fc965800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc96015d420 con 0x7fc9601f3190 2024-09-10T21:51:59.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.234+0000 7fc965800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc96015d2a0 con 0x7fc944005230 2024-09-10T21:51:59.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.234+0000 7fc965800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc9600cd6b0 con 0x7fc9601f69b0 2024-09-10T21:51:59.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.234+0000 7fc95f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9601f3190 0x7fc9600a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:59.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.234+0000 7fc95f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9601f3190 0x7fc9600a9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53566/0 (socket says 172.21.15.7:53566) 2024-09-10T21:51:59.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.234+0000 7fc95f400640 1 -- 172.21.15.7:0/1931458475 learned_addr learned my addr 172.21.15.7:0/1931458475 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:51:59.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.234+0000 7fc95ea00640 1 --2- 172.21.15.7:0/1931458475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc944005230 0x7fc960155eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:59.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.234+0000 7fc95e000640 1 --2- 172.21.15.7:0/1931458475 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9601f69b0 0x7fc96015c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:59.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.235+0000 7fc95f400640 1 -- 172.21.15.7:0/1931458475 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9601f69b0 msgr2=0x7fc96015c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:59.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.235+0000 7fc95f400640 1 --2- 172.21.15.7:0/1931458475 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9601f69b0 0x7fc96015c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:59.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.235+0000 7fc95f400640 1 -- 172.21.15.7:0/1931458475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc944005230 msgr2=0x7fc960155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:59.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.235+0000 7fc95f400640 1 --2- 172.21.15.7:0/1931458475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc944005230 0x7fc960155eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:59.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.235+0000 7fc95f400640 1 -- 172.21.15.7:0/1931458475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9601f29b0 con 0x7fc9601f3190 2024-09-10T21:51:59.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.235+0000 7fc95f400640 1 --2- 172.21.15.7:0/1931458475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9601f3190 0x7fc9600a9440 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fc958001920 tx=0x7fc958033a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:59.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.235+0000 7fc956a00640 1 -- 172.21.15.7:0/1931458475 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9580068c0 con 0x7fc9601f3190 2024-09-10T21:51:59.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.235+0000 7fc956a00640 1 -- 172.21.15.7:0/1931458475 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9580035f0 con 0x7fc9601f3190 2024-09-10T21:51:59.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.235+0000 7fc956a00640 1 -- 172.21.15.7:0/1931458475 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc958003a40 con 0x7fc9601f3190 2024-09-10T21:51:59.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.235+0000 7fc965800640 1 -- 172.21.15.7:0/1931458475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc960158360 con 0x7fc9601f3190 2024-09-10T21:51:59.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.236+0000 7fc965800640 1 -- 172.21.15.7:0/1931458475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9601f2c70 con 0x7fc9601f3190 2024-09-10T21:51:59.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.237+0000 7fc965800640 1 -- 172.21.15.7:0/1931458475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc96015fa70 con 0x7fc9601f3190 2024-09-10T21:51:59.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.238+0000 7fc956a00640 1 -- 172.21.15.7:0/1931458475 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc958003be0 con 0x7fc9601f3190 2024-09-10T21:51:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.239+0000 7fc956a00640 1 --2- 172.21.15.7:0/1931458475 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc940077f70 0x7fc94007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:51:59.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.239+0000 7fc956a00640 1 -- 172.21.15.7:0/1931458475 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5853+0+0 (secure 0 0 0) 0x7fc9580b9dc0 con 0x7fc9601f3190 2024-09-10T21:51:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.239+0000 7fc95ea00640 1 --2- 172.21.15.7:0/1931458475 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc940077f70 0x7fc94007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:51:59.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.240+0000 7fc95ea00640 1 --2- 172.21.15.7:0/1931458475 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc940077f70 0x7fc94007a430 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc950005fd0 tx=0x7fc950009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:51:59.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.243+0000 7fc956a00640 1 -- 172.21.15.7:0/1931458475 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc958082140 con 0x7fc9601f3190 2024-09-10T21:51:59.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.470+0000 7fc965800640 1 -- 172.21.15.7:0/1931458475 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9600742f0 con 0x7fc9601f3190 2024-09-10T21:51:59.471 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.471+0000 7fc956a00640 1 -- 172.21.15.7:0/1931458475 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 107 v107) ==== 95+0+5224 (secure 0 0 0) 0x7fc958086b70 con 0x7fc9601f3190 2024-09-10T21:51:59.472 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:51:59.472 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":107,"btime":"2024-09-10T21:51:59:217026+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3012572644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3012572644},{"type":"v1","addr":"172.21.15.7:6834","nonce":3012572644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7660,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2492331385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2492331385},{"type":"v1","addr":"172.21.15.133:6837","nonce":2492331385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":8094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/148356862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":148356862},{"type":"v1","addr":"172.21.15.7:6835","nonce":148356862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":107,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:51:37.913965+0000","modified":"2024-09-10T21:51:58.220854+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236723},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236723":{"gid":236723,"name":"d","rank":0,"incarnation":105,"state":"up:rejoin","state_seq":9,"addr":"172.21.15.133:6836/772257983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":772257983},{"type":"v1","addr":"172.21.15.133:6836","nonce":772257983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-09-10T21:51:59.472 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 107 2024-09-10T21:51:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.473+0000 7fc965800640 1 -- 172.21.15.7:0/1931458475 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc940077f70 msgr2=0x7fc94007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.473+0000 7fc965800640 1 --2- 172.21.15.7:0/1931458475 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc940077f70 0x7fc94007a430 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fc950005fd0 tx=0x7fc950009500 comp rx=0 tx=0).stop 2024-09-10T21:51:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.473+0000 7fc965800640 1 -- 172.21.15.7:0/1931458475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9601f3190 msgr2=0x7fc9600a9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:51:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.473+0000 7fc965800640 1 --2- 172.21.15.7:0/1931458475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9601f3190 0x7fc9600a9440 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fc958001920 tx=0x7fc958033a60 comp rx=0 tx=0).stop 2024-09-10T21:51:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.473+0000 7fc965800640 1 -- 172.21.15.7:0/1931458475 shutdown_connections 2024-09-10T21:51:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.473+0000 7fc965800640 1 --2- 172.21.15.7:0/1931458475 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc940077f70 0x7fc94007a430 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.473+0000 7fc965800640 1 --2- 172.21.15.7:0/1931458475 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9601f3190 0x7fc9600a9440 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.473+0000 7fc965800640 1 --2- 172.21.15.7:0/1931458475 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9601f69b0 0x7fc96015c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:59.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.473+0000 7fc965800640 1 --2- 172.21.15.7:0/1931458475 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc944005230 0x7fc960155eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:51:59.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.473+0000 7fc965800640 1 -- 172.21.15.7:0/1931458475 >> 172.21.15.7:0/1931458475 conn(0x7fc96006ca60 msgr2=0x7fc96015a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:51:59.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.473+0000 7fc965800640 1 -- 172.21.15.7:0/1931458475 shutdown_connections 2024-09-10T21:51:59.475 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:51:59.473+0000 7fc965800640 1 -- 172.21.15.7:0/1931458475 wait complete. 2024-09-10T21:51:59.894 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 107, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:51:37.913965+0000', 'modified': '2024-09-10T21:51:58.220854+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 117, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 236723}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_236723': {'gid': 236723, 'name': 'd', 'rank': 0, 'incarnation': 105, 'state': 'up:rejoin', 'state_seq': 9, 'addr': '172.21.15.133:6836/772257983', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6835', 'nonce': 772257983}, {'type': 'v1', 'addr': '172.21.15.133:6836', 'nonce': 772257983}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:51:59.894 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_236723:up:rejoin 2024-09-10T21:52:00.895 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:52:01.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.066+0000 7f40bee00640 1 -- 172.21.15.7:0/3082585459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b806e810 msgr2=0x7f40b80cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:01.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.066+0000 7f40bee00640 1 --2- 172.21.15.7:0/3082585459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b806e810 0x7f40b80cd6b0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f40b806e390 tx=0x7f40a402a510 comp rx=0 tx=0).stop 2024-09-10T21:52:01.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.066+0000 7f40bee00640 1 -- 172.21.15.7:0/3082585459 shutdown_connections 2024-09-10T21:52:01.067 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.066+0000 7f40bee00640 1 --2- 172.21.15.7:0/3082585459 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b806e810 0x7f40b80cd6b0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.066+0000 7f40bee00640 1 --2- 172.21.15.7:0/3082585459 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40b8159910 0x7f40b815d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.066+0000 7f40bee00640 1 -- 172.21.15.7:0/3082585459 >> 172.21.15.7:0/3082585459 conn(0x7f40b806ca60 msgr2=0x7f40b806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:01.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.066+0000 7f40bee00640 1 -- 172.21.15.7:0/3082585459 shutdown_connections 2024-09-10T21:52:01.068 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.066+0000 7f40bee00640 1 -- 172.21.15.7:0/3082585459 wait complete. 2024-09-10T21:52:01.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.068+0000 7f40bee00640 1 Processor -- start 2024-09-10T21:52:01.069 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.068+0000 7f40bee00640 1 -- start start 2024-09-10T21:52:01.070 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.069+0000 7f40bee00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40b806e810 0x7f40b80af130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.069+0000 7f40bee00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b80cdbf0 0x7f40b80a9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.069+0000 7f40bee00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40b8159910 0x7f40b80ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.069+0000 7f40bee00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f40b80740c0 con 0x7f40b80cdbf0 2024-09-10T21:52:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.069+0000 7f40bee00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f40b8073f40 con 0x7f40b806e810 2024-09-10T21:52:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.069+0000 7f40bee00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f40b8074240 con 0x7f40b8159910 2024-09-10T21:52:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.069+0000 7f40b7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b80cdbf0 0x7f40b80a9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.069+0000 7f40b7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b80cdbf0 0x7f40b80a9440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53582/0 (socket says 172.21.15.7:53582) 2024-09-10T21:52:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.069+0000 7f40b7400640 1 -- 172.21.15.7:0/3499884777 learned_addr learned my addr 172.21.15.7:0/3499884777 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.069+0000 7f40bce00640 1 --2- 172.21.15.7:0/3499884777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40b8159910 0x7f40b80ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.070+0000 7f40b7e00640 1 --2- 172.21.15.7:0/3499884777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40b806e810 0x7f40b80af130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.070+0000 7f40b7400640 1 -- 172.21.15.7:0/3499884777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40b8159910 msgr2=0x7f40b80ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:01.071 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.070+0000 7f40b7400640 1 --2- 172.21.15.7:0/3499884777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40b8159910 0x7f40b80ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.070+0000 7f40b7400640 1 -- 172.21.15.7:0/3499884777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40b806e810 msgr2=0x7f40b80af130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.070+0000 7f40b7400640 1 --2- 172.21.15.7:0/3499884777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40b806e810 0x7f40b80af130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.070+0000 7f40b7400640 1 -- 172.21.15.7:0/3499884777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b80a9bb0 con 0x7f40b80cdbf0 2024-09-10T21:52:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.070+0000 7f40bce00640 1 --2- 172.21.15.7:0/3499884777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40b8159910 0x7f40b80ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.070+0000 7f40b7e00640 1 --2- 172.21.15.7:0/3499884777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40b806e810 0x7f40b80af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:01.072 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.070+0000 7f40b7400640 1 --2- 172.21.15.7:0/3499884777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b80cdbf0 0x7f40b80a9440 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f40a4000c00 tx=0x7f40a4003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:01.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.070+0000 7f40afe00640 1 -- 172.21.15.7:0/3499884777 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a403f070 con 0x7f40b80cdbf0 2024-09-10T21:52:01.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.070+0000 7f40afe00640 1 -- 172.21.15.7:0/3499884777 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40a40047a0 con 0x7f40b80cdbf0 2024-09-10T21:52:01.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.070+0000 7f40afe00640 1 -- 172.21.15.7:0/3499884777 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a4035030 con 0x7f40b80cdbf0 2024-09-10T21:52:01.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.070+0000 7f40bee00640 1 -- 172.21.15.7:0/3499884777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b80a9e90 con 0x7f40b80cdbf0 2024-09-10T21:52:01.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.071+0000 7f40bee00640 1 -- 172.21.15.7:0/3499884777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b80aa450 con 0x7f40b80cdbf0 2024-09-10T21:52:01.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.072+0000 7f40bee00640 1 -- 172.21.15.7:0/3499884777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b815d370 con 0x7f40b80cdbf0 2024-09-10T21:52:01.073 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.073+0000 7f40afe00640 1 -- 172.21.15.7:0/3499884777 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f40a4004bb0 con 0x7f40b80cdbf0 2024-09-10T21:52:01.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.074+0000 7f40afe00640 1 --2- 172.21.15.7:0/3499884777 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4098077f70 0x7f409807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.074 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.074+0000 7f40afe00640 1 -- 172.21.15.7:0/3499884777 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5853+0+0 (secure 0 0 0) 0x7f40a40bb020 con 0x7f40b80cdbf0 2024-09-10T21:52:01.075 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.074+0000 7f40b7e00640 1 --2- 172.21.15.7:0/3499884777 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4098077f70 0x7f409807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.075+0000 7f40b7e00640 1 --2- 172.21.15.7:0/3499884777 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4098077f70 0x7f409807a430 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f40a8005fd0 tx=0x7f40a800f360 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:01.078 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.078+0000 7f40afe00640 1 -- 172.21.15.7:0/3499884777 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f40b815d370 con 0x7f40b80cdbf0 2024-09-10T21:52:01.311 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.310+0000 7f40bee00640 1 -- 172.21.15.7:0/3499884777 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40b80cd6b0 con 0x7f40b80cdbf0 2024-09-10T21:52:01.312 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.311+0000 7f40afe00640 1 -- 172.21.15.7:0/3499884777 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5236 (secure 0 0 0) 0x7f40a40839f0 con 0x7f40b80cdbf0 2024-09-10T21:52:01.312 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:01.312 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":108,"btime":"2024-09-10T21:52:00:219026+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3012572644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3012572644},{"type":"v1","addr":"172.21.15.7:6834","nonce":3012572644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7660,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2492331385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2492331385},{"type":"v1","addr":"172.21.15.133:6837","nonce":2492331385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":8094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/148356862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":148356862},{"type":"v1","addr":"172.21.15.7:6835","nonce":148356862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:51:37.913965+0000","modified":"2024-09-10T21:52:00.219020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236723},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236723":{"gid":236723,"name":"d","rank":0,"incarnation":105,"state":"up:active","state_seq":10,"addr":"172.21.15.133:6836/772257983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":772257983},{"type":"v1","addr":"172.21.15.133:6836","nonce":772257983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":236723,"qdb_cluster":[236723]},"id":8}]} 2024-09-10T21:52:01.312 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 108 2024-09-10T21:52:01.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.313+0000 7f40bee00640 1 -- 172.21.15.7:0/3499884777 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4098077f70 msgr2=0x7f409807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:01.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.313+0000 7f40bee00640 1 --2- 172.21.15.7:0/3499884777 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4098077f70 0x7f409807a430 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f40a8005fd0 tx=0x7f40a800f360 comp rx=0 tx=0).stop 2024-09-10T21:52:01.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.313+0000 7f40bee00640 1 -- 172.21.15.7:0/3499884777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b80cdbf0 msgr2=0x7f40b80a9440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:01.314 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.313+0000 7f40bee00640 1 --2- 172.21.15.7:0/3499884777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b80cdbf0 0x7f40b80a9440 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f40a4000c00 tx=0x7f40a4003910 comp rx=0 tx=0).stop 2024-09-10T21:52:01.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.313+0000 7f40bee00640 1 -- 172.21.15.7:0/3499884777 shutdown_connections 2024-09-10T21:52:01.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.313+0000 7f40bee00640 1 --2- 172.21.15.7:0/3499884777 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4098077f70 0x7f409807a430 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.313+0000 7f40bee00640 1 --2- 172.21.15.7:0/3499884777 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f40b8159910 0x7f40b80ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.313+0000 7f40bee00640 1 --2- 172.21.15.7:0/3499884777 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f40b80cdbf0 0x7f40b80a9440 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.313+0000 7f40bee00640 1 --2- 172.21.15.7:0/3499884777 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f40b806e810 0x7f40b80af130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.313+0000 7f40bee00640 1 -- 172.21.15.7:0/3499884777 >> 172.21.15.7:0/3499884777 conn(0x7f40b806ca60 msgr2=0x7f40b815cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:01.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.313+0000 7f40bee00640 1 -- 172.21.15.7:0/3499884777 shutdown_connections 2024-09-10T21:52:01.315 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.314+0000 7f40bee00640 1 -- 172.21.15.7:0/3499884777 wait complete. 2024-09-10T21:52:01.714 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:52:01.718 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:52:01.732 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 108, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:51:37.913965+0000', 'modified': '2024-09-10T21:52:00.219020+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 117, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 236723}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_236723': {'gid': 236723, 'name': 'd', 'rank': 0, 'incarnation': 105, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.133:6836/772257983', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6835', 'nonce': 772257983}, {'type': 'v1', 'addr': '172.21.15.133:6836', 'nonce': 772257983}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 236723, 'qdb_cluster': [236723]} 2024-09-10T21:52:01.732 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:52:01.732 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T21:52:01.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.920+0000 7fe00e200640 1 -- 172.21.15.7:0/1040098366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0081604d0 msgr2=0x7fe008070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.920+0000 7fe00e200640 1 --2- 172.21.15.7:0/1040098366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0081604d0 0x7fe008070230 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7fe0080d1190 tx=0x7fdff002a490 comp rx=0 tx=0).stop 2024-09-10T21:52:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.921+0000 7fe00e200640 1 -- 172.21.15.7:0/1040098366 shutdown_connections 2024-09-10T21:52:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.921+0000 7fe00e200640 1 --2- 172.21.15.7:0/1040098366 >> v2:172.21.15.133:3300/0 conn(0x7fe008070770 0x7fe008072fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.921+0000 7fe00e200640 1 --2- 172.21.15.7:0/1040098366 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe0081604d0 0x7fe008070230 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.921+0000 7fe00e200640 1 -- 172.21.15.7:0/1040098366 >> 172.21.15.7:0/1040098366 conn(0x7fe00806ca60 msgr2=0x7fe00806ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.921+0000 7fe00e200640 1 -- 172.21.15.7:0/1040098366 shutdown_connections 2024-09-10T21:52:01.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.921+0000 7fe00e200640 1 -- 172.21.15.7:0/1040098366 wait complete. 2024-09-10T21:52:01.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.922+0000 7fe00e200640 1 Processor -- start 2024-09-10T21:52:01.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.922+0000 7fe00e200640 1 -- start start 2024-09-10T21:52:01.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.923+0000 7fe00e200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe008070770 0x7fe0080ad850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.923+0000 7fe00e200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe008152d20 0x7fe0080b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.923+0000 7fe00e200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0081604d0 0x7fe00814e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.923+0000 7fe00e200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe008073690 con 0x7fe008152d20 2024-09-10T21:52:01.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.923+0000 7fe00e200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe008073510 con 0x7fe008070770 2024-09-10T21:52:01.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.923+0000 7fe00e200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe008073810 con 0x7fe0081604d0 2024-09-10T21:52:01.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.924+0000 7fdfffe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe008152d20 0x7fe0080b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.924+0000 7fdfffe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe008152d20 0x7fe0080b4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53608/0 (socket says 172.21.15.7:53608) 2024-09-10T21:52:01.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.924+0000 7fdfffe00640 1 -- 172.21.15.7:0/3504671743 learned_addr learned my addr 172.21.15.7:0/3504671743 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:01.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.924+0000 7fe00ce00640 1 --2- 172.21.15.7:0/3504671743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe008070770 0x7fe0080ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.924+0000 7fe00d800640 1 --2- 172.21.15.7:0/3504671743 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0081604d0 0x7fe00814e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.924+0000 7fdfffe00640 1 -- 172.21.15.7:0/3504671743 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0081604d0 msgr2=0x7fe00814e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:01.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.924+0000 7fdfffe00640 1 --2- 172.21.15.7:0/3504671743 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0081604d0 0x7fe00814e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.924+0000 7fdfffe00640 1 -- 172.21.15.7:0/3504671743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe008070770 msgr2=0x7fe0080ad850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:01.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.924+0000 7fdfffe00640 1 --2- 172.21.15.7:0/3504671743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe008070770 0x7fe0080ad850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.924+0000 7fdfffe00640 1 -- 172.21.15.7:0/3504671743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0080ae150 con 0x7fe008152d20 2024-09-10T21:52:01.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.924+0000 7fe00d800640 1 --2- 172.21.15.7:0/3504671743 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0081604d0 0x7fe00814e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:01.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.924+0000 7fdfffe00640 1 --2- 172.21.15.7:0/3504671743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe008152d20 0x7fe0080b4010 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fdff0034040 tx=0x7fdff00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:01.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.925+0000 7fe005600640 1 -- 172.21.15.7:0/3504671743 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff003e030 con 0x7fe008152d20 2024-09-10T21:52:01.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.925+0000 7fe005600640 1 -- 172.21.15.7:0/3504671743 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdff00047a0 con 0x7fe008152d20 2024-09-10T21:52:01.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.925+0000 7fe005600640 1 -- 172.21.15.7:0/3504671743 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff0035040 con 0x7fe008152d20 2024-09-10T21:52:01.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.925+0000 7fe00e200640 1 -- 172.21.15.7:0/3504671743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0080ae430 con 0x7fe008152d20 2024-09-10T21:52:01.927 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.925+0000 7fe00e200640 1 -- 172.21.15.7:0/3504671743 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0080ae9f0 con 0x7fe008152d20 2024-09-10T21:52:01.928 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.927+0000 7fe005600640 1 -- 172.21.15.7:0/3504671743 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fdff0004b70 con 0x7fe008152d20 2024-09-10T21:52:01.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.928+0000 7fe005600640 1 --2- 172.21.15.7:0/3504671743 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdfe4077fa0 0x7fdfe407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.928+0000 7fe005600640 1 -- 172.21.15.7:0/3504671743 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5853+0+0 (secure 0 0 0) 0x7fdff00ba7d0 con 0x7fe008152d20 2024-09-10T21:52:01.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.928+0000 7fe0127bc740 1 Processor -- start 2024-09-10T21:52:01.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.929+0000 7fe00ce00640 1 --2- 172.21.15.7:0/3504671743 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdfe4077fa0 0x7fdfe407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.929+0000 7fe0127bc740 1 -- start start 2024-09-10T21:52:01.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.929+0000 7fe00ce00640 1 --2- 172.21.15.7:0/3504671743 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdfe4077fa0 0x7fdfe407a460 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdff8004f60 tx=0x7fdff8007e30 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:01.930 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.929+0000 7fe0127bc740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56245358daa0 0x56245358de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.929+0000 7fe0127bc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562453548c10 0x56245358c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.929+0000 7fe0127bc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453549190 0x56245358cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.929+0000 7fe0127bc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5624534cb430 con 0x562453549190 2024-09-10T21:52:01.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.929+0000 7fe0127bc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5624534bb5d0 con 0x562453548c10 2024-09-10T21:52:01.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.929+0000 7fe0127bc740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5624534aaa00 con 0x56245358daa0 2024-09-10T21:52:01.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.930+0000 7fe00d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453549190 0x56245358cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.930+0000 7fe00ce00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56245358daa0 0x56245358de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.930+0000 7fe00d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453549190 0x56245358cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53610/0 (socket says 172.21.15.7:53610) 2024-09-10T21:52:01.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.930+0000 7fdfffe00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562453548c10 0x56245358c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.931 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.930+0000 7fe00d800640 1 -- 172.21.15.7:0/728820307 learned_addr learned my addr 172.21.15.7:0/728820307 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:01.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.930+0000 7fe00d800640 1 -- 172.21.15.7:0/728820307 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56245358daa0 msgr2=0x56245358de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:01.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.930+0000 7fe00d800640 1 --2- 172.21.15.7:0/728820307 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56245358daa0 0x56245358de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.930+0000 7fe00d800640 1 -- 172.21.15.7:0/728820307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562453548c10 msgr2=0x56245358c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:01.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.930+0000 7fe00d800640 1 --2- 172.21.15.7:0/728820307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562453548c10 0x56245358c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.930+0000 7fe00d800640 1 -- 172.21.15.7:0/728820307 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562453473520 con 0x562453549190 2024-09-10T21:52:01.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.930+0000 7fdfffe00640 1 --2- 172.21.15.7:0/728820307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562453548c10 0x56245358c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:52:01.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.930+0000 7fe00d800640 1 --2- 172.21.15.7:0/728820307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453549190 0x56245358cac0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fe000007f40 tx=0x7fe00000e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4b9e2e3fd0a557 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:01.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.931+0000 7fdffd600640 1 -- 172.21.15.7:0/728820307 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000014070 con 0x562453549190 2024-09-10T21:52:01.933 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.931+0000 7fdffd600640 1 -- 172.21.15.7:0/728820307 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe000004e10 con 0x562453549190 2024-09-10T21:52:01.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.931+0000 7fdffd600640 1 -- 172.21.15.7:0/728820307 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0000108a0 con 0x562453549190 2024-09-10T21:52:01.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.931+0000 7fe0127bc740 1 -- 172.21.15.7:0/728820307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453549190 msgr2=0x56245358cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:01.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.931+0000 7fe0127bc740 1 --2- 172.21.15.7:0/728820307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453549190 0x56245358cac0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fe000007f40 tx=0x7fe00000e7e0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.931+0000 7fe0127bc740 1 -- 172.21.15.7:0/728820307 shutdown_connections 2024-09-10T21:52:01.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.931+0000 7fe0127bc740 1 --2- 172.21.15.7:0/728820307 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562453548c10 0x56245358c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.931+0000 7fe0127bc740 1 --2- 172.21.15.7:0/728820307 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453549190 0x56245358cac0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.931+0000 7fe0127bc740 1 --2- 172.21.15.7:0/728820307 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56245358daa0 0x56245358de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.931+0000 7fe0127bc740 1 -- 172.21.15.7:0/728820307 >> 172.21.15.7:0/728820307 conn(0x5624535441e0 msgr2=0x562453547c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:01.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.932+0000 7fe0127bc740 1 -- 172.21.15.7:0/728820307 shutdown_connections 2024-09-10T21:52:01.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.932+0000 7fe0127bc740 1 -- 172.21.15.7:0/728820307 wait complete. 2024-09-10T21:52:01.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.932+0000 7fe0127bc740 1 -- start start 2024-09-10T21:52:01.934 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.932+0000 7fe0127bc740 1 Processor -- start 2024-09-10T21:52:01.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.933+0000 7fe0127bc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453549190 0x562453547890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.933+0000 7fe0127bc740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56245358daa0 0x5624535a0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.933+0000 7fe0127bc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562453547dd0 0x5624535a0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.933+0000 7fe0127bc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x562453451330 con 0x562453549190 2024-09-10T21:52:01.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.933+0000 7fe0127bc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5624534ae810 con 0x562453547dd0 2024-09-10T21:52:01.937 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.933+0000 7fe0127bc740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x56245321b620 con 0x56245358daa0 2024-09-10T21:52:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.933+0000 7fe0127bc740 10 client.0 _ll_register_callbacks cb 0x562453456420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:52:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.934+0000 7fe00ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453549190 0x562453547890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.934+0000 7fe00ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453549190 0x562453547890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53612/0 (socket says 172.21.15.7:53612) 2024-09-10T21:52:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.934+0000 7fe00ce00640 1 -- 172.21.15.7:0/3444092996 learned_addr learned my addr 172.21.15.7:0/3444092996 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.934+0000 7fdfffe00640 1 --2- 172.21.15.7:0/3444092996 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56245358daa0 0x5624535a0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.934+0000 7fe00d800640 1 --2- 172.21.15.7:0/3444092996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562453547dd0 0x5624535a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.934+0000 7fe00ce00640 1 -- 172.21.15.7:0/3444092996 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56245358daa0 msgr2=0x5624535a0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.934+0000 7fe00ce00640 1 --2- 172.21.15.7:0/3444092996 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56245358daa0 0x5624535a0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.934+0000 7fe00ce00640 1 -- 172.21.15.7:0/3444092996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562453547dd0 msgr2=0x5624535a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.934+0000 7fe00ce00640 1 --2- 172.21.15.7:0/3444092996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562453547dd0 0x5624535a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.934+0000 7fe00ce00640 1 -- 172.21.15.7:0/3444092996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56245320c850 con 0x562453549190 2024-09-10T21:52:01.938 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.934+0000 7fe00ce00640 1 --2- 172.21.15.7:0/3444092996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453549190 0x562453547890 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fdff800fe30 tx=0x7fdff8003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01986c7f75c0fa3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.934+0000 7fdff6000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:52:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.935+0000 7fdff6000640 1 -- 172.21.15.7:0/3444092996 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8025020 con 0x562453549190 2024-09-10T21:52:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.935+0000 7fe0127bc740 1 -- 172.21.15.7:0/3444092996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe000004e10 con 0x562453549190 2024-09-10T21:52:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.935+0000 7fdff6000640 1 -- 172.21.15.7:0/3444092996 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 5853+0+0 (secure 0 0 0) 0x7fdff8014660 con 0x562453549190 2024-09-10T21:52:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.935+0000 7fdff6000640 1 -- 172.21.15.7:0/3444092996 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7fdff802f050 con 0x562453549190 2024-09-10T21:52:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.935+0000 7fe0127bc740 10 client.8358 fetch_fsmap learned FSMap version 108 2024-09-10T21:52:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.935+0000 7fe0127bc740 1 -- 172.21.15.7:0/3444092996 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x5624532836b0 con 0x562453549190 2024-09-10T21:52:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.935+0000 7fdff6000640 1 -- 172.21.15.7:0/3444092996 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe000004e10 con 0x562453549190 2024-09-10T21:52:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.935+0000 7fdff6000640 1 -- 172.21.15.7:0/3444092996 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 108) ==== 3502+0+0 (secure 0 0 0) 0x7fdff8032020 con 0x562453549190 2024-09-10T21:52:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.935+0000 7fe0127bc740 10 client.8358 fetch_fsmap finished waiting for FSMap version 108 2024-09-10T21:52:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.935+0000 7fe0127bc740 10 client.8358 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:52:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.936+0000 7fe0127bc740 20 client.8358 populate_metadata read hostname 'smithi007' 2024-09-10T21:52:01.939 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.936+0000 7fe0127bc740 1 --2- 172.21.15.7:0/3444092996 >> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] conn(0x56245359e6e0 0x5624535aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.936+0000 7fe0127bc740 4 client.8358 mds_command: new command op to 236723 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:52:01.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.936+0000 7fe0127bc740 1 -- 172.21.15.7:0/3444092996 --> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624534bd520 con 0x56245359e6e0 2024-09-10T21:52:01.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.936+0000 7fdfffe00640 1 --2- 172.21.15.7:0/3444092996 >> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] conn(0x56245359e6e0 0x5624535aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.937+0000 7fdfffe00640 1 --2- 172.21.15.7:0/3444092996 >> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] conn(0x56245359e6e0 0x5624535aacd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5502c7a9fefb12f0 server_cookie=7dc5d380966e3e7 in_seq=0 out_seq=0 2024-09-10T21:52:01.940 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.937+0000 7fdff6000640 10 client.8358 ms_handle_connect on v2:172.21.15.133:6835/772257983 2024-09-10T21:52:01.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.937+0000 7fdff6000640 1 -- 172.21.15.7:0/3444092996 <== mds.0 v2:172.21.15.133:6835/772257983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5624534bd520 con 0x56245359e6e0 2024-09-10T21:52:01.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.937+0000 7fdff6000640 10 client.8358 handle_command_reply: tid=0 2024-09-10T21:52:01.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.938+0000 7fe0127bc740 1 client.8358 shutdown 2024-09-10T21:52:01.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.938+0000 7fe0127bc740 1 -- 172.21.15.7:0/3444092996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453549190 msgr2=0x562453547890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:01.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.938+0000 7fe0127bc740 1 --2- 172.21.15.7:0/3444092996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453549190 0x562453547890 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fdff800fe30 tx=0x7fdff8003760 comp rx=0 tx=0).stop 2024-09-10T21:52:01.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.938+0000 7fe0127bc740 1 -- 172.21.15.7:0/3444092996 shutdown_connections 2024-09-10T21:52:01.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.938+0000 7fe0127bc740 1 --2- 172.21.15.7:0/3444092996 >> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] conn(0x56245359e6e0 0x5624535aacd0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.938+0000 7fe0127bc740 1 --2- 172.21.15.7:0/3444092996 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562453547dd0 0x5624535a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.938+0000 7fe0127bc740 1 --2- 172.21.15.7:0/3444092996 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x56245358daa0 0x5624535a0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.941 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.938+0000 7fe0127bc740 1 --2- 172.21.15.7:0/3444092996 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453549190 0x562453547890 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.938+0000 7fdff6000640 0 client.8358 ms_handle_reset on v2:172.21.15.133:6835/772257983 2024-09-10T21:52:01.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.938+0000 7fe0127bc740 1 -- 172.21.15.7:0/3444092996 >> 172.21.15.7:0/3444092996 conn(0x562453548c10 msgr2=0x562453586620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:01.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.938+0000 7fe0127bc740 1 -- 172.21.15.7:0/3444092996 shutdown_connections 2024-09-10T21:52:01.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.938+0000 7fe0127bc740 1 -- 172.21.15.7:0/3444092996 wait complete. 2024-09-10T21:52:01.942 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.938+0000 7fe0127bc740 20 client.8358 trim_cache size 0 max 16384 2024-09-10T21:52:01.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.958+0000 7fe0127bc740 1 Processor -- start 2024-09-10T21:52:01.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.958+0000 7fe0127bc740 1 -- start start 2024-09-10T21:52:01.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.958+0000 7fe0127bc740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562453547dd0 0x5624535859c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.959+0000 7fe0127bc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562453549190 0x562453589500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.959+0000 7fe0127bc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245358daa0 0x562453589a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.959+0000 7fe0127bc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe008200360 con 0x56245358daa0 2024-09-10T21:52:01.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.959+0000 7fe0127bc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe0081f5890 con 0x562453549190 2024-09-10T21:52:01.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.959+0000 7fe0127bc740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe0081f8ab0 con 0x562453547dd0 2024-09-10T21:52:01.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.959+0000 7fe00d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245358daa0 0x562453589a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.959+0000 7fe00d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245358daa0 0x562453589a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53620/0 (socket says 172.21.15.7:53620) 2024-09-10T21:52:01.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.959+0000 7fe00d800640 1 -- 172.21.15.7:0/1862715953 learned_addr learned my addr 172.21.15.7:0/1862715953 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.959+0000 7fe00ce00640 1 --2- 172.21.15.7:0/1862715953 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562453547dd0 0x5624535859c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.959+0000 7fdfffe00640 1 --2- 172.21.15.7:0/1862715953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562453549190 0x562453589500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.959+0000 7fe00d800640 1 -- 172.21.15.7:0/1862715953 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562453547dd0 msgr2=0x5624535859c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.959+0000 7fe00d800640 1 --2- 172.21.15.7:0/1862715953 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562453547dd0 0x5624535859c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.959+0000 7fe00d800640 1 -- 172.21.15.7:0/1862715953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562453549190 msgr2=0x562453589500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.959+0000 7fe00d800640 1 --2- 172.21.15.7:0/1862715953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562453549190 0x562453589500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.959+0000 7fe00d800640 1 -- 172.21.15.7:0/1862715953 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56245358de80 con 0x56245358daa0 2024-09-10T21:52:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.960+0000 7fe00d800640 1 --2- 172.21.15.7:0/1862715953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245358daa0 0x562453589a40 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe000015000 tx=0x7fe00001a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc8f395d0bc49e9c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.960+0000 7fdffcc00640 1 -- 172.21.15.7:0/1862715953 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000028070 con 0x56245358daa0 2024-09-10T21:52:01.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.960+0000 7fdffcc00640 1 -- 172.21.15.7:0/1862715953 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe000010640 con 0x56245358daa0 2024-09-10T21:52:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.960+0000 7fdffcc00640 1 -- 172.21.15.7:0/1862715953 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe000014040 con 0x56245358daa0 2024-09-10T21:52:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.960+0000 7fe0127bc740 1 -- 172.21.15.7:0/1862715953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245358daa0 msgr2=0x562453589a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.960+0000 7fe0127bc740 1 --2- 172.21.15.7:0/1862715953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245358daa0 0x562453589a40 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7fe000015000 tx=0x7fe00001a040 comp rx=0 tx=0).stop 2024-09-10T21:52:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.960+0000 7fe0127bc740 1 -- 172.21.15.7:0/1862715953 shutdown_connections 2024-09-10T21:52:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.960+0000 7fe0127bc740 1 --2- 172.21.15.7:0/1862715953 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x562453549190 0x562453589500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.960+0000 7fe0127bc740 1 --2- 172.21.15.7:0/1862715953 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x56245358daa0 0x562453589a40 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.960+0000 7fe0127bc740 1 --2- 172.21.15.7:0/1862715953 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562453547dd0 0x5624535859c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.960+0000 7fe0127bc740 1 -- 172.21.15.7:0/1862715953 >> 172.21.15.7:0/1862715953 conn(0x562453548c10 msgr2=0x562453542d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.960+0000 7fe0127bc740 1 -- 172.21.15.7:0/1862715953 shutdown_connections 2024-09-10T21:52:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.961+0000 7fe0127bc740 1 -- 172.21.15.7:0/1862715953 wait complete. 2024-09-10T21:52:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.961+0000 7fe0127bc740 1 -- start start 2024-09-10T21:52:01.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.961+0000 7fe0127bc740 1 Processor -- start 2024-09-10T21:52:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.962+0000 7fe0127bc740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453547dd0 0x562453608a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.962+0000 7fe0127bc740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562453549190 0x5624535a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.962+0000 7fe0127bc740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56245358daa0 0x5624535a2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.962+0000 7fe0127bc740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x56245359e560 con 0x562453547dd0 2024-09-10T21:52:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.962+0000 7fe0127bc740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x562453571480 con 0x56245358daa0 2024-09-10T21:52:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.962+0000 7fe0127bc740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x56245359eb80 con 0x562453549190 2024-09-10T21:52:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.962+0000 7fe0127bc740 10 client.0 _ll_register_callbacks cb 0x7fdff800fa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:52:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.962+0000 7fe00ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453547dd0 0x562453608a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.963+0000 7fe00ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453547dd0 0x562453608a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53622/0 (socket says 172.21.15.7:53622) 2024-09-10T21:52:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.963+0000 7fe00ce00640 1 -- 172.21.15.7:0/2771889081 learned_addr learned my addr 172.21.15.7:0/2771889081 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.963+0000 7fe00ce00640 1 -- 172.21.15.7:0/2771889081 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562453549190 msgr2=0x5624535a25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.963+0000 7fe00ce00640 1 --2- 172.21.15.7:0/2771889081 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562453549190 0x5624535a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.963+0000 7fe00ce00640 1 -- 172.21.15.7:0/2771889081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56245358daa0 msgr2=0x5624535a2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.963+0000 7fe00d800640 1 --2- 172.21.15.7:0/2771889081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56245358daa0 0x5624535a2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.963+0000 7fe00ce00640 1 --2- 172.21.15.7:0/2771889081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56245358daa0 0x5624535a2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.963+0000 7fe00ce00640 1 -- 172.21.15.7:0/2771889081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562453608f90 con 0x562453547dd0 2024-09-10T21:52:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.963+0000 7fe00d800640 1 --2- 172.21.15.7:0/2771889081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56245358daa0 0x5624535a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:52:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.963+0000 7fe00ce00640 1 --2- 172.21.15.7:0/2771889081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453547dd0 0x562453608a50 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fdff800fe30 tx=0x7fdff800f780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30120b0ca0df4bb8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.963+0000 7fdff6000640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:52:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.963+0000 7fdff6000640 1 -- 172.21.15.7:0/2771889081 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdff8025030 con 0x562453547dd0 2024-09-10T21:52:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.964+0000 7fdff6000640 1 -- 172.21.15.7:0/2771889081 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 5853+0+0 (secure 0 0 0) 0x7fdff802a800 con 0x562453547dd0 2024-09-10T21:52:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.964+0000 7fe0127bc740 1 -- 172.21.15.7:0/2771889081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe000010640 con 0x562453547dd0 2024-09-10T21:52:01.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.964+0000 7fdff6000640 1 -- 172.21.15.7:0/2771889081 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7fdff803f050 con 0x562453547dd0 2024-09-10T21:52:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.964+0000 7fe0127bc740 10 client.8364 fetch_fsmap learned FSMap version 108 2024-09-10T21:52:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.964+0000 7fe0127bc740 1 -- 172.21.15.7:0/2771889081 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x562453585ee0 con 0x562453547dd0 2024-09-10T21:52:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.964+0000 7fdff6000640 1 -- 172.21.15.7:0/2771889081 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe000010640 con 0x562453547dd0 2024-09-10T21:52:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.964+0000 7fdff6000640 1 -- 172.21.15.7:0/2771889081 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 108) ==== 3502+0+0 (secure 0 0 0) 0x7fdff8042020 con 0x562453547dd0 2024-09-10T21:52:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.964+0000 7fe0127bc740 10 client.8364 fetch_fsmap finished waiting for FSMap version 108 2024-09-10T21:52:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.964+0000 7fe0127bc740 10 client.8364 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:52:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.964+0000 7fe0127bc740 20 client.8364 populate_metadata read hostname 'smithi007' 2024-09-10T21:52:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.964+0000 7fe0127bc740 1 --2- 172.21.15.7:0/2771889081 >> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] conn(0x562453612ba0 0x5624535a1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.964+0000 7fe0127bc740 4 client.8364 mds_command: new command op to 236723 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:52:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.964+0000 7fe0127bc740 1 -- 172.21.15.7:0/2771889081 --> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] -- command(tid 0: {"prefix": "status"}) -- 0x56245358bc20 con 0x562453612ba0 2024-09-10T21:52:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.965+0000 7fdfffe00640 1 --2- 172.21.15.7:0/2771889081 >> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] conn(0x562453612ba0 0x5624535a1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:01.969 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.965+0000 7fdfffe00640 1 --2- 172.21.15.7:0/2771889081 >> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] conn(0x562453612ba0 0x5624535a1120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9b2e34d98091829 server_cookie=11c10878e922b1d7 in_seq=0 out_seq=0 2024-09-10T21:52:01.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.965+0000 7fdff6000640 10 client.8364 ms_handle_connect on v2:172.21.15.133:6835/772257983 2024-09-10T21:52:01.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.966+0000 7fdff6000640 1 -- 172.21.15.7:0/2771889081 <== mds.0 v2:172.21.15.133:6835/772257983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+336 (crc 0 0 0) 0x56245358bc20 con 0x562453612ba0 2024-09-10T21:52:01.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.966+0000 7fdff6000640 10 client.8364 handle_command_reply: tid=0 2024-09-10T21:52:01.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.966+0000 7fe0127bc740 1 client.8364 shutdown 2024-09-10T21:52:01.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.966+0000 7fe0127bc740 1 -- 172.21.15.7:0/2771889081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453547dd0 msgr2=0x562453608a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:01.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.966+0000 7fe0127bc740 1 --2- 172.21.15.7:0/2771889081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453547dd0 0x562453608a50 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fdff800fe30 tx=0x7fdff800f780 comp rx=0 tx=0).stop 2024-09-10T21:52:01.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.966+0000 7fe0127bc740 1 -- 172.21.15.7:0/2771889081 shutdown_connections 2024-09-10T21:52:01.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.966+0000 7fe0127bc740 1 --2- 172.21.15.7:0/2771889081 >> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] conn(0x562453612ba0 0x5624535a1120 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.966+0000 7fe0127bc740 1 --2- 172.21.15.7:0/2771889081 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x56245358daa0 0x5624535a2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.966+0000 7fe0127bc740 1 --2- 172.21.15.7:0/2771889081 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x562453549190 0x5624535a25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.966+0000 7fdff6000640 0 client.8364 ms_handle_reset on v2:172.21.15.133:6835/772257983 2024-09-10T21:52:01.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.966+0000 7fe0127bc740 1 --2- 172.21.15.7:0/2771889081 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x562453547dd0 0x562453608a50 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.966+0000 7fe0127bc740 1 -- 172.21.15.7:0/2771889081 >> 172.21.15.7:0/2771889081 conn(0x562453548c10 msgr2=0x562453542030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:01.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.966+0000 7fe0127bc740 1 -- 172.21.15.7:0/2771889081 shutdown_connections 2024-09-10T21:52:01.970 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.966+0000 7fe0127bc740 1 -- 172.21.15.7:0/2771889081 wait complete. 2024-09-10T21:52:01.971 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.966+0000 7fe0127bc740 20 client.8364 trim_cache size 0 max 16384 2024-09-10T21:52:01.971 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:52:01.971 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2029451a-3c26-48c7-b874-0597197278ac", 2024-09-10T21:52:01.971 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-10T21:52:01.971 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 236723, 2024-09-10T21:52:01.971 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T21:52:01.971 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T21:52:01.971 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T21:52:01.971 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 4.7509483499999998, 2024-09-10T21:52:01.971 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 108, 2024-09-10T21:52:01.972 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 117, 2024-09-10T21:52:01.972 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 117, 2024-09-10T21:52:01.972 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 28.762760162999999 2024-09-10T21:52:01.972 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:52:01.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.967+0000 7fe00e200640 1 -- 172.21.15.7:0/3504671743 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdfe4077fa0 msgr2=0x7fdfe407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.967+0000 7fe00e200640 1 --2- 172.21.15.7:0/3504671743 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdfe4077fa0 0x7fdfe407a460 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fdff8004f60 tx=0x7fdff8007e30 comp rx=0 tx=0).stop 2024-09-10T21:52:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.967+0000 7fe00e200640 1 -- 172.21.15.7:0/3504671743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe008152d20 msgr2=0x7fe0080b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.967+0000 7fe00e200640 1 --2- 172.21.15.7:0/3504671743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe008152d20 0x7fe0080b4010 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fdff0034040 tx=0x7fdff00039e0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.967+0000 7fe00e200640 1 -- 172.21.15.7:0/3504671743 shutdown_connections 2024-09-10T21:52:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.967+0000 7fe00e200640 1 --2- 172.21.15.7:0/3504671743 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdfe4077fa0 0x7fdfe407a460 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.967+0000 7fe00e200640 1 --2- 172.21.15.7:0/3504671743 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe0081604d0 0x7fe00814e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.967+0000 7fe00e200640 1 --2- 172.21.15.7:0/3504671743 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe008152d20 0x7fe0080b4010 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.967+0000 7fe00e200640 1 --2- 172.21.15.7:0/3504671743 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe008070770 0x7fe0080ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.967+0000 7fe00e200640 1 -- 172.21.15.7:0/3504671743 >> 172.21.15.7:0/3504671743 conn(0x7fe00806ca60 msgr2=0x7fe0080ccf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.967+0000 7fe00e200640 1 -- 172.21.15.7:0/3504671743 shutdown_connections 2024-09-10T21:52:01.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:01.967+0000 7fe00e200640 1 -- 172.21.15.7:0/3504671743 wait complete. 2024-09-10T21:52:01.983 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:52:02.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.142+0000 7f21e6000640 1 -- 172.21.15.7:0/2344674208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21e006ce80 msgr2=0x7f21e006d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:02.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.142+0000 7f21e6000640 1 --2- 172.21.15.7:0/2344674208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21e006ce80 0x7f21e006d310 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f21d8005fd0 tx=0x7f21d802a8b0 comp rx=0 tx=0).stop 2024-09-10T21:52:02.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.143+0000 7f21e6000640 1 -- 172.21.15.7:0/2344674208 shutdown_connections 2024-09-10T21:52:02.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.143+0000 7f21e6000640 1 --2- 172.21.15.7:0/2344674208 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21e006ce80 0x7f21e006d310 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:02.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.143+0000 7f21e6000640 1 --2- 172.21.15.7:0/2344674208 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f21e00cf7d0 0x7f21e00acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:02.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.143+0000 7f21e6000640 1 -- 172.21.15.7:0/2344674208 >> 172.21.15.7:0/2344674208 conn(0x7f21e00b1ce0 msgr2=0x7f21e00b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:02.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.143+0000 7f21e6000640 1 -- 172.21.15.7:0/2344674208 shutdown_connections 2024-09-10T21:52:02.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.143+0000 7f21e6000640 1 -- 172.21.15.7:0/2344674208 wait complete. 2024-09-10T21:52:02.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.144+0000 7f21e6000640 1 Processor -- start 2024-09-10T21:52:02.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.144+0000 7f21e6000640 1 -- start start 2024-09-10T21:52:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.145+0000 7f21e6000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21e006ce80 0x7f21e01f2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.145+0000 7f21e6000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21e00cf7d0 0x7f21e00af810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.145+0000 7f21e6000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f21e015bce0 0x7f21e01548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.145+0000 7f21e6000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f21e015ffd0 con 0x7f21e006ce80 2024-09-10T21:52:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.145+0000 7f21e6000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f21e015fe50 con 0x7f21e00cf7d0 2024-09-10T21:52:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.145+0000 7f21e6000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f21e0160150 con 0x7f21e015bce0 2024-09-10T21:52:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.145+0000 7f21e4c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21e006ce80 0x7f21e01f2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.146+0000 7f21e4c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21e006ce80 0x7f21e01f2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53650/0 (socket says 172.21.15.7:53650) 2024-09-10T21:52:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.146+0000 7f21e4c00640 1 -- 172.21.15.7:0/3913082719 learned_addr learned my addr 172.21.15.7:0/3913082719 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:02.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.146+0000 7f21e5600640 1 --2- 172.21.15.7:0/3913082719 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f21e015bce0 0x7f21e01548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:02.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.146+0000 7f21d7e00640 1 --2- 172.21.15.7:0/3913082719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21e00cf7d0 0x7f21e00af810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:02.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.146+0000 7f21e4c00640 1 -- 172.21.15.7:0/3913082719 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f21e015bce0 msgr2=0x7f21e01548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:02.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.146+0000 7f21e4c00640 1 --2- 172.21.15.7:0/3913082719 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f21e015bce0 0x7f21e01548e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:02.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.146+0000 7f21e4c00640 1 -- 172.21.15.7:0/3913082719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21e00cf7d0 msgr2=0x7f21e00af810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:02.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.146+0000 7f21e4c00640 1 --2- 172.21.15.7:0/3913082719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21e00cf7d0 0x7f21e00af810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:02.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.146+0000 7f21e4c00640 1 -- 172.21.15.7:0/3913082719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e00b0310 con 0x7f21e006ce80 2024-09-10T21:52:02.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.146+0000 7f21e4c00640 1 --2- 172.21.15.7:0/3913082719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21e006ce80 0x7f21e01f2b10 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f21d000c3c0 tx=0x7f21d000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.147+0000 7f21dd600640 1 -- 172.21.15.7:0/3913082719 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0017070 con 0x7f21e006ce80 2024-09-10T21:52:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.147+0000 7f21dd600640 1 -- 172.21.15.7:0/3913082719 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21d00035f0 con 0x7f21e006ce80 2024-09-10T21:52:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.147+0000 7f21dd600640 1 -- 172.21.15.7:0/3913082719 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0003a80 con 0x7f21e006ce80 2024-09-10T21:52:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.147+0000 7f21e6000640 1 -- 172.21.15.7:0/3913082719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e01f5890 con 0x7f21e006ce80 2024-09-10T21:52:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.147+0000 7f21e6000640 1 -- 172.21.15.7:0/3913082719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e01f5e50 con 0x7f21e006ce80 2024-09-10T21:52:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.148+0000 7f21e6000640 1 -- 172.21.15.7:0/3913082719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21e01606e0 con 0x7f21e006ce80 2024-09-10T21:52:02.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.149+0000 7f21dd600640 1 -- 172.21.15.7:0/3913082719 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f21d0003c20 con 0x7f21e006ce80 2024-09-10T21:52:02.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.150+0000 7f21dd600640 1 --2- 172.21.15.7:0/3913082719 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f21bc077f70 0x7f21bc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:02.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.150+0000 7f21dd600640 1 -- 172.21.15.7:0/3913082719 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5853+0+0 (secure 0 0 0) 0x7f21d0099320 con 0x7f21e006ce80 2024-09-10T21:52:02.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.150+0000 7f21d7e00640 1 --2- 172.21.15.7:0/3913082719 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f21bc077f70 0x7f21bc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:02.151 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.151+0000 7f21d7e00640 1 --2- 172.21.15.7:0/3913082719 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f21bc077f70 0x7f21bc07a430 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f21c8005fd0 tx=0x7f21c800b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:02.155 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.154+0000 7f21dd600640 1 -- 172.21.15.7:0/3913082719 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f21d0061740 con 0x7f21e006ce80 2024-09-10T21:52:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.383+0000 7f21e6000640 1 -- 172.21.15.7:0/3913082719 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f21e00b2d60 con 0x7f21e006ce80 2024-09-10T21:52:02.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.384+0000 7f21dd600640 1 -- 172.21.15.7:0/3913082719 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v108) ==== 53+0+73 (secure 0 0 0) 0x7f21d0066170 con 0x7f21e006ce80 2024-09-10T21:52:02.385 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:52:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.386+0000 7f21e6000640 1 -- 172.21.15.7:0/3913082719 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f21bc077f70 msgr2=0x7f21bc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.386+0000 7f21e6000640 1 --2- 172.21.15.7:0/3913082719 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f21bc077f70 0x7f21bc07a430 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f21c8005fd0 tx=0x7f21c800b390 comp rx=0 tx=0).stop 2024-09-10T21:52:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.386+0000 7f21e6000640 1 -- 172.21.15.7:0/3913082719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21e006ce80 msgr2=0x7f21e01f2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.386+0000 7f21e6000640 1 --2- 172.21.15.7:0/3913082719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21e006ce80 0x7f21e01f2b10 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f21d000c3c0 tx=0x7f21d000c880 comp rx=0 tx=0).stop 2024-09-10T21:52:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.386+0000 7f21e6000640 1 -- 172.21.15.7:0/3913082719 shutdown_connections 2024-09-10T21:52:02.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.386+0000 7f21e6000640 1 --2- 172.21.15.7:0/3913082719 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f21bc077f70 0x7f21bc07a430 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.386+0000 7f21e6000640 1 --2- 172.21.15.7:0/3913082719 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f21e00cf7d0 0x7f21e00af810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.386+0000 7f21e6000640 1 --2- 172.21.15.7:0/3913082719 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f21e015bce0 0x7f21e01548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.386+0000 7f21e6000640 1 --2- 172.21.15.7:0/3913082719 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f21e006ce80 0x7f21e01f2b10 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.386+0000 7f21e6000640 1 -- 172.21.15.7:0/3913082719 >> 172.21.15.7:0/3913082719 conn(0x7f21e00b1ce0 msgr2=0x7f21e0157990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.386+0000 7f21e6000640 1 -- 172.21.15.7:0/3913082719 shutdown_connections 2024-09-10T21:52:02.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02.386+0000 7f21e6000640 1 -- 172.21.15.7:0/3913082719 wait complete. 2024-09-10T21:52:02.402 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:52:02.402 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:52:02.402 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-10T21:52:02.403 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:52:02.403 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:52:02.403 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:52:02.403 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:52:02.403 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:52:02.403 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:52:02.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:02.403 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-10T21:52:02.438 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:52:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:52:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:52:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:52:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T21:52:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:52:02.439 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:52:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T21:52:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-10T21:52:02.439 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:52:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T21:52:02.439 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-10T21:52:02.439 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:52:02.440 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-10T21:52:02.440 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-10T21:52:02.440 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T21:52:02.440 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42023sec preferred_lft 42023sec 2024-09-10T21:52:02.440 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-10T21:52:02.440 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:52:02.440 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:52:02.440 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-10T21:52:02.440 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-10T21:52:02.440 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:52:02.440 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff 2024-09-10T21:52:02.440 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:52:02.440 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:52:02.440 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::3c3d:dbff:fe02:4c15/64 scope link 2024-09-10T21:52:02.441 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:52:02.441 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:52:02.441 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether f6:f9:69:5a:86:f5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:52:02.441 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f4f9:69ff:fe5a:86f5/64 scope link 2024-09-10T21:52:02.441 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:52:02.441 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:52:02.441 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:52:02.441 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::433:5cff:fe44:4a5b/64 scope link 2024-09-10T21:52:02.441 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:52:02.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:02.441 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:52:02.534 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:52:02.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:02.537 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-10T21:52:02.573 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-10T21:52:02.573 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:52:02.573 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T21:52:02.574 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T21:52:02.574 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:52:02.574 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:52:02.574 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:52:02.574 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:52:02.574 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:52:02.574 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:52:02.574 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:52:02.692 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:52:02.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:52:02.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:02.772 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:52:02.808 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:52:02.808 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:52:02.808 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:52:02.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:02.808 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-10T21:52:02.904 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:52:02.941 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:52:02.976 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:52:02.977 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:52:02.977 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:52:03.050 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:52:03.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:03.051 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:52:03.179 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:52:03.179 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:52:03.207 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:52:03.207 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:52:03.207 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.133:3300,172.21.15.133:3301" 2024-09-10T21:52:03.207 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@2029451a-3c26-48c7-b874-0597197278ac.cephfs=/ 2024-09-10T21:52:03.207 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.133:3300/172.21.15.133:3301" will pass to kernel 2024-09-10T21:52:03.209 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T21:52:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:52:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:52:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:52:03.209 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:52:03.212 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:52:03.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:03.212 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:52:03.267 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:52:03.347 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:52:03.350 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:52:03.358 INFO:teuthology.orchestra.run.smithi007.stdout:client7705 2024-09-10T21:52:03.358 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:52:03.358 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:52:03.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:03.394 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-09-10T21:52:03.475 INFO:teuthology.orchestra.run.smithi007.stdout:./ 2024-09-10T21:52:03.475 INFO:teuthology.orchestra.run.smithi007.stdout:./parent 2024-09-10T21:52:03.476 INFO:teuthology.orchestra.run.smithi007.stdout:./parent/flushed 2024-09-10T21:52:03.476 INFO:teuthology.orchestra.run.smithi007.stdout:./parent/flushed/charlie 2024-09-10T21:52:03.476 INFO:teuthology.orchestra.run.smithi007.stdout:./parent/flushed/alpha 2024-09-10T21:52:03.476 INFO:teuthology.orchestra.run.smithi007.stdout:./parent/unflushed 2024-09-10T21:52:03.476 INFO:teuthology.orchestra.run.smithi007.stdout:./parent/unflushed/jfile 2024-09-10T21:52:03.477 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:52:03.477 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:52:03.477 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:52:03.477 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:52:03.477 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-09-10T21:52:03.477 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:52:03.618 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:52:03.621 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:52:03.629 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:52:03.629 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:52:03.629 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:52:03.629 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:52:03.629 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-09-10T21:52:03.629 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:52:03.770 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627776 2024-09-10T21:52:03.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:52:03.781 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:52:03.781 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:52:03.782 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:52:03.782 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:52:03.782 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-09-10T21:52:03.782 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:52:03.922 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627777 2024-09-10T21:52:03.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:52:03.933 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:52:03.933 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:52:03.933 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:52:03.933 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:52:03.933 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-09-10T21:52:03.933 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:52:04.073 INFO:teuthology.orchestra.run.smithi007.stdout:1099511628280 2024-09-10T21:52:04.076 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:52:04.085 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:52:04.085 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:52:04.085 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:52:04.085 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:52:04.085 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-09-10T21:52:04.085 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:52:04.225 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627778 2024-09-10T21:52:04.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:52:04.236 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:52:04.236 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:52:04.236 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:52:04.237 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:52:04.237 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-09-10T21:52:04.237 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:52:04.376 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627779 2024-09-10T21:52:04.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:52:04.388 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:52:04.388 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:52:04.388 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:52:04.388 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:52:04.388 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-09-10T21:52:04.388 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:52:04.457 INFO:tasks.ceph.mds.d.smithi133.stderr:2024-09-10T21:52:04.457+0000 7f3843800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:52:04.528 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627780 2024-09-10T21:52:04.531 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:52:04.539 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:52:04.539 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:52:04.616 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:52:04.617 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:52:04.692 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:52:04.893 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.892+0000 7f4696800640 1 -- 172.21.15.7:0/2362053494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469015ef90 msgr2=0x7f469006e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:04.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.892+0000 7f4696800640 1 --2- 172.21.15.7:0/2362053494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469015ef90 0x7f469006e810 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f4678005fd0 tx=0x7f467802a5d0 comp rx=0 tx=0).stop 2024-09-10T21:52:04.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.892+0000 7f4696800640 1 -- 172.21.15.7:0/2362053494 shutdown_connections 2024-09-10T21:52:04.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.892+0000 7f4696800640 1 --2- 172.21.15.7:0/2362053494 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469015ef90 0x7f469006e810 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:04.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.892+0000 7f4696800640 1 --2- 172.21.15.7:0/2362053494 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4690074440 0x7f46900ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:04.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.893+0000 7f4696800640 1 -- 172.21.15.7:0/2362053494 >> 172.21.15.7:0/2362053494 conn(0x7f469006ca60 msgr2=0x7f469006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:04.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.893+0000 7f4696800640 1 -- 172.21.15.7:0/2362053494 shutdown_connections 2024-09-10T21:52:04.894 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.893+0000 7f4696800640 1 -- 172.21.15.7:0/2362053494 wait complete. 2024-09-10T21:52:04.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.894+0000 7f4696800640 1 Processor -- start 2024-09-10T21:52:04.895 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.894+0000 7f4696800640 1 -- start start 2024-09-10T21:52:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.895+0000 7f4696800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ed50 0x7f4690158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.895+0000 7f4696800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4690074440 0x7f4690156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.895+0000 7f4696800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f469015ef90 0x7f469015d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.895+0000 7f4696800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4690162240 con 0x7f469006ed50 2024-09-10T21:52:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.895+0000 7f4696800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f46901623c0 con 0x7f469015ef90 2024-09-10T21:52:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.895+0000 7f4696800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4690162540 con 0x7f4690074440 2024-09-10T21:52:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.895+0000 7f468f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ed50 0x7f4690158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.896+0000 7f468f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ed50 0x7f4690158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37882/0 (socket says 172.21.15.7:37882) 2024-09-10T21:52:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.896+0000 7f468f400640 1 -- 172.21.15.7:0/233985565 learned_addr learned my addr 172.21.15.7:0/233985565 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.896+0000 7f468fe00640 1 --2- 172.21.15.7:0/233985565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f469015ef90 0x7f469015d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.896+0000 7f4686a00640 1 --2- 172.21.15.7:0/233985565 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4690074440 0x7f4690156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:04.897 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.896+0000 7f468f400640 1 -- 172.21.15.7:0/233985565 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4690074440 msgr2=0x7f4690156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.896+0000 7f468f400640 1 --2- 172.21.15.7:0/233985565 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4690074440 0x7f4690156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.896+0000 7f468f400640 1 -- 172.21.15.7:0/233985565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f469015ef90 msgr2=0x7f469015d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.896+0000 7f468f400640 1 --2- 172.21.15.7:0/233985565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f469015ef90 0x7f469015d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.896+0000 7f468f400640 1 -- 172.21.15.7:0/233985565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46901f8230 con 0x7f469006ed50 2024-09-10T21:52:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.896+0000 7f468f400640 1 --2- 172.21.15.7:0/233985565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ed50 0x7f4690158200 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f468000c920 tx=0x7f468000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:04.898 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.896+0000 7f4687e00640 1 -- 172.21.15.7:0/233985565 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680007c70 con 0x7f469006ed50 2024-09-10T21:52:04.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.897+0000 7f4687e00640 1 -- 172.21.15.7:0/233985565 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f468000f450 con 0x7f469006ed50 2024-09-10T21:52:04.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.897+0000 7f4687e00640 1 -- 172.21.15.7:0/233985565 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468000f8a0 con 0x7f469006ed50 2024-09-10T21:52:04.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.897+0000 7f4696800640 1 -- 172.21.15.7:0/233985565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46901f8510 con 0x7f469006ed50 2024-09-10T21:52:04.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.897+0000 7f4696800640 1 -- 172.21.15.7:0/233985565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46901f8ad0 con 0x7f469006ed50 2024-09-10T21:52:04.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.898+0000 7f4696800640 1 -- 172.21.15.7:0/233985565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46900ad850 con 0x7f469006ed50 2024-09-10T21:52:04.899 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.899+0000 7f4687e00640 1 -- 172.21.15.7:0/233985565 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f468000fa40 con 0x7f469006ed50 2024-09-10T21:52:04.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.900+0000 7f4687e00640 1 --2- 172.21.15.7:0/233985565 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f466c077f70 0x7f466c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:04.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.900+0000 7f4687e00640 1 -- 172.21.15.7:0/233985565 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5853+0+0 (secure 0 0 0) 0x7f468009a740 con 0x7f469006ed50 2024-09-10T21:52:04.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.900+0000 7f4686a00640 1 --2- 172.21.15.7:0/233985565 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f466c077f70 0x7f466c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:04.901 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.901+0000 7f4686a00640 1 --2- 172.21.15.7:0/233985565 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f466c077f70 0x7f466c07a430 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4678033000 tx=0x7f4678004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:04.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:04.904+0000 7f4687e00640 1 -- 172.21.15.7:0/233985565 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4680014030 con 0x7f469006ed50 2024-09-10T21:52:05.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.110+0000 7f4696800640 1 -- 172.21.15.7:0/233985565 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f4690154c10 con 0x7f469006ed50 2024-09-10T21:52:05.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.111+0000 7f4687e00640 1 -- 172.21.15.7:0/233985565 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v117) ==== 80+0+124 (secure 0 0 0) 0x7f4680063110 con 0x7f469006ed50 2024-09-10T21:52:05.112 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.133:6837/3764354952 2024-09-11T21:51:56.206885+0000 2024-09-10T21:52:05.112 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.133:6834/3764354952 2024-09-11T21:51:56.206885+0000 2024-09-10T21:52:05.112 INFO:teuthology.orchestra.run.smithi007.stderr:listed 2 entries 2024-09-10T21:52:05.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.113+0000 7f4696800640 1 -- 172.21.15.7:0/233985565 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f466c077f70 msgr2=0x7f466c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:05.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.113+0000 7f4696800640 1 --2- 172.21.15.7:0/233985565 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f466c077f70 0x7f466c07a430 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f4678033000 tx=0x7f4678004730 comp rx=0 tx=0).stop 2024-09-10T21:52:05.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.113+0000 7f4696800640 1 -- 172.21.15.7:0/233985565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ed50 msgr2=0x7f4690158200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:05.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.113+0000 7f4696800640 1 --2- 172.21.15.7:0/233985565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ed50 0x7f4690158200 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f468000c920 tx=0x7f468000cde0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.113+0000 7f4696800640 1 -- 172.21.15.7:0/233985565 shutdown_connections 2024-09-10T21:52:05.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.113+0000 7f4696800640 1 --2- 172.21.15.7:0/233985565 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f466c077f70 0x7f466c07a430 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.113+0000 7f4696800640 1 --2- 172.21.15.7:0/233985565 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f469015ef90 0x7f469015d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.113+0000 7f4696800640 1 --2- 172.21.15.7:0/233985565 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4690074440 0x7f4690156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.113+0000 7f4696800640 1 --2- 172.21.15.7:0/233985565 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f469006ed50 0x7f4690158200 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.113+0000 7f4696800640 1 -- 172.21.15.7:0/233985565 >> 172.21.15.7:0/233985565 conn(0x7f469006ca60 msgr2=0x7f469015c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:05.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.113+0000 7f4696800640 1 -- 172.21.15.7:0/233985565 shutdown_connections 2024-09-10T21:52:05.115 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.113+0000 7f4696800640 1 -- 172.21.15.7:0/233985565 wait complete. 2024-09-10T21:52:05.129 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:52:05.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:05.130 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:52:05.190 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:52:05.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:05.191 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:52:05.226 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:52:05.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.426+0000 7faf02000640 1 -- 172.21.15.7:0/1668074793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faefc159910 msgr2=0x7faefc15d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:05.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.426+0000 7faf02000640 1 --2- 172.21.15.7:0/1668074793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faefc159910 0x7faefc15d090 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7faeec009830 tx=0x7faeec02a510 comp rx=0 tx=0).stop 2024-09-10T21:52:05.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.426+0000 7faf02000640 1 -- 172.21.15.7:0/1668074793 shutdown_connections 2024-09-10T21:52:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.426+0000 7faf02000640 1 --2- 172.21.15.7:0/1668074793 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faefc0cddc0 0x7faefc072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.426+0000 7faf02000640 1 --2- 172.21.15.7:0/1668074793 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faefc159910 0x7faefc15d090 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.426+0000 7faf02000640 1 -- 172.21.15.7:0/1668074793 >> 172.21.15.7:0/1668074793 conn(0x7faefc06ca60 msgr2=0x7faefc06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.426+0000 7faf02000640 1 -- 172.21.15.7:0/1668074793 shutdown_connections 2024-09-10T21:52:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.426+0000 7faf02000640 1 -- 172.21.15.7:0/1668074793 wait complete. 2024-09-10T21:52:05.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.428+0000 7faf02000640 1 Processor -- start 2024-09-10T21:52:05.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.428+0000 7faf02000640 1 -- start start 2024-09-10T21:52:05.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.429+0000 7faf02000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faefc06e810 0x7faefc151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:05.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.429+0000 7faf02000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faefc0cddc0 0x7faefc0b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:05.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.429+0000 7faf02000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faefc159910 0x7faefc14c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:05.430 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.429+0000 7faf02000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7faefc073800 con 0x7faefc159910 2024-09-10T21:52:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.429+0000 7faf02000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7faefc073680 con 0x7faefc06e810 2024-09-10T21:52:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.429+0000 7faf02000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7faefc073980 con 0x7faefc0cddc0 2024-09-10T21:52:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.429+0000 7faefbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faefc159910 0x7faefc14c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.429+0000 7faefbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faefc159910 0x7faefc14c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37894/0 (socket says 172.21.15.7:37894) 2024-09-10T21:52:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.429+0000 7faefbe00640 1 -- 172.21.15.7:0/2009644522 learned_addr learned my addr 172.21.15.7:0/2009644522 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.429+0000 7faefb400640 1 --2- 172.21.15.7:0/2009644522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faefc06e810 0x7faefc151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.429+0000 7faefaa00640 1 --2- 172.21.15.7:0/2009644522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faefc0cddc0 0x7faefc0b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.430+0000 7faefbe00640 1 -- 172.21.15.7:0/2009644522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faefc0cddc0 msgr2=0x7faefc0b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.430+0000 7faefbe00640 1 --2- 172.21.15.7:0/2009644522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faefc0cddc0 0x7faefc0b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.430+0000 7faefbe00640 1 -- 172.21.15.7:0/2009644522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faefc06e810 msgr2=0x7faefc151900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.430+0000 7faefbe00640 1 --2- 172.21.15.7:0/2009644522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faefc06e810 0x7faefc151900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.430+0000 7faefbe00640 1 -- 172.21.15.7:0/2009644522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faefc0b21f0 con 0x7faefc159910 2024-09-10T21:52:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.430+0000 7faefbe00640 1 --2- 172.21.15.7:0/2009644522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faefc159910 0x7faefc14c840 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7faef400efd0 tx=0x7faef400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.430+0000 7faef3400640 1 -- 172.21.15.7:0/2009644522 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef4003b70 con 0x7faefc159910 2024-09-10T21:52:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.430+0000 7faef3400640 1 -- 172.21.15.7:0/2009644522 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faef4003e00 con 0x7faefc159910 2024-09-10T21:52:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.430+0000 7faef3400640 1 -- 172.21.15.7:0/2009644522 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef4004280 con 0x7faefc159910 2024-09-10T21:52:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.430+0000 7faf02000640 1 -- 172.21.15.7:0/2009644522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faefc0afe70 con 0x7faefc159910 2024-09-10T21:52:05.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.431+0000 7faf02000640 1 -- 172.21.15.7:0/2009644522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faefc0b0430 con 0x7faefc159910 2024-09-10T21:52:05.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.432+0000 7faf02000640 1 -- 172.21.15.7:0/2009644522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faefc0b0630 con 0x7faefc159910 2024-09-10T21:52:05.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.433+0000 7faef3400640 1 -- 172.21.15.7:0/2009644522 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7faef4004420 con 0x7faefc159910 2024-09-10T21:52:05.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.434+0000 7faef3400640 1 --2- 172.21.15.7:0/2009644522 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7faedc077f70 0x7faedc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:05.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.434+0000 7faef3400640 1 -- 172.21.15.7:0/2009644522 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5853+0+0 (secure 0 0 0) 0x7faef409ace0 con 0x7faefc159910 2024-09-10T21:52:05.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.434+0000 7faefb400640 1 --2- 172.21.15.7:0/2009644522 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7faedc077f70 0x7faedc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:05.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.435+0000 7faefb400640 1 --2- 172.21.15.7:0/2009644522 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7faedc077f70 0x7faedc07a430 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7faeec034000 tx=0x7faeec004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:05.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.438+0000 7faef3400640 1 -- 172.21.15.7:0/2009644522 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7faef40630f0 con 0x7faefc159910 2024-09-10T21:52:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.670+0000 7faf02000640 1 -- 172.21.15.7:0/2009644522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faefc15e8e0 con 0x7faefc159910 2024-09-10T21:52:05.671 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.671+0000 7faef3400640 1 -- 172.21.15.7:0/2009644522 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5236 (secure 0 0 0) 0x7faef4067b20 con 0x7faefc159910 2024-09-10T21:52:05.672 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:05.672 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":108,"btime":"2024-09-10T21:52:00:219026+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3012572644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3012572644},{"type":"v1","addr":"172.21.15.7:6834","nonce":3012572644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7660,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2492331385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2492331385},{"type":"v1","addr":"172.21.15.133:6837","nonce":2492331385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":8094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/148356862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":148356862},{"type":"v1","addr":"172.21.15.7:6835","nonce":148356862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:51:37.913965+0000","modified":"2024-09-10T21:52:00.219020+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":117,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236723},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236723":{"gid":236723,"name":"d","rank":0,"incarnation":105,"state":"up:active","state_seq":10,"addr":"172.21.15.133:6836/772257983","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":772257983},{"type":"v1","addr":"172.21.15.133:6836","nonce":772257983}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":236723,"qdb_cluster":[236723]},"id":8}]} 2024-09-10T21:52:05.672 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 108 2024-09-10T21:52:05.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.673+0000 7faf02000640 1 -- 172.21.15.7:0/2009644522 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7faedc077f70 msgr2=0x7faedc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:05.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.673+0000 7faf02000640 1 --2- 172.21.15.7:0/2009644522 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7faedc077f70 0x7faedc07a430 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7faeec034000 tx=0x7faeec004730 comp rx=0 tx=0).stop 2024-09-10T21:52:05.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.673+0000 7faf02000640 1 -- 172.21.15.7:0/2009644522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faefc159910 msgr2=0x7faefc14c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:05.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.673+0000 7faf02000640 1 --2- 172.21.15.7:0/2009644522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faefc159910 0x7faefc14c840 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7faef400efd0 tx=0x7faef400c6a0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.673+0000 7faf02000640 1 -- 172.21.15.7:0/2009644522 shutdown_connections 2024-09-10T21:52:05.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.673+0000 7faf02000640 1 --2- 172.21.15.7:0/2009644522 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7faedc077f70 0x7faedc07a430 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.673+0000 7faf02000640 1 --2- 172.21.15.7:0/2009644522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7faefc159910 0x7faefc14c840 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.674 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.673+0000 7faf02000640 1 --2- 172.21.15.7:0/2009644522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7faefc0cddc0 0x7faefc0b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.673+0000 7faf02000640 1 --2- 172.21.15.7:0/2009644522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7faefc06e810 0x7faefc151900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.673+0000 7faf02000640 1 -- 172.21.15.7:0/2009644522 >> 172.21.15.7:0/2009644522 conn(0x7faefc06ca60 msgr2=0x7faefc0ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:05.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.673+0000 7faf02000640 1 -- 172.21.15.7:0/2009644522 shutdown_connections 2024-09-10T21:52:05.675 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.673+0000 7faf02000640 1 -- 172.21.15.7:0/2009644522 wait complete. 2024-09-10T21:52:05.689 DEBUG:teuthology.orchestra.run.smithi133:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json tag path /parent mytag123 2024-09-10T21:52:05.875 INFO:teuthology.orchestra.run.smithi133.stdout:{"return_code":0,"scrub_tag":"mytag123","mode":"asynchronous"} 2024-09-10T21:52:05.876 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag123" } 2024-09-10T21:52:05.876 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2024-09-10T21:52:05.953 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.953+0000 7f4f2331bf40 1 Processor -- start 2024-09-10T21:52:05.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.953+0000 7f4f2331bf40 1 -- start start 2024-09-10T21:52:05.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.953+0000 7f4f2331bf40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a331c0d520 0x55a331ce1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:05.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.953+0000 7f4f2331bf40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a331c3a8b0 0x55a331d68b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.953+0000 7f4f2331bf40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a331c3ac90 0x55a331ce0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.953+0000 7f4f2331bf40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a331bb51d0 con 0x55a331c0d520 2024-09-10T21:52:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.953+0000 7f4f2331bf40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a331bc1f90 con 0x55a331c3a8b0 2024-09-10T21:52:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.953+0000 7f4f2331bf40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55a331b56c60 con 0x55a331c3ac90 2024-09-10T21:52:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.954+0000 7f4f1fa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a331c0d520 0x55a331ce1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.954+0000 7f4f1fa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a331c0d520 0x55a331ce1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37908/0 (socket says 172.21.15.7:37908) 2024-09-10T21:52:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.954+0000 7f4f1fa00640 1 -- 172.21.15.7:0/3907231045 learned_addr learned my addr 172.21.15.7:0/3907231045 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.954+0000 7f4f20400640 1 --2- 172.21.15.7:0/3907231045 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a331c3ac90 0x55a331ce0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:05.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.954+0000 7f4f1f000640 1 --2- 172.21.15.7:0/3907231045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a331c3a8b0 0x55a331d68b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:05.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.954+0000 7f4f1fa00640 1 -- 172.21.15.7:0/3907231045 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a331c3ac90 msgr2=0x55a331ce0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.954+0000 7f4f1fa00640 1 --2- 172.21.15.7:0/3907231045 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a331c3ac90 0x55a331ce0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.954+0000 7f4f1fa00640 1 -- 172.21.15.7:0/3907231045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a331c3a8b0 msgr2=0x55a331d68b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.954+0000 7f4f1fa00640 1 --2- 172.21.15.7:0/3907231045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a331c3a8b0 0x55a331d68b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.954+0000 7f4f1fa00640 1 -- 172.21.15.7:0/3907231045 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a331c49f30 con 0x55a331c0d520 2024-09-10T21:52:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.955+0000 7f4f1fa00640 1 --2- 172.21.15.7:0/3907231045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a331c0d520 0x55a331ce1580 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f4f10001df0 tx=0x7f4f1000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a9c4b2d8bbd22f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.955+0000 7f4f1dc00640 1 -- 172.21.15.7:0/3907231045 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1000bde0 con 0x55a331c0d520 2024-09-10T21:52:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.955+0000 7f4f1dc00640 1 -- 172.21.15.7:0/3907231045 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f1000f400 con 0x55a331c0d520 2024-09-10T21:52:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.955+0000 7f4f1dc00640 1 -- 172.21.15.7:0/3907231045 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1000f850 con 0x55a331c0d520 2024-09-10T21:52:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.955+0000 7f4f2331bf40 1 -- 172.21.15.7:0/3907231045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a331c0d520 msgr2=0x55a331ce1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.955+0000 7f4f2331bf40 1 --2- 172.21.15.7:0/3907231045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a331c0d520 0x55a331ce1580 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f4f10001df0 tx=0x7f4f1000b5b0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.955+0000 7f4f2331bf40 1 -- 172.21.15.7:0/3907231045 shutdown_connections 2024-09-10T21:52:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.955+0000 7f4f2331bf40 1 --2- 172.21.15.7:0/3907231045 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a331c3a8b0 0x55a331d68b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.955+0000 7f4f2331bf40 1 --2- 172.21.15.7:0/3907231045 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a331c3ac90 0x55a331ce0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.955+0000 7f4f2331bf40 1 --2- 172.21.15.7:0/3907231045 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a331c0d520 0x55a331ce1580 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.955+0000 7f4f2331bf40 1 -- 172.21.15.7:0/3907231045 >> 172.21.15.7:0/3907231045 conn(0x55a331c551c0 msgr2=0x55a331ce0330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:05.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.955+0000 7f4f2331bf40 1 -- 172.21.15.7:0/3907231045 shutdown_connections 2024-09-10T21:52:05.958 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.955+0000 7f4f2331bf40 1 -- 172.21.15.7:0/3907231045 wait complete. 2024-09-10T21:52:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.956+0000 7f4f2331bf40 1 Processor -- start 2024-09-10T21:52:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.956+0000 7f4f2331bf40 1 -- start start 2024-09-10T21:52:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.957+0000 7f4f2331bf40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a331c0d520 0x55a331d68240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.957+0000 7f4f2331bf40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a331c3a8b0 0x55a331cc6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.957+0000 7f4f2331bf40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a331c3ac90 0x55a331d79440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.957+0000 7f4f2331bf40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55a331cc1240 con 0x55a331c3ac90 2024-09-10T21:52:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.957+0000 7f4f2331bf40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55a331c396e0 con 0x55a331c0d520 2024-09-10T21:52:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.957+0000 7f4f2331bf40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55a331cd5530 con 0x55a331c3a8b0 2024-09-10T21:52:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.957+0000 7f4f20400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a331c3ac90 0x55a331d79440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.957+0000 7f4f20400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a331c3ac90 0x55a331d79440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37914/0 (socket says 172.21.15.7:37914) 2024-09-10T21:52:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.958+0000 7f4f20400640 1 -- 172.21.15.7:0/2917193583 learned_addr learned my addr 172.21.15.7:0/2917193583 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:05.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.958+0000 7f4f1f000640 1 --2- 172.21.15.7:0/2917193583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a331c3a8b0 0x55a331cc6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.958+0000 7f4f1fa00640 1 --2- 172.21.15.7:0/2917193583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a331c0d520 0x55a331d68240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.958+0000 7f4f20400640 1 -- 172.21.15.7:0/2917193583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a331c3a8b0 msgr2=0x55a331cc6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.958+0000 7f4f20400640 1 --2- 172.21.15.7:0/2917193583 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55a331c3a8b0 0x55a331cc6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.958+0000 7f4f20400640 1 -- 172.21.15.7:0/2917193583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a331c0d520 msgr2=0x55a331d68240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.958+0000 7f4f20400640 1 --2- 172.21.15.7:0/2917193583 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55a331c0d520 0x55a331d68240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.958+0000 7f4f20400640 1 -- 172.21.15.7:0/2917193583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a331cc5580 con 0x55a331c3ac90 2024-09-10T21:52:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.958+0000 7f4f20400640 1 --2- 172.21.15.7:0/2917193583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a331c3ac90 0x55a331d79440 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f4f18025fd0 tx=0x7f4f18055be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.958+0000 7f4f17e00640 1 -- 172.21.15.7:0/2917193583 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f18065070 con 0x55a331c3ac90 2024-09-10T21:52:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.958+0000 7f4f17e00640 1 -- 172.21.15.7:0/2917193583 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f18055e40 con 0x55a331c3ac90 2024-09-10T21:52:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.958+0000 7f4f2331bf40 1 -- 172.21.15.7:0/2917193583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a331cc5840 con 0x55a331c3ac90 2024-09-10T21:52:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.959+0000 7f4f17e00640 1 -- 172.21.15.7:0/2917193583 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f18004020 con 0x55a331c3ac90 2024-09-10T21:52:05.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.959+0000 7f4f2331bf40 1 -- 172.21.15.7:0/2917193583 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a331cc5e00 con 0x55a331c3ac90 2024-09-10T21:52:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.961+0000 7f4f17e00640 1 -- 172.21.15.7:0/2917193583 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4f18071030 con 0x55a331c3ac90 2024-09-10T21:52:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.961+0000 7f4f17e00640 1 --2- 172.21.15.7:0/2917193583 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4f00076950 0x7f4f00078e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:05.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.962+0000 7f4f17e00640 1 -- 172.21.15.7:0/2917193583 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5853+0+0 (secure 0 0 0) 0x7f4f180dd690 con 0x55a331c3ac90 2024-09-10T21:52:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.962+0000 7f4f2331bf40 1 --2- 172.21.15.7:0/2917193583 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x55a331d79e30 0x55a331d6aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.962+0000 7f4f1fa00640 1 --2- 172.21.15.7:0/2917193583 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4f00076950 0x7f4f00078e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.962+0000 7f4f2331bf40 1 -- 172.21.15.7:0/2917193583 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:1 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) -- 0x55a331cc0b30 con 0x55a331d79e30 2024-09-10T21:52:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.962+0000 7f4f1f000640 1 --2- 172.21.15.7:0/2917193583 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x55a331d79e30 0x55a331d6aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:05.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.963+0000 7f4f1fa00640 1 --2- 172.21.15.7:0/2917193583 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4f00076950 0x7f4f00078e10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4f10001d90 tx=0x7f4f10000ed0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:05.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.963+0000 7f4f1f000640 1 --2- 172.21.15.7:0/2917193583 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x55a331d79e30 0x55a331d6aa60 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.963+0000 7f4f1f000640 1 -- 172.21.15.7:0/2917193583 <== osd.0 v2:172.21.15.7:6808/651402908 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) ==== 164+0+0 (crc 0 0 0) 0x7f4f0c011770 con 0x55a331d79e30 2024-09-10T21:52:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:error getting xattr cephfs_data/100000001f7.00000000/scrub_tag: (61) No data available 2024-09-10T21:52:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.963+0000 7f4f2331bf40 1 -- 172.21.15.7:0/2917193583 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x55a331d79e30 msgr2=0x55a331d6aa60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.963+0000 7f4f2331bf40 1 --2- 172.21.15.7:0/2917193583 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x55a331d79e30 0x55a331d6aa60 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.963+0000 7f4f2331bf40 1 -- 172.21.15.7:0/2917193583 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4f00076950 msgr2=0x7f4f00078e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.963+0000 7f4f2331bf40 1 --2- 172.21.15.7:0/2917193583 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4f00076950 0x7f4f00078e10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4f10001d90 tx=0x7f4f10000ed0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.963+0000 7f4f2331bf40 1 -- 172.21.15.7:0/2917193583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a331c3ac90 msgr2=0x55a331d79440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.963+0000 7f4f2331bf40 1 --2- 172.21.15.7:0/2917193583 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55a331c3ac90 0x55a331d79440 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f4f18025fd0 tx=0x7f4f18055be0 comp rx=0 tx=0).stop 2024-09-10T21:52:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.963+0000 7f4f20400640 1 -- 172.21.15.7:0/2917193583 reap_dead start 2024-09-10T21:52:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.963+0000 7f4f2331bf40 1 -- 172.21.15.7:0/2917193583 shutdown_connections 2024-09-10T21:52:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.963+0000 7f4f2331bf40 1 -- 172.21.15.7:0/2917193583 >> 172.21.15.7:0/2917193583 conn(0x55a331c551c0 msgr2=0x55a331cdfb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.964+0000 7f4f2331bf40 1 -- 172.21.15.7:0/2917193583 shutdown_connections 2024-09-10T21:52:05.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:05.964+0000 7f4f2331bf40 1 -- 172.21.15.7:0/2917193583 wait complete. 2024-09-10T21:52:05.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:52:05.969 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2024-09-10T21:52:06.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.045+0000 7f661b929f40 1 Processor -- start 2024-09-10T21:52:06.046 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.045+0000 7f661b929f40 1 -- start start 2024-09-10T21:52:06.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.046+0000 7f661b929f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c68e962e10 0x55c68ea36dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.046+0000 7f661b929f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c68e9908b0 0x55c68eabcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.046+0000 7f661b929f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c68e990c90 0x55c68eabd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.047 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.046+0000 7f661b929f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c68e8acc60 con 0x55c68e962e10 2024-09-10T21:52:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.046+0000 7f661b929f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c68e90b1d0 con 0x55c68e990c90 2024-09-10T21:52:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.046+0000 7f661b929f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55c68e9a9310 con 0x55c68e9908b0 2024-09-10T21:52:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.046+0000 7f6618c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c68e962e10 0x55c68ea36dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.046+0000 7f6618c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c68e962e10 0x55c68ea36dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37916/0 (socket says 172.21.15.7:37916) 2024-09-10T21:52:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.046+0000 7f6618c00640 1 -- 172.21.15.7:0/3159077480 learned_addr learned my addr 172.21.15.7:0/3159077480 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.046+0000 7f6619600640 1 --2- 172.21.15.7:0/3159077480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c68e990c90 0x55c68eabd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.046+0000 7f6613e00640 1 --2- 172.21.15.7:0/3159077480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c68e9908b0 0x55c68eabcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.047+0000 7f6618c00640 1 -- 172.21.15.7:0/3159077480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c68e9908b0 msgr2=0x55c68eabcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.047+0000 7f6618c00640 1 --2- 172.21.15.7:0/3159077480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c68e9908b0 0x55c68eabcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.047+0000 7f6618c00640 1 -- 172.21.15.7:0/3159077480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c68e990c90 msgr2=0x55c68eabd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.047+0000 7f6618c00640 1 --2- 172.21.15.7:0/3159077480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c68e990c90 0x55c68eabd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.048 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.047+0000 7f6618c00640 1 -- 172.21.15.7:0/3159077480 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c68ea2e3e0 con 0x55c68e962e10 2024-09-10T21:52:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.047+0000 7f6618c00640 1 --2- 172.21.15.7:0/3159077480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c68e962e10 0x55c68ea36dc0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f660c00b700 tx=0x7f660c00bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab8ad21f9fa998c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.047+0000 7f6612a00640 1 -- 172.21.15.7:0/3159077480 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c00be10 con 0x55c68e962e10 2024-09-10T21:52:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.047+0000 7f6612a00640 1 -- 172.21.15.7:0/3159077480 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f660c00abc0 con 0x55c68e962e10 2024-09-10T21:52:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.047+0000 7f6612a00640 1 -- 172.21.15.7:0/3159077480 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c015460 con 0x55c68e962e10 2024-09-10T21:52:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.048+0000 7f661b929f40 1 -- 172.21.15.7:0/3159077480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c68e962e10 msgr2=0x55c68ea36dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.048+0000 7f661b929f40 1 --2- 172.21.15.7:0/3159077480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c68e962e10 0x55c68ea36dc0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f660c00b700 tx=0x7f660c00bbc0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.048+0000 7f661b929f40 1 -- 172.21.15.7:0/3159077480 shutdown_connections 2024-09-10T21:52:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.048+0000 7f661b929f40 1 --2- 172.21.15.7:0/3159077480 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c68e990c90 0x55c68eabd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.048+0000 7f661b929f40 1 --2- 172.21.15.7:0/3159077480 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c68e9908b0 0x55c68eabcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.048+0000 7f661b929f40 1 --2- 172.21.15.7:0/3159077480 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c68e962e10 0x55c68ea36dc0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.048+0000 7f661b929f40 1 -- 172.21.15.7:0/3159077480 >> 172.21.15.7:0/3159077480 conn(0x55c68e9ab1c0 msgr2=0x55c68e98de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:06.050 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.048+0000 7f661b929f40 1 -- 172.21.15.7:0/3159077480 shutdown_connections 2024-09-10T21:52:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.048+0000 7f661b929f40 1 -- 172.21.15.7:0/3159077480 wait complete. 2024-09-10T21:52:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.049+0000 7f661b929f40 1 Processor -- start 2024-09-10T21:52:06.051 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.049+0000 7f661b929f40 1 -- start start 2024-09-10T21:52:06.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.050+0000 7f661b929f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c68e962e10 0x55c68ea168e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.050+0000 7f661b929f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c68e9908b0 0x55c68ea320b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.052 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.050+0000 7f661b929f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c68e990c90 0x55c68eac15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.050+0000 7f661b929f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c68e9900d0 con 0x55c68e962e10 2024-09-10T21:52:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.050+0000 7f661b929f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c68e98fdf0 con 0x55c68e990c90 2024-09-10T21:52:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.050+0000 7f661b929f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55c68ea2e680 con 0x55c68e9908b0 2024-09-10T21:52:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.050+0000 7f6618c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c68e962e10 0x55c68ea168e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.050+0000 7f6618c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c68e962e10 0x55c68ea168e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37926/0 (socket says 172.21.15.7:37926) 2024-09-10T21:52:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.050+0000 7f6618c00640 1 -- 172.21.15.7:0/3346682364 learned_addr learned my addr 172.21.15.7:0/3346682364 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.050+0000 7f6619600640 1 --2- 172.21.15.7:0/3346682364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c68e990c90 0x55c68eac15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.050+0000 7f6613e00640 1 --2- 172.21.15.7:0/3346682364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c68e9908b0 0x55c68ea320b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.050+0000 7f6618c00640 1 -- 172.21.15.7:0/3346682364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c68e9908b0 msgr2=0x55c68ea320b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.050+0000 7f6618c00640 1 --2- 172.21.15.7:0/3346682364 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c68e9908b0 0x55c68ea320b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.050+0000 7f6618c00640 1 -- 172.21.15.7:0/3346682364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c68e990c90 msgr2=0x55c68eac15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.050+0000 7f6618c00640 1 --2- 172.21.15.7:0/3346682364 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c68e990c90 0x55c68eac15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.050+0000 7f6618c00640 1 -- 172.21.15.7:0/3346682364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c68ea2fbc0 con 0x55c68e962e10 2024-09-10T21:52:06.053 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.051+0000 7f6618c00640 1 --2- 172.21.15.7:0/3346682364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c68e962e10 0x55c68ea168e0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x55c68ea2bd90 tx=0x7f660c015930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.051+0000 7f6611600640 1 -- 172.21.15.7:0/3346682364 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c015d30 con 0x55c68e962e10 2024-09-10T21:52:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.051+0000 7f6611600640 1 -- 172.21.15.7:0/3346682364 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f660c01f420 con 0x55c68e962e10 2024-09-10T21:52:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.051+0000 7f661b929f40 1 -- 172.21.15.7:0/3346682364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c68e9aa890 con 0x55c68e962e10 2024-09-10T21:52:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.051+0000 7f6611600640 1 -- 172.21.15.7:0/3346682364 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f660c01f670 con 0x55c68e962e10 2024-09-10T21:52:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.051+0000 7f661b929f40 1 -- 172.21.15.7:0/3346682364 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c68ea300c0 con 0x55c68e962e10 2024-09-10T21:52:06.054 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.053+0000 7f6611600640 1 -- 172.21.15.7:0/3346682364 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f660c027020 con 0x55c68e962e10 2024-09-10T21:52:06.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.054+0000 7f6611600640 1 --2- 172.21.15.7:0/3346682364 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f65f8076950 0x7f65f8078e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.054+0000 7f6611600640 1 -- 172.21.15.7:0/3346682364 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5853+0+0 (secure 0 0 0) 0x7f660c09a020 con 0x55c68e962e10 2024-09-10T21:52:06.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.054+0000 7f661b929f40 1 --2- 172.21.15.7:0/3346682364 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55c68e95c8e0 0x55c68ead7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.054+0000 7f6613e00640 1 --2- 172.21.15.7:0/3346682364 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f65f8076950 0x7f65f8078e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.054+0000 7f661b929f40 1 -- 172.21.15.7:0/3346682364 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:1 17.12 17:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) -- 0x55c68ea37b50 con 0x55c68e95c8e0 2024-09-10T21:52:06.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.055+0000 7f6619600640 1 --2- 172.21.15.7:0/3346682364 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55c68e95c8e0 0x55c68ead7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.055+0000 7f6613e00640 1 --2- 172.21.15.7:0/3346682364 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f65f8076950 0x7f65f8078e10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6604055000 tx=0x7f6604003020 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.056 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.055+0000 7f6619600640 1 --2- 172.21.15.7:0/3346682364 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55c68e95c8e0 0x55c68ead7fa0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.056+0000 7f6619600640 1 -- 172.21.15.7:0/3346682364 <== osd.3 v2:172.21.15.7:6800/2195665006 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr (12) out=12b] v0'0 uv22 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f6614006020 con 0x55c68e95c8e0 2024-09-10T21:52:06.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.056+0000 7f661b929f40 1 -- 172.21.15.7:0/3346682364 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55c68e95c8e0 msgr2=0x55c68ead7fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.056+0000 7f661b929f40 1 --2- 172.21.15.7:0/3346682364 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x55c68e95c8e0 0x55c68ead7fa0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.056+0000 7f661b929f40 1 -- 172.21.15.7:0/3346682364 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f65f8076950 msgr2=0x7f65f8078e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.056+0000 7f661b929f40 1 --2- 172.21.15.7:0/3346682364 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f65f8076950 0x7f65f8078e10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f6604055000 tx=0x7f6604003020 comp rx=0 tx=0).stop 2024-09-10T21:52:06.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.056+0000 7f661b929f40 1 -- 172.21.15.7:0/3346682364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c68e962e10 msgr2=0x55c68ea168e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.056+0000 7f661b929f40 1 --2- 172.21.15.7:0/3346682364 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c68e962e10 0x55c68ea168e0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x55c68ea2bd90 tx=0x7f660c015930 comp rx=0 tx=0).stop 2024-09-10T21:52:06.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.056+0000 7f6619600640 1 -- 172.21.15.7:0/3346682364 reap_dead start 2024-09-10T21:52:06.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.056+0000 7f661b929f40 1 -- 172.21.15.7:0/3346682364 shutdown_connections 2024-09-10T21:52:06.057 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.056+0000 7f661b929f40 1 -- 172.21.15.7:0/3346682364 >> 172.21.15.7:0/3346682364 conn(0x55c68e9ab1c0 msgr2=0x55c68ea2d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:06.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.056+0000 7f661b929f40 1 -- 172.21.15.7:0/3346682364 shutdown_connections 2024-09-10T21:52:06.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.056+0000 7f661b929f40 1 -- 172.21.15.7:0/3346682364 wait complete. 2024-09-10T21:52:06.061 INFO:teuthology.orchestra.run.smithi007.stdout:mytag123 2024-09-10T21:52:06.061 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2024-09-10T21:52:06.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.137+0000 7fc414bf8f40 1 Processor -- start 2024-09-10T21:52:06.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.137+0000 7fc414bf8f40 1 -- start start 2024-09-10T21:52:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.138+0000 7fc414bf8f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5558c1c7f520 0x5558c1ddeea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.138+0000 7fc414bf8f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5558c1cac8b0 0x5558c1cab500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.138+0000 7fc414bf8f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558c1cacc90 0x5558c1ca7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.138+0000 7fc414bf8f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5558c1bc8c60 con 0x5558c1c7f520 2024-09-10T21:52:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.138+0000 7fc414bf8f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5558c1c271d0 con 0x5558c1cacc90 2024-09-10T21:52:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.138+0000 7fc414bf8f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5558c1cc52d0 con 0x5558c1cac8b0 2024-09-10T21:52:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.138+0000 7fc411e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5558c1c7f520 0x5558c1ddeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.138+0000 7fc411e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5558c1c7f520 0x5558c1ddeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37936/0 (socket says 172.21.15.7:37936) 2024-09-10T21:52:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.138+0000 7fc411e00640 1 -- 172.21.15.7:0/3724208697 learned_addr learned my addr 172.21.15.7:0/3724208697 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.138+0000 7fc40be00640 1 --2- 172.21.15.7:0/3724208697 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5558c1cac8b0 0x5558c1cab500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.138+0000 7fc412800640 1 --2- 172.21.15.7:0/3724208697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558c1cacc90 0x5558c1ca7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.138+0000 7fc411e00640 1 -- 172.21.15.7:0/3724208697 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5558c1cac8b0 msgr2=0x5558c1cab500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.138+0000 7fc411e00640 1 --2- 172.21.15.7:0/3724208697 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5558c1cac8b0 0x5558c1cab500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.138+0000 7fc411e00640 1 -- 172.21.15.7:0/3724208697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558c1cacc90 msgr2=0x5558c1ca7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.138+0000 7fc411e00640 1 --2- 172.21.15.7:0/3724208697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558c1cacc90 0x5558c1ca7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.138+0000 7fc411e00640 1 -- 172.21.15.7:0/3724208697 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558c1d331d0 con 0x5558c1c7f520 2024-09-10T21:52:06.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.138+0000 7fc412800640 1 --2- 172.21.15.7:0/3724208697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558c1cacc90 0x5558c1ca7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:06.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.139+0000 7fc411e00640 1 --2- 172.21.15.7:0/3724208697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5558c1c7f520 0x5558c1ddeea0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fc40400b760 tx=0x7fc40400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11382796b6c0f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.139+0000 7fc410a00640 1 -- 172.21.15.7:0/3724208697 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40400abc0 con 0x5558c1c7f520 2024-09-10T21:52:06.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.139+0000 7fc410a00640 1 -- 172.21.15.7:0/3724208697 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc40400ad60 con 0x5558c1c7f520 2024-09-10T21:52:06.141 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.139+0000 7fc410a00640 1 -- 172.21.15.7:0/3724208697 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4040086b0 con 0x5558c1c7f520 2024-09-10T21:52:06.143 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.139+0000 7fc414bf8f40 1 -- 172.21.15.7:0/3724208697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5558c1c7f520 msgr2=0x5558c1ddeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.139+0000 7fc414bf8f40 1 --2- 172.21.15.7:0/3724208697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5558c1c7f520 0x5558c1ddeea0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fc40400b760 tx=0x7fc40400bc20 comp rx=0 tx=0).stop 2024-09-10T21:52:06.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.140+0000 7fc414bf8f40 1 -- 172.21.15.7:0/3724208697 shutdown_connections 2024-09-10T21:52:06.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.140+0000 7fc414bf8f40 1 --2- 172.21.15.7:0/3724208697 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558c1cacc90 0x5558c1ca7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.140+0000 7fc414bf8f40 1 --2- 172.21.15.7:0/3724208697 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5558c1cac8b0 0x5558c1cab500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.140+0000 7fc414bf8f40 1 --2- 172.21.15.7:0/3724208697 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5558c1c7f520 0x5558c1ddeea0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.140+0000 7fc414bf8f40 1 -- 172.21.15.7:0/3724208697 >> 172.21.15.7:0/3724208697 conn(0x5558c1cc71c0 msgr2=0x5558c1dde360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:06.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.140+0000 7fc414bf8f40 1 -- 172.21.15.7:0/3724208697 shutdown_connections 2024-09-10T21:52:06.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.140+0000 7fc414bf8f40 1 -- 172.21.15.7:0/3724208697 wait complete. 2024-09-10T21:52:06.144 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.140+0000 7fc414bf8f40 1 Processor -- start 2024-09-10T21:52:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.141+0000 7fc414bf8f40 1 -- start start 2024-09-10T21:52:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.141+0000 7fc414bf8f40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5558c1c7f520 0x5558c1d31c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.141+0000 7fc414bf8f40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5558c1cac8b0 0x5558c1ca7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.141+0000 7fc414bf8f40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558c1cacc90 0x5558c1ca75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.141+0000 7fc414bf8f40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5558c1caba40 con 0x5558c1cac8b0 2024-09-10T21:52:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.141+0000 7fc414bf8f40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5558c1d39fc0 con 0x5558c1cacc90 2024-09-10T21:52:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.141+0000 7fc414bf8f40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5558c1ddeba0 con 0x5558c1c7f520 2024-09-10T21:52:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.142+0000 7fc40be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5558c1cac8b0 0x5558c1ca7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.142+0000 7fc40be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5558c1cac8b0 0x5558c1ca7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37944/0 (socket says 172.21.15.7:37944) 2024-09-10T21:52:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.142+0000 7fc411e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5558c1c7f520 0x5558c1d31c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.142+0000 7fc40be00640 1 -- 172.21.15.7:0/2472642541 learned_addr learned my addr 172.21.15.7:0/2472642541 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.142+0000 7fc412800640 1 --2- 172.21.15.7:0/2472642541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558c1cacc90 0x5558c1ca75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.145 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.142+0000 7fc411e00640 1 -- 172.21.15.7:0/2472642541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558c1cacc90 msgr2=0x5558c1ca75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.142+0000 7fc411e00640 1 --2- 172.21.15.7:0/2472642541 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5558c1cacc90 0x5558c1ca75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.142+0000 7fc411e00640 1 -- 172.21.15.7:0/2472642541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5558c1cac8b0 msgr2=0x5558c1ca7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.142+0000 7fc411e00640 1 --2- 172.21.15.7:0/2472642541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5558c1cac8b0 0x5558c1ca7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.142+0000 7fc411e00640 1 -- 172.21.15.7:0/2472642541 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558c1cc2670 con 0x5558c1c7f520 2024-09-10T21:52:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.142+0000 7fc40be00640 1 --2- 172.21.15.7:0/2472642541 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5558c1cac8b0 0x5558c1ca7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:52:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.142+0000 7fc411e00640 1 --2- 172.21.15.7:0/2472642541 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5558c1c7f520 0x5558c1d31c20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc404014000 tx=0x7fc404017a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.143+0000 7fc40aa00640 1 -- 172.21.15.7:0/2472642541 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc404024070 con 0x5558c1c7f520 2024-09-10T21:52:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.143+0000 7fc40aa00640 1 -- 172.21.15.7:0/2472642541 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc404017e50 con 0x5558c1c7f520 2024-09-10T21:52:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.143+0000 7fc414bf8f40 1 -- 172.21.15.7:0/2472642541 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x5558c1cbc290 con 0x5558c1c7f520 2024-09-10T21:52:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.143+0000 7fc40aa00640 1 -- 172.21.15.7:0/2472642541 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4040086b0 con 0x5558c1c7f520 2024-09-10T21:52:06.146 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.143+0000 7fc414bf8f40 1 -- 172.21.15.7:0/2472642541 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) -- 0x5558c1d323c0 con 0x5558c1c7f520 2024-09-10T21:52:06.147 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.145+0000 7fc40aa00640 1 -- 172.21.15.7:0/2472642541 <== mon.2 v2:172.21.15.133:3301/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc40401e030 con 0x5558c1c7f520 2024-09-10T21:52:06.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.146+0000 7fc40aa00640 1 --2- 172.21.15.7:0/2472642541 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc3f0076950 0x7fc3f0078e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.146+0000 7fc40aa00640 1 -- 172.21.15.7:0/2472642541 <== mon.2 v2:172.21.15.133:3301/0 5 ==== osd_map(117..117 src has 1..117) ==== 5853+0+0 (secure 0 0 0) 0x7fc40409a660 con 0x5558c1c7f520 2024-09-10T21:52:06.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.146+0000 7fc40be00640 1 --2- 172.21.15.7:0/2472642541 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc3f0076950 0x7fc3f0078e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.146+0000 7fc414bf8f40 1 --2- 172.21.15.7:0/2472642541 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5558c1cab620 0x5558c1cbdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.146+0000 7fc414bf8f40 1 -- 172.21.15.7:0/2472642541 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:1 17.19 17:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) -- 0x5558c1cbfc30 con 0x5558c1cab620 2024-09-10T21:52:06.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.147+0000 7fc412800640 1 --2- 172.21.15.7:0/2472642541 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5558c1cab620 0x5558c1cbdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.148 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.147+0000 7fc40be00640 1 --2- 172.21.15.7:0/2472642541 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc3f0076950 0x7fc3f0078e10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x5558c1dde300 tx=0x7fc3fc003810 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.147+0000 7fc412800640 1 --2- 172.21.15.7:0/2472642541 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5558c1cab620 0x5558c1cbdcd0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.148+0000 7fc412800640 1 -- 172.21.15.7:0/2472642541 <== osd.5 v2:172.21.15.133:6818/3838498382 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (12) out=12b] v0'0 uv23 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7fc40c00b040 con 0x5558c1cab620 2024-09-10T21:52:06.149 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.148+0000 7fc414bf8f40 1 -- 172.21.15.7:0/2472642541 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5558c1cab620 msgr2=0x5558c1cbdcd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.148+0000 7fc414bf8f40 1 --2- 172.21.15.7:0/2472642541 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5558c1cab620 0x5558c1cbdcd0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.148+0000 7fc414bf8f40 1 -- 172.21.15.7:0/2472642541 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc3f0076950 msgr2=0x7fc3f0078e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.148+0000 7fc414bf8f40 1 --2- 172.21.15.7:0/2472642541 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc3f0076950 0x7fc3f0078e10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x5558c1dde300 tx=0x7fc3fc003810 comp rx=0 tx=0).stop 2024-09-10T21:52:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.148+0000 7fc414bf8f40 1 -- 172.21.15.7:0/2472642541 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5558c1c7f520 msgr2=0x5558c1d31c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.148+0000 7fc414bf8f40 1 --2- 172.21.15.7:0/2472642541 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5558c1c7f520 0x5558c1d31c20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc404014000 tx=0x7fc404017a50 comp rx=0 tx=0).stop 2024-09-10T21:52:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.148+0000 7fc412800640 1 -- 172.21.15.7:0/2472642541 reap_dead start 2024-09-10T21:52:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.148+0000 7fc414bf8f40 1 -- 172.21.15.7:0/2472642541 shutdown_connections 2024-09-10T21:52:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.148+0000 7fc414bf8f40 1 -- 172.21.15.7:0/2472642541 >> 172.21.15.7:0/2472642541 conn(0x5558c1cc71c0 msgr2=0x5558c1dde360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.148+0000 7fc414bf8f40 1 -- 172.21.15.7:0/2472642541 shutdown_connections 2024-09-10T21:52:06.150 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.148+0000 7fc414bf8f40 1 -- 172.21.15.7:0/2472642541 wait complete. 2024-09-10T21:52:06.153 INFO:teuthology.orchestra.run.smithi007.stdout:mytag123 2024-09-10T21:52:06.154 DEBUG:teuthology.orchestra.run.smithi007:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000004.00000000 scrub_tag 2024-09-10T21:52:06.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.229+0000 7f49ce83af40 1 Processor -- start 2024-09-10T21:52:06.230 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.229+0000 7f49ce83af40 1 -- start start 2024-09-10T21:52:06.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.230+0000 7f49ce83af40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560296f50520 0x560296f7c250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.230+0000 7f49ce83af40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560296f7d8b0 0x5602970afb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.230+0000 7f49ce83af40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560296f7dc90 0x560297023350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.230+0000 7f49ce83af40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560296e99c60 con 0x560296f50520 2024-09-10T21:52:06.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.230+0000 7f49ce83af40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560296ef81d0 con 0x560296f7d8b0 2024-09-10T21:52:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.230+0000 7f49ce83af40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x560296f962d0 con 0x560296f7dc90 2024-09-10T21:52:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.231+0000 7f49cba00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560296f50520 0x560296f7c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.231+0000 7f49cba00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560296f50520 0x560296f7c250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37950/0 (socket says 172.21.15.7:37950) 2024-09-10T21:52:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.231+0000 7f49cba00640 1 -- 172.21.15.7:0/2272428984 learned_addr learned my addr 172.21.15.7:0/2272428984 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.231+0000 7f49c3e00640 1 --2- 172.21.15.7:0/2272428984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560296f7d8b0 0x5602970afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.231+0000 7f49cc400640 1 --2- 172.21.15.7:0/2272428984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560296f7dc90 0x560297023350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.231+0000 7f49cba00640 1 -- 172.21.15.7:0/2272428984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560296f7dc90 msgr2=0x560297023350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.231+0000 7f49cba00640 1 --2- 172.21.15.7:0/2272428984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560296f7dc90 0x560297023350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.231+0000 7f49cba00640 1 -- 172.21.15.7:0/2272428984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560296f7d8b0 msgr2=0x5602970afb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.231+0000 7f49cba00640 1 --2- 172.21.15.7:0/2272428984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560296f7d8b0 0x5602970afb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.231+0000 7f49cba00640 1 -- 172.21.15.7:0/2272428984 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560296f98980 con 0x560296f50520 2024-09-10T21:52:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.231+0000 7f49cba00640 1 --2- 172.21.15.7:0/2272428984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560296f50520 0x560296f7c250 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f49bc001df0 tx=0x7f49bc00b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795a542f4d3268f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.232+0000 7f49ca600640 1 -- 172.21.15.7:0/2272428984 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc00bde0 con 0x560296f50520 2024-09-10T21:52:06.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.232+0000 7f49ca600640 1 -- 172.21.15.7:0/2272428984 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49bc00f400 con 0x560296f50520 2024-09-10T21:52:06.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.232+0000 7f49ca600640 1 -- 172.21.15.7:0/2272428984 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc00f850 con 0x560296f50520 2024-09-10T21:52:06.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.232+0000 7f49ce83af40 1 -- 172.21.15.7:0/2272428984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560296f50520 msgr2=0x560296f7c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.232+0000 7f49ce83af40 1 --2- 172.21.15.7:0/2272428984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560296f50520 0x560296f7c250 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f49bc001df0 tx=0x7f49bc00b5b0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.232+0000 7f49ce83af40 1 -- 172.21.15.7:0/2272428984 shutdown_connections 2024-09-10T21:52:06.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.232+0000 7f49ce83af40 1 --2- 172.21.15.7:0/2272428984 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560296f7d8b0 0x5602970afb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.232+0000 7f49ce83af40 1 --2- 172.21.15.7:0/2272428984 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560296f7dc90 0x560297023350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.232+0000 7f49ce83af40 1 --2- 172.21.15.7:0/2272428984 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560296f50520 0x560296f7c250 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.232+0000 7f49ce83af40 1 -- 172.21.15.7:0/2272428984 >> 172.21.15.7:0/2272428984 conn(0x560296f981c0 msgr2=0x5602970af450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:06.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.232+0000 7f49ce83af40 1 -- 172.21.15.7:0/2272428984 shutdown_connections 2024-09-10T21:52:06.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.232+0000 7f49ce83af40 1 -- 172.21.15.7:0/2272428984 wait complete. 2024-09-10T21:52:06.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.233+0000 7f49ce83af40 1 Processor -- start 2024-09-10T21:52:06.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.233+0000 7f49ce83af40 1 -- start start 2024-09-10T21:52:06.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.234+0000 7f49ce83af40 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560296f50520 0x560297002af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.234+0000 7f49ce83af40 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560296f7d8b0 0x5602970bbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.234+0000 7f49ce83af40 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560296f7dc90 0x5602970bc530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.234+0000 7f49ce83af40 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560297004030 con 0x560296f50520 2024-09-10T21:52:06.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.234+0000 7f49ce83af40 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560296f79410 con 0x560296f7dc90 2024-09-10T21:52:06.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.234+0000 7f49ce83af40 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x560296f798b0 con 0x560296f7d8b0 2024-09-10T21:52:06.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.234+0000 7f49cba00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560296f50520 0x560297002af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.234+0000 7f49cba00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560296f50520 0x560297002af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37952/0 (socket says 172.21.15.7:37952) 2024-09-10T21:52:06.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.234+0000 7f49cba00640 1 -- 172.21.15.7:0/711707773 learned_addr learned my addr 172.21.15.7:0/711707773 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:06.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.235+0000 7f49cc400640 1 --2- 172.21.15.7:0/711707773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560296f7dc90 0x5602970bc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.235+0000 7f49c3e00640 1 --2- 172.21.15.7:0/711707773 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560296f7d8b0 0x5602970bbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.235+0000 7f49cba00640 1 -- 172.21.15.7:0/711707773 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560296f7d8b0 msgr2=0x5602970bbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.235+0000 7f49cba00640 1 --2- 172.21.15.7:0/711707773 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560296f7d8b0 0x5602970bbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.235+0000 7f49cba00640 1 -- 172.21.15.7:0/711707773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560296f7dc90 msgr2=0x5602970bc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.235+0000 7f49cba00640 1 --2- 172.21.15.7:0/711707773 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560296f7dc90 0x5602970bc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.235+0000 7f49cba00640 1 -- 172.21.15.7:0/711707773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560297003030 con 0x560296f50520 2024-09-10T21:52:06.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.235+0000 7f49cba00640 1 --2- 172.21.15.7:0/711707773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560296f50520 0x560297002af0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00fdd0 tx=0x7f49bc00fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.235+0000 7f49c9200640 1 -- 172.21.15.7:0/711707773 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc01f070 con 0x560296f50520 2024-09-10T21:52:06.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.236+0000 7f49c9200640 1 -- 172.21.15.7:0/711707773 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49bc01b430 con 0x560296f50520 2024-09-10T21:52:06.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.236+0000 7f49ce83af40 1 -- 172.21.15.7:0/711707773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5602970b9500 con 0x560296f50520 2024-09-10T21:52:06.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.236+0000 7f49c9200640 1 -- 172.21.15.7:0/711707773 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49bc01b8c0 con 0x560296f50520 2024-09-10T21:52:06.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.236+0000 7f49ce83af40 1 -- 172.21.15.7:0/711707773 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602970b9ac0 con 0x560296f50520 2024-09-10T21:52:06.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.238+0000 7f49c9200640 1 -- 172.21.15.7:0/711707773 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f49bc01ba60 con 0x560296f50520 2024-09-10T21:52:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.238+0000 7f49c9200640 1 --2- 172.21.15.7:0/711707773 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f49a8076950 0x7f49a8078e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.239+0000 7f49c9200640 1 -- 172.21.15.7:0/711707773 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5853+0+0 (secure 0 0 0) 0x7f49bc09a200 con 0x560296f50520 2024-09-10T21:52:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.239+0000 7f49ce83af40 1 --2- 172.21.15.7:0/711707773 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x560296f78890 0x5602970bad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.239+0000 7f49ce83af40 1 -- 172.21.15.7:0/711707773 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:1 17.5 17:a367a3e8:::10000000004.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e117) -- 0x5602970bb2d0 con 0x560296f78890 2024-09-10T21:52:06.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.239+0000 7f49c3e00640 1 --2- 172.21.15.7:0/711707773 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f49a8076950 0x7f49a8078e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.239+0000 7f49cc400640 1 --2- 172.21.15.7:0/711707773 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x560296f78890 0x5602970bad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.240+0000 7f49c3e00640 1 --2- 172.21.15.7:0/711707773 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f49a8076950 0x7f49a8078e10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x560296f50eb0 tx=0x7f49b400e040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.240+0000 7f49cc400640 1 --2- 172.21.15.7:0/711707773 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x560296f78890 0x5602970bad90 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.240+0000 7f49cc400640 1 -- 172.21.15.7:0/711707773 <== osd.6 v2:172.21.15.133:6810/3365981160 1 ==== osd_op_reply(1 10000000004.00000000 [getxattr (12) out=12b] v0'0 uv12 ondisk = 0) ==== 164+0+12 (crc 0 0 0) 0x7f49c400b040 con 0x560296f78890 2024-09-10T21:52:06.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.240+0000 7f49ce83af40 1 -- 172.21.15.7:0/711707773 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x560296f78890 msgr2=0x5602970bad90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.240+0000 7f49ce83af40 1 --2- 172.21.15.7:0/711707773 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x560296f78890 0x5602970bad90 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.240+0000 7f49ce83af40 1 -- 172.21.15.7:0/711707773 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f49a8076950 msgr2=0x7f49a8078e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.240+0000 7f49ce83af40 1 --2- 172.21.15.7:0/711707773 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f49a8076950 0x7f49a8078e10 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x560296f50eb0 tx=0x7f49b400e040 comp rx=0 tx=0).stop 2024-09-10T21:52:06.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.240+0000 7f49ce83af40 1 -- 172.21.15.7:0/711707773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560296f50520 msgr2=0x560297002af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.240+0000 7f49ce83af40 1 --2- 172.21.15.7:0/711707773 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560296f50520 0x560297002af0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00fdd0 tx=0x7f49bc00fe00 comp rx=0 tx=0).stop 2024-09-10T21:52:06.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.240+0000 7f49cc400640 1 -- 172.21.15.7:0/711707773 reap_dead start 2024-09-10T21:52:06.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.240+0000 7f49ce83af40 1 -- 172.21.15.7:0/711707773 shutdown_connections 2024-09-10T21:52:06.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.240+0000 7f49ce83af40 1 -- 172.21.15.7:0/711707773 >> 172.21.15.7:0/711707773 conn(0x560296f981c0 msgr2=0x560296f7ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:06.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.241+0000 7f49ce83af40 1 -- 172.21.15.7:0/711707773 shutdown_connections 2024-09-10T21:52:06.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.241+0000 7f49ce83af40 1 -- 172.21.15.7:0/711707773 wait complete. 2024-09-10T21:52:06.245 INFO:teuthology.orchestra.run.smithi007.stdout:mytag123 2024-09-10T21:52:06.246 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.8:0 flush journal 2024-09-10T21:52:06.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.409+0000 7f8a97200640 1 -- 172.21.15.7:0/2797183133 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a9015f980 msgr2=0x7f8a9006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.409+0000 7f8a97200640 1 --2- 172.21.15.7:0/2797183133 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a9015f980 0x7f8a9006d420 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8a900b71b0 tx=0x7f8a7802a4a0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.409+0000 7f8a97200640 1 -- 172.21.15.7:0/2797183133 shutdown_connections 2024-09-10T21:52:06.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.409+0000 7f8a97200640 1 --2- 172.21.15.7:0/2797183133 >> v2:172.21.15.7:3300/0 conn(0x7f8a9015c9c0 0x7f8a9015ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.409+0000 7f8a97200640 1 --2- 172.21.15.7:0/2797183133 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a9015f980 0x7f8a9006d420 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.409+0000 7f8a97200640 1 -- 172.21.15.7:0/2797183133 >> 172.21.15.7:0/2797183133 conn(0x7f8a9006fa60 msgr2=0x7f8a9006fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:06.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.409+0000 7f8a97200640 1 -- 172.21.15.7:0/2797183133 shutdown_connections 2024-09-10T21:52:06.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.409+0000 7f8a97200640 1 -- 172.21.15.7:0/2797183133 wait complete. 2024-09-10T21:52:06.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.410+0000 7f8a97200640 1 Processor -- start 2024-09-10T21:52:06.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.410+0000 7f8a97200640 1 -- start start 2024-09-10T21:52:06.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.411+0000 7f8a97200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a900720e0 0x7f8a90151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.411+0000 7f8a97200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a9015c9c0 0x7f8a901ea520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.411+0000 7f8a97200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a9015f980 0x7f8a901523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.411+0000 7f8a97200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8a900b6360 con 0x7f8a9015f980 2024-09-10T21:52:06.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.411+0000 7f8a97200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8a900b64e0 con 0x7f8a9015c9c0 2024-09-10T21:52:06.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.411+0000 7f8a97200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f8a900b6660 con 0x7f8a900720e0 2024-09-10T21:52:06.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.412+0000 7f8a95200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a9015f980 0x7f8a901523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.412+0000 7f8a87400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a9015c9c0 0x7f8a901ea520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.412+0000 7f8a8fe00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a900720e0 0x7f8a90151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.412+0000 7f8a87400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a9015c9c0 0x7f8a901ea520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:48004/0 (socket says 172.21.15.7:48004) 2024-09-10T21:52:06.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.412+0000 7f8a87400640 1 -- 172.21.15.7:0/1965689607 learned_addr learned my addr 172.21.15.7:0/1965689607 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:06.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.412+0000 7f8a95200640 1 -- 172.21.15.7:0/1965689607 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a900720e0 msgr2=0x7f8a90151540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.412+0000 7f8a95200640 1 --2- 172.21.15.7:0/1965689607 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a900720e0 0x7f8a90151540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.412+0000 7f8a95200640 1 -- 172.21.15.7:0/1965689607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a9015c9c0 msgr2=0x7f8a901ea520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.412+0000 7f8a95200640 1 --2- 172.21.15.7:0/1965689607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a9015c9c0 0x7f8a901ea520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.412+0000 7f8a95200640 1 -- 172.21.15.7:0/1965689607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a901e8550 con 0x7f8a9015f980 2024-09-10T21:52:06.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.412+0000 7f8a95200640 1 --2- 172.21.15.7:0/1965689607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a9015f980 0x7f8a901523b0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800ef00 tx=0x7f8a8800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.412+0000 7f8a8cc00640 1 -- 172.21.15.7:0/1965689607 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88004b20 con 0x7f8a9015f980 2024-09-10T21:52:06.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.413+0000 7f8a8cc00640 1 -- 172.21.15.7:0/1965689607 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a88004dd0 con 0x7f8a9015f980 2024-09-10T21:52:06.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.413+0000 7f8a8cc00640 1 -- 172.21.15.7:0/1965689607 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88003bd0 con 0x7f8a9015f980 2024-09-10T21:52:06.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.413+0000 7f8a97200640 1 -- 172.21.15.7:0/1965689607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a901e8830 con 0x7f8a9015f980 2024-09-10T21:52:06.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.413+0000 7f8a97200640 1 -- 172.21.15.7:0/1965689607 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a901e8df0 con 0x7f8a9015f980 2024-09-10T21:52:06.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.415+0000 7f8a8cc00640 1 -- 172.21.15.7:0/1965689607 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f8a8800b040 con 0x7f8a9015f980 2024-09-10T21:52:06.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.416+0000 7f8a8cc00640 1 --2- 172.21.15.7:0/1965689607 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8a6c077f70 0x7f8a6c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.416+0000 7f8a8cc00640 1 -- 172.21.15.7:0/1965689607 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5853+0+0 (secure 0 0 0) 0x7f8a8809a290 con 0x7f8a9015f980 2024-09-10T21:52:06.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.416+0000 7f8a9a1c3740 1 Processor -- start 2024-09-10T21:52:06.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.416+0000 7f8a9a1c3740 1 -- start start 2024-09-10T21:52:06.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.416+0000 7f8a8fe00640 1 --2- 172.21.15.7:0/1965689607 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8a6c077f70 0x7f8a6c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.417+0000 7f8a8fe00640 1 --2- 172.21.15.7:0/1965689607 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8a6c077f70 0x7f8a6c07a430 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8a800017c0 tx=0x7f8a80009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.417+0000 7f8a9a1c3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa2f130 0x560d1aa2f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.417+0000 7f8a9a1c3740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1a9ea2a0 0x560d1aa2dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.417+0000 7f8a9a1c3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1a9ea820 0x560d1aa2e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.417+0000 7f8a9a1c3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560d1a9491c0 con 0x560d1aa2f130 2024-09-10T21:52:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.417+0000 7f8a9a1c3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560d1a95c870 con 0x560d1a9ea820 2024-09-10T21:52:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.417+0000 7f8a9a1c3740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x560d1a94bca0 con 0x560d1a9ea2a0 2024-09-10T21:52:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.417+0000 7f8a8fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa2f130 0x560d1aa2f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.418+0000 7f8a8fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa2f130 0x560d1aa2f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37984/0 (socket says 172.21.15.7:37984) 2024-09-10T21:52:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.418+0000 7f8a8fe00640 1 -- 172.21.15.7:0/2038781551 learned_addr learned my addr 172.21.15.7:0/2038781551 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.418+0000 7f8a95200640 1 --2- 172.21.15.7:0/2038781551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1a9ea820 0x560d1aa2e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.418+0000 7f8a87400640 1 --2- 172.21.15.7:0/2038781551 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1a9ea2a0 0x560d1aa2dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.418+0000 7f8a8fe00640 1 -- 172.21.15.7:0/2038781551 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1a9ea2a0 msgr2=0x560d1aa2dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.418+0000 7f8a8fe00640 1 --2- 172.21.15.7:0/2038781551 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1a9ea2a0 0x560d1aa2dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.419 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.418+0000 7f8a8fe00640 1 -- 172.21.15.7:0/2038781551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1a9ea820 msgr2=0x560d1aa2e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.418+0000 7f8a8fe00640 1 --2- 172.21.15.7:0/2038781551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1a9ea820 0x560d1aa2e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.420 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.418+0000 7f8a8fe00640 1 -- 172.21.15.7:0/2038781551 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d1a7247d0 con 0x560d1aa2f130 2024-09-10T21:52:06.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.418+0000 7f8a8fe00640 1 --2- 172.21.15.7:0/2038781551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa2f130 0x560d1aa2f510 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000fb20 tx=0x7f8a80012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=304ea795b67ea690 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.418+0000 7f8a84c00640 1 -- 172.21.15.7:0/2038781551 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80025070 con 0x560d1aa2f130 2024-09-10T21:52:06.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.418+0000 7f8a84c00640 1 -- 172.21.15.7:0/2038781551 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a8001b050 con 0x560d1aa2f130 2024-09-10T21:52:06.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.418+0000 7f8a84c00640 1 -- 172.21.15.7:0/2038781551 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80020410 con 0x560d1aa2f130 2024-09-10T21:52:06.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.419+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/2038781551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa2f130 msgr2=0x560d1aa2f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.419+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/2038781551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa2f130 0x560d1aa2f510 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000fb20 tx=0x7f8a80012000 comp rx=0 tx=0).stop 2024-09-10T21:52:06.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.419+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/2038781551 shutdown_connections 2024-09-10T21:52:06.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.419+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/2038781551 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1a9ea820 0x560d1aa2e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.421 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.419+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/2038781551 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1a9ea2a0 0x560d1aa2dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.419+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/2038781551 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa2f130 0x560d1aa2f510 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.419+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/2038781551 >> 172.21.15.7:0/2038781551 conn(0x560d1a9e5870 msgr2=0x560d1a9e9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:06.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.419+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/2038781551 shutdown_connections 2024-09-10T21:52:06.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.419+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/2038781551 wait complete. 2024-09-10T21:52:06.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.420+0000 7f8a9a1c3740 1 -- start start 2024-09-10T21:52:06.422 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.420+0000 7f8a9a1c3740 1 Processor -- start 2024-09-10T21:52:06.424 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.421+0000 7f8a9a1c3740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1a9ea820 0x560d1aa41ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.421+0000 7f8a9a1c3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1aa2f130 0x560d1aa130e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.421+0000 7f8a9a1c3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa13620 0x560d1aa00520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.421+0000 7f8a9a1c3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x560d1a8f2410 con 0x560d1aa13620 2024-09-10T21:52:06.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.421+0000 7f8a9a1c3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560d1a94fab0 con 0x560d1aa2f130 2024-09-10T21:52:06.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.421+0000 7f8a9a1c3740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x560d1a6bc740 con 0x560d1a9ea820 2024-09-10T21:52:06.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.421+0000 7f8a9a1c3740 10 client.0 _ll_register_callbacks cb 0x560d1a8cf360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:52:06.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.421+0000 7f8a95200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa13620 0x560d1aa00520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.421+0000 7f8a95200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa13620 0x560d1aa00520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:37988/0 (socket says 172.21.15.7:37988) 2024-09-10T21:52:06.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.421+0000 7f8a95200640 1 -- 172.21.15.7:0/991769382 learned_addr learned my addr 172.21.15.7:0/991769382 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:06.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.421+0000 7f8a8fe00640 1 --2- 172.21.15.7:0/991769382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1a9ea820 0x560d1aa41ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.421+0000 7f8a87400640 1 --2- 172.21.15.7:0/991769382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1aa2f130 0x560d1aa130e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.422+0000 7f8a95200640 1 -- 172.21.15.7:0/991769382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1a9ea820 msgr2=0x560d1aa41ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.422+0000 7f8a95200640 1 --2- 172.21.15.7:0/991769382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1a9ea820 0x560d1aa41ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.425 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.422+0000 7f8a95200640 1 -- 172.21.15.7:0/991769382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1aa2f130 msgr2=0x560d1aa130e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.422+0000 7f8a95200640 1 --2- 172.21.15.7:0/991769382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1aa2f130 0x560d1aa130e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.422+0000 7f8a95200640 1 -- 172.21.15.7:0/991769382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d1a6ad970 con 0x560d1aa13620 2024-09-10T21:52:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.422+0000 7f8a95200640 1 --2- 172.21.15.7:0/991769382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa13620 0x560d1aa00520 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f8a8805e750 tx=0x7f8a88062ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=606bb831fe667f9a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.422+0000 7f8a7d600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:52:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.422+0000 7f8a7d600640 1 -- 172.21.15.7:0/991769382 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a880883c0 con 0x560d1aa13620 2024-09-10T21:52:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.422+0000 7f8a7d600640 1 -- 172.21.15.7:0/991769382 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 5853+0+0 (secure 0 0 0) 0x7f8a88098b50 con 0x560d1aa13620 2024-09-10T21:52:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.422+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/991769382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a8001b050 con 0x560d1aa13620 2024-09-10T21:52:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.422+0000 7f8a7d600640 1 -- 172.21.15.7:0/991769382 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7f8a8806bb90 con 0x560d1aa13620 2024-09-10T21:52:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.422+0000 7f8a9a1c3740 10 client.8436 fetch_fsmap learned FSMap version 108 2024-09-10T21:52:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.422+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/991769382 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x560d1a68b6a0 con 0x560d1aa13620 2024-09-10T21:52:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.423+0000 7f8a7d600640 1 -- 172.21.15.7:0/991769382 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a8001b050 con 0x560d1aa13620 2024-09-10T21:52:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.423+0000 7f8a7d600640 1 -- 172.21.15.7:0/991769382 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 108) ==== 3502+0+0 (secure 0 0 0) 0x7f8a880619a0 con 0x560d1aa13620 2024-09-10T21:52:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.423+0000 7f8a9a1c3740 10 client.8436 fetch_fsmap finished waiting for FSMap version 108 2024-09-10T21:52:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.423+0000 7f8a9a1c3740 10 client.8436 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-09-10T21:52:06.426 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.423+0000 7f8a9a1c3740 20 client.8436 populate_metadata read hostname 'smithi007' 2024-09-10T21:52:06.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.423+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/991769382 >> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] conn(0x560d1aa27810 0x560d1aa3ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.423+0000 7f8a9a1c3740 4 client.8436 mds_command: new command op to 236723 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:52:06.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.423+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/991769382 --> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560d1a95e7c0 con 0x560d1aa27810 2024-09-10T21:52:06.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.423+0000 7f8a8fe00640 1 --2- 172.21.15.7:0/991769382 >> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] conn(0x560d1aa27810 0x560d1aa3ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.424+0000 7f8a8fe00640 1 --2- 172.21.15.7:0/991769382 >> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] conn(0x560d1aa27810 0x560d1aa3ee60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e643e1774c02cfba server_cookie=c0c243252086ade8 in_seq=0 out_seq=0 2024-09-10T21:52:06.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.424+0000 7f8a7d600640 10 client.8436 ms_handle_connect on v2:172.21.15.133:6835/772257983 2024-09-10T21:52:06.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.425+0000 7f8a7d600640 1 -- 172.21.15.7:0/991769382 <== mds.0 v2:172.21.15.133:6835/772257983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560d1a95e7c0 con 0x560d1aa27810 2024-09-10T21:52:06.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.425+0000 7f8a7d600640 10 client.8436 handle_command_reply: tid=0 2024-09-10T21:52:06.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.425+0000 7f8a9a1c3740 1 client.8436 shutdown 2024-09-10T21:52:06.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.425+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/991769382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa13620 msgr2=0x560d1aa00520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.425+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/991769382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa13620 0x560d1aa00520 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f8a8805e750 tx=0x7f8a88062ce0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.425+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/991769382 shutdown_connections 2024-09-10T21:52:06.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.425+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/991769382 >> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] conn(0x560d1aa27810 0x560d1aa3ee60 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.425+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/991769382 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1aa2f130 0x560d1aa130e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.425+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/991769382 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa13620 0x560d1aa00520 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.425+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/991769382 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1a9ea820 0x560d1aa41ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.425+0000 7f8a7d600640 0 client.8436 ms_handle_reset on v2:172.21.15.133:6835/772257983 2024-09-10T21:52:06.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.425+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/991769382 >> 172.21.15.7:0/991769382 conn(0x560d1a9ea2a0 msgr2=0x560d1a9e27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:06.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.425+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/991769382 shutdown_connections 2024-09-10T21:52:06.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.425+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/991769382 wait complete. 2024-09-10T21:52:06.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.425+0000 7f8a9a1c3740 20 client.8436 trim_cache size 0 max 16384 2024-09-10T21:52:06.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.441+0000 7f8a9a1c3740 1 Processor -- start 2024-09-10T21:52:06.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.441+0000 7f8a9a1c3740 1 -- start start 2024-09-10T21:52:06.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.441+0000 7f8a9a1c3740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1a9ea820 0x560d1a9e6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.442+0000 7f8a9a1c3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1aa13620 0x560d1a9e74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.442+0000 7f8a9a1c3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa2f130 0x560d1aa2d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.442+0000 7f8a9a1c3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8a902014c0 con 0x560d1aa2f130 2024-09-10T21:52:06.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.442+0000 7f8a9a1c3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901f9160 con 0x560d1aa13620 2024-09-10T21:52:06.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.442+0000 7f8a9a1c3740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f8a901f6000 con 0x560d1a9ea820 2024-09-10T21:52:06.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.442+0000 7f8a95200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa2f130 0x560d1aa2d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.442+0000 7f8a95200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa2f130 0x560d1aa2d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38000/0 (socket says 172.21.15.7:38000) 2024-09-10T21:52:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.442+0000 7f8a95200640 1 -- 172.21.15.7:0/3974572350 learned_addr learned my addr 172.21.15.7:0/3974572350 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.442+0000 7f8a8fe00640 1 --2- 172.21.15.7:0/3974572350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1a9ea820 0x560d1a9e6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.442+0000 7f8a87400640 1 --2- 172.21.15.7:0/3974572350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1aa13620 0x560d1a9e74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.442+0000 7f8a95200640 1 -- 172.21.15.7:0/3974572350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1a9ea820 msgr2=0x560d1a9e6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.442+0000 7f8a95200640 1 --2- 172.21.15.7:0/3974572350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1a9ea820 0x560d1a9e6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.442+0000 7f8a95200640 1 -- 172.21.15.7:0/3974572350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1aa13620 msgr2=0x560d1a9e74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.442+0000 7f8a95200640 1 --2- 172.21.15.7:0/3974572350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1aa13620 0x560d1a9e74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.442+0000 7f8a95200640 1 -- 172.21.15.7:0/3974572350 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560d1aa27d20 con 0x560d1aa2f130 2024-09-10T21:52:06.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.443+0000 7f8a95200640 1 --2- 172.21.15.7:0/3974572350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa2f130 0x560d1aa2d630 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800cc40 tx=0x7f8a88098690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3288ec4c2c7fe91a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.443+0000 7f8a7fe00640 1 -- 172.21.15.7:0/3974572350 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a880619a0 con 0x560d1aa2f130 2024-09-10T21:52:06.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.443+0000 7f8a7fe00640 1 -- 172.21.15.7:0/3974572350 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a88079030 con 0x560d1aa2f130 2024-09-10T21:52:06.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.443+0000 7f8a7fe00640 1 -- 172.21.15.7:0/3974572350 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a880881c0 con 0x560d1aa2f130 2024-09-10T21:52:06.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.443+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/3974572350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa2f130 msgr2=0x560d1aa2d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.443+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/3974572350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa2f130 0x560d1aa2d630 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800cc40 tx=0x7f8a88098690 comp rx=0 tx=0).stop 2024-09-10T21:52:06.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.443+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/3974572350 shutdown_connections 2024-09-10T21:52:06.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.443+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/3974572350 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1aa13620 0x560d1a9e74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.443+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/3974572350 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1aa2f130 0x560d1aa2d630 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.443+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/3974572350 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1a9ea820 0x560d1a9e6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.443+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/3974572350 >> 172.21.15.7:0/3974572350 conn(0x560d1a9ea2a0 msgr2=0x560d1aa12d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:06.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.443+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/3974572350 shutdown_connections 2024-09-10T21:52:06.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.443+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/3974572350 wait complete. 2024-09-10T21:52:06.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.444+0000 7f8a9a1c3740 1 -- start start 2024-09-10T21:52:06.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.444+0000 7f8a9a1c3740 1 Processor -- start 2024-09-10T21:52:06.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.445+0000 7f8a9a1c3740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1a9ea820 0x560d1aaaf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.445+0000 7f8a9a1c3740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1aa13620 0x560d1a9ff480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.445+0000 7f8a9a1c3740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1aa2f130 0x560d1a9e6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.445+0000 7f8a9a1c3740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901f8b10 con 0x560d1a9ea820 2024-09-10T21:52:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.445+0000 7f8a9a1c3740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x560d1aaaed70 con 0x560d1aa2f130 2024-09-10T21:52:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.445+0000 7f8a9a1c3740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x560d1a9fef50 con 0x560d1aa13620 2024-09-10T21:52:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.445+0000 7f8a9a1c3740 10 client.0 _ll_register_callbacks cb 0x7f8a880625e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:52:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.446+0000 7f8a8fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1a9ea820 0x560d1aaaf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.446+0000 7f8a8fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1a9ea820 0x560d1aaaf630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38006/0 (socket says 172.21.15.7:38006) 2024-09-10T21:52:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.446+0000 7f8a8fe00640 1 -- 172.21.15.7:0/3956533343 learned_addr learned my addr 172.21.15.7:0/3956533343 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.446+0000 7f8a87400640 1 --2- 172.21.15.7:0/3956533343 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1aa13620 0x560d1a9ff480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.446+0000 7f8a95200640 1 --2- 172.21.15.7:0/3956533343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1aa2f130 0x560d1a9e6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.446+0000 7f8a8fe00640 1 -- 172.21.15.7:0/3956533343 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1aa13620 msgr2=0x560d1a9ff480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.446+0000 7f8a8fe00640 1 --2- 172.21.15.7:0/3956533343 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1aa13620 0x560d1a9ff480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.446+0000 7f8a8fe00640 1 -- 172.21.15.7:0/3956533343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1aa2f130 msgr2=0x560d1a9e6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.446+0000 7f8a8fe00640 1 --2- 172.21.15.7:0/3956533343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1aa2f130 0x560d1a9e6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.446+0000 7f8a8fe00640 1 -- 172.21.15.7:0/3956533343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560d1a9feca0 con 0x560d1a9ea820 2024-09-10T21:52:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.446+0000 7f8a8fe00640 1 --2- 172.21.15.7:0/3956533343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1a9ea820 0x560d1aaaf630 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000f5a0 tx=0x7f8a80000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc625c52e9b08b21 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.446+0000 7f8a7d600640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:52:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.446+0000 7f8a7d600640 1 -- 172.21.15.7:0/3956533343 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80025040 con 0x560d1a9ea820 2024-09-10T21:52:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.446+0000 7f8a7d600640 1 -- 172.21.15.7:0/3956533343 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 5853+0+0 (secure 0 0 0) 0x7f8a80020710 con 0x560d1a9ea820 2024-09-10T21:52:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.446+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/3956533343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a88079030 con 0x560d1a9ea820 2024-09-10T21:52:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.447+0000 7f8a7d600640 1 -- 172.21.15.7:0/3956533343 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7f8a80041050 con 0x560d1a9ea820 2024-09-10T21:52:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.447+0000 7f8a9a1c3740 10 client.8442 fetch_fsmap learned FSMap version 108 2024-09-10T21:52:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.447+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/3956533343 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x560d1aaafb70 con 0x560d1a9ea820 2024-09-10T21:52:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.447+0000 7f8a7d600640 1 -- 172.21.15.7:0/3956533343 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a88079030 con 0x560d1a9ea820 2024-09-10T21:52:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.447+0000 7f8a7d600640 1 -- 172.21.15.7:0/3956533343 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 108) ==== 3502+0+0 (secure 0 0 0) 0x7f8a80044020 con 0x560d1a9ea820 2024-09-10T21:52:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.447+0000 7f8a9a1c3740 10 client.8442 fetch_fsmap finished waiting for FSMap version 108 2024-09-10T21:52:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.447+0000 7f8a9a1c3740 10 client.8442 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-09-10T21:52:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.447+0000 7f8a9a1c3740 20 client.8442 populate_metadata read hostname 'smithi007' 2024-09-10T21:52:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.447+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/3956533343 >> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] conn(0x560d1a9e74f0 0x560d1aab2320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.447+0000 7f8a9a1c3740 4 client.8442 mds_command: new command op to 236723 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-10T21:52:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.447+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/3956533343 --> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560d1aa288e0 con 0x560d1a9e74f0 2024-09-10T21:52:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.447+0000 7f8a87400640 1 --2- 172.21.15.7:0/3956533343 >> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] conn(0x560d1a9e74f0 0x560d1aab2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.448+0000 7f8a87400640 1 --2- 172.21.15.7:0/3956533343 >> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] conn(0x560d1a9e74f0 0x560d1aab2320 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=734586ce661661ad server_cookie=7e7772feae6184ae in_seq=0 out_seq=0 2024-09-10T21:52:06.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.448+0000 7f8a7d600640 10 client.8442 ms_handle_connect on v2:172.21.15.133:6835/772257983 2024-09-10T21:52:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.460+0000 7f8a7d600640 1 -- 172.21.15.7:0/3956533343 <== mds.0 v2:172.21.15.133:6835/772257983 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560d1aa288e0 con 0x560d1a9e74f0 2024-09-10T21:52:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.460+0000 7f8a7d600640 10 client.8442 handle_command_reply: tid=0 2024-09-10T21:52:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.460+0000 7f8a9a1c3740 1 client.8442 shutdown 2024-09-10T21:52:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.460+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/3956533343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1a9ea820 msgr2=0x560d1aaaf630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.460+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/3956533343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1a9ea820 0x560d1aaaf630 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f8a8000f5a0 tx=0x7f8a80000fe0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.460+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/3956533343 shutdown_connections 2024-09-10T21:52:06.461 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.460+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/3956533343 >> [v2:172.21.15.133:6835/772257983,v1:172.21.15.133:6836/772257983] conn(0x560d1a9e74f0 0x560d1aab2320 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.460+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/3956533343 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x560d1aa2f130 0x560d1a9e6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.460+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/3956533343 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x560d1aa13620 0x560d1a9ff480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.460+0000 7f8a7d600640 0 client.8442 ms_handle_reset on v2:172.21.15.133:6835/772257983 2024-09-10T21:52:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.460+0000 7f8a9a1c3740 1 --2- 172.21.15.7:0/3956533343 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x560d1a9ea820 0x560d1aaaf630 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.460+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/3956533343 >> 172.21.15.7:0/3956533343 conn(0x560d1a9ea2a0 msgr2=0x560d1aa518e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.460+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/3956533343 shutdown_connections 2024-09-10T21:52:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.460+0000 7f8a9a1c3740 1 -- 172.21.15.7:0/3956533343 wait complete. 2024-09-10T21:52:06.462 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.460+0000 7f8a9a1c3740 20 client.8442 trim_cache size 0 max 16384 2024-09-10T21:52:06.462 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:52:06.462 INFO:teuthology.orchestra.run.smithi007.stdout: "message": "", 2024-09-10T21:52:06.463 INFO:teuthology.orchestra.run.smithi007.stdout: "return_code": 0 2024-09-10T21:52:06.463 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:52:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.461+0000 7f8a97200640 1 -- 172.21.15.7:0/1965689607 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8a6c077f70 msgr2=0x7f8a6c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.461+0000 7f8a97200640 1 --2- 172.21.15.7:0/1965689607 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8a6c077f70 0x7f8a6c07a430 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f8a800017c0 tx=0x7f8a80009040 comp rx=0 tx=0).stop 2024-09-10T21:52:06.463 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.461+0000 7f8a97200640 1 -- 172.21.15.7:0/1965689607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a9015f980 msgr2=0x7f8a901523b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.461+0000 7f8a97200640 1 --2- 172.21.15.7:0/1965689607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a9015f980 0x7f8a901523b0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800ef00 tx=0x7f8a8800c740 comp rx=0 tx=0).stop 2024-09-10T21:52:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.461+0000 7f8a97200640 1 -- 172.21.15.7:0/1965689607 shutdown_connections 2024-09-10T21:52:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.461+0000 7f8a97200640 1 --2- 172.21.15.7:0/1965689607 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f8a6c077f70 0x7f8a6c07a430 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.461+0000 7f8a97200640 1 --2- 172.21.15.7:0/1965689607 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f8a9015c9c0 0x7f8a901ea520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.461+0000 7f8a97200640 1 --2- 172.21.15.7:0/1965689607 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f8a9015f980 0x7f8a901523b0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.461+0000 7f8a97200640 1 --2- 172.21.15.7:0/1965689607 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f8a900720e0 0x7f8a90151540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.461+0000 7f8a97200640 1 -- 172.21.15.7:0/1965689607 >> 172.21.15.7:0/1965689607 conn(0x7f8a9006fa60 msgr2=0x7f8a900cdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.461+0000 7f8a97200640 1 -- 172.21.15.7:0/1965689607 shutdown_connections 2024-09-10T21:52:06.464 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.461+0000 7f8a97200640 1 -- 172.21.15.7:0/1965689607 wait complete. 2024-09-10T21:52:06.477 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:52:06.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.638+0000 7fca2e600640 1 -- 172.21.15.7:0/1867970591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca28159910 msgr2=0x7fca2815d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.638+0000 7fca2e600640 1 --2- 172.21.15.7:0/1867970591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca28159910 0x7fca2815d090 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fca18009860 tx=0x7fca1802a5a0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.638+0000 7fca2e600640 1 -- 172.21.15.7:0/1867970591 shutdown_connections 2024-09-10T21:52:06.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.638+0000 7fca2e600640 1 --2- 172.21.15.7:0/1867970591 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca28159910 0x7fca2815d090 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.638+0000 7fca2e600640 1 -- 172.21.15.7:0/1867970591 >> 172.21.15.7:0/1867970591 conn(0x7fca2806ca60 msgr2=0x7fca2806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:06.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.639+0000 7fca2e600640 1 -- 172.21.15.7:0/1867970591 shutdown_connections 2024-09-10T21:52:06.640 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.639+0000 7fca2e600640 1 -- 172.21.15.7:0/1867970591 wait complete. 2024-09-10T21:52:06.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.640+0000 7fca2e600640 1 Processor -- start 2024-09-10T21:52:06.641 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.640+0000 7fca2e600640 1 -- start start 2024-09-10T21:52:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.641+0000 7fca2e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca2806e810 0x7fca28143790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.641+0000 7fca2e600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca280cddc0 0x7fca280b2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.641+0000 7fca2e600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca28159910 0x7fca281f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.641+0000 7fca2e600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fca280739d0 con 0x7fca2806e810 2024-09-10T21:52:06.643 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.641+0000 7fca2e600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fca28073850 con 0x7fca280cddc0 2024-09-10T21:52:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.641+0000 7fca2e600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fca28073b50 con 0x7fca28159910 2024-09-10T21:52:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.641+0000 7fca27400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca2806e810 0x7fca28143790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.641+0000 7fca27400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca2806e810 0x7fca28143790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38016/0 (socket says 172.21.15.7:38016) 2024-09-10T21:52:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.641+0000 7fca27400640 1 -- 172.21.15.7:0/1995703375 learned_addr learned my addr 172.21.15.7:0/1995703375 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.642+0000 7fca26a00640 1 --2- 172.21.15.7:0/1995703375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca280cddc0 0x7fca280b2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.642+0000 7fca27e00640 1 --2- 172.21.15.7:0/1995703375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca28159910 0x7fca281f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.642+0000 7fca27400640 1 -- 172.21.15.7:0/1995703375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca28159910 msgr2=0x7fca281f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.642+0000 7fca27400640 1 --2- 172.21.15.7:0/1995703375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca28159910 0x7fca281f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.642+0000 7fca27400640 1 -- 172.21.15.7:0/1995703375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca280cddc0 msgr2=0x7fca280b2370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.642+0000 7fca27400640 1 --2- 172.21.15.7:0/1995703375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca280cddc0 0x7fca280b2370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.644 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.642+0000 7fca27400640 1 -- 172.21.15.7:0/1995703375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca281528a0 con 0x7fca2806e810 2024-09-10T21:52:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.642+0000 7fca27400640 1 --2- 172.21.15.7:0/1995703375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca2806e810 0x7fca28143790 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fca18009980 tx=0x7fca18033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.642+0000 7fca1f400640 1 -- 172.21.15.7:0/1995703375 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18031b50 con 0x7fca2806e810 2024-09-10T21:52:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.642+0000 7fca1f400640 1 -- 172.21.15.7:0/1995703375 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fca18031de0 con 0x7fca2806e810 2024-09-10T21:52:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.642+0000 7fca2e600640 1 -- 172.21.15.7:0/1995703375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca28152b40 con 0x7fca2806e810 2024-09-10T21:52:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.642+0000 7fca1f400640 1 -- 172.21.15.7:0/1995703375 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca18033330 con 0x7fca2806e810 2024-09-10T21:52:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.643+0000 7fca2e600640 1 -- 172.21.15.7:0/1995703375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca281533e0 con 0x7fca2806e810 2024-09-10T21:52:06.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.644+0000 7fca2e600640 1 -- 172.21.15.7:0/1995703375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca28152f30 con 0x7fca2806e810 2024-09-10T21:52:06.646 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.645+0000 7fca1f400640 1 -- 172.21.15.7:0/1995703375 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fca180334d0 con 0x7fca2806e810 2024-09-10T21:52:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.646+0000 7fca1f400640 1 --2- 172.21.15.7:0/1995703375 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fca08077f70 0x7fca0807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.646+0000 7fca1f400640 1 -- 172.21.15.7:0/1995703375 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5853+0+0 (secure 0 0 0) 0x7fca180bb150 con 0x7fca2806e810 2024-09-10T21:52:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.646+0000 7fca26a00640 1 --2- 172.21.15.7:0/1995703375 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fca08077f70 0x7fca0807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.647 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.647+0000 7fca26a00640 1 --2- 172.21.15.7:0/1995703375 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fca08077f70 0x7fca0807a430 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fca14007fd0 tx=0x7fca14009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.650+0000 7fca1f400640 1 -- 172.21.15.7:0/1995703375 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fca1803f030 con 0x7fca2806e810 2024-09-10T21:52:06.880 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.879+0000 7fca2e600640 1 -- 172.21.15.7:0/1995703375 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fca28070b50 con 0x7fca2806e810 2024-09-10T21:52:06.880 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-10T21:52:06.880+0000 7feb3fa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:52:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.882+0000 7fca1f400640 1 -- 172.21.15.7:0/1995703375 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v109) ==== 161+0+0 (secure 0 0 0) 0x7fca18083aa0 con 0x7fca2806e810 2024-09-10T21:52:06.883 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:52:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.885+0000 7fca2e600640 1 -- 172.21.15.7:0/1995703375 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fca08077f70 msgr2=0x7fca0807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.885+0000 7fca2e600640 1 --2- 172.21.15.7:0/1995703375 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fca08077f70 0x7fca0807a430 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fca14007fd0 tx=0x7fca14009040 comp rx=0 tx=0).stop 2024-09-10T21:52:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.885+0000 7fca2e600640 1 -- 172.21.15.7:0/1995703375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca2806e810 msgr2=0x7fca28143790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.885+0000 7fca2e600640 1 --2- 172.21.15.7:0/1995703375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca2806e810 0x7fca28143790 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fca18009980 tx=0x7fca18033ba0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.885+0000 7fca2e600640 1 -- 172.21.15.7:0/1995703375 shutdown_connections 2024-09-10T21:52:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.885+0000 7fca2e600640 1 --2- 172.21.15.7:0/1995703375 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fca08077f70 0x7fca0807a430 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.885+0000 7fca2e600640 1 --2- 172.21.15.7:0/1995703375 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fca280cddc0 0x7fca280b2370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.885+0000 7fca2e600640 1 --2- 172.21.15.7:0/1995703375 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fca28159910 0x7fca281f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.885+0000 7fca2e600640 1 --2- 172.21.15.7:0/1995703375 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fca2806e810 0x7fca28143790 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.885+0000 7fca2e600640 1 -- 172.21.15.7:0/1995703375 >> 172.21.15.7:0/1995703375 conn(0x7fca2806ca60 msgr2=0x7fca2814e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.885+0000 7fca2e600640 1 -- 172.21.15.7:0/1995703375 shutdown_connections 2024-09-10T21:52:06.886 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.885+0000 7fca2e600640 1 -- 172.21.15.7:0/1995703375 wait complete. 2024-09-10T21:52:06.897 INFO:tasks.ceph.mds.d.smithi133.stderr: -1283> 2024-09-10T21:52:04.457+0000 7f3843800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:52:06.901 DEBUG:teuthology.orchestra.run.smithi007:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank cephfs:0 journal reset --force --yes-i-really-really-mean-it 2024-09-10T21:52:06.940 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2024-09-10T21:52:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.978+0000 7f500e97a280 1 -- start start 2024-09-10T21:52:06.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.978+0000 7f500e97a280 1 Processor -- start 2024-09-10T21:52:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.979+0000 7f500e97a280 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5643510bac00 0x564351e8e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.979+0000 7f500e97a280 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643510bb800 0x564351e8e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.979+0000 7f500e97a280 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643510bbc00 0x564351e8eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.979+0000 7f500e97a280 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564351eb0600 con 0x5643510bbc00 2024-09-10T21:52:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.979+0000 7f500e97a280 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564351eb0480 con 0x5643510bb800 2024-09-10T21:52:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.979+0000 7f500e97a280 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x564351eb0300 con 0x5643510bac00 2024-09-10T21:52:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.980+0000 7f500d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643510bbc00 0x564351e8eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.980+0000 7f500ca00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5643510bac00 0x564351e8e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.981 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.980+0000 7f500c000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643510bb800 0x564351e8e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.980+0000 7f500d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643510bbc00 0x564351e8eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38026/0 (socket says 172.21.15.7:38026) 2024-09-10T21:52:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.980+0000 7f500d400640 1 -- 172.21.15.7:0/875606562 learned_addr learned my addr 172.21.15.7:0/875606562 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.980+0000 7f500ca00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5643510bac00 0x564351e8e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3301/0 says I am v2:172.21.15.7:45458/0 (socket says 172.21.15.7:45458) 2024-09-10T21:52:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.980+0000 7f500ca00640 1 -- 172.21.15.7:0/875606562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643510bb800 msgr2=0x564351e8e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.980+0000 7f500ca00640 1 --2- 172.21.15.7:0/875606562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643510bb800 0x564351e8e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.980+0000 7f500ca00640 1 -- 172.21.15.7:0/875606562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643510bbc00 msgr2=0x564351e8eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.980+0000 7f500ca00640 1 --2- 172.21.15.7:0/875606562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643510bbc00 0x564351e8eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.980+0000 7f500ca00640 1 -- 172.21.15.7:0/875606562 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643510f2780 con 0x5643510bac00 2024-09-10T21:52:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.980+0000 7f500d400640 1 --2- 172.21.15.7:0/875606562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643510bbc00 0x564351e8eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:52:06.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.980+0000 7f500ca00640 1 --2- 172.21.15.7:0/875606562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5643510bac00 0x564351e8e000 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x564351ee9290 tx=0x564351e69200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a66b167ae7782873 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.981+0000 7f5009800640 1 -- 172.21.15.7:0/875606562 <== mon.2 v2:172.21.15.133:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5643510e0ea0 con 0x5643510bac00 2024-09-10T21:52:06.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.981+0000 7f5009800640 1 -- 172.21.15.7:0/875606562 <== mon.2 v2:172.21.15.133:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5643510e6540 con 0x5643510bac00 2024-09-10T21:52:06.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.981+0000 7f5009800640 1 -- 172.21.15.7:0/875606562 <== mon.2 v2:172.21.15.133:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5643510e0340 con 0x5643510bac00 2024-09-10T21:52:06.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.981+0000 7f500e97a280 1 -- 172.21.15.7:0/875606562 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({osdmap=0}) -- 0x5643510f3680 con 0x5643510bac00 2024-09-10T21:52:06.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.981+0000 7f500e97a280 1 -- 172.21.15.7:0/875606562 --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_subscribe({fsmap=0}) -- 0x5643510f2000 con 0x5643510bac00 2024-09-10T21:52:06.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.981+0000 7f500e97a280 4 waiting for MDS map... 2024-09-10T21:52:06.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.981+0000 7f5009800640 1 -- 172.21.15.7:0/875606562 <== mon.2 v2:172.21.15.133:3301/0 4 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x564351f9c400 con 0x5643510bac00 2024-09-10T21:52:06.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.981+0000 7f5009800640 1 -- 172.21.15.7:0/875606562 <== mon.2 v2:172.21.15.133:3301/0 5 ==== fsmap(e 109) ==== 2893+0+0 (secure 0 0 0) 0x564351115180 con 0x5643510bac00 2024-09-10T21:52:06.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.981+0000 7f500e97a280 4 Got MDS map 109 2024-09-10T21:52:06.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.981+0000 7f500e97a280 10 main: JournalTool::main 2024-09-10T21:52:06.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.981+0000 7f500e97a280 4 main: JournalTool: connecting to RADOS... 2024-09-10T21:52:06.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.982+0000 7f500e97a280 1 Processor -- start 2024-09-10T21:52:06.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.982+0000 7f500e97a280 1 -- start start 2024-09-10T21:52:06.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.983+0000 7f500e97a280 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564351fda400 0x564351e8f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.983+0000 7f500e97a280 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564351fda800 0x564351e8f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.983+0000 7f500e97a280 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564351fdac00 0x564351e8fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.983+0000 7f500e97a280 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564351eb0180 con 0x564351fda400 2024-09-10T21:52:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.983+0000 7f500e97a280 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564351eb0900 con 0x564351fdac00 2024-09-10T21:52:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.983+0000 7f500e97a280 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x56435111d800 con 0x564351fda800 2024-09-10T21:52:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.983+0000 7f500d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564351fda400 0x564351e8f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.983+0000 7f500d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564351fda400 0x564351e8f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38032/0 (socket says 172.21.15.7:38032) 2024-09-10T21:52:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.983+0000 7f500d400640 1 -- 172.21.15.7:0/133177292 learned_addr learned my addr 172.21.15.7:0/133177292 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.983+0000 7f500c000640 1 --2- 172.21.15.7:0/133177292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564351fdac00 0x564351e8fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.983+0000 7f500ca00640 1 --2- 172.21.15.7:0/133177292 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564351fda800 0x564351e8f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.983+0000 7f500d400640 1 -- 172.21.15.7:0/133177292 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564351fda800 msgr2=0x564351e8f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.983+0000 7f500d400640 1 --2- 172.21.15.7:0/133177292 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564351fda800 0x564351e8f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.983+0000 7f500d400640 1 -- 172.21.15.7:0/133177292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564351fdac00 msgr2=0x564351e8fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.983+0000 7f500d400640 1 --2- 172.21.15.7:0/133177292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564351fdac00 0x564351e8fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.983+0000 7f500d400640 1 -- 172.21.15.7:0/133177292 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643510f23c0 con 0x564351fda400 2024-09-10T21:52:06.987 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.983+0000 7f500c000640 1 --2- 172.21.15.7:0/133177292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564351fdac00 0x564351e8fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:52:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.983+0000 7f500ca00640 1 --2- 172.21.15.7:0/133177292 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564351fda800 0x564351e8f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:52:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.983+0000 7f500d400640 1 --2- 172.21.15.7:0/133177292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564351fda400 0x564351e8f080 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x564351ecef30 tx=0x564351ff2540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25d7c13ed84181ae server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.984+0000 7f5007000640 1 -- 172.21.15.7:0/133177292 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5643510e0ea0 con 0x564351fda400 2024-09-10T21:52:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.984+0000 7f5007000640 1 -- 172.21.15.7:0/133177292 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5643510e6fc0 con 0x564351fda400 2024-09-10T21:52:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.984+0000 7f5007000640 1 -- 172.21.15.7:0/133177292 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5643510e0340 con 0x564351fda400 2024-09-10T21:52:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.984+0000 7f500e97a280 1 -- 172.21.15.7:0/133177292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564351fda400 msgr2=0x564351e8f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.984+0000 7f500e97a280 1 --2- 172.21.15.7:0/133177292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564351fda400 0x564351e8f080 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x564351ecef30 tx=0x564351ff2540 comp rx=0 tx=0).stop 2024-09-10T21:52:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.984+0000 7f500e97a280 1 -- 172.21.15.7:0/133177292 shutdown_connections 2024-09-10T21:52:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.984+0000 7f500e97a280 1 --2- 172.21.15.7:0/133177292 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564351fdac00 0x564351e8fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.984+0000 7f500e97a280 1 --2- 172.21.15.7:0/133177292 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564351fda800 0x564351e8f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.984+0000 7f500e97a280 1 --2- 172.21.15.7:0/133177292 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564351fda400 0x564351e8f080 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.984+0000 7f500e97a280 1 -- 172.21.15.7:0/133177292 >> 172.21.15.7:0/133177292 conn(0x564351fda000 msgr2=0x5643511a5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.984+0000 7f500e97a280 1 -- 172.21.15.7:0/133177292 shutdown_connections 2024-09-10T21:52:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.984+0000 7f500e97a280 1 -- 172.21.15.7:0/133177292 wait complete. 2024-09-10T21:52:06.988 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.985+0000 7f500e97a280 1 Processor -- start 2024-09-10T21:52:06.989 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.985+0000 7f500e97a280 1 -- start start 2024-09-10T21:52:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.986+0000 7f500e97a280 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564351fda400 0x564351e8f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.986+0000 7f500e97a280 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564351fda800 0x564351e8f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.986+0000 7f500e97a280 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564351fdac00 0x564351e8fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.986+0000 7f500e97a280 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564351eb0180 con 0x564351fdac00 2024-09-10T21:52:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.986+0000 7f500e97a280 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564351eb0900 con 0x564351fda400 2024-09-10T21:52:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.986+0000 7f500e97a280 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x564351eb1500 con 0x564351fda800 2024-09-10T21:52:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.986+0000 7f500c000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564351fdac00 0x564351e8fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.986+0000 7f500c000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564351fdac00 0x564351e8fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38034/0 (socket says 172.21.15.7:38034) 2024-09-10T21:52:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.986+0000 7f500c000640 1 -- 172.21.15.7:0/160350026 learned_addr learned my addr 172.21.15.7:0/160350026 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.986+0000 7f500ca00640 1 --2- 172.21.15.7:0/160350026 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564351fda800 0x564351e8f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.991 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.986+0000 7f500c000640 1 -- 172.21.15.7:0/160350026 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564351fda800 msgr2=0x564351e8f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.986+0000 7f500d400640 1 --2- 172.21.15.7:0/160350026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564351fda400 0x564351e8f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.986+0000 7f500c000640 1 --2- 172.21.15.7:0/160350026 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564351fda800 0x564351e8f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.986+0000 7f500c000640 1 -- 172.21.15.7:0/160350026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564351fda400 msgr2=0x564351e8f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.986+0000 7f500c000640 1 --2- 172.21.15.7:0/160350026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564351fda400 0x564351e8f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.987+0000 7f500c000640 1 -- 172.21.15.7:0/160350026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643510f2960 con 0x564351fdac00 2024-09-10T21:52:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.987+0000 7f500ca00640 1 --2- 172.21.15.7:0/160350026 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564351fda800 0x564351e8f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:52:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.987+0000 7f500d400640 1 --2- 172.21.15.7:0/160350026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564351fda400 0x564351e8f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:52:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.987+0000 7f500c000640 1 --2- 172.21.15.7:0/160350026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564351fdac00 0x564351e8fb80 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x564351ecfd40 tx=0x564351ff2660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.987+0000 7f5005200640 1 -- 172.21.15.7:0/160350026 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5643510e0ea0 con 0x564351fdac00 2024-09-10T21:52:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.987+0000 7f5005200640 1 -- 172.21.15.7:0/160350026 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5643510e6700 con 0x564351fdac00 2024-09-10T21:52:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.987+0000 7f500e97a280 1 -- 172.21.15.7:0/160350026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5643510f3860 con 0x564351fdac00 2024-09-10T21:52:06.992 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.987+0000 7f5005200640 1 -- 172.21.15.7:0/160350026 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5643510e0340 con 0x564351fdac00 2024-09-10T21:52:06.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.987+0000 7f500e97a280 1 -- 172.21.15.7:0/160350026 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643510f3a40 con 0x564351fdac00 2024-09-10T21:52:06.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.987+0000 7f500e97a280 4 main: JournalTool: resolving pool 16 2024-09-10T21:52:06.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.989+0000 7f5005200640 1 -- 172.21.15.7:0/160350026 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x564351115500 con 0x564351fdac00 2024-09-10T21:52:06.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.989+0000 7f5005200640 1 --2- 172.21.15.7:0/160350026 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x564352006c00 0x564351e90100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.989+0000 7f5005200640 1 -- 172.21.15.7:0/160350026 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x564352002a00 con 0x564351fdac00 2024-09-10T21:52:06.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.989+0000 7f500e97a280 4 main: JournalTool: creating IoCtx.. 2024-09-10T21:52:06.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.989+0000 7f500e97a280 4 main: Executing for rank 0 2024-09-10T21:52:06.993 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.990+0000 7f500d400640 1 --2- 172.21.15.7:0/160350026 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x564352006c00 0x564351e90100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.990+0000 7f500e97a280 1 -- start start 2024-09-10T21:52:06.995 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.990+0000 7f500d400640 1 --2- 172.21.15.7:0/160350026 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x564352006c00 0x564351e90100 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x564351ecef30 tx=0x564352046cc0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.990+0000 7f500e97a280 1 Processor -- start 2024-09-10T21:52:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.991+0000 7f500e97a280 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564352007400 0x564351e90680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.991+0000 7f500e97a280 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564352007800 0x564351e90c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.991+0000 7f500e97a280 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564352007c00 0x564351e91180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.991+0000 7f500e97a280 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x564351eb1800 con 0x564352007800 2024-09-10T21:52:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.991+0000 7f500e97a280 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x564351eb1980 con 0x564352007400 2024-09-10T21:52:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.991+0000 7f500e97a280 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x564351eb1b00 con 0x564352007c00 2024-09-10T21:52:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.992+0000 7f500ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564352007800 0x564351e90c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.992+0000 7f500ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564352007800 0x564351e90c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38048/0 (socket says 172.21.15.7:38048) 2024-09-10T21:52:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.992+0000 7f500ca00640 1 -- 172.21.15.7:0/653064023 learned_addr learned my addr 172.21.15.7:0/653064023 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:06.996 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.992+0000 7f500d400640 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564352007400 0x564351e90680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.992+0000 7f500c000640 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564352007c00 0x564351e91180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.992+0000 7f500ca00640 1 -- 172.21.15.7:0/653064023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564352007c00 msgr2=0x564351e91180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.992+0000 7f500ca00640 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564352007c00 0x564351e91180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.992+0000 7f500ca00640 1 -- 172.21.15.7:0/653064023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564352007400 msgr2=0x564351e90680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.992+0000 7f500ca00640 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564352007400 0x564351e90680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.992+0000 7f500ca00640 1 -- 172.21.15.7:0/653064023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643510f3c20 con 0x564352007800 2024-09-10T21:52:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.992+0000 7f500d400640 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564352007400 0x564351e90680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.992+0000 7f500ca00640 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564352007800 0x564351e90c00 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x564351ee9f20 tx=0x564351ff2120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c43c0f94f55919d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.993+0000 7f5001600640 1 -- 172.21.15.7:0/653064023 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5643520291e0 con 0x564352007800 2024-09-10T21:52:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.993+0000 7f5001600640 1 -- 172.21.15.7:0/653064023 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5643510e7180 con 0x564352007800 2024-09-10T21:52:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.993+0000 7f5001600640 1 -- 172.21.15.7:0/653064023 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x564352043ba0 con 0x564352007800 2024-09-10T21:52:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.993+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643510f2960 con 0x564352007800 2024-09-10T21:52:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.993+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5643510f3860 con 0x564352007800 2024-09-10T21:52:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.993+0000 7f500e97a280 4 waiting for MDS map... 2024-09-10T21:52:06.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.993+0000 7f5001600640 1 -- 172.21.15.7:0/653064023 <== mon.0 v2:172.21.15.7:3300/0 4 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x5643514b8400 con 0x564352007800 2024-09-10T21:52:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.993+0000 7f5001600640 1 -- 172.21.15.7:0/653064023 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 109) ==== 2893+0+0 (secure 0 0 0) 0x564351115880 con 0x564352007800 2024-09-10T21:52:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.993+0000 7f500e97a280 4 Got MDS map 109 2024-09-10T21:52:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.993+0000 7f500e97a280 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5643514b7800 0x564351e91700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.993+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:1 16.f 16:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5643514b7c00 con 0x5643514b7800 2024-09-10T21:52:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.993+0000 7f500d400640 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5643514b7800 0x564351e91700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:06.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.994+0000 7f500d400640 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5643514b7800 0x564351e91700 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=1ab61752e692940b server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:06.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.996+0000 7f500d400640 1 -- 172.21.15.7:0/653064023 <== osd.0 v2:172.21.15.7:6808/651402908 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v118'2 uv2 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x564351e54000 con 0x5643514b7800 2024-09-10T21:52:06.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.996+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:2 16.14 16:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5643514d8000 con 0x5643514b7800 2024-09-10T21:52:06.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.997+0000 7f500d400640 1 -- 172.21.15.7:0/653064023 <== osd.0 v2:172.21.15.7:6808/651402908 2 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v118'39 uv39 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x564351e54000 con 0x5643514b7800 2024-09-10T21:52:06.999 INFO:teuthology.orchestra.run.smithi007.stdout:writing EResetJournal entry 2024-09-10T21:52:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.997+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:3 16.14 16:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5643514d8400 con 0x5643514b7800 2024-09-10T21:52:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.997+0000 7f500e97a280 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5643514d8800 0x5643514e6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.997+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:4 16.1b 16:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5643514d8c00 con 0x5643514d8800 2024-09-10T21:52:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.997+0000 7f500e97a280 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5643514d9000 0x5643514e6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.997+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:5 16.b 16:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5643514d9400 con 0x5643514d9000 2024-09-10T21:52:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.997+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:6 16.1f 16:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5643514d9800 con 0x5643514d8800 2024-09-10T21:52:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.997+0000 7f500e97a280 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5643514d9c00 0x5643514e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.997+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:7 16.1a 16:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5643514ee000 con 0x5643514d9c00 2024-09-10T21:52:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.997+0000 7f500e97a280 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5643514ee400 0x5643514e7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.997+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:8 16.1e 16:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5643514ee800 con 0x5643514ee400 2024-09-10T21:52:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.998+0000 7f500d400640 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5643514d9000 0x5643514e6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.998+0000 7f500ca00640 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5643514d9c00 0x5643514e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.998+0000 7f500c000640 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5643514d8800 0x5643514e6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.998+0000 7f500c000640 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5643514ee400 0x5643514e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.998+0000 7f500d400640 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5643514d9000 0x5643514e6580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=8ce545a90cedf9eb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.999+0000 7f500ca00640 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5643514d9c00 0x5643514e6b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=acc15414ddebcb77 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.999+0000 7f500c000640 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5643514d8800 0x5643514e6000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=637af557ad186014 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.999+0000 7f500c000640 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5643514ee400 0x5643514e7080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=1e93696dc59024d0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:06.999+0000 7f500d400640 1 -- 172.21.15.7:0/653064023 <== osd.0 v2:172.21.15.7:6808/651402908 3 ==== osd_op_reply(3 200.00000001 [delete] v118'40 uv40 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x564351e54000 con 0x5643514b7800 2024-09-10T21:52:07.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.000+0000 7f500d400640 1 -- 172.21.15.7:0/653064023 <== osd.1 v2:172.21.15.7:6824/4175601737 1 ==== osd_op_reply(5 200.00000003 [delete] v118'6 uv5 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x564351e54000 con 0x5643514d9000 2024-09-10T21:52:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.000+0000 7f500c000640 1 -- 172.21.15.7:0/653064023 <== osd.5 v2:172.21.15.133:6818/3838498382 1 ==== osd_op_reply(4 200.00000002 [delete] v118'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x564351e54500 con 0x5643514d8800 2024-09-10T21:52:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.000+0000 7f500ca00640 1 -- 172.21.15.7:0/653064023 <== osd.3 v2:172.21.15.7:6800/2195665006 1 ==== osd_op_reply(7 200.00000005 [delete] v118'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x564351e54780 con 0x5643514d9c00 2024-09-10T21:52:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.000+0000 7f4fff800640 1 -- 172.21.15.7:0/653064023 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:9 16.14 16:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x564352006800 con 0x5643514b7800 2024-09-10T21:52:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.000+0000 7f500c000640 1 -- 172.21.15.7:0/653064023 <== osd.5 v2:172.21.15.133:6818/3838498382 2 ==== osd_op_reply(6 200.00000004 [delete] v118'9 uv8 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x564351e54500 con 0x5643514d8800 2024-09-10T21:52:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.000+0000 7f500c000640 1 -- 172.21.15.7:0/653064023 <== osd.7 v2:172.21.15.133:6802/2638399040 1 ==== osd_op_reply(8 200.00000006 [delete] v118'3 uv0 ondisk = -2 ((2) No such file or directory)) ==== 156+0+0 (crc 0 0 0) 0x564351e54500 con 0x5643514ee400 2024-09-10T21:52:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500d400640 1 -- 172.21.15.7:0/653064023 <== osd.0 v2:172.21.15.7:6808/651402908 4 ==== osd_op_reply(9 200.00000001 [write 0~48] v118'41 uv41 ondisk = 0) ==== 156+0+0 (crc 0 0 0) 0x564351e54000 con 0x5643514b7800 2024-09-10T21:52:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 4 Successfully wrote new journal pointer and header for rank 8:0 2024-09-10T21:52:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5643514b7800 msgr2=0x564351e91700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5643514b7800 0x564351e91700 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5643514d9000 msgr2=0x5643514e6580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5643514d9000 0x5643514e6580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5643514d9c00 msgr2=0x5643514e6b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5643514d9c00 0x5643514e6b00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5643514d8800 msgr2=0x5643514e6000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5643514d8800 0x5643514e6000 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5643514ee400 msgr2=0x5643514e7080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5643514ee400 0x5643514e7080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500ca00640 1 -- 172.21.15.7:0/653064023 reap_dead start 2024-09-10T21:52:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564352007800 msgr2=0x564351e90c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 --2- 172.21.15.7:0/653064023 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564352007800 0x564351e90c00 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x564351ee9f20 tx=0x564351ff2120 comp rx=0 tx=0).stop 2024-09-10T21:52:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500ca00640 1 -- 172.21.15.7:0/653064023 reap_dead start 2024-09-10T21:52:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 shutdown_connections 2024-09-10T21:52:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 >> 172.21.15.7:0/653064023 conn(0x564352007000 msgr2=0x5643511a5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 shutdown_connections 2024-09-10T21:52:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 -- 172.21.15.7:0/653064023 wait complete. 2024-09-10T21:52:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 -- 172.21.15.7:0/160350026 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x564352006c00 msgr2=0x564351e90100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 --2- 172.21.15.7:0/160350026 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x564352006c00 0x564351e90100 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x564351ecef30 tx=0x564352046cc0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 -- 172.21.15.7:0/160350026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564351fdac00 msgr2=0x564351e8fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.002+0000 7f500e97a280 1 --2- 172.21.15.7:0/160350026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564351fdac00 0x564351e8fb80 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x564351ecfd40 tx=0x564351ff2660 comp rx=0 tx=0).stop 2024-09-10T21:52:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.003+0000 7f500e97a280 1 -- 172.21.15.7:0/160350026 shutdown_connections 2024-09-10T21:52:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.003+0000 7f500e97a280 1 --2- 172.21.15.7:0/160350026 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x564352006c00 0x564351e90100 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.003+0000 7f500e97a280 1 --2- 172.21.15.7:0/160350026 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x564351fdac00 0x564351e8fb80 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.003+0000 7f500e97a280 1 --2- 172.21.15.7:0/160350026 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x564351fda800 0x564351e8f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.003+0000 7f500e97a280 1 --2- 172.21.15.7:0/160350026 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x564351fda400 0x564351e8f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.003+0000 7f500e97a280 1 -- 172.21.15.7:0/160350026 >> 172.21.15.7:0/160350026 conn(0x564351fda000 msgr2=0x5643511a5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.003+0000 7f500e97a280 1 -- 172.21.15.7:0/160350026 shutdown_connections 2024-09-10T21:52:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.003+0000 7f500e97a280 1 -- 172.21.15.7:0/160350026 wait complete. 2024-09-10T21:52:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.003+0000 7f500e97a280 1 -- 172.21.15.7:0/875606562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5643510bac00 msgr2=0x564351e8e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.003+0000 7f500e97a280 1 --2- 172.21.15.7:0/875606562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5643510bac00 0x564351e8e000 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x564351ee9290 tx=0x564351e69200 comp rx=0 tx=0).stop 2024-09-10T21:52:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.003+0000 7f500e97a280 1 -- 172.21.15.7:0/875606562 shutdown_connections 2024-09-10T21:52:07.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.003+0000 7f500e97a280 1 --2- 172.21.15.7:0/875606562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5643510bb800 0x564351e8e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.003+0000 7f500e97a280 1 --2- 172.21.15.7:0/875606562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5643510bbc00 0x564351e8eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.003+0000 7f500e97a280 1 --2- 172.21.15.7:0/875606562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5643510bac00 0x564351e8e000 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.003+0000 7f500e97a280 1 -- 172.21.15.7:0/875606562 >> 172.21.15.7:0/875606562 conn(0x5643510bb000 msgr2=0x5643511a4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:07.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.003+0000 7f500e97a280 1 -- 172.21.15.7:0/875606562 shutdown_connections 2024-09-10T21:52:07.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.003+0000 7f500e97a280 1 -- 172.21.15.7:0/875606562 wait complete. 2024-09-10T21:52:07.009 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'cephfs:0', 'journal', 'reset', '--force', '--yes-i-really-really-mean-it'] in time 0:00:00.108116, result: writing EResetJournal entry 2024-09-10T21:52:07.009 DEBUG:teuthology.orchestra.run.smithi007:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents cephfs_data 2024-09-10T21:52:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.080+0000 7f437fe0c740 1 -- start start 2024-09-10T21:52:07.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.081+0000 7f437fe0c740 1 Processor -- start 2024-09-10T21:52:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.081+0000 7f437fe0c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b8ab4c00 0x5626b8c30000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.081+0000 7f437fe0c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b8ab5800 0x5626b8c30580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.081+0000 7f437fe0c740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b8ab5c00 0x5626b8c30b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.081+0000 7f437fe0c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5626b98a4600 con 0x5626b8ab5800 2024-09-10T21:52:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.081+0000 7f437fe0c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5626b98a4480 con 0x5626b8ab4c00 2024-09-10T21:52:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.081+0000 7f437fe0c740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5626b98a4300 con 0x5626b8ab5c00 2024-09-10T21:52:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.082+0000 7f437ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b8ab5800 0x5626b8c30580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.082+0000 7f437d400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b8ab4c00 0x5626b8c30000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.082+0000 7f437ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b8ab5800 0x5626b8c30580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38064/0 (socket says 172.21.15.7:38064) 2024-09-10T21:52:07.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.082+0000 7f437ca00640 1 -- 172.21.15.7:0/2495022616 learned_addr learned my addr 172.21.15.7:0/2495022616 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:07.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.082+0000 7f437de00640 1 --2- 172.21.15.7:0/2495022616 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b8ab5c00 0x5626b8c30b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.082+0000 7f437ca00640 1 -- 172.21.15.7:0/2495022616 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b8ab5c00 msgr2=0x5626b8c30b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:07.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.082+0000 7f437ca00640 1 --2- 172.21.15.7:0/2495022616 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b8ab5c00 0x5626b8c30b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.082+0000 7f437ca00640 1 -- 172.21.15.7:0/2495022616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b8ab4c00 msgr2=0x5626b8c30000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:07.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.082+0000 7f437ca00640 1 --2- 172.21.15.7:0/2495022616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b8ab4c00 0x5626b8c30000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.082+0000 7f437ca00640 1 -- 172.21.15.7:0/2495022616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626b8ae1680 con 0x5626b8ab5800 2024-09-10T21:52:07.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.082+0000 7f437d400640 1 --2- 172.21.15.7:0/2495022616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b8ab4c00 0x5626b8c30000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:52:07.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.082+0000 7f437ca00640 1 --2- 172.21.15.7:0/2495022616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b8ab5800 0x5626b8c30580 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x5626b9992090 tx=0x5626b8b7d080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=595620796584f1db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.083+0000 7f437a200640 1 -- 172.21.15.7:0/2495022616 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5626b8adaea0 con 0x5626b8ab5800 2024-09-10T21:52:07.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.083+0000 7f437a200640 1 -- 172.21.15.7:0/2495022616 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5626b8b0a8c0 con 0x5626b8ab5800 2024-09-10T21:52:07.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.083+0000 7f437a200640 1 -- 172.21.15.7:0/2495022616 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5626b8adb380 con 0x5626b8ab5800 2024-09-10T21:52:07.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.083+0000 7f437fe0c740 1 -- 172.21.15.7:0/2495022616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626b8ae14a0 con 0x5626b8ab5800 2024-09-10T21:52:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.083+0000 7f437fe0c740 1 -- 172.21.15.7:0/2495022616 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5626b8ae1a40 con 0x5626b8ab5800 2024-09-10T21:52:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.083+0000 7f437fe0c740 4 waiting for MDS map... 2024-09-10T21:52:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.083+0000 7f437a200640 1 -- 172.21.15.7:0/2495022616 <== mon.0 v2:172.21.15.7:3300/0 4 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x5626b99a8600 con 0x5626b8ab5800 2024-09-10T21:52:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.083+0000 7f437a200640 1 -- 172.21.15.7:0/2495022616 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 109) ==== 2893+0+0 (secure 0 0 0) 0x5626b8b0f180 con 0x5626b8ab5800 2024-09-10T21:52:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.083+0000 7f437fe0c740 4 Got MDS map 109 2024-09-10T21:52:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.083+0000 7f437fe0c740 4 datascan.main: Using metadata pool output 2024-09-10T21:52:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.083+0000 7f437fe0c740 4 datascan.main: connecting to RADOS... 2024-09-10T21:52:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.084+0000 7f437fe0c740 1 Processor -- start 2024-09-10T21:52:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.084+0000 7f437fe0c740 1 -- start start 2024-09-10T21:52:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.085+0000 7f437fe0c740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b98a1400 0x5626b8c31080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.085+0000 7f437fe0c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b98a1800 0x5626b8c31600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.085+0000 7f437fe0c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b98a1c00 0x5626b8c31b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.085+0000 7f437fe0c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5626b98a4180 con 0x5626b98a1c00 2024-09-10T21:52:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.085+0000 7f437fe0c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5626b8b17380 con 0x5626b98a1800 2024-09-10T21:52:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.085+0000 7f437fe0c740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5626b8b17800 con 0x5626b98a1400 2024-09-10T21:52:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.085+0000 7f437ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b98a1c00 0x5626b8c31b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.085+0000 7f437ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b98a1c00 0x5626b8c31b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38070/0 (socket says 172.21.15.7:38070) 2024-09-10T21:52:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.085+0000 7f437ca00640 1 -- 172.21.15.7:0/2866710554 learned_addr learned my addr 172.21.15.7:0/2866710554 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.085+0000 7f437de00640 1 --2- 172.21.15.7:0/2866710554 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b98a1400 0x5626b8c31080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.085+0000 7f437d400640 1 --2- 172.21.15.7:0/2866710554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b98a1800 0x5626b8c31600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.085+0000 7f437ca00640 1 -- 172.21.15.7:0/2866710554 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b98a1400 msgr2=0x5626b8c31080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.085+0000 7f437ca00640 1 --2- 172.21.15.7:0/2866710554 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b98a1400 0x5626b8c31080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.085+0000 7f437ca00640 1 -- 172.21.15.7:0/2866710554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b98a1800 msgr2=0x5626b8c31600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.086+0000 7f437ca00640 1 --2- 172.21.15.7:0/2866710554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b98a1800 0x5626b8c31600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.086+0000 7f437ca00640 1 -- 172.21.15.7:0/2866710554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626b8ae1860 con 0x5626b98a1c00 2024-09-10T21:52:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.086+0000 7f437de00640 1 --2- 172.21.15.7:0/2866710554 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b98a1400 0x5626b8c31080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:07.089 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.086+0000 7f437d400640 1 --2- 172.21.15.7:0/2866710554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b98a1800 0x5626b8c31600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:07.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.086+0000 7f437ca00640 1 --2- 172.21.15.7:0/2866710554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b98a1c00 0x5626b8c31b80 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x5626b9993500 tx=0x5626b983e660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb672b4d76fdfa52 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.086+0000 7f4377a00640 1 -- 172.21.15.7:0/2866710554 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5626b8adb1e0 con 0x5626b98a1c00 2024-09-10T21:52:07.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.086+0000 7f4377a00640 1 -- 172.21.15.7:0/2866710554 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5626b8b0ae00 con 0x5626b98a1c00 2024-09-10T21:52:07.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.086+0000 7f4377a00640 1 -- 172.21.15.7:0/2866710554 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5626b8adaea0 con 0x5626b98a1c00 2024-09-10T21:52:07.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.086+0000 7f437fe0c740 1 -- 172.21.15.7:0/2866710554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b98a1c00 msgr2=0x5626b8c31b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.086+0000 7f437fe0c740 1 --2- 172.21.15.7:0/2866710554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b98a1c00 0x5626b8c31b80 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x5626b9993500 tx=0x5626b983e660 comp rx=0 tx=0).stop 2024-09-10T21:52:07.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.086+0000 7f437fe0c740 1 -- 172.21.15.7:0/2866710554 shutdown_connections 2024-09-10T21:52:07.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.086+0000 7f437fe0c740 1 --2- 172.21.15.7:0/2866710554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b98a1800 0x5626b8c31600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.086+0000 7f437fe0c740 1 --2- 172.21.15.7:0/2866710554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b98a1c00 0x5626b8c31b80 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.086+0000 7f437fe0c740 1 --2- 172.21.15.7:0/2866710554 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b98a1400 0x5626b8c31080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.086+0000 7f437fe0c740 1 -- 172.21.15.7:0/2866710554 >> 172.21.15.7:0/2866710554 conn(0x5626b98a1000 msgr2=0x5626b8ba4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:07.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.087+0000 7f437fe0c740 1 -- 172.21.15.7:0/2866710554 shutdown_connections 2024-09-10T21:52:07.090 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.087+0000 7f437fe0c740 1 -- 172.21.15.7:0/2866710554 wait complete. 2024-09-10T21:52:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.087+0000 7f437fe0c740 1 Processor -- start 2024-09-10T21:52:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.087+0000 7f437fe0c740 1 -- start start 2024-09-10T21:52:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.088+0000 7f437fe0c740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b98a1400 0x5626b8c31080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.088+0000 7f437fe0c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b98a1800 0x5626b8c31600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.088+0000 7f437fe0c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b98a1c00 0x5626b8c31b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.093 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.088+0000 7f437fe0c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5626b98a5380 con 0x5626b98a1800 2024-09-10T21:52:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.088+0000 7f437fe0c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5626b98a5500 con 0x5626b98a1c00 2024-09-10T21:52:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.088+0000 7f437fe0c740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5626b98a4180 con 0x5626b98a1400 2024-09-10T21:52:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.088+0000 7f437d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b98a1800 0x5626b8c31600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.088+0000 7f437d400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b98a1800 0x5626b8c31600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38078/0 (socket says 172.21.15.7:38078) 2024-09-10T21:52:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.088+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 learned_addr learned my addr 172.21.15.7:0/3137457218 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.088+0000 7f437de00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b98a1400 0x5626b8c31080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.088+0000 7f437ca00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b98a1c00 0x5626b8c31b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.089+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b98a1400 msgr2=0x5626b8c31080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.089+0000 7f437d400640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b98a1400 0x5626b8c31080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.089+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b98a1c00 msgr2=0x5626b8c31b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.089+0000 7f437d400640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b98a1c00 0x5626b8c31b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.089+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5626b8ae1680 con 0x5626b98a1800 2024-09-10T21:52:07.094 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.089+0000 7f437ca00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b98a1c00 0x5626b8c31b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.089+0000 7f437d400640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b98a1800 0x5626b8c31600 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x5626b98d9890 tx=0x5626b99a0ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.089+0000 7f4375c00640 1 -- 172.21.15.7:0/3137457218 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5626b8ada340 con 0x5626b98a1800 2024-09-10T21:52:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.089+0000 7f4375c00640 1 -- 172.21.15.7:0/3137457218 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5626b8b0ac40 con 0x5626b98a1800 2024-09-10T21:52:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.089+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5626b8ae14a0 con 0x5626b98a1800 2024-09-10T21:52:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.089+0000 7f4375c00640 1 -- 172.21.15.7:0/3137457218 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5626b8adb1e0 con 0x5626b98a1800 2024-09-10T21:52:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.089+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5626b8ae12c0 con 0x5626b98a1800 2024-09-10T21:52:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.089+0000 7f437fe0c740 4 datascan.init: resolving metadata pool 16 2024-09-10T21:52:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.091+0000 7f4375c00640 1 -- 172.21.15.7:0/3137457218 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x5626b8b0f500 con 0x5626b98a1800 2024-09-10T21:52:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.092+0000 7f4375c00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5626b9a11000 0x5626b8c32100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.092+0000 7f4375c00640 1 -- 172.21.15.7:0/3137457218 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x5626b9a18600 con 0x5626b98a1800 2024-09-10T21:52:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.092+0000 7f437fe0c740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-09-10T21:52:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.092+0000 7f437fe0c740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-09-10T21:52:07.095 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.092+0000 7f437fe0c740 4 datascan.main: opening data pool 'cephfs_data' 2024-09-10T21:52:07.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.092+0000 7f437fe0c740 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5626b9a11400 0x5626b8c32680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.092+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:1 17.0 17:00000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b9a11800 con 0x5626b9a11400 2024-09-10T21:52:07.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.092+0000 7f437de00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5626b9a11000 0x5626b8c32100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.096 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.092+0000 7f437ca00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5626b9a11400 0x5626b8c32680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.098 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.093+0000 7f437ca00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5626b9a11400 0x5626b8c32680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.093+0000 7f437de00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5626b9a11000 0x5626b8c32100 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x5626b9859650 tx=0x5626b99a13e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.093+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.7 v2:172.21.15.133:6802/2638399040 1 ==== osd_op_reply(1 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a11400 2024-09-10T21:52:07.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.093+0000 7f4377000640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5626b98a0800 0x5626b8e7c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.093+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:2 17.20 17:04000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b9a10c00 con 0x5626b98a0800 2024-09-10T21:52:07.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.094+0000 7f437de00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5626b98a0800 0x5626b8e7c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.094+0000 7f437de00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5626b98a0800 0x5626b8e7c000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.094+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.1 v2:172.21.15.7:6824/4175601737 1 ==== osd_op_reply(2 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b98a0800 2024-09-10T21:52:07.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.095+0000 7f4377a00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5626b9a10400 0x5626b8e7c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.095+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:3 17.10 17:08000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b9a10000 con 0x5626b9a10400 2024-09-10T21:52:07.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.095+0000 7f437d400640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5626b9a10400 0x5626b8e7c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.095+0000 7f437d400640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5626b9a10400 0x5626b8e7c580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.099 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.096+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 <== osd.3 v2:172.21.15.7:6800/2195665006 1 ==== osd_op_reply(3 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe780 con 0x5626b9a10400 2024-09-10T21:52:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.096+0000 7f4377000640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5626b9a10800 0x5626b8e7cb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.096+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:4 17.30 17:0c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e72c00 con 0x5626b9a10800 2024-09-10T21:52:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.096+0000 7f437ca00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5626b9a10800 0x5626b8e7cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.096+0000 7f437ca00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5626b9a10800 0x5626b8e7cb00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.097+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.0 v2:172.21.15.7:6808/651402908 1 ==== osd_op_reply(4 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a10800 2024-09-10T21:52:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.097+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:5 17.8 17:10000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b9a11800 con 0x5626b9a11400 2024-09-10T21:52:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.097+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.7 v2:172.21.15.133:6802/2638399040 2 ==== osd_op_reply(5 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a11400 2024-09-10T21:52:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.097+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:6 17.28 17:14000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e73000 con 0x5626b9a10400 2024-09-10T21:52:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.097+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 <== osd.3 v2:172.21.15.7:6800/2195665006 2 ==== osd_op_reply(6 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe780 con 0x5626b9a10400 2024-09-10T21:52:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.097+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:7 17.18 17:18000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e73800 con 0x5626b9a10800 2024-09-10T21:52:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.098+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.0 v2:172.21.15.7:6808/651402908 2 ==== osd_op_reply(7 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a10800 2024-09-10T21:52:07.100 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.098+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:8 17.38 17:1c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e73400 con 0x5626b9a11400 2024-09-10T21:52:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.098+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.7 v2:172.21.15.133:6802/2638399040 3 ==== osd_op_reply(8 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a11400 2024-09-10T21:52:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.098+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:9 17.4 17:20000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e73c00 con 0x5626b9a10800 2024-09-10T21:52:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.099+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.0 v2:172.21.15.7:6808/651402908 3 ==== osd_op_reply(9 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a10800 2024-09-10T21:52:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.099+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:10 17.24 17:24000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8e000 con 0x5626b9a10400 2024-09-10T21:52:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.099+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 <== osd.3 v2:172.21.15.7:6800/2195665006 3 ==== osd_op_reply(10 [pgnls start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe780 con 0x5626b9a10400 2024-09-10T21:52:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.099+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:11 17.14 17:28000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8f000 con 0x5626b9a10400 2024-09-10T21:52:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.099+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 <== osd.3 v2:172.21.15.7:6800/2195665006 4 ==== osd_op_reply(11 [pgnls start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe780 con 0x5626b9a10400 2024-09-10T21:52:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.099+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:12 17.34 17:2c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8e400 con 0x5626b9a10800 2024-09-10T21:52:07.103 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.100+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.0 v2:172.21.15.7:6808/651402908 4 ==== osd_op_reply(12 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a10800 2024-09-10T21:52:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.100+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:13 17.c 17:30000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8f400 con 0x5626b9a11400 2024-09-10T21:52:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.100+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.7 v2:172.21.15.133:6802/2638399040 4 ==== osd_op_reply(13 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a11400 2024-09-10T21:52:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.100+0000 7f4377000640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5626b8e8e800 0x5626b8e7d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.100+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:14 17.2c 17:34000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8ec00 con 0x5626b8e8e800 2024-09-10T21:52:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.101+0000 7f437de00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5626b8e8e800 0x5626b8e7d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.101+0000 7f437de00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5626b8e8e800 0x5626b8e7d080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.101+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.4 v2:172.21.15.133:6826/1857710445 1 ==== osd_op_reply(14 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b8e8e800 2024-09-10T21:52:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.101+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:15 17.1c 17:38000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8f800 con 0x5626b98a0800 2024-09-10T21:52:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.102+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.1 v2:172.21.15.7:6824/4175601737 2 ==== osd_op_reply(15 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b98a0800 2024-09-10T21:52:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.102+0000 7f4377000640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5626b8e72800 0x5626b8e7d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.102+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:16 17.3c 17:3c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e72c00 con 0x5626b8e72800 2024-09-10T21:52:07.104 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.102+0000 7f437d400640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5626b8e72800 0x5626b8e7d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.103+0000 7f437d400640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5626b8e72800 0x5626b8e7d600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.103+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 <== osd.2 v2:172.21.15.7:6816/4238877366 1 ==== osd_op_reply(16 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe780 con 0x5626b8e72800 2024-09-10T21:52:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.103+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:17 17.2 17:40000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8fc00 con 0x5626b8e8e800 2024-09-10T21:52:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.103+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.4 v2:172.21.15.133:6826/1857710445 2 ==== osd_op_reply(17 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b8e8e800 2024-09-10T21:52:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.103+0000 7f4377000640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5626b8e73800 0x5626b8e7db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.103+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:18 17.22 17:44000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b9a10c00 con 0x5626b8e73800 2024-09-10T21:52:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.104+0000 7f437ca00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5626b8e73800 0x5626b8e7db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.106 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.104+0000 7f437ca00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5626b8e73800 0x5626b8e7db80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.105+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.5 v2:172.21.15.133:6818/3838498382 1 ==== osd_op_reply(18 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b8e73800 2024-09-10T21:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.105+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:19 17.12 17:48000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e73000 con 0x5626b9a10400 2024-09-10T21:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.105+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 <== osd.3 v2:172.21.15.7:6800/2195665006 5 ==== osd_op_reply(19 [pgnls start_epoch 118 out=81b] v117'22 uv22 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5626b8bbe780 con 0x5626b9a10400 2024-09-10T21:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.105+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:20 17.12 17:4bec3886:::10000000002.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5626b9a11c00 con 0x5626b9a10400 2024-09-10T21:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.106+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 <== osd.3 v2:172.21.15.7:6800/2195665006 6 ==== osd_op_reply(20 10000000002.00000000 [stat out=16b] v0'0 uv22 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5626b8bbe780 con 0x5626b9a10400 2024-09-10T21:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.106+0000 7f437fe0c740 10 datascan.operator(): handling object 1099511627778.0 2024-09-10T21:52:07.107 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.106+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:21 17.12 17:4bec3886:::10000000002.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8e000 con 0x5626b9a10400 2024-09-10T21:52:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.107+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 <== osd.3 v2:172.21.15.7:6800/2195665006 7 ==== osd_op_reply(21 10000000002.00000000 [call] v118'23 uv23 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5626b8bbe780 con 0x5626b9a10400 2024-09-10T21:52:07.108 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.107+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:22 17.32 17:4c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8f000 con 0x5626b9a11400 2024-09-10T21:52:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.108+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.7 v2:172.21.15.133:6802/2638399040 5 ==== osd_op_reply(22 [pgnls start_epoch 118 out=49b] v116'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a11400 2024-09-10T21:52:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.108+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:23 17.a 17:50000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e72c00 con 0x5626b9a11400 2024-09-10T21:52:07.109 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.109+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.7 v2:172.21.15.133:6802/2638399040 6 ==== osd_op_reply(23 [pgnls start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a11400 2024-09-10T21:52:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.109+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:24 17.2a 17:54000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b9a11800 con 0x5626b9a10400 2024-09-10T21:52:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.109+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 <== osd.3 v2:172.21.15.7:6800/2195665006 8 ==== osd_op_reply(24 [pgnls start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe780 con 0x5626b9a10400 2024-09-10T21:52:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.109+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:25 17.1a 17:58000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e73400 con 0x5626b9a10800 2024-09-10T21:52:07.110 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.109+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.0 v2:172.21.15.7:6808/651402908 5 ==== osd_op_reply(25 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a10800 2024-09-10T21:52:07.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.109+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:26 17.3a 17:5c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eaa000 con 0x5626b98a0800 2024-09-10T21:52:07.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.110+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.1 v2:172.21.15.7:6824/4175601737 3 ==== osd_op_reply(26 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b98a0800 2024-09-10T21:52:07.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.110+0000 7f4377a00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5626b8e73c00 0x5626b8e7e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.110+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:27 17.6 17:60000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b9a10000 con 0x5626b8e73c00 2024-09-10T21:52:07.111 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.110+0000 7f437de00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5626b8e73c00 0x5626b8e7e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.111+0000 7f437de00640 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5626b8e73c00 0x5626b8e7e680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.111+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.6 v2:172.21.15.133:6810/3365981160 1 ==== osd_op_reply(27 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b8e73c00 2024-09-10T21:52:07.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.111+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:28 17.26 17:64000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eaa400 con 0x5626b9a10800 2024-09-10T21:52:07.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.112+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.0 v2:172.21.15.7:6808/651402908 6 ==== osd_op_reply(28 [pgnls start_epoch 118 out=49b] v116'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a10800 2024-09-10T21:52:07.112 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.112+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:29 17.16 17:68000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eab800 con 0x5626b8e73800 2024-09-10T21:52:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.112+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.5 v2:172.21.15.133:6818/3838498382 2 ==== osd_op_reply(29 [pgnls start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b8e73800 2024-09-10T21:52:07.113 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.112+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:30 17.36 17:6c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eaa800 con 0x5626b9a10800 2024-09-10T21:52:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.113+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.0 v2:172.21.15.7:6808/651402908 7 ==== osd_op_reply(30 [pgnls start_epoch 118 out=81b] v116'33 uv17 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a10800 2024-09-10T21:52:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.113+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:31 17.36 17:6fe38dc2:::100000001f7.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8ec00 con 0x5626b9a10800 2024-09-10T21:52:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.113+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.0 v2:172.21.15.7:6808/651402908 8 ==== osd_op_reply(31 100000001f7.00000000 [stat out=16b] v0'0 uv17 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a10800 2024-09-10T21:52:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.113+0000 7f437fe0c740 10 datascan.operator(): handling object 1099511628279.0 2024-09-10T21:52:07.114 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.113+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:32 17.36 17:6fe38dc2:::100000001f7.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8f800 con 0x5626b9a10800 2024-09-10T21:52:07.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.115+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.0 v2:172.21.15.7:6808/651402908 9 ==== osd_op_reply(32 100000001f7.00000000 [call] v118'34 uv34 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a10800 2024-09-10T21:52:07.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.115+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:33 17.e 17:70000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8e400 con 0x5626b8e72800 2024-09-10T21:52:07.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.115+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 <== osd.2 v2:172.21.15.7:6816/4238877366 2 ==== osd_op_reply(33 [pgnls start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe780 con 0x5626b8e72800 2024-09-10T21:52:07.116 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.115+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:34 17.2e 17:74000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eaac00 con 0x5626b8e73c00 2024-09-10T21:52:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.115+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.6 v2:172.21.15.133:6810/3365981160 2 ==== osd_op_reply(34 [pgnls start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b8e73c00 2024-09-10T21:52:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.116+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:35 17.1e 17:78000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eabc00 con 0x5626b98a0800 2024-09-10T21:52:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.116+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.1 v2:172.21.15.7:6824/4175601737 4 ==== osd_op_reply(35 [pgnls start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b98a0800 2024-09-10T21:52:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.116+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:36 17.3e 17:7c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eab000 con 0x5626b8e8e800 2024-09-10T21:52:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.116+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.4 v2:172.21.15.133:6826/1857710445 3 ==== osd_op_reply(36 [pgnls start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b8e8e800 2024-09-10T21:52:07.117 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.116+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:37 17.1 17:80000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eaa000 con 0x5626b9a10800 2024-09-10T21:52:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.117+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.0 v2:172.21.15.7:6808/651402908 10 ==== osd_op_reply(37 [pgnls start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a10800 2024-09-10T21:52:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.117+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:38 17.21 17:84000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eab400 con 0x5626b9a10800 2024-09-10T21:52:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.117+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.0 v2:172.21.15.7:6808/651402908 11 ==== osd_op_reply(38 [pgnls start_epoch 118 out=49b] v116'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a10800 2024-09-10T21:52:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.117+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:39 17.11 17:88000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eaac00 con 0x5626b9a10400 2024-09-10T21:52:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.117+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 <== osd.3 v2:172.21.15.7:6800/2195665006 9 ==== osd_op_reply(39 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe780 con 0x5626b9a10400 2024-09-10T21:52:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.117+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:40 17.31 17:8c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eaac00 con 0x5626b98a0800 2024-09-10T21:52:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.118+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.1 v2:172.21.15.7:6824/4175601737 5 ==== osd_op_reply(40 [pgnls start_epoch 118 out=81b] v116'15 uv8 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b98a0800 2024-09-10T21:52:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.118+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:41 17.31 17:8de29a03:::10000000004.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8f400 con 0x5626b98a0800 2024-09-10T21:52:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.118+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.1 v2:172.21.15.7:6824/4175601737 6 ==== osd_op_reply(41 10000000004.00000001 [stat out=16b] v0'0 uv8 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b98a0800 2024-09-10T21:52:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.118+0000 7f437fe0c740 10 datascan.operator(): handling object 1099511627780.1 2024-09-10T21:52:07.119 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.118+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:42 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8e000 con 0x5626b8e73c00 2024-09-10T21:52:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.120+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.6 v2:172.21.15.133:6810/3365981160 3 ==== osd_op_reply(42 10000000004.00000000 [call] v118'14 uv14 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b8e73c00 2024-09-10T21:52:07.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.120+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:43 17.9 17:90000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8e400 con 0x5626b9a10400 2024-09-10T21:52:07.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.120+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 <== osd.3 v2:172.21.15.7:6800/2195665006 10 ==== osd_op_reply(43 [pgnls start_epoch 118 out=49b] v116'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe780 con 0x5626b9a10400 2024-09-10T21:52:07.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.120+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:44 17.29 17:94000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eaa400 con 0x5626b8e8e800 2024-09-10T21:52:07.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.120+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.4 v2:172.21.15.133:6826/1857710445 4 ==== osd_op_reply(44 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b8e8e800 2024-09-10T21:52:07.121 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.120+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:45 17.19 17:98000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eabc00 con 0x5626b8e73800 2024-09-10T21:52:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.121+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.5 v2:172.21.15.133:6818/3838498382 3 ==== osd_op_reply(45 [pgnls start_epoch 118 out=81b] v117'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b8e73800 2024-09-10T21:52:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.121+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:46 17.19 17:997b7129:::100000001f8.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8fc00 con 0x5626b8e73800 2024-09-10T21:52:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.121+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.5 v2:172.21.15.133:6818/3838498382 4 ==== osd_op_reply(46 100000001f8.00000000 [stat out=16b] v0'0 uv23 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b8e73800 2024-09-10T21:52:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.121+0000 7f437fe0c740 10 datascan.operator(): handling object 1099511628280.0 2024-09-10T21:52:07.122 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.121+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:47 17.19 17:997b7129:::100000001f8.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8f000 con 0x5626b8e73800 2024-09-10T21:52:07.123 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.122+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.5 v2:172.21.15.133:6818/3838498382 5 ==== osd_op_reply(47 100000001f8.00000000 [call] v118'24 uv24 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b8e73800 2024-09-10T21:52:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.123+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:48 17.39 17:9c000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8ec00 con 0x5626b8e73800 2024-09-10T21:52:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.123+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.5 v2:172.21.15.133:6818/3838498382 6 ==== osd_op_reply(48 [pgnls start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b8e73800 2024-09-10T21:52:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.123+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:49 17.5 17:a0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e72c00 con 0x5626b8e73c00 2024-09-10T21:52:07.124 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.123+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.6 v2:172.21.15.133:6810/3365981160 4 ==== osd_op_reply(49 [pgnls start_epoch 118 out=81b] v118'14 uv14 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b8e73c00 2024-09-10T21:52:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.124+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:50 17.5 17:a367a3e8:::10000000004.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e72c00 con 0x5626b8e73c00 2024-09-10T21:52:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.124+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.6 v2:172.21.15.133:6810/3365981160 5 ==== osd_op_reply(50 10000000004.00000000 [stat out=16b] v0'0 uv14 ondisk = 0) ==== 164+0+16 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b8e73c00 2024-09-10T21:52:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.124+0000 7f437fe0c740 10 datascan.operator(): handling object 1099511627780.0 2024-09-10T21:52:07.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.124+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:51 17.5 17:a367a3e8:::10000000004.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eab000 con 0x5626b8e73c00 2024-09-10T21:52:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.125+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.6 v2:172.21.15.133:6810/3365981160 6 ==== osd_op_reply(51 10000000004.00000000 [call] v118'15 uv15 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b8e73c00 2024-09-10T21:52:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.125+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:52 17.25 17:a4000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eaac00 con 0x5626b98a0800 2024-09-10T21:52:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.126+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.1 v2:172.21.15.7:6824/4175601737 7 ==== osd_op_reply(52 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b98a0800 2024-09-10T21:52:07.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.126+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:53 17.15 17:a8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e73400 con 0x5626b8e73800 2024-09-10T21:52:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.126+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.5 v2:172.21.15.133:6818/3838498382 7 ==== osd_op_reply(53 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b8e73800 2024-09-10T21:52:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.126+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:54 17.35 17:ac000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eab800 con 0x5626b8e8e800 2024-09-10T21:52:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.126+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.4 v2:172.21.15.133:6826/1857710445 5 ==== osd_op_reply(54 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b8e8e800 2024-09-10T21:52:07.127 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.127+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:55 17.d 17:b0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b9a10000 con 0x5626b9a11400 2024-09-10T21:52:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.127+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.7 v2:172.21.15.133:6802/2638399040 7 ==== osd_op_reply(55 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a11400 2024-09-10T21:52:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.127+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:56 17.2d 17:b4000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eaa800 con 0x5626b9a11400 2024-09-10T21:52:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.127+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.7 v2:172.21.15.133:6802/2638399040 8 ==== osd_op_reply(56 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a11400 2024-09-10T21:52:07.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.127+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:57 17.1d 17:b8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b9a10c00 con 0x5626b9a11400 2024-09-10T21:52:07.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.128+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.7 v2:172.21.15.133:6802/2638399040 9 ==== osd_op_reply(57 [pgnls start_epoch 118 out=49b] v116'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a11400 2024-09-10T21:52:07.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.128+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:58 17.3d 17:bc000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b9a11c00 con 0x5626b9a11400 2024-09-10T21:52:07.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.128+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.7 v2:172.21.15.133:6802/2638399040 10 ==== osd_op_reply(58 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a11400 2024-09-10T21:52:07.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.128+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:59 17.3 17:c0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e73400 con 0x5626b8e8e800 2024-09-10T21:52:07.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.129+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.4 v2:172.21.15.133:6826/1857710445 6 ==== osd_op_reply(59 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b8e8e800 2024-09-10T21:52:07.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.129+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:60 17.23 17:c4000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b9a11800 con 0x5626b8e8e800 2024-09-10T21:52:07.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.129+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.4 v2:172.21.15.133:6826/1857710445 7 ==== osd_op_reply(60 [pgnls start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b8e8e800 2024-09-10T21:52:07.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.129+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:61 17.13 17:c8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8f800 con 0x5626b9a10400 2024-09-10T21:52:07.130 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.130+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 <== osd.3 v2:172.21.15.7:6800/2195665006 11 ==== osd_op_reply(61 [pgnls start_epoch 118 out=49b] v116'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe780 con 0x5626b9a10400 2024-09-10T21:52:07.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.130+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:62 17.33 17:cc000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e73000 con 0x5626b9a11400 2024-09-10T21:52:07.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.130+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.7 v2:172.21.15.133:6802/2638399040 11 ==== osd_op_reply(62 [pgnls start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a11400 2024-09-10T21:52:07.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.130+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:63 17.b 17:d0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8fc00 con 0x5626b9a10800 2024-09-10T21:52:07.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.130+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.0 v2:172.21.15.7:6808/651402908 12 ==== osd_op_reply(63 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a10800 2024-09-10T21:52:07.131 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.130+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:64 17.2b 17:d4000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b9a10000 con 0x5626b8e8e800 2024-09-10T21:52:07.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.131+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.4 v2:172.21.15.133:6826/1857710445 8 ==== osd_op_reply(64 [pgnls start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b8e8e800 2024-09-10T21:52:07.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.131+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:65 17.1b 17:d8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8f000 con 0x5626b8e8e800 2024-09-10T21:52:07.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.131+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.4 v2:172.21.15.133:6826/1857710445 9 ==== osd_op_reply(65 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b8e8e800 2024-09-10T21:52:07.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.131+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:66 17.3b 17:dc000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b9a11800 con 0x5626b8e72800 2024-09-10T21:52:07.132 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.132+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 <== osd.2 v2:172.21.15.7:6816/4238877366 3 ==== osd_op_reply(66 [pgnls start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe780 con 0x5626b8e72800 2024-09-10T21:52:07.133 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.132+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:67 17.7 17:e0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e8ec00 con 0x5626b9a10800 2024-09-10T21:52:07.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.132+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.0 v2:172.21.15.7:6808/651402908 13 ==== osd_op_reply(67 [pgnls start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a10800 2024-09-10T21:52:07.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.132+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:68 17.27 17:e4000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b9a10000 con 0x5626b8e8e800 2024-09-10T21:52:07.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.132+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.4 v2:172.21.15.133:6826/1857710445 10 ==== osd_op_reply(68 [pgnls start_epoch 118 out=49b] v116'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b8e8e800 2024-09-10T21:52:07.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.132+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:69 17.17 17:e8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eaa000 con 0x5626b8e73c00 2024-09-10T21:52:07.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.133+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.6 v2:172.21.15.133:6810/3365981160 7 ==== osd_op_reply(69 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b8e73c00 2024-09-10T21:52:07.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.133+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:70 17.37 17:ec000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e72c00 con 0x5626b9a10800 2024-09-10T21:52:07.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.133+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.0 v2:172.21.15.7:6808/651402908 14 ==== osd_op_reply(70 [pgnls start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a10800 2024-09-10T21:52:07.134 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.133+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:71 17.f 17:f0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eab400 con 0x5626b98a0800 2024-09-10T21:52:07.136 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.133+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.1 v2:172.21.15.7:6824/4175601737 8 ==== osd_op_reply(71 [pgnls start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b98a0800 2024-09-10T21:52:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.133+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:72 17.2f 17:f4000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8e73400 con 0x5626b8e8e800 2024-09-10T21:52:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.134+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.4 v2:172.21.15.133:6826/1857710445 11 ==== osd_op_reply(72 [pgnls start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b8e8e800 2024-09-10T21:52:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.134+0000 7f4377a00640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:73 17.1f 17:f8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eabc00 con 0x5626b8e73c00 2024-09-10T21:52:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.134+0000 7f437de00640 1 -- 172.21.15.7:0/3137457218 <== osd.6 v2:172.21.15.133:6810/3365981160 8 ==== osd_op_reply(73 [pgnls start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe500 con 0x5626b8e73c00 2024-09-10T21:52:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.134+0000 7f4377000640 1 -- 172.21.15.7:0/3137457218 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:74 17.3f 17:fc000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5626b8eab000 con 0x5626b9a10800 2024-09-10T21:52:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.134+0000 7f437ca00640 1 -- 172.21.15.7:0/3137457218 <== osd.0 v2:172.21.15.7:6808/651402908 15 ==== osd_op_reply(74 [pgnls start_epoch 118 out=49b] v116'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5626b8bbe000 con 0x5626b9a10800 2024-09-10T21:52:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.134+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5626b9a10800 msgr2=0x5626b8e7cb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.134+0000 7f437fe0c740 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5626b9a10800 0x5626b8e7cb00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.134+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5626b98a0800 msgr2=0x5626b8e7c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5626b98a0800 0x5626b8e7c000 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5626b8e72800 msgr2=0x5626b8e7d600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5626b8e72800 0x5626b8e7d600 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5626b9a10400 msgr2=0x5626b8e7c580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.137 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5626b9a10400 0x5626b8e7c580 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5626b8e8e800 msgr2=0x5626b8e7d080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5626b8e8e800 0x5626b8e7d080 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 reap_dead start 2024-09-10T21:52:07.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5626b8e73800 msgr2=0x5626b8e7db80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5626b8e73800 0x5626b8e7db80 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5626b8e73c00 msgr2=0x5626b8e7e680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5626b8e73c00 0x5626b8e7e680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5626b9a11400 msgr2=0x5626b8c32680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5626b9a11400 0x5626b8c32680 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 reap_dead start 2024-09-10T21:52:07.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437d400640 1 -- 172.21.15.7:0/3137457218 reap_dead start 2024-09-10T21:52:07.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5626b9a11000 msgr2=0x5626b8c32100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5626b9a11000 0x5626b8c32100 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x5626b9859650 tx=0x5626b99a13e0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.138 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b98a1800 msgr2=0x5626b8c31600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b98a1800 0x5626b8c31600 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x5626b98d9890 tx=0x5626b99a0ea0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 shutdown_connections 2024-09-10T21:52:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5626b9a11000 0x5626b8c32100 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 --2- 172.21.15.7:0/3137457218 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b98a1800 0x5626b8c31600 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 >> 172.21.15.7:0/3137457218 conn(0x5626b98a1000 msgr2=0x5626b8ba4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 shutdown_connections 2024-09-10T21:52:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 -- 172.21.15.7:0/3137457218 wait complete. 2024-09-10T21:52:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 -- 172.21.15.7:0/2495022616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b8ab5800 msgr2=0x5626b8c30580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.135+0000 7f437fe0c740 1 --2- 172.21.15.7:0/2495022616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b8ab5800 0x5626b8c30580 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x5626b9992090 tx=0x5626b8b7d080 comp rx=0 tx=0).stop 2024-09-10T21:52:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.136+0000 7f437fe0c740 1 -- 172.21.15.7:0/2495022616 shutdown_connections 2024-09-10T21:52:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.136+0000 7f437fe0c740 1 --2- 172.21.15.7:0/2495022616 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5626b8ab5c00 0x5626b8c30b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.136+0000 7f437fe0c740 1 --2- 172.21.15.7:0/2495022616 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5626b8ab5800 0x5626b8c30580 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.136+0000 7f437fe0c740 1 --2- 172.21.15.7:0/2495022616 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5626b8ab4c00 0x5626b8c30000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.136+0000 7f437fe0c740 1 -- 172.21.15.7:0/2495022616 >> 172.21.15.7:0/2495022616 conn(0x5626b8ab5000 msgr2=0x5626b8ba5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:07.139 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.136+0000 7f437fe0c740 1 -- 172.21.15.7:0/2495022616 shutdown_connections 2024-09-10T21:52:07.140 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.136+0000 7f437fe0c740 1 -- 172.21.15.7:0/2495022616 wait complete. 2024-09-10T21:52:07.141 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', 'cephfs_data'] in time 0:00:00.132115, result: 2024-09-10T21:52:07.142 DEBUG:teuthology.orchestra.run.smithi007:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --filter-tag mytag123 cephfs_data 2024-09-10T21:52:07.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.215+0000 7fa97e613740 1 -- start start 2024-09-10T21:52:07.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.215+0000 7fa97e613740 1 Processor -- start 2024-09-10T21:52:07.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.216+0000 7fa97e613740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637f58e8c00 0x5637f5a68000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.216+0000 7fa97e613740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637f58e9800 0x5637f5a68580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.216+0000 7fa97e613740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f58e9c00 0x5637f5a68b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.216+0000 7fa97e613740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5637f66d8600 con 0x5637f58e9c00 2024-09-10T21:52:07.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.216+0000 7fa97e613740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5637f66d8480 con 0x5637f58e9800 2024-09-10T21:52:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.216+0000 7fa97e613740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5637f66d8300 con 0x5637f58e8c00 2024-09-10T21:52:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.216+0000 7fa97c600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f58e9c00 0x5637f5a68b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.216+0000 7fa97c600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f58e9c00 0x5637f5a68b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38090/0 (socket says 172.21.15.7:38090) 2024-09-10T21:52:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.216+0000 7fa97c600640 1 -- 172.21.15.7:0/2716963628 learned_addr learned my addr 172.21.15.7:0/2716963628 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.217+0000 7fa97bc00640 1 --2- 172.21.15.7:0/2716963628 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637f58e8c00 0x5637f5a68000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.217+0000 7fa97b200640 1 --2- 172.21.15.7:0/2716963628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637f58e9800 0x5637f5a68580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.217+0000 7fa97c600640 1 -- 172.21.15.7:0/2716963628 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637f58e8c00 msgr2=0x5637f5a68000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.217+0000 7fa97c600640 1 --2- 172.21.15.7:0/2716963628 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637f58e8c00 0x5637f5a68000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.217+0000 7fa97c600640 1 -- 172.21.15.7:0/2716963628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637f58e9800 msgr2=0x5637f5a68580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.217+0000 7fa97c600640 1 --2- 172.21.15.7:0/2716963628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637f58e9800 0x5637f5a68580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.217+0000 7fa97c600640 1 -- 172.21.15.7:0/2716963628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637f5915680 con 0x5637f58e9c00 2024-09-10T21:52:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.217+0000 7fa97c600640 1 --2- 172.21.15.7:0/2716963628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f58e9c00 0x5637f5a68b00 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x5637f67ca060 tx=0x5637f59b0f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e08345fd12c66a90 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.217+0000 7fa978a00640 1 -- 172.21.15.7:0/2716963628 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5637f590eea0 con 0x5637f58e9c00 2024-09-10T21:52:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.218+0000 7fa978a00640 1 -- 172.21.15.7:0/2716963628 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5637f593e8c0 con 0x5637f58e9c00 2024-09-10T21:52:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.218+0000 7fa978a00640 1 -- 172.21.15.7:0/2716963628 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5637f590f380 con 0x5637f58e9c00 2024-09-10T21:52:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.218+0000 7fa97e613740 1 -- 172.21.15.7:0/2716963628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637f59154a0 con 0x5637f58e9c00 2024-09-10T21:52:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.218+0000 7fa97e613740 1 -- 172.21.15.7:0/2716963628 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5637f5915a40 con 0x5637f58e9c00 2024-09-10T21:52:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.218+0000 7fa97e613740 4 waiting for MDS map... 2024-09-10T21:52:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.218+0000 7fa978a00640 1 -- 172.21.15.7:0/2716963628 <== mon.0 v2:172.21.15.7:3300/0 4 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x5637f67dc600 con 0x5637f58e9c00 2024-09-10T21:52:07.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.218+0000 7fa978a00640 1 -- 172.21.15.7:0/2716963628 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 109) ==== 2893+0+0 (secure 0 0 0) 0x5637f5943180 con 0x5637f58e9c00 2024-09-10T21:52:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.218+0000 7fa97e613740 4 Got MDS map 109 2024-09-10T21:52:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.218+0000 7fa97e613740 10 datascan.parse_kwarg: Applying tag filter: 'mytag123' 2024-09-10T21:52:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.218+0000 7fa97e613740 4 datascan.main: Using metadata pool output 2024-09-10T21:52:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.218+0000 7fa97e613740 4 datascan.main: connecting to RADOS... 2024-09-10T21:52:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.219+0000 7fa97e613740 1 Processor -- start 2024-09-10T21:52:07.221 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.219+0000 7fa97e613740 1 -- start start 2024-09-10T21:52:07.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.220+0000 7fa97e613740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637f67c3400 0x5637f5a69080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.220+0000 7fa97e613740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f67c3800 0x5637f5a69600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.220+0000 7fa97e613740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637f67c3c00 0x5637f5a69b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.220+0000 7fa97e613740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5637f66d8180 con 0x5637f67c3800 2024-09-10T21:52:07.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.220+0000 7fa97e613740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5637f594b380 con 0x5637f67c3400 2024-09-10T21:52:07.225 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.220+0000 7fa97e613740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5637f594b800 con 0x5637f67c3c00 2024-09-10T21:52:07.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.220+0000 7fa97bc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f67c3800 0x5637f5a69600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.220+0000 7fa97bc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f67c3800 0x5637f5a69600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38094/0 (socket says 172.21.15.7:38094) 2024-09-10T21:52:07.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.220+0000 7fa97bc00640 1 -- 172.21.15.7:0/34178836 learned_addr learned my addr 172.21.15.7:0/34178836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:07.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.220+0000 7fa97b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637f67c3400 0x5637f5a69080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.220+0000 7fa97b200640 1 --2- 172.21.15.7:0/34178836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637f67c3c00 0x5637f5a69b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.220+0000 7fa97bc00640 1 -- 172.21.15.7:0/34178836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637f67c3c00 msgr2=0x5637f5a69b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:07.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.220+0000 7fa97bc00640 1 --2- 172.21.15.7:0/34178836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637f67c3c00 0x5637f5a69b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.220+0000 7fa97bc00640 1 -- 172.21.15.7:0/34178836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637f67c3400 msgr2=0x5637f5a69080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:07.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.220+0000 7fa97bc00640 1 --2- 172.21.15.7:0/34178836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637f67c3400 0x5637f5a69080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.220+0000 7fa97bc00640 1 -- 172.21.15.7:0/34178836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637f5915860 con 0x5637f67c3800 2024-09-10T21:52:07.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.221+0000 7fa97bc00640 1 --2- 172.21.15.7:0/34178836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f67c3800 0x5637f5a69600 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x5637f67cb4d0 tx=0x5637f67e4ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a8ee5fbcd32572 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.221+0000 7fa976200640 1 -- 172.21.15.7:0/34178836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5637f590eea0 con 0x5637f67c3800 2024-09-10T21:52:07.226 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.221+0000 7fa976200640 1 -- 172.21.15.7:0/34178836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5637f593ee00 con 0x5637f67c3800 2024-09-10T21:52:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.221+0000 7fa976200640 1 -- 172.21.15.7:0/34178836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5637f590f380 con 0x5637f67c3800 2024-09-10T21:52:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.221+0000 7fa97e613740 1 -- 172.21.15.7:0/34178836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f67c3800 msgr2=0x5637f5a69600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.221+0000 7fa97e613740 1 --2- 172.21.15.7:0/34178836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f67c3800 0x5637f5a69600 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x5637f67cb4d0 tx=0x5637f67e4ea0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.221+0000 7fa97e613740 1 -- 172.21.15.7:0/34178836 shutdown_connections 2024-09-10T21:52:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.221+0000 7fa97e613740 1 --2- 172.21.15.7:0/34178836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637f67c3c00 0x5637f5a69b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.222+0000 7fa97e613740 1 --2- 172.21.15.7:0/34178836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f67c3800 0x5637f5a69600 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.222+0000 7fa97e613740 1 --2- 172.21.15.7:0/34178836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637f67c3400 0x5637f5a69080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.222+0000 7fa97e613740 1 -- 172.21.15.7:0/34178836 >> 172.21.15.7:0/34178836 conn(0x5637f67c3000 msgr2=0x5637f59d8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.222+0000 7fa97e613740 1 -- 172.21.15.7:0/34178836 shutdown_connections 2024-09-10T21:52:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.222+0000 7fa97e613740 1 -- 172.21.15.7:0/34178836 wait complete. 2024-09-10T21:52:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.222+0000 7fa97e613740 1 Processor -- start 2024-09-10T21:52:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.222+0000 7fa97e613740 1 -- start start 2024-09-10T21:52:07.227 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.223+0000 7fa97e613740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637f67c3400 0x5637f5a69080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.223+0000 7fa97e613740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f67c3800 0x5637f5a69600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.223+0000 7fa97e613740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637f67c3c00 0x5637f5a69b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.223+0000 7fa97e613740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5637f66d9380 con 0x5637f67c3800 2024-09-10T21:52:07.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.223+0000 7fa97e613740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5637f66d8180 con 0x5637f67c3400 2024-09-10T21:52:07.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.223+0000 7fa97e613740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5637f66d9500 con 0x5637f67c3c00 2024-09-10T21:52:07.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.223+0000 7fa97bc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f67c3800 0x5637f5a69600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.223+0000 7fa97bc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f67c3800 0x5637f5a69600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38110/0 (socket says 172.21.15.7:38110) 2024-09-10T21:52:07.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.223+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 learned_addr learned my addr 172.21.15.7:0/3934142754 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:07.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.223+0000 7fa97b200640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637f67c3400 0x5637f5a69080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.224+0000 7fa97b200640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637f67c3c00 0x5637f5a69b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.224+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637f67c3c00 msgr2=0x5637f5a69b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.228 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.224+0000 7fa97bc00640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637f67c3c00 0x5637f5a69b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.224+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637f67c3400 msgr2=0x5637f5a69080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.224+0000 7fa97bc00640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637f67c3400 0x5637f5a69080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.224+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637f5915680 con 0x5637f67c3800 2024-09-10T21:52:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.224+0000 7fa97bc00640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f67c3800 0x5637f5a69600 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x5637f66f20c0 tx=0x5637f67e4e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.224+0000 7fa974400640 1 -- 172.21.15.7:0/3934142754 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5637f590f040 con 0x5637f67c3800 2024-09-10T21:52:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.224+0000 7fa974400640 1 -- 172.21.15.7:0/3934142754 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5637f593ec40 con 0x5637f67c3800 2024-09-10T21:52:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.224+0000 7fa974400640 1 -- 172.21.15.7:0/3934142754 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5637f590eea0 con 0x5637f67c3800 2024-09-10T21:52:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.224+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5637f59154a0 con 0x5637f67c3800 2024-09-10T21:52:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.224+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637f59152c0 con 0x5637f67c3800 2024-09-10T21:52:07.229 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.224+0000 7fa97e613740 4 datascan.init: resolving metadata pool 16 2024-09-10T21:52:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.226+0000 7fa974400640 1 -- 172.21.15.7:0/3934142754 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x5637f5943500 con 0x5637f67c3800 2024-09-10T21:52:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.227+0000 7fa974400640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5637f684ec00 0x5637f5a6a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.227+0000 7fa974400640 1 -- 172.21.15.7:0/3934142754 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x5637f6844a00 con 0x5637f67c3800 2024-09-10T21:52:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.227+0000 7fa97e613740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-09-10T21:52:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.227+0000 7fa97e613740 4 datascan.main: data pool 'cephfs_data' has ID 17 2024-09-10T21:52:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.227+0000 7fa97e613740 4 datascan.main: opening data pool 'cephfs_data' 2024-09-10T21:52:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.227+0000 7fa97e613740 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5637f684f000 0x5637f5a6a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.227+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:1 16.14 16:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5637f684f400 con 0x5637f684f000 2024-09-10T21:52:07.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.227+0000 7fa97b200640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5637f684ec00 0x5637f5a6a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.227+0000 7fa97b200640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5637f684f000 0x5637f5a6a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.228+0000 7fa97b200640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5637f684ec00 0x5637f5a6a100 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x5637f670b7a0 tx=0x5637f67e5080 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.228+0000 7fa97b200640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5637f684f000 0x5637f5a6a680 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.228+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.0 v2:172.21.15.7:6808/651402908 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv34 ondisk = 0) ==== 160+0+16 (crc 0 0 0) 0x5637f59f2000 con 0x5637f684f000 2024-09-10T21:52:07.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.228+0000 7fa97e613740 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5637f684f800 0x5637f5a6ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.228+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:2 16.1f 16:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5637f684fc00 con 0x5637f684f800 2024-09-10T21:52:07.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.229+0000 7fa97c600640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5637f684f800 0x5637f5a6ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.229+0000 7fa97c600640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5637f684f800 0x5637f5a6ac00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.229+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.5 v2:172.21.15.133:6818/3838498382 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv3 ondisk = 0) ==== 162+0+16 (crc 0 0 0) 0x5637f59f2500 con 0x5637f684f800 2024-09-10T21:52:07.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.229+0000 7fa97e613740 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5637f5cb2000 0x5637f5a6b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.229+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:3 17.0 17:00000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cb2400 con 0x5637f5cb2000 2024-09-10T21:52:07.233 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.230+0000 7fa97bc00640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5637f5cb2000 0x5637f5a6b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.230+0000 7fa97bc00640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5637f5cb2000 0x5637f5a6b180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.230+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.7 v2:172.21.15.133:6802/2638399040 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5cb2000 2024-09-10T21:52:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.231+0000 7fa975800640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5637f5cb3800 0x5637f5cbe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.231+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:4 17.20 17:04000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cb3c00 con 0x5637f5cb3800 2024-09-10T21:52:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.231+0000 7fa97b200640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5637f5cb3800 0x5637f5cbe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.231+0000 7fa97b200640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5637f5cb3800 0x5637f5cbe000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.232+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.1 v2:172.21.15.7:6824/4175601737 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f5cb3800 2024-09-10T21:52:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.232+0000 7fa976200640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5637f5cca000 0x5637f5cbe580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.232+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:5 17.10 17:08000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cca400 con 0x5637f5cca000 2024-09-10T21:52:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.232+0000 7fa97c600640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5637f5cca000 0x5637f5cbe580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.232+0000 7fa97c600640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5637f5cca000 0x5637f5cbe580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.233+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.3 v2:172.21.15.7:6800/2195665006 1 ==== osd_op_reply(5 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cca000 2024-09-10T21:52:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.233+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:6 17.30 17:0c000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5ccac00 con 0x5637f684f000 2024-09-10T21:52:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.233+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.0 v2:172.21.15.7:6808/651402908 2 ==== osd_op_reply(6 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f684f000 2024-09-10T21:52:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.233+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:7 17.8 17:10000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cca800 con 0x5637f5cb2000 2024-09-10T21:52:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.234+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.7 v2:172.21.15.133:6802/2638399040 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5cb2000 2024-09-10T21:52:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.234+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:8 17.28 17:14000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5ccb000 con 0x5637f5cca000 2024-09-10T21:52:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.234+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.3 v2:172.21.15.7:6800/2195665006 2 ==== osd_op_reply(8 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cca000 2024-09-10T21:52:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.234+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:9 17.18 17:18000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f684f400 con 0x5637f684f000 2024-09-10T21:52:07.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.234+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.0 v2:172.21.15.7:6808/651402908 3 ==== osd_op_reply(9 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f684f000 2024-09-10T21:52:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.234+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:10 17.38 17:1c000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5ccb400 con 0x5637f5cb2000 2024-09-10T21:52:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.235+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.7 v2:172.21.15.133:6802/2638399040 3 ==== osd_op_reply(10 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5cb2000 2024-09-10T21:52:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.235+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:11 17.4 17:20000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f66db400 con 0x5637f684f000 2024-09-10T21:52:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.235+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.0 v2:172.21.15.7:6808/651402908 4 ==== osd_op_reply(11 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f684f000 2024-09-10T21:52:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.235+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:12 17.24 17:24000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5ccac00 con 0x5637f5cca000 2024-09-10T21:52:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.235+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.3 v2:172.21.15.7:6800/2195665006 3 ==== osd_op_reply(12 [pgnls-filter start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cca000 2024-09-10T21:52:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.235+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:13 17.14 17:28000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f66db000 con 0x5637f5cca000 2024-09-10T21:52:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.236+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.3 v2:172.21.15.7:6800/2195665006 4 ==== osd_op_reply(13 [pgnls-filter start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cca000 2024-09-10T21:52:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.236+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:14 17.34 17:2c000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5ccb800 con 0x5637f684f000 2024-09-10T21:52:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.236+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.0 v2:172.21.15.7:6808/651402908 5 ==== osd_op_reply(14 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f684f000 2024-09-10T21:52:07.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.236+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:15 17.c 17:30000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cca400 con 0x5637f5cb2000 2024-09-10T21:52:07.240 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.236+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.7 v2:172.21.15.133:6802/2638399040 4 ==== osd_op_reply(15 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5cb2000 2024-09-10T21:52:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.236+0000 7fa975800640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5637f5ccbc00 0x5637f5cbeb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.236+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:16 17.2c 17:34000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cb3c00 con 0x5637f5ccbc00 2024-09-10T21:52:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.237+0000 7fa97bc00640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5637f5ccbc00 0x5637f5cbeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.237+0000 7fa97bc00640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5637f5ccbc00 0x5637f5cbeb00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.238+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.4 v2:172.21.15.133:6826/1857710445 1 ==== osd_op_reply(16 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5ccbc00 2024-09-10T21:52:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.238+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:17 17.1c 17:38000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5ccb000 con 0x5637f5cb3800 2024-09-10T21:52:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.238+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.1 v2:172.21.15.7:6824/4175601737 2 ==== osd_op_reply(17 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f5cb3800 2024-09-10T21:52:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.238+0000 7fa975800640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5637f5cca800 0x5637f5cbf080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.238+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:18 17.3c 17:3c000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5ccb400 con 0x5637f5cca800 2024-09-10T21:52:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.238+0000 7fa97b200640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5637f5cca800 0x5637f5cbf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.239+0000 7fa97b200640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5637f5cca800 0x5637f5cbf080 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.241 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.239+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.2 v2:172.21.15.7:6816/4238877366 1 ==== osd_op_reply(18 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f5cca800 2024-09-10T21:52:07.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.239+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:19 17.2 17:40000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f684fc00 con 0x5637f5ccbc00 2024-09-10T21:52:07.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.239+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.4 v2:172.21.15.133:6826/1857710445 2 ==== osd_op_reply(19 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5ccbc00 2024-09-10T21:52:07.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.239+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:20 17.22 17:44000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cca400 con 0x5637f684f800 2024-09-10T21:52:07.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.240+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.5 v2:172.21.15.133:6818/3838498382 2 ==== osd_op_reply(20 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f684f800 2024-09-10T21:52:07.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.240+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:21 17.12 17:48000000::::head [pgnls-filter start_epoch 118 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f67c2800 con 0x5637f5cca000 2024-09-10T21:52:07.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.240+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.3 v2:172.21.15.7:6800/2195665006 5 ==== osd_op_reply(21 [pgnls-filter start_epoch 118 out=81b] v118'23 uv23 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cca000 2024-09-10T21:52:07.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.240+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:22 17.0 17:00000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cb2800 con 0x5637f5cb2000 2024-09-10T21:52:07.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.241+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.7 v2:172.21.15.133:6802/2638399040 5 ==== osd_op_reply(22 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5cb2000 2024-09-10T21:52:07.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.241+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:23 17.20 17:04000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd6000 con 0x5637f5cb3800 2024-09-10T21:52:07.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.241+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.1 v2:172.21.15.7:6824/4175601737 3 ==== osd_op_reply(23 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f5cb3800 2024-09-10T21:52:07.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.241+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:24 17.10 17:08000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5ccac00 con 0x5637f5cca000 2024-09-10T21:52:07.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.241+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.3 v2:172.21.15.7:6800/2195665006 6 ==== osd_op_reply(24 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cca000 2024-09-10T21:52:07.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.241+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:25 17.30 17:0c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd6400 con 0x5637f684f000 2024-09-10T21:52:07.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.242+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.0 v2:172.21.15.7:6808/651402908 6 ==== osd_op_reply(25 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f684f000 2024-09-10T21:52:07.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.242+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:26 17.8 17:10000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cb2400 con 0x5637f5cb2000 2024-09-10T21:52:07.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.242+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.7 v2:172.21.15.133:6802/2638399040 6 ==== osd_op_reply(26 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5cb2000 2024-09-10T21:52:07.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.242+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:27 17.28 17:14000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd6800 con 0x5637f5cca000 2024-09-10T21:52:07.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.242+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.3 v2:172.21.15.7:6800/2195665006 7 ==== osd_op_reply(27 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cca000 2024-09-10T21:52:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.242+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:28 17.18 17:18000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd6000 con 0x5637f684f000 2024-09-10T21:52:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.243+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.0 v2:172.21.15.7:6808/651402908 7 ==== osd_op_reply(28 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f684f000 2024-09-10T21:52:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.243+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:29 17.38 17:1c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd6c00 con 0x5637f5cb2000 2024-09-10T21:52:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.243+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.7 v2:172.21.15.133:6802/2638399040 7 ==== osd_op_reply(29 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5cb2000 2024-09-10T21:52:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.243+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:30 17.4 17:20000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5ccb800 con 0x5637f684f000 2024-09-10T21:52:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.243+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.0 v2:172.21.15.7:6808/651402908 8 ==== osd_op_reply(30 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f684f000 2024-09-10T21:52:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.243+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:31 17.24 17:24000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd7000 con 0x5637f5cca000 2024-09-10T21:52:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.244+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.3 v2:172.21.15.7:6800/2195665006 8 ==== osd_op_reply(31 [pgnls-filter start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cca000 2024-09-10T21:52:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.244+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:32 17.14 17:28000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5ccb000 con 0x5637f5cca000 2024-09-10T21:52:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.244+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.3 v2:172.21.15.7:6800/2195665006 9 ==== osd_op_reply(32 [pgnls-filter start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cca000 2024-09-10T21:52:07.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.244+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:33 17.34 17:2c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cb3c00 con 0x5637f684f000 2024-09-10T21:52:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.244+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.0 v2:172.21.15.7:6808/651402908 9 ==== osd_op_reply(33 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f684f000 2024-09-10T21:52:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.244+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:34 17.c 17:30000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5ccb400 con 0x5637f5cb2000 2024-09-10T21:52:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.245+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.7 v2:172.21.15.133:6802/2638399040 8 ==== osd_op_reply(34 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5cb2000 2024-09-10T21:52:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.245+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:35 17.2c 17:34000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f684fc00 con 0x5637f5ccbc00 2024-09-10T21:52:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.245+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.4 v2:172.21.15.133:6826/1857710445 3 ==== osd_op_reply(35 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5ccbc00 2024-09-10T21:52:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.245+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:36 17.1c 17:38000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f66db400 con 0x5637f5cb3800 2024-09-10T21:52:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.246+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.1 v2:172.21.15.7:6824/4175601737 4 ==== osd_op_reply(36 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f5cb3800 2024-09-10T21:52:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.246+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:37 17.3c 17:3c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cca400 con 0x5637f5cca800 2024-09-10T21:52:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.246+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.2 v2:172.21.15.7:6816/4238877366 2 ==== osd_op_reply(37 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f5cca800 2024-09-10T21:52:07.248 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.246+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:38 17.2 17:40000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f684f400 con 0x5637f5ccbc00 2024-09-10T21:52:07.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.247+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.4 v2:172.21.15.133:6826/1857710445 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5ccbc00 2024-09-10T21:52:07.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.247+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:39 17.22 17:44000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5ccac00 con 0x5637f684f800 2024-09-10T21:52:07.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.247+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.5 v2:172.21.15.133:6818/3838498382 3 ==== osd_op_reply(39 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f684f800 2024-09-10T21:52:07.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.247+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:40 17.12 17:48000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd6800 con 0x5637f5cca000 2024-09-10T21:52:07.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.248+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.3 v2:172.21.15.7:6800/2195665006 10 ==== osd_op_reply(40 [pgnls-filter start_epoch 118 out=49b] v118'23 uv23 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cca000 2024-09-10T21:52:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.248+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:41 17.32 17:4c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f67c2800 con 0x5637f5cb2000 2024-09-10T21:52:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.248+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.7 v2:172.21.15.133:6802/2638399040 9 ==== osd_op_reply(41 [pgnls-filter start_epoch 118 out=49b] v116'15 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5cb2000 2024-09-10T21:52:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.248+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:42 17.a 17:50000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd7000 con 0x5637f5cb2000 2024-09-10T21:52:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.248+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.7 v2:172.21.15.133:6802/2638399040 10 ==== osd_op_reply(42 [pgnls-filter start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5cb2000 2024-09-10T21:52:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.248+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:43 17.2a 17:54000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f66db000 con 0x5637f5cca000 2024-09-10T21:52:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.249+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.3 v2:172.21.15.7:6800/2195665006 11 ==== osd_op_reply(43 [pgnls-filter start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cca000 2024-09-10T21:52:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.249+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:44 17.1a 17:58000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd6c00 con 0x5637f684f000 2024-09-10T21:52:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.249+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.0 v2:172.21.15.7:6808/651402908 10 ==== osd_op_reply(44 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f684f000 2024-09-10T21:52:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.249+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:45 17.3a 17:5c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd6400 con 0x5637f5cb3800 2024-09-10T21:52:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.250+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.1 v2:172.21.15.7:6824/4175601737 5 ==== osd_op_reply(45 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f5cb3800 2024-09-10T21:52:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.250+0000 7fa975800640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5637f5cd7400 0x5637f5cbf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.250+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:46 17.6 17:60000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd7800 con 0x5637f5cd7400 2024-09-10T21:52:07.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.250+0000 7fa97c600640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5637f5cd7400 0x5637f5cbf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.250+0000 7fa97c600640 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5637f5cd7400 0x5637f5cbf600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.251+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.6 v2:172.21.15.133:6810/3365981160 1 ==== osd_op_reply(46 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cd7400 2024-09-10T21:52:07.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.251+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:47 17.26 17:64000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd6000 con 0x5637f684f000 2024-09-10T21:52:07.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.251+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.0 v2:172.21.15.7:6808/651402908 11 ==== osd_op_reply(47 [pgnls-filter start_epoch 118 out=49b] v116'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f684f000 2024-09-10T21:52:07.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.251+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:48 17.16 17:68000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cb2800 con 0x5637f684f800 2024-09-10T21:52:07.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.252+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.5 v2:172.21.15.133:6818/3838498382 4 ==== osd_op_reply(48 [pgnls-filter start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f684f800 2024-09-10T21:52:07.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.252+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:49 17.36 17:6c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd6c00 con 0x5637f684f000 2024-09-10T21:52:07.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.252+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.0 v2:172.21.15.7:6808/651402908 12 ==== osd_op_reply(49 [pgnls-filter start_epoch 118 out=81b] v118'34 uv34 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5637f59f2000 con 0x5637f684f000 2024-09-10T21:52:07.254 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.252+0000 7fa97e613740 20 datascan.forall_objects: OSD matched oid 100000001f7.00000000 2024-09-10T21:52:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.252+0000 7fa97e613740 10 datascan.operator(): handling object 100000001f7.0 2024-09-10T21:52:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.252+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:50 17.36 17:6fe38dc2:::100000001f7.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cb3000 con 0x5637f684f000 2024-09-10T21:52:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.252+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.0 v2:172.21.15.7:6808/651402908 13 ==== osd_op_reply(50 100000001f7.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (126) out=126b,getxattr (30) out=30b,getxattr] v0'0 uv34 ondisk = 0) ==== 416+0+194 (crc 0 0 0) 0x5637f59f2000 con 0x5637f684f000 2024-09-10T21:52:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.252+0000 7fa97e613740 10 datascan.inject_with_backtrace: inode: 0x0x100000001f7 2024-09-10T21:52:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.252+0000 7fa97e613740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000001/bravo 2024-09-10T21:52:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.252+0000 7fa97e613740 20 datascan.get_frag_of: dirino=0x10000000001 target_dname=bravo 2024-09-10T21:52:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.252+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:51 16.7 16:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cb3400 con 0x5637f5cca000 2024-09-10T21:52:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.253+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.3 v2:172.21.15.7:6800/2195665006 12 ==== osd_op_reply(51 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv8 ondisk = 0) ==== 164+0+95 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cca000 2024-09-10T21:52:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.253+0000 7fa97e613740 20 datascan.get_frag_of: got backtrace parent 0x10000000000/flushed 2024-09-10T21:52:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.253+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:52 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd6800 con 0x5637f5cca800 2024-09-10T21:52:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.253+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.2 v2:172.21.15.7:6816/4238877366 3 ==== osd_op_reply(52 10000000000.00000000 [omap-get-vals-by-keys out=585b] v0'0 uv12 ondisk = 0) ==== 164+0+585 (crc 0 0 0) 0x5637f59f2000 con 0x5637f5cca800 2024-09-10T21:52:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.253+0000 7fa97e613740 20 datascan.read_dentry: oid=10000000000.00000000 dname=flushed frag=*, r=0 2024-09-10T21:52:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.253+0000 7fa97e613740 20 datascan.get_frag_of: fast path, fragtree is fragtree_t(*) 2024-09-10T21:52:07.255 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.253+0000 7fa97e613740 20 datascan.get_frag_of: frag is * 2024-09-10T21:52:07.256 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.253+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:53 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd7800 con 0x5637f5cca000 2024-09-10T21:52:07.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.254+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.3 v2:172.21.15.7:6800/2195665006 13 ==== osd_op_reply(53 10000000001.00000000 [omap-get-header out=274b] v0'0 uv8 ondisk = 0) ==== 164+0+274 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cca000 2024-09-10T21:52:07.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.254+0000 7fa97e613740 10 datascan.find_or_create_dirfrag: read_version = 8 2024-09-10T21:52:07.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.254+0000 7fa97e613740 20 datascan.find_or_create_dirfrag: Dirfrag already exists: 0x0x10000000001 * 2024-09-10T21:52:07.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.254+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:54 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cb2800 con 0x5637f5cca000 2024-09-10T21:52:07.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.254+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.3 v2:172.21.15.7:6800/2195665006 14 ==== osd_op_reply(54 10000000001.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv8 ondisk = 0) ==== 164+0+4 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cca000 2024-09-10T21:52:07.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.254+0000 7fa97e613740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-09-10T21:52:07.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.254+0000 7fa97e613740 20 datascan.read_dentry: bravo_head not found in result 2024-09-10T21:52:07.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.254+0000 7fa97e613740 10 datascan.inject_with_backtrace: Linking inode 0x0x100000001f7 at 0x0x10000000001/bravo with size=5 bytes 2024-09-10T21:52:07.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.254+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:55 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cb3400 con 0x5637f5cca000 2024-09-10T21:52:07.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.255+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.3 v2:172.21.15.7:6800/2195665006 15 ==== osd_op_reply(55 10000000001.00000000 [omap-set-vals] v118'9 uv9 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cca000 2024-09-10T21:52:07.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.255+0000 7fa97e613740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-09-10T21:52:07.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.255+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:56 17.e 17:70000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f66db000 con 0x5637f5cca800 2024-09-10T21:52:07.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.255+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.2 v2:172.21.15.7:6816/4238877366 4 ==== osd_op_reply(56 [pgnls-filter start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f5cca800 2024-09-10T21:52:07.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.256+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:57 17.2e 17:74000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5ccb800 con 0x5637f5cd7400 2024-09-10T21:52:07.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.256+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.6 v2:172.21.15.133:6810/3365981160 2 ==== osd_op_reply(57 [pgnls-filter start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cd7400 2024-09-10T21:52:07.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.256+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:58 17.1e 17:78000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cb2400 con 0x5637f5cb3800 2024-09-10T21:52:07.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.256+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.1 v2:172.21.15.7:6824/4175601737 6 ==== osd_op_reply(58 [pgnls-filter start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f5cb3800 2024-09-10T21:52:07.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.256+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:59 17.3e 17:7c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cca400 con 0x5637f5ccbc00 2024-09-10T21:52:07.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.257+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.4 v2:172.21.15.133:6826/1857710445 5 ==== osd_op_reply(59 [pgnls-filter start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5ccbc00 2024-09-10T21:52:07.259 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.257+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:60 17.1 17:80000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cb3000 con 0x5637f684f000 2024-09-10T21:52:07.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.257+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.0 v2:172.21.15.7:6808/651402908 14 ==== osd_op_reply(60 [pgnls-filter start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f684f000 2024-09-10T21:52:07.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.257+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:61 17.21 17:84000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f66db400 con 0x5637f684f000 2024-09-10T21:52:07.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.258+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.0 v2:172.21.15.7:6808/651402908 15 ==== osd_op_reply(61 [pgnls-filter start_epoch 118 out=49b] v116'24 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f684f000 2024-09-10T21:52:07.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.258+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:62 17.11 17:88000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cb2c00 con 0x5637f5cca000 2024-09-10T21:52:07.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.258+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.3 v2:172.21.15.7:6800/2195665006 16 ==== osd_op_reply(62 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cca000 2024-09-10T21:52:07.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.258+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:63 17.31 17:8c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cb3c00 con 0x5637f5cb3800 2024-09-10T21:52:07.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.258+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.1 v2:172.21.15.7:6824/4175601737 7 ==== osd_op_reply(63 [pgnls-filter start_epoch 118 out=49b] v116'15 uv8 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f5cb3800 2024-09-10T21:52:07.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.258+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:64 17.9 17:90000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f66db000 con 0x5637f5cca000 2024-09-10T21:52:07.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.259+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.3 v2:172.21.15.7:6800/2195665006 17 ==== osd_op_reply(64 [pgnls-filter start_epoch 118 out=49b] v116'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cca000 2024-09-10T21:52:07.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.259+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:65 17.29 17:94000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd7000 con 0x5637f5ccbc00 2024-09-10T21:52:07.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.259+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.4 v2:172.21.15.133:6826/1857710445 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5ccbc00 2024-09-10T21:52:07.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.259+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:66 17.19 17:98000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5ccb400 con 0x5637f684f800 2024-09-10T21:52:07.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.260+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.5 v2:172.21.15.133:6818/3838498382 5 ==== osd_op_reply(66 [pgnls-filter start_epoch 118 out=49b] v118'24 uv24 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f684f800 2024-09-10T21:52:07.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.260+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:67 17.39 17:9c000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd6400 con 0x5637f684f800 2024-09-10T21:52:07.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.260+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.5 v2:172.21.15.133:6818/3838498382 6 ==== osd_op_reply(67 [pgnls-filter start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f684f800 2024-09-10T21:52:07.262 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.260+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:68 17.5 17:a0000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cca400 con 0x5637f5cd7400 2024-09-10T21:52:07.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.261+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.6 v2:172.21.15.133:6810/3365981160 3 ==== osd_op_reply(68 [pgnls-filter start_epoch 118 out=49b] v118'15 uv15 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cd7400 2024-09-10T21:52:07.263 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.261+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:69 17.25 17:a4000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd6000 con 0x5637f5cb3800 2024-09-10T21:52:07.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.262+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.1 v2:172.21.15.7:6824/4175601737 8 ==== osd_op_reply(69 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f5cb3800 2024-09-10T21:52:07.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.262+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:70 17.15 17:a8000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f67c2800 con 0x5637f684f800 2024-09-10T21:52:07.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.262+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.5 v2:172.21.15.133:6818/3838498382 7 ==== osd_op_reply(70 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f684f800 2024-09-10T21:52:07.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.262+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:71 17.35 17:ac000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd6c00 con 0x5637f5ccbc00 2024-09-10T21:52:07.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.263+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.4 v2:172.21.15.133:6826/1857710445 7 ==== osd_op_reply(71 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5ccbc00 2024-09-10T21:52:07.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.263+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:72 17.d 17:b0000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f684fc00 con 0x5637f5cb2000 2024-09-10T21:52:07.264 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.263+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.7 v2:172.21.15.133:6802/2638399040 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5cb2000 2024-09-10T21:52:07.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.263+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:73 17.2d 17:b4000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd6800 con 0x5637f5cb2000 2024-09-10T21:52:07.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.264+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.7 v2:172.21.15.133:6802/2638399040 12 ==== osd_op_reply(73 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5cb2000 2024-09-10T21:52:07.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.264+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:74 17.1d 17:b8000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f684f400 con 0x5637f5cb2000 2024-09-10T21:52:07.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.264+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.7 v2:172.21.15.133:6802/2638399040 13 ==== osd_op_reply(74 [pgnls-filter start_epoch 118 out=49b] v116'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5cb2000 2024-09-10T21:52:07.265 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.264+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:75 17.3d 17:bc000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd7c00 con 0x5637f5cb2000 2024-09-10T21:52:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.265+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.7 v2:172.21.15.133:6802/2638399040 14 ==== osd_op_reply(75 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5cb2000 2024-09-10T21:52:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.265+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:76 17.3 17:c0000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd7800 con 0x5637f5ccbc00 2024-09-10T21:52:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.265+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.4 v2:172.21.15.133:6826/1857710445 8 ==== osd_op_reply(76 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5ccbc00 2024-09-10T21:52:07.266 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.265+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:77 17.23 17:c4000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cb2400 con 0x5637f5ccbc00 2024-09-10T21:52:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.266+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.4 v2:172.21.15.133:6826/1857710445 9 ==== osd_op_reply(77 [pgnls-filter start_epoch 118 out=49b] v116'14 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5ccbc00 2024-09-10T21:52:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.266+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:78 17.13 17:c8000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd6400 con 0x5637f5cca000 2024-09-10T21:52:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.266+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.3 v2:172.21.15.7:6800/2195665006 18 ==== osd_op_reply(78 [pgnls-filter start_epoch 118 out=49b] v116'30 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cca000 2024-09-10T21:52:07.267 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.266+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:79 17.33 17:cc000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd7000 con 0x5637f5cb2000 2024-09-10T21:52:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.267+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.7 v2:172.21.15.133:6802/2638399040 15 ==== osd_op_reply(79 [pgnls-filter start_epoch 118 out=49b] v116'20 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5cb2000 2024-09-10T21:52:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.267+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:80 17.b 17:d0000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5ccb800 con 0x5637f684f000 2024-09-10T21:52:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.267+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.0 v2:172.21.15.7:6808/651402908 16 ==== osd_op_reply(80 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f684f000 2024-09-10T21:52:07.268 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.267+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:81 17.2b 17:d4000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd6c00 con 0x5637f5ccbc00 2024-09-10T21:52:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.268+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.4 v2:172.21.15.133:6826/1857710445 10 ==== osd_op_reply(81 [pgnls-filter start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5ccbc00 2024-09-10T21:52:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.268+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:82 17.1b 17:d8000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5ccb400 con 0x5637f5ccbc00 2024-09-10T21:52:07.269 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.268+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.4 v2:172.21.15.133:6826/1857710445 11 ==== osd_op_reply(82 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5ccbc00 2024-09-10T21:52:07.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.268+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:83 17.3b 17:dc000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd6800 con 0x5637f5cca800 2024-09-10T21:52:07.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.269+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.2 v2:172.21.15.7:6816/4238877366 5 ==== osd_op_reply(83 [pgnls-filter start_epoch 118 out=49b] v116'8 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f5cca800 2024-09-10T21:52:07.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.269+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:84 17.7 17:e0000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f66db000 con 0x5637f684f000 2024-09-10T21:52:07.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.269+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.0 v2:172.21.15.7:6808/651402908 17 ==== osd_op_reply(84 [pgnls-filter start_epoch 118 out=49b] v116'10 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f684f000 2024-09-10T21:52:07.270 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.269+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:85 17.27 17:e4000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd7c00 con 0x5637f5ccbc00 2024-09-10T21:52:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.269+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.4 v2:172.21.15.133:6826/1857710445 12 ==== osd_op_reply(85 [pgnls-filter start_epoch 118 out=49b] v116'4 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5ccbc00 2024-09-10T21:52:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.269+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:86 17.17 17:e8000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cb2800 con 0x5637f5cd7400 2024-09-10T21:52:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.270+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.6 v2:172.21.15.133:6810/3365981160 4 ==== osd_op_reply(86 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cd7400 2024-09-10T21:52:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.270+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:87 17.37 17:ec000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cd7800 con 0x5637f684f000 2024-09-10T21:52:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.270+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.0 v2:172.21.15.7:6808/651402908 18 ==== osd_op_reply(87 [pgnls-filter start_epoch 118 out=49b] v116'22 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f684f000 2024-09-10T21:52:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.270+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:88 17.f 17:f0000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cb3400 con 0x5637f5cb3800 2024-09-10T21:52:07.271 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.271+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.1 v2:172.21.15.7:6824/4175601737 9 ==== osd_op_reply(88 [pgnls-filter start_epoch 118 out=49b] v116'18 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f5cb3800 2024-09-10T21:52:07.272 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.271+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:89 17.2f 17:f4000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f684fc00 con 0x5637f5ccbc00 2024-09-10T21:52:07.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.271+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 <== osd.4 v2:172.21.15.133:6826/1857710445 13 ==== osd_op_reply(89 [pgnls-filter start_epoch 118 out=49b] v116'16 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2780 con 0x5637f5ccbc00 2024-09-10T21:52:07.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.271+0000 7fa975800640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:90 17.1f 17:f8000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f5cb2c00 con 0x5637f5cd7400 2024-09-10T21:52:07.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97c600640 1 -- 172.21.15.7:0/3934142754 <== osd.6 v2:172.21.15.133:6810/3365981160 5 ==== osd_op_reply(90 [pgnls-filter start_epoch 118 out=49b] v116'12 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2500 con 0x5637f5cd7400 2024-09-10T21:52:07.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa976200640 1 -- 172.21.15.7:0/3934142754 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:91 17.3f 17:fc000000::::head [pgnls-filter start_epoch 118 in=93b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5637f684f400 con 0x5637f684f000 2024-09-10T21:52:07.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97b200640 1 -- 172.21.15.7:0/3934142754 <== osd.0 v2:172.21.15.7:6808/651402908 19 ==== osd_op_reply(91 [pgnls-filter start_epoch 118 out=49b] v116'17 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5637f59f2000 con 0x5637f684f000 2024-09-10T21:52:07.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5637f684f000 msgr2=0x5637f5a6a680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5637f684f000 0x5637f5a6a680 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.274 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5637f5cb3800 msgr2=0x5637f5cbe000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5637f5cb3800 0x5637f5cbe000 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5637f5cca800 msgr2=0x5637f5cbf080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5637f5cca800 0x5637f5cbf080 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5637f5cca000 msgr2=0x5637f5cbe580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5637f5cca000 0x5637f5cbe580 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 reap_dead start 2024-09-10T21:52:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5637f5ccbc00 msgr2=0x5637f5cbeb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5637f5ccbc00 0x5637f5cbeb00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5637f684f800 msgr2=0x5637f5a6ac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5637f684f800 0x5637f5a6ac00 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5637f5cd7400 msgr2=0x5637f5cbf600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5637f5cd7400 0x5637f5cbf600 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.275 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5637f5cb2000 msgr2=0x5637f5a6b180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5637f5cb2000 0x5637f5a6b180 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97bc00640 1 -- 172.21.15.7:0/3934142754 reap_dead start 2024-09-10T21:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5637f684ec00 msgr2=0x5637f5a6a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5637f684ec00 0x5637f5a6a100 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x5637f670b7a0 tx=0x5637f67e5080 comp rx=0 tx=0).stop 2024-09-10T21:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f67c3800 msgr2=0x5637f5a69600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f67c3800 0x5637f5a69600 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x5637f66f20c0 tx=0x5637f67e4e40 comp rx=0 tx=0).stop 2024-09-10T21:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 shutdown_connections 2024-09-10T21:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5637f684ec00 0x5637f5a6a100 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.272+0000 7fa97e613740 1 --2- 172.21.15.7:0/3934142754 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f67c3800 0x5637f5a69600 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.273+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 >> 172.21.15.7:0/3934142754 conn(0x5637f67c3000 msgr2=0x5637f59d8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.273+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 shutdown_connections 2024-09-10T21:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.273+0000 7fa97e613740 1 -- 172.21.15.7:0/3934142754 wait complete. 2024-09-10T21:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.273+0000 7fa97e613740 1 -- 172.21.15.7:0/2716963628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f58e9c00 msgr2=0x5637f5a68b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.276 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.273+0000 7fa97e613740 1 --2- 172.21.15.7:0/2716963628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f58e9c00 0x5637f5a68b00 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x5637f67ca060 tx=0x5637f59b0f00 comp rx=0 tx=0).stop 2024-09-10T21:52:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.273+0000 7fa97e613740 1 -- 172.21.15.7:0/2716963628 shutdown_connections 2024-09-10T21:52:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.273+0000 7fa97e613740 1 --2- 172.21.15.7:0/2716963628 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5637f58e9800 0x5637f5a68580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.273+0000 7fa97e613740 1 --2- 172.21.15.7:0/2716963628 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5637f58e9c00 0x5637f5a68b00 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.273+0000 7fa97e613740 1 --2- 172.21.15.7:0/2716963628 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5637f58e8c00 0x5637f5a68000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.273+0000 7fa97e613740 1 -- 172.21.15.7:0/2716963628 >> 172.21.15.7:0/2716963628 conn(0x5637f58e9000 msgr2=0x5637f59d9000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.273+0000 7fa97e613740 1 -- 172.21.15.7:0/2716963628 shutdown_connections 2024-09-10T21:52:07.277 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.273+0000 7fa97e613740 1 -- 172.21.15.7:0/2716963628 wait complete. 2024-09-10T21:52:07.279 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--filter-tag', 'mytag123', 'cephfs_data'] in time 0:00:00.137712, result: 2024-09-10T21:52:07.280 DEBUG:teuthology.orchestra.run.smithi007:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links 2024-09-10T21:52:07.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.350+0000 7f16b5d1c740 1 -- start start 2024-09-10T21:52:07.351 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.350+0000 7f16b5d1c740 1 Processor -- start 2024-09-10T21:52:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.351+0000 7f16b5d1c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9002c00 0x5576d917e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.351+0000 7f16b5d1c740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5576d9003800 0x5576d917e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.351+0000 7f16b5d1c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576d9003c00 0x5576d917eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.351+0000 7f16b5d1c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5576d9e00780 con 0x5576d9002c00 2024-09-10T21:52:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.351+0000 7f16b5d1c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5576d9e00600 con 0x5576d9003c00 2024-09-10T21:52:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.351+0000 7f16b5d1c740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5576d9e00480 con 0x5576d9003800 2024-09-10T21:52:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.351+0000 7f16b3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9002c00 0x5576d917e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.352 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.351+0000 7f16b3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9002c00 0x5576d917e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38114/0 (socket says 172.21.15.7:38114) 2024-09-10T21:52:07.353 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.351+0000 7f16b3400640 1 -- 172.21.15.7:0/4010865225 learned_addr learned my addr 172.21.15.7:0/4010865225 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:07.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.352+0000 7f16b2a00640 1 --2- 172.21.15.7:0/4010865225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5576d9003800 0x5576d917e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.352+0000 7f16b3e00640 1 --2- 172.21.15.7:0/4010865225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576d9003c00 0x5576d917eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.354 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.352+0000 7f16b3400640 1 -- 172.21.15.7:0/4010865225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5576d9003800 msgr2=0x5576d917e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:07.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.352+0000 7f16b3400640 1 --2- 172.21.15.7:0/4010865225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5576d9003800 0x5576d917e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.352+0000 7f16b3400640 1 -- 172.21.15.7:0/4010865225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576d9003c00 msgr2=0x5576d917eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:07.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.352+0000 7f16b3400640 1 --2- 172.21.15.7:0/4010865225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576d9003c00 0x5576d917eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.352+0000 7f16b3400640 1 -- 172.21.15.7:0/4010865225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576d902f680 con 0x5576d9002c00 2024-09-10T21:52:07.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.352+0000 7f16b3400640 1 --2- 172.21.15.7:0/4010865225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9002c00 0x5576d917e000 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x5576d9dfe060 tx=0x5576d9db3200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e82ad70d8871d2af server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.352+0000 7f16b0200640 1 -- 172.21.15.7:0/4010865225 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5576d9028ea0 con 0x5576d9002c00 2024-09-10T21:52:07.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.352+0000 7f16b0200640 1 -- 172.21.15.7:0/4010865225 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5576d90588c0 con 0x5576d9002c00 2024-09-10T21:52:07.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.352+0000 7f16b0200640 1 -- 172.21.15.7:0/4010865225 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5576d9029380 con 0x5576d9002c00 2024-09-10T21:52:07.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.352+0000 7f16b5d1c740 1 -- 172.21.15.7:0/4010865225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576d902f4a0 con 0x5576d9002c00 2024-09-10T21:52:07.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.352+0000 7f16b5d1c740 1 -- 172.21.15.7:0/4010865225 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({fsmap=0}) -- 0x5576d902fa40 con 0x5576d9002c00 2024-09-10T21:52:07.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.352+0000 7f16b5d1c740 4 waiting for MDS map... 2024-09-10T21:52:07.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.353+0000 7f16b0200640 1 -- 172.21.15.7:0/4010865225 <== mon.0 v2:172.21.15.7:3300/0 4 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x5576d9ef0600 con 0x5576d9002c00 2024-09-10T21:52:07.355 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.353+0000 7f16b0200640 1 -- 172.21.15.7:0/4010865225 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 109) ==== 2893+0+0 (secure 0 0 0) 0x5576d905d180 con 0x5576d9002c00 2024-09-10T21:52:07.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.353+0000 7f16b5d1c740 4 Got MDS map 109 2024-09-10T21:52:07.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.353+0000 7f16b5d1c740 4 datascan.main: Using metadata pool output 2024-09-10T21:52:07.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.353+0000 7f16b5d1c740 4 datascan.main: connecting to RADOS... 2024-09-10T21:52:07.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.354+0000 7f16b5d1c740 1 Processor -- start 2024-09-10T21:52:07.356 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.354+0000 7f16b5d1c740 1 -- start start 2024-09-10T21:52:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.354+0000 7f16b5d1c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576d9ef5400 0x5576d917f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.354+0000 7f16b5d1c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9ef5800 0x5576d917f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.355+0000 7f16b5d1c740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5576d9ef5c00 0x5576d917fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.355+0000 7f16b5d1c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5576d9e00300 con 0x5576d9ef5800 2024-09-10T21:52:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.355+0000 7f16b5d1c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5576d9e00000 con 0x5576d9ef5400 2024-09-10T21:52:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.355+0000 7f16b5d1c740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5576d9065800 con 0x5576d9ef5c00 2024-09-10T21:52:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.355+0000 7f16b3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9ef5800 0x5576d917f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.359 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.355+0000 7f16b3400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9ef5800 0x5576d917f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38126/0 (socket says 172.21.15.7:38126) 2024-09-10T21:52:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.355+0000 7f16b3400640 1 -- 172.21.15.7:0/1228654562 learned_addr learned my addr 172.21.15.7:0/1228654562 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.355+0000 7f16b3e00640 1 --2- 172.21.15.7:0/1228654562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576d9ef5400 0x5576d917f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.355+0000 7f16b2a00640 1 --2- 172.21.15.7:0/1228654562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5576d9ef5c00 0x5576d917fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.355+0000 7f16b3400640 1 -- 172.21.15.7:0/1228654562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5576d9ef5c00 msgr2=0x5576d917fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.355+0000 7f16b3400640 1 --2- 172.21.15.7:0/1228654562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5576d9ef5c00 0x5576d917fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.355+0000 7f16b3400640 1 -- 172.21.15.7:0/1228654562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576d9ef5400 msgr2=0x5576d917f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.355+0000 7f16b3400640 1 --2- 172.21.15.7:0/1228654562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576d9ef5400 0x5576d917f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.355+0000 7f16b3400640 1 -- 172.21.15.7:0/1228654562 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576d902f860 con 0x5576d9ef5800 2024-09-10T21:52:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.355+0000 7f16b3400640 1 --2- 172.21.15.7:0/1228654562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9ef5800 0x5576d917f600 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x5576d9dfeed0 tx=0x5576d9dea660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fac471c90323097 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.356+0000 7f16ada00640 1 -- 172.21.15.7:0/1228654562 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5576d90291e0 con 0x5576d9ef5800 2024-09-10T21:52:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.356+0000 7f16ada00640 1 -- 172.21.15.7:0/1228654562 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5576d9058fc0 con 0x5576d9ef5800 2024-09-10T21:52:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.356+0000 7f16ada00640 1 -- 172.21.15.7:0/1228654562 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5576d9028ea0 con 0x5576d9ef5800 2024-09-10T21:52:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.356+0000 7f16b5d1c740 1 -- 172.21.15.7:0/1228654562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9ef5800 msgr2=0x5576d917f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.360 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.356+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/1228654562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9ef5800 0x5576d917f600 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x5576d9dfeed0 tx=0x5576d9dea660 comp rx=0 tx=0).stop 2024-09-10T21:52:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.356+0000 7f16b5d1c740 1 -- 172.21.15.7:0/1228654562 shutdown_connections 2024-09-10T21:52:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.356+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/1228654562 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5576d9ef5c00 0x5576d917fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.356+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/1228654562 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9ef5800 0x5576d917f600 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.356+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/1228654562 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576d9ef5400 0x5576d917f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.356+0000 7f16b5d1c740 1 -- 172.21.15.7:0/1228654562 >> 172.21.15.7:0/1228654562 conn(0x5576d9ef5000 msgr2=0x5576d90f5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.356+0000 7f16b5d1c740 1 -- 172.21.15.7:0/1228654562 shutdown_connections 2024-09-10T21:52:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.356+0000 7f16b5d1c740 1 -- 172.21.15.7:0/1228654562 wait complete. 2024-09-10T21:52:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.357+0000 7f16b5d1c740 1 Processor -- start 2024-09-10T21:52:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.357+0000 7f16b5d1c740 1 -- start start 2024-09-10T21:52:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.358+0000 7f16b5d1c740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9ef5400 0x5576d917f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.358+0000 7f16b5d1c740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5576d9ef5800 0x5576d917f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.361 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.358+0000 7f16b5d1c740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576d9ef5c00 0x5576d917fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.358+0000 7f16b5d1c740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x5576d9e01680 con 0x5576d9ef5400 2024-09-10T21:52:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.358+0000 7f16b5d1c740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x5576d9e01800 con 0x5576d9ef5c00 2024-09-10T21:52:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.358+0000 7f16b5d1c740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x5576d9e01500 con 0x5576d9ef5800 2024-09-10T21:52:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.358+0000 7f16b3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9ef5400 0x5576d917f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.358+0000 7f16b3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9ef5400 0x5576d917f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38130/0 (socket says 172.21.15.7:38130) 2024-09-10T21:52:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.358+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 learned_addr learned my addr 172.21.15.7:0/2009386824 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.358+0000 7f16b3400640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5576d9ef5800 0x5576d917f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.358+0000 7f16b2a00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576d9ef5c00 0x5576d917fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.358+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5576d9ef5800 msgr2=0x5576d917f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.358+0000 7f16b3e00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5576d9ef5800 0x5576d917f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.358+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576d9ef5c00 msgr2=0x5576d917fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.358+0000 7f16b3e00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576d9ef5c00 0x5576d917fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.358+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576d902f680 con 0x5576d9ef5400 2024-09-10T21:52:07.362 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.358+0000 7f16b2a00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576d9ef5c00 0x5576d917fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:07.365 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.359+0000 7f16b3e00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9ef5400 0x5576d917f080 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x5576d9e1c0f0 tx=0x5576d9f04ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.359+0000 7f16abc00640 1 -- 172.21.15.7:0/2009386824 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5576d90291e0 con 0x5576d9ef5400 2024-09-10T21:52:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.359+0000 7f16abc00640 1 -- 172.21.15.7:0/2009386824 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x5576d9058c40 con 0x5576d9ef5400 2024-09-10T21:52:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.359+0000 7f16abc00640 1 -- 172.21.15.7:0/2009386824 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5576d9028ea0 con 0x5576d9ef5400 2024-09-10T21:52:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.359+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5576d902f4a0 con 0x5576d9ef5400 2024-09-10T21:52:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.359+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576d902f2c0 con 0x5576d9ef5400 2024-09-10T21:52:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.359+0000 7f16b5d1c740 4 datascan.init: resolving metadata pool 16 2024-09-10T21:52:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.361+0000 7f16abc00640 1 -- 172.21.15.7:0/2009386824 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x5576d905d500 con 0x5576d9ef5400 2024-09-10T21:52:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.361+0000 7f16abc00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5576d9e17c00 0x5576d9180100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.361+0000 7f16abc00640 1 -- 172.21.15.7:0/2009386824 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x5576d9f64800 con 0x5576d9ef5400 2024-09-10T21:52:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.361+0000 7f16b5d1c740 4 datascan.init: found metadata pool 'cephfs_metadata' 2024-09-10T21:52:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.361+0000 7f16b5d1c740 4 datascan.main: resolving metadata pool 16 2024-09-10T21:52:07.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.362+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5576d9fa0000 0x5576d9180680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.362+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:1 16.0 16:00000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa0400 con 0x5576d9fa0000 2024-09-10T21:52:07.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.362+0000 7f16b3400640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5576d9e17c00 0x5576d9180100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.362+0000 7f16b2a00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5576d9fa0000 0x5576d9180680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.362+0000 7f16b3400640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5576d9e17c00 0x5576d9180100 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x5576d9dfeed0 tx=0x5576d9f04cc0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.362+0000 7f16b2a00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5576d9fa0000 0x5576d9180680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.363+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.0 v2:172.21.15.7:6808/651402908 1 ==== osd_op_reply(1 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa0000 2024-09-10T21:52:07.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.363+0000 7f16b2a00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5576d9fa0400 0x5576d93c2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.363+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:2 16.10 16:08000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de7400 con 0x5576d9fa0400 2024-09-10T21:52:07.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.363+0000 7f16b3e00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5576d9fa0400 0x5576d93c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.363+0000 7f16b3e00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5576d9fa0400 0x5576d93c2000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.364+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 1 ==== osd_op_reply(2 [pgnls start_epoch 118 out=177b] v117'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.364+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:3 16.8 16:10000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de7400 con 0x5576d9fa0000 2024-09-10T21:52:07.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.364+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.0 v2:172.21.15.7:6808/651402908 2 ==== osd_op_reply(3 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa0000 2024-09-10T21:52:07.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.364+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:4 16.18 16:18000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de7400 con 0x5576d9fa0000 2024-09-10T21:52:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.364+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.0 v2:172.21.15.7:6808/651402908 3 ==== osd_op_reply(4 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa0000 2024-09-10T21:52:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.364+0000 7f16b2a00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5576d9de7400 0x5576d93c2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.364+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:5 16.4 16:20000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de7000 con 0x5576d9de7400 2024-09-10T21:52:07.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.365+0000 7f16b3400640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5576d9de7400 0x5576d93c2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.365+0000 7f16b3400640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5576d9de7400 0x5576d93c2580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.366+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.3 v2:172.21.15.7:6800/2195665006 1 ==== osd_op_reply(5 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c780 con 0x5576d9de7400 2024-09-10T21:52:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.366+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:6 16.14 16:28000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de7000 con 0x5576d9fa0000 2024-09-10T21:52:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.366+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.0 v2:172.21.15.7:6808/651402908 4 ==== osd_op_reply(6 [pgnls start_epoch 118 out=149b] v118'41 uv41 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa0000 2024-09-10T21:52:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.366+0000 7f16b2a00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5576d9de7000 0x5576d93c2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.366+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:7 16.c 16:30000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de6800 con 0x5576d9de7000 2024-09-10T21:52:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.366+0000 7f16b2a00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5576d9de7000 0x5576d93c2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.367+0000 7f16b2a00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5576d9de7000 0x5576d93c2b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.367+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.5 v2:172.21.15.133:6818/3838498382 1 ==== osd_op_reply(7 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c000 con 0x5576d9de7000 2024-09-10T21:52:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.367+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:8 16.1c 16:38000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de6800 con 0x5576d9fa0400 2024-09-10T21:52:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.367+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 2 ==== osd_op_reply(8 [pgnls start_epoch 118 out=73b] v117'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.367+0000 7f16b3e00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5576d9de6800 0x5576d93c3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.367+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:9 16.2 16:40000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1400 con 0x5576d9de6800 2024-09-10T21:52:07.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.368+0000 7f16b3e00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5576d9de6800 0x5576d93c3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.368+0000 7f16b3e00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5576d9de6800 0x5576d93c3080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.369+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.4 v2:172.21.15.133:6826/1857710445 1 ==== osd_op_reply(9 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c500 con 0x5576d9de6800 2024-09-10T21:52:07.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.369+0000 7f16b3e00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5576d9fa1400 0x5576d93c3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.369+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:10 16.12 16:48000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1800 con 0x5576d9fa1400 2024-09-10T21:52:07.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.369+0000 7f16b3400640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5576d9fa1400 0x5576d93c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.369+0000 7f16b3400640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5576d9fa1400 0x5576d93c3600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.370+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.6 v2:172.21.15.133:6810/3365981160 1 ==== osd_op_reply(10 [pgnls start_epoch 118 out=81b] v117'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5576d910c780 con 0x5576d9fa1400 2024-09-10T21:52:07.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.370+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:11 16.a 16:50000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1800 con 0x5576d9fa0400 2024-09-10T21:52:07.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.370+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 3 ==== osd_op_reply(11 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.370+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:12 16.1a 16:58000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1800 con 0x5576d9de7400 2024-09-10T21:52:07.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.370+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.3 v2:172.21.15.7:6800/2195665006 2 ==== osd_op_reply(12 [pgnls start_epoch 118 out=49b] v118'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c780 con 0x5576d9de7400 2024-09-10T21:52:07.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.370+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:13 16.6 16:60000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1800 con 0x5576d9fa1400 2024-09-10T21:52:07.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.371+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.6 v2:172.21.15.133:6810/3365981160 2 ==== osd_op_reply(13 [pgnls start_epoch 118 out=77b] v117'9 uv9 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x5576d910c780 con 0x5576d9fa1400 2024-09-10T21:52:07.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.371+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:14 16.16 16:68000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1800 con 0x5576d9de7400 2024-09-10T21:52:07.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.371+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.3 v2:172.21.15.7:6800/2195665006 3 ==== osd_op_reply(14 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c780 con 0x5576d9de7400 2024-09-10T21:52:07.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.371+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:15 16.e 16:70000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1800 con 0x5576d9fa0400 2024-09-10T21:52:07.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.371+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 4 ==== osd_op_reply(15 [pgnls start_epoch 118 out=73b] v117'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.371+0000 7f16b3e00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5576d9fa1800 0x5576d93c3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.371+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:16 16.1e 16:78000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1c00 con 0x5576d9fa1800 2024-09-10T21:52:07.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.372+0000 7f16b2a00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5576d9fa1800 0x5576d93c3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.372+0000 7f16b2a00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5576d9fa1800 0x5576d93c3b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.372+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.7 v2:172.21.15.133:6802/2638399040 1 ==== osd_op_reply(16 [pgnls start_epoch 118 out=49b] v118'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa1800 2024-09-10T21:52:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.372+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:17 16.1 16:80000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1c00 con 0x5576d9fa0400 2024-09-10T21:52:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.373+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 5 ==== osd_op_reply(17 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.373+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:18 16.11 16:88000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1c00 con 0x5576d9de7000 2024-09-10T21:52:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.373+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.5 v2:172.21.15.133:6818/3838498382 2 ==== osd_op_reply(18 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c000 con 0x5576d9de7000 2024-09-10T21:52:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.373+0000 7f16b2a00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5576d9fa1c00 0x5576d93c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.373+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:19 16.9 16:90000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de6c00 con 0x5576d9fa1c00 2024-09-10T21:52:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.373+0000 7f16b3e00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5576d9fa1c00 0x5576d93c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.374+0000 7f16b3e00640 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5576d9fa1c00 0x5576d93c4100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.374+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.1 v2:172.21.15.7:6824/4175601737 1 ==== osd_op_reply(19 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa1c00 2024-09-10T21:52:07.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.374+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:20 16.19 16:98000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de6c00 con 0x5576d9de6800 2024-09-10T21:52:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.375+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.4 v2:172.21.15.133:6826/1857710445 2 ==== osd_op_reply(20 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c500 con 0x5576d9de6800 2024-09-10T21:52:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.375+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:21 16.5 16:a0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de6c00 con 0x5576d9fa0000 2024-09-10T21:52:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.375+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.0 v2:172.21.15.7:6808/651402908 5 ==== osd_op_reply(21 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa0000 2024-09-10T21:52:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.375+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:22 16.15 16:a8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de6c00 con 0x5576d9fa0000 2024-09-10T21:52:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.375+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.0 v2:172.21.15.7:6808/651402908 6 ==== osd_op_reply(22 [pgnls start_epoch 118 out=97b] v117'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa0000 2024-09-10T21:52:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.375+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:23 16.d 16:b0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de6c00 con 0x5576d9fa1400 2024-09-10T21:52:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.376+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.6 v2:172.21.15.133:6810/3365981160 3 ==== osd_op_reply(23 [pgnls start_epoch 118 out=74b] v113'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x5576d910c780 con 0x5576d9fa1400 2024-09-10T21:52:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.376+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:24 16.1d 16:b8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de6c00 con 0x5576d9fa0400 2024-09-10T21:52:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.376+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 6 ==== osd_op_reply(24 [pgnls start_epoch 118 out=73b] v117'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.376+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:25 16.3 16:c0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de6c00 con 0x5576d9de7000 2024-09-10T21:52:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.376+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.5 v2:172.21.15.133:6818/3838498382 3 ==== osd_op_reply(25 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c000 con 0x5576d9de7000 2024-09-10T21:52:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.376+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:26 16.13 16:c8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de6c00 con 0x5576d9fa1c00 2024-09-10T21:52:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.377+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.1 v2:172.21.15.7:6824/4175601737 2 ==== osd_op_reply(26 [pgnls start_epoch 118 out=98b] v117'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa1c00 2024-09-10T21:52:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.377+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:27 16.b 16:d0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de6c00 con 0x5576d9fa1c00 2024-09-10T21:52:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.377+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.1 v2:172.21.15.7:6824/4175601737 3 ==== osd_op_reply(27 [pgnls start_epoch 118 out=76b] v118'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa1c00 2024-09-10T21:52:07.378 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.377+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:28 16.1b 16:d8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de6c00 con 0x5576d9de7000 2024-09-10T21:52:07.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.378+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.5 v2:172.21.15.133:6818/3838498382 4 ==== osd_op_reply(28 [pgnls start_epoch 118 out=49b] v118'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c000 con 0x5576d9de7000 2024-09-10T21:52:07.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.378+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:29 16.7 16:e0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de6c00 con 0x5576d9de7400 2024-09-10T21:52:07.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.378+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.3 v2:172.21.15.7:6800/2195665006 4 ==== osd_op_reply(29 [pgnls start_epoch 118 out=105b] v118'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x5576d910c780 con 0x5576d9de7400 2024-09-10T21:52:07.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.378+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:30 16.17 16:e8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de6c00 con 0x5576d9fa0400 2024-09-10T21:52:07.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.379+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 7 ==== osd_op_reply(30 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.379+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:31 16.f 16:f0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de6c00 con 0x5576d9fa0000 2024-09-10T21:52:07.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.379+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.0 v2:172.21.15.7:6808/651402908 7 ==== osd_op_reply(31 [pgnls start_epoch 118 out=73b] v118'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa0000 2024-09-10T21:52:07.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.379+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:32 16.1f 16:f8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9de6c00 con 0x5576d9de7000 2024-09-10T21:52:07.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.379+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.5 v2:172.21.15.133:6818/3838498382 5 ==== osd_op_reply(32 [pgnls start_epoch 118 out=101b] v118'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x5576d910c000 con 0x5576d9de7000 2024-09-10T21:52:07.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.380+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 601.00000000 2024-09-10T21:52:07.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.380+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:33 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa0800 con 0x5576d9fa0400 2024-09-10T21:52:07.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.380+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 8 ==== osd_op_reply(33 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.380+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 602.00000000 2024-09-10T21:52:07.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.380+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:34 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa0c00 con 0x5576d9fa0400 2024-09-10T21:52:07.381 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.380+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 9 ==== osd_op_reply(34 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.380+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2024-09-10T21:52:07.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.380+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:35 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa0400 2024-09-10T21:52:07.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.381+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 10 ==== osd_op_reply(35 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) ==== 164+0+1169 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.381+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 600.00000000 2024-09-10T21:52:07.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.381+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:36 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f0000 con 0x5576d9fa0400 2024-09-10T21:52:07.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.381+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 11 ==== osd_op_reply(36 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.381+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 603.00000000 2024-09-10T21:52:07.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.381+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:37 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f0400 con 0x5576d9fa0400 2024-09-10T21:52:07.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.381+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 12 ==== osd_op_reply(37 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.381+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2024-09-10T21:52:07.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.381+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-09-10T21:52:07.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.381+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 200.00000000 2024-09-10T21:52:07.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.381+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-09-10T21:52:07.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.381+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 200.00000001 2024-09-10T21:52:07.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.381+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-09-10T21:52:07.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.381+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 606.00000000 2024-09-10T21:52:07.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.381+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:38 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f0800 con 0x5576d9fa0000 2024-09-10T21:52:07.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.382+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.0 v2:172.21.15.7:6808/651402908 8 ==== osd_op_reply(38 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa0000 2024-09-10T21:52:07.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.382+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 607.00000000 2024-09-10T21:52:07.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.382+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:39 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f0c00 con 0x5576d9fa0400 2024-09-10T21:52:07.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.382+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 13 ==== osd_op_reply(39 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.382+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 10000000003.00000000 2024-09-10T21:52:07.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.382+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:40 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f1000 con 0x5576d9fa1400 2024-09-10T21:52:07.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.383+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.6 v2:172.21.15.133:6810/3365981160 4 ==== osd_op_reply(40 10000000003.00000000 [omap-get-vals out=592b] v0'0 uv3 ondisk = 0) ==== 164+0+592 (crc 0 0 0) 0x5576d910c780 con 0x5576d9fa1400 2024-09-10T21:52:07.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.383+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object mds0_openfiles.0 2024-09-10T21:52:07.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.383+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-09-10T21:52:07.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.383+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 608.00000000 2024-09-10T21:52:07.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.383+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:41 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f1400 con 0x5576d9fa0400 2024-09-10T21:52:07.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.383+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 14 ==== osd_op_reply(41 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.383+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 604.00000000 2024-09-10T21:52:07.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.383+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:42 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f0000 con 0x5576d9fa0000 2024-09-10T21:52:07.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.383+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.0 v2:172.21.15.7:6808/651402908 9 ==== osd_op_reply(42 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa0000 2024-09-10T21:52:07.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.384+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 500.00000000 2024-09-10T21:52:07.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.384+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-09-10T21:52:07.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.384+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object mds_snaptable 2024-09-10T21:52:07.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.384+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-09-10T21:52:07.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.384+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 605.00000000 2024-09-10T21:52:07.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.384+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:43 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f0400 con 0x5576d9fa0400 2024-09-10T21:52:07.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.384+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 15 ==== osd_op_reply(43 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.384+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object mds0_inotable 2024-09-10T21:52:07.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.384+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-09-10T21:52:07.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.384+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 100.00000000 2024-09-10T21:52:07.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.384+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:44 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f1800 con 0x5576d9fa1c00 2024-09-10T21:52:07.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.384+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.1 v2:172.21.15.7:6824/4175601737 4 ==== osd_op_reply(44 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa1c00 2024-09-10T21:52:07.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.384+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2024-09-10T21:52:07.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.384+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-09-10T21:52:07.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.384+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 609.00000000 2024-09-10T21:52:07.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.384+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:45 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f1c00 con 0x5576d9de7400 2024-09-10T21:52:07.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.385+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.3 v2:172.21.15.7:6800/2195665006 5 ==== osd_op_reply(45 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c780 con 0x5576d9de7400 2024-09-10T21:52:07.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.385+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 10000000001.00000000 2024-09-10T21:52:07.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.385+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:46 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa0800 con 0x5576d9de7400 2024-09-10T21:52:07.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.385+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.3 v2:172.21.15.7:6800/2195665006 6 ==== osd_op_reply(46 10000000001.00000000 [omap-get-vals out=1683b] v0'0 uv9 ondisk = 0) ==== 164+0+1683 (crc 0 0 0) 0x5576d910c780 con 0x5576d9de7400 2024-09-10T21:52:07.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.385+0000 7f16b5d1c740 20 datascan.scan_links: injected ino detected 2024-09-10T21:52:07.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.385+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 400.00000000 2024-09-10T21:52:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.385+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-09-10T21:52:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.385+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2024-09-10T21:52:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.385+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-09-10T21:52:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.385+0000 7f16b5d1c740 10 datascan.scan_links: step 1: handling object 1.00000000 2024-09-10T21:52:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.385+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:47 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa0c00 con 0x5576d9de7000 2024-09-10T21:52:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.386+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.5 v2:172.21.15.133:6818/3838498382 6 ==== osd_op_reply(47 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x5576d910c000 con 0x5576d9de7000 2024-09-10T21:52:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.386+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:48 16.0 16:00000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa0000 2024-09-10T21:52:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.386+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.0 v2:172.21.15.7:6808/651402908 10 ==== osd_op_reply(48 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa0000 2024-09-10T21:52:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.386+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:49 16.10 16:08000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa0400 2024-09-10T21:52:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.387+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 16 ==== osd_op_reply(49 [pgnls start_epoch 118 out=177b] v117'16 uv16 ondisk = 1) ==== 144+0+177 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.387+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:50 16.8 16:10000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa0000 2024-09-10T21:52:07.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.387+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.0 v2:172.21.15.7:6808/651402908 11 ==== osd_op_reply(50 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa0000 2024-09-10T21:52:07.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.387+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:51 16.18 16:18000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa0000 2024-09-10T21:52:07.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.387+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.0 v2:172.21.15.7:6808/651402908 12 ==== osd_op_reply(51 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa0000 2024-09-10T21:52:07.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.387+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:52 16.4 16:20000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9de7400 2024-09-10T21:52:07.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.388+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.3 v2:172.21.15.7:6800/2195665006 7 ==== osd_op_reply(52 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c780 con 0x5576d9de7400 2024-09-10T21:52:07.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.388+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:53 16.14 16:28000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa0000 2024-09-10T21:52:07.392 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.388+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.0 v2:172.21.15.7:6808/651402908 13 ==== osd_op_reply(53 [pgnls start_epoch 118 out=149b] v118'41 uv41 ondisk = 1) ==== 144+0+149 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa0000 2024-09-10T21:52:07.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.388+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:54 16.c 16:30000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9de7000 2024-09-10T21:52:07.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.388+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.5 v2:172.21.15.133:6818/3838498382 7 ==== osd_op_reply(54 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c000 con 0x5576d9de7000 2024-09-10T21:52:07.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.388+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:55 16.1c 16:38000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa0400 2024-09-10T21:52:07.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.389+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 17 ==== osd_op_reply(55 [pgnls start_epoch 118 out=73b] v117'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.389+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:56 16.2 16:40000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9de6800 2024-09-10T21:52:07.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.389+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.4 v2:172.21.15.133:6826/1857710445 3 ==== osd_op_reply(56 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c500 con 0x5576d9de6800 2024-09-10T21:52:07.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.389+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:57 16.12 16:48000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa1400 2024-09-10T21:52:07.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.389+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.6 v2:172.21.15.133:6810/3365981160 5 ==== osd_op_reply(57 [pgnls start_epoch 118 out=81b] v117'3 uv3 ondisk = 1) ==== 144+0+81 (crc 0 0 0) 0x5576d910c780 con 0x5576d9fa1400 2024-09-10T21:52:07.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.389+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:58 16.a 16:50000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa0400 2024-09-10T21:52:07.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.390+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 18 ==== osd_op_reply(58 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.390+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:59 16.1a 16:58000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9de7400 2024-09-10T21:52:07.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.390+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.3 v2:172.21.15.7:6800/2195665006 8 ==== osd_op_reply(59 [pgnls start_epoch 118 out=49b] v118'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c780 con 0x5576d9de7400 2024-09-10T21:52:07.393 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.390+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:60 16.6 16:60000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa1400 2024-09-10T21:52:07.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.390+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.6 v2:172.21.15.133:6810/3365981160 6 ==== osd_op_reply(60 [pgnls start_epoch 118 out=77b] v117'9 uv9 ondisk = 1) ==== 144+0+77 (crc 0 0 0) 0x5576d910c780 con 0x5576d9fa1400 2024-09-10T21:52:07.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.390+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:61 16.16 16:68000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9de7400 2024-09-10T21:52:07.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.391+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.3 v2:172.21.15.7:6800/2195665006 9 ==== osd_op_reply(61 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c780 con 0x5576d9de7400 2024-09-10T21:52:07.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.391+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:62 16.e 16:70000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa0400 2024-09-10T21:52:07.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.391+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 19 ==== osd_op_reply(62 [pgnls start_epoch 118 out=73b] v117'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.391+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- osd_op(unknown.0.0:63 16.1e 16:78000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa1800 2024-09-10T21:52:07.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.392+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.7 v2:172.21.15.133:6802/2638399040 2 ==== osd_op_reply(63 [pgnls start_epoch 118 out=49b] v118'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa1800 2024-09-10T21:52:07.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.392+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:64 16.1 16:80000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa0400 2024-09-10T21:52:07.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.392+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 20 ==== osd_op_reply(64 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.395 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.392+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:65 16.11 16:88000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9de7000 2024-09-10T21:52:07.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.392+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.5 v2:172.21.15.133:6818/3838498382 8 ==== osd_op_reply(65 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c000 con 0x5576d9de7000 2024-09-10T21:52:07.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.392+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:66 16.9 16:90000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa1c00 2024-09-10T21:52:07.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.393+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.1 v2:172.21.15.7:6824/4175601737 5 ==== osd_op_reply(66 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa1c00 2024-09-10T21:52:07.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.393+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- osd_op(unknown.0.0:67 16.19 16:98000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9de6800 2024-09-10T21:52:07.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.393+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.4 v2:172.21.15.133:6826/1857710445 4 ==== osd_op_reply(67 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c500 con 0x5576d9de6800 2024-09-10T21:52:07.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.393+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:68 16.5 16:a0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa0000 2024-09-10T21:52:07.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.394+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.0 v2:172.21.15.7:6808/651402908 14 ==== osd_op_reply(68 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa0000 2024-09-10T21:52:07.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.394+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:69 16.15 16:a8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa0000 2024-09-10T21:52:07.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.394+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.0 v2:172.21.15.7:6808/651402908 15 ==== osd_op_reply(69 [pgnls start_epoch 118 out=97b] v117'4 uv4 ondisk = 1) ==== 144+0+97 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa0000 2024-09-10T21:52:07.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.394+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:70 16.d 16:b0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa1400 2024-09-10T21:52:07.396 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.394+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.6 v2:172.21.15.133:6810/3365981160 7 ==== osd_op_reply(70 [pgnls start_epoch 118 out=74b] v113'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x5576d910c780 con 0x5576d9fa1400 2024-09-10T21:52:07.397 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.395+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:71 16.1d 16:b8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa0400 2024-09-10T21:52:07.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.395+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 21 ==== osd_op_reply(71 [pgnls start_epoch 118 out=73b] v117'3 uv3 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.395+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:72 16.3 16:c0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9de7000 2024-09-10T21:52:07.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.396+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.5 v2:172.21.15.133:6818/3838498382 9 ==== osd_op_reply(72 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c000 con 0x5576d9de7000 2024-09-10T21:52:07.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.396+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:73 16.13 16:c8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa1c00 2024-09-10T21:52:07.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.396+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.1 v2:172.21.15.7:6824/4175601737 6 ==== osd_op_reply(73 [pgnls start_epoch 118 out=98b] v117'5 uv5 ondisk = 1) ==== 144+0+98 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa1c00 2024-09-10T21:52:07.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.396+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:74 16.b 16:d0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa1c00 2024-09-10T21:52:07.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.396+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.1 v2:172.21.15.7:6824/4175601737 7 ==== osd_op_reply(74 [pgnls start_epoch 118 out=76b] v118'6 uv5 ondisk = 1) ==== 144+0+76 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa1c00 2024-09-10T21:52:07.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.396+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:75 16.1b 16:d8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9de7000 2024-09-10T21:52:07.398 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.397+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.5 v2:172.21.15.133:6818/3838498382 10 ==== osd_op_reply(75 [pgnls start_epoch 118 out=49b] v118'3 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c000 con 0x5576d9de7000 2024-09-10T21:52:07.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.397+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:76 16.7 16:e0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9de7400 2024-09-10T21:52:07.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.397+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.3 v2:172.21.15.7:6800/2195665006 10 ==== osd_op_reply(76 [pgnls start_epoch 118 out=105b] v118'9 uv9 ondisk = 1) ==== 144+0+105 (crc 0 0 0) 0x5576d910c780 con 0x5576d9de7400 2024-09-10T21:52:07.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.397+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:77 16.17 16:e8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa0400 2024-09-10T21:52:07.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.397+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 22 ==== osd_op_reply(77 [pgnls start_epoch 118 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.399 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.397+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:78 16.f 16:f0000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9fa0000 2024-09-10T21:52:07.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.397+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.0 v2:172.21.15.7:6808/651402908 16 ==== osd_op_reply(78 [pgnls start_epoch 118 out=73b] v118'2 uv2 ondisk = 1) ==== 144+0+73 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa0000 2024-09-10T21:52:07.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.398+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:79 16.1f 16:f8000000::::head [pgnls start_epoch 118 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa1000 con 0x5576d9de7000 2024-09-10T21:52:07.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.398+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.5 v2:172.21.15.133:6818/3838498382 11 ==== osd_op_reply(79 [pgnls start_epoch 118 out=101b] v118'9 uv8 ondisk = 1) ==== 144+0+101 (crc 0 0 0) 0x5576d910c000 con 0x5576d9de7000 2024-09-10T21:52:07.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.398+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 601.00000000 2024-09-10T21:52:07.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.398+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:80 16.10 16:09401a81:::601.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa0800 con 0x5576d9fa0400 2024-09-10T21:52:07.401 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.398+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 23 ==== osd_op_reply(80 601.00000000 [omap-get-vals out=5b] v0'0 uv13 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.398+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 602.00000000 2024-09-10T21:52:07.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.398+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:81 16.10 16:0bd6d154:::602.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f1000 con 0x5576d9fa0400 2024-09-10T21:52:07.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.399+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 24 ==== osd_op_reply(81 602.00000000 [omap-get-vals out=5b] v0'0 uv15 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.399+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2024-09-10T21:52:07.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.399+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:82 16.10 16:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f1c00 con 0x5576d9fa0400 2024-09-10T21:52:07.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.399+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 25 ==== osd_op_reply(82 10000000000.00000000 [omap-get-vals out=1169b] v0'0 uv12 ondisk = 0) ==== 164+0+1169 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.399+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 600.00000000 2024-09-10T21:52:07.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.399+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:83 16.10 16:0d82a743:::600.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d9e16c00 con 0x5576d9fa0400 2024-09-10T21:52:07.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.399+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 26 ==== osd_op_reply(83 600.00000000 [omap-get-vals out=5b] v0'0 uv14 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.399+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 603.00000000 2024-09-10T21:52:07.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.399+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:84 16.10 16:0d89b25e:::603.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d9e16800 con 0x5576d9fa0400 2024-09-10T21:52:07.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.400+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 27 ==== osd_op_reply(84 603.00000000 [omap-get-vals out=5b] v0'0 uv16 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.402 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.400+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2024-09-10T21:52:07.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.400+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2024-09-10T21:52:07.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.400+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 200.00000000 2024-09-10T21:52:07.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.400+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000000' 2024-09-10T21:52:07.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.400+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 200.00000001 2024-09-10T21:52:07.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.400+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '200.00000001' 2024-09-10T21:52:07.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.400+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 606.00000000 2024-09-10T21:52:07.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.400+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:85 16.14 16:2f55791f:::606.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d9e16400 con 0x5576d9fa0000 2024-09-10T21:52:07.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.400+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.0 v2:172.21.15.7:6808/651402908 17 ==== osd_op_reply(85 606.00000000 [omap-get-vals out=5b] v0'0 uv36 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa0000 2024-09-10T21:52:07.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.400+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 607.00000000 2024-09-10T21:52:07.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.400+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:86 16.1c 16:3ed09add:::607.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f0c00 con 0x5576d9fa0400 2024-09-10T21:52:07.403 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.400+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 28 ==== osd_op_reply(86 607.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.400+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 10000000003.00000000 2024-09-10T21:52:07.404 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.400+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:87 16.12 16:4def72d4:::10000000003.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f1400 con 0x5576d9fa1400 2024-09-10T21:52:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.401+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.6 v2:172.21.15.133:6810/3365981160 8 ==== osd_op_reply(87 10000000003.00000000 [omap-get-vals out=592b] v0'0 uv3 ondisk = 0) ==== 164+0+592 (crc 0 0 0) 0x5576d910c780 con 0x5576d9fa1400 2024-09-10T21:52:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.401+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object mds0_openfiles.0 2024-09-10T21:52:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.401+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag: 'mds0_openfiles.0' 2024-09-10T21:52:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.401+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 608.00000000 2024-09-10T21:52:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.401+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:88 16.e 16:717a0223:::608.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f0400 con 0x5576d9fa0400 2024-09-10T21:52:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.401+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 29 ==== osd_op_reply(88 608.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.401+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 604.00000000 2024-09-10T21:52:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.401+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- osd_op(unknown.0.0:89 16.15 16:a93a17c2:::604.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f1800 con 0x5576d9fa0000 2024-09-10T21:52:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.402+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.0 v2:172.21.15.7:6808/651402908 18 ==== osd_op_reply(89 604.00000000 [omap-get-vals out=5b] v0'0 uv4 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c000 con 0x5576d9fa0000 2024-09-10T21:52:07.407 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.402+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 500.00000000 2024-09-10T21:52:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.402+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '500.00000000' 2024-09-10T21:52:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.402+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object mds_snaptable 2024-09-10T21:52:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.402+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2024-09-10T21:52:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.402+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 605.00000000 2024-09-10T21:52:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.402+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- osd_op(unknown.0.0:90 16.1d 16:b871830b:::605.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f1000 con 0x5576d9fa0400 2024-09-10T21:52:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.402+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.2 v2:172.21.15.7:6816/4238877366 30 ==== osd_op_reply(90 605.00000000 [omap-get-vals out=5b] v0'0 uv3 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa0400 2024-09-10T21:52:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.402+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object mds0_inotable 2024-09-10T21:52:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.402+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2024-09-10T21:52:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.402+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 100.00000000 2024-09-10T21:52:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.402+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:91 16.13 16:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa0800 con 0x5576d9fa1c00 2024-09-10T21:52:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.402+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.1 v2:172.21.15.7:6824/4175601737 8 ==== osd_op_reply(91 100.00000000 [omap-get-vals out=5195b] v0'0 uv5 ondisk = 0) ==== 156+0+5195 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa1c00 2024-09-10T21:52:07.408 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.402+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2024-09-10T21:52:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.402+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2024-09-10T21:52:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.402+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 609.00000000 2024-09-10T21:52:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.402+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:92 16.7 16:e0b41d06:::609.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f1c00 con 0x5576d9de7400 2024-09-10T21:52:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.403+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.3 v2:172.21.15.7:6800/2195665006 11 ==== osd_op_reply(92 609.00000000 [omap-get-vals out=5b] v0'0 uv7 ondisk = 0) ==== 156+0+5 (crc 0 0 0) 0x5576d910c780 con 0x5576d9de7400 2024-09-10T21:52:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.403+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 10000000001.00000000 2024-09-10T21:52:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.403+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:93 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f0c00 con 0x5576d9de7400 2024-09-10T21:52:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.403+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.3 v2:172.21.15.7:6800/2195665006 12 ==== osd_op_reply(93 10000000001.00000000 [omap-get-vals out=1683b] v0'0 uv9 ondisk = 0) ==== 164+0+1683 (crc 0 0 0) 0x5576d910c780 con 0x5576d9de7400 2024-09-10T21:52:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.403+0000 7f16b5d1c740 20 datascan.scan_links: injected ino detected 2024-09-10T21:52:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.403+0000 7f16b5d1c740 20 datascan.scan_links: adding 0x100000001f7 for future processing to fix dnfirst 2024-09-10T21:52:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.403+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 400.00000000 2024-09-10T21:52:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.403+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag (invalid ino): '400.00000000' 2024-09-10T21:52:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.403+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2024-09-10T21:52:07.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.403+0000 7f16b5d1c740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2024-09-10T21:52:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.403+0000 7f16b5d1c740 10 datascan.scan_links: step 2: handling object 1.00000000 2024-09-10T21:52:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.403+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- osd_op(unknown.0.0:94 16.1f 16:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d93f0400 con 0x5576d9de7000 2024-09-10T21:52:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.404+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 <== osd.5 v2:172.21.15.133:6818/3838498382 12 ==== osd_op_reply(94 1.00000000 [omap-get-vals out=585b] v0'0 uv8 ondisk = 0) ==== 154+0+585 (crc 0 0 0) 0x5576d910c000 con 0x5576d9de7000 2024-09-10T21:52:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.404+0000 7f16b5d1c740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2024-09-10T21:52:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.404+0000 7f16b5d1c740 10 datascan.scan_links: removing dup dentries from 0 objects 2024-09-10T21:52:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.404+0000 7f16b5d1c740 10 datascan.scan_links: processing 0 bad_nlink_inos 2024-09-10T21:52:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.404+0000 7f16b5d1c740 10 datascan.scan_links: processing 1 injected_inos 2024-09-10T21:52:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.404+0000 7f16b5d1c740 10 datascan.scan_links: handling injected_ino 0x100000001f7 2024-09-10T21:52:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.404+0000 7f16b5d1c740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000001:bravo 2024-09-10T21:52:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.404+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:95 16.7 16:e2e07930:::10000000001.00000000:head [omap-get-vals-by-keys in=18b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d9e16c00 con 0x5576d9de7400 2024-09-10T21:52:07.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.404+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.3 v2:172.21.15.7:6800/2195665006 13 ==== osd_op_reply(95 10000000001.00000000 [omap-get-vals-by-keys out=506b] v0'0 uv9 ondisk = 0) ==== 164+0+506 (crc 0 0 0) 0x5576d910c780 con 0x5576d9de7400 2024-09-10T21:52:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.404+0000 7f16b5d1c740 20 datascan.read_dentry: oid=10000000001.00000000 dname=bravo frag=*, r=0 2024-09-10T21:52:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.404+0000 7f16b5d1c740 20 datascan.scan_links: first is now 2 2024-09-10T21:52:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.404+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- osd_op(unknown.0.0:96 16.7 16:e2e07930:::10000000001.00000000:head [omap-set-vals in=506b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5576d9e16800 con 0x5576d9de7400 2024-09-10T21:52:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.406+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.3 v2:172.21.15.7:6800/2195665006 14 ==== osd_op_reply(96 10000000001.00000000 [omap-set-vals] v118'10 uv10 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x5576d910c780 con 0x5576d9de7400 2024-09-10T21:52:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.406+0000 7f16b5d1c740 20 datascan.inject_linkage: Injected dentry 0x0x10000000001/bravo pointing to 0x0x100000001f7 2024-09-10T21:52:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.406+0000 7f16b5d1c740 10 datascan.scan_links: updating inotable 2024-09-10T21:52:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.406+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:97 16.13 16:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d9e16400 con 0x5576d9fa1c00 2024-09-10T21:52:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.406+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.1 v2:172.21.15.7:6824/4175601737 9 ==== osd_op_reply(97 mds0_inotable [read 0~50 out=50b] v0'0 uv4 ondisk = 0) ==== 157+0+50 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa1c00 2024-09-10T21:52:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.406+0000 7f16b5d1c740 10 mds.0.inotable: skip_inos was [0x10000000005~0x1f2,0x100000001f9~0xfffffffe07] 2024-09-10T21:52:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.406+0000 7f16b5d1c740 10 mds.0.inotable: skip_inos now [0x100000001f9~0xfffffffe07] 2024-09-10T21:52:07.411 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.406+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- osd_op(unknown.0.0:98 16.13 16:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa0800 con 0x5576d9fa1c00 2024-09-10T21:52:07.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b3e00640 1 -- 172.21.15.7:0/2009386824 <== osd.1 v2:172.21.15.7:6824/4175601737 10 ==== osd_op_reply(98 mds0_inotable [writefull 0~34] v118'6 uv6 ondisk = 0) ==== 157+0+0 (crc 0 0 0) 0x5576d910c500 con 0x5576d9fa1c00 2024-09-10T21:52:07.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b5d1c740 10 datascan.scan_links: updating snaptable 2024-09-10T21:52:07.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- osd_op(unknown.0.0:99 16.d 16:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e118) -- 0x5576d9fa0c00 con 0x5576d9fa1400 2024-09-10T21:52:07.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b3400640 1 -- 172.21.15.7:0/2009386824 <== osd.6 v2:172.21.15.133:6810/3365981160 9 ==== osd_op_reply(99 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) ==== 157+0+70 (crc 0 0 0) 0x5576d910c780 con 0x5576d9fa1400 2024-09-10T21:52:07.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5576d9fa0000 msgr2=0x5576d9180680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x5576d9fa0000 0x5576d9180680 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5576d9fa1c00 msgr2=0x5576d93c4100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x5576d9fa1c00 0x5576d93c4100 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5576d9fa0400 msgr2=0x5576d93c2000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x5576d9fa0400 0x5576d93c2000 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5576d9de7400 msgr2=0x5576d93c2580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x5576d9de7400 0x5576d93c2580 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5576d9de6800 msgr2=0x5576d93c3080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x5576d9de6800 0x5576d93c3080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5576d9de7000 msgr2=0x5576d93c2b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x5576d9de7000 0x5576d93c2b00 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5576d9fa1400 msgr2=0x5576d93c3600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x5576d9fa1400 0x5576d93c3600 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5576d9fa1800 msgr2=0x5576d93c3b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x5576d9fa1800 0x5576d93c3b80 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 reap_dead start 2024-09-10T21:52:07.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 reap_dead start 2024-09-10T21:52:07.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.408+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 reap_dead start 2024-09-10T21:52:07.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.409+0000 7f16b2a00640 1 -- 172.21.15.7:0/2009386824 reap_dead start 2024-09-10T21:52:07.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.409+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5576d9e17c00 msgr2=0x5576d9180100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.409+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5576d9e17c00 0x5576d9180100 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x5576d9dfeed0 tx=0x5576d9f04cc0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.409+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9ef5400 msgr2=0x5576d917f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.415 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.409+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9ef5400 0x5576d917f080 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x5576d9e1c0f0 tx=0x5576d9f04ea0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.409+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 shutdown_connections 2024-09-10T21:52:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.409+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x5576d9e17c00 0x5576d9180100 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.409+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/2009386824 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9ef5400 0x5576d917f080 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.409+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 >> 172.21.15.7:0/2009386824 conn(0x5576d9ef5000 msgr2=0x5576d90f5000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.409+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 shutdown_connections 2024-09-10T21:52:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.409+0000 7f16b5d1c740 1 -- 172.21.15.7:0/2009386824 wait complete. 2024-09-10T21:52:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.409+0000 7f16b5d1c740 1 -- 172.21.15.7:0/4010865225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9002c00 msgr2=0x5576d917e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.409+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/4010865225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9002c00 0x5576d917e000 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x5576d9dfe060 tx=0x5576d9db3200 comp rx=0 tx=0).stop 2024-09-10T21:52:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.409+0000 7f16b5d1c740 1 -- 172.21.15.7:0/4010865225 shutdown_connections 2024-09-10T21:52:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.410+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/4010865225 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x5576d9003c00 0x5576d917eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.410+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/4010865225 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x5576d9003800 0x5576d917e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.410+0000 7f16b5d1c740 1 --2- 172.21.15.7:0/4010865225 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x5576d9002c00 0x5576d917e000 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.410+0000 7f16b5d1c740 1 -- 172.21.15.7:0/4010865225 >> 172.21.15.7:0/4010865225 conn(0x5576d9003000 msgr2=0x5576d90f4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:07.416 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.410+0000 7f16b5d1c740 1 -- 172.21.15.7:0/4010865225 shutdown_connections 2024-09-10T21:52:07.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.410+0000 7f16b5d1c740 1 -- 172.21.15.7:0/4010865225 wait complete. 2024-09-10T21:52:07.417 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links'] in time 0:00:00.137715, result: 2024-09-10T21:52:07.418 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T21:52:07.422 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T21:52:07.517 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-09-10T21:52:07.517 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-09-10T21:52:07.517 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T21:52:07.517 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-09-10T21:52:07.517 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.517 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-09-10T21:52:07.517 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.517 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-09-10T21:52:07.517 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.517 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-09-10T21:52:07.517 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-09-10T21:52:07.518 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.518 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-09-10T21:52:07.518 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-09-10T21:52:07.518 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.518 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T21:52:07.518 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-09-10T21:52:07.518 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.518 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T21:52:07.518 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T21:52:07.518 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.518 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-09-10T21:52:07.518 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.518 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-09-10T21:52:07.519 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-09-10T21:52:07.519 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-09-10T21:52:07.519 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-09-10T21:52:07.519 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-09-10T21:52:07.519 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-09-10T21:52:07.519 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-09-10T21:52:07.519 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T21:52:07.519 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T21:52:07.519 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-09-10T21:52:07.519 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.519 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T21:52:07.519 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.519 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-09-10T21:52:07.519 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.520 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-09-10T21:52:07.520 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.520 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-09-10T21:52:07.520 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-09-10T21:52:07.520 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-09-10T21:52:07.520 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2024-09-10T21:52:07.520 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on skipped message = False 2024-09-10T21:52:07.520 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 2029451a-3c26-48c7-b874-0597197278ac 2024-09-10T21:52:07.520 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.520 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-09-10T21:52:07.520 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-09-10T21:52:07.520 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.520 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-09-10T21:52:07.521 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2024-09-10T21:52:07.521 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-09-10T21:52:07.521 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.521 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-09-10T21:52:07.521 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-09-10T21:52:07.521 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.521 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-09-10T21:52:07.521 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-09-10T21:52:07.521 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-09-10T21:52:07.521 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug trim objects = true 2024-09-10T21:52:07.521 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.521 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-09-10T21:52:07.521 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-09-10T21:52:07.521 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.521 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-09-10T21:52:07.522 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.522 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-09-10T21:52:07.522 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.522 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-09-10T21:52:07.522 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.522 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-09-10T21:52:07.522 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-09-10T21:52:07.522 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.522 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-09-10T21:52:07.522 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-09-10T21:52:07.522 INFO:teuthology.orchestra.run.smithi007.stdout: bdev async discard = True 2024-09-10T21:52:07.522 INFO:teuthology.orchestra.run.smithi007.stdout: bdev enable discard = True 2024-09-10T21:52:07.522 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore allocator = bitmap 2024-09-10T21:52:07.522 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-09-10T21:52:07.522 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-09-10T21:52:07.523 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 1/20 2024-09-10T21:52:07.523 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 1/20 2024-09-10T21:52:07.523 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-10T21:52:07.523 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-09-10T21:52:07.523 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 4/10 2024-09-10T21:52:07.523 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T21:52:07.523 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-09-10T21:52:07.523 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-09-10T21:52:07.523 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2024-09-10T21:52:07.523 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2024-09-10T21:52:07.523 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.523 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-09-10T21:52:07.523 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-10T21:52:07.523 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-09-10T21:52:07.524 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-10T21:52:07.524 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-10T21:52:07.524 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:52:07.524 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:52:07.524 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-09-10T21:52:07.524 INFO:teuthology.orchestra.run.smithi007.stdout: client mount timeout = 30 2024-09-10T21:52:07.524 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-09-10T21:52:07.524 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.524 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-09-10T21:52:07.524 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-10T21:52:07.524 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-10T21:52:07.524 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-09-10T21:52:07.524 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-10T21:52:07.524 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-09-10T21:52:07.524 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-09-10T21:52:07.525 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:52:07.525 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-09-10T21:52:07.525 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-09-10T21:52:07.525 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:52:07.525 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.525 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T21:52:07.525 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T21:52:07.525 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T21:52:07.525 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.525 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T21:52:07.525 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T21:52:07.525 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T21:52:07.525 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.526 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-09-10T21:52:07.526 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-09-10T21:52:07.526 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.526 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-09-10T21:52:07.526 INFO:teuthology.orchestra.run.smithi007.stdout: mon down mkfs grace = 300 2024-09-10T21:52:07.526 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.526 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-09-10T21:52:07.526 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-09-10T21:52:07.526 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-09-10T21:52:07.526 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-09-10T21:52:07.526 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T21:52:07.526 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T21:52:07.526 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = "" 2024-09-10T21:52:07.526 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-09-10T21:52:07.526 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-09-10T21:52:07.527 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-09-10T21:52:07.527 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-09-10T21:52:07.527 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-09-10T21:52:07.527 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-09-10T21:52:07.527 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-09-10T21:52:07.527 INFO:teuthology.orchestra.run.smithi007.stdout:[mds] 2024-09-10T21:52:07.527 INFO:teuthology.orchestra.run.smithi007.stdout: mds verify scatter = True 2024-09-10T21:52:07.527 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug scatterstat = False 2024-09-10T21:52:07.527 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2024-09-10T21:52:07.527 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2024-09-10T21:52:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T21:52:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2024-09-10T21:52:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2024-09-10T21:52:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2024-09-10T21:52:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2024-09-10T21:52:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2024-09-10T21:52:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2024-09-10T21:52:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2024-09-10T21:52:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T21:52:07.528 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2024-09-10T21:52:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T21:52:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T21:52:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2024-09-10T21:52:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2024-09-10T21:52:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2024-09-10T21:52:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2024-09-10T21:52:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2024-09-10T21:52:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2024-09-10T21:52:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2024-09-10T21:52:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2024-09-10T21:52:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T21:52:07.529 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T21:52:07.530 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2024-09-10T21:52:07.530 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.530 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T21:52:07.530 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.530 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2024-09-10T21:52:07.530 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.530 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2024-09-10T21:52:07.530 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.530 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2024-09-10T21:52:07.530 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2024-09-10T21:52:07.530 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2024-09-10T21:52:07.530 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.7,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2024-09-10T21:52:07.530 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on skipped message = False 2024-09-10T21:52:07.530 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 2029451a-3c26-48c7-b874-0597197278ac 2024-09-10T21:52:07.530 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.530 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2024-09-10T21:52:07.531 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2024-09-10T21:52:07.531 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.531 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2024-09-10T21:52:07.531 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2024-09-10T21:52:07.531 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2024-09-10T21:52:07.531 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.531 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2024-09-10T21:52:07.531 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2024-09-10T21:52:07.531 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.531 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2024-09-10T21:52:07.531 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2024-09-10T21:52:07.531 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2024-09-10T21:52:07.531 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug trim objects = true 2024-09-10T21:52:07.531 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.531 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2024-09-10T21:52:07.532 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2024-09-10T21:52:07.532 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.532 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2024-09-10T21:52:07.532 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.532 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2024-09-10T21:52:07.532 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.532 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2024-09-10T21:52:07.532 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.532 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2024-09-10T21:52:07.532 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2024-09-10T21:52:07.532 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.532 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2024-09-10T21:52:07.532 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2024-09-10T21:52:07.532 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2024-09-10T21:52:07.533 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2024-09-10T21:52:07.533 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = bitmap 2024-09-10T21:52:07.533 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2024-09-10T21:52:07.533 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2024-09-10T21:52:07.533 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 1/20 2024-09-10T21:52:07.533 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 1/20 2024-09-10T21:52:07.533 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-10T21:52:07.533 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2024-09-10T21:52:07.533 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 4/10 2024-09-10T21:52:07.533 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T21:52:07.533 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2024-09-10T21:52:07.533 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2024-09-10T21:52:07.533 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2024-09-10T21:52:07.533 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2024-09-10T21:52:07.533 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.534 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2024-09-10T21:52:07.534 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-10T21:52:07.534 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2024-09-10T21:52:07.534 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-09-10T21:52:07.534 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-09-10T21:52:07.534 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:52:07.534 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:52:07.534 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2024-09-10T21:52:07.534 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 30 2024-09-10T21:52:07.534 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2024-09-10T21:52:07.534 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.534 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2024-09-10T21:52:07.534 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-10T21:52:07.534 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-09-10T21:52:07.535 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2024-09-10T21:52:07.535 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-09-10T21:52:07.535 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2024-09-10T21:52:07.535 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2024-09-10T21:52:07.535 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:52:07.535 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2024-09-10T21:52:07.535 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2024-09-10T21:52:07.535 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:52:07.535 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.535 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T21:52:07.535 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T21:52:07.535 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T21:52:07.535 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.535 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T21:52:07.535 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T21:52:07.536 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T21:52:07.536 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.536 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2024-09-10T21:52:07.536 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2024-09-10T21:52:07.536 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.536 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2024-09-10T21:52:07.536 INFO:teuthology.orchestra.run.smithi133.stdout: mon down mkfs grace = 300 2024-09-10T21:52:07.536 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.536 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2024-09-10T21:52:07.536 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2024-09-10T21:52:07.536 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2024-09-10T21:52:07.536 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2024-09-10T21:52:07.536 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T21:52:07.536 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T21:52:07.536 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2024-09-10T21:52:07.537 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2024-09-10T21:52:07.537 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2024-09-10T21:52:07.537 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2024-09-10T21:52:07.537 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-09-10T21:52:07.537 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-09-10T21:52:07.537 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-09-10T21:52:07.537 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2024-09-10T21:52:07.537 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2024-09-10T21:52:07.537 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = True 2024-09-10T21:52:07.537 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = False 2024-09-10T21:52:07.545 DEBUG:teuthology.orchestra.run.smithi007:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T21:52:07.549 DEBUG:teuthology.orchestra.run.smithi133:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T21:52:07.643 INFO:teuthology.orchestra.run.smithi007.stdout:[global] 2024-09-10T21:52:07.644 INFO:teuthology.orchestra.run.smithi007.stdout: chdir = "" 2024-09-10T21:52:07.644 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T21:52:07.644 INFO:teuthology.orchestra.run.smithi007.stdout: auth supported = cephx 2024-09-10T21:52:07.644 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.644 INFO:teuthology.orchestra.run.smithi007.stdout: filestore xattr use omap = true 2024-09-10T21:52:07.644 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.644 INFO:teuthology.orchestra.run.smithi007.stdout: mon clock drift allowed = 1.000 2024-09-10T21:52:07.644 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.644 INFO:teuthology.orchestra.run.smithi007.stdout: osd crush chooseleaf type = 0 2024-09-10T21:52:07.644 INFO:teuthology.orchestra.run.smithi007.stdout: auth debug = true 2024-09-10T21:52:07.644 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.644 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on old message = true 2024-09-10T21:52:07.645 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on bug = true 2024-09-10T21:52:07.645 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.645 INFO:teuthology.orchestra.run.smithi007.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T21:52:07.645 INFO:teuthology.orchestra.run.smithi007.stdout: mon pg warn max object skew = 0 2024-09-10T21:52:07.645 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.645 INFO:teuthology.orchestra.run.smithi007.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T21:52:07.645 INFO:teuthology.orchestra.run.smithi007.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T21:52:07.645 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.645 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default size = 2 2024-09-10T21:52:07.645 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.645 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow primary affinity = true 2024-09-10T21:52:07.645 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd allow pg remap = true 2024-09-10T21:52:07.645 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on legacy crush tunables = false 2024-09-10T21:52:07.646 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on crush straw calc version zero = false 2024-09-10T21:52:07.646 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on no sortbitwise = false 2024-09-10T21:52:07.646 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on osd down out interval zero = false 2024-09-10T21:52:07.646 INFO:teuthology.orchestra.run.smithi007.stdout: mon warn on too few osds = false 2024-09-10T21:52:07.646 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T21:52:07.646 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T21:52:07.646 INFO:teuthology.orchestra.run.smithi007.stdout: mon_allow_pool_size_one = true 2024-09-10T21:52:07.646 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.646 INFO:teuthology.orchestra.run.smithi007.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T21:52:07.646 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.646 INFO:teuthology.orchestra.run.smithi007.stdout: osd default data pool replay window = 5 2024-09-10T21:52:07.646 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.646 INFO:teuthology.orchestra.run.smithi007.stdout: mon allow pool delete = true 2024-09-10T21:52:07.646 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.647 INFO:teuthology.orchestra.run.smithi007.stdout: mon cluster log file level = debug 2024-09-10T21:52:07.647 INFO:teuthology.orchestra.run.smithi007.stdout: debug asserts on shutdown = true 2024-09-10T21:52:07.647 INFO:teuthology.orchestra.run.smithi007.stdout: mon health detail to clog = false 2024-09-10T21:52:07.647 INFO:teuthology.orchestra.run.smithi007.stdout: mon host = "172.21.15.7,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2024-09-10T21:52:07.647 INFO:teuthology.orchestra.run.smithi007.stdout: ms die on skipped message = False 2024-09-10T21:52:07.647 INFO:teuthology.orchestra.run.smithi007.stdout: fsid = 2029451a-3c26-48c7-b874-0597197278ac 2024-09-10T21:52:07.647 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.647 INFO:teuthology.orchestra.run.smithi007.stdout:[osd] 2024-09-10T21:52:07.647 INFO:teuthology.orchestra.run.smithi007.stdout: osd journal size = 100 2024-09-10T21:52:07.647 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.647 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub load threshold = 5.0 2024-09-10T21:52:07.647 INFO:teuthology.orchestra.run.smithi007.stdout: osd scrub max interval = 600 2024-09-10T21:52:07.647 INFO:teuthology.orchestra.run.smithi007.stdout: osd mclock profile = high_recovery_ops 2024-09-10T21:52:07.647 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.648 INFO:teuthology.orchestra.run.smithi007.stdout: osd recover clone overlap = true 2024-09-10T21:52:07.648 INFO:teuthology.orchestra.run.smithi007.stdout: osd recovery max chunk = 1048576 2024-09-10T21:52:07.648 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.648 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug shutdown = true 2024-09-10T21:52:07.648 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug op order = true 2024-09-10T21:52:07.648 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug verify stray on activate = true 2024-09-10T21:52:07.648 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug trim objects = true 2024-09-10T21:52:07.648 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.648 INFO:teuthology.orchestra.run.smithi007.stdout: osd open classes on start = true 2024-09-10T21:52:07.648 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug pg log writeout = true 2024-09-10T21:52:07.648 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.648 INFO:teuthology.orchestra.run.smithi007.stdout: osd deep scrub update digest min age = 30 2024-09-10T21:52:07.648 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.648 INFO:teuthology.orchestra.run.smithi007.stdout: osd map max advance = 10 2024-09-10T21:52:07.648 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.649 INFO:teuthology.orchestra.run.smithi007.stdout: journal zero on create = true 2024-09-10T21:52:07.649 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.649 INFO:teuthology.orchestra.run.smithi007.stdout: filestore ondisk finisher threads = 3 2024-09-10T21:52:07.649 INFO:teuthology.orchestra.run.smithi007.stdout: filestore apply finisher threads = 3 2024-09-10T21:52:07.649 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.649 INFO:teuthology.orchestra.run.smithi007.stdout: bdev debug aio = true 2024-09-10T21:52:07.649 INFO:teuthology.orchestra.run.smithi007.stdout: osd debug misdirected ops = true 2024-09-10T21:52:07.649 INFO:teuthology.orchestra.run.smithi007.stdout: bdev async discard = True 2024-09-10T21:52:07.649 INFO:teuthology.orchestra.run.smithi007.stdout: bdev enable discard = True 2024-09-10T21:52:07.649 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore allocator = bitmap 2024-09-10T21:52:07.649 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore block size = 96636764160 2024-09-10T21:52:07.649 INFO:teuthology.orchestra.run.smithi007.stdout: bluestore fsck on mount = True 2024-09-10T21:52:07.649 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluefs = 1/20 2024-09-10T21:52:07.649 INFO:teuthology.orchestra.run.smithi007.stdout: debug bluestore = 1/20 2024-09-10T21:52:07.650 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-10T21:52:07.650 INFO:teuthology.orchestra.run.smithi007.stdout: debug osd = 20 2024-09-10T21:52:07.650 INFO:teuthology.orchestra.run.smithi007.stdout: debug rocksdb = 4/10 2024-09-10T21:52:07.650 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T21:52:07.650 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd full ratio = 0.9 2024-09-10T21:52:07.650 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd nearfull ratio = 0.8 2024-09-10T21:52:07.650 INFO:teuthology.orchestra.run.smithi007.stdout: osd failsafe full ratio = 0.95 2024-09-10T21:52:07.650 INFO:teuthology.orchestra.run.smithi007.stdout: osd objectstore = bluestore 2024-09-10T21:52:07.650 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.650 INFO:teuthology.orchestra.run.smithi007.stdout:[mgr] 2024-09-10T21:52:07.650 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-10T21:52:07.650 INFO:teuthology.orchestra.run.smithi007.stdout: debug mgr = 20 2024-09-10T21:52:07.650 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-10T21:52:07.650 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-10T21:52:07.650 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:52:07.651 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:52:07.651 INFO:teuthology.orchestra.run.smithi007.stdout: mgr/telemetry/nag = false 2024-09-10T21:52:07.651 INFO:teuthology.orchestra.run.smithi007.stdout: client mount timeout = 30 2024-09-10T21:52:07.651 INFO:teuthology.orchestra.run.smithi007.stdout: debug client = 20 2024-09-10T21:52:07.651 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.651 INFO:teuthology.orchestra.run.smithi007.stdout:[mon] 2024-09-10T21:52:07.651 INFO:teuthology.orchestra.run.smithi007.stdout: debug ms = 1 2024-09-10T21:52:07.651 INFO:teuthology.orchestra.run.smithi007.stdout: debug mon = 20 2024-09-10T21:52:07.651 INFO:teuthology.orchestra.run.smithi007.stdout: debug paxos = 20 2024-09-10T21:52:07.651 INFO:teuthology.orchestra.run.smithi007.stdout: debug auth = 20 2024-09-10T21:52:07.651 INFO:teuthology.orchestra.run.smithi007.stdout: mon data avail warn = 5 2024-09-10T21:52:07.651 INFO:teuthology.orchestra.run.smithi007.stdout: mon mgr mkfs grace = 240 2024-09-10T21:52:07.651 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:52:07.651 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd reporter subtree level = osd 2024-09-10T21:52:07.652 INFO:teuthology.orchestra.run.smithi007.stdout: mon osd prime pg temp = true 2024-09-10T21:52:07.652 INFO:teuthology.orchestra.run.smithi007.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:52:07.652 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.652 INFO:teuthology.orchestra.run.smithi007.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T21:52:07.652 INFO:teuthology.orchestra.run.smithi007.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T21:52:07.652 INFO:teuthology.orchestra.run.smithi007.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T21:52:07.652 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.652 INFO:teuthology.orchestra.run.smithi007.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T21:52:07.652 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T21:52:07.652 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T21:52:07.652 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.652 INFO:teuthology.orchestra.run.smithi007.stdout: # 1m isn't quite enough 2024-09-10T21:52:07.652 INFO:teuthology.orchestra.run.smithi007.stdout: mon_down_mkfs_grace = 2m 2024-09-10T21:52:07.652 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.653 INFO:teuthology.orchestra.run.smithi007.stdout: mon_warn_on_filestore_osds = false 2024-09-10T21:52:07.653 INFO:teuthology.orchestra.run.smithi007.stdout: mon down mkfs grace = 300 2024-09-10T21:52:07.653 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:07.653 INFO:teuthology.orchestra.run.smithi007.stdout:[client] 2024-09-10T21:52:07.653 INFO:teuthology.orchestra.run.smithi007.stdout: rgw cache enabled = true 2024-09-10T21:52:07.653 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable ops log = true 2024-09-10T21:52:07.653 INFO:teuthology.orchestra.run.smithi007.stdout: rgw enable usage log = true 2024-09-10T21:52:07.653 INFO:teuthology.orchestra.run.smithi007.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T21:52:07.653 INFO:teuthology.orchestra.run.smithi007.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T21:52:07.653 INFO:teuthology.orchestra.run.smithi007.stdout: pid file = "" 2024-09-10T21:52:07.653 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.a] 2024-09-10T21:52:07.653 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.b] 2024-09-10T21:52:07.653 INFO:teuthology.orchestra.run.smithi007.stdout:[mon.c] 2024-09-10T21:52:07.654 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.a] 2024-09-10T21:52:07.654 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.c] 2024-09-10T21:52:07.654 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.b] 2024-09-10T21:52:07.654 INFO:teuthology.orchestra.run.smithi007.stdout:[mds.d] 2024-09-10T21:52:07.654 INFO:teuthology.orchestra.run.smithi007.stdout:[mds] 2024-09-10T21:52:07.654 INFO:teuthology.orchestra.run.smithi007.stdout: mds verify scatter = True 2024-09-10T21:52:07.654 INFO:teuthology.orchestra.run.smithi007.stdout: mds debug scatterstat = True 2024-09-10T21:52:07.654 INFO:teuthology.orchestra.run.smithi133.stdout:[global] 2024-09-10T21:52:07.654 INFO:teuthology.orchestra.run.smithi133.stdout: chdir = "" 2024-09-10T21:52:07.654 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T21:52:07.654 INFO:teuthology.orchestra.run.smithi133.stdout: auth supported = cephx 2024-09-10T21:52:07.655 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.655 INFO:teuthology.orchestra.run.smithi133.stdout: filestore xattr use omap = true 2024-09-10T21:52:07.655 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.655 INFO:teuthology.orchestra.run.smithi133.stdout: mon clock drift allowed = 1.000 2024-09-10T21:52:07.655 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.655 INFO:teuthology.orchestra.run.smithi133.stdout: osd crush chooseleaf type = 0 2024-09-10T21:52:07.655 INFO:teuthology.orchestra.run.smithi133.stdout: auth debug = true 2024-09-10T21:52:07.655 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.655 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on old message = true 2024-09-10T21:52:07.655 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on bug = true 2024-09-10T21:52:07.655 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.655 INFO:teuthology.orchestra.run.smithi133.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T21:52:07.655 INFO:teuthology.orchestra.run.smithi133.stdout: mon pg warn max object skew = 0 2024-09-10T21:52:07.655 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.655 INFO:teuthology.orchestra.run.smithi133.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T21:52:07.656 INFO:teuthology.orchestra.run.smithi133.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T21:52:07.656 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.656 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default size = 2 2024-09-10T21:52:07.656 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.656 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow primary affinity = true 2024-09-10T21:52:07.656 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd allow pg remap = true 2024-09-10T21:52:07.656 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on legacy crush tunables = false 2024-09-10T21:52:07.656 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on crush straw calc version zero = false 2024-09-10T21:52:07.656 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on no sortbitwise = false 2024-09-10T21:52:07.656 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on osd down out interval zero = false 2024-09-10T21:52:07.656 INFO:teuthology.orchestra.run.smithi133.stdout: mon warn on too few osds = false 2024-09-10T21:52:07.656 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T21:52:07.656 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T21:52:07.656 INFO:teuthology.orchestra.run.smithi133.stdout: mon_allow_pool_size_one = true 2024-09-10T21:52:07.657 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.657 INFO:teuthology.orchestra.run.smithi133.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T21:52:07.657 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.657 INFO:teuthology.orchestra.run.smithi133.stdout: osd default data pool replay window = 5 2024-09-10T21:52:07.657 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.657 INFO:teuthology.orchestra.run.smithi133.stdout: mon allow pool delete = true 2024-09-10T21:52:07.657 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.657 INFO:teuthology.orchestra.run.smithi133.stdout: mon cluster log file level = debug 2024-09-10T21:52:07.657 INFO:teuthology.orchestra.run.smithi133.stdout: debug asserts on shutdown = true 2024-09-10T21:52:07.657 INFO:teuthology.orchestra.run.smithi133.stdout: mon health detail to clog = false 2024-09-10T21:52:07.657 INFO:teuthology.orchestra.run.smithi133.stdout: mon host = "172.21.15.7,172.21.15.133,[v2:172.21.15.133:3301,v1:172.21.15.133:6790]" 2024-09-10T21:52:07.657 INFO:teuthology.orchestra.run.smithi133.stdout: ms die on skipped message = False 2024-09-10T21:52:07.657 INFO:teuthology.orchestra.run.smithi133.stdout: fsid = 2029451a-3c26-48c7-b874-0597197278ac 2024-09-10T21:52:07.657 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.658 INFO:teuthology.orchestra.run.smithi133.stdout:[osd] 2024-09-10T21:52:07.658 INFO:teuthology.orchestra.run.smithi133.stdout: osd journal size = 100 2024-09-10T21:52:07.658 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.658 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub load threshold = 5.0 2024-09-10T21:52:07.658 INFO:teuthology.orchestra.run.smithi133.stdout: osd scrub max interval = 600 2024-09-10T21:52:07.658 INFO:teuthology.orchestra.run.smithi133.stdout: osd mclock profile = high_recovery_ops 2024-09-10T21:52:07.658 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.658 INFO:teuthology.orchestra.run.smithi133.stdout: osd recover clone overlap = true 2024-09-10T21:52:07.658 INFO:teuthology.orchestra.run.smithi133.stdout: osd recovery max chunk = 1048576 2024-09-10T21:52:07.658 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.658 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug shutdown = true 2024-09-10T21:52:07.658 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug op order = true 2024-09-10T21:52:07.658 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug verify stray on activate = true 2024-09-10T21:52:07.658 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug trim objects = true 2024-09-10T21:52:07.658 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.658 INFO:teuthology.orchestra.run.smithi133.stdout: osd open classes on start = true 2024-09-10T21:52:07.659 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug pg log writeout = true 2024-09-10T21:52:07.659 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.659 INFO:teuthology.orchestra.run.smithi133.stdout: osd deep scrub update digest min age = 30 2024-09-10T21:52:07.659 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.659 INFO:teuthology.orchestra.run.smithi133.stdout: osd map max advance = 10 2024-09-10T21:52:07.659 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.659 INFO:teuthology.orchestra.run.smithi133.stdout: journal zero on create = true 2024-09-10T21:52:07.659 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.659 INFO:teuthology.orchestra.run.smithi133.stdout: filestore ondisk finisher threads = 3 2024-09-10T21:52:07.659 INFO:teuthology.orchestra.run.smithi133.stdout: filestore apply finisher threads = 3 2024-09-10T21:52:07.659 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.659 INFO:teuthology.orchestra.run.smithi133.stdout: bdev debug aio = true 2024-09-10T21:52:07.659 INFO:teuthology.orchestra.run.smithi133.stdout: osd debug misdirected ops = true 2024-09-10T21:52:07.659 INFO:teuthology.orchestra.run.smithi133.stdout: bdev async discard = True 2024-09-10T21:52:07.659 INFO:teuthology.orchestra.run.smithi133.stdout: bdev enable discard = True 2024-09-10T21:52:07.660 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore allocator = bitmap 2024-09-10T21:52:07.660 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore block size = 96636764160 2024-09-10T21:52:07.660 INFO:teuthology.orchestra.run.smithi133.stdout: bluestore fsck on mount = True 2024-09-10T21:52:07.660 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluefs = 1/20 2024-09-10T21:52:07.660 INFO:teuthology.orchestra.run.smithi133.stdout: debug bluestore = 1/20 2024-09-10T21:52:07.660 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-10T21:52:07.660 INFO:teuthology.orchestra.run.smithi133.stdout: debug osd = 20 2024-09-10T21:52:07.660 INFO:teuthology.orchestra.run.smithi133.stdout: debug rocksdb = 4/10 2024-09-10T21:52:07.660 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T21:52:07.660 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd full ratio = 0.9 2024-09-10T21:52:07.660 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd nearfull ratio = 0.8 2024-09-10T21:52:07.660 INFO:teuthology.orchestra.run.smithi133.stdout: osd failsafe full ratio = 0.95 2024-09-10T21:52:07.660 INFO:teuthology.orchestra.run.smithi133.stdout: osd objectstore = bluestore 2024-09-10T21:52:07.660 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.660 INFO:teuthology.orchestra.run.smithi133.stdout:[mgr] 2024-09-10T21:52:07.661 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-10T21:52:07.661 INFO:teuthology.orchestra.run.smithi133.stdout: debug mgr = 20 2024-09-10T21:52:07.661 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-09-10T21:52:07.661 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-09-10T21:52:07.661 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:52:07.661 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:52:07.661 INFO:teuthology.orchestra.run.smithi133.stdout: mgr/telemetry/nag = false 2024-09-10T21:52:07.661 INFO:teuthology.orchestra.run.smithi133.stdout: client mount timeout = 30 2024-09-10T21:52:07.661 INFO:teuthology.orchestra.run.smithi133.stdout: debug client = 20 2024-09-10T21:52:07.661 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.661 INFO:teuthology.orchestra.run.smithi133.stdout:[mon] 2024-09-10T21:52:07.661 INFO:teuthology.orchestra.run.smithi133.stdout: debug ms = 1 2024-09-10T21:52:07.661 INFO:teuthology.orchestra.run.smithi133.stdout: debug mon = 20 2024-09-10T21:52:07.661 INFO:teuthology.orchestra.run.smithi133.stdout: debug paxos = 20 2024-09-10T21:52:07.662 INFO:teuthology.orchestra.run.smithi133.stdout: debug auth = 20 2024-09-10T21:52:07.662 INFO:teuthology.orchestra.run.smithi133.stdout: mon data avail warn = 5 2024-09-10T21:52:07.662 INFO:teuthology.orchestra.run.smithi133.stdout: mon mgr mkfs grace = 240 2024-09-10T21:52:07.662 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:52:07.662 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd reporter subtree level = osd 2024-09-10T21:52:07.662 INFO:teuthology.orchestra.run.smithi133.stdout: mon osd prime pg temp = true 2024-09-10T21:52:07.662 INFO:teuthology.orchestra.run.smithi133.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:52:07.662 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.662 INFO:teuthology.orchestra.run.smithi133.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T21:52:07.662 INFO:teuthology.orchestra.run.smithi133.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T21:52:07.662 INFO:teuthology.orchestra.run.smithi133.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T21:52:07.662 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.662 INFO:teuthology.orchestra.run.smithi133.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T21:52:07.662 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T21:52:07.662 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T21:52:07.662 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.663 INFO:teuthology.orchestra.run.smithi133.stdout: # 1m isn't quite enough 2024-09-10T21:52:07.663 INFO:teuthology.orchestra.run.smithi133.stdout: mon_down_mkfs_grace = 2m 2024-09-10T21:52:07.663 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.663 INFO:teuthology.orchestra.run.smithi133.stdout: mon_warn_on_filestore_osds = false 2024-09-10T21:52:07.663 INFO:teuthology.orchestra.run.smithi133.stdout: mon down mkfs grace = 300 2024-09-10T21:52:07.663 INFO:teuthology.orchestra.run.smithi133.stdout: 2024-09-10T21:52:07.663 INFO:teuthology.orchestra.run.smithi133.stdout:[client] 2024-09-10T21:52:07.663 INFO:teuthology.orchestra.run.smithi133.stdout: rgw cache enabled = true 2024-09-10T21:52:07.663 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable ops log = true 2024-09-10T21:52:07.663 INFO:teuthology.orchestra.run.smithi133.stdout: rgw enable usage log = true 2024-09-10T21:52:07.663 INFO:teuthology.orchestra.run.smithi133.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T21:52:07.663 INFO:teuthology.orchestra.run.smithi133.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T21:52:07.663 INFO:teuthology.orchestra.run.smithi133.stdout: pid file = "" 2024-09-10T21:52:07.663 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.a] 2024-09-10T21:52:07.664 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.b] 2024-09-10T21:52:07.664 INFO:teuthology.orchestra.run.smithi133.stdout:[mon.c] 2024-09-10T21:52:07.664 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.a] 2024-09-10T21:52:07.664 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.c] 2024-09-10T21:52:07.664 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.b] 2024-09-10T21:52:07.664 INFO:teuthology.orchestra.run.smithi133.stdout:[mds.d] 2024-09-10T21:52:07.664 INFO:teuthology.orchestra.run.smithi133.stdout:[mds] 2024-09-10T21:52:07.664 INFO:teuthology.orchestra.run.smithi133.stdout: mds verify scatter = True 2024-09-10T21:52:07.664 INFO:teuthology.orchestra.run.smithi133.stdout: mds debug scatterstat = True 2024-09-10T21:52:07.672 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-09-10T21:52:07.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.834+0000 7fddd0200640 1 -- 172.21.15.7:0/2232819935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc80b4990 msgr2=0x7fddc8074280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.834+0000 7fddd0200640 1 --2- 172.21.15.7:0/2232819935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc80b4990 0x7fddc8074280 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fddb4005fd0 tx=0x7fddb402a5d0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.834+0000 7fddd0200640 1 -- 172.21.15.7:0/2232819935 shutdown_connections 2024-09-10T21:52:07.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.834+0000 7fddd0200640 1 --2- 172.21.15.7:0/2232819935 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc80b4990 0x7fddc8074280 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.834+0000 7fddd0200640 1 --2- 172.21.15.7:0/2232819935 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddc80b5310 0x7fddc815b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.834+0000 7fddd0200640 1 -- 172.21.15.7:0/2232819935 >> 172.21.15.7:0/2232819935 conn(0x7fddc80b1b60 msgr2=0x7fddc80b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:07.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.834+0000 7fddd0200640 1 -- 172.21.15.7:0/2232819935 shutdown_connections 2024-09-10T21:52:07.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.834+0000 7fddd0200640 1 -- 172.21.15.7:0/2232819935 wait complete. 2024-09-10T21:52:07.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.836+0000 7fddd0200640 1 Processor -- start 2024-09-10T21:52:07.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.836+0000 7fddd0200640 1 -- start start 2024-09-10T21:52:07.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.837+0000 7fddd0200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddc80747c0 0x7fddc814c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.837+0000 7fddd0200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddc80b4990 0x7fddc8140870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.837+0000 7fddd0200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc80b5310 0x7fddc814e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.837+0000 7fddd0200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fddc806e700 con 0x7fddc80b5310 2024-09-10T21:52:07.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.837+0000 7fddd0200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fddc80b3780 con 0x7fddc80747c0 2024-09-10T21:52:07.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.837+0000 7fddd0200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fddc806e880 con 0x7fddc80b4990 2024-09-10T21:52:07.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.837+0000 7fddce200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc80b5310 0x7fddc814e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.837+0000 7fddce200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc80b5310 0x7fddc814e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38160/0 (socket says 172.21.15.7:38160) 2024-09-10T21:52:07.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.837+0000 7fddce200640 1 -- 172.21.15.7:0/737091062 learned_addr learned my addr 172.21.15.7:0/737091062 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:07.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.837+0000 7fddcce00640 1 --2- 172.21.15.7:0/737091062 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddc80b4990 0x7fddc8140870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.838+0000 7fddcd800640 1 --2- 172.21.15.7:0/737091062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddc80747c0 0x7fddc814c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.838+0000 7fddce200640 1 -- 172.21.15.7:0/737091062 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddc80b4990 msgr2=0x7fddc8140870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.838+0000 7fddce200640 1 --2- 172.21.15.7:0/737091062 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddc80b4990 0x7fddc8140870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.838+0000 7fddce200640 1 -- 172.21.15.7:0/737091062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddc80747c0 msgr2=0x7fddc814c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:07.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.838+0000 7fddce200640 1 --2- 172.21.15.7:0/737091062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddc80747c0 0x7fddc814c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:07.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.838+0000 7fddce200640 1 -- 172.21.15.7:0/737091062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc814ef60 con 0x7fddc80b5310 2024-09-10T21:52:07.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.838+0000 7fddce200640 1 --2- 172.21.15.7:0/737091062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc80b5310 0x7fddc814e710 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fddc000c730 tx=0x7fddc000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.838+0000 7fddc5600640 1 -- 172.21.15.7:0/737091062 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0016020 con 0x7fddc80b5310 2024-09-10T21:52:07.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.838+0000 7fddc5600640 1 -- 172.21.15.7:0/737091062 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddc000ce40 con 0x7fddc80b5310 2024-09-10T21:52:07.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.838+0000 7fddd0200640 1 -- 172.21.15.7:0/737091062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc814f240 con 0x7fddc80b5310 2024-09-10T21:52:07.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.839+0000 7fddc5600640 1 -- 172.21.15.7:0/737091062 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0006e60 con 0x7fddc80b5310 2024-09-10T21:52:07.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.839+0000 7fddd0200640 1 -- 172.21.15.7:0/737091062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc814f800 con 0x7fddc80b5310 2024-09-10T21:52:07.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.840+0000 7fddd0200640 1 -- 172.21.15.7:0/737091062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddc806ed40 con 0x7fddc80b5310 2024-09-10T21:52:07.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.841+0000 7fddc5600640 1 -- 172.21.15.7:0/737091062 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fddc0003bf0 con 0x7fddc80b5310 2024-09-10T21:52:07.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.842+0000 7fddc5600640 1 --2- 172.21.15.7:0/737091062 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdda8077fa0 0x7fdda807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:07.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.842+0000 7fddc5600640 1 -- 172.21.15.7:0/737091062 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x7fddc00999c0 con 0x7fddc80b5310 2024-09-10T21:52:07.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.842+0000 7fddcd800640 1 --2- 172.21.15.7:0/737091062 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdda8077fa0 0x7fdda807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:07.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.843+0000 7fddcd800640 1 --2- 172.21.15.7:0/737091062 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdda8077fa0 0x7fdda807a460 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fddc815afb0 tx=0x7fddbc005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:07.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:07.846+0000 7fddc5600640 1 -- 172.21.15.7:0/737091062 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fddc0020020 con 0x7fddc80b5310 2024-09-10T21:52:08.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:08.085+0000 7fddd0200640 1 -- 172.21.15.7:0/737091062 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) -- 0x7fddc814ec50 con 0x7fddc80b5310 2024-09-10T21:52:08.887 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:08.886+0000 7fddc5600640 1 -- 172.21.15.7:0/737091062 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v111) ==== 162+0+0 (secure 0 0 0) 0x7fddc0061d70 con 0x7fddc80b5310 2024-09-10T21:52:08.887 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked joinable; MDS may join as newly active. 2024-09-10T21:52:08.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:08.889+0000 7fddd0200640 1 -- 172.21.15.7:0/737091062 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdda8077fa0 msgr2=0x7fdda807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:08.890 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:08.889+0000 7fddd0200640 1 --2- 172.21.15.7:0/737091062 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdda8077fa0 0x7fdda807a460 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fddc815afb0 tx=0x7fddbc005f90 comp rx=0 tx=0).stop 2024-09-10T21:52:08.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:08.889+0000 7fddd0200640 1 -- 172.21.15.7:0/737091062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc80b5310 msgr2=0x7fddc814e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:08.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:08.889+0000 7fddd0200640 1 --2- 172.21.15.7:0/737091062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc80b5310 0x7fddc814e710 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fddc000c730 tx=0x7fddc000cbf0 comp rx=0 tx=0).stop 2024-09-10T21:52:08.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:08.889+0000 7fddd0200640 1 -- 172.21.15.7:0/737091062 shutdown_connections 2024-09-10T21:52:08.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:08.889+0000 7fddd0200640 1 --2- 172.21.15.7:0/737091062 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdda8077fa0 0x7fdda807a460 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:08.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:08.889+0000 7fddd0200640 1 --2- 172.21.15.7:0/737091062 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fddc80b5310 0x7fddc814e710 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:08.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:08.889+0000 7fddd0200640 1 --2- 172.21.15.7:0/737091062 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fddc80b4990 0x7fddc8140870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:08.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:08.889+0000 7fddd0200640 1 --2- 172.21.15.7:0/737091062 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fddc80747c0 0x7fddc814c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:08.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:08.889+0000 7fddd0200640 1 -- 172.21.15.7:0/737091062 >> 172.21.15.7:0/737091062 conn(0x7fddc80b1b60 msgr2=0x7fddc80b2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:08.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:08.890+0000 7fddd0200640 1 -- 172.21.15.7:0/737091062 shutdown_connections 2024-09-10T21:52:08.891 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:08.890+0000 7fddd0200640 1 -- 172.21.15.7:0/737091062 wait complete. 2024-09-10T21:52:08.914 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:52:09.079 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.078+0000 7f210fa00640 1 -- 172.21.15.7:0/536885734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210806ef20 msgr2=0x7f21080b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:09.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.078+0000 7f210fa00640 1 --2- 172.21.15.7:0/536885734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210806ef20 0x7f21080b4370 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f210806e1f0 tx=0x7f210002aa30 comp rx=0 tx=0).stop 2024-09-10T21:52:09.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.078+0000 7f210fa00640 1 -- 172.21.15.7:0/536885734 shutdown_connections 2024-09-10T21:52:09.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.078+0000 7f210fa00640 1 --2- 172.21.15.7:0/536885734 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210806ef20 0x7f21080b4370 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:09.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.078+0000 7f210fa00640 1 --2- 172.21.15.7:0/536885734 >> v2:172.21.15.133:3300/0 conn(0x7f2108074440 0x7f21080ad850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:09.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.078+0000 7f210fa00640 1 -- 172.21.15.7:0/536885734 >> 172.21.15.7:0/536885734 conn(0x7f210806ca60 msgr2=0x7f210806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:09.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.078+0000 7f210fa00640 1 -- 172.21.15.7:0/536885734 shutdown_connections 2024-09-10T21:52:09.080 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.078+0000 7f210fa00640 1 -- 172.21.15.7:0/536885734 wait complete. 2024-09-10T21:52:09.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.080+0000 7f210fa00640 1 Processor -- start 2024-09-10T21:52:09.081 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.080+0000 7f210fa00640 1 -- start start 2024-09-10T21:52:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.081+0000 7f210fa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210806ef20 0x7f2108157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.081+0000 7f210fa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2108074440 0x7f21081426e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:09.082 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.081+0000 7f210fa00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f210815ef90 0x7f21081555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.081+0000 7f210fa00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f21080b4c40 con 0x7f210806ef20 2024-09-10T21:52:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.081+0000 7f210fa00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f21080b4ac0 con 0x7f2108074440 2024-09-10T21:52:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.081+0000 7f210fa00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f21080b4dc0 con 0x7f210815ef90 2024-09-10T21:52:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.081+0000 7f210d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210806ef20 0x7f2108157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.081+0000 7f210d000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210806ef20 0x7f2108157b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38170/0 (socket says 172.21.15.7:38170) 2024-09-10T21:52:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.081+0000 7f210d000640 1 -- 172.21.15.7:0/4255265394 learned_addr learned my addr 172.21.15.7:0/4255265394 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.081+0000 7f210da00640 1 --2- 172.21.15.7:0/4255265394 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f210815ef90 0x7f21081555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.081+0000 7f20ffe00640 1 --2- 172.21.15.7:0/4255265394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2108074440 0x7f21081426e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.082+0000 7f210d000640 1 -- 172.21.15.7:0/4255265394 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f210815ef90 msgr2=0x7f21081555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.082+0000 7f210d000640 1 --2- 172.21.15.7:0/4255265394 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f210815ef90 0x7f21081555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.082+0000 7f210d000640 1 -- 172.21.15.7:0/4255265394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2108074440 msgr2=0x7f21081426e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.082+0000 7f210d000640 1 --2- 172.21.15.7:0/4255265394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2108074440 0x7f21081426e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:09.083 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.082+0000 7f210d000640 1 -- 172.21.15.7:0/4255265394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21081f90c0 con 0x7f210806ef20 2024-09-10T21:52:09.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.082+0000 7f210d000640 1 --2- 172.21.15.7:0/4255265394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210806ef20 0x7f2108157b50 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f20f800d950 tx=0x7f20f800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:09.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.082+0000 7f2105600640 1 -- 172.21.15.7:0/4255265394 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f8017070 con 0x7f210806ef20 2024-09-10T21:52:09.084 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.082+0000 7f2105600640 1 -- 172.21.15.7:0/4255265394 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20f8009d20 con 0x7f210806ef20 2024-09-10T21:52:09.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.082+0000 7f210fa00640 1 -- 172.21.15.7:0/4255265394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21081e67b0 con 0x7f210806ef20 2024-09-10T21:52:09.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.082+0000 7f2105600640 1 -- 172.21.15.7:0/4255265394 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20f80106c0 con 0x7f210806ef20 2024-09-10T21:52:09.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.082+0000 7f210fa00640 1 -- 172.21.15.7:0/4255265394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21081e6d70 con 0x7f210806ef20 2024-09-10T21:52:09.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.083+0000 7f210fa00640 1 -- 172.21.15.7:0/4255265394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21080b5270 con 0x7f210806ef20 2024-09-10T21:52:09.085 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.084+0000 7f2105600640 1 -- 172.21.15.7:0/4255265394 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f20f8010860 con 0x7f210806ef20 2024-09-10T21:52:09.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.086+0000 7f2105600640 1 --2- 172.21.15.7:0/4255265394 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f20e4077f70 0x7f20e407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:09.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.086+0000 7f2105600640 1 -- 172.21.15.7:0/4255265394 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x7f20f809a2a0 con 0x7f210806ef20 2024-09-10T21:52:09.087 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.086+0000 7f20ffe00640 1 --2- 172.21.15.7:0/4255265394 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f20e4077f70 0x7f20e407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:09.088 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.087+0000 7f20ffe00640 1 --2- 172.21.15.7:0/4255265394 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f20e4077f70 0x7f20e407a430 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f20f0006fd0 tx=0x7f20f0008040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:09.091 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.090+0000 7f2105600640 1 -- 172.21.15.7:0/4255265394 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f20f8062ca0 con 0x7f210806ef20 2024-09-10T21:52:09.322 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.321+0000 7f210fa00640 1 -- 172.21.15.7:0/4255265394 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f21080b30d0 con 0x7f210806ef20 2024-09-10T21:52:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.322+0000 7f2105600640 1 -- 172.21.15.7:0/4255265394 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) ==== 95+0+5227 (secure 0 0 0) 0x7f20f80625a0 con 0x7f210806ef20 2024-09-10T21:52:09.323 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:09.323 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":112,"btime":"2024-09-10T21:52:08:890002+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3012572644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3012572644},{"type":"v1","addr":"172.21.15.7:6834","nonce":3012572644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7660,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2492331385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2492331385},{"type":"v1","addr":"172.21.15.133:6837","nonce":2492331385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":8094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/148356862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":148356862},{"type":"v1","addr":"172.21.15.7:6835","nonce":148356862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:51:37.913965+0000","modified":"2024-09-10T21:52:08.889980+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236978},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236978":{"gid":236978,"name":"d","rank":0,"incarnation":112,"state":"up:replay","state_seq":1,"addr":"172.21.15.133:6836/4068969165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4068969165},{"type":"v1","addr":"172.21.15.133:6836","nonce":4068969165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-09-10T21:52:09.323 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 112 2024-09-10T21:52:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.324+0000 7f210fa00640 1 -- 172.21.15.7:0/4255265394 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f20e4077f70 msgr2=0x7f20e407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.324+0000 7f210fa00640 1 --2- 172.21.15.7:0/4255265394 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f20e4077f70 0x7f20e407a430 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f20f0006fd0 tx=0x7f20f0008040 comp rx=0 tx=0).stop 2024-09-10T21:52:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.324+0000 7f210fa00640 1 -- 172.21.15.7:0/4255265394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210806ef20 msgr2=0x7f2108157b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.324+0000 7f210fa00640 1 --2- 172.21.15.7:0/4255265394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210806ef20 0x7f2108157b50 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f20f800d950 tx=0x7f20f800de10 comp rx=0 tx=0).stop 2024-09-10T21:52:09.325 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.324+0000 7f210fa00640 1 -- 172.21.15.7:0/4255265394 shutdown_connections 2024-09-10T21:52:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.324+0000 7f210fa00640 1 --2- 172.21.15.7:0/4255265394 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f20e4077f70 0x7f20e407a430 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.324+0000 7f210fa00640 1 --2- 172.21.15.7:0/4255265394 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2108074440 0x7f21081426e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.324+0000 7f210fa00640 1 --2- 172.21.15.7:0/4255265394 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f210815ef90 0x7f21081555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.324+0000 7f210fa00640 1 --2- 172.21.15.7:0/4255265394 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f210806ef20 0x7f2108157b50 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.324+0000 7f210fa00640 1 -- 172.21.15.7:0/4255265394 >> 172.21.15.7:0/4255265394 conn(0x7f210806ca60 msgr2=0x7f21080b06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.325+0000 7f210fa00640 1 -- 172.21.15.7:0/4255265394 shutdown_connections 2024-09-10T21:52:09.326 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:09.325+0000 7f210fa00640 1 -- 172.21.15.7:0/4255265394 wait complete. 2024-09-10T21:52:09.743 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:51:37.913965+0000', 'modified': '2024-09-10T21:52:08.889980+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 118, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 236978}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_236978': {'gid': 236978, 'name': 'd', 'rank': 0, 'incarnation': 112, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.133:6836/4068969165', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6835', 'nonce': 4068969165}, {'type': 'v1', 'addr': '172.21.15.133:6836', 'nonce': 4068969165}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:52:09.743 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_236978:up:replay 2024-09-10T21:52:10.744 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:52:10.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.916+0000 7fd23a000640 1 -- 172.21.15.7:0/966547974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2340cddc0 msgr2=0x7fd234166bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:10.917 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.916+0000 7fd23a000640 1 --2- 172.21.15.7:0/966547974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2340cddc0 0x7fd234166bd0 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7fd22c005fd0 tx=0x7fd22c02a940 comp rx=0 tx=0).stop 2024-09-10T21:52:10.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.916+0000 7fd23a000640 1 -- 172.21.15.7:0/966547974 shutdown_connections 2024-09-10T21:52:10.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.916+0000 7fd23a000640 1 --2- 172.21.15.7:0/966547974 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd2340cddc0 0x7fd234166bd0 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:10.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.916+0000 7fd23a000640 1 -- 172.21.15.7:0/966547974 >> 172.21.15.7:0/966547974 conn(0x7fd23406ca60 msgr2=0x7fd23406ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:10.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.916+0000 7fd23a000640 1 -- 172.21.15.7:0/966547974 shutdown_connections 2024-09-10T21:52:10.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.916+0000 7fd23a000640 1 -- 172.21.15.7:0/966547974 wait complete. 2024-09-10T21:52:10.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.918+0000 7fd23a000640 1 Processor -- start 2024-09-10T21:52:10.918 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.918+0000 7fd23a000640 1 -- start start 2024-09-10T21:52:10.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.919+0000 7fd23a000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2340cddc0 0x7fd234150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:10.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.919+0000 7fd23a000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd234159910 0x7fd2341663c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:10.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.919+0000 7fd23a000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23415d090 0x7fd234148420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:10.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.919+0000 7fd23a000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd234167110 con 0x7fd23415d090 2024-09-10T21:52:10.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.919+0000 7fd23a000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd234167290 con 0x7fd234159910 2024-09-10T21:52:10.921 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.919+0000 7fd23a000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fd234167410 con 0x7fd2340cddc0 2024-09-10T21:52:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.919+0000 7fd233e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23415d090 0x7fd234148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.919+0000 7fd233e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23415d090 0x7fd234148420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38186/0 (socket says 172.21.15.7:38186) 2024-09-10T21:52:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.919+0000 7fd233e00640 1 -- 172.21.15.7:0/3196223211 learned_addr learned my addr 172.21.15.7:0/3196223211 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.919+0000 7fd233400640 1 --2- 172.21.15.7:0/3196223211 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2340cddc0 0x7fd234150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.919+0000 7fd232a00640 1 --2- 172.21.15.7:0/3196223211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd234159910 0x7fd2341663c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.920+0000 7fd233e00640 1 -- 172.21.15.7:0/3196223211 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2340cddc0 msgr2=0x7fd234150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.920+0000 7fd233e00640 1 --2- 172.21.15.7:0/3196223211 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2340cddc0 0x7fd234150430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.920+0000 7fd233e00640 1 -- 172.21.15.7:0/3196223211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd234159910 msgr2=0x7fd2341663c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.920+0000 7fd233e00640 1 --2- 172.21.15.7:0/3196223211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd234159910 0x7fd2341663c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.920+0000 7fd233e00640 1 -- 172.21.15.7:0/3196223211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd234165fd0 con 0x7fd23415d090 2024-09-10T21:52:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.920+0000 7fd233400640 1 --2- 172.21.15.7:0/3196223211 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2340cddc0 0x7fd234150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:52:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.920+0000 7fd232a00640 1 --2- 172.21.15.7:0/3196223211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd234159910 0x7fd2341663c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:52:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.920+0000 7fd233e00640 1 --2- 172.21.15.7:0/3196223211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23415d090 0x7fd234148420 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fd22c006010 tx=0x7fd22c031e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.920+0000 7fd22b400640 1 -- 172.21.15.7:0/3196223211 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c004240 con 0x7fd23415d090 2024-09-10T21:52:10.922 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.920+0000 7fd22b400640 1 -- 172.21.15.7:0/3196223211 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd22c0043e0 con 0x7fd23415d090 2024-09-10T21:52:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.920+0000 7fd22b400640 1 -- 172.21.15.7:0/3196223211 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd22c004830 con 0x7fd23415d090 2024-09-10T21:52:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.920+0000 7fd23a000640 1 -- 172.21.15.7:0/3196223211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd234166900 con 0x7fd23415d090 2024-09-10T21:52:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.920+0000 7fd23a000640 1 -- 172.21.15.7:0/3196223211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2341654f0 con 0x7fd23415d090 2024-09-10T21:52:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.922+0000 7fd23a000640 1 -- 172.21.15.7:0/3196223211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd23406e810 con 0x7fd23415d090 2024-09-10T21:52:10.923 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.923+0000 7fd22b400640 1 -- 172.21.15.7:0/3196223211 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd22c02f070 con 0x7fd23415d090 2024-09-10T21:52:10.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.924+0000 7fd22b400640 1 --2- 172.21.15.7:0/3196223211 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd214077fa0 0x7fd21407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:10.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.924+0000 7fd22b400640 1 -- 172.21.15.7:0/3196223211 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x7fd22c0bb640 con 0x7fd23415d090 2024-09-10T21:52:10.925 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.924+0000 7fd233400640 1 --2- 172.21.15.7:0/3196223211 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd214077fa0 0x7fd21407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:10.926 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.925+0000 7fd233400640 1 --2- 172.21.15.7:0/3196223211 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd214077fa0 0x7fd21407a460 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd224001740 tx=0x7fd224009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:10.929 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:10.928+0000 7fd22b400640 1 -- 172.21.15.7:0/3196223211 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd22c041e40 con 0x7fd23415d090 2024-09-10T21:52:11.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:11.160+0000 7fd23a000640 1 -- 172.21.15.7:0/3196223211 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd234155820 con 0x7fd23415d090 2024-09-10T21:52:11.161 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:11.160+0000 7fd22b400640 1 -- 172.21.15.7:0/3196223211 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 114 v114) ==== 95+0+5227 (secure 0 0 0) 0x7fd22c084040 con 0x7fd23415d090 2024-09-10T21:52:11.161 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:11.162 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":114,"btime":"2024-09-10T21:52:10:893672+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3012572644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3012572644},{"type":"v1","addr":"172.21.15.7:6834","nonce":3012572644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7660,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2492331385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2492331385},{"type":"v1","addr":"172.21.15.133:6837","nonce":2492331385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":8094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/148356862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":148356862},{"type":"v1","addr":"172.21.15.7:6835","nonce":148356862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":114,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:51:37.913965+0000","modified":"2024-09-10T21:52:09.897719+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236978},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236978":{"gid":236978,"name":"d","rank":0,"incarnation":112,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.133:6836/4068969165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4068969165},{"type":"v1","addr":"172.21.15.133:6836","nonce":4068969165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":8}]} 2024-09-10T21:52:11.162 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 114 2024-09-10T21:52:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:11.163+0000 7fd23a000640 1 -- 172.21.15.7:0/3196223211 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd214077fa0 msgr2=0x7fd21407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:11.163+0000 7fd23a000640 1 --2- 172.21.15.7:0/3196223211 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd214077fa0 0x7fd21407a460 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fd224001740 tx=0x7fd224009040 comp rx=0 tx=0).stop 2024-09-10T21:52:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:11.163+0000 7fd23a000640 1 -- 172.21.15.7:0/3196223211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23415d090 msgr2=0x7fd234148420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:11.163+0000 7fd23a000640 1 --2- 172.21.15.7:0/3196223211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23415d090 0x7fd234148420 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7fd22c006010 tx=0x7fd22c031e20 comp rx=0 tx=0).stop 2024-09-10T21:52:11.164 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:11.163+0000 7fd23a000640 1 -- 172.21.15.7:0/3196223211 shutdown_connections 2024-09-10T21:52:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:11.163+0000 7fd23a000640 1 --2- 172.21.15.7:0/3196223211 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd214077fa0 0x7fd21407a460 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:11.163+0000 7fd23a000640 1 --2- 172.21.15.7:0/3196223211 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd234159910 0x7fd2341663c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:11.163+0000 7fd23a000640 1 --2- 172.21.15.7:0/3196223211 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd23415d090 0x7fd234148420 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:11.163+0000 7fd23a000640 1 --2- 172.21.15.7:0/3196223211 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd2340cddc0 0x7fd234150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:11.163+0000 7fd23a000640 1 -- 172.21.15.7:0/3196223211 >> 172.21.15.7:0/3196223211 conn(0x7fd23406ca60 msgr2=0x7fd23415e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:11.163+0000 7fd23a000640 1 -- 172.21.15.7:0/3196223211 shutdown_connections 2024-09-10T21:52:11.165 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:11.163+0000 7fd23a000640 1 -- 172.21.15.7:0/3196223211 wait complete. 2024-09-10T21:52:11.583 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 114, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:51:37.913965+0000', 'modified': '2024-09-10T21:52:09.897719+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 118, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 236978}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_236978': {'gid': 236978, 'name': 'd', 'rank': 0, 'incarnation': 112, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.133:6836/4068969165', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6835', 'nonce': 4068969165}, {'type': 'v1', 'addr': '172.21.15.133:6836', 'nonce': 4068969165}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:52:11.583 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_236978:up:rejoin 2024-09-10T21:52:12.584 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:52:12.756 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.755+0000 7f478c600640 1 -- 172.21.15.7:0/3943796361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4784159910 msgr2=0x7f478415d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.755+0000 7f478c600640 1 --2- 172.21.15.7:0/3943796361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4784159910 0x7f478415d090 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f4778005fd0 tx=0x7f477802a5d0 comp rx=0 tx=0).stop 2024-09-10T21:52:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.755+0000 7f478c600640 1 -- 172.21.15.7:0/3943796361 shutdown_connections 2024-09-10T21:52:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.755+0000 7f478c600640 1 --2- 172.21.15.7:0/3943796361 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4784159910 0x7f478415d090 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.755+0000 7f478c600640 1 -- 172.21.15.7:0/3943796361 >> 172.21.15.7:0/3943796361 conn(0x7f478406ca60 msgr2=0x7f478406ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.755+0000 7f478c600640 1 -- 172.21.15.7:0/3943796361 shutdown_connections 2024-09-10T21:52:12.757 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.756+0000 7f478c600640 1 -- 172.21.15.7:0/3943796361 wait complete. 2024-09-10T21:52:12.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.757+0000 7f478c600640 1 Processor -- start 2024-09-10T21:52:12.758 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.757+0000 7f478c600640 1 -- start start 2024-09-10T21:52:12.759 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.758+0000 7f478c600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f478406e810 0x7f47841f4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.758+0000 7f478c600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47840cddc0 0x7f47840b2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.758+0000 7f478c600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4784159910 0x7f47840ae010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.758+0000 7f478c600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f47840739d0 con 0x7f4784159910 2024-09-10T21:52:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.758+0000 7f478c600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4784073850 con 0x7f478406e810 2024-09-10T21:52:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.758+0000 7f478c600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4784073b50 con 0x7f47840cddc0 2024-09-10T21:52:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.758+0000 7f478a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4784159910 0x7f47840ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.758+0000 7f478a600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4784159910 0x7f47840ae010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38194/0 (socket says 172.21.15.7:38194) 2024-09-10T21:52:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.758+0000 7f478a600640 1 -- 172.21.15.7:0/1580048291 learned_addr learned my addr 172.21.15.7:0/1580048291 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.759+0000 7f4789c00640 1 --2- 172.21.15.7:0/1580048291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f478406e810 0x7f47841f4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.759+0000 7f4789200640 1 --2- 172.21.15.7:0/1580048291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47840cddc0 0x7f47840b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.759+0000 7f478a600640 1 -- 172.21.15.7:0/1580048291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47840cddc0 msgr2=0x7f47840b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:12.760 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.759+0000 7f478a600640 1 --2- 172.21.15.7:0/1580048291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47840cddc0 0x7f47840b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:12.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.759+0000 7f478a600640 1 -- 172.21.15.7:0/1580048291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f478406e810 msgr2=0x7f47841f4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:12.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.759+0000 7f478a600640 1 --2- 172.21.15.7:0/1580048291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f478406e810 0x7f47841f4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:12.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.759+0000 7f478a600640 1 -- 172.21.15.7:0/1580048291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47840b0ec0 con 0x7f4784159910 2024-09-10T21:52:12.761 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.759+0000 7f4789200640 1 --2- 172.21.15.7:0/1580048291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47840cddc0 0x7f47840b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.759+0000 7f478a600640 1 --2- 172.21.15.7:0/1580048291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4784159910 0x7f47840ae010 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f477c00c700 tx=0x7f477c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.759+0000 7f4781600640 1 -- 172.21.15.7:0/1580048291 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c003b70 con 0x7f4784159910 2024-09-10T21:52:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.759+0000 7f4781600640 1 -- 172.21.15.7:0/1580048291 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f477c003d30 con 0x7f4784159910 2024-09-10T21:52:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.760+0000 7f478c600640 1 -- 172.21.15.7:0/1580048291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47840b11a0 con 0x7f4784159910 2024-09-10T21:52:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.760+0000 7f4781600640 1 -- 172.21.15.7:0/1580048291 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c00aab0 con 0x7f4784159910 2024-09-10T21:52:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.760+0000 7f478c600640 1 -- 172.21.15.7:0/1580048291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47840b1760 con 0x7f4784159910 2024-09-10T21:52:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.761+0000 7f478c600640 1 -- 172.21.15.7:0/1580048291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47840b5480 con 0x7f4784159910 2024-09-10T21:52:12.762 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.762+0000 7f4781600640 1 -- 172.21.15.7:0/1580048291 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f477c00ac50 con 0x7f4784159910 2024-09-10T21:52:12.763 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.763+0000 7f4781600640 1 --2- 172.21.15.7:0/1580048291 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4764077f70 0x7f476407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:12.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.763+0000 7f4781600640 1 -- 172.21.15.7:0/1580048291 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x7f477c099bb0 con 0x7f4784159910 2024-09-10T21:52:12.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.763+0000 7f4789c00640 1 --2- 172.21.15.7:0/1580048291 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4764077f70 0x7f476407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:12.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.764+0000 7f4789c00640 1 --2- 172.21.15.7:0/1580048291 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4764077f70 0x7f476407a430 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4778033000 tx=0x7f4778004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:12.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.767+0000 7f4781600640 1 -- 172.21.15.7:0/1580048291 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f47840b5480 con 0x7f4784159910 2024-09-10T21:52:12.891 INFO:tasks.ceph.mds.d.smithi133.stderr:2024-09-10T21:52:12.891+0000 7f7b76400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:52:13.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:12.999+0000 7f478c600640 1 -- 172.21.15.7:0/1580048291 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f47840740d0 con 0x7f4784159910 2024-09-10T21:52:13.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.000+0000 7f4781600640 1 -- 172.21.15.7:0/1580048291 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+5238 (secure 0 0 0) 0x7f477c0625b0 con 0x7f4784159910 2024-09-10T21:52:13.001 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:13.001 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":115,"btime":"2024-09-10T21:52:11:895606+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3012572644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3012572644},{"type":"v1","addr":"172.21.15.7:6834","nonce":3012572644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7660,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2492331385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2492331385},{"type":"v1","addr":"172.21.15.133:6837","nonce":2492331385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":8094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/148356862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":148356862},{"type":"v1","addr":"172.21.15.7:6835","nonce":148356862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:51:37.913965+0000","modified":"2024-09-10T21:52:11.895593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236978},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236978":{"gid":236978,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6836/4068969165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4068969165},{"type":"v1","addr":"172.21.15.133:6836","nonce":4068969165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":236978,"qdb_cluster":[236978]},"id":8}]} 2024-09-10T21:52:13.001 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 115 2024-09-10T21:52:13.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.002+0000 7f478c600640 1 -- 172.21.15.7:0/1580048291 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4764077f70 msgr2=0x7f476407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:13.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.002+0000 7f478c600640 1 --2- 172.21.15.7:0/1580048291 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4764077f70 0x7f476407a430 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f4778033000 tx=0x7f4778004730 comp rx=0 tx=0).stop 2024-09-10T21:52:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.002+0000 7f478c600640 1 -- 172.21.15.7:0/1580048291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4784159910 msgr2=0x7f47840ae010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.002+0000 7f478c600640 1 --2- 172.21.15.7:0/1580048291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4784159910 0x7f47840ae010 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f477c00c700 tx=0x7f477c00cbc0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.002+0000 7f478c600640 1 -- 172.21.15.7:0/1580048291 shutdown_connections 2024-09-10T21:52:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.002+0000 7f478c600640 1 --2- 172.21.15.7:0/1580048291 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4764077f70 0x7f476407a430 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.002+0000 7f478c600640 1 --2- 172.21.15.7:0/1580048291 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4784159910 0x7f47840ae010 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.002+0000 7f478c600640 1 --2- 172.21.15.7:0/1580048291 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f47840cddc0 0x7f47840b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.002+0000 7f478c600640 1 --2- 172.21.15.7:0/1580048291 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f478406e810 0x7f47841f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.002+0000 7f478c600640 1 -- 172.21.15.7:0/1580048291 >> 172.21.15.7:0/1580048291 conn(0x7f478406ca60 msgr2=0x7f478415f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.002+0000 7f478c600640 1 -- 172.21.15.7:0/1580048291 shutdown_connections 2024-09-10T21:52:13.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.002+0000 7f478c600640 1 -- 172.21.15.7:0/1580048291 wait complete. 2024-09-10T21:52:13.424 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 115, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:51:37.913965+0000', 'modified': '2024-09-10T21:52:11.895593+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 118, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 236978}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_236978': {'gid': 236978, 'name': 'd', 'rank': 0, 'incarnation': 112, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.133:6836/4068969165', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.133:6835', 'nonce': 4068969165}, {'type': 'v1', 'addr': '172.21.15.133:6836', 'nonce': 4068969165}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 236978, 'qdb_cluster': [236978]} 2024-09-10T21:52:13.424 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:52:13.425 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T21:52:13.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.587+0000 7f54faa00640 1 -- 172.21.15.7:0/1724277904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f4159910 msgr2=0x7f54f415d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:13.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.587+0000 7f54faa00640 1 --2- 172.21.15.7:0/1724277904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f4159910 0x7f54f415d090 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f54f406f230 tx=0x7f54e402a540 comp rx=0 tx=0).stop 2024-09-10T21:52:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.587+0000 7f54faa00640 1 -- 172.21.15.7:0/1724277904 shutdown_connections 2024-09-10T21:52:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.587+0000 7f54faa00640 1 --2- 172.21.15.7:0/1724277904 >> v2:172.21.15.133:3300/0 conn(0x7f54f40cdbf0 0x7f54f415fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.587+0000 7f54faa00640 1 --2- 172.21.15.7:0/1724277904 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54f406e810 0x7f54f40cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.587+0000 7f54faa00640 1 --2- 172.21.15.7:0/1724277904 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f4159910 0x7f54f415d090 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.587+0000 7f54faa00640 1 -- 172.21.15.7:0/1724277904 >> 172.21.15.7:0/1724277904 conn(0x7f54f406ca60 msgr2=0x7f54f406ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.587+0000 7f54faa00640 1 -- 172.21.15.7:0/1724277904 shutdown_connections 2024-09-10T21:52:13.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.587+0000 7f54faa00640 1 -- 172.21.15.7:0/1724277904 wait complete. 2024-09-10T21:52:13.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.589+0000 7f54faa00640 1 Processor -- start 2024-09-10T21:52:13.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.589+0000 7f54faa00640 1 -- start start 2024-09-10T21:52:13.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.590+0000 7f54faa00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54f406e810 0x7f54f41f7950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.590+0000 7f54faa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f40cdbf0 0x7f54f40b20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.590+0000 7f54faa00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54f4159910 0x7f54f40b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.590+0000 7f54faa00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54f4160130 con 0x7f54f40cdbf0 2024-09-10T21:52:13.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.590+0000 7f54faa00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f54f415ffb0 con 0x7f54f406e810 2024-09-10T21:52:13.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.590+0000 7f54faa00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f54f41602b0 con 0x7f54f4159910 2024-09-10T21:52:13.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.590+0000 7f54eb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f40cdbf0 0x7f54f40b20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.590+0000 7f54eb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f40cdbf0 0x7f54f40b20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38208/0 (socket says 172.21.15.7:38208) 2024-09-10T21:52:13.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.590+0000 7f54eb400640 1 -- 172.21.15.7:0/1395098476 learned_addr learned my addr 172.21.15.7:0/1395098476 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:13.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.590+0000 7f54eb400640 1 -- 172.21.15.7:0/1395098476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54f4159910 msgr2=0x7f54f40b25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:13.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.590+0000 7f54eb400640 1 --2- 172.21.15.7:0/1395098476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54f4159910 0x7f54f40b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.590+0000 7f54eb400640 1 -- 172.21.15.7:0/1395098476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54f406e810 msgr2=0x7f54f41f7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:13.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.590+0000 7f54eb400640 1 --2- 172.21.15.7:0/1395098476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54f406e810 0x7f54f41f7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.590+0000 7f54eb400640 1 -- 172.21.15.7:0/1395098476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54f4152960 con 0x7f54f40cdbf0 2024-09-10T21:52:13.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.591+0000 7f54eb400640 1 --2- 172.21.15.7:0/1395098476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f40cdbf0 0x7f54f40b20a0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00b450 tx=0x7f54dc00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:13.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.591+0000 7f54f0c00640 1 -- 172.21.15.7:0/1395098476 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc015020 con 0x7f54f40cdbf0 2024-09-10T21:52:13.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.591+0000 7f54f0c00640 1 -- 172.21.15.7:0/1395098476 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54dc011980 con 0x7f54f40cdbf0 2024-09-10T21:52:13.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.591+0000 7f54faa00640 1 -- 172.21.15.7:0/1395098476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54f40afdc0 con 0x7f54f40cdbf0 2024-09-10T21:52:13.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.591+0000 7f54f0c00640 1 -- 172.21.15.7:0/1395098476 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc011dd0 con 0x7f54f40cdbf0 2024-09-10T21:52:13.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.591+0000 7f54faa00640 1 -- 172.21.15.7:0/1395098476 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54f40b0380 con 0x7f54f40cdbf0 2024-09-10T21:52:13.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.593+0000 7f54f0c00640 1 -- 172.21.15.7:0/1395098476 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f54dc01a400 con 0x7f54f40cdbf0 2024-09-10T21:52:13.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.594+0000 7f54f0c00640 1 --2- 172.21.15.7:0/1395098476 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f54d0077f70 0x7f54d007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.594+0000 7f54f0c00640 1 -- 172.21.15.7:0/1395098476 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x7f54dc09a420 con 0x7f54f40cdbf0 2024-09-10T21:52:13.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.595+0000 7f54fd8a5740 1 Processor -- start 2024-09-10T21:52:13.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.595+0000 7f54f3e00640 1 --2- 172.21.15.7:0/1395098476 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f54d0077f70 0x7f54d007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.595+0000 7f54fd8a5740 1 -- start start 2024-09-10T21:52:13.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.595+0000 7f54f3e00640 1 --2- 172.21.15.7:0/1395098476 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f54d0077f70 0x7f54d007a430 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f54e4033000 tx=0x7f54e40046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:13.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.596+0000 7f54fd8a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 0x55c9d8bcec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.596+0000 7f54fd8a5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8b89a20 0x55c9d8bcd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.596+0000 7f54fd8a5740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8b89fa0 0x55c9d8bcd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.596+0000 7f54fd8a5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8b0c430 con 0x55c9d8bce8b0 2024-09-10T21:52:13.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.596+0000 7f54fd8a5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8afc5d0 con 0x55c9d8b89a20 2024-09-10T21:52:13.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.596+0000 7f54fd8a5740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55c9d8aeba00 con 0x55c9d8b89fa0 2024-09-10T21:52:13.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.596+0000 7f54f3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 0x55c9d8bcec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.596+0000 7f54f3e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 0x55c9d8bcec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38212/0 (socket says 172.21.15.7:38212) 2024-09-10T21:52:13.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.596+0000 7f54f3e00640 1 -- 172.21.15.7:0/2315829299 learned_addr learned my addr 172.21.15.7:0/2315829299 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:13.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.596+0000 7f54eb400640 1 --2- 172.21.15.7:0/2315829299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8b89a20 0x55c9d8bcd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.596+0000 7f54f8a00640 1 --2- 172.21.15.7:0/2315829299 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8b89fa0 0x55c9d8bcd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.596+0000 7f54f3e00640 1 -- 172.21.15.7:0/2315829299 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8b89fa0 msgr2=0x55c9d8bcd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.596+0000 7f54f3e00640 1 --2- 172.21.15.7:0/2315829299 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8b89fa0 0x55c9d8bcd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.596+0000 7f54f3e00640 1 -- 172.21.15.7:0/2315829299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8b89a20 msgr2=0x55c9d8bcd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.596+0000 7f54f3e00640 1 --2- 172.21.15.7:0/2315829299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8b89a20 0x55c9d8bcd390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.596+0000 7f54f3e00640 1 -- 172.21.15.7:0/2315829299 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9d8ab4520 con 0x55c9d8bce8b0 2024-09-10T21:52:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.597+0000 7f54f3e00640 1 --2- 172.21.15.7:0/2315829299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 0x55c9d8bcec90 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f54e40096c0 tx=0x7f54e4039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a714c95bef25924 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.597+0000 7f54e3e00640 1 -- 172.21.15.7:0/2315829299 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e4049070 con 0x55c9d8bce8b0 2024-09-10T21:52:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.597+0000 7f54e3e00640 1 -- 172.21.15.7:0/2315829299 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54e403f070 con 0x55c9d8bce8b0 2024-09-10T21:52:13.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.597+0000 7f54e3e00640 1 -- 172.21.15.7:0/2315829299 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54e403a6a0 con 0x55c9d8bce8b0 2024-09-10T21:52:13.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.598+0000 7f54fd8a5740 1 -- 172.21.15.7:0/2315829299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 msgr2=0x55c9d8bcec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:13.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.598+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/2315829299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 0x55c9d8bcec90 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f54e40096c0 tx=0x7f54e4039000 comp rx=0 tx=0).stop 2024-09-10T21:52:13.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.598+0000 7f54fd8a5740 1 -- 172.21.15.7:0/2315829299 shutdown_connections 2024-09-10T21:52:13.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.598+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/2315829299 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8b89a20 0x55c9d8bcd390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.598+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/2315829299 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8b89fa0 0x55c9d8bcd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.598+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/2315829299 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 0x55c9d8bcec90 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.598+0000 7f54fd8a5740 1 -- 172.21.15.7:0/2315829299 >> 172.21.15.7:0/2315829299 conn(0x55c9d8b84ff0 msgr2=0x55c9d8b88a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:13.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.598+0000 7f54fd8a5740 1 -- 172.21.15.7:0/2315829299 shutdown_connections 2024-09-10T21:52:13.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.598+0000 7f54fd8a5740 1 -- 172.21.15.7:0/2315829299 wait complete. 2024-09-10T21:52:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.599+0000 7f54fd8a5740 1 -- start start 2024-09-10T21:52:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.599+0000 7f54fd8a5740 1 Processor -- start 2024-09-10T21:52:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.600+0000 7f54fd8a5740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8b89fa0 0x55c9d8bb30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.600+0000 7f54fd8a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 0x55c9d8bad060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.600+0000 7f54fd8a5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8bafec0 0x55c9d8bc9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.600+0000 7f54fd8a5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8a92330 con 0x55c9d8bce8b0 2024-09-10T21:52:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.600+0000 7f54fd8a5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8aef810 con 0x55c9d8bafec0 2024-09-10T21:52:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.600+0000 7f54fd8a5740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55c9d885c620 con 0x55c9d8b89fa0 2024-09-10T21:52:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.600+0000 7f54fd8a5740 10 client.0 _ll_register_callbacks cb 0x55c9d8a97420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:52:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.600+0000 7f54eb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 0x55c9d8bad060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.600+0000 7f54eb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 0x55c9d8bad060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38222/0 (socket says 172.21.15.7:38222) 2024-09-10T21:52:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.600+0000 7f54eb400640 1 -- 172.21.15.7:0/661284088 learned_addr learned my addr 172.21.15.7:0/661284088 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.600+0000 7f54f3e00640 1 --2- 172.21.15.7:0/661284088 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8b89fa0 0x55c9d8bb30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.605 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.601+0000 7f54f8a00640 1 --2- 172.21.15.7:0/661284088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8bafec0 0x55c9d8bc9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.601+0000 7f54eb400640 1 -- 172.21.15.7:0/661284088 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8b89fa0 msgr2=0x55c9d8bb30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.601+0000 7f54eb400640 1 --2- 172.21.15.7:0/661284088 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8b89fa0 0x55c9d8bb30b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.601+0000 7f54eb400640 1 -- 172.21.15.7:0/661284088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8bafec0 msgr2=0x55c9d8bc9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.601+0000 7f54eb400640 1 --2- 172.21.15.7:0/661284088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8bafec0 0x55c9d8bc9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.601+0000 7f54eb400640 1 -- 172.21.15.7:0/661284088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9d884d850 con 0x55c9d8bce8b0 2024-09-10T21:52:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.601+0000 7f54f8a00640 1 --2- 172.21.15.7:0/661284088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8bafec0 0x55c9d8bc9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.601+0000 7f54f3e00640 1 --2- 172.21.15.7:0/661284088 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8b89fa0 0x55c9d8bb30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.601+0000 7f54eb400640 1 --2- 172.21.15.7:0/661284088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 0x55c9d8bad060 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009fd0 tx=0x7f54dc09a350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d01563571972c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.601+0000 7f54e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:52:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.601+0000 7f54e0c00640 1 -- 172.21.15.7:0/661284088 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc06bb90 con 0x55c9d8bce8b0 2024-09-10T21:52:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.601+0000 7f54fd8a5740 1 -- 172.21.15.7:0/661284088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54e403f070 con 0x55c9d8bce8b0 2024-09-10T21:52:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.601+0000 7f54e0c00640 1 -- 172.21.15.7:0/661284088 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x7f54dc0618f0 con 0x55c9d8bce8b0 2024-09-10T21:52:13.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.601+0000 7f54e0c00640 1 -- 172.21.15.7:0/661284088 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f54dc061af0 con 0x55c9d8bce8b0 2024-09-10T21:52:13.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.601+0000 7f54fd8a5740 10 client.8550 fetch_fsmap learned FSMap version 115 2024-09-10T21:52:13.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.601+0000 7f54fd8a5740 1 -- 172.21.15.7:0/661284088 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) -- 0x55c9d88c46b0 con 0x55c9d8bce8b0 2024-09-10T21:52:13.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.602+0000 7f54e0c00640 1 -- 172.21.15.7:0/661284088 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54e403f070 con 0x55c9d8bce8b0 2024-09-10T21:52:13.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.602+0000 7f54e0c00640 1 -- 172.21.15.7:0/661284088 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 115) ==== 3502+0+0 (secure 0 0 0) 0x7f54dc0981a0 con 0x55c9d8bce8b0 2024-09-10T21:52:13.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.602+0000 7f54fd8a5740 10 client.8550 fetch_fsmap finished waiting for FSMap version 115 2024-09-10T21:52:13.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.602+0000 7f54fd8a5740 10 client.8550 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:52:13.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.602+0000 7f54fd8a5740 20 client.8550 populate_metadata read hostname 'smithi007' 2024-09-10T21:52:13.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.602+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/661284088 >> [v2:172.21.15.133:6835/4068969165,v1:172.21.15.133:6836/4068969165] conn(0x55c9d8be12f0 0x55c9d8be16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.602+0000 7f54fd8a5740 4 client.8550 mds_command: new command op to 236978 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:52:13.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.602+0000 7f54fd8a5740 1 -- 172.21.15.7:0/661284088 --> [v2:172.21.15.133:6835/4068969165,v1:172.21.15.133:6836/4068969165] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9d8afe520 con 0x55c9d8be12f0 2024-09-10T21:52:13.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.602+0000 7f54f3e00640 1 --2- 172.21.15.7:0/661284088 >> [v2:172.21.15.133:6835/4068969165,v1:172.21.15.133:6836/4068969165] conn(0x55c9d8be12f0 0x55c9d8be16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.603+0000 7f54f3e00640 1 --2- 172.21.15.7:0/661284088 >> [v2:172.21.15.133:6835/4068969165,v1:172.21.15.133:6836/4068969165] conn(0x55c9d8be12f0 0x55c9d8be16d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d1715d818db2e1c server_cookie=5cc2273a2a33ac28 in_seq=0 out_seq=0 2024-09-10T21:52:13.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.603+0000 7f54e0c00640 10 client.8550 ms_handle_connect on v2:172.21.15.133:6835/4068969165 2024-09-10T21:52:13.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.604+0000 7f54e0c00640 1 -- 172.21.15.7:0/661284088 <== mds.0 v2:172.21.15.133:6835/4068969165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c9d8afe520 con 0x55c9d8be12f0 2024-09-10T21:52:13.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.604+0000 7f54e0c00640 10 client.8550 handle_command_reply: tid=0 2024-09-10T21:52:13.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.604+0000 7f54fd8a5740 1 client.8550 shutdown 2024-09-10T21:52:13.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.604+0000 7f54fd8a5740 1 -- 172.21.15.7:0/661284088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 msgr2=0x55c9d8bad060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:13.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.604+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/661284088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 0x55c9d8bad060 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f54dc009fd0 tx=0x7f54dc09a350 comp rx=0 tx=0).stop 2024-09-10T21:52:13.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.604+0000 7f54fd8a5740 1 -- 172.21.15.7:0/661284088 shutdown_connections 2024-09-10T21:52:13.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.604+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/661284088 >> [v2:172.21.15.133:6835/4068969165,v1:172.21.15.133:6836/4068969165] conn(0x55c9d8be12f0 0x55c9d8be16d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.604+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/661284088 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8bafec0 0x55c9d8bc9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.604+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/661284088 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 0x55c9d8bad060 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.604+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/661284088 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8b89fa0 0x55c9d8bb30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.604+0000 7f54fd8a5740 1 -- 172.21.15.7:0/661284088 >> 172.21.15.7:0/661284088 conn(0x55c9d8b89a20 msgr2=0x55c9d8baf180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:13.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.604+0000 7f54e0c00640 0 client.8550 ms_handle_reset on v2:172.21.15.133:6835/4068969165 2024-09-10T21:52:13.609 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.605+0000 7f54fd8a5740 1 -- 172.21.15.7:0/661284088 shutdown_connections 2024-09-10T21:52:13.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.605+0000 7f54fd8a5740 1 -- 172.21.15.7:0/661284088 wait complete. 2024-09-10T21:52:13.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.605+0000 7f54fd8a5740 20 client.8550 trim_cache size 0 max 16384 2024-09-10T21:52:13.624 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.624+0000 7f54fd8a5740 1 Processor -- start 2024-09-10T21:52:13.625 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.624+0000 7f54fd8a5740 1 -- start start 2024-09-10T21:52:13.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.625+0000 7f54fd8a5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8b89fa0 0x55c9d8b83b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.625+0000 7f54fd8a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bafec0 0x55c9d8b84070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.625+0000 7f54fd8a5740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8bce8b0 0x55c9d8bb2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.625+0000 7f54fd8a5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f54f42006f0 con 0x55c9d8bafec0 2024-09-10T21:52:13.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.625+0000 7f54fd8a5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f54f41540c0 con 0x55c9d8b89fa0 2024-09-10T21:52:13.626 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.625+0000 7f54fd8a5740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f54f40b15b0 con 0x55c9d8bce8b0 2024-09-10T21:52:13.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.625+0000 7f54eb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bafec0 0x55c9d8b84070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.625+0000 7f54eb400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bafec0 0x55c9d8b84070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38228/0 (socket says 172.21.15.7:38228) 2024-09-10T21:52:13.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.625+0000 7f54eb400640 1 -- 172.21.15.7:0/1235320230 learned_addr learned my addr 172.21.15.7:0/1235320230 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:13.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.625+0000 7f54f8a00640 1 --2- 172.21.15.7:0/1235320230 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8bce8b0 0x55c9d8bb2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.625+0000 7f54f3e00640 1 --2- 172.21.15.7:0/1235320230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8b89fa0 0x55c9d8b83b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.625+0000 7f54eb400640 1 -- 172.21.15.7:0/1235320230 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8bce8b0 msgr2=0x55c9d8bb2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:13.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.625+0000 7f54eb400640 1 --2- 172.21.15.7:0/1235320230 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8bce8b0 0x55c9d8bb2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.625+0000 7f54eb400640 1 -- 172.21.15.7:0/1235320230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8b89fa0 msgr2=0x55c9d8b83b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:13.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.625+0000 7f54eb400640 1 --2- 172.21.15.7:0/1235320230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8b89fa0 0x55c9d8b83b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.625+0000 7f54eb400640 1 -- 172.21.15.7:0/1235320230 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9d8bb1db0 con 0x55c9d8bafec0 2024-09-10T21:52:13.627 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.626+0000 7f54eb400640 1 --2- 172.21.15.7:0/1235320230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bafec0 0x55c9d8b84070 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f54dc021040 tx=0x7f54dc062590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ddb523a7cb5204 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:13.628 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.626+0000 7f54e3400640 1 -- 172.21.15.7:0/1235320230 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc082070 con 0x55c9d8bafec0 2024-09-10T21:52:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.626+0000 7f54e3400640 1 -- 172.21.15.7:0/1235320230 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54dc079030 con 0x55c9d8bafec0 2024-09-10T21:52:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.626+0000 7f54e3400640 1 -- 172.21.15.7:0/1235320230 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54dc06bb90 con 0x55c9d8bafec0 2024-09-10T21:52:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.626+0000 7f54fd8a5740 1 -- 172.21.15.7:0/1235320230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bafec0 msgr2=0x55c9d8b84070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.626+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/1235320230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bafec0 0x55c9d8b84070 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f54dc021040 tx=0x7f54dc062590 comp rx=0 tx=0).stop 2024-09-10T21:52:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.626+0000 7f54fd8a5740 1 -- 172.21.15.7:0/1235320230 shutdown_connections 2024-09-10T21:52:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.626+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/1235320230 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8bce8b0 0x55c9d8bb2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.626+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/1235320230 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bafec0 0x55c9d8b84070 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.626+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/1235320230 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8b89fa0 0x55c9d8b83b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.626+0000 7f54fd8a5740 1 -- 172.21.15.7:0/1235320230 >> 172.21.15.7:0/1235320230 conn(0x55c9d8b89a20 msgr2=0x55c9d8baccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.627+0000 7f54fd8a5740 1 -- 172.21.15.7:0/1235320230 shutdown_connections 2024-09-10T21:52:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.627+0000 7f54fd8a5740 1 -- 172.21.15.7:0/1235320230 wait complete. 2024-09-10T21:52:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.627+0000 7f54fd8a5740 1 -- start start 2024-09-10T21:52:13.629 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.627+0000 7f54fd8a5740 1 Processor -- start 2024-09-10T21:52:13.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.628+0000 7f54fd8a5740 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8b89fa0 0x55c9d8c488e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.628+0000 7f54fd8a5740 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8bafec0 0x55c9d8bb2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.628+0000 7f54fd8a5740 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 0x55c9d8baef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.628+0000 7f54fd8a5740 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8bad5a0 con 0x55c9d8bce8b0 2024-09-10T21:52:13.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.628+0000 7f54fd8a5740 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x55c9d8bde320 con 0x55c9d8b89fa0 2024-09-10T21:52:13.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.628+0000 7f54fd8a5740 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x55c9d8bddaa0 con 0x55c9d8bafec0 2024-09-10T21:52:13.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.628+0000 7f54fd8a5740 10 client.0 _ll_register_callbacks cb 0x7f54dc062270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:52:13.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.629+0000 7f54f8a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 0x55c9d8baef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.633 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.629+0000 7f54f8a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 0x55c9d8baef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:38238/0 (socket says 172.21.15.7:38238) 2024-09-10T21:52:13.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.629+0000 7f54f8a00640 1 -- 172.21.15.7:0/2276036452 learned_addr learned my addr 172.21.15.7:0/2276036452 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:13.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.629+0000 7f54f3e00640 1 --2- 172.21.15.7:0/2276036452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8b89fa0 0x55c9d8c488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.629+0000 7f54eb400640 1 --2- 172.21.15.7:0/2276036452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8bafec0 0x55c9d8bb2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.629+0000 7f54f8a00640 1 -- 172.21.15.7:0/2276036452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8bafec0 msgr2=0x55c9d8bb2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:13.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.629+0000 7f54f8a00640 1 --2- 172.21.15.7:0/2276036452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8bafec0 0x55c9d8bb2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.629+0000 7f54f8a00640 1 -- 172.21.15.7:0/2276036452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8b89fa0 msgr2=0x55c9d8c488e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:52:13.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.629+0000 7f54f8a00640 1 --2- 172.21.15.7:0/2276036452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8b89fa0 0x55c9d8c488e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.629+0000 7f54f8a00640 1 -- 172.21.15.7:0/2276036452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9d8bcb2e0 con 0x55c9d8bce8b0 2024-09-10T21:52:13.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.629+0000 7f54f8a00640 1 --2- 172.21.15.7:0/2276036452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 0x55c9d8baef60 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00ab00 tx=0x7f54ec00af00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec055016f1a10c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:13.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.629+0000 7f54e0c00640 10 client.0 ms_handle_connect on v2:172.21.15.7:3300/0 2024-09-10T21:52:13.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.630+0000 7f54e0c00640 1 -- 172.21.15.7:0/2276036452 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54ec01b020 con 0x55c9d8bce8b0 2024-09-10T21:52:13.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.630+0000 7f54e0c00640 1 -- 172.21.15.7:0/2276036452 <== mon.0 v2:172.21.15.7:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x7f54ec007da0 con 0x55c9d8bce8b0 2024-09-10T21:52:13.634 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.630+0000 7f54fd8a5740 1 -- 172.21.15.7:0/2276036452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54dc079030 con 0x55c9d8bce8b0 2024-09-10T21:52:13.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.630+0000 7f54e0c00640 1 -- 172.21.15.7:0/2276036452 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_get_version_reply(handle=1 version=115) ==== 24+0+0 (secure 0 0 0) 0x7f54ec017440 con 0x55c9d8bce8b0 2024-09-10T21:52:13.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.630+0000 7f54fd8a5740 10 client.8556 fetch_fsmap learned FSMap version 115 2024-09-10T21:52:13.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.630+0000 7f54fd8a5740 1 -- 172.21.15.7:0/2276036452 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,fsmap=115,monmap=2+}) -- 0x55c9d8c4f5e0 con 0x55c9d8bce8b0 2024-09-10T21:52:13.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.630+0000 7f54e0c00640 1 -- 172.21.15.7:0/2276036452 <== mon.0 v2:172.21.15.7:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54dc079030 con 0x55c9d8bce8b0 2024-09-10T21:52:13.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.630+0000 7f54e0c00640 1 -- 172.21.15.7:0/2276036452 <== mon.0 v2:172.21.15.7:3300/0 5 ==== fsmap(e 115) ==== 3502+0+0 (secure 0 0 0) 0x7f54ec028020 con 0x55c9d8bce8b0 2024-09-10T21:52:13.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.630+0000 7f54fd8a5740 10 client.8556 fetch_fsmap finished waiting for FSMap version 115 2024-09-10T21:52:13.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.630+0000 7f54fd8a5740 10 client.8556 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:52:13.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.630+0000 7f54fd8a5740 20 client.8556 populate_metadata read hostname 'smithi007' 2024-09-10T21:52:13.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.630+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/2276036452 >> [v2:172.21.15.133:6835/4068969165,v1:172.21.15.133:6836/4068969165] conn(0x55c9d8b82960 0x55c9d8bb02a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.630+0000 7f54fd8a5740 4 client.8556 mds_command: new command op to 236978 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:52:13.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.630+0000 7f54fd8a5740 1 -- 172.21.15.7:0/2276036452 --> [v2:172.21.15.133:6835/4068969165,v1:172.21.15.133:6836/4068969165] -- command(tid 0: {"prefix": "status"}) -- 0x55c9d8af9fd0 con 0x55c9d8b82960 2024-09-10T21:52:13.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.631+0000 7f54f3e00640 1 --2- 172.21.15.7:0/2276036452 >> [v2:172.21.15.133:6835/4068969165,v1:172.21.15.133:6836/4068969165] conn(0x55c9d8b82960 0x55c9d8bb02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.635 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.631+0000 7f54f3e00640 1 --2- 172.21.15.7:0/2276036452 >> [v2:172.21.15.133:6835/4068969165,v1:172.21.15.133:6836/4068969165] conn(0x55c9d8b82960 0x55c9d8bb02a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69ff390b3df8d326 server_cookie=d546208608ed63fd in_seq=0 out_seq=0 2024-09-10T21:52:13.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.631+0000 7f54e0c00640 10 client.8556 ms_handle_connect on v2:172.21.15.133:6835/4068969165 2024-09-10T21:52:13.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.631+0000 7f54e0c00640 1 -- 172.21.15.7:0/2276036452 <== mds.0 v2:172.21.15.133:6835/4068969165 1 ==== command_reply(tid 0: 0 ) ==== 8+0+336 (crc 0 0 0) 0x55c9d8af9fd0 con 0x55c9d8b82960 2024-09-10T21:52:13.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.631+0000 7f54e0c00640 10 client.8556 handle_command_reply: tid=0 2024-09-10T21:52:13.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.632+0000 7f54fd8a5740 1 client.8556 shutdown 2024-09-10T21:52:13.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.632+0000 7f54fd8a5740 1 -- 172.21.15.7:0/2276036452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 msgr2=0x55c9d8baef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:13.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.632+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/2276036452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 0x55c9d8baef60 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f54ec00ab00 tx=0x7f54ec00af00 comp rx=0 tx=0).stop 2024-09-10T21:52:13.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.632+0000 7f54fd8a5740 1 -- 172.21.15.7:0/2276036452 shutdown_connections 2024-09-10T21:52:13.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.632+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/2276036452 >> [v2:172.21.15.133:6835/4068969165,v1:172.21.15.133:6836/4068969165] conn(0x55c9d8b82960 0x55c9d8bb02a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.632+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/2276036452 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x55c9d8bce8b0 0x55c9d8baef60 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.632+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/2276036452 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x55c9d8bafec0 0x55c9d8bb2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.632+0000 7f54fd8a5740 1 --2- 172.21.15.7:0/2276036452 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x55c9d8b89fa0 0x55c9d8c488e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.632+0000 7f54fd8a5740 1 -- 172.21.15.7:0/2276036452 >> 172.21.15.7:0/2276036452 conn(0x55c9d8b89a20 msgr2=0x55c9d8bacb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:13.636 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.632+0000 7f54e0c00640 0 client.8556 ms_handle_reset on v2:172.21.15.133:6835/4068969165 2024-09-10T21:52:13.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.632+0000 7f54fd8a5740 1 -- 172.21.15.7:0/2276036452 shutdown_connections 2024-09-10T21:52:13.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.632+0000 7f54fd8a5740 1 -- 172.21.15.7:0/2276036452 wait complete. 2024-09-10T21:52:13.637 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.632+0000 7f54fd8a5740 20 client.8556 trim_cache size 0 max 16384 2024-09-10T21:52:13.637 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:52:13.637 INFO:teuthology.orchestra.run.smithi007.stdout: "cluster_fsid": "2029451a-3c26-48c7-b874-0597197278ac", 2024-09-10T21:52:13.637 INFO:teuthology.orchestra.run.smithi007.stdout: "whoami": 0, 2024-09-10T21:52:13.637 INFO:teuthology.orchestra.run.smithi007.stdout: "id": 236978, 2024-09-10T21:52:13.637 INFO:teuthology.orchestra.run.smithi007.stdout: "want_state": "up:active", 2024-09-10T21:52:13.637 INFO:teuthology.orchestra.run.smithi007.stdout: "state": "up:active", 2024-09-10T21:52:13.637 INFO:teuthology.orchestra.run.smithi007.stdout: "fs_name": "cephfs", 2024-09-10T21:52:13.638 INFO:teuthology.orchestra.run.smithi007.stdout: "rank_uptime": 4.7377571429999996, 2024-09-10T21:52:13.638 INFO:teuthology.orchestra.run.smithi007.stdout: "mdsmap_epoch": 115, 2024-09-10T21:52:13.638 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch": 118, 2024-09-10T21:52:13.638 INFO:teuthology.orchestra.run.smithi007.stdout: "osdmap_epoch_barrier": 118, 2024-09-10T21:52:13.638 INFO:teuthology.orchestra.run.smithi007.stdout: "uptime": 6.6886559280000002 2024-09-10T21:52:13.638 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:52:13.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.633+0000 7f54faa00640 1 -- 172.21.15.7:0/1395098476 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f54d0077f70 msgr2=0x7f54d007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:13.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.633+0000 7f54faa00640 1 --2- 172.21.15.7:0/1395098476 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f54d0077f70 0x7f54d007a430 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f54e4033000 tx=0x7f54e40046c0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.633+0000 7f54faa00640 1 -- 172.21.15.7:0/1395098476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f40cdbf0 msgr2=0x7f54f40b20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:13.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.633+0000 7f54faa00640 1 --2- 172.21.15.7:0/1395098476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f40cdbf0 0x7f54f40b20a0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f54dc00b450 tx=0x7f54dc00b910 comp rx=0 tx=0).stop 2024-09-10T21:52:13.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.633+0000 7f54faa00640 1 -- 172.21.15.7:0/1395098476 shutdown_connections 2024-09-10T21:52:13.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.633+0000 7f54faa00640 1 --2- 172.21.15.7:0/1395098476 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f54d0077f70 0x7f54d007a430 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.633+0000 7f54faa00640 1 --2- 172.21.15.7:0/1395098476 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f54f4159910 0x7f54f40b25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.633+0000 7f54faa00640 1 --2- 172.21.15.7:0/1395098476 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f54f40cdbf0 0x7f54f40b20a0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.633+0000 7f54faa00640 1 --2- 172.21.15.7:0/1395098476 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f54f406e810 0x7f54f41f7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.633+0000 7f54faa00640 1 -- 172.21.15.7:0/1395098476 >> 172.21.15.7:0/1395098476 conn(0x7f54f406ca60 msgr2=0x7f54f415a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:13.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.633+0000 7f54faa00640 1 -- 172.21.15.7:0/1395098476 shutdown_connections 2024-09-10T21:52:13.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.633+0000 7f54faa00640 1 -- 172.21.15.7:0/1395098476 wait complete. 2024-09-10T21:52:13.649 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:52:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.809+0000 7f612ca00640 1 -- 172.21.15.7:0/360973542 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61280cf7d0 msgr2=0x7f61280acd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:13.810 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.809+0000 7f612ca00640 1 --2- 172.21.15.7:0/360973542 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61280cf7d0 0x7f61280acd10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6114009860 tx=0x7f611402a530 comp rx=0 tx=0).stop 2024-09-10T21:52:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.809+0000 7f612ca00640 1 -- 172.21.15.7:0/360973542 shutdown_connections 2024-09-10T21:52:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.809+0000 7f612ca00640 1 --2- 172.21.15.7:0/360973542 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61280cf7d0 0x7f61280acd10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.809+0000 7f612ca00640 1 -- 172.21.15.7:0/360973542 >> 172.21.15.7:0/360973542 conn(0x7f61280b1ce0 msgr2=0x7f61280b20f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.809+0000 7f612ca00640 1 -- 172.21.15.7:0/360973542 shutdown_connections 2024-09-10T21:52:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.809+0000 7f612ca00640 1 -- 172.21.15.7:0/360973542 wait complete. 2024-09-10T21:52:13.811 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.811+0000 7f612ca00640 1 Processor -- start 2024-09-10T21:52:13.812 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.811+0000 7f612ca00640 1 -- start start 2024-09-10T21:52:13.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.812+0000 7f612ca00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f612806ce80 0x7f61281f1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.812+0000 7f612ca00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61280cf7d0 0x7f61281f6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.812+0000 7f612ca00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f612815bce0 0x7f61280b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.812+0000 7f612ca00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f61281607f0 con 0x7f612815bce0 2024-09-10T21:52:13.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.812+0000 7f612ca00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6128160670 con 0x7f612806ce80 2024-09-10T21:52:13.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.812+0000 7f612ca00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f612806d310 con 0x7f61280cf7d0 2024-09-10T21:52:13.814 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.812+0000 7f6127200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f612815bce0 0x7f61280b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.812+0000 7f6127200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f612815bce0 0x7f61280b0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34964/0 (socket says 172.21.15.7:34964) 2024-09-10T21:52:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.812+0000 7f6127200640 1 -- 172.21.15.7:0/2137995458 learned_addr learned my addr 172.21.15.7:0/2137995458 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.812+0000 7f6125e00640 1 --2- 172.21.15.7:0/2137995458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61280cf7d0 0x7f61281f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.812+0000 7f6126800640 1 --2- 172.21.15.7:0/2137995458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f612806ce80 0x7f61281f1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.813+0000 7f6127200640 1 -- 172.21.15.7:0/2137995458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61280cf7d0 msgr2=0x7f61281f6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.813+0000 7f6127200640 1 --2- 172.21.15.7:0/2137995458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61280cf7d0 0x7f61281f6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.813+0000 7f6127200640 1 -- 172.21.15.7:0/2137995458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f612806ce80 msgr2=0x7f61281f1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.813+0000 7f6127200640 1 --2- 172.21.15.7:0/2137995458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f612806ce80 0x7f61281f1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.813+0000 7f6127200640 1 -- 172.21.15.7:0/2137995458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61281f6a40 con 0x7f612815bce0 2024-09-10T21:52:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.813+0000 7f6127200640 1 --2- 172.21.15.7:0/2137995458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f612815bce0 0x7f61280b0cc0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f612000c780 tx=0x7f612000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.813+0000 7f611ea00640 1 -- 172.21.15.7:0/2137995458 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120003b70 con 0x7f612815bce0 2024-09-10T21:52:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.813+0000 7f611ea00640 1 -- 172.21.15.7:0/2137995458 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6120003d10 con 0x7f612815bce0 2024-09-10T21:52:13.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.813+0000 7f612ca00640 1 -- 172.21.15.7:0/2137995458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f612815d680 con 0x7f612815bce0 2024-09-10T21:52:13.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.813+0000 7f611ea00640 1 -- 172.21.15.7:0/2137995458 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f612000aab0 con 0x7f612815bce0 2024-09-10T21:52:13.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.813+0000 7f612ca00640 1 -- 172.21.15.7:0/2137995458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f612815dc40 con 0x7f612815bce0 2024-09-10T21:52:13.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.814+0000 7f612ca00640 1 -- 172.21.15.7:0/2137995458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61280acd10 con 0x7f612815bce0 2024-09-10T21:52:13.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.816+0000 7f611ea00640 1 -- 172.21.15.7:0/2137995458 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f612000ac50 con 0x7f612815bce0 2024-09-10T21:52:13.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.816+0000 7f611ea00640 1 --2- 172.21.15.7:0/2137995458 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6108077fa0 0x7f610807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:13.817 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.817+0000 7f611ea00640 1 -- 172.21.15.7:0/2137995458 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x7f6120099c30 con 0x7f612815bce0 2024-09-10T21:52:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.817+0000 7f6126800640 1 --2- 172.21.15.7:0/2137995458 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6108077fa0 0x7f610807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:13.818 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.818+0000 7f6126800640 1 --2- 172.21.15.7:0/2137995458 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6108077fa0 0x7f610807a460 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6118001220 tx=0x7f6118009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:13.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:13.821+0000 7f611ea00640 1 -- 172.21.15.7:0/2137995458 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f61200626d0 con 0x7f612815bce0 2024-09-10T21:52:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:14.054+0000 7f612ca00640 1 -- 172.21.15.7:0/2137995458 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f61280b3a50 con 0x7f612815bce0 2024-09-10T21:52:14.055 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:14.054+0000 7f611ea00640 1 -- 172.21.15.7:0/2137995458 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v115) ==== 53+0+73 (secure 0 0 0) 0x7f6120061fd0 con 0x7f612815bce0 2024-09-10T21:52:14.055 INFO:teuthology.orchestra.run.smithi007.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-10T21:52:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:14.057+0000 7f612ca00640 1 -- 172.21.15.7:0/2137995458 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6108077fa0 msgr2=0x7f610807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:14.057+0000 7f612ca00640 1 --2- 172.21.15.7:0/2137995458 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6108077fa0 0x7f610807a460 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f6118001220 tx=0x7f6118009040 comp rx=0 tx=0).stop 2024-09-10T21:52:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:14.057+0000 7f612ca00640 1 -- 172.21.15.7:0/2137995458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f612815bce0 msgr2=0x7f61280b0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:14.057+0000 7f612ca00640 1 --2- 172.21.15.7:0/2137995458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f612815bce0 0x7f61280b0cc0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f612000c780 tx=0x7f612000cc40 comp rx=0 tx=0).stop 2024-09-10T21:52:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:14.057+0000 7f612ca00640 1 -- 172.21.15.7:0/2137995458 shutdown_connections 2024-09-10T21:52:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:14.057+0000 7f612ca00640 1 --2- 172.21.15.7:0/2137995458 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6108077fa0 0x7f610807a460 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:14.057+0000 7f612ca00640 1 --2- 172.21.15.7:0/2137995458 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f612815bce0 0x7f61280b0cc0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:14.057+0000 7f612ca00640 1 --2- 172.21.15.7:0/2137995458 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f61280cf7d0 0x7f61281f6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:14.057+0000 7f612ca00640 1 --2- 172.21.15.7:0/2137995458 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f612806ce80 0x7f61281f1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:14.058 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:14.057+0000 7f612ca00640 1 -- 172.21.15.7:0/2137995458 >> 172.21.15.7:0/2137995458 conn(0x7f61280b1ce0 msgr2=0x7f61280af810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:14.057+0000 7f612ca00640 1 -- 172.21.15.7:0/2137995458 shutdown_connections 2024-09-10T21:52:14.059 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:14.057+0000 7f612ca00640 1 -- 172.21.15.7:0/2137995458 wait complete. 2024-09-10T21:52:14.073 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:52:14.073 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:52:14.074 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi007.front.sepia.ceph.com 2024-09-10T21:52:14.074 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:52:14.074 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:52:14.074 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:52:14.074 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-10T21:52:14.074 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:52:14.074 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:52:14.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:14.074 DEBUG:teuthology.orchestra.run.smithi007:> ip addr 2024-09-10T21:52:14.110 INFO:teuthology.orchestra.run.smithi007.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:52:14.110 INFO:teuthology.orchestra.run.smithi007.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:52:14.110 INFO:teuthology.orchestra.run.smithi007.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:52:14.110 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:52:14.110 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 ::1/128 scope host 2024-09-10T21:52:14.110 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:52:14.110 INFO:teuthology.orchestra.run.smithi007.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:52:14.110 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dc brd ff:ff:ff:ff:ff:ff 2024-09-10T21:52:14.110 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f0 2024-09-10T21:52:14.110 INFO:teuthology.orchestra.run.smithi007.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:52:14.111 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:6b:fd:dd brd ff:ff:ff:ff:ff:ff 2024-09-10T21:52:14.111 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp7s0f1 2024-09-10T21:52:14.111 INFO:teuthology.orchestra.run.smithi007.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:52:14.111 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:ea brd ff:ff:ff:ff:ff:ff 2024-09-10T21:52:14.111 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f0 2024-09-10T21:52:14.111 INFO:teuthology.orchestra.run.smithi007.stdout: inet 172.21.15.7/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-10T21:52:14.111 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft 42011sec preferred_lft 42011sec 2024-09-10T21:52:14.111 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::ec4:7aff:febd:15ea/64 scope link 2024-09-10T21:52:14.111 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:52:14.111 INFO:teuthology.orchestra.run.smithi007.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T21:52:14.111 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 0c:c4:7a:bd:15:eb brd ff:ff:ff:ff:ff:ff 2024-09-10T21:52:14.111 INFO:teuthology.orchestra.run.smithi007.stdout: altname enp1s0f1 2024-09-10T21:52:14.111 INFO:teuthology.orchestra.run.smithi007.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-10T21:52:14.112 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff 2024-09-10T21:52:14.112 INFO:teuthology.orchestra.run.smithi007.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-10T21:52:14.112 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:52:14.112 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::3c3d:dbff:fe02:4c15/64 scope link 2024-09-10T21:52:14.112 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:52:14.112 INFO:teuthology.orchestra.run.smithi007.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:52:14.112 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether f6:f9:69:5a:86:f5 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:52:14.112 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::f4f9:69ff:fe5a:86f5/64 scope link 2024-09-10T21:52:14.112 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:52:14.112 INFO:teuthology.orchestra.run.smithi007.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-10T21:52:14.112 INFO:teuthology.orchestra.run.smithi007.stdout: link/ether 06:33:5c:44:4a:5b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:52:14.112 INFO:teuthology.orchestra.run.smithi007.stdout: inet6 fe80::433:5cff:fe44:4a5b/64 scope link 2024-09-10T21:52:14.112 INFO:teuthology.orchestra.run.smithi007.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:52:14.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:14.113 DEBUG:teuthology.orchestra.run.smithi007:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:52:14.204 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:52:14.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:14.207 DEBUG:teuthology.orchestra.run.smithi007:> route 2024-09-10T21:52:14.243 INFO:teuthology.orchestra.run.smithi007.stdout:Kernel IP routing table 2024-09-10T21:52:14.243 INFO:teuthology.orchestra.run.smithi007.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:52:14.243 INFO:teuthology.orchestra.run.smithi007.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-10T21:52:14.243 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-10T21:52:14.243 INFO:teuthology.orchestra.run.smithi007.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:52:14.244 DEBUG:teuthology.orchestra.run.smithi007:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:52:14.244 DEBUG:teuthology.orchestra.run.smithi007:> set -e 2024-09-10T21:52:14.244 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:52:14.244 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:52:14.244 DEBUG:teuthology.orchestra.run.smithi007:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:52:14.244 DEBUG:teuthology.orchestra.run.smithi007:> ') 2024-09-10T21:52:14.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:52:14.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:52:14.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:14.445 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:52:14.480 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:52:14.480 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:52:14.480 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:52:14.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:14.481 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link set brx.0 up 2024-09-10T21:52:14.577 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:52:14.577 DEBUG:teuthology.orchestra.run.smithi007:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:52:14.611 INFO:teuthology.orchestra.run.smithi007.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:52:14.612 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:52:14.612 DEBUG:teuthology.orchestra.run.smithi007:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:52:14.686 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2024-09-10T21:52:14.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:14.686 DEBUG:teuthology.orchestra.run.smithi007:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:52:14.815 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:52:14.815 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:52:14.846 INFO:teuthology.orchestra.run.smithi007.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2024-09-10T21:52:14.846 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2024-09-10T21:52:14.846 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: resolved to: "172.21.15.7:3300,172.21.15.133:3300,172.21.15.133:3301" 2024-09-10T21:52:14.846 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: trying mount with new device syntax: 0@2029451a-3c26-48c7-b874-0597197278ac.cephfs=/ 2024-09-10T21:52:14.846 INFO:teuthology.orchestra.run.smithi007.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.7:3300/172.21.15.133:3300/172.21.15.133:3301" will pass to kernel 2024-09-10T21:52:14.848 INFO:teuthology.orchestra.run.smithi007.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-10T21:52:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: You just mounted a file system that supports labels which does not 2024-09-10T21:52:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-10T21:52:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: applications will generate AVC messages and not be allowed access to 2024-09-10T21:52:14.848 INFO:teuthology.orchestra.run.smithi007.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-10T21:52:14.850 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-10T21:52:14.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:14.851 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:52:14.907 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-10T21:52:14.986 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:14 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:52:14.990 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:52:14.998 INFO:teuthology.orchestra.run.smithi007.stdout:client237050 2024-09-10T21:52:14.998 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:52:14.998 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:52:15.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:15.034 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2024-09-10T21:52:15.116 INFO:teuthology.orchestra.run.smithi007.stdout:./ 2024-09-10T21:52:15.116 INFO:teuthology.orchestra.run.smithi007.stdout:./parent 2024-09-10T21:52:15.116 INFO:teuthology.orchestra.run.smithi007.stdout:./parent/flushed 2024-09-10T21:52:15.117 INFO:teuthology.orchestra.run.smithi007.stdout:./parent/flushed/charlie 2024-09-10T21:52:15.117 INFO:teuthology.orchestra.run.smithi007.stdout:./parent/flushed/alpha 2024-09-10T21:52:15.117 INFO:teuthology.orchestra.run.smithi007.stdout:./parent/flushed/bravo 2024-09-10T21:52:15.117 INFO:teuthology.orchestra.run.smithi007.stdout:./parent/unflushed 2024-09-10T21:52:15.117 INFO:teuthology.orchestra.run.smithi007.stdout:./parent/unflushed/jfile 2024-09-10T21:52:15.118 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:52:15.118 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:52:15.118 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:52:15.118 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:52:15.118 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2024-09-10T21:52:15.118 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:52:15.257 INFO:teuthology.orchestra.run.smithi007.stdout:1 2024-09-10T21:52:15.261 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:52:15.269 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:52:15.269 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:52:15.269 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:52:15.270 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:52:15.270 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent").st_ino) 2024-09-10T21:52:15.270 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:52:15.410 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627776 2024-09-10T21:52:15.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:52:15.422 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:52:15.422 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:52:15.422 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:52:15.422 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:52:15.422 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed").st_ino) 2024-09-10T21:52:15.422 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:52:15.561 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627777 2024-09-10T21:52:15.565 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:52:15.573 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:52:15.573 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:52:15.573 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:52:15.573 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:52:15.573 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/charlie").st_ino) 2024-09-10T21:52:15.573 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:52:15.713 INFO:teuthology.orchestra.run.smithi007.stdout:1099511628280 2024-09-10T21:52:15.716 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:52:15.724 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:52:15.724 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:52:15.724 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:52:15.724 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:52:15.724 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/alpha").st_ino) 2024-09-10T21:52:15.724 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:52:15.863 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627778 2024-09-10T21:52:15.867 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:52:15.875 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:52:15.875 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:52:15.875 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:52:15.875 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:52:15.875 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/flushed/bravo").st_ino) 2024-09-10T21:52:15.875 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:52:16.038 INFO:teuthology.orchestra.run.smithi007.stdout:1099511628279 2024-09-10T21:52:16.042 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:52:16.050 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:52:16.050 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:52:16.050 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:52:16.050 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:52:16.050 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed").st_ino) 2024-09-10T21:52:16.050 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:52:16.190 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627779 2024-09-10T21:52:16.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:52:16.201 DEBUG:teuthology.orchestra.run.smithi007:> stdin-killer -- python3 -c ' 2024-09-10T21:52:16.202 DEBUG:teuthology.orchestra.run.smithi007:> import os 2024-09-10T21:52:16.202 DEBUG:teuthology.orchestra.run.smithi007:> import stat 2024-09-10T21:52:16.202 DEBUG:teuthology.orchestra.run.smithi007:> 2024-09-10T21:52:16.202 DEBUG:teuthology.orchestra.run.smithi007:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parent/unflushed/jfile").st_ino) 2024-09-10T21:52:16.202 DEBUG:teuthology.orchestra.run.smithi007:> ' 2024-09-10T21:52:16.342 INFO:teuthology.orchestra.run.smithi007.stdout:1099511627780 2024-09-10T21:52:16.345 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:52:16.353 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-10T21:52:16.353 DEBUG:teuthology.orchestra.run.smithi007:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:52:16.485 DEBUG:teuthology.orchestra.run.smithi133:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-10T21:52:16.589 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-10T21:52:16.590 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:52:16.590 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:52:16.590 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:52:16.625 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:52:16.625 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:52:16.701 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:52:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.903+0000 7fc69e600640 1 -- 172.21.15.7:0/2572142581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc67c003a50 msgr2=0x7fc67c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.903+0000 7fc69e600640 1 --2- 172.21.15.7:0/2572142581 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc67c003a50 0x7fc67c003e70 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fc688009b80 tx=0x7fc67c005830 comp rx=0 tx=0).stop 2024-09-10T21:52:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.903+0000 7fc69dc00640 1 -- 172.21.15.7:0/2572142581 reap_dead start 2024-09-10T21:52:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.903+0000 7fc69e600640 1 -- 172.21.15.7:0/2572142581 shutdown_connections 2024-09-10T21:52:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.903+0000 7fc69e600640 1 -- 172.21.15.7:0/2572142581 >> 172.21.15.7:0/2572142581 conn(0x7fc6980cc6b0 msgr2=0x7fc6980ccac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:16.904 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.903+0000 7fc69e600640 1 -- 172.21.15.7:0/2572142581 shutdown_connections 2024-09-10T21:52:16.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.903+0000 7fc69e600640 1 -- 172.21.15.7:0/2572142581 wait complete. 2024-09-10T21:52:16.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.904+0000 7fc69e600640 1 Processor -- start 2024-09-10T21:52:16.905 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.905+0000 7fc69e600640 1 -- start start 2024-09-10T21:52:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.905+0000 7fc69e600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc67c003a50 0x7fc6981f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.905+0000 7fc69e600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6981f8ae0 0x7fc6980b37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.905+0000 7fc69e600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc698074290 0x7fc6980b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.905+0000 7fc69e600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc6980c9ce0 con 0x7fc6981f8ae0 2024-09-10T21:52:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.905+0000 7fc69e600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc6980c9b60 con 0x7fc698074290 2024-09-10T21:52:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.905+0000 7fc69e600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc69815e8b0 con 0x7fc67c003a50 2024-09-10T21:52:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.906+0000 7fc68fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6981f8ae0 0x7fc6980b37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:16.908 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.906+0000 7fc69dc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc698074290 0x7fc6980b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.906+0000 7fc68fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6981f8ae0 0x7fc6980b37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34974/0 (socket says 172.21.15.7:34974) 2024-09-10T21:52:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.906+0000 7fc68fe00640 1 -- 172.21.15.7:0/197603141 learned_addr learned my addr 172.21.15.7:0/197603141 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.906+0000 7fc69d200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc67c003a50 0x7fc6981f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.906+0000 7fc68fe00640 1 -- 172.21.15.7:0/197603141 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc67c003a50 msgr2=0x7fc6981f5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.906+0000 7fc68fe00640 1 --2- 172.21.15.7:0/197603141 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc67c003a50 0x7fc6981f5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.906+0000 7fc68fe00640 1 -- 172.21.15.7:0/197603141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc698074290 msgr2=0x7fc6980b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.906+0000 7fc68fe00640 1 --2- 172.21.15.7:0/197603141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc698074290 0x7fc6980b05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.906+0000 7fc68fe00640 1 -- 172.21.15.7:0/197603141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6980b3d20 con 0x7fc6981f8ae0 2024-09-10T21:52:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.906+0000 7fc68fe00640 1 --2- 172.21.15.7:0/197603141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6981f8ae0 0x7fc6980b37e0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fc68000e610 tx=0x7fc68000ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.907+0000 7fc695600640 1 -- 172.21.15.7:0/197603141 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680015070 con 0x7fc6981f8ae0 2024-09-10T21:52:16.909 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.907+0000 7fc695600640 1 -- 172.21.15.7:0/197603141 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc680010980 con 0x7fc6981f8ae0 2024-09-10T21:52:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.907+0000 7fc695600640 1 -- 172.21.15.7:0/197603141 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc680010db0 con 0x7fc6981f8ae0 2024-09-10T21:52:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.907+0000 7fc69e600640 1 -- 172.21.15.7:0/197603141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6980b3ff0 con 0x7fc6981f8ae0 2024-09-10T21:52:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.907+0000 7fc69e600640 1 -- 172.21.15.7:0/197603141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6981f4e60 con 0x7fc6981f8ae0 2024-09-10T21:52:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.908+0000 7fc69e600640 1 -- 172.21.15.7:0/197603141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc69815e370 con 0x7fc6981f8ae0 2024-09-10T21:52:16.910 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.909+0000 7fc695600640 1 -- 172.21.15.7:0/197603141 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc680011460 con 0x7fc6981f8ae0 2024-09-10T21:52:16.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.910+0000 7fc695600640 1 --2- 172.21.15.7:0/197603141 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc674077fa0 0x7fc67407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:16.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.910+0000 7fc695600640 1 -- 172.21.15.7:0/197603141 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x7fc68009a4e0 con 0x7fc6981f8ae0 2024-09-10T21:52:16.911 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.911+0000 7fc69d200640 1 --2- 172.21.15.7:0/197603141 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc674077fa0 0x7fc67407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:16.912 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.911+0000 7fc69d200640 1 --2- 172.21.15.7:0/197603141 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc674077fa0 0x7fc67407a460 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc688035000 tx=0x7fc68802d040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:16.915 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:16.914+0000 7fc695600640 1 -- 172.21.15.7:0/197603141 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc680062800 con 0x7fc6981f8ae0 2024-09-10T21:52:17.125 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.124+0000 7fc69e600640 1 -- 172.21.15.7:0/197603141 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc6980ce740 con 0x7fc6981f8ae0 2024-09-10T21:52:17.126 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.125+0000 7fc695600640 1 -- 172.21.15.7:0/197603141 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 4 entries v118) ==== 80+0+246 (secure 0 0 0) 0x7fc680067230 con 0x7fc6981f8ae0 2024-09-10T21:52:17.126 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.133:6836/772257983 2024-09-11T21:52:06.881059+0000 2024-09-10T21:52:17.126 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.133:6835/772257983 2024-09-11T21:52:06.881059+0000 2024-09-10T21:52:17.126 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.133:6837/3764354952 2024-09-11T21:51:56.206885+0000 2024-09-10T21:52:17.126 INFO:teuthology.orchestra.run.smithi007.stdout:172.21.15.133:6834/3764354952 2024-09-11T21:51:56.206885+0000 2024-09-10T21:52:17.127 INFO:teuthology.orchestra.run.smithi007.stderr:listed 4 entries 2024-09-10T21:52:17.128 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.127+0000 7fc69e600640 1 -- 172.21.15.7:0/197603141 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc674077fa0 msgr2=0x7fc67407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:17.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.127+0000 7fc69e600640 1 --2- 172.21.15.7:0/197603141 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc674077fa0 0x7fc67407a460 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fc688035000 tx=0x7fc68802d040 comp rx=0 tx=0).stop 2024-09-10T21:52:17.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.127+0000 7fc69e600640 1 -- 172.21.15.7:0/197603141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6981f8ae0 msgr2=0x7fc6980b37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:17.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.127+0000 7fc69e600640 1 --2- 172.21.15.7:0/197603141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6981f8ae0 0x7fc6980b37e0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fc68000e610 tx=0x7fc68000ead0 comp rx=0 tx=0).stop 2024-09-10T21:52:17.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.127+0000 7fc69e600640 1 -- 172.21.15.7:0/197603141 shutdown_connections 2024-09-10T21:52:17.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.127+0000 7fc69e600640 1 --2- 172.21.15.7:0/197603141 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc674077fa0 0x7fc67407a460 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:17.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.127+0000 7fc69e600640 1 --2- 172.21.15.7:0/197603141 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc698074290 0x7fc6980b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:17.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.127+0000 7fc69e600640 1 --2- 172.21.15.7:0/197603141 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc6981f8ae0 0x7fc6980b37e0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:17.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.127+0000 7fc69e600640 1 --2- 172.21.15.7:0/197603141 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc67c003a50 0x7fc6981f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:17.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.128+0000 7fc69e600640 1 -- 172.21.15.7:0/197603141 >> 172.21.15.7:0/197603141 conn(0x7fc6980cc6b0 msgr2=0x7fc69815edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:17.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.128+0000 7fc69e600640 1 -- 172.21.15.7:0/197603141 shutdown_connections 2024-09-10T21:52:17.129 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.128+0000 7fc69e600640 1 -- 172.21.15.7:0/197603141 wait complete. 2024-09-10T21:52:17.144 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-10T21:52:17.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:17.144 DEBUG:teuthology.orchestra.run.smithi007:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-10T21:52:17.206 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:52:17.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:17.207 DEBUG:teuthology.orchestra.run.smithi007:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:52:17.242 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:52:17.242 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:52:17.242 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:52:17.282 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:52:17.282 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:52:17.283 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:52:17.283 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:52:17.318 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:52:17.318 INFO:tasks.cephfs.mount:Terminating background process 2024-09-10T21:52:17.318 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:52:17.318 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:52:17.359 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:52:17.360 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:52:17.521 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.520+0000 7f5dfbc00640 1 -- 172.21.15.7:0/1435151354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df406e810 msgr2=0x7f5df40cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:17.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.520+0000 7f5dfbc00640 1 --2- 172.21.15.7:0/1435151354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df406e810 0x7f5df40cd6b0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc005fd0 tx=0x7f5ddc02a5d0 comp rx=0 tx=0).stop 2024-09-10T21:52:17.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.520+0000 7f5dfbc00640 1 -- 172.21.15.7:0/1435151354 shutdown_connections 2024-09-10T21:52:17.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.520+0000 7f5dfbc00640 1 --2- 172.21.15.7:0/1435151354 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df406e810 0x7f5df40cd6b0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:17.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.520+0000 7f5dfbc00640 1 --2- 172.21.15.7:0/1435151354 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5df4159910 0x7f5df415d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:17.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.520+0000 7f5dfbc00640 1 -- 172.21.15.7:0/1435151354 >> 172.21.15.7:0/1435151354 conn(0x7f5df406ca60 msgr2=0x7f5df406ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:17.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.520+0000 7f5dfbc00640 1 -- 172.21.15.7:0/1435151354 shutdown_connections 2024-09-10T21:52:17.522 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.520+0000 7f5dfbc00640 1 -- 172.21.15.7:0/1435151354 wait complete. 2024-09-10T21:52:17.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.522+0000 7f5dfbc00640 1 Processor -- start 2024-09-10T21:52:17.523 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.522+0000 7f5dfbc00640 1 -- start start 2024-09-10T21:52:17.524 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.523+0000 7f5dfbc00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df406e810 0x7f5df40b1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.523+0000 7f5dfbc00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5df40cdbf0 0x7f5df41f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.523+0000 7f5dfbc00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5df4159910 0x7f5df4150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.523+0000 7f5dfbc00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5df40740c0 con 0x7f5df406e810 2024-09-10T21:52:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.523+0000 7f5dfbc00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5df4073f40 con 0x7f5df40cdbf0 2024-09-10T21:52:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.523+0000 7f5dfbc00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f5df4074240 con 0x7f5df4159910 2024-09-10T21:52:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.523+0000 7f5df9200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df406e810 0x7f5df40b1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.523+0000 7f5df9200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df406e810 0x7f5df40b1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:34990/0 (socket says 172.21.15.7:34990) 2024-09-10T21:52:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.523+0000 7f5df9200640 1 -- 172.21.15.7:0/1415908304 learned_addr learned my addr 172.21.15.7:0/1415908304 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.523+0000 7f5df9c00640 1 --2- 172.21.15.7:0/1415908304 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5df4159910 0x7f5df4150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.523+0000 7f5debe00640 1 --2- 172.21.15.7:0/1415908304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5df40cdbf0 0x7f5df41f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.524+0000 7f5df9200640 1 -- 172.21.15.7:0/1415908304 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5df4159910 msgr2=0x7f5df4150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.524+0000 7f5df9200640 1 --2- 172.21.15.7:0/1415908304 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5df4159910 0x7f5df4150df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:17.525 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.524+0000 7f5df9200640 1 -- 172.21.15.7:0/1415908304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5df40cdbf0 msgr2=0x7f5df41f1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:17.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.524+0000 7f5df9200640 1 --2- 172.21.15.7:0/1415908304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5df40cdbf0 0x7f5df41f1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:17.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.524+0000 7f5df9200640 1 -- 172.21.15.7:0/1415908304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df414e8b0 con 0x7f5df406e810 2024-09-10T21:52:17.526 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.524+0000 7f5df9200640 1 --2- 172.21.15.7:0/1415908304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df406e810 0x7f5df40b1680 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f5de400d770 tx=0x7f5de400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:17.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.524+0000 7f5df1600640 1 -- 172.21.15.7:0/1415908304 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de4017070 con 0x7f5df406e810 2024-09-10T21:52:17.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.524+0000 7f5df1600640 1 -- 172.21.15.7:0/1415908304 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5de4009d20 con 0x7f5df406e810 2024-09-10T21:52:17.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.524+0000 7f5df1600640 1 -- 172.21.15.7:0/1415908304 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de40106c0 con 0x7f5df406e810 2024-09-10T21:52:17.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.525+0000 7f5dfbc00640 1 -- 172.21.15.7:0/1415908304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df414eb50 con 0x7f5df406e810 2024-09-10T21:52:17.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.525+0000 7f5dfbc00640 1 -- 172.21.15.7:0/1415908304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df414f110 con 0x7f5df406e810 2024-09-10T21:52:17.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.526+0000 7f5dfbc00640 1 -- 172.21.15.7:0/1415908304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df40cd6b0 con 0x7f5df406e810 2024-09-10T21:52:17.527 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.527+0000 7f5df1600640 1 -- 172.21.15.7:0/1415908304 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f5de4010860 con 0x7f5df406e810 2024-09-10T21:52:17.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.528+0000 7f5df1600640 1 --2- 172.21.15.7:0/1415908304 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5dd0077f70 0x7f5dd007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:17.528 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.528+0000 7f5df1600640 1 -- 172.21.15.7:0/1415908304 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x7f5de409a220 con 0x7f5df406e810 2024-09-10T21:52:17.529 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.528+0000 7f5debe00640 1 --2- 172.21.15.7:0/1415908304 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5dd0077f70 0x7f5dd007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:17.530 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.529+0000 7f5debe00640 1 --2- 172.21.15.7:0/1415908304 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5dd0077f70 0x7f5dd007a430 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc033000 tx=0x7f5ddc004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:17.532 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.532+0000 7f5df1600640 1 -- 172.21.15.7:0/1415908304 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5de40625d0 con 0x7f5df406e810 2024-09-10T21:52:17.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.766+0000 7f5dfbc00640 1 -- 172.21.15.7:0/1415908304 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5df415b0b0 con 0x7f5df406e810 2024-09-10T21:52:17.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.766+0000 7f5df1600640 1 -- 172.21.15.7:0/1415908304 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+5238 (secure 0 0 0) 0x7f5de4067000 con 0x7f5df406e810 2024-09-10T21:52:17.767 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:17.767 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":115,"btime":"2024-09-10T21:52:11:895606+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3012572644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3012572644},{"type":"v1","addr":"172.21.15.7:6834","nonce":3012572644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7660,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2492331385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2492331385},{"type":"v1","addr":"172.21.15.133:6837","nonce":2492331385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":8094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/148356862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":148356862},{"type":"v1","addr":"172.21.15.7:6835","nonce":148356862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:51:37.913965+0000","modified":"2024-09-10T21:52:11.895593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236978},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236978":{"gid":236978,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6836/4068969165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4068969165},{"type":"v1","addr":"172.21.15.133:6836","nonce":4068969165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":236978,"qdb_cluster":[236978]},"id":8}]} 2024-09-10T21:52:17.767 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 115 2024-09-10T21:52:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.769+0000 7f5dfbc00640 1 -- 172.21.15.7:0/1415908304 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5dd0077f70 msgr2=0x7f5dd007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.769+0000 7f5dfbc00640 1 --2- 172.21.15.7:0/1415908304 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5dd0077f70 0x7f5dd007a430 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc033000 tx=0x7f5ddc004730 comp rx=0 tx=0).stop 2024-09-10T21:52:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.769+0000 7f5dfbc00640 1 -- 172.21.15.7:0/1415908304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df406e810 msgr2=0x7f5df40b1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.769+0000 7f5dfbc00640 1 --2- 172.21.15.7:0/1415908304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df406e810 0x7f5df40b1680 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f5de400d770 tx=0x7f5de400dc30 comp rx=0 tx=0).stop 2024-09-10T21:52:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.769+0000 7f5dfbc00640 1 -- 172.21.15.7:0/1415908304 shutdown_connections 2024-09-10T21:52:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.769+0000 7f5dfbc00640 1 --2- 172.21.15.7:0/1415908304 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5dd0077f70 0x7f5dd007a430 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.769+0000 7f5dfbc00640 1 --2- 172.21.15.7:0/1415908304 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5df40cdbf0 0x7f5df41f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.769+0000 7f5dfbc00640 1 --2- 172.21.15.7:0/1415908304 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f5df4159910 0x7f5df4150df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.769+0000 7f5dfbc00640 1 --2- 172.21.15.7:0/1415908304 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f5df406e810 0x7f5df40b1680 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:17.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.769+0000 7f5dfbc00640 1 -- 172.21.15.7:0/1415908304 >> 172.21.15.7:0/1415908304 conn(0x7f5df406ca60 msgr2=0x7f5df415cd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:17.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.769+0000 7f5dfbc00640 1 -- 172.21.15.7:0/1415908304 shutdown_connections 2024-09-10T21:52:17.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.769+0000 7f5dfbc00640 1 -- 172.21.15.7:0/1415908304 wait complete. 2024-09-10T21:52:17.786 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:52:17.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.946+0000 7fc9c1400640 1 -- 172.21.15.7:0/2732649901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9bc0cdbf0 msgr2=0x7fc9bc15fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:17.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.946+0000 7fc9c1400640 1 --2- 172.21.15.7:0/2732649901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9bc0cdbf0 0x7fc9bc15fa70 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fc9bc06f230 tx=0x7fc9b402aa00 comp rx=0 tx=0).stop 2024-09-10T21:52:17.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.946+0000 7fc9c1400640 1 -- 172.21.15.7:0/2732649901 shutdown_connections 2024-09-10T21:52:17.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.946+0000 7fc9c1400640 1 --2- 172.21.15.7:0/2732649901 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9bc0cdbf0 0x7fc9bc15fa70 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:17.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.946+0000 7fc9c1400640 1 --2- 172.21.15.7:0/2732649901 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9bc06e810 0x7fc9bc0cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:17.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.946+0000 7fc9c1400640 1 --2- 172.21.15.7:0/2732649901 >> v2:172.21.15.133:3300/0 conn(0x7fc9bc159910 0x7fc9bc15d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:17.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.946+0000 7fc9c1400640 1 -- 172.21.15.7:0/2732649901 >> 172.21.15.7:0/2732649901 conn(0x7fc9bc06ca60 msgr2=0x7fc9bc06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:17.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.947+0000 7fc9c1400640 1 -- 172.21.15.7:0/2732649901 shutdown_connections 2024-09-10T21:52:17.948 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.947+0000 7fc9c1400640 1 -- 172.21.15.7:0/2732649901 wait complete. 2024-09-10T21:52:17.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.948+0000 7fc9c1400640 1 Processor -- start 2024-09-10T21:52:17.949 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.948+0000 7fc9c1400640 1 -- start start 2024-09-10T21:52:17.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.949+0000 7fc9c1400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9bc06e810 0x7fc9bc1f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:17.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.949+0000 7fc9c1400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9bc0cdbf0 0x7fc9bc0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:17.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.949+0000 7fc9c1400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9bc159910 0x7fc9bc0b2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:17.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.949+0000 7fc9c1400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc160130 con 0x7fc9bc0cdbf0 2024-09-10T21:52:17.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.949+0000 7fc9c1400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc9bc15ffb0 con 0x7fc9bc159910 2024-09-10T21:52:17.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.949+0000 7fc9c1400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc9bc1602b0 con 0x7fc9bc06e810 2024-09-10T21:52:17.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.949+0000 7fc9b9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9bc0cdbf0 0x7fc9bc0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:17.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.949+0000 7fc9b9e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9bc0cdbf0 0x7fc9bc0b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35000/0 (socket says 172.21.15.7:35000) 2024-09-10T21:52:17.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.950+0000 7fc9b9e00640 1 -- 172.21.15.7:0/3672128554 learned_addr learned my addr 172.21.15.7:0/3672128554 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:17.954 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.950+0000 7fc9ba800640 1 --2- 172.21.15.7:0/3672128554 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9bc06e810 0x7fc9bc1f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:17.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.950+0000 7fc9bb200640 1 --2- 172.21.15.7:0/3672128554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9bc159910 0x7fc9bc0b2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:17.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.950+0000 7fc9b9e00640 1 -- 172.21.15.7:0/3672128554 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9bc06e810 msgr2=0x7fc9bc1f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:17.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.950+0000 7fc9b9e00640 1 --2- 172.21.15.7:0/3672128554 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9bc06e810 0x7fc9bc1f3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:17.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.950+0000 7fc9b9e00640 1 -- 172.21.15.7:0/3672128554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9bc159910 msgr2=0x7fc9bc0b2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:17.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.950+0000 7fc9b9e00640 1 --2- 172.21.15.7:0/3672128554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9bc159910 0x7fc9bc0b2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:17.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.950+0000 7fc9b9e00640 1 -- 172.21.15.7:0/3672128554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9bc1553c0 con 0x7fc9bc0cdbf0 2024-09-10T21:52:17.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.950+0000 7fc9ba800640 1 --2- 172.21.15.7:0/3672128554 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9bc06e810 0x7fc9bc1f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:17.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.950+0000 7fc9bb200640 1 --2- 172.21.15.7:0/3672128554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9bc159910 0x7fc9bc0b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:17.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.950+0000 7fc9b9e00640 1 --2- 172.21.15.7:0/3672128554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9bc0cdbf0 0x7fc9bc0b1d50 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800d950 tx=0x7fc9a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:17.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.950+0000 7fc9b2a00640 1 -- 172.21.15.7:0/3672128554 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8017070 con 0x7fc9bc0cdbf0 2024-09-10T21:52:17.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.951+0000 7fc9b2a00640 1 -- 172.21.15.7:0/3672128554 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9a8009d20 con 0x7fc9bc0cdbf0 2024-09-10T21:52:17.955 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.951+0000 7fc9c1400640 1 -- 172.21.15.7:0/3672128554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9bc1556a0 con 0x7fc9bc0cdbf0 2024-09-10T21:52:17.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.951+0000 7fc9b2a00640 1 -- 172.21.15.7:0/3672128554 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9a80106c0 con 0x7fc9bc0cdbf0 2024-09-10T21:52:17.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.951+0000 7fc9c1400640 1 -- 172.21.15.7:0/3672128554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9bc155c60 con 0x7fc9bc0cdbf0 2024-09-10T21:52:17.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.952+0000 7fc9c1400640 1 -- 172.21.15.7:0/3672128554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9bc1607d0 con 0x7fc9bc0cdbf0 2024-09-10T21:52:17.956 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.953+0000 7fc9b2a00640 1 -- 172.21.15.7:0/3672128554 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc9a8010860 con 0x7fc9bc0cdbf0 2024-09-10T21:52:17.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.954+0000 7fc9b2a00640 1 --2- 172.21.15.7:0/3672128554 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc99c077f70 0x7fc99c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:17.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.954+0000 7fc9b2a00640 1 -- 172.21.15.7:0/3672128554 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x7fc9a809a3f0 con 0x7fc9bc0cdbf0 2024-09-10T21:52:17.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.954+0000 7fc9ba800640 1 --2- 172.21.15.7:0/3672128554 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc99c077f70 0x7fc99c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:17.957 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.955+0000 7fc9ba800640 1 --2- 172.21.15.7:0/3672128554 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc99c077f70 0x7fc99c07a430 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac005fd0 tx=0x7fc9ac005d10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:17.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:17.958+0000 7fc9b2a00640 1 -- 172.21.15.7:0/3672128554 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc9a8062d60 con 0x7fc9bc0cdbf0 2024-09-10T21:52:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.190+0000 7fc9c1400640 1 -- 172.21.15.7:0/3672128554 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9bc1f4370 con 0x7fc9bc0cdbf0 2024-09-10T21:52:18.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.191+0000 7fc9b2a00640 1 -- 172.21.15.7:0/3672128554 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+5238 (secure 0 0 0) 0x7fc9a8062660 con 0x7fc9bc0cdbf0 2024-09-10T21:52:18.192 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:18.192 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":115,"btime":"2024-09-10T21:52:11:895606+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3012572644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3012572644},{"type":"v1","addr":"172.21.15.7:6834","nonce":3012572644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7660,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2492331385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2492331385},{"type":"v1","addr":"172.21.15.133:6837","nonce":2492331385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":8094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/148356862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":148356862},{"type":"v1","addr":"172.21.15.7:6835","nonce":148356862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:51:37.913965+0000","modified":"2024-09-10T21:52:11.895593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236978},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236978":{"gid":236978,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6836/4068969165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4068969165},{"type":"v1","addr":"172.21.15.133:6836","nonce":4068969165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":236978,"qdb_cluster":[236978]},"id":8}]} 2024-09-10T21:52:18.192 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 115 2024-09-10T21:52:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.193+0000 7fc9c1400640 1 -- 172.21.15.7:0/3672128554 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc99c077f70 msgr2=0x7fc99c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.193+0000 7fc9c1400640 1 --2- 172.21.15.7:0/3672128554 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc99c077f70 0x7fc99c07a430 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc9ac005fd0 tx=0x7fc9ac005d10 comp rx=0 tx=0).stop 2024-09-10T21:52:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.193+0000 7fc9c1400640 1 -- 172.21.15.7:0/3672128554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9bc0cdbf0 msgr2=0x7fc9bc0b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.193+0000 7fc9c1400640 1 --2- 172.21.15.7:0/3672128554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9bc0cdbf0 0x7fc9bc0b1d50 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800d950 tx=0x7fc9a800de10 comp rx=0 tx=0).stop 2024-09-10T21:52:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.193+0000 7fc9c1400640 1 -- 172.21.15.7:0/3672128554 shutdown_connections 2024-09-10T21:52:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.193+0000 7fc9c1400640 1 --2- 172.21.15.7:0/3672128554 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc99c077f70 0x7fc99c07a430 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:18.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.193+0000 7fc9c1400640 1 --2- 172.21.15.7:0/3672128554 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc9bc159910 0x7fc9bc0b2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.193+0000 7fc9c1400640 1 --2- 172.21.15.7:0/3672128554 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc9bc0cdbf0 0x7fc9bc0b1d50 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.193+0000 7fc9c1400640 1 --2- 172.21.15.7:0/3672128554 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc9bc06e810 0x7fc9bc1f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.193+0000 7fc9c1400640 1 -- 172.21.15.7:0/3672128554 >> 172.21.15.7:0/3672128554 conn(0x7fc9bc06ca60 msgr2=0x7fc9bc15cd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.193+0000 7fc9c1400640 1 -- 172.21.15.7:0/3672128554 shutdown_connections 2024-09-10T21:52:18.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.193+0000 7fc9c1400640 1 -- 172.21.15.7:0/3672128554 wait complete. 2024-09-10T21:52:18.211 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:52:18.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.370+0000 7fe1e4600640 1 -- 172.21.15.7:0/4227096902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1dc06e810 msgr2=0x7fe1dc0cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:18.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.370+0000 7fe1e4600640 1 --2- 172.21.15.7:0/4227096902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1dc06e810 0x7fe1dc0cd6b0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7fe1dc06e390 tx=0x7fe1c802a510 comp rx=0 tx=0).stop 2024-09-10T21:52:18.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.370+0000 7fe1e4600640 1 -- 172.21.15.7:0/4227096902 shutdown_connections 2024-09-10T21:52:18.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.370+0000 7fe1e4600640 1 --2- 172.21.15.7:0/4227096902 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1dc06e810 0x7fe1dc0cd6b0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:18.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.370+0000 7fe1e4600640 1 --2- 172.21.15.7:0/4227096902 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1dc159910 0x7fe1dc15d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:18.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.370+0000 7fe1e4600640 1 -- 172.21.15.7:0/4227096902 >> 172.21.15.7:0/4227096902 conn(0x7fe1dc06ca60 msgr2=0x7fe1dc06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:18.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.370+0000 7fe1e4600640 1 -- 172.21.15.7:0/4227096902 shutdown_connections 2024-09-10T21:52:18.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.370+0000 7fe1e4600640 1 -- 172.21.15.7:0/4227096902 wait complete. 2024-09-10T21:52:18.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.372+0000 7fe1e4600640 1 Processor -- start 2024-09-10T21:52:18.373 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.372+0000 7fe1e4600640 1 -- start start 2024-09-10T21:52:18.374 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.373+0000 7fe1e4600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1dc06e810 0x7fe1dc156e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.373+0000 7fe1e4600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1dc0cdbf0 0x7fe1dc0b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.373+0000 7fe1e4600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1dc159910 0x7fe1dc0afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.373+0000 7fe1e4600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc0740c0 con 0x7fe1dc159910 2024-09-10T21:52:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.373+0000 7fe1e4600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe1dc073f40 con 0x7fe1dc06e810 2024-09-10T21:52:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.373+0000 7fe1e4600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe1dc074240 con 0x7fe1dc0cdbf0 2024-09-10T21:52:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.373+0000 7fe1e2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1dc159910 0x7fe1dc0afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.373+0000 7fe1e2600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1dc159910 0x7fe1dc0afc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35014/0 (socket says 172.21.15.7:35014) 2024-09-10T21:52:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.373+0000 7fe1e2600640 1 -- 172.21.15.7:0/2687053865 learned_addr learned my addr 172.21.15.7:0/2687053865 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.373+0000 7fe1e1c00640 1 --2- 172.21.15.7:0/2687053865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1dc06e810 0x7fe1dc156e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.373+0000 7fe1e1200640 1 --2- 172.21.15.7:0/2687053865 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1dc0cdbf0 0x7fe1dc0b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.374+0000 7fe1e2600640 1 -- 172.21.15.7:0/2687053865 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1dc0cdbf0 msgr2=0x7fe1dc0b1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.374+0000 7fe1e2600640 1 --2- 172.21.15.7:0/2687053865 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1dc0cdbf0 0x7fe1dc0b1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:18.375 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.374+0000 7fe1e2600640 1 -- 172.21.15.7:0/2687053865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1dc06e810 msgr2=0x7fe1dc156e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:18.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.374+0000 7fe1e2600640 1 --2- 172.21.15.7:0/2687053865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1dc06e810 0x7fe1dc156e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:18.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.374+0000 7fe1e2600640 1 -- 172.21.15.7:0/2687053865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1dc1f4c90 con 0x7fe1dc159910 2024-09-10T21:52:18.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.374+0000 7fe1e2600640 1 --2- 172.21.15.7:0/2687053865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1dc159910 0x7fe1dc0afc20 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400c780 tx=0x7fe1d400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:18.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.374+0000 7fe1d9600640 1 -- 172.21.15.7:0/2687053865 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d4004b60 con 0x7fe1dc159910 2024-09-10T21:52:18.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.374+0000 7fe1d9600640 1 -- 172.21.15.7:0/2687053865 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1d4004df0 con 0x7fe1dc159910 2024-09-10T21:52:18.376 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.374+0000 7fe1d9600640 1 -- 172.21.15.7:0/2687053865 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1d400aa90 con 0x7fe1dc159910 2024-09-10T21:52:18.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.374+0000 7fe1e4600640 1 -- 172.21.15.7:0/2687053865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1dc1f4f70 con 0x7fe1dc159910 2024-09-10T21:52:18.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.374+0000 7fe1e4600640 1 -- 172.21.15.7:0/2687053865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1dc1f5530 con 0x7fe1dc159910 2024-09-10T21:52:18.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.376+0000 7fe1e4600640 1 -- 172.21.15.7:0/2687053865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1dc0cd6b0 con 0x7fe1dc159910 2024-09-10T21:52:18.377 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.377+0000 7fe1d9600640 1 -- 172.21.15.7:0/2687053865 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe1d400ac30 con 0x7fe1dc159910 2024-09-10T21:52:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.378+0000 7fe1d9600640 1 --2- 172.21.15.7:0/2687053865 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe1bc077f70 0x7fe1bc07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.378+0000 7fe1d9600640 1 -- 172.21.15.7:0/2687053865 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x7fe1d4099fa0 con 0x7fe1dc159910 2024-09-10T21:52:18.379 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.379+0000 7fe1e1c00640 1 --2- 172.21.15.7:0/2687053865 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe1bc077f70 0x7fe1bc07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:18.380 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.379+0000 7fe1e1c00640 1 --2- 172.21.15.7:0/2687053865 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe1bc077f70 0x7fe1bc07a430 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0005fd0 tx=0x7fe1d000a250 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:18.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.382+0000 7fe1d9600640 1 -- 172.21.15.7:0/2687053865 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe1d4020070 con 0x7fe1dc159910 2024-09-10T21:52:18.583 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.582+0000 7fe1e4600640 1 -- 172.21.15.7:0/2687053865 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe1dc070c60 con 0x7fe1dc159910 2024-09-10T21:52:18.584 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.583+0000 7fe1d9600640 1 -- 172.21.15.7:0/2687053865 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+17065 (secure 0 0 0) 0x7fe1d4062360 con 0x7fe1dc159910 2024-09-10T21:52:18.584 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:18.585 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":118,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:52:06.881081+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:51:35.736733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T21:51:36.873305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.6","mappings":[{"from":0,"to":5}]},{"pgid":"17.3c","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6836/772257983":"2024-09-11T21:52:06.881059+0000","172.21.15.133:6835/772257983":"2024-09-11T21:52:06.881059+0000","172.21.15.133:6837/3764354952":"2024-09-11T21:51:56.206885+0000","172.21.15.133:6834/3764354952":"2024-09-11T21:51:56.206885+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:52:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.586+0000 7fe1e4600640 1 -- 172.21.15.7:0/2687053865 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe1bc077f70 msgr2=0x7fe1bc07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.586+0000 7fe1e4600640 1 --2- 172.21.15.7:0/2687053865 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe1bc077f70 0x7fe1bc07a430 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fe1d0005fd0 tx=0x7fe1d000a250 comp rx=0 tx=0).stop 2024-09-10T21:52:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.586+0000 7fe1e4600640 1 -- 172.21.15.7:0/2687053865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1dc159910 msgr2=0x7fe1dc0afc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.586+0000 7fe1e4600640 1 --2- 172.21.15.7:0/2687053865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1dc159910 0x7fe1dc0afc20 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fe1d400c780 tx=0x7fe1d400cc40 comp rx=0 tx=0).stop 2024-09-10T21:52:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.586+0000 7fe1e4600640 1 -- 172.21.15.7:0/2687053865 shutdown_connections 2024-09-10T21:52:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.586+0000 7fe1e4600640 1 --2- 172.21.15.7:0/2687053865 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe1bc077f70 0x7fe1bc07a430 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.586+0000 7fe1e4600640 1 --2- 172.21.15.7:0/2687053865 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe1dc159910 0x7fe1dc0afc20 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.586+0000 7fe1e4600640 1 --2- 172.21.15.7:0/2687053865 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe1dc0cdbf0 0x7fe1dc0b1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.586+0000 7fe1e4600640 1 --2- 172.21.15.7:0/2687053865 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe1dc06e810 0x7fe1dc156e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:18.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.586+0000 7fe1e4600640 1 -- 172.21.15.7:0/2687053865 >> 172.21.15.7:0/2687053865 conn(0x7fe1dc06ca60 msgr2=0x7fe1dc075890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:18.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.586+0000 7fe1e4600640 1 -- 172.21.15.7:0/2687053865 shutdown_connections 2024-09-10T21:52:18.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.586+0000 7fe1e4600640 1 -- 172.21.15.7:0/2687053865 wait complete. 2024-09-10T21:52:18.603 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:52:18.604 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:52:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.763+0000 7fe90b000640 1 -- 172.21.15.7:0/239217673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe90415ca60 msgr2=0x7fe90415cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.763+0000 7fe90b000640 1 --2- 172.21.15.7:0/239217673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe90415ca60 0x7fe90415cef0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc005fd0 tx=0x7fe8fc02aa90 comp rx=0 tx=0).stop 2024-09-10T21:52:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.763+0000 7fe90b000640 1 -- 172.21.15.7:0/239217673 shutdown_connections 2024-09-10T21:52:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.763+0000 7fe90b000640 1 --2- 172.21.15.7:0/239217673 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe90415ca60 0x7fe90415cef0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.763+0000 7fe90b000640 1 --2- 172.21.15.7:0/239217673 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe904070ba0 0x7fe90406f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.763+0000 7fe90b000640 1 -- 172.21.15.7:0/239217673 >> 172.21.15.7:0/239217673 conn(0x7fe9040b2570 msgr2=0x7fe9040b2980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:18.764 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.763+0000 7fe90b000640 1 -- 172.21.15.7:0/239217673 shutdown_connections 2024-09-10T21:52:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.763+0000 7fe90b000640 1 -- 172.21.15.7:0/239217673 wait complete. 2024-09-10T21:52:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.764+0000 7fe90b000640 1 Processor -- start 2024-09-10T21:52:18.765 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.765+0000 7fe90b000640 1 -- start start 2024-09-10T21:52:18.766 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.765+0000 7fe90b000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe904070ba0 0x7fe904143f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.765+0000 7fe90b000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9040b6050 0x7fe90415d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.765+0000 7fe90b000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe90415ca60 0x7fe904144950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.765+0000 7fe90b000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe9040af940 con 0x7fe9040b6050 2024-09-10T21:52:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.765+0000 7fe90b000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe90406fad0 con 0x7fe90415ca60 2024-09-10T21:52:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.765+0000 7fe90b000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe9040afac0 con 0x7fe904070ba0 2024-09-10T21:52:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.766+0000 7fe8fbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9040b6050 0x7fe90415d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.766+0000 7fe8fbe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9040b6050 0x7fe90415d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35026/0 (socket says 172.21.15.7:35026) 2024-09-10T21:52:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.766+0000 7fe8fbe00640 1 -- 172.21.15.7:0/2799108776 learned_addr learned my addr 172.21.15.7:0/2799108776 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.766+0000 7fe909000640 1 --2- 172.21.15.7:0/2799108776 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe90415ca60 0x7fe904144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:18.767 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.766+0000 7fe903e00640 1 --2- 172.21.15.7:0/2799108776 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe904070ba0 0x7fe904143f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:18.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.766+0000 7fe8fbe00640 1 -- 172.21.15.7:0/2799108776 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe904070ba0 msgr2=0x7fe904143f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:18.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.766+0000 7fe8fbe00640 1 --2- 172.21.15.7:0/2799108776 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe904070ba0 0x7fe904143f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:18.768 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.766+0000 7fe8fbe00640 1 -- 172.21.15.7:0/2799108776 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe90415ca60 msgr2=0x7fe904144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.766+0000 7fe8fbe00640 1 --2- 172.21.15.7:0/2799108776 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe90415ca60 0x7fe904144950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.766+0000 7fe8fbe00640 1 -- 172.21.15.7:0/2799108776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe904144e90 con 0x7fe9040b6050 2024-09-10T21:52:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.767+0000 7fe8fbe00640 1 --2- 172.21.15.7:0/2799108776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9040b6050 0x7fe90415d580 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00d770 tx=0x7fe8ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.767+0000 7fe900c00640 1 -- 172.21.15.7:0/2799108776 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec017070 con 0x7fe9040b6050 2024-09-10T21:52:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.767+0000 7fe900c00640 1 -- 172.21.15.7:0/2799108776 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe8ec009d20 con 0x7fe9040b6050 2024-09-10T21:52:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.767+0000 7fe900c00640 1 -- 172.21.15.7:0/2799108776 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec004280 con 0x7fe9040b6050 2024-09-10T21:52:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.767+0000 7fe90b000640 1 -- 172.21.15.7:0/2799108776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe90415dcd0 con 0x7fe9040b6050 2024-09-10T21:52:18.769 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.767+0000 7fe90b000640 1 -- 172.21.15.7:0/2799108776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe90415e290 con 0x7fe9040b6050 2024-09-10T21:52:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.768+0000 7fe90b000640 1 -- 172.21.15.7:0/2799108776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9040cb960 con 0x7fe9040b6050 2024-09-10T21:52:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.769+0000 7fe900c00640 1 -- 172.21.15.7:0/2799108776 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe8ec0035f0 con 0x7fe9040b6050 2024-09-10T21:52:18.770 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.770+0000 7fe900c00640 1 --2- 172.21.15.7:0/2799108776 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe8e0077f70 0x7fe8e007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.770+0000 7fe900c00640 1 -- 172.21.15.7:0/2799108776 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x7fe8ec099ff0 con 0x7fe9040b6050 2024-09-10T21:52:18.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.770+0000 7fe903e00640 1 --2- 172.21.15.7:0/2799108776 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe8e0077f70 0x7fe8e007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:18.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.771+0000 7fe903e00640 1 --2- 172.21.15.7:0/2799108776 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe8e0077f70 0x7fe8e007a430 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40017c0 tx=0x7fe8f4009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:18.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:18.774+0000 7fe900c00640 1 -- 172.21.15.7:0/2799108776 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe9040cb960 con 0x7fe9040b6050 2024-09-10T21:52:19.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.007+0000 7fe90b000640 1 -- 172.21.15.7:0/2799108776 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe904076090 con 0x7fe9040b6050 2024-09-10T21:52:19.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.008+0000 7fe900c00640 1 -- 172.21.15.7:0/2799108776 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+5238 (secure 0 0 0) 0x7fe904076090 con 0x7fe9040b6050 2024-09-10T21:52:19.010 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:19.010 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":115,"btime":"2024-09-10T21:52:11:895606+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3012572644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3012572644},{"type":"v1","addr":"172.21.15.7:6834","nonce":3012572644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7660,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2492331385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2492331385},{"type":"v1","addr":"172.21.15.133:6837","nonce":2492331385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":8094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/148356862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":148356862},{"type":"v1","addr":"172.21.15.7:6835","nonce":148356862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:51:37.913965+0000","modified":"2024-09-10T21:52:11.895593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236978},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236978":{"gid":236978,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6836/4068969165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4068969165},{"type":"v1","addr":"172.21.15.133:6836","nonce":4068969165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":236978,"qdb_cluster":[236978]},"id":8}]} 2024-09-10T21:52:19.010 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 115 2024-09-10T21:52:19.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.010+0000 7fe90b000640 1 -- 172.21.15.7:0/2799108776 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe8e0077f70 msgr2=0x7fe8e007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:19.011 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.010+0000 7fe90b000640 1 --2- 172.21.15.7:0/2799108776 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe8e0077f70 0x7fe8e007a430 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe8f40017c0 tx=0x7fe8f4009040 comp rx=0 tx=0).stop 2024-09-10T21:52:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.010+0000 7fe90b000640 1 -- 172.21.15.7:0/2799108776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9040b6050 msgr2=0x7fe90415d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.010+0000 7fe90b000640 1 --2- 172.21.15.7:0/2799108776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9040b6050 0x7fe90415d580 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec00d770 tx=0x7fe8ec00dc30 comp rx=0 tx=0).stop 2024-09-10T21:52:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.010+0000 7fe90b000640 1 -- 172.21.15.7:0/2799108776 shutdown_connections 2024-09-10T21:52:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.010+0000 7fe90b000640 1 --2- 172.21.15.7:0/2799108776 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe8e0077f70 0x7fe8e007a430 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.010+0000 7fe90b000640 1 --2- 172.21.15.7:0/2799108776 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe90415ca60 0x7fe904144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.010+0000 7fe90b000640 1 --2- 172.21.15.7:0/2799108776 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe9040b6050 0x7fe90415d580 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.010+0000 7fe90b000640 1 --2- 172.21.15.7:0/2799108776 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe904070ba0 0x7fe904143f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.010+0000 7fe90b000640 1 -- 172.21.15.7:0/2799108776 >> 172.21.15.7:0/2799108776 conn(0x7fe9040b2570 msgr2=0x7fe9040cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.010+0000 7fe90b000640 1 -- 172.21.15.7:0/2799108776 shutdown_connections 2024-09-10T21:52:19.012 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.011+0000 7fe90b000640 1 -- 172.21.15.7:0/2799108776 wait complete. 2024-09-10T21:52:19.027 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:52:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.186+0000 7f351f000640 1 -- 172.21.15.7:0/2860466652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3518159910 msgr2=0x7f351815d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.186+0000 7f351f000640 1 --2- 172.21.15.7:0/2860466652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3518159910 0x7f351815d090 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f35180cfed0 tx=0x7f350802a5f0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.186+0000 7f351f000640 1 -- 172.21.15.7:0/2860466652 shutdown_connections 2024-09-10T21:52:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.186+0000 7f351f000640 1 --2- 172.21.15.7:0/2860466652 >> v2:172.21.15.133:3300/0 conn(0x7f351806e810 0x7f35180cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.187+0000 7f351f000640 1 --2- 172.21.15.7:0/2860466652 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3518159910 0x7f351815d090 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.187+0000 7f351f000640 1 -- 172.21.15.7:0/2860466652 >> 172.21.15.7:0/2860466652 conn(0x7f351806ca60 msgr2=0x7f351806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.187+0000 7f351f000640 1 -- 172.21.15.7:0/2860466652 shutdown_connections 2024-09-10T21:52:19.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.187+0000 7f351f000640 1 -- 172.21.15.7:0/2860466652 wait complete. 2024-09-10T21:52:19.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.188+0000 7f351f000640 1 Processor -- start 2024-09-10T21:52:19.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.188+0000 7f351f000640 1 -- start start 2024-09-10T21:52:19.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.189+0000 7f351f000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f351806e810 0x7f351813b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:19.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.189+0000 7f351f000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35180cdbf0 0x7f35180b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:19.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.189+0000 7f351f000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3518159910 0x7f35180b4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.189+0000 7f351f000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f35180740c0 con 0x7f35180cdbf0 2024-09-10T21:52:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.189+0000 7f351f000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3518073f40 con 0x7f3518159910 2024-09-10T21:52:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.189+0000 7f351f000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f3518074240 con 0x7f351806e810 2024-09-10T21:52:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.190+0000 7f350f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35180cdbf0 0x7f35180b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.190+0000 7f350f400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35180cdbf0 0x7f35180b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35032/0 (socket says 172.21.15.7:35032) 2024-09-10T21:52:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.190+0000 7f350f400640 1 -- 172.21.15.7:0/312124122 learned_addr learned my addr 172.21.15.7:0/312124122 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.190+0000 7f3517e00640 1 --2- 172.21.15.7:0/312124122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f351806e810 0x7f351813b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.190+0000 7f351d000640 1 --2- 172.21.15.7:0/312124122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3518159910 0x7f35180b4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.190+0000 7f350f400640 1 -- 172.21.15.7:0/312124122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f351806e810 msgr2=0x7f351813b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.190+0000 7f350f400640 1 --2- 172.21.15.7:0/312124122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f351806e810 0x7f351813b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.190+0000 7f350f400640 1 -- 172.21.15.7:0/312124122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3518159910 msgr2=0x7f35180b4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.190+0000 7f350f400640 1 --2- 172.21.15.7:0/312124122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3518159910 0x7f35180b4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.192 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.190+0000 7f350f400640 1 -- 172.21.15.7:0/312124122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3518151a30 con 0x7f35180cdbf0 2024-09-10T21:52:19.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.190+0000 7f350f400640 1 --2- 172.21.15.7:0/312124122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35180cdbf0 0x7f35180b0630 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f350000d950 tx=0x7f350000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:19.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.190+0000 7f3514c00640 1 -- 172.21.15.7:0/312124122 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3500017070 con 0x7f35180cdbf0 2024-09-10T21:52:19.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.190+0000 7f3514c00640 1 -- 172.21.15.7:0/312124122 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3500009d20 con 0x7f35180cdbf0 2024-09-10T21:52:19.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.190+0000 7f3514c00640 1 -- 172.21.15.7:0/312124122 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35000106c0 con 0x7f35180cdbf0 2024-09-10T21:52:19.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.191+0000 7f351f000640 1 -- 172.21.15.7:0/312124122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3518151d10 con 0x7f35180cdbf0 2024-09-10T21:52:19.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.191+0000 7f351f000640 1 -- 172.21.15.7:0/312124122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35180b0b70 con 0x7f35180cdbf0 2024-09-10T21:52:19.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.192+0000 7f351f000640 1 -- 172.21.15.7:0/312124122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f351815bda0 con 0x7f35180cdbf0 2024-09-10T21:52:19.193 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.193+0000 7f3514c00640 1 -- 172.21.15.7:0/312124122 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3500010860 con 0x7f35180cdbf0 2024-09-10T21:52:19.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.194+0000 7f3514c00640 1 --2- 172.21.15.7:0/312124122 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f34f4077fa0 0x7f34f407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:19.194 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.194+0000 7f3514c00640 1 -- 172.21.15.7:0/312124122 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x7f350009a1d0 con 0x7f35180cdbf0 2024-09-10T21:52:19.195 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.194+0000 7f3517e00640 1 --2- 172.21.15.7:0/312124122 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f34f4077fa0 0x7f34f407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:19.196 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.195+0000 7f3517e00640 1 --2- 172.21.15.7:0/312124122 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f34f4077fa0 0x7f34f407a460 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3508033000 tx=0x7f3508004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:19.199 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.198+0000 7f3514c00640 1 -- 172.21.15.7:0/312124122 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f3500062c70 con 0x7f35180cdbf0 2024-09-10T21:52:19.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.427+0000 7f351f000640 1 -- 172.21.15.7:0/312124122 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35180cd6b0 con 0x7f35180cdbf0 2024-09-10T21:52:19.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.428+0000 7f3514c00640 1 -- 172.21.15.7:0/312124122 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) ==== 95+0+5238 (secure 0 0 0) 0x7f3500062570 con 0x7f35180cdbf0 2024-09-10T21:52:19.429 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:19.429 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":115,"btime":"2024-09-10T21:52:11:895606+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3012572644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3012572644},{"type":"v1","addr":"172.21.15.7:6834","nonce":3012572644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7660,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2492331385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2492331385},{"type":"v1","addr":"172.21.15.133:6837","nonce":2492331385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":8094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/148356862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":148356862},{"type":"v1","addr":"172.21.15.7:6835","nonce":148356862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":115,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:51:37.913965+0000","modified":"2024-09-10T21:52:11.895593+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":118,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":236978},"failed":[],"damaged":[],"stopped":[],"info":{"gid_236978":{"gid":236978,"name":"d","rank":0,"incarnation":112,"state":"up:active","state_seq":4,"addr":"172.21.15.133:6836/4068969165","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":4068969165},{"type":"v1","addr":"172.21.15.133:6836","nonce":4068969165}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":236978,"qdb_cluster":[236978]},"id":8}]} 2024-09-10T21:52:19.429 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 115 2024-09-10T21:52:19.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.430+0000 7f351f000640 1 -- 172.21.15.7:0/312124122 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f34f4077fa0 msgr2=0x7f34f407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:19.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.430+0000 7f351f000640 1 --2- 172.21.15.7:0/312124122 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f34f4077fa0 0x7f34f407a460 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f3508033000 tx=0x7f3508004730 comp rx=0 tx=0).stop 2024-09-10T21:52:19.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.430+0000 7f351f000640 1 -- 172.21.15.7:0/312124122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35180cdbf0 msgr2=0x7f35180b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:19.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.430+0000 7f351f000640 1 --2- 172.21.15.7:0/312124122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35180cdbf0 0x7f35180b0630 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f350000d950 tx=0x7f350000de10 comp rx=0 tx=0).stop 2024-09-10T21:52:19.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.430+0000 7f351f000640 1 -- 172.21.15.7:0/312124122 shutdown_connections 2024-09-10T21:52:19.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.430+0000 7f351f000640 1 --2- 172.21.15.7:0/312124122 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f34f4077fa0 0x7f34f407a460 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.430+0000 7f351f000640 1 --2- 172.21.15.7:0/312124122 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3518159910 0x7f35180b4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.430+0000 7f351f000640 1 --2- 172.21.15.7:0/312124122 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f35180cdbf0 0x7f35180b0630 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.430+0000 7f351f000640 1 --2- 172.21.15.7:0/312124122 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f351806e810 0x7f351813b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.430+0000 7f351f000640 1 -- 172.21.15.7:0/312124122 >> 172.21.15.7:0/312124122 conn(0x7f351806ca60 msgr2=0x7f351815d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:19.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.430+0000 7f351f000640 1 -- 172.21.15.7:0/312124122 shutdown_connections 2024-09-10T21:52:19.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.430+0000 7f351f000640 1 -- 172.21.15.7:0/312124122 wait complete. 2024-09-10T21:52:19.446 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:52:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.605+0000 7f74b6600640 1 -- 172.21.15.7:0/1174999985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b0159910 msgr2=0x7f74b015d090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.605+0000 7f74b6600640 1 --2- 172.21.15.7:0/1174999985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b0159910 0x7f74b015d090 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f74a0005fd0 tx=0x7f74a002a680 comp rx=0 tx=0).stop 2024-09-10T21:52:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.605+0000 7f74b6600640 1 -- 172.21.15.7:0/1174999985 shutdown_connections 2024-09-10T21:52:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.605+0000 7f74b6600640 1 --2- 172.21.15.7:0/1174999985 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b0159910 0x7f74b015d090 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.605+0000 7f74b6600640 1 -- 172.21.15.7:0/1174999985 >> 172.21.15.7:0/1174999985 conn(0x7f74b006ca60 msgr2=0x7f74b006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.605+0000 7f74b6600640 1 -- 172.21.15.7:0/1174999985 shutdown_connections 2024-09-10T21:52:19.606 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.605+0000 7f74b6600640 1 -- 172.21.15.7:0/1174999985 wait complete. 2024-09-10T21:52:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.606+0000 7f74b6600640 1 Processor -- start 2024-09-10T21:52:19.607 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.607+0000 7f74b6600640 1 -- start start 2024-09-10T21:52:19.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.608+0000 7f74b6600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74b006e810 0x7f74b0152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:19.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.608+0000 7f74b6600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b00cddc0 0x7f74b01557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:19.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.608+0000 7f74b6600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74b0159910 0x7f74b0152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:19.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.608+0000 7f74b6600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f74b00739d0 con 0x7f74b00cddc0 2024-09-10T21:52:19.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.608+0000 7f74b6600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f74b0073850 con 0x7f74b0159910 2024-09-10T21:52:19.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.608+0000 7f74b6600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f74b0073b50 con 0x7f74b006e810 2024-09-10T21:52:19.610 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.608+0000 7f74a6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b00cddc0 0x7f74b01557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.608+0000 7f74a6a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b00cddc0 0x7f74b01557c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35058/0 (socket says 172.21.15.7:35058) 2024-09-10T21:52:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.608+0000 7f74a6a00640 1 -- 172.21.15.7:0/4103850650 learned_addr learned my addr 172.21.15.7:0/4103850650 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.608+0000 7f74af400640 1 --2- 172.21.15.7:0/4103850650 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74b006e810 0x7f74b0152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.608+0000 7f74afe00640 1 --2- 172.21.15.7:0/4103850650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74b0159910 0x7f74b0152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.608+0000 7f74a6a00640 1 -- 172.21.15.7:0/4103850650 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74b006e810 msgr2=0x7f74b0152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.608+0000 7f74a6a00640 1 --2- 172.21.15.7:0/4103850650 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74b006e810 0x7f74b0152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.608+0000 7f74a6a00640 1 -- 172.21.15.7:0/4103850650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74b0159910 msgr2=0x7f74b0152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.608+0000 7f74a6a00640 1 --2- 172.21.15.7:0/4103850650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74b0159910 0x7f74b0152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.608+0000 7f74a6a00640 1 -- 172.21.15.7:0/4103850650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b01f61c0 con 0x7f74b00cddc0 2024-09-10T21:52:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.609+0000 7f74a6a00640 1 --2- 172.21.15.7:0/4103850650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b00cddc0 0x7f74b01557c0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f749800c500 tx=0x7f749800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.609+0000 7f74a7e00640 1 -- 172.21.15.7:0/4103850650 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74980052d0 con 0x7f74b00cddc0 2024-09-10T21:52:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.609+0000 7f74a7e00640 1 -- 172.21.15.7:0/4103850650 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7498005470 con 0x7f74b00cddc0 2024-09-10T21:52:19.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.609+0000 7f74a7e00640 1 -- 172.21.15.7:0/4103850650 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7498012870 con 0x7f74b00cddc0 2024-09-10T21:52:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.609+0000 7f74b6600640 1 -- 172.21.15.7:0/4103850650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b01f6460 con 0x7f74b00cddc0 2024-09-10T21:52:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.609+0000 7f74b6600640 1 -- 172.21.15.7:0/4103850650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b01f6a20 con 0x7f74b00cddc0 2024-09-10T21:52:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.610+0000 7f74b6600640 1 -- 172.21.15.7:0/4103850650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74b0074070 con 0x7f74b00cddc0 2024-09-10T21:52:19.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.611+0000 7f74a7e00640 1 -- 172.21.15.7:0/4103850650 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7498003610 con 0x7f74b00cddc0 2024-09-10T21:52:19.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.613+0000 7f74a7e00640 1 --2- 172.21.15.7:0/4103850650 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f748c077f70 0x7f748c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.613+0000 7f74a7e00640 1 -- 172.21.15.7:0/4103850650 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x7f7498099a00 con 0x7f74b00cddc0 2024-09-10T21:52:19.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.613+0000 7f74af400640 1 --2- 172.21.15.7:0/4103850650 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f748c077f70 0x7f748c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:19.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.614+0000 7f74af400640 1 --2- 172.21.15.7:0/4103850650 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f748c077f70 0x7f748c07a430 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f74a0033000 tx=0x7f74a0004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:19.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.617+0000 7f74a7e00640 1 -- 172.21.15.7:0/4103850650 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7498062380 con 0x7f74b00cddc0 2024-09-10T21:52:19.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.814+0000 7f74b6600640 1 -- 172.21.15.7:0/4103850650 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f74b0156090 con 0x7f74b00cddc0 2024-09-10T21:52:19.816 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.816+0000 7f74a7e00640 1 -- 172.21.15.7:0/4103850650 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+17065 (secure 0 0 0) 0x7f7498061c80 con 0x7f74b00cddc0 2024-09-10T21:52:19.817 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:19.817 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":118,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:52:06.881081+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:51:35.736733+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-10T21:51:36.873305+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"114","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.6","mappings":[{"from":0,"to":5}]},{"pgid":"17.3c","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6836/772257983":"2024-09-11T21:52:06.881059+0000","172.21.15.133:6835/772257983":"2024-09-11T21:52:06.881059+0000","172.21.15.133:6837/3764354952":"2024-09-11T21:51:56.206885+0000","172.21.15.133:6834/3764354952":"2024-09-11T21:51:56.206885+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:52:19.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.818+0000 7f74b6600640 1 -- 172.21.15.7:0/4103850650 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f748c077f70 msgr2=0x7f748c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:19.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.818+0000 7f74b6600640 1 --2- 172.21.15.7:0/4103850650 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f748c077f70 0x7f748c07a430 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f74a0033000 tx=0x7f74a0004730 comp rx=0 tx=0).stop 2024-09-10T21:52:19.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.818+0000 7f74b6600640 1 -- 172.21.15.7:0/4103850650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b00cddc0 msgr2=0x7f74b01557c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.818+0000 7f74b6600640 1 --2- 172.21.15.7:0/4103850650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b00cddc0 0x7f74b01557c0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f749800c500 tx=0x7f749800c9c0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.818+0000 7f74b6600640 1 -- 172.21.15.7:0/4103850650 shutdown_connections 2024-09-10T21:52:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.818+0000 7f74b6600640 1 --2- 172.21.15.7:0/4103850650 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f748c077f70 0x7f748c07a430 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.818+0000 7f74b6600640 1 --2- 172.21.15.7:0/4103850650 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f74b0159910 0x7f74b0152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.818+0000 7f74b6600640 1 --2- 172.21.15.7:0/4103850650 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f74b00cddc0 0x7f74b01557c0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.818+0000 7f74b6600640 1 --2- 172.21.15.7:0/4103850650 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f74b006e810 0x7f74b0152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.818+0000 7f74b6600640 1 -- 172.21.15.7:0/4103850650 >> 172.21.15.7:0/4103850650 conn(0x7f74b006ca60 msgr2=0x7f74b015d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.818+0000 7f74b6600640 1 -- 172.21.15.7:0/4103850650 shutdown_connections 2024-09-10T21:52:19.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.818+0000 7f74b6600640 1 -- 172.21.15.7:0/4103850650 wait complete. 2024-09-10T21:52:19.835 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:52:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.997+0000 7f948ce00640 1 -- 172.21.15.7:0/1616366145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f946c004d40 msgr2=0x7f946c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.997+0000 7f948ce00640 1 --2- 172.21.15.7:0/1616366145 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f946c004d40 0x7f946c0051c0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f948000c610 tx=0x7f946c005830 comp rx=0 tx=0).stop 2024-09-10T21:52:19.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.997+0000 7f9486800640 1 -- 172.21.15.7:0/1616366145 reap_dead start 2024-09-10T21:52:19.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.997+0000 7f948ce00640 1 -- 172.21.15.7:0/1616366145 shutdown_connections 2024-09-10T21:52:19.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.997+0000 7f948ce00640 1 -- 172.21.15.7:0/1616366145 >> 172.21.15.7:0/1616366145 conn(0x7f948806ca60 msgr2=0x7f948806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:19.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.997+0000 7f948ce00640 1 -- 172.21.15.7:0/1616366145 shutdown_connections 2024-09-10T21:52:19.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.997+0000 7f948ce00640 1 -- 172.21.15.7:0/1616366145 wait complete. 2024-09-10T21:52:19.999 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.999+0000 7f948ce00640 1 Processor -- start 2024-09-10T21:52:20.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:19.999+0000 7f948ce00640 1 -- start start 2024-09-10T21:52:20.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.000+0000 7f948ce00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f946c004d40 0x7f94880aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:20.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.000+0000 7f948ce00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f948814f3b0 0x7f948814f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:20.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.000+0000 7f948ce00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94880b1620 0x7f94880b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:20.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.000+0000 7f948ce00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f94880740c0 con 0x7f94880b1620 2024-09-10T21:52:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.000+0000 7f948ce00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f9488073f40 con 0x7f946c004d40 2024-09-10T21:52:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.000+0000 7f948ce00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f9488074240 con 0x7f948814f3b0 2024-09-10T21:52:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.000+0000 7f9486800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94880b1620 0x7f94880b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.000+0000 7f9486800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94880b1620 0x7f94880b1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35080/0 (socket says 172.21.15.7:35080) 2024-09-10T21:52:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.000+0000 7f9486800640 1 -- 172.21.15.7:0/428195971 learned_addr learned my addr 172.21.15.7:0/428195971 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.000+0000 7f9485e00640 1 --2- 172.21.15.7:0/428195971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f946c004d40 0x7f94880aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.000+0000 7f947d400640 1 --2- 172.21.15.7:0/428195971 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f948814f3b0 0x7f948814f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.001+0000 7f9486800640 1 -- 172.21.15.7:0/428195971 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f948814f3b0 msgr2=0x7f948814f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.001+0000 7f9486800640 1 --2- 172.21.15.7:0/428195971 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f948814f3b0 0x7f948814f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.001+0000 7f9486800640 1 -- 172.21.15.7:0/428195971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f946c004d40 msgr2=0x7f94880aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:20.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.001+0000 7f9486800640 1 --2- 172.21.15.7:0/428195971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f946c004d40 0x7f94880aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:20.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.001+0000 7f9486800640 1 -- 172.21.15.7:0/428195971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94881f49e0 con 0x7f94880b1620 2024-09-10T21:52:20.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.001+0000 7f9485e00640 1 --2- 172.21.15.7:0/428195971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f946c004d40 0x7f94880aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:52:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.001+0000 7f9486800640 1 --2- 172.21.15.7:0/428195971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94880b1620 0x7f94880b1aa0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f948002b490 tx=0x7f9480001ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.001+0000 7f947ea00640 1 -- 172.21.15.7:0/428195971 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9480002050 con 0x7f94880b1620 2024-09-10T21:52:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.001+0000 7f947ea00640 1 -- 172.21.15.7:0/428195971 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f948002b9b0 con 0x7f94880b1620 2024-09-10T21:52:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.001+0000 7f947ea00640 1 -- 172.21.15.7:0/428195971 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948002be00 con 0x7f94880b1620 2024-09-10T21:52:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.001+0000 7f948ce00640 1 -- 172.21.15.7:0/428195971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94881f4ca0 con 0x7f94880b1620 2024-09-10T21:52:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.001+0000 7f948ce00640 1 -- 172.21.15.7:0/428195971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94881f5260 con 0x7f94880b1620 2024-09-10T21:52:20.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.002+0000 7f948ce00640 1 -- 172.21.15.7:0/428195971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f948815d380 con 0x7f94880b1620 2024-09-10T21:52:20.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.004+0000 7f947ea00640 1 -- 172.21.15.7:0/428195971 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f948003d3c0 con 0x7f94880b1620 2024-09-10T21:52:20.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.005+0000 7f947ea00640 1 --2- 172.21.15.7:0/428195971 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9464077fa0 0x7f946407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:20.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.005+0000 7f947ea00640 1 -- 172.21.15.7:0/428195971 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5939+0+0 (secure 0 0 0) 0x7f94800b9530 con 0x7f94880b1620 2024-09-10T21:52:20.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.005+0000 7f9485e00640 1 --2- 172.21.15.7:0/428195971 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9464077fa0 0x7f946407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:20.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.006+0000 7f9485e00640 1 --2- 172.21.15.7:0/428195971 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9464077fa0 0x7f946407a460 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f94780124a0 tx=0x7f9478005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:20.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.009+0000 7f947ea00640 1 -- 172.21.15.7:0/428195971 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9480082000 con 0x7f94880b1620 2024-09-10T21:52:20.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.238+0000 7f948ce00640 1 -- 172.21.15.7:0/428195971 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f948815bd30 con 0x7f94880b1620 2024-09-10T21:52:20.240 INFO:tasks.ceph.mon.a.smithi007.stderr:2024-09-10T21:52:20.239+0000 7feb3fa00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T21:52:20.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.241+0000 7f947ea00640 1 -- 172.21.15.7:0/428195971 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v116) ==== 161+0+0 (secure 0 0 0) 0x7f9480081900 con 0x7f94880b1620 2024-09-10T21:52:20.242 INFO:teuthology.orchestra.run.smithi007.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:52:20.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.244+0000 7f948ce00640 1 -- 172.21.15.7:0/428195971 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9464077fa0 msgr2=0x7f946407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:20.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.244+0000 7f948ce00640 1 --2- 172.21.15.7:0/428195971 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9464077fa0 0x7f946407a460 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f94780124a0 tx=0x7f9478005f90 comp rx=0 tx=0).stop 2024-09-10T21:52:20.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.244+0000 7f948ce00640 1 -- 172.21.15.7:0/428195971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94880b1620 msgr2=0x7f94880b1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:20.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.244+0000 7f948ce00640 1 --2- 172.21.15.7:0/428195971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94880b1620 0x7f94880b1aa0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f948002b490 tx=0x7f9480001ac0 comp rx=0 tx=0).stop 2024-09-10T21:52:20.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.244+0000 7f948ce00640 1 -- 172.21.15.7:0/428195971 shutdown_connections 2024-09-10T21:52:20.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.244+0000 7f948ce00640 1 --2- 172.21.15.7:0/428195971 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9464077fa0 0x7f946407a460 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:20.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.244+0000 7f948ce00640 1 --2- 172.21.15.7:0/428195971 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f94880b1620 0x7f94880b1aa0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:20.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.244+0000 7f948ce00640 1 --2- 172.21.15.7:0/428195971 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f948814f3b0 0x7f948814f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:20.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.244+0000 7f948ce00640 1 --2- 172.21.15.7:0/428195971 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f946c004d40 0x7f94880aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:20.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.245+0000 7f948ce00640 1 -- 172.21.15.7:0/428195971 >> 172.21.15.7:0/428195971 conn(0x7f948806ca60 msgr2=0x7f94880cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:20.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.245+0000 7f948ce00640 1 -- 172.21.15.7:0/428195971 shutdown_connections 2024-09-10T21:52:20.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.245+0000 7f948ce00640 1 -- 172.21.15.7:0/428195971 wait complete. 2024-09-10T21:52:20.252 INFO:tasks.ceph.mds.d.smithi133.stderr: -1393> 2024-09-10T21:52:12.891+0000 7f7b76400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:52:20.262 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:52:20.297 INFO:tasks.ceph.mds.d.smithi133.stdout:starting mds.d at 2024-09-10T21:52:20.427 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.426+0000 7f6e67800640 1 -- 172.21.15.7:0/2958575336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e440051c0 msgr2=0x7f6e4400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:20.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.426+0000 7f6e67800640 1 --2- 172.21.15.7:0/2958575336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e440051c0 0x7f6e4400b9c0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f6e58009ed0 tx=0x7f6e44005830 comp rx=0 tx=0).stop 2024-09-10T21:52:20.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.426+0000 7f6e67800640 1 -- 172.21.15.7:0/2958575336 shutdown_connections 2024-09-10T21:52:20.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.426+0000 7f6e67800640 1 --2- 172.21.15.7:0/2958575336 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e440051c0 0x7f6e4400b9c0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:20.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.426+0000 7f6e67800640 1 -- 172.21.15.7:0/2958575336 >> 172.21.15.7:0/2958575336 conn(0x7f6e600b1b60 msgr2=0x7f6e600b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:20.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.426+0000 7f6e67800640 1 -- 172.21.15.7:0/2958575336 shutdown_connections 2024-09-10T21:52:20.428 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.427+0000 7f6e67800640 1 -- 172.21.15.7:0/2958575336 wait complete. 2024-09-10T21:52:20.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.428+0000 7f6e67800640 1 Processor -- start 2024-09-10T21:52:20.429 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.428+0000 7f6e67800640 1 -- start start 2024-09-10T21:52:20.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.429+0000 7f6e67800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e440051c0 0x7f6e60073c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:20.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.429+0000 7f6e67800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e60156330 0x7f6e60155240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:20.431 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.429+0000 7f6e67800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e60155780 0x7f6e601f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.429+0000 7f6e67800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f6e600b5550 con 0x7f6e60155780 2024-09-10T21:52:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.429+0000 7f6e67800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f6e600cf5e0 con 0x7f6e440051c0 2024-09-10T21:52:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.429+0000 7f6e67800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f6e600b56d0 con 0x7f6e60156330 2024-09-10T21:52:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.430+0000 7f6e64e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e440051c0 0x7f6e60073c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.430+0000 7f6e64e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e440051c0 0x7f6e60073c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.133:3300/0 says I am v2:172.21.15.7:50728/0 (socket says 172.21.15.7:50728) 2024-09-10T21:52:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.430+0000 7f6e64e00640 1 -- 172.21.15.7:0/1042276993 learned_addr learned my addr 172.21.15.7:0/1042276993 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.430+0000 7f6e65800640 1 --2- 172.21.15.7:0/1042276993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e60155780 0x7f6e601f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.430+0000 7f6e57e00640 1 --2- 172.21.15.7:0/1042276993 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e60156330 0x7f6e60155240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.430+0000 7f6e64e00640 1 -- 172.21.15.7:0/1042276993 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e60156330 msgr2=0x7f6e60155240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.430+0000 7f6e64e00640 1 --2- 172.21.15.7:0/1042276993 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e60156330 0x7f6e60155240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:20.432 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.430+0000 7f6e64e00640 1 -- 172.21.15.7:0/1042276993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e60155780 msgr2=0x7f6e601f3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:20.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.430+0000 7f6e64e00640 1 --2- 172.21.15.7:0/1042276993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e60155780 0x7f6e601f3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:20.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.430+0000 7f6e64e00640 1 -- 172.21.15.7:0/1042276993 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e601544e0 con 0x7f6e440051c0 2024-09-10T21:52:20.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.430+0000 7f6e64e00640 1 --2- 172.21.15.7:0/1042276993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e440051c0 0x7f6e60073c80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6e50006680 tx=0x7f6e500039c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:20.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.431+0000 7f6e5d600640 1 -- 172.21.15.7:0/1042276993 <== mon.1 v2:172.21.15.133:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e50011700 con 0x7f6e440051c0 2024-09-10T21:52:20.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.431+0000 7f6e5d600640 1 -- 172.21.15.7:0/1042276993 <== mon.1 v2:172.21.15.133:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e50016020 con 0x7f6e440051c0 2024-09-10T21:52:20.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.431+0000 7f6e5d600640 1 -- 172.21.15.7:0/1042276993 <== mon.1 v2:172.21.15.133:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e50011ac0 con 0x7f6e440051c0 2024-09-10T21:52:20.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.431+0000 7f6e67800640 1 -- 172.21.15.7:0/1042276993 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e601f52d0 con 0x7f6e440051c0 2024-09-10T21:52:20.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.431+0000 7f6e67800640 1 -- 172.21.15.7:0/1042276993 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e60154780 con 0x7f6e440051c0 2024-09-10T21:52:20.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.432+0000 7f6e67800640 1 -- 172.21.15.7:0/1042276993 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e60074be0 con 0x7f6e440051c0 2024-09-10T21:52:20.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.433+0000 7f6e5d600640 1 -- 172.21.15.7:0/1042276993 <== mon.1 v2:172.21.15.133:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f6e50011c60 con 0x7f6e440051c0 2024-09-10T21:52:20.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.434+0000 7f6e5d600640 1 --2- 172.21.15.7:0/1042276993 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e3c077fa0 0x7f6e3c07a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:20.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.434+0000 7f6e5d600640 1 -- 172.21.15.7:0/1042276993 <== mon.1 v2:172.21.15.133:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6025+0+0 (secure 0 0 0) 0x7f6e5009aa80 con 0x7f6e440051c0 2024-09-10T21:52:20.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.434+0000 7f6e57e00640 1 --2- 172.21.15.7:0/1042276993 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e3c077fa0 0x7f6e3c07a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:20.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.435+0000 7f6e57e00640 1 --2- 172.21.15.7:0/1042276993 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e3c077fa0 0x7f6e3c07a460 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6e48000c00 tx=0x7f6e48006b70 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:20.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.437+0000 7f6e5d600640 1 -- 172.21.15.7:0/1042276993 <== mon.1 v2:172.21.15.133:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6e60074be0 con 0x7f6e440051c0 2024-09-10T21:52:20.677 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:20.676+0000 7f6e67800640 1 -- 172.21.15.7:0/1042276993 --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6e60154ea0 con 0x7f6e440051c0 2024-09-10T21:52:21.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.246+0000 7f6e5d600640 1 -- 172.21.15.7:0/1042276993 <== mon.1 v2:172.21.15.133:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v117) ==== 104+0+0 (secure 0 0 0) 0x7f6e50020030 con 0x7f6e440051c0 2024-09-10T21:52:21.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.249+0000 7f6e67800640 1 -- 172.21.15.7:0/1042276993 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e3c077fa0 msgr2=0x7f6e3c07a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:21.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.249+0000 7f6e67800640 1 --2- 172.21.15.7:0/1042276993 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e3c077fa0 0x7f6e3c07a460 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f6e48000c00 tx=0x7f6e48006b70 comp rx=0 tx=0).stop 2024-09-10T21:52:21.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.249+0000 7f6e67800640 1 -- 172.21.15.7:0/1042276993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e440051c0 msgr2=0x7f6e60073c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:21.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.249+0000 7f6e67800640 1 --2- 172.21.15.7:0/1042276993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e440051c0 0x7f6e60073c80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6e50006680 tx=0x7f6e500039c0 comp rx=0 tx=0).stop 2024-09-10T21:52:21.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.250+0000 7f6e67800640 1 -- 172.21.15.7:0/1042276993 shutdown_connections 2024-09-10T21:52:21.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.250+0000 7f6e67800640 1 --2- 172.21.15.7:0/1042276993 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6e3c077fa0 0x7f6e3c07a460 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:21.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.250+0000 7f6e67800640 1 --2- 172.21.15.7:0/1042276993 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f6e60155780 0x7f6e601f3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:21.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.250+0000 7f6e67800640 1 --2- 172.21.15.7:0/1042276993 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f6e60156330 0x7f6e60155240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:21.251 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.250+0000 7f6e67800640 1 --2- 172.21.15.7:0/1042276993 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6e440051c0 0x7f6e60073c80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:21.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.250+0000 7f6e67800640 1 -- 172.21.15.7:0/1042276993 >> 172.21.15.7:0/1042276993 conn(0x7f6e600b1b60 msgr2=0x7f6e600adb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:21.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.250+0000 7f6e67800640 1 -- 172.21.15.7:0/1042276993 shutdown_connections 2024-09-10T21:52:21.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.250+0000 7f6e67800640 1 -- 172.21.15.7:0/1042276993 wait complete. 2024-09-10T21:52:21.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:52:21.435 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.434+0000 7f93c7a00640 1 -- 172.21.15.7:0/1019764672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c0145610 msgr2=0x7f93c0073280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:21.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.434+0000 7f93c7a00640 1 --2- 172.21.15.7:0/1019764672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c0145610 0x7f93c0073280 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f93b0005fd0 tx=0x7f93b002a4a0 comp rx=0 tx=0).stop 2024-09-10T21:52:21.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.434+0000 7f93c7a00640 1 -- 172.21.15.7:0/1019764672 shutdown_connections 2024-09-10T21:52:21.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.434+0000 7f93c7a00640 1 --2- 172.21.15.7:0/1019764672 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93c006ff60 0x7f93c00ce150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:21.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.434+0000 7f93c7a00640 1 --2- 172.21.15.7:0/1019764672 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c0145610 0x7f93c0073280 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:21.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.434+0000 7f93c7a00640 1 -- 172.21.15.7:0/1019764672 >> 172.21.15.7:0/1019764672 conn(0x7f93c006ce50 msgr2=0x7f93c006d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:21.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.434+0000 7f93c7a00640 1 -- 172.21.15.7:0/1019764672 shutdown_connections 2024-09-10T21:52:21.436 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.434+0000 7f93c7a00640 1 -- 172.21.15.7:0/1019764672 wait complete. 2024-09-10T21:52:21.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.436+0000 7f93c7a00640 1 Processor -- start 2024-09-10T21:52:21.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.436+0000 7f93c7a00640 1 -- start start 2024-09-10T21:52:21.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.437+0000 7f93c7a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c006ff60 0x7f93c015e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:21.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.437+0000 7f93c7a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93c00ccc30 0x7f93c015ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:21.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.437+0000 7f93c7a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93c0145610 0x7f93c01f5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:21.439 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.437+0000 7f93c7a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f93c00cd410 con 0x7f93c006ff60 2024-09-10T21:52:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.437+0000 7f93c7a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f93c00cd290 con 0x7f93c0145610 2024-09-10T21:52:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.437+0000 7f93c7a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f93c00cd590 con 0x7f93c00ccc30 2024-09-10T21:52:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.437+0000 7f93c5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c006ff60 0x7f93c015e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.437+0000 7f93c5000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c006ff60 0x7f93c015e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35122/0 (socket says 172.21.15.7:35122) 2024-09-10T21:52:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.437+0000 7f93c5000640 1 -- 172.21.15.7:0/462981472 learned_addr learned my addr 172.21.15.7:0/462981472 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.437+0000 7f93c5a00640 1 --2- 172.21.15.7:0/462981472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93c0145610 0x7f93c01f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.437+0000 7f93bfe00640 1 --2- 172.21.15.7:0/462981472 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93c00ccc30 0x7f93c015ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.438+0000 7f93c5000640 1 -- 172.21.15.7:0/462981472 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93c00ccc30 msgr2=0x7f93c015ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.438+0000 7f93c5000640 1 --2- 172.21.15.7:0/462981472 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93c00ccc30 0x7f93c015ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.438+0000 7f93c5000640 1 -- 172.21.15.7:0/462981472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93c0145610 msgr2=0x7f93c01f5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.438+0000 7f93c5000640 1 --2- 172.21.15.7:0/462981472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93c0145610 0x7f93c01f5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:21.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.438+0000 7f93c5000640 1 -- 172.21.15.7:0/462981472 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c00c5060 con 0x7f93c006ff60 2024-09-10T21:52:21.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.438+0000 7f93c5000640 1 --2- 172.21.15.7:0/462981472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c006ff60 0x7f93c015e7c0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f93b0034040 tx=0x7f93b00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:21.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.438+0000 7f93bcc00640 1 -- 172.21.15.7:0/462981472 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b003e030 con 0x7f93c006ff60 2024-09-10T21:52:21.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.438+0000 7f93bcc00640 1 -- 172.21.15.7:0/462981472 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93b00047a0 con 0x7f93c006ff60 2024-09-10T21:52:21.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.438+0000 7f93bcc00640 1 -- 172.21.15.7:0/462981472 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0035040 con 0x7f93c006ff60 2024-09-10T21:52:21.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.438+0000 7f93c7a00640 1 -- 172.21.15.7:0/462981472 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c00cbbf0 con 0x7f93c006ff60 2024-09-10T21:52:21.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.439+0000 7f93c7a00640 1 -- 172.21.15.7:0/462981472 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c0144060 con 0x7f93c006ff60 2024-09-10T21:52:21.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.440+0000 7f93c7a00640 1 -- 172.21.15.7:0/462981472 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93c01442a0 con 0x7f93c006ff60 2024-09-10T21:52:21.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.441+0000 7f93bcc00640 1 -- 172.21.15.7:0/462981472 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f93b0004b90 con 0x7f93c006ff60 2024-09-10T21:52:21.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.441+0000 7f93bcc00640 1 --2- 172.21.15.7:0/462981472 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f93a0077f70 0x7f93a007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:21.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.442+0000 7f93bcc00640 1 -- 172.21.15.7:0/462981472 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 6025+0+0 (secure 0 0 0) 0x7f93b00ba8e0 con 0x7f93c006ff60 2024-09-10T21:52:21.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.442+0000 7f93bfe00640 1 --2- 172.21.15.7:0/462981472 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f93a0077f70 0x7f93a007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:21.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.443+0000 7f93bfe00640 1 --2- 172.21.15.7:0/462981472 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f93a0077f70 0x7f93a007a430 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f93ac005e10 tx=0x7f93ac005da0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:21.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.445+0000 7f93bcc00640 1 -- 172.21.15.7:0/462981472 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f93b0082bb0 con 0x7f93c006ff60 2024-09-10T21:52:21.639 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:21.638+0000 7f93c7a00640 1 -- 172.21.15.7:0/462981472 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f93c00b0e00 con 0x7f93c006ff60 2024-09-10T21:52:22.245 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.245+0000 7f93bcc00640 1 -- 172.21.15.7:0/462981472 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v120) ==== 181+0+0 (secure 0 0 0) 0x7f93b00875e0 con 0x7f93c006ff60 2024-09-10T21:52:22.246 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:52:22.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.248+0000 7f93c7a00640 1 -- 172.21.15.7:0/462981472 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f93a0077f70 msgr2=0x7f93a007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:22.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.248+0000 7f93c7a00640 1 --2- 172.21.15.7:0/462981472 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f93a0077f70 0x7f93a007a430 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f93ac005e10 tx=0x7f93ac005da0 comp rx=0 tx=0).stop 2024-09-10T21:52:22.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.248+0000 7f93c7a00640 1 -- 172.21.15.7:0/462981472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c006ff60 msgr2=0x7f93c015e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:22.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.248+0000 7f93c7a00640 1 --2- 172.21.15.7:0/462981472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c006ff60 0x7f93c015e7c0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f93b0034040 tx=0x7f93b00039e0 comp rx=0 tx=0).stop 2024-09-10T21:52:22.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.248+0000 7f93c7a00640 1 -- 172.21.15.7:0/462981472 shutdown_connections 2024-09-10T21:52:22.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.248+0000 7f93c7a00640 1 --2- 172.21.15.7:0/462981472 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f93a0077f70 0x7f93a007a430 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:22.249 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.248+0000 7f93c7a00640 1 --2- 172.21.15.7:0/462981472 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f93c0145610 0x7f93c01f5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.248+0000 7f93c7a00640 1 --2- 172.21.15.7:0/462981472 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f93c00ccc30 0x7f93c015ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.248+0000 7f93c7a00640 1 --2- 172.21.15.7:0/462981472 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f93c006ff60 0x7f93c015e7c0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.248+0000 7f93c7a00640 1 -- 172.21.15.7:0/462981472 >> 172.21.15.7:0/462981472 conn(0x7f93c006ce50 msgr2=0x7f93c00ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.248+0000 7f93c7a00640 1 -- 172.21.15.7:0/462981472 shutdown_connections 2024-09-10T21:52:22.250 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.248+0000 7f93c7a00640 1 -- 172.21.15.7:0/462981472 wait complete. 2024-09-10T21:52:22.275 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:52:22.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.441+0000 7fc358e00640 1 -- 172.21.15.7:0/359864213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3540af240 msgr2=0x7fc354160220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:22.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.441+0000 7fc358e00640 1 --2- 172.21.15.7:0/359864213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3540af240 0x7fc354160220 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fc354070a50 tx=0x7fc33c02a4b0 comp rx=0 tx=0).stop 2024-09-10T21:52:22.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.441+0000 7fc358e00640 1 -- 172.21.15.7:0/359864213 shutdown_connections 2024-09-10T21:52:22.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.441+0000 7fc358e00640 1 --2- 172.21.15.7:0/359864213 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3540af240 0x7fc354160220 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:22.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.442+0000 7fc358e00640 1 --2- 172.21.15.7:0/359864213 >> v2:172.21.15.133:3300/0 conn(0x7fc35415b960 0x7fc3540acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:22.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.442+0000 7fc358e00640 1 -- 172.21.15.7:0/359864213 >> 172.21.15.7:0/359864213 conn(0x7fc35406ce80 msgr2=0x7fc35406d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:22.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.442+0000 7fc358e00640 1 -- 172.21.15.7:0/359864213 shutdown_connections 2024-09-10T21:52:22.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.442+0000 7fc358e00640 1 -- 172.21.15.7:0/359864213 wait complete. 2024-09-10T21:52:22.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.443+0000 7fc358e00640 1 Processor -- start 2024-09-10T21:52:22.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.443+0000 7fc358e00640 1 -- start start 2024-09-10T21:52:22.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.444+0000 7fc358e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3540af240 0x7fc354150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:22.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.444+0000 7fc358e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc35415b960 0x7fc35415dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:22.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.444+0000 7fc358e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc35415bd40 0x7fc354156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:22.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.444+0000 7fc358e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc3540b2e40 con 0x7fc3540af240 2024-09-10T21:52:22.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.444+0000 7fc358e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc3540d09f0 con 0x7fc35415b960 2024-09-10T21:52:22.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.444+0000 7fc358e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc3540b2fc0 con 0x7fc35415bd40 2024-09-10T21:52:22.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.444+0000 7fc353400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3540af240 0x7fc354150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:22.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.445+0000 7fc353400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3540af240 0x7fc354150610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35132/0 (socket says 172.21.15.7:35132) 2024-09-10T21:52:22.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.445+0000 7fc353400640 1 -- 172.21.15.7:0/3982984619 learned_addr learned my addr 172.21.15.7:0/3982984619 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:22.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.445+0000 7fc34aa00640 1 --2- 172.21.15.7:0/3982984619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc35415b960 0x7fc35415dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:22.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.445+0000 7fc353e00640 1 --2- 172.21.15.7:0/3982984619 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc35415bd40 0x7fc354156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:22.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.445+0000 7fc353400640 1 -- 172.21.15.7:0/3982984619 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc35415bd40 msgr2=0x7fc354156ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:22.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.445+0000 7fc353400640 1 --2- 172.21.15.7:0/3982984619 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc35415bd40 0x7fc354156ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:22.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.445+0000 7fc353400640 1 -- 172.21.15.7:0/3982984619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc35415b960 msgr2=0x7fc35415dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:22.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.445+0000 7fc353400640 1 --2- 172.21.15.7:0/3982984619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc35415b960 0x7fc35415dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.445+0000 7fc353400640 1 -- 172.21.15.7:0/3982984619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3541f2240 con 0x7fc3540af240 2024-09-10T21:52:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.445+0000 7fc34aa00640 1 --2- 172.21.15.7:0/3982984619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc35415b960 0x7fc35415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.445+0000 7fc353400640 1 --2- 172.21.15.7:0/3982984619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3540af240 0x7fc354150610 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fc34400c920 tx=0x7fc34400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.445+0000 7fc34be00640 1 -- 172.21.15.7:0/3982984619 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344007c10 con 0x7fc3540af240 2024-09-10T21:52:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.446+0000 7fc34be00640 1 -- 172.21.15.7:0/3982984619 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc34400f450 con 0x7fc3540af240 2024-09-10T21:52:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.446+0000 7fc34be00640 1 -- 172.21.15.7:0/3982984619 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34400f8a0 con 0x7fc3540af240 2024-09-10T21:52:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.446+0000 7fc358e00640 1 -- 172.21.15.7:0/3982984619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3541f2520 con 0x7fc3540af240 2024-09-10T21:52:22.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.446+0000 7fc358e00640 1 -- 172.21.15.7:0/3982984619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3541f2ae0 con 0x7fc3540af240 2024-09-10T21:52:22.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.447+0000 7fc358e00640 1 -- 172.21.15.7:0/3982984619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc354160220 con 0x7fc3540af240 2024-09-10T21:52:22.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.448+0000 7fc34be00640 1 -- 172.21.15.7:0/3982984619 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc34400fa40 con 0x7fc3540af240 2024-09-10T21:52:22.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.449+0000 7fc34be00640 1 --2- 172.21.15.7:0/3982984619 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc330077fa0 0x7fc33007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:22.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.449+0000 7fc34be00640 1 -- 172.21.15.7:0/3982984619 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5587+0+0 (secure 0 0 0) 0x7fc34409a5b0 con 0x7fc3540af240 2024-09-10T21:52:22.450 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.450+0000 7fc34aa00640 1 --2- 172.21.15.7:0/3982984619 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc330077fa0 0x7fc33007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:22.451 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.450+0000 7fc34aa00640 1 --2- 172.21.15.7:0/3982984619 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc330077fa0 0x7fc33007a460 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc33c034000 tx=0x7fc33c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:22.454 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.453+0000 7fc34be00640 1 -- 172.21.15.7:0/3982984619 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fc344014030 con 0x7fc3540af240 2024-09-10T21:52:22.651 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:22.650+0000 7fc358e00640 1 -- 172.21.15.7:0/3982984619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc35415af10 con 0x7fc3540af240 2024-09-10T21:52:23.252 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.251+0000 7fc34be00640 1 -- 172.21.15.7:0/3982984619 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v121) ==== 169+0+0 (secure 0 0 0) 0x7fc344063080 con 0x7fc3540af240 2024-09-10T21:52:23.252 INFO:teuthology.orchestra.run.smithi007.stderr:pool 'cephfs_data' removed 2024-09-10T21:52:23.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.255+0000 7fc358e00640 1 -- 172.21.15.7:0/3982984619 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc330077fa0 msgr2=0x7fc33007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:23.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.255+0000 7fc358e00640 1 --2- 172.21.15.7:0/3982984619 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc330077fa0 0x7fc33007a460 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fc33c034000 tx=0x7fc33c004730 comp rx=0 tx=0).stop 2024-09-10T21:52:23.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.255+0000 7fc358e00640 1 -- 172.21.15.7:0/3982984619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3540af240 msgr2=0x7fc354150610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:23.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.255+0000 7fc358e00640 1 --2- 172.21.15.7:0/3982984619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3540af240 0x7fc354150610 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fc34400c920 tx=0x7fc34400cde0 comp rx=0 tx=0).stop 2024-09-10T21:52:23.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.255+0000 7fc358e00640 1 -- 172.21.15.7:0/3982984619 shutdown_connections 2024-09-10T21:52:23.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.255+0000 7fc358e00640 1 --2- 172.21.15.7:0/3982984619 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc330077fa0 0x7fc33007a460 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:23.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.255+0000 7fc358e00640 1 --2- 172.21.15.7:0/3982984619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc35415b960 0x7fc35415dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:23.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.255+0000 7fc358e00640 1 --2- 172.21.15.7:0/3982984619 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc35415bd40 0x7fc354156ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:23.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.255+0000 7fc358e00640 1 --2- 172.21.15.7:0/3982984619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc3540af240 0x7fc354150610 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:23.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.255+0000 7fc358e00640 1 -- 172.21.15.7:0/3982984619 >> 172.21.15.7:0/3982984619 conn(0x7fc35406ce80 msgr2=0x7fc3540b3420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:23.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.256+0000 7fc358e00640 1 -- 172.21.15.7:0/3982984619 shutdown_connections 2024-09-10T21:52:23.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.256+0000 7fc358e00640 1 -- 172.21.15.7:0/3982984619 wait complete. 2024-09-10T21:52:23.277 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 27 2024-09-10T21:52:23.440 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.439+0000 7f4681c00640 1 -- 172.21.15.7:0/3530502688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f467c15d150 msgr2=0x7f467c06f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:23.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.439+0000 7f4681c00640 1 --2- 172.21.15.7:0/3530502688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f467c15d150 0x7f467c06f430 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4664009830 tx=0x7f466402a480 comp rx=0 tx=0).stop 2024-09-10T21:52:23.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.439+0000 7f4681c00640 1 -- 172.21.15.7:0/3530502688 shutdown_connections 2024-09-10T21:52:23.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.439+0000 7f4681c00640 1 --2- 172.21.15.7:0/3530502688 >> v2:172.21.15.133:3300/0 conn(0x7f467c145950 0x7f467c074050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:23.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.439+0000 7f4681c00640 1 --2- 172.21.15.7:0/3530502688 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f467c15d150 0x7f467c06f430 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:23.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.439+0000 7f4681c00640 1 -- 172.21.15.7:0/3530502688 >> 172.21.15.7:0/3530502688 conn(0x7f467c0cc6b0 msgr2=0x7f467c0ccac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:23.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.439+0000 7f4681c00640 1 -- 172.21.15.7:0/3530502688 shutdown_connections 2024-09-10T21:52:23.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.439+0000 7f4681c00640 1 -- 172.21.15.7:0/3530502688 wait complete. 2024-09-10T21:52:23.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.441+0000 7f4681c00640 1 Processor -- start 2024-09-10T21:52:23.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.441+0000 7f4681c00640 1 -- start start 2024-09-10T21:52:23.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.442+0000 7f4681c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f467c145570 0x7f467c1f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:23.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.442+0000 7f4681c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f467c145950 0x7f467c1ef850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:23.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.442+0000 7f4681c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f467c15d150 0x7f467c1efd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:23.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.442+0000 7f4681c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f467c0ca610 con 0x7f467c145570 2024-09-10T21:52:23.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.442+0000 7f4681c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f467c0ca490 con 0x7f467c145950 2024-09-10T21:52:23.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.442+0000 7f4681c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f467c0ca790 con 0x7f467c15d150 2024-09-10T21:52:23.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.442+0000 7f467be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f467c145570 0x7f467c1f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:23.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.442+0000 7f467be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f467c145570 0x7f467c1f3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:35140/0 (socket says 172.21.15.7:35140) 2024-09-10T21:52:23.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.442+0000 7f467be00640 1 -- 172.21.15.7:0/20201385 learned_addr learned my addr 172.21.15.7:0/20201385 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:23.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.443+0000 7f4673e00640 1 --2- 172.21.15.7:0/20201385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f467c145950 0x7f467c1ef850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:23.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.443+0000 7f4681200640 1 --2- 172.21.15.7:0/20201385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f467c15d150 0x7f467c1efd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:23.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.443+0000 7f467be00640 1 -- 172.21.15.7:0/20201385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f467c15d150 msgr2=0x7f467c1efd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:23.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.443+0000 7f467be00640 1 --2- 172.21.15.7:0/20201385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f467c15d150 0x7f467c1efd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:23.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.443+0000 7f467be00640 1 -- 172.21.15.7:0/20201385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f467c145950 msgr2=0x7f467c1ef850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:23.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.443+0000 7f467be00640 1 --2- 172.21.15.7:0/20201385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f467c145950 0x7f467c1ef850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:23.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.443+0000 7f467be00640 1 -- 172.21.15.7:0/20201385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f467c1560f0 con 0x7f467c145570 2024-09-10T21:52:23.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.443+0000 7f467be00640 1 --2- 172.21.15.7:0/20201385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f467c145570 0x7f467c1f3850 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f466c00c490 tx=0x7f466c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:23.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.444+0000 7f4678c00640 1 -- 172.21.15.7:0/20201385 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c016020 con 0x7f467c145570 2024-09-10T21:52:23.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.444+0000 7f4681c00640 1 -- 172.21.15.7:0/20201385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f467c136e20 con 0x7f467c145570 2024-09-10T21:52:23.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.444+0000 7f4678c00640 1 -- 172.21.15.7:0/20201385 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f466c012940 con 0x7f467c145570 2024-09-10T21:52:23.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.444+0000 7f4678c00640 1 -- 172.21.15.7:0/20201385 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466c012d90 con 0x7f467c145570 2024-09-10T21:52:23.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.444+0000 7f4681c00640 1 -- 172.21.15.7:0/20201385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f467c0b3c90 con 0x7f467c145570 2024-09-10T21:52:23.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.445+0000 7f4681c00640 1 -- 172.21.15.7:0/20201385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f467c072800 con 0x7f467c145570 2024-09-10T21:52:23.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.446+0000 7f4678c00640 1 -- 172.21.15.7:0/20201385 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f466c01a3b0 con 0x7f467c145570 2024-09-10T21:52:23.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.447+0000 7f4678c00640 1 --2- 172.21.15.7:0/20201385 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4658077f70 0x7f465807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:23.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.447+0000 7f4678c00640 1 -- 172.21.15.7:0/20201385 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7f466c0999c0 con 0x7f467c145570 2024-09-10T21:52:23.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.447+0000 7f4673e00640 1 --2- 172.21.15.7:0/20201385 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4658077f70 0x7f465807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:23.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.448+0000 7f4673e00640 1 --2- 172.21.15.7:0/20201385 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4658077f70 0x7f465807a430 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4664034000 tx=0x7f4664004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:23.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.451+0000 7f4678c00640 1 -- 172.21.15.7:0/20201385 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f467c072800 con 0x7f467c145570 2024-09-10T21:52:23.650 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.649+0000 7f4681c00640 1 -- 172.21.15.7:0/20201385 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "config reset", "num": 27} v 0) -- 0x7f467c15de20 con 0x7f467c145570 2024-09-10T21:52:23.654 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.653+0000 7f4678c00640 1 -- 172.21.15.7:0/20201385 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 27}]=0 v41) ==== 71+0+0 (secure 0 0 0) 0x7f466c020030 con 0x7f467c145570 2024-09-10T21:52:23.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.655+0000 7f4681c00640 1 -- 172.21.15.7:0/20201385 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4658077f70 msgr2=0x7f465807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:23.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.655+0000 7f4681c00640 1 --2- 172.21.15.7:0/20201385 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4658077f70 0x7f465807a430 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f4664034000 tx=0x7f4664004730 comp rx=0 tx=0).stop 2024-09-10T21:52:23.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.655+0000 7f4681c00640 1 -- 172.21.15.7:0/20201385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f467c145570 msgr2=0x7f467c1f3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:23.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.655+0000 7f4681c00640 1 --2- 172.21.15.7:0/20201385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f467c145570 0x7f467c1f3850 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f466c00c490 tx=0x7f466c00c950 comp rx=0 tx=0).stop 2024-09-10T21:52:23.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.656+0000 7f4681c00640 1 -- 172.21.15.7:0/20201385 shutdown_connections 2024-09-10T21:52:23.657 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.656+0000 7f4681c00640 1 --2- 172.21.15.7:0/20201385 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4658077f70 0x7f465807a430 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:23.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.656+0000 7f4681c00640 1 --2- 172.21.15.7:0/20201385 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f467c145950 0x7f467c1ef850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:23.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.656+0000 7f4681c00640 1 --2- 172.21.15.7:0/20201385 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f467c15d150 0x7f467c1efd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:23.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.656+0000 7f4681c00640 1 --2- 172.21.15.7:0/20201385 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f467c145570 0x7f467c1f3850 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:23.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.656+0000 7f4681c00640 1 -- 172.21.15.7:0/20201385 >> 172.21.15.7:0/20201385 conn(0x7f467c0cc6b0 msgr2=0x7f467c0cb760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:23.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.656+0000 7f4681c00640 1 -- 172.21.15.7:0/20201385 shutdown_connections 2024-09-10T21:52:23.658 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.656+0000 7f4681c00640 1 -- 172.21.15.7:0/20201385 wait complete. 2024-09-10T21:52:23.673 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan' 2024-09-10T21:52:23.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.837+0000 7f02bec00640 1 -- 172.21.15.7:0/938093542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b80b01e0 msgr2=0x7f02b80d05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:23.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.837+0000 7f02bec00640 1 --2- 172.21.15.7:0/938093542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b80b01e0 0x7f02b80d05e0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f02ac005fd0 tx=0x7f02ac02a540 comp rx=0 tx=0).stop 2024-09-10T21:52:23.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.837+0000 7f02bec00640 1 -- 172.21.15.7:0/938093542 shutdown_connections 2024-09-10T21:52:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.837+0000 7f02bec00640 1 --2- 172.21.15.7:0/938093542 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02b80b05c0 0x7f02b806ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.837+0000 7f02bec00640 1 --2- 172.21.15.7:0/938093542 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b80b01e0 0x7f02b80d05e0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.837+0000 7f02bec00640 1 -- 172.21.15.7:0/938093542 >> 172.21.15.7:0/938093542 conn(0x7f02b80d0d70 msgr2=0x7f02b80d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.837+0000 7f02bec00640 1 -- 172.21.15.7:0/938093542 shutdown_connections 2024-09-10T21:52:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.837+0000 7f02bec00640 1 -- 172.21.15.7:0/938093542 wait complete. 2024-09-10T21:52:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.839+0000 7f02bec00640 1 Processor -- start 2024-09-10T21:52:23.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.839+0000 7f02bec00640 1 -- start start 2024-09-10T21:52:23.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.840+0000 7f02bec00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02b80b01e0 0x7f02b8145960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:23.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.840+0000 7f02bec00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b80d1860 0x7f02b806f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.840+0000 7f02bec00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b814c850 0x7f02b806fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.840+0000 7f02bec00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f02b80cf380 con 0x7f02b814c850 2024-09-10T21:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.840+0000 7f02bec00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f02b806d3b0 con 0x7f02b80d1860 2024-09-10T21:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.840+0000 7f02bec00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f02b80cf500 con 0x7f02b80b01e0 2024-09-10T21:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.840+0000 7f02be200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b814c850 0x7f02b806fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.840+0000 7f02be200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b814c850 0x7f02b806fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39988/0 (socket says 172.21.15.7:39988) 2024-09-10T21:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.840+0000 7f02be200640 1 -- 172.21.15.7:0/1016866643 learned_addr learned my addr 172.21.15.7:0/1016866643 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.840+0000 7f02bd800640 1 --2- 172.21.15.7:0/1016866643 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02b80b01e0 0x7f02b8145960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.840+0000 7f02bce00640 1 --2- 172.21.15.7:0/1016866643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b80d1860 0x7f02b806f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.841+0000 7f02be200640 1 -- 172.21.15.7:0/1016866643 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02b80b01e0 msgr2=0x7f02b8145960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.841+0000 7f02be200640 1 --2- 172.21.15.7:0/1016866643 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02b80b01e0 0x7f02b8145960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.841+0000 7f02be200640 1 -- 172.21.15.7:0/1016866643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b80d1860 msgr2=0x7f02b806f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:23.842 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.841+0000 7f02be200640 1 --2- 172.21.15.7:0/1016866643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b80d1860 0x7f02b806f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:23.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.841+0000 7f02be200640 1 -- 172.21.15.7:0/1016866643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b814d3b0 con 0x7f02b814c850 2024-09-10T21:52:23.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.841+0000 7f02bd800640 1 --2- 172.21.15.7:0/1016866643 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02b80b01e0 0x7f02b8145960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:23.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.841+0000 7f02be200640 1 --2- 172.21.15.7:0/1016866643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b814c850 0x7f02b806fb10 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f02b000c6f0 tx=0x7f02b000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:23.843 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.841+0000 7f02b5600640 1 -- 172.21.15.7:0/1016866643 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0017070 con 0x7f02b814c850 2024-09-10T21:52:23.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.841+0000 7f02b5600640 1 -- 172.21.15.7:0/1016866643 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02b0003c90 con 0x7f02b814c850 2024-09-10T21:52:23.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.841+0000 7f02b5600640 1 -- 172.21.15.7:0/1016866643 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0003e60 con 0x7f02b814c850 2024-09-10T21:52:23.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.841+0000 7f02bec00640 1 -- 172.21.15.7:0/1016866643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b80b3d40 con 0x7f02b814c850 2024-09-10T21:52:23.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.841+0000 7f02bec00640 1 -- 172.21.15.7:0/1016866643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b814fec0 con 0x7f02b814c850 2024-09-10T21:52:23.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.843+0000 7f02bec00640 1 -- 172.21.15.7:0/1016866643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b80cf9c0 con 0x7f02b814c850 2024-09-10T21:52:23.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.844+0000 7f02b5600640 1 -- 172.21.15.7:0/1016866643 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f02b00052d0 con 0x7f02b814c850 2024-09-10T21:52:23.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.845+0000 7f02b5600640 1 --2- 172.21.15.7:0/1016866643 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0298077fa0 0x7f029807a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:23.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.845+0000 7f02b5600640 1 -- 172.21.15.7:0/1016866643 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7f02b00992a0 con 0x7f02b814c850 2024-09-10T21:52:23.846 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.846+0000 7f02bd800640 1 --2- 172.21.15.7:0/1016866643 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0298077fa0 0x7f029807a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:23.847 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.847+0000 7f02bd800640 1 --2- 172.21.15.7:0/1016866643 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0298077fa0 0x7f029807a460 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f02ac033000 tx=0x7f02ac004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:23.850 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:23.849+0000 7f02b5600640 1 -- 172.21.15.7:0/1016866643 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f02b0061a30 con 0x7f02b814c850 2024-09-10T21:52:24.120 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:24.119+0000 7f02bec00640 1 -- 172.21.15.7:0/1016866643 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]} v 0) -- 0x7f02b8074ac0 con 0x7f02b814c850 2024-09-10T21:52:24.253 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:24.253+0000 7f02b5600640 1 -- 172.21.15.7:0/1016866643 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_orphan_scan"]}]=0 v477) ==== 144+0+0 (secure 0 0 0) 0x7f02b0066460 con 0x7f02b814c850 2024-09-10T21:52:24.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:24.255+0000 7f02bec00640 1 -- 172.21.15.7:0/1016866643 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0298077fa0 msgr2=0x7f029807a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:24.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:24.255+0000 7f02bec00640 1 --2- 172.21.15.7:0/1016866643 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0298077fa0 0x7f029807a460 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f02ac033000 tx=0x7f02ac004730 comp rx=0 tx=0).stop 2024-09-10T21:52:24.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:24.255+0000 7f02bec00640 1 -- 172.21.15.7:0/1016866643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b814c850 msgr2=0x7f02b806fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:24.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:24.255+0000 7f02bec00640 1 --2- 172.21.15.7:0/1016866643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b814c850 0x7f02b806fb10 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f02b000c6f0 tx=0x7f02b000cbb0 comp rx=0 tx=0).stop 2024-09-10T21:52:24.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:24.255+0000 7f02bec00640 1 -- 172.21.15.7:0/1016866643 shutdown_connections 2024-09-10T21:52:24.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:24.255+0000 7f02bec00640 1 --2- 172.21.15.7:0/1016866643 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f0298077fa0 0x7f029807a460 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:24.257 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:24.255+0000 7f02bec00640 1 --2- 172.21.15.7:0/1016866643 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f02b80d1860 0x7f02b806f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:24.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:24.255+0000 7f02bec00640 1 --2- 172.21.15.7:0/1016866643 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f02b814c850 0x7f02b806fb10 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:24.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:24.255+0000 7f02bec00640 1 --2- 172.21.15.7:0/1016866643 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f02b80b01e0 0x7f02b8145960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:24.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:24.255+0000 7f02bec00640 1 -- 172.21.15.7:0/1016866643 >> 172.21.15.7:0/1016866643 conn(0x7f02b80d0d70 msgr2=0x7f02b815efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:24.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:24.256+0000 7f02bec00640 1 -- 172.21.15.7:0/1016866643 shutdown_connections 2024-09-10T21:52:24.258 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:24.256+0000 7f02bec00640 1 -- 172.21.15.7:0/1016866643 wait complete. 2024-09-10T21:52:24.273 INFO:tasks.cephfs_test_runner:test_orphan_scan (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2024-09-10T21:52:24.274 INFO:tasks.cephfs_test_runner: 2024-09-10T21:52:24.274 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-10T21:52:24.274 INFO:tasks.cephfs_test_runner:Ran 7 tests in 588.577s 2024-09-10T21:52:24.274 INFO:tasks.cephfs_test_runner: 2024-09-10T21:52:24.275 INFO:tasks.cephfs_test_runner:OK 2024-09-10T21:52:24.275 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-10T21:52:24.300 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-10T21:52:24.310 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-10T21:52:24.310 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:52:24.310 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:52:24.346 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:52:24.346 DEBUG:teuthology.orchestra.run.smithi007:> set -ex 2024-09-10T21:52:24.346 DEBUG:teuthology.orchestra.run.smithi007:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:52:24.421 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4053402,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249604k,nr_inodes=812401,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6499212k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-10T21:52:24.422 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:52:24.422 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:52:24.461 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:52:24.461 DEBUG:teuthology.orchestra.run.smithi133:> set -ex 2024-09-10T21:52:24.462 DEBUG:teuthology.orchestra.run.smithi133:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:52:24.540 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-10T21:52:24.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:24.540 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:52:24.582 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:52:24.582 INFO:teuthology.orchestra.run.smithi007.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-10T21:52:24.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:24.583 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:52:24.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:24.680 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-10T21:52:24.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:24.937 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-09-10T21:52:25.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:25.232 DEBUG:teuthology.orchestra.run.smithi007:> ip netns list 2024-09-10T21:52:25.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:25.268 DEBUG:teuthology.orchestra.run.smithi007:> sudo ip link delete ceph-brx 2024-09-10T21:52:25.370 INFO:teuthology.orchestra.run.smithi007.stderr:Cannot find device "ceph-brx" 2024-09-10T21:52:25.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:52:25.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:25.374 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-09-10T21:52:25.414 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-10T21:52:25.415 INFO:teuthology.orchestra.run.smithi133.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-10T21:52:25.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:25.415 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T21:52:25.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:25.516 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-10T21:52:25.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:25.801 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2024-09-10T21:52:26.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:26.119 DEBUG:teuthology.orchestra.run.smithi133:> ip netns list 2024-09-10T21:52:26.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:52:26.155 DEBUG:teuthology.orchestra.run.smithi133:> sudo ip link delete ceph-brx 2024-09-10T21:52:26.254 INFO:teuthology.orchestra.run.smithi133.stderr:Cannot find device "ceph-brx" 2024-09-10T21:52:26.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:52:26.257 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-10T21:52:26.270 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:52:26.270 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:52:26.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.440+0000 7f2bdec00640 1 -- 172.21.15.7:0/2765378379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd80cddc0 msgr2=0x7f2bd8072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:26.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.440+0000 7f2bdec00640 1 --2- 172.21.15.7:0/2765378379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd80cddc0 0x7f2bd8072f30 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8076890 tx=0x7f2bd002a9f0 comp rx=0 tx=0).stop 2024-09-10T21:52:26.441 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.440+0000 7f2bdec00640 1 -- 172.21.15.7:0/2765378379 shutdown_connections 2024-09-10T21:52:26.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.440+0000 7f2bdec00640 1 --2- 172.21.15.7:0/2765378379 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd80cddc0 0x7f2bd8072f30 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:26.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.440+0000 7f2bdec00640 1 --2- 172.21.15.7:0/2765378379 >> v2:172.21.15.133:3300/0 conn(0x7f2bd8159910 0x7f2bd815d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:26.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.440+0000 7f2bdec00640 1 -- 172.21.15.7:0/2765378379 >> 172.21.15.7:0/2765378379 conn(0x7f2bd806ca60 msgr2=0x7f2bd806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:26.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.440+0000 7f2bdec00640 1 -- 172.21.15.7:0/2765378379 shutdown_connections 2024-09-10T21:52:26.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.440+0000 7f2bdec00640 1 -- 172.21.15.7:0/2765378379 wait complete. 2024-09-10T21:52:26.442 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.442+0000 7f2bdec00640 1 Processor -- start 2024-09-10T21:52:26.443 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.442+0000 7f2bdec00640 1 -- start start 2024-09-10T21:52:26.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.443+0000 7f2bdec00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2bd806e810 0x7f2bd8150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:26.444 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.443+0000 7f2bdec00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd80cddc0 0x7f2bd80afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.443+0000 7f2bdec00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bd8159910 0x7f2bd80b0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.443+0000 7f2bdec00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8073800 con 0x7f2bd80cddc0 2024-09-10T21:52:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.443+0000 7f2bdec00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd8073680 con 0x7f2bd8159910 2024-09-10T21:52:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.443+0000 7f2bdec00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd8073980 con 0x7f2bd806e810 2024-09-10T21:52:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.443+0000 7f2bcf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd80cddc0 0x7f2bd80afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.443+0000 7f2bd7e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2bd806e810 0x7f2bd8150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.443+0000 7f2bcf400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd80cddc0 0x7f2bd80afc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40000/0 (socket says 172.21.15.7:40000) 2024-09-10T21:52:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.443+0000 7f2bcf400640 1 -- 172.21.15.7:0/1597330909 learned_addr learned my addr 172.21.15.7:0/1597330909 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.443+0000 7f2bdcc00640 1 --2- 172.21.15.7:0/1597330909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bd8159910 0x7f2bd80b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.444+0000 7f2bcf400640 1 -- 172.21.15.7:0/1597330909 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2bd806e810 msgr2=0x7f2bd8150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.444+0000 7f2bcf400640 1 --2- 172.21.15.7:0/1597330909 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2bd806e810 0x7f2bd8150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:26.445 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.444+0000 7f2bcf400640 1 -- 172.21.15.7:0/1597330909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bd8159910 msgr2=0x7f2bd80b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:26.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.444+0000 7f2bcf400640 1 --2- 172.21.15.7:0/1597330909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bd8159910 0x7f2bd80b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:26.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.444+0000 7f2bcf400640 1 -- 172.21.15.7:0/1597330909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd80ae500 con 0x7f2bd80cddc0 2024-09-10T21:52:26.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.444+0000 7f2bdcc00640 1 --2- 172.21.15.7:0/1597330909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bd8159910 0x7f2bd80b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:26.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.444+0000 7f2bcf400640 1 --2- 172.21.15.7:0/1597330909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd80cddc0 0x7f2bd80afc20 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000c430 tx=0x7f2bc000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:26.446 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.444+0000 7f2bd4c00640 1 -- 172.21.15.7:0/1597330909 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0011790 con 0x7f2bd80cddc0 2024-09-10T21:52:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.444+0000 7f2bdec00640 1 -- 172.21.15.7:0/1597330909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd80ae7e0 con 0x7f2bd80cddc0 2024-09-10T21:52:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.444+0000 7f2bd4c00640 1 -- 172.21.15.7:0/1597330909 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bc0017070 con 0x7f2bd80cddc0 2024-09-10T21:52:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.444+0000 7f2bd4c00640 1 -- 172.21.15.7:0/1597330909 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc0011d00 con 0x7f2bd80cddc0 2024-09-10T21:52:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.444+0000 7f2bdec00640 1 -- 172.21.15.7:0/1597330909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd80af080 con 0x7f2bd80cddc0 2024-09-10T21:52:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.445+0000 7f2bdec00640 1 -- 172.21.15.7:0/1597330909 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd815d090 con 0x7f2bd80cddc0 2024-09-10T21:52:26.447 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.446+0000 7f2bd4c00640 1 -- 172.21.15.7:0/1597330909 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f2bc001c460 con 0x7f2bd80cddc0 2024-09-10T21:52:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.447+0000 7f2bd4c00640 1 --2- 172.21.15.7:0/1597330909 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2bb4077f70 0x7f2bb407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.448+0000 7f2bd4c00640 1 -- 172.21.15.7:0/1597330909 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7f2bc0099960 con 0x7f2bd80cddc0 2024-09-10T21:52:26.448 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.448+0000 7f2bd7e00640 1 --2- 172.21.15.7:0/1597330909 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2bb4077f70 0x7f2bb407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:26.449 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.448+0000 7f2bd7e00640 1 --2- 172.21.15.7:0/1597330909 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2bb4077f70 0x7f2bb407a430 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8004630 tx=0x7f2bc800b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:26.452 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.451+0000 7f2bd4c00640 1 -- 172.21.15.7:0/1597330909 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f2bc0062620 con 0x7f2bd80cddc0 2024-09-10T21:52:26.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.644+0000 7f2bdec00640 1 -- 172.21.15.7:0/1597330909 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2bd80b26a0 con 0x7f2bb4077f70 2024-09-10T21:52:26.645 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.645+0000 7f2bd4c00640 1 -- 172.21.15.7:0/1597330909 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7f2bd80b26a0 con 0x7f2bb4077f70 2024-09-10T21:52:26.646 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:26.646 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:52:26.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.647+0000 7f2bdec00640 1 -- 172.21.15.7:0/1597330909 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2bb4077f70 msgr2=0x7f2bb407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:26.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.647+0000 7f2bdec00640 1 --2- 172.21.15.7:0/1597330909 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2bb4077f70 0x7f2bb407a430 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8004630 tx=0x7f2bc800b2e0 comp rx=0 tx=0).stop 2024-09-10T21:52:26.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.647+0000 7f2bdec00640 1 -- 172.21.15.7:0/1597330909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd80cddc0 msgr2=0x7f2bd80afc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:26.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.647+0000 7f2bdec00640 1 --2- 172.21.15.7:0/1597330909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd80cddc0 0x7f2bd80afc20 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f2bc000c430 tx=0x7f2bc000c8f0 comp rx=0 tx=0).stop 2024-09-10T21:52:26.648 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.647+0000 7f2bdec00640 1 -- 172.21.15.7:0/1597330909 shutdown_connections 2024-09-10T21:52:26.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.647+0000 7f2bdec00640 1 --2- 172.21.15.7:0/1597330909 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2bb4077f70 0x7f2bb407a430 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:26.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.647+0000 7f2bdec00640 1 --2- 172.21.15.7:0/1597330909 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2bd8159910 0x7f2bd80b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:26.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.647+0000 7f2bdec00640 1 --2- 172.21.15.7:0/1597330909 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f2bd80cddc0 0x7f2bd80afc20 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:26.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.647+0000 7f2bdec00640 1 --2- 172.21.15.7:0/1597330909 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f2bd806e810 0x7f2bd8150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:26.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.647+0000 7f2bdec00640 1 -- 172.21.15.7:0/1597330909 >> 172.21.15.7:0/1597330909 conn(0x7f2bd806ca60 msgr2=0x7f2bd8159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:26.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.647+0000 7f2bdec00640 1 -- 172.21.15.7:0/1597330909 shutdown_connections 2024-09-10T21:52:26.649 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.647+0000 7f2bdec00640 1 -- 172.21.15.7:0/1597330909 wait complete. 2024-09-10T21:52:26.663 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-10T21:52:26.021161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":295300,"kb_used_data":29156,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754679420,"statfs":{"total":773094113280,"available":772791726080,"internally_reserved":0,"allocated":29855744,"data_stored":23112774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392255,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-175,"num_write_kb":-8306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001518"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":292,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-10T21:52:20.550602+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:52:20.550602+0000","last_peered":"2024-09-10T21:52:20.550602+0000","last_clean":"2024-09-10T21:52:20.550602+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:52:20.550602+0000","last_undegraded":"2024-09-10T21:52:20.550602+0000","last_fullsized":"2024-09-10T21:52:20.550602+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36728,"kb_used_data":5540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335112,"statfs":{"total":96636764160,"available":96599154688,"internally_reserved":0,"allocated":5672960,"data_stored":4835149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509569,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1495040,"data_stored":647013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509568,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32640,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339200,"statfs":{"total":96636764160,"available":96603340800,"internally_reserved":0,"allocated":1486848,"data_stored":647001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509568,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":1490944,"data_stored":647006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509570,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40888,"kb_used_data":5540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330952,"statfs":{"total":96636764160,"available":96594894848,"internally_reserved":0,"allocated":5672960,"data_stored":4835196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509568,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41480,"kb_used_data":6132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330360,"statfs":{"total":96636764160,"available":96594288640,"internally_reserved":0,"allocated":6279168,"data_stored":5431678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542271,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2072576,"data_stored":1231752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542272,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40900,"kb_used_data":5552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330940,"statfs":{"total":96636764160,"available":96594882560,"internally_reserved":0,"allocated":5685248,"data_stored":4837979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:52:26.664 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:52:26.825 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.824+0000 7fd6be200640 1 -- 172.21.15.7:0/699691425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b806ed50 msgr2=0x7fd6b815ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.824+0000 7fd6be200640 1 --2- 172.21.15.7:0/699691425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b806ed50 0x7fd6b815ea20 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fd6b80767b0 tx=0x7fd6a002a530 comp rx=0 tx=0).stop 2024-09-10T21:52:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.824+0000 7fd6be200640 1 -- 172.21.15.7:0/699691425 shutdown_connections 2024-09-10T21:52:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.824+0000 7fd6be200640 1 --2- 172.21.15.7:0/699691425 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b806ed50 0x7fd6b815ea20 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.824+0000 7fd6be200640 1 --2- 172.21.15.7:0/699691425 >> v2:172.21.15.133:3300/0 conn(0x7fd6b80ccdc0 0x7fd6b806e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.824+0000 7fd6be200640 1 -- 172.21.15.7:0/699691425 >> 172.21.15.7:0/699691425 conn(0x7fd6b806ca60 msgr2=0x7fd6b806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.824+0000 7fd6be200640 1 -- 172.21.15.7:0/699691425 shutdown_connections 2024-09-10T21:52:26.826 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.824+0000 7fd6be200640 1 -- 172.21.15.7:0/699691425 wait complete. 2024-09-10T21:52:26.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.826+0000 7fd6be200640 1 Processor -- start 2024-09-10T21:52:26.827 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.826+0000 7fd6be200640 1 -- start start 2024-09-10T21:52:26.828 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.827+0000 7fd6be200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6b806ed50 0x7fd6b8145940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.827+0000 7fd6be200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b80ccdc0 0x7fd6b8158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.827+0000 7fd6be200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6b815ef60 0x7fd6b81f7e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.827+0000 7fd6be200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b815c2c0 con 0x7fd6b80ccdc0 2024-09-10T21:52:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.827+0000 7fd6be200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd6b815c140 con 0x7fd6b806ed50 2024-09-10T21:52:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.827+0000 7fd6be200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fd6b80741d0 con 0x7fd6b815ef60 2024-09-10T21:52:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.827+0000 7fd6aea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b80ccdc0 0x7fd6b8158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.827+0000 7fd6aea00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b80ccdc0 0x7fd6b8158430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40022/0 (socket says 172.21.15.7:40022) 2024-09-10T21:52:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.827+0000 7fd6aea00640 1 -- 172.21.15.7:0/242700619 learned_addr learned my addr 172.21.15.7:0/242700619 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.827+0000 7fd6b7400640 1 --2- 172.21.15.7:0/242700619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6b806ed50 0x7fd6b8145940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.827+0000 7fd6b7e00640 1 --2- 172.21.15.7:0/242700619 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6b815ef60 0x7fd6b81f7e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.828+0000 7fd6aea00640 1 -- 172.21.15.7:0/242700619 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6b815ef60 msgr2=0x7fd6b81f7e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:26.829 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.828+0000 7fd6aea00640 1 --2- 172.21.15.7:0/242700619 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6b815ef60 0x7fd6b81f7e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:26.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.828+0000 7fd6aea00640 1 -- 172.21.15.7:0/242700619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6b806ed50 msgr2=0x7fd6b8145940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:26.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.828+0000 7fd6aea00640 1 --2- 172.21.15.7:0/242700619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6b806ed50 0x7fd6b8145940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:26.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.828+0000 7fd6aea00640 1 -- 172.21.15.7:0/242700619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6b815cfd0 con 0x7fd6b80ccdc0 2024-09-10T21:52:26.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.828+0000 7fd6b7e00640 1 --2- 172.21.15.7:0/242700619 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6b815ef60 0x7fd6b81f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:26.830 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.828+0000 7fd6aea00640 1 --2- 172.21.15.7:0/242700619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b80ccdc0 0x7fd6b8158430 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0009780 tx=0x7fd6a0032d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:26.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.828+0000 7fd6afe00640 1 -- 172.21.15.7:0/242700619 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0040070 con 0x7fd6b80ccdc0 2024-09-10T21:52:26.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.828+0000 7fd6afe00640 1 -- 172.21.15.7:0/242700619 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd6a0005310 con 0x7fd6b80ccdc0 2024-09-10T21:52:26.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.828+0000 7fd6afe00640 1 -- 172.21.15.7:0/242700619 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6a0003800 con 0x7fd6b80ccdc0 2024-09-10T21:52:26.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.828+0000 7fd6be200640 1 -- 172.21.15.7:0/242700619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6b815d2b0 con 0x7fd6b80ccdc0 2024-09-10T21:52:26.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.828+0000 7fd6be200640 1 -- 172.21.15.7:0/242700619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6b815d870 con 0x7fd6b80ccdc0 2024-09-10T21:52:26.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.830+0000 7fd6be200640 1 -- 172.21.15.7:0/242700619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6b8074680 con 0x7fd6b80ccdc0 2024-09-10T21:52:26.831 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.831+0000 7fd6afe00640 1 -- 172.21.15.7:0/242700619 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd6a00039a0 con 0x7fd6b80ccdc0 2024-09-10T21:52:26.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.831+0000 7fd6afe00640 1 --2- 172.21.15.7:0/242700619 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd694077fa0 0x7fd69407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:26.832 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.832+0000 7fd6afe00640 1 -- 172.21.15.7:0/242700619 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7fd6a00bafa0 con 0x7fd6b80ccdc0 2024-09-10T21:52:26.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.832+0000 7fd6b7400640 1 --2- 172.21.15.7:0/242700619 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd694077fa0 0x7fd69407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:26.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.833+0000 7fd6b7400640 1 --2- 172.21.15.7:0/242700619 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd694077fa0 0x7fd69407a460 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800abe0 tx=0x7fd6a800a2d0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:26.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:26.836+0000 7fd6afe00640 1 -- 172.21.15.7:0/242700619 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fd6a0083ce0 con 0x7fd6b80ccdc0 2024-09-10T21:52:27.030 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.029+0000 7fd6be200640 1 -- 172.21.15.7:0/242700619 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd6b8073960 con 0x7fd694077fa0 2024-09-10T21:52:27.031 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.031+0000 7fd6afe00640 1 -- 172.21.15.7:0/242700619 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7fd6b8073960 con 0x7fd694077fa0 2024-09-10T21:52:27.031 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:27.032 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:52:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.033+0000 7fd6be200640 1 -- 172.21.15.7:0/242700619 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd694077fa0 msgr2=0x7fd69407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.033+0000 7fd6be200640 1 --2- 172.21.15.7:0/242700619 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd694077fa0 0x7fd69407a460 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd6a800abe0 tx=0x7fd6a800a2d0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.033+0000 7fd6be200640 1 -- 172.21.15.7:0/242700619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b80ccdc0 msgr2=0x7fd6b8158430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.033+0000 7fd6be200640 1 --2- 172.21.15.7:0/242700619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b80ccdc0 0x7fd6b8158430 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fd6a0009780 tx=0x7fd6a0032d50 comp rx=0 tx=0).stop 2024-09-10T21:52:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.033+0000 7fd6be200640 1 -- 172.21.15.7:0/242700619 shutdown_connections 2024-09-10T21:52:27.034 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.033+0000 7fd6be200640 1 --2- 172.21.15.7:0/242700619 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd694077fa0 0x7fd69407a460 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.033+0000 7fd6be200640 1 --2- 172.21.15.7:0/242700619 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd6b815ef60 0x7fd6b81f7e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.033+0000 7fd6be200640 1 --2- 172.21.15.7:0/242700619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd6b80ccdc0 0x7fd6b8158430 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.033+0000 7fd6be200640 1 --2- 172.21.15.7:0/242700619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd6b806ed50 0x7fd6b8145940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.033+0000 7fd6be200640 1 -- 172.21.15.7:0/242700619 >> 172.21.15.7:0/242700619 conn(0x7fd6b806ca60 msgr2=0x7fd6b80729b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.033+0000 7fd6be200640 1 -- 172.21.15.7:0/242700619 shutdown_connections 2024-09-10T21:52:27.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.033+0000 7fd6be200640 1 -- 172.21.15.7:0/242700619 wait complete. 2024-09-10T21:52:27.050 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-10T21:52:26.021161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":295300,"kb_used_data":29156,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754679420,"statfs":{"total":773094113280,"available":772791726080,"internally_reserved":0,"allocated":29855744,"data_stored":23112774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392255,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-175,"num_write_kb":-8306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001518"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":292,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-10T21:52:20.550602+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:52:20.550602+0000","last_peered":"2024-09-10T21:52:20.550602+0000","last_clean":"2024-09-10T21:52:20.550602+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:52:20.550602+0000","last_undegraded":"2024-09-10T21:52:20.550602+0000","last_fullsized":"2024-09-10T21:52:20.550602+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36728,"kb_used_data":5540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335112,"statfs":{"total":96636764160,"available":96599154688,"internally_reserved":0,"allocated":5672960,"data_stored":4835149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509569,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1495040,"data_stored":647013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509568,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32640,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339200,"statfs":{"total":96636764160,"available":96603340800,"internally_reserved":0,"allocated":1486848,"data_stored":647001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509568,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":1490944,"data_stored":647006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509570,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40888,"kb_used_data":5540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330952,"statfs":{"total":96636764160,"available":96594894848,"internally_reserved":0,"allocated":5672960,"data_stored":4835196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509568,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41480,"kb_used_data":6132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330360,"statfs":{"total":96636764160,"available":96594288640,"internally_reserved":0,"allocated":6279168,"data_stored":5431678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542271,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2072576,"data_stored":1231752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542272,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40900,"kb_used_data":5552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330940,"statfs":{"total":96636764160,"available":96594882560,"internally_reserved":0,"allocated":5685248,"data_stored":4837979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:52:27.050 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:52:27.050 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:52:27.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.210+0000 7f5538400640 1 -- 172.21.15.7:0/3482420380 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f55300cdbf0 msgr2=0x7f5530072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:27.211 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.210+0000 7f5538400640 1 --2- 172.21.15.7:0/3482420380 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f55300cdbf0 0x7f5530072f30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f5528005fd0 tx=0x7f552802aa90 comp rx=0 tx=0).stop 2024-09-10T21:52:27.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.210+0000 7f5538400640 1 -- 172.21.15.7:0/3482420380 shutdown_connections 2024-09-10T21:52:27.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.210+0000 7f5538400640 1 --2- 172.21.15.7:0/3482420380 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f55300cdbf0 0x7f5530072f30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.210+0000 7f5538400640 1 --2- 172.21.15.7:0/3482420380 >> v2:172.21.15.133:3300/0 conn(0x7f553015d090 0x7f55300cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.210+0000 7f5538400640 1 -- 172.21.15.7:0/3482420380 >> 172.21.15.7:0/3482420380 conn(0x7f553006ca60 msgr2=0x7f553006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:27.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.210+0000 7f5538400640 1 -- 172.21.15.7:0/3482420380 shutdown_connections 2024-09-10T21:52:27.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.210+0000 7f5538400640 1 -- 172.21.15.7:0/3482420380 wait complete. 2024-09-10T21:52:27.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.212+0000 7f5538400640 1 Processor -- start 2024-09-10T21:52:27.213 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.212+0000 7f5538400640 1 -- start start 2024-09-10T21:52:27.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.213+0000 7f5538400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55300cdbf0 0x7f553014da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:27.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.213+0000 7f5538400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5530159910 0x7f55300aee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:27.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.213+0000 7f5538400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f553015d090 0x7f55300af380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:27.214 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.213+0000 7f5538400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f5530073800 con 0x7f55300cdbf0 2024-09-10T21:52:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.213+0000 7f5538400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f5530073680 con 0x7f5530159910 2024-09-10T21:52:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.213+0000 7f5538400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f5530073980 con 0x7f553015d090 2024-09-10T21:52:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.213+0000 7f5535a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55300cdbf0 0x7f553014da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.213+0000 7f5535a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55300cdbf0 0x7f553014da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40026/0 (socket says 172.21.15.7:40026) 2024-09-10T21:52:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.213+0000 7f5535a00640 1 -- 172.21.15.7:0/1438761506 learned_addr learned my addr 172.21.15.7:0/1438761506 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.214+0000 7f5536400640 1 --2- 172.21.15.7:0/1438761506 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f553015d090 0x7f55300af380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:27.215 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.214+0000 7f5535000640 1 --2- 172.21.15.7:0/1438761506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5530159910 0x7f55300aee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:27.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.214+0000 7f5535a00640 1 -- 172.21.15.7:0/1438761506 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f553015d090 msgr2=0x7f55300af380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:27.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.214+0000 7f5535a00640 1 --2- 172.21.15.7:0/1438761506 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f553015d090 0x7f55300af380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.214+0000 7f5535a00640 1 -- 172.21.15.7:0/1438761506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5530159910 msgr2=0x7f55300aee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:27.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.214+0000 7f5535a00640 1 --2- 172.21.15.7:0/1438761506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5530159910 0x7f55300aee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.214+0000 7f5535a00640 1 -- 172.21.15.7:0/1438761506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5530157750 con 0x7f55300cdbf0 2024-09-10T21:52:27.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.214+0000 7f5535a00640 1 --2- 172.21.15.7:0/1438761506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55300cdbf0 0x7f553014da50 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f552400c420 tx=0x7f552400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:27.216 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.214+0000 7f552d600640 1 -- 172.21.15.7:0/1438761506 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55240052d0 con 0x7f55300cdbf0 2024-09-10T21:52:27.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.214+0000 7f552d600640 1 -- 172.21.15.7:0/1438761506 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5524007020 con 0x7f55300cdbf0 2024-09-10T21:52:27.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.214+0000 7f552d600640 1 -- 172.21.15.7:0/1438761506 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5524007470 con 0x7f55300cdbf0 2024-09-10T21:52:27.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.215+0000 7f5538400640 1 -- 172.21.15.7:0/1438761506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55301f7e30 con 0x7f55300cdbf0 2024-09-10T21:52:27.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.215+0000 7f5538400640 1 -- 172.21.15.7:0/1438761506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f553014df90 con 0x7f55300cdbf0 2024-09-10T21:52:27.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.216+0000 7f5538400640 1 -- 172.21.15.7:0/1438761506 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5530072f30 con 0x7f55300cdbf0 2024-09-10T21:52:27.217 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.217+0000 7f552d600640 1 -- 172.21.15.7:0/1438761506 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f55240067d0 con 0x7f55300cdbf0 2024-09-10T21:52:27.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.218+0000 7f552d600640 1 --2- 172.21.15.7:0/1438761506 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5510077f70 0x7f551007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:27.218 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.218+0000 7f552d600640 1 -- 172.21.15.7:0/1438761506 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7f5524098f70 con 0x7f55300cdbf0 2024-09-10T21:52:27.219 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.218+0000 7f5535000640 1 --2- 172.21.15.7:0/1438761506 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5510077f70 0x7f551007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:27.220 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.219+0000 7f5535000640 1 --2- 172.21.15.7:0/1438761506 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5510077f70 0x7f551007a430 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f551c0095f0 tx=0x7f551c005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:27.222 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.222+0000 7f552d600640 1 -- 172.21.15.7:0/1438761506 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f5524061660 con 0x7f55300cdbf0 2024-09-10T21:52:27.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.412+0000 7f5538400640 1 -- 172.21.15.7:0/1438761506 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f553006e8b0 con 0x7f5510077f70 2024-09-10T21:52:27.414 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.414+0000 7f552d600640 1 -- 172.21.15.7:0/1438761506 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13498 (secure 0 0 0) 0x7f553006e8b0 con 0x7f5510077f70 2024-09-10T21:52:27.415 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:27.415 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:52:27.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.416+0000 7f5538400640 1 -- 172.21.15.7:0/1438761506 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5510077f70 msgr2=0x7f551007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:27.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.416+0000 7f5538400640 1 --2- 172.21.15.7:0/1438761506 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5510077f70 0x7f551007a430 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f551c0095f0 tx=0x7f551c005f90 comp rx=0 tx=0).stop 2024-09-10T21:52:27.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.416+0000 7f5538400640 1 -- 172.21.15.7:0/1438761506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55300cdbf0 msgr2=0x7f553014da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:27.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.416+0000 7f5538400640 1 --2- 172.21.15.7:0/1438761506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55300cdbf0 0x7f553014da50 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f552400c420 tx=0x7f552400c8e0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.416+0000 7f5538400640 1 -- 172.21.15.7:0/1438761506 shutdown_connections 2024-09-10T21:52:27.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.416+0000 7f5538400640 1 --2- 172.21.15.7:0/1438761506 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f5510077f70 0x7f551007a430 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.417 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.416+0000 7f5538400640 1 --2- 172.21.15.7:0/1438761506 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f5530159910 0x7f55300aee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.416+0000 7f5538400640 1 --2- 172.21.15.7:0/1438761506 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f553015d090 0x7f55300af380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.416+0000 7f5538400640 1 --2- 172.21.15.7:0/1438761506 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f55300cdbf0 0x7f553014da50 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.416+0000 7f5538400640 1 -- 172.21.15.7:0/1438761506 >> 172.21.15.7:0/1438761506 conn(0x7f553006ca60 msgr2=0x7f553015a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:27.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.416+0000 7f5538400640 1 -- 172.21.15.7:0/1438761506 shutdown_connections 2024-09-10T21:52:27.418 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.416+0000 7f5538400640 1 -- 172.21.15.7:0/1438761506 wait complete. 2024-09-10T21:52:27.432 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-10T21:52:26.021161+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":295300,"kb_used_data":29156,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754679420,"statfs":{"total":773094113280,"available":772791726080,"internally_reserved":0,"allocated":29855744,"data_stored":23112774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8392255,"num_objects":-30,"num_object_clones":0,"num_object_copies":-60,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-30,"num_whiteouts":0,"num_read":-258,"num_read_kb":-268,"num_write":-175,"num_write_kb":-8306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-17,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001518"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":292,"reported_epoch":119,"state":"active+clean","last_fresh":"2024-09-10T21:52:20.550602+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:52:20.550602+0000","last_peered":"2024-09-10T21:52:20.550602+0000","last_clean":"2024-09-10T21:52:20.550602+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:52:20.550602+0000","last_undegraded":"2024-09-10T21:52:20.550602+0000","last_fullsized":"2024-09-10T21:52:20.550602+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36728,"kb_used_data":5540,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94335112,"statfs":{"total":96636764160,"available":96599154688,"internally_reserved":0,"allocated":5672960,"data_stored":4835149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509569,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1495040,"data_stored":647013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509568,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32640,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339200,"statfs":{"total":96636764160,"available":96603340800,"internally_reserved":0,"allocated":1486848,"data_stored":647001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509568,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36804,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335036,"statfs":{"total":96636764160,"available":96599076864,"internally_reserved":0,"allocated":1490944,"data_stored":647006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509570,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40888,"kb_used_data":5540,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330952,"statfs":{"total":96636764160,"available":96594894848,"internally_reserved":0,"allocated":5672960,"data_stored":4835196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509568,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41480,"kb_used_data":6132,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330360,"statfs":{"total":96636764160,"available":96594288640,"internally_reserved":0,"allocated":6279168,"data_stored":5431678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542271,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2072576,"data_stored":1231752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542272,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40900,"kb_used_data":5552,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94330940,"statfs":{"total":96636764160,"available":96594882560,"internally_reserved":0,"allocated":5685248,"data_stored":4837979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:52:27.433 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:52:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.592+0000 7fa87f800640 1 -- 172.21.15.7:0/1295441567 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa87806e810 msgr2=0x7fa8780cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.592+0000 7fa87f800640 1 --2- 172.21.15.7:0/1295441567 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa87806e810 0x7fa8780cd6b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa860009830 tx=0x7fa86002a520 comp rx=0 tx=0).stop 2024-09-10T21:52:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.593+0000 7fa87f800640 1 -- 172.21.15.7:0/1295441567 shutdown_connections 2024-09-10T21:52:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.593+0000 7fa87f800640 1 --2- 172.21.15.7:0/1295441567 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa87806e810 0x7fa8780cd6b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.593+0000 7fa87f800640 1 --2- 172.21.15.7:0/1295441567 >> v2:172.21.15.133:3300/0 conn(0x7fa878159910 0x7fa87815d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.593+0000 7fa87f800640 1 -- 172.21.15.7:0/1295441567 >> 172.21.15.7:0/1295441567 conn(0x7fa87806ca60 msgr2=0x7fa87806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.593+0000 7fa87f800640 1 -- 172.21.15.7:0/1295441567 shutdown_connections 2024-09-10T21:52:27.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.593+0000 7fa87f800640 1 -- 172.21.15.7:0/1295441567 wait complete. 2024-09-10T21:52:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.594+0000 7fa87f800640 1 Processor -- start 2024-09-10T21:52:27.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.594+0000 7fa87f800640 1 -- start start 2024-09-10T21:52:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.595+0000 7fa87f800640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa87806e810 0x7fa878151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.595+0000 7fa87f800640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8780cdbf0 0x7fa8781f25b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.595+0000 7fa87f800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa878159910 0x7fa878156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.595+0000 7fa87f800640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fa8780740c0 con 0x7fa878159910 2024-09-10T21:52:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.595+0000 7fa87f800640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fa878073f40 con 0x7fa8780cdbf0 2024-09-10T21:52:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.595+0000 7fa87f800640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fa878074240 con 0x7fa87806e810 2024-09-10T21:52:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.595+0000 7fa87d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa878159910 0x7fa878156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.596+0000 7fa87d800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa878159910 0x7fa878156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40036/0 (socket says 172.21.15.7:40036) 2024-09-10T21:52:27.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.596+0000 7fa87d800640 1 -- 172.21.15.7:0/3613953194 learned_addr learned my addr 172.21.15.7:0/3613953194 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.596+0000 7fa87ce00640 1 --2- 172.21.15.7:0/3613953194 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa87806e810 0x7fa878151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.596+0000 7fa86fe00640 1 --2- 172.21.15.7:0/3613953194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8780cdbf0 0x7fa8781f25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.596+0000 7fa87d800640 1 -- 172.21.15.7:0/3613953194 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa87806e810 msgr2=0x7fa878151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.596+0000 7fa87d800640 1 --2- 172.21.15.7:0/3613953194 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa87806e810 0x7fa878151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.596+0000 7fa87d800640 1 -- 172.21.15.7:0/3613953194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8780cdbf0 msgr2=0x7fa8781f25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.596+0000 7fa87d800640 1 --2- 172.21.15.7:0/3613953194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8780cdbf0 0x7fa8781f25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.596+0000 7fa87d800640 1 -- 172.21.15.7:0/3613953194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8781f5ba0 con 0x7fa878159910 2024-09-10T21:52:27.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.596+0000 7fa87d800640 1 --2- 172.21.15.7:0/3613953194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa878159910 0x7fa878156090 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fa87000cbb0 tx=0x7fa87000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.596+0000 7fa875600640 1 -- 172.21.15.7:0/3613953194 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870016020 con 0x7fa878159910 2024-09-10T21:52:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.597+0000 7fa875600640 1 -- 172.21.15.7:0/3613953194 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa87000add0 con 0x7fa878159910 2024-09-10T21:52:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.597+0000 7fa875600640 1 -- 172.21.15.7:0/3613953194 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa870012680 con 0x7fa878159910 2024-09-10T21:52:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.597+0000 7fa87f800640 1 -- 172.21.15.7:0/3613953194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa878153080 con 0x7fa878159910 2024-09-10T21:52:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.597+0000 7fa87f800640 1 -- 172.21.15.7:0/3613953194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa878153800 con 0x7fa878159910 2024-09-10T21:52:27.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.598+0000 7fa87f800640 1 -- 172.21.15.7:0/3613953194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8781f5d70 con 0x7fa878159910 2024-09-10T21:52:27.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.599+0000 7fa875600640 1 -- 172.21.15.7:0/3613953194 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fa870012820 con 0x7fa878159910 2024-09-10T21:52:27.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.600+0000 7fa875600640 1 --2- 172.21.15.7:0/3613953194 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa854077f70 0x7fa85407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.600+0000 7fa875600640 1 -- 172.21.15.7:0/3613953194 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7fa87009a3c0 con 0x7fa878159910 2024-09-10T21:52:27.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.600+0000 7fa87ce00640 1 --2- 172.21.15.7:0/3613953194 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa854077f70 0x7fa85407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:27.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.601+0000 7fa87ce00640 1 --2- 172.21.15.7:0/3613953194 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa854077f70 0x7fa85407a430 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa868009ac0 tx=0x7fa868005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:27.604 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.604+0000 7fa875600640 1 -- 172.21.15.7:0/3613953194 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fa870020020 con 0x7fa878159910 2024-09-10T21:52:27.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.802+0000 7fa87f800640 1 -- 172.21.15.7:0/3613953194 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa8780cd6b0 con 0x7fa878159910 2024-09-10T21:52:27.804 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.803+0000 7fa875600640 1 -- 172.21.15.7:0/3613953194 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+13270 (secure 0 0 0) 0x7fa8700631a0 con 0x7fa878159910 2024-09-10T21:52:27.804 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:27.804 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":121,"fsid":"2029451a-3c26-48c7-b874-0597197278ac","created":"2024-09-10T21:41:42.809504+0000","modified":"2024-09-10T21:52:23.246106+0000","last_up_change":"2024-09-10T21:41:56.841496+0000","last_in_change":"2024-09-10T21:41:47.424461+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:41:55.889618+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c49a2472-5948-485f-8fef-3848b9e3bce3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6808","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6809","nonce":651402908}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6810","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6811","nonce":651402908}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6814","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6815","nonce":651402908}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6812","nonce":651402908},{"type":"v1","addr":"172.21.15.7:6813","nonce":651402908}]},"public_addr":"172.21.15.7:6809/651402908","cluster_addr":"172.21.15.7:6811/651402908","heartbeat_back_addr":"172.21.15.7:6815/651402908","heartbeat_front_addr":"172.21.15.7:6813/651402908","state":["exists","up"]},{"osd":1,"uuid":"6ad0e4c6-e412-4bf0-906f-aa8fdfecb405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6824","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6825","nonce":4175601737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6826","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6827","nonce":4175601737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6830","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6831","nonce":4175601737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6828","nonce":4175601737},{"type":"v1","addr":"172.21.15.7:6829","nonce":4175601737}]},"public_addr":"172.21.15.7:6825/4175601737","cluster_addr":"172.21.15.7:6827/4175601737","heartbeat_back_addr":"172.21.15.7:6831/4175601737","heartbeat_front_addr":"172.21.15.7:6829/4175601737","state":["exists","up"]},{"osd":2,"uuid":"86e61267-ebb6-4e9f-b436-596efaa9eeb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6816","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6817","nonce":4238877366}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6818","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6819","nonce":4238877366}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6822","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6823","nonce":4238877366}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6820","nonce":4238877366},{"type":"v1","addr":"172.21.15.7:6821","nonce":4238877366}]},"public_addr":"172.21.15.7:6817/4238877366","cluster_addr":"172.21.15.7:6819/4238877366","heartbeat_back_addr":"172.21.15.7:6823/4238877366","heartbeat_front_addr":"172.21.15.7:6821/4238877366","state":["exists","up"]},{"osd":3,"uuid":"1b85f36c-239a-4e3e-9bd8-a5b36382dcb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6800","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6801","nonce":2195665006}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6802","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6803","nonce":2195665006}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6806","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6807","nonce":2195665006}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6804","nonce":2195665006},{"type":"v1","addr":"172.21.15.7:6805","nonce":2195665006}]},"public_addr":"172.21.15.7:6801/2195665006","cluster_addr":"172.21.15.7:6803/2195665006","heartbeat_back_addr":"172.21.15.7:6807/2195665006","heartbeat_front_addr":"172.21.15.7:6805/2195665006","state":["exists","up"]},{"osd":4,"uuid":"4c5f74b5-0324-4f6d-a41c-a5176ea886e4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6826","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6827","nonce":1857710445}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6828","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6829","nonce":1857710445}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6832","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6833","nonce":1857710445}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6830","nonce":1857710445},{"type":"v1","addr":"172.21.15.133:6831","nonce":1857710445}]},"public_addr":"172.21.15.133:6827/1857710445","cluster_addr":"172.21.15.133:6829/1857710445","heartbeat_back_addr":"172.21.15.133:6833/1857710445","heartbeat_front_addr":"172.21.15.133:6831/1857710445","state":["exists","up"]},{"osd":5,"uuid":"6b561866-3e02-4726-a0fb-a1dd3cdae62e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6818","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6819","nonce":3838498382}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6820","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6821","nonce":3838498382}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6824","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6825","nonce":3838498382}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6822","nonce":3838498382},{"type":"v1","addr":"172.21.15.133:6823","nonce":3838498382}]},"public_addr":"172.21.15.133:6819/3838498382","cluster_addr":"172.21.15.133:6821/3838498382","heartbeat_back_addr":"172.21.15.133:6825/3838498382","heartbeat_front_addr":"172.21.15.133:6823/3838498382","state":["exists","up"]},{"osd":6,"uuid":"e0b15a28-b8bc-446a-8b02-3e555e9aa2c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6810","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6811","nonce":3365981160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6812","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6813","nonce":3365981160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6816","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6817","nonce":3365981160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6814","nonce":3365981160},{"type":"v1","addr":"172.21.15.133:6815","nonce":3365981160}]},"public_addr":"172.21.15.133:6811/3365981160","cluster_addr":"172.21.15.133:6813/3365981160","heartbeat_back_addr":"172.21.15.133:6817/3365981160","heartbeat_front_addr":"172.21.15.133:6815/3365981160","state":["exists","up"]},{"osd":7,"uuid":"13799424-09a2-4a3a-adbb-99db0da9d130","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6802","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6803","nonce":2638399040}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6804","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6805","nonce":2638399040}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6808","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6809","nonce":2638399040}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6806","nonce":2638399040},{"type":"v1","addr":"172.21.15.133:6807","nonce":2638399040}]},"public_addr":"172.21.15.133:6803/2638399040","cluster_addr":"172.21.15.133:6805/2638399040","heartbeat_back_addr":"172.21.15.133:6809/2638399040","heartbeat_front_addr":"172.21.15.133:6807/2638399040","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.011775+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.432645+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.872856+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.711857+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.112200+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.328812+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:54.077957+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:41:53.748317+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.133:6836/772257983":"2024-09-11T21:52:06.881059+0000","172.21.15.133:6836/4068969165":"2024-09-11T21:52:20.240147+0000","172.21.15.133:6835/772257983":"2024-09-11T21:52:06.881059+0000","172.21.15.133:6837/3764354952":"2024-09-11T21:51:56.206885+0000","172.21.15.133:6835/4068969165":"2024-09-11T21:52:20.240147+0000","172.21.15.133:6834/3764354952":"2024-09-11T21:51:56.206885+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:52:27.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.805+0000 7fa87f800640 1 -- 172.21.15.7:0/3613953194 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa854077f70 msgr2=0x7fa85407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:27.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.805+0000 7fa87f800640 1 --2- 172.21.15.7:0/3613953194 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa854077f70 0x7fa85407a430 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fa868009ac0 tx=0x7fa868005f90 comp rx=0 tx=0).stop 2024-09-10T21:52:27.806 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.805+0000 7fa87f800640 1 -- 172.21.15.7:0/3613953194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa878159910 msgr2=0x7fa878156090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.805+0000 7fa87f800640 1 --2- 172.21.15.7:0/3613953194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa878159910 0x7fa878156090 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fa87000cbb0 tx=0x7fa87000a430 comp rx=0 tx=0).stop 2024-09-10T21:52:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.805+0000 7fa87f800640 1 -- 172.21.15.7:0/3613953194 shutdown_connections 2024-09-10T21:52:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.805+0000 7fa87f800640 1 --2- 172.21.15.7:0/3613953194 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fa854077f70 0x7fa85407a430 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.805+0000 7fa87f800640 1 --2- 172.21.15.7:0/3613953194 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fa8780cdbf0 0x7fa8781f25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.805+0000 7fa87f800640 1 --2- 172.21.15.7:0/3613953194 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fa878159910 0x7fa878156090 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.805+0000 7fa87f800640 1 --2- 172.21.15.7:0/3613953194 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fa87806e810 0x7fa878151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.805+0000 7fa87f800640 1 -- 172.21.15.7:0/3613953194 >> 172.21.15.7:0/3613953194 conn(0x7fa87806ca60 msgr2=0x7fa87815a070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.805+0000 7fa87f800640 1 -- 172.21.15.7:0/3613953194 shutdown_connections 2024-09-10T21:52:27.807 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:27.805+0000 7fa87f800640 1 -- 172.21.15.7:0/3613953194 wait complete. 2024-09-10T21:52:28.824 INFO:tasks.ceph:Scrubbing osd.0 2024-09-10T21:52:28.824 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:52:29.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:28.998+0000 7fc2c7e00640 1 -- 172.21.15.7:0/4293654067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2c80b3220 msgr2=0x7fc2c80b39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:28.998+0000 7fc2c7e00640 1 --2- 172.21.15.7:0/4293654067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2c80b3220 0x7fc2c80b39e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009830 tx=0x7fc2b802a530 comp rx=0 tx=0).stop 2024-09-10T21:52:29.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:28.998+0000 7fc2c7e00640 1 -- 172.21.15.7:0/4293654067 shutdown_connections 2024-09-10T21:52:29.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:28.998+0000 7fc2c7e00640 1 --2- 172.21.15.7:0/4293654067 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2c80b3220 0x7fc2c80b39e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:28.998+0000 7fc2c7e00640 1 -- 172.21.15.7:0/4293654067 >> 172.21.15.7:0/4293654067 conn(0x7fc2c806ce80 msgr2=0x7fc2c806d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:29.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:28.999+0000 7fc2c7e00640 1 -- 172.21.15.7:0/4293654067 shutdown_connections 2024-09-10T21:52:29.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:28.999+0000 7fc2c7e00640 1 -- 172.21.15.7:0/4293654067 wait complete. 2024-09-10T21:52:29.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.000+0000 7fc2c7e00640 1 Processor -- start 2024-09-10T21:52:29.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.000+0000 7fc2c7e00640 1 -- start start 2024-09-10T21:52:29.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.001+0000 7fc2c7e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2c80b3220 0x7fc2c81f82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:29.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.001+0000 7fc2c7e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2c80b3f20 0x7fc2c8154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:29.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.001+0000 7fc2c7e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2c80d0820 0x7fc2c80b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:29.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.001+0000 7fc2c7e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c815b960 con 0x7fc2c80d0820 2024-09-10T21:52:29.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.001+0000 7fc2c7e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fc2c815b7e0 con 0x7fc2c80b3220 2024-09-10T21:52:29.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.001+0000 7fc2c7e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fc2c815bae0 con 0x7fc2c80b3f20 2024-09-10T21:52:29.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.001+0000 7fc2c7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2c80d0820 0x7fc2c80b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:29.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.002+0000 7fc2c7400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2c80d0820 0x7fc2c80b14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40038/0 (socket says 172.21.15.7:40038) 2024-09-10T21:52:29.004 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.002+0000 7fc2c7400640 1 -- 172.21.15.7:0/3007148269 learned_addr learned my addr 172.21.15.7:0/3007148269 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.002+0000 7fc2c6a00640 1 --2- 172.21.15.7:0/3007148269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2c80b3220 0x7fc2c81f82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.002+0000 7fc2be000640 1 --2- 172.21.15.7:0/3007148269 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2c80b3f20 0x7fc2c8154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.002+0000 7fc2c7400640 1 -- 172.21.15.7:0/3007148269 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2c80b3f20 msgr2=0x7fc2c8154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.002+0000 7fc2c7400640 1 --2- 172.21.15.7:0/3007148269 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fc2c80b3f20 0x7fc2c8154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.002+0000 7fc2c7400640 1 -- 172.21.15.7:0/3007148269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2c80b3220 msgr2=0x7fc2c81f82d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.002+0000 7fc2c7400640 1 --2- 172.21.15.7:0/3007148269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2c80b3220 0x7fc2c81f82d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.002+0000 7fc2c7400640 1 -- 172.21.15.7:0/3007148269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2c8154f00 con 0x7fc2c80d0820 2024-09-10T21:52:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.002+0000 7fc2c6a00640 1 --2- 172.21.15.7:0/3007148269 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fc2c80b3220 0x7fc2c81f82d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:52:29.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.002+0000 7fc2c7400640 1 --2- 172.21.15.7:0/3007148269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2c80d0820 0x7fc2c80b14f0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000c730 tx=0x7fc2c000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.003+0000 7fc2bf400640 1 -- 172.21.15.7:0/3007148269 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c0003b70 con 0x7fc2c80d0820 2024-09-10T21:52:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.003+0000 7fc2bf400640 1 -- 172.21.15.7:0/3007148269 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2c0003d10 con 0x7fc2c80d0820 2024-09-10T21:52:29.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.003+0000 7fc2c7e00640 1 -- 172.21.15.7:0/3007148269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2c81f4790 con 0x7fc2c80d0820 2024-09-10T21:52:29.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.003+0000 7fc2bf400640 1 -- 172.21.15.7:0/3007148269 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2c000aab0 con 0x7fc2c80d0820 2024-09-10T21:52:29.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.003+0000 7fc2c7e00640 1 -- 172.21.15.7:0/3007148269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2c81f8810 con 0x7fc2c80d0820 2024-09-10T21:52:29.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.004+0000 7fc2c7e00640 1 -- 172.21.15.7:0/3007148269 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc2c81f4580 con 0x7fc2c80d0820 2024-09-10T21:52:29.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.005+0000 7fc2bf400640 1 -- 172.21.15.7:0/3007148269 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fc2c000ac50 con 0x7fc2c80d0820 2024-09-10T21:52:29.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.006+0000 7fc2bf400640 1 --2- 172.21.15.7:0/3007148269 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc2a4077f70 0x7fc2a407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:29.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.006+0000 7fc2bf400640 1 -- 172.21.15.7:0/3007148269 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7fc2c0099810 con 0x7fc2c80d0820 2024-09-10T21:52:29.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.006+0000 7fc2bf400640 1 --2- 172.21.15.7:0/3007148269 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x7fc2a407df70 0x7fc2a4080390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:29.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.006+0000 7fc2bf400640 1 -- 172.21.15.7:0/3007148269 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc2a4080a40 con 0x7fc2a407df70 2024-09-10T21:52:29.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.006+0000 7fc2bf400640 1 -- 172.21.15.7:0/3007148269 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7fc2c00a0050 con 0x7fc2c80d0820 2024-09-10T21:52:29.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.006+0000 7fc2be000640 1 --2- 172.21.15.7:0/3007148269 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x7fc2a407df70 0x7fc2a4080390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:29.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.006+0000 7fc2c6a00640 1 --2- 172.21.15.7:0/3007148269 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc2a4077f70 0x7fc2a407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:29.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.007+0000 7fc2be000640 1 --2- 172.21.15.7:0/3007148269 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x7fc2a407df70 0x7fc2a4080390 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:29.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.007+0000 7fc2c6a00640 1 --2- 172.21.15.7:0/3007148269 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc2a4077f70 0x7fc2a407a430 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8034000 tx=0x7fc2b8004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:29.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.008+0000 7fc2bf400640 1 -- 172.21.15.7:0/3007148269 <== osd.0 v2:172.21.15.7:6808/651402908 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc2a4080a40 con 0x7fc2a407df70 2024-09-10T21:52:29.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.021+0000 7fc2c7e00640 1 -- 172.21.15.7:0/3007148269 --> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc2c81f54c0 con 0x7fc2a407df70 2024-09-10T21:52:29.023 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.021+0000 7fc2bf400640 1 -- 172.21.15.7:0/3007148269 <== osd.0 v2:172.21.15.7:6808/651402908 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc2c81f54c0 con 0x7fc2a407df70 2024-09-10T21:52:29.023 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:52:29.023 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:52:29.024 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:52:29.024 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.022+0000 7fc2c7e00640 1 -- 172.21.15.7:0/3007148269 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x7fc2a407df70 msgr2=0x7fc2a4080390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.022+0000 7fc2c7e00640 1 --2- 172.21.15.7:0/3007148269 >> [v2:172.21.15.7:6808/651402908,v1:172.21.15.7:6809/651402908] conn(0x7fc2a407df70 0x7fc2a4080390 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.022+0000 7fc2c7e00640 1 -- 172.21.15.7:0/3007148269 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc2a4077f70 msgr2=0x7fc2a407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.022+0000 7fc2c7e00640 1 --2- 172.21.15.7:0/3007148269 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fc2a4077f70 0x7fc2a407a430 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8034000 tx=0x7fc2b8004730 comp rx=0 tx=0).stop 2024-09-10T21:52:29.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.022+0000 7fc2c7e00640 1 -- 172.21.15.7:0/3007148269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2c80d0820 msgr2=0x7fc2c80b14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.022+0000 7fc2c7e00640 1 --2- 172.21.15.7:0/3007148269 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fc2c80d0820 0x7fc2c80b14f0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fc2c000c730 tx=0x7fc2c000cbf0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.022+0000 7fc2c7400640 1 -- 172.21.15.7:0/3007148269 reap_dead start 2024-09-10T21:52:29.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.022+0000 7fc2c7e00640 1 -- 172.21.15.7:0/3007148269 shutdown_connections 2024-09-10T21:52:29.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.022+0000 7fc2c7e00640 1 -- 172.21.15.7:0/3007148269 >> 172.21.15.7:0/3007148269 conn(0x7fc2c806ce80 msgr2=0x7fc2c8074140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:29.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.022+0000 7fc2c7e00640 1 -- 172.21.15.7:0/3007148269 shutdown_connections 2024-09-10T21:52:29.025 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.022+0000 7fc2c7e00640 1 -- 172.21.15.7:0/3007148269 wait complete. 2024-09-10T21:52:29.038 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-10T21:52:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.198+0000 7f9383600640 1 -- 172.21.15.7:0/2337897941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f937c0cddc0 msgr2=0x7f937c072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.198+0000 7f9383600640 1 --2- 172.21.15.7:0/2337897941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f937c0cddc0 0x7f937c072f30 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f937c0767b0 tx=0x7f937402a8c0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.198+0000 7f9383600640 1 -- 172.21.15.7:0/2337897941 shutdown_connections 2024-09-10T21:52:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.198+0000 7f9383600640 1 --2- 172.21.15.7:0/2337897941 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f937c0cddc0 0x7f937c072f30 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.200 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.198+0000 7f9383600640 1 --2- 172.21.15.7:0/2337897941 >> v2:172.21.15.133:3300/0 conn(0x7f937c159910 0x7f937c15d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.198+0000 7f9383600640 1 -- 172.21.15.7:0/2337897941 >> 172.21.15.7:0/2337897941 conn(0x7f937c06ca60 msgr2=0x7f937c06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.198+0000 7f9383600640 1 -- 172.21.15.7:0/2337897941 shutdown_connections 2024-09-10T21:52:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.198+0000 7f9383600640 1 -- 172.21.15.7:0/2337897941 wait complete. 2024-09-10T21:52:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.200+0000 7f9383600640 1 Processor -- start 2024-09-10T21:52:29.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.200+0000 7f9383600640 1 -- start start 2024-09-10T21:52:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.200+0000 7f9383600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f937c06e810 0x7f937c1539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.201+0000 7f9383600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f937c0cddc0 0x7f937c1501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.201+0000 7f9383600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f937c159910 0x7f937c0afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.201+0000 7f9383600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f937c073800 con 0x7f937c159910 2024-09-10T21:52:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.201+0000 7f9383600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f937c073680 con 0x7f937c0cddc0 2024-09-10T21:52:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.201+0000 7f9383600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f937c073980 con 0x7f937c06e810 2024-09-10T21:52:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.201+0000 7f9381600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f937c159910 0x7f937c0afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.201+0000 7f9381600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f937c159910 0x7f937c0afc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40054/0 (socket says 172.21.15.7:40054) 2024-09-10T21:52:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.201+0000 7f9381600640 1 -- 172.21.15.7:0/1111814522 learned_addr learned my addr 172.21.15.7:0/1111814522 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.201+0000 7f9380c00640 1 --2- 172.21.15.7:0/1111814522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f937c06e810 0x7f937c1539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.201+0000 7f937be00640 1 --2- 172.21.15.7:0/1111814522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f937c0cddc0 0x7f937c1501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:29.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.201+0000 7f9381600640 1 -- 172.21.15.7:0/1111814522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f937c06e810 msgr2=0x7f937c1539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.201+0000 7f9381600640 1 --2- 172.21.15.7:0/1111814522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f937c06e810 0x7f937c1539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.201+0000 7f9381600640 1 -- 172.21.15.7:0/1111814522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f937c0cddc0 msgr2=0x7f937c1501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.201+0000 7f9381600640 1 --2- 172.21.15.7:0/1111814522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f937c0cddc0 0x7f937c1501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.201+0000 7f9381600640 1 -- 172.21.15.7:0/1111814522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f937c153f10 con 0x7f937c159910 2024-09-10T21:52:29.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.202+0000 7f9381600640 1 --2- 172.21.15.7:0/1111814522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f937c159910 0x7f937c0afc20 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f9374009ca0 tx=0x7f9374003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:29.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.202+0000 7f9378c00640 1 -- 172.21.15.7:0/1111814522 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374040070 con 0x7f937c159910 2024-09-10T21:52:29.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.202+0000 7f9378c00640 1 -- 172.21.15.7:0/1111814522 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9374004330 con 0x7f937c159910 2024-09-10T21:52:29.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.202+0000 7f9378c00640 1 -- 172.21.15.7:0/1111814522 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9374004760 con 0x7f937c159910 2024-09-10T21:52:29.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.202+0000 7f9383600640 1 -- 172.21.15.7:0/1111814522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f937c150700 con 0x7f937c159910 2024-09-10T21:52:29.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.202+0000 7f9383600640 1 -- 172.21.15.7:0/1111814522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f937c1f8880 con 0x7f937c159910 2024-09-10T21:52:29.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.203+0000 7f9383600640 1 -- 172.21.15.7:0/1111814522 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f937c0b2b20 con 0x7f937c159910 2024-09-10T21:52:29.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.204+0000 7f9378c00640 1 -- 172.21.15.7:0/1111814522 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f937404d030 con 0x7f937c159910 2024-09-10T21:52:29.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.205+0000 7f9378c00640 1 --2- 172.21.15.7:0/1111814522 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f935c077f70 0x7f935c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:29.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.205+0000 7f9378c00640 1 -- 172.21.15.7:0/1111814522 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7f93740bb300 con 0x7f937c159910 2024-09-10T21:52:29.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.206+0000 7f9380c00640 1 --2- 172.21.15.7:0/1111814522 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f935c077f70 0x7f935c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:29.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.207+0000 7f9380c00640 1 --2- 172.21.15.7:0/1111814522 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f935c077f70 0x7f935c07a430 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f937000bb50 tx=0x7f937000b2e0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:29.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.210+0000 7f9378c00640 1 -- 172.21.15.7:0/1111814522 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f937c150700 con 0x7f937c159910 2024-09-10T21:52:29.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.407+0000 7f9383600640 1 -- 172.21.15.7:0/1111814522 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f937c050fa0 con 0x7f935c077f70 2024-09-10T21:52:29.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.408+0000 7f9378c00640 1 -- 172.21.15.7:0/1111814522 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f937c050fa0 con 0x7f935c077f70 2024-09-10T21:52:29.410 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 0 to deep-scrub 2024-09-10T21:52:29.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.410+0000 7f9383600640 1 -- 172.21.15.7:0/1111814522 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f935c077f70 msgr2=0x7f935c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.410+0000 7f9383600640 1 --2- 172.21.15.7:0/1111814522 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f935c077f70 0x7f935c07a430 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f937000bb50 tx=0x7f937000b2e0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.410+0000 7f9383600640 1 -- 172.21.15.7:0/1111814522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f937c159910 msgr2=0x7f937c0afc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.410+0000 7f9383600640 1 --2- 172.21.15.7:0/1111814522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f937c159910 0x7f937c0afc20 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f9374009ca0 tx=0x7f9374003ba0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.410+0000 7f9383600640 1 -- 172.21.15.7:0/1111814522 shutdown_connections 2024-09-10T21:52:29.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.410+0000 7f9383600640 1 --2- 172.21.15.7:0/1111814522 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f935c077f70 0x7f935c07a430 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.410+0000 7f9383600640 1 --2- 172.21.15.7:0/1111814522 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f937c0cddc0 0x7f937c1501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.412 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.410+0000 7f9383600640 1 --2- 172.21.15.7:0/1111814522 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f937c159910 0x7f937c0afc20 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.410+0000 7f9383600640 1 --2- 172.21.15.7:0/1111814522 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f937c06e810 0x7f937c1539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.410+0000 7f9383600640 1 -- 172.21.15.7:0/1111814522 >> 172.21.15.7:0/1111814522 conn(0x7f937c06ca60 msgr2=0x7f937c15a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:29.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.410+0000 7f9383600640 1 -- 172.21.15.7:0/1111814522 shutdown_connections 2024-09-10T21:52:29.413 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.410+0000 7f9383600640 1 -- 172.21.15.7:0/1111814522 wait complete. 2024-09-10T21:52:29.428 INFO:tasks.ceph:Scrubbing osd.1 2024-09-10T21:52:29.428 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:52:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.591+0000 7fad81200640 1 -- 172.21.15.7:0/3823234572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad7c06ff60 msgr2=0x7fad7c0ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.591+0000 7fad81200640 1 --2- 172.21.15.7:0/3823234572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad7c06ff60 0x7fad7c0ce150 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fad74005fd0 tx=0x7fad7402a980 comp rx=0 tx=0).stop 2024-09-10T21:52:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.591+0000 7fad81200640 1 -- 172.21.15.7:0/3823234572 shutdown_connections 2024-09-10T21:52:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.591+0000 7fad81200640 1 --2- 172.21.15.7:0/3823234572 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad7c06ff60 0x7fad7c0ce150 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.591+0000 7fad81200640 1 --2- 172.21.15.7:0/3823234572 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad7c0cd340 0x7fad7c073280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.591+0000 7fad81200640 1 -- 172.21.15.7:0/3823234572 >> 172.21.15.7:0/3823234572 conn(0x7fad7c06ce50 msgr2=0x7fad7c06d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:29.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.591+0000 7fad81200640 1 -- 172.21.15.7:0/3823234572 shutdown_connections 2024-09-10T21:52:29.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.591+0000 7fad81200640 1 -- 172.21.15.7:0/3823234572 wait complete. 2024-09-10T21:52:29.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.592+0000 7fad81200640 1 Processor -- start 2024-09-10T21:52:29.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.593+0000 7fad81200640 1 -- start start 2024-09-10T21:52:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.593+0000 7fad81200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7c06ff60 0x7fad7c13d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.593+0000 7fad81200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad7c0cd340 0x7fad7c0b0d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.593+0000 7fad81200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad7c145610 0x7fad7c1f4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.593+0000 7fad81200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c06f1b0 con 0x7fad7c06ff60 2024-09-10T21:52:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.593+0000 7fad81200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fad7c06f030 con 0x7fad7c0cd340 2024-09-10T21:52:29.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.593+0000 7fad81200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fad7c06f330 con 0x7fad7c145610 2024-09-10T21:52:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.594+0000 7fad79e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7c06ff60 0x7fad7c13d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.594+0000 7fad79e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7c06ff60 0x7fad7c13d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40064/0 (socket says 172.21.15.7:40064) 2024-09-10T21:52:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.594+0000 7fad79e00640 1 -- 172.21.15.7:0/53204888 learned_addr learned my addr 172.21.15.7:0/53204888 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.594+0000 7fad71400640 1 --2- 172.21.15.7:0/53204888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad7c0cd340 0x7fad7c0b0d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.594+0000 7fad7a800640 1 --2- 172.21.15.7:0/53204888 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad7c145610 0x7fad7c1f4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.594+0000 7fad79e00640 1 -- 172.21.15.7:0/53204888 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad7c145610 msgr2=0x7fad7c1f4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.594+0000 7fad79e00640 1 --2- 172.21.15.7:0/53204888 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fad7c145610 0x7fad7c1f4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.594+0000 7fad79e00640 1 -- 172.21.15.7:0/53204888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad7c0cd340 msgr2=0x7fad7c0b0d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.594+0000 7fad79e00640 1 --2- 172.21.15.7:0/53204888 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fad7c0cd340 0x7fad7c0b0d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.594+0000 7fad79e00640 1 -- 172.21.15.7:0/53204888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad7c13dcd0 con 0x7fad7c06ff60 2024-09-10T21:52:29.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.594+0000 7fad79e00640 1 --2- 172.21.15.7:0/53204888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7c06ff60 0x7fad7c13d790 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c3a0 tx=0x7fad6c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:29.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.595+0000 7fad72a00640 1 -- 172.21.15.7:0/53204888 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c017070 con 0x7fad7c06ff60 2024-09-10T21:52:29.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.595+0000 7fad72a00640 1 -- 172.21.15.7:0/53204888 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad6c0053c0 con 0x7fad7c06ff60 2024-09-10T21:52:29.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.595+0000 7fad72a00640 1 -- 172.21.15.7:0/53204888 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad6c003850 con 0x7fad7c06ff60 2024-09-10T21:52:29.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.595+0000 7fad81200640 1 -- 172.21.15.7:0/53204888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad7c0afcb0 con 0x7fad7c06ff60 2024-09-10T21:52:29.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.595+0000 7fad81200640 1 -- 172.21.15.7:0/53204888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad7c0b0270 con 0x7fad7c06ff60 2024-09-10T21:52:29.598 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.596+0000 7fad81200640 1 -- 172.21.15.7:0/53204888 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fad7c06f850 con 0x7fad7c06ff60 2024-09-10T21:52:29.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.597+0000 7fad72a00640 1 -- 172.21.15.7:0/53204888 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fad6c0039f0 con 0x7fad7c06ff60 2024-09-10T21:52:29.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.598+0000 7fad72a00640 1 --2- 172.21.15.7:0/53204888 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fad58077fc0 0x7fad5807a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:29.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.598+0000 7fad72a00640 1 -- 172.21.15.7:0/53204888 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7fad6c0997b0 con 0x7fad7c06ff60 2024-09-10T21:52:29.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.598+0000 7fad72a00640 1 --2- 172.21.15.7:0/53204888 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x7fad5807dfc0 0x7fad580803e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:29.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.598+0000 7fad72a00640 1 -- 172.21.15.7:0/53204888 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fad6c009e50 con 0x7fad5807dfc0 2024-09-10T21:52:29.600 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.598+0000 7fad72a00640 1 -- 172.21.15.7:0/53204888 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7fad6c099b60 con 0x7fad7c06ff60 2024-09-10T21:52:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.599+0000 7fad71400640 1 --2- 172.21.15.7:0/53204888 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fad58077fc0 0x7fad5807a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.599+0000 7fad7a800640 1 --2- 172.21.15.7:0/53204888 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x7fad5807dfc0 0x7fad580803e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.599+0000 7fad7a800640 1 --2- 172.21.15.7:0/53204888 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x7fad5807dfc0 0x7fad580803e0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:29.601 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.599+0000 7fad71400640 1 --2- 172.21.15.7:0/53204888 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fad58077fc0 0x7fad5807a480 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0b2af0 tx=0x7fad64005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:29.602 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.600+0000 7fad72a00640 1 -- 172.21.15.7:0/53204888 <== osd.1 v2:172.21.15.7:6824/4175601737 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fad6c009e50 con 0x7fad5807dfc0 2024-09-10T21:52:29.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.613+0000 7fad81200640 1 -- 172.21.15.7:0/53204888 --> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fad7c15b970 con 0x7fad5807dfc0 2024-09-10T21:52:29.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.614+0000 7fad72a00640 1 -- 172.21.15.7:0/53204888 <== osd.1 v2:172.21.15.7:6824/4175601737 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fad7c15b970 con 0x7fad5807dfc0 2024-09-10T21:52:29.616 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:52:29.616 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:52:29.616 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:52:29.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.614+0000 7fad81200640 1 -- 172.21.15.7:0/53204888 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x7fad5807dfc0 msgr2=0x7fad580803e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.614+0000 7fad81200640 1 --2- 172.21.15.7:0/53204888 >> [v2:172.21.15.7:6824/4175601737,v1:172.21.15.7:6825/4175601737] conn(0x7fad5807dfc0 0x7fad580803e0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.614+0000 7fad81200640 1 -- 172.21.15.7:0/53204888 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fad58077fc0 msgr2=0x7fad5807a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.614+0000 7fad81200640 1 --2- 172.21.15.7:0/53204888 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fad58077fc0 0x7fad5807a480 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fad7c0b2af0 tx=0x7fad64005f90 comp rx=0 tx=0).stop 2024-09-10T21:52:29.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.614+0000 7fad81200640 1 -- 172.21.15.7:0/53204888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7c06ff60 msgr2=0x7fad7c13d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.614+0000 7fad81200640 1 --2- 172.21.15.7:0/53204888 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fad7c06ff60 0x7fad7c13d790 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7fad6c00c3a0 tx=0x7fad6c00c860 comp rx=0 tx=0).stop 2024-09-10T21:52:29.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.614+0000 7fad7a800640 1 -- 172.21.15.7:0/53204888 reap_dead start 2024-09-10T21:52:29.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.614+0000 7fad81200640 1 -- 172.21.15.7:0/53204888 shutdown_connections 2024-09-10T21:52:29.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.614+0000 7fad81200640 1 -- 172.21.15.7:0/53204888 >> 172.21.15.7:0/53204888 conn(0x7fad7c06ce50 msgr2=0x7fad7c15d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:29.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.615+0000 7fad81200640 1 -- 172.21.15.7:0/53204888 shutdown_connections 2024-09-10T21:52:29.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.615+0000 7fad81200640 1 -- 172.21.15.7:0/53204888 wait complete. 2024-09-10T21:52:29.630 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-10T21:52:29.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.789+0000 7f1e27e00640 1 -- 172.21.15.7:0/2122326261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e200cddc0 msgr2=0x7f1e20072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.789+0000 7f1e27e00640 1 --2- 172.21.15.7:0/2122326261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e200cddc0 0x7f1e20072f30 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f1e200767b0 tx=0x7f1e1802aac0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.789+0000 7f1e27e00640 1 -- 172.21.15.7:0/2122326261 shutdown_connections 2024-09-10T21:52:29.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.789+0000 7f1e27e00640 1 --2- 172.21.15.7:0/2122326261 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e200cddc0 0x7f1e20072f30 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.789+0000 7f1e27e00640 1 --2- 172.21.15.7:0/2122326261 >> v2:172.21.15.133:3300/0 conn(0x7f1e20159910 0x7f1e2015d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.789+0000 7f1e27e00640 1 -- 172.21.15.7:0/2122326261 >> 172.21.15.7:0/2122326261 conn(0x7f1e2006ca60 msgr2=0x7f1e2006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:29.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.790+0000 7f1e27e00640 1 -- 172.21.15.7:0/2122326261 shutdown_connections 2024-09-10T21:52:29.792 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.790+0000 7f1e27e00640 1 -- 172.21.15.7:0/2122326261 wait complete. 2024-09-10T21:52:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.791+0000 7f1e27e00640 1 Processor -- start 2024-09-10T21:52:29.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.791+0000 7f1e27e00640 1 -- start start 2024-09-10T21:52:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.792+0000 7f1e27e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e2006e810 0x7f1e201f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.792+0000 7f1e27e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e200cddc0 0x7f1e200af0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.792+0000 7f1e27e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e20159910 0x7f1e20156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.792+0000 7f1e27e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f1e20073800 con 0x7f1e20159910 2024-09-10T21:52:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.792+0000 7f1e27e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f1e20073680 con 0x7f1e200cddc0 2024-09-10T21:52:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.792+0000 7f1e27e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f1e20073980 con 0x7f1e2006e810 2024-09-10T21:52:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.793+0000 7f1e25e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e20159910 0x7f1e20156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.793+0000 7f1e25e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e20159910 0x7f1e20156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40084/0 (socket says 172.21.15.7:40084) 2024-09-10T21:52:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.793+0000 7f1e25e00640 1 -- 172.21.15.7:0/1958087078 learned_addr learned my addr 172.21.15.7:0/1958087078 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:29.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.793+0000 7f1e24a00640 1 --2- 172.21.15.7:0/1958087078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e200cddc0 0x7f1e200af0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:29.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.793+0000 7f1e25400640 1 --2- 172.21.15.7:0/1958087078 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e2006e810 0x7f1e201f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:29.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.793+0000 7f1e25e00640 1 -- 172.21.15.7:0/1958087078 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e2006e810 msgr2=0x7f1e201f50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.793+0000 7f1e25e00640 1 --2- 172.21.15.7:0/1958087078 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e2006e810 0x7f1e201f50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.793+0000 7f1e25e00640 1 -- 172.21.15.7:0/1958087078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e200cddc0 msgr2=0x7f1e200af0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:29.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.793+0000 7f1e25e00640 1 --2- 172.21.15.7:0/1958087078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e200cddc0 0x7f1e200af0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:29.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.793+0000 7f1e25e00640 1 -- 172.21.15.7:0/1958087078 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e200b3150 con 0x7f1e20159910 2024-09-10T21:52:29.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.793+0000 7f1e24a00640 1 --2- 172.21.15.7:0/1958087078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e200cddc0 0x7f1e200af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:29.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.793+0000 7f1e25400640 1 --2- 172.21.15.7:0/1958087078 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e2006e810 0x7f1e201f50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:29.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.793+0000 7f1e25e00640 1 --2- 172.21.15.7:0/1958087078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e20159910 0x7f1e20156160 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009d60 tx=0x7f1e18003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:29.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.794+0000 7f1e1d600640 1 -- 172.21.15.7:0/1958087078 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e18044070 con 0x7f1e20159910 2024-09-10T21:52:29.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.794+0000 7f1e1d600640 1 -- 172.21.15.7:0/1958087078 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e18004250 con 0x7f1e20159910 2024-09-10T21:52:29.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.794+0000 7f1e1d600640 1 -- 172.21.15.7:0/1958087078 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e180046a0 con 0x7f1e20159910 2024-09-10T21:52:29.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.794+0000 7f1e27e00640 1 -- 172.21.15.7:0/1958087078 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e200b3430 con 0x7f1e20159910 2024-09-10T21:52:29.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.794+0000 7f1e27e00640 1 -- 172.21.15.7:0/1958087078 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e200b39f0 con 0x7f1e20159910 2024-09-10T21:52:29.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.795+0000 7f1e27e00640 1 -- 172.21.15.7:0/1958087078 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e20072f30 con 0x7f1e20159910 2024-09-10T21:52:29.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.796+0000 7f1e1d600640 1 -- 172.21.15.7:0/1958087078 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f1e18036400 con 0x7f1e20159910 2024-09-10T21:52:29.798 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.797+0000 7f1e1d600640 1 --2- 172.21.15.7:0/1958087078 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1e00077fa0 0x7f1e0007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:29.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.797+0000 7f1e1d600640 1 -- 172.21.15.7:0/1958087078 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7f1e180ba150 con 0x7f1e20159910 2024-09-10T21:52:29.799 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.798+0000 7f1e25400640 1 --2- 172.21.15.7:0/1958087078 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1e00077fa0 0x7f1e0007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:29.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.798+0000 7f1e25400640 1 --2- 172.21.15.7:0/1958087078 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1e00077fa0 0x7f1e0007a460 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1e14005fd0 tx=0x7f1e1400f360 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:29.803 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.801+0000 7f1e1d600640 1 -- 172.21.15.7:0/1958087078 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f1e18032030 con 0x7f1e20159910 2024-09-10T21:52:29.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.995+0000 7f1e27e00640 1 -- 172.21.15.7:0/1958087078 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f1e2015bd20 con 0x7f1e00077fa0 2024-09-10T21:52:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.996+0000 7f1e1d600640 1 -- 172.21.15.7:0/1958087078 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1e2015bd20 con 0x7f1e00077fa0 2024-09-10T21:52:29.998 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 1 to deep-scrub 2024-09-10T21:52:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.998+0000 7f1e27e00640 1 -- 172.21.15.7:0/1958087078 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1e00077fa0 msgr2=0x7f1e0007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.998+0000 7f1e27e00640 1 --2- 172.21.15.7:0/1958087078 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1e00077fa0 0x7f1e0007a460 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f1e14005fd0 tx=0x7f1e1400f360 comp rx=0 tx=0).stop 2024-09-10T21:52:30.000 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.998+0000 7f1e27e00640 1 -- 172.21.15.7:0/1958087078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e20159910 msgr2=0x7f1e20156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.998+0000 7f1e27e00640 1 --2- 172.21.15.7:0/1958087078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e20159910 0x7f1e20156160 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009d60 tx=0x7f1e18003ce0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.998+0000 7f1e27e00640 1 -- 172.21.15.7:0/1958087078 shutdown_connections 2024-09-10T21:52:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.998+0000 7f1e27e00640 1 --2- 172.21.15.7:0/1958087078 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f1e00077fa0 0x7f1e0007a460 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.998+0000 7f1e27e00640 1 --2- 172.21.15.7:0/1958087078 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f1e200cddc0 0x7f1e200af0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.998+0000 7f1e27e00640 1 --2- 172.21.15.7:0/1958087078 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f1e20159910 0x7f1e20156160 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.998+0000 7f1e27e00640 1 --2- 172.21.15.7:0/1958087078 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f1e2006e810 0x7f1e201f50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.998+0000 7f1e27e00640 1 -- 172.21.15.7:0/1958087078 >> 172.21.15.7:0/1958087078 conn(0x7f1e2006ca60 msgr2=0x7f1e2015d600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.998+0000 7f1e27e00640 1 -- 172.21.15.7:0/1958087078 shutdown_connections 2024-09-10T21:52:30.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:29.998+0000 7f1e27e00640 1 -- 172.21.15.7:0/1958087078 wait complete. 2024-09-10T21:52:30.017 INFO:tasks.ceph:Scrubbing osd.2 2024-09-10T21:52:30.017 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:52:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.180+0000 7fd43ee00640 1 -- 172.21.15.7:0/937557347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4380aef60 msgr2=0x7fd4380af340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.180+0000 7fd43ee00640 1 --2- 172.21.15.7:0/937557347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4380aef60 0x7fd4380af340 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fd42c009830 tx=0x7fd42c02a510 comp rx=0 tx=0).stop 2024-09-10T21:52:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.181+0000 7fd43ee00640 1 -- 172.21.15.7:0/937557347 shutdown_connections 2024-09-10T21:52:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.181+0000 7fd43ee00640 1 --2- 172.21.15.7:0/937557347 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4380d07d0 0x7fd4380b4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.181+0000 7fd43ee00640 1 --2- 172.21.15.7:0/937557347 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4380aef60 0x7fd4380af340 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.181+0000 7fd43ee00640 1 -- 172.21.15.7:0/937557347 >> 172.21.15.7:0/937557347 conn(0x7fd43806ce80 msgr2=0x7fd43806d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.181+0000 7fd43ee00640 1 -- 172.21.15.7:0/937557347 shutdown_connections 2024-09-10T21:52:30.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.181+0000 7fd43ee00640 1 -- 172.21.15.7:0/937557347 wait complete. 2024-09-10T21:52:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.182+0000 7fd43ee00640 1 Processor -- start 2024-09-10T21:52:30.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.182+0000 7fd43ee00640 1 -- start start 2024-09-10T21:52:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.183+0000 7fd43ee00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43806fea0 0x7fd4380b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.183+0000 7fd43ee00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4380aef60 0x7fd4381561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.183+0000 7fd43ee00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4380d07d0 0x7fd438151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.183+0000 7fd43ee00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd438162130 con 0x7fd43806fea0 2024-09-10T21:52:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.183+0000 7fd43ee00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd438161fb0 con 0x7fd4380d07d0 2024-09-10T21:52:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.183+0000 7fd43ee00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fd4381622b0 con 0x7fd4380aef60 2024-09-10T21:52:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.183+0000 7fd43da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43806fea0 0x7fd4380b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:30.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.184+0000 7fd43da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43806fea0 0x7fd4380b1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40088/0 (socket says 172.21.15.7:40088) 2024-09-10T21:52:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.184+0000 7fd43da00640 1 -- 172.21.15.7:0/930690171 learned_addr learned my addr 172.21.15.7:0/930690171 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.184+0000 7fd43d000640 1 --2- 172.21.15.7:0/930690171 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4380aef60 0x7fd4381561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.184+0000 7fd43e400640 1 --2- 172.21.15.7:0/930690171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4380d07d0 0x7fd438151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.184+0000 7fd43da00640 1 -- 172.21.15.7:0/930690171 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4380aef60 msgr2=0x7fd4381561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.184+0000 7fd43da00640 1 --2- 172.21.15.7:0/930690171 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd4380aef60 0x7fd4381561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.184+0000 7fd43da00640 1 -- 172.21.15.7:0/930690171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4380d07d0 msgr2=0x7fd438151440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.184+0000 7fd43da00640 1 --2- 172.21.15.7:0/930690171 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd4380d07d0 0x7fd438151440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.184+0000 7fd43da00640 1 -- 172.21.15.7:0/930690171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd438155710 con 0x7fd43806fea0 2024-09-10T21:52:30.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.184+0000 7fd43da00640 1 --2- 172.21.15.7:0/930690171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43806fea0 0x7fd4380b1e10 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fd42c02ab00 tx=0x7fd42c032c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.185+0000 7fd435600640 1 -- 172.21.15.7:0/930690171 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c040070 con 0x7fd43806fea0 2024-09-10T21:52:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.185+0000 7fd435600640 1 -- 172.21.15.7:0/930690171 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd42c005310 con 0x7fd43806fea0 2024-09-10T21:52:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.185+0000 7fd43ee00640 1 -- 172.21.15.7:0/930690171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd438158000 con 0x7fd43806fea0 2024-09-10T21:52:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.185+0000 7fd435600640 1 -- 172.21.15.7:0/930690171 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd42c005c90 con 0x7fd43806fea0 2024-09-10T21:52:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.185+0000 7fd43ee00640 1 -- 172.21.15.7:0/930690171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4380b2350 con 0x7fd43806fea0 2024-09-10T21:52:30.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.186+0000 7fd43ee00640 1 -- 172.21.15.7:0/930690171 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd438162760 con 0x7fd43806fea0 2024-09-10T21:52:30.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.187+0000 7fd435600640 1 -- 172.21.15.7:0/930690171 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd42c04c030 con 0x7fd43806fea0 2024-09-10T21:52:30.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.188+0000 7fd435600640 1 --2- 172.21.15.7:0/930690171 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd418077f70 0x7fd41807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:30.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.188+0000 7fd435600640 1 -- 172.21.15.7:0/930690171 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7fd42c0bb1d0 con 0x7fd43806fea0 2024-09-10T21:52:30.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.188+0000 7fd435600640 1 --2- 172.21.15.7:0/930690171 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x7fd41807df70 0x7fd418080390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:30.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.188+0000 7fd435600640 1 -- 172.21.15.7:0/930690171 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd42c046e80 con 0x7fd41807df70 2024-09-10T21:52:30.190 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.188+0000 7fd435600640 1 -- 172.21.15.7:0/930690171 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7fd42c0bb580 con 0x7fd43806fea0 2024-09-10T21:52:30.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.188+0000 7fd43e400640 1 --2- 172.21.15.7:0/930690171 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x7fd41807df70 0x7fd418080390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:30.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.188+0000 7fd43d000640 1 --2- 172.21.15.7:0/930690171 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd418077f70 0x7fd41807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:30.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.189+0000 7fd43e400640 1 --2- 172.21.15.7:0/930690171 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x7fd41807df70 0x7fd418080390 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:30.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.189+0000 7fd43d000640 1 --2- 172.21.15.7:0/930690171 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd418077f70 0x7fd41807a430 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd424007fd0 tx=0x7fd424001370 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:30.191 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.190+0000 7fd435600640 1 -- 172.21.15.7:0/930690171 <== osd.2 v2:172.21.15.7:6816/4238877366 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd42c046e80 con 0x7fd41807df70 2024-09-10T21:52:30.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.203+0000 7fd43ee00640 1 -- 172.21.15.7:0/930690171 --> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd43815dc30 con 0x7fd41807df70 2024-09-10T21:52:30.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.203+0000 7fd435600640 1 -- 172.21.15.7:0/930690171 <== osd.2 v2:172.21.15.7:6816/4238877366 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd43815dc30 con 0x7fd41807df70 2024-09-10T21:52:30.205 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:52:30.205 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:52:30.205 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:52:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.203+0000 7fd43ee00640 1 -- 172.21.15.7:0/930690171 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x7fd41807df70 msgr2=0x7fd418080390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.203+0000 7fd43ee00640 1 --2- 172.21.15.7:0/930690171 >> [v2:172.21.15.7:6816/4238877366,v1:172.21.15.7:6817/4238877366] conn(0x7fd41807df70 0x7fd418080390 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.204+0000 7fd43ee00640 1 -- 172.21.15.7:0/930690171 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd418077f70 msgr2=0x7fd41807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.204+0000 7fd43ee00640 1 --2- 172.21.15.7:0/930690171 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd418077f70 0x7fd41807a430 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd424007fd0 tx=0x7fd424001370 comp rx=0 tx=0).stop 2024-09-10T21:52:30.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.204+0000 7fd43ee00640 1 -- 172.21.15.7:0/930690171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43806fea0 msgr2=0x7fd4380b1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.204+0000 7fd43ee00640 1 --2- 172.21.15.7:0/930690171 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd43806fea0 0x7fd4380b1e10 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fd42c02ab00 tx=0x7fd42c032c10 comp rx=0 tx=0).stop 2024-09-10T21:52:30.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.204+0000 7fd43e400640 1 -- 172.21.15.7:0/930690171 reap_dead start 2024-09-10T21:52:30.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.204+0000 7fd43ee00640 1 -- 172.21.15.7:0/930690171 shutdown_connections 2024-09-10T21:52:30.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.204+0000 7fd43ee00640 1 -- 172.21.15.7:0/930690171 >> 172.21.15.7:0/930690171 conn(0x7fd43806ce80 msgr2=0x7fd4380d00c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:30.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.204+0000 7fd43ee00640 1 -- 172.21.15.7:0/930690171 shutdown_connections 2024-09-10T21:52:30.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.204+0000 7fd43ee00640 1 -- 172.21.15.7:0/930690171 wait complete. 2024-09-10T21:52:30.220 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-10T21:52:30.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.381+0000 7fe2b9600640 1 -- 172.21.15.7:0/2377880639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2b415e100 msgr2=0x7fe2b40d4950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.381+0000 7fe2b9600640 1 --2- 172.21.15.7:0/2377880639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2b415e100 0x7fe2b40d4950 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4009830 tx=0x7fe2a402a5a0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.381+0000 7fe2b9600640 1 -- 172.21.15.7:0/2377880639 shutdown_connections 2024-09-10T21:52:30.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.381+0000 7fe2b9600640 1 --2- 172.21.15.7:0/2377880639 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2b40d0f90 0x7fe2b40b7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.381+0000 7fe2b9600640 1 --2- 172.21.15.7:0/2377880639 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2b415e100 0x7fe2b40d4950 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.381+0000 7fe2b9600640 1 -- 172.21.15.7:0/2377880639 >> 172.21.15.7:0/2377880639 conn(0x7fe2b409d470 msgr2=0x7fe2b40d4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:30.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.381+0000 7fe2b9600640 1 -- 172.21.15.7:0/2377880639 shutdown_connections 2024-09-10T21:52:30.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.381+0000 7fe2b9600640 1 -- 172.21.15.7:0/2377880639 wait complete. 2024-09-10T21:52:30.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.383+0000 7fe2b9600640 1 Processor -- start 2024-09-10T21:52:30.384 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.383+0000 7fe2b9600640 1 -- start start 2024-09-10T21:52:30.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.384+0000 7fe2b9600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b40d0f90 0x7fe2b414c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:30.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.384+0000 7fe2b9600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2b40d4e90 0x7fe2b4156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:30.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.384+0000 7fe2b9600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2b415e100 0x7fe2b41570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:30.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.384+0000 7fe2b9600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b40b8220 con 0x7fe2b40d0f90 2024-09-10T21:52:30.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.384+0000 7fe2b9600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b40b80a0 con 0x7fe2b40d4e90 2024-09-10T21:52:30.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.384+0000 7fe2b9600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fe2b40b83a0 con 0x7fe2b415e100 2024-09-10T21:52:30.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.384+0000 7fe2b2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b40d0f90 0x7fe2b414c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:30.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.384+0000 7fe2b2a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b40d0f90 0x7fe2b414c660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40104/0 (socket says 172.21.15.7:40104) 2024-09-10T21:52:30.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.384+0000 7fe2b2a00640 1 -- 172.21.15.7:0/131571927 learned_addr learned my addr 172.21.15.7:0/131571927 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:30.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.384+0000 7fe2aa000640 1 --2- 172.21.15.7:0/131571927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2b40d4e90 0x7fe2b4156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:30.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.384+0000 7fe2b3400640 1 --2- 172.21.15.7:0/131571927 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2b415e100 0x7fe2b41570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:30.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.384+0000 7fe2b2a00640 1 -- 172.21.15.7:0/131571927 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2b415e100 msgr2=0x7fe2b41570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.384+0000 7fe2b2a00640 1 --2- 172.21.15.7:0/131571927 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2b415e100 0x7fe2b41570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.384+0000 7fe2b2a00640 1 -- 172.21.15.7:0/131571927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2b40d4e90 msgr2=0x7fe2b4156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.384+0000 7fe2b2a00640 1 --2- 172.21.15.7:0/131571927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2b40d4e90 0x7fe2b4156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.384+0000 7fe2b2a00640 1 -- 172.21.15.7:0/131571927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b414cba0 con 0x7fe2b40d0f90 2024-09-10T21:52:30.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.385+0000 7fe2b2a00640 1 --2- 172.21.15.7:0/131571927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b40d0f90 0x7fe2b414c660 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4000c00 tx=0x7fe2a4033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:30.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.385+0000 7fe2ab400640 1 -- 172.21.15.7:0/131571927 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4031d60 con 0x7fe2b40d0f90 2024-09-10T21:52:30.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.385+0000 7fe2ab400640 1 -- 172.21.15.7:0/131571927 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe2a4033e50 con 0x7fe2b40d0f90 2024-09-10T21:52:30.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.385+0000 7fe2ab400640 1 -- 172.21.15.7:0/131571927 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4033310 con 0x7fe2b40d0f90 2024-09-10T21:52:30.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.385+0000 7fe2b9600640 1 -- 172.21.15.7:0/131571927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b4157a30 con 0x7fe2b40d0f90 2024-09-10T21:52:30.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.385+0000 7fe2b9600640 1 -- 172.21.15.7:0/131571927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b41582d0 con 0x7fe2b40d0f90 2024-09-10T21:52:30.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.386+0000 7fe2b9600640 1 -- 172.21.15.7:0/131571927 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe2b40b8850 con 0x7fe2b40d0f90 2024-09-10T21:52:30.389 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.387+0000 7fe2ab400640 1 -- 172.21.15.7:0/131571927 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fe2a404b030 con 0x7fe2b40d0f90 2024-09-10T21:52:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.388+0000 7fe2ab400640 1 --2- 172.21.15.7:0/131571927 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe290077fa0 0x7fe29007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.388+0000 7fe2ab400640 1 -- 172.21.15.7:0/131571927 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7fe2a40ba890 con 0x7fe2b40d0f90 2024-09-10T21:52:30.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.389+0000 7fe2aa000640 1 --2- 172.21.15.7:0/131571927 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe290077fa0 0x7fe29007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:30.391 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.389+0000 7fe2aa000640 1 --2- 172.21.15.7:0/131571927 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe290077fa0 0x7fe29007a460 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe2b41f8270 tx=0x7fe29c009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:30.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.392+0000 7fe2ab400640 1 -- 172.21.15.7:0/131571927 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fe2a403f030 con 0x7fe2b40d0f90 2024-09-10T21:52:30.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.587+0000 7fe2b9600640 1 -- 172.21.15.7:0/131571927 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7fe2b4157c00 con 0x7fe290077fa0 2024-09-10T21:52:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.589+0000 7fe2ab400640 1 -- 172.21.15.7:0/131571927 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fe2b4157c00 con 0x7fe290077fa0 2024-09-10T21:52:30.590 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 2 to deep-scrub 2024-09-10T21:52:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.590+0000 7fe2b9600640 1 -- 172.21.15.7:0/131571927 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe290077fa0 msgr2=0x7fe29007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.590+0000 7fe2b9600640 1 --2- 172.21.15.7:0/131571927 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe290077fa0 0x7fe29007a460 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe2b41f8270 tx=0x7fe29c009040 comp rx=0 tx=0).stop 2024-09-10T21:52:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.590+0000 7fe2b9600640 1 -- 172.21.15.7:0/131571927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b40d0f90 msgr2=0x7fe2b414c660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.590+0000 7fe2b9600640 1 --2- 172.21.15.7:0/131571927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b40d0f90 0x7fe2b414c660 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4000c00 tx=0x7fe2a4033ba0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.591+0000 7fe2b9600640 1 -- 172.21.15.7:0/131571927 shutdown_connections 2024-09-10T21:52:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.591+0000 7fe2b9600640 1 --2- 172.21.15.7:0/131571927 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fe290077fa0 0x7fe29007a460 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.591+0000 7fe2b9600640 1 --2- 172.21.15.7:0/131571927 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fe2b40d4e90 0x7fe2b4156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.591+0000 7fe2b9600640 1 --2- 172.21.15.7:0/131571927 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fe2b415e100 0x7fe2b41570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.591+0000 7fe2b9600640 1 --2- 172.21.15.7:0/131571927 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fe2b40d0f90 0x7fe2b414c660 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.591+0000 7fe2b9600640 1 -- 172.21.15.7:0/131571927 >> 172.21.15.7:0/131571927 conn(0x7fe2b409d470 msgr2=0x7fe2b40cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:30.593 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.591+0000 7fe2b9600640 1 -- 172.21.15.7:0/131571927 shutdown_connections 2024-09-10T21:52:30.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.591+0000 7fe2b9600640 1 -- 172.21.15.7:0/131571927 wait complete. 2024-09-10T21:52:30.609 INFO:tasks.ceph:Scrubbing osd.3 2024-09-10T21:52:30.609 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:52:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.770+0000 7fd4a0c00640 1 -- 172.21.15.7:0/416039315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd49c154ec0 msgr2=0x7fd49c0ae290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.770+0000 7fd4a0c00640 1 --2- 172.21.15.7:0/416039315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd49c154ec0 0x7fd49c0ae290 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7fd49c0b2f20 tx=0x7fd48c02a4d0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.771+0000 7fd4a0c00640 1 -- 172.21.15.7:0/416039315 shutdown_connections 2024-09-10T21:52:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.771+0000 7fd4a0c00640 1 --2- 172.21.15.7:0/416039315 >> v2:172.21.15.133:3300/0 conn(0x7fd49c0cd170 0x7fd49c06eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.771+0000 7fd4a0c00640 1 --2- 172.21.15.7:0/416039315 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd49c1552a0 0x7fd49c0ccc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.771+0000 7fd4a0c00640 1 --2- 172.21.15.7:0/416039315 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd49c154ec0 0x7fd49c0ae290 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.773 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.771+0000 7fd4a0c00640 1 -- 172.21.15.7:0/416039315 >> 172.21.15.7:0/416039315 conn(0x7fd49c06ce50 msgr2=0x7fd49c06d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.771+0000 7fd4a0c00640 1 -- 172.21.15.7:0/416039315 shutdown_connections 2024-09-10T21:52:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.771+0000 7fd4a0c00640 1 -- 172.21.15.7:0/416039315 wait complete. 2024-09-10T21:52:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.772+0000 7fd4a0c00640 1 Processor -- start 2024-09-10T21:52:30.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.772+0000 7fd4a0c00640 1 -- start start 2024-09-10T21:52:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.773+0000 7fd4a0c00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd49c0cd170 0x7fd49c1f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.773+0000 7fd4a0c00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd49c154ec0 0x7fd49c0b1a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.773+0000 7fd4a0c00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd49c1552a0 0x7fd49c0b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.773+0000 7fd4a0c00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c06f1b0 con 0x7fd49c1552a0 2024-09-10T21:52:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.773+0000 7fd4a0c00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fd49c06f030 con 0x7fd49c0cd170 2024-09-10T21:52:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.773+0000 7fd4a0c00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fd49c06f330 con 0x7fd49c154ec0 2024-09-10T21:52:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.774+0000 7fd49a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd49c1552a0 0x7fd49c0b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:30.776 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.774+0000 7fd49a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd49c1552a0 0x7fd49c0b1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40128/0 (socket says 172.21.15.7:40128) 2024-09-10T21:52:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.774+0000 7fd49a800640 1 -- 172.21.15.7:0/2411074359 learned_addr learned my addr 172.21.15.7:0/2411074359 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.774+0000 7fd499e00640 1 --2- 172.21.15.7:0/2411074359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd49c0cd170 0x7fd49c1f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.774+0000 7fd491400640 1 --2- 172.21.15.7:0/2411074359 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd49c154ec0 0x7fd49c0b1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.774+0000 7fd49a800640 1 -- 172.21.15.7:0/2411074359 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd49c154ec0 msgr2=0x7fd49c0b1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.774+0000 7fd49a800640 1 --2- 172.21.15.7:0/2411074359 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fd49c154ec0 0x7fd49c0b1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.774+0000 7fd49a800640 1 -- 172.21.15.7:0/2411074359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd49c0cd170 msgr2=0x7fd49c1f3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.774+0000 7fd49a800640 1 --2- 172.21.15.7:0/2411074359 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fd49c0cd170 0x7fd49c1f3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.774+0000 7fd49a800640 1 -- 172.21.15.7:0/2411074359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49c14ccd0 con 0x7fd49c1552a0 2024-09-10T21:52:30.777 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.774+0000 7fd49a800640 1 --2- 172.21.15.7:0/2411074359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd49c1552a0 0x7fd49c0b1fd0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fd49400cce0 tx=0x7fd49400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:30.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.775+0000 7fd492a00640 1 -- 172.21.15.7:0/2411074359 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49400f450 con 0x7fd49c1552a0 2024-09-10T21:52:30.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.775+0000 7fd492a00640 1 -- 172.21.15.7:0/2411074359 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd49400f700 con 0x7fd49c1552a0 2024-09-10T21:52:30.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.775+0000 7fd492a00640 1 -- 172.21.15.7:0/2411074359 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49400fb50 con 0x7fd49c1552a0 2024-09-10T21:52:30.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.775+0000 7fd4a0c00640 1 -- 172.21.15.7:0/2411074359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd49c14cfb0 con 0x7fd49c1552a0 2024-09-10T21:52:30.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.775+0000 7fd4a0c00640 1 -- 172.21.15.7:0/2411074359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd49c152950 con 0x7fd49c1552a0 2024-09-10T21:52:30.778 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.776+0000 7fd4a0c00640 1 -- 172.21.15.7:0/2411074359 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd49c06f7e0 con 0x7fd49c1552a0 2024-09-10T21:52:30.779 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.777+0000 7fd492a00640 1 -- 172.21.15.7:0/2411074359 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fd4940213f0 con 0x7fd49c1552a0 2024-09-10T21:52:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.778+0000 7fd492a00640 1 --2- 172.21.15.7:0/2411074359 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd478077f70 0x7fd47807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.778+0000 7fd492a00640 1 -- 172.21.15.7:0/2411074359 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7fd49409a5c0 con 0x7fd49c1552a0 2024-09-10T21:52:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.778+0000 7fd492a00640 1 --2- 172.21.15.7:0/2411074359 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x7fd47807df70 0x7fd478080390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.778+0000 7fd492a00640 1 -- 172.21.15.7:0/2411074359 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd478080a40 con 0x7fd47807df70 2024-09-10T21:52:30.780 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.778+0000 7fd492a00640 1 -- 172.21.15.7:0/2411074359 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7fd4940a1050 con 0x7fd49c1552a0 2024-09-10T21:52:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.779+0000 7fd499e00640 1 --2- 172.21.15.7:0/2411074359 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd478077f70 0x7fd47807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.779+0000 7fd491400640 1 --2- 172.21.15.7:0/2411074359 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x7fd47807df70 0x7fd478080390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.779+0000 7fd491400640 1 --2- 172.21.15.7:0/2411074359 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x7fd47807df70 0x7fd478080390 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:30.781 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.779+0000 7fd499e00640 1 --2- 172.21.15.7:0/2411074359 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd478077f70 0x7fd47807a430 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd48c033000 tx=0x7fd48c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:30.784 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.780+0000 7fd492a00640 1 -- 172.21.15.7:0/2411074359 <== osd.3 v2:172.21.15.7:6800/2195665006 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd478080a40 con 0x7fd47807df70 2024-09-10T21:52:30.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.793+0000 7fd4a0c00640 1 -- 172.21.15.7:0/2411074359 --> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd49c06d5b0 con 0x7fd47807df70 2024-09-10T21:52:30.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.794+0000 7fd492a00640 1 -- 172.21.15.7:0/2411074359 <== osd.3 v2:172.21.15.7:6800/2195665006 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd49c06d5b0 con 0x7fd47807df70 2024-09-10T21:52:30.795 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:52:30.796 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:52:30.796 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:52:30.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.794+0000 7fd4a0c00640 1 -- 172.21.15.7:0/2411074359 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x7fd47807df70 msgr2=0x7fd478080390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.794+0000 7fd4a0c00640 1 --2- 172.21.15.7:0/2411074359 >> [v2:172.21.15.7:6800/2195665006,v1:172.21.15.7:6801/2195665006] conn(0x7fd47807df70 0x7fd478080390 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.794+0000 7fd4a0c00640 1 -- 172.21.15.7:0/2411074359 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd478077f70 msgr2=0x7fd47807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.794+0000 7fd4a0c00640 1 --2- 172.21.15.7:0/2411074359 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fd478077f70 0x7fd47807a430 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd48c033000 tx=0x7fd48c004730 comp rx=0 tx=0).stop 2024-09-10T21:52:30.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.794+0000 7fd4a0c00640 1 -- 172.21.15.7:0/2411074359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd49c1552a0 msgr2=0x7fd49c0b1fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.794+0000 7fd4a0c00640 1 --2- 172.21.15.7:0/2411074359 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fd49c1552a0 0x7fd49c0b1fd0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7fd49400cce0 tx=0x7fd49400a620 comp rx=0 tx=0).stop 2024-09-10T21:52:30.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.794+0000 7fd49a800640 1 -- 172.21.15.7:0/2411074359 reap_dead start 2024-09-10T21:52:30.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.794+0000 7fd4a0c00640 1 -- 172.21.15.7:0/2411074359 shutdown_connections 2024-09-10T21:52:30.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.794+0000 7fd4a0c00640 1 -- 172.21.15.7:0/2411074359 >> 172.21.15.7:0/2411074359 conn(0x7fd49c06ce50 msgr2=0x7fd49c0cf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:30.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.794+0000 7fd4a0c00640 1 -- 172.21.15.7:0/2411074359 shutdown_connections 2024-09-10T21:52:30.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.794+0000 7fd4a0c00640 1 -- 172.21.15.7:0/2411074359 wait complete. 2024-09-10T21:52:30.811 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-10T21:52:30.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.970+0000 7f777b200640 1 -- 172.21.15.7:0/3965559576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f777c15df90 msgr2=0x7f777c0c9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.970+0000 7f777b200640 1 --2- 172.21.15.7:0/3965559576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f777c15df90 0x7f777c0c9950 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f776c009830 tx=0x7f776c02a5c0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.970+0000 7f777b200640 1 -- 172.21.15.7:0/3965559576 shutdown_connections 2024-09-10T21:52:30.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.970+0000 7f777b200640 1 --2- 172.21.15.7:0/3965559576 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f777c15df90 0x7f777c0c9950 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.970+0000 7f777b200640 1 -- 172.21.15.7:0/3965559576 >> 172.21.15.7:0/3965559576 conn(0x7f777c0cc6b0 msgr2=0x7f777c0ccac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:30.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.970+0000 7f777b200640 1 -- 172.21.15.7:0/3965559576 shutdown_connections 2024-09-10T21:52:30.972 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.970+0000 7f777b200640 1 -- 172.21.15.7:0/3965559576 wait complete. 2024-09-10T21:52:30.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.972+0000 7f777b200640 1 Processor -- start 2024-09-10T21:52:30.973 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.972+0000 7f777b200640 1 -- start start 2024-09-10T21:52:30.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.973+0000 7f777b200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f777c0ca280 0x7f777c15bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:30.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.973+0000 7f777b200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f777c0cb300 0x7f777c1f7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:30.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.973+0000 7f777b200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f777c15df90 0x7f777c1f8310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:30.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.973+0000 7f777b200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f777c15ee10 con 0x7f777c15df90 2024-09-10T21:52:30.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.973+0000 7f777b200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f777c15ec90 con 0x7f777c0cb300 2024-09-10T21:52:30.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.973+0000 7f777b200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f777c15ef90 con 0x7f777c0ca280 2024-09-10T21:52:30.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.973+0000 7f777a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f777c15df90 0x7f777c1f8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:30.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.973+0000 7f777a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f777c15df90 0x7f777c1f8310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40138/0 (socket says 172.21.15.7:40138) 2024-09-10T21:52:30.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.973+0000 7f777a800640 1 -- 172.21.15.7:0/3903786182 learned_addr learned my addr 172.21.15.7:0/3903786182 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:30.975 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.973+0000 7f7773e00640 1 --2- 172.21.15.7:0/3903786182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f777c0cb300 0x7f777c1f7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:30.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.973+0000 7f7779e00640 1 --2- 172.21.15.7:0/3903786182 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f777c0ca280 0x7f777c15bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:30.976 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.974+0000 7f777a800640 1 -- 172.21.15.7:0/3903786182 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f777c0ca280 msgr2=0x7f777c15bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.974+0000 7f777a800640 1 --2- 172.21.15.7:0/3903786182 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f777c0ca280 0x7f777c15bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.974+0000 7f777a800640 1 -- 172.21.15.7:0/3903786182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f777c0cb300 msgr2=0x7f777c1f7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.974+0000 7f777a800640 1 --2- 172.21.15.7:0/3903786182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f777c0cb300 0x7f777c1f7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.974+0000 7f777a800640 1 -- 172.21.15.7:0/3903786182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777c15c440 con 0x7f777c15df90 2024-09-10T21:52:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.974+0000 7f777a800640 1 --2- 172.21.15.7:0/3903786182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f777c15df90 0x7f777c1f8310 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f777400cbc0 tx=0x7f777400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.974+0000 7f7772000640 1 -- 172.21.15.7:0/3903786182 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7774016020 con 0x7f777c15df90 2024-09-10T21:52:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.974+0000 7f7772000640 1 -- 172.21.15.7:0/3903786182 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f777400ad90 con 0x7f777c15df90 2024-09-10T21:52:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.974+0000 7f777b200640 1 -- 172.21.15.7:0/3903786182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777c15c720 con 0x7f777c15df90 2024-09-10T21:52:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.974+0000 7f7772000640 1 -- 172.21.15.7:0/3903786182 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7774012680 con 0x7f777c15df90 2024-09-10T21:52:30.977 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.974+0000 7f777b200640 1 -- 172.21.15.7:0/3903786182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f777c15cce0 con 0x7f777c15df90 2024-09-10T21:52:30.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.975+0000 7f777b200640 1 -- 172.21.15.7:0/3903786182 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f777c0c9950 con 0x7f777c15df90 2024-09-10T21:52:30.978 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.977+0000 7f7772000640 1 -- 172.21.15.7:0/3903786182 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f7774012820 con 0x7f777c15df90 2024-09-10T21:52:30.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.977+0000 7f7772000640 1 --2- 172.21.15.7:0/3903786182 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7758077f70 0x7f775807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:30.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.978+0000 7f7772000640 1 -- 172.21.15.7:0/3903786182 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7f777409a330 con 0x7f777c15df90 2024-09-10T21:52:30.979 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.978+0000 7f7779e00640 1 --2- 172.21.15.7:0/3903786182 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7758077f70 0x7f775807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:30.980 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.979+0000 7f7779e00640 1 --2- 172.21.15.7:0/3903786182 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7758077f70 0x7f775807a430 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f776c034000 tx=0x7f776c004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:30.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:30.982+0000 7f7772000640 1 -- 172.21.15.7:0/3903786182 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f7774020020 con 0x7f777c15df90 2024-09-10T21:52:31.178 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.177+0000 7f777b200640 1 -- 172.21.15.7:0/3903786182 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f777c161640 con 0x7f7758077f70 2024-09-10T21:52:31.179 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.177+0000 7f7772000640 1 -- 172.21.15.7:0/3903786182 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f777c161640 con 0x7f7758077f70 2024-09-10T21:52:31.179 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 3 to deep-scrub 2024-09-10T21:52:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.180+0000 7f777b200640 1 -- 172.21.15.7:0/3903786182 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7758077f70 msgr2=0x7f775807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.180+0000 7f777b200640 1 --2- 172.21.15.7:0/3903786182 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7758077f70 0x7f775807a430 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f776c034000 tx=0x7f776c004730 comp rx=0 tx=0).stop 2024-09-10T21:52:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.180+0000 7f777b200640 1 -- 172.21.15.7:0/3903786182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f777c15df90 msgr2=0x7f777c1f8310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.180+0000 7f777b200640 1 --2- 172.21.15.7:0/3903786182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f777c15df90 0x7f777c1f8310 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f777400cbc0 tx=0x7f777400a060 comp rx=0 tx=0).stop 2024-09-10T21:52:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.180+0000 7f777b200640 1 -- 172.21.15.7:0/3903786182 shutdown_connections 2024-09-10T21:52:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.180+0000 7f777b200640 1 --2- 172.21.15.7:0/3903786182 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f7758077f70 0x7f775807a430 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.180+0000 7f777b200640 1 --2- 172.21.15.7:0/3903786182 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f777c0cb300 0x7f777c1f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.180+0000 7f777b200640 1 --2- 172.21.15.7:0/3903786182 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f777c15df90 0x7f777c1f8310 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.180+0000 7f777b200640 1 --2- 172.21.15.7:0/3903786182 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f777c0ca280 0x7f777c15bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.182 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.180+0000 7f777b200640 1 -- 172.21.15.7:0/3903786182 >> 172.21.15.7:0/3903786182 conn(0x7f777c0cc6b0 msgr2=0x7f777c0cf060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:31.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.180+0000 7f777b200640 1 -- 172.21.15.7:0/3903786182 shutdown_connections 2024-09-10T21:52:31.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.180+0000 7f777b200640 1 -- 172.21.15.7:0/3903786182 wait complete. 2024-09-10T21:52:31.198 INFO:tasks.ceph:Scrubbing osd.4 2024-09-10T21:52:31.199 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:52:31.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.361+0000 7f2378200640 1 -- 172.21.15.7:0/121390440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237006ff60 msgr2=0x7f2370073280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:31.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.361+0000 7f2378200640 1 --2- 172.21.15.7:0/121390440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237006ff60 0x7f2370073280 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f2368005fd0 tx=0x7f236802aa10 comp rx=0 tx=0).stop 2024-09-10T21:52:31.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.361+0000 7f2378200640 1 -- 172.21.15.7:0/121390440 shutdown_connections 2024-09-10T21:52:31.363 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.361+0000 7f2378200640 1 --2- 172.21.15.7:0/121390440 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237006ff60 0x7f2370073280 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.361+0000 7f2378200640 1 -- 172.21.15.7:0/121390440 >> 172.21.15.7:0/121390440 conn(0x7f237006ce50 msgr2=0x7f237006d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.361+0000 7f2378200640 1 -- 172.21.15.7:0/121390440 shutdown_connections 2024-09-10T21:52:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.361+0000 7f2378200640 1 -- 172.21.15.7:0/121390440 wait complete. 2024-09-10T21:52:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.363+0000 7f2378200640 1 Processor -- start 2024-09-10T21:52:31.364 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.363+0000 7f2378200640 1 -- start start 2024-09-10T21:52:31.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.364+0000 7f2378200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237006ff60 0x7f23701e86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:31.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.364+0000 7f2378200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23700cd340 0x7f23700b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:31.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.364+0000 7f2378200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2370145610 0x7f23700cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:31.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.364+0000 7f2378200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f23700ce4e0 con 0x7f237006ff60 2024-09-10T21:52:31.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.364+0000 7f2378200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f23700ce360 con 0x7f2370145610 2024-09-10T21:52:31.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.364+0000 7f2378200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f237006f530 con 0x7f23700cd340 2024-09-10T21:52:31.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.364+0000 7f2375800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237006ff60 0x7f23701e86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:31.366 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.364+0000 7f2375800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237006ff60 0x7f23701e86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40160/0 (socket says 172.21.15.7:40160) 2024-09-10T21:52:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.364+0000 7f2375800640 1 -- 172.21.15.7:0/4185708356 learned_addr learned my addr 172.21.15.7:0/4185708356 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.364+0000 7f2375800640 1 -- 172.21.15.7:0/4185708356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23700cd340 msgr2=0x7f23700b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.364+0000 7f2375800640 1 --2- 172.21.15.7:0/4185708356 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f23700cd340 0x7f23700b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.364+0000 7f2375800640 1 -- 172.21.15.7:0/4185708356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2370145610 msgr2=0x7f23700cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.364+0000 7f2375800640 1 --2- 172.21.15.7:0/4185708356 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f2370145610 0x7f23700cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.367 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.364+0000 7f2375800640 1 -- 172.21.15.7:0/4185708356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2370130580 con 0x7f237006ff60 2024-09-10T21:52:31.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.365+0000 7f2375800640 1 --2- 172.21.15.7:0/4185708356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237006ff60 0x7f23701e86d0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f236400c3a0 tx=0x7f236400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:31.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.365+0000 7f236d600640 1 -- 172.21.15.7:0/4185708356 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364017070 con 0x7f237006ff60 2024-09-10T21:52:31.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.365+0000 7f236d600640 1 -- 172.21.15.7:0/4185708356 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23640053c0 con 0x7f237006ff60 2024-09-10T21:52:31.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.365+0000 7f236d600640 1 -- 172.21.15.7:0/4185708356 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2364003850 con 0x7f237006ff60 2024-09-10T21:52:31.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.365+0000 7f2378200640 1 -- 172.21.15.7:0/4185708356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23700b1e70 con 0x7f237006ff60 2024-09-10T21:52:31.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.365+0000 7f2378200640 1 -- 172.21.15.7:0/4185708356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2370143bf0 con 0x7f237006ff60 2024-09-10T21:52:31.368 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.366+0000 7f2378200640 1 -- 172.21.15.7:0/4185708356 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f237006f9b0 con 0x7f237006ff60 2024-09-10T21:52:31.369 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.367+0000 7f236d600640 1 -- 172.21.15.7:0/4185708356 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f23640039f0 con 0x7f237006ff60 2024-09-10T21:52:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.368+0000 7f236d600640 1 --2- 172.21.15.7:0/4185708356 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2350077f70 0x7f235007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.368+0000 7f236d600640 1 -- 172.21.15.7:0/4185708356 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7f2364099240 con 0x7f237006ff60 2024-09-10T21:52:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.368+0000 7f236d600640 1 --2- 172.21.15.7:0/4185708356 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x7f235007df70 0x7f2350080390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.368+0000 7f236d600640 1 -- 172.21.15.7:0/4185708356 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2350080a40 con 0x7f235007df70 2024-09-10T21:52:31.370 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.368+0000 7f236d600640 1 -- 172.21.15.7:0/4185708356 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7f23640a0050 con 0x7f237006ff60 2024-09-10T21:52:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.369+0000 7f2374e00640 1 --2- 172.21.15.7:0/4185708356 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2350077f70 0x7f235007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.369+0000 7f2376200640 1 --2- 172.21.15.7:0/4185708356 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x7f235007df70 0x7f2350080390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.369+0000 7f2374e00640 1 --2- 172.21.15.7:0/4185708356 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2350077f70 0x7f235007a430 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f23701f8250 tx=0x7f235c007990 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:31.371 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.369+0000 7f2376200640 1 --2- 172.21.15.7:0/4185708356 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x7f235007df70 0x7f2350080390 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:31.372 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.370+0000 7f236d600640 1 -- 172.21.15.7:0/4185708356 <== osd.4 v2:172.21.15.133:6826/1857710445 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2350080a40 con 0x7f235007df70 2024-09-10T21:52:31.385 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.383+0000 7f2378200640 1 -- 172.21.15.7:0/4185708356 --> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f23700705d0 con 0x7f235007df70 2024-09-10T21:52:31.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.384+0000 7f236d600640 1 -- 172.21.15.7:0/4185708356 <== osd.4 v2:172.21.15.133:6826/1857710445 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f23700705d0 con 0x7f235007df70 2024-09-10T21:52:31.386 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:52:31.386 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:52:31.386 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:52:31.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.384+0000 7f2378200640 1 -- 172.21.15.7:0/4185708356 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x7f235007df70 msgr2=0x7f2350080390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:31.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.384+0000 7f2378200640 1 --2- 172.21.15.7:0/4185708356 >> [v2:172.21.15.133:6826/1857710445,v1:172.21.15.133:6827/1857710445] conn(0x7f235007df70 0x7f2350080390 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.385+0000 7f2378200640 1 -- 172.21.15.7:0/4185708356 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2350077f70 msgr2=0x7f235007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:31.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.385+0000 7f2378200640 1 --2- 172.21.15.7:0/4185708356 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f2350077f70 0x7f235007a430 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f23701f8250 tx=0x7f235c007990 comp rx=0 tx=0).stop 2024-09-10T21:52:31.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.385+0000 7f2378200640 1 -- 172.21.15.7:0/4185708356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237006ff60 msgr2=0x7f23701e86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:31.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.385+0000 7f2378200640 1 --2- 172.21.15.7:0/4185708356 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f237006ff60 0x7f23701e86d0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f236400c3a0 tx=0x7f236400c860 comp rx=0 tx=0).stop 2024-09-10T21:52:31.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.385+0000 7f2376200640 1 -- 172.21.15.7:0/4185708356 reap_dead start 2024-09-10T21:52:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.385+0000 7f2378200640 1 -- 172.21.15.7:0/4185708356 shutdown_connections 2024-09-10T21:52:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.385+0000 7f2378200640 1 -- 172.21.15.7:0/4185708356 >> 172.21.15.7:0/4185708356 conn(0x7f237006ce50 msgr2=0x7f237015df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.385+0000 7f2378200640 1 -- 172.21.15.7:0/4185708356 shutdown_connections 2024-09-10T21:52:31.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.385+0000 7f2378200640 1 -- 172.21.15.7:0/4185708356 wait complete. 2024-09-10T21:52:31.402 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-10T21:52:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.564+0000 7f67fa200640 1 -- 172.21.15.7:0/3411337626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67d80051c0 msgr2=0x7f67d800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.564+0000 7f67fa200640 1 --2- 172.21.15.7:0/3411337626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67d80051c0 0x7f67d800b9c0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f67ec0016c0 tx=0x7f67d8005830 comp rx=0 tx=0).stop 2024-09-10T21:52:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.564+0000 7f67fa200640 1 -- 172.21.15.7:0/3411337626 shutdown_connections 2024-09-10T21:52:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.564+0000 7f67fa200640 1 --2- 172.21.15.7:0/3411337626 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67d80051c0 0x7f67d800b9c0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.564+0000 7f67fa200640 1 -- 172.21.15.7:0/3411337626 >> 172.21.15.7:0/3411337626 conn(0x7f67f406e4b0 msgr2=0x7f67f40b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.564+0000 7f67fa200640 1 -- 172.21.15.7:0/3411337626 shutdown_connections 2024-09-10T21:52:31.566 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.564+0000 7f67fa200640 1 -- 172.21.15.7:0/3411337626 wait complete. 2024-09-10T21:52:31.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.565+0000 7f67fa200640 1 Processor -- start 2024-09-10T21:52:31.567 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.566+0000 7f67fa200640 1 -- start start 2024-09-10T21:52:31.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.566+0000 7f67fa200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d80051c0 0x7f67f41563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:31.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.567+0000 7f67fa200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67f4153e50 0x7f67f41f1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:31.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.567+0000 7f67fa200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67f41f20b0 0x7f67f41f7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:31.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.567+0000 7f67fa200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f67f40b3ee0 con 0x7f67f4153e50 2024-09-10T21:52:31.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.567+0000 7f67fa200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f67f415ebe0 con 0x7f67d80051c0 2024-09-10T21:52:31.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.567+0000 7f67fa200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f67f40b4060 con 0x7f67f41f20b0 2024-09-10T21:52:31.569 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.567+0000 7f67eaa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67f4153e50 0x7f67f41f1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.567+0000 7f67eaa00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67f4153e50 0x7f67f41f1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40176/0 (socket says 172.21.15.7:40176) 2024-09-10T21:52:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.567+0000 7f67eaa00640 1 -- 172.21.15.7:0/3073683140 learned_addr learned my addr 172.21.15.7:0/3073683140 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.567+0000 7f67f3400640 1 --2- 172.21.15.7:0/3073683140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d80051c0 0x7f67f41563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.567+0000 7f67f3e00640 1 --2- 172.21.15.7:0/3073683140 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67f41f20b0 0x7f67f41f7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.567+0000 7f67eaa00640 1 -- 172.21.15.7:0/3073683140 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67f41f20b0 msgr2=0x7f67f41f7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.567+0000 7f67eaa00640 1 --2- 172.21.15.7:0/3073683140 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67f41f20b0 0x7f67f41f7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.567+0000 7f67eaa00640 1 -- 172.21.15.7:0/3073683140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d80051c0 msgr2=0x7f67f41563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.567+0000 7f67eaa00640 1 --2- 172.21.15.7:0/3073683140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d80051c0 0x7f67f41563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.570 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.567+0000 7f67eaa00640 1 -- 172.21.15.7:0/3073683140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f4156fb0 con 0x7f67f4153e50 2024-09-10T21:52:31.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.568+0000 7f67eaa00640 1 --2- 172.21.15.7:0/3073683140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67f4153e50 0x7f67f41f1b70 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f67dc006680 tx=0x7f67dc0066b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:31.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.568+0000 7f67ebe00640 1 -- 172.21.15.7:0/3073683140 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc015070 con 0x7f67f4153e50 2024-09-10T21:52:31.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.568+0000 7f67ebe00640 1 -- 172.21.15.7:0/3073683140 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67dc011920 con 0x7f67f4153e50 2024-09-10T21:52:31.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.568+0000 7f67fa200640 1 -- 172.21.15.7:0/3073683140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f41569f0 con 0x7f67f4153e50 2024-09-10T21:52:31.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.568+0000 7f67ebe00640 1 -- 172.21.15.7:0/3073683140 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67dc011d70 con 0x7f67f4153e50 2024-09-10T21:52:31.571 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.568+0000 7f67fa200640 1 -- 172.21.15.7:0/3073683140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f40ae2b0 con 0x7f67f4153e50 2024-09-10T21:52:31.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.569+0000 7f67fa200640 1 -- 172.21.15.7:0/3073683140 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f40b5730 con 0x7f67f4153e50 2024-09-10T21:52:31.572 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.570+0000 7f67ebe00640 1 -- 172.21.15.7:0/3073683140 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f67dc010420 con 0x7f67f4153e50 2024-09-10T21:52:31.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.571+0000 7f67ebe00640 1 --2- 172.21.15.7:0/3073683140 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f67d0077fa0 0x7f67d007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:31.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.571+0000 7f67ebe00640 1 -- 172.21.15.7:0/3073683140 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7f67dc09ab10 con 0x7f67f4153e50 2024-09-10T21:52:31.573 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.572+0000 7f67f3400640 1 --2- 172.21.15.7:0/3073683140 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f67d0077fa0 0x7f67d007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:31.574 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.572+0000 7f67f3400640 1 --2- 172.21.15.7:0/3073683140 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f67d0077fa0 0x7f67d007a460 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f67e4005fd0 tx=0x7f67e4009500 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:31.577 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.577+0000 7f67ebe00640 1 -- 172.21.15.7:0/3073683140 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f67dc063170 con 0x7f67f4153e50 2024-09-10T21:52:31.771 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.769+0000 7f67fa200640 1 -- 172.21.15.7:0/3073683140 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f67f40d0470 con 0x7f67d0077fa0 2024-09-10T21:52:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.770+0000 7f67ebe00640 1 -- 172.21.15.7:0/3073683140 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f67f40d0470 con 0x7f67d0077fa0 2024-09-10T21:52:31.772 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 4 to deep-scrub 2024-09-10T21:52:31.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.772+0000 7f67fa200640 1 -- 172.21.15.7:0/3073683140 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f67d0077fa0 msgr2=0x7f67d007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:31.774 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.772+0000 7f67fa200640 1 --2- 172.21.15.7:0/3073683140 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f67d0077fa0 0x7f67d007a460 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f67e4005fd0 tx=0x7f67e4009500 comp rx=0 tx=0).stop 2024-09-10T21:52:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.772+0000 7f67fa200640 1 -- 172.21.15.7:0/3073683140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67f4153e50 msgr2=0x7f67f41f1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.772+0000 7f67fa200640 1 --2- 172.21.15.7:0/3073683140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67f4153e50 0x7f67f41f1b70 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f67dc006680 tx=0x7f67dc0066b0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.772+0000 7f67fa200640 1 -- 172.21.15.7:0/3073683140 shutdown_connections 2024-09-10T21:52:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.772+0000 7f67fa200640 1 --2- 172.21.15.7:0/3073683140 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f67d0077fa0 0x7f67d007a460 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.772+0000 7f67fa200640 1 --2- 172.21.15.7:0/3073683140 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f67f41f20b0 0x7f67f41f7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.772+0000 7f67fa200640 1 --2- 172.21.15.7:0/3073683140 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f67f4153e50 0x7f67f41f1b70 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.772+0000 7f67fa200640 1 --2- 172.21.15.7:0/3073683140 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f67d80051c0 0x7f67f41563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.772+0000 7f67fa200640 1 -- 172.21.15.7:0/3073683140 >> 172.21.15.7:0/3073683140 conn(0x7f67f406e4b0 msgr2=0x7f67f40ce640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.773+0000 7f67fa200640 1 -- 172.21.15.7:0/3073683140 shutdown_connections 2024-09-10T21:52:31.775 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.773+0000 7f67fa200640 1 -- 172.21.15.7:0/3073683140 wait complete. 2024-09-10T21:52:31.789 INFO:tasks.ceph:Scrubbing osd.5 2024-09-10T21:52:31.790 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:52:31.811 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:52:31.840 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:52:31.959 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.957+0000 7fdd3e400640 1 -- 172.21.15.7:0/1747402603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd3806ccd0 msgr2=0x7fdd380b4790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.957+0000 7fdd3e400640 1 --2- 172.21.15.7:0/1747402603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd3806ccd0 0x7fdd380b4790 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fdd28005fd0 tx=0x7fdd2802a4a0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.957+0000 7fdd3e400640 1 -- 172.21.15.7:0/1747402603 shutdown_connections 2024-09-10T21:52:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.957+0000 7fdd3e400640 1 --2- 172.21.15.7:0/1747402603 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd380afc80 0x7fdd3815b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.957+0000 7fdd3e400640 1 --2- 172.21.15.7:0/1747402603 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd3806ccd0 0x7fdd380b4790 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.957+0000 7fdd3e400640 1 -- 172.21.15.7:0/1747402603 >> 172.21.15.7:0/1747402603 conn(0x7fdd3806db70 msgr2=0x7fdd3806df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.957+0000 7fdd3e400640 1 -- 172.21.15.7:0/1747402603 shutdown_connections 2024-09-10T21:52:31.960 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.957+0000 7fdd3e400640 1 -- 172.21.15.7:0/1747402603 wait complete. 2024-09-10T21:52:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.959+0000 7fdd3e400640 1 Processor -- start 2024-09-10T21:52:31.961 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.959+0000 7fdd3e400640 1 -- start start 2024-09-10T21:52:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.960+0000 7fdd3e400640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd3806ccd0 0x7fdd381462d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.960+0000 7fdd3e400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd380afc80 0x7fdd381e7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.960+0000 7fdd3e400640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd380b4cd0 0x7fdd381e8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.960+0000 7fdd3e400640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3815bf20 con 0x7fdd380afc80 2024-09-10T21:52:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.960+0000 7fdd3e400640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fdd3815bda0 con 0x7fdd380b4cd0 2024-09-10T21:52:31.962 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.960+0000 7fdd3e400640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fdd3815c0a0 con 0x7fdd3806ccd0 2024-09-10T21:52:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.960+0000 7fdd2fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd380afc80 0x7fdd381e7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.960+0000 7fdd2fe00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd380afc80 0x7fdd381e7ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40186/0 (socket says 172.21.15.7:40186) 2024-09-10T21:52:31.963 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.960+0000 7fdd2fe00640 1 -- 172.21.15.7:0/3339374619 learned_addr learned my addr 172.21.15.7:0/3339374619 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.961+0000 7fdd3d000640 1 --2- 172.21.15.7:0/3339374619 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd3806ccd0 0x7fdd381462d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.961+0000 7fdd3da00640 1 --2- 172.21.15.7:0/3339374619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd380b4cd0 0x7fdd381e8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.961+0000 7fdd2fe00640 1 -- 172.21.15.7:0/3339374619 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd3806ccd0 msgr2=0x7fdd381462d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.961+0000 7fdd2fe00640 1 --2- 172.21.15.7:0/3339374619 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd3806ccd0 0x7fdd381462d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.961+0000 7fdd2fe00640 1 -- 172.21.15.7:0/3339374619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd380b4cd0 msgr2=0x7fdd381e8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.961+0000 7fdd2fe00640 1 --2- 172.21.15.7:0/3339374619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd380b4cd0 0x7fdd381e8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.961+0000 7fdd2fe00640 1 -- 172.21.15.7:0/3339374619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd38146dc0 con 0x7fdd380afc80 2024-09-10T21:52:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.961+0000 7fdd3d000640 1 --2- 172.21.15.7:0/3339374619 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fdd3806ccd0 0x7fdd381462d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:52:31.964 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.961+0000 7fdd3da00640 1 --2- 172.21.15.7:0/3339374619 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fdd380b4cd0 0x7fdd381e8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:52:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.961+0000 7fdd2fe00640 1 --2- 172.21.15.7:0/3339374619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd380afc80 0x7fdd381e7ac0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000b3e0 tx=0x7fdd2000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.961+0000 7fdd35600640 1 -- 172.21.15.7:0/3339374619 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20015020 con 0x7fdd380afc80 2024-09-10T21:52:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.961+0000 7fdd35600640 1 -- 172.21.15.7:0/3339374619 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd20011940 con 0x7fdd380afc80 2024-09-10T21:52:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.961+0000 7fdd35600640 1 -- 172.21.15.7:0/3339374619 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd20011d90 con 0x7fdd380afc80 2024-09-10T21:52:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.961+0000 7fdd3e400640 1 -- 172.21.15.7:0/3339374619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd381470a0 con 0x7fdd380afc80 2024-09-10T21:52:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.961+0000 7fdd3e400640 1 -- 172.21.15.7:0/3339374619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd38147660 con 0x7fdd380afc80 2024-09-10T21:52:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.962+0000 7fdd3e400640 1 -- 172.21.15.7:0/3339374619 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdd3815c6b0 con 0x7fdd380afc80 2024-09-10T21:52:31.965 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.964+0000 7fdd35600640 1 -- 172.21.15.7:0/3339374619 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fdd2001a400 con 0x7fdd380afc80 2024-09-10T21:52:31.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.964+0000 7fdd35600640 1 --2- 172.21.15.7:0/3339374619 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdd14077f70 0x7fdd1407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:31.966 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.965+0000 7fdd35600640 1 -- 172.21.15.7:0/3339374619 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7fdd20099e70 con 0x7fdd380afc80 2024-09-10T21:52:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.965+0000 7fdd35600640 1 --2- 172.21.15.7:0/3339374619 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x7fdd1407df70 0x7fdd14080390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.965+0000 7fdd35600640 1 -- 172.21.15.7:0/3339374619 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdd2000de50 con 0x7fdd1407df70 2024-09-10T21:52:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.965+0000 7fdd35600640 1 -- 172.21.15.7:0/3339374619 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7fdd2009c490 con 0x7fdd380afc80 2024-09-10T21:52:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.965+0000 7fdd3d000640 1 --2- 172.21.15.7:0/3339374619 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdd14077f70 0x7fdd1407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:31.967 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.965+0000 7fdd3da00640 1 --2- 172.21.15.7:0/3339374619 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x7fdd1407df70 0x7fdd14080390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:31.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.966+0000 7fdd3d000640 1 --2- 172.21.15.7:0/3339374619 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdd14077f70 0x7fdd1407a430 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdd28033000 tx=0x7fdd28004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:31.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.966+0000 7fdd3da00640 1 --2- 172.21.15.7:0/3339374619 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x7fdd1407df70 0x7fdd14080390 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:31.968 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.967+0000 7fdd35600640 1 -- 172.21.15.7:0/3339374619 <== osd.5 v2:172.21.15.133:6818/3838498382 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdd2000de50 con 0x7fdd1407df70 2024-09-10T21:52:31.982 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.980+0000 7fdd3e400640 1 -- 172.21.15.7:0/3339374619 --> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fdd380b1150 con 0x7fdd1407df70 2024-09-10T21:52:31.983 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.981+0000 7fdd35600640 1 -- 172.21.15.7:0/3339374619 <== osd.5 v2:172.21.15.133:6818/3838498382 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fdd380b1150 con 0x7fdd1407df70 2024-09-10T21:52:31.983 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:52:31.983 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:52:31.983 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:52:31.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.981+0000 7fdd3e400640 1 -- 172.21.15.7:0/3339374619 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x7fdd1407df70 msgr2=0x7fdd14080390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:31.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.981+0000 7fdd3e400640 1 --2- 172.21.15.7:0/3339374619 >> [v2:172.21.15.133:6818/3838498382,v1:172.21.15.133:6819/3838498382] conn(0x7fdd1407df70 0x7fdd14080390 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.982+0000 7fdd3e400640 1 -- 172.21.15.7:0/3339374619 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdd14077f70 msgr2=0x7fdd1407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:31.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.982+0000 7fdd3e400640 1 --2- 172.21.15.7:0/3339374619 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fdd14077f70 0x7fdd1407a430 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fdd28033000 tx=0x7fdd28004730 comp rx=0 tx=0).stop 2024-09-10T21:52:31.984 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.982+0000 7fdd3e400640 1 -- 172.21.15.7:0/3339374619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd380afc80 msgr2=0x7fdd381e7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:31.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.982+0000 7fdd3e400640 1 --2- 172.21.15.7:0/3339374619 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fdd380afc80 0x7fdd381e7ac0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fdd2000b3e0 tx=0x7fdd2000b8a0 comp rx=0 tx=0).stop 2024-09-10T21:52:31.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.982+0000 7fdd3da00640 1 -- 172.21.15.7:0/3339374619 reap_dead start 2024-09-10T21:52:31.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.982+0000 7fdd3e400640 1 -- 172.21.15.7:0/3339374619 shutdown_connections 2024-09-10T21:52:31.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.982+0000 7fdd3e400640 1 -- 172.21.15.7:0/3339374619 >> 172.21.15.7:0/3339374619 conn(0x7fdd3806db70 msgr2=0x7fdd380b1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:31.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.982+0000 7fdd3e400640 1 -- 172.21.15.7:0/3339374619 shutdown_connections 2024-09-10T21:52:31.985 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:31.982+0000 7fdd3e400640 1 -- 172.21.15.7:0/3339374619 wait complete. 2024-09-10T21:52:31.998 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-10T21:52:32.201 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.199+0000 7f633da00640 1 -- 172.21.15.7:0/2022917443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f631c003a00 msgr2=0x7f631c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:32.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.199+0000 7f633da00640 1 --2- 172.21.15.7:0/2022917443 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f631c003a00 0x7f631c003e20 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f6328009b50 tx=0x7f631c005830 comp rx=0 tx=0).stop 2024-09-10T21:52:32.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.199+0000 7f6337400640 1 -- 172.21.15.7:0/2022917443 reap_dead start 2024-09-10T21:52:32.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.199+0000 7f633da00640 1 -- 172.21.15.7:0/2022917443 shutdown_connections 2024-09-10T21:52:32.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.199+0000 7f633da00640 1 -- 172.21.15.7:0/2022917443 >> 172.21.15.7:0/2022917443 conn(0x7f633806ca60 msgr2=0x7f633806ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:32.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.199+0000 7f633da00640 1 -- 172.21.15.7:0/2022917443 shutdown_connections 2024-09-10T21:52:32.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.199+0000 7f633da00640 1 -- 172.21.15.7:0/2022917443 wait complete. 2024-09-10T21:52:32.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.201+0000 7f633da00640 1 Processor -- start 2024-09-10T21:52:32.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.201+0000 7f633da00640 1 -- start start 2024-09-10T21:52:32.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.202+0000 7f633da00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f631c003a00 0x7f633814c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:32.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.202+0000 7f633da00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63381516a0 0x7f633814f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:32.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.202+0000 7f633da00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6338151b20 0x7f63381466e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:32.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.202+0000 7f633da00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f633806e810 con 0x7f631c003a00 2024-09-10T21:52:32.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.202+0000 7f633da00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f633806e990 con 0x7f6338151b20 2024-09-10T21:52:32.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.202+0000 7f633da00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f633806eb10 con 0x7f63381516a0 2024-09-10T21:52:32.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.202+0000 7f6336a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f631c003a00 0x7f633814c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:32.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.202+0000 7f6336a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f631c003a00 0x7f633814c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40202/0 (socket says 172.21.15.7:40202) 2024-09-10T21:52:32.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.202+0000 7f6336a00640 1 -- 172.21.15.7:0/2105198032 learned_addr learned my addr 172.21.15.7:0/2105198032 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:32.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.203+0000 7f632e000640 1 --2- 172.21.15.7:0/2105198032 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63381516a0 0x7f633814f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:32.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.203+0000 7f6336a00640 1 -- 172.21.15.7:0/2105198032 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63381516a0 msgr2=0x7f633814f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:32.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.203+0000 7f6337400640 1 --2- 172.21.15.7:0/2105198032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6338151b20 0x7f63381466e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:32.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.203+0000 7f6336a00640 1 --2- 172.21.15.7:0/2105198032 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63381516a0 0x7f633814f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:32.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.203+0000 7f6336a00640 1 -- 172.21.15.7:0/2105198032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6338151b20 msgr2=0x7f63381466e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:32.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.203+0000 7f6336a00640 1 --2- 172.21.15.7:0/2105198032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6338151b20 0x7f63381466e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:32.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.203+0000 7f6336a00640 1 -- 172.21.15.7:0/2105198032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6338150270 con 0x7f631c003a00 2024-09-10T21:52:32.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.203+0000 7f632e000640 1 --2- 172.21.15.7:0/2105198032 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63381516a0 0x7f633814f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:52:32.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.203+0000 7f6337400640 1 --2- 172.21.15.7:0/2105198032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6338151b20 0x7f63381466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:52:32.206 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.203+0000 7f6336a00640 1 --2- 172.21.15.7:0/2105198032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f631c003a00 0x7f633814c2a0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f632802b4f0 tx=0x7f6328033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:32.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.203+0000 7f632f400640 1 -- 172.21.15.7:0/2105198032 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6328038040 con 0x7f631c003a00 2024-09-10T21:52:32.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.203+0000 7f632f400640 1 -- 172.21.15.7:0/2105198032 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6328004020 con 0x7f631c003a00 2024-09-10T21:52:32.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.203+0000 7f632f400640 1 -- 172.21.15.7:0/2105198032 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6328004450 con 0x7f631c003a00 2024-09-10T21:52:32.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.203+0000 7f633da00640 1 -- 172.21.15.7:0/2105198032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6338150550 con 0x7f631c003a00 2024-09-10T21:52:32.207 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.204+0000 7f633da00640 1 -- 172.21.15.7:0/2105198032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f633814ac40 con 0x7f631c003a00 2024-09-10T21:52:32.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.205+0000 7f633da00640 1 -- 172.21.15.7:0/2105198032 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63380ad850 con 0x7f631c003a00 2024-09-10T21:52:32.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.206+0000 7f632f400640 1 -- 172.21.15.7:0/2105198032 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f632803b030 con 0x7f631c003a00 2024-09-10T21:52:32.208 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.207+0000 7f632f400640 1 --2- 172.21.15.7:0/2105198032 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6314077f70 0x7f631407a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:32.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.207+0000 7f632f400640 1 -- 172.21.15.7:0/2105198032 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7f63280ba290 con 0x7f631c003a00 2024-09-10T21:52:32.209 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.207+0000 7f632e000640 1 --2- 172.21.15.7:0/2105198032 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6314077f70 0x7f631407a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:32.210 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.208+0000 7f632e000640 1 --2- 172.21.15.7:0/2105198032 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6314077f70 0x7f631407a430 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f633814c250 tx=0x7f6320005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:32.212 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.211+0000 7f632f400640 1 -- 172.21.15.7:0/2105198032 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f6328082fd0 con 0x7f631c003a00 2024-09-10T21:52:32.405 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.404+0000 7f633da00640 1 -- 172.21.15.7:0/2105198032 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f63380aebf0 con 0x7f6314077f70 2024-09-10T21:52:32.406 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.404+0000 7f632f400640 1 -- 172.21.15.7:0/2105198032 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f632001c440 con 0x7f6314077f70 2024-09-10T21:52:32.407 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 5 to deep-scrub 2024-09-10T21:52:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.407+0000 7f633da00640 1 -- 172.21.15.7:0/2105198032 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6314077f70 msgr2=0x7f631407a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.407+0000 7f633da00640 1 --2- 172.21.15.7:0/2105198032 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6314077f70 0x7f631407a430 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f633814c250 tx=0x7f6320005f90 comp rx=0 tx=0).stop 2024-09-10T21:52:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.407+0000 7f633da00640 1 -- 172.21.15.7:0/2105198032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f631c003a00 msgr2=0x7f633814c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.407+0000 7f633da00640 1 --2- 172.21.15.7:0/2105198032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f631c003a00 0x7f633814c2a0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f632802b4f0 tx=0x7f6328033670 comp rx=0 tx=0).stop 2024-09-10T21:52:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.407+0000 7f633da00640 1 -- 172.21.15.7:0/2105198032 shutdown_connections 2024-09-10T21:52:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.407+0000 7f633da00640 1 --2- 172.21.15.7:0/2105198032 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f6314077f70 0x7f631407a430 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:32.409 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.407+0000 7f633da00640 1 --2- 172.21.15.7:0/2105198032 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f6338151b20 0x7f63381466e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:32.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.407+0000 7f633da00640 1 --2- 172.21.15.7:0/2105198032 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f63381516a0 0x7f633814f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:32.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.407+0000 7f633da00640 1 --2- 172.21.15.7:0/2105198032 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f631c003a00 0x7f633814c2a0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:32.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.407+0000 7f633da00640 1 -- 172.21.15.7:0/2105198032 >> 172.21.15.7:0/2105198032 conn(0x7f633806ca60 msgr2=0x7f633815d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:32.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.407+0000 7f633da00640 1 -- 172.21.15.7:0/2105198032 shutdown_connections 2024-09-10T21:52:32.410 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.407+0000 7f633da00640 1 -- 172.21.15.7:0/2105198032 wait complete. 2024-09-10T21:52:32.424 INFO:tasks.ceph:Scrubbing osd.6 2024-09-10T21:52:32.424 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:52:32.589 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.587+0000 7f3718e00640 1 -- 172.21.15.7:0/1089384986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37140cdc40 msgr2=0x7f37140ce0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.587+0000 7f3718e00640 1 --2- 172.21.15.7:0/1089384986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37140cdc40 0x7f37140ce0b0 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f370c005fd0 tx=0x7f370c02aa90 comp rx=0 tx=0).stop 2024-09-10T21:52:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.587+0000 7f3718e00640 1 -- 172.21.15.7:0/1089384986 shutdown_connections 2024-09-10T21:52:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.587+0000 7f3718e00640 1 --2- 172.21.15.7:0/1089384986 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37140cdc40 0x7f37140ce0b0 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.587+0000 7f3718e00640 1 --2- 172.21.15.7:0/1089384986 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f371406fc00 0x7f3714070040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.587+0000 7f3718e00640 1 -- 172.21.15.7:0/1089384986 >> 172.21.15.7:0/1089384986 conn(0x7f371406ce50 msgr2=0x7f371406d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.587+0000 7f3718e00640 1 -- 172.21.15.7:0/1089384986 shutdown_connections 2024-09-10T21:52:32.590 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.587+0000 7f3718e00640 1 -- 172.21.15.7:0/1089384986 wait complete. 2024-09-10T21:52:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.589+0000 7f3718e00640 1 Processor -- start 2024-09-10T21:52:32.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.589+0000 7f3718e00640 1 -- start start 2024-09-10T21:52:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.590+0000 7f3718e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f371406fc00 0x7f37140c8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:32.594 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.590+0000 7f3718e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37140cdc40 0x7f371415d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.590+0000 7f3718e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37141446e0 0x7f37140ca970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.590+0000 7f3718e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f37140b3280 con 0x7f37140cdc40 2024-09-10T21:52:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.590+0000 7f3718e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f37140b3100 con 0x7f37141446e0 2024-09-10T21:52:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.590+0000 7f3718e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f37140b3400 con 0x7f371406fc00 2024-09-10T21:52:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.590+0000 7f3709400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37140cdc40 0x7f371415d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.590+0000 7f3711e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f371406fc00 0x7f37140c8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:32.595 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.590+0000 7f3709400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37140cdc40 0x7f371415d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40218/0 (socket says 172.21.15.7:40218) 2024-09-10T21:52:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.590+0000 7f3709400640 1 -- 172.21.15.7:0/1208894043 learned_addr learned my addr 172.21.15.7:0/1208894043 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.590+0000 7f3712800640 1 --2- 172.21.15.7:0/1208894043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37141446e0 0x7f37140ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.591+0000 7f3709400640 1 -- 172.21.15.7:0/1208894043 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f371406fc00 msgr2=0x7f37140c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.591+0000 7f3709400640 1 --2- 172.21.15.7:0/1208894043 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f371406fc00 0x7f37140c8fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.591+0000 7f3709400640 1 -- 172.21.15.7:0/1208894043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37141446e0 msgr2=0x7f37140ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.591+0000 7f3709400640 1 --2- 172.21.15.7:0/1208894043 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f37141446e0 0x7f37140ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:32.596 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.591+0000 7f3709400640 1 -- 172.21.15.7:0/1208894043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37141f2af0 con 0x7f37140cdc40 2024-09-10T21:52:32.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.591+0000 7f3709400640 1 --2- 172.21.15.7:0/1208894043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37140cdc40 0x7f371415d740 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00d950 tx=0x7f36fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:32.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.591+0000 7f370aa00640 1 -- 172.21.15.7:0/1208894043 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc017070 con 0x7f37140cdc40 2024-09-10T21:52:32.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.591+0000 7f370aa00640 1 -- 172.21.15.7:0/1208894043 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36fc009d20 con 0x7f37140cdc40 2024-09-10T21:52:32.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.591+0000 7f370aa00640 1 -- 172.21.15.7:0/1208894043 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36fc0106c0 con 0x7f37140cdc40 2024-09-10T21:52:32.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.591+0000 7f3718e00640 1 -- 172.21.15.7:0/1208894043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37141f2dd0 con 0x7f37140cdc40 2024-09-10T21:52:32.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.591+0000 7f3718e00640 1 -- 172.21.15.7:0/1208894043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37141f3390 con 0x7f37140cdc40 2024-09-10T21:52:32.597 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.592+0000 7f3718e00640 1 -- 172.21.15.7:0/1208894043 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f37140b3920 con 0x7f37140cdc40 2024-09-10T21:52:32.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.594+0000 7f370aa00640 1 -- 172.21.15.7:0/1208894043 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f36fc010860 con 0x7f37140cdc40 2024-09-10T21:52:32.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.595+0000 7f370aa00640 1 --2- 172.21.15.7:0/1208894043 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f36f0077f70 0x7f36f007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:32.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.595+0000 7f370aa00640 1 -- 172.21.15.7:0/1208894043 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7f36fc099eb0 con 0x7f37140cdc40 2024-09-10T21:52:32.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.595+0000 7f370aa00640 1 --2- 172.21.15.7:0/1208894043 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x7f36f007df70 0x7f36f0080390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:32.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.595+0000 7f370aa00640 1 -- 172.21.15.7:0/1208894043 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36f0080a40 con 0x7f36f007df70 2024-09-10T21:52:32.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.595+0000 7f370aa00640 1 -- 172.21.15.7:0/1208894043 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7f36fc09c4d0 con 0x7f37140cdc40 2024-09-10T21:52:32.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.595+0000 7f3711e00640 1 --2- 172.21.15.7:0/1208894043 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f36f0077f70 0x7f36f007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:32.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.595+0000 7f3712800640 1 --2- 172.21.15.7:0/1208894043 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x7f36f007df70 0x7f36f0080390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:32.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.596+0000 7f3711e00640 1 --2- 172.21.15.7:0/1208894043 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f36f0077f70 0x7f36f007a430 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f37140ce140 tx=0x7f3704009040 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:32.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.596+0000 7f3712800640 1 --2- 172.21.15.7:0/1208894043 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x7f36f007df70 0x7f36f0080390 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:32.599 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.597+0000 7f370aa00640 1 -- 172.21.15.7:0/1208894043 <== osd.6 v2:172.21.15.133:6810/3365981160 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f36f0080a40 con 0x7f36f007df70 2024-09-10T21:52:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.610+0000 7f3718e00640 1 -- 172.21.15.7:0/1208894043 --> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3714134350 con 0x7f36f007df70 2024-09-10T21:52:32.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.611+0000 7f370aa00640 1 -- 172.21.15.7:0/1208894043 <== osd.6 v2:172.21.15.133:6810/3365981160 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3714134350 con 0x7f36f007df70 2024-09-10T21:52:32.613 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:52:32.613 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:52:32.613 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:52:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.611+0000 7f3718e00640 1 -- 172.21.15.7:0/1208894043 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x7f36f007df70 msgr2=0x7f36f0080390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.611+0000 7f3718e00640 1 --2- 172.21.15.7:0/1208894043 >> [v2:172.21.15.133:6810/3365981160,v1:172.21.15.133:6811/3365981160] conn(0x7f36f007df70 0x7f36f0080390 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.611+0000 7f3718e00640 1 -- 172.21.15.7:0/1208894043 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f36f0077f70 msgr2=0x7f36f007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.611+0000 7f3718e00640 1 --2- 172.21.15.7:0/1208894043 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f36f0077f70 0x7f36f007a430 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f37140ce140 tx=0x7f3704009040 comp rx=0 tx=0).stop 2024-09-10T21:52:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.611+0000 7f3718e00640 1 -- 172.21.15.7:0/1208894043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37140cdc40 msgr2=0x7f371415d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.611+0000 7f3718e00640 1 --2- 172.21.15.7:0/1208894043 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f37140cdc40 0x7f371415d740 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00d950 tx=0x7f36fc00de10 comp rx=0 tx=0).stop 2024-09-10T21:52:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.611+0000 7f3712800640 1 -- 172.21.15.7:0/1208894043 reap_dead start 2024-09-10T21:52:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.611+0000 7f3718e00640 1 -- 172.21.15.7:0/1208894043 shutdown_connections 2024-09-10T21:52:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.611+0000 7f3718e00640 1 -- 172.21.15.7:0/1208894043 >> 172.21.15.7:0/1208894043 conn(0x7f371406ce50 msgr2=0x7f371415ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:32.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.612+0000 7f3718e00640 1 -- 172.21.15.7:0/1208894043 shutdown_connections 2024-09-10T21:52:32.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.612+0000 7f3718e00640 1 -- 172.21.15.7:0/1208894043 wait complete. 2024-09-10T21:52:32.627 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-10T21:52:32.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.787+0000 7fab78600640 1 -- 172.21.15.7:0/1645838538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab580041a0 msgr2=0x7fab58004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:32.789 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.787+0000 7fab78600640 1 --2- 172.21.15.7:0/1645838538 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab580041a0 0x7fab58004800 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7fab5c009d00 tx=0x7fab58005830 comp rx=0 tx=0).stop 2024-09-10T21:52:32.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.787+0000 7fab76600640 1 -- 172.21.15.7:0/1645838538 reap_dead start 2024-09-10T21:52:32.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.787+0000 7fab78600640 1 -- 172.21.15.7:0/1645838538 shutdown_connections 2024-09-10T21:52:32.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.787+0000 7fab78600640 1 -- 172.21.15.7:0/1645838538 >> 172.21.15.7:0/1645838538 conn(0x7fab7006ca60 msgr2=0x7fab7006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:32.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.787+0000 7fab78600640 1 -- 172.21.15.7:0/1645838538 shutdown_connections 2024-09-10T21:52:32.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.787+0000 7fab78600640 1 -- 172.21.15.7:0/1645838538 wait complete. 2024-09-10T21:52:32.790 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.790+0000 7fab78600640 1 Processor -- start 2024-09-10T21:52:32.791 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.790+0000 7fab78600640 1 -- start start 2024-09-10T21:52:32.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab78600640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab580041a0 0x7fab701f8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:32.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab78600640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab7015e020 0x7fab700c7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:32.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab78600640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab701f51d0 0x7fab700c4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:32.793 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab78600640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7fab7015edb0 con 0x7fab7015e020 2024-09-10T21:52:32.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab78600640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7fab7015ec30 con 0x7fab701f51d0 2024-09-10T21:52:32.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab78600640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7fab7015c300 con 0x7fab580041a0 2024-09-10T21:52:32.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab75200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab7015e020 0x7fab700c7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:32.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab75200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab7015e020 0x7fab700c7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40238/0 (socket says 172.21.15.7:40238) 2024-09-10T21:52:32.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab75200640 1 -- 172.21.15.7:0/2690497460 learned_addr learned my addr 172.21.15.7:0/2690497460 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:32.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab75c00640 1 --2- 172.21.15.7:0/2690497460 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab580041a0 0x7fab701f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:32.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab76600640 1 --2- 172.21.15.7:0/2690497460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab701f51d0 0x7fab700c4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:32.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab75200640 1 -- 172.21.15.7:0/2690497460 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab580041a0 msgr2=0x7fab701f8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:32.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab75200640 1 --2- 172.21.15.7:0/2690497460 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab580041a0 0x7fab701f8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:32.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab75200640 1 -- 172.21.15.7:0/2690497460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab701f51d0 msgr2=0x7fab700c4f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:32.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab75200640 1 --2- 172.21.15.7:0/2690497460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab701f51d0 0x7fab700c4f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:32.794 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab75200640 1 -- 172.21.15.7:0/2690497460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab701f8790 con 0x7fab7015e020 2024-09-10T21:52:32.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab76600640 1 --2- 172.21.15.7:0/2690497460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab701f51d0 0x7fab700c4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:32.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab75200640 1 --2- 172.21.15.7:0/2690497460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab7015e020 0x7fab700c7460 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fab5c02b620 tx=0x7fab5c005cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:32.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab6d600640 1 -- 172.21.15.7:0/2690497460 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c037050 con 0x7fab7015e020 2024-09-10T21:52:32.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab6d600640 1 -- 172.21.15.7:0/2690497460 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fab5c043070 con 0x7fab7015e020 2024-09-10T21:52:32.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab6d600640 1 -- 172.21.15.7:0/2690497460 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c004bf0 con 0x7fab7015e020 2024-09-10T21:52:32.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab78600640 1 -- 172.21.15.7:0/2690497460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab701f8a70 con 0x7fab7015e020 2024-09-10T21:52:32.795 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.791+0000 7fab78600640 1 -- 172.21.15.7:0/2690497460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab701f4cb0 con 0x7fab7015e020 2024-09-10T21:52:32.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.792+0000 7fab78600640 1 -- 172.21.15.7:0/2690497460 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab701f4aa0 con 0x7fab7015e020 2024-09-10T21:52:32.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.794+0000 7fab6d600640 1 -- 172.21.15.7:0/2690497460 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7fab5c03e450 con 0x7fab7015e020 2024-09-10T21:52:32.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.794+0000 7fab6d600640 1 --2- 172.21.15.7:0/2690497460 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fab50077fa0 0x7fab5007a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:32.796 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.795+0000 7fab6d600640 1 -- 172.21.15.7:0/2690497460 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7fab5c0ba880 con 0x7fab7015e020 2024-09-10T21:52:32.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.795+0000 7fab75c00640 1 --2- 172.21.15.7:0/2690497460 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fab50077fa0 0x7fab5007a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:32.797 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.796+0000 7fab75c00640 1 --2- 172.21.15.7:0/2690497460 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fab50077fa0 0x7fab5007a460 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fab64006fd0 tx=0x7fab64006b10 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:32.800 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.799+0000 7fab6d600640 1 -- 172.21.15.7:0/2690497460 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7fab5c04c020 con 0x7fab7015e020 2024-09-10T21:52:32.997 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.995+0000 7fab78600640 1 -- 172.21.15.7:0/2690497460 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7fab7006e810 con 0x7fab50077fa0 2024-09-10T21:52:32.998 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.996+0000 7fab6d600640 1 -- 172.21.15.7:0/2690497460 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fab7006e810 con 0x7fab50077fa0 2024-09-10T21:52:32.998 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 6 to deep-scrub 2024-09-10T21:52:33.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.998+0000 7fab78600640 1 -- 172.21.15.7:0/2690497460 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fab50077fa0 msgr2=0x7fab5007a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:33.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.998+0000 7fab78600640 1 --2- 172.21.15.7:0/2690497460 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fab50077fa0 0x7fab5007a460 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fab64006fd0 tx=0x7fab64006b10 comp rx=0 tx=0).stop 2024-09-10T21:52:33.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.999+0000 7fab78600640 1 -- 172.21.15.7:0/2690497460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab7015e020 msgr2=0x7fab700c7460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:33.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.999+0000 7fab78600640 1 --2- 172.21.15.7:0/2690497460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab7015e020 0x7fab700c7460 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fab5c02b620 tx=0x7fab5c005cb0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.999+0000 7fab78600640 1 -- 172.21.15.7:0/2690497460 shutdown_connections 2024-09-10T21:52:33.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.999+0000 7fab78600640 1 --2- 172.21.15.7:0/2690497460 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7fab50077fa0 0x7fab5007a460 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.999+0000 7fab78600640 1 --2- 172.21.15.7:0/2690497460 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7fab701f51d0 0x7fab700c4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.999+0000 7fab78600640 1 --2- 172.21.15.7:0/2690497460 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7fab7015e020 0x7fab700c7460 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.999+0000 7fab78600640 1 --2- 172.21.15.7:0/2690497460 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7fab580041a0 0x7fab701f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.999+0000 7fab78600640 1 -- 172.21.15.7:0/2690497460 >> 172.21.15.7:0/2690497460 conn(0x7fab7006ca60 msgr2=0x7fab700b32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:33.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.999+0000 7fab78600640 1 -- 172.21.15.7:0/2690497460 shutdown_connections 2024-09-10T21:52:33.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:32.999+0000 7fab78600640 1 -- 172.21.15.7:0/2690497460 wait complete. 2024-09-10T21:52:33.016 INFO:tasks.ceph:Scrubbing osd.7 2024-09-10T21:52:33.016 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:52:33.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.178+0000 7f3c39000640 1 -- 172.21.15.7:0/4210010708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c340cd340 msgr2=0x7f3c340ce150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:33.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.178+0000 7f3c39000640 1 --2- 172.21.15.7:0/4210010708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c340cd340 0x7f3c340ce150 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c005fd0 tx=0x7f3c2c02aa10 comp rx=0 tx=0).stop 2024-09-10T21:52:33.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.178+0000 7f3c39000640 1 -- 172.21.15.7:0/4210010708 shutdown_connections 2024-09-10T21:52:33.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.178+0000 7f3c39000640 1 --2- 172.21.15.7:0/4210010708 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c340cd340 0x7f3c340ce150 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.180 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.178+0000 7f3c39000640 1 --2- 172.21.15.7:0/4210010708 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c34145610 0x7f3c34073280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.178+0000 7f3c39000640 1 -- 172.21.15.7:0/4210010708 >> 172.21.15.7:0/4210010708 conn(0x7f3c3406ce50 msgr2=0x7f3c3406d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:33.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.178+0000 7f3c39000640 1 -- 172.21.15.7:0/4210010708 shutdown_connections 2024-09-10T21:52:33.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.178+0000 7f3c39000640 1 -- 172.21.15.7:0/4210010708 wait complete. 2024-09-10T21:52:33.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.180+0000 7f3c39000640 1 Processor -- start 2024-09-10T21:52:33.181 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.180+0000 7f3c39000640 1 -- start start 2024-09-10T21:52:33.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.181+0000 7f3c39000640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c3406ff60 0x7f3c3413fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:33.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.181+0000 7f3c39000640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c340cd340 0x7f3c340b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:33.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.181+0000 7f3c39000640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c34145610 0x7f3c340ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:33.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.181+0000 7f3c39000640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3406f1b0 con 0x7f3c340cd340 2024-09-10T21:52:33.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.181+0000 7f3c39000640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f3c3406f030 con 0x7f3c3406ff60 2024-09-10T21:52:33.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.181+0000 7f3c39000640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f3c3406f330 con 0x7f3c34145610 2024-09-10T21:52:33.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.181+0000 7f3c29400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c340cd340 0x7f3c340b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:33.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.181+0000 7f3c29400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c340cd340 0x7f3c340b0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40250/0 (socket says 172.21.15.7:40250) 2024-09-10T21:52:33.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.181+0000 7f3c29400640 1 -- 172.21.15.7:0/801429726 learned_addr learned my addr 172.21.15.7:0/801429726 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:33.183 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.181+0000 7f3c32800640 1 --2- 172.21.15.7:0/801429726 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c34145610 0x7f3c340ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:33.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.181+0000 7f3c31e00640 1 --2- 172.21.15.7:0/801429726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c3406ff60 0x7f3c3413fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:33.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.182+0000 7f3c29400640 1 -- 172.21.15.7:0/801429726 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c34145610 msgr2=0x7f3c340ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:33.184 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.182+0000 7f3c29400640 1 --2- 172.21.15.7:0/801429726 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f3c34145610 0x7f3c340ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.182+0000 7f3c29400640 1 -- 172.21.15.7:0/801429726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c3406ff60 msgr2=0x7f3c3413fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:33.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.182+0000 7f3c29400640 1 --2- 172.21.15.7:0/801429726 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f3c3406ff60 0x7f3c3413fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.182+0000 7f3c29400640 1 -- 172.21.15.7:0/801429726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c340b08d0 con 0x7f3c340cd340 2024-09-10T21:52:33.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.182+0000 7f3c29400640 1 --2- 172.21.15.7:0/801429726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c340cd340 0x7f3c340b0390 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00c450 tx=0x7f3c1c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:33.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.182+0000 7f3c2aa00640 1 -- 172.21.15.7:0/801429726 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c0052d0 con 0x7f3c340cd340 2024-09-10T21:52:33.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.182+0000 7f3c2aa00640 1 -- 172.21.15.7:0/801429726 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c1c005470 con 0x7f3c340cd340 2024-09-10T21:52:33.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.182+0000 7f3c2aa00640 1 -- 172.21.15.7:0/801429726 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c012870 con 0x7f3c340cd340 2024-09-10T21:52:33.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.182+0000 7f3c39000640 1 -- 172.21.15.7:0/801429726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c341409e0 con 0x7f3c340cd340 2024-09-10T21:52:33.185 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.182+0000 7f3c39000640 1 -- 172.21.15.7:0/801429726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c340afb90 con 0x7f3c340cd340 2024-09-10T21:52:33.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.183+0000 7f3c39000640 1 -- 172.21.15.7:0/801429726 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3c3406f7e0 con 0x7f3c340cd340 2024-09-10T21:52:33.186 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.185+0000 7f3c2aa00640 1 -- 172.21.15.7:0/801429726 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f3c1c0035f0 con 0x7f3c340cd340 2024-09-10T21:52:33.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.185+0000 7f3c2aa00640 1 --2- 172.21.15.7:0/801429726 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3c10077f70 0x7f3c1007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:33.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.185+0000 7f3c2aa00640 1 -- 172.21.15.7:0/801429726 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7f3c1c099590 con 0x7f3c340cd340 2024-09-10T21:52:33.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.186+0000 7f3c2aa00640 1 --2- 172.21.15.7:0/801429726 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x7f3c1007df70 0x7f3c10080390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:33.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.186+0000 7f3c2aa00640 1 -- 172.21.15.7:0/801429726 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3c10080a40 con 0x7f3c1007df70 2024-09-10T21:52:33.187 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.186+0000 7f3c2aa00640 1 -- 172.21.15.7:0/801429726 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_get_version_reply(handle=1 version=121) ==== 24+0+0 (secure 0 0 0) 0x7f3c1c0a0050 con 0x7f3c340cd340 2024-09-10T21:52:33.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.186+0000 7f3c31e00640 1 --2- 172.21.15.7:0/801429726 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3c10077f70 0x7f3c1007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:33.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.186+0000 7f3c32800640 1 --2- 172.21.15.7:0/801429726 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x7f3c1007df70 0x7f3c10080390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:33.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.187+0000 7f3c31e00640 1 --2- 172.21.15.7:0/801429726 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3c10077f70 0x7f3c1007a430 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3c24004590 tx=0x7f3c2400b390 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:33.188 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.187+0000 7f3c32800640 1 --2- 172.21.15.7:0/801429726 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x7f3c1007df70 0x7f3c10080390 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:33.189 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.188+0000 7f3c2aa00640 1 -- 172.21.15.7:0/801429726 <== osd.7 v2:172.21.15.133:6802/2638399040 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3c10080a40 con 0x7f3c1007df70 2024-09-10T21:52:33.202 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.202+0000 7f3c39000640 1 -- 172.21.15.7:0/801429726 --> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3c340b1b80 con 0x7f3c1007df70 2024-09-10T21:52:33.203 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.202+0000 7f3c2aa00640 1 -- 172.21.15.7:0/801429726 <== osd.7 v2:172.21.15.133:6802/2638399040 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3c340b1b80 con 0x7f3c1007df70 2024-09-10T21:52:33.203 INFO:teuthology.orchestra.run.smithi007.stdout:{ 2024-09-10T21:52:33.203 INFO:teuthology.orchestra.run.smithi007.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:52:33.203 INFO:teuthology.orchestra.run.smithi007.stdout:} 2024-09-10T21:52:33.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.202+0000 7f3c39000640 1 -- 172.21.15.7:0/801429726 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x7f3c1007df70 msgr2=0x7f3c10080390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:33.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.202+0000 7f3c39000640 1 --2- 172.21.15.7:0/801429726 >> [v2:172.21.15.133:6802/2638399040,v1:172.21.15.133:6803/2638399040] conn(0x7f3c1007df70 0x7f3c10080390 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.202+0000 7f3c39000640 1 -- 172.21.15.7:0/801429726 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3c10077f70 msgr2=0x7f3c1007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:33.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.202+0000 7f3c39000640 1 --2- 172.21.15.7:0/801429726 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f3c10077f70 0x7f3c1007a430 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f3c24004590 tx=0x7f3c2400b390 comp rx=0 tx=0).stop 2024-09-10T21:52:33.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.202+0000 7f3c39000640 1 -- 172.21.15.7:0/801429726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c340cd340 msgr2=0x7f3c340b0390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:33.204 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.202+0000 7f3c39000640 1 --2- 172.21.15.7:0/801429726 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f3c340cd340 0x7f3c340b0390 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00c450 tx=0x7f3c1c00c910 comp rx=0 tx=0).stop 2024-09-10T21:52:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.202+0000 7f3c32800640 1 -- 172.21.15.7:0/801429726 reap_dead start 2024-09-10T21:52:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.202+0000 7f3c39000640 1 -- 172.21.15.7:0/801429726 shutdown_connections 2024-09-10T21:52:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.202+0000 7f3c39000640 1 -- 172.21.15.7:0/801429726 >> 172.21.15.7:0/801429726 conn(0x7f3c3406ce50 msgr2=0x7f3c340ccc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.202+0000 7f3c39000640 1 -- 172.21.15.7:0/801429726 shutdown_connections 2024-09-10T21:52:33.205 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.202+0000 7f3c39000640 1 -- 172.21.15.7:0/801429726 wait complete. 2024-09-10T21:52:33.218 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-10T21:52:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.380+0000 7f4964a00640 1 -- 172.21.15.7:0/3490607130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4944003fd0 msgr2=0x7f4944004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.380+0000 7f4964a00640 1 --2- 172.21.15.7:0/3490607130 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4944003fd0 0x7f4944004800 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f494c00c290 tx=0x7f4944005830 comp rx=0 tx=0).stop 2024-09-10T21:52:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.380+0000 7f495e800640 1 -- 172.21.15.7:0/3490607130 reap_dead start 2024-09-10T21:52:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.380+0000 7f4964a00640 1 -- 172.21.15.7:0/3490607130 shutdown_connections 2024-09-10T21:52:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.380+0000 7f4964a00640 1 -- 172.21.15.7:0/3490607130 >> 172.21.15.7:0/3490607130 conn(0x7f496006ca60 msgr2=0x7f496006ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:33.382 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.380+0000 7f4964a00640 1 -- 172.21.15.7:0/3490607130 shutdown_connections 2024-09-10T21:52:33.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.380+0000 7f4964a00640 1 -- 172.21.15.7:0/3490607130 wait complete. 2024-09-10T21:52:33.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.383+0000 7f4964a00640 1 Processor -- start 2024-09-10T21:52:33.383 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.383+0000 7f4964a00640 1 -- start start 2024-09-10T21:52:33.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f4964a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4944003fd0 0x7f4960155500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:33.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f4964a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4960152a50 0x7f496014e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:33.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f4964a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49600b0c70 0x7f49600b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:33.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f4964a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f49600732c0 con 0x7f49600b0c70 2024-09-10T21:52:33.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f4964a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4960073140 con 0x7f4944003fd0 2024-09-10T21:52:33.386 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f4964a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4960073440 con 0x7f4960152a50 2024-09-10T21:52:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f495e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49600b0c70 0x7f49600b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f495e800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49600b0c70 0x7f49600b1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:40274/0 (socket says 172.21.15.7:40274) 2024-09-10T21:52:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f495e800640 1 -- 172.21.15.7:0/1687600886 learned_addr learned my addr 172.21.15.7:0/1687600886 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f495de00640 1 --2- 172.21.15.7:0/1687600886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4944003fd0 0x7f4960155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f495d400640 1 --2- 172.21.15.7:0/1687600886 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4960152a50 0x7f496014e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f495e800640 1 -- 172.21.15.7:0/1687600886 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4960152a50 msgr2=0x7f496014e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f495e800640 1 --2- 172.21.15.7:0/1687600886 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4960152a50 0x7f496014e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f495e800640 1 -- 172.21.15.7:0/1687600886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4944003fd0 msgr2=0x7f4960155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f495e800640 1 --2- 172.21.15.7:0/1687600886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4944003fd0 0x7f4960155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f495e800640 1 -- 172.21.15.7:0/1687600886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4960155ba0 con 0x7f49600b0c70 2024-09-10T21:52:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f495e800640 1 --2- 172.21.15.7:0/1687600886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49600b0c70 0x7f49600b1120 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f496015d5d0 tx=0x7f4958011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f4956000640 1 -- 172.21.15.7:0/1687600886 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f495800fc80 con 0x7f49600b0c70 2024-09-10T21:52:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f4956000640 1 -- 172.21.15.7:0/1687600886 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f495800fe20 con 0x7f49600b0c70 2024-09-10T21:52:33.387 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f4956000640 1 -- 172.21.15.7:0/1687600886 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958004d80 con 0x7f49600b0c70 2024-09-10T21:52:33.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f4964a00640 1 -- 172.21.15.7:0/1687600886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49601f5f90 con 0x7f49600b0c70 2024-09-10T21:52:33.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.384+0000 7f4964a00640 1 -- 172.21.15.7:0/1687600886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4960152770 con 0x7f49600b0c70 2024-09-10T21:52:33.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.385+0000 7f4964a00640 1 -- 172.21.15.7:0/1687600886 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49601f5db0 con 0x7f49600b0c70 2024-09-10T21:52:33.388 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.386+0000 7f4956000640 1 -- 172.21.15.7:0/1687600886 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4958019410 con 0x7f49600b0c70 2024-09-10T21:52:33.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.388+0000 7f4956000640 1 --2- 172.21.15.7:0/1687600886 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4940077f70 0x7f494007a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:33.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.388+0000 7f4956000640 1 -- 172.21.15.7:0/1687600886 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7f495809a2d0 con 0x7f49600b0c70 2024-09-10T21:52:33.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.388+0000 7f495de00640 1 --2- 172.21.15.7:0/1687600886 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4940077f70 0x7f494007a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:33.390 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.390+0000 7f495de00640 1 --2- 172.21.15.7:0/1687600886 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4940077f70 0x7f494007a430 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4950009cd0 tx=0x7f4950005f90 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:33.394 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.393+0000 7f4956000640 1 -- 172.21.15.7:0/1687600886 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f49601f5db0 con 0x7f49600b0c70 2024-09-10T21:52:33.587 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.586+0000 7f4964a00640 1 -- 172.21.15.7:0/1687600886 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f496015d090 con 0x7f4940077f70 2024-09-10T21:52:33.588 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.586+0000 7f4956000640 1 -- 172.21.15.7:0/1687600886 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f496015d090 con 0x7f4940077f70 2024-09-10T21:52:33.588 INFO:teuthology.orchestra.run.smithi007.stderr:instructed osd(s) 7 to deep-scrub 2024-09-10T21:52:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.589+0000 7f4964a00640 1 -- 172.21.15.7:0/1687600886 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4940077f70 msgr2=0x7f494007a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.589+0000 7f4964a00640 1 --2- 172.21.15.7:0/1687600886 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4940077f70 0x7f494007a430 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f4950009cd0 tx=0x7f4950005f90 comp rx=0 tx=0).stop 2024-09-10T21:52:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.589+0000 7f4964a00640 1 -- 172.21.15.7:0/1687600886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49600b0c70 msgr2=0x7f49600b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.589+0000 7f4964a00640 1 --2- 172.21.15.7:0/1687600886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49600b0c70 0x7f49600b1120 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f496015d5d0 tx=0x7f4958011ed0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.589+0000 7f4964a00640 1 -- 172.21.15.7:0/1687600886 shutdown_connections 2024-09-10T21:52:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.589+0000 7f4964a00640 1 --2- 172.21.15.7:0/1687600886 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4940077f70 0x7f494007a430 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.589+0000 7f4964a00640 1 --2- 172.21.15.7:0/1687600886 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f49600b0c70 0x7f49600b1120 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.591 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.589+0000 7f4964a00640 1 --2- 172.21.15.7:0/1687600886 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4960152a50 0x7f496014e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.589+0000 7f4964a00640 1 --2- 172.21.15.7:0/1687600886 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4944003fd0 0x7f4960155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.589+0000 7f4964a00640 1 -- 172.21.15.7:0/1687600886 >> 172.21.15.7:0/1687600886 conn(0x7f496006ca60 msgr2=0x7f49600b22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:33.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.589+0000 7f4964a00640 1 -- 172.21.15.7:0/1687600886 shutdown_connections 2024-09-10T21:52:33.592 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.589+0000 7f4964a00640 1 -- 172.21.15.7:0/1687600886 wait complete. 2024-09-10T21:52:33.606 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:52:33.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.832+0000 7f445be00640 1 -- 172.21.15.7:0/921942372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c0b01e0 msgr2=0x7f445c0d05e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:33.833 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.832+0000 7f445be00640 1 --2- 172.21.15.7:0/921942372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c0b01e0 0x7f445c0d05e0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f445c156a50 tx=0x7f444c02a560 comp rx=0 tx=0).stop 2024-09-10T21:52:33.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.832+0000 7f445be00640 1 -- 172.21.15.7:0/921942372 shutdown_connections 2024-09-10T21:52:33.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.832+0000 7f445be00640 1 --2- 172.21.15.7:0/921942372 >> v2:172.21.15.133:3300/0 conn(0x7f445c0d1860 0x7f445c06cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.832+0000 7f445be00640 1 --2- 172.21.15.7:0/921942372 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f445c0b05c0 0x7f445c06ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.832+0000 7f445be00640 1 --2- 172.21.15.7:0/921942372 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c0b01e0 0x7f445c0d05e0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.832+0000 7f445be00640 1 -- 172.21.15.7:0/921942372 >> 172.21.15.7:0/921942372 conn(0x7f445c0d0d70 msgr2=0x7f445c0d1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:33.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.832+0000 7f445be00640 1 -- 172.21.15.7:0/921942372 shutdown_connections 2024-09-10T21:52:33.834 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.832+0000 7f445be00640 1 -- 172.21.15.7:0/921942372 wait complete. 2024-09-10T21:52:33.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.834+0000 7f445be00640 1 Processor -- start 2024-09-10T21:52:33.835 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.834+0000 7f445be00640 1 -- start start 2024-09-10T21:52:33.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.835+0000 7f445be00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f445c0b01e0 0x7f445c148260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:33.836 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.835+0000 7f445be00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c0d1860 0x7f445c146220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.835+0000 7f445be00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c15bfa0 0x7f445c09de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.835+0000 7f445be00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f445c0cf1f0 con 0x7f445c15bfa0 2024-09-10T21:52:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.835+0000 7f445be00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f445c0cf070 con 0x7f445c0d1860 2024-09-10T21:52:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.835+0000 7f445be00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f445c0cf370 con 0x7f445c0b01e0 2024-09-10T21:52:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.835+0000 7f445b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c15bfa0 0x7f445c09de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.835+0000 7f445b400640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c15bfa0 0x7f445c09de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:53726/0 (socket says 172.21.15.7:53726) 2024-09-10T21:52:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.835+0000 7f445b400640 1 -- 172.21.15.7:0/611405483 learned_addr learned my addr 172.21.15.7:0/611405483 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.835+0000 7f445b400640 1 -- 172.21.15.7:0/611405483 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f445c0b01e0 msgr2=0x7f445c148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.835+0000 7f445b400640 1 --2- 172.21.15.7:0/611405483 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f445c0b01e0 0x7f445c148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.835+0000 7f445b400640 1 -- 172.21.15.7:0/611405483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c0d1860 msgr2=0x7f445c146220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:33.837 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.835+0000 7f445a000640 1 --2- 172.21.15.7:0/611405483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c0d1860 0x7f445c146220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:33.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.836+0000 7f445b400640 1 --2- 172.21.15.7:0/611405483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c0d1860 0x7f445c146220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:33.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.836+0000 7f445b400640 1 -- 172.21.15.7:0/611405483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445c148870 con 0x7f445c15bfa0 2024-09-10T21:52:33.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.836+0000 7f445a000640 1 --2- 172.21.15.7:0/611405483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c0d1860 0x7f445c146220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:52:33.838 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.836+0000 7f445b400640 1 --2- 172.21.15.7:0/611405483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c15bfa0 0x7f445c09de30 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f445400ea80 tx=0x7f445400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:33.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.836+0000 7f4452a00640 1 -- 172.21.15.7:0/611405483 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454004b20 con 0x7f445c15bfa0 2024-09-10T21:52:33.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.836+0000 7f4452a00640 1 -- 172.21.15.7:0/611405483 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4454004ce0 con 0x7f445c15bfa0 2024-09-10T21:52:33.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.836+0000 7f445be00640 1 -- 172.21.15.7:0/611405483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445c148b50 con 0x7f445c15bfa0 2024-09-10T21:52:33.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.836+0000 7f4452a00640 1 -- 172.21.15.7:0/611405483 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454005280 con 0x7f445c15bfa0 2024-09-10T21:52:33.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.836+0000 7f445be00640 1 -- 172.21.15.7:0/611405483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f445c149110 con 0x7f445c15bfa0 2024-09-10T21:52:33.839 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.837+0000 7f445be00640 1 -- 172.21.15.7:0/611405483 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f445c09aa30 con 0x7f445c15bfa0 2024-09-10T21:52:33.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.838+0000 7f4452a00640 1 -- 172.21.15.7:0/611405483 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4454005420 con 0x7f445c15bfa0 2024-09-10T21:52:33.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.839+0000 7f4452a00640 1 --2- 172.21.15.7:0/611405483 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f443c077f70 0x7f443c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:33.840 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.839+0000 7f4452a00640 1 -- 172.21.15.7:0/611405483 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7f445409a880 con 0x7f445c15bfa0 2024-09-10T21:52:33.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.840+0000 7f445aa00640 1 --2- 172.21.15.7:0/611405483 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f443c077f70 0x7f443c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:33.841 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.840+0000 7f445aa00640 1 --2- 172.21.15.7:0/611405483 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f443c077f70 0x7f443c07a430 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f444c033000 tx=0x7f444c0046c0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:33.844 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:33.843+0000 7f4452a00640 1 -- 172.21.15.7:0/611405483 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4454063530 con 0x7f445c15bfa0 2024-09-10T21:52:34.035 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:34.034+0000 7f445be00640 1 -- 172.21.15.7:0/611405483 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f445c0cf890 con 0x7f443c077f70 2024-09-10T21:52:34.036 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:34.036+0000 7f4452a00640 1 -- 172.21.15.7:0/611405483 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13357 (secure 0 0 0) 0x7f445c0cf890 con 0x7f443c077f70 2024-09-10T21:52:34.037 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:34.037 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:52:34.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:34.038+0000 7f445be00640 1 -- 172.21.15.7:0/611405483 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f443c077f70 msgr2=0x7f443c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:34.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:34.038+0000 7f445be00640 1 --2- 172.21.15.7:0/611405483 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f443c077f70 0x7f443c07a430 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f444c033000 tx=0x7f444c0046c0 comp rx=0 tx=0).stop 2024-09-10T21:52:34.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:34.038+0000 7f445be00640 1 -- 172.21.15.7:0/611405483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c15bfa0 msgr2=0x7f445c09de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:34.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:34.038+0000 7f445be00640 1 --2- 172.21.15.7:0/611405483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c15bfa0 0x7f445c09de30 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f445400ea80 tx=0x7f445400ef40 comp rx=0 tx=0).stop 2024-09-10T21:52:34.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:34.038+0000 7f445be00640 1 -- 172.21.15.7:0/611405483 shutdown_connections 2024-09-10T21:52:34.039 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:34.038+0000 7f445be00640 1 --2- 172.21.15.7:0/611405483 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f443c077f70 0x7f443c07a430 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:34.038+0000 7f445be00640 1 --2- 172.21.15.7:0/611405483 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f445c0d1860 0x7f445c146220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:34.038+0000 7f445be00640 1 --2- 172.21.15.7:0/611405483 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f445c15bfa0 0x7f445c09de30 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:34.038+0000 7f445be00640 1 --2- 172.21.15.7:0/611405483 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f445c0b01e0 0x7f445c148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:34.038+0000 7f445be00640 1 -- 172.21.15.7:0/611405483 >> 172.21.15.7:0/611405483 conn(0x7f445c0d0d70 msgr2=0x7f445c0ae860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:34.038+0000 7f445be00640 1 -- 172.21.15.7:0/611405483 shutdown_connections 2024-09-10T21:52:34.040 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:34.038+0000 7f445be00640 1 -- 172.21.15.7:0/611405483 wait complete. 2024-09-10T21:52:34.055 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":433,"stamp":"2024-09-10T21:52:34.022777+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":279080,"kb_used_data":12904,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754695640,"statfs":{"total":773094113280,"available":772808335360,"internally_reserved":0,"allocated":13213696,"data_stored":6402912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.769497"},"pg_stats":[{"pgid":"1.0","version":"17'72","reported_seq":296,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T21:52:24.554502+0000","last_change":"2024-09-10T21:41:57.870063+0000","last_active":"2024-09-10T21:52:24.554502+0000","last_peered":"2024-09-10T21:52:24.554502+0000","last_clean":"2024-09-10T21:52:24.554502+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:52:24.554502+0000","last_undegraded":"2024-09-10T21:52:24.554502+0000","last_fullsized":"2024-09-10T21:52:24.554502+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:41:56.841496+0000","last_clean_scrub_stamp":"2024-09-10T21:41:56.841496+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:27:27.485725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509571,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32660,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1503232,"data_stored":652772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509571,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32660,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1503232,"data_stored":652772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32660,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1503232,"data_stored":652772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509570,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36820,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335020,"statfs":{"total":96636764160,"available":96599060480,"internally_reserved":0,"allocated":1503232,"data_stored":652772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509572,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36820,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335020,"statfs":{"total":96636764160,"available":96599060480,"internally_reserved":0,"allocated":1503232,"data_stored":652772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509570,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37400,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334440,"statfs":{"total":96636764160,"available":96598466560,"internally_reserved":0,"allocated":2097152,"data_stored":1243140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542273,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2097152,"data_stored":1243140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542274,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36824,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":1503232,"data_stored":652772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:52:34.058 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-10T21:41:56.841496+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=41, tm_sec=56, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=52, tm_sec=27, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:52:34.058 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-10T21:52:54.059 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:52:54.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.229+0000 7f9880a00640 1 -- 172.21.15.7:0/2090827566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f987c0b50e0 msgr2=0x7f987c0739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:54.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.229+0000 7f9880a00640 1 --2- 172.21.15.7:0/2090827566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f987c0b50e0 0x7f987c0739c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f9874005fd0 tx=0x7f987402a8b0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.230+0000 7f9880a00640 1 -- 172.21.15.7:0/2090827566 shutdown_connections 2024-09-10T21:52:54.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.230+0000 7f9880a00640 1 --2- 172.21.15.7:0/2090827566 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f987c0b50e0 0x7f987c0739c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.231 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.230+0000 7f9880a00640 1 --2- 172.21.15.7:0/2090827566 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f987c1514b0 0x7f987c15b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.230+0000 7f9880a00640 1 -- 172.21.15.7:0/2090827566 >> 172.21.15.7:0/2090827566 conn(0x7f987c06ca60 msgr2=0x7f987c06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:54.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.230+0000 7f9880a00640 1 -- 172.21.15.7:0/2090827566 shutdown_connections 2024-09-10T21:52:54.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.230+0000 7f9880a00640 1 -- 172.21.15.7:0/2090827566 wait complete. 2024-09-10T21:52:54.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.231+0000 7f9880a00640 1 Processor -- start 2024-09-10T21:52:54.232 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.232+0000 7f9880a00640 1 -- start start 2024-09-10T21:52:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.232+0000 7f9880a00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f987c0b50e0 0x7f987c14f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.233+0000 7f9880a00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f987c0cded0 0x7f987c1f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.233+0000 7f9880a00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f987c1514b0 0x7f987c14fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.233+0000 7f9880a00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f987c074290 con 0x7f987c1514b0 2024-09-10T21:52:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.233+0000 7f9880a00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f987c074110 con 0x7f987c0b50e0 2024-09-10T21:52:54.234 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.233+0000 7f9880a00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f987c074410 con 0x7f987c0cded0 2024-09-10T21:52:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.233+0000 7f987a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f987c1514b0 0x7f987c14fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.233+0000 7f987a800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f987c1514b0 0x7f987c14fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39212/0 (socket says 172.21.15.7:39212) 2024-09-10T21:52:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.233+0000 7f987a800640 1 -- 172.21.15.7:0/2687621646 learned_addr learned my addr 172.21.15.7:0/2687621646 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.233+0000 7f9871400640 1 --2- 172.21.15.7:0/2687621646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f987c0cded0 0x7f987c1f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.233+0000 7f9879e00640 1 --2- 172.21.15.7:0/2687621646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f987c0b50e0 0x7f987c14f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.233+0000 7f987a800640 1 -- 172.21.15.7:0/2687621646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f987c0cded0 msgr2=0x7f987c1f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.233+0000 7f987a800640 1 --2- 172.21.15.7:0/2687621646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f987c0cded0 0x7f987c1f5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.233+0000 7f987a800640 1 -- 172.21.15.7:0/2687621646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f987c0b50e0 msgr2=0x7f987c14f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.233+0000 7f987a800640 1 --2- 172.21.15.7:0/2687621646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f987c0b50e0 0x7f987c14f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.235 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.233+0000 7f987a800640 1 -- 172.21.15.7:0/2687621646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987c0af4b0 con 0x7f987c1514b0 2024-09-10T21:52:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.234+0000 7f987a800640 1 --2- 172.21.15.7:0/2687621646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f987c1514b0 0x7f987c14fe50 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f9874009b30 tx=0x7f98740316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.234+0000 7f9872a00640 1 -- 172.21.15.7:0/2687621646 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874040070 con 0x7f987c1514b0 2024-09-10T21:52:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.234+0000 7f9872a00640 1 -- 172.21.15.7:0/2687621646 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9874004040 con 0x7f987c1514b0 2024-09-10T21:52:54.236 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.234+0000 7f9872a00640 1 -- 172.21.15.7:0/2687621646 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9874004450 con 0x7f987c1514b0 2024-09-10T21:52:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.234+0000 7f9880a00640 1 -- 172.21.15.7:0/2687621646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987c0af1d0 con 0x7f987c1514b0 2024-09-10T21:52:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.234+0000 7f9880a00640 1 -- 172.21.15.7:0/2687621646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f987c157e20 con 0x7f987c1514b0 2024-09-10T21:52:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.235+0000 7f9880a00640 1 -- 172.21.15.7:0/2687621646 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f987c0739c0 con 0x7f987c1514b0 2024-09-10T21:52:54.237 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.237+0000 7f9872a00640 1 -- 172.21.15.7:0/2687621646 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f98740045f0 con 0x7f987c1514b0 2024-09-10T21:52:54.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.237+0000 7f9872a00640 1 --2- 172.21.15.7:0/2687621646 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9858077f70 0x7f985807a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:54.238 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.238+0000 7f9872a00640 1 -- 172.21.15.7:0/2687621646 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7f98740bb4c0 con 0x7f987c1514b0 2024-09-10T21:52:54.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.238+0000 7f9879e00640 1 --2- 172.21.15.7:0/2687621646 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9858077f70 0x7f985807a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:54.239 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.239+0000 7f9879e00640 1 --2- 172.21.15.7:0/2687621646 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9858077f70 0x7f985807a430 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f987c1eb840 tx=0x7f986c00a370 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:54.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.242+0000 7f9872a00640 1 -- 172.21.15.7:0/2687621646 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f9874084180 con 0x7f987c1514b0 2024-09-10T21:52:54.433 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.432+0000 7f9880a00640 1 -- 172.21.15.7:0/2687621646 --> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f987c0aef80 con 0x7f9858077f70 2024-09-10T21:52:54.434 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.433+0000 7f9872a00640 1 -- 172.21.15.7:0/2687621646 <== mgr.4123 v2:172.21.15.133:6800/4032855124 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13364 (secure 0 0 0) 0x7f987c0aef80 con 0x7f9858077f70 2024-09-10T21:52:54.434 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:54.434 INFO:teuthology.orchestra.run.smithi007.stderr:dumped all 2024-09-10T21:52:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.435+0000 7f9880a00640 1 -- 172.21.15.7:0/2687621646 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9858077f70 msgr2=0x7f985807a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.435+0000 7f9880a00640 1 --2- 172.21.15.7:0/2687621646 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9858077f70 0x7f985807a430 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f987c1eb840 tx=0x7f986c00a370 comp rx=0 tx=0).stop 2024-09-10T21:52:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.435+0000 7f9880a00640 1 -- 172.21.15.7:0/2687621646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f987c1514b0 msgr2=0x7f987c14fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.435+0000 7f9880a00640 1 --2- 172.21.15.7:0/2687621646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f987c1514b0 0x7f987c14fe50 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f9874009b30 tx=0x7f98740316e0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.436+0000 7f9880a00640 1 -- 172.21.15.7:0/2687621646 shutdown_connections 2024-09-10T21:52:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.436+0000 7f9880a00640 1 --2- 172.21.15.7:0/2687621646 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f9858077f70 0x7f985807a430 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.436+0000 7f9880a00640 1 --2- 172.21.15.7:0/2687621646 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f987c1514b0 0x7f987c14fe50 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.437 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.436+0000 7f9880a00640 1 --2- 172.21.15.7:0/2687621646 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f987c0cded0 0x7f987c1f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.436+0000 7f9880a00640 1 --2- 172.21.15.7:0/2687621646 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f987c0b50e0 0x7f987c14f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.436+0000 7f9880a00640 1 -- 172.21.15.7:0/2687621646 >> 172.21.15.7:0/2687621646 conn(0x7f987c06ca60 msgr2=0x7f987c06ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:54.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.436+0000 7f9880a00640 1 -- 172.21.15.7:0/2687621646 shutdown_connections 2024-09-10T21:52:54.438 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.436+0000 7f9880a00640 1 -- 172.21.15.7:0/2687621646 wait complete. 2024-09-10T21:52:54.453 INFO:teuthology.orchestra.run.smithi007.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-09-10T21:52:54.027347+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":279068,"kb_used_data":12904,"kb_used_omap":12,"kb_used_meta":265971,"kb_avail":754695652,"statfs":{"total":773094113280,"available":772808347648,"internally_reserved":0,"allocated":13213696,"data_stored":6402912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":272354865},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002891"},"pg_stats":[{"pgid":"1.0","version":"121'74","reported_seq":306,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-09-10T21:52:34.322477+0000","last_change":"2024-09-10T21:52:34.322477+0000","last_active":"2024-09-10T21:52:34.322477+0000","last_peered":"2024-09-10T21:52:34.322477+0000","last_clean":"2024-09-10T21:52:34.322477+0000","last_became_active":"2024-09-10T21:41:57.869089+0000","last_became_peered":"2024-09-10T21:41:57.869089+0000","last_unstale":"2024-09-10T21:52:34.322477+0000","last_undegraded":"2024-09-10T21:52:34.322477+0000","last_fullsized":"2024-09-10T21:52:34.322477+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"121'74","last_scrub_stamp":"2024-09-10T21:52:34.322381+0000","last_deep_scrub":"121'74","last_deep_scrub_stamp":"2024-09-10T21:52:34.322381+0000","last_clean_scrub_stamp":"2024-09-10T21:52:34.322381+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:53:11.299864+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":6,"up_from":15,"seq":64424509575,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32660,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1503232,"data_stored":652772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509575,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32660,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1503232,"data_stored":652772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509574,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32660,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339180,"statfs":{"total":96636764160,"available":96603320320,"internally_reserved":0,"allocated":1503232,"data_stored":652772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509574,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36820,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335020,"statfs":{"total":96636764160,"available":96599060480,"internally_reserved":0,"allocated":1503232,"data_stored":652772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509576,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36820,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335020,"statfs":{"total":96636764160,"available":96599060480,"internally_reserved":0,"allocated":1503232,"data_stored":652772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509574,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37392,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334448,"statfs":{"total":96636764160,"available":96598474752,"internally_reserved":0,"allocated":2097152,"data_stored":1243140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542277,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33232,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338608,"statfs":{"total":96636764160,"available":96602734592,"internally_reserved":0,"allocated":2097152,"data_stored":1243140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542278,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36824,"kb_used_data":1468,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":1503232,"data_stored":652772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:52:54.454 DEBUG:teuthology.orchestra.run.smithi007:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-10T21:52:54.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.609+0000 7f4e46e00640 1 -- 172.21.15.7:0/1833183905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e400b5170 msgr2=0x7f4e400b5550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:54.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.610+0000 7f4e46e00640 1 --2- 172.21.15.7:0/1833183905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e400b5170 0x7f4e400b5550 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4e30009830 tx=0x7f4e3002a540 comp rx=0 tx=0).stop 2024-09-10T21:52:54.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.610+0000 7f4e46e00640 1 -- 172.21.15.7:0/1833183905 shutdown_connections 2024-09-10T21:52:54.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.610+0000 7f4e46e00640 1 --2- 172.21.15.7:0/1833183905 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e400b5170 0x7f4e400b5550 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.610+0000 7f4e46e00640 1 -- 172.21.15.7:0/1833183905 >> 172.21.15.7:0/1833183905 conn(0x7f4e400b1b60 msgr2=0x7f4e400b1f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:54.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.610+0000 7f4e46e00640 1 -- 172.21.15.7:0/1833183905 shutdown_connections 2024-09-10T21:52:54.611 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.610+0000 7f4e46e00640 1 -- 172.21.15.7:0/1833183905 wait complete. 2024-09-10T21:52:54.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.611+0000 7f4e46e00640 1 Processor -- start 2024-09-10T21:52:54.612 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.611+0000 7f4e46e00640 1 -- start start 2024-09-10T21:52:54.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.612+0000 7f4e46e00640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e400b5170 0x7f4e4013ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:54.613 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.612+0000 7f4e46e00640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e40142260 0x7f4e4015ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:54.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.612+0000 7f4e46e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e4015cb70 0x7f4e4015f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:54.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.612+0000 7f4e46e00640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f4e400cee60 con 0x7f4e4015cb70 2024-09-10T21:52:54.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.612+0000 7f4e46e00640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f4e400cece0 con 0x7f4e40142260 2024-09-10T21:52:54.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.612+0000 7f4e46e00640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f4e400cefe0 con 0x7f4e400b5170 2024-09-10T21:52:54.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.613+0000 7f4e44e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e4015cb70 0x7f4e4015f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:54.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.613+0000 7f4e44e00640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e4015cb70 0x7f4e4015f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39214/0 (socket says 172.21.15.7:39214) 2024-09-10T21:52:54.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.613+0000 7f4e44e00640 1 -- 172.21.15.7:0/4060970894 learned_addr learned my addr 172.21.15.7:0/4060970894 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:54.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.613+0000 7f4e37400640 1 --2- 172.21.15.7:0/4060970894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e40142260 0x7f4e4015ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:54.614 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.613+0000 7f4e3fe00640 1 --2- 172.21.15.7:0/4060970894 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e400b5170 0x7f4e4013ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:54.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.613+0000 7f4e44e00640 1 -- 172.21.15.7:0/4060970894 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e400b5170 msgr2=0x7f4e4013ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:54.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.613+0000 7f4e44e00640 1 --2- 172.21.15.7:0/4060970894 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e400b5170 0x7f4e4013ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.613+0000 7f4e44e00640 1 -- 172.21.15.7:0/4060970894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e40142260 msgr2=0x7f4e4015ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:54.615 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.613+0000 7f4e44e00640 1 --2- 172.21.15.7:0/4060970894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e40142260 0x7f4e4015ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.613+0000 7f4e44e00640 1 -- 172.21.15.7:0/4060970894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e401f66a0 con 0x7f4e4015cb70 2024-09-10T21:52:54.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.613+0000 7f4e44e00640 1 --2- 172.21.15.7:0/4060970894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e4015cb70 0x7f4e4015f260 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800cbd0 tx=0x7f4e3800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:54.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.614+0000 7f4e3cc00640 1 -- 172.21.15.7:0/4060970894 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e38016020 con 0x7f4e4015cb70 2024-09-10T21:52:54.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.614+0000 7f4e3cc00640 1 -- 172.21.15.7:0/4060970894 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e3800ae50 con 0x7f4e4015cb70 2024-09-10T21:52:54.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.614+0000 7f4e3cc00640 1 -- 172.21.15.7:0/4060970894 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e38012680 con 0x7f4e4015cb70 2024-09-10T21:52:54.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.614+0000 7f4e46e00640 1 -- 172.21.15.7:0/4060970894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e401f6980 con 0x7f4e4015cb70 2024-09-10T21:52:54.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.614+0000 7f4e46e00640 1 -- 172.21.15.7:0/4060970894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e401f6f40 con 0x7f4e4015cb70 2024-09-10T21:52:54.616 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.615+0000 7f4e46e00640 1 -- 172.21.15.7:0/4060970894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e401f7140 con 0x7f4e4015cb70 2024-09-10T21:52:54.617 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.616+0000 7f4e3cc00640 1 -- 172.21.15.7:0/4060970894 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f4e38012a30 con 0x7f4e4015cb70 2024-09-10T21:52:54.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.617+0000 7f4e3cc00640 1 --2- 172.21.15.7:0/4060970894 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e1c077f70 0x7f4e1c07a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:54.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.617+0000 7f4e3cc00640 1 -- 172.21.15.7:0/4060970894 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7f4e3809a6a0 con 0x7f4e4015cb70 2024-09-10T21:52:54.618 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.618+0000 7f4e3fe00640 1 --2- 172.21.15.7:0/4060970894 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e1c077f70 0x7f4e1c07a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:54.619 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.618+0000 7f4e3fe00640 1 --2- 172.21.15.7:0/4060970894 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e1c077f70 0x7f4e1c07a430 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4e30034000 tx=0x7f4e30004730 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:54.622 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.621+0000 7f4e3cc00640 1 -- 172.21.15.7:0/4060970894 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f4e401f7140 con 0x7f4e4015cb70 2024-09-10T21:52:54.815 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.814+0000 7f4e46e00640 1 -- 172.21.15.7:0/4060970894 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f4e400cf470 con 0x7f4e4015cb70 2024-09-10T21:52:54.819 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.818+0000 7f4e3cc00640 1 -- 172.21.15.7:0/4060970894 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v42)=0 v42) ==== 123+0+0 (secure 0 0 0) 0x7f4e38020020 con 0x7f4e4015cb70 2024-09-10T21:52:54.820 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.819+0000 7f4e3cc00640 1 -- 172.21.15.7:0/4060970894 <== mon.0 v2:172.21.15.7:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f4e38062820 con 0x7f4e4015cb70 2024-09-10T21:52:54.822 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.821+0000 7f4e46e00640 1 -- 172.21.15.7:0/4060970894 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e1c077f70 msgr2=0x7f4e1c07a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:54.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.821+0000 7f4e46e00640 1 --2- 172.21.15.7:0/4060970894 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e1c077f70 0x7f4e1c07a430 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f4e30034000 tx=0x7f4e30004730 comp rx=0 tx=0).stop 2024-09-10T21:52:54.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.821+0000 7f4e46e00640 1 -- 172.21.15.7:0/4060970894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e4015cb70 msgr2=0x7f4e4015f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:54.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.821+0000 7f4e46e00640 1 --2- 172.21.15.7:0/4060970894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e4015cb70 0x7f4e4015f260 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800cbd0 tx=0x7f4e3800a430 comp rx=0 tx=0).stop 2024-09-10T21:52:54.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.821+0000 7f4e46e00640 1 -- 172.21.15.7:0/4060970894 shutdown_connections 2024-09-10T21:52:54.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.821+0000 7f4e46e00640 1 --2- 172.21.15.7:0/4060970894 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f4e1c077f70 0x7f4e1c07a430 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.821+0000 7f4e46e00640 1 --2- 172.21.15.7:0/4060970894 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f4e40142260 0x7f4e4015ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.821+0000 7f4e46e00640 1 --2- 172.21.15.7:0/4060970894 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f4e4015cb70 0x7f4e4015f260 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.821+0000 7f4e46e00640 1 --2- 172.21.15.7:0/4060970894 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f4e400b5170 0x7f4e4013ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:54.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.821+0000 7f4e46e00640 1 -- 172.21.15.7:0/4060970894 >> 172.21.15.7:0/4060970894 conn(0x7f4e400b1b60 msgr2=0x7f4e400cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:54.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.821+0000 7f4e46e00640 1 -- 172.21.15.7:0/4060970894 shutdown_connections 2024-09-10T21:52:54.823 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:54.821+0000 7f4e46e00640 1 -- 172.21.15.7:0/4060970894 wait complete. 2024-09-10T21:52:54.838 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:52:54.839 DEBUG:teuthology.orchestra.run.smithi007:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:52:55.001 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.000+0000 7f46dc200640 1 -- 172.21.15.7:0/3831883258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46bc003a00 msgr2=0x7f46bc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:55.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.000+0000 7f46dc200640 1 --2- 172.21.15.7:0/3831883258 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46bc003a00 0x7f46bc003e20 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f46c8009b50 tx=0x7f46bc005830 comp rx=0 tx=0).stop 2024-09-10T21:52:55.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.000+0000 7f46da200640 1 -- 172.21.15.7:0/3831883258 reap_dead start 2024-09-10T21:52:55.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.000+0000 7f46dc200640 1 -- 172.21.15.7:0/3831883258 shutdown_connections 2024-09-10T21:52:55.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.000+0000 7f46dc200640 1 -- 172.21.15.7:0/3831883258 >> 172.21.15.7:0/3831883258 conn(0x7f46d406ca60 msgr2=0x7f46d406ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:55.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.000+0000 7f46dc200640 1 -- 172.21.15.7:0/3831883258 shutdown_connections 2024-09-10T21:52:55.002 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.000+0000 7f46dc200640 1 -- 172.21.15.7:0/3831883258 wait complete. 2024-09-10T21:52:55.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.002+0000 7f46dc200640 1 Processor -- start 2024-09-10T21:52:55.003 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.002+0000 7f46dc200640 1 -- start start 2024-09-10T21:52:55.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.003+0000 7f46dc200640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46bc003a00 0x7f46d41509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:55.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.003+0000 7f46dc200640 1 --2- >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46d40afc20 0x7f46d40b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:55.005 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.003+0000 7f46dc200640 1 --2- >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46d41522c0 0x7f46d4152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.003+0000 7f46dc200640 1 -- --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_getmap magic: 0 -- 0x7f46d40732c0 con 0x7f46bc003a00 2024-09-10T21:52:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.003+0000 7f46dc200640 1 -- --> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] -- mon_getmap magic: 0 -- 0x7f46d4073140 con 0x7f46d40afc20 2024-09-10T21:52:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.003+0000 7f46dc200640 1 -- --> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] -- mon_getmap magic: 0 -- 0x7f46d4073440 con 0x7f46d41522c0 2024-09-10T21:52:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.003+0000 7f46d9800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46bc003a00 0x7f46d41509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.003+0000 7f46d9800640 1 --2- >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46bc003a00 0x7f46d41509f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.7:3300/0 says I am v2:172.21.15.7:39224/0 (socket says 172.21.15.7:39224) 2024-09-10T21:52:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.003+0000 7f46d9800640 1 -- 172.21.15.7:0/2665481836 learned_addr learned my addr 172.21.15.7:0/2665481836 (peer_addr_for_me v2:172.21.15.7:0/0) 2024-09-10T21:52:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.004+0000 7f46da200640 1 --2- 172.21.15.7:0/2665481836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46d41522c0 0x7f46d4152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.004+0000 7f46d8e00640 1 --2- 172.21.15.7:0/2665481836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46d40afc20 0x7f46d40b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.004+0000 7f46d9800640 1 -- 172.21.15.7:0/2665481836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46d41522c0 msgr2=0x7f46d4152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.004+0000 7f46d9800640 1 --2- 172.21.15.7:0/2665481836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46d41522c0 0x7f46d4152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.004+0000 7f46d9800640 1 -- 172.21.15.7:0/2665481836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46d40afc20 msgr2=0x7f46d40b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.004+0000 7f46d9800640 1 --2- 172.21.15.7:0/2665481836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46d40afc20 0x7f46d40b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.004+0000 7f46d9800640 1 -- 172.21.15.7:0/2665481836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d40b2ad0 con 0x7f46bc003a00 2024-09-10T21:52:55.006 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.004+0000 7f46d8e00640 1 --2- 172.21.15.7:0/2665481836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46d40afc20 0x7f46d40b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:52:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.004+0000 7f46d9800640 1 --2- 172.21.15.7:0/2665481836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46bc003a00 0x7f46d41509f0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f46c8036040 tx=0x7f46c80336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.004+0000 7f46d1600640 1 -- 172.21.15.7:0/2665481836 <== mon.0 v2:172.21.15.7:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8038040 con 0x7f46bc003a00 2024-09-10T21:52:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.004+0000 7f46d1600640 1 -- 172.21.15.7:0/2665481836 <== mon.0 v2:172.21.15.7:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f46c8004020 con 0x7f46bc003a00 2024-09-10T21:52:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.004+0000 7f46d1600640 1 -- 172.21.15.7:0/2665481836 <== mon.0 v2:172.21.15.7:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c8004470 con 0x7f46bc003a00 2024-09-10T21:52:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.004+0000 7f46dc200640 1 -- 172.21.15.7:0/2665481836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d4151820 con 0x7f46bc003a00 2024-09-10T21:52:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.005+0000 7f46dc200640 1 -- 172.21.15.7:0/2665481836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d4151ac0 con 0x7f46bc003a00 2024-09-10T21:52:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.006+0000 7f46dc200640 1 -- 172.21.15.7:0/2665481836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d415d090 con 0x7f46bc003a00 2024-09-10T21:52:55.007 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.007+0000 7f46d1600640 1 -- 172.21.15.7:0/2665481836 <== mon.0 v2:172.21.15.7:3300/0 4 ==== mgrmap(e 5) ==== 100601+0+0 (secure 0 0 0) 0x7f46c803b030 con 0x7f46bc003a00 2024-09-10T21:52:55.008 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.008+0000 7f46d1600640 1 --2- 172.21.15.7:0/2665481836 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f46b4077fa0 0x7f46b407a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:52:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.008+0000 7f46d1600640 1 -- 172.21.15.7:0/2665481836 <== mon.0 v2:172.21.15.7:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5115+0+0 (secure 0 0 0) 0x7f46c80ba2d0 con 0x7f46bc003a00 2024-09-10T21:52:55.009 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.008+0000 7f46d8e00640 1 --2- 172.21.15.7:0/2665481836 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f46b4077fa0 0x7f46b407a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:52:55.010 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.009+0000 7f46d8e00640 1 --2- 172.21.15.7:0/2665481836 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f46b4077fa0 0x7f46b407a460 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f46c0001eb0 tx=0x7f46c0000ed0 comp rx=0 tx=0).ready entity=mgr.4123 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:52:55.013 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.012+0000 7f46d1600640 1 -- 172.21.15.7:0/2665481836 <== mon.0 v2:172.21.15.7:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+196006 (secure 0 0 0) 0x7f46c804c020 con 0x7f46bc003a00 2024-09-10T21:52:55.242 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.242+0000 7f46dc200640 1 -- 172.21.15.7:0/2665481836 --> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46d40cfb30 con 0x7f46bc003a00 2024-09-10T21:52:55.243 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.242+0000 7f46d1600640 1 -- 172.21.15.7:0/2665481836 <== mon.0 v2:172.21.15.7:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 117 v117) ==== 95+0+3911 (secure 0 0 0) 0x7f46c8083010 con 0x7f46bc003a00 2024-09-10T21:52:55.243 INFO:teuthology.orchestra.run.smithi007.stdout: 2024-09-10T21:52:55.243 INFO:teuthology.orchestra.run.smithi007.stdout:{"epoch":117,"btime":"2024-09-10T21:52:21:242221+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7465,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6834/3012572644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6832","nonce":3012572644},{"type":"v1","addr":"172.21.15.7:6834","nonce":3012572644}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":96},{"gid":7660,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6837/2492331385","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6834","nonce":2492331385},{"type":"v1","addr":"172.21.15.133:6837","nonce":2492331385}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":7918,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.133:6836/2715741183","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.133:6835","nonce":2715741183},{"type":"v1","addr":"172.21.15.133:6836","nonce":2715741183}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":117},{"gid":8094,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.7:6835/148356862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.7:6833","nonce":148356862},{"type":"v1","addr":"172.21.15.7:6835","nonce":148356862}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":97}],"filesystems":[]} 2024-09-10T21:52:55.243 INFO:teuthology.orchestra.run.smithi007.stderr:dumped fsmap epoch 117 2024-09-10T21:52:55.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.244+0000 7f46dc200640 1 -- 172.21.15.7:0/2665481836 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f46b4077fa0 msgr2=0x7f46b407a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:55.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.244+0000 7f46dc200640 1 --2- 172.21.15.7:0/2665481836 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f46b4077fa0 0x7f46b407a460 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f46c0001eb0 tx=0x7f46c0000ed0 comp rx=0 tx=0).stop 2024-09-10T21:52:55.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.244+0000 7f46dc200640 1 -- 172.21.15.7:0/2665481836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46bc003a00 msgr2=0x7f46d41509f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:52:55.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.244+0000 7f46dc200640 1 --2- 172.21.15.7:0/2665481836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46bc003a00 0x7f46d41509f0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f46c8036040 tx=0x7f46c80336f0 comp rx=0 tx=0).stop 2024-09-10T21:52:55.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.245+0000 7f46dc200640 1 -- 172.21.15.7:0/2665481836 shutdown_connections 2024-09-10T21:52:55.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.245+0000 7f46dc200640 1 --2- 172.21.15.7:0/2665481836 >> [v2:172.21.15.133:6800/4032855124,v1:172.21.15.133:6801/4032855124] conn(0x7f46b4077fa0 0x7f46b407a460 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:55.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.245+0000 7f46dc200640 1 --2- 172.21.15.7:0/2665481836 >> [v2:172.21.15.133:3300/0,v1:172.21.15.133:6789/0] conn(0x7f46d40afc20 0x7f46d40b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:55.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.245+0000 7f46dc200640 1 --2- 172.21.15.7:0/2665481836 >> [v2:172.21.15.133:3301/0,v1:172.21.15.133:6790/0] conn(0x7f46d41522c0 0x7f46d4152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:55.246 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.245+0000 7f46dc200640 1 --2- 172.21.15.7:0/2665481836 >> [v2:172.21.15.7:3300/0,v1:172.21.15.7:6789/0] conn(0x7f46bc003a00 0x7f46d41509f0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:52:55.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.245+0000 7f46dc200640 1 -- 172.21.15.7:0/2665481836 >> 172.21.15.7:0/2665481836 conn(0x7f46d406ca60 msgr2=0x7f46d40ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:52:55.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.245+0000 7f46dc200640 1 -- 172.21.15.7:0/2665481836 shutdown_connections 2024-09-10T21:52:55.247 INFO:teuthology.orchestra.run.smithi007.stderr:2024-09-10T21:52:55.245+0000 7f46dc200640 1 -- 172.21.15.7:0/2665481836 wait complete. 2024-09-10T21:52:55.261 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-10T21:52:55.262 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-10T21:52:55.262 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:52:55.262 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:52:55.275 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:52:55.276 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:52:55.276 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:52:55.330 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:52:55.330 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:52:55.330 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:52:55.346 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:52:55.347 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:52:55.347 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:52:55.403 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:52:55.403 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-10T21:52:55.403 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-10T21:52:55.403 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:52:55.448 INFO:tasks.ceph.osd.0:Stopped 2024-09-10T21:52:55.448 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-10T21:52:55.449 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:52:55.493 INFO:tasks.ceph.osd.1:Stopped 2024-09-10T21:52:55.494 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-10T21:52:55.494 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:52:55.552 INFO:tasks.ceph.osd.2:Stopped 2024-09-10T21:52:55.552 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-10T21:52:55.552 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:52:55.603 INFO:tasks.ceph.osd.3:Stopped 2024-09-10T21:52:55.603 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-10T21:52:55.603 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:52:55.665 INFO:tasks.ceph.osd.4:Stopped 2024-09-10T21:52:55.665 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-10T21:52:55.665 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:52:55.723 INFO:tasks.ceph.osd.5:Stopped 2024-09-10T21:52:55.723 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-10T21:52:55.724 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:52:55.791 INFO:tasks.ceph.osd.6:Stopped 2024-09-10T21:52:55.791 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-10T21:52:55.792 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:52:55.841 INFO:tasks.ceph.osd.7:Stopped 2024-09-10T21:52:55.841 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-10T21:52:55.841 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-10T21:52:55.842 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:52:55.908 INFO:tasks.ceph.mgr.y:Stopped 2024-09-10T21:52:55.908 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-10T21:52:55.909 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:52:55.978 INFO:tasks.ceph.mgr.x:Stopped 2024-09-10T21:52:55.978 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-10T21:52:55.979 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T21:52:55.979 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:52:56.000 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T21:52:56.000 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-10T21:52:56.000 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:52:56.018 INFO:tasks.ceph.mon.b:Stopped 2024-09-10T21:52:56.018 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T21:52:56.018 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:52:56.077 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T21:52:56.077 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-10T21:52:56.078 DEBUG:teuthology.orchestra.run.smithi007:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | egrep -v 'object missing on disk' | egrep -v 'Invalid tag char' | head -n 1 2024-09-10T21:52:56.165 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:52:56.165 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-10T21:52:56.405 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:52:56.406 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-10T21:52:56.534 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:52:56.534 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-10T21:52:56.642 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:52:56.642 DEBUG:teuthology.orchestra.run.smithi007:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-10T21:52:56.777 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi133.front.sepia.ceph.com 2024-09-10T21:52:56.777 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-10T21:52:57.038 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi133.front.sepia.ceph.com 2024-09-10T21:52:57.038 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-10T21:52:57.196 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi133.front.sepia.ceph.com 2024-09-10T21:52:57.197 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-10T21:52:57.332 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi133.front.sepia.ceph.com 2024-09-10T21:52:57.332 DEBUG:teuthology.orchestra.run.smithi133:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-10T21:52:57.447 INFO:tasks.ceph:Archiving mon data... 2024-09-10T21:52:57.449 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899691/data/mon.a.tgz 2024-09-10T21:52:57.449 DEBUG:teuthology.orchestra.run.smithi007:> mktemp 2024-09-10T21:52:57.485 INFO:teuthology.orchestra.run.smithi007.stdout:/tmp/tmp.1jTpPfJhk3 2024-09-10T21:52:57.486 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.1jTpPfJhk3 2024-09-10T21:52:57.672 DEBUG:teuthology.orchestra.run.smithi007:> sudo chmod 0666 /tmp/tmp.1jTpPfJhk3 2024-09-10T21:52:57.767 DEBUG:teuthology.orchestra.remote:smithi007:/tmp/tmp.1jTpPfJhk3 is 394KB 2024-09-10T21:52:57.857 DEBUG:teuthology.orchestra.run.smithi007:> rm -fr /tmp/tmp.1jTpPfJhk3 2024-09-10T21:52:57.892 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899691/data/mon.b.tgz 2024-09-10T21:52:57.892 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-09-10T21:52:57.927 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.xQRzTn0ayt 2024-09-10T21:52:57.927 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.xQRzTn0ayt 2024-09-10T21:52:58.117 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.xQRzTn0ayt 2024-09-10T21:52:58.211 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.xQRzTn0ayt is 398KB 2024-09-10T21:52:58.298 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.xQRzTn0ayt 2024-09-10T21:52:58.333 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899691/data/mon.c.tgz 2024-09-10T21:52:58.333 DEBUG:teuthology.orchestra.run.smithi133:> mktemp 2024-09-10T21:52:58.408 INFO:teuthology.orchestra.run.smithi133.stdout:/tmp/tmp.G94tIKyhUe 2024-09-10T21:52:58.408 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.G94tIKyhUe 2024-09-10T21:52:58.585 DEBUG:teuthology.orchestra.run.smithi133:> sudo chmod 0666 /tmp/tmp.G94tIKyhUe 2024-09-10T21:52:58.677 DEBUG:teuthology.orchestra.remote:smithi133:/tmp/tmp.G94tIKyhUe is 399KB 2024-09-10T21:52:58.766 DEBUG:teuthology.orchestra.run.smithi133:> rm -fr /tmp/tmp.G94tIKyhUe 2024-09-10T21:52:58.801 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-10T21:52:58.802 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T21:52:58.806 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T21:52:58.934 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T21:52:58.969 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T21:52:58.969 INFO:tasks.ceph:Archiving crash dumps... 2024-09-10T21:52:58.971 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/lib/ceph/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899691/remote/smithi007/crash 2024-09-10T21:52:58.972 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T21:52:59.030 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/lib/ceph/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899691/remote/smithi133/crash 2024-09-10T21:52:59.031 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T21:52:59.084 INFO:tasks.ceph:Shutting down logrotate 2024-09-10T21:52:59.085 DEBUG:teuthology.orchestra.run.smithi007:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:52:59.089 DEBUG:teuthology.orchestra.run.smithi133:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:52:59.188 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T21:52:59.243 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T21:52:59.298 INFO:tasks.ceph:Compressing logs... 2024-09-10T21:52:59.298 DEBUG:teuthology.orchestra.run.smithi007:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T21:52:59.303 DEBUG:teuthology.orchestra.run.smithi133:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T21:52:59.353 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-10T21:52:59.353 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57012.log 2024-09-10T21:52:59.354 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45357.log 2024-09-10T21:52:59.355 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51082.log 2024-09-10T21:52:59.355 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57012.log.gz 2024-09-10T21:52:59.356 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35683.log 2024-09-10T21:52:59.357 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47523.log 2024-09-10T21:52:59.357 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45357.log.gz 2024-09-10T21:52:59.357 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57931.log 2024-09-10T21:52:59.357 INFO:teuthology.orchestra.run.smithi007.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.51082.log.gz 2024-09-10T21:52:59.359 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44901.log 2024-09-10T21:52:59.359 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47523.log.gz 2024-09-10T21:52:59.359 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57931.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57931.log.gz 2024-09-10T21:52:59.360 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66299.log 2024-09-10T21:52:59.361 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-10T21:52:59.361 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35963.log 2024-09-10T21:52:59.361 INFO:teuthology.orchestra.run.smithi133.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.35683.log.gz 2024-09-10T21:52:59.361 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41157.log 2024-09-10T21:52:59.361 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35803.log 2024-09-10T21:52:59.362 INFO:teuthology.orchestra.run.smithi133.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.35963.log.gz 2024-09-10T21:52:59.362 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.1.41157.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41157.log.gz 2024-09-10T21:52:59.362 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-10T21:52:59.362 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35883.log 2024-09-10T21:52:59.362 INFO:teuthology.orchestra.run.smithi133.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.35803.log.gz 2024-09-10T21:52:59.364 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-10T21:52:59.364 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35883.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T21:52:59.364 INFO:teuthology.orchestra.run.smithi133.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35883.log.gz 2024-09-10T21:52:59.364 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41572.log 2024-09-10T21:52:59.367 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44901.log: gzip -5 --verbose 87.3% -- -- replaced with /var/log/ceph/ceph-client.admin.44901.log.gz /var/log/ceph/ceph-client.admin.43999.log 2024-09-10T21:52:59.367 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-10T21:52:59.367 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66843.log 2024-09-10T21:52:59.367 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66299.log.gz 2024-09-10T21:52:59.367 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36019.log 2024-09-10T21:52:59.367 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43999.log.gz 2024-09-10T21:52:59.367 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61380.log 2024-09-10T21:52:59.367 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66843.log.gz 2024-09-10T21:52:59.367 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36019.log: 77.2% -- replaced with /var/log/ceph/ceph-client.admin.36019.log.gzgzip 2024-09-10T21:52:59.368 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.57114.log 2024-09-10T21:52:59.368 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61569.log 2024-09-10T21:52:59.368 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61380.log.gz 2024-09-10T21:52:59.368 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42245.log 2024-09-10T21:52:59.368 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57114.log.gz 2024-09-10T21:52:59.368 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40758.log 2024-09-10T21:52:59.368 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61569.log.gz 2024-09-10T21:52:59.368 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42245.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42245.log.gz 2024-09-10T21:52:59.368 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53188.log 2024-09-10T21:52:59.372 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-10T21:52:59.372 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.3.41572.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41572.log.gz 2024-09-10T21:52:59.372 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35923.log 2024-09-10T21:52:59.372 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-10T21:52:59.372 INFO:teuthology.orchestra.run.smithi133.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T21:52:59.372 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35923.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-10T21:52:59.373 INFO:teuthology.orchestra.run.smithi133.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.35923.log.gz 2024-09-10T21:52:59.373 INFO:teuthology.orchestra.run.smithi133.stderr:gzip/var/log/ceph/ceph-osd.6.log: -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T21:52:59.373 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-10T21:52:59.375 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67221.log 2024-09-10T21:52:59.375 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40758.log.gz 2024-09-10T21:52:59.375 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70863.log 2024-09-10T21:52:59.376 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53188.log.gz 2024-09-10T21:52:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57752.log 2024-09-10T21:52:59.376 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67221.log.gz 2024-09-10T21:52:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70863.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.70863.log.gz 2024-09-10T21:52:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57283.log 2024-09-10T21:52:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66144.log 2024-09-10T21:52:59.376 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57752.log.gz 2024-09-10T21:52:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53104.log 2024-09-10T21:52:59.376 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57283.log.gz 2024-09-10T21:52:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63968.log 2024-09-10T21:52:59.376 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66144.log.gz 2024-09-10T21:52:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53104.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53104.log.gz 2024-09-10T21:52:59.376 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51420.log 2024-09-10T21:52:59.377 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.63968.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63968.log.gz 2024-09-10T21:52:59.377 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57367.log 2024-09-10T21:52:59.377 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.51420.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.51420.log.gz 2024-09-10T21:52:59.377 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54812.log 2024-09-10T21:52:59.378 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-10T21:52:59.378 INFO:teuthology.orchestra.run.smithi133.stderr: 92.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T21:52:59.381 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58883.log 2024-09-10T21:52:59.381 INFO:teuthology.orchestra.run.smithi007.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.57367.log.gz 2024-09-10T21:52:59.381 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54980.log 2024-09-10T21:52:59.381 INFO:teuthology.orchestra.run.smithi007.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54812.log.gz 2024-09-10T21:52:59.381 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70779.log 2024-09-10T21:52:59.381 INFO:teuthology.orchestra.run.smithi007.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.58883.log.gz 2024-09-10T21:52:59.381 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51918.log 2024-09-10T21:52:59.381 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.54980.log.gz 2024-09-10T21:52:59.381 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56844.log 2024-09-10T21:52:59.381 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.70779.log.gz 2024-09-10T21:52:59.381 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69342.log 2024-09-10T21:52:59.381 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51918.log.gz 2024-09-10T21:52:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35723.log 2024-09-10T21:52:59.383 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.admin.35723.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-10T21:52:59.383 INFO:teuthology.orchestra.run.smithi133.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.35723.log.gz 2024-09-10T21:52:59.384 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56844.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56844.log.gz 2024-09-10T21:52:59.384 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47355.log 2024-09-10T21:52:59.384 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58337.log 2024-09-10T21:52:59.384 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69342.log.gz 2024-09-10T21:52:59.384 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69282.log 2024-09-10T21:52:59.384 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47355.log.gz 2024-09-10T21:52:59.384 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41182.log 2024-09-10T21:52:59.384 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58337.log.gz 2024-09-10T21:52:59.385 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35818.log 2024-09-10T21:52:59.385 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69282.log.gz 2024-09-10T21:52:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41182.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.41182.log.gz 2024-09-10T21:52:59.386 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61779.log 2024-09-10T21:52:59.387 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35818.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35818.log.gz 2024-09-10T21:52:59.387 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45529.log 2024-09-10T21:52:59.387 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48624.log 2024-09-10T21:52:59.388 INFO:teuthology.orchestra.run.smithi133.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35763.log 2024-09-10T21:52:59.389 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61779.log.gz 2024-09-10T21:52:59.389 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69894.log 2024-09-10T21:52:59.389 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45529.log.gz 2024-09-10T21:52:59.389 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57199.log 2024-09-10T21:52:59.389 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48624.log.gz 2024-09-10T21:52:59.390 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69894.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69894.log.gz 2024-09-10T21:52:59.390 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58061.log 2024-09-10T21:52:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65808.log 2024-09-10T21:52:59.392 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57199.log.gz 2024-09-10T21:52:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69205.log 2024-09-10T21:52:59.392 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58061.log.gz 2024-09-10T21:52:59.392 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41905.log 2024-09-10T21:52:59.394 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35843.log 2024-09-10T21:52:59.394 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35763.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.35763.log.gz 2024-09-10T21:52:59.394 INFO:teuthology.orchestra.run.smithi133.stderr:/var/log/ceph/ceph-client.admin.35843.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.35843.log.gz 2024-09-10T21:52:59.395 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.65808.log.gz 2024-09-10T21:52:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69978.log 2024-09-10T21:52:59.395 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69205.log.gz 2024-09-10T21:52:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48264.log 2024-09-10T21:52:59.395 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41905.log.gz 2024-09-10T21:52:59.395 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42239.log 2024-09-10T21:52:59.395 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.69978.log.gz 2024-09-10T21:52:59.396 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45925.log 2024-09-10T21:52:59.396 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48264.log.gz 2024-09-10T21:52:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42239.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42239.log.gz 2024-09-10T21:52:59.397 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57462.log 2024-09-10T21:52:59.398 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53020.log 2024-09-10T21:52:59.398 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45925.log.gz 2024-09-10T21:52:59.398 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45841.log 2024-09-10T21:52:59.398 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57462.log.gz 2024-09-10T21:52:59.399 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40439.log 2024-09-10T21:52:59.400 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53020.log.gz 2024-09-10T21:52:59.400 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45841.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45841.log.gz 2024-09-10T21:52:59.400 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64994.log 2024-09-10T21:52:59.401 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65120.log 2024-09-10T21:52:59.401 INFO:teuthology.orchestra.run.smithi007.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.40439.log.gz 2024-09-10T21:52:59.402 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46794.log 2024-09-10T21:52:59.402 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.64994.log.gz 2024-09-10T21:52:59.402 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65120.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-10T21:52:59.403 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46794.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.65120.log.gz 2024-09-10T21:52:59.404 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61695.log 2024-09-10T21:52:59.404 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46794.log.gz 2024-09-10T21:52:59.404 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62498.log 2024-09-10T21:52:59.405 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41224.log 2024-09-10T21:52:59.405 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61695.log.gz 2024-09-10T21:52:59.406 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52392.log 2024-09-10T21:52:59.406 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62498.log.gz 2024-09-10T21:52:59.407 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41224.log: gzip 87.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.41224.log.gz --verbose 2024-09-10T21:52:59.407 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.55731.log 2024-09-10T21:52:59.408 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40800.log 2024-09-10T21:52:59.409 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52392.log.gz 2024-09-10T21:52:59.409 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61989.log 2024-09-10T21:52:59.409 INFO:teuthology.orchestra.run.smithi007.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.55731.log.gz 2024-09-10T21:52:59.410 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41573.log 2024-09-10T21:52:59.410 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40800.log.gz 2024-09-10T21:52:59.411 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61989.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.61989.log.gz 2024-09-10T21:52:59.411 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47481.log 2024-09-10T21:52:59.412 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50543.log 2024-09-10T21:52:59.413 INFO:teuthology.orchestra.run.smithi007.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.41573.log.gz 2024-09-10T21:52:59.413 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47481.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47481.log.gz 2024-09-10T21:52:59.414 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65766.log 2024-09-10T21:52:59.415 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41570.log 2024-09-10T21:52:59.415 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50543.log.gz 2024-09-10T21:52:59.416 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65766.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.65766.log.gz 2024-09-10T21:52:59.416 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50244.log 2024-09-10T21:52:59.417 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60788.log 2024-09-10T21:52:59.417 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41570.log.gz 2024-09-10T21:52:59.418 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50244.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50244.log.gz 2024-09-10T21:52:59.418 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47649.log 2024-09-10T21:52:59.419 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35910.log 2024-09-10T21:52:59.419 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60788.log.gz 2024-09-10T21:52:59.420 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47649.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47649.log.gz 2024-09-10T21:52:59.420 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71307.log 2024-09-10T21:52:59.421 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35910.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35910.log.gz 2024-09-10T21:52:59.421 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69609.log 2024-09-10T21:52:59.422 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71307.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.71307.log.gz 2024-09-10T21:52:59.422 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56676.log 2024-09-10T21:52:59.422 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49614.log 2024-09-10T21:52:59.424 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56676.log: 92.0% -- replaced with /var/log/ceph/ceph-client.admin.69609.log.gz 2024-09-10T21:52:59.424 INFO:teuthology.orchestra.run.smithi007.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.56676.log.gzgzip 2024-09-10T21:52:59.424 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.71559.log 2024-09-10T21:52:59.425 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41056.log 2024-09-10T21:52:59.425 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49614.log.gz 2024-09-10T21:52:59.426 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61821.log 2024-09-10T21:52:59.426 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71559.log.gz 2024-09-10T21:52:59.427 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41056.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.41056.log.gz 2024-09-10T21:52:59.427 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70517.log 2024-09-10T21:52:59.428 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41308.log 2024-09-10T21:52:59.428 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61821.log.gz 2024-09-10T21:52:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70517.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.70517.log.gz 2024-09-10T21:52:59.429 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40634.log 2024-09-10T21:52:59.430 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43222.log 2024-09-10T21:52:59.430 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41308.log.gz 2024-09-10T21:52:59.431 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40634.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40634.log.gz 2024-09-10T21:52:59.431 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62540.log 2024-09-10T21:52:59.432 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.43222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62416.log 2024-09-10T21:52:59.432 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43222.log.gz 2024-09-10T21:52:59.433 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62540.log: gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.62540.log.gz --verbose 2024-09-10T21:52:59.433 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.62996.log 2024-09-10T21:52:59.434 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48812.log 2024-09-10T21:52:59.434 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62416.log.gz 2024-09-10T21:52:59.435 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62996.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62996.log.gz 2024-09-10T21:52:59.435 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45441.log 2024-09-10T21:52:59.436 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.48812.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.51768.log 2024-09-10T21:52:59.436 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.0.48812.log.gz 2024-09-10T21:52:59.437 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45441.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45441.log.gz 2024-09-10T21:52:59.437 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65036.log 2024-09-10T21:52:59.438 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56886.log 2024-09-10T21:52:59.438 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65036.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51768.log.gz 2024-09-10T21:52:59.439 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70694.log 2024-09-10T21:52:59.439 INFO:teuthology.orchestra.run.smithi007.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.65036.log.gz 2024-09-10T21:52:59.440 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60125.log 2024-09-10T21:52:59.440 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56886.log.gz 2024-09-10T21:52:59.441 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68958.log 2024-09-10T21:52:59.441 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70694.log.gz 2024-09-10T21:52:59.442 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60125.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60125.log.gz 2024-09-10T21:52:59.442 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66633.log 2024-09-10T21:52:59.443 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.68958.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.0.68958.log.gz /var/log/ceph/ceph-client.admin.35964.log 2024-09-10T21:52:59.443 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-10T21:52:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66633.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66633.log.gz 2024-09-10T21:52:59.444 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51960.log 2024-09-10T21:52:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35964.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35964.log.gz 2024-09-10T21:52:59.447 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60167.log 2024-09-10T21:52:59.448 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56101.log 2024-09-10T21:52:59.448 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51960.log.gz 2024-09-10T21:52:59.448 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51232.log 2024-09-10T21:52:59.448 INFO:teuthology.orchestra.run.smithi007.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60167.log.gz 2024-09-10T21:52:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49478.log 2024-09-10T21:52:59.451 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.56101.log.gz 2024-09-10T21:52:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51232.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51232.log.gz 2024-09-10T21:52:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48390.log 2024-09-10T21:52:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49478.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49478.log.gz 2024-09-10T21:52:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41896.log 2024-09-10T21:52:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48390.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.48390.log.gz 2024-09-10T21:52:59.451 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50872.log 2024-09-10T21:52:59.452 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44859.log 2024-09-10T21:52:59.452 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41896.log.gz 2024-09-10T21:52:59.453 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50872.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50872.log.gz 2024-09-10T21:52:59.453 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47145.log 2024-09-10T21:52:59.453 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44859.log: gzip -5 --verbose 87.3% -- -- replaced with /var/log/ceph/ceph-client.admin.44859.log.gz 2024-09-10T21:52:59.454 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.65598.log 2024-09-10T21:52:59.454 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47145.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47145.log.gz -5 2024-09-10T21:52:59.454 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47565.log 2024-09-10T21:52:59.455 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44041.log 2024-09-10T21:52:59.456 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65598.log.gz 2024-09-10T21:52:59.456 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47565.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47565.log.gz 2024-09-10T21:52:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58019.log 2024-09-10T21:52:59.457 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46578.log 2024-09-10T21:52:59.457 INFO:teuthology.orchestra.run.smithi007.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.44041.log.gz 2024-09-10T21:52:59.458 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58019.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.58019.log.gz 2024-09-10T21:52:59.458 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42240.log 2024-09-10T21:52:59.459 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46578.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46578.log.gz 2024-09-10T21:52:59.459 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48942.log 2024-09-10T21:52:59.460 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42240.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42240.log.gz 2024-09-10T21:52:59.460 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59048.log 2024-09-10T21:52:59.461 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49093.log 2024-09-10T21:52:59.461 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48942.log.gz 2024-09-10T21:52:59.462 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59048.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59048.log.gz 2024-09-10T21:52:59.462 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66675.log 2024-09-10T21:52:59.463 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49093.log: gzip 85.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.49093.log.gz 2024-09-10T21:52:59.463 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48474.log 2024-09-10T21:52:59.464 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66675.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66675.log.gz 2024-09-10T21:52:59.464 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55905.log 2024-09-10T21:52:59.465 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45189.log 2024-09-10T21:52:59.465 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48474.log.gz 2024-09-10T21:52:59.467 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68383.log 2024-09-10T21:52:59.467 INFO:teuthology.orchestra.run.smithi007.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55905.log.gz 2024-09-10T21:52:59.467 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45189.log: gzip -5 87.1% --verbose -- replaced with /var/log/ceph/ceph-client.admin.45189.log.gz -- 2024-09-10T21:52:59.467 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.49992.log 2024-09-10T21:52:59.469 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68383.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.68383.log.gz 2024-09-10T21:52:59.469 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58103.log 2024-09-10T21:52:59.469 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49992.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.49992.log.gz 2024-09-10T21:52:59.469 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.62073.log 2024-09-10T21:52:59.470 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58103.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.58103.log.gz 2024-09-10T21:52:59.470 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56167.log 2024-09-10T21:52:59.470 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53887.log 2024-09-10T21:52:59.471 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62073.log.gz 2024-09-10T21:52:59.472 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56167.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56167.log.gz 2024-09-10T21:52:59.472 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59849.log 2024-09-10T21:52:59.472 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53887.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53887.log.gz 2024-09-10T21:52:59.472 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62031.log 2024-09-10T21:52:59.473 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56718.log 2024-09-10T21:52:59.473 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59849.log.gz 2024-09-10T21:52:59.474 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53929.log 2024-09-10T21:52:59.474 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62031.log.gz 2024-09-10T21:52:59.475 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56718.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56718.log.gz 2024-09-10T21:52:59.475 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62763.log 2024-09-10T21:52:59.476 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55539.log 87.2% 2024-09-10T21:52:59.476 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.53929.log.gz 2024-09-10T21:52:59.476 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62763.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62763.log.gz 2024-09-10T21:52:59.477 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64754.log 2024-09-10T21:52:59.477 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65292.log 2024-09-10T21:52:59.478 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.55539.log.gz 2024-09-10T21:52:59.479 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35797.log 2024-09-10T21:52:59.479 INFO:teuthology.orchestra.run.smithi007.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.64754.log.gz 2024-09-10T21:52:59.480 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53062.log 2024-09-10T21:52:59.480 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.65292.log.gz 2024-09-10T21:52:59.480 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35797.log.gz 2024-09-10T21:52:59.481 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55623.log 2024-09-10T21:52:59.482 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53062.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53062.log.gz 2024-09-10T21:52:59.482 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54691.log 2024-09-10T21:52:59.483 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58525.log 2024-09-10T21:52:59.483 INFO:teuthology.orchestra.run.smithi007.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.55623.log.gz 2024-09-10T21:52:59.484 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54691.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54691.log.gzgzip 2024-09-10T21:52:59.484 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.69651.log 2024-09-10T21:52:59.485 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.58525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50826.log 2024-09-10T21:52:59.485 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.58525.log.gz 2024-09-10T21:52:59.486 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45023.log 2024-09-10T21:52:59.487 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50826.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50826.log.gz 2024-09-10T21:52:59.487 INFO:teuthology.orchestra.run.smithi007.stderr: 91.9% -- replaced with /var/log/ceph/ceph-client.admin.69651.log.gz 2024-09-10T21:52:59.487 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.43679.log 2024-09-10T21:52:59.488 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45023.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46868.log 2024-09-10T21:52:59.488 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45023.log.gz 2024-09-10T21:52:59.489 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.2.43679.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.43679.log.gz 2024-09-10T21:52:59.489 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48008.log 2024-09-10T21:52:59.490 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71601.log 2024-09-10T21:52:59.490 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46868.log.gz 2024-09-10T21:52:59.491 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45485.log 2024-09-10T21:52:59.491 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48008.log.gz 2024-09-10T21:52:59.492 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44943.log 2024-09-10T21:52:59.492 INFO:teuthology.orchestra.run.smithi007.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.71601.log.gz 2024-09-10T21:52:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45485.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45485.log.gz 2024-09-10T21:52:59.493 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47103.log 2024-09-10T21:52:59.494 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52218.log 2024-09-10T21:52:59.494 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44943.log.gz 2024-09-10T21:52:59.495 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47103.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47103.log.gz 2024-09-10T21:52:59.495 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46977.log 2024-09-10T21:52:59.496 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40930.log 2024-09-10T21:52:59.496 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52218.log.gz 2024-09-10T21:52:59.496 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46977.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46977.log.gz 2024-09-10T21:52:59.496 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53230.log 2024-09-10T21:52:59.497 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40930.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.40930.log.gz 2024-09-10T21:52:59.498 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50339.log 2024-09-10T21:52:59.498 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53230.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.53230.log.gz 2024-09-10T21:52:59.498 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.65358.log 2024-09-10T21:52:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50339.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50339.log.gz 2024-09-10T21:52:59.499 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56474.log 2024-09-10T21:52:59.500 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65358.log: 87.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.65358.log.gz 2024-09-10T21:52:59.500 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.67263.log 2024-09-10T21:52:59.501 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36079.log 2024-09-10T21:52:59.501 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56474.log.gz 2024-09-10T21:52:59.502 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35994.log 2024-09-10T21:52:59.502 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.67263.log.gz 2024-09-10T21:52:59.503 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50629.log 2024-09-10T21:52:59.503 INFO:teuthology.orchestra.run.smithi007.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36079.log.gz 2024-09-10T21:52:59.504 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35994.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35994.log.gz 2024-09-10T21:52:59.504 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50784.log 2024-09-10T21:52:59.505 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52707.log 2024-09-10T21:52:59.505 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50629.log.gz 2024-09-10T21:52:59.506 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50784.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50784.log.gz 2024-09-10T21:52:59.506 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72166.log 2024-09-10T21:52:59.507 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52707.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.52707.log.gz 2024-09-10T21:52:59.507 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54435.log 2024-09-10T21:52:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72166.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72166.log.gz 2024-09-10T21:52:59.508 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47061.log 2024-09-10T21:52:59.509 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.54435.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54435.log.gz 2024-09-10T21:52:59.509 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49866.log 2024-09-10T21:52:59.510 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47061.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47061.log.gz 2024-09-10T21:52:59.510 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64796.log 2024-09-10T21:52:59.511 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42246.log 87.4% 2024-09-10T21:52:59.511 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49866.log.gz 2024-09-10T21:52:59.512 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71391.log 2024-09-10T21:52:59.512 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64796.log.gz 2024-09-10T21:52:59.513 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42246.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42246.log.gz 2024-09-10T21:52:59.513 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46757.log 2024-09-10T21:52:59.513 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71391.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.71391.log.gz 2024-09-10T21:52:59.514 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67095.log 2024-09-10T21:52:59.515 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48052.log 2024-09-10T21:52:59.515 INFO:teuthology.orchestra.run.smithi007.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46757.log.gz 2024-09-10T21:52:59.516 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67095.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.67095.log.gz 2024-09-10T21:52:59.516 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65724.log 2024-09-10T21:52:59.516 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48052.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48052.log.gz 2024-09-10T21:52:59.516 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40716.log 2024-09-10T21:52:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65724.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65724.log.gz 2024-09-10T21:52:59.517 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49698.log 2024-09-10T21:52:59.518 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49656.log 2024-09-10T21:52:59.518 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40716.log.gz 2024-09-10T21:52:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49698.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49698.log.gz 2024-09-10T21:52:59.519 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48558.log 2024-09-10T21:52:59.520 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49656.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60265.log 2024-09-10T21:52:59.520 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49656.log.gz 2024-09-10T21:52:59.521 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50419.log 2024-09-10T21:52:59.521 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.48558.log.gz 2024-09-10T21:52:59.522 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60265.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60265.log.gz 2024-09-10T21:52:59.522 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51166.log 2024-09-10T21:52:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50419.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50419.log.gz 2024-09-10T21:52:59.523 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46075.log 2024-09-10T21:52:59.524 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64693.log 2024-09-10T21:52:59.524 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51166.log.gz 2024-09-10T21:52:59.525 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46075.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46075.log.gz 2024-09-10T21:52:59.525 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71433.log 2024-09-10T21:52:59.526 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48096.log 2024-09-10T21:52:59.526 INFO:teuthology.orchestra.run.smithi007.stderr: 90.2% -- replaced with /var/log/ceph/ceph-client.admin.64693.log.gz 2024-09-10T21:52:59.527 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71433.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.71433.log.gz 2024-09-10T21:52:59.527 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.64603.log 2024-09-10T21:52:59.528 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68560.log 2024-09-10T21:52:59.528 INFO:teuthology.orchestra.run.smithi007.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48096.log.gz 2024-09-10T21:52:59.529 INFO:teuthology.orchestra.run.smithi133.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-10T21:52:59.529 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53571.log 2024-09-10T21:52:59.529 INFO:teuthology.orchestra.run.smithi007.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64603.log.gz 2024-09-10T21:52:59.530 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44565.log 2024-09-10T21:52:59.530 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68560.log.gz 2024-09-10T21:52:59.531 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50956.log 2024-09-10T21:52:59.531 INFO:teuthology.orchestra.run.smithi007.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53571.log.gz 2024-09-10T21:52:59.532 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44565.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44565.log.gz 2024-09-10T21:52:59.532 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54097.log 2024-09-10T21:52:59.533 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72039.log 2024-09-10T21:52:59.533 INFO:teuthology.orchestra.run.smithi007.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50956.log.gz 2024-09-10T21:52:59.534 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54097.log: 86.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.54097.log.gz 2024-09-10T21:52:59.534 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.54247.log 2024-09-10T21:52:59.534 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72039.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T21:52:59.534 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72039.log.gz 2024-09-10T21:52:59.535 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54247.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54247.log.gz 2024-09-10T21:52:59.535 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61737.log 2024-09-10T21:52:59.536 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46009.log 2024-09-10T21:52:59.537 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61737.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61737.log.gz 2024-09-10T21:52:59.538 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52134.log 2024-09-10T21:52:59.538 INFO:teuthology.orchestra.run.smithi007.stderr: 89.5%/var/log/ceph/ceph-client.admin.46009.log: -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T21:52:59.539 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36049.log 2024-09-10T21:52:59.539 INFO:teuthology.orchestra.run.smithi007.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.46009.log.gz 2024-09-10T21:52:59.539 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56970.log 2024-09-10T21:52:59.540 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.52134.log.gz 2024-09-10T21:52:59.541 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36049.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36049.log.gz 2024-09-10T21:52:59.541 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59428.log 2024-09-10T21:52:59.541 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68095.log 2024-09-10T21:52:59.542 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56970.log.gz 2024-09-10T21:52:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.59428.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59428.log.gz 2024-09-10T21:52:59.542 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57977.log 2024-09-10T21:52:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.68095.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.68095.log.gz 2024-09-10T21:52:59.543 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64670.log 2024-09-10T21:52:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57977.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57977.log.gz 2024-09-10T21:52:59.544 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53272.log 2024-09-10T21:52:59.545 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58145.log 2024-09-10T21:52:59.545 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.64670.log.gz 2024-09-10T21:52:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53272.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53272.log.gz 2024-09-10T21:52:59.546 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72123.log 2024-09-10T21:52:59.547 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69852.log 2024-09-10T21:52:59.547 INFO:teuthology.orchestra.run.smithi007.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.58145.log.gz 2024-09-10T21:52:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72123.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.72123.log.gz 2024-09-10T21:52:59.548 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57156.log 2024-09-10T21:52:59.549 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67137.log 2024-09-10T21:52:59.549 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69852.log.gz 2024-09-10T21:52:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57156.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57156.log.gz 2024-09-10T21:52:59.550 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49353.log 2024-09-10T21:52:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71031.log 2024-09-10T21:52:59.551 INFO:teuthology.orchestra.run.smithi007.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.67137.log.gz 2024-09-10T21:52:59.551 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58938.log 2024-09-10T21:52:59.552 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49353.log.gz 2024-09-10T21:52:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71031.log: gzip 87.3% -5 -- replaced with /var/log/ceph/ceph-client.admin.71031.log.gz --verbose 2024-09-10T21:52:59.553 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.55839.log 2024-09-10T21:52:59.553 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66717.log 2024-09-10T21:52:59.554 INFO:teuthology.orchestra.run.smithi007.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.admin.58938.log.gz 2024-09-10T21:52:59.554 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50118.log 2024-09-10T21:52:59.555 INFO:teuthology.orchestra.run.smithi007.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.55839.log.gz 2024-09-10T21:52:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66717.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66717.log.gz 2024-09-10T21:52:59.556 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47824.log 2024-09-10T21:52:59.557 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49092.log 2024-09-10T21:52:59.557 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50118.log.gz 2024-09-10T21:52:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47824.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47824.log.gzgzip 2024-09-10T21:52:59.558 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41881.log 2024-09-10T21:52:59.558 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63724.log 2024-09-10T21:52:59.559 INFO:teuthology.orchestra.run.smithi007.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.49092.log.gz 2024-09-10T21:52:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41881.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41881.log.gz 2024-09-10T21:52:59.560 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59240.log 2024-09-10T21:52:59.561 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55234.log 2024-09-10T21:52:59.561 INFO:teuthology.orchestra.run.smithi007.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.63724.log.gz 2024-09-10T21:52:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59240.log: gzip -5 --verbose 87.4% -- -- replaced with /var/log/ceph/ceph-client.admin.59240.log.gz /var/log/ceph/ceph-client.admin.54614.log 2024-09-10T21:52:59.562 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-10T21:52:59.562 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.55234.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.55234.log.gz 2024-09-10T21:52:59.563 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54055.log 2024-09-10T21:52:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54614.log: gzip 87.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.54614.log.gz --verbose 2024-09-10T21:52:59.564 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.41902.log 2024-09-10T21:52:59.564 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54055.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54055.log.gz 2024-09-10T21:52:59.565 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66379.log 2024-09-10T21:52:59.565 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41563.log 2024-09-10T21:52:59.566 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41902.log.gz 2024-09-10T21:52:59.566 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66379.log: gzip -5 --verbose -- 87.4% /var/log/ceph/ceph-client.admin.41266.log -- replaced with /var/log/ceph/ceph-client.admin.66379.log.gz 2024-09-10T21:52:59.566 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-10T21:52:59.567 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62582.log 2024-09-10T21:52:59.567 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41563.log.gz 2024-09-10T21:52:59.568 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41266.log: gzip 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41266.log.gz -5 2024-09-10T21:52:59.568 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60450.log 2024-09-10T21:52:59.569 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41574.log 2024-09-10T21:52:59.569 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62582.log.gz 2024-09-10T21:52:59.570 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42442.log 2024-09-10T21:52:59.571 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41574.log: 92.4% -- replaced with /var/log/ceph/ceph-client.admin.60450.log.gz 2024-09-10T21:52:59.571 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose 87.2% -- -- replaced with /var/log/ceph/ceph-client.admin.41574.log.gz /var/log/ceph/ceph-client.admin.52936.log 2024-09-10T21:52:59.571 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-10T21:52:59.572 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42442.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52026.log 2024-09-10T21:52:59.572 INFO:teuthology.orchestra.run.smithi007.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.42442.log.gz 2024-09-10T21:52:59.573 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52936.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52936.log.gz 2024-09-10T21:52:59.573 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62626.log 2024-09-10T21:52:59.574 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52026.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52026.log.gz 2024-09-10T21:52:59.574 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45781.log 2024-09-10T21:52:59.575 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47229.log 2024-09-10T21:52:59.575 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62626.log.gz 2024-09-10T21:52:59.576 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45781.log: gzip -5 87.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.45781.log.gz 2024-09-10T21:52:59.576 INFO:teuthology.orchestra.run.smithi007.stderr: -- /var/log/ceph/ceph-client.admin.64507.log 2024-09-10T21:52:59.577 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47229.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47229.log.gz 2024-09-10T21:52:59.577 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66461.log 2024-09-10T21:52:59.578 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62199.log 2024-09-10T21:52:59.578 INFO:teuthology.orchestra.run.smithi007.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.64507.log.gz 2024-09-10T21:52:59.579 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49950.log 2024-09-10T21:52:59.579 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66461.log.gz 2024-09-10T21:52:59.580 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63639.log 2024-09-10T21:52:59.580 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62199.log.gz 2024-09-10T21:52:59.581 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49950.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49950.log.gz 2024-09-10T21:52:59.581 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42484.log 2024-09-10T21:52:59.581 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45103.log 2024-09-10T21:52:59.582 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63639.log.gz 2024-09-10T21:52:59.582 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42484.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42484.log.gz 2024-09-10T21:52:59.583 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71073.log 2024-09-10T21:52:59.583 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45103.log: gzip -5 --verbose -- 87.5% /var/log/ceph/ceph-client.admin.48138.log -- replaced with /var/log/ceph/ceph-client.admin.45103.log.gz 2024-09-10T21:52:59.583 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-10T21:52:59.584 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71073.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71073.log.gz 2024-09-10T21:52:59.584 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53761.log 2024-09-10T21:52:59.585 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61206.log 2024-09-10T21:52:59.585 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48138.log.gz 2024-09-10T21:52:59.586 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53761.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.53761.log.gz 2024-09-10T21:52:59.586 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68518.log 2024-09-10T21:52:59.587 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71475.log 2024-09-10T21:52:59.587 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.61206.log.gz 2024-09-10T21:52:59.588 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68518.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68518.log.gz 2024-09-10T21:52:59.588 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52326.log 2024-09-10T21:52:59.589 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71685.log 2024-09-10T21:52:59.589 INFO:teuthology.orchestra.run.smithi007.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.71475.log.gz 2024-09-10T21:52:59.590 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49018.log 2024-09-10T21:52:59.590 INFO:teuthology.orchestra.run.smithi007.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.52326.log.gz 2024-09-10T21:52:59.591 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71685.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50914.log 2024-09-10T21:52:59.591 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71685.log.gz 2024-09-10T21:52:59.592 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49018.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45231.log 2024-09-10T21:52:59.592 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.49018.log.gz 2024-09-10T21:52:59.593 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71997.log 2024-09-10T21:52:59.593 INFO:teuthology.orchestra.run.smithi007.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50914.log.gz 2024-09-10T21:52:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45231.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45231.log.gz 2024-09-10T21:52:59.594 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52791.log 2024-09-10T21:52:59.595 INFO:teuthology.orchestra.run.smithi133.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-10T21:52:59.595 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71997.log: gzip -5 --verbose 86.7% -- -- replaced with /var/log/ceph/ceph-client.admin.71997.log.gz 2024-09-10T21:52:59.595 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.62912.log 2024-09-10T21:52:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52791.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52791.log.gz 2024-09-10T21:52:59.596 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44446.log 2024-09-10T21:52:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54854.log 2024-09-10T21:52:59.597 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62912.log.gz 2024-09-10T21:52:59.597 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44446.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44446.log.gz 2024-09-10T21:52:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47313.log 2024-09-10T21:52:59.598 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70821.log 2024-09-10T21:52:59.599 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54854.log.gz 2024-09-10T21:52:59.599 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47313.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47313.log.gz 2024-09-10T21:52:59.599 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67179.log 2024-09-10T21:52:59.600 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62670.log 2024-09-10T21:52:59.600 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70821.log.gz 2024-09-10T21:52:59.601 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67179.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67179.log.gz 2024-09-10T21:52:59.601 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71811.log 2024-09-10T21:52:59.602 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41908.log 2024-09-10T21:52:59.602 INFO:teuthology.orchestra.run.smithi007.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.62670.log.gz 2024-09-10T21:52:59.603 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53803.log 2024-09-10T21:52:59.603 INFO:teuthology.orchestra.run.smithi007.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.71811.log.gz 2024-09-10T21:52:59.604 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41908.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41908.log.gz 2024-09-10T21:52:59.604 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58271.log 2024-09-10T21:52:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53803.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53803.log.gz 2024-09-10T21:52:59.605 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56928.log 2024-09-10T21:52:59.606 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46831.log 2024-09-10T21:52:59.610 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.58271.log.gz 2024-09-10T21:52:59.610 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56928.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56928.log.gz 2024-09-10T21:52:59.610 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45273.log 2024-09-10T21:52:59.610 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46831.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46831.log.gz 2024-09-10T21:52:59.610 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54181.log 2024-09-10T21:52:59.610 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45273.log: gzip -5 87.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.45273.log.gz -- 2024-09-10T21:52:59.610 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.52833.log 2024-09-10T21:52:59.610 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41567.log 2024-09-10T21:52:59.611 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.54181.log.gz 2024-09-10T21:52:59.611 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52833.log: 87.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.52833.log.gz 2024-09-10T21:52:59.611 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36274.log 2024-09-10T21:52:59.612 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53613.log 2024-09-10T21:52:59.612 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41567.log.gz 2024-09-10T21:52:59.612 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36274.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36274.log.gz 2024-09-10T21:52:59.613 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59524.log 2024-09-10T21:52:59.614 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53613.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53613.log.gz 2024-09-10T21:52:59.614 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48516.log 2024-09-10T21:52:59.614 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56059.log 2024-09-10T21:52:59.615 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59524.log.gz 2024-09-10T21:52:59.616 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45613.log 2024-09-10T21:52:59.616 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48516.log.gz 2024-09-10T21:52:59.616 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71349.log 2024-09-10T21:52:59.617 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56059.log.gz 2024-09-10T21:52:59.617 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47966.log 2024-09-10T21:52:59.617 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45613.log.gz 2024-09-10T21:52:59.618 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71349.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.71349.log.gz 2024-09-10T21:52:59.618 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55396.log 2024-09-10T21:52:59.619 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47966.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47966.log.gz 2024-09-10T21:52:59.619 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53701.log 2024-09-10T21:52:59.620 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55396.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55396.log.gz 2024-09-10T21:52:59.620 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54139.log 2024-09-10T21:52:59.621 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53701.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-10T21:52:59.621 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53701.log.gz 2024-09-10T21:52:59.622 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54139.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54139.log.gz 2024-09-10T21:52:59.622 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49180.log 2024-09-10T21:52:59.623 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49572.log 2024-09-10T21:52:59.624 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47271.log 2024-09-10T21:52:59.624 INFO:teuthology.orchestra.run.smithi007.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.49180.log.gz 2024-09-10T21:52:59.625 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54733.log 2024-09-10T21:52:59.625 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49572.log.gz 2024-09-10T21:52:59.626 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55689.log 2024-09-10T21:52:59.627 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47271.log.gz 2024-09-10T21:52:59.627 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47439.log 2024-09-10T21:52:59.627 INFO:teuthology.orchestra.run.smithi007.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54733.log.gz 2024-09-10T21:52:59.628 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55689.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55689.log.gz 2024-09-10T21:52:59.629 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68273.log 2024-09-10T21:52:59.629 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40972.log 2024-09-10T21:52:59.630 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47439.log.gz 2024-09-10T21:52:59.631 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68273.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68273.log.gz 2024-09-10T21:52:59.631 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60637.log 2024-09-10T21:52:59.632 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40972.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40972.log.gz 2024-09-10T21:52:59.632 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61055.log 2024-09-10T21:52:59.633 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65381.log 2024-09-10T21:52:59.633 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.60637.log.gz 2024-09-10T21:52:59.634 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61055.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63164.log 2024-09-10T21:52:59.634 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.61055.log.gz 2024-09-10T21:52:59.635 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65208.log 2024-09-10T21:52:59.635 INFO:teuthology.orchestra.run.smithi007.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.admin.65381.log.gz 2024-09-10T21:52:59.636 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55797.log 2024-09-10T21:52:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65208.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.63164.log.gz 2024-09-10T21:52:59.637 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66801.log 2024-09-10T21:52:59.638 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.65208.log.gz 2024-09-10T21:52:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72206.log 2024-09-10T21:52:59.639 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.55797.log.gz 2024-09-10T21:52:59.639 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57325.log 2024-09-10T21:52:59.640 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72206.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66801.log.gz 2024-09-10T21:52:59.640 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72206.log.gz 2024-09-10T21:52:59.641 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50076.log 2024-09-10T21:52:59.642 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57325.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57325.log.gz 2024-09-10T21:52:59.642 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67053.log 2024-09-10T21:52:59.643 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50076.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50076.log.gz 2024-09-10T21:52:59.643 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44775.log 2024-09-10T21:52:59.644 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70232.log 2024-09-10T21:52:59.645 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67053.log.gz 2024-09-10T21:52:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68704.log 2024-09-10T21:52:59.646 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44775.log.gz 2024-09-10T21:52:59.646 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.70232.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70232.log.gz 2024-09-10T21:52:59.647 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35893.log 2024-09-10T21:52:59.648 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61314.log 2024-09-10T21:52:59.649 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35893.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.68704.log.gz 2024-09-10T21:52:59.649 INFO:teuthology.orchestra.run.smithi007.stderr: 76.4% -- replaced with /var/log/ceph/ceph-client.admin.35893.log.gz 2024-09-10T21:52:59.649 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45399.log 2024-09-10T21:52:59.650 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66759.log 2024-09-10T21:52:59.650 INFO:teuthology.orchestra.run.smithi007.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.61314.log.gz 2024-09-10T21:52:59.651 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.45399.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.52623.log 2024-09-10T21:52:59.651 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45399.log.gz 2024-09-10T21:52:59.652 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66759.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66759.log.gz 2024-09-10T21:52:59.652 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57708.log 2024-09-10T21:52:59.653 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52623.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52623.log.gz 2024-09-10T21:52:59.653 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58187.log 2024-09-10T21:52:59.654 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41910.log 2024-09-10T21:52:59.655 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57708.log.gz 2024-09-10T21:52:59.655 INFO:teuthology.orchestra.run.smithi133.stderr: 93.1% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-10T21:52:59.656 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64419.log 2024-09-10T21:52:59.656 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58187.log.gz 2024-09-10T21:52:59.657 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59783.log 2024-09-10T21:52:59.657 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41910.log.gz 2024-09-10T21:52:59.658 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52665.log 2024-09-10T21:52:59.658 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.64419.log.gz 2024-09-10T21:52:59.659 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51852.log 2024-09-10T21:52:59.660 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52665.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59783.log.gz 2024-09-10T21:52:59.660 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52665.log.gz 2024-09-10T21:52:59.660 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72081.log 2024-09-10T21:52:59.661 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43873.log 2024-09-10T21:52:59.661 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51852.log.gz 2024-09-10T21:52:59.662 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58860.log 2024-09-10T21:52:59.663 INFO:teuthology.orchestra.run.smithi007.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.72081.log.gz 2024-09-10T21:52:59.663 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43873.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43873.log.gz 2024-09-10T21:52:59.664 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41901.log 2024-09-10T21:52:59.665 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58229.log 2024-09-10T21:52:59.665 INFO:teuthology.orchestra.run.smithi007.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.58860.log.gz 2024-09-10T21:52:59.666 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41901.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.41901.log.gz 2024-09-10T21:52:59.666 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53447.log 2024-09-10T21:52:59.667 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59891.log 2024-09-10T21:52:59.667 INFO:teuthology.orchestra.run.smithi007.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.58229.log.gz 2024-09-10T21:52:59.668 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53447.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53447.log.gz 2024-09-10T21:52:59.668 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53146.log 2024-09-10T21:52:59.669 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71643.log 2024-09-10T21:52:59.670 INFO:teuthology.orchestra.run.smithi007.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.59891.log.gz 2024-09-10T21:52:59.670 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53146.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29764.log 2024-09-10T21:52:59.671 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53146.log.gz 2024-09-10T21:52:59.672 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64420.log 2024-09-10T21:52:59.672 INFO:teuthology.orchestra.run.smithi007.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.71643.log.gz 2024-09-10T21:52:59.672 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.tmp-client.admin.29764.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29764.log.gz 2024-09-10T21:52:59.673 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42874.log 2024-09-10T21:52:59.674 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65121.log 2024-09-10T21:52:59.674 INFO:teuthology.orchestra.run.smithi007.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.64420.log.gz 2024-09-10T21:52:59.674 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42874.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42874.log.gz 2024-09-10T21:52:59.675 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66545.log 2024-09-10T21:52:59.676 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36870.log 2024-09-10T21:52:59.676 INFO:teuthology.orchestra.run.smithi007.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.65121.log.gz 2024-09-10T21:52:59.677 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45571.log 2024-09-10T21:52:59.677 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66545.log.gz 2024-09-10T21:52:59.678 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36870.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36870.log.gz 2024-09-10T21:52:59.678 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36247.log 2024-09-10T21:52:59.679 INFO:teuthology.orchestra.run.smithi133.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-10T21:52:59.679 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64906.log 2024-09-10T21:52:59.680 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45571.log.gz 2024-09-10T21:52:59.680 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36247.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36247.log.gz 2024-09-10T21:52:59.680 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42133.log 2024-09-10T21:52:59.681 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64906.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.64906.log.gz 2024-09-10T21:52:59.682 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36578.log 2024-09-10T21:52:59.683 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42133.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42133.log.gz 2024-09-10T21:52:59.683 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40538.log 2024-09-10T21:52:59.684 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36578.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55993.log 2024-09-10T21:52:59.685 INFO:teuthology.orchestra.run.smithi007.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36578.log.gz 2024-09-10T21:52:59.685 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40538.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40538.log.gz 2024-09-10T21:52:59.685 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-10T21:52:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59132.log 2024-09-10T21:52:59.687 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.55993.log.gz 2024-09-10T21:52:59.687 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57624.log 2024-09-10T21:52:59.689 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59132.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.59132.log.gz 2024-09-10T21:52:59.689 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66204.log 2024-09-10T21:52:59.690 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41140.log 2024-09-10T21:52:59.690 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57624.log.gz 2024-09-10T21:52:59.691 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66204.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66204.log.gz 2024-09-10T21:52:59.691 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52978.log 2024-09-10T21:52:59.692 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.41140.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47744.log 2024-09-10T21:52:59.693 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41140.log.gz 2024-09-10T21:52:59.693 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52978.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52978.log.gz 2024-09-10T21:52:59.694 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66969.log 2024-09-10T21:52:59.694 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.47744.log: /var/log/ceph/ceph-client.admin.52068.log 2024-09-10T21:52:59.695 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47744.log.gz 2024-09-10T21:52:59.695 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66969.log: 87.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.66969.log.gz -5 2024-09-10T21:52:59.695 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42244.log 2024-09-10T21:52:59.696 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40888.log 2024-09-10T21:52:59.697 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42244.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.52068.log.gz 2024-09-10T21:52:59.698 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64852.log 2024-09-10T21:52:59.698 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42244.log.gz 2024-09-10T21:52:59.699 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71859.log 2024-09-10T21:52:59.699 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40888.log.gz 2024-09-10T21:52:59.700 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64852.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.64852.log.gz 2024-09-10T21:52:59.700 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47019.log 2024-09-10T21:52:59.701 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61272.log 2024-09-10T21:52:59.702 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.71859.log.gz 2024-09-10T21:52:59.702 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47019.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.47019.log.gz 2024-09-10T21:52:59.702 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63230.log 2024-09-10T21:52:59.703 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.61272.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.57796.log 2024-09-10T21:52:59.704 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61272.log.gz 2024-09-10T21:52:59.704 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63230.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63230.log.gz 2024-09-10T21:52:59.704 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59005.log 2024-09-10T21:52:59.705 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56209.log 2024-09-10T21:52:59.706 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57796.log.gz 2024-09-10T21:52:59.707 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59005.log: 87.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63038.log 2024-09-10T21:52:59.707 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.59005.log.gz 2024-09-10T21:52:59.707 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49824.log 2024-09-10T21:52:59.708 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56209.log.gz 2024-09-10T21:52:59.709 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61905.log 2024-09-10T21:52:59.709 INFO:teuthology.orchestra.run.smithi007.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.63038.log.gz 2024-09-10T21:52:59.710 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49824.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49824.log.gz 2024-09-10T21:52:59.710 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62954.log 2024-09-10T21:52:59.711 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57847.log 2024-09-10T21:52:59.711 INFO:teuthology.orchestra.run.smithi007.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.61905.log.gz 2024-09-10T21:52:59.712 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42568.log 2024-09-10T21:52:59.712 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62954.log.gz 2024-09-10T21:52:59.713 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57847.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.57847.log.gz 2024-09-10T21:52:59.713 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54896.log 2024-09-10T21:52:59.714 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53971.log 2024-09-10T21:52:59.715 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42568.log.gz 2024-09-10T21:52:59.716 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54896.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.54896.log.gzgzip 2024-09-10T21:52:59.716 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.47913.log 2024-09-10T21:52:59.717 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53971.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53971.log.gz 2024-09-10T21:52:59.717 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63122.log 2024-09-10T21:52:59.718 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62721.log 2024-09-10T21:52:59.718 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47913.log.gz 2024-09-10T21:52:59.719 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55473.log 2024-09-10T21:52:59.719 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63122.log.gz 2024-09-10T21:52:59.720 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62721.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62721.log.gz 2024-09-10T21:52:59.720 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44083.log 2024-09-10T21:52:59.721 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55473.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-10T21:52:59.723 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44083.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.55473.log.gz 2024-09-10T21:52:59.723 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51702.log 2024-09-10T21:52:59.723 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44083.log.gz 2024-09-10T21:52:59.723 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58721.log 2024-09-10T21:52:59.725 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41572.log 2024-09-10T21:52:59.725 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.51702.log.gz 2024-09-10T21:52:59.726 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42526.log 2024-09-10T21:52:59.726 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58721.log.gz 2024-09-10T21:52:59.727 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41572.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41572.log.gz 2024-09-10T21:52:59.728 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58798.log 2024-09-10T21:52:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42526.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42526.log.gz 2024-09-10T21:52:59.729 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65976.log 2024-09-10T21:52:59.730 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48222.log 2024-09-10T21:52:59.731 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.58798.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58798.log.gz 2024-09-10T21:52:59.732 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66018.log 2024-09-10T21:52:59.732 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65976.log.gz 2024-09-10T21:52:59.733 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51646.log 2024-09-10T21:52:59.733 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48222.log.gz 2024-09-10T21:52:59.736 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66018.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66018.log.gz 2024-09-10T21:52:59.736 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41907.log 2024-09-10T21:52:59.736 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.51646.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.69810.log 2024-09-10T21:52:59.736 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.51646.log.gz 2024-09-10T21:52:59.736 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41907.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41907.log.gz 2024-09-10T21:52:59.737 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49740.log 2024-09-10T21:52:59.738 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69810.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.69810.log.gz 2024-09-10T21:52:59.738 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42652.log 2024-09-10T21:52:59.739 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49740.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49740.log.gz 2024-09-10T21:52:59.740 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47397.log 2024-09-10T21:52:59.740 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64306.log 2024-09-10T21:52:59.741 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42652.log.gz 2024-09-10T21:52:59.742 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47397.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47397.log.gz 2024-09-10T21:52:59.742 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45655.log 2024-09-10T21:52:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64306.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64306.log.gz 2024-09-10T21:52:59.743 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63080.log 2024-09-10T21:52:59.745 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54756.log 2024-09-10T21:52:59.745 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45655.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45655.log.gz 2024-09-10T21:52:59.746 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63080.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.63080.log.gz 2024-09-10T21:52:59.746 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42242.log 2024-09-10T21:52:59.747 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54756.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54756.log.gz 2024-09-10T21:52:59.747 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43813.log 2024-09-10T21:52:59.748 INFO:teuthology.orchestra.run.smithi007.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-10T21:52:59.748 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50501.log 2024-09-10T21:52:59.749 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42242.log.gz 2024-09-10T21:52:59.749 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43813.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43813.log.gz 2024-09-10T21:52:59.750 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-10T21:52:59.751 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40844.log 2024-09-10T21:52:59.751 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50501.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50501.log.gz 2024-09-10T21:52:59.752 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56583.log 2024-09-10T21:52:59.753 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.40844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46720.log 2024-09-10T21:52:59.753 INFO:teuthology.orchestra.run.smithi007.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.40844.log.gz 2024-09-10T21:52:59.754 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56583.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56583.log.gz 2024-09-10T21:52:59.755 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49330.log 2024-09-10T21:52:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46720.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46720.log.gz 2024-09-10T21:52:59.756 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45967.log 2024-09-10T21:52:59.758 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70947.log 2024-09-10T21:52:59.758 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49330.log.gz 2024-09-10T21:52:59.759 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45967.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45967.log.gz 2024-09-10T21:52:59.759 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44125.log 2024-09-10T21:52:59.760 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70947.log: gzip 87.2% -- replaced with /var/log/ceph/ceph-client.admin.70947.log.gz -5 2024-09-10T21:52:59.760 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42236.log 2024-09-10T21:52:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45145.log 2024-09-10T21:52:59.762 INFO:teuthology.orchestra.run.smithi007.stderr: 94.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-10T21:52:59.762 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44125.log.gz 2024-09-10T21:52:59.762 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62336.log 2024-09-10T21:52:59.763 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42236.log.gz 2024-09-10T21:52:59.764 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45145.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45145.log.gz 2024-09-10T21:52:59.764 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70905.log 2024-09-10T21:52:59.765 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62336.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T21:52:59.765 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62336.log.gz 2024-09-10T21:52:59.766 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53367.log 2024-09-10T21:52:59.766 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.70905.log.gz 2024-09-10T21:52:59.767 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57889.log 2024-09-10T21:52:59.768 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42694.log 2024-09-10T21:52:59.769 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53367.log.gz 2024-09-10T21:52:59.770 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59675.log 2024-09-10T21:52:59.770 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57889.log.gz 2024-09-10T21:52:59.770 INFO:teuthology.orchestra.run.smithi007.stderr: 86.8% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T21:52:59.771 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42694.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42694.log.gz 2024-09-10T21:52:59.771 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66503.log 2024-09-10T21:52:59.772 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59090.log 2024-09-10T21:52:59.772 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66503.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59675.log.gz 2024-09-10T21:52:59.773 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71913.log 2024-09-10T21:52:59.773 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66503.log.gz 2024-09-10T21:52:59.774 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50691.log 2024-09-10T21:52:59.774 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59090.log.gz 2024-09-10T21:52:59.775 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71913.log: 87.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63780.log 2024-09-10T21:52:59.776 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71913.log.gz 2024-09-10T21:52:59.776 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45739.log 2024-09-10T21:52:59.777 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50691.log.gz 2024-09-10T21:52:59.778 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59174.log 2024-09-10T21:52:59.778 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63780.log.gz 2024-09-10T21:52:59.779 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45739.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45739.log.gz 2024-09-10T21:52:59.779 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48432.log 2024-09-10T21:52:59.780 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52893.log 2024-09-10T21:52:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48432.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59174.log.gz 2024-09-10T21:52:59.781 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48432.log.gz 2024-09-10T21:52:59.781 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47607.log 2024-09-10T21:52:59.782 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48180.log 2024-09-10T21:52:59.783 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52893.log.gz 2024-09-10T21:52:59.783 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47607.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47607.log.gz 2024-09-10T21:52:59.783 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50034.log 2024-09-10T21:52:59.784 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64229.log 2024-09-10T21:52:59.784 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48180.log.gz 2024-09-10T21:52:59.785 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50034.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50034.log.gz 2024-09-10T21:52:59.786 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62241.log 2024-09-10T21:52:59.786 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56802.log 2024-09-10T21:52:59.787 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64229.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.64229.log.gz 2024-09-10T21:52:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62241.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.62241.log.gz 2024-09-10T21:52:59.788 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46263.log 2024-09-10T21:52:59.789 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60385.log 2024-09-10T21:52:59.789 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56802.log.gz 2024-09-10T21:52:59.789 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.46263.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.46263.log.gz 2024-09-10T21:52:59.790 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59741.log 2024-09-10T21:52:59.791 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60385.log: gzip -5 --verbose 87.1% -- /var/log/ceph/ceph-client.admin.59590.log -- replaced with /var/log/ceph/ceph-client.admin.60385.log.gz 2024-09-10T21:52:59.791 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-10T21:52:59.793 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44649.log 2024-09-10T21:52:59.793 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59741.log.gz 2024-09-10T21:52:59.794 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.59590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46697.log 2024-09-10T21:52:59.794 INFO:teuthology.orchestra.run.smithi007.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59590.log.gz 2024-09-10T21:52:59.795 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67329.log 2024-09-10T21:52:59.795 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44649.log.gz 2024-09-10T21:52:59.796 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46697.log: gzip 87.3% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.46697.log.gz -- 2024-09-10T21:52:59.796 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.69936.log 2024-09-10T21:52:59.797 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67329.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67329.log.gz 2024-09-10T21:52:59.797 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43915.log 2024-09-10T21:52:59.798 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44733.log 2024-09-10T21:52:59.798 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69936.log.gz 2024-09-10T21:52:59.800 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43915.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43915.log.gz 2024-09-10T21:52:59.800 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60976.log 2024-09-10T21:52:59.801 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36216.log 2024-09-10T21:52:59.801 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44733.log.gz 2024-09-10T21:52:59.802 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.60976.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.60976.log.gz 2024-09-10T21:52:59.802 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52487.log 2024-09-10T21:52:59.802 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36216.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36216.log.gz 2024-09-10T21:52:59.803 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50998.log 2024-09-10T21:52:59.804 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49264.log 2024-09-10T21:52:59.804 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52487.log.gz 2024-09-10T21:52:59.805 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50998.log: gzip 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50998.log.gz -5 2024-09-10T21:52:59.805 INFO:teuthology.orchestra.run.smithi007.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60408.log 2024-09-10T21:52:59.806 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67764.log 2024-09-10T21:52:59.807 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60408.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.49264.log.gz 2024-09-10T21:52:59.807 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53845.log 2024-09-10T21:52:59.808 INFO:teuthology.orchestra.run.smithi007.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.60408.log.gz 2024-09-10T21:52:59.808 INFO:teuthology.orchestra.run.smithi133.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-10T21:52:59.809 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67907.log 2024-09-10T21:52:59.809 INFO:teuthology.orchestra.run.smithi007.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.67764.log.gz 2024-09-10T21:52:59.810 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53845.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.53845.log.gz 2024-09-10T21:52:59.810 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57666.log 2024-09-10T21:52:59.812 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69472.log 2024-09-10T21:52:59.812 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67907.log.gz 2024-09-10T21:52:59.813 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57666.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57666.log.gz 2024-09-10T21:52:59.813 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70044.log 2024-09-10T21:52:59.814 INFO:teuthology.orchestra.run.smithi007.stderr:gzip/var/log/ceph/ceph-client.admin.69472.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41392.log 2024-09-10T21:52:59.814 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70044.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.69472.log.gz 2024-09-10T21:52:59.814 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70044.log.gz 2024-09-10T21:52:59.815 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61653.log 2024-09-10T21:52:59.816 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42610.log 2024-09-10T21:52:59.816 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41392.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41392.log.gz 2024-09-10T21:52:59.817 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61653.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61653.log.gz 2024-09-10T21:52:59.817 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45315.log 2024-09-10T21:52:59.818 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52284.log 2024-09-10T21:52:59.818 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42610.log.gz 2024-09-10T21:52:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45315.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45315.log.gz 2024-09-10T21:52:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44607.log 2024-09-10T21:52:59.820 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52284.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52284.log.gz 2024-09-10T21:52:59.821 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60595.log 2024-09-10T21:52:59.822 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65555.log 2024-09-10T21:52:59.823 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44607.log.gz 2024-09-10T21:52:59.823 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48965.log 2024-09-10T21:52:59.823 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60595.log.gz 2024-09-10T21:52:59.824 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65555.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.65555.log.gz 2024-09-10T21:52:59.824 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60492.log 2024-09-10T21:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36781.log 2024-09-10T21:52:59.826 INFO:teuthology.orchestra.run.smithi007.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.48965.log.gz 2024-09-10T21:52:59.827 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60492.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57054.log 2024-09-10T21:52:59.827 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36781.log: 92.3% -- replaced with /var/log/ceph/ceph-client.admin.60492.log.gz 2024-09-10T21:52:59.828 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50160.log 2024-09-10T21:52:59.828 INFO:teuthology.orchestra.run.smithi007.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.36781.log.gz 2024-09-10T21:52:59.829 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67517.log 2024-09-10T21:52:59.830 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57054.log.gz 2024-09-10T21:52:59.830 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50160.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50160.log.gz 2024-09-10T21:52:59.830 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66060.log 2024-09-10T21:52:59.831 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.0.67517.log: --verbose -- /var/log/ceph/ceph-client.admin.44356.log 2024-09-10T21:52:59.831 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67517.log.gz 2024-09-10T21:52:59.832 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66060.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66060.log.gz 2024-09-10T21:52:59.832 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71955.log 2024-09-10T21:52:59.833 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66589.log 2024-09-10T21:52:59.834 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44356.log.gz 2024-09-10T21:52:59.834 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71955.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.71955.log.gz 2024-09-10T21:52:59.834 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70610.log 2024-09-10T21:52:59.835 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41559.log 2024-09-10T21:52:59.836 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66589.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.66589.log.gz 2024-09-10T21:52:59.836 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70610.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70610.log.gz 2024-09-10T21:52:59.836 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41098.log 2024-09-10T21:52:59.837 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51124.log 2024-09-10T21:52:59.838 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41559.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41559.log.gz 2024-09-10T21:52:59.838 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41098.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41098.log.gz 2024-09-10T21:52:59.838 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69561.log 2024-09-10T21:52:59.839 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51550.log 2024-09-10T21:52:59.840 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51124.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51124.log.gz 2024-09-10T21:52:59.840 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53657.log 2024-09-10T21:52:59.841 INFO:teuthology.orchestra.run.smithi007.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.69561.log.gz 2024-09-10T21:52:59.841 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36189.log 2024-09-10T21:52:59.842 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.51550.log.gz 2024-09-10T21:52:59.842 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.53657.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53657.log.gz 2024-09-10T21:52:59.843 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35938.log 2024-09-10T21:52:59.843 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5/var/log/ceph/ceph-client.admin.36189.log: --verbose -- /var/log/ceph/ceph-client.admin.67841.log 2024-09-10T21:52:59.844 INFO:teuthology.orchestra.run.smithi007.stderr: 76.9% -- replaced with /var/log/ceph/ceph-client.admin.36189.log.gz 2024-09-10T21:52:59.845 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.35938.log: 75.4% -- replaced with /var/log/ceph/ceph-client.admin.35938.log.gz 2024-09-10T21:52:59.845 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53529.log 2024-09-10T21:52:59.846 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65850.log 2024-09-10T21:52:59.846 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67841.log: /var/log/ceph/ceph-client.admin.53529.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53529.log.gz 2024-09-10T21:52:59.847 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71769.log 2024-09-10T21:52:59.847 INFO:teuthology.orchestra.run.smithi007.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.67841.log.gz 2024-09-10T21:52:59.848 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69416.log 2024-09-10T21:52:59.848 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.65850.log.gz 2024-09-10T21:52:59.849 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71769.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.71769.log.gz 2024-09-10T21:52:59.849 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54938.log 2024-09-10T21:52:59.850 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56760.log 2024-09-10T21:52:59.850 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54938.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.69416.log.gz 2024-09-10T21:52:59.851 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54938.log.gz 2024-09-10T21:52:59.851 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61121.log 2024-09-10T21:52:59.852 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.56760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36298.log 2024-09-10T21:52:59.853 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56760.log.gz 2024-09-10T21:52:59.853 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69305.log 2024-09-10T21:52:59.853 INFO:teuthology.orchestra.run.smithi007.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.61121.log.gz 2024-09-10T21:52:59.854 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44691.log 2024-09-10T21:52:59.855 INFO:teuthology.orchestra.run.smithi007.stderr: 77.0% -- replaced with /var/log/ceph/ceph-client.admin.36298.log.gz 2024-09-10T21:52:59.856 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69305.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.69305.log.gz 2024-09-10T21:52:59.856 INFO:teuthology.orchestra.run.smithi007.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.62115.log 2024-09-10T21:52:59.856 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65892.log 2024-09-10T21:52:59.857 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44691.log.gz 2024-09-10T21:52:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62115.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62115.log.gz 2024-09-10T21:52:59.858 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69538.log 2024-09-10T21:52:59.859 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68644.log 2024-09-10T21:52:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65892.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.65892.log.gz/var/log/ceph/ceph-client.admin.69538.log: 2024-09-10T21:52:59.860 INFO:teuthology.orchestra.run.smithi007.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.69538.log.gz 2024-09-10T21:52:59.860 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69693.log 2024-09-10T21:52:59.861 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45883.log 2024-09-10T21:52:59.861 INFO:teuthology.orchestra.run.smithi007.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68644.log.gz 2024-09-10T21:52:59.862 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51040.log 2024-09-10T21:52:59.863 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.45883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61611.log 2024-09-10T21:52:59.864 INFO:teuthology.orchestra.run.smithi007.stderr: 92.2% -- replaced with /var/log/ceph/ceph-client.admin.69693.log.gz 2024-09-10T21:52:59.864 INFO:teuthology.orchestra.run.smithi007.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45883.log.gz 2024-09-10T21:52:59.864 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41014.log 2024-09-10T21:52:59.864 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51040.log.gz 2024-09-10T21:52:59.865 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61611.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61611.log.gz 2024-09-10T21:52:59.866 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54013.log 2024-09-10T21:52:59.867 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66885.log 2024-09-10T21:52:59.867 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41014.log.gz 2024-09-10T21:52:59.868 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.54013.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54013.log.gz 2024-09-10T21:52:59.868 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36134.log 2024-09-10T21:52:59.869 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70652.log 2024-09-10T21:52:59.869 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66885.log.gz 2024-09-10T21:52:59.870 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50585.log 2024-09-10T21:52:59.870 INFO:teuthology.orchestra.run.smithi007.stderr: 76.5% -- replaced with /var/log/ceph/ceph-client.admin.36134.log.gz 2024-09-10T21:52:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70652.log: 87.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68602.log 2024-09-10T21:52:59.872 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.70652.log.gz 2024-09-10T21:52:59.872 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62806.log 2024-09-10T21:52:59.873 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50585.log.gz 2024-09-10T21:52:59.874 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44523.log 2024-09-10T21:52:59.874 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.68602.log.gz 2024-09-10T21:52:59.875 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62806.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62806.log.gz 2024-09-10T21:52:59.875 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61863.log 2024-09-10T21:52:59.876 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50742.log 2024-09-10T21:52:59.876 INFO:teuthology.orchestra.run.smithi007.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44523.log.gz 2024-09-10T21:52:59.877 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63418.log 2024-09-10T21:52:59.877 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61863.log.gz 2024-09-10T21:52:59.878 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50742.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.50742.log.gz 2024-09-10T21:52:59.878 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66927.log 2024-09-10T21:52:59.879 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.63418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63701.log 2024-09-10T21:52:59.879 INFO:teuthology.orchestra.run.smithi007.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63418.log.gz 2024-09-10T21:52:59.880 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71517.log 2024-09-10T21:52:59.881 INFO:teuthology.orchestra.run.smithi007.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.66927.log.gz 2024-09-10T21:52:59.882 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.63701.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63701.log.gz 2024-09-10T21:52:59.882 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48306.log 2024-09-10T21:52:59.883 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44167.log 2024-09-10T21:52:59.883 INFO:teuthology.orchestra.run.smithi007.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.71517.log.gz 2024-09-10T21:52:59.884 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48306.log: 87.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64948.log 2024-09-10T21:52:59.884 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.48306.log.gz 2024-09-10T21:52:59.885 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65934.log 2024-09-10T21:52:59.885 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44167.log.gz 2024-09-10T21:52:59.886 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.64948.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.64948.log.gz 2024-09-10T21:52:59.886 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63562.log 2024-09-10T21:52:59.887 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60680.log 2024-09-10T21:52:59.888 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65934.log: /var/log/ceph/ceph-client.admin.63562.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65934.log.gz 2024-09-10T21:52:59.888 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.63562.log.gz 2024-09-10T21:52:59.888 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49782.log 2024-09-10T21:52:59.889 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61947.log 2024-09-10T21:52:59.890 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60680.log.gz 2024-09-10T21:52:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49782.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49782.log.gz 2024-09-10T21:52:59.890 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62870.log 2024-09-10T21:52:59.891 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62157.log 2024-09-10T21:52:59.892 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.61947.log.gz 2024-09-10T21:52:59.892 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62870.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62870.log.gz 2024-09-10T21:52:59.892 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48348.log 2024-09-10T21:52:59.894 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.62157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44817.log 2024-09-10T21:52:59.894 INFO:teuthology.orchestra.run.smithi007.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.62157.log.gz 2024-09-10T21:52:59.894 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.48348.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48348.log.gz 2024-09-10T21:52:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52581.log 2024-09-10T21:52:59.895 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68462.log 2024-09-10T21:52:59.896 INFO:teuthology.orchestra.run.smithi007.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.44817.log.gz 2024-09-10T21:52:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52581.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52581.log.gz 2024-09-10T21:52:59.897 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70737.log 2024-09-10T21:52:59.898 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65640.log 2024-09-10T21:52:59.898 INFO:teuthology.orchestra.run.smithi007.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68462.log.gz 2024-09-10T21:52:59.899 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68770.log 2024-09-10T21:52:59.899 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70737.log.gz 2024-09-10T21:52:59.900 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65640.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71727.log 2024-09-10T21:52:59.900 INFO:teuthology.orchestra.run.smithi007.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65640.log.gz 2024-09-10T21:52:59.901 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.68770.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46655.log 2024-09-10T21:52:59.902 INFO:teuthology.orchestra.run.smithi007.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68770.log.gz 2024-09-10T21:52:59.903 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70989.log 2024-09-10T21:52:59.903 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.71727.log.gz 2024-09-10T21:52:59.903 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.46655.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46655.log.gz 2024-09-10T21:52:59.904 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60722.log 2024-09-10T21:52:59.905 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.70989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65513.log 2024-09-10T21:52:59.905 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.70989.log.gz 2024-09-10T21:52:59.906 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60722.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-10T21:52:59.906 INFO:teuthology.orchestra.run.smithi007.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.60722.log.gz 2024-09-10T21:52:59.907 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44278.log 2024-09-10T21:52:59.907 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65513.log.gz 2024-09-10T21:52:59.908 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55046.log 2024-09-10T21:52:59.910 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.44278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50202.log 2024-09-10T21:52:59.910 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.44278.log.gz 2024-09-10T21:52:59.911 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.55046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36161.log 2024-09-10T21:52:59.911 INFO:teuthology.orchestra.run.smithi007.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.55046.log.gz 2024-09-10T21:52:59.913 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.50202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72248.log 2024-09-10T21:52:59.913 INFO:teuthology.orchestra.run.smithi007.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50202.log.gz 2024-09-10T21:52:59.913 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36161.log: gzip 75.6% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.36161.log.gz -- 2024-09-10T21:52:59.914 INFO:teuthology.orchestra.run.smithi007.stderr: /var/log/ceph/ceph-client.admin.66102.log 2024-09-10T21:52:59.915 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.72248.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-10T21:52:59.915 INFO:teuthology.orchestra.run.smithi007.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.72248.log.gz 2024-09-10T21:52:59.916 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.66102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52749.log 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66102.log.gz 2024-09-10T21:52:59.916 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-10T21:52:59.918 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41566.log 2024-09-10T21:52:59.919 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.52749.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52749.log.gz 2024-09-10T21:52:59.920 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43375.log/var/log/ceph/ceph-client.admin.41566.log: 2024-09-10T21:52:59.920 INFO:teuthology.orchestra.run.smithi007.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41566.log.gz 2024-09-10T21:52:59.921 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71265.log 2024-09-10T21:52:59.922 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43375.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43375.log.gz 2024-09-10T21:52:59.923 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61475.log 2024-09-10T21:52:59.924 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.71265.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.71265.log.gz 2024-09-10T21:52:59.925 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51593.log 2024-09-10T21:52:59.926 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.61475.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61475.log.gz 2024-09-10T21:52:59.927 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65682.log 2024-09-10T21:52:59.928 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.51593.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.51593.log.gz 2024-09-10T21:52:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69379.log 2024-09-10T21:52:59.929 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.65682.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65682.log.gz 2024-09-10T21:52:59.930 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36104.log 2024-09-10T21:52:59.932 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.69379.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.69379.log.gz 2024-09-10T21:52:59.932 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60553.log 2024-09-10T21:52:59.933 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.36104.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36104.log.gz 2024-09-10T21:52:59.934 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59957.log 2024-09-10T21:52:59.935 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60553.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60553.log.gz 2024-09-10T21:52:59.936 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67011.log/var/log/ceph/ceph-client.admin.59957.log: 2024-09-10T21:52:59.937 INFO:teuthology.orchestra.run.smithi007.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.59957.log.gz 2024-09-10T21:52:59.938 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56397.log 2024-09-10T21:52:59.939 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.67011.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67011.log.gz 2024-09-10T21:52:59.939 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60041.log 2024-09-10T21:52:59.940 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.0.56397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56397.log.gz 2024-09-10T21:52:59.941 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42736.log 2024-09-10T21:52:59.942 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.60041.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.60041.log.gz 2024-09-10T21:52:59.943 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.42736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43957.log 2024-09-10T21:52:59.944 INFO:teuthology.orchestra.run.smithi007.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.42736.log.gz 2024-09-10T21:52:59.945 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45697.log 2024-09-10T21:52:59.946 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.43957.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43957.log.gz 2024-09-10T21:52:59.947 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47187.log/var/log/ceph/ceph-client.admin.45697.log: 2024-09-10T21:52:59.948 INFO:teuthology.orchestra.run.smithi007.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.45697.log.gz 2024-09-10T21:52:59.948 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41350.log 2024-09-10T21:52:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.47187.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47187.log.gz 2024-09-10T21:52:59.950 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49908.log 2024-09-10T21:52:59.951 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.41350.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41350.log.gz 2024-09-10T21:52:59.952 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60343.log 2024-09-10T21:52:59.953 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.49908.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49908.log.gz 2024-09-10T21:52:59.954 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57542.log/var/log/ceph/ceph-client.admin.60343.log: 2024-09-10T21:52:59.955 INFO:teuthology.orchestra.run.smithi007.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60343.log.gz 2024-09-10T21:52:59.956 INFO:teuthology.orchestra.run.smithi007.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57241.log 2024-09-10T21:52:59.957 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57542.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57542.log.gz 2024-09-10T21:52:59.958 INFO:teuthology.orchestra.run.smithi007.stderr:/var/log/ceph/ceph-client.admin.57241.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57241.log.gz 2024-09-10T21:52:59.997 INFO:teuthology.orchestra.run.smithi133.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-10T21:53:00.011 INFO:teuthology.orchestra.run.smithi007.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-10T21:53:00.026 INFO:teuthology.orchestra.run.smithi133.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-10T21:53:00.054 INFO:teuthology.orchestra.run.smithi007.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-10T21:53:00.064 INFO:teuthology.orchestra.run.smithi133.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-10T21:53:00.080 INFO:teuthology.orchestra.run.smithi133.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-10T21:53:00.084 INFO:teuthology.orchestra.run.smithi133.stderr: 2024-09-10T21:53:00.084 INFO:teuthology.orchestra.run.smithi133.stderr:real 0m0.748s 2024-09-10T21:53:00.084 INFO:teuthology.orchestra.run.smithi133.stderr:user 0m3.671s 2024-09-10T21:53:00.084 INFO:teuthology.orchestra.run.smithi133.stderr:sys 0m0.382s 2024-09-10T21:53:00.168 INFO:teuthology.orchestra.run.smithi007.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-10T21:53:00.354 INFO:teuthology.orchestra.run.smithi007.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-10T21:53:00.377 INFO:teuthology.orchestra.run.smithi007.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-10T21:53:00.515 INFO:teuthology.orchestra.run.smithi007.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-10T21:53:00.518 INFO:teuthology.orchestra.run.smithi007.stderr: 2024-09-10T21:53:00.518 INFO:teuthology.orchestra.run.smithi007.stderr:real 0m1.187s 2024-09-10T21:53:00.518 INFO:teuthology.orchestra.run.smithi007.stderr:user 0m3.515s 2024-09-10T21:53:00.518 INFO:teuthology.orchestra.run.smithi007.stderr:sys 0m1.568s 2024-09-10T21:53:00.519 INFO:tasks.ceph:Archiving logs... 2024-09-10T21:53:00.520 DEBUG:teuthology.misc:Transferring archived files from smithi007:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899691/remote/smithi007/log 2024-09-10T21:53:00.521 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T21:53:01.430 DEBUG:teuthology.misc:Transferring archived files from smithi133:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899691/remote/smithi133/log 2024-09-10T21:53:01.431 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T21:53:01.946 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T21:53:01.957 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T21:53:01.957 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T21:53:01.962 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T21:53:02.092 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T21:53:02.101 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T21:53:02.101 DEBUG:teuthology.orchestra.run.smithi007:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:53:02.106 DEBUG:teuthology.orchestra.run.smithi133:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:53:02.135 INFO:teuthology.orchestra.run.smithi007.stderr:bash: line 1: ntpq: command not found 2024-09-10T21:53:02.139 INFO:teuthology.orchestra.run.smithi133.stderr:bash: line 1: ntpq: command not found 2024-09-10T21:53:02.142 INFO:teuthology.orchestra.run.smithi007.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T21:53:02.142 INFO:teuthology.orchestra.run.smithi007.stdout:=============================================================================== 2024-09-10T21:53:02.143 INFO:teuthology.orchestra.run.smithi007.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 -319us[ -315us] +/- 48ms 2024-09-10T21:53:02.143 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 53 +1668us[+1668us] +/- 87ms 2024-09-10T21:53:02.143 INFO:teuthology.orchestra.run.smithi007.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 -884us[ -884us] +/- 77ms 2024-09-10T21:53:02.143 INFO:teuthology.orchestra.run.smithi007.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:53:02.146 INFO:teuthology.orchestra.run.smithi133.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T21:53:02.147 INFO:teuthology.orchestra.run.smithi133.stdout:=============================================================================== 2024-09-10T21:53:02.147 INFO:teuthology.orchestra.run.smithi133.stdout:^* hv01.front.sepia.ceph.com 3 6 377 55 -328us[ -331us] +/- 48ms 2024-09-10T21:53:02.147 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 56 +1669us[+1666us] +/- 87ms 2024-09-10T21:53:02.147 INFO:teuthology.orchestra.run.smithi133.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 -890us[ -893us] +/- 77ms 2024-09-10T21:53:02.147 INFO:teuthology.orchestra.run.smithi133.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T21:53:02.148 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T21:53:02.160 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T21:53:02.160 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T21:53:02.174 DEBUG:teuthology.orchestra.run.smithi007:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T21:53:02.320 DEBUG:teuthology.orchestra.run.smithi133:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T21:53:02.433 DEBUG:teuthology.orchestra.run.smithi007:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T21:53:02.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:53:02.496 DEBUG:teuthology.orchestra.run.smithi133:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T21:53:02.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:53:02.555 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T21:53:02.565 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T21:53:02.579 INFO:teuthology.task.internal:Duration was 1188.925758 seconds 2024-09-10T21:53:02.579 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T21:53:02.593 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T21:53:02.593 DEBUG:teuthology.orchestra.run.smithi007:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T21:53:02.597 DEBUG:teuthology.orchestra.run.smithi133:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T21:53:02.684 INFO:teuthology.orchestra.run.smithi133.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T21:53:02.686 INFO:teuthology.orchestra.run.smithi007.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T21:53:03.136 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T21:53:03.137 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi007.front.sepia.ceph.com 2024-09-10T21:53:03.137 DEBUG:teuthology.orchestra.run.smithi007:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T21:53:03.189 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi133.front.sepia.ceph.com 2024-09-10T21:53:03.189 DEBUG:teuthology.orchestra.run.smithi133:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T21:53:03.252 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T21:53:03.253 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T21:53:03.256 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T21:53:03.315 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T21:53:03.315 DEBUG:teuthology.orchestra.run.smithi007:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T21:53:03.319 DEBUG:teuthology.orchestra.run.smithi133:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T21:53:03.512 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T21:53:03.523 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T21:53:03.523 DEBUG:teuthology.orchestra.run.smithi007:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T21:53:03.579 DEBUG:teuthology.orchestra.run.smithi133:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T21:53:03.635 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T21:53:03.673 DEBUG:teuthology.orchestra.run.smithi007:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T21:53:03.677 DEBUG:teuthology.orchestra.run.smithi133:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T21:53:03.744 INFO:teuthology.orchestra.run.smithi007.stdout:kernel.core_pattern = core 2024-09-10T21:53:03.748 INFO:teuthology.orchestra.run.smithi133.stdout:kernel.core_pattern = core 2024-09-10T21:53:03.777 DEBUG:teuthology.orchestra.run.smithi007:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T21:53:03.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:53:03.810 DEBUG:teuthology.orchestra.run.smithi133:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T21:53:03.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:53:03.844 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T21:53:03.854 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T21:53:03.855 DEBUG:teuthology.misc:Transferring archived files from smithi007:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899691/remote/smithi007 2024-09-10T21:53:03.855 DEBUG:teuthology.orchestra.run.smithi007:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T21:53:03.920 DEBUG:teuthology.misc:Transferring archived files from smithi133:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899691/remote/smithi133 2024-09-10T21:53:03.921 DEBUG:teuthology.orchestra.run.smithi133:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T21:53:03.983 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T21:53:03.983 DEBUG:teuthology.orchestra.run.smithi007:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T21:53:03.987 DEBUG:teuthology.orchestra.run.smithi133:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T21:53:04.022 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T21:53:04.033 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T21:53:04.034 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T21:53:04.043 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T21:53:04.044 DEBUG:teuthology.orchestra.run.smithi007:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T21:53:04.063 DEBUG:teuthology.orchestra.run.smithi133:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T21:53:04.093 INFO:teuthology.orchestra.run.smithi007.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 21:53 /home/ubuntu/cephtest 2024-09-10T21:53:04.097 INFO:teuthology.orchestra.run.smithi133.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 21:53 /home/ubuntu/cephtest 2024-09-10T21:53:04.099 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T21:53:04.109 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T21:53:04.133 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} subvol_versions/create_subvol_version_v1 tasks/forward-scrub} duration: 1188.9257583618164 flavor: default owner: scheduled_hyelloji@teuthology success: true 2024-09-10T21:53:04.133 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T21:53:04.240 INFO:teuthology.run:pass